# Tutorial: Creating a Multiple Choice Scanner with OpenCV

Categories Computer Vision, Uncategorized
EDIT (July 14, 2016): A better way to extract would be to use page markers.
This is a tutorial on creating a multiple choice scanner similar to the Scantron system. We will take a photo of a multiple choice answer sheet and we will find the corresponding letter of the bubbles. I will be using OpenCV 2.4.3 for this project.

## Algorithm

We can split the algorithm into 9 parts:
1. Perform image preprocessing to make the image black & white (binarization)
2. Use hough transform to find the lines in the image
3. Find point of intersection of lines to form the quadrilateral
4. Apply a perspective transform to the quadrilateral
5. Use hough transform to find the circles in the image
6. Sort circles into rows and columns
7. Find circles with area 30% or denser and designate these as “filled in”
Thanks to this tutorial for helping me find POI and using perspective transformation

### 1. Image Preprocesssing

I like to use my favourite binarization method for cleaning up the image:
– First apply a gaussian blur to blur the image a bit to get rid of random dots
– Use adaptive thresholding to set each pixel to black or white
``` cv::Size size(3,3);
cv::GaussianBlur(img,img,size,0);
adaptiveThreshold(img, img,255,CV_ADAPTIVE_THRESH_MEAN_C, CV_THRESH_BINARY,75,10);
cv::bitwise_not(img, img);```

We get a nice clean image with distinct shapes marked in white. However, we do get a few dots of white but they shouldn’t affect anything.

### 2. Hough transfrom to get lines

Use a probabilistic Hough line detection to find the sides of the rectangle. It works by going to every point in the image and checking if a line exists for all the angles. This is the most expensive operation in the whole process because it has to check every point and angle.
``` cv::Mat img2;
cvtColor(img,img2, CV_GRAY2RGB);
vector<Vec4i> lines;
HoughLinesP(img, lines, 1, CV_PI/180, 80, 400, 10);
for( size_t i = 0; i < lines.size(); i++ )
{
Vec4i l = lines[i];
line( img2, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0,0,255), 3, CV_AA);
}```

## 3. Find POI of lines

From: http://opencv-code.com/tutorials/automatic-perspective-correction-for-quadrilateral-objects/

However, we need to sort the points from top left to bottom right:

``` bool comparator(Point2f a,Point2f b){
return a.x<b.x;
}
void sortCorners(std::vector<cv::Point2f>& corners, cv::Point2f center)
{
std::vector<cv::Point2f> top, bot;
for (int i = 0; i < corners.size(); i++)
{
if (corners[i].y < center.y)
top.push_back(corners[i]);
else
bot.push_back(corners[i]);
}
sort(top.begin(),top.end(),comparator);
sort(bot.begin(),bot.end(),comparator);
cv::Point2f tl = top[0].x;
cv::Point2f tr = top[top.size()-1];
cv::Point2f bl = bot[0];
cv::Point2f br = bot[bot.size()-1];
corners.clear();
corners.push_back(tl);
corners.push_back(tr);
corners.push_back(br);
corners.push_back(bl);
}
// Get mass center
cv::Point2f center(0,0);
for (int i = 0; i < corners.size(); i++)
center += corners[i];
center *= (1. / corners.size());
sortCorners(corners, center);```

## 4. Apply a perspective transform

At first I used a minimum area rectangle for extracting the region and cropping it but i got a slanted image. Because the picture was taken at an angle, the rectangle we took a picture of, has become a trapezoid. However, if you’re using a scanner, than this shouldn’t be too much an issue.
However, we can fix this with a perspective transform and OpenCV supplies a function for doing so.
``` // Get transformation matrix
cv::Mat transmtx = cv::getPerspectiveTransform(corners, quad_pts);
// Apply perspective transformation
cv::warpPerspective(img3, quad, transmtx, quad.size());```

### 5. Find circles

We use Hough transform to find all the circles using a provided function for detecting them.
```
cvtColor(img,cimg, CV_BGR2GRAY);
vector<Vec3f> circles;
HoughCircles(cimg, circles, CV_HOUGH_GRADIENT, 1, img.rows/16, 100, 75, 0, 0 );
for( size_t i = 0; i < circles.size(); i++ )
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
// circle center
circle( testImg, center, 3, Scalar(0,255,0), -1, 8, 0 );
// circle outline```

### 6. Sort circles into rows and columns

Now that we have the valid circles we should sort them into rows and columns. We can check if two circles are in a row with a simple test:
y1 = y coordinate of centre of circle 1
y2 = y coordinate of centre of circle 2
r = radius
y2-r > y1 and y2+r<y1
If two circles pass this test, then we can say that they are in the same row. We do this to all the circle until we have figure out which circles are in which rows.Row is an array of data about each row and index. The double part of the pair is the y coord of the row and the int is the index of arrays in bubble (used for sorting).

``` vector<vector<Vec3f> > bubble;
vector<pair<double,int> > row;
for(int i=0;i<circles.size();i++){
bool found = false;
int r = cvRound(circles[i][2]);
int x = cvRound(circles[i][0]);
int y= cvRound(circles[i][1]);
for(int j=0;j<row.size();j++){
int y2 = row[j].first;
if(y-r<y2&&y+r>y2){
bubble[j].push_back(circles[i]);
found = true;
break;
}
}
if(!found){
int l = row.size();
row.push_back(make_pair(y,l));
vector<Vec3f> v;
v.push_back(circles[i]);
bubble.push_back(v);
}
found = false;
}```

Then sort the rows by y coord and inside each row sort by x coord so you will have a order from top to bottom and left to right.

```bool comparator2(pair<double,int> a,pair<double,int> b){
return a.first<b.first;
}
bool comparator3(Vec3f a,Vec3f b){
return a[0]<b[0];
}
....
sort(row.begin(),row.end(),comparator2);
for(int i=0;i<bubble.size();i++){
sort(bubble[i].begin(),bubble[i].end(),comparator3);
}```

### 7. Check bubble

Now that we have each circle sorted, in each row we can check if the density of pixels is 30% or higher which will indicate that it is filled in.
We can use countNonZero to count the filled in pixels over the area of the region.
In each row, we look for the highest filled density over 30% and it will most likely be the answer that is highlighted. However, if none are found then it is blank.
```for(int i=0;i<row.size();i++){
double max = 0;
int ind = -1;
for(int j=0;j<bubble[row[i].second].size();j++){
Vec3f cir = bubble[row[i].second][j];
int r = cvRound(cir[2]);
int x = cvRound(cir[0]);
int y= cvRound(cir[1]);
Point c(x,y);
// circle outline
circle( img, c, r, Scalar(0,0,255), 3, 8, 0 );
Rect rect(x-r,y-r,2*r,2*r);
Mat submat = cimg(rect);
double p =(double)countNonZero(submat)/(submat.size().width*submat.size().height);
if(p>=0.3 && p>max){
max = p;
ind = j;
}
}
if(ind==-1)printf("%d:-",i+1);
else printf("%d:%c",i+1,'A'+ind);
cout<<endl;
}
}```

### 70 Comments

• Richiely Batista
August 30, 2013

Hi!
Nice post.
Could you send me the code, please!?

my e-mail is: [email protected]

• Gabriel Rosini
November 11, 2013

Hello!
Could you send me the code?

my e-mail is: [email protected]

• Didid Ikhsan
November 12, 2013

hello
a very interesting post.
Could you send me the code?

my e-mail is: [email protected]

• Thana seelan
January 15, 2014

i want this code plzzz
Could you send me the code?
my email ID: [email protected]

• Sanya
January 26, 2014

Hi Michael.. Very nice post.. Is the code available too?
Please mail me at [email protected]

• Yuri Alberto
February 25, 2014

Que mas hermano, puedo publicar una version en españor en mi blog y con todo el codigo. Hello, I can show your post in my Blog. It's work and code complete.

• Xuân Nguyễn
March 18, 2014

Hi Michael.. Very nice post!
Please mail me at [email protected]

• Nurzhan Mussabekov
May 11, 2014

Hi Michael…Could you send me the code, plzz?
my email ID: [email protected]

• GUNDA PRAVEEN KUMAR
May 19, 2014

Hi Micheal
could you send me the code please
my email id: [email protected]

• bac nguyen
June 9, 2014

Hi Michael…Could you send me the code?
my email ID: [email protected]

• Ümit Çelik
June 10, 2014

Hi Michael, Could you send me the code and image? I am struggling the all day to get the same result but somethings go wrong always,
I am working on Mac and opencv version 2.4.9

• ayoungprogrammer
June 11, 2014

Sorry I only have code in Windows

• bac nguyen
June 11, 2014

Could you send me the code in Windows? please!!

• fdiedler
June 19, 2014

Hi Michael,
Can you send me your code please ?
email : [email protected]

Thanks 🙂

• Carlos Henrique Lemos
June 24, 2014

Hi Michael,
Can you send me your code please ?
email : [email protected]

Thanks 🙂

• Germanno Teles
July 2, 2014

Hi Michael,
Can have the code too?
[email protected]

• Mark
July 25, 2014

Hi!
Could you send me the code?
Email: [email protected]

• ilyas
August 7, 2014

Hi!
Could you send me the code?
Email: [email protected]

• Dofs Almario
September 1, 2014

Hi,
Great code. Could you send mo the code.
Email: [email protected]

Thanks

• Issam Farran
September 14, 2014

Hi Michael,
This is simply great, and has already saved me a LOT of effort & time.
Can you please send me the code to [email protected] ? I"ll appreciate it a lot.

• Abed Khaled
September 14, 2014

can you send me the code please
it would be appreciated

[email protected]

• teacherfelix
October 15, 2014

hello Michael… thanks a lot for sharing your knowledge… it will give us teachers savings in our TIME in checking.. i am hoping that you will also share the code to me… here is my email address: [email protected]

• mustafa göl
June 8, 2015

Hi Michael, your tutorial is very helpful. There is one problem. I found more than one line for my edges, totaly 11. I changed the parameters of functions but i could not overcome. I think that merging or averaging lines is reasonable but I couldn't find the way. Do you have any suggestion? Thanks.

• Elsy Noh
December 6, 2015

Hi Michael, thanks for sharing your knowledge.
could you send me the code?
email: [email protected]

• Jake Bardinas
March 11, 2016

can u send me the code ??? please thank u 🙂

[email protected]

• aleksandar micic
March 29, 2016

Can you send me a code..and is there some tutorial how to run this?
[email protected]
And thank you.

• Mickey Friedman
April 26, 2016

H! Can you send me the code/give me instructions on how to run it? Thank you!

[email protected]

• Madu Acalud
May 25, 2016

Tnx a lot Michael. It was very helpful

I was checking you post on markers to find ROI but the link (
http://blog.ayoungprogrammer.com/2014/03/tutorial-omr-scanning-form-recognition.html) was not available.

• Khagesh Desai
July 12, 2016

Hi Michael,
This is superb.
Can you please send me the code to [email protected] ? I”ll appreciate it a lot.

• Madhup
September 15, 2016

Hi Michael,
Great tutorial. I am beginner to Python and am trying to create an OMR sheet reading set-up for my company. It would be great if you could please share the code you have used. It would be great help

• Paulo
October 2, 2016

Great help for everyone. Can you share your code? Thanks in advance. Please send to [email protected]

• brutalremus
October 30, 2016

Hi, ı want to desing this Project with java(android). Can you help me? Please dear outhor

• Dave
November 5, 2016

Please can you send me the source code to [email protected]

Thanks

• kérisson
November 17, 2016

the http://opencv-code.com/tutorials/automatic-perspective-correction-for-quadrilateral-objects/ is inaccessible. How to resolve?

• ayoungprogrammer
November 22, 2016

I have updated my post

• Airah
February 9, 2017

Hi do u have codes for android base multiple choice exam scanner?

• Airah
February 9, 2017

Pls reply thanks

• ayoungprogrammer
February 23, 2017

Sorry, I don’t have any code for that

• vishal
February 23, 2017

Hi Michael…Could you send me the code?
my email ID [email protected]

• ayoungprogrammer
February 23, 2017
• amir
March 17, 2017

hi , I tried your code …. but the the result of the exam doesnot appear , whats the problem you think ?

• Muhammad Usman Ghani
May 17, 2017

can you send the [email protected]

• Tarun Bhagat
May 23, 2017

Nice..
but can you help me for android..

• Vamsi Vudatala
May 25, 2017

nice post.. Could you send me the code plzz..my e-mail is [email protected]

• sid
June 14, 2017

please can u send me the code [email protected]

• ayoungprogrammer
June 15, 2017
• Abhishek Kateliya
June 29, 2017

Hi Michael, this is indeed very helpful post, but can you suggest me java code for the same,
Ive been trying this since a month.
Please help

• ayoungprogrammer
July 7, 2017

Sorry, I don’t have Java code available for this.

• martin
July 12, 2017

hi can you send this project?
thank [email protected]

• camila
July 21, 2017

Hola! el tutorial muy claro, pero tengo un problema, me levanta las imágenes pero no me da el resultado final…

• Adam
November 1, 2017

fatal error: opencv2/highgui/highgui.hpp: No such file and directory

• Veda
January 7, 2018

Hi! Can you send me the code please?

[email protected]. Thank you!

• Python OpenCV, Houghline detection around paper – program faq
March 15, 2018

[…] I need to form a bounding rectangle around it to carry on the further process from the tutorials here and […]

• ben
April 5, 2018

Couldn’t you have used findContours() to find the bondary of the answer sheet? Is using HoughLinesP better? I’m new to image processing.

• Minh
April 8, 2018

I got the error at line cv::Point2f tl = top[0];
cv::Point2f tr = top[top.size() – 1];
cv::Point2f bl = bot[0];
cv::Point2f br = bot[bot.size() – 1];
when I tried with the other image.

• vc
April 27, 2018

if i change the image and instead use OMR of more than 5 questions, will it still work? if not what all do i have to change??

• janith
May 3, 2018

Reply
Hello There!
Could you send me the code

• jv
May 23, 2018

I wonder if you have a tutorial on how to create a customize answer sheet. It’s a big help for me.

• Malik
June 2, 2018

Hi, it’s very helpful please send me code [email protected]
Thanks

• TVD
June 25, 2018

Hello, i using java but error load text

• omkar kachare
October 8, 2018

Hi! i got error ..printf was not declared in this scope
please help me.

• Dr. Ashfak
December 14, 2018

Hi,

Please send code of this to me on email id: [email protected]

• Abdul Latif
January 9, 2019

Great !
I realy need this code

• malik
March 26, 2019

it is possible we can read if multi marks?
what needs to be changed in the code to get multi marks,

• Thomas
January 8, 2020

Hi can i have the code to my email for Study purposes? Thanks in advance!
Email: [email protected]

• OpenCV | Andreas' Blog
April 8, 2020

[…] Tutorial: Creating a Multiple Choice Scanner with OpenCV github.com/ayoungprogrammer/MultipleChoiceScanner, A multiple choice scanner […]