OpenCV Basics Parviz

OpenCV Basics
Header files
o #include <cv.h>
o #include <cvaux.h>
o #include <highgui.h>
o #include <cxcore.h>
Image data types
o IPL_DEPTH_<bit_depth>(S|U|F)
Here possible values for <bit_depth> are 1,8,16,32 and 64
S = Signed
U = Unsigned
F = Float
1 bit depth images should be unsigned
8 bit depth images should be signed or unsigned
16 bit depth images should be signed or unsigned
32 bit depth images should be signed or float
64 bit depth images should be float
o E.g.:
IPL_DEPTH_16S means an 16-bit signed image
IPL_DEPTH_64F means a 64-bit float image
Matrix data types
o CV_<bit_depth>(S|U|F)C<number_of_channels>
Here possible values for <bit_depth> are 8,16,32 and 64
S = Signed integer
U = Unsigned integer
F = Float
Here possible values for <number_of_channels> are 1,2,3,4
8 bit depth matrices should be signed or unsigned
16 bit depth matrices should be signed or unsigned
32 bit depth matrices should be signed or float
64 bit depth matrices should be float
o E.g.:
CV_16SC1 means an 16-bit signed single-channel matrix
CV_64FC3 means a 64-bit float matrix with 3 channels
Function naming conventions
o cvActionTargetMod(.....)
Action = the functionality (e.g. load, show)
Target = the target image area (e.g. contour, image)
Mod = optional modifiers (e.g. argument type)
o E.g.:
cvLoadImage(...)
cvFitLine(...)

Capturing Images & Videos

Simple OpenCV Program-Displaying an Image from File
Open Microsoft Visual Studio and create a new win32 console application. Then write the following program. If you have not
install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
#include <highgui.h>

int main()
{
IplImage* img = cvLoadImage("C:/MyPic.jpg");
cvNamedWindow("MyWindow");
cvShowImage("MyWindow", img);
cvWaitKey(0);
cvDestroyWindow("MyWindow");
cvReleaseImage(&img);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////

Before you run this program, put any image(MyPic.jpg) in C: drive.

You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to
extract it using Winrar or other suitable software)

Result of above OpenCV Application

Explanation

Let's see the new OpenCV functions, found in the above example.
#include "stdafx.h"
#include <cv.h>
These 3 lines are pre-processor directives to include some header files for your application. In almost every program you have
to insert these 3 line at the top of your program.
IplImage is a data structure to store images in the memory.
cvLoadImage(...) is a function which loads an image from the location "C:/MyPic.jpg". Supported image formats of cvLoadImage
function are BMP, DIB, JPEG, JPG, JPE, PNG, PBM, PGM, PPM, SR, RAS, TIFF and TIF.

By default, the loaded image is forced to be a 3-channel color image. This default can be modified by using a flag variable inside
the cvLoadImage function. Then
IplImage* img = cvLoadImage("C:/MyPic.jpg" , flag);

if flag: >0 the loaded image is forced to be a 3 channel color image
=0 the loaded image is forced to be a 1 channel grayscale image
<0 the loaded image is loaded with number of channels in the file

cvNamedWindow(...) function creates a window with the title of 'MyWindow'
cvShowImage(..) function shows the image pointed by 'img' variable on the 'MyWindow' window.
cvWaitKey(0);
cvWaitKey(0) function wait for key press infinitely. If any key is pressed, this function returns the ASCII value of the key and your
program will go to the next line.
This function destroy the opened window, with the title of "MyWindow"
This function releases the memory allocated for the image , pointed by 'img' variable. After releasing the image, you cannot
access the image, 'MyPic.jpg'. But you can use the variable 'img', for any other thing. If you create something, make sure you
release it before terminating the program. This is probably the very first thing you should check when fixing memory leakage
problems with OpenCV.

Summary
When running this program, the image of 'MyPic.jpg' is created in the memory and its memory address is loaded into the
variable 'img'. Then a window named 'MyWindow' is opened. After that the image, pointed by 'img' variable is loaded to the
window, 'MyWindow'. The window, 'MyWindow' with the image, 'MyPic.jpg' will be shown until any key is pressed.

Simple OpenCV Program-Creating a Blank Image

This program is also very much similar to the previous application.

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

int main(){
IplImage* img = cvCreateImage(cvSize(800,500),IPL_DEPTH_8U,1);
cvZero(img);
cvWaitKey(0);
return 0;
}

///////////////////////////////////////////////////////////////////////////////////////


Result of above OpenCV Application

New OpenCV functions which are not found earlier are explained here
cvCreateImage(cvSize(800,500),IPL_DEPTH_8U,1);
This function create a 8 bit depth unsigned single-channel image which has a hight of 800 pixels and width of 500 pixels.

The first argument, cvSize(800,500) defines the height and width of the image.

The 2nd argument, IPL_DEPTH_8U defines the bit_depth of the image. Here bit_depth is unsigned 8 bits. Here'U' is stands for
'unsigned'. For more clarification, visit OpenCV Basics.

The 3rd argument defines the number of channels. Here it is single channel.

cvZero(img);
This function assigns zero to all pixel values of the image which is pointed by the 'img variable.

All other lines are same as the previous application.

Summary
In this application, it is created an image and assigned zero to all pixel values. Therefore we can see a black image as the result
of the application.

Simple OpenCV Program-Playing a Video From a File
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{
//load the video file to the memory
CvCapture *capture = cvCaptureFromAVI("C:/Wildlife.avi");

if( !capture ) return 1;

//obtain the frames per seconds of that video
int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );

//create a window with the title "Video"
cvNamedWindow("Video");

while(true) {
//grab and retrieve each frames of the video sequencially
IplImage* frame = cvQueryFrame( capture );

if( !frame ) break;

//show the retrieved frame in the "Video" window
cvShowImage( "Video", frame );

int c;

if(fps!=0){

//wait for 1000/fps milliseconds
c = cvWaitKey(1000/fps);
}else{
//wait for 40 milliseconds
c = cvWaitKey(40);
}

//exit the loop if user press "Esc" key (ASCII value of "Esc" is 27)
if((char)c==27 ) break;
}

//destroy the opened window
cvDestroyWindow("Video");
//release memory
cvReleaseCapture( &capture );

return 0;

}
///////////////////////////////////////////////////////////////////////////////////////


cvCaptureFromAVI("C:/Wildlife.avi");
The function allocates and initializes the CvCapture structure for reading the video stream from the specified AVI file,
"C:/Wildlife.avi" (Other video formats also can be used, but they are not optimized e.g.- wmv,dat,mpg).
cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );
Returns the frames per second of the captured video.
If this function fails to get frames per second, it will return zero.

For the first argument, it should be given a video capturing structure.

For the 2nd argument, there are few options.
They are
o CV_CAP_PROP_FPS - gives the frame rate
o CV_CAP_PROP_FRAME_WIDTH - gives the width of frames in the video steam
o CV_CAP_PROP_FRAME_HEIGHT - gives the height of frames in the video stream
o CV_CAP_PROP_FRAME_COUNT - gives the number of frames in the video stream
o CV_CAP_PROP_POS_MSEC - gives the current position in miliseconds in the running video stream
o CV_CAP_PROP_POS_FRAMES - gives the 0-based index of the frame to be captured next
This method can be used only for capturing from file. It cannot be used for capturing from cameras.
cvQueryFrame( capture );
The function grabs the next frame from the captured video and returns the pointer. The returned image should not be released
by the user.

cvWaitKey(1000/fps);
The function waits for 1000/fps milliseconds. If a key was pressed before the specified time, it returns the ASCII value of the
pressed key and the program will go to the next line. If any key is not pressed, the function returns -1, after waiting for 1000/fps
milliseconds.
If cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ) function returns zero, cvWaitKey(40) statement will execute.

The function releases the CvCapture structure allocated by cvCaptureFromAVI function.

Important Note:
User should never release images, returned by cvQueryFrame function because it will cause a run time error.

Summary
At first, this program captures a video from a file. Then program enters into a infinite loop. In that loop, it grabs frames from the
captured video sequentially, shows in a window and wait for 1000/fps milliseconds.
Using cvWaitKey function is very important because cvShowImage function need time to show the image and cvWaitkey will
give that necessary time. And waiting 1000/fps milliseconds will make sure the frame rate of output video is same as the original
frame rate.

Simple OpenCV Program-Playing a Video From a Webcam or Camera

The only difference of the following program from the above program is the first line after the main function.

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

int main()
{
//initialize and allocate memory to load the video stream from camera
CvCapture *capture = cvCaptureFromCAM(0);


//create a window with the title "Video"

while(true) {
//grab and retrieve each frames of the video sequentially

if( !frame ) break;

//show the retrieved frame in the "Video" window
cvShowImage( "Video", frame );

//wait for 40 milliseconds
int c = cvWaitKey(40);

//exit the loop if user press "Esc" key (ASCII value of "Esc" is 27)
}

//destroy the opened window
cvDestroyWindow("Video");
//release memory

return 0;

}
///////////////////////////////////////////////////////////////////////////////////////


cvCaptureFromCAM(0);
The function allocates and initialized the CvCapture structure for reading a video stream from the camera. Here the '0' means
the index of the camera to be used. You can use 1,2,3.. instead of 0, if your computer is attached to more than 1 camera. If
there is only one camera or it does not matter what camera to use, -1 can be passed as the index.

cvWaitKey(40)
cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ) cannot be used for capturing from cameras. Therefore we cannot
obtain the frame rate of video stream. Therefore we have to put a custom value for the parameter ofcvWaitKey function. You
can use any value more than 20 for that parameter. But you must try different values for that parameter and see the difference.

Saving Images & Videos
In this tutorial, it is explained how to save images and videos to the hard disk using OpenCV.
If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio.
Simple OpenCV Program-Saving an Image

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

int main()
{
//loads the image

//check whether the image is loaded
if(!img){
printf("Could not load image file\n");
}

//saves the image as NewMyPic.jpg in the working folder
int result=cvSaveImage("D:/NewMyPic.jpg",img);

//check whether the image is saved
if(!result){
printf("Could not save\n");
}


cvWaitKey(0);


return 0;
}

///////////////////////////////////////////////////////////////////////////////////////


cvSaveImage("D:/NewMyPic.jpg",img)
The function saves the image in the specified location to the specified file name and returns a non zero value. If this function
fails to save the image, it will return 0.

The image format is chosen depending on the file name extension. Only 8 bit single channel or 3 channel with 'BGR' channel
order images can be saved. If the depth or channel order of the image is different,
use'cvCvtScale' and 'cvCvtColor' functions to convert it before using this function.

If you want more explanation about various methods in the above computer application , please refer toCapturing Images &
Videos.

Simple OpenCV Program-Saving a Video

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

int main()
{
CvCapture *capture = cvCaptureFromAVI("C:/video.avi");

int frame_width=(int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH); //get the width of frame
int frame_height=(int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT); //get the height of frame
int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ); //get the frames per second

//creating video writer structure
CvVideoWriter *writer = 0;
writer=cvCreateVideoWriter("savedVideo.avi",CV_FOURCC('M', 'P', '4', '2'),fps,cvSize(frame_width,frame_height),1);

if( !writer ) {
//release memory
return 0;
}

while(true) {
//grab and retrieve each frames of the video sequentially
if( !frame ) break;

//writing frame by frame to the output file
cvWriteFrame(writer,frame);
}

//release memory
cvReleaseVideoWriter(&writer);

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////


cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH)
This function returns the width of frames in pixels
cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT)
This function returns the height of frames in pixels

cvCreateVideoWriter("savedVideo.avi",CV_FOURCC('M', 'P', '4', '2'),fps,cvSize(frame_width,frame_height),1)
This fuction creates a Video Writer structure.
The 1st argument, "savedVideo.avi" is the name of the output file. The output file will be saved inside the project folder.
Here I have used CV_FOURCC('M', 'P', '4', '2') as the four character code of codec to compress the video. If the output file
cannot be opened, you can try any of the following four character code of codecs.
All four character code of codecs are listed below
CV_FOURCC('D', 'I', 'V', '3') for MPEG-4.3 codec
CV_FOURCC('M', 'P', '4', '2') for MPEG-4.2 codec
CV_FOURCC('D', 'I', 'V', 'X') for MPEG-4 codec
CV_FOURCC('P','I','M','1') for MPEG-1 codec
CV_FOURCC('U', '2', '6', '3') for H263 codec
CV_FOURCC('I', '2', '6', '3') for H263I codec
CV_FOURCC('F', 'L', 'V', '1') for FLV1 codec
For Windows users, it is possible to use -1 instead of the above codecs in order to choose compression method and additional
compression parameters from a dialog box.

The 3rd parameter, fps is the frames per seconds of the video stream.

The 4th parameter, cvSize(frame_width,frame_height) defines the size of frames in the video.

If the last parameter is a non zero value the encoder will expect and encode color frames. If it is zero, the encoder will work with
grayscale frames. This flag is currently supported on Windows only. Here I have used 1, that's means I am telling the encoder to
encode this video as color frames.
cvWriteFrame(writer,frame)
This function adds frame by frame to the output video file.
The 1st parameter defines the Video Writer structure.
The 2nd parameter defines the frame which is to be added to the output file.

cvReleaseVideoWriter(&writer)
This function finishes writing to video file and releases the Video Writer structure.

Filtering Images
Image filtering is a important part of computer vision. Most of computer vision applications, filtering should be done before
anything else. OpenCV has lots of in-build filtering methods for images. Here is the list of methods used in this tutorial for
filtering images
Eroding
Dilating
Smoothing
Inverting
Adjusting Brightness
Adjusting Contrast
Here is the original image which I am going to filter using above methods.

Original Image


Eroding

Eroding is a simple way of filtering images. Here is how it can be done with OpenCV.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{
//display the original image

//erode and display the eroded image
cvErode(img, img, 0, 2);
cvNamedWindow("Eroded");
cvShowImage("Eroded", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Eroded");

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////

Eroded Image

cvErode(img, img, 0, 2)
The 1st parameter is the source image.
The 2nd parameter is the destination image which is to be the eroded image.
Here the 3rd parameter is the structuring element used for erosion. If it is 0, a 33 rectangular structuring element is used.
The 4th parameter is the number of times, erosion is applied.
This function can process images in place. That means same variable can be used for the 1st and 2nd parameters.

If you want more explanation about various methods in the above computer application , please refer toCapturing Images &
Videos.

Dilating

Dilating is something like opposite of the eroding an image. Here is the OpenCV code.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{

//dilate and display the dilated image
cvDilate(img, img, 0, 2);
cvNamedWindow("Dilated");
cvShowImage("Dilated", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Dilated");

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////

Dilated Image

cvDilate(img, img, 0, 2)
The 2nd parameter is the destination image which is to be the dilated image.
Here the 3rd parameter is the structuring element used for dilation. If it is 0, a 33 rectangular structuring element is used.
The 4th parameter is the number of times, dilation is applied.

Smoothing
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{

//erode and display the smoothed image
cvSmooth(img, img, CV_GAUSSIAN,3,3);
cvNamedWindow("Smoothed");
cvShowImage("Smoothed", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Smoothed");

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////


Smoothed Image

cvSmooth(img, img, CV_GAUSSIAN,3,3)
The 2nd parameter is the destination image.
The 3rd parameter is the method, used for smoothing the image. There are several methods that you can use for this
parameter.

o CV_BLUR_NO_SCALE
o CV_BLUR
o CV_GAUSSIAN
o CV_MEDIAN
o CV_BILATERAL
Every of these methods has some features and restrictions listed below.
CV_BLUR_NO_SCALE supports for single channel images only.
CV_BLUR and CV_GAUSSIAN support for 1 or 3 channel, 8 and 32 bit floating point images. These two methods can process
images in place.

CV_MEDIAN and CV_BILATERAL support for 1 or 3 channel, 8 bit images and can not process images in place.

The 4th parameter is the aperture width. (It is a technical term. It defines how much you are going to smooth the image). You
have to use positive odd number (1,3,5,7,...)

The 5th parameter is the aperture height. (It is a technical term. It defines how much you are going to smooth the image). This
parameter is ignored by CV_MEDIAN and CV_BILATERAL methods. You have to use positive odd number (1,3,5,7,...)

Inverting

Inverting an image is like taking the negative of an image.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{

//invert and display the inverted image
cvNot(img, img);
cvNamedWindow("Inverted");
cvShowImage("Inverted", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Inverted");

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////


Inverted Image

cvNot(img, img)
This function inverts every bit in every element of the image in the 1st parameter and places the result in the image in the 2nd
parameter.

e.g - For a 8 bit image, the value 0 will be mapped to (255-0)=255
the value 46 will be mapped to (255-46)=209

For a 16 bit image, the value 0 will be mapped to (65535-0)=65535
the value 46 will be mapped to (65535-46)=65489

Adjusting Brightness

You can adjust brightness of an image by adding or subtracting some constant from the each and every pixel of the original
image. Here is the simple example of increasing the brightness of an image with OpenCV.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{

//increase the brightness and display the brightened image
cvAddS(img, cvScalar(70,70,70), img);
cvNamedWindow("Brightened");
cvShowImage("Brightened", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Brightened");

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Brightened Image


cvAddS(img, cvScalar(70,70,70), img)
1st parameter is the source image.
3rd parameter is the destination image.
2nd parameter is the value, we are going to add to each and every pixel value of the source image. Here I have
used cvScalar(70,70,70) because the source image consists of 3 channels. (Color images always consists of 3 channels to
represent red, green and blue). If the source image is a gray scale image which consists of 1 channel, you may
use cvScalar(70) as the 2nd parameter. If you want to decrease the brightness, you may usecvScalar(-50,-50,-50) as the 2nd
parameter.

Adjusting Contrast

You can adjust contrast of an image by multiplying or dividing each and every pixel of the original image by some constant. Here
is the simple example of increasing the contrast of an image with OpenCV.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{

//increase the contrast and display the image
cvScale(img, img, 2);
cvNamedWindow("Contrast");
cvShowImage("Contrast", img);

cvWaitKey(0);

//cleaning up
cvDestroyWindow("Contrast");

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Contrasted Image

cvScale(img, img, 2)
1st parameter is the source image.
2nd parameter is the destination image.
3rd parameter is the constant by which each and every pixel values are multiplied. If you use value 1, there is no change. If you
use a value more than 1, destination image will have a high contrast than the source image. If you use a value less than
1, destination image will have a low contrast than the source image. (e.g - To decrease the contrast, you can use cvScale(img,
img, 0.5))

Track Bars

Using Track Bars with Images
Track bars are very useful in lots of occasions. It enables users to change various parameters while the computer application is
running.
In the following example, I have added a track bar to change the brightness of an image.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

int main()
{
int bright=50;


//create trackbars
cvCreateTrackbar("brightness", "MyWindow", &bright, 100, NULL);

//load the original image

//create a blank image
IplImage* des = cvCreateImage(cvGetSize(img),IPL_DEPTH_8U,3);

while(1){

//change the brightness of the image
cvAddS(img, cvScalar(bright-50,bright-50,bright-50), des);

cvShowImage("MyWindow", des);

int c=cvWaitKey(40);
if(c==27){
//cleaning up
cvReleaseImage(&des);
break;
}
}

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////

Using Track Bars

Explanation

Here are the new OpenCV functions, found in the above c++ application.
cvCreateTrackbar("brightness", "MyWindow", &bright, 100, NULL);
This function creates a track bar in a window, defined by the 2nd parameter.
The 1st parameter is the name of the track bar. As you can see in the above picture, that name is displayed in the left side of the
track bar.
The 3rd parameter is a pointer to an integer variable. It is the variable which is controlled by the track bar. When the posi tion of
the track bar changes, the value of that variable will also be changed. That value is displayed the the left corner of the track bar
in front of the name of the track bar.
The 4th parameter is the maximum value of the track bar. Minimum is always 0.
The 5th parameter is a name of the callback function which will be called whenever the position of the track bar changes.

If you want more explanation about various methods in above computer application , refer to Capturing Images &
Videos and Filtering Images.

Track Bars with Callback Function

In the above application, I have used 'NULL' for the 5th parameter of the function, cvCreateTrackbar. Here I am going to
explain, how we use a callback function for the 5th parameter.
In the following example, I have added a track bar to change the contrast of an image.The advantage of using the callback
function is that it is not required to use a while loop as in the above example.
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

IplImage* img;
IplImage* des;

//callback function
void changeContrast(int contrast){
if(contrast<10){
cvScale(img, des, 1/(double)(11-contrast));
}
else if(contrast>=10){
cvScale(img, des, (contrast-9));
}
cvShowImage("MyWindow", des);
}

int main()
{
int contrast=10;


//create trackbars
cvCreateTrackbar("contrast", "MyWindow", &contrast, 21, changeContrast);

img = cvLoadImage("C:/MyPic.jpg");

//create a blank image
des = cvCreateImage(cvGetSize(img),IPL_DEPTH_8U,3);

changeContrast(10);

cvWaitKey(0);

//cleaning up
cvReleaseImage(&des);

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Explanation
In addition to 'main' method, I have used a another method,'changeContrast(int contrast)' as the callback function of the track
bar.
I have used 2 global variables, img and des because they should be accessed from both methods.
Examine closely the 5th parameter of cvCreateTrackbar("contrast", "MyWindow", &contrast, 21, changeContrast). It is just
name of the callback function. This function will be called whenever the position of the track bar changes.

Mouse Events
OpenCV supports for detecting mouse events. It is very simple to do that. We can detect the specific mouse event and (x,y)
coordinates of the mouse pointer in image coordinates (not window coordinates).
Here is a simple example of detecting a mouse left click event and its (x,y) coordinates.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

//callback function
void mouseEvent(int evt, int x, int y, int flags, void* param){
if(evt==CV_EVENT_LBUTTONDOWN){
printf("%d %d\n",x,y);
}
}

int main()
{


//assigning the callback function for mouse events
cvSetMouseCallback("MyWindow", mouseEvent, 0);

//load and display an image

//wait for key press
cvWaitKey(0);

//cleaning up

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Mouse Events

Summary
In this application, it detects mouse left button press on the image window and prints out the (x,y) coordinates.

Here are the new OpenCV functions, found in the above example.

cvSetMouseCallback("MyWindow", mouseEvent, 0)
The 1st parameter is the name of the window with should be monitored for mouse events.
The 2nd parameter is the name of the function which will be called whenever mouse events occur in the specified window.This
function is prototyped as follows.
void mouseEvent(int evt, int x, int y, int flags, void* param)
o For the 1st parameter, it will be one of following
CV_EVENT_MOUSEMOVE - when the mouse pointer moves over the specified window
CV_EVENT_LBUTTONDOWN - when the left button of the mouse is pressed on the specified
window
CV_EVENT_RBUTTONDOWN - when the right button of the mouse is pressed on the specified
window
CV_EVENT_MBUTTONDOWN - when the middle button of the mouse is pressed on the specified
window
CV_EVENT_LBUTTONUP - when the left button of the mouse is released on the specified window
CV_EVENT_RBUTTONUP - when the right button of the mouse is released on the specified window
CV_EVENT_MBUTTONUP - when the middle button of the mouse is released on the specified
window

o For the 2nd and 3rd parameters, it will be the x and y coordinates of the position of mouse pointer as
illustrated in the above image.
o For the 4th parameter, it will be one of following
CV_EVENT_FLAG_LBUTTON - when the left button of the mouse is pressed on the specified
window
CV_EVENT_FLAG_RBUTTON - when the right button of the mouse is pressed on the specified
window
CV_EVENT_FLAG_MBUTTON - when the middle button of the mouse is pressed on the specified
window
CV_EVENT_FLAG_CTRLKEY - when the 'CTRL' key is pressed and any mouse event is occured
CV_EVENT_FLAG_SHIFTKEY - when the 'SHIFT' key is pressed and any mouse event is occured
CV_EVENT_FLAG_ALTKEY - when the 'ALT' key is pressed and any mouse event is occured

o The 5th parameter is user defined parameter passed by the 3rd parameter of cvSetMouseCallbackfunction.

More Advance Visual C++ Example
This is also similar to the above example. But this is a little bit advanced. Try to understand the difference and predict the result.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

//callback function
void mouseEvent(int evt, int x, int y, int flags, void* param){
if(evt==CV_EVENT_MOUSEMOVE && flags==CV_EVENT_FLAG_CTRLKEY){
printf("%d %d\n",x,y);
}
}

int main()
{


//assigning the callback function for mouse events
cvSetMouseCallback("MyWindow", mouseEvent, 0);

//load and display an image

//wait for key press
cvWaitKey(0);

//cleaning up

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////

You can download this visual OpenCV c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to

Rotating Images & Videos

Rotating Images

There are several ways to rotate images with OpenCV. The main problem in rotating images is it leaves some blank space in
the four corners of the image. There are several ways to cope with the blank area

Here is an example of rotating an image while preserving its size.

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

IplImage *rotateImage(const IplImage *src, int angleDegrees)
{
IplImage *imageRotated = cvCloneImage(src);

if(angleDegrees!=0){
CvMat* rot_mat = cvCreateMat(2,3,CV_32FC1);

// Compute rotation matrix
CvPoint2D32f center = cvPoint2D32f( cvGetSize(imageRotated).width/2, cvGetSize(imageRotated).height/2 );
cv2DRotationMatrix( center, angleDegrees, 1, rot_mat );

// Do the transformation
cvWarpAffine( src, imageRotated, rot_mat );
}

return imageRotated;
}

int main()
{

IplImage* img;
IplImage* rotated_img;

int angle=0;

//creating the window with a track bar
cvCreateTrackbar("Angle", "MyWindow", &angle, 360, 0);

while(true){

//rotate the image
rotated_img=rotateImage(img,angle);

//display the rotated image
cvShowImage("MyWindow", rotated_img);

//clean up
cvReleaseImage(&rotated_img);

//if user press 'ESC' button, program quit the while loop
if(c==27) break;
}


return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Rotated Image

Explanation
cvCloneImage(const IplImage* src)
Makes a identical(cloned) copy of the 'src' image including header and region of interrest(ROI)

Returns IplImage* points to the cloned image

Arguements -
o IplImage* src - source image to be cloned

cvCreateMat(int rows, int columns, int types)
Creates matrix header and allocates matrix data
Returns CvMat* points to the matrix

Arguements -
o int rows - number of rows of the matrix
o int columns - number of columns of the matrix
o int types - CV_<no. of bits of the image data><data type of image data>C<no. of channel>
<no. of bits of the image data> = 8,16,32, ....
<data type of image data> = 'S' for signed data, 'U' for unsigned data and 'F' for float data
<no. of channel> = 1,2,3

eg. : CV_8UC1 = 8 bit unsigned image with a single channel
CV_32FC3 = 32 bit float image with 3 channels
cvPoint2D32f (float x, float y)
It is a basic structure for 2D point with floating point cordinates

typedef struct CvPoint2D32f
{
float x;
float y;
}
CvPoint2D32f;

/* Constructor */
inline CvPoint2D32f cvPoint2D32f( double x, double y );
cv2DRotationMatrix(CvPoint2D32f center, double angle, double scale, CvMat* mapMatrix)
Computes affine matrix of 2D rotation

Arguements -
o CvPoint2D32f center - specifies the ceter of rotation
o double angle - rotation angle in degrees (Positive value means counter clockwise rotation and negative value
means clockwise rotation)
o double scale -Isotrophic scale factor
o CvMat* mapMatrix - the pointer to the resultant matrix which should be 2x3 matrix

cvWarpAffine(const CvArr* src, CvArr* dst, const CvMat* mapMatrix)
Applies an affine transformation to the source image
Arguements -
o const CvArr* src - source image
o CvArr* dst - destination image which is the affine transformed image of the source image
o const CvMat* mapMatrix - 2x3 transformation matrix

If you are not familiar with other methods of the above application, please refer to my previous OpenCV tutorials

Rotate with Zoom

This is also very much similar to the above example. Try to understand the code by comparing the above example.

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

IplImage *rotateImage(const IplImage *src, int angleDegrees, double zoom)
{
IplImage *imageRotated = cvCloneImage(src);

CvMat* rot_mat = cvCreateMat(2,3,CV_32FC1);

// Compute rotation matrix
CvPoint2D32f center = cvPoint2D32f( cvGetSize(imageRotated).width/2, cvGetSize(imageRotated).height/2 );
cv2DRotationMatrix( center, angleDegrees, zoom, rot_mat );

// Do the transformation
cvWarpAffine( src, imageRotated, rot_mat );

}

int main()
{

IplImage* img;

int angle=0;
int zoom=24;

//creating the window with 2 track bars
cvCreateTrackbar("Zoom", "MyWindow", &zoom, 99, 0);

while(true){

//rotate the image
rotated_img=rotateImage( img, angle, (zoom+1)/25.0 );


//clean up

if(c==27) break;
}


return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Rotated and Zoomed Image

Explanation
This program is almost similar to the previous one except the fact that the image can be zoomed out or in.
The only differece is in the cv2DRotationMatrix( center, angleDegrees, zoom, rot_mat ) method which has 'zoom' variable as
the third argument of this method instea of 1.

Here the value of 'zoom' variable should be larger than 0.

Rotating Images in a Different Way

///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

IplImage* rotateImage(const IplImage* src, int angleDegrees)
{
//take the dimention of original image
int w = src->width;
int h = src->height;

// Make a new image for the result
CvSize newSize;
newSize.width = cvRound(w);
newSize.height = cvRound(h);
IplImage *imageRotated = cvCreateImage( newSize, src->depth, src->nChannels );

// Create a map_matrix, where the left 2x2 matrix is the transform and the right 2x1 is the dimensions.
float m[6];
CvMat M = cvMat(2, 3, CV_32F, m);

float angleRadians = angleDegrees * ((float)CV_PI / 180.0f);
m[0] = (float)( cos(angleRadians) );
m[1] = (float)( sin(angleRadians) );
m[3] = -m[1];
m[4] = m[0];
m[2] = w*0.5f;
m[5] = h*0.5f;

// Transform the image
cvGetQuadrangleSubPix( src, imageRotated, &M);

}

int main()
{

IplImage* img;
int angle=0;

//creating the window with a track bar

while(true){

//rotate the image
rotated_img=rotateImage(img,angle);


//clean up

if(c==27) break;
}


return 0;
}
///////////////////////////////////////////////////////////////////////////////////////


Rotated Image

Explanation

cvRound(double x)
Returns nearest nterger value of x

Arguement -
o double x - the input floating point number

cvCreateImage( CvSize size, int depth, int channels)
Creates an image header and allocate image data
Returns IplImage* which points to the created blank image

Arguements -
o CvSize size - size of the cretaed blank image
o int depth - depth of the created blank image (8,16,32,...)
o int channels - number of channels of the created blank image (1,3)

cvMat(int rows, int columns, int type, void data)
Initialize the matrix header and assigns data to it. The matrix is filled row-wise

Arguements -
o int rows - number of rows of the matrix
o int columns - number of columns of the matrix
o int types - CV_<no. of bits of the image data><data type of image data>C<no. of channel>
(More details can be found in the explanation of the method, 'cvCreateMat' in the 1st application in the top of this page)

cvGetQuadrangleSubPix(const CvArr* src, CvArr* dst, const CvMat* mapMatrix)
Retrieves the pixel quadrangle from an image with sub-pixel accuracy.
The values of pixels at non-integer coordinates are retrieved using bilinear interpolation.
When the function needs pixels outside of the image, it uses replication border mode to reconstruct the values.
Every channel of multiple-channel images is processed independently.

Arguements -
o const CvArr* src - Source image
o CvArr* dst - Destination image
o const CvMat*mapMatrix - Transformation 2x3 matrix

Object Detection & Tracking using Color

Object detection and segmaentation is the most important and challenging fundamental task of computer vision. It is a critical
part in many applications such as image search, image auto-annotation and scene understanding. However it is still an open
problem due to the complexity of object classes and images.

The easiest way to detect and segment an object from an image is the color based methods . The colors in the object and the
background should have a significant color difference in order to segment objects sucessfully using color based methods.

Simple Example of Detecting Red objects

In this example, I am going to convet a video into a binary image based on the red color. (Red color area of the video is
assined to '1' and other area is assigned to '0' in the binary image)

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"

#include <cv.h>

//This function threshold the HSV image and create a binary image
IplImage* GetThresholdedImage(IplImage* imgHSV){
IplImage* imgThresh=cvCreateImage(cvGetSize(imgHSV),IPL_DEPTH_8U, 1);
cvInRangeS(imgHSV, cvScalar(170,160,60), cvScalar(180,256,256), imgThresh);
return imgThresh;
}

int main(){
CvCapture* capture =0;

capture = cvCaptureFromCAM(0);
if(!capture){
printf("Capture failure\n");
return -1;
}

IplImage* frame=0;

cvNamedWindow("Ball");

//iterate through each frames of the video
while(true){

frame = cvQueryFrame(capture);
if(!frame) break;

frame=cvCloneImage(frame);
cvSmooth(frame, frame, CV_GAUSSIAN,3,3); //smooth the original image using Gaussian kernel

IplImage* imgHSV = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 3);
cvCvtColor(frame, imgHSV, CV_BGR2HSV); //Change the color format from BGR to HSV
IplImage* imgThresh = GetThresholdedImage(imgHSV);

cvSmooth(imgThresh, imgThresh, CV_GAUSSIAN,3,3); //smooth the binary image using Gaussian kernel

cvShowImage("Ball", imgThresh);
cvShowImage("Video", frame);

//Clean up used images
cvReleaseImage(&imgHSV);
cvReleaseImage(&imgThresh);
cvReleaseImage(&frame);

//Wait 50mS
//If 'ESC' is pressed, break the loop
}

cvDestroyAllWindows() ;
cvReleaseCapture(&capture);

return 0;
}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


Explanation
OpenCV usually captures images and videos in 8-bit, unsigned integer, BGR format. In other words, captured images can be
considered as 3 matrices, BLUE,RED and GREEN with integer values ranges from 0 to 255.

How BGR image is formed
In the above image, each small box represents a pixel of the image. In real images, these pixels are so small that human eye
cannot differentiate.

Usually, one can think that BGR color space is more suitable for color based segmentation. But HSV color space is the most
suitable color space for color based image segmentation. So, in the above application, I have converted the color space of
original image of the video from BGR to HSV image.

HSV color space is consists of 3 matrices, 'hue', 'saturation' and 'value'. In OpenCV, value range for 'hue', 'saturation' and
'value' are respectively 0-179, 0-255 and 0-255. 'Hue' represents the color, 'saturation' represents the amount to which that
respective color is mixed with white and 'value' represents the amount to which that respective color is mixed with black.

In the above application, I have considered that the red object has 'hue', 'saturation' and 'value' in between 170-180, 160-255,
60-255 respectively. Here the 'hue' is unique for that specific color distribution of that object. But 'saturation' and 'value' may be
vary according to the lighting condition of that environment.

Hue values of basic colors
o Orange 0-22
o Yellow 22- 38
o Green 38-75
o Blue 75-130
o Violet 130-160
o Red 160-179
There is a open source software, called 'ColorWheelHSV', that enables you to find the 'hue', 'saturation' and 'value' for a specific
object easily. You can download it from here.

These are approximate values. You have to find the exact range of 'hue' values according to the color of the object. I found that
the range of 170-179 is perfect for the range of hue values of my object. The 'saturation' and 'value' is depend on the lighting
condition of the environment as well as the surface of the object.

How to find the exact range of 'hue', 'saturation' and 'value' for a object is discussed later in this post.

Now let's discuss new OpenCV methods in the above application.

cvInRangeS(const CvArr* src, CvScalar lower, CvScalar upper, CvArr* dst)
Checks that each array element of 'src' lies between 'lower' and 'upper'. If so, array element in the relevant location of 'dst' is
assigned '255' , otherwise '0'.

Arguments -
o const CvArr* src - source array which is the image
o CvScalar lower - inclusive lower bound (In the above application, it is cvScalar(170,160,60) because my lower
bound for 'hue','saturation' and 'value' is 170,160 and 60 respectively)
o CvScalar upper - exclusive upper bound (In the above application, it is cvScalar(180,256,256) because my
upper bound for 'hue','saturation' and 'value' is 179,255 and 255 respectively)
o CvArr* dst - destination array which is the binary image (must have 8-bit unsigned integer or 8-bit signed
integer type)

Output images of a webcam is little bit noisy. So, I smooth each frame of the video and each binary thresholded images with a
Gaussian 3x3 kernel.

All other methods in the above application have been discussed in early OpenCV tutorials.

Simple Example of Tracking Red objects

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include <cv.h>

IplImage* imgTracking;
int lastX = -1;
int lastY = -1;

cvInRangeS(imgHSV, cvScalar(170,160,60), cvScalar(180,2556,256), imgThresh);
return imgThresh;
}

void trackObject(IplImage* imgThresh){
// Calculate the moments of 'imgThresh'
CvMoments *moments = (CvMoments*)malloc(sizeof(CvMoments));
cvMoments(imgThresh, moments, 1);
double moment10 = cvGetSpatialMoment(moments, 1, 0);
double moment01 = cvGetSpatialMoment(moments, 0, 1);
double area = cvGetCentralMoment(moments, 0, 0);

// if the area<1000, I consider that the there are no object in the image and it's because of the noise, the area is not
zero
if(area>1000){
// calculate the position of the ball
int posX = moment10/area;
int posY = moment01/area;

if(lastX>=0 && lastY>=0 && posX>=0 && posY>=0)
{
// Draw a yellow line from the previous point to the current point
cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX, lastY), cvScalar(0,0,255), 4);
}

lastX = posX;
lastY = posY;
}

free(moments);
}

int main(){

if(!capture){
return -1;
}

IplImage* frame=0;
if(!frame) return -1;

//create a blank image and assigned to 'imgTracking' which has the same size of original video
imgTracking=cvCreateImage(cvGetSize(frame),IPL_DEPTH_8U, 3);
cvZero(imgTracking); //covert the image, 'imgTracking' to black


while(true){

if(!frame) break;

cvSmooth(frame, frame, CV_GAUSSIAN,3,3); //smooth the original image using Gaussian kernel


cvSmooth(imgThresh, imgThresh, CV_GAUSSIAN,3,3); //smooth the binary image using Gaussian kernel

//track the possition of the ball
trackObject(imgThresh);

// Add the tracking image and the frame
cvAdd(frame, imgTracking, frame);



//Wait 10mS
}

cvDestroyAllWindows() ;
cvReleaseImage(&imgTracking);

return 0;
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


Object Tracking

Explanation

In this application, I use moments to calculate the position of the center of the object. We have to calculate 1st order spacial
moments around x-axis and y-axis and the 0th order central moments of the binary image.

0th order central moments of the binary image is equal to the white area of the image in pixels.
X coordinate of the position of the center of the object = 1st order spacial moment around x-axis / 0th order central
moment
Y coordinate of the position of the center of the object = 1st order spacial moment around y-axis / 0th order central
moment
If there are 2 or more objects in the image, we cannot use this method. And noise of the binary image is also should be at
minimum level to get accurate results.

In the above application, I considered that if the white area of the binary image is less than 1000 in pixels, there is no objects in
the image because there may be noisy white areas that may have been identified as the object. Usually my object always have
an area more than 1000 pixels.

Now, let's discuss new OpenCV methods that can be found in the above application.

malloc(size_t x)
Allocate a block of memory which have x bytes and returns a pointer to the beginning of the block.
The content of the newly allocated block of memory is not initialized, remaining with undetermined values.

cvMoments(
const CvArr* arr, CvMoments* moments, int isBinary=0
)
Calculates all of the spacial and central moments up to the third order

Arguments -
o const CvArr* arr - source image which we are going to find the moment
o CvMoments* moments - Pointer to a memory block to store calculated moments
o int isBinary - If this argument is equal to non-zero value, all non-zero pixel values are considered as 1 when
calculating moments.

free(void* ptr)
A block of memory previously allocated using a call to malloc is deallocated, making it available again for further allocations.

cvGetSpatialMoment(CvMoments* ptr, int x_order, int y_order)
Retrieve calculated spacial moments from ptr memory block

Arguments -
o CvMoments* ptr - pointer to allocated memory block which store all moments
o int x_order - order of x (>=0)
o int y_order - order of y (>=0)

e.g. -
cvGetSpatialMoment(moments, 1, 0) retrieves 1st order spacial moment around x-axis
cvGetSpatialMoment(moments, 0, 1) retrieves 1st order spacial moment around y-axis

cvGetCentralMoment ( CvMoments* ptr, int x_order, int y_order )
Retrieve calculated central moments from ptr memory block

Arguments -
CvMoments* ptr - pointer to allocated memory block which store all moments
int x_order - order of x (>=0)
int y_order - order of y (>=0)
e.g. -
cvGetCentralMoment(moments, 0, 0) retrieves 0th order central moment

cvLine (CvArr* img, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1)
Draw a line between 2 points, pt1 and pt2

Arguments -
o CvArr* img - source image
o CvPoint pt1 - starting point of the line
o CvPoint pt2 - ending point of the line
o CvScalar color - color of the line (in the Blue, Green, Red order)
o int thickness - thickness of the line in pixels

Except to the 'void trackObject(IplImage* imgThresh)' method, all other code in the above application is almost same to the 1st
application in this post.

How to Find Exact Range for 'Hue', 'Saturation' and 'Value' for a Given Object
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include <cv.h>

int lowerH=0;
int lowerS=0;
int lowerV=0;

int upperH=180;
int upperS=256;
int upperV=256;


cvInRangeS(imgHSV, cvScalar(lowerH,lowerS,lowerV), cvScalar(upperH,upperS,upperV), imgThresh);

return imgThresh;

}

//This function create two windows and 6 trackbars for the "Ball" window
void setwindowSettings(){

cvCreateTrackbar("LowerH", "Ball", &lowerH, 180, NULL);
cvCreateTrackbar("UpperH", "Ball", &upperH, 180, NULL);

cvCreateTrackbar("LowerS", "Ball", &lowerS, 256, NULL);
cvCreateTrackbar("UpperS", "Ball", &upperS, 256, NULL);

cvCreateTrackbar("LowerV", "Ball", &lowerV, 256, NULL);
cvCreateTrackbar("UpperV", "Ball", &upperV, 256, NULL);
}

int main(){

if(!capture){
return -1;
}

IplImage* frame=0;

setwindowSettings();

while(true){

if(!frame) break;





//Wait 80mS

}

cvDestroyAllWindows();

return 0;
}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


Finding the optimum 'hue', 'saturation' and 'value' ranges for an object is a 4 step process.

1) At the beginning, track bars are placed so that maximum ranges for 'hue', 'saturation' and 'value' are set. In other
words, 'hue', 'saturation' and 'value'have ranges 0-179, 0-255 and 0-255 respectively. So, we will see a complete white image in
the 'Ball' window.

2) First, adjust 'LowerH' and 'UpperH' track bars so that the gap between 'LowerH' and 'UpperH' is minimized. Here you have to
be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimi ze the
gap.

3) Then, adjust 'LowerS' and 'UpperS' track bars so that the gap between 'LowerS' and 'UpperS' is minimized. Here you have to
be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimize the
gap.

4) Finally, adjust 'LowerV' and 'UpperV' track bars so that the gap between 'LowerV' and 'UpperV' is minimized. Here you have
to be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimize
the gap.

Now you know the optimum 'hue', 'saturation' and 'value' ranges for the object. It is 163-179, 126-217 and 68-127 in my case as
you can see in the below picture.

Identification of Shapes of Object using Contours

In the previous tutorial, we could detect and track an object using color separation. But we could not identify the shape of the
object there. In this tutorial, let's see how to identify a shape and position of an object using contours with OpenCV.

Using contours with OpenCV, you can get a sequence of points of vertices of each white patch (White patches are considered
as polygons). As example, you will get 3 points (vertices) for a triangle, and 4 points for quadrilaterals. So, you can identify any
polygon by the number of vertices of that polygon. You can even identify features of polygons such as convexity,
concavity, equilateral and etc by calculating and comparing distances between vertices.

Let's see how this can be done with OpenCV. All you need, is a binary image in which your objects should be white and the
background should be black.

Now I am going to identify triangles and quadrilaterals and heptagon in the above image using a C++ application with OpenCV.
I'll draw a line along the perimeter of every identified polygon with colors blue for triangle, green for quadrilaterals and red for
heptagons. Here is the code.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>
using namespace std;

int main()
{

IplImage* img = cvLoadImage("C:/Users/SHERMAL/Desktop/FindingContours.png");

//show the original image
cvNamedWindow("Raw");
cvShowImage("Raw",img);

//converting the original image into grayscale
IplImage* imgGrayScale = cvCreateImage(cvGetSize(img), 8, 1);
cvCvtColor(img,imgGrayScale,CV_BGR2GRAY);

//thresholding the grayscale image to get better results
cvThreshold(imgGrayScale,imgGrayScale,128,255,CV_THRESH_BINARY);

CvSeq* contours; //hold the pointer to a contour in the memory block
CvSeq* result; //hold sequence of points of a contour
CvMemStorage *storage = cvCreateMemStorage(0); //storage area for all contours

//finding all contours in the image
cvFindContours(imgGrayScale, storage, &contours, sizeof(CvContour), CV_RETR_LIST,
CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));

//iterating through each contour
while(contours)
{
//obtain a sequence of points of contour, pointed by the variable 'contour'
result = cvApproxPoly(contours, sizeof(CvContour), storage, CV_POLY_APPROX_DP,
cvContourPerimeter(contours)*0.02, 0);

//if there are 3 vertices in the contour(It should be a triangle)
if(result->total==3 )
{
//iterating through each point
CvPoint *pt[3];
for(int i=0;i<3;i++){
pt[i] = (CvPoint*)cvGetSeqElem(result, i);
}

//drawing lines around the triangle
cvLine(img, *pt[0], *pt[1], cvScalar(255,0,0),4);

}

//if there are 4 vertices in the contour(It should be a quadrilateral)
else if(result->total==4 )
{
CvPoint *pt[4];
}

//drawing lines around the quadrilateral
}

//if there are 7 vertices in the contour(It should be a heptagon)
else if(result->total ==7 )
{
CvPoint *pt[7];
}

//drawing lines around the heptagon
}

//obtain the next contour
contours = contours->h_next;
}

//show the image in which identified shapes are marked
cvNamedWindow("Tracked");
cvShowImage("Tracked",img);

cvWaitKey(0); //wait for a key press

//cleaning up
cvReleaseMemStorage(&storage);
cvReleaseImage(&imgGrayScale);

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

As you can see, triangles are marked with blue, quadrilaterals are marked with green and heptagons are marked with red. So,
now it is obvious that this method is capable of identifying shapes.

Explanation

Here I have converted the original image in to gray scale. It is because this method works only with gray scale image with single
channel. To get better results, I threshold the gray-scale image using 'cvThreshold' function. You can use your own way to
threshold the image. Then I find all contours in the thresholded image and identify and track all triangles, quadrilaterals and
heptagons.

Let's discuss new OpenCV functions, found in this application.

cvThreshold( const Mat& src, Mat& dst, double threshVal, double max, int thresholdType )
applies a fix level threshold to the each element of 'src' array write a value to corresponding array element of 'dst'

Arguements -
o const Mat& src - Source array (This should be single channel)
o Mat& dst - Destination array which has the same size and same type as the 'src'
o double threshVal - Threshold value
o double max - Maximum value to use with 'THRESH_BINARY' and 'THRESH_BINARY_INV' which are
thresholding types
o int thresholdType - You can use one of the following for this arguement
THRESH_BINARY
dst(x,y)=max, if src(x,y) > ThreshVal
dst(x,y)=0, if src(x,y) < ThreshVal
THRESH_BINARY_INV
dst(x,y)=0, if src(x,y) > ThreshVal
dst(x,y)=max, if src(x,y) < ThreshVal

THRESH_TOZERO
dst(x,y)=src(x,y), if src(x,y) > ThreshVal
dst(x,y)=0, if src(x,y) < ThreshVal
THRESH_TOZERO_INV
dst(x,y)=0, if src(x,y) > ThreshVal
dst(x,y)=src(x,y), if src(x,y) < ThreshVal
THRESH_TRUNC
dst(x,y)=threshVal, if src(x,y) > ThreshVal
dst(x,y)=src(x,y), if src(x,y) < ThreshVal

In the above application, I have used 'THRESH_BINARY', because I want to assign 255 (white) where the objects are located
and 0 (black) elsewhere.

cvCreateMemStorage(int byteSize)
Creates memory storage which has the capacity specified by the parameter 'byteSize'. But if byteSize=0, the allocated capacity
is the default value(usually 64 Kb)

cvFindContours( CvArr* img, CvMemStorage* str, CvSeq** first_contour, int header_size, int mode, int method,
CvPoint offset )
Find all contours in a binary image
Arguments -
o CvArr* img - Source image (This should be 8 bit single channel). All non-zero pixels are considered as 1 and
all zero remain zero.
o CvMemStorage* str - Memory blocks to store all obtained contours
o CvSeq** first_contour - store a pointer to the first contour in the memory block, 'str'
o int header_size - size of the sequence header
o int mode - mode of retrieval of contours from the image
You have to choose one of the following
CV_RETR_LIST - Retrieves all of the contours and put them in a list
CV_RETR_EXTERNAL - Retrieves only the extreme outer contours
CV_RETR_CCOMP - Retrieves all of the contours and organizes them into a two-level hierarchy: on
the top level are the external boundaries of the components, on the second level are the boundaries
of the holes
CV_RETR_TREE - Retrieves all of the contours and reconstructs the full hierarchy of nested
contours

o int method - Approximation method
You have to choose one of the following

CV_CHAIN_CODE - Outputs contours in the Freeman chain code
CV_CHAIN_APPROX_NONE - Translates all of the points from the chain code into points
CV_CHAIN_APPROX_SIMPLE - Compresses horizontal, vertical, and diagonal segments and
leaves only their end points
CV_CHAIN_APPROX_TC89_L1,CV_CHAIN_APPROX_TC89_KCOS - Applies one of the flavors of
the Teh-Chin chain approximation algorithm.
CV_LINK_RUNS - uses a completely different contour retrieval algorithm by linking horizontal
segments of 1s. Only the 'CV_RETR_LIST' retrieval mode can be used with this method.

o CvPoint offset - Offset by which every contour point should be shifted. This is useful when we have set ROI
(Region Of Interest) in the image. Normally we set the offset to 'cvPoint(0,0)'

cvApproxPoly( const void* src, int header_size, CvMemStorage* storage, int method, double para1, int para2 )
Approximate polygonal curves with specified precision

arguments -
o const void* src - Sequence of points
o int header_size - size of the sequence header
o CvMemStorage* storage - memory block that contains all contours
o int method - Approximation method. (The only method, available to use for this argument is
'CV_POLY_APPROX_DP')
o double para1 - approximation accuracy
o int para2 - Determines whether the single sequence should be approximated or all sequences in the same
level or below

cvGetSeqElem( const CvSeq* seq, int index )
Returns a pointer to the element of 'seq' at 'index'

cvReleaseMemStorage( CvMemStorage** storage )
Deallocate memory blocks which have been allocated by 'cvCreateMemStorage()' function

Real World Example

The above example is not really useful in practical situation. Usually, there are lots of noises in an image such as irregular
lighting, shadows, camera irregularities and etc. So, above application as it is, cannot be used to identify shapes in a real image.
It should be modified to cope with these noises. And images usually have 3 channels (BGR color). So, it should be converted
into grey-scale which has only one channel.

Here is a real world image of an arena of a robot soccer, taken from a camera.

Here, we are going to detect and mark the perimeter of each triangle in the image with a blue line. Let's see the modified
OpenCV c++ application which accomplish the above task.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"
#include <cv.h>

int main()
{

IplImage* img = cvLoadImage("C:/Users/SHERMAL/Desktop/DetectingContours.jpg");

//show the original image
cvNamedWindow("Original");
cvShowImage("Original",img);

//smooth the original image using Gaussian kernel to remove noise
cvSmooth(img, img, CV_GAUSSIAN,3,3);

IplImage* imgGrayScale = cvCreateImage(cvGetSize(img), 8, 1);
cvCvtColor(img,imgGrayScale,CV_BGR2GRAY);

cvNamedWindow("GrayScale Image");
cvShowImage("GrayScale Image",imgGrayScale);

cvThreshold(imgGrayScale,imgGrayScale,100,255,CV_THRESH_BINARY_INV);

cvNamedWindow("Thresholded Image");
cvShowImage("Thresholded Image",imgGrayScale);

CvSeq* contour; //hold the pointer to a contour

cvFindContours(imgGrayScale, storage, &contour, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE,
cvPoint(0,0));

while(contour)
{
//obtain a sequence of points of the countour, pointed by the variable 'countour'
result = cvApproxPoly(contour, sizeof(CvContour), storage, CV_POLY_APPROX_DP,
cvContourPerimeter(contour)*0.02, 0);

//if there are 3 vertices in the contour and the area of the triangle is more than 100 pixels
if(result->total==3 && fabs(cvContourArea(result, CV_WHOLE_SEQ))>100 )
{
CvPoint *pt[3];
}

//drawing lines around the triangle

}

contour = contour->h_next;
}

//show the image in which identified shapes are marked
cvNamedWindow("Tracked");
cvShowImage("Tracked",img);

cvWaitKey(0); //wait for a key press

//cleaning up

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Gray scale Image

Thresholded Image

Triangles Detected

In the same way, any shapes with any sizes can be detected with OpenCV.

Explanation

To reduce the noise level of the original image, I have smoothed the original image with a Gaussian kernel.
Further you can change the 5th argument of cvApproxPoly() function to cope with the noise. In the above example, I have
used cvContourPerimeter(contour)*0.02 as the 5th argument of cvApproxPoly(). You can
try cvContourPerimeter(contour)*0.01 or cvContourPerimeter(contour)*0.04 or any other value and see the difference of
the output yourselves.
Still there may be very small triangles, formed due to the noise. Therefore all triangles with areas less than 100 pixels are
filtered out.

cvContourArea(const CvArr* contour, CvSlice slice)
Calculate the area enclosed by sequence of contour points.
o const CvArr* contour - array of vertices of the contour
o CvSlice slice - starting and ending point of the contour. 'CV_WHOLE_SEQ' will take the whole contour to
calculate the area
The orientation of contour affects the area sign. So, this function may return a negative value. So, it should be used fabs()
function to get the absolute value.

fabs(double x)

This function returns the absolute value of any floating point number. ( This is a C function, not a OpenCV function)

Tracking two Triangles in a Video

Here I am going to track the two triangles in a video. The blue triangle is marked with red and the green triangle is marked with
blue.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>

IplImage* imgTracking=0;

int lastX1 = -1;
int lastY1 = -1;

int lastX2 = -1;
int lastY2 = -1;

void trackObject(IplImage* imgThresh){
CvSeq* contour; //hold the pointer to a contour

cvFindContours(imgThresh, storage, &contour, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE,
cvPoint(0,0));

while(contour)
{
//obtain a sequence of points of the countour, pointed by the variable 'countour'
result = cvApproxPoly(contour, sizeof(CvContour), storage, CV_POLY_APPROX_DP,
cvContourPerimeter(contour)*0.02, 0);

//if there are 3 vertices in the contour and the area of the triangle is more than 100 pixels
if(result->total==3 && fabs(cvContourArea(result, CV_WHOLE_SEQ))>100 )
{
CvPoint *pt[3];
}

int posX=( pt[0]->x + pt[1]->x + pt[2]->x )/3;
int posY=( pt[0]->y + pt[1]->y + pt[2]->y )/3;

if(posX > 360 ){
if(lastX1>=0 && lastY1>=0 && posX>=0 && posY>=0){
// Draw a red line from the previous point to the current point
cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX1, lastY1), cvScalar(0,0,255), 4);
}

lastX1 = posX;
lastY1 = posY;
}
else{
if(lastX2>=0 && lastY2>=0 && posX>=0 && posY>=0){
// Draw a blue line from the previous point to the current point
cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX2, lastY2), cvScalar(255,0,0), 4);
}

lastX2 = posX;
lastY2 = posY;
}
}

contour = contour->h_next;
}

}

int main(){
CvCapture *capture = cvCaptureFromAVI("E:/Projects/Robot/IESL Robot/robot/a.avi");

if(!capture){
return -1;
}

IplImage* frame=0;
if(!frame) return -1;

//create a blank image and assigned to 'imgTracking' which has the same size of original video
imgTracking=cvCreateImage(cvGetSize(frame),IPL_DEPTH_8U, 3);
cvZero(imgTracking); //covert the image, 'imgTracking' to black


while(true){

if(!frame) break;

//smooth the original image using Gaussian kernel
cvSmooth(frame, frame, CV_GAUSSIAN,3,3);

IplImage* imgGrayScale = cvCreateImage(cvGetSize(frame), 8, 1);
cvCvtColor(frame,imgGrayScale,CV_BGR2GRAY);

cvThreshold(imgGrayScale,imgGrayScale,100,255,CV_THRESH_BINARY_INV);

//track the possition of the ball
trackObject(imgGrayScale);

// Add the tracking image and the frame
cvAdd(frame, imgTracking, frame);



//Wait 10mS
}

cvReleaseImage(&imgTracking);

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

OpenCV Basics Parviz

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

OpenCV Basics Parviz

Uploaded by

Copyright:

Available Formats

OpenCV Basics

You might also like