Intel Open Source Computer Vision Library Version 4.

0-Beta
Installation and Getting Started Guide for Windows

Troy L. McDaniel
http://www.public.asu.edu/~tmcdani
Center for Cognitive Ubiquitous Computing
Arizona State University
 Table of Contents

1. Introduction 2

2. Installation 2

2.1. Installing DirectX and DirectX Extras 2

2.2. Installing OpenCV v4.0-Beta 3

3. Setting Up Visual C++ 6.0 For OpenCV 4

3.1. Setting Up the System PATH Environmental Variable 4

3.2. Specifying Directories in Visual C++ 6.0 6

3.3. Creating a Visual C++ Project That Uses OpenCV 9

4. Getting Started With OpenCV 18

5. Troubleshooting 19

5.1. Error: cannot open include file <filename.h> 19

5.2. Error: unresolved external symbol <symbol> 19

5.3. Error: bad number of channels/unsupported format 20

5.4. Missing Files 20

5.5. Other Hints 20

6. Example Programs 20

6.1. Displaying Video From a Web Cam (Web Cam Required) 20

6.2. Edge Detection (Web Cam Required) 21

6.3. Motion Detection (Web Cam Required) 22

6.4. Pyramidal Lucas-Kanade Motion Tracker with Optical Flow Field Visualization
(Web Cam Required) 24

7. Future Work 29

8. Final Words 29

1
 1. Introduction
The Intel Open Source Computer Vision Library, or just OpenCV, is a library of image
processing and computer vision algorithms. It’s open source so anybody can contribute to it.
OpenCV utilizes Intel’s Image Processing Library (IPL), which is a set of low-level image
processing functions. OpenCV uses DirectX, which is a set of APIs developed by Microsoft for
creating multimedia applications and games.
OpenCV is quickly gaining popularity for developing real-time computer vision
applications. Some examples of applications include face recognizers, object recognizers, and
motion trackers, just to name a few. The library has especially gained popularity in the computer
vision research community. It allows researchers to get demos or research projects up and
running quickly, and take advantage of the large collection of algorithms that are already
available.
This guide will take you step-by-step through the installation of OpenCV and the setup of
Visual C++ 6.0. This guide covers installation on Windows XP; however, OpenCV may be
installed on other operating systems including Windows 95/98/2000/NT and Linux. Furthermore,
we use here Visual C++ 6.0, but .NET is possible as well. After installation and setup, some
advice on getting started with learning OpenCV is given. Also, this guide offers a
troubleshooting section where some common problems are solved, as well as a section listing
some example programs to help better familiarize you with OpenCV.

2. Installation
This section covers installing DirectX 9.0 SDK Update (December 2004), DirectX 9.0
SDK Update (December 2004) Extras, and OpenCV v4.0-Beta.

2.1. Installing DirectX and DirectX Extras

Required to run OpenCV is Microsoft DirectX 9.0 SDK Update (December 2004) and
Microsoft DirectX 9.0 SDK Update (December 2004) Extras. (Later versions of the update
and extras will work as well.)
Go to http://www.microsoft.com/downloads/details.aspx?FamilyID=4e825a37-0c94-
4421-9ec8-156e52525d11&displaylang=en. The top of the page should read “DirectX 9.0 SDK
Update – (December 2004)”. Click the download button on the right hand side. Click save and
save the file to the desktop, or some temporary folder if desired. A self-extracting zip file called
“dxsdk_dec2004.exe” of size 222 MB will be downloaded. Once this file has successfully
downloaded, double click it and follow the instructions to install it. Next, we’ll install the extras.
Go to http://www.microsoft.com/downloads/details.aspx?FamilyID=a8667cc6-22ca-
4195-a4fa-71dec1f454f9&displaylang=en. The top of the page should read “DirectX 9.0 SDK
Update (December 2004) Extras”. Follow the same instructions as before to download the file. A
self-extracting zip file called “dxsdk_dec2004_extras.exe” of size 35.1 MB will be downloaded.
Once this file has successfully downloaded, double click it and extract the files to a temporary
folder. Created at this location will be a folder called “Extras”. Take this folder and put it in the

2
DirectX 9.0 SDK folder, which by default should have the location C:\Program Files\Microsoft
DirectX 9.0 SDK (December 2004).
This completes the installation of DirectX. Where DirectX is stored is very important
since later we’ll be specifying paths so Visual C++ 6.0 can find DirectX. Next, we began the
installation of Open CV v4.0-Beta.

2.2. Installing OpenCV v4.0-Beta

In this section, we’ll install OpenCV v4.0-Beta. Even though the library will be installed
at the end of this section, we still won’t be able to use it. Using the library requires setting up
Visual Studio to work with it, which takes some work. The main OpenCV site is
http://www.intel.com/research/mrl/research/opencv/. You should familiarize yourself with this
site. Furthermore, joining Yahoo! OpenCV group is a very smart move; it is a major OpenCV
resource.
From the main OpenCV site, click on “Download” under “Featured Content” on the right
hand side. Then click on the download link, i.e., http://sourceforge.net/projects/opencvlibrary/. We’re
now at the OpenCV project page at SourceForge. The top of the page should read “Project: Open
Computer Vision Library: Summary”. Scroll down to “Latest File Releases”. Find the package
called “opencv-win”, and click the download button. Fig. 2-1 shows which file to download.

Figure 2-1: Download List.

On the next page, click “OpenCV_b4a.exe” under “beta4”. Now, choose a close mirror
site to download the file from. With respective to your selected mirror site, click on the picture of
a binary file under the column labeled “Download”. Click save and save the file to a temporary
folder. A self-extracting zip file of size 12.4 MB called “OpenCV_b4a.exe” will be downloaded.
Double click the file and follow the instructions to install OpenCV v4.0-Beta, which
requires at least 36.2 MB. To make life easier, install OpenCV in the default folder, i.e.,
“C:\Program Files\OpenCV”. Leave the check box under additional tasks checked so the system
will add the specified path to the system PATH variable. (Actually, this feature doesn’t seem to
work in the current release of OpenCV, so we’ll go in and add in manually later.) That’s it! In
the next section, we’ll set up Visual C++ 6.0 to work with OpenCV.

3
 3. Setting Up Visual C++ 6.0 For OpenCV
We now begin the process of setting up Visual C++ 6.0 to work with OpenCV. The
explanation found here is meant for users with minimal experience in Visual Studio, and thus,
details will be carefully explained. We begin with setting up the system PATH variable.

3.1. Setting Up the System PATH Environmental Variable

Although we selected the option to take care of what we’re about to do during
installation, this feature seems to not be working correctly in the current release of OpenCV. But
that’s not a problem; we’ll just do it manually. From the taskbar, click ”start” and then click
“Control Panel”. Switch to “Classic View” (upper left corner) if the current view is “Category
View”. Once in “Classic View”, double click on the “System” icon to bring up the “System
Properties” dialog box. Click the “Advance” tab. And then click the “Environmental Variables”
button towards the bottom of the dialog (Fig. 3-1).

Figure 3-1: “System Properties” Dialog Box.

The “Environmental Variables” dialog box should now be open. Under the second list,
i.e., “System Variables”, find the variable called “Path”. Select it, and then click “Edit” (Fig. 3-
2).

4
 Fig. 3-2: “Environmental Variables” Dialog Box.

Add the path of the OpenCV “bin” folder in the “Variable value” text box. To do this, we
may append the path to the end of the list, making sure that semicolons separate the entries (Fig.
3-3). If the default destination was chosen during installation, your path should be “C:\Program
Files\OpenCV\bin”. If not, find the path to OpenCV’s “bin” folder, and add it in the “Variable
value” text box. Click “OK” on all open dialogs, and close the Control Panel. You must now
restart your computer for the PATH variable to be updated. Now, OpenCV will be able to
find the required DLLs it needs. If you need to run an application on another computer that
doesn’t have OpenCV set up, you’ll have to bring not only the application but the DLLs as
well. This completes setting up the PATH variable. Next, we’ll set up our directory listings.

Figure 3-3: ‘Edit System Variable” Dialog Box.

5
3.2. Specifying Directories in Visual C++ 6.0

Visual C++ 6.0 must be able to find files such as include, source, executable, and library
files. We need to go in and manually tell the IDE where to find the ones it needs.
Start Microsoft Visual C++ 6.0. The programming environment without a loaded
project/file is depicted in Fig. 3-4.

Figure 3-4: Microsoft Visual C++ 6.0 IDE

6
 From the menu, click “Tools”, and then “Options…”, (Fig. 3-5). The “Options” dialog
box should now be shown.

Figure 3-5: Go To “Tools” Then “Options…”.

In the “Options” dialog box, find the “Directories” tab and click it. The “Options” dialog
box with the “Directories” tab selected is shown in Fig. 3-6. Currently, we’re looking at the
directories specified for the include files. You can switch to other file types such as library
files by clicking the arrow under “Show directories for”. Right now, let’s leave it on “Include
files”.
Visual C++ should have it’s include files already set up. If there is no folder for the
include files of DirectX 9.0 SDK (December 2004) listed, then specify it. If you chose the
default folder, your path will be the same as shown in Fig. 3-6. To add another entry, click where
the blue highlight is in Fig. 3-6. Don’t type the entry directly; instead, browse for DirectX’s
“include” folder. And finally, move this entry to the top of the list by clicking, holding, and
dragging it. The entry for the DirectX “include” folder must be at the top of the “Include
files” directory list.

7
 Figure 3-6: “Options” Dialog Box With the “Directories” Tab Selected

We have more directories to specify. Assuming the default destination folder during
installation, add the following directories. (Your directories may be different if you chose to
install OpenCV in a different location. If so, change the paths accordingly.)

• C:\Program Files\OpenCV\cv\include
• C:\Program Files\OpenCV\cxcore\include
• C:\Program Files\OpenCV\otherlibs\highgui

8
 The directories for the needed include files are finished. Next, switch to “Library files”.
The listing for the library files is shown in Fig. 3-7.

Figure 3-7: Library Files Listing.

Visual C++ should have it’s library files already set up. If there is no directory for the
library files of DirectX 9.0 SDK (December 2004) listed, then specify it. If you chose the default
directory, your path will be the same as shown in Fig. 3-7. Go ahead and add it as we did before
if it’s missing. (Don’t forget to move it to the top as well.) Finally, add the directory “C:\Program
Files\OpenCV\lib” to the list of directories.
Switch to “Executable files”. Same as before, check for a DirectX listing. If it is missing,
add “C:\Program Files\Microsoft DirectX 9.0 SDK (December 2004)\Utilities\Bin\x86”, and
move it to the top. Lastly, add “C:\PROGRAM FILES\OPENCV\BIN”.
Specification of the directories is finished. Be sure to click “OK” on the “Options”
dialog box, or you’ll be doing all of this over again. In the next section, we’ll get a project up
and running.

3.3. Creating a Visual C++ Project that Uses OpenCV

Let’s go ahead and get a project up and running. The settings we specify in this section
are specific to a project. If we create another project, these settings will have to be made to that
one as well. However, the settings we made in the previous section, i.e., adding the new
directories, are inherited by all Visual C++ projects.

9
 Start Microsoft Visual C++ 6.0. From the menu, click “File” and then click “New…”
(Fig. 3-8).

Figure 3-8: Go to “File” Then “New…”.

10
 A dialog box for creating a new file/project/etc. will appear. We’re interested in creating
a new project, so click on the “Projects” tab if it’s not already selected. Several fields are on the
right hand side. The following is all depicted in Fig. 3-9. For “Project name”, enter
“imageViewer”. You may choose any location you wish. We recommend a folder on your
desktop called “imageViewer” to make things simple. Check “Create new workspace” if it’s not
checked. Check “Win32” as the platform if it’s not checked. Now, select “Win32 Console
Application” from the project list on the left, as shown in Fig. 3-9. Finally, click “OK”.

Figure 3-9: Dialog Box For Creating New Files/Project/Etc.

11
 A second dialog box will appear asking about what type of console application we wish
to create. Leave “An empty project” selected and click “Finish” (Fig. 3-10). Once “Finish” is
clicked, a skeleton project will be created with no files. Another window will appear informing
you about the successful creation of a skeleton console application project. Click “OK”.

Figure 3-10: “Console Application Type” Dialog Box

12
 Before creating the actual application, we need to modify the project settings to our
liking. From the menu, click “Projects” and then click “Settings…” (Fig. 3-11).

Figure 3-11: Go To “Projects” Then “Settings…”.

13
 Once “Settings…” is clicked, the “Project Settings” dialog box will appear. Under
“Settings For” at the top left, switch to “All Configurations” (Fig. 3-12).

Figure 3-12: All Configurations.

Click the “Link” tab, and then select “General” for the “Category” (Fig. 3-13).

Figure 3-13: Go To “Link” Then Select “General”.

14
 Under “Object/library Modules” (Fig. 3-14), add the following libraries.

• cv.lib
• cxcore.lib
• highgui.lib
• strmbase.lib

Figure 3-14: Object/library modules.

15
 Once the libraries have been entered, click “OK”. Now, we’ll add a file to our project.
From the menu, click “File” and then click “New…”, as before. Instead of the “Projects” tab, we
need the “Files” tab, so click it (Fig. 3-15). From the choices, choose “C++ Source File”. On the
right hand side, enter a file name. (We use “imageViewer” as a file name). The location should
already be specified. “Add to project” should already be checked, and set to “imageViewer”. If
not, change it. Once done, click “OK”.

Figure 3-15: Adding a File.

After clicking “OK”, the file should be open and ready for coding. Type the following
code into the source file.

#include <cv.h> /* required to use OpenCV */

#include <highgui.h> /* required to use OpenCV's highgui */

void main(int argc, char *argv[]) {

IplImage* image=0; /* pointer to an image */

if(argv[1] != 0)
image = cvLoadImage(argv[1], 1); // 1 for color
if(image != 0) {
cvNamedWindow("Display", CV_WINDOW_AUTOSIZE); // create a window
cvShowImage("Display", image); // show image in window
cvWaitKey(0); // wait until user hits a key
cvDestroyWindow("Display");
}
}

16
 After entering the code, compile the program and then build the executable, either a
Debug or Release version. To compile, from the menu click “Build” and then click “Compile
<filename.cpp>” (Fig. 3-16.). Once compilation is complete, from the menu click “Build” and
then click “Build <filename.exe>” (Fig.3-16). If you receive errors, either 1) review the program
to make sure it was entered correctly; 2) skim through the guide to see if you missed any steps or
did any steps incorrectly; or 3) visit section 5 for an outline of popular issues.

Figure 3-16: Compiling and Building

To run this program, the command line must be used. From the taskbar, click “start” and
then click “Run…”. Type “cmd”. Using the “change directory” command (“cd”), change your
current directory to the directory holding your executable. Once you’ve reached the correct
folder, type “imageViewer <filename>”, where filename is the file name of the image you wish
to display. Of course you’ll need an image before you can display one. Probably the easiest way
to get an image if you don’t have one is by simply finding one on the Web (Google’s image
search works fine).
Fig. 3-17 depicts the series of commands we issued to display an image called “lena.jpg”.
Remember that you’ll have to set up your project like we did in this section each time you create
a new one (minus the code of course). In the next section, you’ll “learn how to learn” OpenCV.

17
 Figure 3-17: Viewing Lena, An Image Made Popular By Image Processing Researchers.

4. Getting Started With OpenCV

By now, we’ve installed and set up OpenCV. We’ve even created a simple project. Now,
it’s time to “learn how to learn” OpenCV. A question every single newcomer asks is “Where do I
begin?”. OpenCV is a very large library, and therefore can quickly become overwhelming. This
section attempts help you get started with OpenCV.
Open the “OpenCV” folder. Next, open the “docs” folder. In this folder are a number of
documents. Begin with “faq.htm”. This document answers a number of popular questions. Next,
visit “index.htm”. This is a very important resource that you’ll be visiting time and time again. In
fact, you should make a shortcut to it on your desktop. Carefully read through “index.htm”. At
the end of “index.htm” are four reference manuals, i.e., “CXCORE”, “CV”, “HighGUI”, and
“CVCAM” reference manuals, and an extensive, however outdated, reference manual for the
entire library. Use the first four reference manuals as your APIs when developing applications.
The outdated manual is good for understanding the algorithms used in OpenCV, but not for use
as an API.
What we recommend is to first read through the “CXCORE” reference manual. This
manual is an excellent starting place. Here, you’ll learn about the basic data structures used by
OpenCV; the different operations on Arrays such as accessing elements, copying Arrays, etc.;
dynamic data structures such as sequences and trees; how to draw shapes or display text; and
other topics as well.

18
 The next file to visit is the “HighGUI” reference manual. This API is very useful since it
allows you to use DirectShow without any knowledge of the DirectShow API. Finally, you
should visit the “CV” reference manual. Here you’ll find many of OpenCV’s image processing
and computer vision algorithms, but not all as we’ve found out. If you can’t find an
implementation of an algorithm you’d like to use, check the Yahoo! OpenCV forum, discussed
next, before reinventing the wheel.
The Yahoo! OpenCV forum a truly invaluable resource. It’s very useful for getting help
with OpenCV, or communicating with other computer vision researchers that are using OpenCV.
The last step in our Getting Started section is to visit the “samples” folder, i.e.,
“C:\Program Files\OpenCV\samples\c”. Here, you’ll find a number of useful and interesting
sample programs that you can learn from. The next section gives some advice on
troubleshooting.

5. Troubleshooting
Many things can go wrong whilst setting up Visual C++ 6.0 for OpenCV. Furthermore,
for newcomers it is quite challenging writing the first, somewhat complex, program in OpenCV.
This section attempts to outline many of the problems we experienced during our own process of
learning OpenCV.

5.1. Error: cannot open include file <filename.h>

Obviously, Visual C++ 6.0 cannot find an “include” file. See section 3.2, and make sure
you’ve included those paths mentioned. It’s possible to use other projects besides just these, for
example, “cvcam” or “cvaux”. If you forgot to include the path to a particular project’s “include”
files, you’ll have to add it like we did before. If you’re not sure where an “include” file is, you
may do a search for it (ctrl+f).

5.2. Error: unresolved external symbol <symbol>

Here, there’s a good chance you forgot to include a “library” file in the “Object/library
module” field of the “Project Settings” Dialog Box. See section 3.3, and make sure you’ve used
“strmbase.lib”, “cxcore.lib”, “cv.lib”, and “highgui.lib”, at the minimum. If you want to use
other projects such as “cvcam” or “cvaux”, you’ll of course have to add additional libraries like
we did in section 3.3.

19
5.3. Error: bad number of channels/unsupported format

Popular error many newcomers receive. Basically, the number of channels corresponds to
color channels. For example, 1 is grayscale, 3 is BGR. Certain functions accept only images of a
certain number of channels. Furthermore, functions may accept images of only a particular bit
depth. As an example, Sobel will accept only grayscale images with a bit depth of 16 bits.

5.4. Missing Files

You may be missing a DLL or a library file of a particular project, for example, “cvcam”.
You’ll have to manually go in and compile/build the respective project. These files can be
accessed in the “_make” folder within the “OpenCV” folder. Open up the appropriate workspace
and find the project in question. Make sure you set the project as the active project as well as
setting the configuration of the project to your liking. Now, you’re ready to compile and build.

5.5. Other Hints

Read and make sure you understand the data structures you’re using perfectly.
Furthermore, you should understand precisely what a function accepts, does, and its end result.
Due to seemingly infinite number of problems you might encounter during your use of OpenCV,
we cannot enumerate every situation here. If you cannot find help here, please see the very
helpful “faq.htm”, or the life-saving Yahoo! OpenCV forum.

6. Example Programs
In this section, we’ll cover some example programs to further your knowledge of
OpenCV. Each program requires a web cam or some other video recording device whose video
data can be downloaded in real-time to a computer via a USB cable. The first program is a
simple program to feed in video from a web cam and display it. The next program performs
some spatial processing on the fed in video. The third program performs some temporal
processing on the fed in video. And the fourth program is a tracking application with
visualization of the optical flow field. You should advance in order since the programs are in
order of difficulty. Enjoy!

6.1. Displaying Video From a Web Cam (Web Cam Required)

This simple program will allow video fed in from a web cam to be display on the
monitor. See section 3.3 to learn how to create a project and set it up properly. (Note: This file
need not be run from the command line as we did before.)

20
#include <cv.h>
#include <highgui.h>

void main(int argc,char *argv[])

{
int c;
IplImage* color_img;
CvCapture* cv_cap = cvCaptureFromCAM(-1); // -1 = only one cam or doesn't matter

cvNamedWindow("Video",1); // create window

for(;;) {
color_img = cvQueryFrame(cv_cap); // get frame
if(color_img != 0)
cvShowImage("Video", color_img); // show frame
c = cvWaitKey(10); // wait 10 ms or for key stroke
if(c == 27)
break; // if ESC, break and quit
}
/* clean up */
cvReleaseCapture( &cv_cap );
cvDestroyWindow("Video");
}

6.2. Edge Detection (Web Cam Required)

This program applies a spatial filter, namely the Sobel operator, to detect vertical edges.
As an aside, you may modify the filter if you wish to detect horizontal edges. Furthermore, it’s
possible to use Sobel to detect both types of edges. See section 3.3 to learn how to create a
project and set it up properly. (Note: This file need not be run from the command line as we did
before.)

#include <cv.h>
#include <highgui.h>

void main(int argc,char *argv[])

{
int c;
IplImage* color_img=0;
IplImage *gray_img=0;
IplImage *sobel_img16=0;
IplImage *sobel_img8=0;
CvCapture* cv_cap = cvCaptureFromCAM(-1);

/* create windows */
cvNamedWindow("Video", CV_WINDOW_AUTOSIZE);
cvNamedWindow("Edges", CV_WINDOW_AUTOSIZE);

/* get a frame and use it to allocate space and define specs for other images */
color_img = cvQueryFrame(cv_cap);

21
 if(!color_img)
return;
gray_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
sobel_img16 = cvCreateImage( cvSize(color_img->width,color_img->height),
IPL_DEPTH_16S, 1);
sobel_img8 = cvCreateImage( cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
/* origin (start) of image must be the same */
gray_img->origin = color_img->origin;
sobel_img16->origin = color_img->origin;
sobel_img8->origin = color_img->origin;
for(;;) {
color_img = cvQueryFrame(cv_cap); // get a frame
if(!color_img)
break;
/* convert to grayscale */
cvCvtColor(color_img, gray_img, CV_BGR2GRAY);
/* apply Sobel operator; requires as 16 bit destination */
cvSobel(gray_img, sobel_img16, 1, 0, 3);
/* display requires an 8 bit image, so we must convert it */
cvConvertScale(sobel_img16, sobel_img8, 1, 0);

/* display images */
cvShowImage("Edges",sobel_img8);
cvShowImage("Video", color_img);
c = cvWaitKey(10); // wait for 10 ms or key stroke
if(c == 27)
break; // if ESC, break and quit
}
/* clean up */
cvReleaseCapture(&cv_cap);
cvDestroyAllWindows();
}

6.3. Motion Detection (Web Cam required)

This next program performs some temporal processing on video data to detect motion.
This program basically computes the difference between two frames. See section 3.3 to learn
how to create a project and set it up properly. (Note: This file need not be run from the command
line as we did before.)

22
#include <cv.h>
#include <highgui.h>
#include <math.h>

void main(int argc,char *argv[])

{
int c, i, j;
IplImage* color_img=0;
IplImage* gray_img=0;
IplImage* prev_img=0;
IplImage* curr_img=0;
IplImage* difference=0;
CvCapture* cv_cap = cvCaptureFromCAM(-1);

/* Create windows */
cvNamedWindow("Video", CV_WINDOW_AUTOSIZE);
cvNamedWindow("Output", CV_WINDOW_AUTOSIZE);

/* Allocate space for other images */

color_img = cvQueryFrame(cv_cap);
if(!color_img)
return;
gray_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
prev_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
curr_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
difference = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
gray_img->origin = color_img->origin;
prev_img->origin = color_img->origin;
curr_img->origin = color_img->origin;
difference->origin = color_img->origin;

/* convert to grayscale */
cvCvtColor( color_img, gray_img, CV_BGR2GRAY );
/* get a current image now so we'll have a previous image when we begin */
cvCopy(gray_img, curr_img, 0);
for(;;) {
color_img = cvQueryFrame(cv_cap); // get a frame
if(!color_img)
break;
/* convert to grayscale */
cvCvtColor( color_img, gray_img, CV_BGR2GRAY );
/* current becomes previous and new frame because current */
cvCopy(curr_img, prev_img, 0);
cvCopy(gray_img, curr_img, 0);

23
 /* compute the difference */
for(i = 0; i < color_img->height; i++)
for(j = 0; j < color_img->width; j++)
(difference->imageData + i*color_img->width)[j] =
abs((prev_img->imageData + i*color_img->width)[j] -
(curr_img->imageData + i*color_img->width)[j]);

/* show images */
cvShowImage("Video", color_img);
cvShowImage("Output", difference);
c = cvWaitKey(10); // wait for 10 ms or keystroke
if(c == 27) // ESC
break;
}
/* clean up */
cvReleaseCapture(&cv_cap);
cvDestroyWindow("Video");
cvDestroyWindow("Output");
}

6.4. Pyramidal Lucas-Kanade Motion Tracker with Optical Flow Field

Visualization (Web cam Required)

Tracker based on motion using a famous algorithm. Click on the video to create “track
balls”. The display with the arrows is the optical flow field. Bigger arrows indicate larger
displacements/velocities. See section 3.3 to learn how to create a project and set it up properly.
(Note: This file need not be run from the command line as we did before.)

#include <cv.h>
#include <highgui.h>
#include <math.h>
#include <stdio.h>

const int MAX_COUNT = 2000;

int count = 0, startOfTrackPoints = 0;
CvPoint2D32f* points[2] = {0,0};
IplImage *color_img=0;

/* Listener for mouse clicks */

void OnMouseClick(int event, int x, int y, int flags)
{
if(color_img->origin)
y = color_img->height - y;
if(event == CV_EVENT_LBUTTONDOWN && count < MAX_COUNT) {
points[1][count] = points[0][count] = cvPoint2D32f(x, y); // Add point
count++;
}
}

24
void main(int argc,char *argv[])
{
int c, i, j, step, win_size, flags, color;
char* status = 0;
CvCapture* cv_cap = cvCaptureFromCAM(-1);
IplImage *gray_img, *prev_img, *curr_img;
IplImage *pyramid, *prev_pyramid, *optical_flow_field;
CvPoint p1, p2, p3, p4, p5;

printf("Welcome to Troy's LK demo.\n");

printf("Click on the 'Video' window to add trackballs.\n");
printf("The 'Field' window shows the optical flow with larger \n");
printf("arrows corresponding to larger displacements.\n\n");

printf("Keyboard commands:\n");
printf(" ESC to quit\n");
printf(" c to clear all the trackballs\n");
printf(" n to cycle through the different trackball colors\n");

/* Initialization and space allocation */

color_img = cvQueryFrame(cv_cap);
if(!color_img)
return;
gray_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
optical_flow_field = cvCreateImage( cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
prev_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
curr_img = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
pyramid = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
prev_pyramid = cvCreateImage(cvSize(color_img->width,color_img->height),
IPL_DEPTH_8U, 1);
gray_img->origin = color_img->origin;
optical_flow_field->origin = color_img->origin;
prev_img->origin = color_img->origin;
curr_img->origin = color_img->origin;
pyramid->origin = color_img->origin;
prev_pyramid->origin = color_img->origin;
points[0] = (CvPoint2D32f*)cvAlloc(MAX_COUNT*sizeof(points[0][0]));
points[1] = (CvPoint2D32f*)cvAlloc(MAX_COUNT*sizeof(points[0][0]));
status = (char*)cvAlloc(MAX_COUNT);
flags = 0;
step = 10;
win_size = 10;
color = 0;

25
/* Create windows */
cvNamedWindow("Video", CV_WINDOW_AUTOSIZE);
cvNamedWindow("Field", CV_WINDOW_AUTOSIZE);

/* Set up mouse event listener */

cvSetMouseCallback("Video", OnMouseClick);

/* Initialize the points to the points on the image separated

by distances of 10 pixels */
for(i = 0; i < color_img->height; i+=step) {
for(j = 0; j < color_img->width; j+=step) {
points[0][count] = cvPoint2D32f(j, i);
points[1][count] = cvPoint2D32f(j, i);
count++; // count the number of points
}
}

/* Where track points begin */

startOfTrackPoints = count;

/* Use the "first" frame */

color_img = cvQueryFrame(cv_cap);
if(!color_img)
return;
cvCvtColor( color_img, gray_img, CV_BGR2GRAY );
/* Smooth to improve accuracy */
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvCopy(curr_img, prev_img, 0);
cvCopy(gray_img, curr_img, 0);

/* Begin the main infinite loop until break */

for(;;) {
color_img = cvQueryFrame(cv_cap);
if(!color_img)
break;
cvCvtColor( color_img, gray_img, CV_BGR2GRAY );
/* Smooth to improve accuracy */
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvSmooth(gray_img, gray_img, CV_GAUSSIAN, 3, 0, 0);
cvCopy(curr_img, prev_img, 0);
cvCopy(gray_img, curr_img, 0);
cvCopy(pyramid, prev_pyramid, 0);
for(i = startOfTrackPoints; i < count; i++)
points[0][i] = points[1][i];

26
/* Calculate the optical flow using pyramidal iterative LK */
cvCalcOpticalFlowPyrLK(prev_img, curr_img, prev_pyramid, pyramid,
points[0], points[1], count, cvSize(win_size, win_size), 3, status, 0,
cvTermCriteria(CV_TERMCRIT_ITER|CV_TERMCRIT_EPS,20,0.03), flags);

/* Threshold to ignore noise */

for(i = startOfTrackPoints; i < count; i++)
if(abs((int)(points[0][i].x - points[1][i].x)) > 50 ||
abs((int)(points[0][i].y - points[1][i].y)) > 50)
points[1][i] = points[0][i];

/* Set flags */
flags |= CV_LKFLOW_PYR_A_READY;

/* Refresh the optical flow field */

for (i = 0; i < optical_flow_field->height; i++)
for (j = 0; j < optical_flow_field->width; j++)
(optical_flow_field->imageData+
optical_flow_field->widthStep*i)[j] = (char)255;

/* Draw the optical flow field; the greater the velocity, the bigger the
respective arrow and arrowhead will be for better visualization */
for (i = 0; i < startOfTrackPoints; i++) {
/* Threshold velocities: not drawing very small or very large velocities */
if((abs((int)(points[0][i].x - points[1][i].x)) >= 1 ||
abs((int)(points[0][i].y - points[1][i].y)) >= 1) &&
(abs((int)(points[0][i].x - points[1][i].x)) <= 20 &&
abs((int)(points[0][i].y - points[1][i].y)) <= 20)) {
/* Arrow */
cvLine(optical_flow_field, cvPointFrom32f(points[0][i]),
cvPointFrom32f(points[1][i]), cvScalar(0, 0, 0,0), 1, 8, 0);
/* Arrowhead */
p1 = cvPoint((int)(cvPointFrom32f(points[0][i]).x +
0.7*(cvPointFrom32f(points[1][i]).x -
cvPointFrom32f(points[0][i]).x)),
(int)(cvPointFrom32f(points[0][i]).y +
0.7*(cvPointFrom32f(points[1][i]).y -
cvPointFrom32f(points[0][i]).y)));
p2 = cvPoint((int)(cvPointFrom32f(points[0][i]).x +
0.7*(cvPointFrom32f(points[1][i]).x -
cvPointFrom32f(points[0][i]).x)),
(int)cvPointFrom32f(points[1][i]).y);
p3 = cvPoint((int)cvPointFrom32f(points[1][i]).x,
(int)(cvPointFrom32f(points[0][i]).y +
0.7*(cvPointFrom32f(points[1][i]).y -
cvPointFrom32f(points[0][i]).y)));
p4 = cvPoint((int)(p1.x + 0.5*(p2.x-p1.x)),
(int)(p1.y + 0.5*(p2.y-p1.y)));
p5 = cvPoint((int)(p1.x + 0.5*(p3.x-p1.x)),
(int)(p1.y + 0.5*(p3.y-p1.y)));
cvLine(optical_flow_field, cvPointFrom32f(points[1][i]),

27
 p4, cvScalar(0, 0, 0,0), 1, 8, 0);
cvLine(optical_flow_field, cvPointFrom32f(points[1][i]),
p5, cvScalar(0, 0, 0,0), 1, 8, 0);
}
else {
/* Draws a dot */
cvLine(optical_flow_field, cvPointFrom32f(points[0][i]),
cvPointFrom32f(points[0][i]), cvScalar(0, 0, 0,0), 1, 8, 0);
}
}
for(i = startOfTrackPoints; i < count; i++) {
/* Draw trackballs */
if(color == 0)
cvCircle(color_img, cvPoint((int)points[1][i].x, (int)points[1][i].y),
5, cvScalar(0,255,0,0), 3, 8, 0);
else if(color == 1)
cvCircle(color_img, cvPoint((int)points[1][i].x, (int)points[1][i].y),
5, cvScalar(255,0,0,0), 3, 8, 0);
else
cvCircle(color_img, cvPoint((int)points[1][i].x, (int)points[1][i].y),
5, cvScalar(0,255,255,0), 3, 8, 0);
}

/* Show images */
cvShowImage("Video", color_img);
cvShowImage("Field", optical_flow_field);

/* Wait */
c = cvWaitKey(10);
/* Check for ESC */
if(c == 27)
break;
else if(c == 'c')
startOfTrackPoints = count;
else if(c == 'n') {
color++;
color = color % 3;
}

/* Release windows */
cvReleaseCapture(&cv_cap);
cvDestroyWindow("Video");
cvDestroyWindow("Field");
}

28
 7. Future Work
As newer versions of OpenCV appear, we’ll try to keep this guide updated as much as
possible. Also, we’ll continue expanding the troubleshooting section as we run into other
problems many others are experience. Problems will include not only installation and setup
problems, but problems dealing with computer vision algorithms in the OpenCV library.

8. Final Words
We hope this guide has helped you. If we help at least one person, then it was worth
writing. Thanks and happy OpenCV-ing!

29