A Basic Introduction

to OpenCV for Image

Processing
Qing Chen (David)
E-mail: qchen@discover.uottawa.ca
DiscoverLab
School of Information Technology & Engineering
University of Ottawa
J anuary 15, 2007
2
Outline
1. Introduction
2. Image data structure in OpenCV
3. Basic operations for images
4. Working with videos
5. References and resources
3
1. Introduction
General description
Open source computer vision library in C/C++.
Optimized and intended for real-time applications.
OS/hardware/window-manager independent.
Generic image/video loading, saving, and acquisition.
Both low and high level API.
Provides interface to Intel's Integrated Performance
Primitives (IPP) with processor specific optimization
(Intel processors).
4
1. Introduction
Features:
Image data manipulation (allocation, release, copying, setting, conversion).
Image and video I/O (file and camera based input, image/video file output).
Matrix and vector manipulation and linear algebra routines.
Various dynamic data structures (lists, queues, sets, trees, graphs).
Basic image processing (filtering, edge detection, corner detection, sampling
and interpolation, color conversion, morphological operations, histograms,
image pyramids).
Structural analysis (connected components, contour processing, distance
transform, various moments, template matching, Hough transform, polygonal
approximation, line fitting, ellipse fitting, Delaunay triangulation).
Camera calibration (finding and tracking calibration patterns, calibration,
fundamental matrix estimation, homography estimation, stereo correspondence).
Motion analysis (optical flow, motion segmentation, tracking).
Object recognition (eigen-methods, HMM).
Basic GUI (display image/video, keyboard and mouse handling, scroll-bars).
Image labeling (line, conic, polygon, text drawing).
5
1. Introduction
OpenCV modules:
cv - Main OpenCV functions.
cvaux - Auxiliary (experimental) OpenCV
functions.
cxcore - Data structures and linear algebra
support.
highgui - GUI functions.
6
2. Image data structure in OpenCV
Load and display an image in OpenCV:
#i ncl ude " cv. h" / / mai n OpenCV f unct i ons
#i ncl ude " hi ghgui . h" / / OpenCV GUI f unct i onsi ncl ude <st di o. h>
i nt mai n( )
{
/ * decl ar e a new I pl I mage poi nt er , t he basi c
i mage dat a st r uct ur e i n OpenCV */
IplImage* newImg;
/ * l oad an i mage named " appl e. bmp" , 1 means
t hi s i s a col or i mage */
newImg = cvLoadImage("apple.bmp",1);
/ / cr eat e a new wi ndow
cvNamedWi ndow( " Wi ndow" , 1) ;
/ / di spl ay t he i mage i n t he wi ndow
cvShowI mage( " Wi ndow" , newI mg) ;
/ / wai t f or key t o cl ose t he wi ndow
cvWai t Key( 0) ;
cvDest r oyWi ndow( " Wi ndow" ) ; / / dest r oy t he wi ndow
cvRel easeI mage( &newI mg ) ; / / r el ease t he memor y f or t he i mage
r et ur n 0;
}
7
2. Image data structure in OpenCV
IplImage is the basic image data structure in OpenCV
8
3. Basic operations for images
Threshold
#i ncl ude "cv. h"
#i ncl ude "hi ghgui . h"
#i ncl ude "mat h. h"
i nt mai n( )
{
I pl I mage* sr c;
I pl I mage* col or Thr esh;
I pl I mage* gr ay;
I pl I mage* gr ayThr esh;
i nt t hr eshol d = 120, maxVal ue = 255;
i nt t hr eshol dType = CV_THRESH_BI NARY;
sr c = cvLoadI mage( "appl e. bmp", 1) ;
col or Thr esh = cvCl oneI mage( sr c ) ;
gr ay = cvCr eat eI mage( cvSi ze( sr c- >wi dt h, sr c- >hei ght ) , I PL_DEPTH_8U, 1 ) ;
cvCvt Col or ( sr c, gr ay, CV_BGR2GRAY ) ;
gr ayThr esh = cvCl oneI mage( gr ay ) ;
cvNamedWi ndow( " sr c", 1 ) ; cvShowI mage( "sr c", sr c ) ;
cvNamedWi ndow( " gr ay", 1 ) ; cvShowI mage( "gr ay", gr ay ) ;
cvThreshold(src, colorThresh, threshold, maxValue, thresholdType);
cvThreshold(gray, grayThresh, threshold, maxValue, thresholdType);
cvNamedWi ndow( " col or Thr esh" , 1 ) ; cvShowI mage( " col or Thr esh" , col or Thr esh ) ;
cvNamedWi ndow( " gr ayThr esh", 1 ) ; cvShowI mage( "gr ayThr esh", gr ayThr esh ) ;
cvWai t Key( 0) ;
cvDest r oyWi ndow( "sr c" ) ;
cvDest r oyWi ndow( "col or Thr esh" ) ;
cvDest r oyWi ndow( "gr ay" ) ;
cvDest r oyWi ndow( "gr ayThr esh" ) ;
cvRel easeI mage( &sr c ) ;
cvRel easeI mage( &col or Thr esh ) ;
cvRel easeI mage( &gr ay ) ;
cvRel easeI mage( &gr ayThr esh ) ;
r et ur n 0;
}
9
3. Basic operations for images
Canny edge detection
#i ncl ude "cv. h"
#i ncl ude "hi ghgui . h"
i nt mai n( )
{
I pl I mage* newI mg; / / or i gi nal i mage
I pl I mage* gr ayI mg; / / gr ay i mage f or t he conver si on of t he or i gi nal i mage
I pl I mage* cannyI mg; / / gr ay i mage f or t he canny edge det ect i on
/ / l oad or i gi nal i mage
newI mg = cvLoadI mage( " appl e. bmp", 1) ;
/ / cr eat e a si ngl e channel 1 byt e i mage ( i . e. gr ay- l evel i mage)
gr ayI mg = cvCr eat eI mage( cvSi ze( newI mg- >wi dt h, newI mg- >hei ght ) , I PL_DEPTH_8U, 1 ) ;
/ / conver t or i gi nal col or i mage ( 3 channel r gb col or i mage) t o gr ay- l evel i mage
cvCvt Col or ( newI mg, gr ayI mg, CV_BGR2GRAY ) ;
cannyI mg = cvCr eat eI mage( cvGet Si ze( newI mg) , I PL_DEPTH_8U, 1) ;
/ / canny edge det ect i on
cvCanny(grayImg, cannyImg, 50, 150, 3);
cvNamedWi ndow( "sr c" , 1) ;
cvNamedWi ndow( "canny", 1) ;
cvShowI mage( " sr c", newI mg ) ;
cvShowI mage( " canny" , cannyI mg ) ;
cvWai t Key( 0) ;
cvDest r oyWi ndow( "sr c" ) ;
cvDest r oyWi ndow( "canny" ) ;
cvRel easeI mage( &newI mg ) ;
cvRel easeI mage( &gr ayI mg ) ;
cvRel easeI mage( &cannyI mg ) ;
r et ur n 0;
}
10
3. Basic operations for images
Contour detection
#i ncl ude "cv. h"
#i ncl ude "cxcor e. h"
#i ncl ude "hi ghgui . h"
i nt mai n( )
{
I pl I mage* newI mg = NULL;
I pl I mage* gr ayI mg = NULL;
I pl I mage* cont our I mg = NULL;
/ / par amet er s f or t he cont our det ect i on
CvMemSt or age * st or age = cvCr eat eMemSt or age( 0) ;
CvSeq * cont our = 0;
i nt mode = CV_RETR_EXTERNAL;
mode = CV_RETR_CCOMP; / / det ect bot h out si de and i nsi de cont our
cvNamedWi ndow( "sr c", 1) ;
cvNamedWi ndow( "cont our ", 1) ;
/ / l oad or i gi nal i mage
newI mg = cvLoadI mage( "appl ebw. bmp", 1) ;
/ / cr eat e a si ngl e channel 1 byt e i mage ( i . e. gr ay- l evel i mage)
gr ayI mg = cvCr eat eI mage( cvSi ze( newI mg- >wi dt h, newI mg- >hei ght ) , I PL_DEPTH_8U, 1 ) ;
/ / conver t or i gi nal col or i mage ( 3 channel r gb col or i mage) t o gr ay- l evel i mage
cvCvt Col or ( newI mg, gr ayI mg, CV_BGR2GRAY ) ;
cvShowI mage( "sr c" , newI mg ) ;
/ / make a copy of t he or i gi nal i mage t o dr aw t he det ect ed cont our
cont our I mg = cvCr eat eI mage( cvGet Si ze( newI mg) , I PL_DEPTH_8U, 3) ;
cont our I mg=cvCl oneI mage( newI mg ) ;
/ / f i nd t he cont our
cvFindContours(grayImg, storage, &contour, sizeof(CvContour), mode, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
/ / dr aw t he cont our
cvDrawContours(contourImg, contour, CV_RGB(0, 255, 0), CV_RGB(255, 0, 0), 2, 2, 8);
cvShowI mage( "cont our ", cont our I mg ) ;
cvWai t Key( 0) ;
cvDest r oyWi ndow( " sr c" ) ; cvDest r oyWi ndow( "cont our " ) ;
cvRel easeI mage( &newI mg ) ; cvRel easeI mage( &gr ayI mg ) ; cvRel easeI mage( &cont our I mg ) ;
cvRel easeMemSt or age( &st or age) ;
r et ur n 0;
}
11
3. Basic operations for images
Dilate/Erode
#i ncl ude "cv. h"
#i ncl ude "cxcor e. h"
#i ncl ude "hi ghgui . h"
i nt mai n( )
{
I pl I mage* newI mg = NULL;
I pl I mage* di l at eI mg = NULL;
I pl I mage* er odeI mg = NULL;
cvNamedWi ndow( "sr c", 1) ;
cvNamedWi ndow( "di l at e", 1) ;
cvNamedWi ndow( "er ode", 1) ;
/ / l oad or i gi nal i mage
newI mg = cvLoadI mage( "appl e. bmp", 1) ;
cvShowI mage( "sr c" , newI mg ) ;
/ / make a copy of t he or i gi nal i mage
di l at eI mg=cvCl oneI mage( newI mg ) ;
er odeI mg=cvCl oneI mage( newI mg ) ;
/ / di l at e i mage
cvDilate(newImg,dilateImg,NULL,4);
/ / er ode i mage
cvErode(newImg,erodeImg,NULL,4);
cvShowI mage( "di l at e", di l at eI mg ) ;
cvShowI mage( "er ode", er odeI mg ) ;
cvWai t Key( 0) ;
cvDest r oyWi ndow( " sr c" ) ; cvDest r oyWi ndow( "di l at e" ) ; cvDest r oyWi ndow( "er ode" ) ;
cvRel easeI mage( &newI mg ) ; cvRel easeI mage( &di l at eI mg ) ; cvRel easeI mage( &er odeI mg ) ;
r et ur n 0;
}
12
3. Basic operations for images
Flood and Fill
#i ncl ude "cv. h"
#i ncl ude "cxcor e. h"
#i ncl ude "hi ghgui . h"
i nt mai n( )
{
I pl I mage* newI mg = NULL;
I pl I mage* f f I mg = NULL;
/ / f l ood and f i l l par amet er s
i nt l o_di f f , up_di f f ; / / t he l ow and up f l ood r andge whi ch can be adj ust ed
CvConnect edComp comp;
CvPoi nt f l oodSeed; / / t he or i gi nal pi xel wher e t he f l ood begi ns
CvScal ar f l oodCol or ;
l o_di f f =8;
up_di f f =8;
f l oodCol or = CV_RGB( 255, 0, 0 ) ; / / set t he f l ood col or t o r ed
cvNamedWi ndow( "sr c", 1) ;
cvNamedWi ndow( "f l ood&f i l l ", 1) ;
/ / l oad or i gi nal i mage
newI mg = cvLoadI mage( "appl e. bmp", 1) ;
cvShowI mage( "sr c" , newI mg ) ;
/ / make a copy of t he or i gi nal i mage
f f I mg=cvCl oneI mage( newI mg ) ;
f l oodSeed=cvPoi nt ( 60, 60) ; / / f l oodi ng st ar t f r ompi xel ( 60, 60)
/ / Fl ood and Fi l l f r ompi xel ( 60, 60) wi t h col or r ed and t he f l ood r ange of ( - 8, +8)
cvFloodFill( ffImg, floodSeed, floodColor, CV_RGB( lo_diff, lo_diff, lo_diff ),
CV_RGB( up_diff, up_diff, up_diff ), &comp, 8, NULL);
cvShowI mage( "f l ood&f i l l ", f f I mg ) ;
cvWai t Key( 0) ;
cvDest r oyWi ndow( " sr c" ) ; cvDest r oyWi ndow( "f l ood&f i l l " ) ;
cvRel easeI mage( &newI mg ) ; cvRel easeI mage( &f f I mg ) ;
r et ur n 0;
}
13
3. Basic operations for images
Rotate and Scale
#i ncl ude "cv. h"
#i ncl ude "hi ghgui . h"
#i ncl ude "mat h. h"
i nt mai n( )
{
I pl I mage* sr c;
I pl I mage* dst ;
i nt del t a;
i nt angl e;
sr c = cvLoadI mage( "appl e. bmp", 1) ;
dst = cvCl oneI mage( sr c ) ;
del t a = 1; angl e = 0;
cvNamedWi ndow( "sr c", 1 ) ;
cvShowI mage( "sr c" , sr c ) ;
f or ( ; ; )
{
f l oat m[ 6] ;
doubl e f act or = ( cos( angl e*CV_PI / 180. ) + 1. 1) *3;
CvMat M = cvMat ( 2, 3, CV_32F, m) ;
i nt w = sr c- >wi dt h;
i nt h = sr c- >hei ght ;
m[ 0] = ( f l oat ) ( f act or *cos( - angl e*2*CV_PI / 180. ) ) ;
m[ 1] = ( f l oat ) ( f act or *si n( - angl e*2*CV_PI / 180. ) ) ;
m[ 2] = w*0. 5f ;
m[ 3] = - m[ 1] ;
m[ 4] = m[ 0] ;
m[ 5] = h*0. 5f ;
cvGetQuadrangleSubPix( src, dst, &M, 1, cvScalarAll(0));
cvNamedWi ndow( "dst " , 1 ) ; cvShowI mage( "dst ", dst ) ;
i f ( cvWai t Key( 5) == 27 )
br eak;
angl e = ( angl e + del t a) %360;
}
r et ur n 0;
}
14
4. Working with videos
Video capture from a file:
CvCapture* cvCaptureFromFile( const char* filename );
Video capture from a camera:
CvCapture* cvCaptureFromCAM( int index );
example:
/ / capt ur e f r omvi deo devi ce #0
CvCapt ur e* capt ur e = cvCapt ur eFr omCAM( 0) ;
15
4. Working with videos
Grab a frame:
cvGrabFrame( CvCapture* capture );
Get the image grabbed with cvGrabFrame:
cvRetrieveFrame( CvCapture* capture );
example:
I pl I mage* i mg = 0;
i f ( ! cvGr abFr ame( capt ur e) ) { / / capt ur e a f r ame
pr i nt f ( "Coul d not gr ab a f r ame\ n\ 7") ;
exi t ( 0) ; }
/ / r et r i eve t he capt ur ed f r ame
i mg=cvRet r i eveFr ame( capt ur e) ;
Release the capture source:
cvReleaseCapture(&capture);
For a better understanding of video processing with OpenCV, refer to
the face detection example under the dir: C:\Program
Files\OpenCV\samples\c\facedetect.c
16
5. References and resources
http://www.intel.com/technology/computing/opencv/index.htm
OpenCV official webpage.
http://opencvlibrary.sourceforge.net/
OpenCV documentation and FAQs.
http://tech.groups.yahoo.com/group/OpenCV/
OpenCV forum at Yahoo Groups.
http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/cvis
ion.htm
This is a good walkthrough for OpenCV and the Microsoft
DirectShow technology by Prof. Robert Laganire of university of
Ottawa. The configuration of OpenCV for MS .Net is also included.
http://ai.stanford.edu/~dstavens/cs223b/stavens_opencv_optical_flo
w.pdf
This is another OpenCV introduction focused on Optical Flow, the
installation of OpenCV is also included.
17
About myself
I am a Ph.D. candidate at the DiscoverLab, School of Information
Technology and Engineering, University of Ottawa. My general
research interests include image processing and computer vision.
My current research topic is focused on real-time vision-based hand
gesture recognition for Human Computer Interface (HCI).
For more information about me, please refer to my webpage:
http://www.discover.uottawa.ca/~qchen