Food Living Outside Play Technology Workshop

Face detection and tracking with Arduino and OpenCV

by techbitar on December 9, 2011

Table of Contents

Face detection and tracking with Arduino and OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Intro: Face detection and tracking with Arduino and OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

File Downloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Step 1: Installation and integration issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Step 2: Attach servos and camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Step 3: Wiring the parts together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Step 4: Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Related Instructables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/
Intro: Face detection and tracking with Arduino and OpenCV
INTRODUCTION

In this project I have assembled a face detection and tracking system. You can see the video of the final project here: http://youtu.be/ZaV_aVZL87o (for some reason I
don't see the embed video button anymore)

Basically, the webcam sends video frames to OpenCV running on a Windows PC. If OpenCV detects a face it will track it and calculate its center's X,Y coordinates. The
coordinates are then passed on to the Arduino via a serial USB connection. The Arduino controls the movement of the webcam with the help of two pan/tilt servos to
follow the detected face.

OpenCV (Open Source Computer Vision Library: http://opencv.willowgarage.com/wiki/ ) is an open-source library that includes several hundreds of real-time computer
vision algorithms. The OpenCV 2.x library is a C++ API.

This is an integration project between hardware and software tools. The image processing C++ code samples are provided with the openCV library and all I did was to
modify the sample code for this project. I removed some of the unnecessary code and added serial communications to it so it can send X,Y values to Arduino.

CREDIT

This project would not have been possible without the team who developed OpenCV. I also benefited from Ryan Owens' tutorial (http://www.sparkfun.com/tutorials/304)
which is based on Processing, OpenFrameworks, and an earlier version of OpenCV. I tried to install Processing and OpenFrameworks with no luck. So Instead, I opted
for Microsoft Visual C++ 2010 Express and the latest version of OpenCV which is 2.3.1 without any middleware or wrappers.

TOOLS

Software Required

Arduino IDE 1.0 for Windows

OpenCV 2.3.1 SuperPack For Windows
Microsoft Visual C++ 2010 Express SP1
Serial C++ Library for Win32 (by Thierry Schneider)

Code Required

- OpenCV C++ (attached) techbitarFaceDetection.cpp (based on OpenCV's example facedetect.cpp)

- Arduino's (attached) cam_servo.ino (based on Ryan Owens' example SerialServoControl.pde)

Hardware Required

- PC preferably running Windows 7 SP1. The faster CPU the better.

- Arduino Uno or compatible + power source.
- Standard servos X 2.
- Webcam w/UBS interface.
- Breadboard.
- Jumper wires.
- Hobby wire to tie pan/tilt servos and webcam together.

Image Notes
1. This contraption here is just deadweight to pin down the servo/cam assembly so it does not move randomly when the servos turn.

File Downloads

cam_servo.ino (3 KB)
[NOTE: When saving, if you see .tmp as the file ext, rename it to 'cam_servo.ino']

techbitarFaceDetection.cpp (3 KB)
[NOTE: When saving, if you see .tmp as the file ext, rename it to 'techbitarFaceDetection.cpp']
http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/
Step 1: Installation and integration issues
1) Download and install the OpenCV-2.3.1-win-superpack.exe if you don't wish to deal with generating the support files yourself. Everything you need from OpenCV to
build this project has already been generated in this download.

http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.3.1/

2) Download and install Microsoft Visual C++ 2010 Express

http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express

The OpenCV installation documentation explains how to make Visual C++ aware of the OpenCV support files (include, bin, etc). This is not a one-click job.
Careful attention must be given to how Visual C++ must be configured to recognize OpenCV files.

The OpenCV team tested version 2.3.1 and Visual C++ 2010 on Windows 7 SP1. If you are using a different configuration, be prepared for a few hiccups.

Step 2: Attach servos and camera

I did not want to affix any of the project parts permanently because I like to take my projects apart after I am done. So I used hobby wire, which is nothing more than a stiff
wire, to tie the servos and the webcam together.

I wrapped the base of the webcam to the pan servo horn. Then I wrapped a cable around the horn of the tilt servo and the body of the pan servo.

To keep the whole servo/webcam assembly from moving randomly during operation, I used a clamp to tie it down to the soldering helping hands. It kept is steady during
servo rotation.

It ain't pretty but it works.

http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/
 Image Notes
1. I used hobby wire to wrap the base of the webcam to the pan servo.
2. I used hobby wire to wrap the base of the webcam to the pan servo's horn.
3. I used hobby wire to wrap the pan servo to the tilt servo's horn.
4. To pin the servos and cam down, I tied the assembly to a heavy clamp and the solder helper hands.
5. To pin the servos and cam down, I tied the assembly to a heavy clamp and the solder helper hands.
6. To pin the servos and webcam down, I tied the assembly to a heavy clamp and the soldering helping hands.

Step 3: Wiring the parts together

The wiring is straight forward. I used a breadboard to make the connections.

SERVOS
The yellow/signal wire for the pan (X axis) servo goes to digital pin 9.
The yellow/signal wire for the tilt (y axis) servo goes to digital pin 10.
The red/Vcc wires of both servos go to the Arduino's 5V pin.
The black/GND wires of both servos go to Arduino's GND pin.

WEBCAM
The webcam's USB goes to the PC. The C++ code will identify it via a number representing the USB port its connected to. .

ARDUINO
The Arduino Uno is connected to the PC via USB. Take note of the COM port the USB is connected to. You can find the com port from the Arduino Tools/Serial Ports
menu. You will see a check mark next to the active USB port. This is the com port that you will use in your C++ code to communicate with Arduino.

You must change the C++ code to match the PC's com port and baud rate with Arduino's. Also, the C++ code must be told which USB port the webcam is
using.

http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/
Step 4: Resources
I found these websites to be informative:

OpenCV Face Detection/Tracking Guides

Face Tracking with a Pan/Tilt Servo Bracket by zagGrad

http://www.sparkfun.com/tutorials/304

Arduino + Servo + openCV Tutorial [openFrameworks]

http://www.creativeapplications.net/tutorials/arduino-servo-opencv-tutorial-openframeworks

OpenCV & Microsoft Visual C++ Integration

Getting Started with OpenCV 2.3 in Microsoft Visual Studio 2010 in Windows
http://siddhantahuja.wordpress.com/2011/07/18/getting-started-with-opencv-2-3-in-microsoft-visual-studio-2010-in-windows-7-64-bit

OpenCV 2.1.0 with Visual Studio 2010

http://blog.aguskurniawan.net/post/OpenCV-210-with-Visual-Studio-2010.aspx

How to build applications with OpenCV inside the Microsoft Visual Studio
http://opencv.itseez.com/doc/tutorials/introduction/windows_visual_studio_Opencv/windows_visual_studio_Opencv.html

Using OpenCV 2.3.1 with Visual Studio 2010 (tutorial)

http://www.anlak.com/using-opencv-2-3-1-with-visual-studio-2010-tutorial

Related Instructables

Car No.03 Car No.02

Steering Wheel Steering Wheel
open cv+ Using openCV interactive
arduino+ many Drive R/C Car Drive R/C Car
1.0 with python with Arduino, with Arduino, Halloween
hours == Easy Home
2.5 in Windows Webcam and Webcam and mirror by
AWSOME!!!! Surveillance by
XP by nak net.USB etc. But net.USB etc. by lighttamer
(video) by kyle robonerd
this is Crawler Kaeru no Ojisan
brinkerhoff and 4WS (4
wheel steering)
by Kaeru no
Ojisan

http://www.instructables.com/id/Face-detection-and-tracking-with-Arduino-and-OpenC/