Recon Outpost Research & Analysis Systems

RECON OUTPOST
RESEARCH & ANALYSIS SYSTEMS
Derek Budde
Software Engineering, University Lusófona Porto, Portugal
derekbudde@gmail.com
Supervised by Drª Daniela

Cruz
Abstract. Recon Outpost system is designed to make available tools for home securityand investigators that
need to research surrounding ambient with video data in real time. The system can analyse and identify
biometric faces in live video, and provide real time surveillance in adverse weather conditions.
Keywords: Infrared, Night Vision, Face Detection, Target Detection, Face Detection, Arduino,
Biometric, Identification, Surveillance Systems.
Recon Outpost Research & Analysis Systems
Software Engineering, University Lusófona do Porto
“Give a shelterless man tools and he will buil d a home.

Inspire him, encourage him, give him vision, and he will build an empire.”
Rick Beneteau
1
TABLE OF CONTENTS
GLOSSARY...................................................................................................................................................................................4
INTRODUCTION .........................................................................................................................................................................4
WHAT IS FACE RECOGNITION? ..........................................................................................................................................5
DIFFERENT APPROACHES OF FACE RECOGNITION.........................................................................................................5
WHY SELECT PCA BASED EIGENFACES ALGORITHM FOR THE PROJECT? ...................................................................6
BASIC CONCEPTS ..................................................................................................................................................................6
OBJECTIVE ..................................................................................................................................................................................
7
TARGET MARKET ..................................................................................................................................................................7
SWOT ANALYSIS OF OUR PRODUCT .............................................................................................................................9
TARGET PROFILE...............................................................................................................................................................9
PLANNING .............................................................................................................................................................................9
WBS - GANTT CHART .................................................................................................................................................... 10
RESOURCES PLAN.......................................................................................................................................................... 11
Low fidelity prototype sketch ..................................................................................................................................... 13
SYSTEM AND SOFTW ARE DESIGN .......................................................................................................................................
14
USER REQUIREMENTS ...................................................................................................................................................... 14
USE CASE ............................................................................................................................................................................ 14
FACE DETECTION........................................................................................................................................................... 15
FACE RECOGNITION...................................................................................................................................................... 15
CAMERA CONTROL ....................................................................................................................................................... 16
INFRARED VIDEO ........................................................................................................................................................... 17
GREEN FILTER ................................................................................................................................................................ 17
TRACE CONTOURS ........................................................................................................................................................ 18
CLASS DIAGRAMS .............................................................................................................................................................. 19
SYSTEM ARCHITECTURE........................................................................................................................................................
20
SOFTWARE ......................................................................................................................................................................... 20
HARDW ARE ........................................................................................................................................................................ 23
DEPLO YMENT DIAGRAM ................................................................................................................................................. 24
ENTITY RELATIONSHIP DIAGRAM................................................................................................................................... 25
TECHNOLOGY OPTIONS ................................................................................................................................................... 25
IMPLEMENTATION.................................................................................................................................................................
26
CLASSES............................................................................................................................................................................... 26
ALGORITHMS AND CODE................................................................................................................................................. 26
FACE DETECTION........................................................................................................................................................... 29
FACE RECOGNITION...................................................................................................................................................... 30
IR IMAGE......................................................................................................................................................................... 32
NIGHT IMAGE ................................................................................................................................................................ 32
TRACE IMAGE ................................................................................................................................................................ 33
LOGIN .............................................................................................................................................................................. 34
SENSOR CONTROLS ....................................................................................................................................................... 35
RECON OUTPOST TOOL.........................................................................................................................................................
36
TESTS ........................................................................................................................................................................................
36
TESTING OBJECTIVE ...................................................................................................................................................... 36
TESTING TOOLS ............................................................................................................................................................. 37
TESTING .......................................................................................................................................................................... 37
FUTURE OPTIONS IMPLEMENTATIONS ..............................................................................................................................
38
CONCLUSION ..........................................................................................................................................................................
238
BIBLIOGRAPHY........................................................................................................................................................................
38
List of Figures
Figure 1 - Face Recognition.....................................................................................................................................................5
Figure 2 -Photometric stereo image......................................................................................................................................6
Figure 3 - Geometric facial recognition. ...............................................................................................................................6
Figure 4 - Samsung 17 Quad CCTV Observation System ....................................................................................................8
Figure 5 - - TR-500 NotiFace Recognition CCTV Surveillance System (500 face version).............................................8
Figure 6 - Swot analysis............................................................................................................................................................9
Figure 7 – System Main Idea................................................................................................................................................ 10
Figure 8 - WBS ........................................................................................................................................................................ 10
Figure 9 - Main menu screen ............................................................................................................................................... 13
Figure 10 - Splash screen ...................................................................................................................................................... 13
Figure 11 - Login screen ........................................................................................................................................................ 13
Figure 12 - Detection screen ................................................................................................................................................ 13
Figure 13 - Recognition screen ............................................................................................................................................ 13
Figure 14 - Save data screen ................................................................................................................................................ 13
Figure 15 - Night vision screen ............................................................................................................................................ 13
Figure 16 - Use Case.............................................................................................................................................................. 14
Figure 17 - Class Diagram ..................................................................................................................................................... 19
Figure 18 - System ................................................................................................................................................................. 20
Figure 19 - Main System AD ................................................................................................................................................. 20
Figure 20 - Login process ...................................................................................................................................................... 21
Figure 21 - Face Detecting AD ............................................................................................................................................. 21
Figure 22 - Camera Control AD............................................................................................................................................ 22
Figure 23 - Face Recognition AD2 ....................................................................................................................................... 22
Figure 24 - Night vision AD ................................................................................................................................................... 23
Figure 25 - Arduino AD.......................................................................................................................................................... 23
Figure 26 - System and Arduino AD .................................................................................................................................... 24
Figure 27 - Deployment Diagram ........................................................................................................................................ 24
Figure 28 - All components of the system ......................................................................................................................... 24
Figure 29 - ER Chen´s notation (Peter Chen's 1976 paper). ........................................................................................... 25
Figure 30 - Research diagram.............................................................................................................................................. 28
Figure 31 - Histogram Equalization..................................................................................................................................... 29
Figure 32 - Sequence diagram for Face Detection ........................................................................................................... 30
Figure 33 - Photometric Normalization Techniques – Shahrin Azuan Nazeer and Marzuki Khalid ........................ 32
Figure 34 - Gaussian Pyramid Decomposition.................................................................................................................. 33
Figure 35 - Canny edge detector steps .............................................................................................................................. 34
3
GLOSSARY
Visual Studio C# 2008, Visual Studio supports Visual C# with a full -featured code editor, compiler, project
templates, designers, code wizards, a powerful and easy-to-use debugger, and other tools. The .NET Framework
class library provides access to many operating system services and other useful, well - designed classes that
speed up the development cycle significantly.
Emgu CV, cross platform .Net wrapper to the Intel OpenCV image-processing library. Allowing OpenCV
functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc.
Arduino, Arduino programming language based on Wiring, Wiring is an open-source programming framework
for microcontroller.
Eigenfaces - Refers to an appearance-based approach to face recognition that seeks to capture the variation in
a collection of face images and use this information to encode and compare images of individual faces in a
holistic (as opposed to a parts-based or feature-based) manner.
PCA (Principal Component Analysis) - Transform each original image of the training set into a corresponding
eigenface. An important feature of PCA is that one can reconstruct any original image from the training set by
combining the eigenfaces.
Threshold - Minimum or maximum value (established for an attribute, characteristic, or parameter) which
serves as a benchmark for comparison or guidance and any breach of which may call for a complete review of
the situation or the redesign of a system.
Image Pyramid - An image pyramid is a collection of images, all arising from a single original image that are
successively downsampled until some desired stopping point is reached. There are two common kinds of image
pyramids:
Gaussian pyramid: Used to downsample images.
Laplacian pyramid: Used to reconstruct an upsampled image from an image lower in the pyramid (with less
resolution). (http://docs.opencv.org/doc/tutorials/imgproc/pyramids/pyramids.html )
INTRODUCTION
Humans have been using physical characteristics such as face, voice, etc. to recognize each other for thousands
of years. With new advances in technology, biometrics has become an emerging technology for recognizing
individuals using their biological traits. Now, biometrics is becoming part of day to day life, where in a person is
recognized by his/her personal biological characteristics. Our goal is to develop an inexpensive security
surveillance system, which will be able to detect and identify facial and body characteristics in adverse weather
conditions. There are many factors which influence this type of methods i.e. lighting condition background
noise, fog and rain.
A particular attention is given to face recognition. Face recognition refers to an automated or semi -automated
process of matching facial images. Many techniques are available to apply face recognition one of them is
Principle Component Analysis (PCA). PCA is a way of identifying patterns in data and expressing the data in
such a way to highlight their similarities and differences.
4
WHAT IS FACE RECOGNITION?
Face recognition is the task of identifying an already detected object as a known or unknown face, and
in more advance cases, telling exactly who’s face it is (Figure 1).
Often the problem of face recognition is confused with the problem of face detection.
Face Detection is to identify an object as a "face" and locate it in the input image.
Face Recognition on the other hand is to decide if the "face" is someone known, or unknown, using for this
purpose a database of faces in order to validate this input face.
Figure 1 - Face Recognition
DIFFERENT APPROACHES OF FACE RECOGNITION
There are two predominant approaches to the face recogniti on problem: Geometric (feature based) and
photometric (view based). As researcher interest in face recognition continued, many different algorithms were
developed, three of which have been well studied in face recognition literature.
Recognition algorithms can be divided into two main approaches:
1. Geometric: Is based on geometrical relationship between facial landmarks, or in other words the
spatial configuration of facial features. That means that the main geometrical fea tures of the face such
as the eyes, nose and mouth are first located and then faces are classified on the basis of various
geometrical distances and angles between features. (Figure 3).
2. Photometric stereo: Used to recover the shape of an object from a number of images taken under
different lighting conditions. The shape of the recovered object is defined by a gradient map, which
is made up of an array of surface normals (Zhao and Chellappa, 2006) (Figure 2).
Popular recognition algorithms include:
3. Principal Component Analysis using Eigenfaces,(PCA)

4. Linear Discriminate Analysis,
5. Elastic Bunch Graph Matching using the Fisherface algorithm,
6. The Hidden Markov model,
7. The neuronal motivated dynamic link matching.
5
Figure 2 -Photometric stereo image.
Figure 3 - Geometric facial recognition.
WHY SELECT PCA BASED EIGENFACES ALGORITHM FOR THE PROJECT?
PCA based Eigenface method is at the most primary level and simplest of efficient face recognition algorithms
and Eigenfaces method for recognition is as supported by EmguCV library as is Viola -Jones method for
detection. PCA based Eigenfaces method is not 100% efficient, in fact, on the average of 96% with light
variation, 85% with orientation variation, and 64% with size variation (Turk & Pentland 1991, p. 590). No face
recognition algorithm is 100% efficient.
BASIC CONCEPTS
Viola-Jones - The Viola–Jones [3] object detection framework is the first object detection framework to provide
competitive object detection rates in real -time proposed in 2001 by Paul Viola and Michael Jones. Although it
can be trained to detect a variety of object classes, it was motivated primarily by the problem of face
detection. This algorithm is implemented in EmguCV as HaarDetectObjects ().
Face detector uses a method that Paul Viola and Michael Jones published in 2001. Usually called simply the
Viola-Jones method, or even just Viola-Jones, this approach to detecting objects in images combines four key
concepts:
Simple rectangular features, called Haar features

An Integral Image for rapid feature detection The
AdaBoost machine-learning method
A cascaded classifier to combine many features
effici ently
6
The features that Viola and Jones used are based on Haar wavelets. Haar wavelets are single wavelength
square waves (one high interval and one low interval). In two dimensions, a square wave is a pair of adjacent
rectangles - one light and one dark.
Haar Cascade - Haar-like features (so called because they are computed in a similar method as the
coefficients in Haar wavelet transforms) and a cascade of boosted tree classifiers as a statistical model.
Canny Edge Detector - The Canny operator was designed to be an optimal edge detector (according to
particular criteria). It takes as input a grey scale image, and produces as output an image showing the positions
of tracked intensity discontinuities. (Canny filter) for edge detection, visualize a derivative of a Gaussian
function, simulate non-classical receptive field inhibition or surround suppression and use it for object contour
detection. The basic algorithm deployed for edge detection is that of J. Canny [1]. You can use an additional
surround suppression step that will eliminate texture edges [2]. You should use this step if you are specifically
interested in the detection of object contours and region boundaries.
MCvTermCriteria – Is a class which represents OpenCV structure for terminating iterative algorithms. It is
composed of two numbers the first being the number of iterations and the second one is demanded accuracy.
For some algorithm it makes sense to iterate until the accuracy is not bellow certain threshold. For eigenfaces
algorithm it is the number of iterations which is important and it will impact the number of eigenfaces being
created [6].
OBJECTIVE
The goal of this project is to create a prototype to reduce the amount of climacteric factors to provide facial
recognition and facial tracking in day light surveillance and night vision surveillance for wide areas. Also, it
implements pan and tilt support to give the ability to rotate the cameras by software control.
The prototype is built to test the performance of the current face tracking and face recognition technology in
real life conditions.
The results will determinate if the prototype is suitable for commercialization, and set the current parameters
to determinate in which state the prototype performs in maximum efficiency. If the field test results are
acceptable for home users, the prototype will be adapted for military and law enforcement.
For this purpose there are some basic prerequisites.
- The prototype must be able to power up and run on a 12v dc car battery.
- The prototype must be portable.
- The software must have a direct and objective interface for inexperience users.
The project is divided in modules, where each module will provide one different feature to the project. With this
approach each individual module can be developed and updated separatel y from the main module and also
provide to the client the ability to choose with module best suits his needs.
TARGET MARKET
Throughout the world, there is an increase need for video surveillance systems to secure our schools,
businesses, hospitals, ports, homes and other critical environments. According to a new report from market
research firm IMS Research, the global video surveillance market is expected to grow from $12.6 billion in
7
2012 to $23.2 billion in 2017. That’s a more than 80 percent increase in the next five years and a sure sign
of even more growth to come. (http://www.imsresearch.com/).
The industry of CCTV video surveillance sometimes does not provide an adequate
system. The most common problems are:
Video surveillance cost installation (Figure 4)

Most systems are fixed, can’t be transported to other locations
(Figure 4) Infrared light to bright or to low can’t be adjusted
Few camera have rotation or is too expensive for home users
Facial recognition software price (Figure 5)
Night rain or snow interferes infrared illumination
Figure 4 - Samsung 17 Quad CCTV Observation System
Figure 5 - - TR-500 NotiFace Recognition CCTV Surveillance System (500 face version)
The challenges remain in addressing these issues in one product, ready to operate in most case scenarios.
- Portable and integrated system (plug and play) - Guarantees that any person, anywhere or in a mobile
vehicle can star using the product.
- The special lens and software fi lters provide a clear infrared picture regardless of weather conditions.
- Never seen features in other systems of video surveillance - Trace image, that provide analysis in real
time of moving objects or people.
- Simple touch screen interface.
- No need for separated hardware to control camera rotation.(e.g. joystick)
8
- System software modules can be purchased individually
SWOT ANALYSIS OF OUR PRODUCT
Strengths Weakness
Low cost Hardware

Home made product
Free Updates Video
No budget
Trace Feature
No recording video
Camera Rotation Control
No motion detection
Adjustable IR Intensity
No Internet streaming
Portable system
image
Opportunities Threats
Market increase
80 percent in the next five Low public appreciation
year No global or local presence
Competitor products does in market.
not meet costumers Competitive advantage
demands or features of our of other companies
product
Figure 6 - Swot analysis
TARGET PROFILE
Our customer can be divided in two groups:
Residential User
Commercial Owners
Residential User Commercial Owners

Gender: Male/Female Gender: Male/Female
Age: 21/60 Age: 21/60
Education: Basic (read Education: Basic (read
and write) and write)
Location: any Location: any
IT knowledge: Low experience IT knowledge: Low experience
Profession: any Profession: any
Social Class: High and Middle class Social Class: Low and middle business
Main User: Head of household Main User: owner, security personal,
Secondary User: Dependent members private investigator
Secondary User: Staff members
Motivation: Prevent theft, Monitor business, Outdoor surveillance, Check on family, ID employees.
PLANNING
The project schedule identifies and organizes project tasks into a sequence of events that form a project
management plan. The process of building this schedule enabled us to identify risk points and develop the
proper linkage of events; it also assists in resource planning and allows us to establish milestones for the
development of the project.
9
Our concept map represents ideas and information that the system must
perform.
See images
advers Trace Filter
weather
ID Faces
can contain Green Filter

can
s image
Save Faces
Face can contain
recognition Night s
Vision
can
contain
s
Face Rotate
can can
detection Camera
s
has
has Adjust IR
brightness
Recon
Outpost
Figure 7 – System
Main Idea
WBS - GANTT CHART
Figure 8 - WBS
10
Figure 8 - Gantt chart timeline
Figure 9 - Gantt chart table information
RESOURCES PLAN
Types of labor required for the project: Programmer, Robotics/Electronics and Project manager
Roles and key responsibilities for each labor type:
- Programmer- Software Development

- Robotics/Electronics – Hardware Development
- Project manager - Planning, execution and closing of the project
Items of equipment to be used and their purposes:
- Main Equipment:
- PC for software development
- Webcams for image acquiring
- Arduino and electronics for robotics
11
Software Tools:
- Visual Studi o C# 2008

- EmguCV
- Adobe Photoshop CS3
Complete list of equipment: Attachment 29 - Equipment list
Phase 1 Phase 3
Preparing O.S
- Using windows restricti ons, tweaking registry
Project Documentati on
- Initi al development idea
- Requirement analysis Programming Code
- Main structure
- Blue prints, Interface sketches, etc.
- Methods to use EmguCv
- Using windows l ibraries
Project schedule - Adapti ng controls to interface
- Project tasks( WBS, Gantt ) - Code opti mizati on
- Available resources (Materials, Tools, etc.) - Hardware code
- Budget - Preliminary tests
Project viability Hardware assembling

- Research and small scale experimental - Soldering, wiring and adapting
Phase 2 Phase 4
Programming Tools( C#, EmguCV)

Field Testi ng
Hardware preparation - Adjusti ng soft ware parameters
- Disassembling, modifying and researching. - Fixing hardware problems
Software preparation Project Documentati on

- Researching EmguCV implementation, - Ending project
parameters, routines.
- Preparing Graphical Interface and adapting for
touch screen.
12
LOW FIDELITY PROTOTYPE SKETCH
Figure 9 - Main menu screen Figure 10 - Splash screen
Figure 11 - Login screen

Figure 12 - Detection screen
Figure 14 - Save data screen
Figure 13 - Recognition screen
Figure 15 - Night vision screen
13
SYSTEM AND SOFTWARE DESIGN
The final system will provide the user new tools for detecting and recognizing faces in real time video stream
also provide real time video for adverse weather conditions. The user will have the ability to rotate camera
position, providing a wider range of view. All this features have the purpose to update older video surveillances
systems. To accomplish this mission we consulted an expert Americo Santos (Surveillance Security Expert) the
following requirement was the result of our interview/research.
USER REQUIREMENTS
1. Functional requirements
a. The system must be able to identify human faces in live video.
b. The system must be able to search for faces in images as an input and sea rch for a
matching face in folder, and then show the results.
c. The results should be viewed by showing the name of the face match of the input to the
most-similar face in folder.
d. The user should be able to choose (click on) to change the camera position.
e. The system must be able to draw contours of moving objects in a live video.
f. The system must be able to display a live infrared video.
2. Non Functional Requirements

a. The user should be able to adjust (click on) Infrared intensity.
b. User should be able to apply a green spectrum filter (click on) in a live video.
c. Critical errors and information may be shown in a textbox where it has auto-scroll when
the space of the box is not enough.
d. The face should be localized by detecting inner and outer boundaries, background must
be ignored.
e. The user should be able to save detected faces and names for future comparison.
USE CASE
Figure 16 - Use Case
Actors Description:
14
Users: Authorize the system to analyses a video image and expects the system
to display the results.
Use Cases Description
Request Face Detection The user requests the system to find a human face in video.
Request Face Recognition The user requests the system to match a detected face.
Perform Camera Control Allows the user to rotate the camera up, down, left and right
Request Infrared video The user request to view stream from Infrared camera
Request Green Filter The user request to apply green filter on Infrared stream
Request Trace Contours The user request to apply trace filter on Infrared stream
FACE DETECTION

Case No. 1
Name Request Face Detection
Actor User
Description The user requests the system to find a human fa ce in video.
Pre-condition System must have receiving webcam stream
Post-condition System displays highlighted face(s).
Flow of Events
Steps Description
1 User click button to start detecting faces
2 System starts webcam stream
3 System loads algorithm (Haar) to detect face in video
4 System highlight detected faces with rectangle square
5 System return video image with highlighted face(s).
Alternative Flows
Steps Description
2.a Webcam disconnected
5.a No face(s) have been detected in video
Entry Conditions
User has correct authentication to system and webcam is functional
Exit Conditions
User click back button
Sequence Diagram: Attachment 3 - Face Detection SD

Activity Diagram: Attachment 4 - Face Detection AD
FACE RECOGNITION

Case No. 2
Name Request Face Recognition
Actor User
Description The user requests the system to match a detected face.
Pre- System must have detected a face(s)
condition
15
Post- System displays name(s) of face(s).

condition
Flow of Events
Steps Description
1 User click button to recognize a highlighted face
2 System loads algorithm (Haar) to detect face in image
3 System highlight detected faces with rectangle square
4 System loads algorithm (PCA) to match a stored face
5 System return name(s) of the most similar face matched.
Alternative Flows
Steps Description
2.a No face(s) have been detected in image
5.a Name doesn’t match face(s)
Entry Conditions
Image was previous captured by system
Exit Conditions
Sequence Diagram: Attachment 5 - Face Recognition SD

Activity Diagram: Attachment 6 - Face Recognition AD
CAMERA CONTROL

Case No. 3
Name Perform Camera Control
Actor User
Description Allows the user to rotate the camera up, down, left and right
Pre-condition Arduino interface must be connected
Post- Display message of the new position
condition
Flow of Events
Steps Description
1 User click button to rotate camera position
2 System connect to Arduino by serial port
3 System send new values position to Arduino
4 Arduino rotates the camera to the new position
5 System return new position
Alternative Flows
Steps Description
2.a System can´t open serial port
Entry Conditions
Arduino must be connected and serial port available
Exit Conditions
Sequence Diagram: Attachment 7 - Camera Control SD

Activity Diagram: Attachment 8 - Camera Control AD
16
INFRARED VIDEO

Case No. 4
Name Request Infrared Video
Actor User
Description The user request to view stream from Infrared camera
Pre- System must be receiving video stream from webcam
condition
Post- System displays infrared image
condition
Flow of Events
Steps Description
1 User click button to display infrared video
2 System checks camera connection
3 System starts displaying video stream
Alternative Flows
Steps Description
2.a Doesn’t receive image
Entry Conditions
Camera must be connected
Exit Conditions
Sequence Diagram: Attachment 9 - Infrared SD

Activity Diagram: Attachment 10 - Infrared AD
GREEN FILTER

Case No. 5
Name Request Green Filter
Actor User
Description The user request to apply green filter on Infrared stream
condition
Post- System displays infrared image in green spectrum
condition
Flow of Events
Steps Description
1 User click button to display green spectrum
3 System copies streaming video frame to memory
4 System loop pixels and removes red and blue colours.
5 System shows live video stream in the green spectrum
Alternative Flows
Steps Description
Entry Conditions
17

Exit Conditions
Sequence Diagram: Attachment 11 - Green Filter SD

Activity Diagram: Attachment 12 - Green Filter AD
TRACE CONTOURS

Case No. 5
Name Request Trace Contours
Actor User
Description The user request to apply trace filter on Infrared stream
condition
Post- System displays contours of moving objects in stream
condition
Flow of Events
Steps Description
1 User click button to display green spectrum
3 System copies streaming video frame to memory
4 System applies Image Pyramid algorithm to image
5 System applies Canny edge detector algorithm to image
6 System shows white contours of moving objects
Alternative Flows
Steps Description
Entry Conditions
Exit Conditions
Sequence Diagram: Attachment 13 - Trace Contours SD

Activity Diagram: Attachment 14 - Trace Contours AD
18
CLASS DIAGRAMS
Figure 17 - Class Diagram
Class Diagram: Attachment 15 - Class Diagram
19
SYSTEM ARCHITECTURE
System architecture is divided in two parts: Software and Hardware.
SOFTWARE
In the figure below has a representation of the system overview main structure, that is divide in three parts.
Figure 18 - System
1. Main System
The main system validates the user login, and provides an interface for the other modules.
Figure 19 - Main System AD
The authentication process stage is deployed before the main interface.
20
Figure 20 - Login process
2. Module Tracking
This stage consist in detecting, recognize and adding a face to the system. We will star with the face
detection main process and mounting up the components to full scale.
Figure 21 - Face Detecting AD
All algorithms used are explained in implementation chapter.
21
Figure 22 - Camera Control AD
Bellow the main diagram with all stages
Figure 23 - Face Recognition AD2
3. Module Night Vision

Module responsible to apply filters to improve the quality of the infrared illuminated video.
22
Figure 24 - Night vision AD
In the following attachments can represent a better dependency of the system internal
structure.
Attachment 30 - Dependency Graphs Main

Attachment 31- Dependency Graphs MD1
Attachment 32 - Dependency Graphs MD2
HARDWARE
In this section we will represent the hardware interaction. Please see “Deployment Diagram”. “tarting with
the Arduino internal and mounting up the components to full scale.
Figure 25 - Arduino AD
Now we represent the full cycle interaction with Arduino and C#.
23
Note: Webcams image streams are not represented here, we assume that the
USB controllers processes is the
O.S. responsibility.
Figure 26 - System and Arduino AD
DEPLOYMENT DIAGRAM
Figure 27 - Deployment Diagram
Figure 28 - All components of the system
24
Hardware schematic and assembling: can be viewed in (Hardware

attachment).
ENTITY RELATIONSHIP DIAGRAM
Our system does not use a database to store data, why:
We don't have enormous data sets.

The application will never use the full features of databases software’s.
Power loss can corrupt database, due to the nature of the prototype.
Additional cost and time associated for development and (probable) to restore corrupted database.
Plain text files in a file system
Very simple to create and edit.

Easy for users to manipulate with simple tools (i.e. text editors, grep, etc.).
Efficient storage of binary documents.
K.I.S.S: Keep It Small and Simple.
Figure 29 - ER Chen´s notation (Peter Chen's 1976 paper).
All images (faces) are stored in simple files (.bmp) in a folder (TrainedFaces). Each i mage name has a number
that corresponds to a name positioned in the file separated by the delimiter. “NumLabels” represents the total
of names and images in file and in the folder.
Future implementation of a database such as SQLite or other can be developed in use with a backup system or
a use of UPS.
TECHNOLOGY OPTIONS
25
The project is developed over Microsoft Windows O.S. and Arduino electronics. Windows O.S. will provide
the capability for develop a graphical interface and easy integration with the external hardware. Arduino
electronics will provide the ability to control the hardware.
In the project it will be used Visual Studio C# language to develop the software and Arduino language. The
combinations of these two languages provide a stable software interface and communication with the
hardware.
EmguCV simplifies the implementation of the algorithms in C# code and provides a set of tools for
development.
IMPLEMENTATION
CLASSES
Classes generated from C# code - Attachment 1 - S.C. MD1
Attachment 2 - S.C. MD2
ALGORITHMS AND CODE
Algorithms - For the algorithms was carried out an extended research to select the proper algorithms that are
best suited for the development of the project. Being selected Emgu CV that provides the tools for their
implementation, documentation and use.
EmguCV internal architecture can be viewed: Attachment 33 - EmguCV Architecture.
Software – In this section we will point out the parts that was researched and studied to be implemented,
adjusted and optimized ‘in the real world' conditions to our project.
Splash Screen
Creation of input graph and adaptation to the prototype screen.

Adaptation and use of Windows libraries to hide the Windows taskbar (based on research).
Creation of the activation time window.
Login Form
Creation of the graphic adaptation and routines for buttons and functions of the virtual keypad input.
Creation of the routines for the algorithm (Reading, writing and implementation of the calculation).
Mathematical calculation (based on research).
Main Form
Creation of the interaction process between screens (Splash Screen, Login, and Main Form).
Adaptation of the parameter (WM_VSCROLL) Windows libraries (based on research).
Creation of the function that checks other modules installed in the system.
Adaptation of the function to shutdown computer. Windows command (based on research).
26
Tracking Mode - Face Detection
Creation of the control functions for the position and rotation of the camera by the seria l port
and display the results.
Creation of the control function to check connection hardware (PC-Arduino).
Implementation of the process to capturing images of the camera (based on functionality
EmguCV and research for optimization and parameters use).
Implementation of the face detection algorithm (based on functionality EmguCV and
research for optimization and parameters use).
Creation of graphics video overlay. (Using the functionality provided by
EmguCV). Creation of the passage of the selected image between screens.
Tracking Mode - Face Recognition
Implementation of face recognition algorithm (based on functionality EmguCV and

Creation of graphics video overlay. (Using the functionality provided by
EmguCV). Creation of the passage of the selected image between screens.
Tracking Mode – Database
Creation of the function for the virtual keyboard and the graphical adaptation for the interface
form. Implementation process of saving data (based on functionality EmguCV and research for
optimization and parameters use).
Night Mode – Night Vision
Implementation of the process to capturing images of the camera (based on functionality

EmguCV and research for optimization and parameters use).
Implementation process of removing pixels (based on functionality EmguCV and
research for optimization).
Implementation of the process of image contours (based on functionality EmguCV and
Creation of the control functions for brightness intensity of infrared.
Arduino Code
Creation of code interaction between hardware and software (based on the functionality and
use of
the Arduino library and library’s for IC 74HC595N).
Artwork
All creation
Hardware – After the hardware components selected as well as a research of the characteristics,
it is concluded that the following changes shall be carried out).
Modifying the webcam hardware (based on research and previous knowledge)

Creation of a printed circuit board for the expansion input ports of Arduino (based on
research and previous knowledge).
27
Creation of a system for adjusting the intensity of infrared LEDs (based on research and use
of IC 74HC595N library).
Rotation control system of the motors (based on Arduino library and previous knowledge).
Creation and modification (Metrics dimensions, assembly, wiring and enclosures prototype)
personal electronics skills.
based on research for

optimization and parameters use
user32.dll from O.S to

Splash Screen use
Hide Taskbar
contain
user32.dll -
WM_VSCROLL from
Main Form use
contain O.S to Scroll up and
Recon Outpost down
use
contain user32.dll from O.S to
Login Form Shutdown PC
use
calls
calls Mathematical
calculation
Face Tracking contain Face Detection
contain use
EmguCV functions for
receive image
Face Recognition
Night Mode use analyses and apply
algorithms
contain
use
Save Face use
contain
Arduino
send t o library for IC
Green Filter 74HC595N
Trace contours
IR image
Figure 30 - Research diagram
28
FACE DETECTION
The system is developed based on EmguCV that provides the libraries for the implementation of the
algorithms for face detection and recognition.
The face detector uses the responses to a series of simple filters to classify regions of an image as either a face
or not a face. The filters are called Haar filters and are calculated by taking the sum of pixels within a number
of rectangles, multiplying each sum by a weight and adding the results.
The implementation in simple steps, the system loads the Haar file(haarcascade_frontalface_default.xml) to
detect frontal faces in the image.
This file contains the metrics for detecting faces. For this metrics please see (Cascade Classifier Training –
www.docs.opencv.org/doc/user_guide/ug_traincascade .html).
After loading the metrics for the Haar filter each image (fr ame) acquired by the webcam is scanned by the
parameters set in the code.
Convert the frame to a grayscale image and perform Histogram Equalization.
Histogram Equalization: Equalization implies mapping one distribution (the given histogram) to another
distribution (a wider and more uniform distribution of intensity values) so the intensity values are spreaded
over the whole range (http://docs.opencv.org).
Figure 31 - Histogram Equalization
And for the detection:
Scales Increase Rate - specifies how quickly EMguCV should increase the scale for face detections with
each pass it makes over an image.
Minimum Neighbors Threshold - sets the cutoff level for discarding or keeping rectangle groups as
“face” or not, based on how many raw detections are in the group.
Canny Pruning Flag - skips image regions that are unlikely to contain a face, reducing computational overhead
and possibly eliminating some false detections.
Minimum Detection Scale - is the size of the smallest face to search for in a image. The default for this by
setting the scale is 25x25.
Code:
haar = new HaarCascade(@"haarcascade_frontalface_default.xml");
_i mgProcessado = _i mgOriginal.Convert<Gray, byte>();
_i mgProcessado._EqualizeHist();
29
var faces = _i mgProcessado.DetectHaarCascade(

haar,
_s caleFactor,
_s caleDistanc,
HAAR_DETECTION_TYPE.DO_CANNY_PRUNING,
new Size(_imgProcessado.Width / 8, _i
mgProcessado.Height / 8))[0];
Figure 32 - Sequence diagram for Face Detection
FACE RECOGNITION
For the face recognition, EmguCv uses a series of algorithms and mathematical calculations combined. This
process is implemented by the PCA algorithm please sees for more details:
(www.ehu.es/ccwintco/uploads/e/eb/PFC-IonMarques.pdf).
The first step is to load the trained faces (Folder containing a set of images of human faces) and the file
containing the name for each of these faces. The extracted face from the face detector is compared (projected)
in the set of Eigenfaces. This means the target face featur es are compared (distance) which the features of the
Eigenfaces and returns the most similar face, this return face corresponds to a name in the file
(TrainedLabels.txt).
30
To achieve this, the MCvTermCriteria (EmguCV parameter) must receive the total sum of trained faces and
the maxIteration (MCvTermCriteria parameter), this parameter determines the number of Eigenfaces to be
created, please see (www.face-rec.org/algorithms/PCA/jcn.pdf).
The EigenObjectRecognizer (EmguCV) object recognizer using the PCA (Principle Components Analysis) will be
set to receive:
The array of trained faces and the array of names.

The thresholds (EigenObjectRecognizer parameter) that compares the facial feature distance between
the target image and the set of Eigenfaces, the smaller the value number, the more likely an examined
image will be treated as unrecognized face, big values will return the most closest match to one of the
images in the Eigenface set.
The MCvTermCriteria paramenters defined above.
This will return the correspondent name of the most similar face match. Our threshold is set to always return a
most similar face of the Eigenface set despite the (distance) facial features between them.
This will ensure that even if the target face is affected by some other conditions, the algorithm will return the
most similar face in folder.
Code:
string _LoadFaces;
string _NomesInfo = File.ReadAllText(Application.StartupPath +
"/TrainedFaces/TrainedLabels.txt");
string[] _listNamesTxt = _NomesInfo.Split('%');
int _numLabels = Convert.ToInt16(_listNamesTxt[0]);
_numbTrainFaces = _numLabels;
for (int _tra indFaces = 1; _tra indFaces < _nu mLabels + 1; _tra indFaces++)
{
_tra iningImages.Add(new Image<Gray, byte>(Application.StartupPath + "/TrainedFaces/" +
_Loa dFaces));
_l abels.Add(_listNamesTxt[_traindFaces]);
…
Bitmap default_ig = new Bitmap(face_PICbox.Image);
_gra y = new Image<Gray, byte>(new Bitmap(default_ig));
_res ult = _gra y.Resize(100, 100, Emgu.CV.CvEnum.INTER.CV_INTER_CUBIC);
MCvTermCriteria termCrit = new MCvTermCriteria(_numbTrainFaces, 0.001);
EigenObjectRecognizer recognizer = new EigenObjectRecognizer(
_tra iningImages.ToArray(),
_l abels.ToArray(),
500000,
ref termCrit);
_na me = recognizer.Recognize(_result).Label;
Code Sequence Diagram: Attachment 17 -Face recognition code SD
Graphical representation of the process:
31
Figure 33 - Photometric Normalization Techniques – Shahrin Azuan Nazeer and Marzuki Khalid
IR IMAGE
An infrared (IR) light emitting diode (LED) is an electronic method of creating infrared light. This makes an IR
LED illuminator a method of lighting up an area using infrared light. Infrared is outside the visual range of the
human eye. On the other hand, if the area is monitored by a device capable of seeing into the IR range, then
the area may be brightly lit to that device alone.
(http://www.wisegeek.com/what-is-an-ir-led-illuminator.htm) see (webcam modification).
The code just captures images from the second webcam and displays it for the user.
Code:
capWebCam = new Capture(0);
capWebCam.SetCa ptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_HEIGHT, 240);
capWebCam.SetCa ptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_WIDTH, 320);
…
imgOriginal = capWebCam.QueryFrame();
vdbox_IMGBOX.Image = imgOriginal;
Code Sequence Diagram: Attachment 22 - S.D. IR
NIGHT IMAGE
Now that our webcam is capturing the near infrared spectrum, we can apply filters to the streaming video. The
filter is (Night Mode) that removes the Red and Blue pixels from the image, displaying only the green pixels.
This is because the human eye can distinguish more shades of green than any other colour.
32
(http://www.physicscentral.com/explore/action/infraredlight.cfm). This is a simple loop cycle that interacts

over an array of the pixels (image) removing it from the video. EMguCV provides direct acess to the image data.
Code:
vdbox_IMGBOX.Image = imgOriginal;
imgCapture = imgOriginal.Copy();
Suppress(2);
vdbox_IMGBOX.Image =
imgCapture;
Suppress(0);
vdbox_IMGBOX.Image =
imgCapture;
private void Suppress(int
spectrum){
for (int i = 0; i < imgOriginal.Height; i++)
{ for (int j = 0; j < imgOriginal.Width; j+
+){
imgCapture.Data[i,
Code Sequence j, spectrum]21
Diagram: Attachment = 0;}
- S.D. Night Mode
}
}
TRACE IMAGE
For the filter (Trace Mode) the image is subject to various transformations. The first transformation is the Image
Pyramid [9]. We perform Gaussian pyramid decomposition (PyrDown) each reduction in size removes high
frequency components from the image. What we are left with are the low frequencies at each level. Then we
reverse the process (PyrUP) we take the reduced image, scale it up to the original size, and subtract the
difference. See (Figure 29).
Figure 34 - Gaussian Pyramid Decomposition
Now we apply the Canny edge detector is an edge detection operator that uses a multi -stage algorithm to detect
a wide range of edges in images. It was developed by John F. Canny in 1986. The Canny edge detector uses two
parameters:
Thresh - The threshold to find initial segments of strong edges.
ThreshLinking - The threshold used for edge linking.
To remove the background and redraw the image in black and white we use a ThresholdBinary that will convert
the intensity of the pixel, if the pixel is higher than thresh, then the new pixel intensity is set to zero, otherwise,
it is set to max value.
33
In Figure 30 we can see the steps of the image transformation.
Figure 35 - Canny edge detector steps
Code:
private void traceIMG()

{
Image<Gray, Byte> grayFrame = imgOriginal.Convert<Gray, Byte>()
.PyrDown()
.PyrUp()
.Ca nny(new Gray(180), new Gray(90));
Image<Gray, byte> binaryImage = grayFrame.ThresholdBinary(new Gray(1), new Gray(255));

for (var contour = binaryImage.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
RETR_TYPE.CV_RETR_CCOMP);
contour != null;
contour = contour.HNext)
{
grayFrame.Draw(contour, new Gray(255), -1);
}
vdbox_IMGBOX.Image = grayFrame;
}
Code Sequence Diagram: Attachment 20 - S.D. Trace
LOGIN
The login is composed by 6 digits e.g. (233987), this digits are divided in to two groups of three digits e.g. (233)
and (987). The first group of three digits is where we apply the mathematical calculations.
The number is duplicated e.g. (233233), now we divide the number by 13 e.g. (233233/13= 17941).This
result is stored in a binary file as e.g. (100011000010101 “data.bin”).
To check if the login is valid, the system retrieves the value from the file, and divides by 11 e. g.(17941/11 =
1631), and divides the result by 7 e.g.( 1631/7 = 233). This way prevents that the first group of digits or stored
directly in the code and provides a unique identification for the soft ware. The second group of digits e.g. (987)
it doesn’t implement any verification in this code, futur e work this group will be used to verify hardware
authenticity (Arduino).
Code:
void validatePass(){
i f (Fi le.Exists(@"data.bin")){
// pass 233987
i f (unlockBox.Text.Length == 6){
string codFile = passRead();
string cod = unlockBox.Text;
string sub = cod.Substring(0,
3);
34
string sub2 = cod.Substring(3, 3);

string codeVerificacao = "987";
i f (codFile == sub &&

codeVerificacao == sub2)
{ this.Close();
}else { errorPass = true; }
}else { errorPass = true; }
}else{ MessageBox.Show("Please contact support: Error F24");
foreach (Control c in this.Controls){ c.Enabled = false; }
Application.Exit(); }}
string passRead(){
string testfile = @"data.bin";
long valueCalc;
StreamReader fs = new StreamReader(testfile);
string str = fs .ReadLine();
fs .Cl ose();
valueCalc = Convert.ToInt64(str, 2) / 11 / 7;
string final = Convert.ToString(valueCalc);
return (final);
}Code Sequence Diagram: Attachment 23 - SD login
SENSOR CONTROLS
The sensors are responsible to rotate the camera in the X-axis and Y-axis, and provide control of the infrared
light intensity. In camera rotation, the X-axis value is (10 to 140) degrees and the Y-axis is (80 to 140) degrees.
This is due to the dimensions and the assembly position of the servos (motors) on the hardware. The default
position in the X-axis is 80 degrees and on the Y -axis is 140 degrees. With this starting position (default), the
system increases or decreases 10 degrees in the axis position when user clicks the buttons. To accomplish this C#
opens the serial port and sends the value to Arduino.
Code Rotation Up Control:
private void btn_UP_Click(object sender, EventArgs e){

i f (conectado == true)
{ try{
i f (posicaoY >= 80 &&
posicaoY <= 130){
posicaoY += 10;
enviarTX = "0" + posicaoY.ToString() + "b";
sp_COM.Wri te(enviarTX);
}else{
enviarTX = "Max rotation reached";
}
txtbox_LOG.AppendText("TX=" + enviarTX
+ " | " + posicaoY.ToString() + "º Graus" +
"\r\n");
}catch{
conectado
Code Sequence= false;
Diagram: Attachment 18 - S.D. Rotate Camera
chkTimeSerial.Start();}}}
This code represents the movement in the upper direction of the servo (motor), for the left, right, center and
down the code receives other values; the structure of the implementation is the same.
Code Brightness Control:
The same sequence of events of the camera rotation is followed to control the brightness of the infrared led´s.
Only in this case the maximum and the minimum value ranges are 255 and 25 units and
35
increases and decreases by 25 units each click. These units are established by the IC (Integrated Circuit).
See attachment chapter (IR Illumination array).
Code:
private void brightDown_Click(object sender, EventArgs e){

i f (conectado == true)
{ try{
i f (brightX <= 250 &&
brightX >= 125){
brightX -= 25;
string enviarTX = brightX.ToString() + "e";
txtbox_LOG.AppendText("Intensity= " +
enviarTX + "\r\n");
}
else i f (brightX >= 25 && brightX <= 100)
{
brightX -= 25;
string enviarTX = "0" + brightX.ToString() + "e";
txtbox_LOG.AppendText("Intensity= " +
enviarTX + "\r\n");
}
}catch{
Code Sequence Diagram: Attachment 24 - S.D. Brightness
conectado = false;
chkTimeSerial.Start();}}}
RECON OUTPOST TOOL
Software:
Attachment 29 – Login
Attachment 30 - Main Interface
Attachment 31 - Interface Face Detection
Attachment 32 - Interface Face Recognition
Attachment 33 - Interface Database
Attachment 34 - Interface Night Mode
Attachment 35 - Interface Infrared Mode
Attachment 36 - Interface Trace Mode
Hardware:
Attachment – Chapter Hardware
TESTS
TESTING OBJECTIVE
Detection Performance - Test the performance of frontal face for recogniti on metrics.
Measure limitations – Test the performance of the system in a different environment conditions.
All tests are adapted to the nature of the prototype.
36
TESTING TOOLS
Detection Performance: The Yale Face Database contains 165 grayscale images in GIF format of 15
individuals. There are 11 images per subject, one per different facial expression or configuration: center -light,
w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink.
(http://vision.ucsd.edu/content/yale-face-database).
This database was converted and rescale to bmp for mat and the names where inserted in the
(TrainedLabels.txt) file.
Hardware used: Full assembled prototype. (Attachment 46 - Prototype Recon Outpost)
O.S used: Windows XP Professional SP3
TESTING
Test Case ID Description

001 Detection performance under low lights conditions.
002 Detection performance under other face angles
003 Trace Mode distance range
Test Case ID: 001

Test Description: Detection performance under low lights conditions
Purpose: Compare who much light conditions affect detection and recognition under 90
lux. of illumination in a 10m2 area.
Input Data: Yale Face Database plus three more subjects.
Except Result: The system will detect and recognize the three new subjects
Actual Result: In a straight line towards camera two subjects were detected and
recognized in the first pass, the other subject was identified and
recognized on the second pass.
Pass / Fail: Pass
Remarks: N/A
Test Case ID: 002

Test Description: Detection performance under other face angles
Purpose: Test metrics of haarcascade_frontalface _default .xml” for different face
angles.
Input Data: Three test subject in strait line path
Except Result: The system will detect and recognize the three subjects in different angles.
Actual Result: System fails at z-axis face rotation after +/- [-30º +30º]
System fails at x-axis face rotation after +/- [-15º +30º]
System fails at y-axis face rotation after +/- [-20º +20º]
Pass / Fail: Pass
Remarks: N/A
Test Case ID: 003
37
Test Description: Trace Mode distance range

Purpose: Test distance of the Trace Mode under infrared light illumination in exterior
environment.
Input Data: One test subject in strait line path
Except Result: The system will trace contours at 3mts distance.
Actual Result: System starts to identify subject at 3.5mts and complete the contour of the
body at 2.5mts from the camera.
Pass / Fail: Pass
Remarks: N/A
Results: We tested the three main algorithms in the software to measure the limits and capability of the system.
The real time performance of the software under the systems hardware proven to be acceptable in the
conditions tested. Other tests were performed and the result can be viewed at (http://youtu.be/Hf7Fnb0ZaCA).
FUTURE OPTIONS IMPLEMENTATIONS
For future implementations that will improve the system is recommended:
1. The sensibility of the Minimum Neighbors Threshold and the Scales Increase Rate may be adjusted by
the user.
2. Save and Record video images.
3. Motion sensor trigger
4. Internet streaming video image and hardware control
CONCLUSION
By the end of this project we were able to implement a complete low cost surveillance prototype that provides
complete face detection and recognition also provided the system with night vision capability options. The
success of our prototype is demonstrated in the test results. The system does not need sophistical installation
and the portability of the prototype aloes the user to diversify the use.
Future Work:
Project Night Watch, system capable of detecting and recognizing faces for over 1km of distance in 0.1 lux of
illumination. The system is compose by objective optics and infrared laser illumination. The system is remotely
controlled, the video is send by internet streaming connection and the system can rotate the cameras in
various angles, the system can be accessed from anywhere in the world.
The authors wishes to thank everyone who was involved in this project especially project supervisor Drª
Daniela Cruz (my teacher!!), Americo Santos (Surveillance Security Expert), Alison Cynthia Budde (test member
crew), special thanks to Jeremy and Julia Ashford (my uncle and aunt for their special support) Douglas Charles
Budde (Graphic Design Expert) and special thanks to Our Lady of Fatima for the blessing.
BIBLIOGRAPHY
1. J. F. Canny: A computational approach to edge detection. IEEE Trans. Pattern Analysis and
Machine Intelligence, 8 (6), 1986, 679-698.
2. C. Grigorescu, N. Petkov and M. A. Westenberg: Contour and boundary detection improved by
surround suppression of texture edges, Image and Vision Computing, 22 (8), 2004, 609-622.
38
3. Paul Viola - Robust Real-time Object Detection - Mitsubishi Electric Research Labs, Cambridge, MA
02142
4. Face Recognition Using Eigenfaces, Matthew A. Turk and Alex P. Pentland, MIT Vision and
Modeling
Lab, CVP‘ ’91.
5. Eigenfaces for Recognition, Matthew A. Turk and Alex P. Pentland, Journal of Cognitive
Neuroscience ’91.
6. Jan Fajfr: Face recognition in RIA applications, 20 October 2011.
7. Emgu CV: OpenCV in .NET: http://www.emgu.com/wiki/index.php/Main_Page.
8. Clint Cielto: Multiple face detection and recognition in real time, September 2011
9. OpenCV 2.4.5.0 documentation - http://docs.opencv.org/genindex.html
10. Robin Hewitt: Face Recognition With Eigenface: SERVO Magazine, February 2007
11. Arduino Forum, http://forum.arduino.cc
39
RECON OUTPOST
RESEARCH & ANALYSIS SYSTEMS
ATTACHMENTS REPORT
1
Table of Contents
Diagrams...................................................................................................................................... 4
Software Interface...................................................................................................................... 25
Hardware................................................................................................................................... 29
Webcam Modification:............................................................................................................ 30
IR Illumination array:............................................................................................................... 31
Assembling Hardware: ............................................................................................................ 32
Schematic Wiring Diagram:...................................................................................................... 35
2
Table of Images
Attachment 1 - S.C. MD1.............................................................................................................................................................. 4
Attachment 2 - S.C. MD2.............................................................................................................................................................. 5
Attachment 3 - S.C. MD3.............................................................................................................................................................. 6
Attachment 4 - Face Detection SD............................................................................................................................................... 7
Attachment 5 - Face Detection AD .............................................................................................................................................. 8
Attachment 6 - Face Recognition SD ........................................................................................................................................... 8
Attachment 7 - Face Recognition AD ........................................................................................................................................... 9
Attachment 8 - Ca mera Control SD.............................................................................................................................................. 9
Attachment 9 - Ca mera Control AD ........................................................................................................................................... 10
Attachment 10 - Infrared SD...................................................................................................................................................... 10
Attachment 11 - Infrared AD...................................................................................................................................................... 11
Attachment 12 - Green Fil ter SD................................................................................................................................................ 11
Attachment 13 - Green Fil ter AD ............................................................................................................................................... 12
Attachment 14 - Trace Contours SD .......................................................................................................................................... 12
Attachment 15 - Trace Contours AD .......................................................................................................................................... 13
Attachment 16 - Class Diagram Abstract................................................................................................................................... 13
Attachment 17 -Face recognition code SD ................................................................................................................................ 14
Attachment 18 - S.D. Rotate Ca mera......................................................................................................................................... 15
Attachment 19 - S.D Select Module........................................................................................................................................... 16
Attachment 20 - S.D. Trace ........................................................................................................................................................ 17
Attachment 21 - S.D. Night Mode.............................................................................................................................................. 18
Attachment 22 - S.D. IR.............................................................................................................................................................. 19
Attachment 23 - SD login ........................................................................................................................................................... 19
Attachment 24 - S.D. Brightness ................................................................................................................................................ 20
Attachment 25 - Dependency Graphs Main .............................................................................................................................. 21
Attachment 26- Dependency Graphs MD1................................................................................................................................ 22
Attachment 27 - Dependency Graphs MD2............................................................................................................................... 23
Attachment 28 - EmguCV Architecture...................................................................................................................................... 24
Attachment 29 – Login............................................................................................................................................................... 25
Attachment 30 - Main Interface ................................................................................................................................................ 25
Attachment 31 - Interface Face Detection ................................................................................................................................ 26
Attachment 32 - Interface Face Recognition ............................................................................................................................. 26
Attachment 33 - Interface Database Manager.......................................................................................................................... 27
Attachment 34 - Interface Night Mode ..................................................................................................................................... 27
Attachment 35 - Interface Infrared Mode ................................................................................................................................. 28
Attachment 36 - Interface Trace Mode ..................................................................................................................................... 28
Attachment 37 - Equipment list................................................................................................................................................. 29
Attachment 38 - Webcam Schematic ........................................................................................................................................ 30
Attachment 39 - Webcam Filter................................................................................................................................................. 31
Attachment 40 - Arduino IR Array ............................................................................................................................................. 32
Attachment 41 - PCB La yout...................................................................................................................................................... 32
Attachment 42 - Ca m Assembling.............................................................................................................................................. 33
Attachment 43 - Servo Arduino ................................................................................................................................................. 33
Attachment 44 - Unit Assembling.............................................................................................................................................. 34
Attachment 45 - Wire Diagram.................................................................................................................................................. 35
Attachment 46 - Prototype Recon Outpost............................................................................................................................... 36
3
Diagrams
4
5
6
Attachment 4 - Face Detection SD
7
Detect faces in
image
Wait for stream
No face
No detected
Image
Highlight
Haar Cascade detected faces
algorithm
Attachment 5 - Face Detection AD
Attachment 6 - Face Recognition SD
8
Highlight
Image
detected faces
Haar Cascade
algorithm PCA algorithm
Detect faces in Search for

image match in folder
face No
detected
Attachment 7 - Face Recognition AD
Attachment 8 - Camera Control SD
9
Wait for new
position
no is port yes
open
Calculate
Display error
position
Send comand
Return Position
to hardware
Attachment 9 - Camera Control AD
Attachment 10 - Infrared SD
10
Attachment 11 - Infrared AD
Attachment 12 - Green Filter SD
11
Connect
Camera
no Wait for yes

stream
Copy frame to
Display error
memory
Remove red
and blue colors
Return Image
Attachment 13 - Green Filter AD
Attachment 14 - Trace Contours SD
12
Wait for image
no yes
image
Canny Edge
Image Pyramid
Detector
algorithm
algorithm
Trace Contours
Return Image
Attachment 15 - Trace Contours AD
Attachment 16 - Class Diagram Abstract
13
Attachment 17 -Face recognition code SD
14
Attachment 18 - S.D. Rotate Camera
15
Attachment 19 - S.D Select Module
16
Attachment 20 - S.D. Trace
17
Attachment 21 - S.D. Night Mode
18
Attachment 22 - S.D. IR
Attachment 23 - SD login
19
Attachment 24 - S.D. Brightness
20
Attachment 25 - Dependency Graphs Main
21
Attachment 26- Dependency Graphs MD1
22
Attachment 27 - Dependency Graphs MD2
23
Attachment 28 - EmguCV Architecture
24
Software Interface
Attachment 29 – Login
Attachment 30 - Main Interface
25
Attachment 31 - Interface Face Detection
Attachment 32 - Interface Face Recognition
26
Attachment 33 - Interface Database Manager
Attachment 34 - Interface Night Mode
27
Attachment 35 - Interface Infrared Mode
Attachment 36 - Interface Trace Mode
28
Hardware
Modelo: 7 UV LED
Link: http://www.securitycamera2000.com/products/Mini-420TVL-1%7B47%7D3-Sony-CCD-
Color-Camera-Audio%7B47%7DVideo-with-12-IR-LED.html
Descricao: UV Leds 365nm
Modelo: 12 IR LED
Link: http://www.securitycamera2000.com/products/Mini-420TVL-1%7B47%7D3-Sony-CCD-
Color-Camera-Audio%7B47%7DVideo-with-12-IR-LED.html
Descricao: IR Leds
Modelo: Microsoft LifeCam HD-5000

Link: http://www.microsoft.com/hardware/en-us/p/lifecam-hd-5000
Descricao: LifeCam HD-5000 offers a 720p HD sensor and automatically sharpens your images
with
Auto Focus.
Modelo: Intel D525MW Fanless Dual Core Atom Mini-ITX Board

Link: http://www.mini-itx.com/store/?c=47
Descricao: 1.8GHz Dual Core, Intel GMA 3150 Video,4GB of system memory.
Modelo: Servo - Small

Link: http://www.ptrobotics.com/product.php?id_product=1089
Descricao: low-cost, high quality servo for all your mechatronic needs.
Modelo: Pan & Tilt Bracket Medium Kit

Descricao: Pan and Tilt assembly for horizontal surface mount.
Modelo: Arduino Uno R3

Descricao: Arduino Uno is a microcontroller board based on the ATmega328.
Modelo: Lilliput UM1010-NP/T

Link: http://www.lilliputuk.com/monitors/usb/um1010/
Descricao: 10" touch screen USB monitor
Modelo: PicoPSU-150-XT DC-DC Power Converter, 150 W

Link: https://www.logicsupply.eu/power-supplies/dc-converters/picopsu-150-xt/
Descricao: The picoPSU-150-XT is the world's tiniest 12-volt DC-DC ATX power supply unit (PSU).
Modelo: HP Webcam HD-2200

Link: http://h10010.www1.hp.com/wwpc/ca/en/ho/WF06c/A1-329290-3736366-3736367-
Attachment 37 -3736367-5081841-5081844.html
Equipment list
Descricao: 720P video at up to 30 frames per second.
29
Webcam Modification:
Theory: The light sensors used in digital cameras are either CCDs (charge-coupled detectors) or
CMOS detectors — the latter is more sensitive and appears in newer, fancier, and pricier webcams.
Both types are more sensitive to infrared light near the visible spectrum than they are to visible
light.
So for a CCD/CMOS camera to match what a human sees, all of the infrared light (wavelengths
longer than 700 nanometers) must be cut out with a filter. By removing this IR-cutout filter,
“hacking the webcam” we can restore the response to infrared light.
Objective: Remove manufacturers IR-cutout filter from webcam.
Procedure:
Disassembling webcam casing

Removing filter
Disable auto focus
Hardware:
QT. Model: Description
1 Microsoft LifeCam HD-5000 720p HD sensor and automatically sharpens your images
with Auto Focus.
Schematic Diagram:
Attachment 38 - Webcam Schematic
Removing the filter:
The IR-blocking filter is a plastic coating, reflecting reddish, on the lens surface closest to the light
detector CCD or CMOS. Take a sharp, small penknife, and scrape the coating off.
30
Attachment 39 - Webcam Filter
References:
http://www.lpi.usra.edu/education/fieldtrips/2005/activities/ir_spectrum/ir_webcam.html
IR Illumination array:
Theory: IR LED illuminator a method of lighting up an area using infrared light. Since infrared is
outside the visual range of the vast majority of people, this is essentially bathing an area in totally
invisible light. On the other hand, if the area is monitored by a device capable of seeing into the
IR range, then the area may be brightly lit to that device alone.
Objective: Create an array of IR bright leds, with brightness control via software.
Procedure:
Create PCB for electronic components
Hardware:

1 Capacitor 0.1F - Panasonic Capacitor
8 220-ohm Resistor - Panasonic Resistor
1 74HC595N-Philips 8-bit s shift register
8 5mm IR LED - 940nm Super-bright IR Leds
1 PCB fiberglass Fiberglass Printed Circuit Board
31
Schematic Diagram:
Attachment 40 - Arduino IR Array
PCB Layout:
Attachment 41 - PCB Layout
References:
http://www.elcojacobs.com/shiftpwm/
http://arduino.cc/en/Tutorial/ShiftOut
Assembling Hardware:
Objective: Provide information of the
sensors to the software.
Procedure:
Integrate all the modules

Assembler PC
connections
Hardware:
1 Triple External casing Metal body housing for security cams
1 Hoya 52mm Infrared R72 Filter R72 passes only infrared rays above 720nm
1 Intel D525MW Board Fanless Dual Core Atom Mini-ITX Board
32
2 Servo - Small High quality servo for all your mechatronic
1 Pan & Tilt Bracket Medium Kit Pan and Tilt assembly for horizontal surface mount
1 Arduino Uno R3 Microcontroller board based on the ATmega328
1 Lilliput UM1010-NP/T 10" touch screen USB monitor
1 PicoPSU-150-XT DC-DC Power 12-volt DC-DC ATX power supply unit
1 HP Webcam HD-2200 720P video at up to 30 frames per second.
1 Mini PC Case External casing for motherboard
Module Units:
1. Vision unit, composes by optics, webcams and

sensors.
2. Electronic unit, compose by Arduino and servos.
3. Brain unit, compose by computer.
Assembling Vision Unit:
UV Leds Laser Microsoft LifeCam IR Leds

Module HD-5000
Hoya 52mm
Infrared
R72 Filter
USB Cam Ultrasonic Range

HP Webcam HD-2200 Finder - Maxbotix LV-
EZ1
Attachment 42 - Cam Assembling
Assembling Electronic Unit:
Attachment 43 - Servo Arduino
33
Assembling Brain Unit:
90º degrees
flip
Touch Screen
PC Casing
Monitor
16 cm
5 cm
6 cm
19 cm 26 cm
Power
Supply
HD
Motherboard
Attachment 44 - Unit Assembling
34
35
Attachment 45 - Wire Diagram
LCD
CAM HD
CAM HD IR
ARDUINO
Schematic Wiring Diagram:
Servo X Input = 05
Servo Y Input =
06 Sonar Input =
07 Laser Input =
RECON OUTPOST PROJECT
09 UV Input =
Main Hardware Wire
10 IR Leds Input
Connections Layout REV.1.1
= 08 IR Leds
Input = 12 IR
Leds Input = 13
Attachment 46 - Prototype Recon Outpost
36
Read this manual before installing.
Always follow instructions for proper use.
1
Table of Contents
1. Introduction ...................................................................................................................................4
1.1 Audience Assumptions.........................................................................................................4
1.2 Document Overview ............................................................................................................4
2. Recon Outpost Hardware Setup.......................................................................................................4
2.1 System Components..................................................................................................................4
2.2 System Connections ..................................................................................................................5
3. Using Recon Outpost System......................................................................................................6
3.1 LOGIN..............................................................................................................................6
3.2 MAIN WINDOW ...............................................................................................................6
3.3 Tracking Mode Module – FACE DETECTION........................................................................6
3.3.1 Tracking Mode Module – FACE RECOGNITION................................................................7
3.3.2 Tracking Mode Module – DATABASE MANAGER.............................................................8
3.4 Night Vision Mode Module ...............................................................................................8
3.4.1 Night Vision Mode Module – NIGHT VISION ...................................................................8
4. Troubleshooting.......................................................................................................................... 10
2
LEGAL NOTICE:
Recon Outpost product is designed to be a prototype for testing purpose to test a concept.
Recon Outpost disclaims liability associated with the use of non-default hardware.
Recon Outpost makes no representations concerning the legality of certain product applications
such as the making, transmission, or recording of video of others without their knowledge and/or
consent. We encourage you to check and comply with all applicable local, state, and federal
laws and regulations before engaging in any form of surveillance.
To reduce the risk of electric shock, do not remove cover (or back). No user serviceable
parts inside. Refer servicing to qualified service personnel.
Do not use the camera in extreme environments where high temperatures or high humidity
exists. Use the camera under conditions where temperatures are between -4°F ~ 122°F (-
20°C
~ 50°C), and humidity is below 85%.
Whether or not the camera is used outdoors, never point it toward the sun. Use caution
when operating the camera in the vicinity of spot lights or other bright lights and light reflecting
objects.
If installed close to a TV, radio transmitter, magnet, electric motor transformer or audio
speakers the magnetic field generated may interfere with or distort the image.
To prevent damage, do not drop any component or subject it to strong shock or

vibration.
Do not install or operate in small, unventilated areas. Heat buildup can significantly reduce
the performance and operating life of the product and may cause a fire.
3
1. Introduction
Welcome to Recon Outpost Configuration User’s Guide document. This document provides
sufficient guidance for hardware and software users to securely use the product in accordance
with the requirements. This document is specifically targeted at the final user.
1. Audience Assumptions
This document assumes the audience is generally familiar with computer peripheral installation.
2. Document Overview
This document has the following chapters:
Chapter 1, “Introduction”, introduces the purpose and structure of the document and the
assumptions of the audience.
Chapter 2, “Recon Outpost Hardware installation”, describes the evaluated installation.
Chapter 3, “Using Recon Outpost System”, describes the use and environment of the
software.
2. Recon Outpost Hardware Setup
1. System Components
1 Unit pre-assembled
3 USB cables
1 Power Supply
1 Car battery cable
4
90º degrees
flip
Cameras
Touch Screen
Monitor
5 cm
Camera Interface
PC Casing 19 cm
Figure 1 - System dimensions
2. System Connections
All hardware is pre-assembled don´t attempt to open or modify any components, unless you
know what or you doing.
This unit is a PROTOTYPE don’t expose to water, humidity or ignore common sense
with electrical devices.
1. Determine where the unit will be mounted.

2. Monitor - Connect the USB-Y cable to the available USB slots in the pc casing.
3. Camera Interface - Connect the other 3 (three) USB cables to the available USB slots in
the pc casing. Se e Fig.2
4. Power - Connect the power connectors direct to a DC12V car battery or use the power
supply to connect to outlets in home. Max. 220V - 70W. Se e Fig.2
POW ER
USB
Figure 2 - PC Unit back cover, 2 (two) USB front
5
3. Using Recon Outpost System
3.1 LOGIN
a. To enter the passwords just tap the numeric keys.
b. The password is provided on the cover of this
manual.
Here you enter password

to login the system
Figure 3 - Login
3.2 MAIN WINDOW

a. By default Face Tracking comes in the package, if you purchase other Modules please follow
the
instruction for their installation.
b. Select the Module you want to monitor, by tapping on the button. All available modules
and functions are highlighted with a light green color.
Exit:
Shutdown the system
Help Monitor:
Available module descriptions
UP/DOWN:
Scroll text up and down
Figure 4 - Main Window
3.3 Tracking Mode Module – FACE DETECTION

This module performs face detection and face recognition, please follow the instructions describe
in this manual for efficient use of all features.
6
1. Video Monitor: In this area the face detection crosshair will attempt to recognize all
humans face in video. If no face is detected, crosshair will not appear.
2. Terminal: The terminal area will show all errors and messages from the system.
3. Identification Analysis: Allows to identifying the detected face.
TIP: Wait to crosshair to appear, this means the face has been detected.
4. Camera Rotation: Allows rotating the camera position.

5. Back: Go’s to initial window.
Figure 5 - Face detection
3.3.1 Tracking Mode Module – FACE RECOGNITION

The system will search database for all detected faces and display their name. If the system
doesn’t find a match, the system will display the most approximated face result that has the
most similar traces to the detected face.
1. In the video area the detected face or faces or displayed in the upper right area.
2. Name of the detected face.
3. Button to insert the face in the database.
4. If more than one face is detected this button allows you to display the next face in the
upper right display area
5. Back: Go’s to FACE DETECTION window.
7
Figure 6 - Face recognition
3.3.2 Tracking Mode Module – DATABASE MANAGER

1. Tap each key to enter a name.
2. After entering the name, tap SAVE button to insert in to database, the detected face will be
save and ready to be detected next time it show in FACE RECOGNITION window.
3. Displays the face that will be saved.\
4. Back: Go’s to FACE RECOGNITION window in case you need to add another face to database.
4
2
1 3
Figure 7 - Database Manager
4. Night Vision Mode Module

This module allows you to view live video in adverse weather conditions. Gives users the
capability to analysis and observe details that are not visible to the naked eye.
1. Night Vision Mode Module – NIGHT VISION
1. Video Monitor
2. Brightness: Adjust the intensity of the infrared leds.
3. Back: Go’s to initial window.
4. Video Mode:
8
1. NIGHT - Applies green filter to video.Fig.8
2. IR - Change video to gray tons. Fig.9
3. TRACE - Is an edge detection operator that uses a multi-stage algorithm to detect a
wide range of edges in images. This will draw all edges in black and white. Fig.10
5. Camera Rotation: Allows rotating the camera position.
Figure 8 - Night Vision
Figure 9 -IR
Figure 10 - Trace
9
4. Troubleshooting
System doesn’t start or no image appears on

screen? Check power cable and USB connections.
System has a delay boot up. Please wait at least
2min.
No face is detected?
Try decreasing the distance of the camera to the target, the minimum size face for detection
is 40x30 pixels
When I try to identify a face, system shows no face detected.

To identify a face, the face must have been detected by the crosshair in Face Detection.
Terminal displays the message” HARDWARE FAILUER: Check USB cable”.

Check or replace USB cable.
Motor keeps running, when system is powered on.

Sometimes this can happen; our engineers are working day and night on this issue…
For any question or help: help.reconoutpost@mail.com
10

Recon Outpost Research & Analysis Systems

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recon Outpost Research & Analysis Systems

Uploaded by

Copyright:

Available Formats

RECON OUTPOST

RESEARCH & ANALYSIS SYSTEMS

Supervised by Drª Daniela

“Give a shelterless man tools and he will buil d a home.

Gaussian pyramid: Used to downsample images.

WHAT IS FACE RECOGNITION?

Figure 1 - Face Recognition

DIFFERENT APPROACHES OF FACE RECOGNITION

Recognition algorithms can be divided into two main approaches:

Popular recognition algorithms include:

3. Principal Component Analysis using Eigenfaces,(PCA)

Figure 2 -Photometric stereo image.

Figure 3 - Geometric facial recognition.

WHY SELECT PCA BASED EIGENFACES ALGORITHM FOR THE PROJECT?

Simple rectangular features, called Haar features

For this purpose there are some basic prerequisites.

system. The most common problems are:

Video surveillance cost installation (Figure 4)

Figure 4 - Samsung 17 Quad CCTV Observation System

- System software modules can be purchased individually

SWOT ANALYSIS OF OUR PRODUCT

Low cost Hardware

Figure 6 - Swot analysis

Our customer can be divided in two groups:

Residential User Commercial Owners

can contain Green Filter

WBS - GANTT CHART

Figure 8 - Gantt chart timeline

Figure 9 - Gantt chart table information

Roles and key responsibilities for each labor type:

- Programmer- Software Development

Items of equipment to be used and their purposes:

- Visual Studi o C# 2008

Complete list of equipment: Attachment 29 - Equipment list

Project viability Hardware assembling

Programming Tools( C#, EmguCV)

Software preparation Project Documentati on

LOW FIDELITY PROTOTYPE SKETCH

Figure 9 - Main menu screen Figure 10 - Splash screen

Figure 11 - Login screen

Figure 14 - Save data screen

Figure 13 - Recognition screen

Figure 15 - Night vision screen

SYSTEM AND SOFTWARE DESIGN

2. Non Functional Requirements

Figure 16 - Use Case

Use Cases Description

Sequence Diagram: Attachment 3 - Face Detection SD

Use Cases Description

Post- System displays name(s) of face(s).

Sequence Diagram: Attachment 5 - Face Recognition SD

Use Cases Description

Sequence Diagram: Attachment 7 - Camera Control SD

Use Cases Description

Sequence Diagram: Attachment 9 - Infrared SD

Use Cases Description

Camera must be connected

Sequence Diagram: Attachment 11 - Green Filter SD

Use Cases Description

Sequence Diagram: Attachment 13 - Trace Contours SD

Figure 17 - Class Diagram

Class Diagram: Attachment 15 - Class Diagram