Real-Time Facial

Recognition

MATCH!

EECE 279: Real-Time Systems Design

Vanderbilt University
Ames Brown & Jason Cherry

Topics Of Discussion

Why real-time face recognition?

What is difficult about real-time face recognition?
In general how is face recognition done?
Eigenfaces
Other face recognition algorithms
Opposition
Future of face recognition

Why Real-Time Face

Recognition?
Security
Fight terrorism
Find fugitives

Personal information access

ATM
Sporting events
Home access (no keys or passwords)
Any other application that would want personal identification

Improved human-machine interaction

Personalized advertising
Beauty search

Real-Time Face Recognition

System Requirements
Want the system to be inexpensive enough to use
at many locations
Match within seconds
Before the person walks away from the advertisement
Before the fugitive has a chance to run away

Ability to handle a large database

Ability to do recognition in varying environments

What Is Difficult About RealTime Face Recognition

Lighting variation
Orientation variation (face angle)
Size variation
Large database
Processor intensive
Time requirements

General Image Types

Still image (digital photograph)
AmIU

Dynamic image
Video camera

FERET database
Contains images of 1,196 individuals, with up to
5 different images captured for each
individual
Often used to test face recognition algorithms
Information on obtaining the database can be
found here:
http://www.itl.nist.gov/iad/humanid/feret/

General Face Recognition

Steps
Face Detection
Face Normalization
Face Identification

Face Detection
In General
Locate face in a given image
Separate it from the scene

Different Approaches
Motion detecting and head
tracking
Face Space distance

Face Detection: Face Space

Face Detection: Motion

Detecting and Head Tracking

Face Detection: Motion

Detecting and Head Tracking

Face Normalization
Adjustment

Expression
Rotation
Lighting
Scale
Head tilt
Eye location

Face Normalization: FERET

face2norm
1. Image is rotated to align the eyes (eye coordinates must be
known).
2. The image is scaled to make the distance between the eyes
constant. The image is also cropped to a smaller size that
is nearly just the face.
3. A mask is applied that zeros out pixels not in an oval that
contains the typical face. The oval is generated
analytically.
4. Histogram equalization is used to smooth the distribution
of gray values for the non-masked pixels.
5. The image is normalized so the non-masked pixels have
mean zero and standard deviation one.

Face Identification
Application of a face recognition algorithm

PCA Algorithms
Principle Component Analysis
Look at the principal components of the
distribution of faces, or the eigenvectors of the
covariance matrix of the set of face images
Eigenfaces

Eigenfaces Algorithm
Eigenfaces Initialization
1. Acquire an initial set
of face images (the
training set)

Eigenfaces Algorithm
2. Calculate the eigenfaces
from the training set,
keeping only the M images
that correspond to the
highest eigenvalues. These
M images define the face
space. As new faces are
experienced, the eigenfaces
can be updated or
recalculated

Eigenfaces Algorithm
3. Calculate the corresponding distribution in M-dimensional
weight space for each known individual, by projecting
their face images onto the face space.

Eigenfaces Algorithm
Eigenfaces Recognition
1. Calculate a set of weights
based on the input image
and the M eigenfaces by
projecting the input image
onto each of the
eigenfaces.
2. Determine if the image is
a face at all by checking
to see if the image is
sufficiently close to face
space.

3. If it is a face, classify the

weight pattern as either a
known person or as
unknown.
4. (Optional) Update the
eigenfaces and/or weight
patterns.

Eigenfaces Problems
Recognition performance decreases quickly as the
head size, or scale, is misjudged. The head size in
the input image must be close to that of the
eigenfaces for the system to work well
In the case where every face image is classified as
known, a sample system achieved approximately
96% correct classification averaged over lighting
variation, 85% correct averaged over orientation
variation, and 64% correct averaged over size
variation

Parameter Based Facial

Recognition
Facial image is analyzed and reduced to small set
of parameters describing prominent facial features
Major features analyzed are: eyes, nose, mouth
and cheekbone curvature
These features are then matched to a database
Advantage: recognition task is not very expensive
Disadvantage: the image processing required is
very expensive and parameter selection must be
unambiguous to match an individuals face

Template Based Facial

Recognition
Salient regions of the facial image are extracted
These regions are then compared on a pixel-bypixel basis with an image in the database
Advantage is that the image preprocessing is
simpler
Disadvantage is the database search and
comparison is very expensive

Real-Time System using

Template Recognition
Implemented on a IBM PC w/ a video camera,
image digitizer, and custom VLSI image correlator
chip (340 Mop/second).
Needed single frontal facial image under semicontrolled lighting conditions
Took the system 2 to 3 seconds to identify a user
from 173 images of 34 persons
88% recognition rate

How the System Worked

Artificial Neural Networks in

Real-Time Face Recognition
Use many of the same algorithms described
before but with back propagation ANNs
Disadvantages: Complex and difficult to train
Difficult to implement
Sensitive to lighting variation

There are many face

recognition algorithms

LDA (Linear Discriminant Analysis)

Bayesian Classifier
Gabor Wavelet Algorithm
Elastic graphs

Get more information and source code at

http://www.cs.colostate.edu/evalfacerec/index.html

Not Everyone Loves Face

Recognition

Critics say it produces too many false positives

Invasion of privacy
To easy to misuse for wrong purposes
Technology is not accurate enough given the
current technology and algorithms

Future Of Face Recognition

Some consider the problem impossible

No standard way of approaching the problem
Advancements in hardware and software
Slow integration into society in limited
environments
Very large potential market

Questions?

References
M. Turk and A. Pentland. Eigenfaces for Recognition.
Journal of Cognitive Neuroscience, 3(1), 1991
C. Nastar and M. Mitschke. Real-Time Face Recognition
Using Feature Combination. In Proceedings of the Third
IEEE International Conference on Automatic Face and
Gesture Recognition, Nara, Japan, April 1998
J. Gilbert and W. Yang. A Real-Time Face Recognition
System using Custom VLSI Hardware. Harvard
Undergraduate Honors Thesis in Computer Science, 1993.

Topics Of Discussion

Why real-time face recognition?

What is difficult about real-time face recognition?
In general how is face recognition done?
Eigenfaces
Other face recognition algorithms
Opposition
Future of face recognition

Real-Time Facial
Recognition

MATCH!

EECE 279: Real-Time Systems Design

Vanderbilt University
Ames Brown & Jason Cherry