Detecting Faces in Images: A Survey
, David J. Kriegman,
, andNarendra Ahuja,
—Images containing faces are essential to intelligent vision-based human computer interaction, and research efforts in faceprocessing include face recognition, face tracking, pose estimation, and expression recognition. However, many reported methodsassume that the faces in an image or an image sequence have been identified and localized. To build fully automated systems thatanalyze the information contained in face images, robust and efficient face detection algorithms are required. Given a single image, thegoal of face detection is to identify all image regions which contain a face regardless of its three-dimensional position, orientation, andlighting conditions. Such a problemis challenging because faces are nonrigid and have a high degree of variability in size, shape, color,and texture. Numerous techniques have been developed to detect faces in a single image, and the purpose of this paper is tocategorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics, andbenchmarking. After analyzing these algorithms and identifying their limitations, we conclude with several promising directions forfuture research.
—Face detection, face recognition, object recognition, view-based recognition, statistical pattern recognition, machinelearning.
the ubiquity of new information technology andmedia, more effective and friendly methods forhuman computer interaction (HCI) are being developedwhich do not rely on traditional devices such as keyboards,mice, and displays. Furthermore, the ever decreasing price/performance ratio of computing coupled with recentdecreases in video image acquisition cost imply thatcomputer vision systems can be deployed in desktop andembedded systems , , . The rapidly expand-ing research in face processing is based on the premise thatinformation about a user’s identity, state, and intent can beextracted from images, and that computers can then reactaccordingly, e.g., by observing a person’s facial expression.In the last five years, face and facial expression recognitionhave attracted much attention though they have beenstudied for more than 20 years by psychophysicists,neuroscientists, and engineers. Many research demonstra-tions and commercial applications have been developedfrom these efforts. A first step of any face processing systemis detecting the locations in images where faces are present.However, face detection from a single image is a challen-ging task because of variability in scale, location, orientation(up-right, rotated), and pose (frontal, profile). Facialexpression, occlusion, and lighting conditions also changethe overall appearance of faces.We now give a definition of
Given anarbitrary image, the goal of face detection is to determinewhether or not there are any faces in the image and, if present, return the image location and extent of each face.The challenges associated with face detection can beattributed to the following factors:
The images of a face vary due to the relativecamera-face pose (frontal, 45 degree, profile, upsidedown), and some facial features such as an eye or thenose may become partially or wholly occluded.
Presence or absence of structural components.
Facial features such as beards, mustaches, andglasses may or may not be present and there is agreat deal of variability among these componentsincluding shape, color, and size.
The appearance of faces aredirectly affected by a person’s facial expression.
Faces may be partially occluded by otherobjects. In an image with a group of people, somefaces may partially occlude other faces.
Face images directly vary fordifferent rotations about the camera’s optical axis.
When the image is formed,factors such as lighting (spectra, source distributionand intensity) and camera characteristics (sensorresponse, lenses) affect the appearance of a face.There are many closely related problems of facedetection.
aims to determine the imageposition of a single face; this is a simplified detectionproblem with the assumption that an input image containsonly one face , . The goal of
facial feature detection
isto detect the presence and location of features, such as eyes,nose, nostrils, eyebrow, mouth, lips, ears, etc., with theassumption that there is only one face in an image , .
compares an input image(probe) against a database (gallery) and reports a match, if
34 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24, NO. 1, JANUARY 2002
M.-H. Yang is with Honda Fundamental Research Labs, 800 CaliforniaStreet, Mountain View, CA 94041. E-mail: firstname.lastname@example.org.
D.J. Kriegman is with the Department of Computer Science and BeckmanInstitute, University of Illinois at Urbana-Champaign, Urbana, IL 61801.E-mail: email@example.com.
N. Ahjua is with the Department of Electrical and Computer Engineeringand Beckman Institute, University of Illinois at Urbana-Champaign,Urbana, IL 61801. E-mail: firstname.lastname@example.org. Manuscript received 5 May 2000; revised 15 Jan. 2001; accepted 7 Mar. 2001.Recommended for acceptance by K. Bowyer.For information on obtaining reprints of this article, please send e-mail to:email@example.com, and reference IEEECS Log Number 112058.