A Scalable Real-Time Algorithm for Facial Recognition

Anand Gupta, Anhad Mohananey, Ishan Nigam and Sahil Goel
Netaji Subhas Institute of Technology, University of Delhi, New Delhi, India. e-mail:

Abstract. An interesting and evergreen problem in the field of Computer Vision is the identification of humans by recognition through their faces. The problem has been addressed by a number of researchers in the past, employing a range of techniques. Even though the algorithms in these techniques perform accurately in real-time, they lack in design to scale temporally with the size of the dataset. To mitigate this limitation, we propose a scaleinvariant real-time algorithm. It works with the application of two successive filters, which are based on the minimum Euclidean distance algorithm and the Principal Component Analysis based Eigen face algorithm. The proposed algorithm has been validated on the wellknown Essex face dataset. In a comparative study with algorithms present in the literature, it is found that the proposed algorithm achieves better accuracy than the current state-of the art and scales well with an increase in the dataset size. Keywords: Eigen faces, Euclidean distance, facial recognition, normalized correlation.

1. Introduction For a long time, the process of identification of humans has primarily been through the individual faces. It is but natural that the approach be extended to automated recognition of humans. Automated facial recognition has become an increasingly important area of research with applications in security, including surveillance and detecting fraud, and also has increasing applications on social networks such as automatic face tagging in photographs. One of the landmark approaches in facial recognition has been the Principal Component Analysis approach applied to Eigen Faces. The notion of using principal components to represent human faces has been developed by Sirovich and Kirby in [1] and was first applied to face recognition by Turk and Pentaland in [2]. This approach has been further improved upon using two dimensional PCA by Yang et al. in [12]. A common feature of these algorithms is that the run-time of the algorithm depends on the number of subjects in the database. 1.1 Related work Several approaches [1–8], have been proposed in the Computer Vision literature for human identification using face recognition, 45 years since Bledsoes work on modeling automated facial recognition [3]. In [2], Eigen faces have been used for recognition, wherein face images are projected
K. R. Venugopal, K. B. Raja and L. M. Patnaik (Eds.) ICSIP 2013, pp. 78–85. © Elsevier Publications 2013.

Etemad and Chellappa [7] have employed discriminant analysis for face recognition which. although simple. 2. Section 4 summarizes the experiments carried out to validate the algorithm along with the comparison of its performance to the existing algorithms.4 Organisation We have organized the paper as follows: Section 2 describes the extraction of landmarks on the test image. The algorithm in [6] uses face parts such as eyes. 1. 3. mouth and hair as states in the Hidden Markov Model for face recognition and detection. which have been used to calculate the geometrical features. 1. direct Linear Discriminant Analysis algorithm has been applied. nose. which is a very small subset of the entire dataset. 79 .3 Contributions We present a face recognition algorithm that uses geometrical features to create a subset of matched face images prior to the application of the PCA based Eigen face algorithm. Normalized correlation is used to localize the eye region. The arrangement reduces the number of inputs to the Eigen face algorithm so that it runs on a predetermined number of subject faces.2 Motivation The motivation for our proposed algorithm lies in the fact that even though the PCA based Eigen face approach is a well-researched topic in facial recognition. Section 5 explains the results of our experiments. PCA has been used to produce an optimal least-squares decomposition of a training set. Section 6 concludes the research effort and elaborates on the future scope of our research. the run-time of the system depends on the number of individuals present in the dataset. has performed unsatisfactorily on larger datasets. Section 3 elucidates the approach used to recognize faces using the landmarks as features. The contribution of our work can be summarized as follows: 1. which aids in the extraction of facial landmarks. The organisation of the proposed algorithm is diagrammatically represented in Figure 1. In [8]. as shown in the work of Zhao et al. Hence.A Scalable Real-Time Algorithm for Facial Recognition onto a feature space that has best encoded the variation amongst the known face images. where the recognition time is directly proportional to the size of the dataset. a subset of closest matches is created which is supplied to the PCA based Eigen face algorithm. The minimum Euclidean distance filter can be applied as a preprocessing stage in any face recognition algorithm to optimize the time complexity. The Euclidean distances of the extracted features of the test image are computed with respect to the Euclidean distances of the faces in the dataset. The run-time of the proposed algorithm is largely independent of the size of the dataset in comparison to the PCA algorithm. in [13]. The proposed algorithm resolves this limitation by introducing a minimum Euclidean distance filter before the PCA based Eigen face algorithm. In [4]. A general algorithm has been introduced to reduce the load on the face recognition step through the minimum Euclidean distance filter. it calls for an improvement to mitigate this limitation. The aim of our proposed work is to develop a scalable real-time system that can identify individuals through low-resolution images. 1. Hence.

2. 80 . Irises (Points b and c) Outer end-points of the eyes (Points a and d ) Nostrils (Points e and f ) End-points of the lips (Points g and h ) All the geometrical landmarks are diagrammatically represented in Figure 2. 3. Ishan Nigam and Sahil Goel Figure 1. Feature values are calculated by taking the normalized distances between the eight detected primary geometrical landmarks and three secondary landmarks.Anand Gupta. 2. represent static facial points which do not move with small changes in the subject’s expressions. derived from primary feature points. Organisation. which are employed in the Face Recognition algorithm. Anhad Mohananey. Feature Extraction Geometrical landmarks are localised on the subject’s face to compute the values of features. Secondary points. The primary geometrical landmarks are as follows: 1. 4.

v) = x . The test image is evaluated using the patterns applied on the normalized correlation algorithm. The obtained image matrix is multiplied by the original matrix. the left and the right irises. The extreme points in the image correspond to the required points a and d . 2. y [t ( x ¯ x .2 Locating the end-points of the eye (points a and d) The rectangular region obtained from normalized correlation is extended to the left and right horizontally. The rectangular area obtained from normalized correlation is divided into two equal parts corresponding to the left and the right eye regions. The obtained image is processed using a Gabor filter. y ) − f¯u .e. y − v) − t 2. After processing this region with a Gabor filter. y − v) − t¯] 2 x . It involves the test image being matched to pattern images. i. which are redundant and are not a part of the iris. A Gabor tfilter is applied to remove the high frequency components present in the image. y [ f (x . i. the region represents the points between the 2 irises.v ][t (x − u . resulting in an image in which the iris is the darkest portion. Smaller sized regions. Implementation of Normalized Correlation closely follows the formula [14]: γ (u . Hence. The above steps. Geometrical landmarks. The minimum threshold distance condition ensures that both the points chosen are not a part of the same nostril.1 Locating the irises (points b and c) Normalized correlation [11] is applied on the face using a pattern as a combination of both eyes.e.. y ) − f u . y [ f ( x .. The pattern with the highest match value is considered to be the match. Histogram Equalization is applied to contrast the image.A Scalable Real-Time Algorithm for Facial Recognition Figure 2. after being applied to both the left and right parts. Morphological opening operations are performed using a disk structure. Horizontally. the region enclosing the eyes is detected in the form of a rectangle.v ] ¯]2 − u . 2.3 Locating the nostrils (points e and f ) The region vertically below the area encompassed by the normalized correlation result of the eyes is considered. 81 . the end points of the eye. are used to locate points b and c. are not considered. as the disk structure has a predefined radius (which depends on the resolution of the input image). the 2 darkest points having a minimum threshold distance between them represent the nostril points e and f .

82 . the following features are computed: f 1 = ec/ae f 6 = e f /ae f 2 = d f /ae f 7 = a f /ae f 3 = gh /ae f 8 = de/ae f 4 = bc/ae f 9 = nl /ae f 5 = ad /ae f 10 = zl /ae where ec is the distance between the geometrical land-marks e and c. The extreme points of the image obtained represent the points g and h . Facial Recognition Facial recognition is carried out in 2 stages: 3. a unique face match is obtained. Ishan Nigam and Sahil Goel 2. They correspond to points in the 10-dimensional Euclidean Space. During feature extraction. where p and q are 10 dimensional vectors. and so on. 3. Thus.midpoint of iris points b and c After the 8 landmarks points are located.midpoint of nostril points e and f z . The features are then normalized so that they are resilient to changes in the distance of the subject from the camera. The 5 shortest distances correspond to the top-5 matches from the database. The features computed in this section are used as input to the face recognition module. After filtering out the higher intensity range. the image is complemented. A Gabor filter is applied to the greyscale image.5 Secondary feature points l . These features correspond to a static frontal view of the face. The pre-filtering of faces in the previous step has improved the accuracy and the speed of the algorithm. Principal Component Analysis (PCA) [5] based Eigen Face algorithm is applied to obtain a unique match. 10 features for the test image are calculated.Anand Gupta.4 Locating the lips end-points (points g and h) The region below the nose is considered for this procedure. ae is the distance between the geometrical landmarks a and e. Anhad Mohananey.midpoint of lip end-points g and h n . 3. q ) = (q1 − p1 )2 + (q2 − p2 )2 + · · · + (q10 − p10 )2 . Hence.2 Eigen face approach After obtaining the top 5 matches. 2. top 5 matches are obtained for a test image. Euclidean Distances are computed from the test features to points corresponding to the faces in the database.1 Minimum Euclidean distance approach Median features for all faces in the dataset are computed prior to executing this stage of the algorithm. d ( p.

A comparison with existing PCA and geometry based techniques is given in Table 2.8% 74.0. 5. This step is carried out to validate our proposed algorithm. Eight primary and three secondary landmarks are localized on the basis of the rectangular eye region. These landmarks are used to construct geometrical features to be used in the face recognition steps.8% 99.8% 99. Experiments are conducted on the Essex Face Dataset [9] using a random set of 50 faces with 20 images for each individual. Results Results of feature extraction are summarized in Table 1.8% 99.10% is achieved using our algorithm.Rank 1 Euclidean Distance – Match in top 5 Eigen Based Face Recognition (in Top 5) Total Success 99.8% 99. Module Normalized Correlation Detecting Point A Detecting Point B Detecting Point C Detecting Point D Detecting Point E Detecting Point F Detecting Point G Detecting Point H Euclidean Distance – Top Match . The Euclidean distance of the subject is computed with respect to the points corresponding to the median feature vectors of the faces in the database in the 10-dimensional Euclidean space. 2GHz system in MATLAB 7. Normalized correlation is applied on the test image to locate the rectangular region around the eyes. which uses an amalgam of both the aforementioned techniques.6% 99. The recognition rate and the run-time are used as parameters to evaluate the algorithms. The 5 points which correspond to the shortest distances from the test image are taken to be the top-5 matches from the database.1% 83 . Experiments are also conducted to compare changes in the run-time of both PCA and our algorithm on varying the size of the dataset and the corresponding results are plotted on a graph. Principal Component Analysis (PCA) based Eigen Face algorithm trained on one image per face is applied to the top-5 matches to obtain a unique face match. The images are converted to grey-scale prior to application of the face recognition algorithm. The results show that application of our procedure improved accuracy of both PCA and geometry Table 1.1% 100% 93.8% 99. Experiments The algorithm is validated on an Intel Core 2 Duo.8% 99. To compare results with existing algorithms.A Scalable Real-Time Algorithm for Facial Recognition 4.0% 93.8% 99. An accuracy of 93. PCA based Eigen Face Recognition [10] and Minimal Euclidean-Distance approaches are applied on the complete set of 50 faces. Experimental results for feature extraction and Euclidean distance.12.

84 .01% 16 Seconds Our Algorithm 93. our algorithm outperforms PCA in datasets larger than this threshold.85% 28 Seconds Minimum Euclidean Distance 74. Although PCA performs better on small datasets (< 30 faces).Anand Gupta. Anhad Mohananey. Conclusions and Further Research In the present paper. experimental results show that accuracy of both PCA (92.19 seconds in comparison to the Eigen-Face Algorithm. The run-time of existing Eigen-Face reduces significantly due to the decreased size of input to the Eigen Face algorithm. wherein recognition time increases with increase in the size of the dataset.10% 19 Seconds based algorithms. the present algorithm performs faster than PCA-based Eigen Face Algorithm for datasets larger than 30 faces. Module Recognition Accuracy Recognition Time for Unique Face PCA Based Eigen Face Algorithm 92. wherein run-time increases linearly when experiments are conducted on a dataset of 50 faces.85%) and Euclidean Distance (74. 6.1%. It is observed that the run-time of our proposed algorithm is invariant to the size of the dataset. As observed in Graph 1. Also. Graph 1 portray changes in run time for both PCA and our algorithm when the size of the dataset is varied. as applied on the Essex Face Dataset. Experimental results have shown that the run-time of the algorithm remains nearly constant at 0.01%) have been improved by the present algorithm. Execution time analysis. Hence. Comparison of techniques as applied on the Essex Face dataset. we have developed a real-time face recognition algorithm for low-resolution images whose run-time is independent of the size of dataset. Table 2. the algorithm presented in the paper is scalable and real-time with an improved recognition rate. as opposed to PCA. Ishan Nigam and Sahil Goel Graph 1. which has given an accuracy of 93.

D.: A Direct LDA Algorithm for High-dimensional Data-with Application to Face Recognition. we aim to extend the algorithm for non-frontal face recognition. In Proceeding of Perception. U. M. [13] Zhao. 3(1). In Proceeding of the Journal of Optical Society of America A. 30. K. [14] Lewis.: Low-Dimensional Procedure for the Characterization Of Human Faces. D. Japan. May (1995). J. 2(1). particularly for systems with large face databases. In Proceeding of Journal of Cognitive Neuroscience. [4] Moon. In Proceeding of International Conference on Automatic Face and Gesture Recognition (IEEE). USA. [5] Wold. W. August (1966).: Two-dimensional PCA: A New Approach to Appearance-Based Face Representation and Recognition. Chellappa. In Vision Interface 5(120123).: Face Detection and Recognition Using Hidden Markov Models. PR1. Another limitation of our algorithm is that faces which are not part of the dataset show positive result.: Principal Component Analysis.essex.: Eigen Faces for Recognition. 71–86 (1991). [6] Nefian. and Hayes. 85 .: Discriminant Analysis of Principal Components for Face Recognition. and Krishnaswamy.. In Proceeding of IEEE Transactions on Pattern Analysis and Machine Intelligence.. Illinois. This limitation can be mitigated by using a threshold in the Euclidean Distance stage and hence. In Proceeding of International Conference on Image Processing (ICIP 98).html. 141–145. and Chellappa.10. 303–321 (2001). R.: Fast Template Matching. use of geometrical features limits the method to recognition of frontal faces. 1724–1733 (1997). Nara. Chicago. W. we look to address the following issues in the future. 519–524 (1987). Rep. 14(8). [8] Yu. In Proceeding of the Journal of Optical Society of America A. S. 26(1). We wish to address this concern by incorporating features present in non-frontal faces in the recognition process. Therefore.: Computational and Performance Aspects of PCA-based Face Recognition Algorithms. J. [7] Etemad.A Scalable Real-Time Algorithm for Facial Recognition Experimental results presented show that the novel methodology can be a promising tool for automated face recognition in areas such as security. [3] Bledsoe. P. K. Panoramic Research Inc. P. J. [2] Turk.: Template Matching Using Fast Normalized Cross Correlation. 4387. R. 4(3). and Phillips. A. V. K. 1. Esbensen. Pattern Recognition 34. 131–137 (2004). W. and Yang.ac. F. Y.mathworks.com/matlabcentral/fileexchange/17032-pca-based-face-recognition-system [11] Briechle. In Proceeding of the Journal of Pattern Recognition Society. M. [10] http://www. A. Firstly. 37–52 August (1987).: The Model Method in Facial Recognition. In Proceeding of Multivariate Statistical Workshop for Geologists and Geochemists. [12] Yang. A. H. Frangi.uk/mv/allfaces/faces94. CA. and Hanebeck. H.. October 4–7 (1998). Palo Alto. H. J.: Discriminant Analysis for Recognition of Human Face Images. References [1] Sirovich. 15–19. and Kirby. April 14–16 (1998). In Proceeding of SPIE on Optical Pattern Recognition XII. 15. L. M. Zhang.. To make the present algorithm more effective and pragmatic. giving negative matches for faces not belonging to the dataset. Chemometrics and Intelligent Laboratory Systems. and Geladi. 336–341. J. 2067–2070 (2001). 95–102 (2001). and Yang. [9] http://cswww. and Pentland. A. P.