This action might not be possible to undo. Are you sure you want to continue?
Kyung-A Kim 1 , Se-Young Oh 2 , Hyun-Chul Choi 3 Dept. of Electrical Eng., Pohang University of Science and Technology, Pohang, Kyungbuk, 790-784, South Korea 1 email@example.com 2 firstname.lastname@example.org 3 email@example.com
This paper presents a novel algorithm for the extraction of the facial feature (eyebrow, eye, nose and mouth) fields from 2-D gray-level face images. The fundamental philosophy is that eigenfeatures, derived from the eigenvalues and eigenvectors of the gray-level data set constructed from the feature fields, are very useful to locate these fields efficiently. In addition, multi-resolution images, derived from a 2-D DWT (Discrete Wavelet Transform), are used to save the search time of the facial features. The experimental results indicate that the proposed algorithm is robust against facial feature size and slight variations of pose.
heuristic assumptions are used. Also, to enhance performance of the system, we combine a few single feature detectors. In Section 2, the basic idea of the proposed algorithm is reviewed briefly. A facial feature extractor, which uses the proposed algorithm, is introduced in Section 3. And methods to improve the performance of the extraction system are presented in Section 4. Experimental results are reported in Section 5.
2. The Basic Idea
There are many disadvantages in pixel-based sample representation approach, for example, the data is not compact and there is much redundant information. Besides, when we search for a facial feature in the whole face image (original size), computational cost is very expensive. Therefore, we use a 2-D discrete wavelet transform (2-D DWT) to emphasize the change information in the gray scale and to reduce the search region. After the transform, the dimension could be reduced from 256x256 to 128x128, 64x64 and 32x32, which can save the computational cost. Figure 1 gives an example of the decomposition of an image. The Haar wavelet is adopted in our experiment. On the other hand, the theory of the principal component analysis (PCA) is widely used in pattern recognition as well as in classification. The PCA has a special characteristic which can retrieve original image using dominant eigenvectors (eigenfeatures). We use a sliding window template of the facial features represented in the eigenfeature space to locate facial features in face images. Images of facial features do not change radically when projected into the feature space, while the projection of non-feature images that are not facial features appears very different. We define the “facial feature map” based upon the “face map” by M.Turk and A.Pentland . The facial feature map is calculated as the distance between the local subimage and facial feature template after PCA filtering at every location in the image. The PCA filter is defined as follows. The original image is PCA transformed into
Face recognition is useful for many applications, such as identity authentication, security access, e-commerce etc., which has received greater interest among experts recently. To recognize face, automatic extraction of facial features from a person’s face is a very important process. Many researchers have proposed methods to find the facial feature regions [1,3-5,8] or to locate the face region [6-7,9-10] in an image. These methods can be classified by their use of three types of information: template matching, intensity and geometrical features. In general, template matching requires many templates to accommodate varying pose whereas the intensity method requires good lighting conditions. And geometrical method is not robust enough for the various pose. In this paper, we present a novel algorithm for the extraction of the facial feature fields from 2-D graylevel face images. We use a sliding window template of the facial features represented in the eigenfeature space to locate facial features in face images. Here, the eigenfeature space is defined as eigenvector space from the training set which consists of a particular facial feature one of eyes, eyebrows, nose and mouth. To speed up execution time, multi-resolution images and
3 levels) with scale-frequency resolution. Figure 4 shows examples of the detection result. Step2 : Perform a Haar transform of the input image at several levels (In our experiments. Facial Component Extraction We use knowledge of the eigenfeature space and a 2D-DWT to detect facial features in face images. Step1 : Compute eigenvectors (eigenfeatures) of the facial features (eyebrow. Step4-1: Generate a binary edge image (canny . HL2 and LL3 images are used to detect facial features. 8x8 for the eyebrow-eye and the nosemouth region respectively. while the other obtains the region containing both nose and mouth. Figure 3 represents the system block diagram. LL1. the lowest value indicates the presence of a feature (see Figure 2). The dominant PCA components are then mapped back into the original image space and the result is called a PCA filtered image. In our test. One gets coarse region of eyebrows and eyes. Therefore a tradeoff between accuracy Facial features Figure 3. Here. eye. because the use of the former images is enough to obtain good performance. latter images are not used. Single Feature Detector (eye) Single Feature Detector (eyebrow) Single Feature Detector (nose) Single Feature Detector (mouth) If failed Figure 2. Example of the feature map for the eye. The system block diagram manually marked samples. mouth) with manually marked feature from the training set. Step3 : Determine coarse regions of eyebrows. eyes. nose and mouth.the eigenfeature space. Example of the Multi-Resolution Image. To improve the accuracy of detection with higher cost. LL2. Compute the facial feature map and detect features. Double Feature Detector 1 (eyes and eyebrows region) Figure 1. Among the results of the transform. 3. LH2. The coarse region detectors were trained on randomly selected 30 frontal images with Input image HL3 LL2 Compute eigenvectors (eigenfeatures) of facial features. The detailed algorithm follows: HL3 HL2 LH1 and computational cost exists. nose. Double Feature Detector 2 (nose and mouth region) Generate a binary edge image or image with certain threshold within coarse region. LH1 and HL1 images can be used. In the facial feature map. HL2 Decompose the image using 2D-DWT(Multi-Resolution Image). LL1 HL1 Determine the coarse regions for eyebrows. nose and mouth using the LL3(32x32) image. Size of each sample is 6x15. Step4 : Compute the feature map as follows. a feature detection system is made up of 2 double feature detectors. This step not only reduces the size of the search window in the next step but also improves performance of the facial feature detection. eyes.
Combined usage of the two single feature detectors for increased robustness We developed a feature detector system which consists of the two double feature detectors and four single feature detectors (see Figure 3). (b) binary image obtained by applying a certain threshold to the summed image of the LH2 and HL2. The inverse case may appear. (LH2+HL2) and HL2 images.1.1 Eye-eyebrow detector (1) Eye detector First. Double feature detector1 extracts a region containing the eyebrows and . select four candidates (two eyes and two eyebrows) that have the lowest value in the feature map. (a) From left to right. rather than the single feature detector. In the feature map. Performance improvement of the feature detection system To improve the feature detection performance. (a) (b) Figure 4. (b) Mouth Detector and (c) Combined Detector 4. LL2. the characteristic information of the facial morphology simplifies the process thereby reducing the execution time. The single nose detector cannot extract the exact nose. This extracts both eyes and eyebrows with over 80% of accuracy. (b) Eyebrow Detector and (c) Combined Detector (a) (b) Figure 5. whereas the eyebrow detector makes up for the eye detector’s defects.mouth. To save time. the eyebrow detector does not 4. The single eye detector cannot extract the right eyebrow. LH2. If four candidates satisfy heuristic assumptions (the relations between eyebrows and eyes). however. First row of the Figure 7 also demonstrates the advantage of using the nose-mouth detector. eyes while double feature detector2 extracts that of the nose and mouth. is more proper because each system assists each other to avoid defects of the single detector system. the system combined with a few detectors. Example of the results of (a) Eye Detector. Example of the coarse feature region by Step3. only those pixels with the 1’s value within the coarse region (result of the previous step) are computed for the facial feature map. top to bottom. the lowest value indicates the presence of a feature . Step4-2: Compute the facial feature map using a PCA information in LL2 image. 4. (a) (b) (c) Figure 7. Example of the results of (a) Nose Detector. clockwise. Also. Second row of the Figure 6 shows the advantage of detecting both eyes and eyebrows over the single feature detectors. (a) and (b) represent the coarse region containing eyebrow-eye and nose. this problem is solved as the mouth detector detects a nose as well as mouth. (a) (b) (c) Figure 6.1.or sobel edge) or an image with certain threshold within a coarse region (result of the previous step) using summation of LH2 and HL2 images (see Figure 5). which contain high frequency features.
the eyebrow detector operates to detect missed features.2 Nose-mouth detector (1) Nose and mouth detector Select two candidates that have the lowest value in the feature map of each single feature detector system. The nose is above the mouth. (1)The eyebrows and eyes remain within the upper half of the face region. eye. 20x40. The results demonstrate the robust performance of the detector which extracts narrow eyes as well as eyes in spectacles. If the lowest value in the eyebrow feature map is below some chosen threshold. The algorithm is very fast (with the average execution time of 0.3]. All images are D . (2) Eyebrow detector Considering both heuristic assumptions and the result of the eye detector. Their exact position. The certain distance between eyes and eyebrows is defined as the averaged distance from the experimental result. For simplicity.13% 96.17% eye 4/(214) 55/(1710) 98. C: Performance of the facial feature extraction in the frontal face DB. Size of each feature in the training samples is 20x50.0862sec) and produced good results. Left-right eyes and eyebrows are symmetric. A: Number of missed features in the frontal face DB. The proposed algorithm is robust enough for the test data.51% 5. (3)The relation between eyebrows and eyes a. The performance of feature extraction have achieved correct hit rate of 92. Eyes keep a certain distance from eyebrows. we can select the sample features in the rotated space as well as the frontal space. 30x60 for eyebrow. Table 1. To improve the generalization performance. To improve the performance in the rotated face. however. using 20~30 frontal images. (2)The nose and mouth remain within the lower half of the face region. facial features hidden by hair were not considered. The nose is kept apart from the mouth by a certain distance which is taken to be an averaged distance from experimental statistics. respectively. with tolerance for some tilting and rotation. which is the rotated facial images. it works within the coarse region. Inter-eye and eyebrow distances are similar.13% for the training feature samples consisting of frontal faces. Table 1 shows the experimental results. c. Extraction performance.23~98.operate (see the first row of the Figure 6). Database) face data. features are extracted (see Figure 7). 26x50. Otherwise. 4.78% nose 5/(107) 40/(856) 95. our algorithm was applied to different people with various poses.46% 94.2. with the person in an upright frontal position. b. Considering both heuristic assumptions (the relations between nose and mouth) and the results of nose-mouth detector. Experimental Results We have used the IMDB (Intelligent Multimedia Lab.23% 90. b. The eyebrows are above eyes. B: Number of missed features in the rotated face DB.33% 95. (Figures within parentheses represent the total number of the features) eyebrow A B C 16/(206) 160/(1628) 92. The feature detectors were trained as described before. 4. Also. Figure 9 shows the representative experimental results.33% Mouth 7/(107) 47/(856) 93. D: Performance in the rotated face DB. Dotted rectangles represent examples of mis-extraction in rotated faces. the extraction performance of the eyes is higher than any other algorithms [1. The horizontal center of the nose and the mouth is near the horizontal center line of the face. the system achieved generalization of 90. The eye and eyebrow detectors were trained on selected 20 frontal images while the nose and mouth detectors were learned from 30 frontal images with manually marked features.17~96. Heuristic Assumptions made to simplify the detection process The following assumptions are made in order to simplify the process of finding the coarse region (see Step3) and also to improve the feature detection accuracy.78% for the test set. nose and mouth. In this experiment. Figure 8 shows the examples of the training feature set. can vary according to the height of the forehead and the facial pose. d. 256x256 pixels in 256 grey levels and are taken against a homogeneous background.1. which consists of 107 Asian faces (56 males and 51 females) . c. the position is selected for the remaining feature to be found (see the second row of the Figure 6). selection of the good samples which represent various shapes is important. Especially. (4) The relation between nose and mouth a.
Cases include male and female faces. Examples from the results of the facial detector. with and without eyeglasses. Examples of the training feature set Figure 9. . Dotted boxes indicate errors.Figure 8.
Proceedings of the 1995 Fourth IEEE International Conference on Universal Personal Communications Record.781-787.K. Wu. Sun. Podolak and Seong-Whan Lee. Japan. 2002. Poggio. Volume2. Vol. Second.P. Facial Component Extraction and Face Recognition with Support Vector Machines.713-735. 1991.  Dihua Xi. 1998. Asian Face Image Database PF01. tional Conference on Pattern Recognition. binary images (edge or threshold) and heuristic assumptions.778-782. October 1993. pp. Feature-based human face detection. Lam.  Matthew Turk and Alex Pentland. Conclusion A facial feature extraction algorithm is presented based on eigenfeatures and multi-resolution images with the following three merits. Q. The proposed system can be applied to 3D face modeling. Journal of Cognitive Neuroscience. Face recognition: features versus templates. Igor T. Brunelli. pp. Fujumura.  Intelligent Multimedia Laboratory.17(2).09 second for each image through the use of the 2-D DWT. Applied Intellignece. R. C. 1995. A robust approach to face and eyes detection from images with cluttered background. The extraction time takes less than 0.H. Number 1. although the detector system is trained using a relatively small feature sample set of 2~3% of the total data. First.  Weimin Huang.110-113.1360-1364.  Yankang Wang. Tokyo. Automatic extraction of eye and mouth fields from monochrome face image using fuzzy technique. can be directly applied to face recognition without additional processing.Pinto-Elias.Juell.6. Volume:29 Issue:5. T. IEEE Computer Society. IEEE Transactions on. 2001. 2002. References  Yeon-Sik Ryu and Se-Young Oh. the eigenfeatures and geometric information of the features. Automatic facial feature detection and location. Image Vision Computer. 1996. H. Pattern Recognition.171-185. Department of Information Science. the training and extraction time of the proposed system is less than that of the existing that we know of any algorithms [1. Proceedings of the 14th Interna- . pp.1042 -1052.3]. Pattern Analysis and Machine Intelligence. Volume3. Technical Report. R. Picture processing by computer complex and recognition of human faces. A. Volume:15 Issue:10.  Kin Choong Yow. Nakamura. Eigenfaces for Recognition. The training time is needed only to compute eigenfeatures. Third.  R.Kuroda. J. making it faster than SVM or MLP [1. Proceeding of the 14th International Conference on Pattern Recognition. 1997.  R. Volume:15 Issue:9.76-81. J. pp. Volume 1. 1973. Cipolla. Automatic Extraction of Eye and Mouth Fields from a Face Images using Eigenfeatures and Ensemble Networks.3].  P. face tracking and detection in mobile robots and face recognition. that is. M. Department of Computer Science and Engineering. pp. Kyoto University. the result of the proposed feature detection system. Sossa-Azuela. it has good generalization performance. pp. A hierarchical neural network for human face detection. pp. Pohang University of Science and Technology. Massachusetts Institute of Technology. Proceedings of the Fifth International Conference on Automatic Face and Gesture Recognition. pp.Marsh.  Takeo Kanade. 1998. coarse region extraction. Technical Report.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.