2019 International Conference on Robots & Intelligent System (ICRIS)

Face Detection Based on Viola-Jones Algorithm Applying Composite Features

LU Wen-yao1，YANG Ming1*
1
School of Computer and Information Science, Southwest University, Chongqing，400715
*
corresponding author’s
email: 850282765@qq.com

Abstract: Viola-Jones' face detection algorithm was following:

jointly proposed by Paul Viola and Michael Jones. (1) detection of face images by Haar features;
Although it realized face real-time detection to some (2) the calculation speed of Haar features is
extent, its false detection rate is not low. Because the improved through the integral graph;
block features in the Viola-Jones algorithm can't handle
purely rigid objects, such as chopsticks and cups, so if
there are rigid objects in the face image, Viola-Jones' face
detection algorithm is prone to generate false detection of
faces. In this paper, we propose to apply the composite
features based on Viola-Jones algorithm to improve the
above problems, and prove the feasibility of this method
through experiments.
Keywords: Viola-Jones; Compound Features; Face
Detection

Face recognition is a hot research field in pattern

recognition. Face detection is a key step in face
recognition. The premise of improving face recognition
rate is to extract effective face features. At present, when
there is no rigid object in the face image to be detected in
some methods, the face recognition rate is considerable,
but the presence of rigid objects is inevitable in practice.
Therefore, this paper proposes to process the face image
through a face detection based on Viola-Jones algorithm Figure 1 Cascade Strong Classifier
using composite features to ensure that the face
recognition rate will not be affected and maintain a (3) training the face data set through the cascaded
certain degree of observability. The contributions of this AdaBoost classifier;
article are: (4) The trained detection classifier is used for the
(1) Viola-Jones algorithm is used to detect the inner final face image detection.
face of the rectangle in the image. Then the complex face
features of the rectangle region are analyzed by 1.1 Haar characteristics
discriminant analysis, and the composite face features are There are four Haar features used in the Viola-Jones
input into the classifier for face recognition. algorithm: edge features, linear features, center features,
(2) When constructing complex features of face in and diagonal features.
rectangular frame, we need to use global and local
features extracted from face inside rectangular frame.

1 Classical Viola-Jones algorithm

Viola-jones algorithm has higher detection speed in
real-time face detection. The characteristics of this
algorithm are:
(1) The feature value of the face image is extracted
by the use of the integral map to ensure the speed at
which the feature is extracted, and secondly ensure the
correct rate of face detection by using the AdaBoost
strong classifier.
(2) The traditional AdaBoost classifier is modified.
In the traditional AdaBoost classifier, the weak classifier
is constructed by many small decision trees, and then
cascaded into a strong classifier. The AdaBoost cascade
processing classifier in Viola-Jones algorithm is shown in
Figure 1 below. Each new classifier built from this is a
strong classifier, which ensures a high detection rate.
The main steps of the Viola-Jones algorithm are the

978-1-7281-2632-6/19/$31.00 ©2019 IEEE 82

DOI 10.1109/ICRIS.2019.00029
 Viola-Jones algorithm has the problem that rigid objects
make the recognition rate decrease. To solve this problem,
we proposes a face detection method based on
Viola-Jones algorithm using composite features. The
main process of this method is as follows:
(1) Input an image, and determine a rectangular
frame of the human face by the Viola-Jones algorithm;
(2) After the faces inside the rectangular frame are
calibrated, they are processed into four types of

sub-images. As shown in the figure below, x H is the

ENM
calibrated face inside the rectangle, x , x SEG is the
ENM
sub-image obtained by cutting off part of the face, x
is the sub-image composed of the main features of eyes,

nose and mouth, x SEG is the sub-images divided into

FSDD
four parts, and x , x IVS is a sub-image obtained by
selecting useful pixels based on discriminant analysis .
(3) Use NLDA (Zero Space Linear Discriminant
Analysis) to extract features from the obtained full face
Figure 2 Four Haar features image and four sub-images.
(4) Evaluate the validity of all extracted features
As shown in the figure above, there are two regions (global and local features) by discriminant distance
in the Haar feature template, namely, the black region and (5) Select feature regions with large distance from
the white region. The difference value between the pixels discriminant values to form new composite feature
of the black and white regions is the eigenvalue to be vectors, and then input them to classifier for face
calculated. For the features of center, edge and diagonal recognition.
type, the formula for calculating the eigenvalue is as
follows:
v Sumwhite  Sumblack
（1）
However, the black-and-white areas with linear
characteristics occupy different areas, so it is impossible
to calculate the difference directly. The number of pixels
of the two colors in the rectangular region should be kept
the same. The numerical formulas for calculating the
characteristics are as follows: （a） (b) (c) (d)
v Sumwhite  2Sumblack (e)
H SEG
IVS ENM
Figure 3. (a) x , (b) x , (c) x , (e) x
FSDD
（2） , (d) x .
1.2 Integral graph Face in rectangular and four types of sub-images
Because the generated Haar feature dimension is
huge, in order to calculate the feature quickly, the integral 2.1 Constructing New Compound Characteristics
map is used to divide the image cell, and the grayscale After determining the rectangular frame of the face,
image of the two-dimensional coordinates is established, the projection vector of the full face ( x H ) and the
and the values of each pixel point are quickly located,
sub-image ( x , L  {IVS , ENM , FSDD, SEG} ) can be
L
thereby the pixel sum of the image region is ensured to
calculate in the efficient state. In the calculation of obtained by using NLDA (Zero Space Linear
integral graph, the upper left corner of image cell is taken Discriminant Analysis). They are
as the starting point. Any point in the image to the starting H
WOpt  R muC 1 ,WOpt
L
 R muC 1 respectively. The global
point can form a rectangle, and the sum of all pixels in the
rectangle can be calculated. Then the pixels are saved as and local feature vectors
H H H H T L L L L T
integral graph pixels of this region. Thus, when we need y [ y , y ,! , y
1 2 C 1 ] ,y [ y , y ,! , y
1 2 ]
C 1
to extract the pixels of a certain region, we can get them can be expressed according to the following formula (3).
directly.
2 Face Detection Using Compound Features Based on
Viola-Jones Algorithms
In the previous part, we pointed out that the

83
 difference can be avoided effectively and the accuracy of
yH H T H
(WOpt ) x face recognition can be guaranteed. Figure 4 is a classifier
flow based on the Viola-Jones algorithm using composite
yL L T L
(WOpt ) x features.
（3）
Then the basic vector
YPool [Y1 ,!, Y2(c1) ]T
[ y1H , y2H ,! yCH1 , y1L , y2L ,!, yCL1 ]T is constructed
H L
by using the vector y , y obtained by NLDA. The
weight information of distinguishing features is used to
evaluate the validity of basic features in face recognition.
Then the feature with larger weight is selected to form the
composite feature vector. The discriminant weights of
each basic feature are measured by discriminant distance.
For Y j , it is the j-th feature of Y Pool . Its within-class
j j
distance ( DW ) and between-class distance ( DB ) can
be expressed by the following formulas (4), (5).
2 ( C 1) c
1
¦ ¦ N ¦ (Y
2
D
W
j
j
i
 Y ji )
j 1 i 1 i Y Pool ci

（4）
2 ( C 1) c
Ni
¦ ¦ N (Y
Figure 4 A classifier using composite features based on
DBj j
i
 Yj )2 Viola-Jones algorithm
j 1 i 1
（5） 3 Experimental results and analysis
i FDDB is the test data set used in this experiment.
Y j , Y j defines the mean value of the features
This data set contains 2,845 images with a total of 5,171
belonging to the class ci in the basic feature vector and faces. The face images in this data set have angle,
the mean value of the each feature in the basic feature illumination and scene changes.
vector. The discriminant distance of the j-th feature can be In the experiment, we randomly selected 100 images
expressed by equation (6): from the FDDB data set for face detection. And we count
the number of faces actually included in the selected 100
Fj DWj  E DBj pictures, the number of false detection and missed
（6） detection. On this basis, a total of 10 experiments were
The discriminant distance of the j-th feature is used conducted and 1000 pictures were used as experimental
data. By comparing the composite feature method based
to measure the difference of each feature information. E
on Viola-Jones algorithm with the original Viola-Jones
is a parameter determined by the sample distribution. By algorithm in experiments, the accuracy of textual method
comparing the effects caused by different E , we finally used in the face recognition process is illustrated. And the
chose 2 as the value of E . The discriminant distance of phenomenon that the rigid object in the previous picture
affects the experimental effect is improved.
each feature in formula (6) is stored in the discriminant Table 1, table 2 and table 3 as shown below,are the
distance vector of the same size as Y Pool , namely specific experimental results on the FDDB data set of the
T original Viola-Jones algorithm and the application of
F [ F1 , F2 ,! , F2 ( c 1) ] . Then, according to the composite feature method based on Viola-Jones algorithm
discriminant distance Fj of each feature, a composite in this paper respectively .
feature vector （yCF） is constructed by selecting features
with larger Fj value. Finally, the constructed composite Table 1 Experimental results of original Viola-Jones
features are input into the classifier for face recognition. algorithm on FDDB data set
Experim Actual Missing False face Missin Error
2.2 Classifier Based on Viola-Jones Algorithms Applied ent face face number g detect
Compound Features number number number detecti ion
The original Viola-Jones algorithm does not handle on rate rate
the huge differences caused by the variation of different 01 126 16 3 0.127 0.024
rigid objects. By using the method of this paper, this 02 150 34 3 0.227 0.020
03 128 23 1 0.180 0.008

84
 04 134 19 0 0.142 0 original face effectively, which makes a good foundation
05 152 22 1 0.145 0.007 for ensuring the accuracy of face recognition.
06 153 16 4 0.105 0.026
07 131 18 4 0.137 0.031
REFERENCES
08 126 14 1 0.111 0.008
09 147 18 2 0.122 0.014 [1] Liu Yihui, Bai Li. Salient Information of Mass Spectra of
10 125 9 2 0.072 0.016 Prostate Cancer Dataset[C]. Proc. of IEEE GRC’08. Hangzhou,
China:[s. n.], 2008
[2] C.Sagonas,G.Tzimiropoulos,S.Zafeiriou,and M.Pantic, “A
Table 2 Experiments of using composite feature semiautomatic methodology for facial landmark annotation,” in
based Viola-Jones method on FDDB data set Proc. CVPR,Jun. 2013, pp. 896–903.
Experim Actual Missing False face Missi Error [3] Z. Zhang, P. Luo, C. C. Loy, and X. Tang, “Learning deep
ent face face number ng detect
number number number detecti ion
representation for face alignment with auxiliary attributes,”
on rate IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 5, pp.
rate 918–930, May 2016
01 126 5 1 0.040 0.008 [4] Wu Kai, Zhu Hengliang, Hao Yangyang, Malizhuang.
02 150 8 0 0.053 0 Cascade regression multi-pose face registration [J]. Chinese
03 128 8 0 0.063 0 Journal of Image and Graphics. 2017, 22 (2): 0257-0264
04 134 6 0 0.044 0 [5] Bai Xiaoming, Wang Chengzhang, Shiqin. Color face
05 152 10 0 0.065 0 recognition based on two-dimensional linear discriminant
06 153 10 0 0.065 0 analysis [J]. Journal of Beijing University of Technology. 2010,
07 131 7 0 0.053 0 12 (36): 1717-1721.
08 126 8 0 0.063 0 [6] Wang Hongyong, Liao Haibin, Duan Xinhua, Ding Miao.
09 147 9 0 0.061 0 Adaptive Face Recognition Based on Singular Value and
10 125 4 0 0.032 0
Feature Fusion Matrix [J]. Computer Engineering and
Application, 2010, 46 (7): 162-174.
Table 3 Comprehensive comparison of the two [7] Bai Xiaoming, Wang Chengzhang, Shiqin. Color face
algorithms recognition based on two-dimensional linear discriminant
Total Total Total Total Gross analysis [J]. Journal of Beijing University of Technology, 2010,
Algorithm
name
face missing error missing error 36 (12): 1717-1721.
number count count rate rate [8] Wei Chongchong, Wang Xiaopeng, Yan Jianwei. A Face
Viola-Jone Recognition Method for Eliminating Illumination Effects [J].
189 21 0.138 0.015
s Electronic Test, 2012, (7): 19-23.
1372
Article [9] Wang Shoujia. Research on Image-based human detection,
75 1 0.055 0.0007
method tracking and face recognition [D]. Jilin University, 2013.
[10] Ren S, Cao X, Wei Y, et al. Face alignment at 3000 fps via
4 Conclusion regressing local binary features[C]//Proceedings of the IEEE
By analyzing the experimental results in the previous Conference on Computer Vision and Pattern Recognition. 2014:
section, we can know that the original Viola-Jones 1685-1692.
algorithm has a lot of missed and false face detections on [11] Yi D, Lei Z, Li S Z. Shared representation learning for
the FDDB data set. However, the method used in this heterogenous face recognition[C]//Automatic Face and Gesture
paper is obviously lower than Viola-Jones algorithm in Recognition (FG), 2015 11th IEEE International Conference and
Workshops on. IEEE, 2015, 1: 1-7.
terms of both missed detection rate and false detection [12] Zhu J Y, Zheng W S, Lai J H, et al. Matching nir face to vis
rate. From this, we can see that the combination of face using transduction[J]. Information Forensics and Security,
Viola-Jones algorithm and the constructed composite IEEE Transactions on, 2014, 9(3): 501-514.
features avoids the missed and false detection of the

85