2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

A Comparative Study between LBP and Haar-like

features for Face Detection Using OpenCV
Kushsairy Kadir #1, Mohd Khairi Kamaruddin *1.Haidawati Nasir#2, Sairul I Safie#3,
Zulkifli Abdul Kadir Bakti#1
#1
British Malaysian Institute, Universiti Kuala Lumpur, Selangor
#2
Malaysian Institute of Information Technology, Universiti Kuala Lumpur, Kuala Lumpur
#3
Malaysian Institue of Industrial Technology, Universiti Kuala Lumpur, Johor

kushsairy@bmi.unikl.edu.my

Abstract—Face Detection is an important step in any face
recognition systems, for the purpose of localizing and extracting
face region from the rest of the images. There are many
techniques, which have been proposed from simple edge
detection techniques to advance techniques such as utilizing
pattern recognition approaches. This paper evaluates two
methods of face detection, her features and Local Binary Pattern
features based on detection hit rate and detection speed. The
algorithms were tested on Microsoft Visual C++ 2010 Express
with OpenCV library. The experimental results show that Local
Binary Pattern features are most efficient and reliable for the
implementation of a real-time face detection system.
I. INTRODUCTION
Face detection is a technique of using any various methods
to detect a face in an image or video. Nowadays there are
many devices and websites which use face detection
technology to detect a face such as Facebook that offer to tag
friends on uploaded images and then will automatically tags
the face whenever the face appear in any of the photos.
Currently, we can find that most laptops come with webcams
that utilize software that has a face detection function.
Therefore, face detection is not new in the computer vision
industry.
Face detection is a computer technology that determines the
locations and sizes of human faces in arbitrary (digital)
images. Face detection can be regarded as a specific case of
object-class detection. In object-class detection, the task is to
find the locations and sizes of all objects in an image that
belongs to a given class.
One of the classical techniques for face detection was
presented by Paul Viola and Michael Jones [1] which use
weak classifier cascade for face detection. There are four
components in the technique proposed by Viola and Jones.
First, is to generate Haar-like features [2] that are digital
image features used in object recognition. Second, is the
introduction of a new image representation called the Integral
Image [1] which allows the features used by the detector to be
computed very quickly. The third component is a simple and
efficient classifier which is built using AdaBoost [3] learning
algorithm to select a small number of critical visual features
from a very large set of potential features. The fourth
component is a method for combining classifiers in a cascade
which allows background regions of the image to be quickly
discarded while spending more computation on promising
face-like regions.
Local Binary Patterns proposed by Timo Ojala et al [4] has
been extensively used for the face detection algorithm. Hadid
et al [5] was the first to apply LBP histogram features in face
detection, showing exceptionally high discriminative power in
face or non-face classification.
This paper focused on comparative studies of two face
detection algorithm. The remainder of this paper is organized
as in Section II discusses the methodology of the algorithms
and Section III shows the result of the testing. For the testing
the same training set and samples are used.
II. METHODOLOGY
Every face detection algorithm has different operation and
procedures to detect a face. This section explains the
components of both algorithms and step by step towards its
function. First, is the face detection using Haar-like features,
and second, is the face detection using Local Binary Pattern
features.
A. Haar-like Features
The features that Viola and Jones used are based on Haar
wavelets [6]. Haar wavelets are single wavelength square
waves (one high interval and one low interval). In two
dimensions, a square wave is a pair of adjacent rectangles, one
light and the other is dark. This method is quite similar to
binary method that separates the obvious different:
Figure 1: Haar-like Features[7]

2014 4th International Conference on Engineering Technology335

and Technopreneuship (ICE2T)
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)
The actual rectangle combinations used for visual object
detection are not true Haar wavelets. Instead, they are the
modified rectangle combinations better suited to visual
recognition tasks. Because of that difference, these features
are called Haar features or Haar-like features [7].
The presence of a Haar-like feature is determined by
subtracting the average dark-region pixel value from the
average light-region pixel value [8]. If the difference is above
the threshold (set during learning), that feature is said to be
present. Examples of Haar-like features are shown in Figure 1.
Figure 2: Haar-like features application.
To detect a face by using Haar-like features, it can come
one feature per image or ten features or even hundreds
features per image as shown in Figure 2. The number of
features depends on the structure, occlusion and skin tone of
the face in an image.
To determine the presence or absence of hundreds of Haar-
like features at every image location and at several scales
efficiently, Viola and Jones used a technique called an Integral
Image [1]. In general, integrating means adding small units
together, which in this case the small units are pixel values.
The integral value for each pixel is the sum of all the pixels
above it and to its left. Starting from the top left, and
traversing to the right and down, the entire image can be
integrated with a few integer operations per pixel [8].
The idea is to compute all sums in the very beginning and
save them for future computations. This can be done by
forming a summed area table for the frame being processed,
also known as computing its integral image representation. In
an integral image, the area of any rectangular region in the
image can be computed by using only four array accesses [9].
The Figure 3 shows the example.
336
Figure 3: Integral image
The blue matrices represent the original images, while the
purple ones represent the images after the integral
transformation. If we were to compute the shaded area in the
first image, we would have to sum all pixels individually,
reaching the answer of 20 after about six memory accesses.
Using the integral image, what is needed is a single access
(but this only because we were in the border). In case we are
not in the border, all it would require would be at max four
array accesses, independently of the size of the region. So, for
the scanning process, we no longer use every tiny pixel, we
will make the process using integral images.
Figure 4: Integral Image
The value at each pixel location, (x,y), contains the sum of
all pixel values within a rectangular region that has one corner
at the top left of the image and the other at a location (x,y). To
find the average pixel value in this rectangle, divide the value
at (x,y) by the rectangle’s area. Conveniently, A+B+C+D is
the Integral Image’s value at location 4, A+B is the value for
location 2, A+C is the value for location 3 and A is the value
for location 1. So, with an Integral Image, the sum of pixel
values for any rectangle in the original image with three
integer operations: (x4,y4) – (x2,y2) – (x3,y3) + (x1,y1) or
D=A+B+C+D-(A+B)-(A+C)+A.
B. Local Binary Patterns
Local Binary Patterns (LBP) [4] [10] is a simple yet very
efficient texture operator which labels the pixels of an image
by thresholding the neighbourhood of each pixel with the
value of the centre pixel and considers the result as a binary
number. Due to its discriminative power and computational
simplicity, LBP texture operator has become a popular
approach to various applications.
The LBP texture method has provided excellent results in
various applications. Perhaps the most important property of
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

the LBP operator in real-world applications is its robustness to

monotonic gray-scale changes caused, for example, by
illumination variations. Another important property is its
computational simplicity, which makes it possible to analyse
images in challenging real-time settings.
The LBP texture analysis operator, introduced by Ojala et
al [4], is defined as a grayscale invariant texture measure,
derived from a general definition of texture in a local
neighbourhood. The original LBP operator forms labels for
the image pixels by thresholding the 3 x 3 neighbourhood of
each pixel with the centre value and considering the result as a
binary number. The histogram of these 28 = 256 different
labels can then be used as a texture descriptor [11]. Figure 7: Example of texture primitives [13]

Each bin (LBP code) can be regarded as a micro-texton

[13]. Local primitives which are codified by these bins include
different types of curved edges, spots, flat areas, line end or
corners.

III. RESULTS
The algorithms were tested on Microsoft Visual C++ 2010
Express with OpenCV library. This testing running on
Windows 7 64bit with Intel Core i3 M380 (2.53GHz), 6GB
Figure 5: The basic LBP operation RAM and SATA Hard Disk. The settings for the algorithm
testing are using 1.1 scale factor with four minimum
neighbours threshold and not using any flag, with 25x25
minimum detection scale. Hear and LBP had already been
tested by using multiple sets of face database:
i) The Color FERET Database, NIST USA [14] with
1127 images.
ii) MIT CBCL Database [15] with 2000 images
Figure 6: Neighbourhood set for different (P,R). The pixel iii) Taarlab Face Database, Brown University [16] with
values are bilinear interpolated whenever the sampling point is 759 images.
not in the centre of a pixel.
All data sets that have been selected are focus on frontal
The operator has been extended to use neighbourhoods of image and with 30 degree occlusion to right and left.
different sizes. Using a circular neighbourhoods and bilinear
interpolating values at non-integer pixel coordinates would TABLE 1
allow any radius and number of pixels in the neighbourhood Haar Result
[11]. In the following, the notation (P,R) will be used for pixel Algorithm Dataset Detected Hit Rate Detection
neighbourhood, which means P sampling points on a circle of Faces Speed (ms)
radius of R. Another extension of the original operator is the Haar Color 976/1127 86.6% 235.4117
definition of so called uniform patterns [12]. This extension FERET
was inspired by the fact that some binary pattern occurs more MIT 1670/2000 83.5% 255.5048
commonly in texture images than others. A LBP is called Taarlab 647/759 85.2% 231.5865
uniform [10] if the binary pattern contains at most two bitwise
Overall 3293/3886 84.7% 241
transitions from 0 to 1 or vice versa when the pit pattern is
traversed circularly. For example, the patterns 00000000 (0
TABLE 2
transitions), 01110000 (2t transitions) and 11001111 (2
LBP Result
transitions) are uniform whereas the patterns 11001001 (4
Algorithm Dataset Detected Hit Rate Detection
transitions) and 01010011 (6 transitions) are not. In the
Faces Speed (ms)
computation of the LBP labels, uniform patterns are used so
that there is a separate label for each uniform pattern and all LBP Color 1004/1127 89% 95.44924
the non-uniform patterns are labelled with a single label [11]. FERET
For example, when using (8,R) neighbourhoods, there are a MIT 1779/2000 89% 101.8864
total of 256 patterns, 58 of which are uniform, which yields in Taarlab 674/759 88.8% 104.9335
59 different labels. Overall 3457/3886 89% 101

337
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)
The color FERET data set is a mix database that consists of
different skin tone faces, faces with glasses, various racial
faces and 30 degrees of face left and right occlusion with
semi-controlled environment.
Table 1 and Table 2 shows the results of Haar and LBP
algorithms tested on the three databases. The result shows for
Haar algorithm the detection rate for Color FERET database is
84.7% compare to LBP detection rate, which is 89%. The
result shows that LBP detected more faces by the difference of
3.4% more hit rate than Haar. Haar and LBP have the same
result in number of subjects detected which both algorithm
missed one subject totally. Half of Haar undetected images are
faces with 30 degree left occlusion which means that Haar
vulnerable in face with left occlusion detection.
MIT face database is a grayscale face database with a wide
range of background light. It contains face images of 10
subjects (200 for each subject). LBP reaches the same hit rate
(89%) as it had been used on Color FERET database while
Haar has a lower hit rate percentage (83.5%) on MIT data set
compared to it’s detection on Color FERET database. The
result from this data set means that LBP is more robust than
Haar in a wide range of environment such as background light.
Figure 8: Sample results taken from Color FERET data set
testing using LBP algorithm.
Figure 8 shows the sample of result that was taken from
Color FERET data set testing by using LBP algorithm. Figure
8 also shows that LBP algorithm is capable to detect faces
with different skin tone, hairy face, faces with glasses and
different racial face’s structure.
Taarlab database includes multiple images for over 200
individuals of many different races with majority of Asian
faces. This database has a consistent lighting with various
emotions and even wearing sunglasses. LBP still lead the
detection hit rate percentage by 3.6% more than Haar
detection hit rate in testing with Taarlab data set.
338
The above two experiments result show that two algorithms
of face detection can detect face accurately and reach the
requirement if there are faces in video. In other words, the
effectiveness of both algorithms almost are the same if the
data sets of sequential static image are used to detect faces.
Although LBP detected 4.3% more faces than Haar, the
result outputs for both algorithms are almost the same which
both algorithms detected almost all subjects in these three data
sets. Both algorithms do not have detection problems with
racial faces, hairy faces, difference skin tone faces or faces
with glasses.
Based on the results of the experiment, the LBP face
detection algorithm is the most reliable algorithm to find faces
in images because LBP detected 4.3% more faces than Haar
on hit rate aspect and LBP detection speed is 140% faster than
has detection speed on the overall result. This could be a huge
advantage in the implementation on embedded system for
real time face detection.
IV. CONCLUSION
This paper discusses two methods or algorithms that have
been used in face detection, Haar features and Local Binary
Patterns (LBP). Comparative study has been made between
these two algorithms by using three face databases as samples
which are Color FERET database, MIT CBCL database and
Taarlab database. All three databases provide single face
images with various types of faces. Results have been
evaluated based on detection speed for every image and
detection’s hit rate which leads to accuracy.
Based on overall results, it can be concluded that LBP has
overall faster detection speed compared to Haar by 140%.
LBP also detected more faces than Haar by extra 4%. From
here, we can say that LBP is better than Haar in the aspect of
detection accuracy and detection speed.
ACKNOWLEDGMENT
This study is supported by Majlis Amanah Rakyat (MARA)
under Skim Geran Penyelidikan Institusi MARA.
REFERENCES
[1] Paul Viola and Michael Jones, Rapid Object Detection using a Boosted
Cascade of Simple Features. Accepted Conference on Computer
Vision and Pattern Recognition, 2001.
[2] Constantine P., M. Orent T. Poggio. A General Framework for Object
Detection. International Conference on Computer Vision, 1998.
[3] Y. Freund and R.E. Schapire. A Decision-Theoretic Generalization of
on-Line and an Application to Boosting. Journal of Computer and
System Sciences, 1997.
[4] T. Ojala, M. Pietikainen, and D. Harwood, A comparative study of
texture measures with classification based on feature distributions,
Pattern Recognition, 1996.
[5] A. Hadid, M.Pietikainen and T.Ahonen. A discriminative feature space
for detecting and recognizing aces. In Proceedings of the 2004 IEEE
computer society conference on Computer vision and pattern
recognition, Washington, DC, USA, 2004, CVPR ’04, pp. 797-804,
IEEE Computer Society.
[6] A. Haar, For Orthogonal Function Systems Theory, German
Mathematical Research Journal, 1910.
[7] R. Lienhart and J. Maydt. An Extended Set of Haar-like Features for
Rapid Object Detection, 2004.
[8] R. Hewitt, 2007 “Seeing with OpenCV: How Face Detection Works”,
webpage on Cognotics, Resources for Cognitive Robotics [Online].
 2014 4th International Conference on Engineering Technology and Technopreneuship (ICE2T)

Available: : [12] T. Ojala and M. Pietikainen, Multiresolution Gray-Scale and Rotation

http://www.cognotics.com/opencv/servo_2007_series/part_2/sidebar.ht
ml
[9] C. Souza, 2012, “Haar-feature Object Detection in C#”, webpage on
CODE PROJECT [Online]. Available:
http://www.codeproject.com/Articles/441226/Haar-feature-Object-
Detection-in-Csharp
[10] T. Ahonen, A. Hadid, and M. Pietikainen, Face description with local
binary patterns: Application to face recognition, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 2006.
[11] A. Hadid, G. Zhao, T. Ahonen, and M. Pietikainen, Face analysis
using Local binary Patterns, World Scientific Volume, 2008, pp.347-
350.
Invariant Texture Classification with local Binary Patterns, Pattern
Analysis and Machine Intelligence, 2002.
[13] J. Chang-yeon, Face Detection using LBP features, CS 229 Final
Project Report, 2008.
[14] “The Color FERET Database”, webpage on NIST [Online]. Available:
http://www.nist.gov/itl/iad/ig/colorferet.cfm/.
[15] 2004, “Stimulus images courtesy” of M. J. Tarr, Center for the Neural
Basis of Cognition and Department of Psychology, Carnegie Mellon
University (Brown University), webpage on The CNBC Wiki [Online].
Available:http://cbcl.mit.edu/software-datasets/heisele/facerecognition-
database.html/.
[16] 2008, “The MIT CBCL Face Recognition Database”, webpage on MIT
CBCL [Online]. Available: http://wiki.cnbc.cmu.edu/Face_Place/

339