March-April 2020

ISSN: 0193-4120 Page No. 14249 - 14255

Skin Tone Recognition and Face Detection using

Local Binary Pattern and Sparse Coding
Naluguru Udaya Kumar1, A. Ushasree2, Ramashri Tirumala3, Shijin Kumar P.S4, Abhilash K.S5
1
Associate Professor, Department of Electronics and Communication Engineering, Marri Laxman Reddy Institute of
Technology and Management, Hyderabad, India.
2
Assistant Professor, Department of Electronics and Communication Engineering, Gokaraju Rangaraju Institute of
Engineering and Technology, Hyderabad, India.
3
Professor, Department of Electronics and Communication Engineering, Sri Venkateswara University College of
Engineering, Tirupati, India.
4
Associate Professor, Department of Electronics and Communication Engineering, Marri Laxman Reddy Institute of
Technology and Management, Hyderabad, India.
5
Principal, Jai Bharath College of Management and Engineering Technology, Perumbavoor, India.

Article Info Abstract

Volume 83
Page Number: 14249 - 14255
Publication Issue:
March - April 2020
Article History
Article Received: 24 July 2019
Revised: 12 September 2019
Accepted: 15 February 2020
Publication: 20 April 2020
Face detection deals with the identification and delineation of facial features from a
collection of object features in the image. Due to increased intra class variance in facial
images, low level features deficit adequate meaning. This will lead to unsatisfactory
retrieval results. An advanced method for face detection is implemented in this paper to
overcome these problems. This system utilizes human attributes containing semantic cues of
facial images to improve detection efficiency by computing semantic code words. Initially
face region is detected from the image of interest. Then Local Binary Pattern (LBP) is
obtained from the face region detected. These LBP features are the input to the sparse
coding along with the facial skin tone to optimize the retrieval process. Sparse coding is
applied as an offline step for constructing sparse code words of all images in the data set.
Inverted indexing is applied in the online stages for efficient face image detection. By
combining these methods efficient face image detection can be achieved by utilizing the
advantages of higher level attributes and lower level features.
Keywords: Face Detection, Skin Tone Recognition, Local Binary Pattern, Optimal
Sparse Coding.

I. INTRODUCTION time and can refine the dataset to narrow down to

face images of the candidate. To surmount such
Face detection has numerous applications such as
drawbacks of low level features, the proposed
face tagging, face image clustering, crime
system combine facial colour attribute with the low
investigation, automatic face annotation, etc. Facial
level features. Facial attributes are seldom precisely
attributes are important parameters in matching face
exploited in existing face detection algorithms.
images [1]. These are high level features in the
Appropriate conflation of attributes and features is
images and very useful in retrieval tasks. Even
anticipated to produce improvement in recognition
though face images are partially damaged or
accuracy. Sparse coding provides assuring results in
occluded, facial attributes helps in retrieval process.
various applications such as feature extraction, noise
Facial attributes are physical characteristics
reduction, image compression and classification.
including colour, mouth shape, nose size and face
similarity. These attributes seldom change over life
14249
Published by: The Mattingley Publishing Co., Inc.

Utilizing the benefits of sparse coding, proposed
system introduces a face detection scheme based on
inverted index generated from sparse representation.
This work is intended to identify the problems in
face image retrieval is to introduce a comprehensive
coding technique for face the retrieval application.
Further the performance is improved by using face
attribute information into this algorithm. Here a
retinal neural network inspects small regions of
given image and check for face. The neural
networks are directly applied to the input image
portions and the results are arbitrated [2].
Initial step in face detection is the segmentation of
human skin to determine probable regions
corresponds to human face. Segmentation utilizes a
model based method to represent and differentiate
skin colour from background colours. From the
initial segmentation outputs, human faces are
separated by using adaptive shape analysis. Each
human face is located using a view based face
detection methodology. Eigen maps learning method
provided robust representation in varying
illumination condition. Instead of intensity values,
gradient orientation was considered. Han et al. [3]
implemented cosine kernel to generate image
correlation to ensure robustness towards noise and
other outliers. Su et al. [4] utilized Dual Tree
Complex Wavelet Transform (DTCWT) to extract
image features. An extreme learning machine
(ELM) is assigned to classify images. In ELM,
iterative modification of weight is unnecessary and
it is highly agile than the gradient descent method
proposed by Nikan et al [5]. A proper number of
DCT coefficients, are discarded to reduce the
illumination effect Nabatchian et al. [6] on the
image. Yang et al. [7] discussed the generation of
dual eigen spaces by extracting the algebraic
features of samples and performed face
identification using a dual layer minimum distance
classifier.
The remainder of this paper is arranged as follows.
Section II presents the proposed face detection and
Published by: The Mattingley Publishing Co., Inc.
March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255
skin tone recognition scheme in detail. The
experimental results with output images and
performance analysis are illustrated in Section III.
Finally, section IV provides conclusion for this
paper.
II. METHODOLOGY
The work flow of proposed facial image retrieval
system is delineated in Fig.1. A new face detection
algorithm combining Haar Wavelets and Adaptive
Boosting (AdaBoost) Classifier is applied on all
images in the database to find the face location.
Along with face detection, proposed system find
local patches of face (Eyes, Nose, and Mouth).
Fig. 1. Block Diagram of the Proposed System
Facial attributes like Face colour, Patch Similarity
are found out separately for all images. Image
patches are extracted and uniform LBP feature
descriptor are calculated to use as local features.
These local feature descriptors are then quantized to
generate code words using optimal sparse coding. It
is performed in the training stage of retrieval system.
For efficient detection, attribute embedded inverted
index is then created. When a query image is given,
the above procedures are followed to generate sparse
code words. These code words with binary attributes
are used to retrieve images. Therefore inverted
indexing is done in the testing stage of these
systems.
14250
 March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255

A. Face Detection
Proposed face detection method has mainly three
steps. In the first step a unique image descriptor
named integral image is computed for faster feature
evaluation. The integral image is calculated using
point operations on the image. The next step is to
construct a classifier by taking minimum features
using AdaBoost [8]. In a sub image the number of
Haar feature is higher than the count of all image
Fig. 3. Integral Image
pixels. The third step is to successively combine
complex classifiers in cascade to improve the speed The integral image is given by the equation,
of face detector by targeting prominent image areas.
I ( x, y ) = ∑
x '≤ x , y '≤ y
i ( x ', y ') (1)

where, I(x,y) is the integral image and i(x,y) is the

original image.

p ( x, y=
) p ( x, y − 1) + i ( x, y ) (2)

I ( x, y ) = I ( x − 1, y ) + p ( x, y ) (3)

AdaBoost is the machine learning process used to

Fig. 2. Rectangular Features
Rectangle combinations are used for visual detection
of objects in face image. Haar features contain
rectangle combinations optimized for visual
recognition tasks. Basic rectangular features used in
select particular Haar features along with optimum
threshold levels. AdaBoost conjoins N number of
classifiers to produce an efficient classifier. The
weak classifiers are combined and assigned with
weight to produce strong classifier.
C( x ) = a1c1 ( x ) + a2 c2 ( x ) + a3 c3 ( x ) + ........... (4)

the proposed detector are elucidated in Fig.2. The

where, C(x) is the strong classifier and it is a
existence of Haar features is measured by
weighted combination of lot of weak classifiers
subtracting the mean of low intensity pixels from the
ci(x).
mean of high intensity pixels. If the difference thus
obtained is higher than the threshold, feature is said Consider the training stage with images of face and
to be present. non-face. During the initial stage all these images
have equal weights. And by using a weak classifier
To efficiently check the existence of Haar features
or feature with minimum error rate classify the
in all locations, integral image technique is used.
images. Sometimes non faces are classified as faces
The integral of a pixel is the summation of intensity
and faces are classified as non faces. In the next step
values of all pixel intensities above and left to it.
AdaBoost increment the weights of misclassified
Starting from the top left and moving towards right,
ones and decrement the weights of classified ones. It
whole image can be integrated. Rectangle features
is done to give more attention to the misclassified
are computed instantaneously from integral image
images. After the update of weights the images are
which is given in Fig.3.
classified like in first step and repeat the process. In
14251
Published by: The Mattingley Publishing Co., Inc.

testing stage, when a new image comes the strong
classifier classify the images correctly. A series of
AdaBoost classifiers is combined to form a chain of
filters, as shown in Fig.4. Each of these filters is an
AdaBoost classifier with minimum number of weak
classifiers.
Fig. 4. Cascading of Classifiers
B. Skin Colour Detection
In skin colour detection, the initial step is the
development of a skin attribute model. Input images
are transformed from RGB to Y Cb Cr space. Then
the output image is transformed into binary image
using the following threshold properties of the
colour space. 130 < Y < 164; 97 <Cb< 138; 140 <
Cr< 160.
Fig. 5. Image after skin colour segmentation
Pixels lying in this range are assigned as white and
others are assigned as black. The output will be a
Published by: The Mattingley Publishing Co., Inc.
March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255
binary image consist of 1 (skin) and 0s (not-skin).
Morphological operations are then applied to
separate the loosely connected (overlapped) skin
areas. Initially, closing is performed on binary
image. The dilated image is then multiplied
segmented output to retain the number of holes. In
the later stage, these holes are used to filter out some
non-face regions. Finally, stage cross correlation is
obtained between gray scale region and template
face. By choosing a suitable threshold, faces are
classified into black or white.
Fig. 6. Result After Morphological Processing
C. Local Binary Pattern Features
Local Binary Pattern (LBP) [9] is a nonparametric
method that identifies local structures in images by
the comparison of pixels with its neighbouring
pixels. Significant properties of LBP are monotonic
illumination tolerance and computational simplicity.
The LBP operator marks all pixels in the image
using decimal numbers. These numbers are known
as LBP codes. The local structures are encoded
around each pixel using these codes. Every pixel
present in the image is compared with their 8-
neighbors. This operation is performed by the
subtraction of central pixel intensity in a 3×3
neighbourhood. From the resultant, negative values
are coded as 0, and the rest as 1. For every input
pixel, a binary value will be obtained by modifying
the resultant in clockwise direction. The decimal
equivalent of resultant binary are used to label given
pixel.
14252
 March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255

Algorithm 1: Optimal Sparse Coding

Step 1: Initialize the residual function ‘r(0)’ as input signal
Step 2: Compute inner product ‘tl*’, between residual element and
dictionary element.
Step 3: Find the position of ‘tl*’ in the dictionary, s (tl*).
Step 4: Update residual as; r ( l + 1)= r ( l ) − s ( tl * ) .d ( tl * ) .

∑ s (tl *) d (tl *)
l

Step 5: Stopping condition: x − < E

i =1

Fig. 7. Application of LBP on an Image III. RESULTS AND DISCUSSIONS

Major limitation of LBP operator is the incapability
of small 3×3 kernel to obtain striking features
present in big structures. Face image is initially split
into tiny regions and LBP features are extracted.
These features are concatenated to form a feature
histogram. In LBP, the face images are considered
as a combination of micro patterns invariant to grey
scale transforms. A global representation of facial
image is obtained by combining these micro
patterns. LBP is considered as uniform, if there exist
two or more bitwise transformations from (0 to 1) or
(1 to 0). An example for LBP is shown in Fig. 8.
MATLAB is used as simulation tool to obtain the
results. Images from LFW dataset are used as input
images for the experiment. In LFW data set, face of
person in the image is labelled with name. The
extracted local image patches and LBP features are
obtained. Skin colour is considered as an attribute of
the faces and is classified. The simulation results
obtained for LFW dataset are given below.

Fig. 9. (a) Detected Face, (b) Extracted Face

Fig. 8. Local Binary Pattern
D. Optimal Sparse Coding
The basic principle behind sparse coding is the
representation of image signals as a linear
combination of atoms from the dictionary. All the
given signals are represented using multiple
combinations of separate atoms. For the sparse
representation of facial attributes, the dictionary is
first divided into distinct sparse codes for images
having distinct attributes. For one attribute of human
face, the dictionary is divided into two distinct
subsets. Images having positive attribute value
utilize one subset and images having negative
attribute values utilize the second. Fig. 10. (a) Face Patch (b) LBP (c) Histogram of
LBP
14253
Published by: The Mattingley Publishing Co., Inc.
 March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255

To obtain the skin colour, the segmented image is
initially considered. Then the input gray image is
extracted along with the template and the face
region. The average gray level values are obtained
and are compared with a predefined threshold. For a
given input image, segmented image and matched
output are shown in Fig. 11. The skin tone detection
output is illustrated in Fig. 12.
Fig. 11. (a) Input Image, (b) Skin Segmented
image, (c) Template Matched Output
Mean of performance parameters are calculated to
compare the performance of the proposed system
with existing systems. Average accuracy of the
proposed system is 98.65%, which is higher
compared to accuracy of existing face detection
methods. The accuracy obtained using the
combination of Principal Component Analysis
(PCA) and Artificial Neural Network (ANN) is
96.94%, which is little bit lower than the accuracy of
proposed method. Average specificity of the
proposed system is 98.11% and all other methods
exhibits lower specificity during detection. The
specificity obtained using Support Vector Machine
(SVM) is 74.86%, which is very much less than the
specificity of proposed method. Average sensitivity
of proposed method is 99.29%, which is higher than
the sensitivity of PCA-ANN method. Table 1 shows
a detailed comparison of performance parameters
while using various methods for face detection and
it is graphically represented in Fig. 13.
V. CONCLUSION

Fig. 12. Skin Tone Detection Output in Here face detection is carried out by using optimal
Command Window sparse coding. It provides high detection accuracy
while minimizing computation time. In order to
Table 1. Performance of Face Detection improve retrieval results, facial skin colour based
Algorithms attribute classification is also performed with low
Accuracy Specificity Sensitivity level features. Facial attributes provide prominent
Method
(%) (%) (%) morphological descriptions of an individual. Most of
SVM 78.95 74.86 77.46
KNN 80.91 79.93 79.97 these attributes remain same over the life time. LBP
FFNN 85.50 61.62 87.60 is tolerant to illumination fluctuations and reduced
PCA-ANN 96.94 90.10 98.28 computational complexity. But LBP facial features
IPSONN 86.12 93.27 96.32
Proposed 98.65 98.11 99.29 are viewed as a combination of distinct tiny patterns.
Optimal sparse code words for the input images are
found out and the faces are retrieved based on
similarity measures.
REFERENCES
[1] N. Kumar, A.C. Berg, P.N Belhumeur, and S.K.
Nayar, “Describable visual attributes for face
verification and image search,” in IEEE
Transactions on Pattern Analysis and Machine
Fig. 13. Comparison of Classification Intelligence (PAMI), Special Issue on Real World
Performance Face Recognition, Oct 2011.
14254
Published by: The Mattingley Publishing Co., Inc.
 March-April 2020
ISSN: 0193-4120 Page No. 14249 - 14255

[2] M. Douzeand A. Ramisa and C. Schmid,

“Combining Attributes and Fisher Vectors for
Efficient Image Retrieval,” IEEE Conference on
Computer Vision and Pattern Recognition, 2011.
[3] Han, H., S. Shan, X. Chen, and W. Gao (2013b).
Face recognition the problem of compensating for
changes in illumination direction International
Journal on Pattern Recognition, Vol. 46(Issue 6),
pp. 1691–1699.
[4] Su, Y., S. Shan, X. Chen, and W. Gao (2009).
Hierarchical ensemble of global and local classifiers
for face recognition. IEEE Transaction on Image
Processing, Vol. 18(Issue 8), pp. 1–8.
[5] Nikan, S. and M. Ahmadi (2015). Performance
evaluation of different feature extractors and
classifiers for recognition of human faces with low
resolution images. International Journal of
Intelligent Systems and Applications in
Engineering, Vol. 3 (Issue 2), pp. 72–77.
[6] Nabatchian, A., E. Abdel-Raheem, and M. Ahmadi
(2011). Illumination invariant feature extraction and
mutual-information-based local matching for face
recognition under illumination variation and
occlusion. Pattern Recognition, Vol. 41(Issue 1), pp.
12576–2587.
[7] Yang, M., L. Zhang, S. C. Shiu, and D. Zhang
(2013). Robust kernel representation with statistical
local features for face recognition. IEEE
Transaction on Neural Network Learning Systems,
Vol. 24(Issue 1), pp. 900–912.
[8] Murugananthan, V., & Shivakumar, B. L. (2019).
An Exploration on Data Mining for Face Detection
based on Real time Face Tracking. Journal of
Computer Technology & Applications, 5(3), 46-51.
[9] Li, L., Feng, X., Xia, Z., Jiang, X., & Hadid, A.
(2018). Face spoofing detection with local binary
pattern network. Journal of visual communication
and image representation, 5(4), 182-192.

14255
Published by: The Mattingley Publishing Co., Inc.