Design Creative & Thinking

Lab Project Report

On

Case Study: The Challenges

in Face Detection System

BY
SHREYA

Department of Computer Science & Engineering,

School of Engineering & Technology
July 2022
1
 CANDIDATE’S DECLARATION

I/we hereby declare that the project entitled “Case Study: The Challenges in
Face Detection System” submitted in partial fulfillment for requirements of the
Design Creative & Thinking Lab completed under the guidance of Dr. Vivek
Kumar Singh, Sharda University is an authentic work.

Further, I/we declare that I/we have not submitted this work for the
award/ requirements of any other degree/course elsewhere.

Signature and name of the student(s) with date

CERTIFICATE by The Project Guide

It is certified that the above statement made by the students is correct to

the best of my/our knowledge.

Signature of the project guide with dates and their designation

2
 Abstract
The face is perhaps the most essential tool for human identification and is regularly employed
in a variety of contexts due to the vast number of its distinguishing feature. In expressing
individual's identity and feelings, facial features are crucial. Numerous strategies have been
applied so far in this field. The goal of this work is to outline the difficulties and problems
that can occur when trying to identify and analyze faces using a recognition system due to
factors like light, ageing, facial expressions, pose variation, complex background, skin color
and other more general problems. This paper explains all the above mentioned problems in
detail and it also includes the proposed methods by various researchers focusing on tackling
these problems.

3
 Table of Contents

S no. Content Page no.

1. Candidate’s Declaration 2
2. Guide’s Certificate 2
3. Abstract 3
4. Introduction 7-8
5. Survey of the State of the Art 9
6. Datasets and Evaluation Measures 10
7. Discussion 11
8. Summary and Conclusion 12
9. References 13

4
 List of Figures

S no. Name Page no.

1. This picture demonstrates the various challenges

in face detection like occlusion, too many 7
photos, low resolution, pose variation,
illumination, and complex background.

2. Block Diagram for Introduction of Challenges 8

Faced by Face Detection System.

5
List of Symbols, Abbreviations and Nomenclature

ABBREVIATION FULL NAME

CNN Convolutional Neural
Network
SOC Structured Occlusion Coding
3D Three Dimension
HOG Histograms of Oriented
Gradients
SIFT Scale Invariant Feature
Transform
SURF Speeded Up Robust Features
RPN Region Proposal Network
ILSVRC Image-Net Large Scale
Visual Recognition
Challenge
FDDB Face Detection Dataset and
Benchmark

6
Chapter 1: Introduction
Face detection methods utilize technology to detect and identify human features in
digital videos or photos. Trees, hills, roads, cars, buildings, and other elements of
the digital photo are not taken into account. Researchers working on computer
vision, pattern classification, and fingerprints have recently been quite interested in
it. Applications like security and forensics involve face detection and identification
systems. Face detection technology is quite helpful in daily life. Of all biometric
technologies, face detection and classification technologies are the most precise.
Face detection techniques are increasingly being applied in real-world applications
and goods, which is fascinating. Face detection is often used as a first step in other
applications such as face tracking, analysis and, most crucially, face recognition. It
aids face analysis by indicating the sections of the video or image that should be
examined when establishing a person's age, gender, and emotions. It also aids in
face recognition because it is included in the system's algorithm, which aids in
selecting which part of the video and image is required to create a face print. It has
a wide range of applications in domains such as security, biometrics, and personal
safety.

Fig 1: This picture demonstrates the various challenges in face detection like occlusion, too
many photos, low resolution, pose variation, illumination, complex background.( Kumar, A.,
Kaur, A., & Kumar, M. (2018). Face detection techniques: a review. Artificial Intelligence
Review, 52(2), 927-948. doi: 10.1007/s10462-018-9650-2) [11]

Numerous, face detection approaches have been suggested in the literature, which are
effectively performing to detect human faces. However, still there are some challenges such
as odd expressions, facial occlusion, illuminations, complex background, too many faces,
resolution, and skin color shown in fig1. Odd expressions: human’s face might have strange
facial expression. It is hard for these systems to categorize it as a human face. Occlusion:
when human face is obscured by hair, a hat, a hand, glasses, or a scarf, it is called occlusion.
7
It is possible that you will get a false negative. Illuminations: it is possible that an image does
not have consistent lighting effects; some areas of the photograph may be overexposed, while
others may be underexposed. Another area is very dimly lit. This, too, can lead to false
negatives. Complex Background: background information is difficult to come by. When
plenty of items are present in a digital photo, the validity of these systems suffers. Too many
faces: face detection technology may find difficulty in differentiating human faces in digital
photo. Low resolution: when the resolution of an image is low, it becomes more challenging
in detecting faces. Skin tone: when an individual’s skin tone falls outside of the algorithm's
known gradient, their face may not be recognized.

In this report, we will focus on the case study on the above challenges present in the state-of-
the-art methods. The various solutions proposed by different researchers to tackle these
problems are included here as well.

Challenges Present In the Face

Detection System

Illumination Complex Background Pose

variation

Odd Occlusion Low Resolution

Expression

Fig 2: Block Diagram for Introduction of Challenges Faced by Face Detection System.

This block diagram represents the pictorial form of the challenges present in the already
developed face detection system. The various problems encountered are illumination, odd
expression, complex background, occlusion, facial-expression, low resolution, shown in fig 2.

8
Chapter 2: Survey of the State of the Art
Hwang et al., provides a reliable face detection technology for enormous data sets taken
under unpredictable lighting situations. The recommended approach suggests a pre-
processing technique for integrating and standardizing the variations of a facial image. [1]
Sompura et al., proposed a fast, efficient algorithm with a higher recognition rate of faces in
various settings, such as lighting, head attitude, expressions, and so on. Using Principal
Component Analysis and Local Binary Pattern, the suggested system collects global and local
features. [2] Abbad et al., proposed a method of 3D detection technology based on the
geometrical and localized form of characteristics, to overcome the challenges of
strange expressions. [3] Yang et al., proposed a system that has three tiers and employs a
hierarchical knowledge-based strategy. The two higher tiers are built using mosaic photos
with various resolutions. An enhanced edge detection technique is suggested at the lower
level. In this study, the scale issue is addressed such that the software can find unidentified
human faces in a complicated black-and-white image that range in size. [4] Ruan et al.,
proposed a quick face detection technique that can produce reliable results. Researchers
employ illumination adjustment to lessen the computational burden of the feature-based
scheme while enhancing the performance of the color-based scheme. Our approach works
well with different facial expressions such half-profile faces, faux faces, close eyes, open
mouths, dark/bright vision, and close vision. It is important to emphasize that our technology
can correctly distinguish between cartoon and human faces. [5]

Wan et al., proposed the DeepMaskNet approach, a trainable module that may be
incorporated into current CNN architectures. DeepMaskNet learns how to properly generate
various feature map masks for various occluded face images using end-to-end learning
monitored only by personal identity labels. It instinctively assigns greater weight to hidden
units which are activated by facial portions that are not obscured and significantly lower
compared to hidden units which are activated by obscured human face. [6] Xiang et al.,
proposed the structured occlusion coding (SOC) to address occlusion issues . Here, there are
two layers of organized coding. On the one hand, for recognition, we need an organized
lexicon. Contrarily, we suggest utilizing organized inside this formulation. In particular, SOC
concurrently classifies the image and separates the occlusion. This method transforms the
challenge of identifying an obscured image into a search for an organized sparse solution on
an occlusion-appended dictionary. [7] Aksasse et al., proposed an innovative method for
pose-invariant facial identification. They suggested a method for face alignment using a
single 3D face model in relation to FaceGen Modeler in an effort to increase the rate of face
recognition.[8] Bowyer et al., developed a new Gabor-based technique that models face
deformations brought on by changing position by altering the grids where the Gabor features
are selected using a mesh. Additionally, to enhance recognition performance across pose, a
statistical model of the scores obtained using the Gabor features is applied. [9] Shekhar et al.
proposed a method of dictionary learning strategy for identifying the low-resolution probe
image. The ability of this algorithm to tolerate resolution and lighting changes is a key
component.[10]

9
Chapter 3: Datasets and Evaluation Measures
The difficulties in face detection technology are what cause the efficiency and detection ratio
of face recognition software to decrease. These difficulties include complex backgrounds, an
excessive number of faces in the photographs, strange expressions, illuminations, low
image resolution, occlusion, skin tone, distance, orientation, etc.

Some of the Dataset used for face detection:

 Wider face dataset - The publicly accessible WIDER dataset is used to create the
benchmark dataset for face detection known as WIDER FACE. With a significant
degree of variation in scale, position, and occlusion, researchers select 32,203 images
and classify 393,703 faces. The 61 event classes were used to structure the WIDER
FACE dataset. Then choose 40 per cent, 10 per cent, and 50 per cent of the data for
training, validation, and testing sets for each event class at random. They use the same
assessment methodology as the PASCAL VOC dataset. Researchers do not publish
the ground truth for the bounding boxes for the test photos, just like the MALF and
Caltech datasets. Users must submit their final prediction data, which will be
reviewed by researchers.
 Face detection dataset and benchmark - This dataset includes 5'171 faces which have
been identified in 2'845 photos with such a variety of challenges, including
facial occlusions, challenging poses, and poor image resolutions. These photos are
used to test with significant appearance changes, significant blur degradations, and
high occlusions, all of which are common when identifying a face in unrestricted real-
life circumstances.

10
Chapter 4: Discussions
The last several years have seen tremendous advancement in face detection, with new
benchmarks being reached every year. Since its inception in the 1990s, face detection seems
to have been a difficult study area. Prior to the landmark work proposed by Viola and Jones
in 2000, despite numerous investigations, the practical performance of facial recognition was
far from ideal. Face detection has advanced significantly since the innovative research of
Viola-Jones. Innovators Viola and Jones (2004), develop a face detector technology with
excellent accuracy and efficiency using Haar features and AdaBoost, and their work serves as
an inspiration for a number of subsequent methods.

Early methods primarily concentrated on extracting various kinds of manually created

features with computer vision domain expertise and developing efficient classifiers for
detection using conventional machine learning techniques. Such approaches have limitations
because each component of the detection pipeline is typically not optimized as a whole,
necessitating the assistance of professionals in computer vision to create useful features.
Much work has gone into developing more complex features including HOG, SIFT, SURF,
and aggregate channel features (ACF) to address the first issue. Numerous detectors that had
been generated separately for various viewpoints or positions have been combined to increase
the robustness of detection. However, the supervised learning of such models typically took
more time, and the improvement in detection rate was only marginal.

Deep learning techniques, particularly CNN (deep convolutional neural networks), have
shown notable improvements in facial identification over the past few years in a variety of
computer vision tasks. Deep learning techniques have surpassed traditional computer vision
algorithms in many well-known benchmark tests, including the ILSVRC Challenge, by
avoiding the hand-crafted design process. Recently, scientists used the Quick R-CNN, one of
the most advanced generalized object detectors, and obtained encouraging outcomes.
Additionally, end-to-end optimization has been achieved through cooperative training on
CNN cascades, RPN, and Faster R-CNN. ResNet and hard negative mining were used with a
quicker R-CNN face detection method to make notable improvements in detection accuracy
on face detection method benchmarks like FDDB.

11
Chapter 5: Summary and Conclusion
In this paper, I have discussed various problems in face detection systems in great depth.
Face detection technology is the most extensively utilized biometric system, hence the
system's efficiency must be considered. Furthermore, facial photographs are frequently taken
in a natural setting. Backgrounds for images might be complicated, and lighting variations
can be considerable. I have talked about issues like odd expressions, facial occlusion,
illuminations, complex background, too many faces, resolution, and skin color and how to
deal with them. They can be regulated using proper methods. Several algorithms have been
developed to deal with these issues. However, when these issues arise, it is difficult to
recognize the human face. I have also mentioned the proposed method by various researchers
on respective issues. According to my research, deep learning using CNN gives accurate
results. However, there is plenty of space for improvements. Furthermore, about facial
detection and recognition, there are varieties of challenges ahead, as well as plenty of
opportunities. In the future, I will work on this topic and will try to propose a solution for the
mentioned issue. 

12
References
[1] Hwang, W., Wang, H., Kim, H., Kee, S. C., & Kim, J. (2010). Face recognition
system using multiple face model of hybrid Fourier feature under uncontrolled
illumination variation. IEEE transactions on image processing, 20(4), 1152-1165.
[2] Sompura, M., & Gupta, V. (2015). An efficient face recognition with ANN using
hybrid feature extraction methods. International Journal of Computer
Applications, 117(17).
[3] Abbad, A., Abbad, K., & Tairi, H. (2018). 3D face recognition: Multi-scale
strategy based on geometric and local descriptors. Computers & Electrical
Engineering, 70, 525-537.
[4] Yang, G., & Huang, T. S. (1994). Human face detection in a complex
background. Pattern recognition, 27(1), 53-63.
[5] Pai, Y. T., Ruan, S. J., Shie, M. C., & Liu, Y. C. (2006, July). A simple and
accurate color face detection algorithm in complex background. In 2006 IEEE
International Conference on Multimedia and Expo (pp. 1545-1548).IEEE.
[6] Wan, W., & Chen, J. (2017, September). Occlusion robust face recognition based
on mask learning. In 2017 IEEE international conference on image processing
(ICIP) (pp. 3795-3799). IEEE.
[7] Wen, Y., Liu, W., Yang, M., Fu, Y., Xiang, Y., & Hu, R. (2016). Structured
occlusion coding for robust face recognition. Neurocomputing, 178, 11-24.
[8] Singh, S., & Prasad, S. V. A. V. (2018). Techniques and challenges of face
recognition: A critical review. Procedia computer science, 143, 536-543.
[9] Cament, L. A., Galdames, F. J., Bowyer, K. W., & Perez, C. A. (2015). Face
recognition under pose variation with local Gabor features enhanced by active
shape and statistical models. Pattern Recognition, 48(11), 3371-3384.
[10] Shekhar, S., Patel, V. M., & Chellappa, R. (2017). Synthesis-based robust low
resolution face recognition. arXiv preprint arXiv:1707.02733.
[11] Kumar, A., Kaur, A., & Kumar, M. (2018). Face detection techniques: a review.
Artificial Intelligence Review, 52(2), 927-948. doi: 10.1007/s10462-018-9650-2

13