VISVESVARAYA TECHNOLOGICAL UNIVERSITY “JNANA

SANGAMA”, BELAGAVI-590018

AN TECHNICAL SEMINAR REPORT ON

Multi-Face Recognition Processing using Haar Cascades and Eigenface Method

Submitted in Partial Fulfilment for the Award of Degree of

Bachelor of Engineering
In
Electronics and Communication Engineering
Submitted by

JOYCELITA DIAS
(USN:4MK17EC003)

UNDER THE GUIDANCE OF

Ms. Sookshma Adiga
Assistant professor
dept of ECE
MIT Kundapura
 MOODLAKATTE INSTITUTE OF TECHNOLOGY
MOODLAKATTE, KUNDAPURA - 576217

DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING

CERTIFICATE

This is to certify that Miss. Joycelita Dias (4MK17EC003), a bonafide student of Moodlakatte
Institute of Technology Kundapura in partial fulfilment for the award of Bachelor of
Engineering in Electronics and Communication Engineering of the Visvesvaraya
Technological University, Belagavi. during the academic year 2020-2021. It is certified that all
corrections/suggestions indicated for Internal Assessment have been incorporated in the report.
The report of Internship/Professional Practice has been approved as it satisfies the academic
requirements in respect of Internship/Professional Practice prescribed for the said degree.

(Signature of Guide) (Signature of HOD) (Signature of Principal)

Ms. Sookshma Adiga Mr.Balnageshwara S Dr. Chandra Rao Madane

Asst.prof dept of ECE HOD Dept of ECE MIT Kundapura
MIT Kundapura MIT Kundapura
 ACKNOWLEDGEMENT

This technical involves the collection an analysis of information from a wide variety of source and the
efforts of many beyond us.

I take this opportunity to acknowledge our profound gratitude to principal Dr. Chandra Rao
Madane, MIT Kundapura for his support and suggestion and support and suggestion provided
during during our study in the institution.

I convey our sincere thank Mr, Balnageshwara S, assistant professor and head, Department of
electronics and communication, MIT Kundapura for his suggestion and support for completion of
the technical seminar work.

I sincerely thank the collage management for all infrastructure and facilities provided during our
study in this internship.

I deeply express our sincere thanks to our Teaching and non-Teaching staff of Electronics and
Communication Engineering department for encouraging and who helped us directly in directly
for the successful of the work. Finally I would like to thank parents for supporting in hundreds of
little ways that means a lot to us.

JOYCELITA DIAS

(4MK17EC003)
 ABSTRACT
Face recognition is a technical way of identifying a human face. One of the biometric
information processes involves fingerprinting, DNA, palm printing, hand
morphology, iris recognition, retina, iris scanning and signature is a facial
recognition device. These are used by a Multiple facial recognition program to map
face features from image or video. It compares the details to a database of recognized
face to match each other. When a person recorded in the database comes under our
surveillance camera. The problem with biometric authentication is its lengthy
process and the accuracy of tests. A hybrid process of Haar Cascades and Eigenface
methods was used for the proposed face recognition technique. In this system, they
detect multiple faces in a single detection cycle i.e., up to 55 faces.

Conventional method of identification based on the exclusive knowledge like a

social security number or a password are not all together reliable. Somehow
password can be forgotten or compromised. But a face is undeniably connected to
its owner. It cannot be borrowed or stolen. Face recognition technology may solve
this problem since a face is undeniably connected to its owner expect in case of
identical twins. The system can compare scans to records stored in a central or local
database. The software first captures an image of all the authorized persons and
stores the information into the database.

Keywords: Face recognition, biometric identification, fast face recognition process,

Haar Cascade method, Eigenface method.
 TABLE OF CONTENTS

CHAPTER NO TITLE PAGE NO.

ACKNOWLEDGEMENT i
ABSTRACT ii
TABLE OF CONTENTS iii
TABLE OF FIGURES
iv

1. INTRODUCTION 1-3

1.1 Problem statement 2

1.2 Objectives 3

2. LITERATURE REVIEW 4-5

3. METHODOLOGY 6-8

4. IMPLEMENTATION 9-13

4.1 Implementation methods 9-10

4.2 Haar Cascade Technique 10-13

5. RESULTS 14-17

6. CONCLUSION 18

REFERENCES
 LIST OF FIGURES

FIGURE NO. FIGURE NAME Page No.

1.1 Multi-face Recognition 1

3.1 The proposed face recognition process 6

3.2 Process Flow 7

3.3 Face detection process on the image 8

4.2.1 An image with Haar like feature 11

4.2.2 Finding the integral image 11

5.1 Comparison of multi-faces recognition 16

Process

5.2 Result of facial image recognition 17

based on distance
Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CHAPTER 1

INTRODUCTION

Face recognition has been used as an authentication process in various fields and
especially in computer security related activities, such as homeland security, building access
security, criminal identification, as well as user identification in small mobile devices. Face
recognition also plays a significant role in the research field of biometric and computer
vision. The goal of a face recognition system is to have a negligible misclassification rate.
Biometric technology is used for authentication and it may analyze human behaviour. Each
biometric system has its own advantages and disadvantages, so that proper consideration is
required when selecting one to use in an application.
Face detection algorithms are considered to require intensive computation, which
makes it is difficult to perform face detection. Most of the face recognition processes are
implemented on a single face at a time. For a single face to be recognized, it may need a short
period of time, but for many faces/people, using single face recognition, it will take a lot of
time. Thus, it is necessary to develop a system for multiple faces recognition in one go to
speed up the recognizing process. This paper presents a study on multiple faces recognition
using a hybrid method of Haar Cascades and Eigenface.This study aims to improve the
performance of face recognition process using the Haar Cascades and Eigenface method.

Figure 1.1 Multi-Face Recognition

Dept. Of ECE, MITK Page 1

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

OpenCV can be used as a face detector type that works with the Haar-cascade
classifier. From an image, a face detector will test each part of the image and classify it as
"face" or "not face". This facial classification uses a fixed scale, for example 50 × 50 pixels.
If the face of the image is larger or smaller than the fixed scale, the classifier continuously
processes the image, searching for a face. The classifier can use data stored in an XML file to
decide how to classify each image location.
Face recognition is a technical way of identifying a human face.One of the biometric
information processes involves fingerprinting, DNA, palm printing, hand morphology, iris
recognition, retina , iris scanning and signature is a facial recognition device.These are used
by a Multiple facial recognition program to map face features from image or video.It
compares the details to a database of recognized face to match each other. When a person
recorded in the database comes under our surveillance camera. The problem with biometric
authentication is its lengthy process and the accuracy of tests.A hybrid process of Haar
Cascades and Eigenface methods was used for the proposed face recognition technique. In
this system, they detect multiple faces in a single detection cycle i.e. upto 55 faces.

The next part of this paper presents previous works that are related to face
recognition, especially ones that can identify multiple faces. It is then followed by the
methodology used in the study presented in this paper. Later, the results of the experiments
conducted are presented and discussed in section IV. Then, the conclusion of this study can
be found in section V.

1.1 PROBLEM STATEMENT

This system cannot tell the difference between identical twins. Less effective in low
light.Unable to capture if the facial expressions vary frequently. This project is to improve
the face detection system by using Haar Classifier to get higher accuracy result. Haar
Classifier is used for face detection because it can detect the desire image very fast. The
algorithm has been used for the detection which achieved high detection accuracy. It is
necessary to extract the face region from the face detection process and separate the face
from the background pattern, which provides the basis for the subsequent extraction of the
face difference features.

Dept. Of ECE, MITK Page 2

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

1.2 OBECTIVES

Face recognition System has turn into a mots emphasizing research area since it plays a
major important role in a human computer interaction.The face can articulate emotion sooner
than people verbalize or even understands than their posture. This synopsis report describes
integrated system for facial expression recogintion.Modeling the facial emotion and its
intensity.Human face recognition ability helps to develop a non human face recognition
system. To design real time face detection system. To utilize the face detection system based
on Haar Classifier.

Dept. Of ECE, MITK Page 3

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CHAPTER 2

LITERATURE REVIEW

Face recognition is one of the fastest growing research areas and it has been used in
many domains. In the field of education, it is shown to have the ability to detect, analyze and
process emotions in order to get positive teaching effects such as perception, understanding
and expressing emotions [1]. This approach consists of three stages: Feature extraction,
subset feature and emotion classifier. A Haar Cascades method is used to detect the input
image, a face, as the basis for the feature extraction of the eyes and mouth, and then the Sobel
edge detection is used to obtain the characteristic value [1].

There are two major approaches for feature extraction, typically holistic feature and
local feature. In the holistic feature-based approach, the features are extracted from the face
as a whole, which may sometimes be affected by occlusion and expression changes. Whereas
in local features-based approaches, these are overcome as patches of the image are
considered. Also, they are scale and rotation invariant [2]. This study used the holistic feature
approach using Eigen face.

Facial recognition systems have been used in small mobile environments as well, to
recognize images and video. It uses multiple object detection using the Viola-Jones cascade
classifier in the OpenCV library [3]. A frontal face is good for face recognition accuracy.
Non-frontal facial images can be reconstructed to frontal face images to increase the accuracy
of the facial recognition [4]. Another way to increase the accuracy of the face recognition can
be through filtering techniques [5]. As well the similarity measurement can be used in face
recognition by utilizing the probabilistic histograms of the maximum likelihood manner [6].
Robust face detection based on Viola Jones face detection algorithm [7, 8, 9] can also be
implemented using a graphics processing unit (GPU) [7, 8, 10].

Dept. Of ECE, MITK Page 4

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

The implementation model of this study is to help the security control. This approach
has also been implemented by detecting face as in [1, 7, 8, 9]. This study also conducted
multiple face detection tasks such as in [9,10] using sequenced images.

In video-based face recognition systems, the illumination and pose variation problems
are predominant. Most of the efficient face recognition systems are developed for controlled
or indoor environments, hence some fail to give accurate recognition in outdoor
environments with variations in illumination [2,8]. A video-based face recognition system
based on the Viola Jones face detection algorithm is an essential building block in many
applications, such as video surveillance and tracking [7, 8, 9].

Dept. Of ECE, MITK Page 5

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CHAPTER 3

METHODOLOGY
The face recognition process for multi-faces that is proposed in this study is
depictedin Figure 2. This face recognition process is going to be implemented in the
registration of computer-based exams. The process flow of the facial recognition during
registration is shown in Figure 3.1 .

Figure 3.1. The proposed face recognition process

The proposed process is started with pre-processing of the training data by converting
the RGB images to gray-scale images and reducing the images to 8-bits color. Then
implementing the Haar Cascade method to the images by going through the Haar feature
phase, integral image and cascade classifier.

The next stage is the feature extraction process, where important features for the face
recognition are extracted from the pre-processed images. The Eigenface method is combined
with PCA to perform the feature extraction process for the training data. The test data can be
directly processed using PCA for their feature extraction.

Dept. Of ECE, MITK Page 6

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

Figure 3.2 Process flow

Following the feature extraction process is the recognition process, where the
PCA_train and PCA_test data will go through the similarity distance calculation based on the
Euclidian distance technique. Based on the Euclidian distance calculations, the result for the
recognized image (read: face) is concluded. In the implementation of the proposed face
recognition process, it begins with capturing images using a webcam to get training data,
which then goes through the face detection process, feature extraction process, and then the
core of the face recognition to get the results. The results are sent to the database. The

Dept. Of ECE, MITK Page 7

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

training data in the database is later used in the face recognition process of the test data from
the registration on the day of the exam to authenticate the participants of the exam prior to
accessing the exam questions in the computer system.

During the face detection process the webcam will detect the face by putting boxes to
the detected faces as can be seen in Figure 3.2

Figure 3.3 Face detection process on the image

Dept. Of ECE, MITK Page 8

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CHAPTER 4

IMPLEMENTATION

4.1 Implementation Methods

1. Geometric Based / Template Based:-

Face recognition algorithms classified as geometry based or template based algorithms.
The template-based methods can be constructed using statistical tools like SVM [Support
Vector Machines], PCA [Principal Component Analysis], LDA [Linear Discriminant
Analysis], Kernel methods or Trace Transforms. The geometric feature based methods analyse
local facial features and their geometric relationship. It is also known as a feature-based
method.

2. Piecemeal / Wholistic:-

The relation between the elements or the connection of a function with the whole face
not undergone into the amount, many researchers followed this approach, trying to deduce the
most relevant characteristics. Some methods attempted to use the eyes, a combination of
features and so on. Some Hidden Markov Model methods also fall into this category, and
feature processing is very famous in face recognition.

3. Appearance-Based / Model-Based:-

The appearance-based method shows a face regarding several images. An image

considered as a high dimensional vector. This technique is usually used to derive a feature
space from the image division. The sample image compared to the training set. On the other
hand, the model-based approach tries to model a face. The new sample implemented to the
model and the parameters of the model used to recognise the image.

Dept. Of ECE, MITK Page 9

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

4. Template / Statistical / Neural Networks Based:-

In template matching the patterns are represented by samples, models, pixels, textures,
etc. The recognition function is usually a correlation or distance measure.In the Statistical
approach, the patterns expressed as features.

4.2 Haar Cascade Technique

The technique starts with the Haar feature phase where face detection is performed by
using a higher differentiator between ‘face’ and ‘not face’.

As shown in Figure 3, there are rectangular features on some parts of the image which
are in two forms: black (dark) and white (bright). Based on these rectangles, the Haar-like
feature is calculated. The Haar-like feature is derived from the difference between the sum of
pixels of the dark area and the sum of pixels of the bright area, as expressed in the following
formula:

F(Haar) = ∑ Fwhite - ∑

FblackWhere:

∑ Fblack = sum of pixels of the dark area

∑ Fwhite= sum of pixels of the bright

areaF(Haar) = the Haar-like feature

When the Haar-like feature is higher than a certain threshold, it can be stated that a
face or faces are within the area. To efficiently filter a high number of faces within the image,
an integral image technique is used.

Dept. Of ECE, MITK Page 10

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

Figure 4.2.1 An image with Haar-like feature

The Haar feature can be calculated based on the Integral image. Integral image is a
technique for calculating the feature value quickly by changing the value of each pixel into a
new image

Representation. The integral image value is the pixel accumulation value of the top to
left calculation. For example, the pixel value (a, b) has an accumulative value of pixels (x, y).
After integrating, the pixel at (x,y) contains the sum of all pixel values on the shaded
rectangle. The pixel values in rectangle D is ((x4,y4) + (x1,y1)) – ((x3,y3) + (x2,y2)).

Figure 4.2.2 Finding the Integral Image

Dept. Of ECE, MITK Page 11

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

One example of counting of Integral Image is as follows:

SUM: (D + A) - (B + C)

The Haar feature value is calculated based on the Integral image value that is defined
as the distinguishing value between the box (white and black pixels), which is usually known
as the threshold value. This threshold is a parameter to determine whether the object is a face
or not. The Cascade Classifier process combines many weak classifiers to become sharper
images (strong classifiers) by assigning weights to the weak classifiers image. Cascade
Classifier indirectly this can be said as a filter chain. Each filter is a classifier by a separate
boost with a relatively small number of weak classifiers.

The threshold value for each filter is set whenever the filter process occurs, so the
value of the threshold is dynamic. Filters at each level have been trained to classify imagery
that has passed the previous stage. During the classification process, if one of the filters failed
to pass, then the image can be said to be a non-face. If the image is not enough, means a
strong classifier, then the process is repeated until the weights are met by raising the value of
the threshold. When the image can pass through each filter in the chain, it means that the area
is the face. To get accurate values, the processing of haar-like features is performed. If at
stage-1 the results do not meet the criteria, the result is rejected. Then, the algorithm will
move to sub next window that is phase-2 and will count feature back, if the result received
fits in the threshold, then it will be continued at the next filter stage so that the sub window
step decreases and moves closer to the real image value.

Filters at each level are trained to classify previously filtered images (training set is a
database of faces). During its use, if one of the filters fails, that region of the image is
classified as "Not Face". When the filter successfully passes that region, it then enters the
next filter. Image regions that have been through all filters will be considered as "Face".

The face detection program is processed as follows:

1. The face image of the capture result is changed from RGB to Greyscale.

2. Normalization of lighting.

3. Haar Cascade detection.

4. Facial extract with eigenface calculation for face recognition.

Dept. Of ECE, MITK Page 12

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

5. Face recognition process with PCA calculation (composing flat-vector image matrix and
determining the vector of the image) and the calculation of eigenvector and eigenvalue (by
computing co-variance).

Dept. Of ECE, MITK Page 13

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CHAPTER 5

RESULTS

Experiments using the proposed face recognition process involving several factors
have been conducted to see the performance of the system developed based on the proposed
technique. The first experiment was to look at the position factor of the faces that were
being recognized. Data in Table 1 shows the results from this experiment. The results show
that only when the face is in the side facing position for 30 degrees, then the face was fail to
be detected.

Table 5.1. Detection results for various face position

Position Degree of the position Detection results (%)

Normal 0 100
Head up 25 100
Head down 25 100
Side facing 1 15 100
Side facing 2 30 0

The second experiment studied the distance factor of the face detection process.
Table 5.2 shows the results of this experimentation. The data shows that face detection can
be successfully performed up to 200 cm.

The next experiment is to see how many different faces can be detected in a single
instance during the face recognition process. The results are presented in Table 5.3. The
data in Table 5.3 shows that out of 60 faces that need to be detected, 55 can be successfully
recognized. This means that the success rate is 91.67% for the multi-faces recognition
process.

Table 5.2. Detection results at various distance

Dept. Of ECE, MITK Page 14

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

Distance Targeted distance Detection distance

D1 20 20
D2 70 70
D3 120 120
D4 170 170
D5 200 200

The next experiment is to see how many different faces can be detected in a single
instance during the face recognition process. The results are presented in Table 5.3. The
data in Table 5.3 shows that out of 60 faces that need to be detected, 55 can be successfully
recognized. This means that the success rate is 91.67% for the multi-faces recognition
process.

Table 5.3. Results of multi-faces recognition process

Location Targeted number of faces Detected number of faces

Classroom 1 2 2
Classroom 2 3 3
Classroom 3 5 5
Classroom 4 6 6
Open space 60 60

Dept. Of ECE, MITK Page 15

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

Figure 5.1 Comparison of multi-faces recognition process

The data in Table 5.4 shows the following:

1. The facial recognition process is successfully optimized within the specified target.

2. When the face is turned away, the facial image is still able to be recognized, even
thoughthe selected method is for straight face detection.

Table 5. 4. Multi-face recognition optimization

Face Position Image targets to be The results of the face

recognized image to be recognized
1 facial image (normal) 1 1
2 facial image (optimization) 2 2
3 facial image (optimization) 3 3

Figure 5.1shows that during test 1, 2 and 3, the face of user-1 and the face of user-2
can be recognized. Recognizing the facial image can be performed successfully up to 200 cm.

Dept. Of ECE, MITK Page 16

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

Figure 5.2 Results of facial image recognition based on distance.

Dept. Of ECE, MITK Page 17

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

CONCLUSION
The process of facial recognition with the Haar Cascade and Eigenface method is able
to detect and recognize the face both during the day and night (with good light) as shown in
the test results. Although the type of detection is for straight faces (frontal faces), it is still
able to detect the face when it is facing to the side until about 15o (degrees). The facial
recognition process with the Haar Cascade and Eigenface method is able to optimize facial
recognition with more than one face with accuracy up to 91, 67%. The facial recognition
process with the Haar Cascade and Eigenface method can be successfully performed at a
distance of more than 200 cm using a webcam. For future work, while the current
development is for straight faces, it can to be improved to recognize faces at varying angles.
Followed by the optimization of the facial recognition process for use on a small mobile
device.

Dept. Of ECE, MITK Page 18

Multi-Faces Recognition Process Using Haar Cascades And Eigenface Methods 2020-21

REFERENCES

[1] D. Yang, AbeerAlsadoon, P.W.C. Prasad, A.K. Singh, A. Elchouemi, “An Emotion
Recognition Model Based on Facial Recognition in Virtual Learning Environment”,
Procedia Computer Science, Vol. 125, pp.2-10, 2018.
[2] V. Mohanraj, M. Vimalkumar, M. Mithila, V. Vaidehi, “Robust Face Recognition
System in Video using Hybrid Scale Invariant Feature Transform”, Procedia
Computer Science, Vol. 93, pp. 503-512, 2016.
[3] N. Kazanskiy, V. Protsenko, P. Serafimovich, “Performance analysis of real-time face
detection system based on stream data mining frameworks”, Procedia Engineering,
Vol. 201, pp. 806-816, 2017.
[4] J. Kavitha, T.T. Mirnalinee, “Automatic Frontal Face Reconstruction Approach for
Pose Invariant Face Recognition”, Procedia Computer Science, Vol. 87, pp. 300-305,
2016.
[5] Vinay A., G. Kathiresan, D. A. Mundroy, H. NiharNandan, C. Sureka, K.N. B.
Murthy, S. Natarajan, “Face Recognition using Filtered Eohsift”, Procedia Computer
Science, Vol. 79, pp. 543-552, 2016.
[6] G. Srikote, A. Meesomboon, “Face Recognition Performance Improvement Using
Derivative of Accumulated Absolute Difference Based on Probabilistic Histogram”,
Procedia Computer Science, Vol. 86, pp. 265-268, 2016.
[7] V. Jain, D. Patel, “A GPU Based Implementation of Robust Face Detection System”,
Procedia Computer Science, Vol. 87, pp. 156-163, 2016.
[8] V.E. MachacaArceda, K.M. FernándezFabián, P.C. Laguna Laura, J.J. Rivera Tito,
J.C. Gutiérrez Cáceres, “Fast Face Detection in Violent Video Scenes”, Electronic
Notes in Theoretical Computer Science, Vol. 329, pp. 5-26, 2016.
[9] P. Irgens, C. Bader, T. Lé, D. Saxena, C. Ababei, “An efficient and cost effective
FPGA based implementation of the Viola-Jones face detection algorithm”,
HardwareX, Vol. 1, pp. 68-75, 2017.

Dept. Of ECE, MITK Page 19