Software Engineering project Report

on:
Face Detection using Open CV Machine
Learning

By

Sediqa Sultani Under Guidance Of:

Diba Sayed
Mohibullah Amiri Eng. Majid Niazi

Hewad Institue of Higher Education

1
Table of contents

Abstract ............................................................................................................................................. 3
1. Introduction ............................................................................................................................... 3
FACE RECOGNIZATION: .......................................................................................................... 3
DIFFERENT APPROACHES OF FACE RECOGNITION: ................................................... 3
1.2 FACE DETECTION: .............................................................................................................. 4
2. Problem Statement ........................................................................................................................ 4
3. Proposed System ........................................................................................................................... 4
4. Project Design ............................................................................................................................... 5
5. Tools used ..................................................................................................................................... 7
Deep learning ................................................................................................................................ 8
What Is Deep Learning? ........................................................................................................... 8
How Deep Learning Works ...................................................................................................... 9
How our algorithm works ............................................................................................................... 10
Module for face detection ............................................................................................................. 11
Conclusion ...................................................................................................................................... 13
Future Enhancement ....................................................................................................................... 13
References ....................................................................................................................................... 14

2
Abstract
dentifying a person with an image has been popularised through the mass media.
However,it is less robust to fingerprint or retina scanning. This report describes the
face detection and recognition mini-project undertaken for the visual perception and
autonomy module at Plymouth university. It reports the technologies available in the
Open-Computer-Vision (OpenCV) library and methodology to implement them
using Python. For face detection, Haar-Cascades were used and for face recognition
Eigenfaces, Fisherfaces and Local binary pattern histograms were used. The
methodology is described including flow charts for each stage of the system. Next,
the results are shown including plots and screen-shots followed by a discussion of
encountered challenges.

1. Introduction
Face recognition is the task of identifying an already detected object as a known or
unknown face. Often the problem of face recognition is confused with the problem
of face detection Face Recognition on the other hand is to decide if the "face" is
someone known, or unknown, using for this purpose a database of faces in order to
validate this input face.

FACE RECOGNIZATION:
DIFFERENT APPROACHES OF FACE RECOGNITION:
There are two predominant approaches to the face recognition problem: Geometric
(Feature based) and photometric (view based). As researcher interest in face
recognition continued, many different algorithms were developed, three of which
have been well studied in face recognition literature.

Recognition algorithms can be divided into two main approaches:

1. Geometric: Is based on geometrical relationship between facial landmarks, or in

other words the spatial configuration of facial features. That means that the main
geometrical features of the face such as the eyes, nose and mouth are first located and

3
then faces are classified on the basis of various geometrical distances and angles
between features.

2. Photometric stereo: Used to recover the shape of an object from a number of mages
taken under different lighting conditions. The shape of the recovered object is defined
by a gradient map.

Popular recognition algorithms include:

1. Principal Component Analysis using Eigenfaces, (PCA)

2. Linear Discriminate Analysis,

3. Elastic Bunch Graph Matching using the Fisher face algorithm

1.2 FACE DETECTION:

2. Problem Statement
The main aim or objective of this paper is to provide or develop a system that will
use the camera of the computer or the system that would detect and recognize the
person’s face or the face of the individual using the tool in OpenCV called as the
Open Face and python programming language in deep learning domain.
3. Proposed System
Face detection involves separating image windows into two classes; one containing
faces (tanning the background (clutter). It is difficult because although
commonalities exist between faces, they can vary considerably in terms of age, skin
color and facial expression. The problem is further complicated by differing lighting
conditions, image qualities and geometries, as well as the possibility of partial
occlusion and disguise. An ideal face detector would therefore be able to detect the
presence of any face under any set of lighting conditions, upon any background.
The face detection task can be broken down into two steps. The first step is a

4
classification task that takes some arbitrary image as input and outputs a binary
value of yes or no, indicating whether there are any faces present in the image. The
second step is the face localization task that aims to take an image as input and
output the location of any face or faces within that image as some bounding box
with (x, y, width, height).
The face detection system can be divided into the following steps: -
1. Pre-Processing: To reduce the variability in the faces, the images are processed
before they are fed into the network. All positive examples that is the face images
are obtained by cropping images with frontal faces to include only the front view.
All the cropped images are then corrected for lighting through standard algorithms.
2. Classification: Neural networks are implemented to classify the images as faces
or nonfaces by training on these examples. We use both our implementation of the
neural network and the Matlab neural network toolbox for this task. Different
network configurations are experimented with to optimize the results.
3. Localization: The trained neural network is then used to search for faces in an
image and if present localize them in a bounding box.
4. Project Design
The model we used is built with Keras using Convolutional Neural Networks
(CNN). A convolutional neural network is a special type of deep neural network
which performs extremely well for image classification purposes. A CNN basically
consists of an input layer, an output layer and a hidden layer which can have multiple
numbers of layers. A convolution operation is performed on these layers using a
filter that performs 2D matrix multiplication on the layer and filter.
The CNN model architecture consists of the following layers:

• Convolutional layer; 32 nodes, kernel size 3 • Convolutional layer; 32 nodes,

kernel size 3
5
• Convolutional layer; 64 nodes, kernel size 3
• Fully connected layer; 128 nodes
The final layer is also a fully connected layer with 2 nodes. In all the layers, a Relu
activation function is used except the output layer in which we used Softmax.

Similar to how a child learns to recognize objects, we need to show an algorithm

millions of pictures before it is be able to generalize the input and make predictions
for images it has never seen before. Computers ‘see’ in a different way than we do.
Their world consists of only numbers. Every image can be represented as 2-
dimensional arrays of numbers, known as pixels. But the fact that they perceive
images in a different way, doesn’t mean we can’t train them to recognize patterns,
like we do. We just have to think of what an image is in a different way

6
To teach an algorithm how to recognise objects in images, we use a specific type of
Artificial Neural Network: a Convolutional Neural Network (CNN). Their name
stems from one of the most important operations in the network: convolution.
Convolutional Neural Networks are inspired by the brain. Research in the 1950s and
1960s by D.H Hubel and T.N Wiesel on the brain of mammals suggested a new
model for how mammals perceive the world visually. They showed that cat and
monkey visual cortexes include neurons that exclusively respond to neurons in their
direct environment.

5. Tools used
In this Python project, we will be using OpenCV for gathering the images from
webcam and feed them into a Deep Learning model which will classify whether the
person’s eyes are ‘Open’ or ‘Closed’. the requirement for this Python project is a
webcam through which we will capture images. You need to have Python (3.6

7
version recommended) installed on your system, then using pip, you can install the
necessary packages.

1. OpenCV – pip install opencv-python (face and eye detection).

2. TensorFlow – pip install tensorflow (keras uses TensorFlow as backend).

3. Keras – pip install keras (to build our classification model).

The approach we will be using for this Python project is as follows:

Step 1 – Take image as input from a camera.

Step 2 – Detect the face in the image and create a Region of Interest (ROI).

Step 3 – Detect the eyes from ROI and feed it to the classifier.

Deep learning
For the purpose of working with algorithms of AI we need brief knowledge in deep learning which
will make easy the things for making projects and solving the real case problems which will helpful
for the humanity and will be a way for new technologies to come out.

What Is Deep Learning?

Deep learning is an artificial intelligence

(AI) function that imitates the workings of
the human brain in processing data and
creating patterns for use in decision
making. Deep learning is a subset of
machine learning in artificial intelligence that has networks capable of learning

8
unsupervised from data that is unstructured or unlabeled. Also known as deep
neural learning or deep neural network.

• Deep learning is an AI function that mimics the workings of the human brain
in processing data for use in detecting objects, recognizing speech, translating
languages, and making decisions.
• Deep learning AI is able to learn without human supervision, drawing from
data that is both unstructured and unlabeled.
• Deep learning, a form of machine learning, can be used to help detect fraud or
money laundering, among other functions.

How Deep Learning Works

Deep learning has evolved hand-in-hand with the digital era, which has brought
about an explosion of data in all forms and from every region of the world. This data,
known simply as big data, is drawn from sources like social media, internet search
engines, e-commerce platforms, and online cinemas, among others. This enormous
amount of data is readily accessible and can be shared through fintech applications
like cloud computing.

However, the data, which normally is unstructured, is so vast that it could take
decades for humans to comprehend it and extract relevant information. Companies
realize the incredible potential that can result from unraveling this wealth of
information and are increasingly adapting to AI systems for automated support.

Camera
In this project we need a camera to take pictures when we run our code. The camera
will capture multiple photos and will detect the eyes is it open or close.
Operating system

9
It is an open-source project. We can run the code of this project in any operating
system. But we used windows 10, 64-bit operating system and x64-based processer.

How our algorithm works

Step 1 – Take Image as Input from a Camera
With a webcam, we will take images as input. So to access the webcam, we made an
infinite loop that will capture each frame. We use the method provided by OpenCV,
cv2.VideoCapture(0) to access the camera and set the capture object (cap). cap.read()
will read each frame and we store the image in a frame variable.
Step 2 – Detect Face in the Image and Create a Region of Interest (ROI)
To detect the face in the image, we need to first convert the image into grayscale as
the OpenCV algorithm for object detection takes gray images in the input. We don’t
need color information to detect the objects. We will be using haar cascade
classifier to detect faces. This line is used to set our classifier face =
cv2.CascadeClassifier(‘ path to our haar cascade xml file’). Then we perform the
detection using faces = face.detectMultiScale(gray). It returns an array of detections
with x,y coordinates, and height, the width of the boundary box of the object. Now
we can iterate over the faces and draw boundary boxes for each face.

Step 3 – Detect the eyes from ROI and feed it to the classifier
The same procedure to detect faces is used to detect eyes. First, we set the cascade
classifier for eyes in leye and reye respectively then detect the eyes using left_eye =
leye.detectMultiScale(gray). Now we need to extract only the eyes data from the full

10
image. This can be achieved by extracting the boundary box of the eye and then we
can pull out the eye image from the frame with this code.

l_eye only contains the image data of the eye. This will be fed into our CNN classifier
which will predict if eyes are open or closed. Similarly, we will be extracting the
right eye into r_eye.

Module for face detection

This is the piece of code for face recognition or we can say region of interest in our
head when camera is going to take video or photos. Here we use the cv2 library
which is used for finding the region of interest and it will read all the input images
and will find the region of interest (face) in photo.

For video capturing of the region of interest import cv2

face_cascade =
cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
# Read the input image #img
= cv2.imread('test.png') cap =
cv2.VideoCapture('test.mp4')
while cap.isOpened():
_, img = cap.read()

gray = cv2.cvtColor(img,
cv2.COLOR_BGR2GRAY) faces =
face_cascade.detectMultiScale(gray, 1.1, 4)

11
for (x, y, w, h) in faces: cv2.rectangle(img,
(x, y), (x+w, y+h), (255, 0, 0), 3)
cv2.imshow('img', img) if cv2.waitKey(1) &
0xFF == ord('q'): break cap.release()
photo capturing of the region of interest

import cv2
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

# Read the input image img =

cv2.imread('test.png') gray =
cv2.cvtColor(img,
cv2.COLOR_BGR2GRAY) faces =
face_cascade.detectMultiScale(gray, 1.1, 4)

for (x, y, w, h) in faces:

cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 3)

# Display the output

cv2.imshow('img', img)
cv2.waitKey()

OUTPUT OF THE CODE

12
Conclusion

Face recognition is a method of identifying or verifying the identity of an individual

using their face. Face recognition systems can be used to identify people in photos,
video, or in real-time. Law enforcement may also use mobile devices to identify
people during police stops.

But face recognition data can be prone to error, which can implicate people for
crimes they haven’t committed. Facial recognition software is particularly bad at
recognizing African Americans and other ethnic minorities, women, and young
people, often misidentifying or failing to identify them, disparately impacting
certain groups.

Future Enhancement
Future Enhancement Assistance has an immensely boundless scope in future. It can
be amended as and when requirement emerges, as it is versatile in terms of the

13
extension. There are some facets which can be further modified such as recognized
distance can be extended, Graphics processing unit (GPU) can be used for a large
amount of the database and quick processing, data storage can be made server-based and
can be integrated with multiple cameras at the same time.

References

https://www.irjet.net/archives/V7/i10/IRJET-V7I10219.pdf

https://www.mygreatlearning.com/blog/face-recognition/

https://www.researchgate.net/publication/267426877_Facial_Recognition_using_OpenCV

https://www.pyimagesearch.com/2018/06/18/face-recognition-with-opencv-python-and-deep-
learning/

https://techvidvan.com/tutorials/face-recognition-project-python-opencv/

14