MACHINE LEARNING INSPIRED REAL-TIME

FACE DETECTION FOR DEPARTMENT SURVEILLANCE

Salman Shahid
Ahmad Waheed
Syed Muhammad Habib Arif

Supervised By

Dr. Adeel Asghar

Submitted for the partial fulfillment of BS Computer Science degree to the

Faculty of Engineering & CS
DEPARTMENT OF COMPUTER SCIENCE

Army Public College of Management Sciences October, 2022

i
 It is certified that project report titled „Title of your project‟ submitted by
Name of Student1, Name of Student2 and Name of Student3 for the
partial fulfillment of the requirement of “Bachelor’s/Master’s Degree in
Computer Science” is approved.

COMMITEE

Name of Dean
Acting Dean Engineering & CS: Signature:

Noman Malik
HoD Computer Science: Signature:

Name of Coordinator
Project Coordinator: Signature:

Supervisor Name
Supervisor: Signature:

ii
 DECLARATION

We hereby declare that our dissertation is entirely our work and

genuine / original. We understand that in case of discovery of any
PLAGIARISM at any stage, our group will be assigned an F
(FAIL) grade and it may result in withdrawal of our Bachelor’s
degree.

Group members:

Name Signature

Name of Student 1

Name of Student 2

Name of Student 3

iii
 Chapter 1
INTRODUCTION

1
Face detection is one of the finest inventions of the current era. As we all are aware that we
can improve our daily lives with continuous advancements in technology and we can say that
these technological advancements surely have made our lives much easier. We might have
seen in our life that some people get caught just by the use of surveillance systems and the
sole purpose of our research is to identify every person just by their faces so that any accident
or theft attempt could be avoided. There are some existing systems already in today’s world
that have achieved so much in recent times. Our system is intended to help an organization in
making the environment more disciplined through the use of technology. Our system is going
to make a difference by providing more accuracy to prohibited activities i.e. smoking
detection, alert, and marking the attendance of staff and students.

We have seen in recent years how the world has changed in terms of technology and face
detection was a massive change. Existing systems are present in the market and sometimes
an error can make one’s life miserable, we are considering the building is an accurate system
for the identification of a person and detecting a smoking person on the premises of the
organization.

Figure a. Figure b.

2
1.1. Project Domain

Our system lies in the domain of Machine Learning which is a type of Artificial Intelligence.
The proposed system will be based on supervised learning, which is a type of machine
learning. The proposed system will also utilize the concepts of digital image processing for
the acquisition of images of a person’s face. The Problem that our system will solve is
surveillance. Our system will keep the track of unknown and known people using the
database. The system will also be able to mark the attendance and maintain the record so that
it can be fetched later when needed. Furthermore, our proposed system will also provide an
alert to the administration if some prohibited activity is recorded i.e. smoking, this will
ultimately be very helpful as smoking can be very dangerous sometimes.

1.2. Problem Identification

The problem we are here to tackle is to maintain the check and balances for the people that
are associated with the institution. It is very difficult to maintain the attendance of every
faculty member manually. This practice can be very unpleasant and very exhausting for the
person managing the records. Moreover, it is certain to have the discipline of the campus
attained, and just to keep an eye on everyone so that he/she doesn’t smoke we had to appoint
a person, especially for their activities. We are well aware of the fact that we are moving into
an automated world where the manpower is quite reduced, so just to ensure that we needed to
come up with a solution.
The existing systems are present in the market but the problem with those systems is that no
system in our country has implemented face detection in such a way that these problems can
be solved. There is still there is need for a well-developed system that can perform
exceptionally well in different environments.

3
1.2.1 Proposed Solution

Being in the field of technology the problems mentioned above encouraged us to put forward
a solution to these problems. We came up with this machine learning-inspired real-time face
detection idea which is going to be solely used for department surveillance, the system’s
features will help us in maintaining the attendance record and discipline in the institute. Our
proposed system is going to be application-oriented in terms of surveillance. The system will
provide real-time detection 24/7.

1.2.2 Objectives

By the time we finish our build, we are hoping to have a clear picture and knowledge of the
face detection systems and algorithms and we aim:
 To design a system that is very easy to use and administration-friendly.
 To develop a system that can keep a track of known and unknown personnel.
 To improve the accuracy as opposed to the previous systems.
 To increase the efficiency of the surveillance system.
 To build a system that validates the person whether the person belongs to the
institution or not.

4
Flow Chart

1.2.3 Scope of the Project

The scope of our system is the identification of people. We decided to go with supervised
learning. Our model will be already trained and testing will be done in real-time. Our system
will be an application-based tool that will be a suitable tool for the administration of various
institutions and can also be used in many other private institutions where nobody is supposed
to break in. Our system will also allow us to detect any smoking person on the premises, It
will also maintain a record for everyone e.g. attendance and it will automatically insert the
data in our learning management system.

5
1.3. Effectiveness / Usefulness of the System

The systems that are already present haven’t been so accurate and that makes them less
productive. This field is so delicate that these systems are prone to have accuracy in them, we
will try to develop a system that will clearly show its working and effectiveness by its
accuracy and precision. Our system will also be able to maintain the record and that makes it
different from other tools and techniques. Application of the system is going to be quite easy
and will certainly be much useful.

1.4. Resource Requirement

Our system will require some resources to be built and those requirements are listed below:

1.4.1 Hardware Requirement

As our system is application-oriented so it is not going to need some extra hardware. We just
need a computer with normal specs and a CCTV camera setup.

1.4.2 Software Requirement

For our system to be built we would need python because the script is going to be written in
python.

6
MATLAB might also be used in the development of our system.

1.4.3 Data Requirement

Datasets are required for our project. We will work our way with the university’s
existing database so that we can maintain the record and identify the person whether
he/she is from the campus or not.

7
 Chapter 2

BACKGROUND AND
EXISTING SYSTEMS

8
 Machine Learning Inspired Real Time Face Detection is a model which lets
us to detect faces of authorized and unauthorized personnel entering in the
premises of the campus. The system will be able to detect faces of any
unauthorized personnel and generate alert for the administration to acknowledge
there is been entry of an outsider.

2.1. Literature Review

Face Detection and Recognition from Distance Based on Deep Learning, Yan et al.
developed a system that provides a better and more accurate face detection system that
overcomes the problem of distance based facial recognition. Deep learning algorithm CNN
were used to recognize faces and MLP structures to implement visual object classification
and localization. The proposed system provides an accuracy of 90.18% for facial recognition
at a distance. The limitation was that the training dataset must be reduced as much as
possible in order to improve system’s efficiency and availability. [1]

The research paper "Real-Time Smart Attendance System using Face Recognition
Techniques" by Shreyak Sawhney et al. The authors propose a realtime smart attendance
system to automate the attendance process and improve accuracy. The proposed system uses
OpenCV and LBPH to identify individuals and record attendance. The authors describe the
implementation of the attendance system, which involves capturing an image of each
individual's face, processing it, and recording attendance in a database. The LBPH algorithm
achieves high accuracy up to 95% and computational efficiency, outperforming traditional
attendance systems. The proposed system relies on individuals being present and visible in
front of the camera for accurate attendance recording, but may not be suitable for large-scale
deployments. [2]

The research paper "A real time face emotion classification and recognition using deep
learning model" by Shaik Asif Hussain and Ahlam Salim Abdallah Al Balushi.Deep
learning-based methods have improved accuracy and efficiency in facial emotion
recognition. Deep learningbased approach to facial emotion recognition uses a CNN to
classify expressions. The authors explain the process of training and testing a deep learning
model for facial emotion recognition. Deep learning model outperforms state-of-the-art
methods in facial emotion recognition with an average accuracy of 90%. The paper does not
discuss ethical and social implications of facial emotion recognition technology, privacy
concerns, and potential biases. [3]

“Artificial Intelligence Based Real Time Attendance System Using Face Recognition” by
Khandaker Mohammad Mohi Uddin; Aditta Chakraborty; Md. Abdul Hadi; Md Ashraf
Uddin; Samrat Kumar Dey. This proposes method for face detection by using the DLib and
ResNet-34. Using two cameras arrival and departure times of an individual are calculated
sequentially. 96.03% accuracy was observed by camera-1 to recognize the face and at
daytime when the light is off camera-2 shows the best accuracy of 96.62%. [4]

Face Detection and Segmentation Based on Improved Mask R-CNN, Kaihan Lin et al.
presented a system that overcame the problem of segmenting each face from background
image simultaneously. The system was based upon an improvised version of R-CNN which
they named G-Mask. Furthermore, ResNet-101, RPN, RoIAlign, Generalized Intersection
over Union (GIoU) were utilized for the proposed system. The proposed system attained
auspicious results. The accuracy on the training set was more than 99% whereas on the
verification set it was 90.18% [5]
 Table 2.1: Summary of Reviewed Literature

Year Authors Contribution Techniques Limitations/Remarks

Yan et al. Detects and recognizes faces  CNN requires a large amount of training data to
in low-resolution images  face alignment train the deep CNN
2016
captured from a distance. method

Shaik Asif Real-time face emotion  pre-trained Requires powerful computing system
Hussain and recognition system using a VGG16
2020
Ahlam Salim deep learning model  FER-2013
Abdallah Al
Balushi

Shreyak Real-time smart attendance  OpenCV Requires a well-calibrated camera to capture

2019 Sawhney, system using face recognition  support vector high-quality face images
Rishabh Garg, techniques. machine
Divanshu (SVM)
Sharma, and  Local Binary
Nupur Goyal Patterns (LBP)

Khandaker Artificial intelligence-based  ResNet-50 Accuracy can be compromised due to makeup

2021 Mohammad real-time attendance system  OpenCV and masks.
Mohi Uddin and using face recognition
Aditta
Chakraborty

Kaihan Lin, Jian Improved Mask R-CNN  ResNet-50 Requires significant computational resources.
2019  FPN and an
Chen, Yu Fu, (Region-based Convolutional
Zhimin Zhang, Neural Networks) method for anchor-free

and Weiming face detection and RPN

Hu segmentation.
2.2. Related Systems/Applications

There are no such applications or systems related to our proposed solution out in the market
in terms of department surveillance. We intend to make an exceptional system so that our
surveillance practices can be improvised and the shortcomings of the system can be taken
care of.

2.3. Identified Problem from Existing Work

One of the key concerns is that the accuracy is subjected. Some of the systems mentioned
above require sophisticated computational systems which in today’s world cost a fortune, our
system can be set up with a generic computer i.e. a laptop, pc or raspberry pi with the help of
a webcam or a CCTV camera. We aim to overcome the problems of the existing system and
improve the accuracy by the utilization of modern techniques and modern machine learning
paradigms.

2.1. Selected Boundary for Proposed Solution

The proposed solution is an effort to maximize the accuracy of the face detection systems.
This project will be able to provide access in the premises to the authorized personnel only
whereas un authenticated person will be identified and will not be allowed unless he chooses
to give his/her information. If a visitor wishes to enter the campus he has to enter his CNIC
and face impression at the entrance.
Other functionalities that were not included in the proposed system are smoking detection
and attendance maintenance as we are bound to provide with the real time face detection
system only.
 Chapter 3

SYSTEM REQUIREMENTS AND

SPECIFICATIONS
3.1 System Specification
Here are some system specifications for our proposed solution:

Hardware:

 A computer with standard specs running a supported operating system

 A CCTV camera setup capable of capturing high-quality images of individuals'
faces

Software:

 Python 3.x installed with necessary packages such as OpenCV, NumPy, and
TensorFlow for face detection and recognition
 Potentially MATLAB for additional data analysis and processing
 A database management system (e.g. MySQL or PostgreSQL) for storing and
managing attendance and other records.
 A web-based interface for administrators to view real-time surveillance footage,
manage attendance records, and view reports.

Data:

 A dataset of facial images for training and testing face detection and recognition
algorithms
 A database of known individuals (e.g. students, faculty, staff) and their
corresponding biographical information (e.g. name, photo ID,
department/program affiliation)
 A database of prohibited activities (e.g. smoking) and associated alerts and
notifications for security personnel

Overall, the system will be designed to provide real-time surveillance and detection of
individuals within a designated area, with a focus on accurately identifying and tracking
known individuals while also detecting and alerting security personnel to any
unauthorized or prohibited activities. The system will be user-friendly and easy to
manage for administrators, while also providing detailed reports and analytics for further
analysis and optimization.
3.2 System Modules

A module refers to a self-contained, independent unit of code that performs a specific

function or task. In the context of system design, modules can be thought of as building
blocks or components that make up the larger system. Designing a system into modules
has several benefits. It helps to break down the system into smaller, manageable
components that can be developed and tested independently. This makes it easier to
identify and fix errors, as well as modify and maintain the system over time. Furthermore,
a modular design can enhance the scalability, flexibility, and reusability of the system.

3.2.1 Programming Module

This module is responsible for the software aspect of the system. It involves writing the
code that will be used to implement the system's functionalities. This module may also
include libraries, frameworks, or other software tools that are required to develop the
system.

3.2.2 Hardware Module

This module deals with the physical components of the system. It includes the computer,
webcam, and other peripherals that are necessary for the system to function. This module
may also include any additional hardware required to enhance the performance of the
system, such as graphics cards or processors.

3.2.3 Database Module

This module is responsible for the storage and retrieval of data used by the system. It
involves designing and implementing a database that can efficiently store and manage the
data required by the system. This module may also include software tools that are used to
interact with the database, such as SQL query builders or database management systems.
3.3. Functional Requirements
Below are some of the functional requirements for the proposed solution:

3.3.1 Face Detection

 The system must be capable of detecting human faces accurately and efficiently in
real time.
 It must be to handle the ever changing lighting conditions, angles, and facial
expressions.
 The face detection algorithm must be optimized to make certain high detection
rates and low false positives.
3.3.2 User Module

3.3.2.1 Register
User can be registered with details such as Name, employee id and contact no

3.3.2.2 Login
User can be logged in if he has registered himself.

3.3.2.3 View live surveillance

User or Admin can open the interface where he can monitor the activities.

3.3.2.4 Visitor’s Entry

This module is given to the user at the gate of the campus so that he can make the
entries of the outsiders if they want to make an entry in the university with basic
details such as CNIC, name and phone number.
3.4. Non-Functional Requirements

3.4.1 Performance

 The system should be highly responsive and provide real-time face detection and
recognition without significant delays. It should be capable of handling a large
number of simultaneous face detection requests efficiently.

3.4.2 Accuracy

 The face detection and recognition algorithms employed by the system should
have a high accuracy rate. The system should minimize false positives and false
negatives to ensure reliable identification and authentication of individuals.
3.4.3 Scalability
 The system should be scalable to accommodate future growth and increasing
demands. It should be able to handle a growing number of users, cameras, and
facial data in a seamless manner without compromising performance.

3.4.4 Reliability

 The system should be reliable and operate consistently without unexpected

failures or downtime. It should have robust error handling mechanisms and be
able to recover gracefully from errors or system failures.
3.4.4 Security
 The system should implement strong security measures to protect sensitive data
and prevent unauthorized access. This includes encryption of data transmission,
secure storage of facial data, and strict access controls to ensure the privacy and
confidentiality of individuals' information.

3.4.6 Usability

 The system should have a user-friendly interface that is easy to navigate and
understand. It should provide clear instructions and feedback to users during face
detection and recognition processes.
3.4.7 Compatibility

 The system should be compatible with different types of cameras, operating

systems, and hardware configurations commonly found in the university
environment. It should support a wide range of devices to ensure flexibility and
ease of integration.

3.4.8 Maintainability

 The system should be designed and implemented in a modular and maintainable

manner. It should be easy to update, modify, and extend to accommodate future
requirements or technological advancements.

3.4.9 Compliance

 The system should comply with relevant laws, regulations, and standards related
to data protection, privacy, and surveillance in the educational sector. It should
adhere to ethical considerations and ensure transparency in its operation.

3.4.10 Performance Monitoring and Logging

 The system should provide mechanisms for monitoring its performance and
logging relevant information such as system activities, user actions, and system
errors. These logs can be used for troubleshooting, system optimization, and
auditing purposes.
 Chapter 4

SYSTEM MODELING AND DESIGN

4.2 Use Case Diagrams

4.2.1 Face Detection System

Figure 4.2.1

4.2.2 Registration Use Case

 Figure 4.2.2

4.2.3 Login Use Case

 Figure 4.2.3

4.2.4 Visitor’s Entry Use Case

 Figure 4.2.4

4.3 Activity diagram

 Figure 4.3

4.4 Data Flow Diagram

4.4.1 Data Flow Diagram level 0

Figure 4.4.1

4.4.2 Data Flow Diagram level 1

Figure 4.4.2

4.5 Sequence Diagram

 Figure 4.5

4.6 Class Diagram

 Figure 4.6

4.7 Deployment Diagram

Figure 4.7