REAL TIME COVID-19 HUMAN MASK ALERT

SYSTEM USING DEEP LEARNING

A PROJECT REPORT

Submitted by
MAHALAKSHMI T (171001049)
DEVISREE S (171001017)
DHIVYA M (171001117)

in partial fulfilment for the award of the degree of

BACHELOR OF TECHNOLOGY IN
INFORMATION TECHNOLOGY

RAJALAKSHMI ENGINEERING COLLEGE,

THANDALAM

ANNA UNIVERSITY, CHENNAI

MARCH 2021

i
 ANNA UNIVERSITY, CHENNAI
BONAFIDE CERTIFICATE

Certified that this project report titled “REAL TIME COVID-19 HUMAN

MASK ALERT SYSTEM USING DEEP LEARNING” is the bonafide work of

MAHALAKSHMI T (171001049), DEVISREE S (171001017) , DHIVYA M(

171001117) who carried out the work under my supervision. Certified further that to the

best of my knowledge the work reported herein does not form part of any other thesis or

dissertation on the basis of which a degree or award was conferred on an earlier occasion

on this or any other candidate.

SIGNATURE SIGNATURE

Dr. L.PRIYA Mrs. M.SINDHUJA

HEAD OF THE DEPARTMENT SUPERVISOR
Professor Assistant Professor [SG]
Information Technology, Information Technology,
Rajalakshmi Engineering College, Rajalakshmi Engineering
College, Thandalam, College, Thandalam,
Chennai – 602 105. Chennai – 602 105.
This project report is submitted for viva voce examination to be held on …………….at
Rajalakshmi Engineering College, Thandalam.

EXTERNAL EXAMINER INTERNAL EXAMINER

ii
 ABSTRACT

The end of 2019 witnessed the outbreak of Coronavirus Disease 2019 (COVID-19),
which has continued to be the cause of plight for millions of lives and businesses
even in 2020. As the world recovers from the pandemic and plans to return to a state
of normalcy, there is a wave of anxiety among all individuals, especially those who
intend to resume in person activity.
Businesses are constantly overhauling their existing infrastructure and processes to
be more efficient, safe, and usable for employees, customers, and the community.
With the ongoing pandemic, it’s even more important to have advanced analytics
apps and services in place to mitigate risk. In a world battling against the Novel
Coronavirus Disease (COVID-19) pandemic, technology has been a lifesaver. With
the aid of technology, ‘work from home’ has substituted our normal work routines
and has become a part of our daily lives. However, for some sectors, it is impossible
to adapt to this new norm. As the pandemic slowly settles and such sectors become
eager to resume in-person work, individuals are still sceptical of getting back to the
office. 65% of employees are now anxious about returning to the office.
For public safety and health, authorities are recommending the use of face masks and
coverings to control the spread of COVID-19. Face mask detection has become a
crucial task to help global society. However, it is not feasible to manually track the
implementation of this policy. Technology holds the key here.
Computer Vision provides a better solution to this. Using a combination of image
classification, object detection, object tracking, and video analysis, The proposed
method developed a robust system that can detect the presence and absence of face
masks in images as well as videos.
Keywords: Machine Learning, Deep Learning, MobileNetV2, Keras.

iii
 ACKNOWLEDGEMENT

We thank the Almighty God for the successful completion for the project.
Sincere thanks to our Chairman Mr. S. Meganathan for his sincere endeavor
in educating us in his premier institution.

`
We heartly thank our chairperson Dr. (Mrs.) Thangam Meganathan for

her motivation and inspiration that paved for the completion of our project.

We also express our gratitude to our principal Dr. S. N. Murugesan who helped
us in providing the required facilities in completing the project.

We would like to thank our Head of Department Dr. (Mrs.) L. Priya for
her guidance and encouragement for completion of project.

We would like to thank Mrs. M. Sindhuja our supervisor for constantly guiding
us and motivating us throughout the course of the project. We express our
gratitude to our parents and friends for extending their full support to us.

We thank our Project Coordinator Mrs. M. Sindhuja for her invaluable guidance,
ideas, advice and encouragement for the successful completion of this project.

iv
 TABLE OF CONTENTS

TITLE PAGENO

ABSTRACT iii
ACKNOWLEDGEMENT iv
LIST OF FIGURES vii
LIST OF SYMBOLS viii
LIST OF ABBREVIATIONS ix

1. INTRODUCTION
1.1 SYSTEM OVERVIEW 1
1.2 SCOPE OF THE PROJECT 5

2. LITERATURE SURVEY 6

3. SYSTEM ANALYSIS
3.1 EXISTING SYSTEM 9
3.2 PROPOSED SYSTEM 9
3.2.1 ARCHITECTURE DIAGRAM 10
3.3 REQUIREMENTS SPECIFICATION 11
3.3.1 HARDWARE SPECIFICATION 11
3.3.2 SOFTWARE REQUIREMENTS 11
3.4 SOFTWARE SPECIFICATION 11
3.4.1 PYTHON WITH ANACONDA 11

v
4. SYSTEM DESIGN
4.1 SYSTEM DESIGN 12
4.1.1 UML DIAGRAM 12
4.1.2 LEVEL 0 DFD 13
4.1.3 LEVEL 1 DFD 14
4.1.4 LEVEL 2 DFD 15

5. SYSTEM IMPLEMENTATION
5.1 BUILDING A MASK DETECTOR MODEL 17
5.2 MASK DETECTION THROUGH WEB CAM 19
5.3 BUILDING AN ALERT SYSTEM 20

6. IMPLEMENTATION AND RESULT

6.1 MASK DETECTOR MODEL 21
6.2 INCORPORATING WEB CAM TO OUR MODEL 22
6.3 FACE MASK ALERT SYSTEM 24

7. CONCLUSION AND FUTURE ENHANCEMENTS

7.1 CONCLUSION 26
7.2 FUTURE ENHANCEMENTS 27

APPENDIX 1 28
APPENDIX 2 34

REFERENCES 35

vii
 LIST OF FIGURES

FIGURE NUMBER FIGURE NAME PAGE NUMBER

3.1 ARCHITECTURE DIAGRAM 10

4.1 UML DIAGRAM 12
4.2 LEVEL 0 DFD 13
4.3 LEVEL 1 DFD 14
4.4 LEVEL 2 DFD 15
6.1 MODEL EVALUATION REPORT 20
6.2 ACCURACY AND LOSS GRAPH 21
6.3 WITH MASK 22
6.4 WITH OUT MASK 23
6.5 WITH AND WITHOUT MASK 23
6.6 SMS 24
6.7 MAIL 24

viii
SYMBOLS AND NOTATIONS

An association between an actor and use

case indicate that actor and the use case
communicate with each other. An actor could
be associated to one or several use cases.

Actor
Standard UML icon for actor is “stick
man” icon the name of the actor above or
below the icon

Use case
Every use case must have a name. Use case
is shown as ellipse containing the name of
the use case.

viii
 LIST OF ABBREVIATIONS

CNN CONVOLUTIONAL NEURAL NETWORK

ML MACHINE LEARNING
DL DEEP LEARNING
WHO WORLD HEALTH ORGANIZATION

ix
 CHAPTER 1
INTRODUCTION

1.1 SYSTEM OVERVIEW

Coronaviruses are a group of related RNA viruses that cause diseases

in mammals and birds. In humans and birds, they cause respiratory tract
infections that can range from mild to lethal. Mild illnesses in humans include
some cases of the common cold (which is also caused by other viruses,
predominantly rhinoviruses), while more lethal varieties can
cause SARS, MERS, and COVID-19. In cows and pigs they cause diarrhea,
while in mice they cause hepatitis and encephalomyelitis.
Coronaviruses constitute the subfamily Orthocoronavirinae, in the
family Coronaviridae, order Nidovirales, and realm Riboviria. They
are enveloped viruses with a positive-sense single-stranded RNA genome and
a nucleocapsid of helical symmetry. The genome size of coronaviruses ranges
from approximately 26 to 32 kilobases, one of the largest among RNA viruses.
They have characteristic club-shaped spikes that project from their surface,
which in electron micrographs create an image reminiscent of the solar corona,
from which their name derives.

Coronaviruses are large, roughly spherical particles with unique surface

projections. Their size is highly variable with average diameters of 80 to 120 nm.
Extreme sizes are known from 50 to 200 nm in diameter. The total molecular
weight is on average 40,000 kDa. They are enclosed in an envelope embedded
with a number of protein molecules. The lipid bilayer envelope, membrane
proteins, and nucleocapsid protect the virus when it is outside the host cell.

Machine learning (ML) is the study of computer algorithms that improve

automatically through experience and by the use of data. It is seen as a part
of artificial intelligence. Machine learning algorithms build a model based on
1
sample data, known as "training data", in order to make predictions or decisions
without being explicitly programmed to do so. Machine learning algorithms are
used in a wide variety of applications, such as email filtering and computer
vision, where it is difficult or unfeasible to develop conventional algorithms to
perform the needed tasks.

A subset of machine learning is closely related to computational statistics, which

focuses on making predictions using computers; but not all machine learning is
statistical learning. The study of mathematical optimization delivers methods,
theory and application domains to the field of machine learning. Data mining is
a related field of study, focusing on exploratory data
analysis through unsupervised learning. In its application across business
problems, machine learning is also referred to as predictive analytics.

Machine learning involves computers discovering how they can perform tasks
without being explicitly programmed to do so. It involves computers learning
from data provided so that they carry out certain tasks. For simple tasks assigned
to computers, it is possible to program algorithms telling the machine how to
execute all steps required to solve the problem at hand; on the computer's part, no
learning is needed. For more advanced tasks, it can be challenging for a human
to manually create the needed algorithms. In practice, it can turn out to be more
effective to help the machine develop its own algorithm, rather than having
human programmers specify every needed step.

Today,they are pleased to announce the availability of MobileNetV2 to power the

next generation of mobile vision applications. MobileNetV2 is a significant
improvement over MobileNetV1 and pushes the state of the art for mobile visual
recognition including classification, object detection and semantic segmentation.
MobileNetV2 is released as part of TensorFlow-Slim Image Classification
Library, or you can start exploring MobileNetV2 right away in Colaboratory.
Alternately, you can download the notebook and explore it locally using Jupyter.

2
MobileNetV2 is also available as modules on TF-Hub, and pretrained
checkpoints can be found on github.MobileNetV2 builds upon the ideas from
MobileNetV1, using depth wise separable convolution as efficient building
blocks. However, V2 introduces two new features to the architecture:
1) linear bottlenecks between the layers, and
2) shortcut connections between the bottlenecks1. The basic structure is shown
below.
The intuition is that the bottlenecks encode the model’s intermediate inputs and
outputs while the inner layer encapsulates the model’s ability to transform from
lower-level concepts such as pixels to higher level descriptors such as image
categories. Finally, as with traditional residual connections, shortcuts enable
faster training and better accuracy. You can learn more about the technical details
in our paper, “MobileNet V2: Inverted Residuals and Linear Bottlenecks”.
A face mask detection is a technique to find out whether someone is wearing a
mask or not. It is similar to detect any object from a scene. Many systems have
been introduced for object detection. Deep learning techniques are highly used in
medical applications. Recently, deep learning architectures have shown a
remarkable role in object detection. These architectures can be incorporated in
detecting the mask on a face.
Recently, the growth of COVID-19 can be reduced by detecting the facial mask
in a smart city network. This paper aims at designing a system to find out whether
a person is using a mask or not and informing the corresponding authority in a
smart city network. Firstly, CCTV cameras are used to capture real-time video
footage of different public places in the city. From that video footage, facial
images are extracted and these images are used to identify the mask on the face.
The learning algorithm is MobileNetV2 with Convolutional Neural Network
(CNN) where the convolution layer is replaced by MobileNetV2, which is used
for feature extraction from the images then these features are learned by multiple
hidden layers. Whenever the architecture identifies people without face mask this

3
information is transferred through the city network to the corresponding authority
to take necessary actions. The proposed system appraised promising output on
data collected from different sources. The proposed method also represented a
system that can ensure proper enforcement of the law on people who are not
following basic health guidelines in this pandemic situation.

4
 1.2 SCOPE OF THE PROJECT
To develop a prediction and alert system for face masks using dataset images,
and find an accurate identification of human face mask and alert the security
guard through SMS and email.

5
 CHAPTER 2

LITERATURE SURVEY

Object detection is one of the trending topics in the field of image processing and
computer vision. Ranging from small scale personal applications to large scale
industrial applications, object detection and recognition is employed in a wide
range of industries. Some examples include image retrieval, security and
intelligence, OCR, medical imaging and agricultural monitoring.

In the meantime, many systems have been developed for COVID-19 in smart city
networks. BlueDot and HealthMap services have been introduced in [1] “Smart
City Technologies,” Smart Cities, 2020 by G. Halegoua.

BlueDot method was first used to mark the cluster of unusual pneumonia in
Wuhan which finally detected the disease as a pandemic. It also predicted that the
virus would spread from Wuhan to Bangkok, Taipei, Singapore, Tokyo and Hong
Kong. HealthMap service, based on San Francisco, spotted the patients with a
cough which is the initial sign of COVID-19, using Artificial Intelligence (AI)
and big data.

A study on using facemask to restrict the growth of COVID-19 is introduced in

[2] “Uso de máscara facial para limitar a transmissão da COVID-19,” Epidemiol.
e Serv. saude Rev. do Sist. Unico Saude do Bras., vol. 29, no. 2, by L. P. Garcia.
The study indicated that the masks that are adequately fit, effectively interrupt the
spread of droplets expelled when coughing or sneezing. Masks that are not
perfectly fitted, also capable of retaining airborne particles and viruses.

6
Allam and Jones [3] “On the Coronavirus (COVID-19) Outbreak and the Smart
City Network: Universal Data Sharing Standards Coupled with Artificial
Intelligence (AI) to Benefit Urban Health Monitoring and Management,”
proposed a framework on smart city networks focusing on how data sharing
should be performed during the outbreak of COVID-19. The proposed system
discussed the prospects of Urban Health Data regarding the safety issues of the
economy and national security. In the system, the data is collected from various
points of the city using sensors, trackers, and from laboratories.

A face mask detecting model named Retina Facemask combining with a cross-
class object removal algorithm is proposed by Jiang et al. [4]. The developed
model includes one stage detector consisting feature pyramid network that results
in slightly higher precision and recall than the baseline result. For reducing the
shortage of datasets, they have applied transfer learning, a well-known deep
learning technique. Gupta et al.

“Enabling and Enforcing Social Distancing Measures using Smart City and ITS
Infrastructures: A COVID-19 Use Case,” [5] proposed a model to enforce the
social distance using smart city and Intelligent Transportation System (ITS)
during COVID-19 pandemic. Their model described the deploying sensors in
different places of the city to monitor the real-time movement of objects and
offered a data-sharing platform.

A noticeable contribution of a smart city in controlling the spread of coronavirus

in South Korea is explained by Won Sonn and Lee [6]. A time-space cartographer
speeded up the contact tracking in the city including patient movement, purchase
history, cell phone usages, and cell phone location. Real-time monitoring has
been carried out on CCTV cameras in the hallways of residential buildings. Singh
et al.

7
[7] “Internet of things (IoT) applications to fight against COVID-19 pandemic,”
put their focus on how IoT can fight against COVID-19. The developed system
emphasizes on inter-connected devices or operations to track the patients along
with wary cases. A well-informed group using interconnected devices is formed
to identify the clusters significantly.

A remarkable pandemic control model without lockdown in a smart city has been
outlined by Sonn et al. [8]. The patients have been interviewed and their past
movement has been monitored. They have claimed that some patients tried to
conceal about their past mobility but real-time tracking system found the exact
information. Jaiswal et al.

“Smart Solution for Reducing the COVID-19 Risk using Smart City
Technology,” [9] proposed a way to minimize the risk during COVID-19. Their
proposed model used the position of technology to track infected people. Drones
and Robot technologies have been applied as medical personnel for providing
adequate services to infected people.

The development of smart cities under COVID-19 and controlling the pandemic
in China has been reviewed by Wang et al. [18]. The continuous supply of
essential materials and contactless logistic distribution of systems to society made
the way to reduce the spread of coronavirus. ITS and real-time map reflection
methods have been used to block the movement of vehicles during the pandemic.
In addition, driverless vehicles have been used to monitor the scenarios across the
city.

8
 CHAPTER 3
SYSTEM ANALYSIS

3.1 EXISTING SYSTEM

The existing system faces difficulty in classifying the faces covered by hands as
it resembles the face with mask. So, the existing model results in inaccurate
predictions whenever the model deals with a face covered by hands. Also in the
existing system no alert system is built to notify the authorities.

3.2 PROPOSED SYSTEM

The proposed system aims to predict whether a person wears a mask or not and
if the person doesn’t wear a mask then notification is sent to the security guard
via mail and SMS. The techniques like MobileNetV2 as an image classifier to
classify faces into masked and unmasked images. Then an email is sent by
configuring the methods in smtplib module and using requests module. The sms
is sent by creating a trial account in twilio and the corresponding twilio module
is installed and made use of to send the sms. When the mask detector detects that
the person doesn’t wear a mask, a pop up is displayed and an alarm sound is
played. The sound is played with the help of winsound python module.

9
 3.2.1 ARCHITECTURE DIAGRAM

Figure 3.1 Architecture diagram

The dataset consists of around 4000 images where there are 1900 masked images
and 1900 unmasked images. The input images are pre-processed with the help of
one hot encoding method where the categorical data is converted into numerical
values which in our case is 0 and 1. The pre-processed data is divided into training
and testing test where training dataset undergoes data augmentation where a
single image is modified to obtain multiple images. This is done to enrich our
training dataset. The enriched data is then given to the mobile Net V2 architecture
For training the model and for evaluation of the model accuracy metric is used.

10
 3.3 REQUIREMENT SPECIFICATION

3.3.1 HARDWARE REQUIREMENTS

3.3.1.1 System : Intel CORE i5 8th Gen

3.3.1.2 RAM : 8 Gb

3.3.1.3 Integrated Web Cam

3.3.2SOFTWARE REQUIREMENT

ANACONDA, SPYDER

3.4 SOFTWARE SPECIFICATION

3.4.1 PYTHON USING ANACONDA

Anaconda (Python distribution) Anaconda is a free and open-source distribution

of the Python and R programming languages for scientific computing (data
science, machine learning applications, large-scale data processing, predictive
analytics, etc.), that aims to simplify package management and deployment.
Anaconda is a Python prepackaged distribution of Python which contains a
number of Python modules and packages, including spyder. Spyder is a free and
open source scientific environment written in Python, for Python, and designed
by and for scientists, engineers and data analysts. It features a unique combination
of the advanced editing, analysis, debugging, and profiling functionality of a
comprehensive development tool with the data exploration, interactive execution,
deep inspection, and beautiful visualization capabilities of a scientific package.

11
 CHAPTER 4

SYSTEM DESIGN

4.1 SYSTEM DESIGN

This section focuses on describing the individual components of the system and
how they interact with each other. The system design in explained by using UML
diagrams and data flow diagrams. Unified modelling language is a rich language
to model software solutions, application structures, system behavior and business
processes. A data flow diagram is a graphical representation of the “flow” of data
through an information system, modelling its process aspects.

4.1.1 UML DIAGRAM

Figure 4.1.1 UML Diagram

The Unified Modelling Language (UML) is a general purpose,

12
developmental, modelling language in the field of software engineering. The
actors involved with this system are the system administrators and the external
users. The user feeds the input data to the model. The data is classified and is
validated.

The model captures each frame from the real time video stream and is
preprocessed to the form that can be fed to our mask detector model. The
prediction model which is trained and evaluated classifies the data. Based on the
predictions, an alert mechanism is activated.

4.1.2 LEVEL 0 DFD

Figure 4.2 Level 0 DFD

The level 0 DFD shows an abstract flow of data. It is called as a Context Diagram.
The visitor provides the input data to the system. The data is analyzed by the Face
Mask Alert System. Then an alert message is sent to the security guard if the
visitor doesn’t wear a mask.

13
4.1.3 LEVEL 1 DFD

Figure 4.3 Level 1 DFD

The level 1 DFD provides a more detailed view of the Context Level Diagram.
The level 1 DFD shows the separate flow of data for both the Face detection and
mask detection models. The data that is being fed undergoes separate analysis
process to predict the required output.

14
4.1.4 LEVEL 2 DFD

Figure 4.4 Level 2 DFD

Dataset initially undergoes data preprocessing where several processes like one
hot encoding is done to convert the image input into numpy array for further
processing. The preprocessed data is then split into training set and test set. The
training set undergoes data augmentation to enrich our dataset. The MobileNetV2
model is then constructed and trained using the augmented data. The model is
then evaluated. Video Stream class is used to start the web cam and each frame
from the camera is captured and fed to the face detector which predicts the face
and its location. If the result obtained from mask detector is ‘no mask’, then a
notification message is sent to the security guard via SMS and mail. An alarm
sound is also activated.

15
 CHAPTER 5

SYSTEM IMPLEMENTATION

The proposed method introduces a two-stage architecture for detecting masked

and unmasked faces and localizing them.

5.1 BUILDING MASK DETECTOR MODEL:

Software dependencies for the proposed method:
i) TensorFlow version >= 1.15.2
ii) keras version 2.3.1
iii) imutils version 0.5.3
iv) numpy version 1.18.2
v) OpenCV-python version 4.2.0
vi) matplotlib==3.2.1
vii) scipy==1.4.1

The dataset images for masked and unmasked faces were collected from public
domain like Kaggle and google images. The dataset has 3833 images, divided
into two classes
i) with mask: - 1,915 images
ii) without mask: - 1,918 images.

In this step, the images that are present in the dataset are converted to arrays with
which the deep learning model is created. In train_mask_detector.py, necessary
imports are done. Then grab the list of images in the dataset directory and
initialize lists consisting of data and class respectively. The image list contains
the images in a uniform size of 224 and 224.
16
Each image in the image list is pre-processed using Mobile Net. Then the image
is appended into image list and label into the labels list. Since the labels are still
in the form of string - ‘with mask’ and ‘without mask’, to convert it into a
numerical value which is done with the help of LabelBinarizer from sklearn. Pre-
processing module.
With the help of this module, the labels are converted to categorical variables like
0s and 1s. Now to convert the image list and labels list into numpy array as deep
learning modules learn only with numpy array. This is done with the help of np.
Array () method.

Then with the help of train_test_split () method, the dataset is divided into
training and testing data, with the test size = 0.2 which splits 20% of the dataset
into test set and the remaining into training dataset.

The architecture that is used for training the model is similar to that of
Convolutional Neural Network, but the convolution step that is used for image
processing is neglected and instead a Mobile Net is introduced. So, after the input
image is processed as an array, it is then sent to Mobile Net and then Average
Pooling is done and the input is flattened and is given to a fully connected network
which then determines the output.

The major advantage of using a Mobile Net in the place of convolution is that
Mobile Net provides faster processing speed when compared to CNN. It also uses
lesser parameters when compared to CNN, but they tend to be less accurate. But
for our use case, it will serve the purpose. The learning rate is given 1 e ^ -4 which
is given as low as possible which in turn gives better accuracy.

17
The batch size is given as 32 and the number of epochs as 20.
ImageDataGenerater performs image augmentation to enrich our dataset. Then a
base model is created whose weights are given as imagenet which in turn adjust
the weights to process images. Then the head model is given the base model
output and a neural network is created using CNN architecture. To avoid
overfitting, dropout function is used. Since the output is a binary value, softmax
activation function is used. The model is then compiled using adam optimizer
with the loss function as binary cross entropy.

5.2 MASK DETECTION THROUGH WEB CAM

In order to detect the face in an image that is being fed, two pretrained models are
used. Hence the faces are detected using a pretrained model and then the mask is
detected. To incorporate camera to our model the proposed method uses OpenCV.
To do this, initially facenet is used which loads the pretrained models. To use the
model, readNet () method is used which is under dnn in OpenCV.

The mask detector model is loaded with the help of load_model () method. To
enable the camera, use Video Stream in which src = 0 which indicates the
webcam. Inside a while loop, every frame is read, and for each frame the face is
detected and a rectangular mask is drawn over the face and the accuracy of the
person wearing the mask or not is returned.

The accuracy is then labelled as ‘with mask’ or ‘without mask’ based on the
prediction that is done by our model. This is then applied to the rectangular box.
The rectangular box is then given green color by specifying (0,255,0) when the
person wears a mask and is given red color by specifying (0,0,255) when the

18
person doesn't wear a mask. The label is the displayed with the help of format
string. The label font is set as Hershey simplex.

For every frame the rectangle is drawn using rectangle () method and then color
coding is done to this rectangle. The frames are displayed with the help of imshow
() method. The while loop stops when the ‘q’ button is pressed. Finally, video
stream is closed and the frames are destroyed.

5.3 BUILDING AN ALERT SYSTEM

The alert system consists of a pop up along with a beep sound when the person is
identified to be not wearing a mask. This is achieved with the help of winsound
and easygui modules in python. Along with this a notification message is also
sent to the security guard via mail and sms. The email is sent through requests
and smtplib modules in python. The sms is sent by creating an account in twilio
and installing the corresponding twilio package in python.

19
 CHAPTER 6

IMPLEMENTATION AND RESULT

6.1 MASK DETECTOR MODEL

• Input: Dataset

• Data Pre-processing: Label Binarizer class

• Data Partitioning: Using train_test_split

• Data Augmentation: Enrich training data

• Training MobileNetV2 model

• Model Evaluation: metric used - accuracy

• Output: Mask detector model

Figure 6.1 Model evaluation report

20
 Figure 6.2 Accuracy and loss graph

6.2 INCORPORATING WEB CAM TO OUR MODEL

• Load face detector and mask detector model

• Start the video stream via web cam and read each frame.

• Detect the face using face detector and get the detections from mask
detector.

• Draw a bounding box around the face in each frame and display the
prediction result above the bounding box.

21
 Figure 6.3 WITH MASK
This screenshot encloses the face within a rectangular frame of green color and
displays 'Mask' with 99.43% prediction.

Figure 6.4 WITHOUT MASK

This screenshot encloses the face within a rectangular frame of red color and displays
'No mask' with 100% prediction.

22
 Figure 6.5 WITH AND WITHOUT MASK
This screen shot indicates that the face mask detector is able to detect and classify multiple
faces in a given face. In the given frame, the girl's face is predicted to be covered in a mask
with 98.36% and Bill gates 's face is predicted to be without mask with 99.99% confirmation.

6.3 FACE MASK ALERT SYSTEM

• Input: mask prediction result

• process:

• If no mask: send alert message to the security guard via a SMS and mail.

• If the person wears a mask, a bounding box is drawn around the face with
the label “mask”

• Output: a pop up message if the access is denied with an alarm sound.

SMS and E-mail is sent to the person in-charge.

23
Figure 6.6 SMS

Figure 6.7 Mail

24
 CHAPTER 7

CONCLUSION AND FUTURE ENHANCEMENT

7.1 CONCLUSION

Face detection is a key link of subsequent face-related applications, such as face

recognition, facial expression recognition, and face hallucination, because its
effect directly affects the subsequent applications performance. Therefore, face
detection has become a research hotspot in the field of pattern recognition and
computer vision and has been widely studied in the past two decades.

In this paper, a two-stage Face Mask Detector was presented. The first stage uses
a pre-trained model for robust face detection and trained a neural network that
can detect the presence of a mask in the face that is being detected by the
pretrained models. In the second phase, the model that was trained and tested was
loaded. A Camera is used to detect the presence of face masks on multiple faces.
In the third phase, a pop-up message appears if the access is denied with an alarm
sound. SMS and E-mail is sent to the person in charge.

Hence, the proposed method is able to detect face masks for human objects from
RGB channel images containing localized objects. Proposed Mobilenet V2
network can detect multiple faces from a single image. The method can find
applications in advanced tasks such as facial part detection.

In times of the COVID-19 pandemic, with the world looking to return to

normalcy and people resuming in-person work, this system can be easily
deployed for automated monitoring of the use of face masks at workplaces,
which will help make them safer.

25
7.2 FUTURE ENHANCEMENT

There are a number of aspects that needs to be planned shortly:

1. The proposed method can further be improved by incorporating GPS

module to provide the location of the person who is not wearing a mask
and this location can be sent as a notification.
2. Stage 1 and Stage 2 models can be easily replaced with improved models
in the future, that would give better accuracy and lower latency
3. And will consider improving the enriching the dataset used in the proposed
model to result in better learning.

26
 APPENDIX 1-SAMPLE CODING

1. BUILDING MASK DETECTOR MODEL

INIT_LR = 1e-4

EPOCHS = 20

BS = 32

DIRECTORY = r"C:\Mask Detection\CODE\Face-Mask-Detection-

master\dataset"

CATEGORIES = ["with_mask", "without_mask"]

print("[INFO] loading images...")

data = []

labels = []

for category in CATEGORIES:

path = os.path.join(DIRECTORY, category)

for img in os.listdir(path):

img_path = os.path.join(path, img)

image = load_img(img_path, target_size=(224, 224))

image = img_to_array(image)

image = preprocess_input(image)

data.append(image)

labels.append(category)

lb = LabelBinarizer()

27
labels = lb.fit_transform(labels)

labels = to_categorical(labels)

data = np.array(data, dtype="float32")

labels = np.array(labels)

(trainX, testX, trainY, testY) = train_test_split(data, labels,

test_size=0.20, stratify=labels, random_state=42)

aug = ImageDataGenerator(

rotation_range=20,

zoom_range=0.15,

width_shift_range=0.2,

height_shift_range=0.2,

shear_range=0.15,

horizontal_flip=True,

fill_mode="nearest")

baseModel = MobileNetV2(weights="imagenet", include_top=False,

input_tensor=Input(shape=(224, 224, 3)))

headModel = baseModel.output

headModel = AveragePooling2D(pool_size=(7, 7))(headModel)

headModel = Flatten(name="flatten")(headModel)

headModel = Dense(128, activation="relu")(headModel)

headModel = Dropout(0.5)(headModel)

headModel = Dense(2, activation="softmax")(headModel)

28
model = Model(inputs=baseModel.input, outputs=headModel)

for layer in baseModel.layers:

layer.trainable = False

print("[INFO] compiling model...")

opt = Adam(lr=INIT_LR, decay=INIT_LR / EPOCHS)

model.compile(loss="binary_crossentropy", optimizer=opt,

metrics=["accuracy"])

print("[INFO] training head...")

H = model.fit(

aug.flow(trainX, trainY, batch_size=BS),

steps_per_epoch=len(trainX) // BS,

validation_data=(testX, testY),

validation_steps=len(testX) // BS,

epochs=EPOCHS)

print("[INFO] evaluating network...")

predIdxs = model.predict(testX, batch_size=BS)

predIdxs = np.argmax(predIdxs, axis=1)

print(classification_report(testY.argmax(axis=1), predIdxs,

print("[INFO] saving mask detector model...")

model.save("mask_detector.model", save_format="h5")

29
2. MASK DETECTION THROUGH WEB CAM

# load our serialized face detector model from disk

prototxtPath = r"face_detector\deploy.prototxt"

weightsPath = r"face_detector\res10_300x300_ssd_iter_140000.caffemodel"

faceNet = cv2.dnn.readNet(prototxtPath, weightsPath)

# load the face mask detector model from disk

maskNet = load_model("mask_detector.model")

# initialize the video stream

print("[INFO] starting video stream...")

vs = VideoStream(src=0).start()

# loop over the frames from the video stream

while True:

frame = vs.read()

frame = imutils.resize(frame, width = 400)

# detect faces in the frame and determine if they are wearing a

# face mask or not

(locs, preds) = detect_and_predict_mask(frame, faceNet, maskNet)

30
 for(box,pred) in zip(locs,preds):

(startX, startY, endX, endY) = box

(mask, withoutMask) = pred

label = "Mask" if mask > withoutMask else "No Mask"

color = (0,255,0) if label == "Mask" else (0,0,255)

label = "{}: {:.2f}%".format(label, max(mask, withoutMask) * 100)

cv2.putText(frame, label, (startX, startY - 10),

cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 2)

cv2.rectangle(frame, (startX, startY), (endX, endY), color, 2)

cv2.imshow("Frame",frame)

key = cv2.waitKey(1) & 0xFF

if key == ord("q"):

break

cv2.destroyAllWindows()

vs.stop()

31
3. FACE MASK ALERT SYSTEM

account_sid = "ACe368ef538ae07c1644c0fad7ceac2928"

auth_token = "ca0581d49ec31ab308e0506d7b37cb08"

client = Client(account_sid, auth_token)

def email_alert(subject, body, to):

msg = EmailMessage()

msg.set_content(body)

msg['subject'] = subject

msg['to'] = to

user = "mahalakshmithirumurthy@gmail.com"

password = "qvlhwmdprzztfrpz"

msg['from'] = user

server = smtplib.SMTP("smtp.gmail.com", 587)

server.starttls()

server.login(user,password)

server.send_message(msg)

server.quit()

32
APPENDIX-2 OUTPUT

33
 REFERENCES

1. H. Lau et al., “Internationally lost COVID-19 cases,” J. Microbiol.

Immunol. Infect., vol. 53, no. 3, pp. 454–458, 2020.
2. L. Li et al., “COVID-19 patients’ clinical characteristics, discharge rate,
and fatality rate of meta-analysis,” J. Med. Virol., vol. 92, no. 6, pp. 577–
583, Jun. 2020.
3. M. Z. Islam, M. M. Islam, and A. Asraf, “A Combined Deep CNNLST
Network for the Detection of Novel Coronavirus (COVID-19) Using X-
ray Images,” Informatics in Medicine Unlocked, vol. 20, pp. 100412, Aug.
2020.
4. L. J. Muhammad, M. M. Islam, S. S. Usman, and S. I. Ayon, “Predictive
Data Mining Models for Novel Coronavirus (COVID-19)Infected Patients’
Recovery,” SN Comput. Sci., vol. 1, no. 4, p. 206, Jun. 2020.
5. L. Liu et al., “Deep Learning for Generic Object Detection: A Survey,” Int.
J. Comput. Vis., vol. 128, no. 2, pp. 261–318, Sep. 2018.
6. L. Calavia, C. Baladrón, J. M. Aguiar, B. Carro, and A. Sánchez-
Esguevillas, “A Semantic Autonomous Video Surveillance System for
Dense Camera Networks in Smart Cities,” Sensors, vol. 12, no. 8, pp.
10407–10429, Aug. 2012.
7. G. Halegoua, “Smart City Technologies,” Smart Cities, 2020,
doi:10.7551/mitpress/11426.003.0005.
8. Z. Allam and D. S. Jones, “On the Coronavirus (COVID-19) Outbreak and
the Smart City Network: Universal Data Sharing Standards Coupled with
Artificial Intelligence (AI) to Benefit Urban Health Monitoring and
Management,” Healthcare, vol. 8, no. 1, p. 46, 2020.
9. M. Jiang, X. Fan, and H. Yan, “RetinaMask: A Face Mask detector,” 2020.
[Online]. Available: http://arxiv.org/abs/2005.03950.

34
10. M. Gupta, M. Abdelsalam, and S. Mittal, “Enabling and Enforcing Social
Distancing Measures using Smart City and ITS Infrastructures: A COVID-
19 Use Case,” 2020. [Online]. Available:https://arxiv.org/abs/2004.09246.
11. J. Won Sonn and J. K. Lee, “The smart city as time-space cartographer
in COVID-19 control: the South Korean strategy and democratic
control of surveillance technology,” Eurasian Geogr. Econ., pp. 1–
11, May. 2020.
12. R. P. Singh, M. Javaid, A. Haleem, and R. Suman, “Internet of things(IoT)
applications to fight against COVID-19 pandemic,” Diabete Metab. Syndr.
Clin. Res. Rev., vol. 14, no. 4, pp. 521–524, Jul. 2020.
13. J. W. Sonn, M. Kang, and Y. Choi, “Smart city technologies for pandemic
control without lockdown,” Int. J. Urban Sci., vol. 24, no.2, pp. 149
151,2020.
14. R. Jaiswal, A. Agarwal, and R. NEGI, “Smart Solution for Reducing the
COVID-19 Risk using Smart City Technology,” IET Smart Cities,vol.2,pp.
82–88, 2020.
15. X. Wang, X. Le, and Q. Lu, “Analysis of China’s Smart City Upgrade and
Smart Logistics Development under the COVID-19 Epimedic,” J.Phys.
Conf. Ser., vol. 1570, p. 012066, 2020.
16. Face Mask Detection| Kaggle. [Online]. Available:
https://www.kaggle.com/andrewmvd/face-mask-detection, accessed on:
Jul. 27, 2020.
17. GitHub-prajnasb/observations. [Online]. Available: https://github.com
/prajnasb/observations, accessed on: Jul. 27, 2020.
18. A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A Survey of the
Recent Architectures of Deep Convolutional Neural Networks,” Artif.
Intell. Rev., Jan. 2019.
19. I. S. Cardenas et al., “Telesuit: design and implementation of an
immersive user-centric telepresence control suit,” in Proceedings of the

35
 23rd International Symposium on Wearable Computers – ISWC ’19, New
York, NY, USA, 2019, pp. 261–266.
20. D. Y. Kim, I. S. Cardenas, and J.-H. Kim, “Engage/Disengage: Control
Triggers for Immersive Telepresence Robots,” in Proceedings of the 5th
International Conference on Human Agent Interaction, New York, NY,
USA, 2017, pp. 495–499.

36