COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN

COVID-19 Risk Assessment through Multiple Face

Mask Detection using MobileNetV2 DNN
Marielet Guillermo*1, Athena Rosz Ann Pascua*1, Robert Kerwin Billones*2, Edwin
Sybingco*1, Alexis Fillone*3, Elmer Dadios*2
*1
Electronics and Communications Department
*2
Manufacturing Engineering and Management Department
*3
Transportation and Civil Engineering Department
Email: marielet_guillermo@dlsu.edu.ph
Gokongwei College of Engineering, De La Salle University
Manila, Philippines

Abstract. The current pandemic caused by COVID-
19 outbreak is continuously putting lives into peril. It
has already spread to over 200 countries, some of
which are still fighting heavily to survive. This study
aims to promote the importance of disease control and
preventive measures such as the use of face masks in
crowded places. An active face masks detection and
monitoring system can help authorities to identify
people who might be vulnerable to infectious diseases
such as coronavirus. The problem in strategic
planning such as allocation of personnel in high risk
areas, due to lack of reliable and prompt means to
identify COVID-19 cases, is giving much headache to
Filipino community especially to the Philippine
Government. An artificial neural network-based
system capable of detecting if people in the crowd are
wearing face masks, will be discussed
comprehensively in this paper. The implementation of
the study resulted to 99% in all training and testing
key parameters.
Keywords: face and face mask detection, deep neural
network (DNN), computer vision, artificial dataset,
MobileNetV2
1. INTRODUCTION
Coronavirus disease (COVID-19) emerged in early
December 2019 and since then, it ravaged 214 countries
in a span of months, causing 571,527 deaths (as of July
13, 2020) and affecting many aspects of life in almost
every part of the globe [1]. School classes, work, supply
chains, travels and more are halted or modified in the best
effort of reducing the impact of the pandemic [2].
COVID-19 is a concerning disease which is caused by a
novel coronavirus. It comes in various forms of
complications in human beings ranging from mild
symptoms including respiratory illnesses to severe
progressive pneumonia, organ failure, cardiovascular
complications, and ultimately, death [3]. Another cause
of concern is the unavailability of clinical preventive
vaccines or pharmaceutical drugs which can cure the said
disease. Another factor to point out is the high effectivity
of the disease to transfer from a human host to another,
triggering about 13.1 million confirmed cases worldwide
due to human to human transmission [1]. Studies suggest
that prevention of spread of the disease like COVID-19,
including physical or social distancing of at least 1 meter,
should drastically reduce the risk of infection. Evidence
also suggests the role of using face masks and eye
protection in reducing the infection rate of the disease.
Although none of these measures ensure complete
protection against COVID-19, these help in buying time
for the healthcare system to expand, react, and respond.
[2].
Flattening the curve of COVID-19 cases is not as easy as
it looks. There has to be more ways to do COVID-19 risk
analysis. To complement analysis, based on rapid testing,
and improve existing systems, the researchers suggest
having a risk assessment in crowded places, based on the
usage of face masks. In this way, strategic response can
be done actively while waiting for swab test results. With
this in mind, the researchers innovate a system capable of
multiple face mask detection in public places using
MobileNetV2 Deep Neural Network (DNN).
2. REVIEW OF RELATED LITERATURE
COVID-19 cases increase daily in the Philippines, hence
the need for a stricter protocol in preventing the spread of
this virus. The World Health Organization (WHO) has
advised several public health and social measures [4] to
slow down or possibly stop the spread of COVID-19.
These measures must be instigated with the full
cooperation of the entire society [5].
Certain documentaries written during COVID-19, MERS
(Middle East Respiratory Syndrome), and SARS (Severe
Acute Respiratory Syndrome) outbreak, prove that the
current protocol to maintain at least 1 m physical
distancing relates with a great decrease in spread and
infection, more so, with distances over 2 m. These data
also attest that wearing face masks and Personal
Protective Equipment (PPEs) protects individuals (both
health-care workers and the public) against contagion.
Eye protection (face shield) also serves additional
security. However, none of these interventions offers full
defense from infection thus, having the need for risk
assessment and several related deliberations [2][5]. There
are not enough analyses given the surging amount of
confirmed cases and patients who experience critical

The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
1
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
China, 2020
Oct.31-Nov.3, 2020
 COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN

respiratory rupture and cardiovascular difficulties [6].

Hence, the need to explore artificial intelligence (AI). It
is also necessary to know how and to what extent, on a
global scale, AI could help in improving and renewing
health care methods [7].
Optimizing treatment procedures and emergency
administration, and improving diagnosis methods such as
medical imaging and vision processing, are only few of
immense advantages, AI can offer. However, its potential
support to medical practices and assistance to health-care
systems against the COVID-19 seem to be underutilized
at present. One section that takes a special interest in AI’s Fig. 3. Phase 3: Face Mask Detector Testing
valuable information is the image-based system allowing 3.1. Artificial Face Mask Dataset Creation
COVID-19 medical diagnosis faster and more precise One unique implementation of this study is creating an
[8][9]. Regardless of the abundance of research made in artificial dataset. Other neural network and vision-based
the area of COVID-19, there are not a lot of existing studies [17] [18] usually gather raw images and videos,
literature efforts to analyze the effects and role of then annotate it to be used as datasets. Here, the
developing technologies like neural networks, let alone researchers used Prajna Bhandary’s study on adding face
artificial intelligence, and deep learning [10]–[12]. masks to face images using computer vision.
On the other hand, there were few researches relevant to In this phase, raw face images were applied to a face
public security which can be a stepping stone towards detection model, which is responsible for calculating the
addressing concerns in COVID-19 [13]–[16]. For bounding box location of the face and pulling out the
instance, study on intruder face mask detection can region of interest. dlib’s facial landmark detector maps
actually be diverted to detecting whether or not people facial structure from the region of interest (ROI) by
are strictly following preventive measures such as estimating the location of 68 (x,y)-coordinates.
wearing medical face masks.
Seizing AI systems to deal with COVID-19 related
matters can fill the gap between AI-based systems and
medical methods and practices. AI experts’ usage of AI
programs can help create connections between different
parameters and speed up the means to achieve optimum
results. The future of AI systems to overcome COVID-
19 related hurdles can be inferred through this study.

3. METHODOLOGY
The implementation of this study was made possible
through three (3) major phases namely: artificial face
mask dataset creation, face mask detector training, and
face mask detector testing. Each of these phases are
subdivided into blocks as illustrated below:
Fig. 4. 68 facial landmark coordinates [19]
Knowing the location of nose and mouth, the loaded
mask image is then applied to ROI and positioned
accordingly to the raw face image. As a result, the
original face image now has a mask on it.

Fig. 1. Phase 1: Artificial Face Mask Dataset Creation

Fig. 5. Sample Artificial Face Mask Dataset Output

For more number of dataset, the researchers included
artificial face mask dataset output from Prajna, publicly
available images, and some from volunteer researchers
Fig. 2. Phase 2: Face Mask Detector Training with their consent.

The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
2
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
China, 2020
Oct.31-Nov.3, 2020
 COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN
3.2. Face Mask Detector Training
In total, over 600 images, with or without masks, were
used as a dataset. These were preprocessed then used to
train the MobileNetV2 classifier model. Pre-processing
steps include resizing images to 224×224 pixels,
conversion to array format, and scaling the pixel
intensities in the input image to the range [-1, 1]. Part of
data preparation as well are encoding of labels, partition
of dataset (80% for training and 20% for testing), and
arrangement for the improvement of data.
The model was fine-tuned with pre-trained ImageNet
weights before the actual training. This is necessary to set
a paradigm while saving acceptable time. The following
hyper parameter constants were specified:
● initial learning rate = 0.001
● number of training epochs = 20
● batch size = 32
With all these organized, the model is trained and part of
this is making predictions on the test set, getting the
highest probability class label indices. Finally, the results
of training are published and saved as a newly trained
model. The results of this training such as accuracy and
loss curves will be illustrated and discussed on the next
section of this paper.
The researchers chose to use MobileNetV2 classifier
model as this architecture is highly efficient that it can be
applied to devices with limitations on computing
capability [20]. Deploying a face mask detector to these
types of devices potentially decreases assembly cost. As
described in [21], this model is an enhancement of
MobileNetV1 [22], custom-fit for computer vision
applications to address the problem of network
complexity through inverted residual linear bottleneck
layer. In the experiment section of [21], it is observed that
linear layers prevent destruction of information
especially that the manifold of interest is lying in a low-
dimensional subspace of the higher-dimensional
activation space in deep neural networks. The
performance on ImageNet classification of MobileNetV2
as compared to other models [23] operating in a
constrained computational setting is shown in the
succeeding table and figure below.
Table. 1. Performance on ImageNet including Multiply-Adds and
running time for a single large core of Google Pixel 1 phone
using TF-Lite [21]
The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
Fig. 6. Performance curve of MobileNetV2 vs other model [21]
Using MobileNetV2 classifier model for this research
indeed resulted to a high accuracy to be shown on the
next section. Not to mention, acceptable training time
given a limited computing power of the machine used.
3.3. Face Mask Detector Testing
The first step in this phase is to load the trained face mask
classifier so it can be later on used as soon as the face
ROIs from images/videos at stake are ready. Next is to
also load the face detection model. For this one, the
researchers used a deep neural network (DNN) based
face detector model that was created with a single-shot
detector (SSD) framework using ResNet-10 architecture
as a backbone. It was also heavily trained in the Caffe
framework on a huge and available online dataset. Hence,
the researchers chose using this model for face detection
instead of training on their own.
Steps 2 and 3 in this phase are similar to that in phase 1.
Only, test data for images are different from the training
data used. In addition, static videos and video streams
directly from webcam are tested. Next step is to apply the
face mask classifier model on the extracted ROI to
determine whether the people captured in the test data are
wearing masks or not. Results are shown through loading
the original test data but this time, with labels on the
faces. The label includes the identified classification –
“mask” in green and “no mask” in red with probability
percentage of the classified item.
4. DATA AND RESULTS
This section discusses the results of the training and
testing phase for face mask detection. This will showcase
the effectiveness and efficiency of neural network
execution for this study.
4.1. Training Phase
In summary, here are the parameters used and equivalent
learning rates for each epoch associated with the study:
Table. 2. Hyper parameter constants
3
China, 2020
Oct.31-Nov.3, 2020
 COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN

Table. 3. Learning Rate mapping with Epoch

Fig. 8. Loss and Accuracy

The first three (3) parameters in the first table are set by
the researchers and the 4th one was based on gathered 4.2. Testing Phase
available data. For decay and weight updates, refer to To assess the accuracy of the trained model, the
formulas below: researchers used a different set of images from the data
used in the previous phase. Static videos and feeds from
𝐷𝑒𝑐𝑎𝑦 =
𝐼𝑛𝑖𝑡𝑖𝑎𝑙 𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝑅𝑎𝑡𝑒
Eq. (1) the webcam were also tested. See test results below:
𝐸𝑝𝑜𝑐ℎ

𝑇𝑟𝑎𝑖𝑛𝑖𝑛𝑔 𝐷𝑎𝑡𝑎
𝑊𝑒𝑖𝑔ℎ𝑡 𝑈𝑝𝑑𝑎𝑡𝑒𝑠 = Eq. (2)
𝐸𝑝𝑜𝑐ℎ

To get the Learning Rate (α) for each epoch, the working
formula is:
1
𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝑅𝑎𝑡𝑒 (𝛼) = 𝐼𝐿𝑅 ∗
1+(𝐷𝑒𝑐𝑎𝑦(𝐸𝑝𝑜𝑐ℎ 𝑁𝑜.∗𝑊))
Eq. (3)

where: ILR is Initial Learning Rate

W is Weight Updates

Fig.9 Single face images test data (face images from volunteers)
It can be seen from Table 3 that the learning rate
decreases as the no. of epoch increases. This behavior
implies that the neural network takes smaller steps in
learning, allowing it to descend into areas of the loss
landscape that are more optimal [24] [25]. In other words,
obtaining higher accuracy through finding an area with
reasonably low loss [26][27].
The result of training neural network model is shown in
the figures below:

Fig.10 Multiple face images test data (face images from publicly
available images)

Fig. 7. Neural Network Evaluation

Fig.11-a Static video frames test data (from publicly available

videos)

The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
4
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
China, 2020
Oct.31-Nov.3, 2020
 COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN
Fig.11-b Static video frames test data (from publicly available
videos)
Fig.11-c Static video frames test data (from publicly available
videos)
Fig.11-d Static video frames test data (from publicly available
videos)
Fig. 12-a Video Stream Frames Test Data
The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
Fig. 12-b Video Stream Frames Test Data
From the images and video frames above, the probability
percentage of classified items, whether “mask” or “no
mask” is approximately 99%. As the risk of getting
COVID-19 is higher in crowded places, the researchers
also tested the multiple face mask detection system in
publicly available videos with mass gathering. See
Figures 11-b to 11-d.
5. CONCLUSION AND RECOMMENDATIONS
Based on Loss and Accuracy figure, the curves for
training accuracy and loss are almost linear at a value of
1 and 0 respectively. Thus, almost 100% accuracy and
minimal loss. Looking further into the performance
metrics, it can be seen in the neural network evaluation
figure that accuracy, precision, recall, and F1 score are
all approximately 99%.
Accuracy measures the actual correctly predicted
observation over the total observations while high
precision measured indicates low false positive rate. High
recall means almost all correctly classified items were
correctly predicted. F1 score simply refers to the
weighted average of precision and recall. All these
parameters being at 99% value indicate that the classifier
model predictions are highly reliable. This was proven
further with the testing results shown in the figures under
Testing Phase section.
Despite the dataset used for training is artificial and there
are no videos included, the performance of the face mask
detector is superb. However, the researchers still
recommend training the model with actual image and
video datasets for it to maintain great performance when
tested on different video conditions and where more
people are needed to be detected.
It is also recommended for future researchers to extend
the study to having real time analytics to help people in
authority respond much faster to pandemic situations.
While the study is a great enabler towards COVID-19
risk assessment in crowded places, it should be continued
until there is already a full defense against this disease.
Acknowledgements
The researchers would like to thank the De La Salle
University (DLSU) Intelligent Systems Laboratory (ISL),
Department of Electronics and Communications
Engineering, and Department of Manufacturing and
Engineering Management for the support and learning
resources they provided, and to volunteers for the image
data set.
5
China, 2020
Oct.31-Nov.3, 2020
 COVID-19 Risk Assessment through Multiple Face Mask Detection using MobileNetV2 DNN

REFERENCES: landmark features to determine the best subset for finding face
orientation,” ICCIDS 2019 - 2nd Int. Conf. Comput. Intell. Data Sci.
[1] Q. Pham, D. C. Nguyen, T. Huynh-The, W. Hwang and P. N. Pathirana, Proc., pp. 17–20, 2019, doi: 10.1109/ICCIDS.2019.8862093
"Artificial Intelligence (AI) and Big Data for Coronavirus (COVID-
[20] A. Mu, “Food categories classification and ingredients estimation
19) Pandemic: A Survey on the State-of-the-Arts," in IEEE Access,
using CNNs on Raspberry Pi 3,” J. Chem. Inf. Model., vol. 53, no. 9,
doi: 10.1109/ACCESS.2020.3009328.
pp. 1689–1699, 2019, doi: 10.1017/CBO9781107415324.004.
[2] The effect of control strategies to reduce social mixing on outcomes of
[21] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen,
the COVID-19 epidemic in Wuhan, China: a modelling study Kiesha
“MobileNetV2: Inverted Residuals and Linear Bottlenecks,” 2018,
Prem*, Yang Liu*, Timothy W Russell, Adam J Kucharski, Rosalind
doi: 10.1109/CVPR.2018.00474.
M Eggo, Nicholas Davies, Centre for the Mathematical Modelling of
Infectious Diseases COVID-19 Working Group†, Mark Jit, Petra [22] A. G. Howard et al., “MobileNets: Efficient Convolutional Neural
Klepac Lancet Public Health 2020 Published Online March 25, 2020 Networks for Mobile Vision Applications,” 2017, [Online]. Available:
https://doi.org/10.1016/ S2468-2667(20)30073-6 http://arxiv.org/abs/1704.04861.
[3] M. B. Jamshidi et al., "Artificial Intelligence and COVID-19: Deep [23] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning
Learning Approaches for Diagnosis and Treatment," in IEEE Access, Transferable Architectures for Scalable Image Recognition,” Proc.
vol. 8, pp. 109581-109595, 2020, doi: IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 8697–
10.1109/ACCESS.2020.3001973. 8710, 2018, doi: 10.1109/CVPR.2018.00907.
[4] EUROPEAN CENTRE FOR DISEASE PREVENTION AND [24] P. Mishra and K. Sarawadekar, “Polynomial Learning Rate Policy
CONTROL, “Considerations relating to social distancing measures in with Warm Restart for Deep Neural Network,” IEEE Reg. 10 Annu.
response to the COVID-19 epidemic,” no. March, pp. 1–10, 2020, Int. Conf. Proceedings/TENCON, vol. 2019-October, pp. 2087–2092,
[Online]. Available: https://www.ecdc.europa.eu/en/publications- 2019, doi: 10.1109/TENCON.2019.8929465.
data/considerations-relating-social-distancing-measures-response- [25] Y. Kanada, “Optimizing neural-network learning rate by using a
COVID-19-second. genetic algorithm with per-epoch mutations,” Proc. Int. Jt. Conf.
[5] Coronavirus disease 2019 (COVID-19) Situation Report – 72 World Neural Networks, vol. 2016-October, pp. 1472–1479, 2016, doi:
Health Organization Data as reported by national authorities by 10:00 10.1109/IJCNN.2016.7727372.
CET 1 April 2020 https://www.who.int/publications-detail/infection- [26] R. Bhati, S. Jain, N. Maltare, and D. K. Mishra, “A comparative
prevention-and-control-during-health-care-when-novel-coronavirus- analysis of different neural networks for face recognition using
(ncov)infection-is-suspected-20200125 principal component analysis, wavelets and efficient variable learning
[6] T. Guo et al., ‘‘Cardiovascular implications of fatal outcomes of rate,” 2010 Int. Conf. Comput. Commun. Technol. ICCCT-2010, pp.
patients with coronavirus disease 2019 (COVID-19),’’ JAMA 526–531, 2010, doi: 10.1109/ICCCT.2010.5640486.
Cardiol., early access, Mar. 27, 2020, doi: [27] R. K. C. Billones et al., "Vehicle-Pedestrian Classification with Road
10.1001/jamacardio.2020.1017. Context Recognition Using Convolutional Neural Networks," 2018
[7] P. Hamet and J. Tremblay, ‘‘Artificial intelligence in medicine,’’ IEEE 10th International Conference on Humanoid, Nanotechnology,
Metabolism, vol. 69, pp. S36–S40, Apr. 2017. Information Technology,Communication and Control, Environment
and Management (HNICEM), Baguio City, Philippines, 2018, pp. 1-6,
[8] W. Naudé, ‘‘Artificial Intelligence against COVID-19: An early
doi: 10.1109/HNICEM.2018.8666257.
review,’’ IZA Inst. Labor Econ., Maastricht, The Netherlands, Tech.
Rep. 13110, Apr. 2020.
[9] T. T. Nguyen, G. Waurn, and P. Campus, “Artificial Intelligence in the
Battle against Coronavirus ( COVID-19 ): A Survey and Future
Research Directions,” Researchgate.Net, 2020, doi:
10.13140/RG.2.2.36491.23846.Artificial.
[10] V. Chamola, V. Hassija, V. Gupta and M. Guizani, "A Comprehensive
Review of the COVID-19 Pandemic and the Role of IoT, Drones, AI,
Blockchain, and 5G in Managing its Impact," in IEEE Access, vol. 8,
pp. 90225-90265, 2020, doi: 10.1109/ACCESS.2020.2992341.
[11] A. Beikmohammadi and K. Faez, “Leaf Classification for Plant
Recognition with Deep Transfer Learning,” Proc. - 2018 4th Iran.
Conf. Signal Process. Intell. Syst. ICSPIS 2018, pp. 21–26, 2018, doi:
10.1109/ICSPIS.2018.8700547.
[12] O. Cakiroglu, “Mask R-CNN ile Derin Yüz Sezici Gerçekleme Design
of a Deep Face Detector by Mask R-CNN,” 2019 27th Signal Process.
Commun. Appl. Conf., pp. 1–4, 2019.
[13] G. Deore, R. Bodhula, V. Udpikar, and V. More, “Study of masked
face detection approach in video analytics,” Conf. Adv. Signal Process.
CASP 2016, pp. 196–200, 2016, doi: 10.1109/CASP.2016.7746164.
[14] W. Bu, J. Xiao, C. Zhou, M. Yang, and C. Peng, “A cascade
framework for masked face detection,” 2017 IEEE Int. Conf. Cybern.
Intell. Syst. CIS 2017 IEEE Conf. Robot. Autom. Mechatronics, RAM
2017 - Proc., vol. 2018-January, pp. 458–462, 2017, doi:
10.1109/ICCIS.2017.8274819.
[15] T. Meenpal, A. Balakrishnan, and A. Verma, “Facial Mask Detection
using Semantic Segmentation,” 2019 4th Int. Conf. Comput. Commun.
Secur. ICCCS 2019, pp. 1–5, 2019, doi:
10.1109/CCCS.2019.8888092.
[16] M. S. Ejaz and M. R. Islam, “Masked face recognition using
convolutional neural network,” 2019 Int. Conf. Sustain. Technol. Ind.
4.0, STI 2019, vol. 0, pp. 24–25, 2019, doi:
10.1109/STI47673.2019.9068044.
[17] M. Guillermo et al., “Detection and Classification of Public Security
Threats in the Philippines Using Neural Networks,” 2020, doi:
10.1109/LifeTech48969.2020.1570619075.
[18] R. K. C. Billones et al., “Vision-based passenger activity analysis
system in public transport and bus stop areas,” 2019, doi:
10.1109/HNICEM.2018.8666357.
[19] H. M. Shah, A. Dinesh, and T. S. Sharmila, “Analysis of facial

The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020)
6
CITIC Jingling Beijing, China, Beijing,
Hotel Beijing, Oct.31-Nov.3,
China, 2020
Oct.31-Nov.3, 2020