Real-time Face Mask Detection Using Machine
Learning/ Deep Feature-Based Classifiers For Face
Mask Recognition
Sweety Reddy
Computer Science Engineering Computer Science Engineering
University of Petroleum and Energy University of Petroleum and Energy
Studies
Dehradun, India
sweetyr2000@gmail.com silkygoel90@gmail.com
2021 IEEE Bombay Section Signature Conference (IBSSC) | 978-1-6654-1758-7/21/$31.00 ©2021 IEEE | DOI: 10.1109/IBSSC53889.2021.9673170
Abstract — In this research, we put forward an architecture
that combines recent deep learning algorithms with geometry
techniques to create robust models that can handle aspects like
detection, tracking and validation. This dissertation focuses on
creating a model that efficiently utilizes a mix of conventional
machine learning and deep learning techniques to categorize
the facemasks effectively. Our model comprises two aspects in
this hour of need: For dimensionality reduction through
feature extraction, the initial element is produced with
InceptionV3. The facemask classification procedure is
developed with the Logistic Regression (LR) method. Deep
learning (DL) is used to efficiently train an architecture using a
dataset of photos of people's faces with and without and partial
face masks to extract features. The retrieved traits are now
passed into various classification algorithms namely Random
Forest, Logistic Regression, CNN, Support Vector Machine,
AdaBoost and K-Nearest Neighbors to appropriately classify
the masks position. Hence, we can project that employing
Transfer Learning (TL) and Deep Learning together can
detect a properly or improperly worn face mask with high
accuracy. This system design stops transmitting this fatal virus
by detecting individuals in urban areas who are not wearing
facemasks effectively.
Keywords—Face Mask Detection, Machine Learning, Deep
Learning, Classification, VGG16, VGG19
I. INTRODUCTION
SARS-CoV-2, a coronavirus discovered in 2019 has
triggered a worldwide pandemic with unthinkable
consequences in our daily lives, turning the world upside
down. As of July 13, 2021, 187,086,096 confirmed COVID-
19 cases worldwide with 4,042,921 fatalities [1]. The World
Health Organization (WHO) labeled it a public health
emergency of worldwide concern, the greatest degree of
alarm under international law. As time passed by and we
learned more about the SARS-CoV-2 virus, it became clear
that face coverings, alongside other social distancing
measures, were stemming the spread of the virus. Wearing
protective face masks is now part of the new normal since
we believe that prevention is better than cure. Even in the
near future, face masks are a part of the survival
recommendation norms given to the public as COVID is
believed to still be around us for quite some years before it
may disappear completely [2]. Consequently, identifying
face masks has become an important responsibility in
assisting global civilization.
In order to implement this, our research applies a
sophisticated approach. The suggested model accurately
978-1-6654-1758-7/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.
Silky Goel Rahul Nijhawan
Computer Science Engineering
University of Petroleum and Energy
Studies Studies
Dehradun, India Dehradun, India
rahulnijhawan@ddn.upes.ac.in
recognizes the face from the image and then determines
whether a mask is worn correctly, partially or not at all.
The machine learning algorithms with various pre-trained
deep learning architectures have been trained on 66% of our
dataset and analyzed by the remaining 34% i.e. our test
dataset. For each of the algorithms, we obtained Area Under
Curve (AUC), Recall, F1, Precision and Classification
Accuracy (CA) varied with popular models for feature
extraction by visualizing the obtained results in the form of a
ROC Curve and Confusion Matrix. A principal challenge
confronted by our team in the research was to be able to find
and amalgamate the images to the category of partial masks;
this category consists of a mixture of mouth-chin, nose-
mouth and only chin-covered images that help the model to
produce a better and accurate analysis of whether a mask is
worn correctly or not as shown in Fig. 1. Another major
problem was finding images of faces with masks having the
right kind of orientation.
Fig. 1. Sample images of no mask, partial mask and full mask.
The following is how the paper is structured: Section II
provides information on our study and a few references to
our work. Section III consists of a wide range of techniques
such as K-Nearest Neighbor, Logistic Regression, Support
Vector Machine, Random Forest etc. and further explains the
approach involved and three distinct feature extraction
models: Inception V3, VGG-16 and VGG-19. The acquired
outcomes from employing the above procedure are compared
through confusion matrix and other parameters such as AUC,
Recall, F1-score, Precision and Accuracy in Section IV.
Eventually, Section V summarizes our paper findings.
II. RELATED WORK
Facemask detection models which became a critical
model during the COVID-19 epidemic, have received a lot of
attention in recent months. While personally checking
whether individuals are wearing facemasks in public and
Fig. 2. The architecture for the proposed model for mask detection using pre-trained CNN models.
using facemasks to help maintain social distancing at
crowded places by utilizing computer vision and Raspberry
Pi that can generate accurate feedback through the reports.
After analysis, we figured that this study uses three datasets.
The first dataset is the Simulated Masked Face Dataset
(SMFD), the second is Labeled Faces (LF) and the third is
Real-World Masked Face Dataset (RWMFD) [2]. In SMFD,
the SVM learning algorithm obtained 99.49 percent
accuracy. LFW obtained 100 percent testing accuracy
whereas RMFD scored 99.64 percent.
Another significant contribution in this domain is the
efficient detector, Retina Face Mask Detector which is a face
mask analyzer that detects whether individuals are wearing
their masks or not [3]. This architecture is indeed a single-
stage detector that includes a novel ambient attention module
focused on face mask identification and a pyramid network
feature that integrates semantic data with multiple feature
maps-this algorithmic technique eliminates the background
attention component artifacts with higher union crossovers
and weak confidences. The Retina Face Mask achieves state-
of-the-art results on facemask datasets that are 2.3 percent
and 1.5 percent higher than the expected result respectively,
and mask detection precision that is 11.0 percent and 5.9
percent better than the standard result.
III. METHODOLOGY
We imported our dataset which was divided into images
with full masks, partial masks (mouth-chin / only chin) and
normal faces without masks as shown in Fig. 1. The
following feature extraction models are then implemented on
our dataset: Inception V3, VGG-16 and VGG-19. The
resultant data is sampled with the training and testing dataset
ratios remaining constant at 66:34. There are two
experiments conducted to establish a system for identifying
appropriately worn face masks. Firstly, a comparison of three
different deep features and six machine learning algorithms
was carried out. Following that, a comparison of the best
tuples against the average accuracies in the 5-fold cross-
validation technique has been computed for all efficacy trials
(one partition for the training set and one for the test set) [4].
Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.
busy places is cumbersome, researchers developed a real-
time automated model that can determine whether people are
We utilized NN, LR, KNN, Support Vector Machine
(SVM), K-Nearest Neighbors, AdaBoost and Random
Forest techniques employed in the image classification
process. For each of the three various feature extraction
models, LR has the most remarkable accuracy in our result
observation [5].
In this statistical analysis, conducted for image
classification [6] performance measures like Recall, F1-
Score, Precision and Accuracy have been calculated by
using the following equations (1), (2), (3) and (4).
Precision = TP/ (TP + FP) (1)
Recall = TP/ (TP + FN) (2)
F-Measure = 2TP/ (2TP + FP + FN) (3)
Accuracy = (TP + TN)/ (TP + TN + FP + FN) (4)
AUC Score is generally spoken about after defining the
ROC curves as shown in Fig. 3, Fig. 4 and Fig. 5. This chart
visualizes the trade-off between True Positive Rate (TPR)
and False Positive Rate (FPR) where its values are given by
equations (5) and (6):
True Positive Rates (TPR) = TP/ (TP + FN) (5)
False Positive Rates (FPR) = FP/ (FP + TN) (6)
Here,
TP = True Positive, TN = True Negative, FP = False
Positive, FN = False Negative.
CNN [7] has been mainly built for image classification
tasks. The convolution layers, pooling and activation
function layers are layered to a certain depth one over
another in this architecture. Finally, when performing the
classification, an output layer is added to a fully linked layer
which then calculates the picture’s classification probabilities
provided as an input to CNN which further uses a SoftMax
layer for mapping them onto specific classes. The
convolution operations are then performed by the findings, we experimented with three distinct feature
convolutional layers on these images by utilizing many extraction models [13]: Inception V3, VGG-16, and VGG-
filters with extremely tiny sizes relative to the provided 19. For all three distinct feature extraction models, we
photos (such as 3 X 3, 5 x 5, etc.). Training is used to learn observed our results through ROC curve and various
the weights associated with these filters. CNN-based network parameters like AUC, Accuracy, F1 score, Precision and
training takes a lengthy time, measured in hours or days. Recall [14].
Additionally, CNN models require a large amount of data • Inception V3 feature model
to be processed to provide informative feature maps. The
introduction of Transfer Learning (TL) [8,9] has effectively The following Fig. 3 demonstrates the ROC curve for
overcome this problem. Initial layer weights that are frozen Inception V3 model [15] and different machine learning
are used in these instances along with their pre-trained classifiers.
models. Now, our dataset is utilized to retrain the final few
layers to extract just the abstract features from those images
as the core traits are consistent across all of the others:
InceptionV3, Visual Geometry Group VGG16 and VGG19.
CNN models include a cutting-edge architecture that
employs this principle.

IV. PROPOSED APPROACH

Our study uses a mix of both conventional machine
learning and advanced deep learning techniques to detect the
facemasks. There are two components to the model: the first
element is built using deep convolutional, pre-trained neural
network models based on VGG16 ,VGG19 and Inception V3
architectures for object detection task to detect the presence
of masks correctly during feature extraction without causing Fig. 3. ROC curve representing Inception v3 and different
over-fitting, the second component is constructed using a classifiers.
number of machine learning methods namely AdaBoost, LR,
CNN, KNN and SVM. This technique achieves the TABLE I: MACHINE LEARNING ALGORITHMS UTILIZED IN
maximum accuracy of 96.3% using Inception V3 model and INCEPTION V3 MODEL.
Logistic Regression classifier. Performance Measures
Model
AUC Accuracy F1 Precision Recall
Proposed architecture, as shown in Fig. 2 utilizes a fine-
LR 99.4% 96.3% 95.3% 96.2% 96.3%
tuned feature extraction model, Inception V3 that had been NN 99.2% 94.5% 94.3% 94.7% 94.8%
pre-trained on the Normal Face without masks dataset. This
SVM 98.7% 94.8% 94.7% 94.7% 94.7%
image recognition algorithm has been demonstrated to KNN 97.0% 91.1% 90.8% 91.3% 90.7%
achieve better than 78.1 percent accuracy on the ImageNet RF 96.7% 91.3% 91.0% 90.6% 90.7%
dataset and is the culmination of numerous ideas explored by AdaBoost 87.7% 86.3% 86.3% 85.8% 85.5%
researchers over the years. Inception V3 convolutional neural
network architecture improves on previous versions by Label Through the observed outcomes in Table I and ROC curves
Smoothing, factorized 7 x 7 convolutions, and uses an shown in Fig. 3 the Logistic Regression technique provides
auxiliary classifier that is further used to transmit label us with the best accuracy of 96.3 % and AUC value of
information farther down the network [10]. This proposed 99.4%.
work employs Logistic Regression as the primary classifier • VGG16 feature model [15]
and as illustrated in Table I. Our images are categorized as
either normal face, entirely or partially masked.
V. EXPERIMENTAL RESULTS
Our proposed model was trained on 767 rigorously
selected images which were then classified into three
categories/classes: Normal face without a mask (127
pictures), Face with complete mask (449 images) and Face
with partial mask [mouth-chin / nose-mouth /only chin] (191
images) [11].Configuration: Core i3, 1.60GHz CPU, 4GB
RAM.

A. Models Used
We employed popular image classification classifiers
namely Logistic Regression, SVM [12], K-Nearest Fig. 4. ROC curve representing the VGG16 model and
Neighbors, Random Forest and AdaBoost. To get alternative several classifiers.

Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.
TABLE II: COMPARISON OF MACHINE LEARNING ALGORITHMS categorization model. They are commonly utilized because
UTILIZED IN VGG-16.
they provide a more accurate picture of a model's
Performance Measures performance than classification accuracy.
Model
AUC Accuracy F1 Precision Recall
LR 99.1% 94.1% 94.1% 94.1% 94.1% TABLE IV: CONFUSION MATRIX FOR LOGISTIC REGRESSION
NN 96.0% 89.0% 88.9% 89.1% 89.0% CLASSIFIER IN INCEPTION V3 MODEL
SVM 96.6% 88.7% 88.5% 88.6% 88.7%
KNN 87.4% 79.1% 78.0% 77.8% 79.1%
RF 88.0% 79.1% 78.0% 77.8% 79.2%
AdaBoost 75.4% 71.6% 71.8% 72.2% 71.6%

Through the observed outcomes in Table II and ROC

curve shown in fig. 4. The Logistic Regression technique
provides us with the best accuracy of 94.1% percent and
AUC of 99.1%.
• VGG19 feature model Model for feature extraction: Inception V3 Table IV
reveals that there are 449 mask-wearing faces, 7 of which are
incorrectly categorized as partially masked pictures. Three
pictures of exposed faces without masks are misclassified as
partially masked faces out of 127. At the same time, one of
the 191 somewhat masked faces is misclassified as wearing
no mask and 17 as that being fully masked [17, 18].
TABLE V: CONFUSION MATRIX FOR LOGISTIC REGRESSION
CLASSIFIER IN VGG16 MODEL

Fig. 5. ROC curve representing the VGG19 model and

Model for Feature Extraction: VGG-16 [19]. Table V
several classifiers
displays 449 faces with masks, 11 of which have been
TABLE III: COMPARISON OF MACHINE LEARNING ALGORITHMS misclassified as partially masked faces. Three pictures of
UTILIZED IN VGG-19 exposed faces without masks are misclassified as partially
Performance Measures masked faces out of 127. While 191 somewhat masked faces
Model are misclassified as faces without any mask, 6 are
AUC Accuracy F1 Precision Recall
LR 99.2% 94.6% 94.6% 94.7%
misclassified as fully masked [20].
94.7%
NN 97.6% 92.7% 92.6% 92.7% 92.7% TABLE VI: CONFUSION MATRIX FOR LOGISTIC REGRESSION
SVM 96.9% 90.6% 90.4% 91.0% 90.6% AND VGG19 MODEL
KNN 89.9% 84.0% 82.9% 83.7% 84.0%
RF 92.2% 84.6% 83.8% 84.4% 84.6%
AdaBoost 80.4% 71.7% 77.8% 77.9% 77.7%

Through the observed outcomes in Table III and ROC curve

shown in fig. 5. The Logistic Regression technique provides
us with an accuracy of 94.7 % and AUC of 99.2%.
Finally, based on our observations, we determined
that the Logistic Regression method is best suited for our
model for classification, and Inception V3 model as feature
extraction model. The highest accuracy achieved was
96.1%.
VI. RESULT
We applied the proposed hybrid model using previously
described deep learning architectures and compared the
outcomes of images with various methods and models for
feature extraction, after which we accomplished the
following outcomes:
Confusion Matrix [16]: A confusion matrix is a tabular
breakdown of a classifier's accurate and erroneous
predictions. It is used to assess the effectiveness of a
Model for Feature Extraction: VGG-19. Table VI
displays 449 faces with masks, 11 of which have been
misclassified as partially masked pictures and 1 without a
mask. Three pictures of exposed faces without masks are
misclassified as partially masked faces, while one image is
misclassified as fully masked. While 191 partially masked
faces are misclassified as no mask, 3 are misclassified as
totally masked [21].
VII. CONCLUSION
We presented a hybrid method in this paper that makes
use of deep learning architectures linked with the transfer
learning methodology associating INCEPTION V3 PLUS
LOGISTIC REGRESSION obtained the highest accuracy of

Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.
96.1 percent among the two other popular deep learning
architectures (VGG 16 and VGG 19) and several different
predictive analytics approaches that were evaluated. In
assessed pictures, our implementation outperformed other
state-of-the-art algorithms for the task of face mask
recognition. We analyzed pictures of three different mask
detection scenarios by extracting bottleneck characteristics
where we utilized a dataset of 767 pictures where 66% of the
images were utilized for training the model, while the
remaining 34% were utilized to assess the model. Since our
approach differentiates a particular image into one of three
groups, it eventually computes distinct categorization
probabilities for a particular image according to each class
involved. Image is thus categorized into the class with the
highest probability. This hybrid technique used to identify
face masks here is very novel. In general, there aren't enough
instances to train a deep architecture from the cradle to the
grave. Our technique provides a realistic solution to employ
CNN’s which eliminates their need to generate hand-crafted
features.
In reality, this model can be employed in several
instances to determine if a person is correctly wearing their
masks or not. There are numerous such applications for face
mask detection for example: in airports, visitors' faces can be
promptly and effectively captured in the system at the entry.
Hospitals can use a face mask detection system to determine
whether or not their employees are wearing masks.
Furthermore, for quarantined individuals who are forced to
wear a mask, the system can keep an eye out and identify
whether or not the mask is present. It may also be used in
offices to determine whether employees are adhering to
workplace safety regulations. This framework can soon
achieve great heights by expanding the collection of images
that can transform this CNN network to be more powerful in
the near future.
Finally, this research opens up new avenues for future
research. The suggested approach is not restricted to mask
detection and may be implemented into any high-resolution
video surveillance system. Secondly, this framework may be
extended to recognize facial landmarks while wearing a
facemask for biometric purposes.
REFERENCES
[1] https://www.who.int/emergencies/diseases/novel-
coronavirus-2019/events-as-they-happen
[2] K. Suresh, M. Palangappa and S. Bhuvan, "Face Mask
Detection by using Optimistic Convolutional Neural
Network," 2021 6th International Conference on Inventive
Computation Technologies (ICICT), 2021, pp. 1084-1089,
doi: 10.1109/ICICT50816.2021.9358653.
[3] M. Jiang, Fan, X. and Yan, H., 2020. Retinamask: A face
mask detector. arXiv preprint arXiv:2005.03950.
[4] R. Boutaba, Salahuddin, M.A., Limam, N., Ayoubi, S.,
Shahriar, N., Estrada-Solano, F. and Caicedo, O.M., 2018.
"A comprehensive survey on machine learning for
networking: evolution, applications and research
opportunities,” Journal of Internet Services and
Applications, 9(1), pp.1-99, 2018.
[5] R. Nijhawan, Raman, B. and Das, J., “Meta-classifier
approach with ANN, SVM, rotation forest, and random
Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.
forest for snow cover mapping,” in Proc. of 2nd
International Conference on Computer Vision & Image
Processing (pp. 279-287), Springer, Singapore.
[6] S. Gupta, A. Panwar, S. Goel, A. Mittal, R. Nijhawan and
A. K. Singh, "Classification of Lesions in Retinal Fundus
Images for Diabetic Retinopathy Using Transfer
Learning," International Conference on Information
Technology (ICIT), 2019, pp. 342-347.
[7] A. Chavda, J. Dsouza, S. Badgujar and A. Damani,
"Multi-Stage CNN Architecture for Face Mask
Detection," 2021 6th International Conference for
Convergence in Technology (I2CT), 2021, pp. 1-8.
[8] G. J Chowdary, Punn, N.S., Sonbhadra, S.K. and
Agarwal, S., “Face mask detection using transfer learning
of InceptionV3,” In International Conference on Big Data
Analytics, 2020, pp. 81-90.
[9] M. E. H. Chowdhury et al., "Can AI Help in Screening
Viral and COVID-19 Pneumonia?," IEEE Access, vol. 8,
pp.132665-132676, 2020.
[10] Tripathi, M., “Analysis of Convolutional Neural Network
based Image Classification Techniques,” Journal of
Innovative Image Processing (JIIP), vol. 3, no. 2, pp.100-
117, 2018.
[11] A. Cabani, Hammoudi, K., Benhabiles, H. and Melkemi
M., “MaskedFace-Net–A dataset of correctly/incorrectly
masked face images in the context of COVID-19,” Smart
Health, pp.1-5, 2021.
[12] D. Varshni, K. Thakral, L. Agarwal, R. Nijhawan and A.
Mittal, "Pneumonia Detection Using CNN based Feature
Extraction," 2019 IEEE International Conference on
Electrical, Computer and Communication Technologies
(ICECCT), 2019, pp.1-7.
[13] R. Nijhawan, H. Sharma, H. Sahni and A. Batra, "A Deep
Learning Hybrid CNN Framework Approach for
Vegetation Cover Mapping Using Deep Features," 2017
13th International Conference on Signal-Image
Technology & Internet-Based Systems (SITIS), 2017, pp.
192-196.
[14] S. S. Rawat, A. Bisht and R. Nijhawan, "A Classifier
Approach using Deep Learning for Human Activity
Recognition," 2019 Fifth International Conference on
Image Information Processing (ICIIP), 2019, pp. 486-490.
[15] Y. -C. Hsieh, C. -L. Chin, C. -S. Wei, I. -M. Chen, P. -Y.
Yeh and R. -J. Tseng, "Combining VGG16, Mask R-CNN
and Inception V3 to identify the benign and malignant of
breast microcalcification clusters," 2020 International
Conference on Fuzzy Theory and Its Applications
(iFUZZY),2020, pp.1-4.
[16] S. Gupta, A. Panwar and D. Rawat, "A Comparison
among Distinct Deep Learning Techniques for Real-
Time Testingof Covid-19 Infected Patient using Chest
Radiography," 2021 2nd International Conference on
Secure Cyber Computing and Communications
(ICSCCC), 2021, pp.394-398.
[17] H. S. Chhabra, Srivastava, A.K. and Nijhawan, R., 2020.
“A hybrid deep learning approach for automatic fish
classification,” in Proceedings of ICETIT, 2019, pp. 427-
436.
[18] S. Gupta, A. Panwar and K. Mishra, "Skin Disease
Classification using Dermoscopy Images through Deep
Feature Learning Models and Machine Learning
 Classifiers," IEEE EUROCON 2021 - 19th International
Conference on Smart Technologies, 2021, pp. 170-174.
[19] S. Visa, Ramsay, B., Ralescu, A.L.,Van Der Knaap and
E., “Confusion matrix-based feature selection,” in Proc.
MAICS, 2011, pp.120-127.
[20] A. Panwar, R. Yadav, K. Mishra and S. Gupta, "Deep
Learning Techniques for the Real Time Detection of
Covid19 and Pneumonia using Chest Radiographs," IEEE
EUROCON 2021 - 19th International Conference on
smartTechnologies, 2021, pp.250-253.
[21] Gupta, S., Aggarwal, P., Chaubey, N. and Panwar, A.,
“Accurate prognosis of Covid-19 using CT scan images
with deep learning model and machine learning
classifiers,” Indian Journal of Radio & Space Physics
(IJRSP), vol. 50, no. 1, pp.19-24, 2021.

Authorized licensed use limited to: SRM University Amaravathi. Downloaded on November 17,2022 at 07:15:13 UTC from IEEE Xplore. Restrictions apply.