Missing Person Identification Using Face
Recognition
Simran Arora#1, Deepesh Kumar Rai*2, Gaurav Vishwakarma#3, Susanto Mandal*4
#
Computer Science Department, Dr.A.P.J Abdul Kalam Technical University, India
Abstract—In the world, a countless number of people
are missing every day which includes kids, teens, mentally
challenged, old-aged people with Alzheimer's, etc. Most of them
remain untraced. This paper proposes a system that would help
the police and the public by accelerating the process of searching
using face recognition. When a person goes missing, the people
related to that person, or the police can upload the picture of the
person which will get stored in the database. When the public
encounter a suspicious person, they can capture and upload the
picture of that person into our portal. The face recognition model
in our system will try to find a match in the database with the
help of face encodings. It is performed by comparing the face
encodings of the uploaded image to the face encodings of the
images in the database. If a match is found, it will be notified to
the police and the people related to that person along with the
location. It also has additional features like informing the police
authorities and missing person’s relative using their specified e-
mail. Also, there is a feature to detect the missing person using
the video which can be implemented in CCTV and video
surveillance systems to broaden its reach and enhance the
working.
I. INTRODUCTION
A missing individual is regularly portrayed on the
grounds that the person who is frequently a little child or a
grown-up who is lost, intentionally or automatically. There are
different classifications of missing instances of which just 43%
of missing cases' reasons are known, 99% are adolescent
runways, 2500 cases are on account of family issues and
around 500 cases are seized by outsiders (which incorporate
the both teenagers and grown-ups).
“In India, there are no budgets allocated to finding
missing people”, claimed by an official source. Media can be
used to find missing people for instance the use of newspapers.
Media appeals may be the quickest and most effective way of
raising awareness of your missing person and helping in the
continuing search for him or her.
Nevertheless, not everyone feels comfortable using
the media. Different newspapers and magazines have different
interviewing techniques and styles. Whilst many journalists
will be sympathetic, others may appear forceful, cold or
aggressive or behave in other ways, which seem insensitive to
what you are going through. Some people do not trust the
media or want their circumstances made public; others feel
overwhelmed by the thought of dealing with journalists and
being asked probing and personal questions about their
missing friend or relatives.
Additionally, publicity may put already vulnerable
people at greater risk by forcing them further away if they do
not wish to be found.
Kidnappers can continue to victimize their victims,
as they will be aware through media. However, the use of
facial recognition technique makes it easier for us to find the
missing people and this will cater for all the disadvantages of
using media.
The objective of this project is to identify/recognise
the missing person based on their previously available data.
This project can be used by police to recognise missing person
easily and based on video footage we can also identify missing
person from the crowd. The objective of this study is two-fold:
• Matching a face with available database accurately.
• Applying principal component analysis for finding
distinguishable features from many images to get the
similarity for the target image.
We have developed a system that can be used by
police or investigation department to recognize missing
person from their faces. The method of face recognition used
is fast, robust, reasonably simple and accurate with a
relatively simple and easy to understand algorithms and
technique.
When a suspicious person is found, the picture at
that instance of time is compared with the images uploaded
by the guardian/police department at the time of missing
through the face recognition model. If a match is found, it will
be notified to the police and the guardian in the form of an
alert e-mail along with the location of where the person is
found. If not found, a new record will be created in the
database with the uploaded picture. By this way, it decreases
the time taken to search for a person’s detail after he is found.
Sometimes, the person has been missing for a long period of
time. The age gap is reflected in the image as ageing affects
the structure of the face, including shape, texture, etc. The
appearance of the person can vary due to ageing, filters, pose,
lightings etc. All these factors were considered before
choosing the face recognition algorithm.
Sometimes, the person has been missing for a long
period of time. The age gap is reflected in the image as ageing
affects the structure of the face, including shape, texture, etc.
The appearance of the person can vary due to ageing, filters,
pose, lightings etc. All these factors were considered before
choosing the face recognition algorithm.
II. LITERATURE SURVEY
A. The Eigen Face Method
Firstly, Kirby and Sirvoich demonstrated Eigenfaces
method for recognition. Pentland and Turk made
improvements on this research by employing Eigenfaces
method based on Principle Component Analysis for the same
reason the eigenfaces are the principal components of a
distribution of faces, or equivalently, the eigenvectors of the
covariance matrix of the set of face images, where an image
with N pixels is considered a point (or vector) in N-
dimensional space
B. Fisher Face Method
Image recognition using fisherface method is based
on the reduction of face space dimension using Principal
Component Analysis (PCA) method, then apply Fisher's
Linear Discriminant (FDL) method or also known as Linear
Discriminant Analysis (LDA) method to obtain feature of
image characteristic.
C. Face Recognition Through Geometric Features
In the first phase a set of fiducial points are examined
in each face and the geometric facts such as distances between
these points are explored and the image closest to the query
face is nominated.
D. Hidden Markov Model (HMM)
HMM generally employed on images with variations
due to lighting, orientation and facial expression and thus it
has more advancements over than the approaches for treating
images using HMM, space sequences are considered. This
procedure is named as a Hidden Markov Model this is why
because the states are invisible, only the output is vivid to the
external use
E. Active Appearance Model (AAM)
2D Morphable Method Faces are highly distinct and
able to be deformed. Classifying by pose, expression, lighting,
and faces can have various looks in the images. Coots, Taylor,
and Edwards [56] presented Active Appearance Model which
is strongly capable of explaining the view of face in set of
model parameters. AAM is an integrated statistical model,
implemented on the basis of a training set comprising labelled
images
III. METHODOLOGY
The proposed system makes use of Face Recognition
for missing peoples’ identification. The architecture of our
framework is presented in figure 1.
Figure 1. The Architecture of the proposed People Identification System
Here the public or police who finds a suspicious
person (child, mentally challenged person, etc.) on the road
uploads a picture of that person into the portal. Our algorithm
extracts the face encodings of the image as shown in Figure 2
and compare with that of the face encodings of the previously
existing images in the database. If a match is found, an alert e-
mail will be sent to both the concerned police officer and the
parent/guardian of that person in the image. If a match is not
found, then the person will be provided with the option of
registering that face as a new entry to our database with the
remarks.
Whenever public or police upload an image, the face
encodings of the image are extracted and then compared to the
face encodings of the images stored in the database. If the
distance between the encoding of the uploaded image and the
encoding of the image in the database is less than or equal to
the threshold, then the face in both the images is of the same
person. If that is the case, the user is notified that a match is
found along with the picture from the database that matched
with the uploaded picture. If the distance between the
encodings is more than the threshold, it means that the faces in
the images are not of the same person. By this way, our
proposed system will help in identifying the missing people.
Figure 2. Comparing Face encodings of two images
The model we have used involves five main steps to
perform face recognition.
A. Preprocessing
To reduce the variability in the faces, the images are
processed before they are fed into the network. All positive
examples that is the face images are obtained by cropping
Department of CSE Page 3 images with frontal faces to include
only the front view. All the cropped images are then corrected
for lighting through standard algorithms.
A face recognition system, based on computing the
distance between unprocessed gray- level images, fails to
recognize all the faces in the database and will confuse the
faces. The approaches to this problem can be classified in three
main categories:
• Illumination Normalization: In this approach, face images
are preprocessed to normalize the illumination. For
example, gamma correction, logarithmic transforms
histogram equalization few of the methods are used here.
• Invariant features extraction: This approach attempts to
extract facial features invariant to illumination variations.
For example, edge maps, derivatives of the gray-level,
Gabor-like filters and Fisher-face etc. are few of the
methods applicable here.
• Face modeling: illumination variations are mainly due to
the 3D shape of human faces under lighting in different
 directions. There are researchers trying to construct a
generative 3D face model that can be used to render face
image with different pose and under varying lighting
conditions.

It has been focused only on the first category, the

process, called illumination normalization, attempts to
transform an image with an arbitrary lighting condition to an
image with standard lighting condition. Subsequently the
Gamma Intensity Correction method, the Logarithm transform
method, Discrete Cosine transform method and the Histogram
Equalization method (global and local) are analysed to process Figure 3 Convolutional neural networks (CNN)
face images normalizing lighting variations.
C. Localization
B. Classification
The trained neural network is then used to search for
Neural networks are implemented to classify the
faces in an image and if present localize them in a bounding
images as faces or nonfaces by training on these examples. We
box. Various Feature of Face on which the work has done on:-
use both our implementation of the neural network and the
Position Scale Orientation Illumination.
MATLAB neural network toolbox for this task. Different
network configurations are experimented with to optimize the
results. The way of localizing faces is the HAAR-like
features object recognition algorithm. It describes objects by
their changes in lightness.
Image classification refers to the labelling of images
into one of a number of predefined classes. There are
potentially n number of classes in which a given image can be What lightness changes can we see that describe a
classified. Manually checking and classifying images could be general face? The forehead is lighter than the area of the eyes.
a tedious task especially when they are massive in number (say The bridge of the nose is lighter than the wings of the nose.
10,000) and therefore it will be very useful if we could The mouth is darker than the chin.
automate this entire process using computer vision.
HAAR-like features describe these changes in
The features have been extracted using a lightness using four types of features:
convolutional neural network. This is because deep learning
models have achieved state of the art results in the feature
extraction process.

Convolutional neural networks (CNN) are a special

architecture of artificial neural networks. CNNs uses some of
its features of visual cortex and have therefore achieved state
of the art results in computer vision tasks. Convolutional
neural networks are comprised of two very simple elements,
namely convolutional layers and pooling layers. Figure 4. HAAR Features
Green stands for more lightness, red for less lightness.
Although simple, there are near-infinite ways to To describe a face, several of these features (in fact, several
arrange these layers for a given computer vision problem. The thousand) are grouped together to form a classifier:
elements of a convolutional neural network, such as
convolutional and pooling layers, are relatively
straightforward to understand.

The challenging part of using convolutional neural

networks in practice is how to design model architectures that
best use these simple elements.

Figure 5. HAAR Features grouped together

 As you can see, the features can also be inverted.
• To detect a face in an image, the image gets scanned using
the classifier.
• To localize tilted heads the classifier has to be rotated.
• To calculate the lightness differences quickly you should
use integral images.
• To optimize the scanning process its suggested to first
scan roughly and then close in on the responsive areas.
D. Data
The Data is being stored in numpy matrix. Numpy
Matrix is a n x n dimensional matrix used to represent pixels
of images. It has capability of storing both grey scale images
and colored images.
Data being stored in numpy matrix is compared with
data being trained in neural net. Depending on comparison
output is being produced.
3) Distinctive Character Location: All facial-scan
systems attempt to match visible facial features in a
fashion similar to the way people recognize one
another. The features most often utilized in facial-
scan systems are those least likely to change
significantly over time: upper ridges of the eye
sockets, areas around the cheekbones, sides of the
mouth, nose shape, and the position of major features
relative to each other. Behavioural changes such as
alteration of hairstyle, changes in makeup, growing
or shaving facial hair, adding or removing eyeglasses
are behaviours that impact the ability of facial- scan
systems to locate distinctive features, facial-scan
systems are not yet developed to the point where they
can overcome such variables.
4) Template Creation: Enrolment templates are
normally created from a multiplicity of processed
facial images. These templates can vary in size from
less than 100 bytes, generated through certain
vendors and to over 3K for templates. The 3K
template is by far the largest among technologies
considered physiological biometrics. Larger
templates are normally associated with behavioural
biometrics.

Figure 6. Face is stored in n x n matrix

E. Functionalities
The functionalities performed by the system on the
input attributes to give the desired output are as follows:

1) Image Acquisition (Input image): Facial-scan

technology can acquire faces from almost any static Figure 7. Image Recognition Flow
camera or video system that generates images of
sufficient quality and resolution. Enrolment is
essential for verification and identification. IV. RESULTS AND DISCUSSION
Enrolment images define the facial characteristics to After implementation following are the results
be used in all future authentication events. summarized based on our observations:
• The system has reasonably fast response, and the
2) Image Processing (Face Detection): Images are
matching templates step- which happen to have a small
cropped such that the ovoid facial image remains, and
time relative to the total response time- is the only step
colour images are normally converted to black and
that depends upon the size of the dataset, luckily, it’s only
white in order to facilitate the initial comparisons
logarithmic proportional to dataset sizes, this implies that
based on grayscale characteristics. First the presence
the system is scalable to large dataset and will have fast
of faces or face in a scene must be detected. Once the
response on big data.
face is detected, it must be localized and
• Increasing the volume of data did not hurt the system
Normalization process may be required to bring the
response performance not it is accuracy in detecting the
dimensions of the live facial sample in alignment
faces.
with the one on the template.
• Finally, we get to see that the faces being scanned through
the systems are whether being identified or not as a
missing person or normal citizen either.
These are the salient features based on our discussion
over the system:
• All identification and authentication technologies operate
using the 4- stage method.
• Capturing the physical and behavioural sample during
enrolment
• Extracting the unique data from the sample and a template
is created and the data is stored in NumPy matrix
• Comparing the template with the new sample
• Then the system checks if the feature extracted in the
matrix is a match or non-match
• Our project consists of two comparisons:
a. Verification: The system compares the given
individual with who they say they are and gives a yes
or no decision.
b. Identification: The system compares the given
individual to all the other individuals in the database
and gives the ranked list of matches.
V. CONCLUSION
Our system replaces the manual method of scanning
through the database for each picture to check the match by an
efficient face recognition method which finishes the work on
time.
Approximate match of human face at
different angles using optical flow. In the future, we
are planning to extend this system further by increasing the
reach of our system to public.
Though our system has a small limitation i.e., when
the age of the person is between the age 0 and 10 the accuracy
drops. This is due to the incomplete growth of facial features
at that age. We look forward to overcoming this limitation in
the future.
REFERENCES
[1] Huang, Gary B. And Erik G. Learned-Miller. “Labeled Faces in the
Wild: Updates and New Reporting Procedures”, Department of
Computer Science, University of Massachusetts Amherst, Amherst,
MA, USA, Tech Report, 2014, pp 14–003
[2] S. Chandran, Pournami & Balakrishnan, Byju & Rajasekharan,
Deepak & N Nishakumari, K & Devanand, P & M Sasi, P. (2018).
“Missing Child Identification System Using Deep Learning and
Multiclass SVM”. 113-116. 10.1109/RAICS.2018.8635054
[3] Rohit Satle , Vishnuprasad Poojary , John Abraham , Mrs. Shilpa
Wakode, “MISSING CHILD IDENTIFICATION USING FACE
RECOGNITION SYSTEM” Vol.3, Issue.1, July – August 2016
[4] S. B. Arniker et al., "RFID based missing person identification
system," International Conference on Informatics, Electronics &
Vision (ICIEV), Dhaka, 2014, pp. 1-4.
[5] Birari Hetal, “Android Based Application - Missing Person Finder”,
in Iconic Research and Engineering Journals, Vol.1, Issue 12, JUN
2018.
[6] Thomas M. Omweri, “Using a Mobile Based Web Service to
Search for Missing People – A Case Study of Kenya”, in
International Journal of Computer Applications Technology and
Research, Vol. 4, Issue 7, 507 - 511, 2015.
[7] Sumeet Pate, “Robust Face Recognition System for E-Crime Alert”,
in International Journal for Research in Engineering Application
and Management, Issue 1, MAR, 2O16
[8] Peace Muyambo, 2018, An Investigation on the Use of LBPH
Algorithm for Face Recognition to Find Missing People in
Zimbabwe, INTERNATIONAL JOURNAL OF ENGINEERING
RESEARCH & TECHNOLOGY (IJERT) Volume 07, Issue 07
(July
2018)