SDG

No.10

KNOW-THEM
Domain-Machine Learning
Team
Guide • Deepika Lakshmi.M.B
Ms.S.Swathi • Minu Dhakshina .B.S
• Aanandhini.S
INTRODUCTION
Visually challenged are one of the marginalized
people who face day to day problems without seeing
the world. The rapid advancement in technology has
now helped the visually challenged people to
explore the world through the mobile phones. Many
features and applications has been developed to help
them use the smart phones effectively like the normal
people. A survey showed that 75% of the visually
impaired has started to use smart phones. Few
features that helps them use these devices are virtual
assistants ,text to voice convertors, image
recognizers, etc.
PROBLEM STATEMENT
• Understanding the emotions through facial reactions
of others while speaking to them is important to
understand the people. Visually challenged people use
their other senses effectively to understand other’s
emotions, but at times it becomes a challenging task
• Finding whether the person they face is a one whom
they have already met or not is difficult.
What is Know Them ?
Know Them is a mobile application that assists
the visually impaired people to understand /
know how the person opposite to them or
person to whom they speak react through their
facial expression. Know Them uses some Ai
principles to recognise the emotions of other
person and conveys it to the user through
some text to voice conversion techniques.
KnowThem comes under the SDG goal.no 10- Reduced Inequalities &
SDG goal.no 1 No Poverty ,as it can enrich the visually challenged
communities and enrich their day-to-day life
EXISTING SYSTEM

LOOK OUT :
It is a an Assisted Vision that uses ML algorithms to
know every objects around them.

Different feature included in Know Them :

Identifying the emotion of the opposite person
through the emotion feature extraction algorithms.
 WORK FLOW

SCAN THE FACE FACIAL

FACE FEATURE APPROVE
USING REAR EXPRESSION
DETECTION EXTRACTION EMOTION
CAMERA DETECTION

START
CONVEY THE
EMOTION DETECTED
TO THE USER
THROUGH SPEECH
ARCHITECTURE DIAGRAM
WORK FLOW

CLASSIFICATION
MODULE:1 Face Detection:
The first module for the project is face detection. It is just detecting the presence of a face in an
image with multiple objects.

MODULE:2 Facial Emotion Extraction:

Features from the images are extracted and images get trained.

MODULE:3 Emotion Classification

According to the extracted features, the images get classified using the Machine Learning
algorithms. The images are classified as Happy, Sad, Neutral, Surprise, Disgust and Angry.

MODULE:4 Text to Voice Conversion:

The output of how the image is classified is converted to voice format. As this project is being
focused for visually challenged the output should be in voice format and so text to voice conversion is
another module to convert the form of the output.
Screenshots of the code
The sample code for detecting the faces in the
captured image
Code used to train the model
Code used to test the features in the images
Code used to classify the images according
to the detected emotions
Result of the implemented modules

The implemented program has detected various facial objects in the above image.
REQUIREMNTS OF Know Them
Facial expressions are a great indicator of the state of a mind for a person. KnowThem can sense the emotions of
the user using their facial expressions through the camera . The emotions are recognized using machine learning
methods and includes Artificial intelligence concepts .
Our work is based and works on python libraries. The principle mechanism of this project will implemented in a
mobile application.

Software Requirements
• OpenCv - For face recognition ,detecting the face, training the module and image processing
• Pyttsx3 - For the conversion of text to speech in a program it works offline
• Keras , Tensorflow
• Other Python modules
• Android Studio for app development

Hardware requirements
• Smart Phone
• Rear Camera
• Speaker
 Know Them recognises the emotion of the person
whom the visually challenged(user) faces and
How Know Them helps the visually categorises their emotion into various types as
challenged? mentioned in the table.

Emotions
• Knowing the emotions of others will help the Happy
visually impaired to understand others and brings Blushing
an active feeling. Excited
• Boosts self-confidence to meet or face others when Surprised
they are out on their own without anyone to assist Grateful
them. Contended
• Knowing the facial expressions and reactions of
Tired
others will be one thing that the visually challenged
Cunning
would always wish to know. Hence it brings them
Shocked
fulfilment.
Angry
Sorry
Sad
No reaction
FUTURE ENHANCEMENT
IDENTIFYING THE PERSON
In future, enhancement of the project will be done by adding an advantage of
Identifying whether the people around the visually challenged people is already know to
him or not. This can be accomplished by creating a memory space to store the information
of the frequently and recently met people.
 REFERENCES
• https://files.eric.ed.gov/fulltext/EJ1149519.pdf - Use of Mobile Applications for People Who Are Visually
Impaired

• https://www.researchgate.net/publication/326224279_Text-to-Speech_Device_for_Visually_Impaired_Peo
ple

• https://ieeexplore.ieee.org/abstract/document/5626038 - Smartphone based face recognition tool for

the blind

• https://ieeexplore.ieee.org/document/7392581 - Face detection and recognition application for

Android