Gender Based Emotion

Detection System
Name of the students

ANSHIKA JAIN (19102052) (A2)

VISHESH JAIN (19102103) (A4)

ANMOL ARORA (19102110) (A4)

-SUBMITTED TO

Dr. P.K. Yadav

Dr. Juhi Gupta
ABSTRACT

● Our project ‘Facial Emotion Detection System’ detects the emotion portrayed
on the face of a person, in real time.
● One of the pioneers of understanding communication, Prof. Albert Mehrabin,
has concluded in his study [2], that in face-to-face communication, emotions
are transmitted in proportion of 55% through facial expressions. That means
that if the computer could capture and understand the emotions of its
"interlocutor", communication would be more natural and appropriate,
especially if we think of scenarios where a computer would play the role of a
tutor.
● Human emotion detection is implemented in many areas requiring additional security or
information about the person. It can be seen as a second step to face detection where we
may be required to set up a second layer of security, where along with the face, the
emotion is also detected. This can be useful to verify that the person standing in front of
the camera is not just a 2-dimensional representation [1] of that person.

● Another important domain where we see the importance of emotion detection is for
business promotions. Most of the businesses thrive on customer responses to all their
products and offers. If an artificial intelligent system can capture and identify real time
emotions based on user image or video, they can make a decision on whether the
customer liked or disliked the product or offer.
Scopes and Applications-

The scope of this system is to tackle with the problems that can arise in
day to day life. Some of the scopes are:
1. The system can be used to detect and track a user’s state of mind.
2. The system can be used in mini-marts, shopping center to view the
feedback of the customers to enhance the business,
3. The system can be installed at busy places like airport, railway station
or bus station for detecting human faces and facial expressions of each
person. If there are any faces that appeared suspicious like angry or
fearful, the system might set an internal alarm.
4. The system can also be used for educational purpose such as one can get
feedback on how the student is reacting during the class.
5. This system can be used for lie detection amongst criminal suspects
during interrogation
6. This system can help people in emotion related -research to improve the
processing of emotion data.
7. Clever marketing is feasible using emotional knowledge of a person
which can be identified by this system.
SYSTEM DIAGRAM-
● A webcam is used to capture, detect and recognize the expressions of a person.
Rectangular frame on the face region is obtained in the camera; this detection of
face region from a non-face region is done by making use of different algorithms.
Around 60 Images of a person are captured, preprocessed and stored in a folder
with his/her ID and Name. During Face Detection: Pretrained dataset is used to
match the face in video camera.
● If person’s face matches with that of trained dataset, then his/her ID and Name is
shown. The classification of the acquired face is done using convolutional neural
network by making use of FER2013 database. The highest probability of
expression acquired with respect to the features, is shown as the facial expression.
One out of seven expressions are displayed with the detected image of the person.
DATASET USED: FER2013

●FER2013 is an open-source dataset which is first created for an ongoing project by

Pierre-Luc Carrier and Aaron Courville, then shared publicly for a Kaggle
competition.
●The FER2013 database was introduced during the ICML2013 Challenges in
Representation Learning.
●FER2013 is a large-scale and unconstrained database collected automatically by the
Google image search API. All images have been registered and resized to 48*48 pixels
after rejecting wrongfully labelled frames and adjusting the cropped region.
●This dataset consists of 35,887 grayscale, 48x48 sized face images with various
emotions -7 emotions, all labelled-.
Emotion labels in the dataset:

0: -4593 images- Angry

1: -547 images- Disgust

2: -5121 images- Fear

3: -8989 images- Happy

4: -6077 images- Sad

5: -4002 images- Surprise

6: -6198 images- Neutral

● During the competition, 28,709 images and 3,589 images were shared with the participants as training and public test
sets respectively and the remaining 3,589 images were kept as private test set to findthe winner of the competition.
The dataset was set to accessible to everyone after completing the competition.
● The FER-2013 dataset was created by gathering the results of a Google image search of each emotion and synonyms
of the emotions. The images in FER-2013 consist of both posed and un-posed headshots.
Fig.2 Example images from the FER2013
dataset.
The above figure illustrates variability in illumination, age, pose, expression intensity, and occlusions
that occur under realistic conditions. Images in the same column depict identical expressions,
namely anger, disgust,fear, happiness, sadness, surprise, as well as neutral.

The data file contains 3 columns — Class, Image data, and Usage

a) Emotion class: is a digit between 0 to 6 and represents the emotion depicted in the
corresponding picture. Each emotion is mapped to an integer as shown below.

0- 'Angry’ 1-'Disgust’ 2-'Fear’ 3-'Happy’ 4-'Sad’ 5-'Surprise’ 6-'Neutral'

b) Image data: is a string of 2,304 numbers and these are the pixel intensity values of
ourimage.

c) Usage: It denotes whether the corresponding data should be used to train the network
or test it.
 NORMALIZATION

● Normalization let’s you see how things are different than the average.

● Based on the distances between those points, we get to detect the new data.
Algorithm used-

1. Viola Jones Algorithm:

● Haar-like feature considers neighboring rectangular regions at a specific location in a

detection window, sums up the pixel intensities in each region and calculates the difference
between these sums. This difference is then used to categorize subsections of an image. In
the detection of human faces, Commonly, the areas around the eyes are darker than the
areas on the cheeks.

● Haar-like feature for face detection is therefore a set of two neighboring rectangular areas
above the eye and cheek regions.
The algorithm has four stages:

a) Haar Feature Selection

b) Creating Integral Images

c) Ad boost Training

d) Cascading Classifiers
Local Binary Pattern (LBP):

Local Binary Pattern (LBP) is a simple yet very efficient texture operator which labels the
pixels of an image by thresholding the neighborhood of each pixel and considers the result as a
binary number. It was first described in 1994 (LBP) and has since been found to be a powerful
feature for texture classification. It has further been determined that when LBP is combined
with histograms of oriented gradients (HOG) descriptor, it improves the detection performance
considerably on some datasets.

Using the LBP combined with histograms the face images are represented with a simple data
vector.
Applying the LBP operation:

The first computational step of the LBPH is to create an intermediate image that
describes the original image in a better way, by highlighting the facial characteristics.
To do so, the algorithm uses a concept of a sliding window, based on the parameter’s
radius and neighbors’ image below shows this procedure:
CONCLUSION-

● Facial expression plays an important role in communication and thus identifying

the correct expression is as essential as knowing the exact matter of the
communication. This project proposes an approach for recognizing the category of
facial expressions. Face Detection and Extraction of expressions from facial images
has been achieved successfully and is useful in many applications, such as robotics
vision, video surveillance, digital cameras, security and human-computer
interaction. This project’s objective is to develop a FACE EMOTION
RECOGNITION implementing the computer visions and enhancing the advanced
feature extraction and classification in face expression recognition.
FUTURE SCOPE OF WORK-

● Face expression recognition systems have improved a lot over the past decade. The focus
has definitely shifted from posed expression recognition to spontaneous expression
recognition. Promising results can be obtained under face registration errors, fast
processing time, and high correct recognition rate (CRR) and significant performance
improvements can be obtained in our system.
● System is fully automatic and has the capability to work with images feed. It is able to
recognize spontaneous expressions. Our system can be used in Digital Cameras wherein
the image can be captured only when the person smiles.
● In security systems which can identify a person, in any form of expression he presents
himself. Rooms in homes can set the lights, television to a person’s taste when they enter
the room. Doctors can use the system to understand the intensity of pain or illness of a
deaf patient. Our system can be used to detect and track a user’s state of mind, and in
mini-marts, shopping center to view the feedback of the customers to enhance the
business etc.
Future replacement:

In future we can add features like:

1. Registering person details automatically when camera detects a human face.

2. Registering multiple person’s details simultaneously.

3. Doctors can use the system to understand the intensity of pain or illness of a deaf patient.
REFERENCES-
Chibelushi, C. C., & Bourel, F. (2003). Facial expression recognition: A brief tutorial overview.
CVonline: On-Line Compendium of Computer Vision, 9.
Sokolova, M., Japkowicz, N., & Szpakowicz, S. (2006, December). Beyond accuracy, F-score
and ROC: a family of discriminant measures for performance evaluation. In Australasian Joint
Conference on Artificial Intelligence (pp. 1015- 1021). Springer Berlin Heidelberg.
Michel, P., & El Kaliouby, R. (2005). Facial expression recognition using support vector
machines. In The 10th International Conference on Human-Computer Interaction, Crete, Greece.
Michel, P., & El Kaliouby, R. (2003, November). Real time facial expression recognition in video
using support vector machines. In Proceedings of the 5th international conference on Multimodal
interfaces (pp. 258-264). ACM.
Zhang, S., Zhao, X., & Lei, B. (2012). Facial expression recognition based on local binary
patterns and local fisher discriminant analysis. WSEAS Trans. Signal Process, 8(1), 21-31.30