Facial Emotion Detection using Convolution

Neural Network
A progress report for EC708 Minor Project

Submitted By

Amit Prakash (2016UGEC051)

Ujjwal Gupta (2016UGEC012)

Under the supervisison of

Dr. Akhilesh Kumar

Associate Professor
Department of Electronics and Communication Engineering

DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING

NATIONAL INSTITUTE OF TECHNOLOGY, JAMSHEDPUR. `
September 30, 2019.
 CERTIFICATE

It is certified that the work contained in this project entitled ” Facial Emotion
Detection using Convolution Neural Network”, by Mr. Amit Prakash (Roll No.
2016UGEC012) and Mr. Ujjwal Gupta (Roll No. 2016UGEC012) has been carried out
under the supervision of Dr. Akhilesh Kumar, Associate Professor, NIT Jamshedpur
and this work has not been submitted elsewhere for a degree.

Dr. Akhilesh Kumar

Associate Professor
Department of Electronics and Communication Engineering
NIT Jamshedpur.

Amit Prakash (Roll no. - 2016UGEC051)

Ujjwal Gupta (Roll no. -2016UGEC012)

 ABSTRACT

Understanding one’s emotion have always been a complicated task. In this project
we are presented the results of seven emotional states (Neutral, Happy, Sad,
Surprise, Anger, Fear and Disgust) based on facial expressions. The model takes an
image or a video frame as an input and predicts the probability of above give
emotional states. The features have been extracted using Convolution Neural
Network Model- Mini Xception.
1 Origin of the Project:
Facial expressions play an important role in recognition of emotions and are used in the
process of non-verbal communication, as well as to identify people. They are very
important in daily emotional communication, just next to the tone of voice. They are also
an indicator of feelings, allowing a man to express an emotional state. People, can
immediately recognize an emotional state of a person. As a consequence, information on
the facial expressions are often used in automatic systems of emotion recognition. The
aim of the research, presented in this article, is to recognize seven basic emotional states:
neutral, joy, surprise, anger, sadness, fear and disgust based on facial expressions.

2. Objective:

This project is mainly aimed at:

• Predicting the emotion of the person in the image or the video and categorize it into
Angry, Disgust, Fear, Happy, Sad, Surprise and Neutral with an accuracy greater than
60% using Convolution Neural Network.

3. Machine Learning:
Machine Learning is the learning in which machine can learn by its own without being
explicitly programmed. It is an application of AI that provide system the ability to
automatically learn and improve from experience.
“Machine Learning is said to learn from experience E w.r.t some class of task T and a
performance measure P if learners performance at the task in the class as measured by
P improves with experiences.”

4. Training CNN model-Mini Xception:

Here comes the exciting architecture which is comparatively small and achieves almost
state-of-art performance of classifying emotion on this data-set. This architecture is
different from the most common CNN architecture Common architectures uses fully
connected layers at the end where most of parameters resides. Also, they use standard
convolutions. Modern CNN architectures such as Xception leverage from the combination
of two of the most successful experimental assumptions in CNNs: the use of residual
modules and depth-wise separable convolutions.

The architecture of the Mini Xception - CNN model is given as follows.

There are various techniques that can be kept in mind while building a deep neural
network and is applicable in most of the computer vision problems. Below are few of
those techniques which are used while training the CNN model below.

1. Data Augmentation : More data is generated using the training set by applying
transformations. It is required if the training set is not sufficient enough to learn
representation.

2. Kernel_regularizer : It allows to apply penalties on layer parameters during

optimization. These penalties are incorporated in the loss function that the
network optimizes.

3. Global Average Pooling : It reduces each feature map into a scalar value by taking
the average over all elements in the feature map.

4. Depthwise Separable Convolution : These convolutions are composed of two

different layers: depth-wise convolutions and point-wise convolutions. Depth-wise
separable convolutions reduces the computation with respect to the standard
convolutions by reducing the number of parameters.

5. Flow Chart:
6. Time Schedule:

Time Schedule
Milestones August September October November
Literature survey, Problem formulation
Building architecture and flow diagram
Cleaning data
Formulating code
Hyper parameter tuning

7. Expected Outcome:

Plot the Bar-Graph of emotions with their corresponding probabilities and show the
emotion with maximum probability as result.