Lab Report

Experiment No:

Experiment Name: Facial expression recognition using keras

Course code : CSE-316

Course Title : Artificial Intelligence Lab

Submitted to:

Md. Mahfujur Rahman

Lecturer,Department of CSE

Daffodil International University

Submitted by:
ID:192-15-2853 , ID:192-15-2864

ID:192-15-2875 , ID:192-15-2888

Section : PC-B

Department of CSE

Daffodil International University

Submission Date: 30/08/2021

1 . Introduction:
Our team members information:

Amran Ali (192-15-2853)

Md. Ismail Hossen Sarker (192-15-2864)

Md.Arafat Islam (192-15-2875)

Jannat Ara Jhuma (192-15-2888)

We are all 3rd-year BSC in CSE students on Design, use and

interaction. We all think AI as a field is very interesting and are looking
forward to having a lot of professional discussions about the topic
through our project work.

2. Abstract:
Keras is a very powerful open source Python library which is runs on
top of top of other open source machine libraries like TensorFlow,
Theano etc, used for developing and evaluating deep learning models
and leverages various optimization techniques. Features — Keras fully
supports recurrent neural networks and convolution neural networks
Keras runs smoothly on both CPU and GPU Keras NN are written in
Python which advocates simplicity and great debugging power Keras is
known for its incredibly expressive, flexible, minimal structure Keras is
consistent, simple and extensible API Keras is also known for its highly
computational scalability Extensive support for various platforms and
backends.
3. Objective:
Inspecting and manipulating the data Creating our own custom FER
model from scratch Applications of FER systems Introduction Facial
Emotion Recognition (commonly known as FER) is one of the most
researched field of computer vision till date and is still in continuous
evaluation and improvement. The idea is, can we detect the emotion a
person is having by his/her facial expressions? Of-course as a human
we can very easily do that and in-fact we do this thousands of times
every day. But can we make our so called dumb-machines intelligent
enough so they can also achieve human level performance on these
emotion recognizing task? See, because we humans have emotions
that’s why we are easily able to detect other’s emotions but that’s not
the case with machines. The advancements in the field of computer
vision are skyrocketing, these tasks are no more that difficult, at-least
we are able to achieve good performance with very less amount of
efforts. We will see how that can be done with few lines of python code.

4. Methodology:
In this paper, deep learning with the convolutional neural network
approach is used. The Keras Application Programming Interface and
OpenCV framework were used [12]. OpenCV is used for the automatic
detection of faces and drawing bounding boxes around them. OpenCV
consists of many pre- trained classifiers for face, eyes, smile etc [13-
16]. The model here is trained with the dataset used in the ICMR
(International Conference on Machine Learning). This dataset can be
taken from the Kaggle website. All the images of the dataset are of size
48*48. To select some images from every category, utility function can
be used [17-20].
5. Work Process and Methods:
The facial expressions datasets of different people. We need to check
the class imbalance problem before modeling and perform the data
augmentation and check which class has lesser data and provide more
of the data of the minority class. So, the number of images contained in
each section should be checked [21-23]. These command lines can be
used for this purpose, for expression in
os.listdir("train/"):print(str(len(os.listdir("train/" + expression))) + " " +
expression + " images")The training data should be set based on
expressions. Dataset is split into 80% training and 20% test. During
training, the loss of the neural network is minimized using the mini batch
gradient descent[24-25]. In the paper, two data generators are used.
One is for the training and the other is for testing.
6. Facial Expression & Emotion:
According to the basic definition of emotion by Ekman and Friesen,
emotions are divided into six classes, namely happy, sad, surprise, fear,
disgust, angry. In this paper talks about seven face detection classifiers
using open CV and one of its classiers for drawing the boundary box
around the face to detect the correct expression. For training the CNN
models we have used 48x48 grey–scale images from dataset. The FER
data set is divided into two folders called test and train, further divided
into separate folder each containing one of the seven types of FER data
base.

To build the training model, training and validation batches are

generated with the FER dataset image size 48x48 and batch size as 64
as per the memory size of CPU/GPU to speed up the training process.
ImageDataGenerator() class is used to accept values or images to treat
camera captured image as a horizontal mirror image. These images are
used to generate the training set. Test and training seds are generated
by keeping the various parameters same.

7. Conclusion:
Facial expression recognition is done using deep learning with the
convolutional neural network approach CNN and automatic detection of
faces and drawing bounding boxes around them are done using
different data set’s. The activation is done using the RELU activation
function in the work. The validation is done with the real time images
with different expression. The proposed model gives the correct
expression name using the trained CNN network
8. Reference:
[1] F. Ahmed, H. Bari, and E. Hossain. Person-independent facial
expression recognition based on compound local binary pattern (clbp).
Int.Arab J. Inf. Technol., 11(2):195–203, 2014.

[2] F. Bashar, A. Khan, F. Ahmed, and M. H. Kabir. Robust facial

expression recognition based on median ternary pattern (mtp).
InElectrical Information and Communication Technology (EICT),
2013International Conference on, pages 1–5. IEEE, 2014.

[3] P. Carcagn, M. Coco, M. Leo, and C. Distante. Facial expression

recognition and histograms of oriented gradients: a comprehensive
study. SpringerPlus, 4(1):1, 2015.

[4] C.-C. Chang and C.-J. Lin. Libsvm: a library for support vector
machines. ACM Transactions on Intelligent Systems and
Technology(TIST), 2(3):27, 2011.

[5] J. Chen, Z. Chen, Z. Chi, and H. Fu. Facial expression recognition

based on facial components detection and hog features. In
InternationalWorkshops on Electrical and Computer Engineering
Subfields, pages 884–888, 2014.

[6] J. de Andrade Fernandes, L. N. Matos, and M. G. dos Santos

Aragao. Geometrical approaches for facial expression recognition using
support vector machines. In Graphics, Patterns and Images
(SIBGRAPI), 201629th SIBGRAPI Conference on, pages 347–354.
IEEE, 2016.

[7] M. M. Donia, A. A. Youssif, and A. Hashad. Spontaneous facial

expression recognition based on histogram of oriented gradients
descriptor.Computer and Information Science, 7(3):31, 2014.
[8] P. Ekman. An argument for basic emotions. Cognition &
emotion,6(3-4):169–200, 1992.

[9] T. Gritti, C. Shan, V. Jeanne, and R. Braspenning. Local features

based facial expression recognition with face registration errors. In
AutomaticFace & Gesture Recognition, 2008. FG’08. 8th IEEE
InternationalConference on, pages 1–8. IEEE, 2008.

[10] A. A. Gunawan et al. Face expression detection on kinect using

active appearance model and fuzzy logic. Procedia Computer
Science,59:268–274, 2015.

[11] Murali Krishna, B., Siva Kumar, M., Rajesh, J., Inthiyaz, S.,
Mounica, J., Bhavani, M., Adidela, C.N.FPGA implementation by using
XBee transceiver2016 Indian Journal of Science and Technology9(17).

[12] Vijaya Lakshmi, A., Ghali, V.S., Muzammil Parvez, M., Chandra
Sekhar Yadav, G.V.P., Gopi Tilak, V.Fuzzy C-means clustering based
anomalies detection in quadratic frequency modulated thermal wave
imagingInternational Journal of Recent Technology and
Engineering8(3), pp. 4047-4051

[13] Muzammil Parvez, M., Shanmugam, J., Mohan Rao, K.R.R.,

Lakshmana, C., Shameem, S.Alive node and network lifetime analysis
of DEEC protocol and DEEC protocolJournal of Advanced Research in
Dynamical and Control Systems 10(6 Special Issue), pp. 1379-1384

[14] Vijayalakshmi, A., Ghali, V.S., Chandrasekhar Yadav, G.V.P., Gopi

Tilak, V., Muzammil Parvez, M.Machine learning based automatic
defect detection in non stationary thermal wave imagingARPN Journal
of Engineering and Applied Sciences 15(2), pp. 172-178

[15] Maddisetti L., Senapati R.K., Ravindra J.V.R. (2019), ‘Supervised

machine learning for training a neural network as 5:2 compressor’,
International Journal of Innovative Technology and Exploring
Engineering, 8(10), PP.2079-2084.

[16] Raja C., Balaji L. (2019), ‘An Automatic Detection of Blood Vessel
in Retinal Images Using Convolution Neural Network for Diabetic
Retinopathy Detection’, Pattern Recognition and Image Analysis, 29(3),
PP.533-545.

[17] Vijaya Prasad K., Kishore P.V.V., Srinivasa Rao O. (2019),

‘Skeleton based view invariant human action recognition using
convolutional neural networks’, International Journal of Recent
Technology and Engineering, 8(2), PP.4860-4867

[18] .Inthiyaz, S., Prasad, M.V.D., Usha Sri Lakshmi, R., Sri Sai, N.T.B.,
Kumar, Ahammad,S.H.Agriculture based plant leaf health assessment
tool: A deep learning perspective International Journal of Emerging
Trends in Engineering Research 7(11), pp. 690- 694

[19] Myla, S., Marella, S.T., Goud, A.S., Ahammad, S.H., Kumar,
G.N.S., Inthiyaz, SDesign decision taking system for student career
selection for accurate academic system , 2019 International Journal of
Scientific and Technology Research8(9), pp. 2199-2206

[20] Inthiyaz, S., Madhav, B.T.P., Kishore, P.V.V.Flower image

segmentation with PCA fused colored covariance and gabor texture
features based level sets2018.Ain Shams Engineering Journal9(4), pp.
3277-3291

[21] Prasad M.V.D., Lakshmamma B.J., Chandana A.H., Komali K.,

Manoja M.V.N., Kumar P.R., Prasad C.R., Inthiyaz S., Kiran P.S. (2018)
, ‘An efficient classification of flower images with convolutional neural
networks’,International Journal of Engineering and Technology(UAE),
7 (1.1),PP. 384- 391.
[22] Srinivasa Rao Y., Ravi Kumar G., Kesava Rao G. (2017),’A new
approach for classification of fault in transmission line with combination
of wavelet multi resolution analysis and neural Networks’,International
Journal of Power Electronics and Drive Systems,8(1),PP.505-512.

[23] Rahman M.Z.U., Reddy B.M.K. (2017),’Efficient sar image

segmentation using bias field estimation’,Journal of Scientific and
Industrial Research,76(6),PP.335-338.

[24] Sirisha B., Paidimarry C.S., Chandrasekhara Sastry A.S.

(2017),’Performance evaluation of distance metrics for terra SAR-X
image alignment’,Journal of Advanced Research in Dynamical and
Control Systems,9(5),PP.206-219.

[25] Suryanarayana, G., Dhuli, R. Super-Resolution Image

Reconstruction Using Dual-Mode Complex Diffusion-Based Shock
Filter and Singular Value Decomposition. Circuits Syst Signal Process
36, 3409–3425.

Thank You