Emotion Based Music Player System
Salvina Desai Vrushali Paul
Information Technology Information Technology
Datta Meghe College of Engineering Datta Meghe College of Engineering
Navi Mumbai Navi Mumbai
salvinadesai12@gmail.com vrushalipaul02@gmail.com
Sayali Jadhav
Information Technology
Datta Meghe College of Engineering
Navi Mumbai
sayalijadhav8291@gmail.com
Abstract— This paper proposes an intelligent agent that
sorts a music collection based on the emotions conveyed
by each song and then suggests an appropriate playlist to
the user based on his or her current mood. The user's
local music collection is initially clustered based on the
emotion the song conveys, i.e., the mood of the song. This
is calculated taking into consideration the lyrics of the
song, as well as the melody. Every time the user wishes to
generate a mood-based playlist, the user takes a picture
of themselves at that instant. This image is subjected to
facial detection and emotion recognition techniques,
recognising the emotion of the user. The music that best
matches this emotion is then recommended to the user as
a playlist. Face emotion recognition is a form of image
processing. Facial emotion recognition is the process of
converting the movements of a person's face into a
digital database using various image processing
techniques. Facial Emotion Recognition recognises the
face's emotions. Recently, a number of new technologies
for facial expression recognition have been developed.
This technique, with an equal contribution from AI, can
revolutionise the robotics industry. Human-robot
interaction will be taken to a whole new level when
robots can understand human emotions and play music
based on them. Index Terms: music classification,
emotion recognition, intelligent music player.
I INTRODUCTION
Everyday, each and every person undergoes a lot of
troubles and the reliever of all the stress that is encountered
is Music. Music plays a very important role in strengthening
and the life of the individual because it is an important
medium entertainment for music lovers and listeners. In
today's world, with ever-increasing progress in the field
multimedia and technology, various music players were
developed with functions like fast forward, reverse, variable
playback speed (time search and compression), local
playback, streaming playback with multicast streams and
incl volume modulation, genre classification, etc. Although
these functions satisfy the basic requirements of the user, yet
the user has to face the task of manually scrolling through
the playlist songs and choosing songs based on his current
mood and behaviour.
Pranita Gode
Information Technology
Datta Meghe College of Engineering
Navi Mumbai
pranitagode19@gmail
Imagine yourself in a world where humans interact with
computers. You are sitting in front of your personal
computer that can listen, talk, or even scream aloud. It has
the ability to gather information about you and interact with
you through special techniques like facial recognition,
speech recognition, etc. It can even understand your
emotions at the touch of the mouse. It verifies your identity,
feels your presence, and starts interacting with you . The
vital part of hearing the song has to be in a facilitated way,
that is the player is able to play the song in accordance to the
person’s mood. The paper proposes emotion-based music
player is a approach that helps automatically play songs
based on the user's emotions. Emotion recognition is an
aspect of artificial intelligence that is becoming increasingly
relevant, for the purpose of automating various processes
that are relatively more tedious to perform manually. The
human face is an important organ of the human body,
especially playing an important role in extracting a person's
behaviour and emotional state. Identifying a person’s state
of mind based on emotions they display is an important part
of making efficient automated decisions best suited to the
person in question, for a variety of applications. One
important aspect of this would be in the entertainment field,
for the purpose of providing recommendations to a person
based on their current mood. We study this from the
perspective of providing a person with customised music
recommendations based on their state of mind, as detected
from their facial expressions. Most music connoisseurs have
extensive music collections that are often sorted only based
on parameters such as artist, album, genre and no. of times
played. However, this often leaves the users with the
arduous task of making mood based playlists, i.e. sorting the
music based on the emotion conveyed by the songs
something that is much more essential to the listening
experience than it often appears to be. This task only
increases in complexity with larger music collections, and
automating the process would save many users the effort
spent in doing the same manually, while improving their
overall experience and allowing for a better enjoyment of
the music. it recognizes the emotion on the user's face and
plays songs according to that emotion. 
 II RELATED WORK
Charles Darwin is the first scientist to recognize that facial
expression is one of the most powerful and immediate
means for human being to communicate their emotions,
intentions and opinions to each other. Rosalind Picard
(1997) describes why emotions are important to the
computing community.   recognition based on the geometric features of a face is
There are two aspects of affective computing: giving the
computer the ability to detect emotions and giving the
computer the ability to express emotions. Not only are
emotions crucial for rational decision making as Picard
describes, but emotion detection is an important step to an
adaptive computer system. An adaptive, smart computer
system has been driving our efforts to detect a person’s
emotional state. An important element of incorporating
emotion into computing is for productivity for a computer
user. A study (Dryer & Horowitz, 1997) has shown that
people with personalities that are similar or complement
each other collaborate well. Dryer (1999) has also shown
that people view their computer as having a personality. For
these reasons, it is important to develop computers which
can work well with its user.
 In the year 2011 Ligang Zhang and Dian Tjondronegoro
developed a facial emotion recognition system (FER) they
used dynamic 3D Gabor feature approach and obtained the
highest correct recognition rate (CRR) on the JAFFE
database and FER is among the top performers on the Cohn-
Kanade (CK) database using above approach. They testified
the effectiveness of the proposed approach through
recognition performance, computational time, and
comparison with the state-of the art performance. And
concluded that patch-based Gabor features show a better
performance over point based Gabor features in terms of
extracting regional features, keeping the position
information, achieving a better recognition performance, and
requiring a less number.
Till 2010 as per the survey done by R A Patil, Vineet Sahula
and A. S. Mandal CEERI Pilani on expression recognition,
The problem is divided into three sub problems face
detection, feature extraction and facial expression
classification. Most of the existing systems assume that the
presence of the face in a scene is ensured. Most of the
systems deal with only feature extraction and classification,
as- suming that face is already detected.
The algorithms which we have selected for detecting
emotions are from the papers, Image Edge Detection
Algorithm Based on Improved Canny Operator of 2013 and
Rapid Object Detection using a Boosted Cascade of Simple
Features by Viola and Jones. In Image Edge Detection
Algorithm Based on Improved Canny Operator paper
improved canny edge detection algorithm is proposed.
Because, the traditional canny algorithm has difficulty in
treating images which contain the salt & pepper noise, and it
does not have the adaptive ability in the variance of the
Gaussian filtering. For this reason, a new canny algorithm is
presented in this paper, in which open-close filtering instead
of Gaussian filtering. In this paper, the traditional canny
operator is improved by using morphology filtering to
preterit the noise image. The final edge image can reduce
effectively the influence of noise, keep the edge strength and
more complete details, get a more satisfactory subjective
result. And by using objective evaluation standards,
compared with the traditional Canny operator, information
entropy, average gradient, peak signal to noise ratio,
correlation coefficient and the distortion degree also have
increased significantly. So, the new algorithm is an effective
and practical method of edge detection. Automatic face
recognition is all about extracting those meaningful features
from an image, putting them into a useful representation and
performing some kind of classification on them.  Face
probably the most intuitive approach to face recognition.
METHODOLOGY
There are several methods for handling the implementation
and the consequent conversion from the old to the new
computerized system.
The most secure method for conversion from the old system
to the new system is to run the old and new system in
parallel. In this approach, a person may operate in the
manual older processing system as well as start operating
the new computerized system. This method offers high
security, because even if there is a flaw in the computerized
system, we can depend upon the manual system. However,
the cost for maintaining two systems in parallel is very high.
This outweighs its benefits.
Another commonly method is a direct cut over from the
existing manual system to the computerized system. The
change may be within a week or within a day. There are no
parallel activities. However, there is no remedy in case of a
problem. This strategy requires careful planning.
A working version of the system can also be implemented in
one part of the organization and the personnel will be
piloting the system and changes can be made as and when
required. But this method is less preferable due to the loss of
entirety of the system.
A. User Emotion Recognition
For this model, the Cohn Kanade extended dataset is used. A
python script is used to fetch images containing faces along
with their emotional descriptor values. The images are
contrast enhanced by contrast limiting adaptive histogram
equalization and converted to grayscale in order to maintain
uniformity and increase effectiveness of the classifiers. A
cascade classifier, trained with face images is used for face
detection where the image is split into fixed-size windows.
Each window is passed to the cascade classifiers and is
accepted if it is passes through all the classifiers, otherwise
it is rejected. The detected faces are then used to train the
face, which works on reduction of variance between classes.
Fisher face recognition method proves to be efficient as it
works better with additional features such as spectacles and
facial hair. It is also relatively invariant to lighting. A picture
is taken during the runtime of the application, which after
preprocessing, is predicted to belong to one of the emotion
classes by the fisher face classifier. The model also permits
the user to customize the model in order to reduce the
variance within the classes further, initially or periodically,
such that only variance would be that of emotion change
B. Music Recommendation
The emotion detected from the image processing is given as
input to the clusters of music to select a specific cluster. In
order to avoid the interfacing with a music app or music
modules which would involve extra installation, the support
from the operating system is used instead to play the music
file. The playlist selected by the clusters are played by
creating a subprocess, as the forked subprocess returns the
control back to the python script on the completion of its
execution, so that other songs can be played. This makes the
program play music on any system regardless of its music
player.

C. Haar cascade classifiers

The core basis for Haar classifier object detection is the

Haar-like features. These features, rather than using the
intensity values of a pixel, use the change in contrast values
between adjacent rectangular groups of pixels. The contrast
variances between the pixel groups are used to determine
relative light and dark areas. Two or three adjacent groups
With a relative contrast variance form a Haar-like feature.
Haar-like features are used to detect an image. Haar features
can easily be scaled by increasing or decreasing the size of
the pixel group being examined. This allows features to be
used to detect objects of various sizes.

D. Implementation plan
The implementation plan includes a description of all the
activities that must occur to implement the new system and
to put it into operation. It identifies the personnel
responsible for the activities and prepares a time chart for
implementing the system. The implementation plan consists
of the following steps.
List all files required for implementation. Identify all data
required to build new files during the implementation. List
all new documents and procedures that go into the new
system.
The implementation plan should anticipate possible
problems and must be able to deal with them. The usual
problems may be missing documents; mixed data formats
between current and files, errors in data translation, missing
data etc.
E.DFD
A data flow diagram (DFD) is a graphical representation of
the flow of data through an information system. A data
flowdiagram can also be used for the visualization of data
processing (structured design). It is common practice for a
designer to draw a context-level DFD first which shows the
interaction between the system and outside entities. This
context-level DFD is then exploded to show more detail of
the system being modeled.  recognition. You should also do things like removing the
Symbols:
The four components of a data flow diagram (DFD) are:
External Entities/Terminators are outside of the system
being modeled. Terminators represent where information
comes from and where it goes. In designing a system, we
have no idea about what these terminators do or how they do
it.
Processes modify the inputs in the process of generating the
outputs comes to rest. A DFD does not say anything about
the relative timing of the processes, so a data store might be
a place to accumulate data over a year for the annual
accounting process.
III . DESIGN
Now that you have detected a face, you can use that face
image for Face Recognition. However, if you tried to simply
perform face recognition directly on a normal photo image,
you will probably get less than 10% accuracy! 
It is extremely important to apply various image pre-
processing techniques to standardize the images that you
supply to a face recognition system. Most face recognition
algorithms are extremely sensitive to lighting conditions, so
that if it was trained to recognize a person when they are in a
dark room, it probably won’t recognize them in a bright
room, etc. This problem is referred to as "lumination
dependent", and there are also many other issues, such as the
face should also be in a very consistent position within the
images (such as the eyes being in the same pixel
coordinates), consistent size, rotation angle, hair and
makeup, emotion (smiling, angry, etc), position of lights (to
the left or above, etc). This is why it is so important to use a
good image preprocessing filters before applying face
pixels around the face that aren't used, such as with an
elliptical mask to only show the inner face region, not the
hair and image background, since they change more than the
face does For simplicity, the face recognition system I will
show you is Eigen faces using greyscale images. So I will
show you how to easily convert color images to greyscale
(also called 'grayscale'), and then easily apply Histogram
Equalization as a very simple method of automatically
standardizing the brightness and contrast of your facial
images. For better results, you could use color face
recognition (ideally with color histogram fitting in HSV or
another color space instead of RGB), or apply more
processing stages such as edge enhancement, contour
detection, motion detection, etc. Also, this code is resizing
images to a standard size, but this might change the aspect
ratio of the face.
 IV. FUTURE WORK 

The future scope in the system would to design a mechanism

that would be helpful in music therapy treatment and
provide the music therapist the help needed to treat the
patients suffering from disorders like mental stress, anxiety,
acute depression and trauma. The proposed system also
tends to avoid in future the unpredictable results produced in
extreme bad light conditions and very poor camera
resolution. Extend to detect more facial features, gestures
and other emotional states (stress level, lie detection,
etc.)Facial recognition can be used for authentication
purpose. Android Development. Can detect sleepy mood
while driving. Can be used to determine mood of physically
challenged & mentally challenged people.

REFERENCES

[1] Adolf, F. How-to builds a cascade of boosted classifiers

based on Haar-like features.
http://robotik.inflomatik.info/other/opencv/OpenCV_Object
Detection_HowTo.pdf, June 20 2003.
[2] Bradski, G. Computer vision face tracking for use in a
perceptual user interface. Intel Technology Journal, 2nd
Quarter, 1998.
[3] Cristinacce, D. and Cootes, T. Facial feature detection
using AdaBoost with shape constraints. British Machine
Vision Conference, 2003.
[4] The Facial Recognition Technology (FERET) Database.
National Institute of Standards and Technology, 2003.
http://www.itl.nist.gov/iad/humanid/feret
[5] Lienhart, R. and Maydt, J. An extended set of Haar-like
features for rapid object detection. IEEE ICIP 2002, Vol. 1,
pp. 900-903, Sep. 2002
[6] Menezes, P., Barreto, J.C. and Dias, J. Face tracking
based on Haar-like features and Eigen faces. 5th IFAC
Symposium on Intelligent Autonomous Vehicles, Lisbon,
Portugal, July 5-7, 2004.
[7] Muller, N., Magaia, L. and Herbst B.M. Singular value
decomposition, Eigen faces, and 3D reconstructions. SIAM
Review, Vol. 46 Issue 3, pp. 518–545. Dec. 2004.
[8] Open Computer Vision Library Reference Manual.
Intel Corporation, USA, 2001.
[9] Viola, P. and Jones, M. Rapid object detection using
boosted cascade of simple features. IEEE Conference
on Computer Vision and Pattern Recognition, 2001.