Acknowledgement

I am extremely thankful to all my supporters whose input and feedback has made my project to turn
into a reality. This includes my college, Softwarica college of IT and e-commerce which has helped
me provide necessary environment to conduct an academic research for this thesis. I am thankful to
my module leader, Mr. Manoj Shrestha for supervising and motivating me throughout my thesis and
helping me with guidelines for research, implementation, and documentation. I have tried my best to
gather relevant document for this project. Furthermore, I don’t claim all the information presented
in this thesis are included perfectly. There may be shortcoming, factual errors, or mistakes in
implementation. Only I am responsible for those and I will try to give a better volume in future. Lastly,
I would like to show my gratitude to all my lecturers, friends, and family for supporting me
throughout this process.
Abstract
Psychological problems like depression, pessimism, anxiety, etc. are one of the major health issues in
this century (Scielosp, 2020) . An estimated 264 million people suffer from depression globally (WHO,
2020). An article published in American psychiatric association states $44 billion dollars are lost
because of workplace depression (Assosiation, 2020). Depression is not an untreatable condition
however, half of the employees are untreated (Assosiation, 2020). This number is only going up,
especially after the recent global pandemic. A survey was done among 3900 people who had a
previous case of covid-19 between May 2020 to January 2021. The study showed, 52% of them
suffered major depression (Reinberg, 2021). A more recent research that performed a meta-analysis
of multiple studies, that included 288,830 participants from multiple countries has found one in three
adults are experiencing symptom of depression because of covid-19 (Graciano, 2021). This gives an
idea of a serious issue that is coming. A proper solution should be considered to minimize the impact.
In the proposed system, an attempt to track emotional data of employees and usage of music as
therapy is being done. The system gets raw input data from webcam of computers and uses
convolutional neural networks to extract emotion data. The data is sent to a server and an
appropriate music is returned to user.
Keywords
Computer vision, Depression detection, Convolutional neural network, Facial feature extraction, Image
processing, Music as therapy

Figure 1: Word cloud

Table of Contents
1 introduction ..................................................................................................................................................................... 8
1.1 Aim ........................................................................................................................................................................... 10
1.2 Objective ................................................................................................................................................................ 10
1.2.1 Primary objectives ................................................................................................................................... 10
1.2.2 Secondary objectives............................................................................................................................... 10
1.3 Problem Statement ............................................................................................................................................ 11
1.4 Research question.............................................................................................................................................. 13
1.5 Background........................................................................................................................................................... 13
2 Ethical consideration and Data protection........................................................................................................ 13
2.1 Ethical consideration ........................................................................................................................................ 13
2.2 GDPR ........................................................................................................................................................................ 14
3 Literature review ......................................................................................................................................................... 16
3.1 Related works ...................................................................................................................................................... 16
4 Research methodology .............................................................................................................................................. 19
4.1 Data source ........................................................................................................................................................... 19
4.2 Primary source .................................................................................................................................................... 20
4.2.1 Focus group................................................................................................................................................. 20
4.2.2 Survey tools................................................................................................................................................. 21
4.3 Secondary sources ............................................................................................................................................. 22
4.4 Challenges ............................................................................................................................................................. 23
5 Data collection............................................................................................................................................................... 24
6 Application development ......................................................................................................................................... 28
6.1 Waterfall model .................................................................................................................................................. 28
6.2 Tools ........................................................................................................................................................................ 30
6.3 Technique .............................................................................................................................................................. 31
6.4 Use case diagram ................................................................................................................................................ 32
7 Technical specifications ............................................................................................................................................ 33
7.1 Security................................................................................................................................................................... 33
7.2 Usability ................................................................................................................................................................. 33
 7.3 Scalability .............................................................................................................................................................. 34
7.4 Automation testing ............................................................................................................................................ 35
7.5 Reliability............................................................................................................................................................... 36
8 Application architecture ........................................................................................................................................... 37
8.1 Integration of product ...................................................................................................................................... 39
9 Discussion ....................................................................................................................................................................... 40
10 PEST Analysis ........................................................................................................................................................... 41
11 Issue log ...................................................................................................................................................................... 43
12 Conclusion .................................................................................................................................................................. 43
13 future work ................................................................................................................................................................ 44
14 Bibliography .............................................................................................................................................................. 45
15 Appendix..................................................................................................................................................................... 48
15.1 Appendix A: Survey questionnaire ............................................................................................................. 48
15.2 Appendix B: System configuration .............................................................................................................. 51
15.3 Appendix C: Code ............................................................................................................................................... 66
15.4 Appendix D: User Interface ............................................................................................................................ 66
Table of Figures
Figure 1: Word cloud ............................................................................................................................................................. 4
Figure 2: Effect of music on brain ..................................................................................................................................... 9
Figure 3: Health report in late June 2021 .................................................................................................................... 11
Figure 4: Mental health status after Covid-19 ........................................................................................................... 12
Figure 5: Ethical Consideration ....................................................................................................................................... 14
Figure 6: GDPR implementation...................................................................................................................................... 15
Figure 7: The depression tracker by team of Dr Louis Philippe Morenci ...................................................... 16
Figure 8: Uses of wearable technologies in mental health data collection .................................................... 17
Figure 9: Results of research done by N.S. Parameswaran .................................................................................. 18
Figure 10: Research Approach ......................................................................................................................................... 19
Figure 11: Qualitative vs Quantitative .......................................................................................................................... 20
Figure 12: Focus group on survey .................................................................................................................................. 21
Figure 13: Survey tool used in this research .............................................................................................................. 21
Figure 14: Secondary data sources ................................................................................................................................ 22
Figure 15: Challenges faced during this research .................................................................................................... 23
Figure 16: People who find office stressful................................................................................................................. 25
Figure 17: People that have been affected by covid-19 ......................................................................................... 25
Figure 18: People affected mentally by covid-19 ..................................................................................................... 26
Figure 19: People who welcome an idea of AI based mental health detector .............................................. 26
Figure 20: Suggestions for improvement .................................................................................................................... 27
Figure 21: Names of people involved in survey ........................................................................................................ 27
Figure 22: Waterfall model vs Agile............................................................................................................................... 29
Figure 23: Tools used in development of this system ............................................................................................ 30
Figure 24: Technologies used in the development of system ............................................................................. 32
Figure 25: Use case diagram of our application........................................................................................................ 32
Figure 26: Usability of proposed system ..................................................................................................................... 34
Figure 27: Scalability of proposed system .................................................................................................................. 35
Figure 28: Testing of the system ..................................................................................................................................... 36
Figure 29: Reliability of the system ............................................................................................................................... 36
Figure 30: NodeJS architecture ........................................................................................................................................ 37
Figure 31: Initiating the process ..................................................................................................................................... 39
Figure 32: Emotional analysis being performed ...................................................................................................... 39
Figure 33: Music is played when sadness count reaches 10 ............................................................................... 40
Figure 34: PEST analysis .................................................................................................................................................... 42
Figure 35: Issue log ............................................................................................................................................................... 43
1 INTRODUCTION
“Depression is a prison where you are both suffering prisoner and the cruel jailer”

- Dr Dorothy Rowe, Australian Psychologist

-

The mortality statistics show human life expectancy has increased all-time high in last century. While
the more famous killers like malaria, Polio, tuberculosis have been almost tackled by humanity, a new
killer with name of mental illness has been on rise (Scielosp, 2020). More than 264 million people
were suffering from a variation of depression worldwide in 2020. (WHO, 2020) Anxiety and
depression are two of the common reasons for various diseases globally. In developing countries like
Nepal, their condition is yet to be evaluated (Ajay Risal, 2016).

The modern life is extraordinarily easier than past. An average person does not have to fear predators
and live the hardship of highly physical life of past. While the mankind has moved on past these with
rise of technology, the wiring in the brain is still prevalent. Anxiety is a natural emotion that is
triggered in the condition of “Fight or Flight” (interestingengineering, 2020). The modern life is
complex, and many factors can lead to this condition being triggered. The factors can be anything like
Financial issues, Relationship issues, Family issues, Health issues or other reasons. Depression is the
leading reason for disability in the world (WHO, 2020). The modern world runs on money and
businesses, industries, tech firms, etc. play a crucial role in economy. These businesses thrive when
the people working there are satisfied. The stressful environment dealing with myriad of timelines
and schedules can be too stressing sometimes (Assosiation, 2020). Depression can lead to issues like
lower work performance, employee satisfaction, absenteeism, or low engagement, absenteeism, or
employees skipping the work (Assosiation, 2020). It can also be life threatening. However, it is not
untreatable. It is important to find out the suspect and help them to right direction. Depression can
be challenging to figure out as some symptoms are subtle and the patient themselves are not aware
of it. Although the typical symptoms like sadness and hopelessness are easy to find, things like forced
happiness, Fatigue and Lack of concentration are easy to figure out (medicalnewstoday, 2019). A
research has found, music can be used as a therapy to improve depressive tendencies (Aalbers, 2017).
So, we aim to create a system that tracks user activities and applies music as therapy.

The proposed system tries to tackle this by monitoring the employees periodically. The system makes
use of Convolutional neural network to process raw video/image data and send the response to
server. The server sends back a music according to mood. This system uses music as therapy to
prevent mental health issues from getting worse.
Figure 2: Effect of music on brain
1.1 Aim
The aim of this project is to create an appropriate solution that can be used in workplaces to battle
mental illness. It should be able to effectively track depressive tendencies among employees and
make use of music as therapy to help them.

1.2 Objective
1.2.1 PRIMARY OBJECTIVES
• Create a real time mental health tracker software.
• Understand effect of mental health in workplaces and ways to tackle it.
• Improve portability matrix by choosing dataset model with smallest memory footprint.
• Make use of music as therapy to prevent depressive tendencies among employees.
• Help managers figure out condition of their employees.
• Investigate the usage of artificial intelligence to solve complex human problems.
• Collect data about how usage of AI can change workplace environment and total productivity.
• Design a pleasant user interface with easy to use visualization tool.
• Obtain bachelor’s degree in computing.

1.2.2 SECONDARY OBJECTIVES

• Study cost effectiveness of this solution.
• Gather data about the accuracy of dataset used to process images in proposed system.
• To estimate effect of music as therapy as a treatment.
• To ensure privacy issues are effectively solved.
1.3 Problem Statement
While the rise of technology has helped mankind achieve new feats every day, there are still some
responses hardcoded by nature into our brain. One of the important natural emotion is anxiety.
Anxiety is a condition that is triggered during the condition of “Fight or Flight”. This was an important
survival instinct and saved human from predators or dangers (interestingengineering, 2020). The
modern life is not highly physical as of past and It is not as simple as well. There are many factors
that may put the person in a situation of “Fight or Flight” despite predators being nowhere near.
These factors may be financial issues, relationships, social issues , work or other reasons.

According to WHO, Depression is a leading cause of disability in the world (WHO, 2020). Depression
can lead to issues like lower work performance, employee satisfaction, absenteeism, or low
engagement, absenteeism, or employees skipping the work (Assosiation, 2020). Despite it being a
notorious killer, it is not incurable. The tricky part about finding the depression is finding out. It does
not have a fixed reason. It can occur to anyone from children to adults for some reasons or no reason
at all (Schimelpfening, 2020). According to the national alliance on mental illness, 46% of people
committing suicides are suffering from some case of depression (nami, 2019). This issue has been
increasing since the recent pandemic. According to centres for disease control and prevention
(cdc.gov, 2020), a whopping 40% of U.S. adults reported struggling with mental health or substance
use.

Figure 3: Health report in late June 2021

The fear of being sick, unemployment and being isolated inside the house for long period of time is
likely to play the roles here. In another study, it was found 18% of covid-19 patients got a type of
mental issue (ries, 2020). That is almost a person per five people. This risk was doubled within three
months as compared to healthy people. Almost six percent of adults suffering from covid-19
developed a mental health issue, compared to 3.4 percent of adults without case of covid-19 (ries,
2020).

Figure 4: Mental health status after Covid-19

From these reports, we can understand depression is a major issue currently and it is affecting many
people. At the top of being a life-threatening condition, it can also affect in professional works and
productivity (Davis, 2003). A necessary measure must be taken in order to battle this hindrance.
1.4 Research question

How can convolutional neural network be used to create a mental health detection system?

1.5 Background
The stress, deadlines, long hours and deskbound nature of modern work culture is one the main
reason people are destroying their health. Arounds 86% of American workers sit all day at work
(Giang, 2014). Another major issue that is uprising in the last century is Depression.

According to WHO, Depression is a leading cause of disability in the world (WHO, 2020). Depression
affects more than 17 million American adults every year (healthline, 2021). Surprisingly, as notorious
depression really is It is not an incurable disease. It can be diagnosed if found in earlier phases.

2 ETHICAL CONSIDERATION AND DATA PROTECTION

2.1 Ethical consideration

Ethical considerations guide the way when collecting research data. A research done with respect to
ethics is the best for society. So, Ethical issues are considered seriously in this paper. The ethical
concerns maintained here are as follows

• No personal data is collected by server. All the processing of images is done in client side. The
user source of emotion data is untraceable and is used only once to send back music. The data
passes through an encrypted connection. No logs are kept so User Privacy can be guaranteed.
• It is made sure people are well informed about the objective of research honestly and interviews
were carried out only if permission was given.
• The copyrights, patents, etc of sources are respected. All the journals, papers, etc. are well
referenced using Harvard-Coventry references.
• The research can lead to getting an unpopular opinion here and there. Even if the opinions
sounded offensive to us, the opinion was respected. The research was not biased in any ways.
• The people involved in research are from various cultural background. Hence, a cultural
difference is likely to occur anytime. A proper filtering of data is our social responsibility to
prevent people from going on rampage.

Figure 5: Ethical Consideration

2.2 GDPR
General data protection regulation (GDPR) is a regulation created in 2016 to protect European
citizens. It makes sure businesses protect the personal data of all the European citizens and respect
their privacy. This is mandatory to all the businesses and failure to comply can cost dearly (Nadeau,
2020).

Businesses have been continuously evolving and making digital transition past decade. They have
been using different services and large quantity of data regularly. However, a regulating body was
not present so GDPR was created. Since its inception, it has been quite successful. Big corporations
like Google, British Airways, Facebook were fined when they did not follow (eset, 2020). So, It has
been quite a success so far. Some principles set by GDPR are:
• The lawfulness, fairness and transparency states businesses should ask for consent first and
lawfully get the data of EU citizens. The data should not be sold to third party without consent
from owner.
• The personal limitation states individuals should be informed about all the reasons for data
collection.
• The data collection states businesses should not collect any data more than required. The
collected data must be justified and well documented.
• The accuracy states the data should always be up to date.
• The Storage limitations makes sure data is deleted after a certain period. This period should be
justified and well documented.
• The Integrity and confidentiality states businesses should implement a anonymisation system
should be implemented to protect user data from attackers.

Figure 6: GDPR implementation

3 LITERATURE REVIEW

3.1 Related works

The national institute of mental health estimates that 40 million of US population experienced a kind
of mental issue in 2015. Among them, 16 million has had at least one case of major depressive
episode. This leads to more than 30% of total population being affected from mental health issues
eventually (saleh, 2017). These has been few related works that tried to address this situation in the
past with usage of neural networks. They are presented here.

In 2017, Dr louis Philippe Morenci and his team on collaboration with McLean hospital studied on a
possible use of Convolutional neural network in depressive patients (saleh, 2017). The system uses
facial landmark detection technique to get position of 68 facial landmarks of the face. These muscle
change can be tracked with these landmarks and finally predict the emotion user is showing. The
early study was done with 500 volunteers who were patients and regular people. The people were
asked to fill a set of questionnaires before and after interacting with the system. Among those 500,
15-20% showed symptoms of depression and same range for PTSD. The study contained all the types
of population which represented real world. The eventual goal of the application was to be used at
screening however, initially it was used to track patient mood during treatment. According to Dr
louis, the goal of this system was never to treat depression as that is the job of doctor. The calibration
was not perfect in the beginning, but they managed to get 78% correlation with time. This is a
important improvement as limited people were used.

Figure 7: The depression tracker by team of Dr Louis Philippe Morenci

A study paper published by team of Enrique Garcia-Ceja deals with the usage of smartphones and
wearable technology for mental health monitoring (Ceja, 2018). This survey varies from previous one
as it presents an overview of how various devices can be used for a wide variety of mental condition
than focusing on particular states. It makes use of multimodal sensing devices to get wide number
of data. Previously, researcher had used a supervised neural network with a predefined state. This
resulted into a small group of emotion data. The new method collected everything unsupervised. The
unsupervised clustering allowed possibility of finding new expressions and states. The study also
made use of other wearable technologies like smartwatch and bands (Ceja, 2018).. The ability of
capturing real time data of mental health patients introduced a new possibility to treat people much
efficiently.

Figure 8: Uses of wearable technologies in mental health data collection

A paper published by Namboodiri Sandhya Parameswaran (Parameswaram, 2019) made the use of
SVM classifier to train the feature data extracted using Gabor filter bank. The author experimented
with five different student multiple times and performed an experimental analysis. The experiment
resulted into following output
 Figure 9: Results of research done by N.S. Parameswaran

Out of five different videos, first and second video resulted into matching actual and predicted result.
The fourth video was satisfactory as it showed High depression for actual mild depression. The third
and fifth however wrong. The overall system accuracy was 64.34%. This system was used in schools
to find student with depressive tendencies. A teacher or class monitor was informed accordingly.

Another research done by J. Singh used similar approach to decode depressive disorder with the use
of computer vision. A study was conducted between group of four hundred volunteers. Among the
total of 254 males and 147 females, The proposed system was able to attain 74 percent identification
success (Jaiteg Singh, 2020).
4 RESEARCH METHODOLOGY
Both the quantitative and qualitative analysis has been used to address the research objective of this
paper. Primary data has been used for quantitative analysis and Secondary data has been used for
qualitative analysis. The survey was used for primary data collection and literatures, past papers,
journals, etc were used for secondary data collection.

Figure 10: Research Approach

4.1 Data source

The thesis needed a great amount of data to address the research question properly. Both the primary
and secondary data are properly used. Among qualitative and quantitative analysis, we have
specifically chosen to use quantitative approach. The qualitative approach takes data that can be
biased, inaccurate or highly variable depending on authors. Quantitative approach deals with data
that can be measured hence, they are more factual and consistent. There are other differences among
them that can be seen below
 Figure 11: Qualitative vs Quantitative

4.2 Primary source

The data which are collected by researcher directly is called primary data. The sources to get primary
data can be anything like observations, surveys, questionnaires, interviews, etc. Primary source
provides first-hand evidence and are more credible as evidence. The major advantage of primary
sources over secondary is the data can be transposed into numbers. This gives them an ability to be
presented as statistics. A survey was conducted using google forms. The survey included
questionnaires needed for thesis statement. The usage of google form made a lot easier for us because
ease of access encouraged more people to participate. It was also particularly helpful at the current
case of global pandemic.

The collected data can be represented in any graph, charts and calculated statistical data like mean,
median or average. For the survey several participants are given questionnaires with multiple choice
questions and some optional written questions. The close ended MCQs however, are mandatory.
These questions allow us to get exact data for our research question.

4.2.1 FOCUS GROUP

The group of people selected in this survey were mostly fulltime office workers and few worked
parttime. It was easier to find a focus group as recent pandemic has affected almost everyone. The
participants also vary on age, gender and location. The have some experience of working during the
pandemic and can effectively provide an output for us.
 Figure 12: Focus group on survey

4.2.2 SURVEY TOOLS

An online tool called google forms was used to carry out the survey. It is an online service provided
by google free of cost. Since It is a cloud-based solution, a large area of people can be included in
research. Sharing the questionnaire also becomes an easy task as links can be generated. This has
especially helped as people were anxious to come for the on-field survey because of global pandemic.
Google form also provides an automatic visualization tool that shows data in pi charts. This enables
us to perform quantitative analysis faster.

Figure 13: Survey tool used in this research

4.3 Secondary sources
Secondary sources are the data created by individual who was not present at the event. It may include
past researches, journals, newspapers, government surveys and statistics, articles, etc. Actually, the
data from international newspapers and research papers published in various online sites were used
quite a lot here. Internet is a great source for secondary data. However, there are few problems with
secondary sources. Research topic are always selected trying to make it as unique as possible. This
may result to issues finding exact survey data needed for the cause. The ideas and data collected from
secondary sources were saved and referenced to topic.

Figure 14: Secondary data sources

4.4 Challenges
Several issues occurred during the period of whole research process. These challenges resulted to
wrong findings, missing the deadline, etc. The challenges that were encountered are stated and
explained below:

• The topic was quite unique, and the research was based on recent pandemic so, there was limited
resources online. so, most of the research took more time than necessary. This resulted into
issues with time management and deadlines were difficult to meet.
• The methodology played crucial part in data collection. As data collection is the major part of this
research, we had to be careful to select appropriate methodology.
• Finding relevant information was difficult because of ongoing pandemic. Even though internet
has helped tremendously, it was difficult to get access to books, papers, etc.
• It was difficult to conduct find people for survey because of pandemic. Most of the working people
around me were unfamiliar with google sheets so I had to attach a tutorial for them. Initially,
finding variation in people for research was troublesome.

Figure 15: Challenges faced during this research

How do we solve the issues?
As these challenges started to be more frequent, a way to solve these has to be thought. A time
management tool called Gantt chart was used to divide the tasks into smaller sub tasks. Time
management was recorded to be way easier after usage of it. Similarly, the e-library provided by
college was used to get digital versions of relevant books. The ability to find any book in mere
seconds was better than searching books in library. A number of research papers from sites like
researchgate were used. The issue with finding proper population for our survey was solved with
usage of google forms. Since, people could just fill a form and be done with it compared to more
traditional travel to survey field, the number increased. This resulted in better quality of survey.

5 DATA COLLECTION
A number of office workers were presented with a list of questionnaires and the responses were
recorded. Total of 30 people participated during this process. Initially, a huge list with multiple
questions was presented but people engagement was low. So, we had to remove the unnecessary
questions. The participants increased after removing those questions. Participants were presented
with following questions.

• Is working at office stressful?

• Have you been affected by covid-19?
• Has covid-19 affected your mental health directly or indirectly?
• How likely would you welcome the idea of an AI based mental health detector in your office for
your personal use? (Nobody gets the access to your photo/video, not even your employer)
• Suggestions for improvement? (Optional)
• Name (optional)

The questions that did not matter much for research were made optional. All the data entered by user
are represented by google in appropriate charts like bar graph and pie-chart. The results of our
survey are explained below
 Figure 16: People who find office stressful

It is found that a humongous 76.7% of total people find working at office stressful. This is a huge
problem since modern humans spend most of their life at work. The answer to this question can be
quite difficult as work can sometime be lovely and sometime difficult. To address this, we allowed
people an ability to choose both yes and no if wanted. However, people still picked yes.

Figure 17: People that have been affected by covid -19

Among 30 people present, 93.3% of them had a case of covid-19 in the past. We can be sure these
people have experienced the issues we are trying to address in this paper. The resulting 36.7% of
people were not affected by it.
 Figure 18: People affected mentally by covid-19

Despite only 63.3% people being affected from covid-19, 90% of the total reported to being affected
mentally because of pandemic directly or indirectly. This is a important data as it shows not only
people with cases of covid-19 are in danger from workplace depression.

Figure 19: People who welcome an idea of AI based mental health detector
When asked how people like the idea of installing an AI based mental detector that only tracks the
emotion of users, there were mixed responses. 10% of them discarded the idea, 20% of them
welcomed the idea but were not too sure. However, Majority (70%) of people were very likely to
allow installation. This should be a good data to instigate the process.

Figure 20: Suggestions for improvement

An optional field asking for suggestions was added too but only 4 of the response was recorded. They
wanted the system to run faster and improve technical factors. There was not any complain about
trust issues. However, we should still take ethical issues seriously and provide a most secure system.

Figure 21: Names of people involved in survey

Similarly, another optional field recorded the names of people that took survey. Only 6 people were
comfortable to share name.

6 APPLICATION DEVELOPMENT
A web application was developed as a final product. The application is easily accessible with the help
of browser as it is powered by JavaScript. The application allows user to record their face with
webcam in real time. Users are shown their emotion data on screen and they can get some idea too.
The application also shows music according to mood of users. The application tries to use music as
therapy to cure the mood of user by this process.

6.1 Waterfall model

A process of utilizing a sequential model that divides the software development lifecycle to multiple
phases is known by waterfall model (Guru99, 2021). Each phase must be completed before going to
another phase in this model. It consists of different phases like requirement analysis, design, coding
implementation, testing, deploy and maintenance. Once we go to next step, it is impossible to revert
back to fix errors. Hence, it has had many problems. This model has been decreasing in use after
popularities of agile methodology (ReQtest, 2019). However, there are still certain use cases where
waterfall model has several advantages. In projects where requirements are not changing and
application is quite small, it can be used. Agile is an overkill for small projects like this. Since the
requirement of our project was set at the beginning and application is quite simple, we can easily use
waterfall model.
 Figure 22: Waterfall model vs Agile

Steps followed in the development process of this application are as follows:

Analyse the requirements: All the requirements necessary for the system were considered in this
phase. A suitable programming language, database and development environment is chosen in this
phase.

Designing the system: An overall system architecture taking consideration of requirements from
previous phase is designed in this phase. The application architecture is designed in this step with
tools such as class diagram, entity diagram, use-case diagram, etc.

Implementation: This phase covers all the implementation part. The coding takes place and the
running status of application is confirmed.

Testing: After the implementation phase, the functionality of software is almost completed.
However, there are likely to be some bugs and issues. Each unit of the application are tested for faults
and failures.

Deployment: The product is deployed after functional and non-functional testing in this phase. It is
then provided to the client. This is the one of the crucial phases as customer gets access to the
working software after this.

Maintenance: Even though the software is provided to client, some bugs or security vulnerabilities
might occur in future. This phase tries to solve those issues with patches and updates.
6.2 Tools
Visual studio code: Visual studio is a lightweight text editor that can turn into a full fletched IDE
with plugins. It also provides some key features like debugger and version control by default. It is
also the most popular text editor currently (hackr.io, 2021). The installation and configuration can
be found at appendix.

Microsoft office: Microsoft office is one of the most popular suites of office and productivity
software. We have utilized the MS word for writing the paper and MS project for creating a time
management Gantt chart.

Postman: Postman is a software used to test APIs. It makes the developer workflow easier by
minimizing the tedious process of writing the tests in code. The collection feature can be used to
create an abstract test endpoint and customized as needed.

Git and Github: Git is the most popular version control system in the world and Github is an online
platform providing hosting, management, and collaboration between developers. Both of them are
free to use and are made used on tremendously in the project.

Google Chrome: Google chrome is the most popular browser used currently (statcounter, 2021). It
is developed by google and based on chromium browser. All the JavaScript codes has been checked
using this browser.

Canva: Canva is a popular cloud-based design tool. It offers a free version with some limitations. The
infographics and charts necessary for this project are made with the use of it.

Figma: Figma is another cloud-based design tool. It was used for custom designs and images.

Figure 23: Tools used in development of this system

6.3 Technique
According to the research, a web application that can record an input video and show emotion data
with some music need to be created in the end. The classification of the images is done on client side
/ in browser. This was possible because JavaScript powers this application. The backend was
developed using express.js framework.

• NodeJS: Nodejs is a cross platform and opensource JavaScript runtime environment. It is

powered by v8 engine of chrome in the core. This framework is used to run JavaScript in the
server. Nodejs is a highly performant framework so, it was chosen compared to other
frameworks like Laravel or CodeIgniter.
• Express.js: Expressjs is a framework built on top of NodeJS. It is used to make designing and
developing REST APIs easily. It is used for generating the backend for our application.
• MongoDB: MongoDB is a high performant open source database solution. It is based on
NoSQL technology and is popular for its scalability and flexibility.
• FaceAPI.js: Faceapi.js is JavaScript framework built on top of TensorFlow.js library. It can be
used to create CNN powered applications using JavaScript.

Node.js is a programming runtime that runs asynchronously, making it faster than traditional
languages like PHP. Node.js also includes nice package called npm. Npm package enables user to get
access to npm repository and install any libraries as required. Nodejs is quite lightweight and
requires only few modules to run. This was not the cases for previous frameworks like Laravel. The
asynchronous nature of NodeJS enables the software to run multiple operation at once. This
scalability is the reason the application is developed on NodeJS.

There are various frameworks like Express, Nest, Meteor, etc for development of backend using
NodeJS. However, express is the most popular among them. and rightly so, Express provides a
minimal syntax and easy to understand architecture. On top of that, It also provides a nifty tool called
express generator.

MongoDB was chosen as the preferred database as it is a highly performant database solution. It is
a NoSQL database and highly scalable as file based noSQL database are faster to read and write as
SQL based database. The system fills the gap of a highly required solution, so users are likely to
increase exponentially. Therefore, the industry standard combination of NodeJS and MongoDB is
used.
 Figure 24: Technologies used in the development of system

6.4 Use case diagram

Figure 25: Use case diagram of our application

7 TECHNICAL SPECIFICATIONS

7.1 Security
Despite NodeJS being a highly performant framework, it is still new. It was developed by Ryan Dahl
in 2009. This makes it quite prone to issues occasionally. Well, none of them are too threatening as
NodeJS has been used by likes of LinkedIn, Netflix, Uber, etc. The good news is any major issue can
be fixed with some package updates. Since our application deals with important data like emotion, it
is necessary to prepare a secure and robust solution. We have used following tools to make this
system secure and robust.

Authentication: A proper login is required to access the application. Following tools are used to
achieve a robust authentication system.

Bcrypt.js: Bcrypt.js is a JavaScript library that provides the application with ability to hash
passwords to create their salts. This protects the user from rainbow table attacks. It can also be used
to verify the password with hash from database. This is one of the most secure solution to store
passwords.

CORS: The default browser function is to avoid people get access to file from outside the domain. This
can be solved with a NodeJS package called CORS. This allows our backend to be separate from our
frontend. The isolation between backend and frontend is one of the reasons NodeJS is preferred over
other frameworks.

Regular updates are necessary to protect the system. This could be tedious process if it was for
others. Utilizing npm, we can update all the out of date packages at one and keep the system secure.

Using these packages, the security of application was ensured. The usage of bcrypt ensure all the
sensitive data are encrypted and regular update solves breaking changes and package security issues.

7.2 Usability
Despite our system being pretty small visually, it is not difficult to make it look messy. A good design
always attracts the users. A careful selection of colour palette is to be implemented and
complementary fonts are to be used. Most of the application is hardcoded and checked so errors are
very unlikely to occur. A proper DOM manipulation can yield highly performant applications. All the
codes are commented properly making sure that even non-technical person can understand the flow
of data.
 Figure 26: Usability of proposed system

7.3 Scalability
The application fills the demand of a major problem solution. The pandemic especially has increased
this applications potential exponentially. Hence, the application is to be developed considering the
scalability factor of it. The usage of NodeJS which uses non-blocking I/O and asynchronous operation
solves the performance issue of the server. Even though server is performant, using NoSQL instead
of traditional SQL could leverage faster performance. MongoDB also offers a cloud solution called
MongoDB atlas. Using this cloud solution ensures the data are not compromised in future. The Pay as
you go pricing model allows the system to always work performantly.
 Figure 27: Scalability of proposed system

7.4 Automation testing

The process of using tools to generate testing report automatically is called automated testing.
Automated testing provides better documentation that human counterpart in most of the cases and
are cheaper to operate after initial setup. There are two kind of tests that can be performed, black
box and white box testing. The black box test is performed in our case to compare expected result
with actual result and pass the test. Jest and Supertest are used in testing of Schema and routes for
the backend. A temporary mongo-shelf server is used to create a temporary server. This prevents the
data in actual MongoDB be tampered during the tests. The frontend of the application is a single page,
so an automated testing was not as necessary.
 Figure 28: Testing of the system

7.5 Reliability
The most important aspect of any application is reliability. Nobody wants to use a software that is
going to break on them down the road. This is especially true for a software like ours which deals
with developing a solution for a mental health issue. The application makes use of modern NodeJS
ecosystem and uses MongoDB for database. MongoDB is a industry proven database for highly
performant systems. The libraries and modules used in the application were chosen carefully such
that no breaking changes occur in future.

Figure 29: Reliability of the system

8 APPLICATION ARCHITECTURE
NodeJS makes use of single threading event loop to achieve concurrency in operation. It is based on
event-based model. The requests are sent to event loop which are sent to thread pool and event pool
accordingly.

Figure 30: NodeJS architecture

Figure 3: Proposed architecture of depression management system
8.1 Integration of product
The frontend of the application is hosted in codesandbox.

Figure 31: Initiating the process

Users are presented with a camera screen. The faces are tracked and a small rectangular box shows
emotion data.

Figure 32: Emotional analysis being performed

The application displays the rectangle according to mood and shows values too. The faces are tracked
every second and if a person is sad for more than 10 secs, they are presented with a motivating music.
 Figure 33: Music is played when sadness count reaches 10

The music stops playing when user is seen happy again.

9 DISCUSSION

One of the main reasons why workers are going to choose this solution is it does all the processing
client side. This prevents any anxiety users may have about turning the camera on most of the time.

The system makes use of Convolutional neural network to process raw video/image data and send
the response to server. The server sends back a music according to mood. This system uses music as
therapy to prevent mental health issues from getting worse.
10 PEST ANALYSIS
The PEST analysis was used to organize, evaluate, and track multiple factors that can impact the
overall process at present or in future. It can help us discover opportunities and threats due to
Political, Economic, Social and Technological factors (GroupMap, 2021).

The application was made completely following cyber laws and policies to ensure no legal issues
occur at past. Artificial intelligence is a new technology and many governments are yet to amend laws
on this topic. Our system, however, does all the processing in client side and only emotion data are
sent to server. The emotion data do not contain any personal information and they are not kept in
logs. Another advantage of processing client side is a huge server is not necessary anymore. This
solves the possible economic issues. A social analysis found; people were quite welcoming about the
application. The questionnaire for this is included in the appendix at the bottom. Finally, a technical
analysis was performed, and it was revealed that there is a low probability for a new technology to
evolve.
Figure 34: PEST analysis
11 ISSUE LOG

Figure 35: Issue log

12 CONCLUSION
The study done in this paper was done with image data in variable time. Some were taken after a few
hours’ session of writing paper and some at the morning. The presence of Angry, Disgust were
prominent in the photos after intensive work. The Happy and Neutral faces were prominent. The
free, portable and pretrained models provided by faceAPI-js were used for image processing. These
models were originally made for Keras. however, it has been converted to a JavaScript supported
.json file by good folks of internet. These models are trained with utilization of RAFDB, a huge
collection of diverse facial expressions. This is a huge training set and should be sufficient for general
use. If accuracy is to be improved in future, A custom set of local image data can be use to train the
model using Microsoft FERPlus. At first, implementation of this system seemed too difficult however
after a through research and hours of hard work, a prototype was possible. Learning from others
research paper has helped a lot to understand underlying process of the Convolutional neural
network, training the models with dataset, etc. The classifier was quite reliable and found the correct
reason most of the time, especially for angry and happy faces.
Depression is a complex human condition and different people show different emotion. This system
can only show results based on physical attributes associated with depression. To get actual idea of
the situation, other activities of employees such as work efficiency, attendance, etc are to be
considered. Though music as therapy is performed by the system, a regular sport session or a
refreshment would help too. The goal is to loosen up employees and keep them distracted from
stressful tasks while keeping the productivity same. By taking consideration of above activities, the
mental health of employees can be tracked, and necessary measures can be taken to solve it
effectively.

13 FUTURE WORK
After completing the through research and development of the system, there are numbers of
improvements that can be made. Some of them are:

• The people who answered the questionnaire during the research were friends of my age. Though
more than a dozen of people participated, only a tiny group of age was considered. This can be
improved by asking the questionnaires again to diverse age group.
• The dataset used in training the used machine learning model was not local. hence, a lot of faces
were of foreign people. Though this can be fine for most of the cases, it can be improved by usage
of a custom dataset of local faces.
• The application uses the most portable model, tiny face detector and runs totally on client side.
This system should run without any problems in desktop systems. However, Problems may arise
on mobile devices. So, native apps can be made in future to address this issue.
• The functionality of the system is done but UI/UX aspect were not prioritized in this iteration.
This can be solved in future versions with help of graphic designers.
• This system is highly versatile and can run anywhere. Currently, we are considering using it in a
workplace, but these can be implemented in other institutions like schools, hostels, etc with few
tweaks.
• There are limited, royalty free music used in the system currently. Despite them being shown
according to mood, they are not optimized for a usage of music as therapy. A custom-made
collection of music can do a much better job.
14 BIBLIOGRAPHY
Aalbers, S., 2017. [Online]
Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6486188

Ajay Risal, K. M., 2016. [Online]

Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4831105/

Assosiation, A. P., 2020. Depression: A Costly Condition for Businesses. [Online]

Available at: http://workplacementalhealth.org/Mental-Health-Topics/Depression

cdc.gov, 2020. [Online]

Available at: https://www.cdc.gov/mmwr/volumes/69/wr/mm6932a1.htm

Ceja, E. G., 2018. Mental health monitoring with multimodal sensing and machine learning: A survey.
[Online]
Available at: https://www.sciencedirect.com/science/article/pii/S1574119217305692#b23

Davis, J. L., 2003. Depression Affects Work Productivity. [Online]

Available at: https://www.webmd.com/depression/news/20030617/depression-affects-work-
productivity

eset, w. b., 2020. Two years later, has GDPR fulfilled its promise?. [Online]
Available at: https://www.welivesecurity.com/2020/05/25/two-years-later-has-gdpr-fulfilled-its-
promise/

Giang, V., 2014. [Online]

Available at: https://www.businessinsider.com/ways-your-office-job-is-killing-you-2014-
1#sitting-all-day-could-shave-years-off-your-life-1

Graciano, F., 2021. world economic forum. [Online]

Available at: https://www.weforum.org/agenda/2021/02/covid19-mental-health-depression-
anxiety/

GroupMap, 2021. [Online]

Available at: https://www.groupmap.com/portfolio/pest-analysis/

Guru99, 2021. What is Waterfall Model in SDLC? Advantages & Disadvantages. [Online]
Available at: https://www.guru99.com/what-is-sdlc-or-waterfall-model.html

hackr.io, 2021. Best Web Development IDE. [Online]

Available at: https://hackr.io/blog/web-development-ide

healthline, 2021. Work Depression: How to Take Care of Your Mental Health on the Job. [Online]
Available at: https://www.healthline.com/health/depression/work-depression
interestingengineering, 2020. [Online]
Available at: https://interestingengineering.com/can-technology-cause-anxiety-and-depression

Jaiteg Singh, G. G., 2020. Decoding depressive disorder using computer vision. [Online]
Available at: https://link.springer.com/article/10.1007/s11042-020-10128-9

medicalnewstoday, 2019. [Online]

Available at: https://www.medicalnewstoday.com/articles/325513

Nadeau, M., 2020. General Data Protection Regulation (GDPR): What you need to know to stay
compliant. [Online]
Available at: https://www.csoonline.com/article/3202771/general-data-protection-regulation-
gdpr-requirements-deadlines-and-facts.html
[Accessed 2021].

nami, 2019. [Online]

Available at: https://www.nami.org/About-Mental-Illness/Common-with-Mental-Illness/Risk-of-
Suicide

Parameswaram, N., 2019. A computer vision based image processing system for depression detection
among students for counseling. [Online]
Available at:
https://www.researchgate.net/publication/331147769_A_computer_vision_based_image_processi
ng_system_for_depression_detection_among_students_for_counseling

Reinberg, S., 2021. webmd. [Online]

Available at: https://www.webmd.com/lung/news/20210316/half-of-covid-survivors-struggle-
with-depression#1

ReQtest, 2019. Waterfall Methodology is Dead? Long Live Waterfall!. [Online]

Available at: https://reqtest.com/agile-blog/what-is-waterfall-methodology/

ries, J., 2020. healthline. [Online]

Available at: https://www.healthline.com/health-news/people-with-covid-19-more-likely-to-
develop-depression-anxiety-and-dementia

saleh, M., 2017. Computer Vision Applications in Mental Health: An Interview with Dr. LP Morency.
[Online]
Available at: https://www.medgadget.com/2017/06/computer-vision-applications-mental-health-
interview-dr-lp-morency.html

Schimelpfening, N., 2020. [Online]

Available at: https://www.verywellmind.com/depression-facts-you-should-know-1067617
Scielosp, 2020. [Online]
Available at: https://www.scielosp.org/article/bwho/2000.v78n4/411-411/
[Accessed 2021].

scielosp, 2020. scielosp. [Online]

Available at: https://www.scielosp.org/article/bwho/2000.v78n4/411-411/

statcounter, 2021. Browser rankings corresponding to market share. [Online]

Available at: https://gs.statcounter.com/browser-market-share

WHO, 2020. [Online]

Available at: https://www.who.int/news-room/fact-sheets/detail/depression

WHO, 2020. [Online]

Available at: https://www.who.int/news-room/fact-sheets/detail/depression

WHO, 2020. WHO. [Online]

Available at: https://www.who.int/teams/mental-health-and-substance-use/mental-health-in-the-
workplace
15 APPENDIX

15.1 Appendix A: Survey questionnaire

15.2 Appendix B: System configuration

Node.js installation:

The latest and stable version of Node.js is downloaded from official website. It is available here:

https://nodejs.org/en/download/
Installation of mongoDB database:

Installation file for MongoDB community version was downloaded from

https://www.mongodb.com/download-center/community
MongoDB compass was installed as follows
Configuring Express.js
Other modules necessary for our backend are added:

The dependencies required for the backend and frontend are Bcrypt.js, Body-parser, Cors, Json web
token (JWT), Mongoose, Morgan, Multer and validator.

Visual studio code installation:

Download Visual Studio Code - Mac, Linux, Windows

Following steps are taken to install vs code:
15.3 Appendix C: Code

The frontend code is available at https://codesandbox.io/s/motivation-bot-forked-

dfssp?file=/src/TinyFaceDetectWebcam.html

It can also be found at

15.4 Appendix D: User Interface