Project Report on

“SENTIMENTAL ANALYSIS BY SPEECH VIDEO RECOGNITION USING

MACHINE LEARNING”

Project Report Submitted to

(Savitribai Phule Pune University)

In Partial Fulfillment of Requirement for the Award of

MASTERS OF BUSINESS ADMINISTRATION

By

SAYALI RAJARAM KOLTE

UNDER THE GUIDANCE OF

DR. ZAMARRUD ANSARI

SINHGAD INSTITUTE OF BUSINESS ADMINISTRATION AND RESEARCH,

KONDHWA (BK), PUNE.

2019-2021
 DECLARATION

I, the undersigned, hereby declare that the Dissertation Report “ SENTIMENTAL ANALYSIS BY SPEECH
VIDEO RECOGNITION USING MACHINE LEARNING” Written and submitted by me to SAVITRIBAI
PHULE PUNE UNIVERSITY, in partial fulfilment of the requirement for the award of MASTER OF
BUSINESS ADMINISTRATION is my original work and the conclusions drawn are based on the material
collected by myself.

Place: Pune

Date: Sayali Rajaram Kolte

 ACKNOWLEDGEMENT

The success of any task lies upon the efforts made by a person but it cannot be achieved without the
cooperation of others. I would like to thank Sinhgad Institute of Business Administration and Research,
Pune for giving me this opportunity. I take this opportunity to express my profound gratitude and deep
regard to my dissertation guide Prof.Zamarrud mam for her exemplary guidance, monitoring and constant
encouragement throughout the Dissertation Report.

Place: Pune

Date: Sayali Rajaram Kolte

 Contents

1 Introduction:

1.1 Introduction To Study…...............................................................................................................7

1.2 Statement of the Problem…...........................................................................................................8
1.3 Purpose of Study….......................................................................................................................8
1.4 Objective of Study.........................................................................................................................8
1.5 Significance of Study…................................................................................................................8
1.6 Scope of Study..............................................................................................................................9
1.7 Limitations of Study….................................................................................................................9

2 Conceptual Background:

2.1 User Interface..............................................................................................................................11

2.2 HardwareInterface.......................................................................................................................11
2.3 Software Interface.......................................................................................................................11
2.4 Communication Interface…........................................................................................................11
2.5 What is backend development….................................................................................................12
2.6 Existing System Architecture.....................................................................................................12
2.7 Proposed System Architecture....................................................................................................14
2.8 Dataflow Diagrams.....................................................................................................................15
2.9 UML Diagrams...........................................................................................................................17
2.10 Research Methodology..............................................................................................................23
2.11 Types of Research.....................................................................................................................24
2.12 Performance Requirement........................................................................................................25
2.13 Safety Requirements..................................................................................................................27
2.14 Security Requirements…..........................................................................................................28
2.15 Software Quality Attributes......................................................................................................28
2.16 System Requirements................................................................................................................28
3 Data Presentation,Analysis and Interpretations:

3.1 Data Source….............................................................................................................................31

3.2 Sampling….................................................................................................................................31
3.3 Data Analysis & Interpretations…..............................................................................................32
3.4 Results….....................................................................................................................................33

4 Findings,Suggestion and Conclusion:

4.1Findings........................................................................................................................................38
4.2 Conclusion...................................................................................................................................38
4.3 Suggestions..................................................................................................................................38

5 Bibliography
 CHAPTER I

INTRODUCTION
1.1 Introduction To Study

In a large proportion of these videos, people depict their opinions about products, movies, social issues,
political issues, etc. The capability of detecting the sentiment of the speaker in the video can serve two
basic functions: (i) it can enhance the retrieval of the particular video in question, thereby, increasing its
utility, and (ii) the combined sentiment of a large number of videos on a similar topic can help in
establishing the general sentiment. It is important to note that automatic sentiment detection using text is
a mature area of research, and significant attention has been given to product reviews , we focus our
attention on dual sentiment detection in videos based on audio and text analysis. We focus on videos
because the nature of speech in these videos is more natural and spontaneous which makes automatic
sentiment processing challenging. In Particular, automatic speech recognition (ASR) of natural audio
streams and text spoken in audio is difficult and the resulting transcripts are not very accurate. The
difficulty stems from a variety of factors including (i) noisy audio due to non-ideal recording conditions,
(ii) foreign accents,
(iii) spontaneous speech production, and (iv)diverse range of topics. Our approach towards sentiment
extraction uses two main systems, namely, automatic speech recognition (ASR) system and text-based
sentiment extraction system. For text based sentiment extraction, we propose a new method that uses
POS (part-of-speech) tagging to extract text features and Maximum Entropy modelling to predict the
polarity of the sentiments (positive or negative) using the text features. An important feature of our
method is the ability to identify the individual contributions of the text features towards sentiment
estimation.
We evaluate the proposed sentiment estimation on both publically available text databases and videos.
On the text datasets, This provides us with the capability of identifying key words/phrases within the
video that carry important information. By indexing these key words/phrases, retrieval systems can
enhance the ability of users to search for relevant information.
1.2 Statement of the Problem

There is unstructured, ungrammatical language. Since voice notes, words, sentences are not clear
sometimes. This issue can lead to un-usual analysis. The fact that the voice of a person is not always
correct. During fast speaking, user may have misspelled a particular word and make the analysis step
harder. The difficulty lies in the fact that there could be mixed opinions in a document, and with the
creative nature of natural languages, people may express the same opinion in vastly different ways,
sometimes without using any opinion words .How can voice of any person be automatically and
accurately classified with respect to their sentiment?

1.3 Purpose of Study

Sentiment analysis – otherwise known as opinion mining – is a much bandied about but often
misunderstood term. In essence, it is the process of determining the emotional tone behind a series
of words, used to gain an understanding of the the attitudes, opinions and emotions expressed within an
online mention.

1.4 Objective of Study

● To analyses the strengths and the limitations of systems based video and text
● To increase accuracy of sentiment analysis system
● To extract efficient information through speech
● To analyse the huge volume of data coming from video, such as homemade videos, audio reviews

1.5 Significance of Study

Sentiment analysis (SA) is a process of extensive exploration of data stored on the Web to identify and
categorize the views expressed in a part of the text. The intended outcome of this process is to assess the
author's attitude toward a particular topic, movie, product, etc. The result is positive, negative, or
neutral. These study illustrated different techniques in SA approach for extracting and analytics
sentiments associated with the polarity of positive, negative, or neutral on the topic selected. Social
networks SA can be a useful source of information and data.
1.6 Scope of Study

Admin have a train dataset. In that the video can upload to the user dashboard. Then user can select the
video from dataset and convert the video of audio and text and analysis of speech in video to give as
sentiments.

Sentiment analysis is target-oriented, aiming to identify opinions or attitudes towards topics or

entities(e.g., product, movie). Emotion recognition, on the other hand, focuses on recognizing either the
emotion expressed in text or evoked by the text, with no attachment to a specific target.

1.7 Limitations of Study

● The study is limited to few ratios because of non availability of videos.

● The study is used on available videos such as downloaded video in a local system.
● The reliability and accuracy of calculation depends more on information found in data set.
● The data is collected in two types:-
● Primary data- It will be collected with the help of available videos.
● Secondary data- Through the social media.
 CHAPTER II

CONCEPTUAL BACKGROUND
2.1 User Interfaces

User interact with the system using web application where user need to face the Login page to enter the
login credentials. If the user does not have the login credential or a first time user need to register to our
system by entering basic details like Name, address, contact number emailed and password for login.
After login they will get the respective options for interacting with the system and user gets option to
perform the basic functionality according to requirement.

• Front-end software: Java

• Back-end software: SQLYog

2.2 Hardware Interfaces

Since neither the mobile application nor the web portal have any designated hardware, it does not have
any direct hardware interfaces.

• Windows

• A browser which supports CSS, HTML JavaScript

2.3 Software Interfaces

System will used Java language to develop project. Eclipse platform is used. We used SQLYog in
backend. Tomcat server is used.

2.4 Communication Interfaces

The communication between the different parts of the system is important since they depend on each
other. However, in what way the communication is achieved is not important for the system and is
therefore handled by the underlying operating systems for the web application having internet
connection.
 Front-end :

• All about what a user able to see.

• It’s may be buttons, images, layout, input field etc.

• To design these components in WEB you need XML and JAVA programming.

• Therefore these programming are called front-end programming.

2.5 What is Back End Development?

Now that front end is ready, it likely needs data to display in one form or another. This data resides on
the back end, on a server that hosts user data. To create a server that communicates purposefully and
efficiently with the front end, many technolo- gies may be used. Some platforms include .Net, Java,
LAMP, Ruby, and Python.

2.6 Existing System Architecture

We assumed web based application which gives sentiment analysis of images,text only. Their was no
analysis of video .Also user was unable to see the visual results.
In our system there are four main modules:

(User):-
In this, User get registered with system and add images.

(Admin):-
Admin is responsible for adding user and managing other module.

(Upload image or text):-

Uploading the image or text

(Result):-
Displaying the results
 Upload
User
Video

Analyze the video

with audio and text

Show results
2.7 Proposed System Architecture

System is divided into four parts according to functions performed by individuals. In this system we
need to collect the data from different resources like videos, in- the system in three subsections:

(User):- User in system can get registered with system with unique id and password. After successfully
registration user can login with system and after get authentication by admin user can access system
and can upload the video.

(Admin):- Admin in this system is responsible for adding user authentication and managing other
modules like adding video and verify the video format.
2.8 Dataflow Diagrams

2.8.1 Data Flow Diagram Level 0

A data-flow diagram (DFD 0) is a way of representing a flow of a data of a process or a system.The

DFD also provides information about the outputs and inputs of each entity and the process itself.The
below figure shows the data flow diagram,the main module in this figure are video

Figure: Data Flow Diagram Level 0

2.8.2 Data Flow Diagram Level 1

A data-flow diagram (DFD 1) is a way of representing a flow of a data of a process or a system.The

DFD also provides information about the outputs and inputs of each entity and the process itself. The
main module in this figure are User, browse video,analysis of audio ,conversion to
text,classification,compare,display result.

Figure : Data Flow Diagram Level 1

2.9 UML Diagrams

2.9.1 Use Case Diagram

A use case diagram is a dynamic or behavior diagram in UML.Use case diagrams model the
functionality of a system using actors and use cases. Use cases are a set of actions, services, and
functions that the system needs to perform.Its shows the relationship between user database and
system.

Figure : Use Case Diagram

2.9.2 Sequence Diagram
Sequence Diagrams are time focus and they show the order of the interaction visually by using the
vertical axis of the diagram to represent time what messages are sent and when. The below figure
shows the sequence diagram with include various step like user,system,database,text mining,Analysis.

Figure: Sequence Diagram

2.9.3 E-R Diagram

An entity relationship diagram (ERD) shows the relationships of entity sets stored in a database. An
entity in this context is an object, a component of data. An entity set is a collection of similar entities.
These entities can have attributes that define its properties.

Figure : E-R Diagram

2.9.4 Class Diagram

In software engineering, a class diagram in the Unified Modeling Language (UML) is a type of static
structure diagram that describes the structure of a system by show- ing the system’s classes, their
attributes, operations (or methods), and the relationships among objects. Below figure shows class
diagram which detail of user, admin, database structure.

Figure: Class Diagram

2.9.5 Activity Diagram

Activity diagram is another important diagram in UML to describe the dynamic aspects of the system.
Activity diagram is basically a flowchart to represent the flow from one activity to another activity.
The activity can be described as an operation of the system. The control flow is drawn from one
operation to another .The Below figure shows various activity carried out in this sentiment recognition
project

Figure: Activity Diagram

2.9.6 Deployment Diagram

A deployment diagram is a UML diagram type that shows the execution architecture of a system,
including nodes such as hardware or software execution environments, and the middleware connecting
them. Deployment diagrams are typically used to visualize the physical hardware and software of a
system.

Figure : Deployment Diagram

2.10 Research Methodology
Software models are ways of expressing a software design. For object-oriented software, an object
modeling language such as UML is used to develop and express the software design. In this project, the
waterfall model is used for the purpose of development.

Figure : SDLC Model

As this model is very simple to understand and use so this model can be easily used in our project.
Hence, we have decided to use it for project implementation. In a waterfall model, each phase must be
completed fully before the next phase can begin. In our project, after completion of requirement , the
design phase will be started. This type of software development model is basically used for the project
which has clear requirement.

Using this Waterfall model, we will come to know at the end of each phase, a review takes place to
determine if the project is on the right path and whether or not to continue or discard the project. In this
model software testing starts only after the development is complete. In waterfall model phases do not
overlap. As this model has many advantages hence it can be used for our project.
2.11 Types of Research:
Research is broadly classified into these main categories :

● Basic Research
● Applied Research
● Qualitative Research
● Quantitative Research

Market research is systematic problem analysis model building and fact finding for the purpose of
important decision making and control in the marketing of good and service.

The research process can be said to be circular over a period of time. exploratory research may be define
hypothesis that are their tested verified by conclusive research but in the latter process the conclusive
research may develop new ideas opportunities of new difficulties.

With the ever increasing complexity of marketing and business activity, market research has also
increase complexity. Today carrying out research relating to customers products and market require
specialized skills and sophisticated techniques market research has emerged as highly specified function
of marketing management.

2.11.1 Basic Research:

Basic research is an investigation on basic principles and reasons for occurrence of a particular event or
process or phenomena is also called as theoretical research. Study or Investigation of some neutral
phenomena or relating to pure science are termed as basic research.

2.11.2 Applied Research:

In an applied research one solves certain problems employing well known and accepted theories and
principle. Most of the experimental research case studies and inter-disciplinary research are essentially
applied research.
2.12 Performance Requirements

1 .TITLE: Understanding Video quality

DESC: As we consider input to a system is a video so the type of video and quality of video is important to
convert analyse audio and convert it to text.

2 .TITLE: Filtering echo and audience voice in a video of a speaker.

DESC: The video contents other surrounding voice so we need to perform filtering that part of audio so
that result will be more accurate.

3 .TITLE: Keyword in text with different meanings.

DESC: After considering the text in a video they have different meaning ac- cording to different situation.

4 . TITLE: Good internet speed to process the video and audio to get better re- sult.
DESC: Depending on internet speed the background task to fetch the result of system may change.

2.12.1 Better Utilization of Our Budget

Since you are still in the process of developing and testing the app, you have not yet utilized your
expenses on app promotion and maintenance.

2.12.2 Saves Maintenance Effort

By the time you release your system to the consumers, you would have already had addressed the
prospective challenges. Tracking your system performance is the very first step to assess and validate it.
Below are some of the KPIs that are directly proportional to its usability.

2.12.3 Time to First Byte

If it takes more than 2-3 seconds to load your system, no matter how well designed it is, chances that the
consumers will abandon it will rise. Additionally, you will have to observe if the amount of data that
travels the network directly impacts the network bandwidth and consequently your system performance.
Additionally, you will have to observe if the amount of data that travels the network directly impacts the
network bandwidth and consequently your apps performance.

You can adopt the following measures to optimize network delivery:

• Compress the data sent and received over the network.

• Use flush early techniques to avoid overloading the backend.

• Tweak your server configuration.

• Monitor your database schema and configuration.

2.12.4 Average Time to Render Screen

The time your app screen takes to load the content, images, videos, animations on the mobile screen is
essential to mobile app performance.
Following are the factors that largely impact screen rendering:

• Improper screen dimensions

• Unscaled heavy images

• Inconsistent font sizes

You should be taking effective steps to rule out these possibilities and improve mobile rendering of your
app. If your app functions fast but take longer to render content on the screen, it is a negative
performance signal.

2.12.5 System Crash Rates

Crashes refer to shutting down of the system abruptly while the user is using it. If your app crashes on a
frequent basis, then it's time to take the desired steps to curb this. You should be troubleshooting crash
 scenarios on multiple devices and operating systems for measuring the following:
You should be troubleshooting crash scenarios on multiple devices and operating systems for measuring
the following:

• What % of users faced it?

• What is the stack trace of the crash?

• Why did the app crash? Was it because of a failed network/IO/API service request?

2.12.6 System Load Per period

This is a measure of the maximum your system can stretch to handle video type to input. This way you
can be well informed of the scenarios which your cus- tomers might encounter if they try to access
something different format than predefined. By checking on how video format your system can take so
that you know when you need to scale it.
To make sure your mobile app performance improves to withstand the extraload, you should consider
practicing:

• Thread management Managing multi-threading servers capable of handling multiple concurrent

transactions at a time
Task scheduling Schedule processing tasks effectively to reduce the burden on the CPU.

• I/O Scheduling Schedule I/O operations for efficient consumption of storage.

2.13 Safety Requirements

System registration is way to spam check for increasing the security. Hence, security is provided from
unwind phone should be maintained. Database should be properly maintained. Information transmission
should be securely transmitted to server without any changes in information. The main security concern
is for users account hence proper login mechanism should be used to avoid hacking. The table id
registration is way to spam check for increasing the security. Hence, security is provided from unwanted
use of recognition software.
2.14 Security Requirements

User should contain login ID and password. Applications designed with security in mind are safer than
those where security is an afterthought. Traditionally security issues are first considered during the
Design phase of the Software Development Life Cycle (SDLC) once the Software Requirements
Specification (SRS) has been frozen. Login credentials will provide security to application.
Authentication is main feature for system.

2.15 Software Quality Attributes

• Software functional quality reflects how well it complies with or conforms to a given design, based on
functional requirements or specifications. That attribute can also be described as the fitness for purpose
of a piece of software or how it compares to competitors in the marketplace as a worthwhile product.

• Software structural quality refers to how it meets non-functional requirements that support the delivery
of the functional requirements, such as robustness or maintainability, the degree to which the software
was produced correctly.

2.16 System Requirements

2.16.1 Database Requirements

We are using SQLYog database. SQLYog is customizable. The open-source GPL license allows
programmers to modify the SQLYog software. Project is having tables. It consists of registration table,
login table and history table etc

Use case name Save User and Admin Data

Priority High
Precondition Data are saved in database
 Basic path 1. Create database 2. Save information in
database
Alternative None
Path
Post condition None
Other None

2.16.2 Software Requirements

We are using Eclipse Oxygen and the latest version of Tomcat Server for storing the database.

Sr. Software Details(Technical details with

No. Component Purpose)
1 Operating System Windows 8 and 10
2 Technology Java Version 7 & above
3 Tool Eclipse oxygen
4 Server Tomcat Server
5 Database SQLYog
 2.16.3 Hardware Requirements

Android Phone:-

Sr.No. Component Details(Technical details with

Purpose)
1 Hardware Pentium
2 Speed 1.1 GHz
3 RAM 1GB
4 Hard Disk 20 GB
 CHAPTER III
DATA PRESENTATION, ANALYSIS AND
INTERPRETATION
3.1 Data Sources:
After identifying and defining the research problem and determining specific information required
solving the problem, the researcher's task is to look for the type of source of data, which may yield the
desired results. There are two types of data available to researcher, these are:-

□ Primary Data
□ Secondary Data

3.1.1 Primary Data:

Primary Data are generated when particular problem in hand is investigated by researcher employing a
mail questionnaire, telephone surveys, Personal interview etc

3.1.2 Secondary Data:

Secondary Data on the other hand includes that data which is collected from some earlier research work
and is applicable or usable in the study the researcher has presently undertaken.

3.2 Sampling :

Sample Unit:
Individual video is selected as a sample Unit. That also included the existing video of local system.

Sample Size:
The targeted sample size was 26 videos.

Sampling Technique:
Sample was selected at a random basis from the data set and it was approached by collecting different
videos from social media.
3.3 Data Analysis And Interpretation

In Data analysis and interpretation chapter research of primary data and secondary data it will help to
analyze the data correctly. The bar graph is based on the sentiments that was predicted from the video
.Each and every bar is presented in well manner in order to get proper idea and information.
3.4 Results

Figure: Home Page

Figure: User Registration

 Figure: Login

Figure: Upload Video

Figure : Analyse Video

Figure : Complete Video Analysis

Figure: View Result
 CHAPTER IV

FINDINGS, SUGGESTIONS AND CONCLUSION

4.1 FINDINGS

First and foremost, it saves time and effort because the process of sentiment extraction is fully automated
it’s the algorithm that analyses the sentiment analysis, and so human participation is sparse.

It counts the number of sentimental words that appear in a given video. If the number of happy,sad, angry
or appearances is greater than the number of other word appearances, the system returns that sentiment,
and vice versa.

4.2 Conclusion

Until now we were unable to carry out sentimental analysis of speech video with simultaneous text
appearance on screen. Hence, we are proposing a system for sentiment analysis through video and
text.The proposed system will be used to obtain the transcripts for the videos. Trained dataset will be
used for analyzing sentiment using classifier.

4.3 Suggestions
● In this ,in future we can use videos of the various formats.
● We can upload videos directly from any social media.
● It provide the facility to the user to display the sentiments analysis of the natural talk.
 CHAPTER V

BIBLIOGRAPHY
5 BIBLIOGRAPHY

[1] Maghilnan S,Rajesh Kumar M, Senior IEEE ,Member School of Electronic Engineering VIT
University Tamil Nadu, India , Sentimental Analysis on Speaker Specific Speech Data,International
Conference on Intelligent Com- puting and Control 2017

[2] Svetlana Kim, YongIk Yoon, Fusion Sentimental Analysis in Self- Growth Broadcasting, IEEE
International Conference on Big Data and Smart Com- puting 2018.

[3] Lakshmish Kaushik, Abhijeet Sangwan, John H L. Hansen, Sen- timent extraction from natural audio
streams, IEEE International Conference on Big Data and Smart Computing 2018.

[4] Aishwarya Murarka, Kajal Shivarkar, Sneha, Vani Gupta, Prof.Lata Sankpal, Sentiment Analysis of
Speech, International Journal of Advanced Research in Computer and Communication Engineering,
11, November 2017.

[5] Zhong-Qiu Wang1 and Ivan Tashev Learning utterance-level representations for speech emotion and
age/gender recognition. 2978-1-5090- 4117- 6/17/31.002017IEEEICASSP2017.

[6] Bin Wen , Shanrong Duan , Bin Rao , Wenhua Dai, Research on Word Sen- timental Classification
based on Transductive Learning, 8th International Symposium on Computational Intelligence and
Design2015.
[7] G, Castellano, L. Kessous, G. Caridakis, G Emotion recognition through multiple modalities: Face,
body gesture, speech, affect and emotion in hu- mancomputer interaction, In C. Peter amp; R. Beale
(Eds.), Lecture notes in computer science 4868 (pp. 92103). [Berlin Heidelberg: Springer. 2008

[8] M. Wollmer, F. Weninger, T. Knaup, B. Schuller, C. Sun, K. Sagae, L.P. Morency, Youtube movie
reviews: Sentiment analysis in an audiovisual context, Intell Erik Boiy, Marie-Francine Moens, A
Machine Learning Approach to Sentiment Analysis in Multilingual Web Texts, August 13, 2008.

[9] Ana Ramirez Lopez,Rahim Saeidi,Lauri Juavela ,Paavo Alku,Normal To- Shouted Speech Spectral
Mapping For Speaker Recognition Under Vocal Effort Mismatch,Department Of Signal Process+ing
and Acoustics School of Electrical Engg,Aalto University Finland IEEE@2017
[10] Nattapong Kurpukdee,Swait Kasuriya, Vataya Chunwijitra A Study of Support Vector Machines
for Emotional Speech Recognition,IC-ICTES IEEE @2017

[11] Anderw Rosenberge ,Kartik Audhkhasi ,Abhinav Sethy ,Bhuvana Ramab- handharan Michel
Pincheny,End-to-End speech recognition keyword search on low-resourced language,IEEE @201