You are on page 1of 39

A PRELIMINARY PROJECT

REPORT ON

STOCK PREDICTION USING TECHNICAL AND


SENTIMENTAL ANALYSIS
SUBMITTED TO THE SAVITRIBAI PHULE PUNE UNIVERSITY, PUNE
IN THE PARTIAL FULFILLMENT FOR THE AWARD OF THE DEGREE
OF

BACHELOR OF ENGINEERING
IN
INFORMATION TECHNOLOGY
BY

DIGVIJAY SHARAD CHOUDHARI 71700796M

AJINKYA ANIL DHOTRE 71700833K

APURV SANDEEP PIMPLE 71700713J

UNDER THE GUIDANCE OF


Dr. A. M. BAGADE

DEPARTMENT OF INFORMATION TECHNOLOGY


PUNE INSTITUTE OF COMPUTER TECHNOLOGY
Sr No. 27, Pune Satara Road, Dhankawadi, Pune - 411 043
2019-2020
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CERTIFICATE
This is to certify that the preliminary project report entitled

“STOCK PREDICTION USING TECHNICAL AND


SENTIMENTAL ANALYSIS”
Submitted by

Digvijay Sharad Choudhari 71700796M

Ajinkya Anil Dhotre 71700833K

Apurv Sandeep Pimple 71700713J

is a record of bonafide work carried out by him under the supervision and guidance of Dr.
A. M. Bagade in partial fulfillment of the requirement of Savitribai Phule Pune University
for the award of the Degree of Bachelor of Engineering (Information Technology).
This project report has not been earlier submitted to any other Institute or University for
the award of any degree or diploma.

Dr. A. M. Bagade Dr. Anant Madhukar Bagade


Internal Guide Head of Department
Department of Information Technology Department of Information Technology

External Examiner Dr. P. T. Kulkarni


Date: Principal
P. I.C.T., Pune

Place: PICT, Pune.


Date:

PICT - INFORMATION TECHNOLOGY - 2019-20


2
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

ACKNOWLEDGEMENT

The creation of this report required a lot of guidance from many people and it is
always a pleasure to remember them. We are grateful to our institute for making available
all the resources necessary for research for my seminar topic. We would also like to thank
all the people who have been contributory in successful completion of this report.
We would like to thank our Internal Guide Dr. A. M. Bagade for guiding us at
each and every step of the project since it’s commencement. We would also like to thank
our External Guides, Mr. S. S. Pande and Mr. T. A. Rane for analyzing our work and
providing us with their valuable inputs throughout the project.
We would like to extend our gratitude towards all the staff members of the IT
department for their constant support and encouragement. Also, our deepest gratitude
towards Dr. A. M. Bagade, Head of Information Technology Department, PICT and
Dr. P. T. Kulkarni, Principal, PICT for providing us with this wonderful opportunity to
extend our knowledge and explore new horizons.
Finally we appreciate all our fellow colleagues and group members who have
worked hard to the best of their abilities to do research for this seminar. Being with them
and working with them has truly showed us the importance of teamwork and they have
always motivated us to carry on.

Digvijay Sharad Choudhari

Ajinkya Anil Dhotre

Apurv Sandeep Pimple

PICT - INFORMATION TECHNOLOGY - 2019-20


3
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

LIST OF FIGURES

Figure No. Name Page No.

Fig1 System Architecture 19

Fig2 Data Flow Diagram 20

Fig3 Use Case Diagram 21

Fig4 Activity Diagram 22

Fig5 Sequence Diagram 23

Fig6 State Diagram 24

Fig7 Class Diagram 25

Fig8 Component Diagram 26

Fig9 Steps for Technical 29


Analysis
Fig10 Steps for Sentimental 30
Analysis

Fig11 Example Image 34

PICT - INFORMATION TECHNOLOGY - 2019-20


4
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

LIST OF ABBREVIATIONS

Abbreviation Full Form


ROC Rate of Change

MA Moving Average

FE Feature Extraction

SMO Social Media Optimization

MKL Multiple Kernel Learning

SVR Support Vector Regression

Forex Foreign Exchange Market

NSE National Stock Exchange

PICT - INFORMATION TECHNOLOGY - 2019-20


5
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CONTENTS

CERTIFICATE 2

ACKNOWLEDGEMENT 3

LIST OF FIGURES 4

LIST OF ABBREVIATIONS 5

CONTENTS 6

ABSTRACT 8

INTRODUCTION 9
1.1 BACKGROUND 9
1.2 RELEVANCE 9
1.3 PROJECT UNDERTAKEN 9
1.4 ORGANIZATION OF PROJECT REPORT 10

LITERATURE REVIEW 11
2.1 EXISTING METHODOLOGIES 12
2.2 PROPOSED METHODOLOGIES 14

REQUIREMENT SPECIFICATION AND ANALYSIS 15


3.1 PROBLEM DEFINITION 15
3.2 CONCEPT 15
3.3 SCOPE 15
3.3 OBJECTIVE 16
3.4 PROJECT REQUIREMENTS 16
3.4.1 DATASETS 16
3.4.2 FUNCTIONAL REQUIREMENTS 16
3.4.3 NON-FUNCTIONAL REQUIREMENTS 17
3.4.4 HARDWARE REQUIREMENTS 17
3.4.5 SOFTWARE REQUIREMENTS 17
3.5 PROJECT PLAN 17

PICT - INFORMATION TECHNOLOGY - 2019-20


6
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

3.5.1 PROJECT RESOURCES 17


3.5.2 PROJECT PLAN 3.0 18
SYSTEM ANALYSIS AND DESIGN 19
4.1 ARCHITECTURE 19
4.2 DFD 20
4.3 BEHAVIORAL DIAGRAMS 21
4.3.1 USE CASE DIAGRAM 21
4.3.2 ACTIVITY DIAGRAM 22
4.3.3 SEQUENCE DIAGRAM 23
4.3.4 STATE DIAGRAM 24
4.3.4 CLASS DIAGRAM 25
4.3.5 COMPONENT DIAGRAM 26
4 .4 ALGORITHM AND METHODOLOGIES 27

IMPLEMENTATION 29
5.1 STAGES OF IMPLEMENTATION 29
5.1.1 TECHNICAL ANALYSIS 29
5.1.2 SENTIMENTAL ANALYSIS 30
5.2 IMPLEMENTATION TECHNIQUES 31

EVALUATION METHODS 33
6.1 EVALUATION METHODS 33
6.1.1 EXAMPLE SET OF FEATURES 33
6.1.2 EXAMPLE SET OF MODELS 33
6.2 EXAMPLE 34

CONCLUSIONS 35
7.1 CONCLUSION 35
7.2 LIMITATIONS 35
7.3 FUTURE SCOPE 35

REFERENCES 36

APPENDICES 38
A] BASE PAPER(S) 38
B] PLAGIARISM REPORT 39

PICT - INFORMATION TECHNOLOGY - 2019-20


7
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

ABSTRACT

Predicting stock market movements is a well-known problem of interest. Now-a-


days social media is perfectly representing the public sentiment and opinion about current
events. Especially, Twitter has attracted a lot of attention from researchers for studying
the public sentiments. Stock market prediction on the basis of public sentiments
expressed on Twitter has been an intriguing field of research. The approach through
sentimental analysis is to observe how well the changes in stock prices i.e. the rise and
fall are correlated to the opinion of people that are expressed by them on Twitter.

Sentimental analysis helps in analyzing the public sentiments on Twitter, this


approach is our approach through using make of sentimental analysis. Another approach
in the same topic of our project is using technical analysis. We model the stock price
movement as a function of these input features and solve it as a regression problem in a
multiple kernel learning regression framework. The machine learning coupled with
fundamental and/ or technical analysis also yields satisfactory results for stock market
prediction. We also evaluated the model for taking buy-sell decision at the end of day
which is also known as intraday trading.

PICT - INFORMATION TECHNOLOGY - 2019-20


8
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 1

INTRODUCTION

1.1 BACKGROUND

Sentiment analysis is contextual mining of text which identifies and extracts subjective information in
source material, and helping a business to understand the social sentiment of their brand, product or
service while monitoring online conversations. This concept came into existence since the innovation
of social networking websites such as Twitter, Facebook, Tinder etc. and people started posting their
honest online reviews about any particular brand or product on the Internet. Stock Prediction is one
such field which is borne out of this. Sentimental Analysis encompasses several domains such as
Natural Language Processing, text analysis, computational linguistics, and biometrics. Currently
many traders and brokers use indicators based on mathematical formulae. Many of them even don't
know how they work. Technical indicators will be in use if market or company stock repeats pattern;
but many times it doesn't. Through this project we have attempted to overcome most of the above
flaws, make the system cheaper and easier to use and understand.

1.2 RELEVANCE

Stock market prediction task is a fascinating topic and it divides researchers and academics into two
groups, people who believe we'll devise mechanisms to predict the market and people who believe that
the market is efficient and whenever new information comes up the market absorbs it by correcting
itself, hence there's no space for prediction.
Stock price prediction can be used to gain insight about market behavior over time, spotting trends that
would otherwise not have been noticed. With the increasing computational power of the computer,
machine learning will be an efficient method to solve this problem. However, the public stock data-
sets is too limited for any machine learning algorithm alone to work with, while asking for more
features may cost thousands of dollars every day.
In our project we will perform sentimental analysis of Twitter data (news or comments) to gain insight
on customer behavior. It will also help us analyze the public sentiments. The researchers found a
major correlation between stock returns and individual’s reactions. In fact, valuable data in the domain
of stock market should include several features like time, targeted audience, and brand but the most
important feature for the decision makers who are looking to invest in the stock market are time and
brand.

PICT - INFORMATION TECHNOLOGY - 2019-20


9
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

1.3 PROJECT UNDERTAKEN

Output of sentimental analysis can’t be the only deciding factor in predicting process. Use of technical
indicators to check the actual movement of the market is essential. Many times market movement can’t
be predicted due to pseudo force and it can’t sustain more or less than the upper and lower circuit. So
here investors should be aware of retracement of market movement at any time.
In the proposed model of our project “Stock Prediction using Sentimental and Technical Analysis”,
we extract features not only from time series data source (price and volume) but also from sentimental
analysis results mining from Twitter. The aggregation of features is done and multiple kernel learning
framework is used to learn and predict stock movement.

1.4 ORGANIZATION OF PROJECT REPORT

In this chapter we have attempted to give a brief overview of the project. In Chapter 2 we talk in detail
about the literature review. Chapter 3 of this report deals with requirement specification and analysis.
Chapter 4 delves into the design aspect of the project. Chapter 5 talks in detail about the
implementation of the project. In Chapter 6 we discuss the conclusion and future scope.

PICT - INFORMATION TECHNOLOGY - 2019-20


10
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 2

LITERATURE REVIEW

Sr. No Author Name and Paper Description


Year

1 Alessia D’Andrea, ‘Approaches, Tools Concept of


Fernando Ferri, and Applications for sentimental analysis
Patrizia Grifoni, Sentiment Analysis is described and a
Tiziana Guzzo, IJCA, Implementation ’ model is proposed
Sept 2015[1]. along with some
ML methods.
2 Ashish Sharma, ‘Survey of Stock Machine learning
Dinesh Bhuriya , Market Prediction approaches are
Upendra Singh, Using Machine described. 4 types of
ICECA, 2017[17]. Learning Approach’ regression methods
are discussed.

3 Isnaini Nuzula 'The Integration of Technical analysis


Agustin, 2019[5]. Fundamental and of some methods are
Technical Analysis in done to predict
Predicting the Stock stock market.
Price'
4 Venkata Sasank ‘Sentiment Analysis Sentimental analysis
Pagolu, Kamal Nayan of Twitter Data for model is proposed
Reddy Challa, Predicting Stock and explained.
Ganapati Panda, Market Movements ’
Babita Majhi,
2016[15]

4 Shangkun DENG, ‘Combining Concept of MKL is


Takashi Technical Analysis stated and model is
MITSUBUCHI, Kei with Sentiment proposed which will
SHIODA, Tatsuro Analysis for Stock extract features from
SHIMADA, Akito Price Prediction’ both technical and
SAKURAI, 2011[10]. sentimental analysis.

PICT - INFORMATION TECHNOLOGY - 2019-20


11
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

5 Dr. Devpriya Soni, 'Optimised Prediction Literature survey of


Sparsh Agarwal, Model For Stock different methods is
Tushar Agarwal, Market Trend done along with
Pooshan Arora, Kopal Analysis' technical analysis,
Gupta, 2018[13]. and optimised
model is proposed.

6 Dr. Devpriya Soni, 'Optimised Prediction Literature survey of


Sparsh Agarwal, Model For Stock different methods is
Tushar Agarwal, Market Trend done along with
Pooshan Arora, Kopal Analysis' technical analysis,
Gupta, 2018[13]. and optimised
model is proposed.

7 Vivek Rajput, Sarika ‘Stock Market Literature survey of


Bobde, 2016[6]. Forecasting 5 different methods
Techniques: is done and 7
Literature Survey’ various methods are
explained.

8 Jan Ivar Larsen, ‘Predicting Stock Basics of trading in


2010[3]. Prices Using stock market and
Technical Analysis technical analysis is
and Machine explained. ML
Learning’ model is proposed
for various feature
extraction agents.

9 Dev Shah , Haruna 'Predicting the Effects Dictionary based


Isah, Farhana of News Sentiments model is proposed
Zulkernine, 2018[14]. on the Stock Market' and explained.

PICT - INFORMATION TECHNOLOGY - 2019-20


12
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

2.1 EXISTING METHODOLOGIES

1. Currently many traders and brokers use indicators based on mathematical


formulae. Many of them even don't know how they work. Technical indicators
will be in use if market or company stock repeats pattern; but many times it
doesn't.
2. Market movement can change drastically if some news or decision from
organization body/ government comes up. This can lead to huge loss to investors.
3. Currently investors get news from social media but their collective analysis is not
done properly.
4. For sentiment analysis, researchers have primarily focused on the time feature in
which data was collected from Twitter. The researchers found a significant
correlation between stock returns and individual’s reactions. In fact, valuable data
in the domain of stock market should include several features like time, targeted
audience, and brand but the most important for the decision makers who are
looking to invest in the stock market are time and brand.
5. Current prediction models mainly focus on results of sentimental analysis or
technical analysis individually.

PICT - INFORMATION TECHNOLOGY - 2019-20


13
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

2.2 PROPOSED METHODOLOGIES

1. Output of sentimental analysis can’t be the only deciding factor in predicting


process. Use of technical analysis to check the actual movement of the market is
essential. Many times market movement can’t be predicted due to pseudo force
and it can’t sustain more or less than the upper and lower circuit. So here investors
should be aware of retracement of market movement at any time.
2. In the proposed model, we extract features not only from sentimental analysis
results mining from Twitter, but also from time series data source (price and
volume) .
3. Using the historical data of a stock to analyse how the movement of the stock has
been in the last one year it becomes easy to predict the future movement of the
stock.

PICT - INFORMATION TECHNOLOGY - 2019-20


14
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 3

REQUIREMENT SPECIFICATION AND ANALYSIS

3.1 PROBLEM DEFINITION

We aim to create a system for stock prediction based on a set of technical trading
rules and sentimental analysis of date present on social media. The aim of the research is
to check if it is possible to obtain a set of trading pattern, which could be used to take
decisions while trading such as a Buy, Sell, Exit, Stop Loss, etc.

3.2 CONCEPT

The sentiments from various investors, investment firms and traders are important
to calculate an idea where the market can go, but this data cant be trusted completely as
there can be a trap used by various operators who are financially strong to manipulate the
market and create a state of confusion among the common retail investors . Hence
technical analysis of the stocks is also important to predict the future performance of a
stock or a company .

3.3 SCOPE

1. The system is designed to predict Indian stock market particularly.


2. The technical analysis is done using at a year’s historical data of a
particular stock, but only one year’s data is to be extracted as increasing
the number of past year’s data leads to decrease in the accuracy of the
predictions.
3. Data(tweets) from Twitter is taken to know people's opinions to analyze
sentiments.
4. The system can be used alongside with the existing assistance systems for
gaining more information of the market.

PICT - INFORMATION TECHNOLOGY - 2019-20


15
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

3.4 OBJECTIVES

1. To achieve technical analysis using historic data source of a stock.


2. To analyze the correlation between stock market movements of a company
and sentiments in tweets.
3. To combine the features extracted from multiple sources.

3.5 PROJECT REQUIREMENTS

3.5.1 DATASETS

1. User defined bag of words text file (positive and negative files).
2. Historical Stock Price from Yahoo Finance.
3. Tweets from project members via Twitter.

3.5.2 FUNCTIONAL REQUIREMENTS

1. For predicting the sentiments from the tweets fetched all the words
that are commonly used in stock market should be updated in the
user defined bag of words.
2. The prediction of the price of the stock will be done only
through technical analysis, sentiment analysis will only give an
idea about the market’s sentiments.

PICT - INFORMATION TECHNOLOGY - 2019-20


16
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

3.5.3 NON-FUNCTIONAL REQUIREMENTS

1. The accuracy of the predicted price will be dependent on the


accuracy of the data, date of purchase, quantity of stock purchased,
high and low value range as well as opening and closing figures.
2. User should be at least aware of basic stock market terms and
concepts.

3.5.4 HARDWARE REQUIREMENTS

1. System with 8GB RAM, INTEL i5 processor


2. GPU for training the model

3.5.5 SOFTWARE REQUIREMENTS


1. Platform used:- VS code, idle.
2. Language: Python
3. OS:- windows/Ubuntu.

3.6 PROJECT PLAN

3.6.1 PROJECT RESOURCES

1. Python Libraries.

2. Twitter API for fetching tweets.


3. Data pre-processing and data cleaning module.
4. High performance GPU for model training.

PICT - INFORMATION TECHNOLOGY - 2019-20


17
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 4

SYSTEM ANALYSIS AND DESIGN

4.1 ARCHITECTURE

Fig2. System Architecture

PICT - INFORMATION TECHNOLOGY - 2019-20


18
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.2 DFD1

Fig3. Data Flow Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


19
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3 BEHAVIORAL DIAGRAMS

4.3.1 USE CASE DIAGRAM

Fig4. Use Case Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


20
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3.2 ACTIVITY DIAGRAM

Fig5. Activity Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


21
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3.3 SEQUENCE DIAGRAM

Fig6. Sequence Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


22
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3.4 STATE DIAGRAM

Fig7. State Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


23
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3.4 CLASS DIAGRAM

Fig8. Class Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


24
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.3.5 COMPONENT DIAGRAM

Fig9. Component Diagram

PICT - INFORMATION TECHNOLOGY - 2019-20


25
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

4.4 ALGORITHM AND METHODOLOGIES

1. NATURAL LANGUAGE PROCESSING


Natural language processing (NLP) may be a subfield of
linguistics, computing , information engineering, and AI concerned with
the interactions between computers and human (natural) languages,
especially the way to program computers to process and analyze large
amounts of tongue data.Challenges in tongue processing frequently involve
speech recognition, tongue understanding, and tongue generation.

2. NAIYE BAYES ALGORITHM


Naive Bayes classifier uses applied mathematics to classify data.
Naive Bayes classifier algorithms make use of Bayes' theorem . The key
insight of Bayes' theorem is that the probability of an event can be adjusted
as new data is introduced.What makes a naive Bayes classifier naive is its
assumption that each one attributes of a knowledge point into account are
independent of every other. A classifier sorting fruits into apples and
oranges would know that apples are red, round and are a certain size, but
would not assume all these things at once. Oranges are round too, after all.

PICT - INFORMATION TECHNOLOGY - 2019-20


26
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

3. TIME SERIES PREDICTION


Time series analysis comprises methods for analyzing statistic data
so as to extract meaningful statistics and other characteristics of the
info . Time series forecasting is that the use of a model to predict
future values supported previously observed values. While
multivariate analysis is usually employed in such how on test
theories that the present values of 1 or more independent statistic
affect the present value of once more series, this sort of study of
your time series isn't called "time series analysis", which focuses on
comparing values of one statistic or multiple dependent time series
at different points in time. Interrupted statistic analysis is that the
analysis of interventions on one statistic ..

4. AUTO REGRESSION
Autoregression is basically a time series model that uses
observations from previous time steps or from the past data as input to a
regression of y on x to predict the worth at the following time step. It is a
simple concept that may end up in accurate forecasts on a spread of your
time series problems which is a great way to predict the future price of a
company's stock.

5. ARIMA FROM STATS MODEL


Auto regressive integrated moving average(ARIMA) is statistical
model for time series prediction. The standard notation for the model is
ARIMA(p,d,q) where p is total lag observations also called lag order, d is
number of times that the raw observations are differenced also called
degree of differencing, and q is size of moving average window. Model
predicts future state by checking difference between values in time series
instead of actual stock value.

PICT - INFORMATION TECHNOLOGY - 2019-20


27
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

6. FORWARD MOVING AVERAGE

It is the most used technical indicator used to find momentum of


trading stock. It is calculated for fixed time frame by adding next
successive items to previously calculated average of group of items. The
most common time frame for predicting stock momentum is 14 days, 22
days and 50 days. It is used to understand short term fluctuations. Simple
moving average(SMA) and exponential moving average(EMA) are two
basic types of moving forward average.

PICT - INFORMATION TECHNOLOGY - 2019-20


28
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 5

IMPLEMENTATION

5.1 STAGES OF IMPLEMENTATION

5.1.1 Technical Analysis

Fig10. Steps for technical analysis

PICT - INFORMATION TECHNOLOGY - 2019-20


29
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

5.1.2 Sentimental Analysis

Fig11. Steps for sentimental analysis

PICT - INFORMATION TECHNOLOGY - 2019-20


30
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

5.2 IMPLEMENTATION TECHNIQUES

1. Raw data preprocessing component:


It takes content of news and comments as input, and also takes historic price and
volume data as raw data for further process the content of tweets require cleaning which
helps us in creating the sentiment analysis.

2. Feature extraction component:


It breaks down each and every word in the fetched tweets and compares it with
bag of words, if that word exists in bag of words then it assigns it labels from that
respective bag of words.

3. Sentiment analysis component:


It analyzes the overall sentiments (objective, subjective, positive, negative), a
threshold value for overall sentiments is set to classify each sentiment. The output of
process will be the actual percentage of positive and negative sentiment from the fetched
data.

4. Technical analysis component:


It formulates different indicators based on price and volume. These indicators
generate signals movement of market. Then we combine outputs of each signal using
simple combination or using Neural Network combination. The trend of market is
predicted as output of technical analysis.

PICT - INFORMATION TECHNOLOGY - 2019-20


31
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

5. Correlation:
If the previous day stock price is more than the current day stock price, the current
day is marked with a numeric value of 0, else marked with a numeric value of 1. This
correlation analysis turns out to be a classification problem which can be solved using
machine learning classification algorithms. The accuracy of model increases as it gets
trained on more data.

PICT - INFORMATION TECHNOLOGY - 2019-20


32
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 6

EVALUATION METHODS

5.1 MODEL TRAINING

5.1.1 EXAMPLE SET OF FEATURES


1. Price ROC, MACD
2. Volume ROC
3. Frequency of news
4. Frequency of comments
5. Number of positive, negative, neutral news and comments

5.1.2 EXAMPLE SET OF MODELS


1. Sentimental analysis + technical analysis (MKL regression model)
2. Sentimental analysis + technical analysis (SVR regression model)
3. Technical analysis only (SVR regression model)
4. Sentimental analysis only (SVR regression model)

PICT - INFORMATION TECHNOLOGY - 2019-20


33
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

5.2 EXAMPLE

Fig12. Example

Stock value of Yes Bank went down when the rumors were spread in market.
Bank took some days to file a complaint and it was again set to retain its position.
Considering this scenario we can say that use of sentimental analysis can make
excellent profit; but we can’t only rely on it. Along with sentimental analysis we must
apply technical analysis to minimize risk factor in trading. The retracement of market at
any time can be well predicted using technical indicators i.e. based on volume, price,
percentage change in buyers and sellers, etc.

PICT - INFORMATION TECHNOLOGY - 2019-20


34
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

CHAPTER 7

CONCLUSIONS

7.1 CONCLUSION

The proposed system will help beginner traders as a decision support tool and help
them take decisions accordingly. Collective analysis of news regarding market have been
made easy using machine learning algorithms.

7.2 LIMITATIONS

1. Sentimental analysis will only work on english language and not other regional
languages.
2. Input datasets has been taken from Twitter itself.

7.3 FUTURE SCOPE

1. We have considered only twitter data for analyzing people's sentiments which may
be biased because not all the people who trade share their opinions on Twitter.
The study can be extended by incorporating data from various platforms like
moneycontrol.com, stock twits, Yahoo Finance, etc.
2. This project is the initial phase of development where the algorithms are selected
and are being developed. As we proceed further the algorithm will be optimized
for more technical parameters and more complex parameters, to get much more
real world output values. And also, we can extend this project for other markets
like foreign stock markets, commodities, Forex trading, etc.

PICT - INFORMATION TECHNOLOGY - 2019-20


35
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

REFERENCES

[1] “Approaches, Tools and Applications for Sentiment Analysis


Implementation”, Alessia D’Andrea, Fernando Ferri, Patrizia Grifoni,
Tiziana Guzzo, International Journal of Computer Applications (0975 –
8887) Volume 125 – No.3, September 2015.

[2] “Stock Market Classification Model Using Sentiment Analysis on


Twitter Based on Hybrid Naive Bayes Classifiers”, Ghaith Abdulsattar
A.Jabbar Alkubaisi, Siti Sakira Kamaruddin & Husniza Husni, Computer and
Information Science; Vol. 11, No. 1; 2018, ISSN 1913-8989 E-ISSN 1913-
8997

[3] “Stocks Market Prediction Using Support Vector Machine”, Zhen Hu,
Jie Zhu, and Ken Tse, 2013 6th International Conference on Information
Management, Innovation Management and Industrial Engineering.

[4] “Sentiment Analysis in Twitter using Machine Learning Techniques”,


Neethu M S, Rajasree R, 4th ICCCNT 2013, July 4 - 6, 2013, Tiruchengode,
India.

[5] “The Integration of Fundamental and Technical Analysis in Predicting


the Stock Price”, Isnaini Nuzula Agustin, Volume 18, Nomor 2, Mei 2019,
pp 93-102. Studi Manajemen, Fakultas Ekonomi, Universitas Kristen
Maranatha. ISSN 1411-9293 | e-ISSN 2579-4094.

[6] “STOCK MARKET FORECASTING TECHNIQUES:


LITERATURE SURVEY”, Vivek Rajput, Sarika Bobde, International
Journal of Computer Science and Mobile Computing, Vol.5 Issue.6, June-
2016.

[7] “Quantitative Analysis of Stock Market Prediction for Accurate


Investment Decisions in Future”, Surbhi Sharma, Baijnath Kaushik, Journal
of Artificial Intelligence, ISSN 1994-5450, 2018.

[8] “A Genetic Programming Based Stock Price Predictor together with


Mean-Variance Based Sell/Buy Actions”, Ramin Rajabioun, Ashkan

PICT - INFORMATION TECHNOLOGY - 2019-20


36
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

Rahimi-Kian, Proceedings of the World Congress on Engineering 2008.


[9] “Optimised Prediction Model For Stock Market Trend Analysis”, Dr.
Devpriya Soni, Sparsh Agarwal, Tushar Agarwal, Pooshan Arora, Kopal
Gupta, Proceedings of 2018 Eleventh International Conference on
Contemporary Computing (IC3), 2-4 August, 2018, Noida, India.

[10] “Predicting the Effects of News Sentiments on the Stock Market”, Dev
Shah , Haruna Isah, Farhana Zulkernine, 2018 IEEE International
Conference on Big Data.

[11] “Varsity by Zerodha.” [Online]. Available: https://zerodha.com/varsity

[12] moneycontrol website[online]. Available:


https://www.moneycontrol.com

PICT - INFORMATION TECHNOLOGY - 2019-20


37
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

APPENDICES

A] BASE PAPER(S)

1. “Combining Technical Analysis with Sentiment Analysis for Stock


Price Prediction”, Shangkun DENG, Takashi MITSUBUCHI, Kei
SHIODA, Tatsuro SHIMADA, Akito SAKURAI, 2011 Ninth IEEE
International Conference on Dependable, Autonomic and Secure
Computing.
https://ieeexplore.ieee.org/document/6118898?anchor=citations

2. “Sentiment Analysis of Twitter Data for Predicting Stock Market


Movements”, Venkata Sasank Pagolu, Kamal Nayan Reddy Challa,
Ganapati Panda, Babita Majhi, International conference on Signal
Processing, Communication, Power and Embedded System
(SCOPES)-2016.
https://arxiv.org/pdf/1610.09225.pdf

3. “Survey of Stock Market Prediction Using Machine Learning


Approach”, ASHISH SHARMA, DINESH BHURIYA , UPENDRA
SINGH , International Conference on Electronics, Communication and
Aerospace Technology, ICECA 2017.
https://ieeexplore.ieee.org/document/8212715

4. “A comparative study of supervised machine learning algorithms for


stock market trend prediction”, Indu Kumar, Kiran Dogra, Chetna
Utreja, Premlata Yadav, 978-1-5386-1974-2/18/2018 IEEE.
https://ieeexplore.ieee.org/document/8473214

5. “Predicting Stock Prices Using Technical Analysis and Machine


Learning”, Jan Ivar Larsen, Norwegian University of Science and
Technology Department of Computer and Information Science, 2010.
https://core.ac.uk/download/pdf/52104888.pdf

PICT - INFORMATION TECHNOLOGY - 2019-20


38
STOCK PREDICTION USING TECHNICAL AND SENTIMENTAL ANALYSIS

B] PLAGIARISM REPORT

PICT - INFORMATION TECHNOLOGY - 2019-20


39

You might also like