Approaches of Sentiment Analysis in Finance - Articles by Fervent

12/04/2023, 19:35 Approaches of Sentiment Analysis in Finance - Articles by Fervent
Do you want unlimited access to all our Finance

and Investing Courses? Learn More >>
(https://www.ferventlearning.com/risk-tolerance-quiz/)
FERVENT | FINANCE
COURSES, INVESTING
COURSES
Approaches of Sentiment
Analysis in Finance
 Share  Tweet  Share
In this article, we’re going to learn about the two core approaches for
estimating sentiment for sentiment analysis in Finance.
We ethically use cookies on our website to give you the best possible user experience. By clicking “Accept All”, you
Just
consent to the a quick
use of ALLrecap though,
the cookies. recall
However, that
you may we
visit saidSettings"
"Cookie sentiment analysis
to provide a controlled consent.
(https://www.ferventlearning.com/sentiment-analysis-an-overview/),
Cookie Settings Accept All
at
least in finance, involves quantifying and exploiting sentiment or emotions
f t fi t t
https://www.ferventlearning.com/approaches-of-sentiment-analysis-in-finance/ 1/13
for some sort of investment purpose.
Or indeed, it’s about estimating sentiment. And then linking it to other firm
characteristics to gain a better understanding of how sentiment can drive
firm performance.
Further recall that we said sentiment itself can include things like
positivity; as well as negativity, uncertainty, narcissism. And a whole host
of other human emotions that you might think of when you think of the
word sentiment.
Approaches of Sentiment Analysis in

Finance
Now, when we think about estimating sentiment for sentiment analysis in
finance, there are broadly two approaches you can take.
The first is what’s called the lexicon or dictionary based approach.
And the second is broadly some sort of “machine learning” / “artificial

intelligence” approach.
This holds regardless of whether you’re looking to estimate investor

sentiment, or create a sentiment indicator for the sentiment of the stock
market / financial market as a whole.
consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Let’s consider both, starting with the lexicon or dictionary based approach.
Lexicon based approach for sentiment analysis in

finance
The lexicon based approach is perhaps the most common way of

estimating sentiment. You’ll see why later on in the post. For now though,
let’s crack on with learning how it works.
Related Course: Investment Analysis with Natural

Language Processing (NLP)
This Article features concepts that are covered extensively in our course
on Investment Analysis with Natural Language Processing (NLP)
(https://www.ferventlearning.com/courses/investment-analysis-with-
natural-language-processing-nlp/).
If you’re interested in learning how to leverage the power of text data for
investment analysis while working with real world data, you should
definitely check out the course
(https://www.ferventlearning.com/courses/investment-analysis-with-
natural-language-processing-nlp/).
Start with a Prior
For the lexicon based approach, we typically start with some sort of a prior
or belief or opinion on what constitutes words relating to sentiment.
Be that positive sentiment, negative sentiment, uncertainty, or indeed any

other type of sentiment.
The key idea is that we start with an opinion on what we believe are
words that plausibly relate to the sentiment. And we call that collection of
words a sentiment language.
And by the way, we tend to denote the sentiment language, as (the

Greek letter “psi“).
Importantly,
consent to sentiment
the use of ALL the language
cookies. However, is visit
you may the"Cookie as sentiment
sameSettings" to providelexicon . Which
a controlled consent.
is the same
Cookie Settings
as sentiment dictionary.
Accept All
People who apply textual analysis tend to use these terms
People who apply textual analysis tend to use these terms

interchangeably.
Consistent with other aspects in finance, we just use a bunch of words that
mean exactly the same thing. Because it tends to make us sound clever.
But jokes aside, we can either create the sentiment language from scratch,
or we can work with existing sentiment language.
If we create the sentiment language from scratch, then we need to start

with our own priors or our own beliefs.
Priors on which words we think plausibly relate to positivity, negativity, or

any type of sentiment that we’re looking at.
For instance, if we wanted to create a custom dictionary or lexicon for

positivity / positive sentiment…
Then we might argue that words like “happy”, “positive”, “growth”, “increase”,
or “excitement” are all words that plausibly relate to positivity.
We would then say that those words together make up our positive
sentiment language or .
Similarly, if we wanted to create a lexicon for negative sentiment, then we

might argue that words like “sad”, “disappoint”, “decline”, “decrease” are
words that plausibly reflect negative sentiment.
Alternatively, Agree With Others’ Priors
Alternatively, we could work with existing dictionaries or lexicons.

And
consent to in that
the use of ALLcase, we’reHowever,
the cookies. you mayagreeing
essentially visit "Cookiewith thetopriors,
Settings" provide aor beliefs,consent.
controlled or
opinions Accept
Cookie Settings
of other
All
people.
The people who’ve created these dictionaries of a bunch of words which
The people who’ve created these dictionaries of a bunch of words, which

they think plausibly relate to a specific kind of sentiment.
Want to go further in Financial

Sentiment Analysis?
Get the Investment Analysis with NLP Study

Pack (for FREE!).
Fields marked with an

* are required
Your Name
Your Email
I agree to receive emails

from Fervent and accept
the Privacy Policy. *
Estimating
consent to Sentiment
the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.

Regardless of whether you use a custom sentiment language or an
existing sentiment language
existing sentiment language…
The idea is that once you have this sentiment language…
You can estimate sentiment as a function of the words in a given

document that belong to a sentiment language.
Strictly, you’d want to look at the cleaned words which belong to a

sentiment language. For instance, which of the cleaned words belong to
positive sentiment, and which of those cleaned words belong to negative
sentiment, etc.
Now, you don’t really need to worry about what we mean by cleaned words
just yet. We have got a separate post on how to clean text data
(https://www.ferventlearning.com/how-to-clean-text-data/).
The only thing you need to know for now is that we don’t really work with
the words in a given document.
Instead, we work with the cleaned words in a given document.
So that’s the words after we’ve performed text cleaning.
In a nutshell, we’re literally just looking at the cleaned words inside a

document and identifying which of those cleaned words relate to positive
language. And which of those relate to negative language. Or which of
those relate to the specific sentiment language that we’re trying to
explore.
All right. So that’s as far as the lexicon based approach to estimating

sentiment for sentiment analysis in finance goes.
Machine learning based approach for sentiment
analysis
Cookie Settings inAll finance
Accept
Perhaps more fancy than the standard lexicon based approach, the
machine learning approach is arguably more objective. However, data
limitations mean we can’t quite leverage its power (yet). But here’s how
you’d go about applying it.
Start with a subsample
In the machine learning / artificial intelligence approach, we typically start

with a sub-sample of the Corpus which displays sentiment.
Remember, a Corpus is just the entire sample of text data that you’re
working with.
So essentially, you would start with a sub-sample of that Corpus which

displays or shows sentiment. Put differently, a sub-sample for which
you already have estimates of sentiment.
And a typical, classic example, is the case of movie reviews.
Outside of Finance, when people are looking to estimate sentiment, the

classic example is the IMDB movie reviews dataset
(https://www.imdb.com/interfaces/) where you’ve got a bunch of movie
reviews, which are labeled as either “positive” or “negative”.
And so in that case, you’d have a sub-sample of the data, which is labeled
as either positive or negative.
Train,
consent to the use Test, Iterate
of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.

You would then apply some sort of classification algorithm or machine
learning algorithm to classify or estimate sentiment for other sample
learning algorithm to classify or estimate sentiment for other sample

texts.
In other words, you would train the algorithm on a sub-sample

dataset that has sentiment labels. And then you would apply the algorithm
to other data to classify them as either positive or negative.
Keep in mind that the machine learning approach isn’t quite as simple and
straightforward as we’ve made it seem.
It can in fact get far more complicated than what we’ve written here.
Issues with applying it in Finance
But the reason we’re not really going into much more detail is that,
fundamentally, applying this approach for sentiment analysis in Finance is
quite problematic.
LACK OF CLASSIFIED FINANC IAL TEXT
And that is because we don’t really have financial text data with sentiment
labels in Finance!
There’s plenty of IMDB movie reviews, for instance, that have labels of
either positive or negative sentiment, or some sort of rating scale between
1 and 10.
And you could easily say, or plausibly argue, that any movie with a rating
greater than five is plausibly positive. Any movie with a rating below five is
plausibly negative. And any movie with a rating of 5 is plausibly neutral.
But we don’t quite have that sort of luxury in finance.

There’s
consent to noALL
the use of database
the cookies. or no existing
–However, database
you may visit – at least
"Cookie Settings" at the
to provide time of
a controlled consent.
writing, which
Cookie Settings
has already classified companies based on their level of
Accept All
sentiment.
We don’t have a database of each company on the S&P500 for example,

classified as either displaying positive tone, or negative tone, or net
positive tone, or any other sentiment for that matter.
FINDING IDENTIFIERS TO MER GE ON
Now, you might argue that it should be fairly trivial to create such a
database.
For instance, why not just scrape the Twitter / social media accounts of
companies, and classify their tweets on sentiment?
As trivial as that actually is, the issue then becomes one of connecting that
data to stock returns.
Or to any of the company’s financial fundamentals for that matter.
Because in Finance, conducting sentiment analysis usually involves linking

sentiment to firms.
The end result could be creating a trading strategy or sentiment investing

strategy (https://www.ferventlearning.com/sentiment-investing-guide/)
that exploits the sentiment of firms over the long term; or indeed, one that
uses some sort of “sentiment signals” to determine the buy / sell / hold
decision for a given trading strategy.
RELATED: Sentiment Investing Guide

(https://www.ferventlearning.com/sentiment-investing-guide/)
But
consent to thedoing sothe
use of ALL will be challenging
cookies. However, you maybecause theSettings"
visit "Cookie Twitter to profiles / socialconsent.
provide a controlled
media profiles
Cookie Settings
of companies don’t tend to have their tickers, CIK number,
Accept All
CUSIP, or other firm level identifiers in their profile descriptions!
That in turn makes merging sentiment score data with other financial data
tricky.
And so at least today, machine learning approaches in finance are largely
focused on traditional asset pricing.
For instance, machine learning algorithms are used to try and explain the
cross-section of returns. Or even the volatility of returns.
But as far as estimating sentiment for sentiment analysis in finance goes…
The literature – both academic and practitioner – largely relies on the

lexicon or dictionary based approach.
Wrapping Up
In summary, we learned that sentiment can broadly be estimated using a
lexicon / dictionary based approach, or a machine learning approach.
Applying the machine learning approach in finance can be quite

problematic. Because, at least today, there’s no existing database of
companies labeled based on their level of sentiment.
And so for the most part, both academics and practitioners largely rely on
using the lexicon or dictionary based approach for estimating sentiment.
Perhaps most importantly, we learned that for the dictionary / lexicon

based approach, you can estimate sentiment as a function of the cleaned
words which belong to a sentiment language.
Related Course: Investment Analysis with Natural
Language
Cookie Settings Accept All Processing (NLP)
Do you want to build a rigorous investment analysis system that leverages

the power of text data with Python?
EXPLORE THE COURSE

(HT TPS://WWW.FERVENTLEARN I N G.COM/COURSE S/I N VE STM E N T
-ANALYSIS-WITH-NATURAL-L AN GUAGE - P ROCESSI N G- N L P /)
 Share  Tweet  Share
You must be logged in (https://www.ferventlearning.com/wp-login.php?

redirect_to=https%3A%2F%2Fwww.ferventlearning.com%2Fapproaches-of-sentiment-
analysis-in-finance%2F) to post a comment.
Do You Want Unlimited Access To All

Our Courses?
YES! TELL ME MORE

(HT TPS://WWW.FERVENTLEARN I N G.COM/ALL-ACCESS- PASS/)
Copyright © 2023, Fervent · Privacy Policy (https://ferventlearning.com/privacy-policy/) · Terms and Conditions

(https://ferventlearning.com/terms-conditions/)
Logos of institutions used are owned by those respective institutions. Neither Fervent nor the institutions endorse
each other's products / services.

Approaches of Sentiment Analysis in Finance - Articles by Fervent

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Approaches of Sentiment Analysis in Finance - Articles by Fervent

Uploaded by

Copyright:

Available Formats

12/04/2023, 19:35 Approaches of Sentiment Analysis in Finance - Articles by Fervent

Do you want unlimited access to all our Finance

for some sort of investment purpose.

Approaches of Sentiment Analysis in

The first is what’s called the lexicon or dictionary based approach.

And the second is broadly some sort of “machine learning” / “artificial

This holds regardless of whether you’re looking to estimate investor

Cookie Settings Accept All

Lexicon based approach for sentiment analysis in

The lexicon based approach is perhaps the most common way of

Related Course: Investment Analysis with Natural

Cookie Settings Accept All

Start with a Prior

Be that positive sentiment, negative sentiment, uncertainty, or indeed any

And by the way, we tend to denote the sentiment language, as (the

People who apply textual analysis tend to use these terms

People who apply textual analysis tend to use these terms

If we create the sentiment language from scratch, then we need to start

Priors on which words we think plausibly relate to positivity, negativity, or

For instance, if we wanted to create a custom dictionary or lexicon for

Similarly, if we wanted to create a lexicon for negative sentiment, then we

Alternatively, Agree With Others’ Priors

Alternatively, we could work with existing dictionaries or lexicons.

The people who’ve created these dictionaries of a bunch of words which

The people who’ve created these dictionaries of a bunch of words, which

Want to go further in Financial

Get the Investment Analysis with NLP Study

Fields marked with an

I agree to receive emails

Cookie Settings Accept All

existing sentiment language…

The idea is that once you have this sentiment language…

You can estimate sentiment as a function of the words in a given

Strictly, you’d want to look at the cleaned words which belong to a

Instead, we work with the cleaned words in a given document.

So that’s the words after we’ve performed text cleaning.

In a nutshell, we’re literally just looking at the cleaned words inside a

All right. So that’s as far as the lexicon based approach to estimating

Start with a subsample

In the machine learning / artificial intelligence approach, we typically start

So essentially, you would start with a sub-sample of that Corpus which

And a typical, classic example, is the case of movie reviews.

Outside of Finance, when people are looking to estimate sentiment, the

Cookie Settings Accept All

learning algorithm to classify or estimate sentiment for other sample

In other words, you would train the algorithm on a sub-sample

Issues with applying it in Finance

LACK OF CLASSIFIED FINANC IAL TEXT

But we don’t quite have that sort of luxury in finance.

We don’t have a database of each company on the S&P500 for example,

FINDING IDENTIFIERS TO MER GE ON

Or to any of the company’s financial fundamentals for that matter.

Because in Finance, conducting sentiment analysis usually involves linking

The end result could be creating a trading strategy or sentiment investing

RELATED: Sentiment Investing Guide

But as far as estimating sentiment for sentiment analysis in finance goes…

The literature – both academic and practitioner – largely relies on the

Applying the machine learning approach in finance can be quite

Perhaps most importantly, we learned that for the dictionary / lexicon

Do you want to build a rigorous investment analysis system that leverages

EXPLORE THE COURSE

 Share  Tweet  Share

You must be logged in (https://www.ferventlearning.com/wp-login.php?

Cookie Settings Accept All