Natural Language Processing

Unit 6 Applications
Sentiment Analysis; Text Entailment; Robust and Scalable Machine Translation; Question
Answering in Multilingual Setting; Cross Lingual Information Retrieval (CLIR).

Natural Language Processing (NLP) is an emerging technology that derives various forms of AI
that we see in the present times and its use for creating a seamless as well as interactive interface
between humans and machines will continue to be a top priority for today’s and tomorrow’s
increasingly cognitive applications.

6.1 Sentiment Analysis

Sentiment analysis is the process of classifying whether a block of text is positive, negative, or,
neutral. The goal which Sentiment analysis tries to gain is to be analyzed people’s opinions in a
way that can help businesses expand. It focuses not only on polarity (positive, negative &
neutral) but also on emotions (happy, sad, angry, etc.). It uses various Natural Language
Processing algorithms such as Rule-based, Automatic, and Hybrid.
Example:
if we want to analyze whether a product is satisfying customer requirements, or is there a need
for this product in the market? We can use sentiment analysis to monitor that product’s reviews.
Sentiment analysis is also efficient to use when there is a large set of unstructured data, and we
want to classify that data by automatically tagging it.Net Promoter Score (NPS) surveys are used
extensively to gain knowledge of how a customer perceives a product or service. Sentiment
analysis also gained popularity due to its feature to process large volumes of NPS responses and
obtain consistent results quickly.

Sentiment

Why perform Sentiment Analysis?

Sentiment analysis is the contextual meaning of words that indicates the social sentiment of a
brand and also helps the business to determine whether the product they are manufacturing is
going to make a demand in the market or not.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

According to the survey,80% of the world’s data is unstructured. The data needs to be analyzed
and be in a structured manner whether it is in the form of emails, texts, documents, articles, and
many more.
1. Sentiment Analysis is required as it stores data in an efficient, cost-friendly.
2. Sentiment analysis solves real-time issues and can help you solve all real-time scenarios.
Types of Sentiment Analysis
1. Fine-grained sentiment analysis: This depends on the polarity base. This category can be
designed as very positive, positive, neutral, negative, or very negative. The rating is done on
a scale of 1 to 5. If the rating is 5 then it is very positive, 2 then negative, and 3 then neutral.
2. Emotion detection: The sentiments happy, sad, angry, upset, jolly, pleasant, and so on come
under emotion detection. It is also known as a lexicon method of sentiment analysis.
3. Aspect-based sentiment analysis: It focuses on a particular aspect for instance if a person
wants to check the feature of the cell phone then it checks the aspect such as the battery,
screen, and camera quality then aspect based is used.
4. Multilingual sentiment analysis: Multilingual consists of different languages where the
classification needs to be done as positive, negative, and neutral. This is highly challenging
and comparatively difficult.
How does Sentiment Analysis work?
There are three approaches used:
1. Rule-based approach: Over here, the lexicon method, tokenization, and parsing come in
the rule-based. The approach is that counts the number of positive and negative words in the
given dataset. If the number of positive words is greater than the number of negative words
then the sentiment is positive else vice-versa.
2. Machine Learning Approach: This approach works on the machine learning technique.
Firstly, the datasets are trained and predictive analysis is done. The next process is the
extraction of words from the text is done. This text extraction can be done using different
techniques such as Naive Bayes, Support Vector machines, hidden Markov model, and
conditional random fields like this machine learning techniques are used.
3. Neural network Approach: In the last few years neural networks have evolved at a very
rate. It involves using artificial neural networks, which are inspired by the structure of the
human brain, to classify text into positive, negative, or neutral sentiments. it has Recurrent
neural networks, Long short-term memory, Gated recurrent unit, etc to process sequential
data like text.
4. Hybrid Approach: It is the combination of two or more approaches i.e. rule-based
and Machine Learning approaches. The surplus is that the accuracy is high compared to the
other two approaches.
Applications
Sentiment Analysis has a wide range of applications as:
1. Social Media: If for instance the comments on social media side as Instagram, over here all
the reviews are analyzed and categorized as positive, negative, and neutral.
2. Customer Service: In the play store, all the comments in the form of 1 to 5 are done with the
help of sentiment analysis approaches.
3. Marketing Sector: In the marketing area where a particular product needs to be reviewed as
good or bad.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

4. Reviewer side: All the reviewers will have a look at the comments and will check and give
the overall review of the product.
Challenges of Sentiment Analysis
There are major challenges in the sentiment analysis approach:
1. If the data is in the form of a tone, then it becomes really difficult to detect whether the
comment is pessimist or optimistic.
2. If the data is in the form of emoji, then you need to detect whether it is good or bad.
3. Even the ironic, sarcastic, comparing comments detection is really hard.
4. Comparing a neutral statement is a big task.
Example of Sentiment Analysis
 Sentiment Analysis of Hindi Text – Python
 Facebook Sentiment Analysis using python
 Twitter Sentiment Analysis using Python
 Sentiment Analysis with Recurrent Neural Networks (RNN)
 Emotion Detection using Bidirectional LSTM
 Sentiment Classification Using BERT

______________________________________________________________________________

Types of Sentiment Analysis

Sentiment analysis focuses on the polarity of a text (positive, negative, neutral) but it also goes
beyond polarity to detect specific feelings and emotions (angry, happy, sad, etc), urgency
(urgent, not urgent) and even intentions (interested v. not interested).

Depending on how you want to interpret customer feedback and queries, you can define and
tailor your categories to meet your sentiment analysis needs. In the meantime, here are some of
the most popular types of sentiment analysis:

Graded Sentiment Analysis

If polarity precision is important to your business, you might consider expanding your polarity
categories to include different levels of positive and negative:

 Very positive
 Positive
 Neutral
 Negative
 Very negative

This is usually referred to as graded or fine-grained sentiment analysis, and could be used to
interpret 5-star ratings in a review, for example:

 Very Positive = 5 stars

 Very Negative = 1 star
MIT CORER, Barshi
Department of Computer Science & Engineering
 Natural Language Processing

Emotion detection

Emotion detection sentiment analysis allows you to go beyond polarity to detect emotions, like
happiness, frustration, anger, and sadness.

Many emotion detection systems use lexicons (i.e. lists of words and the emotions they convey)
or complex machine learning algorithms.

One of the downsides of using lexicons is that people express emotions in different ways. Some
words that typically express anger, like bad or kill (e.g. your product is so bad or your customer
support is killing me) might also express happiness (e.g. this is bad ass or you are killing it).

Aspect-based Sentiment Analysis

Usually, when analyzing sentiments of texts you’ll want to know which particular aspects or
features people are mentioning in a positive, neutral, or negative way.

That's where aspect-based sentiment analysis can help, for example in this product review: "The
battery life of this camera is too short", an aspect-based classifier would be able to determine
that the sentence expresses a negative opinion about the battery life of the product in question.

Multilingual sentiment analysis

Multilingual sentiment analysis can be difficult. It involves a lot of preprocessing and resources.
Most of these resources are available online (e.g. sentiment lexicons), while others need to be
created (e.g. translated corpora or noise detection algorithms), but you’ll need to know how to
code to use them.

Alternatively, you could detect language in texts automatically with a language classifier, then
train a custom sentiment analysis model to classify texts in the language of your choice.

Why Is Sentiment Analysis Important?

Since humans express their thoughts and feelings more openly than ever before, sentiment
analysis is fast becoming an essential tool to monitor and understand sentiment in all types of
data.

Automatically analyzing customer feedback, such as opinions in survey responses and social
media conversations, allows brands to learn what makes customers happy or frustrated, so that
they can tailor products and services to meet their customers’ needs.

For example, using sentiment analysis to automatically analyze 4,000+ open-ended responses in
your customer satisfaction surveys could help you discover why customers are happy or unhappy
at each stage of the customer journey.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

Maybe you want to track brand sentiment so you can detect disgruntled customers immediately
and respond as soon as possible. Maybe you want to compare sentiment from one quarter to the
next to see if you need to take action. Then you could dig deeper into your qualitative data to see
why sentiment is falling or rising.

The overall benefits of sentiment analysis include:

 Sorting Data at Scale

Can you imagine manually sorting through thousands of tweets, customer support conversations,
or surveys? There’s just too much business data to process manually. Sentiment analysis helps
businesses process huge amounts of unstructured data in an efficient and cost-effective way.

 Real-Time Analysis

Sentiment analysis can identify critical issues in real-time, for example is a PR crisis on social
media escalating? Is an angry customer about to churn? Sentiment analysis models can help you
immediately identify these kinds of situations, so you can take action right away.

 Consistent criteria

It’s estimated that people only agree around 60-65% of the time when determining the sentiment
of a particular text. Tagging text by sentiment is highly subjective, influenced by personal
experiences, thoughts, and beliefs.

By using a centralized sentiment analysis system, companies can apply the same criteria to all of
their data, helping them improve accuracy and gain better insights.

The applications of sentiment analysis are endless. So, to help you understand how sentiment
analysis could benefit your business, let’s take a look at some examples of texts that you could
analyze using sentiment analysis.

Then, we’ll jump into a real-world example of how Chewy, a pet supplies company, was able to
gain a much more nuanced (and useful!) understanding of their reviews through the application
of sentiment analysis.

Sentiment Analysis Examples

To understand the goal and challenges of sentiment analysis, here are some examples:

Basic examples of sentiment analysis data

 Netflix has the best selection of films

 Hulu has a great UI
 I dislike like the new crime series

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

 I hate waiting for the next series to come out

More challenging examples of sentiment analysis

 I do not dislike horror movies. (phrase with negation)

 Disliking horror movies is not uncommon. (negation, inverted word order)
 Sometimes I really hate the show. (adverbial modifies the sentiment)
 I love having to wait two months for the next series to come out! ( sarcasm)
 The final episode was surprising with a terrible twist at the end (negative term used in a
positive way)
 The film was easy to watch but I would not recommend it to my friends. (difficult to
categorize)
 I LOL’d at the end of the cake scene (often hard to understand new terms)

6.2 Textual Entailment

• Textual entailment recognition is the task of deciding, given two text fragments, whether the
meaning of one text is entailed (can be inferred) from another text. This task captures generically
a broad range of inferences that are relevant for multiple applications.
• For example, a QA system has to identify texts that entail the expected answer:

• Other QA ex: “Who owns Overture?” “Yahoo acquired Overture” “Overture’s acquisition by
Yahoo”, …

 In IE entailment holds between different text variants that express the same target relation:
(X kill Y X acquire Y)
• In MDS a redundant sentence or expression, to be omitted from the summary, should be
entailed from other expressions in the summary.
• In IR the concept denoted by a query expression should be entailed from relevant retrieved
documents.
• In MT evaluation a correct translation should be semantically equivalent to the gold standard
translation, and thus both translations have to entail each other.
• Thus, in a similar spirit to WSD and NER which are recognized as generic tasks, modeling
textual entailment may consolidate and promote broad research on applied semantic inference.

Textual Entailment Applications

 Question-Answering

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

 Information Extraction
 Information Retrieval
 Multi-Document Summarization
 Named Entity Recognition
 Temporal and Spatial Normalization
 Semantic Parsing
 Natural Language Generation

Textual entailment (TE), also known as Natural Language Inference (NLI), in natural language
processing is a directional relation between text fragments. The relation holds whenever the truth
of one text fragment follows from another text. In the TE framework, the entailing and entailed
texts are termed text (t) and hypothesis (h), respectively. Textual entailment is not the same as
pure logical entailment – it has a more relaxed definition: "t entails h" (t ⇒ h) if, typically, a human
reading t would infer that h is most likely true. (Alternatively: t ⇒ h if and only if, typically, a
human reading t would be justified in inferring the proposition expressed by h from the proposition
expressed by t.) The relation is directional because even if "t entails h", the reverse "h entails t" is
much less certain.
Determining whether this relationship holds is an informal task, one which sometimes overlaps
with the formal tasks of formal semantics (satisfying a strict condition will usually imply
satisfaction of a less strict conditioned); additionally, textual entailment partially subsumes word
entailment.

Examples
Textual entailment can be illustrated with examples of three different relations: [5]
An example of a positive TE (text entails hypothesis) is:

 text: If you help the needy, God will reward you.

hypothesis: Giving money to a poor man has good consequences.
An example of a negative TE (text contradicts hypothesis) is:

 text: If you help the needy, God will reward you.

hypothesis: Giving money to a poor man has no consequences.
An example of a non-TE (text does not entail nor contradict) is:

 text: If you help the needy, God will reward you.

hypothesis: Giving money to a poor man will make you a better person.

Ambiguity of natural language

A characteristic of natural language is that there are many different ways to state what one wants
to say: several meanings can be contained in a single text and the same meaning can be expressed
by different texts. This variability of semantic expression can be seen as the dual problem of
language ambiguity. Together, they result in a many-to-many mapping between language
expressions and meanings. The task of paraphrasing involves recognizing when two texts have the
MIT CORER, Barshi
Department of Computer Science & Engineering
 Natural Language Processing

same meaning and creating a similar or shorter text that conveys almost the same information.
Textual entailment is similar but weakens the relationship to be unidirectional. Mathematical
solutions to establish textual entailment can be based on the directional property of this relation,
by making a comparison between some directional similarities of the texts involved.

Approaches
Textual entailment measures natural language understanding as it asks for a semantic
interpretation of the text, and due to its generality remains an active area of research. Many
approaches and refinements of approaches have been considered, such as word embedding, logical
models, graphical models, rule systems, contextual focusing, and machine learning .Practical or
large-scale solutions avoid these complex methods and instead use only surface syntax or lexical
relationships, but are correspondingly less accurate. However, even state-of-the-art systems are
still far from human performance; a study found humans to be in agreement on the dataset 95.25%
of the time, while algorithms from 2016 had not yet achieved 90%.

Applications
Many natural language processing applications, like question answering, information
extraction, summarization, multi-document summarization, and evaluation of machine
translation systems, need to recognize that a particular target meaning can be inferred from
different text variants. Typically entailment is used as part of a larger system, for example in a
prediction system to filter out trivial or obvious predictions.Textual entailment also has
applications in adversarial stylometry, which has the objective of removing textual style without
changing the overall meaning of communication.

6.3 Robust and scalable Machine Translation

Machine translation (MT), process of translating one source language or text into another
language, is one of the most important applications of NLP. We can understand the process of
machine translation with the help of the following flowchart −

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

Types of Machine Translation Systems

There are different types of machine translation systems. Let us see what the different types are.
1. Bilingual MT System
Bilingual MT systems produce translations between two particular languages.
2. Multilingual MT System
Multilingual MT systems produce translations between any pair of languages. They may be either
uni-directional or bi-directional in nature.

Approaches to Machine Translation (MT)

Let us now learn about the important approaches to Machine Translation. The approaches to MT
are as follows −
1. Direct MT Approach
It is less popular but the oldest approach of MT. The systems that use this approach are capable of
translating SL (source language) directly to TL (target language). Such systems are bi-lingual and
uni-directional in nature.
2. Interlingua Approach

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

The systems that use Interlingua approach translate SL to an intermediate language called
Interlingua (IL) and then translate IL to TL. The Interlingua approach can be understood with the
help of the following MT pyramid −

3. Transfer Approach
Three stages are involved with this approach.
 In the first stage, source language (SL) texts are converted to abstract SL-oriented
representations.
 In the second stage, SL-oriented representations are converted into equivalent target
language (TL)-oriented representations.
 In the third stage, the final text is generated.
4. Empirical MT Approach
This is an emerging approach for MT. Basically, it uses large amount of raw data in the form of
parallel corpora. The raw data consists of the text and their translations. Analogy based, example-
based, memory-based machine translation techniques use empirical MTapproach.

6.4 Multilingual Question Answering

Multilingual question answering is a sub-field of question answering or QA, a discipline within

the computer science that is closely related to both the field of information retrieval and natural
language processing. It focuses on creating systems that don’t only answer questions but also allow
the user to access information in other languages than that used for asking questions.
The main idea of the question answering (QA) system is to automatically provide an answer to the
user’s query. To do that, the system obviously needs an access to a database of relevant information
such as the knowledge base for instance. However, most QA systems rely on information obtained
from various collections of natural language documents which are in most cases unstructured, e.g.
collection of reference texts, news reports, world wide web pages, etc.

Several multilingual QA systems have been built in the last few years, all seeking to overcome not
only the challenges of monolingual QA but also to overcome the issue of accessing and retrieving
information in multiple languages. The majority, however, are based on translating relevant
sections of the question – usually with the aid of machine translation system - which are then used

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

to access to a collection containing relevant information. Most commonly used multilingual QA

systems also rely heavily on the monolingual system to answer the question.

Though current multilingual QA system needs further development, it is highly important to

mention that, just like its monolingual counterpart, it too depends greatly on a good corpus or
database. Without it, even the most sophisticated QA system cannot provide satisfactory results.

The traditional architecture of a monolingual QA system considers three basic modules: (i)
question classification, where the type of expected answer is determined, (ii) passage retrieval,
where the passages with the greatest probability to contain the answer are obtained from the target
document collection, and (iii) answer extraction, where candidate answers are ranked and the final
answer recommendation of the system is produced. In addition, a multilingual QA system must
include two other modules, one for question translation and another for information merging. The
purpose of the first module is to translate the input question to all target languages, whereas the
second module is intended to integrate the information extracted from these languages into one
single ranked list.

Figures 1 and 2 show two different architectures for multilingual QA. For the sake of simplicity,
in both cases we do not consider the module for question classification. On one hand, Figure 1
shows a multilingual QA architecture that does the information merging at passage level. The idea
of this approach is to perform in parallel the recovery of relevant passages from all collections
(i.e., from all different languages), then integrate these passages into one single ranked list, and
finally, extract the answer from the combined set of passages. On the contrary, Figure 2 illustrates
an architecture that achieves the information merging at answer level. In this case, the idea is to
perform the complete QA process independently in all languages, and after that, integrate the sets
of answers into one single ranked list.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

It is important to mention that merging processes normally rely on the translation of information
to a common language. This translation is required for some merging strategies in order to be able
to compare and rank the passages and answers extracted from different languages. The two
proposed architectures have different advantages and disadvantages. For instance, doing the
information merging at passage level commonly allows obtaining better translations for named
entities (possible answers) since they are immerse in an extended context. On the other hand, doing
the merging at answer level has the advantage of a clear (unambiguous) comparison of the
multilingual information. In other words, comparing two answers (named entities) is a
straightforward step, whereas comparing two passages requires the definition of a similarity
measure and the determination of a criterion about how similar two different passages should be
in order to be considered as equal. This previous problem is not present in monolingual QA
ensembles, since in that case all individual QA systems search on the same document collection.

Given several sets of relevant passages obtained from different languages, the procedure for
passage merging considers the following two basic steps:

1. Translate all passages to one common language. This translation can be done by means of any
translation method or online translation machine. However, we suggest translating all passages

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

into the original question’s language in order to avoid translation errors in at least one passage set.
It is important to clarify that translation is only required by the CombSUM and CombMNZ
strategies. Nevertheless, all passages should be translated to one common language before entering
the answer extraction module.

2. Combine the sets of passages according to a selected merging strategy. In the case of using the
Round Robin or RSV approaches, the combination of passages is straightforward. In contrast,
when applying CombSUM or CombMNZ, it is necessary to determine the occurrence of a given
passage in two or more collections. Given that it is practically impossible to obtain exactly the
same passage from two different collections, it is necessary to define a criterion about how similar
two different passages should be in order to be considered as equal. In particular, we measure the
similarity of two passages by the Jaccard function (calculated as the cardinality of their vocabulary
intersection divided by the cardinality of their vocabulary union) and consider them as equal only
if their similarity is greater than a given specified threshold (empirically we set the threshold value
to 0.5). The procedure for answer merging is practically the same as that for passage merging. It
also includes one step for answer translation and another step for answer combination. However,
the combination of answers is much simpler than the combination of passages, since they are
directly comparables. In this case, the application of all merging strategies is straightforward.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

6.5 Cross Lingual Information Retrieval (CLIR).

Cross-lingual Information Retrieval is the task of retrieving relevant information when the
document collection is written in a different language from the user query. Figure 1 below shows a
typical architecture of a CLIR system. There are many situations where CLIR becomes essential
because the information is not in the user’s native language.

Figure 1. Typical architecture of a CLIR system .

Translation Approaches
CLIR requires the ability to represent and match information in the same representation space even
if the query and the document collection are in different languages. The fundamental problem in
CLIR is to match terms in different languages that describe the same or a similar meaning. The
strategy of mapping between different language representations is usually machine translation. In
CLIR, this translation process can be in several ways.
 Document translation is to map the document representation into the query representation
space, as illustrated in Figure 2.
 Query translation is to map the query representation into the document representation space,
as illustrated in Figure 3.
 Pivot language or Interlingua is to map both document and query representations to a third
space.

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

Figure 2. Document Translation for CLIR

Figure 3. Query Translation for CLIR.

Query translation is generally considered the most appropriate approach.
The query is short and thus fast to translate than the document, and it is more flexible and allows
more interaction with users if the user understands the translation. However, query translation can
suffer from translation ambiguity, and this problem is even more obvious for the short query text
due to the limited context. By contrast, document translation can provide more accurate translation
thanks to richer contexts. Document translation also has the advantage that once the translated
document is retrieved, the user can directly understand it, while the query translation still needs a
post-retrieval translation. However, several experiments show that there is no clear evidence of one
approach or the other using the same machine translation system , and the effectiveness is more
dependent on the translation direction .

MIT CORER, Barshi

Department of Computer Science & Engineering
 Natural Language Processing

Cross-language information retrieval (CLIR) is a subfield of information retrieval dealing with

retrieving information written in a language different from the language of the user's query. The
term "cross-language information retrieval" has many synonyms, of which the following are
perhaps the most frequent: cross-lingual information retrieval, translingual information retrieval,
multilingual information retrieval. The term "multilingual information retrieval" refers more
generally both to technology for retrieval of multilingual collections and to technology which has
been moved to handle material in one language to another. The term Multilingual Information
Retrieval (MLIR) involves the study of systems that accept queries for information in various
languages and return objects (text, and other media) of various languages, translated into the user's
language. Cross-language information retrieval refers more specifically to the use case where users
formulate their information need in one language and the system retrieves relevant documents in
another. To do so, most CLIR systems use various translation techniques. CLIR techniques can be
classified into different categories based on different translation resources:

 Dictionary-based CLIR techniques

 Parallel corpora based CLIR techniques
 Comparable corpora based CLIR techniques
 Machine translator based CLIR techniques
CLIR systems have improved so much that the most accurate multi-lingual and cross-lingual adhoc
information retrieval systems today are nearly as effective as monolingual systems. Other related
information access tasks, such as media monitoring, information filtering and routing, sentiment
analysis, and information extraction require more sophisticated models and typically more
processing and analysis of the information items of interest. Much of that processing needs to be
aware of the specifics of the target languages it is deployed in.
Mostly, the various mechanisms of variation in human language pose coverage challenges for
information retrieval systems: texts in a collection may treat a topic of interest but use terms or
expressions which do not match the expression of information need given by the user. This can be
true even in a mono-lingual case, but this is especially true in cross-lingual information retrieval,
where users may know the target language only to some extent. The benefits of CLIR technology
for users with poor to moderate competence in the target language has been found to be greater
than for those who are fluent. Specific technologies in place for CLIR services
include morphological analysis to handle inflection, decompounding or compound splitting to
handle compound terms, and translations mechanisms to translate a query from one language to
another.

******************

MIT CORER, Barshi

Department of Computer Science & Engineering