Natural Language Processing

Project Review-3

Cyber Bullying Detection System

Using Sentiment Analysis
Team Members:

Harsh Khandelwal- 17BIT0063

Vidit- 17BIT0246
Mohit Verma- 17BIT0284
Prishita Ray- 17BCE2405

Course Code: CSE4022

Slot: C2+TC2
Professor: Jagalingam Pushparaj
 Acknowledgement
We would like to thank our course instructor, Prof. Jagalingam Pushparaj for
guiding us throughout the project, and for solving our doubts, our group
members for contributing towards the project and ensuring its successful
completion. We would also like to thank VIT for providing us the opportunity to
work on this project.
 Certificate
This is to certify that the group members have successfully completed the project
component for Natural Language Processing with the topic Cyber Bullying Detection
System using Sentiment Analysis having the course code CSE4022 for the C2+TC2 slot
under the guidance of Prof. Jagalingam Pushparaj.

Name of group member: Name of Faculty:

Signature: Signature:
 Abstract
Cyberbullying includes sending, posting, or sharing negative, harmful, false, or mean
content about someone else over digital devices like cell phones, computers, and tablets
through SMS, text, and apps, or online in social media, forums, or gaming where people
can view, participate in, or share content. In this project we aim to build a system which
can be proved as a perfect cyber bullying detection system. For addressing this major issue
we will apply the knowledge of Natural Language Processing to develop a system that can
understand the sentiment behind a given text. The system will be trained using the Naïve
Bayes Analyzer and Pattern Analyzer on a predefined labelled dataset. Thereafter, the
performance of the system trained using each of these classifiers will be tested and
compared on actual comments posted on social media platforms.
 1.Introduction

With the advent of the digital era and social media platforms, the misuse of technology has
also started to increase. People post comments on almost any received content. This has
led to cyber bullying, that is demeaning, hurting or attempting to spoil the image of a
person who has uploaded some content through the use of abusive language, sarcastic
comments, or using derogatory words that are aimed towards attacking the sentiments of
the concerned person. This is even more noticeable in the way people react to pictures,
videos or any text posted by celebrities on their profile pages. Each post garners thousands
of comments, some of which are extremely controversial and hurtful. Moreover, with such
a vast user base, these comments get shared to multiple people, without the knowledge of
the victim. These are the worst outcomes of social media that we often overlook.
Therefore, a robust cyber bullying detection system is necessary to analyse these
comments, decide whether they are derogatory, such that these miscreants can be
targeted and necessary action can be taken against them.

In [1], the authors devise a model which encodes emotional semantics into vectors and is
trained by multi-task learning six different emotion-related tasks, including
emotion/sentiment analysis, sarcasm classification, stress detection, abusive language
classification, insult detection, and personality recognition. Sub-word level representations
in LSTM (Subword-LSTM) architecture have been introduced in [2] as a linguistic prior that
enables learning information about sentiment value of important morphemes in noisy text.
A new deep convolutional neural network approach has been proposed in [3] that exploits
from character- to sentence-level information to perform sentiment analysis of short texts
using the Stanford Sentiment Treebank, and Stanford Twitter Sentiment corpus.

The authors have proposed an unsupervised lexicon building method for the detection of
polar clauses in [4], which convey positive or negative aspects in a specific domain. In a
typical document, there are around 8% of conditional sentences that are one of the
commonly used language constructs in text. The study in [5] determines whether opinions
expressed on different topics in a conditional sentence are positive, negative or neutral.

Based on the above literature review, though the method suggested in [1] using emotion-
related tasks seems to be a useful approach when data involves only text messages, it may
not give accurate and quick results in social media posts that involve emoticons, and
factual positive or factual negative comments also, that may not be very evident . The idea
of [2] is quite practical as social media posts do contain a lot of noise, and it is difficult to
find the actual words. However, given that there are a lot of misspelt words and figures
involved, and an ever growing corpus of words used while messaging, attempting to find
root words can be a major task in itself and may not give good results. Usage of CNNs in
[3], to perform sentiment analysis is the closest to our implementation, as it trains on
actual message test based corpora. The approach in [4] is similar to that in [2], and again
may not give the right results. We consider the labels positive, negative and neutral as in
[5] in our approach as well with a few other additional labels.

To counter these problems, we propose and implement a cyber bullying detection system,
that classifies a given comment into positive, negative, factual negative, neutral or abusive
based on the subjectivity and polarity scores of that comment. The most innovative part of
our project is that we are not taking the decision only based on the comments because
study says that friendly comments can also be insulting but they should not be considered
as a part of cyber bullying. We will map the comments with the reactions posted from that
user id and if even we find that the comments are using inappropriate language but the
reaction posted by that user id is positive then we may not consider that comment in cyber
bullying category. Moreover, our model is trained on text from movie reviews that are very
similar to those posted in social media comments, using Naïve Bayes and Pattern
analyzers, that are likely to give a good accuracy. In addition, we are also testing our model
on actual Facebook comments to give us a picture of how it would be applicable to the
real-world scenario.
 2.Literature Review

(i)Authors: Peng Xu, Andrea Madotto, Chien-Sheng Wu, Ji Ho Parkand Pascale Fung

Title: Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training

Conceptual Model: Emo2Vec which encodes emotional semantics into vectors. Emo2Vec is
trained by multi-task learning six different emotion-related tasks, including
emotion/sentiment analysis, sarcasm classification, stress detection, abusive language
classification, insult detection, and personality recognition. Evaluation of Emo2Vec shows
that it outperforms existing affect-related representations, such as Sentiment-Specific
Word Embedding and DeepMoji embeddings with much smaller training corpora. When
concatenated with GloVe, Emo2Vec achieves competitive performances to state-of-the-art
results on several tasks using a simple logistic regression classifier

Methodology Used: Convolutional Neural Networks, Multitask Learning

Dataset used: The dataset, uses documents with emotional hashtags on the cease and
filter out any documents with URLs, quotations, or less than five phrases. The overall wide
variety of documents is about 1.9 million with four training: joy (36.5%), unhappiness
(33.Eight%), anger (23.5%), and fear (6%). The dataset is randomly break up right into a
teach (70%), validation (15%), and test set (15%) for experiments.

Related Work: For sentiment analysis, numerous classification models have been explored.
Transfer learning from the large corpus is also investigated to train a large model on a huge
emoji tweet corpus, which boosts the performance of affect related tasks. Multi-task
training has achieved great success in various natural language tasks, such as machine
translation, multilingual tasks, semantic parsing jointly learns POS tagging, chunking,
dependency parsing, semantic relatedness, and textual entailment by considering linguistic
hierarchy and achieves state-of-the-results on five datasets. For sentiment analysis, jointly
trains ternary and fine-grained classification with are current neural network and achieves
new state of-the-art results.

Future Works: In this paper, the author proposes Emo2Vec to represent emotion with
vectors using a multi-task training framework. Six affect-related tasks are utilized, including
emotion/sentiment analysis, sarcasm classification, stress detection, abusive language
classification, insult detection, and personality recognition. The authors empirically show
how Emo2Vec leverages multi-task training to learn a generalized emotion representation.
In addition, Emo2Vec outperforms existing affect-related embeddings on more than ten
different datasets. By combining Emo2Vec with GloVe, logistic regression can achieve
competitive performances on several state-of-the-art results.
(ii)Author: Aditya Joshi, Prabhu Ameya Pandurang, Manish Shrivastava, Vasudeva Varma
Title: Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code
Mixed Text

Conceptual Model: Sentiment analysis (SA) using code-mixed data from social media has
several applications in opinion mining ranging from customer satisfaction to social
campaign analysis in multilingual societies. Advances in this area are impeded by the lack
of a suitable annotated dataset. The author introduce introduce a Hindi-English (Hi-En)
code-mixed dataset for sentiment analysis and perform empirical analysis comparing the
suitability and performance of various state-of-the-art SA methods in social media. In this
paper, we introduce learning sub-word level representations in LSTM (Subword-LSTM)
architecture instead of character-level or word-level representations. This linguistic prior in
the architecture enables us to learn the information about sentiment value of important
morphemes. This also seems to work well in highly noisy text containing misspellings as
shown in our experiments which is demonstrated in morpheme-level feature maps learned
by our model. Also, the authors hypothesize that encoding this linguistic prior in the
Subword-LSTM architecture leads to the superior performance. Our system attains
accuracy 4-5% greater than traditional approaches on our dataset, and also outperforms
the available system for sentiment analysis in Hi-En codemixed text by 18%

Methodology: The author proposes a method of generating sub-word level

representations through 1-D convolutions on character inputs for a given sentence.
Formally, let C be the set of characters and T be an set of input sentences. The sentence s
T is made up of a sequence of characters [c1,....,cl] where l is length of the input.

Datasets: Comments from social media handles of famous celebrities like Salman Khan,
Narendra Modi.

Conclusion: The author introduces Sub-Word Long Short Term Memory model to learn
sentiments in a noisy Hindi-English Code Mixed dataset. We discuss that due to the
unavailability of NLP tools for Hi-En Code Mixed text and noisy nature of such data, several
popular methods for Sentiment Analysis are not applicable. The solutions that involve
unsupervised word representations would again fail due to sparsity in the dataset. Sub-
Word LSTM interprets sentiment based on morpheme-like structures and the results thus
produced are significantly better than baselines. Further work should explore the effect of
scaling of RNN and working with larger datasets on the results. In the new system, we
would like to explore more deep neural network architectures that are able to capture
sentiment in Code Mixed and other varieties of noisy data from the social web.
(iii)Authors: C´ıcero Nogueira dos Santos, Ma´ıra Gatti

Title: Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts

Conceptual Model: Sentiment analysis of short texts such as single sentences and Twitter
messages is challenging because of the limited contextual information that they normally
contain. Effectively solving this task requires strategies that combine the small text content
with prior knowledge and use more than just bag-of-words. In this work we propose a new
deep convolutional neural network that exploits from character- to sentence-level
information to perform sentiment analysis of short texts. We apply our approach for two
corpora of two different domains: the Stanford Sentiment Treebank (SSTb), which contains
sentences from movie reviews; and the Stanford Twitter Sentiment corpus (STS), which
contains Twitter messages. For the SSTb corpus, our approach achieves state-of-the-art
results for single sentence sentiment prediction in both binary positive/negative
classification, with 85.7% accuracy, and fine-grained classification, with 48.3% accuracy.
For the STS corpus, our approach achieves a sentiment prediction accuracy of 86.4%.

Methodology: Given a sentence, CharSCNN computes a score for each sentiment label τ
T. In order to score a sentence, the network takes as input the sequence of words in the
sentence, and passes it through a sequence of layers where features with increasing levels
of complexity are extracted. The network extracts feature from the character-level up to
the sentence-level. The main novelty in our network architecture is the inclusion of two
convolutional layers, which allows it to handle words and sentences of any size.

Dataset: Movie Reviews from IMDB, Twitter posts

Related works: There are a few works on neural network architectures for sentiment
analysis. A semi-supervised approach based on recursive autoencoders for predicting
sentiment distributions. The method learns vector space representation for multi-word
phrases and exploits the recursive nature of sentences. It is proposed a matrix-vector
recursive neural network model for semantic compositionality, which has the ability to
learn compositional vector representations for phrases and sentences of arbitrary length.
The vector captures the inherent meaning of the constituent, while the matrix captures
how the meaning of neighbouring words and phrases are changed.

Conclusion: The author present a new deep neural network architecture that jointly uses
character-level, word level and sentence-level representations to perform sentiment
analysis. The main contributions of the paper are: (1) the idea of using convolutional
neural networks to extract from character- to sentence level features; (2) the
demonstration that a feed-forward neural network architecture for sentiment analysis of
sentences; (3) the definition of new state-of the-art results for SSTb and STS corpora.
(iv)Authors: Hiroshi Kanayama, Tetsuya Nasukawa

Title: Fully Automatic Lexicon Expansion for Domain-oriented Sentiment Analysis

Conceptual Model: This paper proposes an unsupervised lexicon building method for the
detection of polar clauses, which convey positive or negative aspects in a specific domain.
The lexical entries to be acquired are called polar atoms, the minimum human-
understandable syntactic structures that specify the polarity of clauses. As a clue to obtain
candidate polar atoms, we use context coherency, the tendency for same polarities to
appear successively in contexts. Using the overall density and precision of coherency in the
corpus, the statistical estimation picks up appropriate polar atoms among candidates,
without any manual tuning of the threshold values. The experimental results show that the
precision of polarity assignment with the automatically acquired lexicon was 94% on
average, and our method is robust for corpora in diverse domains and for the size of the
initial lexicon.

Methodology: The flow of sentiment analysis system involves three steps. The first step is
sentence delimitation: the input document is divided into sentences. The second step is
proposition detection: propositions which can form polar clauses are identified in each
sentence. The third step is polarity assignment: the polarity of each proposition is
examined by considering the polar atoms. This section describes the last two processes.

Dataset: The author used Japanese corpora from discussion boards in four different
domains.

Related Work: Sentiment analysis has been extensively studied in recent years. The target
of SA in this paper is wider than in previous work. Separated facts from opinions and
assigned polarities only to opinions. In contrast, this system detects factual polar clauses as
well as sentiments. Unsupervised learning for sentiment analysis is also being studied. For
example, Hatzivassiloglou and McKeown (1997) labeled adjectives as positive or negative,
relying on semantic orientation. Turney (2002) used collocation with “excellent” or “poor”
to obtain positive and negative clues for document classification.

Conclusion: The author proposed an unsupervised method to acquire polar atoms for
domain-oriented SA, and demonstrated its high performance. The lexicon can be expanded
automatically by using unannotated corpora, and tuning of the threshold values is not
required. Therefore, even end-users can use this approach to improve the sentiment
analysis. These features allow them to do on-demand analysis of more narrow domains,
such as the domain of digital cameras of a specific manufacturer, or the domain of mobile
phones from the female users’ point of view.
(v)Authors: Ramanathan Narayanan, Bing Liu, Alok Choudhary

Topic: Sentiment Analysis of Conditional Sentences

Conceptual Model: This paper studies sentiment analysis of conditional sentences. The
aim is to determine whether opinions expressed on different topics in a conditional
sentence are positive, negative or neutral. Conditional sentences are one of the commonly
used language constructs in text. In a typical document, there are around 8% of such
sentences. Due to the condition clause, sentiments expressed in a conditional sentence
can be hard to determine. For example, in the sentence, if your Nokia phone is not good,
buy this great
Samsung phone, the author is positive about “Samsung phone” but does not express an
opinion on “Nokia phone” (although the owner of the “Nokia phone” may be negative
about it). However, if the sentence does not have “if”, the first clause is clearly negative.
Although “if” commonly signifies a conditional sentence, there are many other words and
constructs that can express conditions. This paper first presents a linguistic analysis of such
sentences, and then builds some supervised learning models to determine if sentiments
expressed on different topics in a conditional sentence are positive, negative or neutral.
Experimental results on conditional sentences from 5 diverse domains are given to
demonstrate the effectiveness of the proposed approach.

Related Work: There are several research directions in sentiment analysis (or opinion
mining). One of the main directions is sentiment classification, which classifies the whole
opinion document (e.g., a product review) as positive or negative (e.g., Pang et al, 2002;
Turney, 2002; Dave et al, 2003; Ng et al. 2006; McDonald et al, 2007). It is clearly different
from this work as we are interested in conditional sentences.

Future Works: The author will further improve the classification accuracy and study
related problems, e.g., identifying topics/features. Although there are some special
conditional sentences that do not use easily recognizable conditional connectives and
identifying them are useful, such sentences are very rare and spending time and effort on
them may not be cost-effective at the moment.
 3.Methodology
3.1 Datasets Used

We have created a corpus of common negative phrases, words and facts, using the
Selenium library to scrape the web and extract them.

For training, we have used the movie reviews dataset, available in the nltk corpus. This
dataset has actual review text annotated with their associated sentiments.

To generate the testing data, we have used the SocioFi tool that is a third party application
to extract actual Facebook comments in reaction to posts on the social media platform.
This data is not labelled, and our model predicts the sentiment label based on the text in
the comments.

3.2 Tools Used

In this project, we will be using the TextBlob library which is one of most efficient Natural
Language Processing libraries in python.

TextBlob: TextBlob is a Python (2 and 3) library for processing textual data. It divides the
preprocessing involved with sentiment analysis into common natural language processing
(NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis,
classification, translation, and more.

To implement our project, we will be using the Jupyter Notebook.

3.3 Implementation

i)To generate the corpus of negative and abusive phrases or words and factual negative
phrases, we will use import and use the Selenium driver. The driver will scrape the web,
and create a corpus of these phrases and words, that we will use as the abusive word
corpus.

ii) From the TextBlob library, we import the nltk corpus. It includes the movie reviews
dataset with labelled reactions that we will use as our training set.

iii) Next, we create a Naïve Bayes classifier, that trains on the review text available in
the movie reviews, using the abusive word corpus, which generates a polarity and
subjectivity score, for each of these texts, and a final label as abusive, negative, factual
negative, neutral, positive etc. is predicted. Based on the actual label and the predicted
label, the model learns.

iv) Similarly, we create a Support Vector Machines classifier, that trains on the same
movie reviews dataset using the cyber bullying corpus.

v) For generating the testing data, we will use SocioFi that is a third party application to
scrape Facebook and retrieve comments on posts. We will extract all the comments and
reactions from Facebook along with the user id that are posted.

vi) The models are independently tested on these extracted Facebook comments. The
models check the sentiment and subjectivity of each and every comment and set certain
parameters according to the user beyond which negative sentiment should not be
tolerated.
vii) For the comments which are predicted negative by the system, w the abusive words
in that comment are found and mapped to the abusive word corpus.

viii) These comments are then classified into labels such as abusive, negative, factual
negative, neutral and positive based on the models’ predictions, after they generate the
polarity and subjectivity (fact-based) scores for each of these comments.

ix) After doing the above process we store the user ids and map it with the reaction
user ids. If any user has posted any abusive stuff on the post but the reaction posted seems
to be positive then it will not be booked under cyber bullying because it may have some
sarcastic comment according to the admin of that post.

x) Finally we plot a bar graph for the percentage of comments in each category
predicted by both the models separately.

3.4 Naïve Bayes Analyzer

The Naïve Bayesian classifier works as follows: Suppose that there exist a set of
training data, D, in which each tuple is represented by an n-dimensional feature
vector, X=x 1,x 2,..,x n , indicating n measurements made on the tuple from n
attributes or features. Assume that there are m classes, C 1,C 2,...,C m . Given a
tuple X, the classifier will predict that X belongs to C i if and only if:
P(C i |X)>P(C j |X), where i,j [1,m]a n d i≠j.
P(C i |X) is computed as:
The Naïve Bayes Analyzer has been pretrained on the movie reviews dataset present in nltk
corpus.

3.5 Pattern Analyzer

The Pattern Analyzer is based on the pattern library, it is pretrained on the pattern library
dataset.

Pattern is a web mining module for the Python programming language . It has many tools
for data mining including sentiment analysis tools. Input text that can be a string, text,
sentence, chunk, word or a synset (a set of one or more synonyms), is divided into two
types: facts and opinions. Opinions carry people’s sentiments. The module has several
useful functions:

The sentiment() function returns a (polarity, subjectivity) tuple, an ordered set of values,
for the given sentence, based on the adjectives it contains, where polarity is a value
between -1.0 and +1.0 and subjectivity between 0.0 and 1.0.
The positive() function returns True if the given sentence’s polarity is above the threshold.

3.6 Performance Evaluation

Since the testing data is not labelled, to check the accuracy of the model, we need to
manually label the comments based on human intuition. We can then use a metric such as
RMSE to check its performance.
 4. Code and Outputs
Testing Dataset:

Testing Reactions(Manual Labelling)

Code and Outputs (Naïve Bayes Classifier)
ii) Code and Outputs (Pattern Analyzer)
 5. Results and Discussion
We compare the performances of the Naïve Bayes Analyzer and the Pattern Analyzer on
the same testing dataset.
The x-axis in the graph corresponds to the categories in order as follows: Positive, negative,
neutral, abusive, others.

The y-axis in the graph corresponds to the percentage of comments in each category in the
testing dataset.

From the graph we can see that, the pattern analyser classifies factual comments into
either neutral or others categories, whereas the Naïve Bayes analyser correctly classifies
factual comments to the facts category rather than classifying them as neutral. Their
performances in the other categories is similar as observed from the graphs.
 6.Conclusion
It is observed that the Naïve Bayes Analyzer gives more accuracy in terms of classifying
factual negative data, than does the Pattern Analyzer in TextBlob. Hence, it is more reliable
in predicting whether a particular comment can be considered as an example of cyber
bullying.
 7.References
[1] Emo2Vec: Learning Generalized Emotion Representation by
Multi-task Training by Peng Xu, Andrea Madotto, Chien-Sheng Wu, Ji
Ho Parkand Pascale Fung
[2] Towards Sub-Word Level Compositions for Sentiment Analysis of
HindiEnglish Code Mixed Text by Aditya Joshi, Prabhu Ameya
Pandurang, Manish Shrivastava, Vasudeva Varma
[3] Deep Convolutional Neural Networks for Sentiment Analysis of
Short Texts by C´ıcero Nogueira dos Santos, Ma´ıra Gatti
[4] Fully Automatic Lexicon Expansion for Domain-oriented
Sentiment Analysis by Hiroshi Kanayama, Tetsuya Nasukawa
[5] Sentiment Analysis of Conditional Sentences by Ramanathan
Narayanan, Bing Liu, Alok Choudhary