Capstone-final-review

Automatic Hate/ abusive speech

identification in online social network

Name: Rishi Tej Rao Kulakarni

Reg.no: 19bit0029
INTRODUCTION
 INTRODUCTION:
 The hate speech is a harmful form of speech that targets individuals or groups based on their identity, such as their
race, ethnicity, religion, gender, sexual orientation, or disability. The rise of online platforms has made it easier for
hate speech to spread, leading to increased social polarization and discrimination. To address this problem, hate
speech detection has become an essential tool for online platforms and law enforcement agencies to identify and
remove harmful content. initial the hate speech is more in the English but by the growing the internet world the hate
content with the multiple languages and code- mixed languages are becoming more and more, to identify this
language and the code- mixed languages in online social media platforms is first most important and the code
mixed the reginal language is growing more. The lot of people and the organizations were victims of these hate
content in online social media, during the elections also the targeted parties also getting victims of these groups.

The social media provides the multiple language to address there felling one such language is Telugu,
which is the one the oldest spoken language in the India which is written from the left to right, now a days the
people can speak and write multiple languages one such is Telugu-English, the usage of the Telugu-English
codemixed language becoming vast in the online media in terms of hate speech it is very difficult to find out this
type of speech in social media.
Ex:
Tel-Eng: Maa manchi akka
English: My good sister
In India most of the research of the hate detection is restricted to the code-mixed of Hindi-English only, but India has
the multiple diversity and the languages, the study of the Telugu-English code-mixed language is not improvised,
using code-mixed they are targeting the organization, groups, and family, to reduce this type of attacks we need the
automatic detection in the online social media.

The main aim of the paper is to address the problem of hate speech using the dataset of Telugu-
English code-mixed language which has two labels: hatespeech (HS), non-hate speech (NHS). Collected the text data
from the different sources like Facebook, twitter etc, the extracted tweets from these platforms where label with
binary classes called HS, NHS. with the Telugu-English known person.

Proposed the hate speech detection models for the Telugu-English mixed language using the
new dataset which is created by extracting the tweets from the Facebook, twitter. 2 Here we used the two types of
feature extraction methods called Term frequency (TF)- Inverse document frequency (IDF) and Count vectorizer and
multiple machine learning classifiers algorithms.
BACKGROUND:
Speech that targets, threatens, or disparages an individual or group because of their race, ethnicity, national origin,
religion, sexual orientation, gender identity, or other qualities is referred to as hate speech. It can have a big influence
on people and communities, causing them to feel afraid, intimidated, and excluded. Because of the rise in hate
speech occurrences brought on by social media and other online platforms, it is more crucial than ever to provide
efficient tools for identifying and combating such speech. The practise of detecting hate speech entails analysing vast
amounts of text using machine learning and natural language processing techniques in order to spot language that is
likely to be hurtful or hateful. With the purpose of detecting hate speech, numerous groups and research teams have
created datasets and models, frequently combining rule-based methods with machine learning techniques. Yet,
identifying hate speech is a challenging process that calls for constant work to address the underlying causes of such
speech, careful consideration of context and cultural standards, and other factors.
MOTIVATION:
Online communities and individuals may become hostile and threatening due to hate speech. Particularly negative
effects on marginalised groups, including racial and ethnic minorities, people, and religious minorities. Internet
venues that fail to take action against hate speech risk harming their reputation as a business. Online platforms may
demonstrate that they take their responsibilities seriously and are dedicated to fostering a pleasant online
environment by identifying and responding to hate speech. A secure and inclusive online environment, the protection
of vulnerable groups, compliance with laws and regulations, and brand reputation all depend on the ability to
identify hate speech on social media.
PROJECT STATEMENT:
In the growing internet world the data in the social media is increasing the social media became the open platform to
express their opinions in public but day by day the abusive content in the social media is increasing initial it is
responsible for the respective platform to take care of their users, the comment made in English it is easy to review
but later on region specific and multi lingual people are increasing and try to comment more and more on the mixed
words so till now in the India most of the reach papers where on the Hindi English code mixed word but there is lot
of abusive content on the online social media with different region specific language in these paper I am working on
the Telugu English code mixed language in these code mixed there is no such work done so here working on this
topic here we first use the NLP for the feature extraction of data on the dataset which was prepared by own we use
different feature extraction of Uni-gram, bi-gram, tri-gram further more after that for the training the dataset we use
the different machine learning models for the detection of the abusive/ Offensive hate speech on online social media.
OBJECTIVES:
1)To reduce the amount of hate speech and promote positive and respectful communication online.
2) To extract the feature from the text data.
3) Improve the model to obtain the better accuracy level.
4) Validate using the k-fold method.
5) To automatically identify and flag potentially offensive or harmful language in text, such as comments on social
media or other online platforms

SCOPE OF THE PROJECT:

This project is to detect the abusive or hate speech tweets which are written in the code-mixed language of
Telugu-English language. Hate speech is increasing on the social media platform like twitter, Facebook,
what’sup, to automatically identify and flag potentially offensive or harmful language in text, such as
comments on social media or other online platforms. reduce this hate speech and protect vulnerable groups
from targeted abuse and harassment. Using the feature extraction and machine learning.
LITERATURE REVIEW
SUMMARY OF EXISTING WORK:
 CHALLENGES PRESENT IN EXISTING SYSTEM
Lack of clear definition: The absence of a precise and generally acknowledged definition of hate speech is one of the main
problems. It might be difficult to identify hate speech because different nations and organizations interpret it differently.

Contextual understanding: Because hate speech frequently depends on context, identifying it necessitates knowledge of
the cultural, social, and historical elements that influence the meaning of particular words and phrases. Current methods
might have trouble accurately classifying hate speech in various settings.

Language complexity: Slang, colloquialisms, and metaphors are just a few of the languages and expressions that can be
used to communicate hate. Due to this intricacy, it may be difficult for detection algorithms to correctly identify and classify
hate speech.

Bias: Current systems may be biased as a result of the training data utilized or the prejudices of the people who created
them. These prejudices may result in false positives or incorrect identification of hate speech.

Privacy concerns: Systems for detecting hate speech sometimes rely on gathering and examining vast volumes of data,
which presents privacy issues. People could feel uneasy with the concept of their online activity being tracked and analyzed
without their permission.

Dynamic nature of hate speech: Over time, hate speech changes, and fresh variations appear frequently. Detection
systems might find it difficult to keep up with these changes, which would eventually reduce their effectiveness.
This above shows the clear challenge facing by the current hate speech detection models, and there is no restriction the post
or comments to write in the particular language due to these, there are multiple languages and people of knowing bilingual
languages, who can post in combining of languages de to these it is very tough to identify the language of the post and which
context the post is posted.
 REQUIREMENTS
HARDWARE REQUIREMENTS
Process: intel I 5 (8th gen)
Ram: 4 GB (minimum)
Internal space: 15 – 30 GB

SOFTWARE REQUIREMENTS
Software used: python interpreter (Google-Colab) with the google engine in
the backend to run.
Language used: python
BUDGET:

GANTT CHART :
ANALYSIS AND DESIGN
Proposed methodology:
Prepare the dataset:
Extract the tweets from the online social media networks like Facebook, twitter etc,
identify the code mixed of Telugu-English language tweets and extract, and prepare the
dataset of hate and the normal tweets, tweets are extracted using the tools like Octoparse.
After extract the tweets now create a new column called label, after that label each tweet as
hate-Speech (HS), non-hate speech (NHS).
 

Pre-processing:
Preprocess the text data by removing stop words, punctuation, and other irrelevant
information. Also, perform data cleaning to eliminate duplicates, remove URLs and
hashtags.
 
Feature extraction:
Use natural language processing (NLP) techniques to extract features from the preprocessed text
data. Feature extraction methods like (Tf - Idf) is one of the Technique in which we first find the term
frequency and after that we will find the inverse document frequency, and another method is Count-
vectorizer in which we calculate the how many times the word repeat in the sentence.
After that in each method extract the Uni-gram, bi-gram, tri-gram.
 

Train the dataset:

After feature extraction now train the model k-cross validation technique in which we split the
dataset into the k-folds, in each iteration we split the dataset in n-folds, n-1 for the training and one
for the testing, and repeating the processing for the n-iterations for the better results. Now in the each
iteration for the training the models with the classifier models like SVM, Random forest, Logistic
Regression , Decision tree, Evaluate the performance of your model using metrics like precision,
recall, and F1-score.

 
New data test:
After training the testing the models now test with some new data.
 
ARCHITECTURE DIAGRAM:
Use-case diagram:
Complete design:
MODULE DISCRIPTION:
1)Dataset:

Collect the tweet from the social media, and prepare the dataset.

1.1) used the Octoparse tool to extract the tweets from the online social media like

Facebook, Twitter etc.

2) pre-processing:

Now clean the dataset by removing the stop words.

 
2.1) by importing the regular expression module to clean the text in the dataset
2.2) by the importing the Stop word clear the repeating
2.3) now create a cropus and store all the cleaned tweets and store them in the
cropus.
3) feature extraction:
 
Extract the feature from the dataset using the NLP techniques, there I have used the two
types of methods in the NLP called TF-IDF, bow methods
 
3.1) after the pre-processing use the TF-IDF, for the feature extraction where TF is
term frequency which means it calculates each term in sentence and IDF is the
inverse document frequency, calculate term in the whole sentence or
document.
 
 
3.2) the second method is bow method in which we use the Count vectorizer, in these
method, if the term present the sentence it will mark as 1 if not 0 such the if the
word repeats more time it will count that many times such that we get know how
import that word.
 
 
3.3) in the both methods we can extract the Uni-gram, bi-gram, tri-gram, and n-gram.
 4) k-fold cross validation:
 
In this method we used the we split the dataset into the k- number of folds and

in that k-number of folds we train the test, we train the k-1 folds and 1 test
and
iterate till the k-times.
 

5) machine learning models:

We use the machine models like SVM, Random-forest, Logistic regression,

decision tree to train and test the folds.

5.1) evaluate each model with the accuracy, f-1 score, recall, precision, confusion
matrix.
 
6) now test the model with the new data.
IMPLEMENTATION AND TESTING
DATA SET
I manually gathered tweets and comments from twitter to create the dataset for this study.
First, based on the selection criteria for public pages, I chose 15 different public twitter
pages from categories that each contained three to six chosen pages. After then, we
gathered every post made on the page between September 2023 and December 2023 and
noted the comments made on each one. The mixed TeluguEnglish postings and comments
were then screened by removing non-textual information. 5000 tweets are the outcome of
this process. 3000 posts and comments on separate pages were being filtered using the
keywords in total. The use of keywords made it easier to weed out postings and comments,
finally 1400 tweets where annotated and labeled with the Hate speech (HS), Non hate
speech (NHS).
Sample code:
1) using the tf-idf:
 
1.1) import the libraries:
 
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix,Confusion
MatrixDisplay
 
1.2) to read the dataset:

text=pd.read_csv('/content/data_set2.csv')

 
 
 
 
1.3) to get information of the data set:

text[' label']
 
text['tweet’]

1.4) pre-processing of the dataset:

The dataset is cleaned and tweets where stored in the corpus.
import nltk
nltk.download('stopwords')
 

import nltk
import re
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
ps= PorterStemmer()
corpus=[]
for i in range(0,len(text)):
     title=re.sub('[^a-zA-Z]',' ',text['tweet'][i])
     title=title.lower()
     title=title.split()
     title=[ps.stem(word) for word in title if not word in stopwords.words('english')]
     title='  '.join(title)
     corpus.append(title)
1.5) now the feature extraction:

from sklearn.feature_extraction.text import TfidfVectorizer
tfidf= TfidfVectorizer(ngram_range=(1,2)).fit(corpus)

To get the features from the feature extraction :

count_tokens=tfidf.get_feature_names_out()
count_tokens
 

Now the transform the dataset into the if-idf vectorization :

article_vect = tfidf.transform(corpus)
 
 
1.6) to train the model:

First the give the label of the X,Y:

Y=text[' label']
X=tfidf.transform(corpus)
Using Logistic Regression:

logreg= LogisticRegression()
logreg.fit(x_train,y_train)
logreg_predict=logreg.predict(x_test)
logreg_acc=accuracy_score(logreg_predict,y_test)
print("Test accuracy:{:.2f}%".format(logreg_acc*100))

Using SVM():

svcp=SVC()
svcp.fit(x_train,y_train)
svcp_predict=svcp.predict(x_test)
svcp_acc=accuracy_score(svcp_predict,y_test)
print("Test accuracy:{:.2f}%".format(svcp_acc*100))
1.7) now using the k-fold corss validation:

Logistic Regression:
from sklearn.model_selection import cross_val_score
score=cross_val_score(logreg,X,Y,cv=15)

SVM():

from sklearn.model_selection import cross_val_score
score=cross_val_score(svcp,X,Y,cv=15)
RESULTS:
LR:

SVM:
DT:

RF:
Confusion matrix:
LR():

SVM():
RF:

DT:
2) bow method:
 

2.1) import the libraries:

 
import pandas as pd
import numpy as np

2.2) to read the dataset:

text_data=pd.read_csv('/content/data_set2.csv')

2. 3) to get information of the data set:

text_data[' label']
 
text_data['tweet’]

 
 
 
2.4) pre-processing of the dataset:
 

import nltk
import re
nltk.download('stopwords')
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
ps= PorterStemmer()
corpus=[]
for i in range(0,len(text_data)):
     title=re.sub('[^a-zA-Z]',' ',text_data['tweet'][i])
     title=title.lower()
     title=title.split()
     title=[ps.stem(word) for word in title if not word in stopwords.words('english')]
     title=' '.join(title)
     corpus.append(title)
2.5) now the feature extraction:

 
from sklearn.feature_extraction.text import CountVectorizer
bow_article=CountVectorizer(ngram_range=(1,2)).fit(corpus)
count_tokens=bow_article.get_feature_names_out()
article_vect=bow_article.transform(corpus)
count_tokens
 
 
2.6) now to train the model:

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

from sklearn.metrics import accuracy_score, classification_report, confusion_matrix,Confusion
MatrixDisplay
 
 
X=bow_article.transform(corpus)
Y=text_data[' label']
Using Logistic Regression:

logreg= LogisticRegression()
logreg.fit(x_train,y_train)
logreg_predict=logreg.predict(x_test)
logreg_acc=accuracy_score(logreg_predict,y_test)
print("Test accuracy:{:.2f}%".format(logreg_acc*100))

Using SVM():

svcp=SVC()
svcp.fit(x_train,y_train)
svcp_predict=svcp.predict(x_test)
svcp_acc=accuracy_score(svcp_predict,y_test)
print("Test accuracy:{:.2f}%".format(svcp_acc*100))
Using Decision tree:

dec=DecisionTreeClassifier()
dec.fit(x_train,y_train)
dec_predict=dec.predict(x_test)
dec_acc=accuracy_score(dec_predict,y_test)
print("Test accuracy:{:.2f}%".format(dec_acc*100))

Using Random forest:

rf=RandomForestClassifier(n_estimators=100, random_state=42)
rf.fit(x_train,y_train)
rf_predict=rf.predict(x_test)
rf_acc=accuracy_score(rf_predict,y_test)
print("Test accuracy:{:.2f}%".format(rf_acc*100))
2.7) now using the k-fold corss validation:

Logistic Regression():

from sklearn.model_selection import cross_val_score
score=cross_val_score(logreg,X,Y,cv=5)

SVM():

from sklearn.model_selection import cross_val_score
score=cross_val_score(svcp,X,Y,cv=5)
decision tree:

from sklearn.model_selection import cross_val_score
score=cross_val_score(dec,X,Y,cv=5)

random forest:

from sklearn.model_selection import cross_val_score
score=cross_val_score(rf,X,Y,cv=5)
RESULTS:
LG():

SVM():
DT:

RF:
Confusion matrix:
LR():

SVM():
DT():

RF():
 SAMPLE OUTPUT:
Corpus:
After feature extraction:
To check hate or non-hate:
CONCLUSIONS AND FUTURE WORK:
In this study, a machine learning approach is suggested for identifying hate speech on social media. The study makes
an effort to create, put into practice, and evaluate text feature extraction and machine learning techniques specifically
for hate speech identification for the Telugu-English mixed code language. Understanding and defining hate and
non-hate speech on social media, looking into the many methods already in use to address the issue, and being
familiar with Telugu are all necessary for the research to be carried out successfully. Also, it's crucial to recognize the
many approaches used to build and design models with the ability to detect hate speech. These techniques include
gathering posts and comments to create the dataset, creating annotation standards, pre-processing and features
extraction using TF-IDF, CV, and mixed n-grams models training using SVM, DT, LR, and RF, and model testing.
The models were then contrasted using the findings of a 5-fold CV evaluation metric. The authors manually
classified the posts and comments in this paper into three categories: hate (HS), non-hate (NHS), and talks. This
resulted dataset of 1400 post with binary classes, based on these binary classes the machine models were developed,
these models SVM, DT, LR, and RF along with the two feature extraction methods along ngram, when we compare
using the average accuracy the logistic regression with the count vectorizer given the better result with 73.8%,
compared to SVM, DT, RF , on the other side when we compare the result with the F1-score, the LG with the bigram
given the better result of 87.0% compared to SVM, DT, RF. Finally the model based on the LR with the CV + bigram
gives the slightly better performance that compared with DT, SVM and RF.

In future work try to build the more size of the dataset

including the multiple classes such as HS,NHS, offensive etc, such that hate and offensive can be differentiated ,
using the more number of feature extraction techniques and multiple machine and deep learning techniques to
improve the score of accuracy and F-1, and try to combined the multiple languages and prepare a dataset which can
train and test of all languages, individual and code mixed, and implement on the online social media network such
Facebook, twitter , what’s up etc, to stop the hate speech and provide the user and hate free environment.