IEEE - 45670
Extensive Survey on Feature Extraction and
Feature Selection Techniques for Sentiment
Classification in Social Media
S.Sathish Kumar1
1Research Scholar,
Department of Computer Science,
Hindustan College of Arts and Science,
Coimbatore 641028.
ABSTRACT - Data Mining is a process of generating new
information from the existing datasets involving the
machine learning, statistics and database systems. Major
role of data mining is to turn raw data into useful
information by using mathematical algorithms for
predicting future events. Data mining is known as
knowledge discovery in data (KDD).The key properties of
data mining are automatic discovery of patterns,
prediction of likely outcomes, creation of actionable
information and focuses on large datasets and database.
Mining the information related to a content-oriented
extraction on the basics of feature of data. For accessing
the features of data that is data retrieval can be done by
feature extraction mechanism. Various feature extraction
methods have been used along with a feature selection
algorithm for mining the large datasets.
Keywords: Knowledge Discovery, Feature Selection
Algorithm, Data Retrieval
I. INTRODUCTION
In machine learning feature selection is the process
of selecting a subset of related features, termed as
variable selection, attribute selection for building the
model. A selection of features is independent of any
machine learning algorithms residing within the data.
The evolution of features can be done using
dependent and independent measures [5]. The
dependent measures of features subset are evaluated
and performance of algorithm is monitored.
Evaluation is done without algorithms for
independent measures. The different types of
independent measures are distance, information,
dependency and consistency.[15] The feature
selection removes the redundant data using feature
selection algorithms involving filter wrapper and
embedded techniques [7].
Filter technique is classified into multivariate and
univariate without considering the classifiers. In
10th ICCCNT 2019
July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India
Dr.Aruchamy Rajini2
2 Assistant Professor,
Department of Computer Science,
NGM College,Pollachi 624001
Tamil nadu.
wrapper technique dependency of features are
considered for training and testing in feature space
[6]. Embedded methods works better than wrapper
method and integrate feature selection process into
training model process. Feature extraction is
classified into linear and non-linear. Issues of pattern
recognition are an important part of feature
extraction. Learning mechanisms is classified into
supervised, unsupervised and reinforcement learning
[12].
To build a classifier model, feature classifications
use the available resources such as Filter approach,
Wrapper approach and embedded approach. Feature
extraction and feature selection is a process of
reducing the inputs for processing and analyzing the
inputs [3]. More number of features will not be
informative, due to presence of irrelevant information
or redundant features in a class. Relevant and non-
redundant feature is to be extracted and selected for
effective and efficient classification. Feature
selection methods is divided into lexicon based
methods and statistical methods [1]. The terms are
eliminated and features are selected from the list of
vocabulary features identified, among them are
information gain (IG), CHI-Statistic (CHI), Gain
Ratio (GR), Document Frequency (DF) and Relief-F
[2].Residual part of the paper is planned as follows.
Section II provides related works. In section III,
provides proposed sentimental classification model
Section IV provides concludes the paper with future
process.
II.RELATED WORKS
MondherBouazizi and TomoakiOhtsuki [14] 2019,
had proposed sentiment analysis refers to the
automatic collection, aggregation, and classification
of data collected online into different emotion
classes. The proposed approach of multi-class
 IEEE - 45670
classification achieves an accuracy of 60.2% for 7
different sentiment classes which, compared to an
accuracy of 81.3% for binary classification,
emphasizes the effect of having multiple classes.
Hai Ha Do et al. [15] 2019, had anticipated current
research focus for sentiment analysis was the
improvement of granularity at aspect level,
representing two distinct aims: aspect extraction and
sentiment classification of product reviews and
sentiment classification of target-dependent tweets.
Deep learning approaches have emerged as a
prospect for achieving these aims with their ability to
capture both syntactic and semantic features of text
without requirements for high-level feature
engineering, as was the case in earlier methods. In
that article, they aim to provide a comparative review
of deep learning for aspect-based sentiment analysis
to place different approaches in context.
Bowen Zhang et al. [16] 2019, had suggested
sentiment analysis was an important task in natural
language processing. Previous studies have shown
that integrating the knowledge rules into
conventional classifiers can effectively improve the
sentiment analysis accuracy. A critic learning based
convolutional neural network, which can address the
two shortcomings. Our method was composed of
three key parts, a feature-based predictor, a rule-
based predictor and a critic learning network.
Extensive experiments are carried out, and the results
show that the proposed method achieves better
performance than state-of-the-art methods in
sentiment analysis.
AsadAbdi et al. [17] 2019, had proposed sentiment
analysis concerns the study of opinions expressed in
a text. They present a deep-learning-based method to
classify a user's opinion expressed in reviews (called
RNSA).The RNSA employs the Recurrent Neural
Network (RNN) which was composed by Long
Short-Term Memory (LSTM) to take advantage of
sequential processing and overcome several flaws in
traditional methods, where order and information
about the word are vanished.
Shiyang Liao et al. [18] 2017, had planned an
approach to understand situations in the real world
with the sentiment analysis of Twitter data base on
deep learning techniques. Recently, deep learning
was able to solve problems in computer vision or
voice recognition, and convolution neural network
(CNN) works well for image analysis and image
classification. The result shows that it achieves better
accuracy performance in twitter sentiment
classification than some of traditional method such as
the SVM and Naive Bayes methods.
10th ICCCNT 2019
July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India
Classification algorithm such as Naïve Bayes
(NB), K-Nearest Neighbor (KNN), Logistic Model
Tree (LMT) and Radial Basis Function (RBF) in text
classification are compared. Interaction between
feature subset search and model selection in wrapper
based approach results in high performance than filter
based approach. The selection of classification
algorithm in a wrapper based approach decides the
accuracy of classifier. For choosing the best
classification algorithm to work with feature
selection algorithms in a wrapper based approach
plays a vital role for sentiment classification.
Categorization of content into polarity levels such as
positive, negative, and neutral is known as sentiment
classification.
3.1 DATA PREPROCESSING
Data preprocessing is a data
mining technique that involves transforming
raw data into an understandable format. Real
world data is often incomplete, inconsistent and
lacking in certain behaviors or trends and likely to
contain many errors. Data preprocessing includes
 Cleaning
 Integration
 Transformation
 Reduction
Cleaning process fills in the missing values
in data and identifies outliers. Cleaning also smooth
outs the noisy data. The Integration uses multiple
databases and files. Transformation is a process
involving normalization, aggregation and
generalization. The data are reduced in attribute
numbers is reduction [4].
3.2 FEATURE EXTRACTION
Feature extraction consists of transforming
arbitrary data, such as text or images, into numerical
features which is usable for machine learning [12]. It
efficiently represents interesting parts of an image as
a compact feature vector starts from initial set of
measured data and builds derived values by
dimensionality reduction.
3.2.1 Types of Stemmer
A stem is a natural group of words with equal (or
very similar) meaning. This method describes the
base of particular word. Inflectional and derivational
stemming are two types of method [8]. The stemming
algorithms can be classified as follows,
Truncating [1. Lovins 2. Porters 3. Paice/Husk 4.
Dawson]
Statistical [1. N-Gram 2. HMM 3. YASS]
 IEEE - 45670
Mixed [1.Inflectional & Derivational a) Krovetz b)
Xerox 2. Corpus Based 3. Context Sensitive]
Steming:
1 probability of occurrence associated with each word.
CJ= ----
|CJ| Σdi … (1)
di is the document vector in the set Cj
j is the number of documents in Cluster Cj.
3.2.2 Stop Words Removal
The major forms of pre-processing are to filter out
useless data. The useless words (data) are referred to
as stop words in natural language processing [16].
The usual words like a, an, but, and, of, the etc. is
removed while indexing the entries.
3.2.3 Tokenization
The given document is considered as a string and
identifying single word in document i.e. the given
document string is divided into one unit or token, that
has no extrinsic or exploitable meaning or value [9].
Through a tokenization system, the token is a
reference (i.e. identifier) that maps back to the
sensitive data. Using tokens created from random
numbers, original data gets mapped to token using
methods which render tokens infeasible to reverse in
the absence of the tokenization system.
3.2.4 Normalization
Normalization divides the larger tables into smaller
tables and links them using relationships [14]. The
series of restructuring a relational database into a
normal form, in order to reduce data redundancy
improves data integrity. Repeated storage of same
information leads to update anomaly problem, which
can be overcome with the help of normalization
process.
X new = x −µ
--------
O … (2)
3.2.5 N-Gram
N-gram model sequences of natural languages
utilize the statistical properties of n-grams. The n-
items contiguous sequence for given sample of text
or speech is called as n-grams or shingles [10]. Based
on the size of the n-gram, is classified as,
 Unigram (Size 1)
 Bigram or Diagram(Size 2)
 Trigram (Size 3)
3.2.5.1 Unigram
Each word occurrence in a document is
10th ICCCNT 2019
July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India
independent of all other word occurrences for the
unigram posits [11] [13]. The document generation
process as a sequence of dice rolls with a fixed
The product of the word probabilities provides the
chance of observing a given document.
P(wi ∣ w1…wi−1 )≈P(wi )= c (wi ) ∑w̃ c (w̃ ) … (3)
3.2.5.2 Bigram
Every bigram’s frequency distribution in a
string is commonly used for simple statistical
analysis of text in many applications. This includes in
computational linguistics, cryptography, speech
recognition, and so on.
3.2.5.3 Trigram
The Cryptanalytic frequency analysis has found 16
common character-level trigrams in English. Context
is very important for the varying analysis of rankings
and percentages, which are easily derived by drawing
from different sample sizes, different document
types: poetry, science-fiction, technology
documentation and writing levels.
FEATURE SELECTION
Feature selection differs from dimensionality
reduction, but these methods seek to reduce the
number of attributes in the dataset [15].
Dimensionality reduction method is for creating new
combinations of attributes, where as feature selection
methods include and exclude attributes present in the
data without changing them. Feature selection is used
for Enabling the machine learning algorithm to train
faster. Reduces the complexity of a model and makes
it easier to interpret. Improves the accuracy of a
model if the right subset is chosen. Reduces over
fitting. Filter methods are generally used as a
preprocessing step. The selection of features is
independent of any machine learning algorithms.
Instead, features are selected on the basis of their
scores in various statistical tests for their correlation
with the outcome variable. In wrapper methods, a
subset of features and train a model using them.
Based on the inferences that are draw from the
previous model, subsets are added or removed. These
methods are usually computationally very expensive
[12]. Embedded methods combine the qualities’ of
filter and wrapper methods. It’s implemented by
algorithms that have their own built-in feature
selection methods.
3.3.1 Chi Square (CHI)
The Chi Squared Statistic (CHI) measures
the association between the word feature and its
associated class or category [15]. CHI as a common
 IEEE - 45670
statistical test represents divergence from the
distribution expected based on the assumption that
the feature occurrence is perfectly independent of the
class value.
… (4)
3.3.2 Information Gain (IG)
IG targets at the removal of features that
produce less information based on the presence and
absence of terms in the document. Information gain is
utilized as a feature goodness criterion in machine
learning based classification [12]. IG measures
information obtained for class prediction of an
arbitrary text document by evaluating the presence or
absence of a feature in the text document
… (5)
Where,
R- response value.
|T| is the number of observations
4.3 Odds Ratio (OR)
Odds ratio measures the odds of the word
occurring in the positive class normalized by that of
the negative class [5]. The basic idea is that the
distribution of features on the relevant documents is
different from the distribution of features on the non
relevant documents.
Odds ratio PG1(1-PG1) …….(6)
=PG /2 (1 − PG) 2
3.4 Sequential Minimal Optimization (SMO)
Sequential Minimal Optimization trains a
support vector machine requires the solution of a
very large quadratic programming (QP) optimization
problem. SMO breaks this large QP problem into a
series of smallest possible QP problems [18][14].
These small QP problems are solved analytically,
which avoids using a time-consuming numerical QP
optimization as an inner loop. The amount of
memory required for SMO is linear in the training set
size, which allows SMO to handle very large training
sets. Because matrix computation is avoided, SMO
scales somewhere between linear and quadratic in the
training set size for various test problems, while the
standard chunking SVM algorithm scales somewhere
between linear and cubic in the training set size.
SMO’s computation time is dominated by SVM
evaluation, hence SMO is fastest for linear SVMs and
sparse data sets. On real- world sparse data sets,
SMO can be more than 1000 times faster than the
chunking algorithm [14].
10th ICCCNT 2019
July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India
III PROPOSED SENTIMENTAL
CLASSIFICATON MODEL
Sentiment classification is a special task of
text classification whose objective is to classify a text
according to the sentimental polarities of opinions it
contains favorable or unfavorable, positive or
negative. Sentimental analysis is used to improve the
accuracy of the classifiers for business decisions
made to improve the business. Statements received
from web consist of words which process
classification model. Classification algorithms are
used for enhancing business in e-commerce sites and
where the accuracy of the classifiers has to be
increased. Meta heuristic algorithms can be used in
selecting the features to enhance the accuracy of the
classifiers [12].
Classification algorithm such as Naïve Bayes
(NB), K-Nearest Neighbor (KNN), Logistic Model
Tree (LMT) and Radial Basis Function (RBF) in text
classification are compared. Interaction between
feature subset search and model selection in wrapper
based approach results in high performance than filter
based approach. The selection of classification
algorithm in a wrapper based approach decides the
accuracy of classifier. For choosing the best
classification algorithm to work with feature
selection algorithms in a wrapper based approach
plays a vital role for sentiment classification.
Categorization of content into polarity levels such as
positive, negative, and neutral is known as sentiment
classification.
Naïve Bayes algorithm for sentimental analysis is a
process of finding that, a document can be classified
as a positive or a negative sentiment. KNN is quite
different from the other classification algorithms
[11]. Instead of training a model by using the
available data set, this model uses lazy learning
approach. This algorithm will not generalize the
model before a query is made. Based on the input
query, the model will form a specific model.
Fig. 1. Diagrammatic representation of accuracies
for Naïve Bayes and KNN.14
 IEEE - 45670
Classification involves the process of extracting
the features in the given statement and classifying the
input statement based on the polarity of features
extracted. Once the input reviews are classified, the
classification accuracy of the different algorithms is
measured by comparing the actual sentiment of the
reviews with the classified sentiment [13]. Feature
extraction is a process in data mining that involves
the steps for reducing the amount of resources
required to describe a large set of data. Major
problem in mining and analysis of a complex data is
availability of large number of attributes in the data
set. Applying feature extraction techniques to the
data set before it is given as input to the classifier
results in improving the accuracy of the classifier
model [10].
In Figure 1, the NB and also KNN classification
technical on considering accuracy are compared.
From this above figure for 100 training dataset, it is
perceived that the Naïve bayes technique offer 56.78
accuracy, KNN techniqueoffer47.64.Similarly,for all
the training dataset, the accuracy is varied. Hence,
from the Figure 1 it is oblivious that the NB give the
better accuracy.
IV CONCLUSION
Efficient sentimental classification models the
algorithms used at the step of feature selection plays
an important role. Process involved in feature
selection improves the overall accuracy of the
classifier. Features elected based on the mathematical
formulas are easily implemented and used with a
classifier. The increasing demand sentimental
analysis is to improve the accuracy of the classifiers
based on which important business decisions can be
made to improve the business. In general, for feature
selection the list of features are not apparently fixed.
Statements or tweets received from web consist of
perplexing words which makes process of
classification more difficult. To make use of the
classification algorithms for enhancing business in e-
commerce sites the accuracy of the classifiers has to
be increased.
REFERENCES
[1]EmaKusen and Mark Strembeck,"Something
draws near, I can feel it: An analysis of human and
bot emotion-exchange motifs on Twitter",Online
Social Networks and Media, Vol.10-11, pp.1-17,
2019.
[2]KiichiTago and QunJin,"Influence Analysis of
Emotional Behaviors and User Relationships Based
on Twitter Data",TSINGHUAScience and
Technology, Vol.23, No.1, pp.104-113, 2018.
10th ICCCNT 2019
July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India
[3]Avinash Chandra Pandey, Dharmveer Singh
Rajpoot and MukeshSaraswat,"Twitter sentiment
analysis using hybrid cuckoo search
method",Information Processing and Management,
Vol.53, pp.764-779, 2017.
[4]Neha Singh, Nirmalya Roy and
AryyaGangopadhyay,"Analyzing The Emotions of
Crowd For Improving The Emergency Response
Services",Pervasive and Mobile Computing, Vol.58,
pp.1-33, 2019.
[5]Chae Won Park and DaeRyongSeo,"Sentiment
Analysis of Twitter Corpus Related to Artificial
Intelligence Assistants",International Conference on
Industrial Engineering and Applications, pp.495-498,
2018.
[6]KashfiaSailunaz and RedaAlhajj,"Emotion and
Sentiment Analysis from Twitter Text",Journal of
Computational Science, pp.1-42, 2019
[7]AnkushChatterjee, Umang Gupta, Manoj
Kumar Chinnakotla, RadhakrishnanSrikanth, Michel
Galley and PuneetAgrawal,"Understanding emotions
in text using deep learning and big data",Computers
in Human Behavior, Vol.93, pp.309-317, 2019.
[8]RavinderAhuja, Aakarsha Chug, ShrutiKohli,
Shaurya Gupta and PratyushAhuja,"The Impact of
Features Extraction on the Sentiment
Analysis",Procedia Computer Science, Vol.152,
pp.341-348, 2019.
[9]M. Ghiassi and S. Lee,"A Domain Transferable
Lexicon Set for Twitter Sentiment AnalysisUsing a
Supervised Machine Learning Approach",Expert
Systems With Applications, Vol.106, pp.197-216,
2018.
[10]FazeelAbid, Muhammad Alam, Muhammad
Yasir and Chen Li,"Sentiment analysis through
recurrent variants latterly on convolutional neural
network of Twitter",Future Generation Computer
Systems, Vol.95, pp.292-308, 2019.
[11]Eric S.Tellez, Sabino Miranda-Jiménez, Mario
Graff, Daniela Moctezuma, Oscar S.Siordia and Elio
A.Villasenor,"A case study of Spanish text
transformations for twitter sentiment
analysis",ExpertSystems with Applications, Vol.81,
pp.457-471, 2017.
[12]PragyaTripathi, Santosh Kr Vishwakarma and
Ajay Lala,"Sentiment Analysis of English Tweets
Using RapidMiner",International Conference on
Computational Intelligence and Communication
Networks, pp.668-672, 2015.
[13]Ahmed Sulaiman M Alharbi and Elise
Donckerde,"Twitter Sentiment Analysis with a Deep
Neural Network: An Enhanced Approach using User
Behavioral Information",Cognitive Systems
 IEEE - 45670

Research, Vol.54, pp.50-61, 2019. convolutional neural networks with

[14] MondherBouazizi and rules",Neurocomputing, Vol.356, pp.21-30, 2019.
TomoakiOhtsuki,"Multi-Class Sentiment Analysis on [17] AsadAbdi, SitiMariyamShamsuddin,
Twitter: Classification Performance and ShafaatunnurHasan and JalilPiran,"Deep learning-
Challenges",Big Data Mining and Analytics, Vol.2, based sentiment classification of evaluative text
No.3, pp.181-194, 2019. based on Multi-feature fusion",Information
[15] Hai Ha Do, PWC Prasad, Angelika Maag and Processing & Management, Vol.56, No.4, pp.1245-
AbeerAlsadoon,"Deep Learning for Aspect-Based 1259, 2019.
Sentiment Analysis: A Comparative Review",Expert [18] Shiyang Liao, Junbo Wang, Ruiyun Yu,
Systems with Applications, Vol.118, pp.272-299, Koichi Sato and ZixueCheng,"CNN for situations
2019. understanding based on sentiment analysis of twitter
[16]Bowen Zhang, XiaofeiXu, Xutao Li, Xiaojun data",Procedia Computer Science, Vol.111, pp.376-
Chen, Yunming Ye and ZhongjieWang,"Sentiment 381, 2017.
analysis through critic learning for optimizing

10th ICCCNT 2019

July 6 -8, 2019 - IIT, Kanpur,
Kanpur, India