International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
Improved Feature Extraction and Classification -
Sentiment Analysis
M.Trupthi1, Suresh Pabboju2,
Research Scholar, Information Technology Department,
Computer Science Department, JNTUH, CBIT, Hyderabad, Telangana State,
Hyderabad, Telangana State, India.
Abstract—Sentiment analysis is becoming a popular area
of research and social media analysis, especially around user
reviews and tweets. It is a special case of text mining generally
focused on identifying opinion polarity, People are usually
interested to seek positive and negative opinions containing
likes and dislikes, shared by users. Therefore reviews or
product features have got significant role in sentiment analysis.
In addition to sufficient work being performed in text
analytics, feature extraction in sentiment analysis is now
becoming an active area of research. Feature based sentiment
analysis include feature extraction, sentiment classification and
finally the sentiment evaluation. In this paper, explored the
machine learning classification approaches with different
feature selection schemes to obtain a sentiment analysis model
for the movie review dataset and the results obtained are
compared to identify the best possible approach.
Keywords— Feature Extraction, Bag of Words,
Classification, Bigram Collocation, Information Features,
Evaluation
I. INTRODUCTION
Text mining is the process of discovering useful and
interesting knowledge from unstructured text. In order to
discover knowledge from unstructured text data, the first
step is to convert text data into a manageable representation.
A common practice is to model text in a document as a set
of word features, i.e., “bag of words” (BOW). Often, some
feature selection techniques are applied, such as stop-word
removal or stemming, to only keep meaningful features and
improve the accuracy using supervised classification
algorithm [1].
Sentiment Analysis or Opinion Mining is a challenging one
in Text Mining and Natural Language Processing for feature
extraction, classification and summarization of sentiments.
Sentiment Analysis also assists individuals and
organizations who are interested in knowing what other
people comment about a particular product, service topic,
issue and event to find an optimal choice for which they are
looking for. Sentiment Analysis in the context of
Government Intelligence aims at extracting public views on
government policies and decisions to infer possible public
reaction on implementation of certain policies [2].
Sentiments with respect to movie reviews, the reviews are
categorized into pos and neg categories. In this proposed
work simple Naïve Bayes Classifier is considered as a
baseline, using Boolean word feature extraction.
978-1-4673-8810-8/16/$31.00 ©2016 IEEE
G.Narasimha ,
3
Computer sciencience Department,
JNTUH, Jagital, Telangana State, India.
Sentiment analysis can be performed at three different
levels: document, sentence and aspect level. The document
level sentiment analysis aims at classifying the entire
document as positive or negative, (Pang et al, [3]; Turney,
[4]). The sentence level sentiment analysis is closely related
to subjectivity analysis. At this level each sentence is
analyzed and its opinion is determined as positive, negative
or neutral, (Riloff et al, [4]; Terveen et al, [5]). The aspect
leave l sentiment analysis aims at identifying the target of
the opinion. The basis of this approach is that every opinion
has a target and an opinion without a target is of limited use,
(Hu and Liu, [6]). The simplest strategy uses a bag-of-words
model. Which create lists of 'positive' and 'negative' words
and judge a document based on whether it has a
preponderance of positive or negative words. Creating such
lists is not easy; some of the words are likely to be quite
different for different kinds of topics. Initially, take a
collection of documents on some topic and rate them (by
hand) as positive or negative and then use this collection to
train a classifier model
Limitations:
The bag-of-words model does quite well in assessing short
reviews which are clearly addressed to a single object, but
they have several limitations:
 some important words are ambiguous in their opinion:
"low" is positive in "low price" but negative in "low
quality"
 the assessment does not reveal which aspects of the
product led to the positive or negative sentiment,
although this may be more crucial to the decision maker
to do the overall assessment
 If the review compares the item to other items, the bag-
of-words approach is unable to distinguish references to
the different items
Overcoming all three limitations requires a richer model
which can capture some of the structures of the language.
NGrams- Unigrams presents the simplest model for the
n-gram approach. It consists of all the individual words
present in the text. The bigram model defines a pair of
adjacent words. Each pair of words forms a single bigram.
The higher order grams can be formed in the similar way by
taking together the n adjacent words. Higher order n-grams
are more efficient in capturing the context as they provide
better understanding of the word position.
An n-gram defines a subsequence of n items from a given
sequence. It is used in various fields of natural language
 International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
processing and genetic sequence analysis. An n-gram model
defines a method for finding a set of n-gram words from a
given document. The commonly used models include
unigrams (n=1), bigrams (n=2) and trigrams (n=3). However
the value of n can be extended to higher level grams. The n-
gram model can be better explained with the following
examples:
Text: “Honesty is the best policy.”
Unigrams: “honesty”, “is”, “the”, “best”, “policy”.
Bigrams: “honesty is”, “is the”, “the best”, “best policy”.
Trigrams: “honesty is the”, “is the best”, “the best policy”.
NLP and Machine Learning approaches were used for the
process. Multiple experiments were carried out using
different feature sets and parameters to obtain maximum
accuracy.
II. LITERATURE SURVEY
Turney (2002) presented a simple unsupervised learning
algorithm to classify the reviews based on recommended
(thumbs up) or not recommended (thumbs down) reviews
online. The sentiment classification of a review is predicted
by the average semantic orientation (SO) of adjective or
adverb phrases in the review. Opinions are usually
expressed by adjectives and adverbs. They used Point-wise
Mutual Information (PMI) and Information Retrieval (IR) to
measure the similarity of pairs of words or phrases, which is
to calculate semantic orientation (SO) of a word or phrase
by subtracting mutual information between the word or
phrase and the reference word “excellent” from the mutual
information between the word or phrase and the reference
word “ poor”. The mutual information is the co-occurrence
of the two words or phrase among millions of online
documents. Using 14410 reviews in 4 domain areas, they
obtained 4% accuracy for the bank and automobile datasets,
and 66% accuracy for the movie review datasets. They
argued that movie reviews were difficult to classify, since
movie reviews usually contain description words such as
“bad scene” or “good scene” which are not sentiment words.
Although they received a decent result, the way they
calculated the semantic orientation (SO) of phrases was not
efficient enough as it involved retrieving millions of online
documents to get the co-occurrence of two words.
A. Finding appropriate features
Sentiment analysis is a kind of text classification task. In
supervised learning, a number of machine learning
algorithms can be used in text classification to classify text.
When using machine learning models, the major focuses of
supervised learning have two aspects: constructing
appropriate feature spaces and choosing appropriate
classification algorithms. In the Nature Language Processing
(NLP) tasks, features are also called terms or tokens. It is
important to find out right features when using machine
learning models for text mining. In sentiment analysis, many
efforts have focused on finding right features to improve
978-1-4673-8810-8/16/$31.00 ©2016 IEEE
classification performance. If a particular feature tends to be
highly consistent in the texts of a certain class (positive class
or negative class), then the algorithm will generalize that
this feature is a good indicator of that class (Brooke, 2009).
For example, beautiful may be a good indicator to
generalize a text as positive. To date, many features have
been applied in sentiment analysis, such as unigrams,
bigrams, trigrams, even higher level n-grams [6].
B. N-grams
An n-gram model is a type of probabilistic language model
for predicting the next word conditioned on a sequence of
previous words using Markov models. The probabilistic
expression is P(xi| xi-(n-1)…., xi-1 ). N-gram of size 1 is referred
to as unigram, size 2 as bigram, and size 3 as trigram. Since
n-grams are used for capturing dependencies between single
words that stay in a text sequentially, the combination of
words does not necessarily have syntactical or semantic
relations. Unigrams performed much better than bigrams
when used as features for feature spaces in Pang et al.
(2002), while bigrams and trigrams contributed higher
performance than unigrams in (Dave et al., 2003; Ng et al.,
21 2006). In Pang et al.(2002), unigrams also outperformed
adjectives when treated as features.
In feature spaces of machine learning models for text
classifications, the size of document vectors or sentence
vectors that are composed of Bag-of-Word features is
usually big because it depends on the size of vocabularies in
the whole corpus (dataset). For example, a corpus contains
5000 sentences, and the average number of vocabularies in
one sentence is 5, then the size of document vectors or
sentence vectors will be 25000 when unigrams are treated as
features. Large sized vectors can slow the system down and
they are inefficient. Common ways to get rid of less
effective features are applying feature selection methods.
One way is to use stop words. Stop words are usually
domain specific, so it is important to find out the domain
dependent stop word list.
Feature Extraction
Preprocessing
Movie Review
Data
Classification
Classification
positive or
Data negative class
Preprocessing Knowledge
Discovery
Evaluation
Fig. 2.1 Model For classification
C. Machine learning classification method
Text classifications using machine learning methods usually
focus on finding right features, appropriate feature
 International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
weighting values, feature selection methods, and right
machine-learning algorithms. The supervised machine
learning algorithms are usually corpus-based classification
methods, which are to find out co-occurrence patterns (e.g.,
frequency) of words in the corpus to determine the
sentiments of words or phrases. Bayes Theorem is the basic
theorem of many classification algorithms in text
classifications. The theorem provides a way to calculate the
probability of hypothesis base d on its prior probability
(Mitchell 2003, p156).
Bayes theorem is expressed as , which
is the cornerstone of Bayesian learning methods, because it
provides a way to calculate the posterior
probability , from prior
probabilities , in which, h is the
hypothesis. More intuitively, h is the target classification in
space H, and D is the training dataset. We are often
interested in determining the best hypothesis h from the
space H. In our problem, the target classification (the best
probable hypothesis) is positive or negative. So, the Bayes
theorem provides a direct method for calculating such
probabilities.
D. Classification performance evaluation
The evaluation metrics provide greater insight into
the performance characteristics of a binary classifier.
Accuracy (Ai) is commonly used as a measure for
categorization techniques. Accuracy values, however, are
much less reluctant to variations in the number of correct
decisions than precision and recall:
Precision (πi) is determined as the conditional probability
that a random document d is classified under ci, or what
would be deemed the correct category. It represents the
classifiers ability to place a document as being under the
correct category as opposed to all documents place in that
category, both correct and incorrect:
Precision measures the exactness of a classifier. A higher
precision means less false positives, while a lower precision
means more false positives. This is often at odds with recall,
as an easy way to improve precision is to decrease recall.
Recall (ρi) is defined as the probability that, if a random
document dx should be classified under category (ci), this
decision is taken.
Recall measures the completeness, or sensitivity, of a
classifier. Higher recall means less false negatives, while
lower recall means more false negatives. Improving recall
can often decrease precision because it gets increasingly
harder to be precise as the sample space increases.
978-1-4673-8810-8/16/$31.00 ©2016 IEEE
F-measure Metric: Precision and recall can be combined to
produce a single metric known as F-measure, which is the
weighted mean of precision and recall.
In the final phase of this work the results are evaluated to
find the issues, improvements and ways to extend the work.
A summary of the obtained results and future scope is also
discussed. The results obtained are compared to the previous
works to obtain a comparative summary of the existing work
and the proposed work.
III. PROPOSED METHODOLOGY
This section presents the proposed technique to improve the
feature extraction and evaluation is done which is required
for sentiment analysis. The proposed approach uses a
combination of NLP techniques and supervised learning [7].
In the first stage a pre-processing model is proposed to
optimize the dataset. In the second stage experiments are
performed using NLP and the machine learning methods to
obtain the performance of various feature selection schemes.
The process was carried out in the following steps, 1)
Identified the corpus on which the sentiment features has to
be extracted 2) Feature Extraction BoW 3) Classification- A
binary classifier is trained on the training set and then
applied to the test set 4) Better Feature Extraction – Stop
word filtering – Bigram Collocations 5) Eliminate Low
Information Features/High Information Feature Selection 6)
Comparison of Evaluations.
A. Classification
Consider movie_reviews.csv file, Each row in the dataset
contains the text of the review, based on the tone of the review
we need to classify as positive (1), or negative(1). Next few
instances should be trained with an algorithm using the reviews
and classifications train.csv, and then make predictions on the
reviews in test.csv. It will be able to calculate error using the
actual classifications in test.csv, and see how good our
predictions were [8].
Naive Bayes classification algorithm, A naive bayes classifier
works by figuring out the probability of different attributes of the
data being associated with a certain class. This is based on bayes’
theorem
This basically states “the probability of A given that B is true
equals the probability of B given that A is true times the
probability of A being true, divided by the probability of B being
true.”
B. BigramCollocation
Finding collocations requires first calculating the
frequencies of words and their appearance in the context of
other words. Often the collection of words will then requires
filtering to only retain useful content terms. Each ngram of
words may then be scored according to association measure,
in order to determine the relative likelihood of each ngram
being a collocation. The BigramCollocationFinder`` class
provide these functionalities, dependent on being provided a
function which scores a ngram given appropriate frequency
 International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
counts. A number of standard association measures are
provided in bigram_measures and trigram_measures.
C. HIGH INFORMATION FEATURE SELECTION
To find the highest information features, we need to
calculate information gain for each word. Information
gain for classification is a measure of how common a
feature is in a particular class compared to how common it is
in all other classes. A word that occurs primarily in positive
movie reviews and rarely in negative reviews is high
information. For example, the presence of the word
“magnificent” in a movie review is a strong indicator that
the review is positive. That makes “magnificent” a high
information word. The point is to use only the most
informative features and ignore the rest.
One of the best metrics for information gain is chi
square. NLTK includes this in the BigramAssocMeasures
class in the metrics package. To use it, first we need to
calculate a few frequencies for each word, its overall
frequency and its frequency within each class. This is done
with a FreqDist for overall frequency of words, and a
Conditional FreqDist where the conditions are the class
labels. Once we have those numbers, we can score words
with the BigramAssocMeasures .chi_sqfunction, then sort
the words by score and take the top 10000. We then put
these words into a set, and use a set membership test in our
feature selection function to select only those words that
appear in the set. Now each file is classified based on the
presence of these high information words.
Fig. 3.1 Proposed model for improving feature extraction
D. Evaluation
There are various methods to determine effectiveness;
however, recision, recall, and accuracy are most often used.
To determine these, one must first begin by understanding if
the classification of a sentiment was a true positive (TP),
false positive (FP), true negative (TN), or false negative
(FN)
978-1-4673-8810-8/16/$31.00 ©2016 IEEE
Table 3.1 Evaluation of a Sentiment Analysis
TP Determined as a sentiment being
classified correctly as relating to a
Feature.
FP Determined as a sentiment that is said
to be related to the Feature incorrectly.
FN Determined as a sentiment that is not
marked as related to a Feature but
should be.
TN Sentiments that should not be marked
as being in a particular Feature and are
not.
IV IMPLEMENTATION
A. Experimental Setup
The corpus [9] has 1000 positive files and 1000 negative
files. In this paper, 3/4 of the files are used as the training
set, and the rest as the test set. So it should give us 1500
training instances and 500 test instances. The classifier
training method provides a list of tokens in the form of
(feats, label), where feats is a feature dictionary and label is
the classification label. In our case, feats will be of the form
{word: True} and label will be one of ‘pos’ or ‘neg’.
B. Bag of Words Feature Extraction
In the experiment, bag of words model was used to
extract simple words as features, which is a simple
dictionaries mapping a feature name to a feature value. A
feature structure is a mapping from feature identifiers to
feature values, where each feature value is either a basic
value (such as a string or an integer), or a nested feature
structure. From the corpus for every word its feature name is
assigned a value True.
Example:
def word_feats(words):
return dict.fromkeys(words.split(), True)
print(word_feats("I love this sandwich."))
{'I': True, 'this': True, 'love': True, 'sandwich.': True}
C. Classification
Classification using machine learning can be in two steps:
1. Learning the model using the training dataset
2. Applying the trained model to the test dataset.
Sentiment analysis is a text classification problem and thus
any existing supervised classification method can be
applied. Naïve Bayes classifier is a simple probabilistic
classifier that is based on the Bayes theorem. This
classification technique assumes that the presence or
absence of any feature in the document is independent of the
presence or absence of any other feature. Naïve Bayes
classifier considers a document as a bag of words and
assumes that the probability of a word in the document is
independent of its position in the document and the presence
of other word. For a document d and class c, the experiment
was performed using Naive Bayes classifier for classifying
the movie reviews and the below results are obtained.
 International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
on multiple words. Another possibility is the abundance of
neutral words, such words degrade the sentiment. Because
the classifier treats all words same and assign each word as
either pos or neg. so it is better to remove neutral or
meaningless words from the featuresets to improve metrics,
and only classify sentiment rich words.
In order to improve feature extraction two
modifications should be considered in word_feats 1) filter
out stopwords and 2) include bigram collocations which
significantly improves the accuracy. The classifier should
run on new feature dictionary and use the new features to
train the classifier.
Fig. 4.1 Training Set vs Test Set and Accuracy using A. 1) Stopword Filtering: Stopwords are words that are
Naive Bayes classifier
generally considered useless. Mostly these words are
In the above Fig. 4.1 the 10 most informative ignored because they are so common that including them
features are shown, which are highly descriptive would greatly increase the size of the index without
adjectives. Accuracy is found to be 73% but improving precision or recall. NLTK comes with
apparently people agree on sentiment only around a stopwords corpus that includes a list of 128 english
80% of the time. stopwords. The results after using filtered out bag of
D. Measuring Precision and Recall for Positive and words.
Negative Reviews Accuracy: 0.726
Here it is required to build 2 sets for each classification label pos precision: 0.649867374005 pos recall: 0.98
a reference set of correct values, and a test set of observed neg precision: 0.959349593496 neg recall: 0.472
values. Then we train a Naive Bayes Classifier. It will
collect reference values and observed values for each label Fig.4.3 stopword filtered bag of words
(pos or neg), then use those sets to calculate the precision, The results show that accuracy decreased by .2%, and pos
precision and neg recall dropped!
recall, and F-measure of the naive bayes classifier.
2) Bigram Collocations: Including bigrams will improve
E. precision: 0.651595744680851
pos pos recall: 0.98 accuracy. The hypothesis is that people say things like
pos F-measure: 0.7827476038338657 “not great”, which is a negative expression that the bag
F.
neg precision: 0.9596774193548387 neg recall: 0.476 of words model could interpret as positive since it seems
neg F-measure: 0.6363636363636364
“great” as a separate word. The BigramCollocation
Fig. 4.2 Results- precision, recall, and F-measure using naive Finder maintains 2 internal FreqDists, one for individual
bayes classifier. word frequencies, another for bigram frequencies. Once
From the above results: it has these frequency distributions, it can score
1. Nearly every file that is pos is correctly identified, with individual bigrams using a scoring function such as chi-
98% recall. This means very few false negatives in the pos square. These scoring function measure the collocation
class. correlation of 2 words, basically whether the bigram
2. But, a file given a pos classification is only 65% likely to occurs about as frequently as each individual word.
be correct. Not so good precision leads to 35% false Most Informative Features
positives for the pos label. magnificent = True pos : neg = 15.0 : 1.0
3. Any file that is identified as neg is 96% likely to be outstanding = True pos : neg = 13.6 : 1.0
correct (high precision). This means very few false insulting = True neg : pos = 13.0 : 1.0
positives for the neg class. vulnerable = True pos : neg = 12.3 : 1.0
('matt', 'damon') = True pos : neg = 12.3 : 1.0
4. But many files that are neg are incorrectly classified.
('give', 'us') = True neg : pos =12.3 : 1.0
Low recall causes 52% false negatives for the neg label. ludicrous = True neg : pos =11.8 : 1.0
The above results are not satisfactory, because in general uninvolving = True neg : pos = 11.7 : 1.0
people use positives words in negative reviews, such as the avoids = True pos : neg = 11.7 : 1.0
word “not” is preceded by positive word “great” like “not ('absolutely', 'no') = True neg : pos = 10.6 : 1.0
great” which is a negative word. Accuracy: 0.816
pos precision: 0.753205128205 pos recall: 0.94
E. Better Feature Selection neg precision: 0.920212765957 neg recall: 0.692
The classifier uses the bag of words, which considers every
word as independent; it cannot learn that “not great” is a Fig. 4.4 Evaluation after including Bigram collocation
negative. In such cases it is better to train the classifier The above results show that the bigram collocation has
improved the accuracy to almost 9% pos precision has

978-1-4673-8810-8/16/$31.00 ©2016 IEEE

 International Conference on Advances in Human Machine Interaction (HMI - 2016),
March 03-05, 2016, R. L. Jalappa Institute of Technology, Doddaballapur, Bangalore, India
increased over 10% with only 4% drop in recall neg recall
has increased over 21% with just under 4% drop in
precision. So it appears that the bigram hypothesis is correct
and it is better to include significant bigrams which can
enhance the classifier effectiveness.
F. Eliminate Low Information Features/High Information
Feature Selection
In the previous classification, experiment was carried by
considering the 200 best bigrams which shows a high
improvement on the accuracy and pos precision. When the
classification model has hundreds or thousands of features,
many (if not most) of the features are low information.
Individually they are harmless, but in aggregate, low
information features can decrease performance.
For example, the presence of the word “magnificent” in
a movie review is a strong indicator that the review is
positive. That makes “magnificent” a high information
word. So eliminating low information features gives a better
model by removing noisy data. It can save from overfitting
and also from the curse of dimensionality. So only the
higher information features are considered it in turn increase
the performance which also decreases the size of the model,
which results in less memory usage along with faster
training and classification. Using the same evaluate
classifier method as on classifying with bigrams but using
Information Features the results obtained are evaluating best
word features:
Most Informative Features
magnificent = True pos : neg = 15.0 : 1.0
outstanding = True pos : neg = 13.6 : 1.0
insulting = True neg : pos = 13.0 : 1.0
vulnerable = True pos : neg = 12.3 : 1.0
ludicrous = True neg : pos = 11.8 : 1.0
avoids = True pos : neg = 11.7 : 1.0
uninvolving = True neg : pos = 11.7 : 1.0
astounding = True pos : neg = 10.3 : 1.0
fascination = True pos : neg = 10.3 : 1.0
idiotic = True neg : pos = 9.8 : 1.0
Accuracy: 0.93
pos precision: 0.890909090909 pos recall: 0.98
neg precision: 0.977777777778 neg recall: 0.88
FIG. 4.5 HIGH INFORMATION FEATURE SELECTION
G. Comparison on Improving Feature Extraction
The accuracy is over 20% higher when using only the best 10000
words and pos precision has increased almost 24% while neg
recall improved over 40%. These are huge increases with no
reduction in pos recall and even a slight increase in neg
978-1-4673-8810-8/16/$31.00 ©2016 IEEE
precision. Primarily in positive movie reviews and rarely in
negative reviews is high information.
The accuracy is over 20% higher when using only the best
10000 words and pos precision has increased almost
24% while neg recall improved over 40%. These are huge
increases with no reduction in pos recall and even a slight
increase in neg precision. Primarily in positive movie
reviews and rarely in negative reviews is high information.
V. Conclusion and Future Scope
The proposed work presents an approach for sentiment
analysis by comparing the different combination with
various feature selection schemes. We successfully analyzed
the different schemes for feature selection and their effect on
sentiment analysis. The classification using High
Information Features results in more accuracy than Bigram
Collocation. The model proposed in this paper is just an
initial step towards the improvement in the techniques for
sentiment analysis. It is worth exploring the capabilities of
the model for the dynamic data and extending the research
using hybrid techniques considerable scope for
improvement in the corpus creation and the work can also
be extended to improve the results using various
classification algorithms and also consider reducing
dimensionality.
References
[1] Mr. Rahul Patel , Mr. Gaurav Sharma, “A survey on text mining
techniques”, International Journal Of Engineering And Computer
Science,Volume 3 Issue 5, 5621-5625, May-2014.
[2] Stylios George et al, “Public Opinion Mining for Governmental
Decisions” Electronic Journal of eGovernment Volume. 8 Issue 2
2010, (pp203-214)
[3] B. Pang et al, 2002, Thumbs up sentiment classification using
machine learning techniques, Proceedings of the ACL-02
conference on Empirical methods in natural language processing,
vol.10, 79-86.
[4] Riloff, E &Wiebe, J., 2003, Learning extraction patterns for
subjective expressions, EMNLP‟03.
[5] Loren Terveen et al, 1997, PHOAKS: A system for sharing
recommendations, Communications of the Association for
Computing Machinery (CACM), 40(3):59–62;8
[6] Zhang H, Yu Z, XU M, Shi Y, Feature-level Sentiment Analysis
for Chinese Product Reviews, IEEE, 2011.
[7] Indurkhya N., Damereau F.J., (Eds). 2010. Handbook of Natural
Language Processing. 2nd Ed., Chapman & Hall/CRC, Boca
Raton.
[8] H.M.Wallach,“Topic modeling: beyond bag-of-words,” in
Proceedings of the 23rd International Conference on Machine
Learning (ICML '06), pp. 977–984, Pittsburgh, Pa, USA, June 06.
[9] L. Zhuang, F. Jing and X.Y. Zhu, "Movie review mining and
summarization," in Proceedings of the 15th ACM international
conference on Information and knowledge management (CIKM
'06), ACM, New York, NY, USA, 2006.
[10] A. McCallum, and K. Nigam, “A comparison of event models for
naïve Bayes text classification”, Journal of Machine Learning
Research, Vol. 3, 2003, pp. 1265–1287.