Received August 1, 2020, accepted September 23, 2020, date of publication October 1, 2020, date of current version October

13, 2020.
Digital Object Identifier 10.1109/ACCESS.2020.3028260

Experimental Results on Customer Reviews

Using Lexicon-Based Word Polarity
Identification Method
KHALED ABDALGADER AND AYSHA AL SHIBLI
Faculty of Computing and Information Technology, Sohar University, Sohar 311, Oman
Corresponding author: Khaled Abdalgader (komar@su.edu.om)
This work was supported by The Research Council (TRC), Oman through the Smart City Platform under Grant TRC/SRG/SC/SU/01/2019.

ABSTRACT Determining the polarities of words in a given context has been in existence since the inception
of computational linguistics, text mining, and sentiment analysis. Due to its fundamental role in determining
the overall semantic orientation of natural language expressions, it is considered one of the most challenging
issues facing these areas of research. This paper introduces a new implementation of the lexicon-based word
polarity identification method on several customer reviews datasets. Herein, we use a variation of a lexicon-
based word polarity identification method that operates by computing the semantic relatedness between
the context expansion set of the target word and a synonym expansion set comprising the synonyms of
all words surrounding the target word within the original text fragment. The polarity of the target word
is determined as that for which the semantic relatedness between these two meaningful sets is the highest.
Unlike most existing lexicon-based multi-polarity word identification methods, the used method is not based
on estimating pairwise relatedness at term-level, but instead, it is based on measuring semantic relatedness at
the fragment-level. This enables the exploration and capture of a higher degree of semantic and sentimental
information, and is more consistent with people’ understanding through the consideration of the larger
context in which the word appears. Its performance can be further improved by incorporating an initial
step in which the relative negation scope of words in the given text fragment is managed while determining
their sentiment orientation. The implementation results demonstrate that the used variation of the lexicon-
based word polarity identification method performs favourably against compared methods, as evaluated on
numerous benchmark datasets through stand-alone and end-to-end evaluation models.

INDEX TERMS Semantic similarity, sentiment scores, semantic orientation, multi-polarity words, negation
scope.

I. INTRODUCTION and then retrieves information to provide a real answer to

Identification of multi-polarity words is the process of deter-
mining the correct polarity or semantic orientation for words
as they occur in given text fragments. This process is an
important and intermediate function in many text-processing
activities. These include opinion mining and sentiment anal-
ysis, wherein enterprises and customers may become better
informed with respect to a particular product or service, and
hence enable better buying decisions or finding new business
trends or opportunities [1]–[6]; information retrieval, which
the application determines semantic orientation of a query
The associate editor coordinating the review of this manuscript and
approving it for publication was Muhammad Afzal . can be broadly categorized as belonging to one of the two
the posted query [7]–[10]; and smart city services, where
people can express their views and opinions on a particular
service through social media networks, thereby make better
identification of citizen’s interests and concerns [11]–[15].
Therefore, the task of automating identification and analysis
of indiscernible knowledge or opinions on human generated
texts would eliminate the vast effort and time it would require
to perform such a task manually.
Various polarity identification methods have been pro-
posed in literature to determine and classify the sentiment
orientation in a document-level text [16]–[18], sentence-level
text [19], and word, or feature level [20]. These methods

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 8, 2020 179955
 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
approaches: machine learning-based methods [21]–[25] and
lexicon-based methods [26]–[32]. Machine learning-based
methods use a supervised learning mechanism to motivate a
classifier from a textual collection of training data contain-
ing of a set of manual sentiment labels. This indicates the
polarity information (i.e., positive, negative, and neutral) and
an assigned numeric value to score how positive or negative
a given word is [15], [16]. Once a classifier has been estab-
lished by predicting the syntactic and semantic features, it can
then be used to extract the polarity of the target word in a
given textual context. The lexicon-based methods, however,
are usually unsupervised and utilize thesauri, dictionaries,
or lexicons to determine the polarity score of each word, and
they do not require any such training corpus [33]. These meth-
ods determine the overall sentiment orientation of a given
context as one where the total positive scores of each word
is matched with the total of their negative scores. The whole
textual context is classified to be negative if the summation of
the negative scores is higher, and vice versa [26], [27], [34].
In this study, we exclusively focus on lexicon-based methods.
Many approaches to lexicon-based polarity identification
rely on predetermined lexicon words that are often expressed
subjectively in the specific domain [7], [15], [16]. Naturally,
some words definitely change their polarity across different
domains or contexts, where most of these approaches do not
appropriately identify the actual polarity in which words are
being used within the target context [26], [27], [35], [36].
Overlooking the effects of considering the actual polarity of
each word according to its context could lead to poor accuracy
in overall performance of the polarity identification process.
For example, the word ‘‘small’’ has a negative polarity in the
sentence ‘‘The train’s seats are very small,’’ and it could be
understood as positive when appearing in a different context,
as in ‘‘The trolley is small and can easily fit in my luggage.’’
Nevertheless, this occurs even in closely related domains.
For example, ‘‘unpredictable’’ is a negative aspect for a car’s
steering but a positive one for a car’s shape. This situation
is different from polysemy (i.e., a word having multiple
meanings) as a word can carry the same meaning across
domains while having different polarities. A limitation of this
approach is in determining the words’ polarities individually
without considering the polarity assigned to neighbouring
words. In practice, the performance usually suffered from
the lack of utilizing the semantic orientation presented in the
target context.
Another challenging phenomenon in lexicon-based word
polarity identification methods is dealing with the words
that are affected either by morphological (i.e., prefixes and
suffixes) or syntactic (i.e., no, not, rather, etc.) negations
[37]–[43]. However, it is extremely important to predict
words that are affected by negation-terms before starting to
identify their polarities. The scope of negation can affect a
single word that appears directly after a negation-term or may
even encompass to affect the whole context. For example,
in the sentence ‘‘This book is not readable, but it looks nice’’
the negation scope here only affects the word that follows
179956
the negation-term. In contrast, the scope of negation in the
sentence ‘‘We are not waiting for you since a long time’’ is
extended to the end of the sentence; thus, the negation affects
the whole sentence. To overcome such negation challenges,
some methods [39], [40], [43] invert the polarity of negated
words to handle the negations. Furthermore, these methods
are operated as a stand-alone, and they do not incorporate the
negation handling process while determining the polarity of
words. Therefore, it is evident that predicting the negation
scope occurring within the context will improve the overall
performance of word in the polarity identification task.
Considering these issues, the contributions of this study
are as follows. First, we introduce a new implementation
of the word polarity identification task on several customer
reviews datasets. This implementation uses a new variation of
lexicon-based method that determines the polarity of a target
word by calculating the semantic similarity between its Word-
Net’s [44] synonyms, gloss, and usage-example (referred as
context expansion), and the synonyms context provided by
all synonym-words of all words surrounding the target word
within the given text fragment (referred as synonyms expan-
sion). This allows it to explore and capture closely related
sentimental information to determine the correct polarity of
the word. The method also identifies the polarity of the
words simultaneously by progressively incorporating these
polarity-assigned words in the synonyms expansion context
that contains all words surrounding the target word. This
again enables our variation method to exploit more related
semantic and sentiment information. Second, we present a
simplified process that improves the performance of polar-
ity identification by handling the relative negation scope
of words while identifying their polarities. Finally, this is
believed to be the maiden research evaluating the lexicon-
based multi-polarity identification using two different eval-
uation models. Empirical results demonstrate that the used
method achieves better performance than that achieved by the
compared lexicon-based methods as evaluated on different
standard customer review datasets.
The remainder of the paper is organized in the following
manner. Section II describes relevant work in the area of
lexicon-based sentiment analysis. Section III describes the
used word polarity identification method. Section IV presents
a walk-through example illustrating how the used method
works. Experimental results are presented in Section V, and
Section VI concludes the paper.
II. RELATED WORK
Various approaches for lexicon-based word polarity iden-
tification have been developed in recent years to analyze
emotions, attitudes, and opinions about particular objects or
events [30]–[32]. These methods can be generally classified
into two groups: corpus-based [47]–[50] and dictionary-
based [34], [45], [46], [51], [52]. In corpus-based methods,
the polarity of each word is determined based on term-
frequency (i.e., word co-occurrence) in the classified and
more dynamic seed words sets [26], [27], whereas in the
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
dictionary-based methods, the polarity of each word is com-
puted by utilizing fixed lexical and lexicon resources such
as WordNet [44] and SentiWordNet [52], respectively. Both
approaches basically operate by extracting the opinion and
emotion targets, known as sentiment attributes, from the
unstructured text. Following this, sentimental information is
labeled or rank scored to demonstrate how positive, negative,
or neutral a word is. These targets are identified based on
lexical/lexicon or corpus resources that have been applied.
Finally, the highest value resulting from the summation or
average is considered to determine the overall semantic ori-
entation for the given text fragment. However, apart from
sentiment scores, the aspect of exploiting the sentential con-
text such as senses, negation, or intensification surrounding a
word are not considered in the majority of these approaches.
Most of the lexicon-based methods are inspired by the
assumption that word polarity usually organizes in the form
of an ordinal scale, where this scale may be numerical values
(e.g., 0 can be ‘‘very negative’’ to 1 being ‘‘very positive’’),
or annotated labels (e.g., positive, negative, neutral) [26],
[27], [34], [46], [51]. The first word polarity identification
method that capitalized on this assumption is attributed to
Hatzivassiloglou and McKeown [26]. This method deter-
mines the polarity of adjective terms by exploiting the pairs of
their conjoined linguistic features (and, or, but, etc.) extracted
from a free-annotated textual collection. The method adopts
the linguistic distributional hypothesis that the conjunction
but usually links adjectives that have opposite orientation and
the conjunction and/or links adjectives that have equal orien-
tation to expand the seed document sets. A graph has been
created to represent the extracted terms (adjectives) as nodes
connected by edges that indicate their relationships (e.g.,
equal-orientation or opposite-orientation). Finally, a non-
hierarchical clustering was applied to classify a group of
nodes (adjectives) into a Positive class and a Negative class,
which is based on the similar relationship induced by the
edges.
While Hatzivassiloglou and McKeown’s method and many
of its variants [27]–[31] are based on the notion of word
polarities being identified individually at term-level (i.e.,
without considering the other parts of speech and the ori-
entation assigned to the neighbouring words in the context),
Kanayama and Nasukawa [53], followed by Ding et al. [30],
Qiu et al. [54] and Zhang and Liu [55] opine that involving
only a single part of speech is just one of many possible
polarity identification methods and propose methods based
on the idea of coherency (i.e., at clause-level). They deal
with the three main concepts of part of speech type, fea-
ture context, and dependency relations. Their approaches are
motivated by the belief that words in a context must be related
in meaning to the text-fragment to be coherent. This means
that words that have the same semantic orientation usually
occur together in the context. The methods are evaluated
by checking their performance at classifying words using
precompiled sets of positive and negative words. However,
these methods are limited by the fact that they can only
VOLUME 8, 2020
identify the polarity of words that appear within some limited
distance from each other in the given context (i.e., a fixed-
sized window surrounding the target word).
The family of dictionary-based methods [34], [45], [46],
[51], [52], [56], which have become immensely popular in
recent years, are based on utilizing lexicon resources such
as SentiWordNet [52], WordNet-Affect [57] which are built
using WordNet [44] lexical information. They have been
developed to annotate their synsets with sentiment scores
(positively, negatively, and objectively) ranged from 0 to
1 for each sentimental class. However, these lexicon resources
face the challenge of their synsets being ordinarily orga-
nized based on the common senses of the words that are
extracted from WordNet. Therefore, incorrect identification
of the actual meaning (sense) from the possible senses of the
target word would impact the performance of word polarity
identification. For example, if the words room, hotel, and
unexpected appear near the word small, we can simply rec-
ognize that the intended sentiment of small is of negative
polarity, and if it appears near words fit, easily, and bucket,
it has a positive polarity. Thereby, the actual polarity of a
word needs to be identified in the textual context in which it
occurs, and this presents yet more linguistic challenges, such
as context limitation, negation, and intensity handling.
The majority of lexicon-based approaches for word
polarity identification do not handle the issue of lan-
guage ambiguity, wherein word senses can carry different
polarities depending on the context of target words being
used. However, little work has been done to consider the
actual polarity of the word senses in the sentiment analysis
task [58]–[65]. The method proposed by Esuli and Sebastiani
[45] determines the semantic orientation of word senses (i.e.,
polarity of subjective words) based on quantitative analysis
of the glosses associated to WordNet’s synsets. They man-
ually create a lexicon resource [52] in which each WordNet
synsets (i.e., word-sense) is assigned three scores demonstrat-
ing how positive, negative, or neutral the senses contained
in the synset are. The labelling was based on applying a
PageRank method [51] for ranking the synsets according to
their semantic property. Wiebe and Mihalcea [63] proposed
another method that relied on exploring large labelled opinion
datasets for annotates of WordNet’s synsets as objective
or subjective in order to identify the semantic orientation.
Interestingly, Rentoumi et al. [66] have tackled the polarity
identification issue by disambiguating all target words first,
and then associating the senses-assigned words to models of
sentimental information using graphical representations. This
was done to produce better related contextual and sub-word
information [67].
With few exceptions, the typical lexicon-based approaches
for identifying word polarity use some functions of the lexical
relations matching through utilizing the lexicon (dictionary)
resources and produce a relative numerical score based only
on the original constituent word individually [26], [27], [45],
[51], [60], [62], [63], [65]. However, these relationships
may not be absolutely real as not all word-senses may be
179957
 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
syntactically and semantically connected, and this lack of
connectivity may result in identifying polarity without con-
sidering different aspects such as the surrounding context
related to the target word and linguistic features (i.e., variabil-
ity in the negation scope), thereby possibly leading to a lack of
coherence in meanings. The research described in this paper,
however, is inspired by the belief that the performance of such
identification could be improved through analysing the orig-
inal context and then expanding it to simultaneously explore
and identify the accurate polarity of the target word. In the fol-
lowing section, a lexicon-based word polarity identification
method is modified that encompasses a wide range of lexical
relations, and outperforms other compared approaches.
III. POLARITY IDENTIFICATION METHOD
In this section, a new variation of lexicon-based word polarity
identification method is presented in detail. Unlike exist-
ing methods, which rely on either a word’s pairwise mea-
suring [27], [35], [58], [58]–[60], [62], [63] or contextual
overlapping [45], [54], [53], [64], the modified method deter-
mines the polarity of a target word by calculating the semantic
relatedness between its context expansion and synonyms
expansion sets, which provided by all related WordNet’s asso-
ciated semantic information (i.e., synset). This calculation
of the semantic information was performed at text-fragment
level. The actual polarity of the target word is assigned
as the WordNet’s synset (with its sentimental information
retrieved from SentiWordNet) for which the semantic similar-
ity between context expansion set and synonyms expansion
set is greatest. Significantly, it also identified the polarity of
the target words simultaneously by progressively incorporat-
ing these polarity-assigned words in the synonyms expansion
context, thereby they could be used to identify the polarity
of other unidentified words, thus contributing an assignation
to other words that have not had their polarity determined.
Performance was further improved by handling the relative
negation scope of words while identifying their polarities and
other related issues.
A. MODIFIED LEXICON-BASED METHOD
The target text comprising the words to be identified by
semantic orientation (polarity) is first represented as the set
that it consists: Target Text = {word i |i = 1...N } , where N is
the total number of words in original text fragment (i.e., given
text). Now, assume that wordi is the word whose polarity we
need to identify. Below Context Expansion represents the set
of WordNet’s associated information (i.e., synonyms, gloss,
and usage example) corresponding to the available synsets of
wordi :
n o
Context_Expansion = akword i | k = 1 . . . Nword i
where Nword i is the total number of the possible synsets
for wordi in WordNet and akword i is the union set of non-
stopwords in the k th WordNet associated information of
179958
wordi :
akword i = synonymskword i ∪ glosskword i ∪ examplekword i
 
= i = 1 . . . Nak ≤ N1 + N2 + N3
word i
where N1 , N2 and N3 are the number of non-stopwords
in WordNet’s synonyms, gloss, and usage example, respec-
tively, and they can be expressed as:
synonymskword i = word 1i | i = . . . N1

glosskword i = word 2i | i = . . . N2

examplekword i = word 3i | i = . . . N3

Synonyms Expansion is the set containing all available
synonym-words of the words in Target Text set except the
target word wordi :
Synonyms_Expansion
n
= s_wordssword j | s_wordssword j ∈ word j _all_synonyms,
and j 6= i}
where s is the synset of the word in the WordNet in which the
first one has been considered for a word whose polarity has
not been identified during the whole identification process.
The polarity of target word wordi is determined as the k
score for which context expansion set is semantically most
related to the synonyms expansion set (see Subsection 3.4).
The pseudocode that demonstrates how the proposed method
operates is described in Algorithm 1.
It is important to note that all words in both expanded sets
(line 7 of the algorithm) are treated with the first WordNet’s
synset, where they usually are organized from most frequent
to least frequent. This means that considering the first synset
is likely to expand the context in order to capture enough
semantic information between words being compared (but
not always granted). While most WordNet semantic relations
do not extend beyond parts of speech, context expansion
improves this lack of connectivity by enabling a greater rela-
tional connectivity between words compared with the current
approaches, which only consider the constituent words. For
example, the semantic connection between the noun con-
tainer and the adjective fleshy does not exist, where they
are usually considered to be topically related. Therefore,
we may discover some words that can improve the connec-
tivity between these semantically related words by looking at
their synonyms.
The semantic orientation between context expansion and
synonyms expansion sets is calculated after preforming a pre-
processing step to the WordNet’s gloss and usage example
in the context expansion set. This pre-processing function
adopts a sequence of words as input and retunes their morpho-
logical base form. Since the gloss and usage example contain
a description of the word meaning, (it is better to utilize as
much of their meaning as possible). This requires the removal
of stopwords, prefixes, suffixes, and other linguistic features
in order to keep words in their original form. However,
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method

Algorithm 1 Word Polarity Identification Algorithm 2 Antonym Word Replacing

Input: Target Text = {word i |i = 1...N } & Input: Word
Context_Expansion = {akword i | k = 1 . . . Nword i } Output: Inferred Antonym ant_word
Output: WordNet synset Polarity = {Pi |i = . . . N }, where Antonym Word Replacing
Pi is the synset with the associated SentiWordNet 1: //DETECT WORD FOLLOWING NEGATION
word-polarity scores of wordi . 2: if antonym(Word) available in WordNet then
Word Polarity Identification 3: ant_word ← first possible antonym-word
1: // NEGATION HANDLING 4: else:
2: for i=1 to N do 5: for synonym in all Word’s-synsets in WordNet do
3: Synonyms_Expansioni = {s_wordssword j | 6: ant_word ← most similar antonym-word to
s_wordssword j ∈ word j _all_synonyms, and j 6 = i} // Word
with progressively incorporating the 7: end for
polarity-identified words in this set. 8: end if
4: highest_semantic ← 0
5: Pi ← 1 // first synset as an initial step.
6: for j=1 to Nword i do
7: sim_score ← semantic_orientation
(context_preprocessing (Context_Expansionj ),
Synonyms_Expansioni )
8: if sim_score > highest_semantic then
9: sim_score ← highest_semantic
10: Pi = j // with the highest SentiWordNet score
word-polarity for the current synset.
11: end if
FIGURE 1. Antonym word replacement. (a) showing the selection of the
12: end for first possible antonym of the word bad from WordNet. (b) illustrating
13:end for how to explore the symmetric relation to obtain the antonym of the word
small from WordNet.

Note that in line 6 of Algorithm 2, in which semantic

morphological stemming was not performed to the synonym-
words in both expansion contexts. This was because there was
a desire to take advantage of their part of speech information
that effectively enhanced the relational connectivity between
words. Further detailed information on how this algorithm
operates is presented in Section IV.
B. NEGATION SCOPE HANDLING
Negations affect the meaning of other words; therefore, it is
significant to detect their scope before beginning to identify
the polarity of words. This can typically be done through
determining the word following negation, and then removing
this negation term and replacing it with its possible antonym-
word. In this preliminary step, two main classes of negation
words were defined: morphological negations (e.g., dimin-
ishers, prefixes and suffixes) and syntactic negations (e.g.,
no, not, never, nobody, etc.). These negations are used as a
primary guide to identify the scope of a negation. In other
words, it is done to detect the word that needs to be replaced
by an antonym-word.
Let us suppose that we have the sentence: The room is not
small, in which the word small is affected by the negation
term not. In this case, we handle this issue by removing the
term not and replacing the next word small by its closest
possible and most related antonym-word using WordNet’s
context. The full process is described in Algorithm 2.
similarity between a negation detected word and its possi-
ble antonym word (after exploring its symmetric relation,
i.e., synonyms and antonyms) is estimated using a WordNet-
based word-to-word similarity measure (shortest path) as was
done by Rada et al. [68]. We also consider the first WordNet’s
synset for both cases (line 3 and 6) described in the antonym
word replacing algorithm. For clarity, Fig. 1 illustrates how
antonym replacement can be performed in both cases above.
C. PRE-PROCESSING AND TEXT REPRESENTATION
After handling the negations in the given text, another pre-
processing step is performed in which all the stopwords
(e.g., the, a, as, and, etc.) and noise (e.g., ‘‘, /, ∗ ,. etc.) are
removed from the target text. This is due to them contributing
little to the overall meaning and because they do not convey
any semantic information that affect the understanding of
the context. Thus, the final form of the given text-fragment
after handling negation is Target_Text = {word1 , word2 , . . . ,
wordn }.
Regarding text representation, however, the proposed
method uses a matrix structure to represent the processed
textual context. The matrix used consists of two-dimensions
(mxn), where m (horizontal) and n (vertical) are called row
and column, respectively. That is, the synonyms expansion
and context expansion sets are represented as a matrix of
X = (xi,j ) when their semantic orientation is calculated.
Each element (xi,j ) of the matrix is a numerical score that

VOLUME 8, 2020 179959

 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
represents the semantic similarity between wordi and wordj ,
and can be found in the ith row (synonyms expansion set) and
jth column (context expansion set). Consequently, the size of
a matrix is dynamic and defined by the number of words in
the sets. More detailed information on matrix representation
is demonstrated in the next section.
D. EXPANDED CONTEXTS SEMANTIC CALCULATION
To calculate the semantic relatedness between the synonyms
expansion and context expansion sets as shown in line 7 of
algorithm 1, they are represented in a matrix structure, where
the rows list words in the synonyms expansion set and the
columns list words in the context expansion set. All words
in both sets are organized in the order as they appear in
their original context. Let Synonyms_Expansion and Con-
text_Expansion be the word sets of the two contexts whose
semantic similarity are to be computed. Suppose that Syn-
onyms Expansion is the set that contains all non-stopwords
(synonym-words) of the given text-fragment except the target
word whose polarity we aim to identify, and Context Expan-
sion is the union set that encompasses all non-stopwords
and morphed (stemmed) words from the associated WordNet
synonyms, gloss, and usage examples corresponding to the
synset k for the target word:
Synonyms_Expansion = {word 1i |i = 1. . .N1 , i 6 = j};
Context_Expansion = akword 1j = {word 2i |j = 1. . .N2 }
For each word in the Synonyms Expansion set, the semantic
similarity value is measured at the cross point with all words
in the Context_Expansion set. The word-to-word shortest
path [68] similarity measure was applied to estimate the
semantic similarity between words across the compared sets.
Following this, the semantic orientation list is constructed,
where each element of this list corresponds to a word in the
Synonyms_Expansion set, thereby its size is dependent on
the total number of the remaining words in this set (N1 ).
The elements value of the list is obtained by picking the
highest cross point similarity score in each column belong-
ing to the word in the Synonyms_Expansion set. Once the
semantic orientation list is formed for each possible synset
k of the target word, the Context_Expansion set that has the
highest similarity score will be used to retrieve its sentimental
information from SentiWordNet to determine the polarity of
the target word. This is only for determining the polarity of
each single word in the given context. The following section
presents how the overall polarity (orientation) of the given
context can be determined.
E. TOTAL SENTIMENT SCORE DETERMINATION
After performing word polarity identification using the mod-
ified method, retrieving their sentiment information from
SentiWordNet, a list (i.e., polarity = {Pi | i = . . . N})
was obtained that comprised of two scores (positive and
negative) corresponding to each polarity-assigned word.
Finally, overall polarity of the given text was determined by
179960
separately aggregating the scores of all polarity-identified
words in the list. There were two cases to consider depending
on whether the overall sentiment orientation was positive,
negative, or neutral:
Case 1: hard classification, where the given text could
belong to a single class (positive, negative, or neu-
tral) for which aggregated sentiment orientation
score was highest.
Case 2: soft classification, where the given text could
belong to all classes (positive, negative, and neu-
tral) with different degrees of membership. In this
case, all the retrieved sentiment orientation scores
have been considered.
Therefore, the output’s class membership values polarityci ,
which demonstrated the highest score that shows the mem-
bership of a particular given text to class c; i.e., Max =
{polarityci }. In the event that a soft classification was required,
this was to be achieved by assigning a particular given text
to all classes with different degrees of membership. This
meant that overall sentiment orientation of the given text
could belong to all classes of positive, negative, and neutral
at the same time but with different aggregated scores. The
experimental testing in Sections IV and V demonstrates these
two cases.
F. COMPUTATIONAL COMPLEXITY
The used word polarity identification method utilizes the
semantic and sentimental information provided by all words
in the given text to identify their polarities. Importantly,
this utilization does not lead to a significant increase in the
calculations required for running it. To demonstrate, sup-
pose that T is the total number of words in Target_Text, t
is the number of words in each Synonyms_Expansion set,
a the number of words in each Context_Expansion set, and
k the possible number of WordNet’s synsets for each word
in the given context (Target_Text). Each word whose polarity
needs to be identified will require a calculation of semantic
orientation between each of its Context_Expansion set and the
Synonyms_Expansion set (synonym-words of its surround-
ing original words). The maximum number of calculations
needed to identify P the polarity of all words in the given text
is therefore O( TK =0 K = sum(k)). Each of these semantic
orientation calculations needs O(t. a)Psimilarity computations
between words, which leads to O( K sum=0 sum = (t.a)) as a
maximum overall word-to-word similarity computation. For
the purpose of clarification, let T be equal to 3 and K equal 5
(i.e., 5, 5, 4, 3, 3 words in each synonyms expansion sets and
6, 7, 5, 4, 7 words in context expansion set for each target
word respectively). The method proposed would therefore
require a maximum of 118 computations to estimate the
semantic similarity between words.
Interestingly, these computational requirements will
not usually be more expansive compared to the current
approaches that utilize a limited context provided within
a size-fixed window. In cases dealing with long textual
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method

contexts, however, it is of course possible to limit the given TABLE 1. Word polarity progressively assigned.
context by using only the context provided within the deter-
mined window of words.

IV. A WALK-THROUGH EXAMPLE

This section provides an example demonstrating how the
modified method operates in order to identify the polarities
of words in the given context as presented in the above
section. Consider the following four sentences, which contain
the multi-polarity words subtle, filled, and ambiguous words
virus and bank.
Sentence 1: The current medications are not sufficient for
a subtle virus.
Sentence 2: The computer hackers have obtained subtle
answers when they were attacked by a virus.
Sentence 3: She filled her pail to cross the south bank of
the river.
Sentence 4: The people won’t cry when seeing their bank
accounts filled with enough money.
Firstly, there is a demonstration using sentence 1. The
TargetText set is constructed to contains all words in the given
text (sentence 1):
Target Text
= {‘‘The,’’ ‘‘current,’’ ‘‘medications,’’ ‘‘are,’’
‘‘not,’’ ‘‘sufficient,’’ ‘‘for,’’ ‘‘a,’’ ‘‘subtle,’’ ‘‘virus’’}
The negation handling step is performed, thus obtaining the
following set without stopwords:
Target Text = {‘‘current,00 ‘‘medications,00
‘‘insufficient,00 ‘‘subtle,00 ‘‘virus00 }
Now, after dealing with words affected by negation terms,
the modified word polarity identification method starts by
processing the first word in TargetText set, which is current.
This word has four synsets in WordNet. For reasons of space,
here is the Context_Expansion sets only:
Context_Expansionsynset−1 = {‘‘electric_current,’’‘‘flow,’’
‘‘electricity,’’‘‘conductor,’’‘‘current,’’‘‘measure,’’
‘‘ampere’’}
Context_Expansionsynset−2 = {‘‘stream,’’‘‘steady,’’‘‘flow,’’
‘‘fluid,’’‘‘usually,’’‘‘natural,’’‘‘cause,’’‘‘raft,’’
‘‘float,00 ‘‘downstream,00
‘‘current,’’‘‘felt,’’‘‘air,’’‘‘hose,’’‘‘eject,’’‘‘water’’}
Context_Expansionsynset−3 = {‘‘stream,’’ ‘‘course,’’
‘‘flow,’’ ‘‘dominant,’’‘‘suggestive,’’ ‘‘running,’’ ‘‘water,’’ ,
‘‘successive,’’‘‘event,’’ ‘‘idea,’’ ‘‘two,’’‘‘development,’’
‘‘run,’’ ‘‘american,’’ ‘‘history,’’ ‘‘consciousness,’’ }
‘‘thought,’’‘‘current’’
Context_Expansionsynset−4 = {‘‘occur,’’ ‘‘belonging,’’
‘‘present,’’ ‘‘time,’’ ‘‘current,’’‘‘event,’’ ‘‘topic,’’
‘‘negotiation,’’ ‘‘psychoanalytic,’’ ‘‘theory,’’ ‘‘ship,’’}
‘‘position00
While processing the word current, the Synonyms_
Expansion set that resulted from remining words (‘‘medica-
tions,’’ ‘‘insufficient,’’ ‘‘subtle,’’ ‘‘virus’’) is:
Synonyms_Expansion ={‘‘medicine,’’ ‘‘medication,’’
‘‘medicament,’’ ‘‘medicinal_drug,’’ ‘‘insufficient,’’
‘‘deficient,’’‘‘elusive,’’ ‘‘subtle,’’ ‘‘virus’’}
Note that, as described in the previous section, in construct-
ing the both expanded sets’ pre-processing steps have been
applied to these sets as per line 7 of the algorithm 1.
We now calculate the semantic similarity between Syn-
onyms_Expansion set and each Context_Expansion set of the
word current as per line 7 of algorithm 1, thus resulting in four
similarity score lists corresponding to each possible synsets
of the word current.
Listsynset−1
= [0.066, 0.111, 0.111, 0.111, 0.0, 0.0, 0.0, 0.0, 0.125]
Listsynset−2
= [0.071, 0.142, 0.142, 0.142, 0.0, 0.0, 0.0, 0.0, 0.166]
Listsynset−3
= [0.111, 0.142, 0.142, 0.142, 0.0, 0.0, 0.0, 0.0, 0.166]
Listsynset−4
= [0.090, 0.111, 0.111, 0.111, 0.0, 0.0, 0.0, 0.0, 0.111]
As can be observed, the first highest similarity score
appears in the second list (Listsynset−2 ), which then can be
considered as the actual synset for the word current, and
therefore, its sentimental information from SentiWordNet can
be retrieved to identify the polarity of this word.
Repeating this same process simultaneously to the remain-
ing words in TargetText set results in the following table,
where the polarity-assigned words are progressively incorpo-
rated in the set:
The meaning (e.g., definitions or glosses) associated
with these polarity-identified words, together with their

VOLUME 8, 2020 179961

 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
sentimental information, are as follows:
current 0n : a steady flow of a fluid
(usually from natural causes)
Positive Score: 0.0; Negative Score: 0.0
medications0n : (medicine) something that treats
or prevents or alleviates the symptoms of disease
Positive Score: 0.25; Negative Score: 0.0
insufficient 0a : of a quantity not able to fulfill a need
or requirement
Positive Score: 0.125; Negative Score: 0.875
subtle0s : working or spreading in a hidden and
usually injurious way
Positive Score: 0.0; Negative Score: 0.625
virus0n : (virology) ultramicroscopic infectious agent that
replicates itself only within cells of living hosts;
many are pathogenic; a piece of nucleic acid
(DNA or RNA) wrapped in a thin coat of protein
Positive Score: 0.0; Negative Score: 0.0
It can be deciphered clearly that the proposed method
assigned the correct synsets to words subtle and virus which
are the same as a human would assign them as they appeared
in this context. Now, let us conduct the same procedure on
sentence 2, sentence 3, and sentence 4, respectively. Note that
only the final polarity-assigned set for each sentence is illus-
trated here, together with the associated meaning information
for words subtle, virus, filled, and bank only.
polaritysentence2
= {(computer 0n , 0.0, 0.0), (hackers1n , 0.0, 0.0),
(obtained 2v , 0.37, 0.0), (subtle0s , 0.5, 0.25),
(answers9v , 0.0, 0.0), (attacking0v , 0.0, 0.0),
(virus2n , 0.37, 0.0)}
polaritysentence3
= {(filled 1v , 0.25, 0.37), (pail 0n , 0.0, 0.0), (cross10
v , 0.0,
0.0), (sough0v , 0.0, 0.0), (bank 0n , 0.0, 0.0), (river 0n ,
0.0, 0.0)}
polaritysentence4
= {(people4v , 0.0, 0.0), (laugh1n , 0.0, 0.0), (seeing2v ,
0.0, 0.0), (bank 1n , 0.0, 0.0), (accounts2n , 0.0, 0.0), (filled 9s ,
0.25, 0.0), (enough0n , 0.0, 0.0), (money2n ,
0.0, 0.0)}
The numbers that correspond to each word in the list above
are its positive and negative scores, respectively. For further
clarification, the following are meanings that have been iden-
tified for multi-polarity and ambiguous words that appeared
in sentence 2, sentence 3, and sentence 4, respectively.
179962
Sentence 2:
subtle0s : difficult to detect or grasp by the mind or
analyze
virus2n : a software program capable of reproducing
itself and usually capable of causing great
harm to files or other programs on the same
computer
Sentence 3:
filled 1v : become full
bank 0n : sloping land (especially the slope beside a
body of water)
Sentence 4:
bank 1n : a financial institution that accepts deposits
and channels the money into lending
activities
filled 9s : (usually followed by ‘‘with’’ or used as a com-
bining form) generously supplied with
The words subtle and virus that appeared in sentence 2 have
been assigned different meanings from those which appeared
in sentence 1 and the same can be seen for words in sentence
3 and sentence 4. This means that the method can capture a
high degree of semantic and sentimental information in order
to identify the actual meaning (polarity) of words based on
the context they appear within. Some of these synset-assigned
words, however, are different from what a human would
allocate, especially when related to the part of speech. This
is quite reasonable due to the possible range of synsets in
WordNet.
For the overall polarity of the given text, a summation for
all retrieved sentimental scores separately was performed,
then the highest summed value was considered to be the
actual overall polarity and then the text was classified as
positive or negative (Note that the neutral scores were not
considered). Fig. 2 illustrates both classification cases (hard
and soft classification) for the overall sentiment orientation
of the four exemplar sentences.
V. EXPERIMENTAL RESULTS
In the experimental session, the effectiveness of the new
variation method was evaluated in two different tasks
(stand-alone and end-to-end) through several standard sen-
timent analysis datasets. All results presented below are
based on the use of the lexical and lexicon knowledge
resources which are commonly known as WordNet and
SentiWordNet.
A. STAND-ALONE EXPERIMENTS
In this section, the modified method was applied in two
different experimental models, both word- and sentence-
level polarity identification, using eleven benchmark datasets.
Firstly, the standard datasets and evaluation criteria that have
been used in both experiment models are described. Then,
the results obtained from these two different experiments
together with a comparative performance of other existing
approaches are discussed.
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
FIGURE 2. Overall sentiment classification. (a) hard classification. (b) soft
classification.
1) BENCHMARK DATASETS AND EVALUATION CRITERIA
As the modified method was experimented on two stand-
alone models, the used datasets based on when they
are applied are described here in detail. For word-
level experiments, three benchmark datasets were used:
SemEval-2015-Task10-(A) [69], SemEval-2016-Task4-(B)
[70] and SemEval-2017-Task4-(B) [71]. In sentence-level
experiments, however, eight benchmark datasets were
used: the SemEval-2015-Task10-(B) [69], the SemEval-
2016 Task4-(A) [70], the SemEval-2017 Task4-(A) [71],
the Stanford Twitter Sentiment (STS) [72], the 5-Prodects
Customer Review [73], the 3-Prodects Customer Review
[74], the Movie Review [75], and the Software Review [76].
Only the test data have been used as a result of the proposed
method being unsupervised and not needing training from
annotated data. Note that for SemEval datasets, they were
not included in any task, and they were downloaded after the
evaluation was completed.
SemEval-2015-Task10-(A), SemEval-2016-Task4-(B) and
SemEval-2017-Task4-(B) were designed for the task of deter-
mining whether a particular word or topic was positive or
negative (two-point scale) in the given context. The test data
that was used included Twitter ID, the target word/topic,
VOLUME 8, 2020
the polarity toward it and the related context. Since the
positive and negative messages only were considered, these
datasets contained 2,907 (1,899 positive and 1,008 neg-
ative), 10,551 (8,212 positive and 2,339 negative) and
6,185 (2,463 positive and 3,722 negative) Twitter messages,
respectively.
SemEval-2015-Task10-(B), SemEval-2016 Task4-(A) and
SemEval-2017 Task4-(A) were created for the task of
classifying whether the given context was of positive, neg-
ative, or neutral sentiment. The test data that was used
contained messages obtained from Twitter and a polarity tag
toward it. These datasets contained 1,405 (1,040 positive and
365 negative), 10,290 (7,059 positive and 3,231 negative) and
6,347 (2,375 positive and 3,972 negative) Twitter messages,
respectively.
STS (STS-Test) dataset were manually labelled and
included 182 positive and 177 negative text-fragments
retrieved from Twitter APIs. The 5-Prodects Customer
Review contained 3,945 labeled sentences collected from
Amazon.com, which belonged to four domains: cellphone,
MP3 Player, DVD Player, and two types of digital camera.
The 3-Prodects Customer Review was also retrieved from
Amazon.com and it contained 2,099 polarity annotated sen-
tences related to three domains: Computer, Wireless Router,
and Speaker. Finally, the movie and software datasets com-
prised a total of 3,000 positive reviews and 2,915 negative
reviews.
All these datasets were organized as a collection of text-
fragments, where each one was labeled with a tag, indicating
whether the fragment was positive or negative. While the pro-
cess of polarity identification could be considered as a classi-
fication task, a classification threshold needed to be defined
(i.e., the target text-fragment was classified as a positive or
negative if the summation of all retrieved word-sentimental
scores was highest). Therefore, four classification measures
were used to evaluate the results: accuracy, precision, recall,
and F-measure. Whereby accuracy was the number of correct
predictions divided by the total number of text-fragments.
Precision was the number of correct true predictions divided
by the total number of true predictions. Recall was the number
of correct true predictions divided by the actual number of
correctly identified fragments for both positive and negative
tagged collections. F-measure was the weighted average of
precision and recall scores. These were calculated as follows:
Accuracy = (True − Positive + True − Negative)
/(True − Positive + False − Positive
+False − Negative + True − Negative),
Precision = True − Positive/(True − Positive
+False − Positive),
Recall = True − Positive/(True − Positive
+False − Negative), and
F − measure = 2 × (Precision × Recall)
/(Precision + Recall)).
179963
 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
TABLE 2. Performance (%) on SemEval-2015-Task10-(A),
SemEval-2016-Task4-(B) and SemEval-2017-Task4(B) Datasets.
FIGURE 3. Performance (% F-measure) of other polarity identification
methods and baselines on SemEval-2015-Task10(A),
SemEval-2016-Task4-(B) and SemEval-2017-Task4-(B).
2) WORD-LEVEL POLARITY IDENTIFICATION RESULTS
Table 2 indicates the accuracy, precision, recall, and
F-measures resulting from applying the proposed method
to the SemEval-2015-Task10-(A), SemEval-2016-Task4-(B)
and SemEval-2017-Task4-(B) datasets. The first section of
the table shows performance with handling negation scope
as a preliminary step; the second shows performance without
considering the negations. When the negation handling is
applied, the best performance was achieved. This led to the
belief that the ability of the proposed method, with negation
handling, to identify the actual polarity of words was appro-
priate for use within a sentiment analysis task.
Comparing the performance of this method with other
methods that have been reported in literature, Fig. 3 illustrates
the comparative performance of other methods that have
achieved the best results in SemEval. For SemEval-2015-
Task10(A), SemEval-2016-Task4-(B) and SemEval-2017-
Task4-(B), the F-measure of 92.28%, 93.63% and 90.56%
achieved by this proposed method is higher than any of the
F-measures obtained in [69]–[71] methods, respectively.
As can be observed from Figure 3, the results of this
proposed method exceed those of the best results in SemEval
tasks as well as far exceeding the baseline, and it would seem
better than any results reported in literature to date. It also
179964
TABLE 3. Performance (%) on SemEval-2015-Task10-(B),
SemEval-2016-Task4-(A), SemEval-2017-Task4-(A), STS, 5-Prodects Review,
3-Prodects Review, Movie Review and Software Review Datasets.
shows a comparative result in terms of error reduction as this
method demonstrated the best performance overall with the
fewest errors.
3) SENTENCE-LEVEL POLARITY IDENTIFICATION RESULTS
Performance of the sentence-level sentiment identification
using the modified method is shown in Table 3 (perfor-
mance with negation are shown). The results are based on the
average of accuracy, precision, recall, and F-measure values
calculated over multiple runs of 10 times. It can be observed
that on each of the eight datasets (SemEval-2015-Task10-
(B), SemEval-2016 Task4-(A), SemEval-2017 Task4-(A),
Stanford Twitter Sentiment (STS), and 5-Prodects Cus-
tomer Review, 3-Prodects Customer Review, Movie Review,
and Software Review), the classification performance were
always consistent through all the datasets (i.e., different
domains). This suggests that the proposed method may essen-
tially be better able to identify the polarity of a broad cov-
erage context while at the same time achieving favourable
results. The improvement in performance by this method
was due to the following reasons. First, the used method
explores all the semantic and contextual information of the
target text at the fragment-text level. Second, the method
also identified the polarity of the words simultaneously by
progressively incorporating these polarity-assigned words in
the target context. Finally, all the relative negation scope of
words were considered while determining the polarity.
It is informative to compare the results obtained by the
modified method with other well-known approaches that
have achieved the best results in SemEval competitions.
Figure 4 illustrates results based on the SemEval-2015-
Task10-(B), the SemEval-2016 Task4-(A), and the SemEval-
2017 Task4-(A) only, as well as several baselines. As can
be observed from this figure, our modified method outper-
forms the compared methods with F-measures of 84.86%,
87%, and 70.58% as compared with F-measure values of
64.84%, 63.33%, and 67.70% reported in [69]–[71], respec-
tively. It can also be observed that this approach reached
the statistical significance level as will be discussed later
in this section. Therefore, given that this method performs
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method

TABLE 4. Extract from hotels dataset.

FIGURE 4. Performance (% F-measure) of other polarity identification

methods and baselines on SemEval-2015-Task10(B),
SemEval-2016-Task4-(A) and SemEval-2017-Task4-(A).

very favorably against other methods and baselines, it can be

concluded that the modified approach is highly effective and
is obviously better than the approaches it was compared to.
To measure the statistical significance level, the t-test was
used [77] to evaluate the differences in performance between
our modified method and other methods using F-measure val-
ues presented in Figure 4. The results obtained from the t-test
indicate that the used method was statistically significant at
the 95% confidence level. The obtained p-value for compar-
ing results with approaches that were run on SemEval datasets
and associated baselines were 0.021557 and 0.000292. This
means that its performance differed significantly from the
other approaches on the three SemEvals.

B. END-TO-END EXPERIMENTS
A more intuitive appreciation of the modified method results
can be obtained when applying it within two sentence-level
sentiment analysis tasks: hotel review sentiment analysis and
Sohar fort review sentiment analysis.

1) HOTEL REVIEW SENTIMENT ANALYSIS

While this method is obviously applicable to polarity iden-
tification tasks, the method is generic in nature and can in
principle be applied to any sentiment analysis domain. In this
section, the method is applied for identifying the polarity of
several hotel review datasets.
To assess our method’s effectiveness, a small but chal-
lenging hotel review dataset was compiled from randomly
retrieved comments from the booking.com website. This
dataset contains a total of 250 labeled reviews regarding
different hotels located in the city of Muscat in Oman, and averages obtained from over 10 trials. It can be observed
it was grouped into 5 categories (1-star, 2-star, 3-star, 4-star, that the use of the modified method leads to considerably
and 5-star). Each group consists of 50 reviews where each better performance in sentence-level sentiment analysis, as it
review comprises two text-fragments in which one segment provides excellent recall and precision values on this type of
was annotated as positive and other annotated as negative dataset.
(i.e., sentiment tags). An excerpt of this dataset is shown Figure 5 illustrates the results obtained by analyzing the
in Table 4. opinions of customers regarding five categories of hotel ser-
Table 5 shows the results of applying the method to the vices located in Muscat. Each class of hotel review dataset
hotel review datasets and evaluated using the four classifica- was processed separately and the overall sentimental scores
tion measures described above. The values in the table are are normalized between 1 to 100 (i.e., soft classification).

VOLUME 8, 2020 179965

 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method

TABLE 5. Performance (%) on hotels review dataset. TABLE 6. Sohar fort dataset.

FIGURE 5. Observed opinions of the customers regarding some hotel’s

services in Muscat city, Oman.

For 5-star to 1-star, the overall positive and negative scores

are (86%, 14%), (79%, 19%), (77%, 33%), (47%, 53%), and
(35%, 65%), respectively. This indicates that the customers
of 5-star and 4-star hotels had a very positive opinion on
the services provided, and there were almost entirely pos-
itive reviews for 3-star hotels. Therefore, the overall per-
centage of customers with a negative opinion was relatively
low compared with positive opinions in these three classes.
However, for 2-star and 1-star classes, the percentage of
negative reviews was higher than the percentage of positive
views, which was not the case for 5-star, 4-star, and 3-star
hotels. This means that the customers of 2-star and 1-star
hotels were not happy with the services provided in these two
classes of hotel, and those hotels’ administrations should take
appropriate action to address this concern.

2) SOHAR FORT REVIEW SENTIMENT ANALYSIS

In this section, our method was applied to another sentiment
analysis domain that contains unlabeled review dataset. The
unlabeled review dataset about the Sohar city’s fort was
constructed based on reviews from the TripAdvisor web-
site in order to perform further evaluation of the presented
method. The dataset contained a total of 20 review-fragments
from twenty different people posting their opinions regarding
Sohar fort. The issue of deciding opinions in the collected Figure 6 depicts the results obtained by applying our mod-
reviews as positive or negative was conducted by ten human ified method to the opinions of people about the Sohar fort
participants who were then asked to rate the opinion of each dataset. Again, the overall sentimental scores are normalized
review on a scale of very positive (starting from Review 1) to on a scale between 1 to 100. The solid line corresponds
very negative (ending with Review 20) as shown in Table 6. to the degree of positiveness, the grey line corresponds to

179966 VOLUME 8, 2020

K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
FIGURE 6. Performance (%) of the proposed method on the Sohar fort
dataset.
the degree of negativeness. It can be seen that about 40%
(9 reviews) of the visitors posted positively, and around 35%
(6 reviews) and 25% (5 reviews) of them posted negatively
and neutrally, respectively. It can be further observed that the
performance of the method is in accordance with the human
opinions regarding the polarity annotated classes. These types
of result, therefore, definitely leads to the situation wherein
the decision makers can undertake the right action regard-
ing any particular concerns raised. It was observed from
this experiment that the performance of our method is very
promising and shows the potential of applying it in more
sentiment analysis application.
VI. CONCLUSION
This paper has presented a new implementation of the
lexicon-based word polarity identification method on sev-
eral customer reviews based on the use of context expan-
sion. Unlike the existing approaches, which measure pairwise
relatedness between words at term-level over a fixed-size
context window around the target word, the modified method
determines the polarity of a target word by calculating the
semantic relatedness between context expansion contained
in WordNet’s synonyms, gloss, and usage-examples; further-
more, the synonyms expansion consists of all synonym-words
of all words surrounding (but not including) the target word
within the given text fragment. The actual polarity of the
target word is assigned as the WordNet’s synset (with its
sentimental information retrieved from SentiWordNet) for
which the semantic similarity between context expansion set
and synonyms expansion set is greatest. This was inspired
by the belief that people coherently identify the polarity of
words based on the whole context in which they appear. The
VOLUME 8, 2020
method also identifies the polarity of the words simultane-
ously by progressively incorporating these polarity-assigned
words in the synonyms expansion context that contains all
words surrounding the target word. This allows it to explore
and capture highly related sentimental information in order
to determine an accurate polarity of the word. While Word-
Net and SentiWordNet do not make available the knowledge
level regarding words similar to what a human being does,
it provides text-fragments of such information through its
definitional glosses, usage-examples, and semantic relations
between words. The results have indicated this method to
have achieved excellent performance as evaluated on several
benchmark datasets through different models of evaluation.
Apart from the use of context expansions, another con-
tribution of this method was handling the relative negation
scope of words while identifying their polarities, thereby
providing better performance in the polarity identification
process. The detection of the negation scope was performed
by determining the word following the negation term, and
then removing this negation term and replacing it with its
possible antonym-word. The empirical results have illustrated
that incorporating such negation handling as a preliminary
step obviously effects a better overall performance.
The experimental results have demonstrated that our
modified method achieves considerable performance as mea-
sured with stand-alone and end-to-end evaluation tasks.
However, an important future direction of research lies in
applying word polarity identification in the context of more
all-encompassing sentiment analysis applications using dif-
ferent lexicon resources and languages, such as Arabic. It has
been demonstrated that the incorporation of negation scope
detection can lead to improvements in sentiment analysis
performance. The feasibility of using the word polarity identi-
fication technique in other sentiment analysis tasks applicable
for smart city services is currently being explored.
ACKNOWLEDGMENT
The authors would also like to thank Sohar University for
valuable support. This publication reflects the views only
of the authors, and the TRC cannot be held responsible for
any use which may be made of the information contained
therein.
REFERENCES
[1] M. Hu and B. Liu, ‘‘Mining and summarizing customer reviews,’’ in Proc.
KDD, Seattle, WA, USA, 2004, pp. 168–177.
[2] R. Feldman, ‘‘Techniques and applications for sentiment analysis,’’ Com-
mun. ACM, vol. 56, no. 4, pp. 82–89, Apr. 2013.
[3] W. Zhao, Z. Guan, L. Chen, X. He, D. Cai, B. Wang, and Q. Wang,
‘‘Weakly-supervised deep embedding for product review sentiment analy-
sis,’’ IEEE Trans. Knowl. Data Eng., vol. 30, no. 1, pp. 185–197, Jan. 2018.
[4] R. S. Jagdale, V. S. Shirsat, and S. N. Deshmukh, ‘‘Sentiment analysis
on product reviews using machine learning techniques,’’ in Cognitive
Informatics and Soft Computing (Advances in Intelligent Systems and
Computing), vol. 768, P. Mallick, V. Balas, A. Bhoi, and A. Zobaa, Eds.
Singapore: Springer, 2019, pp. 639–647.
[5] N. Ko, B. Jeong, S. Choi, and J. Yoon, ‘‘Identifying product opportunities
using social media mining: Application of topic modeling and chance
discovery theory,’’ IEEE Access, vol. 6, pp. 1680–1693, 2018.
179967
 K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method
[6] F. Ali, D. Kwak, P. Khan, S. El-Sappagh, A. Ali, S. Ullah, K. H. Kim,
and K.-S. Kwak, ‘‘Transportation sentiment analysis using word embed-
ding and ontology-based topic modeling,’’ Knowl.-Based Syst., vol. 174,
pp. 27–42, Jun. 2019.
[7] E. Riloff, J. Wiebe, and W. Phillips, ‘‘Exploiting subjectivity classification
to improve information extraction,’’ in Proc. 20th Nat. Conf. Artif. Intell.,
Pittsburgh, PA, USA, 2005, pp. 1106–1111.
[8] A.-M. Popescu and O. Etzioni, ‘‘Extracting product features and opinions
from reviews,’’ in Natural Language Processing and Text Mining. London,
U.K.: Springer, 2007, pp. 9–28.
[9] H. Takamura, T. Inui, and M. Okumura, ‘‘Extracting emotional polarity of
words using spin model,’’ in Proc. 43rd Annu. Meeting Assoc. Comput.
Linguistics. Ann Arbor, MI, USA: The Association for Computational
Linguistics, 2005, pp. 133–140.
[10] S. Mukund, R. Srihari, and E. Peterson, ‘‘An information-extraction sys-
tem for urdu—A resource-poor language,’’ ACM Trans. Asian Lang. Inf.
Process., vol. 9, no. 4, pp. 1–43, Dec. 2010.
[11] L. Abberley, N. Gould, K. Crockett, and J. Cheng, ‘‘Modelling road
congestion using ontologies for big data analytics in smart cities,’’ in Proc.
Int. Smart Cities Conf. (ISC), Wuxi, China, Sep. 2017, pp. 1–6.
[12] J. F. F. Pereira, ‘‘Social media text processing and semantic anal-
ysis for smart cities,’’ 2017, arXiv:1709.03406. [Online]. Available:
https://arxiv.org/abs/1709.03406
[13] C. Musto, G. Semeraro, M. de Gemmis, and P. Lops, ‘‘Developing smart
cities services through semantic analysis of social streams,’’ in Proc. 24th
Int. Conf. World Wide Web, Florence, Italy, 2015, pp. 18–22.
[14] A. Al Nuaimi, A. Al Shamsi, A. Al Shamsi, and E. Badidi, ‘‘Social media
analytics for sentiment analysis and event detection in smart cities,’’ in
Proc. 4th Int. Conf. Natural Lang. Comput. (NATL), Apr. 2018, pp. 57–64.
[15] S. Singhal, S. Maheshwari, and M. Meena, ‘‘Survey of challenges in
sentiment analysis,’’ in Recent Findings in Intelligent Computing Tech-
niques (Advances in Intelligent Systems and Computing), vol. 709, P. Sa,
S. Bakshi, I. Hatzilygeroudis, and M. Sahoo, Eds. Singapore: Springer,
2018.
[16] E. Cambria, ‘‘Affective computing and sentiment analysis,’’ IEEE
Intell. Syst., vol. 31, no. 2, pp. 102–107, Mar./Apr. 2016, doi:
10.1109/MIS.2016.31.
[17] S. Behdenna, F. Barigou, and G. Belalem, ‘‘Sentiment analysis at document
level,’’ in Smart Trends in Information Technology and Computer Com-
munications (Communications in Computer and Information Science),
vol. 628, A. Unal, M. Nayak, D. Mishra, D. Singh, and A. Joshi, Eds.
Singapore: Springer, 2016, pp. 159–168.
[18] A. Abdi, S. M. Shamsuddin, S. Hasan, and J. Piran, ‘‘Automatic sentiment-
oriented summarization of multi-documents using soft computing,’’ Soft
Comput., vol. 23, pp. 10551–10568, Dec. 2018, doi: 10.1007/S00500-018-
3653-4.
[19] R. Arulmurugan, K. R. Sabarmathi, and H. Anandakumar, ‘‘Classifica-
tion of sentence level sentiment analysis using cloud machine learning
techniques,’’ Cluster Comput., vol. 22, pp. 1199–1209, Sep. 2017, doi:
10.1007/s10586-017-1200-1.
[20] T. Wu, D. S. Weld, and J. Heer, ‘‘Local decision pitfalls in interactive
machine learning: An investigation into feature selection in sentiment
analysis,’’ ACM Trans. Comput.-Hum. Interact., vol. 26, no. 4, pp. 1–27,
Jul. 2019.
[21] A. Alarifi, A. Tolba, Z. Al-Makhadmeh, and W. Said, ‘‘A big data approach
to sentiment analysis using greedy feature selection with cat swarm
optimization-based long short-term memory neural networks,’’ J. Super-
comput., vol. 76, no. 6, pp. 4414–4429, Jun. 2020, doi: 10.1007/s11227-
018-2398-2.
[22] M. T. Al-Sharuee, F. Liu, and M. Pratama, ‘‘Sentiment analysis: Dynamic
and temporal clustering of product reviews,’’ Appl. Intell., Mar. 2020, doi:
10.1007/s10489-020-01668-6.
[23] N. K. Singh, D. S. Tomar, and A. K. Sangaiah, ‘‘Sentiment analysis:
A review and comparative analysis over social media,’’ J. Ambient Intell.
Hum. Comput., vol. 11, no. 1, pp. 97–117, Jan. 2020, doi: 10.1007/s12652-
018-0862-8.
[24] A. Krouska, C. Troussas, and M. Virvou, ‘‘Comparative evaluation of
algorithms for sentiment analysis over social networking services,’’ J. UCS,
vol. 23, no. 8, pp. 755–768, 2017.
[25] J. Song, K. T. Kim, B. Lee, S. Kim, and H. Y. Youn, ‘‘A novel classifi-
cation approach based on Naïve Bayes for Twitter sentiment analysis,’’
KSII Trans. Internet Inf. Syst., vol. 11, no. 6, pp. 2996–3012, 2017, doi:
10.3837/tiis.2017.06.011.
179968
[26] V. Hatzivassiloglou and K. R. McKeown, ‘‘Predicting the semantic orien-
tation of adjectives,’’ in Proc. 35th Annu. Meeting Assoc. Comput. Linguis-
tics. Madrid, Spain: The Association for Computational Linguistics, 1997,
pp. 174–181.
[27] P. D. Turney and M. L. Littman, ‘‘Measuring praise and criticism: Inference
of semantic orientation from association,’’ ACM Trans. Inf. Syst., vol. 21,
no. 4, pp. 315–346, Oct. 2003.
[28] M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, ‘‘Lexicon-
based methods for sentiment analysis,’’ Comput. Linguistics, vol. 37, no. 2,
pp. 267–307, Jun. 2011.
[29] B. Agarwal and N. Mittal, ‘‘Semantic orientation-based approach for sen-
timent analysis,’’ in Prominent Feature Extraction for Sentiment Analysis.
Cham, Switzerland: Springer, 2016, pp. 77–88.
[30] A. Nawaz, S. Asghar, and S. H. A. Naqvi, ‘‘A segregational approach for
determining aspect sentiments in social media analysis,’’ J. Supercomput.,
vol. 75, no. 5, pp. 2584–2602, May 2019, doi: 10.1007/s11227-018-2664-
3.
[31] S. Bandari and V. V. Bulusu, ‘‘Survey on ontology-based sentiment
analysis of customer reviews for products and services,’’ in Data Engi-
neering and Communication Technology (Advances in Intelligent Sys-
tems and Computing), vol. 1079, K. Raju, R. Senkerik, S. Lanka, and
V. Rajagopal, Eds. Singapore: Springer, 2020.
[32] M. E. Mowlaei, M. S. Abadeh, and H. Keshavarz, ‘‘Aspect-based senti-
ment analysis using adaptive aspect-based lexicons,’’ Expert Syst. Appl.,
vol. 148, Jun. 2020, Art. no. 113234.
[33] C. Hung and H.-K. Lin, ‘‘Using objective words in SentiWordNet to
improve word-of-mouth sentiment classification,’’ IEEE Intell. Syst.,
vol. 28, no. 2, pp. 47–54, Mar. 2013.
[34] L. Gatti, M. Guerini, and M. Turchi, ‘‘SentiWords: Deriving a high preci-
sion and high coverage lexicon for sentiment analysis,’’ IEEE Trans. Affect.
Comput., vol. 7, no. 4, pp. 409–421, Oct. 2016.
[35] C. Song, X.-K. Wang, P.-F. Cheng, J.-Q. Wang, and L. Li, ‘‘SACPC:
A framework based on probabilistic linguistic terms for short text senti-
ment analysis,’’ Knowl.-Based Syst., vol. 194, Apr. 2020, Art. no. 105572.
[36] W. Medhat, A. Hassan, and H. Korashy, ‘‘Sentiment analysis algo-
rithms and applications: A survey,’’ Ain Shams Eng. J., vol. 5, no. 4,
pp. 1093–1113, Dec. 2014.
[37] K. Ravi and V. Ravi, ‘‘A survey on opinion mining and sentiment anal-
ysis: Tasks, approaches and applications,’’ Knowl.-Based Syst., vol. 89,
pp. 14–46, Nov. 2015.
[38] M. Giatsoglou, M. G. Vozalis, K. Diamantaras, A. Vakali, G. Sarigiannidis,
and K. C. Chatzisavvas, ‘‘Sentiment analysis leveraging emotions and
word embeddings,’’ Expert Syst. Appl., vol. 69, pp. 214–224, Mar. 2017.
[39] A. Hogenboom, P. van Iterson, B. Heerschop, F. Frasincar, and U. Kaymak,
‘‘Determining negation scope and strength in sentiment analysis,’’ in Proc.
IEEE Int. Conf. Syst., Man, Cybern., Oct. 2011, pp. 2589–2594.
[40] U. Farooq, H. Mansoor, A. Nongaillard, Y. Ouzrout, and M. A. Qadir,
‘‘Negation handling in sentiment analysis at sentence level,’’ J. Comput.,
vol. 12, no. 5, pp. 470–478, 2017.
[41] D. Gautam, N. Maharjan, R. Banjade, L. J. Tamang, and V. Rus, ‘‘Long
short term memory based models for negation handling in tutorial dia-
logues,’’ in Proc. 31st Int. Flairs Conf. (FLAIRS), 2018, pp. 1–6.
[42] R. Banjade, N. B. Niraula, and V. Rus, ‘‘Towards detecting intra- and inter-
sentential negation scope and focus in dialogue,’’ in Proc. FLAIRS Conf.,
2016, pp. 198–203.
[43] J. Barnes, E. Velldal, and L. Vrelid, ‘‘Improving sentiment analysis with
multi-task learning of negation,’’ Natural Lang. Eng., vol. 1, no. 1,
pp. 1–25, 2019.
[44] C. E. Fellbaum, WordNet: An Electronic Lexical Database. Cambridge,
MA, USA: MIT Press, 1998.
[45] A. Esuli and F. Sebastiani, ‘‘Determining the semantic orientation of terms
through gloss classification,’’ in Proc. 14th ACM Int. Conf. Inf. Knowl.
Manage. (CIKM), Bremen, Germany, 2005, pp. 617–624.
[46] D. T. Santosh, K. S. Babu, S. D. V. Prasad, and A. Vivekananda, ‘‘Opinion
mining of online product reviews from traditional LDA topic clusters using
feature ontology tree and SentiWordNet,’’ Int. J. Educ. Manage. Eng.,
vol. 6, no. 6, pp. 34–44, 2016.
[47] M. E. Moussa, E. H. Mohamed, and M. H. Haggag, ‘‘A generic lexicon-
based framework for sentiment analysis,’’ Int. J. Comput. Appl., vol. 42,
pp. 463–473, Jun. 2018, doi: 10.1080/1206212X.2018.1483813.
[48] A. Jurek, M. D. Mulvenna, and Y. Bi, ‘‘Improved lexicon-based senti-
ment analysis for social media analytics,’’ Secur. Informat., vol. 4, no. 1,
pp. 1–13, Dec. 2015.
VOLUME 8, 2020
K. Abdalgader, A. Al Shibli: Experimental Results on Customer Reviews Using Lexicon-Based Word Polarity Identification Method

[49] A. Pak and P. Paroubek, ‘‘Twitter as a corpus for sentiment analysis [71] S. Rosenthal, N. Farra, and P. Nakov, ‘‘SemEval-2017 task 4: Sentiment
and opinion mining,’’ in Proc. 7th Conf. Int. Lang. Resour. Eval., 2010, analysis in Twitter,’’ in Proc. 11th Int. Workshop Semantic Eval. (SemEval),
pp. 1320–1326. Vancouver, BC, Canada, 2017, pp. 502–518.
[50] D. Rice and C. Zorn, ‘‘Corpus-based dictionaries for sentiment analysis [72] A. Go, R. Bhayani, and L. Huang, ‘‘Twitter sentiment classification using
of specialized vocabularies,’’ in Proc. New Directions Analyzing Text Data distant supervision,’’ Stanford, CA, USA, CS224N Project Rep., 2009,
Workshop (NDATAD), Sep. 2013, pp. 1–16. pp. 1–12.
[51] A. Esuli and F. Sebastiani, ‘‘Pageranking wordnet synsets: An application [73] M. Hu and B. Liu, ‘‘Mining and summarizing customer reviews,’’ in Proc.
to opinion mining,’’ in Proc. 45th Annu. Meeting Assoc. Comput. Linguis- ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), 2004,
tics (ACL), 2007, pp. 424–431. pp. 164–168.
[52] A. Esuli and F. Sebastiani, ‘‘SentiWordNet: A publicly available lexical [74] Q. Liu, Z. Gao, B. Liu, and Y. Zhang, ‘‘Automated rule selection for aspect
resource for opinion mining,’’ in Proc. 5th Conf. Lang. Resour. Eval. extraction in opinion mining,’’ in Proc. 24th Int. Conf. Artif. Intell. (IJCAI),
(LREC), vol. 6, 2006, pp. 417–422. 2015, pp. 1291–1297.
[75] B. Pang and L. Lee, ‘‘A sentimental education: Sentiment analysis using
[53] H. Kanayama and T. Nasukawa, ‘‘Fully automatic lexicon expansion for
subjectivity summarization based on minimum cuts,’’ in Proc. Assoc.
domain-oriented sentiment analysis,’’ in Proc. Conf. Empirical Meth-
Comput. Linguistics (ACL), Barcelona, Spain, 2004, pp. 271–278.
ods Natural Lang. Process. (EMNLP), Sydney, NSW, Australia, 2006,
[76] J. Blitzer, M. Dredze, and F. Pereira, ‘‘Biographies, bollywood, boom-
pp. 22–23.
boxes and blenders: Domain adaptation for sentiment classification,’’
[54] G. Qiu, B. Liu, J. Bu, and C. Chen, ‘‘Expanding domain sentiment lexicon
in Proc. 45th Annu. Meeting Assoc. Comput. Linguistics (ACL), 2007,
through double propagation,’’ in Proc. 21st Int. Joint Conf. Artif. Intell.,
pp. 440–447.
Pasadena, CA, USA, 2009 pp. 1199–1204.
[77] R. A. Johnson and G. K. Bhattacharyya, Statistics: Principles and Meth-
[55] L. Zhang and B. Liu, ‘‘Identifying noun product features that imply opin- ods. Hoboken, NJ, USA: Wiley, 2014.
ions,’’ in Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL), Portland,
OR, USA, 2011, pp. 575–580.
[56] S. Taj, B. B. Shaikh, and A. F. Meghji, ‘‘Sentiment analysis of news
articles: A lexicon based approach,’’ in Proc. 2nd Int. Conf. Comput., Math.
Eng. Technol. (iCoMET), Sukkur, Pakistan, Jan. 2019, pp. 1–5.
[57] C. Strapparava and A. Valitutti, ‘‘WordNet-affect: An affective extension
of WordNet,’’ in Proc. 4th Int. Conf. Lang. Resour. Eval. (LREC), Lisbon,
Portugal, vol. 4, 2004, pp. 1083–1086.
[58] C. Akkaya, J. Wiebe, A. Conrad, and R. Mihalcea, ‘‘Improving the impact
of subjectivity word sense disambiguation on contextual opinion analysis,’’
in Proc. CoNLL, 2011, pp. 87–96.
[59] C. Hung and S.-J. Chen, ‘‘Word sense disambiguation based sentiment
lexicons for sentiment classification,’’ Knowl.-Based Syst., vol. 110,
pp. 224–232, Oct. 2016.
[60] T. Wilson, J. Wiebe, and P. Hoffmann, ‘‘Recognizing contextual polarity: KHALED ABDALGADER received the B.Sc.
An exploration of features for phrase-level sentiment analysis,’’ Comput. degree in computer science from Sebha University,
Linguistics, vol. 35, no. 3, pp. 339–433, 2009. Libya, in 2001, the M.Sc. degree from University
[61] M. Marchand, R. Besançon, O. Mesnard, and A. Vilnat, ‘‘Domain adapta- Utara Malaysia, in 2004, and the Ph.D. degree in
tion for opinion mining: A study of multipolarity words,’’ J. Lang. Technol. natural language processing from La Trobe Uni-
Comput. Linguistics, vol. 29, no. 1, pp. 17–31, 2014. versity, Australia, in 2012. He was a Lecturer with
[62] C. Akkaya, J. Wiebe, and R. Mihalcea, ‘‘Subjectivity word sense dis- the Department of Computer Science, Sebha Uni-
ambiguation,’’ in Proc. Conf. Empirical Methods Natural Lang. Pro- versity, from 2004 to 2007. Since 2013, he has
cess. Singapore: The Association for Computational Linguistics, 2009, been an Assistant Professor with the Faculty of
pp. 190–199. Computing and Information Technology, Sohar
[63] J. Wiebe and R. Mihalcea, ‘‘Word sense and subjectivity,’’ in Proc. 21st University, Oman. His research interests include natural language processing
Int. Conf. Comput. Linguistics, 44th Annu. Meeting Assoc. Comput. Lin- and understanding, particularly word sense disambiguation, semantic text
guistics. Sydney, NSW, Australia: The Association for Computational similarity, text mining, sentiment analysis, and knowledge discovery from
Linguistics, 2006, pp. 1065–1072. textual collections.
[64] B. R. Razon and D. L. Salle, ‘‘Word sense disambiguation of opinionated
words using extended gloss overlap,’’ in Proc. 8th Nat. Natural Lang.
Process. Res. Symp., Manila, Philippines, 2011, pp. 1–5.
[65] M. Tamara, B. Alexandra, and M. Andres, ‘‘Word sense disambiguation in
opinion mining: Pros and cons,’’ J. Res. Comput. Sci., vol. 46, pp. 119–130,
2010.
[66] V. Rentoumi, G. Giannakopoulos, V. Karkaletsis, and G. A. Vouros, ‘‘Sen-
timent analysis of figurative language using a word sense disambigua-
tion approach,’’ in Proc. Int. Conf. RANLP, Borovets, Bulgaria, 2009,
pp. 370–375.
[67] K. Abdalgader and A. Skabar, ‘‘Unsupervised similarity-based word sense
disambiguation using context vectors and sentential word importance,’’
ACM Trans. Speech Lang. Process., vol. 9, no. 1, pp. 1–21, May 2012.
[68] R. Rada, H. Mili, E. Bicknell, and M. Blettner, ‘‘Development and appli-
cation of a metric on semantic nets,’’ IEEE Trans. Syst., Man, Cybern., AYSHA AL SHIBLI received the B.Sc. degree in
vol. 19, no. 1, pp. 17–30, Jan./Feb. 1989. information technology and the M.Sc. degree in
[69] S. Rosenthal, P. Nakov, S. Kiritchenko, S. Mohammad, A. Ritter, and computer science from Sohar University, Oman,
V. Stoyanov, ‘‘SemEval-2015 task 10: Sentiment analysis in Twitter,’’ in in 2010 and 2016, respectively. From 2010 to
Proc. 9th Int. Workshop Semantic Eval. (SemEval), Denver, CO, USA, 2016, she was a Teaching Assistant with the Fac-
2015, pp. 451–463. ulty of Computing and Information Technology,
[70] P. Nakov, A. Ritter, S. Rosenthal, F. Sebastiani, and V. Stoyanov, Sohar University, where she is currently a Lecturer.
‘‘SemEval2016 task 4: Sentiment analysis in Twitter,’’ in Proc. 10th Her research interests include the fields of artificial
Int. Workshop Semantic Eval. (SemEval), San Diego, CA, USA, 2016, intelligence, data mining, and smart systems.
pp. 1–18.

VOLUME 8, 2020 179969