This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE TRANSACTIONS ON KNOWLEDGE
IEEE Transactions AND DATA
on Knowledge ENGINEERING,
and Data VOL. (?,Volume:
Engineering NO. ?, AUGUST 2017 1,
30, Issue: Jan. 1 2018 ) 1

Weakly-supervised Deep Embedding for

Product Review Sentiment Analysis
Wei Zhao, Ziyu Guan∗ , Long Chen, Xiaofei He, Fellow, IAPR, Deng Cai, Beidou Wang and Quan Wang

Abstract—Product reviews are valuable for upcoming buyers in helping them make decisions. To this end, different opinion mining
techniques have been proposed, where judging a review sentence’s orientation (e.g. positive or negative) is one of their key
challenges. Recently, deep learning has emerged as an effective means for solving sentiment classification problems. A neural network
intrinsically learns a useful representation automatically without human efforts. However, the success of deep learning highly relies on
the availability of large-scale training data. We propose a novel deep learning framework for product review sentiment classification
which employs prevalently available ratings as weak supervision signals. The framework consists of two steps: (1) learning a high level
representation (an embedding space) which captures the general sentiment distribution of sentences through rating information; (2)
adding a classification layer on top of the embedding layer and use labeled sentences for supervised fine-tuning. We explore two kinds
of low level network structure for modeling review sentences, namely, convolutional feature extractors and long short-term memory. To
evaluate the proposed framework, we construct a dataset containing 1.1M weakly labeled review sentences and 11,754 labeled review
sentences from Amazon. Experimental results show the efficacy of the proposed framework and its superiority over baselines.

Index Terms—Deep learning, opinion mining, sentiment classification, weak-supervision.

1 I NTRODUCTION

W ITH the booming of e-commerce, people are getting used

to consuming online and writing comments about their
purchase experiences on merchant/review Websites. These opin-
ionated contents are valuable resources both to future customers
for decision-making and to merchants for improving their products
and/or service. However, as the volume of reviews grows rapidly,
people have to face a severe information overload problem. To
alleviate this problem, many opinion mining techniques have been
proposed, e.g. opinion summarization [8], [19], opinion polling
[54], and comparative analysis [26]. The key challenge is how to
accurately predict the sentiment orientation of review sentences.
Popular sentiment classification methods generally fall into
two categories: (1) lexicon-based methods and (2) machine learn-
ing methods. Lexicon-based methods [8], [19], [47] typically take
the tack of first constructing a sentiment lexicon of opinion words
(e.g. “wonderful”, “disgusting”), and then design classification
rules based on appeared opinion words and prior syntactic knowl-
edge. Despite effectiveness, this kind of methods require substan-
tial efforts in lexicon construction and rule design. Furthermore,
lexicon-based methods cannot well handle implicit opinions, i.e.
objective statements such as “I bought the mattress a week ago,
and a valley appeared today”. As pointed out in [12], this is also an
important form of opinions. Factual information is usually more
helpful than subjective feelings. Lexicon-based methods can only
deal with implicit opinions in an ad-hoc way [52].
The first machine learning based sentiment classification work
[35] applied popular machine learning algorithms such as Naive
Bayes to the problem. After that, most research in this direc-
tion revolved around feature engineering for better classification
performance. Different kinds of features have been explored,
e.g. n-grams [6], Part-of-speech (POS) information and syntactic
relations [32], etc. Feature engineering also costs a lot of human
efforts, and a feature set suitable for one domain may not generate
good performance for other domains [34].
In recent years, deep learning has emerged as an effective
means for solving sentiment classification problems [14], [21],
[40], [41], [44], [45]. A deep neural network intrinsically learns a
high level representation of the data [2], thus avoiding laborious
work such as feature engineering. A second advantage is that deep
models have exponentially stronger expressive power than shallow
models. However, the success of deep learning heavily relies on
the availability of large-scale training data [1], [2]. Labeling a
large number of sentences is very laborious.

* Corresponding author
• W. Zhao and Q. Wang are with the School of Computer Sci-
ence and Technology, Xidian University, Xi’an, CN 710127. E-mail:
ywzhao@mail.xidian.edu.cn; qwang@xidian.edu.cn
• Z. Guan and L. Chen are with the School of Information and Tech-
nology, Northwest University of China, Xi’an, CN 710127. E-mail:
ziyuguan@nwu.edu.cn; longchen@stumail.nwu.edu.cn
• X. He and D. Cai are with the State Key Lab of CAD&CG, College of
Computer Science, Zhejiang University, Hangzhou, CN 310027. E-mail:
{xiaofeihe, dengcai}@cad.zju.edu.cn
• B. Wang is with the School of Computing Science, Simon Fraser University,
Burnaby, BC V5A 1S6, Canada. E-mail: beidouw@sfu.ca
Manuscript received April 19, 2005; revised August 26, 2015.
Fig. 1. A 5-stars review with negative words.
Fortunately, most merchant/review Websites allow customers
to summarize their opinions by an overall rating score (typically
in 5-stars scale). Ratings reflect the overall sentiment of customer
reviews and have already been exploited for sentiment analysis
[27], [37]. Nevertheless, review ratings are not reliable labels for
the constituent sentences, e.g. a 5-stars review can contain negative
sentences and we may also see positive words occasionally in 1-

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
star reviews. An example is shown in Figure 1. Therefore, treating
binarized ratings as sentiment labels could confuse a sentiment
classifier for review sentences.
Despite the promising performance of deep learning on senti-
ment classification, no previous work tried to leverage the preva-
lently available ratings for training deep models. In this work,
we propose a novel deep learning framework for review sentence
sentiment classification. The framework treats review ratings as
weak labels to train deep neural networks. For example, with
5-stars scale we can deem ratings above/below 3-stars as posi-
tive/negative weak labels respectively. The framework generally
consists of two steps. In the first step, rather than predicting
sentiment labels directly, we try to learn an embedding space
(a high level layer in the neural network) which reflects the
general sentiment distribution of sentences, from a large number
of weakly labeled sentences. That is, we force sentences with
the same weak labels to be near each other, while sentences
with different weak labels are kept away from one another. To
reduce the impact of sentences with rating-inconsistent orientation
(hereafter called wrong-labeled sentences), we propose to penalize
the relative distances among sentences in the embedding space
through a ranking loss. In the second step, a classification layer
is added on top of the embedding layer, and we use labeled
sentences to fine-tune the deep network. The framework is dubbed
Weakly-supervised Deep Embedding (WDE). Regarding network
structure, two popular schemes are adopted to learn to extract
fixed-length feature vectors from review sentences, namely, con-
volutional feature extractors [21] and Long Short-Term Memory
(LSTM) [16], [18]. With a slight abuse of concept, we will refer
to the former model as Convolutional Neural Network based
WDE (WDE-CNN); the latter one is called LSTM based WDE
(WDE-LSTM). We then compute high level features (embedding)
by synthesizing the extracted features, as well as the contextual
aspect information (e.g. screen of cell phones) of the product. The
aspect input represents prior knowledge regarding the sentence’s
orientation.
The main contributions of this paper are summarized as
follows:
1) We propose a new deep learning framework WDE which
can leverage the vast amount of weakly labeled review
sentences for sentiment analysis. The framework first
tries to capture the sentiment distribution of the data by
embedding training on weakly labeled sentences. Then
it uses a few labeled sentences for deep network fine-
tuning, as well as for prediction model learning. We
empirically demonstrate this “weakly pre-training + su-
pervised fine-tuning” idea is feasible. The idea could also
be useful for exploiting other kinds of weakly labeled
data (e.g. tagging data [17]).
2) We devise a general neural network architecture for
WDE and instantiate it by two popular neural network
schemes for modeling text data: CNN and LSTM. We
compare WDE-CNN and WDE-LSTM in terms of their
effectiveness, efficiency and specialties on this sentiment
classification task.
3) To evaluate WDE we construct a dataset containing
1.1M weakly labeled review sentences and 11,754
labeled review sentences from three domains
of Amazon, i.e. digital cameras, cell phones
and laptops. The dataset can be downloaded at
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2
https://www.dropbox.com/s/aji68llxmtcuu5l/data.zip.
Experimental results show that WDE is effective and
outperforms baseline methods.
The rest of the paper is organized as follows: the next section
outlines related work. Section 3 presents the WDE framework
and explains the design of each step in details. Experiments are
described in Section 4, and finally, Section 5 concludes our work
and discusses about future work.
2 R ELATED W ORK
Sentiment analysis is a long standing research topic. Readers can
refer to [25] for a recent survey. Sentiment classification is one
of the key tasks in sentiment analysis and can be categorized as
document level, sentence level and aspect level [25]. Traditional
machine learning methods for sentiment classification can gener-
ally be applied to the three levels [25]. Our work falls into the
last category since we consider aspect information. In the next we
review two subtopics closely related to our work.
2.1 Deep Learning for Sentiment Classification
In recent years, deep learning techniques have been exploited to
address text related problems, e.g. information retrieval [38], ques-
tion answering [36] and text categorization [53]. In the sentiment
analysis community, researchers have explored different deep
models for sentiment classification. Glorot et al. used stacked de-
noising auto-encoder to train review representation in an unsuper-
vised fashion, in order to address the domain adaptation problem
of sentiment classification [14]. Socher et al. [39]–[41] proposed
a series of Recursive Neural Network (RecNN) models for sen-
timent classification. These methods learn vector representations
of variable-length sentences through compositional computation
recursively. Kim investigated using CNN for sentence sentiment
classification and found it outperformed RecNN [21]. A variant
CNN with dynamic k-max pooling and multiple convolutional lay-
ers was proposed in [20]. Researchers have also investigated using
sequential models such as Recurrent Neural Network (RNN) such
as Long Short-Term Memory (LSTM) for sentiment classification
[44]. In the following, we review neural models for text embedding
and aspect level sentiment classification.
Text embedding learning. There is an increasing interest in
learning distributed text embedding by neural models for text
understanding tasks among which sentiment classification is a
popular one. Le and Mikolov [24] developed an unsupervised
embedding learning method for sentences, paragraphs and docu-
ments. Two simple network models were proposed which were
inspired by [30]. Kiros et al. [22] proposed the unsupervised
skip-thoughts model which generalized the skip-gram model [30]
to the sentence level. The key idea was to use the embedding
representation of a sentence to predict its surrounding sentences. A
supervised sentence embedding learning framework was proposed
by Wieting et al. [51], where sentence similarity in the embedding
space was trained according to a paraphrase database via a margin
loss. Six network architectures were tested and compared.
Aspect-based neural models for sentiment classification. Re-
cently, neural models have been proposed for aspect level (or target
dependent) sentiment classification. Several works were built on
the RecNN models: Dong et al. proposed a variant of RecNN
for target dependent sentiment classification on Twitter [9]. The
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
method used a set of composition functions and syntactic rela-
tionships between words to control the propagation of sentiments
to targets. Nguyen and Shirai [33] further extended the method
of [9] by using syntactic information from both the dependency
and constituent trees of sentences. In [23], Lakkaraju et al. used
RecNN to jointly perform aspect detection and sentiment predic-
tion. Researchers also explored adapting LSTM for aspect level
sentiment classification. Tang et al. proposed two LSTM based
models for this task [43]. The first one used two LSTMs to model
preceding and succeeding contexts of the target words respectively
and built target specific representation by a concatenation of the
outputs of the two LSTMs. The second one, which performed bet-
ter, explicitly took the target vector as an input for each time step.
The target vector was simply an average of target word vectors.
Wang et al. [49] introduced the attention mechanism into LSTM to
learn aspect specific sentence representation. Similar to our work,
aspects were modeled as embedding vectors and used for attention
estimation. One of the state-of-art methods was proposed by Tang
et al. [45] which is based on deep memory network. The key
idea was to use content- and location-based attention models to
learn the weight of each context word and combine context words
and the aspect word (after linear transformation) by summation,
to learn sentence representation in different levels. However, the
model was only naturally defined for single-word aspects, and it
did not try to model implicit aspects [25].
None of the above works tried to use review ratings to train
deep sentiment classifiers for sentences. This is not a trivial
problem since ratings are too noisy to be used directly as sentence
labels (see Section 3 and experiments for discussions of this issue).
To our knowledge, the WDE framework is the first attempt to
make use of rating information as weak labels for training deep
sentence sentiment classifiers. While some of the studies described
above also revolved around text embedding by neural networks,
they were not concerned with weak supervision. Our work is
orthogonal to theirs in that we could use/adapt their methods to
pre-train the deep models. In this paper, our focus is to investigate
whether the proposed WDE framework can effectively exploit
weakly labeled data, thus we do not employ those methods for pre-
training. Note that although we instantiate WDE with CNN and
LSTM which are two popular neural schemes for text, the idea of
WDE could also be applied to other types of deep models as well,
such as the existing aspect-based neural models aforementioned.
The major contribution of this work is a weakly-supervised deep
learning framework, rather than specific deep models.
2.2 Exploiting Ratings in Sentence Sentiment Classifi-
cation
Rating information has been exploited in sentence sentiment clas-
sification. Qu et al. incorporated ratings as weak labels in a proba-
bilistic framework for sentence level sentiment classification [37].
However, their method still required careful feature design and
relied on base predictors. While our method automatically learns
a meaningful sentence representation for sentiment classification.
Täckström and McDonald used conditional random fields to
combine review level and sentence level sentiment labels for sen-
tence sentiment analysis [42]. This method also required feature
engineering. Maas et al. [27] proposed to learn sentiment-bearing
word vectors by incorporating rating information in a probabilistic
model. For sentiment classification, they simply averaged the word
vectors of a document as its representation. A similar work is
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
3
[46], which developed a variant of the C&W neural model [5] for
learning sentiment-bearing word vectors from weak tweet labels
derived from emoticons. The tweet representation was obtained
by min, max and avg pooling on word vectors. Although this kind
of methods can generate sentence representations automatically,
the representations were derived by simple pooling of the learned
word vectors. In comparison, our method generates a sentence
representation by feeding word vectors through an expressive
deep neural network. Moreover, we directly optimize sentence
representation, rather than word vectors. We take the above two
methods as baselines in experiments.
3 W EAKLY - SUPERVISED D EEP E MBEDDING
The classic deep learning methods take an “unsupervised training
then supervised fine-tuning” scheme, where restricted Boltzmann
machines (RBM) or auto-encoders are used to pre-train network
parameters from large quantities of unlabeled data [1]. This works
well when the data distribution is correlated with label prediction
[1]. Nevertheless, in sentiment analysis the word co-occurrence
information is usually not well correlated with sentiment predic-
tion [27], which motivates us to exploit large-scale rating data for
training deep sentiment classifiers.
Fig. 2. Percentages of wrong-labeled sentences by ratings in our labeled
review dataset. The overall percentage is 13.4%.
However, ratings are noisy labels for review sentences and
would mislead classifier training if directly used in supervised
training. In this paper, we adopt a simple rule to assign weak
labels to sentences with 5-stars rating scale:

pos, if s is in a 4 or 5-stars review
`(s) = , (1)
neg, if s is in a 1 or 2-stars review
where `(s) denotes the weak sentiment label of sentence s. Note
we follow previous works on aspect level sentiment analysis
[8], [19] to only consider positive and negative sentiment labels.
The reason is that when commenting on various aspects of a
product, people hardly express neutral opinions. Figure 2 shows
the percentages of wrong-labeled sentences by `(s), estimated in
our labeled review dataset (detailed description of the dataset is
in Section 4.1). We can see the noise level is moderate but not
ignorable.
The general idea behind WDE is that we use large quantities
of weakly labeled sentences to train a good embedding space so
that a linear classifier would suffice to accurately make sentiment
predictions. Here good embedding means in the space sentences
with the same sentiment labels are close to one another, while
those with different labels are kept away from each other. In the
following, we first present the network architecture and explain the
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
specific design choices for WDE-CNN and WDE-LSTM. Then
we discuss how to train it with large-scale rating data, followed by
supervised fine-tuning on labeled sentences.
Fig. 3. The network architecture in general for sentence sentiment
classification.
3.1 Network Architecture in General
The general architecture of the neural network designed for WDE
is shown in Figure 3. At the first layer, the network takes a review
sentence as input and extracts a fixed-length low-level feature
vector from the sentence. Unlike many traditional methods for
sentiment analysis, no feature engineering is required and the
extractor is learned automatically. Specific implementation of the
extractor will be discussed in the following for WDE-CNN and
WDE-LSTM respectively. The low-level feature vector is then
passed through a hidden layer, adding sufficient nonlinearity, and
the output is used to compute the embedding representation of the
sentence. The embedding representation also takes the sentence’s
aspect contextual information into consideration. An aspect is a
topic on which customers can comment with respect to a sort of
entities. For instance, battery life is an aspect for cell phones.
We use a learnable context vector to represent an aspect. The
motivation for incorporating aspect information as the context of a
sentence is that similar comments in different contexts could be of
opposite orientations, e.g. “the screen is big” vs. “the size is big”.
In the weakly-supervised training phase, the goal is to learn
an embedding space which can properly reflect data’s semantic
distribution. Hence, the network used in this phase contains only
the layers up to the embedding layer. The final classification layer
(drawn in dotted lines in Figure 3) is added in the following
supervised training phase, in order to learn the final prediction
model.
3.2 Network Architecture of WDE-CNN
The network architecture of WDE-CNN, depicted in Figure 4, is
a variant of the CNNs described in [5], [21]. In what follows, we
use upper case bold letters such as W to denote matrices and
lower case bold letters such as x to denote column vectors. The
i-th element in vector x is denoted by x(i).
Input Layer. An input sentence of length t is a word sequence
s =< w1 w2 . . . wT >. Each word w in the vocabulary is
described by a word vector x. Let k be the length of x and n be
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
4
Fig. 4. The network architecture for WDE-CNN.
the total number of words in the vocabulary. The trainable word
lookup table X is then a k × n matrix with word vectors as its
columns. The input layer simply maps s =< w1 w2 . . . wT > to
its corresponding word vector representation < x1 x2 . . . xT >.
The lookup table is initialized using the publicly available 300-
dimensional word vectors trained on 100 billion words from
Google News by word2vec [31]. Out-of-sample words are ini-
tialized randomly.
Convolutional Layer and Max pooling Layer. The convolutional
layer applies a set of filters on the sentence. Each filter w ∈ Rhk
is applied to a window of h words to produce a local feature value:
u(t) = f (wT xt:(t+h−1) + b), (2)
where xt:(t+h−1) represents the concatenated vector
[xTt xTt+1 . . . xTt+h−1 ]T , u(t) is the computed feature value at
position t, b is the bias of the current filter, and f (·) is a non-linear
activation function such as hyperbolic tangent. Computing u(t) at
all possible positions in s yields a (T −h+1)-dimensional feature
vector (i.e. a feature map) u = [u(1)u(2) . . . u(T − h + 1)]T .
Then the max pooling layer performs a max operation over
each feature map uj to find the most salient value of the filter’s
corresponding feature as its final value [5]
v(j) = max{uj (t)}. (3)
t
This pooling scheme keeps the most important indicator of a
feature and naturally leads to a fixed-length vector output v at
the max pooling layer.
A filter with window size h is intrinsically a feature extractor
which performs “feature selection” from the h-gram features of
a sentence. When the input h-gram matches its w, we will
obtain a high feature value, indicating this h-gram activates the
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017 5

feature. This resembles the traditional feature selection in senti-
ment classification [34], but is done automatically by the network.
Since traditional machine learning based methods often exploit
unigrams, bigrams and trigrams [34], we also employ filters with
different window sizes, i.e. h = 1, 2, 3.
Hidden Layer and Embedding Layer. The fixed-length feature
vector v is then fed to the fully connected hidden layer and
embedding layer to extract nonlinear higher level features. For
the hidden layer, the computation is straightforward with weigh
matrix Wh and bias vector bh :
h = f (Wh v + bh ).
The embedding layer gets its input from two sources: the output of
the hidden layer h, and context vector as of sentence s. Context
vectors of all aspects constitute the context lookup table A (as
columns). The embedding layer output is computed as
 
h
 
y = f We + be . (5)
as
Classification Layer. This layer is fully connected to the embed-
ding layer and outputs sentiment prediction for the input sentence.
model to explicitly capture long-term dependencies in sequential
data. Hence, we propose another instantiation of WDE by Long
short-term memory (LSTM) [18]. LSTM is a popular technique
for recurrent neural networks (RNNs). An RNN progressively
updates its hidden state given the current input and the previous
hidden state from the last time step, making it a natural choice for
modeling sequential data such as natural language sentences.
LSTM has recently attracted much attention due to its ability
to learn long-term dependencies by using a gating mechanism.
A LSTM maintains a structure called memory cell which can be
viewed as a continuous analogy of a memory circuit. The memory
(4) cell controls the read, write and reset operations of its internal
state through output, input and forget gates respectively, allowing
the gradient information to be back-propagated through many time
steps. The forward computation of a block of memory cells at time
t is as follows [16]:
dt =g(Wd xt + Ud zt−1 + bd ) (6)
it =σ(Wi xt + Ui zt−1 + bi ) (7)
ft =σ(Wf xt + Uf zt−1 + bf ) (8)
ot =σ(Wo xt + Uo zt−1 + bo ) (9)
It is not until the supervised fine-tuning phase that the layer will ct =it  dt + ft  ct−1 (10)
be added to the network. We defer the description of this layer to zt =ot  g(ct ) (11)
Section 3.5.
where {W? , U? , b? }?∈{d,i,f ,o} is the set of model parameters,
Classification Layer  denotes the element-wise product between vectors, and each
element in the vectors dt , it , ft and ot corresponds to the outcome
Embedding
Layer y value of one memory cell’s input unit, input gate, forget gate and
output gate, respectively, at time t. ct represents the internal state
Hidden of the memory cells and zt represents their outputs at time t. σ(·)
h
Layer
Context
is the sigmoid activation function. g(·) is the activation function
Max Pooling v Lookup of inputs and outputs (usually hyperbolic tangent). The above
Layer ......
max max max as Table A
definition is a variant of the vanilla LSTM [16]. We omit peephole
connections from internal states to gates since it has been reported
that removing them does not influence LSTM’s performance
¬
{z} ...... significantly [16]. Readers are referred to [16], [18] for a detailed
description of LSTM and discussion of its advantages.
® ¬ Taking LSTM as a building block, we design the network
concat( z , z )
architecture of WDE-LSTM as depicted in Figure 5. The input
layer is the same with that of WDE-CNN. We also use the 300-
® ...... dimensional word vectors trained with Google News to initialize
{z}
the word lookup table. The LSTM layer contains two LSTMs to
form a bidirectional RNN [15]. More formally, we have
¬ →
−z =LSTM (x , → −z )
LSTMb LSTMb ... LSTMb LSTMb z0 t f t t−1
LSTM ←
−
Layer ®
z0
z t =LSTMb (xt , ←−
z t+1 )
LSTMf LSTMf ... LSTMf LSTMf
→
− ←−
zt =concat( z t , z t ), (12)
where LSTMf /LSTMb represents forward/backward LSTM, and
Input Layer x ... →
−z t and ←−
z t are their output at time t respectively. zt is then
Word
constructed as a concatenation of →
−
z t and ←−
z t . Such a bidirectional
Lookup structure generates a more comprehensive feature representation
Table X of each word by encoding contextual information from the whole
Input Sentence s: w1 w2 ... w T -1 wT sentence. On the other hand, memory cells in LSTMs can be
viewed as feature extractors for detecting sequential patterns in
Fig. 5. The network architecture for WDE-LSTM. sentences. Therefore, we again perform max pooling over the
whole word sequence of a sentence to obtain the most salient
value of each extractor, forming the fixed-length feature vector v:
3.3 Network Architecture of WDE-LSTM
v(j) = max{zt (j)}. (13)
The convolutional filters in WDE-CNN can only capture input t
patterns from a text window, e.g., of 3 words. It is hard for a CNN The layers above max pooling layer is the same as in WDE-CNN.

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
3.4 Embedding Training with Ratings
With the weak label definition in Eq. (1), we can divide review
sentences into two sets: P = {s|`(s) = pos} and N =
{s|`(s) = neg}. Since P and N contain wrong-labeled sentences,
they cannot directly be used to train a classifier. Therefore, we
propose to first train an embedding space that captures the general
sentiment distribution of sentences. Intuitively, we should let sen-
tences in P /N stick together, while keeping P and N separated.
A straightforward training scheme could be adapted from [50]
by stochastic gradient descent (SGD): we sample sentence pairs
and reduce distances for same-label pairs and increase distances
for opposite-label pairs. However, when wrong-labeled sentences
are sampled, there is still a relatively high chance that we make
a wrong move. To alleviate this issue, we propose to penalize
relative distances for sentence triplets. The training objective is
defined as a ranking loss [5]
X the Embedding Layer (Figure 3). Hyperbolic tangent is employed
Lweak = max (0, λ − dst(s1 , s3 ) + dst(s1 , s2 )) ,
<s1 ,s2 ,s3 >
(14)
where λ is the margin parameter, dst(·) is the Euclidean distance
between sentences computed by their embedding layer represen-
tation:
dst(si , sj ) = kysi − ysj k2 , (15)
and < s1 , s2 , s3 > denotes a valid triplet with `(s1 ) = `(s2 ) 6=
`(s3 ). Eq. 14 means we require the distance between same-label
sentences s1 and s2 to be shorter than that between s1 and a
sentence s3 with the opposite label by at least λ. A sample triplet
is generated as follows. First, we randomly choose P or N as the
focus. Suppose we choose P . Then two sentences s1 and s2 are
sampled from P in turn, and a sentence s3 is sampled from N .
The case for N as the focus is just a mirror case.
Fig. 6. Comparison between (a) pair-based training and (b) triplet-based
training. Please see the text for detailed explanations.
Figure 6 illustrates the advantages of triplet-based training
over pair-based training via a toy example. We use circles and
triangles to represent sentences in P and N respectively. Black
nodes denote wrong-labeled sentences. Since the majority of
sentences are with correct labels, they would gather together in the
training process. Wrong-labeled sentences would go towards the
wrong clusters, but with slower speeds. In both training methods,
undesirable moves could happen when wrong-labeled sentences
are sampled. For clarity, we just show three such cases that are
representative for respective methods. The three cases in Fig-
ure 6(a) all result in undesirable moves: sentences with different
orientations become closer (cases 1 and 2), while same-orientation
sentences become more separated (case 3). In Figure 6(b), case
1 generates only undesirable moves: since s3 (black triangle) is
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
6
closer to s1 (white circle) than s2 (black circle), the algorithm will
drag s3 away from s1 and drag s2 toward s1 . Cases 2 and 3 lead
to a mixed behavior: one move is desirable while the other one
is not. Therefore, cases 2 and 3 in Figure 6(b) are not as harmful
as the cases in Figure 6(a). Furthermore, in triplet-based training
there will not be a move if the difference in distances exceeds the
margin λ, since the derivative of Lweak becomes 0. This is useful
in that we will not make things too bad. For example, in case 2 of
Figure 6(b) s2 is actually a negative sentence and should not be
too close to s1 . Notice s3 is far away from s1 . Hence, the distance
difference may already exceed λ and there will be no move for
this triplet. As a comparison, cases 1 and 2 in Figure 6(a) will
continually move s1 and s2 toward each other until their distance
becomes 0, which is the worst result.
Training details. Embedding training is done by taking the
derivative of Lweak in (14) with respect to all the parameters under
as the activation function for all the layers. The detailed derivation
of the back-propagation procedure for WDE can be found in the
appendix. We do SGD over sampled sentence triplets with Ada-
Grad update rule [10]. Each mini-batch consists of 1024 sentence
triplets. To cope with the overfitting problem, we do early stopping
in the training process according to the network’s performance on
the validation set. We devise a measure for validation. Intuitively,
a good embedding space will keep intra-class instances near each
other and separate instances from different classes, providing a
proper prior model for supervised training. Hence, we propose
to assess the intra-class affinity and inter-class separability of the
embedding on the validation set as an indicator of the model’s
generalization ability. Specifically, the measure is defined as
1 P
|Ψ| (si ,sj )∈Ψ dst(si , sj )
Dst ratio = 1 P , (16)
|Φ| (si ,sj )∈Φ dst(si , sj )
where Ψ and Φ are sets of inter-class sentence pairs and intra-class
sentence pairs in the validation set respectively. It is essentially
the ratio between inter-class average distance and intra-class
average distance. A better model would lead to a higher value
of Dst ratio. In the training process, the learning rate  is first
set to 0.1. After every 1M triplets are processed, Dst ratio is
evaluated. If it deteriorates apparently, we start to adjust  as
 = 0.5 × . The training stops if either  is below a preset
threshold (e.g. 10−4 ) or Dst ratio can no longer be increased
significantly. The hyperparameters are tuned on the validation
set. We find the performance becomes relatively stable when the
hyperparameters are large enough. Considering both effectiveness
and efficiency, for WDE-CNN and WDE-LSTM we set context
vector size to 50 and set both hidden layer size and embedding
layer size to 300. For WDE-CNN, the number of filters for each
window size is 200 (a total of 600 filters). For WDE-LSTM, both
forward LSTM and backward LSTM contain 300 memory cells (a
total of 600, matching the number of filters in WDE-CNN for fair
comparison). The training is accelerated using GPU.
CNN vs. LSTM As discussed before, WDE-LSTM can naturally
capture long-term dependencies while WDE-CNN is limited by
the window sizes of convolutional filters. Nevertheless, WDE-
CNN is more efficient than WDE-LSTM. With the above hy-
perparameter setting, WDE-CNN needs about half an hour for
processing 1M triplets on a Nvidia GTX 980ti GPU, while for
WDE-LSTM the corresponding time is about 5 hours. The low
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
efficiency of WDE-LSTM stems from two facts: (1) there are
much more parameters in LSTM layer than in CNN layer; (2)
when computing derivatives for a LSTM cell’s weights, each
word position must be considered, while for a convolutional filter,
we only need to care about the word position which wins the
max operation. Moreover, the parallelization can only be done
sentence-wise in WDE-LSTM, while for WDE-CNN we can do
word-level parallelization.
3.5 Supervised Fine-tuning
After obtaining a good enough sentence representation by the
embedding layer, we add a classification layer on the top (Figure 4)
to further train the network using labeled sentences. The classifi-
cation layer simply performs standard affine transformation of the
embedding layer output y and then applies a softmax activation
function [3] to the result for label prediction. In this work, we
focus on binary sentiment prediction (i.e. positive or negative)
since we only consider sentences which comment on specific
aspects of an entity. This kind of sentences hardly contain neutral
sentences. Nevertheless, WDE could also be adapted to multi-class
prediction problems. For binary prediction, the classification layer
is equivalent to a logistic regression model. We train the network
using standard SGD, since AdaGrad can easily “forget” the prior
model learned in the first phase. The mini-batch size is set to 64.
A similar early stopping strategy is adopted as in the embedding
training phase (except that we employ prediction accuracy as the
validation measure).
4 E XPERIMENTS
In this section, we present the empirical evaluation of WDE on
reviews collected from Amazon.com.
TABLE 1
Statistics of the labeled dataset.
Positive Negative Total
Subjective 3750 2024 5774
Objective 1860 4120 5980
Total 5610 6144 11754
4.1 Data and Preprocessing
We collected Amazon customer reviews of 3 domains: digital
cameras, cell phones and laptops. All unlabeled reviews were
extracted from the Amazon data product dataset [29]. In particular,
we extracted all the reviews from 12 leaf categories closely related
to the above three domains (3-stars reviews were ignored). For
the labeled dataset, we crawled latest reviews in 2015 for random
products in the above 12 categories, in order to be disjoint with the
unlabeled data. We tried to keep a balance between reviews with 4
& 5-stars and those with 1 & 2-stars. We then summarized product
aspects and their keywords by the popular method in [8] with
manual calibration1 . A total of 107 aspects were extracted from the
obtained reviews. Next, all reviews were split into sentences and
those with no aspect keywords were discarded. In case a sentence
mentioned multiple aspects in different clauses, we split it to
1. Although more recent methods such as [4] could be used for aspect
extraction, we choose [8] for simplicity. Comparison between different aspect
extraction methods is out of scope of this paper.
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
7
treat each clause as a single-aspect sentence2 . After preprocessing,
we obtained a vocabulary of 148,183 terms, an unlabeled set of
1,143,721 sentences with rating information only (named 1.1M
dataset), and 11,754 sentences for labeling. We labeled each
sentence with respect to its subjectivity and orientation. For each
labeling task, three students were instructed to perform labeling,
and each sentence was labeled by all the three students. In case
the labeling results are inconsistent, they discussed to reach a
consensus. The statistics of the labeled dataset is shown in Table 1.
We can see the dataset is roughly balanced. We used Fleiss’s kappa
[13] to measure the degree of agreement between the raters. For
the subjectivity and orientation labeling tasks, we achieved Fleiss’s
kappa values of 0.81 and 0.79 respectively. The labeled dataset
was randomly split into training set (50%), validation set (20%)
and test set (30%) and we maintain the proportion as shown in
Table 1.
4.2 Baselines and Evaluation Settings
The main research question we want to answer is, whether the
proposed weakly-supervised training method for deep networks
can help boost performance. We compare WDE-CNN and WDE-
LSTM3 with the following baseline methods for review sentence
sentiment classification:
Lexicon: this is the popular lexicon-based method proposed in [8].
SVM: the support vector machine with n-gram features [35] is
widely employed as a baseline for sentiment classification. We
use up to tri-grams since this setting is shown to yield good
performance for product reviews. Liblinear [11] is used to train
the classifier.
NBSVM: NBSVM combines Naive Bayes and NB-enhanced
SVM to predict sentiment labels [48]. It generates good perfor-
mance on many sentiment classification datasets.
SSWE: SSWE learns sentiment-bearing word vectors by a neural
network applied on weakly labeled data. We use min, max and avg
pooling [46] on word vectors to generate the sentence representa-
tion which is then fed to a classifier.
SentiWV: this is the sentiment word vector learning method on
rating data described in Section 2 [27]. We also use the afore-
mentioned three pooling functions to generate sentence vectors.
Liblinear is used to train the classifiers for SentiWV and SSWE.
MemNet: MemNet is the deep memory network proposed in [45]
for aspect level sentiment classification. The method is naturally
defined only for single-word aspects. For multi-word aspects, we
follow the suggestion of averaging the constituting word vectors
[45]. For implicit aspects, the word that is the most relevant to the
aspect according to word vectors is treated as the aspect word.
CNN-rand & CNN-rand11m: we train the same CNN based
network (Figure 4) on labeled data with random parameter ini-
tialization. For the sake of fairness, we also initialize the word
lookup table by word vectors trained by word2vec [31]. Besides
the publicly available word vectors trained on Google News,
we also employ word vectors trained on the 1.1M dataset. The
corresponding baselines are named CNN-rand and CNN-rand11m,
respectively.
CNN-weak: we train the same CNN based network on 1.1M
dataset by treating weak labels defined in Eq. (1) as real labels.
2. This is also to be fair to the classic machine learning baseline methods
which do not take aspect information as input.
3. The source code of WDE can be downloaded at
https://www.dropbox.com/s/yvo4x4c6dqtx3nm/code.zip.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017 8

TABLE 2
Performance comparison.
0.85
Accuracy Macro-F1
Method
All Subj Obj All Subj Obj

Accuracy
Lexicon .722 .827 .621 .721 .812 .613 0.8
SVM .818 .838 .800 .818 .821 .765 WDE-CNN
WDE-LSTM
NBSVM .826 .844 .808 .825 .831 .773 CNN-rand
SSWE .835 .857 .815 .834 .826 .804 0.75 MemNet
Sentiwv
SentiWV .808 .806 .809 .807 .786 .771 SSWE
MemNet .839 .841 .837 .838 .829 .803 SVM
NBSVM
CNN-rand .847 .861 .835 .847 .848 .802
0.7
CNN-rand11m .849 .861 .836 .848 .850 .804 0 20 40 60 80 100
CNN-weak .771 .773 .770 .771 .755 .741 Percentage of training data (%)
LSTM-rand .845 .863 .829 .845 .852 .794
(a)
LSTM-rand11m .850 .861 .839 .849 .853 .809
WDE-CNN .877 .886 .868 .876 .875 .843
WDE-LSTM .879 .889 .870 .879 .878 .844
0.85

This baseline will answer whether rating data can be used to train
sentence sentiment classifiers directly4 .
LSTM-rand & LSTM-rand11m: The setting of these baselines
is the same as that of CNN-rand and CNN-rand11m except that
we employ the LSTM based network (Figure 5).
All methods (except CNN-weak and Lexicon) are trained on
the training set and evaluated on the test set. WDE-CNN, WDE-
LSTM, SSWE and Senti-WV have a pre-training phase on the
1.1M dataset. The validation set is used for parameter tuning of
all the methods and early stopping of neural network training. We
employ Accuracy and Macro-F1 as the evaluation metrics.
4.3 Performance Comparison
The results are shown in Table 2. We also report performance for
subjective sentences and objective sentences separately. The key
observations are as follows. Lexicon performs poorly on objective
sentences, since factual statements would not contain opinion
words. When no opinion word is detected, we can only make
random predictions. The machine learning methods all achieve ac-
ceptable performance, on both subjective and objective sentences.
One exception is CNN-weak, which is trained on weakly labeled
sentences. We find its validation performance fluctuates drastically
during training. This indicates directly binarizing ratings as labels
for supervised training is not a good idea. SSWE performs better
than traditional classifiers (SVM and NBSVM) by applying a
neural model on 1.1M dataset. However, it just uses word vectors
to encode the useful information in the 1.1M dataset. The classifier
is still a “shallow” linear model. The two neural network methods,
CNN-rand and LSTM-rand, generate comparable results, with
LSTM-rand exhibiting a slightly lower overall performance. The
reason could be that the LSTM based model has more parameters
and consequently needs more training data. The performance
of CNN-rand11m and LSTM-rand11m is marginally better than
CNN-rand and LSTM-rand. The reason could be that word vectors
trained on the 1.1M dataset are “customized” for reviews. These
four baselines cannot beat WDE since (1) word2vec does not
explicitly capture sentiment information (as discussed in [27], co-
occurrence information is not enough); (2) WDE pre-trains the
whole network (except the last prediction layer) while word2vec
only pre-trains input word vectors. MemNet is slightly worse
than CNN-rand and LSTM-rand. This could be because Mem-
Net cannot naturally handle multi-word and implicit aspects. In
4. The result is similar by using the LSTM based network.
Macro-F1
0.8
WDE-CNN
WDE-LSTM
CNN-rand
0.75 MemNet
Sentiwv
SSWE
SVM
NBSVM
0.7
0 20 40 60 80 100
Percentage of training data (%)
(b)
Fig. 7. Impact of labeled training data size on each method’s perfor-
mance.
WDE, aspects are modeled by separate semantic vectors, thus
avoiding the above issues. Finally, the WDE framework encodes
both a large number of weak supervision signals and supervision
signals in a deep neural network and beats all the baselines. By
incorporating a large set of weakly labeled training data, WDE-
LSTM performs slightly better than WDE-CNN. In Section 4.6,
we will demonstrate by case studies that WDE-LSTM can handle
sentences with more complicated structures.
4.4 Varying the Size of Training Set
Next we examine the impact of the size of labeled training data
on each method’s performance. CNN-weak and Lexicon are not
involved since they do not depend on labeled training data. We
randomly select d% training data to train the classifiers and test
them on the test set, with d ranging from 10 to 90. For each d, we
generate the training set 30 times and the averaged performance
is reported. Figure 7 shows the results. The curve of LSTM-
rand is slightly lower than that of CNN-rand, and the margin
increases a bit as the training set becomes smaller5 . This is in
conformity with the observation in the previous subsection. The
reason could be that LSTM based model requires more training
data. Secondly, we can see that as the number of available training
instances decreases, the performance of CNN-rand, NBSVM and
SVM drops faster than that of WDE-CNN, WDE-LSTM, SSWE
and SentiWV. This should be because the latter methods have
gained prior knowledge about the sentiment distribution through
pre-training, though with different capabilities. With 10% training
5. Since the curves of CNN-rand11m, LSTM-rand and LSTM-rand11m
overlap with that of CNN-rand heavily, we omit them for clarity.

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017 9

set (nearly 600 instances), WDE-CNN and WDE-LSTM can still

achieve around 80% accuracy on the test set. According to t-test,
WDE-CNN and WDE-LSTM significantly outperform the other
methods with p-value < 0.01. WDE-LSTM performs slight better
than WDE-CNN when the labeled training set is sufficiently large.

0.89 1.25

Inter-/Intra-class Avg. Dist. Ratio

Classification Accuracy

0.88 1.2

0.87 1.15

0.86 1.1

0.85 1.05
Accuracy of final prediction model
Inter-/intra-class average distance ratio
0.84 1
0 20 40 60 80 100
No. of triplets processed (m)

Fig. 8. Investigating the relationship between Dst ratio (Eq. (16)) and
the accuracy of the final prediction model: for every 1M triplets pro- Fig. 9. Visualization of the embedding space by t-SNE. The upper
cessed in the embedding training phase, we save the model and feed it subfigure: after random initialization; the lower subfigure: after training
to the supervised training phase; then we plot both Dst ratio and the on 10M triplets.
final classification accuracy.

TABLE 3
Example sentences for which WDE-LSTM makes correct predictions
4.5 Performance Evaluation for Embedding Training while WDE-CNN fails.
While traditional neural network training algorithms often use the
prediction performance of the model on validation set to assess its Sentence in test set Label
Battery capacity is not the greatest, but it is ok. Pos
generalization ability, it is difficult to apply this strategy in the first
The internet drops randomly while the yoga in the same room Neg
phase of WDE since the target model is not a prediction model. is absolutely fine.
Hence, in section 3.4 we devised a measure called Dst ratio None of these cameras has an articulating front view screen. Neg
(Eq. (16)) for the first phase of WDE which could be able to reflect
the model’s generalization ability. Here we investigate whether
Dst ratio can reflect the generalization performance of the final set in detail and find that: (1) WDE-CNN is better at handling
prediction model. We save the learned model in the embedding
sentences with simple structures, e.g. “Sound is not that good.”
training phase after every 1M triplets are processed (the early
(2) for more complicatedly structured sentences, WDE-LSTM can
stopping strategy is not used). Then the saved models are fed to the
do better. Table 3 shows some examples for which WDE-LSTM
supervised fine-tuning phase to get the classification performance.
makes correct predictions while WDE-CNN fails. The second
The two curves of Dst ratio and classification accuracy are
column shows their true labels. Regarding the first two sentences,
shown in Figure 8. The results of WDE-CNN and WDE-LSTM
there is an attitude transition between the two clauses. For WDE-
are very similar, so we just show those of WDE-CNN. Figure 8
CNN, conflicting local patterns (e.g. “not the greatest” and “is ok”)
indicates that Dst ratio can to some extent forecast the final
could lead to wrong predictions. In the last sentence, the negation
prediction model’s performance. Therefore, Dst ratio could be
word “none” is far from the words conveying the user’s opinion,
used as the validation measure for embedding training.
so it is hard for WDE-CNN to capture the dependence with limited
To get an intuitive understanding of the learned embedding
window sizes. By modeling sentences holistically using LSTMs,
space, we visualize it by t-SNE [28]. We sample 2000 sentences
WDE-LSTM is able to predict correctly for these sentences.
from the 1.1M dataset to manually label them and plot them
as projected by t-SNE. The visualization results are shown in We also compare WDE-CNN (WDE-LSTM) with its coun-
Fig. 9. We only report the results of WDE-CNN since those of terparts, CNN-rand and CNN-rand11m (LSTM-rand and LSTM-
WDE-LSTM are similar. The upper subfigure shows the status rand11m). Table 4 shows three example sentences for which
of the embedding space right after random initialization of the WDE-CNN generates correct predictions while its counterparts
neural network. The lower one shows its status after training give wrong predictions. Table 5 is for WDE-LSTM. The second
on 10M triplets. We also show the Dst ratio scores on top of column of the tables presents the most similar sentences of the
the subfigures. From Fig. 9 we see that the general sentiment corresponding example sentences in the 1.1M dataset. We use
distribution of sentences can be captured by the embedding space. Latent Semantic Indexing [7] to assess sentence similarity. Most
of these example sentences are objective ones. We find objective
sentences are more likely to be diverse in expression (without
4.6 Case studies popular opinion words), so it is harder to make correct predictions
As stated in Section 3.3, the LSTM layer is able to capture for them. With pre-training on the 1.1M dataset WDE-CNN and
long-term dependencies. Compared to WDE-CNN, WDE-LSTM WDE-LSTM have a much higher chance to be trained on similar
could be more capable of “perceiving” a sentence holistically. We expressions, as shown in Tables 4 and 5. This could explain their
examine the classification results of the two methods on the test superior performance on those sentences.

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017 10

0.89
TABLE 4 WDE-CNN
Example sentences for which WDE-CNN makes correct predictions WDE-LSTM
CNN-rand
while its counterparts, CNN-rand and CNN-rand11m, fail to give correct 0.88
answers.
0.87

Accuracy
Sentence in test set The most similar sentence in
1.1M dataset 0.86
I was using a motorola t720i with It dropped all calls every two min-
verizon and I had a lot of dropped utes.
calls and I had troubles syncing 0.85

data.
This thing was amazing for about The phone has an issue that causes 0.84
0 5 10 15 20 25 30
20 minutes, then I started charging it to reboot at random or simply λ
it and it bricked out, started re- freeze up.
booting randomly and then stopped
turning on altogether. Fig. 10. Impact of λ on classification performance.
No future updates for this device There will (obviously) be no up-
huawei refused to release any up- dates to push this phone any fur-
dates for this device. ther and I don’t think it would be 5 C ONCLUSION
necessary to do so.
In this work we proposed a novel deep learning framework
named Weakly-supervised Deep Embedding for review sentence
TABLE 5 sentiment classification. WDE trains deep neural networks by
Example sentences for which WDE-LSTM makes correct predictions exploiting rating information of reviews which is prevalently
while its counterparts, LSTM-rand and LSTM-rand11m, fail to give available on many merchant/review Websites. The training is a
correct answers.
2-step procedure: first we learn an embedding space which tries
Sentence in test set The most similar sentence in to capture the sentiment distribution of sentences by penalizing
1.1M dataset relative distances among sentences according to weak labels
It leaks it leaks it leaks. Basically, at an angle, you can see inferred from ratings; then a softmax classifier is added on top
a horizontal light leaking at the top
of the embedding layer and we fine-tune the network by labeled
edge of the screen.
Bluetooth does not work on this The Bluetooth does not work half data. Experiments on reviews collected from Amazon.com show
phone. the time. that WDE is effective and outperforms baseline methods.
You can shoot in very low light Low light photos come out with Two specific instantiations of the framework, WDE-CNN and
without dark backgrounds looking color intact (no flash) - ideal for
grainy. museum items.
WDE-LSTM, are proposed. Compared to WDE-LSTM, WDE-
CNN has fewer model parameters, and its computation is more
easily parallelized on GPUs. Nevertheless, WDE-CNN cannot
well handle long-term dependencies in sentences. WDE-LSTM
4.7 Effect of λ in WDE is more capable of modeling the long-term dependencies in
sentences, but it is less efficient than WDE-CNN and needs more
The margin parameter λ in Eq. (14) controls the extent to which training data. For future work, we plan to investigate how to com-
we require weakly labeled positive instances to be separated bine different methods to generate better prediction performance.
from weakly labeled negative ones. A small value of λ may not We will also try to apply WDE on other problems involving weak
effectively capture the sentiment distribution while too large λ labels.
could amplify the impact of wrong-labeled sentences. Take the
case 2 in Figure 6(b) as an illustrative example. First, let us
ACKNOWLEDGMENT
assume s2 in case 2 is white (i.e. correctly labeled). In that case,
a small λ may result in no move since the margin requirement This research was supported by the National Natural Sci-
is already satisfied. Consequently, the positive class will not be ence Foundation of China (Grant Nos. 61672409, 61522206,
compact since s2 is still far from the majority of the class. When 61373118), the Major Basic Research Project of Shaanxi Province
s2 represents a wrong-labeled sentence (as shown in Figure 6(b)), (Grant No. 2017ZDJC-31) and the Science and Technology Plan
large margin values tend to force s2 to move toward s1 until they Program in Shaanxi Province of China (Grant No. 2017KJXX-
are very close to each other, making the classifier training hard. 80). The content of the information does not necessarily reflect
the position or the policy of the Government, and no official
Here we investigate λ’s impact on the classification perfor-
endorsement should be inferred.
mance. Recall that the embedding layer is a 300-dimensional
vector, and the output range of its neural nodes is [−1, 1]. It forms
a hypercube where the maximal √ distance between any two points R EFERENCES
in the hypercube is dia = 1200 ≈ 35. Hence, we vary λ [1] Y. Bengio. Learning deep architectures for ai. Foundations and trends
R
from 1 to 30. Figure 10 plots the accuracy curves of WDE-CNN in Machine Learning, 2(1):1–127, 2009.
and WDE-LSTM. We also show the best baseline performance [2] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A
review and new perspectives. IEEE TPAMI, 35(8):1798–1828, 2013.
achieved by CNN-rand for comparison. We find the performance [3] C. M. Bishop. Pattern recognition and machine learning. springer, 2006.
drops quickly when λ > 15, and when λ < 15 we can easily find [4] L. Chen, J. Martineau, D. Cheng, and A. Sheth. Clustering for simulta-
a value leading to good performance. Moreover, when λ is set to neous extraction of aspects and features from reviews. In NAACL-HLT,
a relatively high value (> 0.5dia), the network is more easily to pages 789–799, 2016.
[5] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and
be trapped in saturating regions [2] after long time training. In this P. Kuksa. Natural language processing (almost) from scratch. JMLR,
paper we set λ = 5. 12:2493–2537, 2011.

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
Transactions on Knowledge and Data Engineering
IEEE Transactions on Knowledge and Data Engineering ( Volume: 30, Issue: 1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017
[6] K. Dave, S. Lawrence, and D. M. Pennock. Mining the peanut gallery:
Opinion extraction and semantic classification of product reviews. In
WWW, pages 519–528, 2003.
[7] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and
R. Harshman. Indexing by latent semantic analysis. Journal of the
American society for information science, 41(6):391, 1990.
[8] X. Ding, B. Liu, and P. S. Yu. A holistic lexicon-based approach to
opinion mining. In WSDM, pages 231–240, 2008.
[9] L. Dong, F. Wei, C. Tan, D. Tang, M. Zhou, and K. Xu. Adaptive recur-
sive neural network for target-dependent twitter sentiment classification.
In ACL, pages 49–54, 2014.
[10] J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for
online learning and stochastic optimization. JMLR, 12:2121–2159, 2011.
[11] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin.
Liblinear: A library for large linear classification. JMLR, 9:1871–1874,
2008.
[12] R. Feldman. Techniques and applications for sentiment analysis. Com-
munications of the ACM, 56(4):82–89, 2013.
[13] J. L. Fleiss. Measuring nominal scale agreement among many raters.
Psychological bulletin, 76(5):378, 1971.
[14] X. Glorot, A. Bordes, and Y. Bengio. Domain adaptation for large-scale
sentiment classification: A deep learning approach. In ICML, pages 513–
520, 2011.
[15] A. Graves and J. Schmidhuber. Framewise phoneme classification
with bidirectional lstm and other neural network architectures. Neural
Networks, 18(5):602–610, 2005.
[16] K. Greff, R. K. Srivastava, J. Koutnı́k, B. R. Steunebrink, and J. Schmid-
huber. Lstm: A search space odyssey. arXiv preprint arXiv:1503.04069,
2015.
[17] H. Halpin, V. Robu, and H. Shepherd. The complex dynamics of
collaborative tagging. In WWW, pages 211–220, 2007.
[18] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural
computation, 9(8):1735–1780, 1997.
[19] M. Hu and B. Liu. Mining and summarizing customer reviews. In
SIGKDD, pages 168–177, 2004.
[20] N. Kalchbrenner, E. Grefenstette, and P. Blunsom. A convolutional neural
network for modelling sentences. In ACL, 2014.
[21] Y. Kim. Convolutional neural networks for sentence classification. In
EMNLP, pages 1746–1751, 2014.
[22] R. Kiros, Y. Zhu, R. R. Salakhutdinov, R. Zemel, R. Urtasun, A. Torralba,
and S. Fidler. Skip-thought vectors. In NIPS, pages 3294–3302, 2015.
[23] H. Lakkaraju, R. Socher, and C. Manning. Aspect specific sentiment
analysis using hierarchical deep learning. In NIPS Workshop on Deep
Learning and Representation Learning, 2014.
[24] Q. V. Le and T. Mikolov. Distributed representations of sentences and
documents. In ICML, volume 14, pages 1188–1196, 2014.
[25] B. Liu. Sentiment analysis and opinion mining. Morgan & Claypool
Publishers, 2012.
[26] B. Liu, M. Hu, and J. Cheng. Opinion observer: analyzing and comparing
opinions on the web. In WWW, pages 342–351, 2005.
[27] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts.
Learning word vectors for sentiment analysis. In ACL, pages 142–150,
2011.
[28] L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. JMLR,
9(Nov):2579–2605, 2008.
[29] J. McAuley, R. Pandey, and J. Leskovec. Inferring networks of substi-
tutable and complementary products. In SIGKDD, pages 785–794, 2015.
[30] T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of
word representations in vector space. arXiv preprint arXiv:1301.3781,
2013.
[31] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed
representations of words and phrases and their compositionality. In NIPS,
pages 3111–3119, 2013.
[32] T. Mullen and N. Collier. Sentiment analysis using support vector
machines with diverse information sources. In EMNLP, volume 4, pages
412–418, 2004.
[33] T. H. Nguyen and K. Shirai. Phrasernn: Phrase recursive neural network
for aspect-based sentiment analysis. In EMNLP, pages 2509–2514, 2015.
[34] B. Pang and L. Lee. Opinion mining and sentiment analysis. Foundations
and trends in information retrieval, 2(1-2):1–135, 2008.
[35] B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: sentiment classi-
fication using machine learning techniques. In EMNLP, pages 79–86,
2002.
[36] X. Qiu and X. Huang. Convolutional neural tensor network architecture
for community-based question answering. In IJCAI, pages 1305–1311,
2015.
1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
11
[37] L. Qu, R. Gemulla, and G. Weikum. A weakly supervised model for
sentence-level semantic orientation analysis with multiple experts. In
EMNLP-CoNLL, pages 149–159, 2012.
[38] Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil. A latent semantic model
with convolutional-pooling structure for information retrieval. In CIKM,
pages 101–110, 2014.
[39] R. Socher, B. Huval, C. D. Manning, and A. Y. Ng. Semantic compo-
sitionality through recursive matrix-vector spaces. In EMNLP-CoNLL,
pages 1201–1211, 2012.
[40] R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning.
Semi-supervised recursive autoencoders for predicting sentiment distri-
butions. In EMNLP, pages 151–161, 2011.
[41] R. Socher, A. Perelygin, J. Y. Wu, J. Chuang, C. D. Manning, A. Y. Ng,
and C. Potts. Recursive deep models for semantic compositionality over
a sentiment treebank. In EMNLP, volume 1631, page 1642, 2013.
[42] O. Täckström and R. McDonald. Semi-supervised latent variable models
for sentence-level sentiment analysis. In ACL, pages 569–574, 2011.
[43] D. Tang, B. Qin, X. Feng, and T. Liu. Target-dependent sentiment
classification with long short term memory. CoRR, abs/1512.01100,
2015.
[44] D. Tang, B. Qin, and T. Liu. Deep learning for sentiment analysis:
successful approaches and future challenges. Wiley Interdisciplinary
Reviews: Data Mining and Knowledge Discovery, 5(6):292–303, 2015.
[45] D. Tang, B. Qin, and T. Liu. Aspect level sentiment classification with
deep memory network. arXiv preprint arXiv:1605.08900, 2016.
[46] D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin. Learning
sentiment-specific word embedding for twitter sentiment classification.
In ACL, volume 1, pages 1555–1565, 2014.
[47] P. D. Turney. Thumbs up or thumbs down?: semantic orientation applied
to unsupervised classification of reviews. In ACL, pages 417–424, 2002.
[48] S. Wang and C. D. Manning. Baselines and bigrams: Simple, good
sentiment and topic classification. In ACL, pages 90–94, 2012.
[49] Y. Wang, M. Huang, X. Zhu, and L. Zhao. Attention-based lstm for
aspect-level sentiment classification. In EMNLP, pages 606–615, 2016.
[50] J. Weston, F. Ratle, and R. Collobert. Deep learning via semi-supervised
embedding. In ICML, pages 1168–1175, 2008.
[51] J. Wieting, M. Bansal, K. Gimpel, and K. Livescu. Towards universal
paraphrastic sentence embeddings. arXiv preprint arXiv:1511.08198,
2015.
[52] L. Zhang and B. Liu. Identifying noun product features that imply
opinions. In ACL, pages 575–580, 2011.
[53] X. Zhang, J. Zhao, and Y. LeCun. Character-level convolutional networks
for text classification. In NIPS, pages 649–657, 2015.
[54] J. Zhu, H. Wang, M. Zhu, B. K. Tsou, and M. Ma. Aspect-based
opinion polling from customer reviews. IEEE Transactions on Affective
Computing, 2(1):37–49, 2011.
Wei Zhao received the B.S., M.S. and Ph.D.
degrees from Xidian University, Xi’an, China, in
2002, 2005 and 2015, respectively. He is cur-
rently an associated professor in the School
of Computer Science and Technology at Xid-
ian University. His research direction is pattern
recognition and intelligent systems, with specific
interests in attributed graph mining and search,
machine learning, signal processing and preci-
sion guiding technology.
Ziyu Guan received the B.S. and Ph.D. degrees
in Computer Science from Zhejiang University,
China, in 2004 and 2010, respectively. He had
worked as a research scientist in the Univer-
sity of California at Santa Barbara from 2010
to 2012. He is currently a full professor in the
School of Information and Technology of North-
west University, China. His research interests
include attributed graph mining and search, ma-
chine learning, expertise modeling and retrieval,
and recommender systems.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TKDE.2017.2756658, IEEE
IEEE Transactions on Knowledge and Data Engineering Transactions on Knowledge
( Volume: and30,
DataIssue:
Engineering
1, Jan. 1 2018 )
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. ?, NO. ?, AUGUST 2017 12

Long Chen Long Chen received the B.S. degree

in Electronic Information Engineering from Xi’an
Jiaotong University City College, Xi’an, China,
in 2012 and received the M.S. degree in Elec-
tronics and Communications Engineering from
Northwest University, Xi’an, China, in 2015. He
is currently working toward the Ph.D. degree with
the School of Information Science and Technol-
ogy, Northwest University , Xi’an, China. His re-
search interests include deep learning,sentiment
analysis,text mining and natural language pro-
cessing.

Xiaofei He received the BS degree in Computer

Science from Zhejiang University, China, in 2000
and the Ph.D. degree in Computer Science from
the University of Chicago, in 2005. He is a Pro-
fessor in the State Key Lab of CAD&CG at Zhe-
jiang University, China. Prior to joining Zhejiang
University, he was a Research Scientist at Ya-
hoo! Research Labs, Burbank, CA. His research
interests include machine learning, information
retrieval, and computer vision.

Deng Cai is an Associate Professor in the State

Key Lab of CAD&CG, College of Computer
Science at Zhejiang University, China. He re-
ceived the Ph.D. degree in computer science
from University of Illinois at Urbana Champaign
in 2009. Before that, he received his B.S. degree
and M.S. degree from Tsinghua University in
2000 and 2003 respectively, both in automation.
His research interests include machine learning,
data mining and information retrieval. He is a
member of the IEEE.

Beidou Wang received his B.S. degree from

Zhejiang University, China, in 2011. He is cur-
rently in the duo Ph.D. program of Zhejiang
University, China and Simon Fraser University,
Canada. His research interests include social
network mining, active learning, cross domain
recommendation, recommendation explanation
and recommendation in social networks.

Quan Wang was born in 1970. He received

the B.S., M.S. and Ph.D. degrees in Computer
Science and Technology from Xidian University,
Xi’an, China. He is now a full professor in Xidian
University. His current research interests include
input and output technologies and systems, im-
age processing and image understanding.

1041-4347 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.