Expert Systems With Applications 241 (2024) 122512

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Product feature sentiment analysis based on GRU-CAP considering Chinese

sarcasm recognition
Zeng Wang a, 1, Shi-jie Hu b, 2, Wei-dong Liu b, *, 3
a
Architecture and Design College, Nanchang University, Nanchang, 330031, China
b
Advanced Manufacturing School, Nanchang University, Nanchang 330031, China

A R T I C L E I N F O A B S T R A C T

Keywords: Sentiment analysis based on product online reviews is an important measure in grasping customer’s satisfaction
Product online reviews with products, which has the problem of inadequate feature extraction and difficulty in identifying the sentiment
Sarcasm detection polarity of texts with irony. To solve the problem, this paper proposes a novel method of product feature
Gate recurrent units
sentiment analysis based on gate recurrent units and capsule neural network (GRU-CAP). Firstly, product online
Capsule neural network
Product feature extraction
reviews are preprocessed to build a review data set. The feature extraction model of GRU-CAP is then used to
Sentiment analysis extract the product features contained in the data set and generate the corresponding product feature index. After
combining the feature index with the data set to form new input data, the recognition of positive/negative
emotions, sarcastic/non-sarcastic semantics is carried out through the sentiment analysis model of GRU-CAP.
Finally, the recognition results are integrated to achieve fine-grained product feature sentiment analysis. The
effectiveness and practicability of the method is validated by an example product-mobile phone on an e-com­
merce platform. The results show that the method can extract product features and identify sentiment polarity
from product online review texts with irony more accurately, so as to provide better product design support for
enhancing customer satisfaction with products more effectively.

1. Introduction Product feature sentiment analysis mainly involves two core

With the increasingly fierce market competition, enterprises need to
quickly and accurately grasp consumer demands and highlight them in
product design to stand out in the industry (Park et al., 2018). In recent
years, due to the rapid development of e-commerce and social media
technology, vast majority of customers choose to purchase products and
post product online reviews on various e-commerce platforms to express
their views and concerns. Compared with traditional questionnaire data,
product online review data has the benefits of large volume, low
acquisition costs and active sharing of consumers rather than passive
Q&A, which can more truthfully reflect consumer satisfaction and
emotional demand for related products (Filieri et al., 2021; Li et al.,
2013; Zeng et al., 2014). Therefore, accurate sentiment analysis of
customers’ product online reviews can not only improve the efficiency
of product design, but also strengthen customer orientation and enhance
the competitiveness of products in the market.
research contents. One is to extract product features from online reviews
using related techniques of keyword extraction. The other is sentiment
analysis based on extracted product features. At present, many scholars
have conducted in-depth research on these contents.
Although the existing studies have made some achievements in the
sentiment analysis of product online reviews, there are still two
shortcomings:
On the one hand, when extracting product features from product
online reviews, existing studies tend to focus more on extracting explicit
product features rather than taking into account the effect of extracting
implicit product features. Explicit product features are features that are
explicitly expressed in a review, usually as nouns or noun phrases. Im­
plicit product features refer to features that are implied or not clearly
expressed in a review. Such features do not appear as specific nouns, but
usually appear as adjectives or phrasal verbs, so they have great
concealment (Tubishat et al., 2018). Since the feature words or the

* Corresponding author.
E-mail addresses: wangzeng@ncu.edu.cn (Z. Wang), liuwd@ncu.edu.cn (W.-d. Liu).
1
ORCID: 0000-0002-7810-9669.
2
ORCID: 0009-0005-6950-6581.
3
ORCID: 0000-0002-2276-2159.

https://doi.org/10.1016/j.eswa.2023.122512
Received 24 February 2023; Received in revised form 27 September 2023; Accepted 7 November 2023
Available online 11 November 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
Z. Wang et al.
Table 1
Linguistic features of sarcastic text in Internet slang.
Linguistic features Example
Set Collocation 这手机(this phone)用的可以(used be)再烫一点(more hotter)
吗？
Specific Adverbs 真有你的(good for you)、服了、服气(I’m speechless)等
Specific 呵呵、笑死、吐了(LOL)等
Interjections
Internet Buzzwords 芭比Q、老6、真的会谢(holy terrible)等
establishment of candidate feature-sentiment word sets is often used as a
screening condition for extracting these implicit features (Chen et al.,
2015; Sun et al., 2015), the effectiveness of these methods is directly
related to the established feature vocabulary and word sets. Besides,
users have to adjust them when they are applied to different scenarios,
such adjustments often require excessive resources to complete, which
means these methods have poor generalization ability. Further more, the
previous methods for implicit product feature extraction are complex in
calculation due to simple integration and stacking of network and
mechanism rather than design based on the characteristics of data itself.
All these deficiencies may lead to the discovery of the wrong product
features from the comments further leads the results of sentiment
analysis to the wrong product features, and ultimately result in the
authenticity and reliability of customer demand acquisition, which has a
negative impact on the follow-up sentiment analysis.
On the other hand, due to the current habit of using Internet lan­
guage, a considerable number of customers will use irony to emphasize
their negative emotions towards a certain thing when expressing opin­
ions on the Internet (Dress et al., 2008; Sarsam et al., 2020). As a means
of expression and writing, irony needs to be understood from the context
to get its true meaning, which is the opposite of what can be understood
literally (Naseem et al., 2020). Nowadays, texts with irony in Internet
language usually have four types of language features (Table 1) (Ben
et al., 2021), and many related studies have also focused on these fea­
tures. Rajadesingan et al. (2015) studied sarcastic semantic phenomena
and sentence features of video comments on Twitter, and proposed a
SCUBA sarcastic recognition model based on set collocations and spe­
cific interjections. Ben et al. (2021) further put forward a sarcastic
recognition method based on fuzzy theory, which finally obtained more
realistic results of sarcastic recognition by combining the linguistic
features of irony.
As an important way for customers to express their product experi­
ence, irony also exist in product online reviews (Zeng et al., 2014).
However, to the best of our knowledge, there are few case studies that
apply sarcastic semantics to sentiment analysis of product online re­
views. If the sentiment polarity of sarcastic comments can be correctly
identified, the quality of the overall sentiment analysis can be improved
to a certain extent, and it is helpful to further accurately grasp the
consumer demand for products.
As a representative technique in deep learning, gate recurrent unit
(GRU) and capsule neural network (CAP) provide new ideas and
methods with enlightening and potential value to overcome the above
two shortcomings. For the product feature extraction part, GRU can
extract explicit product features more accurately according to its
forgetting and storing characteristics, while CAP can extract implicit
product features by recognizing the spatial and semantic relationships
between features; For the sentiment analysis part, CAP can identify the
sarcastic sentences with obvious linguistic features through its own
structural features. At the same time, combined with GRU, it can more
accurately complete the sentiment analysis at the product feature level
by virtue of its own advantages in processing sequence data.
In view of this, this paper proposes a novel method for product
feature sentiment analysis that integrates GRU and CAP (GRU-CAP) and
considers sarcasm detection. First of all, the initial online review text
data of this study is collected by web crawlers on e-commerce website
2
Expert Systems With Applications 241 (2024) 122512
platforms, and then text preprocessing is performed on the data,
including the attachment of data labels, the deletion of stop words, the
segmentation of long sentences according to particular punctuation, and
the filtering and deletion of invalid comments. Then, the processed
online review data is input into Word2Vec model to output as word
vectors, and the product online review data set composed of several
word vectors is adopted as the final input data of GRU-CAP feature
extraction model.
After data preprocessing is completed, the product online review
data set is fed into the GRU-CAP feature extraction model. The model
outputs the classification results of each word in the comments from two
different sets of channels, obtains the final word classification result in a
comprehensive and weighted method, and extracts the feature words
according to the results.
After that, the extracted product features are coded with index values
and fused with the product online review data set to form a new matrix,
which is input into the GRU-CAP sentiment analysis model. In this
model, data is fed into the sarcastic recognition channel and the senti­
ment analysis channel in parallel to compute more adequate. After a
certain number of rounds of learning and weight iteration, the sentiment
polarity (positive/negative) of comment sentences is fused and output
(Fig. 1).
The main contributions of this paper are summarized as follows:
• On the basis of comprehensively considering the shortcomings of
current product feature sentiment analysis research and comparing
the characteristics and advantages of different representative deep
learning techniques, a novel deep learning parallel architecture
model combining gate recurrent unit and capsule network is pro­
posed to improve the efficiency and accuracy of feature extraction
and sentiment analysis.
• Considering the influence of sarcastic semantics phenomenon on
product feature sentiment analysis, a novel sentiment analysis model
combining sarcasm recognition and GRU-CAP network is proposed.
The model adds a sarcasm recognition channel on the basis of
traditional binary sentiment analysis, which can further improve the
accuracy of product feature sentiment analysis.
The rest of this paper is organized as follows. An overview of the
literature is given in Section 2. Section 3 introduces the theoretical basis
of the proposed method and our GRU-CAP model. In Section 4, we
conduct a case study based on a certain type of mobile phone, present
the related results for discussion, and verify the feasibility of GRU-CAP
model through extensive experiments. In Section 5, we give some brief
conclusions and prospects for future research.
2. Literature review
2.1. Current status of product feature extraction research
Feature extraction is the basis of fine-grained product feature senti­
ment analysis, and its precision will directly affect the results of subse­
quent sentiment analysis. Depending on different principles used, the
existing research can be divided into the following three categories:
2.1.1. Rules-based product feature extraction method
This method filters and compares texts according to certain gram­
matical rules to achieve product feature extraction. Jakob and Gurevych
(2010) constructed the basic template features of product features by
combining product feature words, parts of speech and syntactic features,
and extracted product features by using a conditional random field
(CRF) model. Some scholars have suggested that the more frequently a
word appears in a document, the more important it is (Hu & Bing, 2004).
Based on this idea, several new methods of product feature extraction
have emerged. Shen et al. (2021)) established a product feature dictio­
nary based on term frequency inverse document frequency (TF-IDF)
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Fig. 1. Overall research plan of proposed method.

3
Z. Wang et al.
method to explore consumer demand. Quan and Ren (2014) further
combined the theory of point-wise mutual information (PMI) with TF-
IDF to extract product features through evaluating the relationship be­
tween candidate features and text domain entities.
2.1.2. Machine learning-based product feature extraction method
This method uses the correlation algorithm in machine learning to
establish keyword extractors to extract product features in review texts.
Zha et al. (2014) proposed a feature extraction method for electronic
products through an improved support vector machine (SVM) model.
Wang and Lu (2014) put forward a two level hierarchical hidden Markov
model (HHMM-2) for product feature extraction in online reviews. Zhou
and Lin (2018) combined binary weighted network with node ranking
algorithm and proposed a machine learning keyword extractor based on
dependency analysis. Similarly, various unsupervised machine learning
method are also applied to achieve product feature extraction. This type
of method uncovers latent patterns and underlying characteristics from
the given data. Joung and Kim (2021) added an key word extraction in
preprocessing according to a set of product manual documents, which
reduce the effect of noise key words to latent Dirichlet allocation (LDA)
result. The experiment showed that this measurement make the LDA
model gain a higher quality topic result, which is better for product
feature extraction and subsequent sentiment analysis. Further more, to
enhance the interpretability of neural networks in feature extraction and
achieve a more perfect balance between predictability and explain­
ability, Joung and Kim (2022) proposed an explainable neural network
xNN based on the Shapley additive explanation (SHAP) and an approach
with the automation of Kano four tasks to conduct the Kano analysis
from online reviews. The results exhibited that the proposed approach
presents a more reliable performance than the previous method
combining product feature word clustering and neural network. Wang
et al. (2018) constructed a new framework of applying online product
reviews using LDA and topic difference analysis to analyze customer
preference. Zhou et al. (2020) proposed a machine-learning approach to
customer needs analysis for product ecosystems. Firstly, fastText was
applied for data preprocessing, LDA and associate rule is further used for
product feature extraction and sentiment analysis. Finally, Kano model
was used for categorized customer needs.
Unlike supervised learning methods that require large-scale labeled
data, these above methods save a certain amount of manpower and time
resources when switching application scenarios. Compared with these
methods, the proposed method in this paper mine the product features
involved in the review more accurately from the semantic content level
regardless of whether it is in the form of a noun or noun phrase.
2.1.3. Deep learning-based product feature extraction method
Compared with machine learning, deep learning is more suitable for
mining text data rich in semantic information due to the longer stack
combination of nonlinear layers in the network it builds, thus it is
favored by scholars. Li et al. (2022) proposed a bidirectional simple
recurrent units (Bi-SRU) model, which combines deep network structure
with multi-head attention mechanism to extract features at the word and
grammar levels. Alamoudi and Alghamdi (2021) used pre-trained deep
learning models to extract product features, eliminating the multi-
faceted complexity of traditional supervised learning models. Tang
et al. (2019) combined partial network of convolutional neural network
(CNN) with recurrent neural network (RNN) to form a new parallel
convolution gated circulation neural network, which not only over­
comes the shortcomings of CNN algorithm, but also mines text feature
words at a deeper semantic level.
2.2. Current status of sentiment analysis research based on product online
reviews
The existing sentiment analysis research based on product online
reviews mainly relies on the knowledge of nature language processing
4
Expert Systems With Applications 241 (2024) 122512
(NLP) and statistics, which can be divided into the following three cat­
egories according to specific research methods:
2.2.1. Sentimental feature dictionary-based sentiment analysis method
This method completes the sentiment analysis of product online re­
views by matching an emotional expression dictionary and a corre­
sponding product feature dictionary. Shen et al. (2021) first built an
attribute dictionary and a sentiment dictionary through an online crowd
sourcing platform, then adopted a recognition model of review spam
based on information entropy theory to filter online reviews, and finally
calculated the attribute attention and attribute sentiment by time
sequence analysis to obtain consumer demand trends for product fea­
tures. Bora (2012) used the filtered emotion words to construct an
emotion thesaurus, completed the task of emotion tendency classifica­
tion of Twitter comment text, and achieved a good accuracy in the final
verification. Turney and Littman (2003) extracted emotional two-word
phrases and calculated their average emotional semantic quantity
values, and then judged their emotional tendencies based on the values.
The SO_CAL model proposed by Taboada et al. (2011) comprehensively
considered the impact of the polarity reversal of degree adverbs and
negative adverbs on affective tendency. Although this sentiment anal­
ysis method is simple and convenient to operate, the analysis results are
easy to be affected by the delicacy of the dictionary built, and have high
volatility and insufficient stability.
2.2.2. Machine learning-based sentiment analysis method
The method uses text classifiers constructed by machine learning
algorithms to analyze product online reviews. To dig out the needs of
consumers for short-life electronic products, Zhang et al. (2021) pro­
posed a method combining latent Dirichlet allocation (LDA) with
SnowNLP for feature topic extraction and sentiment polarity classifica­
tion in product online reviews. Wawre and Deshmukh (2016) analyzed
product online reviews using basic vocabulary and predicted sentiment
tendencies using models including simple Bayes and maximum entropy.
Yang et al. (2013) proposed a Chinese text analysis method based on
Bayesian model and verified its effectiveness by analyzing consumer
reviews of a hotel on a website. Liu et al. (2013) proposed a multi-class
classification SVM method based on adaptive mechanism to transform
traditional two-class sentiment classification issues into an adaptive
topic issue under the premise of multiple topics. Compared with the
traditional method of constructing sentimental dictionary, the machine
learning-based method greatly improves the stability and can under­
stand the semantic relationship in text data to a certain extent. However,
this method often requires a huge manually annotated data set, and
when the data capacity is not large enough or the ratio of positive and
negative samples is seriously unbalanced, the trained classifier cannot
achieve the expected classification effect.
2.2.3. Deep learning-based sentiment analysis method
This method builds a deep hierarchical complex neural network to
analyze the sentiment of review texts. Zhang et al. (2019) proposed a
three-channel CNN model to improve three aspects of text sentiment
analysis. Pilehvar and Collier (2016) used a more complex long short-
term memory (LSTM) model for sentiment analysis. Liu et al. (2019)
proposed a new model combining the attention mechanism with CNN. It
adds an attention layer after the convolutional layer of traditional CNN
model to perform nonlinear calculations to obtain local sentimental
features, and uses the innovative activation function NLReLU to further
improve its performance. Tang et al. (2015) proposed two models,
TD_LSTM and TC_LSTM. The TD_LSTM model divides product online
reviews according to the location of product feature attributes, and then
uses two independent LSTMs to obtain sentiment features and calculate
the sentiment tendency of the comment text. On the basis of the
TD_LSTM, the TC_LSTM model concatenates the word vectors of product
features with the word vectors of each word, and inputs them into the
LSTM network to establish the semantic relationship between product
 Z. Wang et al.
Fig. 2. Structure of LSTM unit & gate recurrent unit. Structure of LSTM unit shown in (a), Structure of gate recurrent unit shown in (b).
features and text words.
2.3. Sarcasm recognition
Sarcasm is a sophisticated language phenomenon, which would
cause much confusion to exist sentiment classification systems.
Recently, many methods have been proposed to identify sarcasm, which
can be broadly divided into two categories depending on the source of
the data. The first kind of methods focus on text data and identify
sarcastic semantics according to the inconsistency and contradiction of
its own text semantics. Wang et al. (2022) took mask operations on key
sentiment words in original text to obtain mask text data, and then input
this mask text data into BERT model to obtain the generated text.
Finally, according to the similarity between the generated text and the
original text, it is determined whether the original text has sarcastic
semantics. Zhang et al. (2021) proposed that the text to be recognized is
considered as a quantum superposition of a set of separate words, and
the contextual interaction between adjacent text is described as the
interaction between a quantum system and its surrounding environ­
ment. A complex-valued fuzzy network was proposed for sarcasm
recognition by leveraging the mathematical formalisms of quantum
theory and fuzzy logic.
The second kind of methods is based on extra information such as
users’ history information and common sense knowledge. Ray et al.
(2022) proposed a multi-modal sarcasm recognition method including
text, audio and video and expanded the sarcasm detection dataset
(MUStARD). Plepi and Flek (2021) proposed a graph attention networks
leveraging users’ historical tweets and the social information from a
user’s conversational neighborhood, and the results indicated that the
model contributes to interpreting the authors’ sarcastic intentions.
3. Methodology & framework
In this section, we first briefly introduce the theoretical basis and
core techniques of the proposed GRU-CAP method, and then describe
the feature extraction model and the sentiment analysis model in the
method in detail.
3.1. Gate recurrent unit
GRU is an optimization algorithm of LSTM. Both of them are derived
from RNN, which has been widely applied into various fields because of
its excellent sequence processing ability. However, the RNN model still
has two major defects and deficiencies (Alom et al., 2019; Geng et al.,
2020; Sherstinsky, 2020). First, the memory mechanism of the hidden
5
­
Expert Systems With Applications 241 (2024) 122512
layer of the model is limited due to the defects of its basic structure. If
the scale of the processed data sequence is too large, it is difficult for
RNN to transfer the earlier information to the later time step. Second,
the model has the problem of vanishing gradient. Since the first deriv­
ative of the activation function used in the hidden layer of RNN is close
to 0 when the independent variable is small or large, as the number of
training increases, the gradient change of the model gradually decreases
or even returns to zero, which seriously affects the training process of
the model. To address these issues, LSTM has been proposed as an
improved algorithm by modifying the internal structure of RNN. The
core concepts of LSTM are cell state and gate structure. The cell state is
equivalent to the path of information transmission, even the information
of earlier time step can be carried to the cells of later time step, which
overcomes the limitation of the memory mechanism of RNN. Informa­
tion is updated and removed through the four-gate structure (Fig. 2(a)).
The gate structure during training not only learns what information to
save or forget, but also greatly eliminates the vanishing gradient
problem.
On the basis of LSTM, GRU further integrates the cell state with the
hidden layer state, and transforms the four-gate structure into the
double-gate structure of the renewal gate and the reset gate, which
effectively reduces the computational complexity of network parameters
(Alom et al., 2019; Geng et al., 2020) (Fig. 2(b)). The detailed operation
process is as follows:
First, calculate the update information of the unit, which includes
deciding what previously stored information to forget and how much
current input to absorb. The formula is as follows:
zt = σ(Wz ⋅[ht− 1 , xt ]) (1)
rt = σ(Wr ⋅[ht− 1 , xt ]) (2)
Where zt represents the proportion of information inherited from the
previous cell unit, rt represents the absorb rate of the input information
for the current cell unit, Wz and Wr are the weight matrices of the update
gate and reset gate, ht− 1 is the information data for the previous unit, xt
is the input data for the current unit, σ represents the sigmoid function
which compresses values into a range from zero to one.
Second, determine the information to be transported to the next
sequence unit and the information to be output as current unit as
∼
ht = tanh(Wh ⋅[rt × ht - 1 , xt ]) (3)
∼
ht = (1 - zt ) × ht - 1 + zt × ht (4)
Where ht represents the information data for the current unit, ̃
ht repre
Z. Wang et al.
Fig. 3. Basic structure of capsule network proposed by Hinton et al in 2017.
sents the input information data for candidates, Wh is the hidden layer
weight coefficient matrix.
3.2. Capsule network
The concept of capsule network (Hinton et al., 2011; Sabour et al.,
2017) referring to a set of neurons of a certain size in the hidden layer of
a neural network as a capsule. Capsule network is a variant of CNN. In
CNN, the relationship between deep and shallow hidden layers is
determined by convolution operations, and its state is a static connec­
tion. In a capsule networks, a capsule is defined as a set of neurons,
which can be composed of vectors or matrices. Multiple capsules make
up a hidden layer, and the relationship between the two layers is
determined by dynamic routing algorithm. Different from CNN, the
composition of capsules is flexible, and the dynamic routing algorithm
does not calculate through templates, but calculates the depth of two
hidden layers between each capsule. The calculation method of dynamic
routing achieves the dynamic connection between the two layers, so that
the model can automatically select more effective capsules and improve
the performance.
The basic capsule network structure is composed of input layer,
convolution layer, primary capsule layer and digital capsule layer
(Fig. 3). The input layer contains the input data of the capsule network.
The convolution layer is used to extract multiple local features from the
input data and activate them through the ReLu activation function. The
primary capsule layer encapsulates the local features of the convolution
layer into capsules containing multiple features using a larger stride,
less-channel convolution kernel and filter, and its length represents the
Fig. 4. Architecture of the proposed GRU-CAP feature extraction model.
6
Expert Systems With Applications 241 (2024) 122512
number of features it contains. The digital capsule layer is the fully
connected layer of the capsule network. The connection probability
between each capsule in the primary capsule layer and the capsule in the
digital capsule layer is calculated by dynamic routing algorithm, so that
the feature information contained in the capsule can be better used to
predict the classification. The dynamic routing algorithm is shown as
follows:
The core of dynamic routing algorithm is to find the best coupling
coefficient cij . The weight value determines the degree of closeness be­
tween each capsule j in layer L +1 and each capsule i in layer L to
calculate the final output classification of the capsule j. Specifically, the
following operations are included in each iteration:
(i) Normalization
exp(bij )
cij = ∑k (5)
i=1 exp(bij )
Where cij presents the coupling coefficient, bij presents the prior prob­
ability of each capsule i in layer L to each capsule j in layer L + 1, k is the
total number of capsules in layer L.
(ii) Calculate the linear combination input of the high-level nerve
capsule
∑
sj = u j|i
cij ̂ (6)
i
u j|i = Wij ui
̂ (7)
Where sj is the input vector, ̂
u j|i is the prediction vector of each capsule i
Z. Wang et al.
Fig. 5. Architecture of the proposed GRU-CAP sentiment analysis model.
in layer L to each capsule j in layer L + 1, ui is the output of each capsule
i in layer L.
(iii) Calculate and compress the output of the high-level nerve
capsule
⃦ ⃦
⃦ 2⃦
⃦sj ⃦ s the model to realize product feature extraction, and its structure consists
vj = ⃦ ⃦⋅⃦ j ⃦ (8)
1 + ⃦s2j ⃦ ⃦sj ⃦
Where vj is the output of capsules in layer L + 1.
(iv) Update the weights
bij ←bij + ̂
u j|i ⋅vj (9)
After several iterations of the dynamic routing algorithm, the capsule
network adjusts the weight according to the loss function until the error
reaches the set threshold and completes the network training. The loss
function used by the network is expressed as
Lk = Tk max(0, m+ − ‖vk ‖)2 + λ(1 − Tk )max(0, ‖vk ‖ − m− )2 (10)
Where k is the number of categories, Tk is a classification function, Tk =
1 if and only if the category k appears, otherwise Tk = 0. ‖vk ‖ represents
the modulus length of the vector vk , which is actually the probability of
belonging to class k. m+ and m− are the edge thresholds, m+ is the upper
edge threshold, which penalizes the false positives to deal with the sit­
uation where the category is predicted to exist but does not actually
exist; m− is the lower edge threshold, which is used to deal with the
situation where the category is predicted to exist but are not correctly
classified. λ is the sparse coefficient, which is applied to adjust the bal­
ance between m+ and m− .
3.3. The model proposed in this paper
Based on the aforementioned techniques of gate recurrent unit and
capsule network, we propose two innovative models, namely, GRU-CAP
feature extraction model and GRU-CAP sentiment analysis model.
3.3.1. GRU-CAP feature extraction model
The proposed feature extraction model combining gate recurrent
unit and capsule network can extract both relevant explicit and implicit
product features from product online reviews more adequately. Its ar­
chitecture is shown in Fig. 4.
The GRU-CAP feature extraction model consists of three major parts
7
Expert Systems With Applications 241 (2024) 122512
(Fig. 4). The first part is the word-embedding layer applied to deal with
data entry processing, in which the product online reviews are trans­
formed into 128-dimension word embedded data Datasetvec 1
as the input
of the feature extraction model through the Word2Vec model.
The second part is the hidden layer, which is also an essential part of
of two sets of channels (Fig. 4). One of the channels is a capsule channel
composed of convolution layer 1, primary capsule layer 1 and digit
capsule layer 1, which is used to separate product features from
redundant data of other non-related text features. In the capsule chan­
nel, the convolution layer 1 first roughly extract valid information fea­
tures from input data, and then the primary capsule layer 1 will be
connected to the convolution layer 1 to further mine the detailed local
features based on the extracted information features. Finally, the digital
capsule layer 1 uses Eq. (8) to reshape the extracted local features to
obtain the feature extraction result Scorei1 of the capsule channel.
Another channel is a gate recurrent channel composed of the gate
recurrent unit layer 1 and the max-pooling layer. Similar to the capsule
channel, the textual semantic relationships of input data will be mined
to get the feature extraction result Scorei2 under supervised learning.
The third part performs feature classification and obtains the final
extraction results. The results Scorei1 and Scorei2 obtained from the two
different sets of channels in the second part are calculated through the
weighted integration of Eq. (11), and the final feature extraction result
Scorefin is obtained through the calculation of Eq. (12).
Socrei3 = α⋅Scorei1 + β⋅Scorei2 (11)
Scorei3
Scorefin = Max(∑k i
) (i = 1, 2, 3, ⋯, k) (12)
i=1 Score3
⎧
⎪
⎪ eloss2
⎨α =
⎪
eloss1 + eloss2
(13)
⎪
⎪ eloss1
⎪
⎩β =
eloss1 + eloss2
Where α and β are weight coefficients, and their initial values are both
0.5. During the training process of the model, they will be automatically
adjusted according to the ratio of the calculated loss value of the capsule
channel and the gate recurrent channel according to formula (13). loss1
and loss2 are the calculated loss values of the capsule channel and the
gate recurrent channel respectively, i is the category of features while k
Z. Wang et al.
Fig. 6. A product online review on Jingdong e-commerce platform.
represents the max number of categories. Scorefin is the final probability
value of the category to which the feature belongs after comprehensive
calculation, as well as the final output result and the corresponding
feature label.
3.3.2. GRU-CAP sentiment analysis model
Aiming at the phenomenon of sarcastic semantics in product online
reviews, this paper proposes a sentiment analysis model combining gate
recurrent network and capsule network (Fig. 5). It considers sarcasm
recognition on the basis of traditional sentiment analysis model, which
can identify the sentiment polarity of reviews more accurately.
Similar to the GRU-CAP feature extraction model, the sentiment
analysis model also includes three parts. The first part is responsible for
processing the input data. By inserting a feature label data corre­
sponding to the sentence into each comment sentence data, the data
1
fusion of Datasetvec and feature labels is completed, and the product
features in the comment sentence are expressed. The data after fusion is
2
recorded as Datasetvec , which is denoted as the input data of the senti­
ment analysis model.
The second part is to realize the capture and learning of multi-
semantic features. The GRU-CAP sentiment analysis model can ach­
ieve more fine-grained sentiment analysis by simultaneously mining
multiple features composed of sentiment semantic features and sarcastic
semantic features in online review text data. Similar to the feature
extraction model, the second part of GRU-CAP sentiment analysis model
also consists of two sets of channels. One is the sarcastic recognition
channel, which is composed of convolution layer 2, primary capsule
layer 2 and digital capsule layer 2.
When product review enters the sarcastic recognition channel, the
convolution operation is performed in the convolution layer 2 to obtain
preliminary semantic features, which contain potential sarcastic se­
mantic information such as the local features of some word vectors with
sarcastic semantic features. Secondly, preliminary semantic features
enter the primary capsule layer 2 and output deep semantic features,
which not only contains specific semantic features, but also includes
some relationships between these semantic features, such as the distance
between features, modular value differences, etc. These information can
precisely correspond to the sarcastic semantic features in product online
reviews. For example, specific semantic features reflect the specific
sarcastic words in the text, while the distance and modular value dif­
ferences between features can reflect the contradictions in the text.
Finally, all these information is fed into the digital capsule layer 2 and
mapped to determine whether the text contains sarcastic semantic.
Meanwhile, a discriminant value of sarcastic semantic ε is added for
each comment sentence. If the model judges that there is sarcastic se­
mantics in the review sentence in the current operation, then ε = 1,
otherwise ε = 0.
The other is the sentiment analysis channel, which is composed of
8
Expert Systems With Applications 241 (2024) 122512
gate recurrent unit layer 2 and max pooling layer. It is responsible for
binary classification of the sentiment polarity of the currently operating
comment sentences based on the mined emotional semantic features.
After the calculation, the channel will add a judgment value of sentiment
polarity σ to each comment sentence. If the sentiment analysis channel
judges the sentiment polarity of the comment sentence in the current
operation is positive, then σ = 1, otherwise σ = 0.
After completing all the aforementioned processes, the third part
obtains the final results of sentiment analysis. In this part, the semantic
value of sarcasm and the value of sentiment polarity are integrated by
the discriminant Eq. (14), and both the product features of the review
sentence and the result of sentiment polarity analysis are output after
calculation.
{
0, (ε = 1) ∪ (ε = 0 ∩ σ = 0)
ϛ= (14)
1, σ = 1
Where ϛ = 0 indicates that the final sentiment polarity of the current
comment sentence is negative, while ϛ = 1 indicates the opposite situ­
ation.
4. Experiment
In this section, we will present the application of the proposed
method, along with the corresponding results of discussion and analysis.
4.1. Data acquisition and preprocessing
The dataset applied in this paper was collected by web crawler
software ‘Octopus’ in Jingdong website, which is one of the most
influential e-commerce platforms in China. We chose Iphone 12 as our
experimental subject. As shown in Fig. 6, there are several kinds of in­
formation contained in users’ reviews including user ID, account level,
user rating of product, review text information, review publication time,
and photos of products. With crawler tools, we collected a total of
40,350 online review which contain the information mentioned before.
After collecting the necessary data, the preprocessing operation is set
up to ensure the quality of dataset. Words of 4 characters or less in the
text information dataset and meaningless text with emoticons are first
excluded from the review. Secondly, punctuation is removed from the
text and a Chinese stop word list is applied to delete stop words in the
rest of review text. After completing the above steps, the labeling task for
review text is enabled. A focus group of 2 experts was set up to complete
the task of tagging label and verification. The experts come from the
academia and industry in the field of industrial design, including an
industrial design teacher from university and a designer from company.
They all have a keen insight for designing distinctive appearance of
mobile phones and are familiar with the process and skills of tagging
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Table 2 subtexts to ensure that each subtext contains only one product feature.
Data set label construction. The sentiment labels and sarcastic labels are determined based on text
Label_attribution Number semantic information and the star rating of the review. When the in­
formation provided by the two may be inconsistent, the label is deter­
Feature label 42,830
Positive sentiment label 29,120 mined mainly by analyzing the text semantic information. Table 2 shows
Negative sentiment label 13,710 the statistical results after data label determination. Part of the repre­
Sarcastic label 6853 sentative product online reviews and their labels are shown in Table 3.
Non-sarcastic label 35,977 After the data preprocessing was completed, the online reviews were
1
first transformed into matrix data Datasetvec with a dimension of 128 by
the Word2Vec model in the GRU-CAP feature extraction model, and the
Table 3 matrix data corresponding to the preprocessed online review text was
Representative product online review text. obtained. As shown in Fig. 7, the number of rows of the matrix is 128,
ID Feature Sentiment Sarcastic Content of review and the number of columns corresponds to the number of words in the
label label label comment sentence.
1 Appearance Positive Non- 很(very)/喜欢(like)/颜色 After completing the above steps, to better train and validate the
sarcastic (color)/新(new)/手机(phone)/ model, the dataset was divided into three parts according to the struc­
新(fresh)/心情(emotion) ture of train set, test set and validation set at a ratio of 6:2:2 (Geng et al.,
2 Performance Negative Sarcastic 网络(net)/真(really)/是(is)/超
级卡(delayed)/现在(now)/都
2020) (Table 4).
(almost)/是(is)/5G/这个(this)/
反应(reaction)/真是(really)/没 4.2. Implementation & results
谁了(terrible)
3 Battery Negative Sarcastic 充电(charge)/插头(plug)/不配
In this section, we present the details of the model experiment and
(do not allocate)/真(really)/是
(is)/绝了(awesome) the relevant results.
4 Price Positive Non- 优惠(discount)/力度(strength)/
sarcastic 大(high)/发货(deliver ability)/
快(fast)/手机(phone)/正品
(quality product)/好(good)
5 Screen Negative Non- 新机(new phone)/面容锁屏(face Table 4
sarcastic lock screen)/坏的(breakdown)/ Dataset description of product feature level senti­
没有(did not)/处理好(handle
ment analysis.
well)
6 Price Negative Sarcastic 买(purchase)/第二天(second Dataset Number
day)/出(launch)/优惠券
Train Set 25,698
(coupon)/真行(LOL)
Test Set 8566
7 Camera Positive Non- 拍照效果(photographic
Validation Set 8566
sarcastic performance)/秒杀(better
Total 42,830
than)/同级别(same level)/所有
(all)/手机(phone)/清晰度/
(articulation)/让我(let me)/目
瞪口呆(surprised)
Table 5
8 Multimedia Negative Sarcastic 音效(AUDIO)/还(may)/是(is)/
Parameters of the proposed GRU-CAP feature extraction model.
小了点(a bit small)
Parameter Attribute

Input 128-dimensional word vector matrix

labels. Therefore, the reliability of data label is guaranteed. Output Probability toward product features
There are three types of labels need to accomplish for review text, Scale of capsule hidden layer 128–256-32*8–8–8
namely feature labels, sentiment labels and sarcastic labels. According Dimension of word vector 128
to the fundamental functions and features of the mobile phone, product Size of Word2Vec capture window 5
Batch size 128
feature labels are divided into 8 categories (Huang et al., 2022),
Epoch 50
including screen, battery, camera, system, multimedia, appearance, Iteration number of routing Algorithm 1
price and performance. We calibrated the product feature label ac­ Initial standard deviation 0.01
cording to the content in the review text. When multiple product fea­ Number of GRU 128
tures are involved in a review, we split the review text into multiple Dropout rate 0.5

Fig. 7. Matrix data of a representative review text.

9
Z. Wang et al.
Fig. 8. Performance of GRU-CAP feature extraction model on different product features.
Fig. 9. Performance of GRU-CAP feature extraction model on the validation set. The total loss of the model is shown in Fig. a. and the performance on the validation
set is shown in Fig. b.
4.2.1. Parameter setting of GRU-CAP feature extraction model
As shown in Table 5, some critical parameters are set for GRU-CAP
feature extraction model. Some of them including the size of Word2­
Vec capture window, initial standard deviation, number of GRU,
dropout rate are set according to experience, and the other such as the
dimension of word vector, iteration number of routing Algorithm and
epoch are set by experiment. The experiment will display in Section
Discussion.
4.2.2. Product feature extraction
According to the above parameters, product feature extraction was
completed through the proposed GRU-CAP feature extraction model,
and the corresponding product feature indexes and the classification
performance of the extracted 8 types of features were obtained.
10
Expert Systems With Applications 241 (2024) 122512
Although the extraction results of battery and multimedia are slightly
lower than those of the other 6 types, the overall F1_Score is above 90%
(Fig. 8).
The total loss of the GRU-CAP feature extraction model and the ac­
curacy on the validation set are obtained after the experiment (Fig. 9).
The total loss tends to be stable and reaches the minimum threshold
when the number of steps is about 10000. Meanwhile, although the
performance of the model on the validation set fluctuates, the perfor­
mance of synthesis precision index and the overall F1_Score indicate that
the proposed model can not only fully extract the product features in
online reviews, but also operate smoothly.
4.2.3. Product feature sentiment analysis
2
Datasetvec 1
was obtained by fusing Datasetvec with the feature label
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Fig. 10. Matrix data of a review text with product feature index.

sentiment analysis model is more complex than that of the feature

Table 6
extraction model. Considering the training time and effect, the number
Parameters of the proposed GRU-CAP sentiment analysis model.
of training rounds was correspondingly reduced. In the experiment, the
Parameter Attribute GRU-CAP sentiment analysis model took a total of 479 s to complete the
Input 128-dimensional word vector matrix training and validation, and its total loss and the performance on the
Output Type of sentiment polarity validation set are shown in Fig. 11.
Scale of capsule hidden layer 128–256-32*8–8-2
As it can be seen, the total loss of the model enters a relatively stable
Batch size 128
Epoch 20
state when the number of steps is about 350, and continues to decrease,
Iteration number of routing algorithm 1 and finally stabilizes at 2500 steps. The final calculated loss at this time
Learning rate 1e-3 is 0.0026. The validation performance tends to be stable and keeps in the
Number of GRU 128 range of 92% ~ 96% when the number of steps is about 600 and the final
Dropout rate 0.5
validation F1_Score is 93.35% after the loss reaches the threshold.

index, and then used as the input data of the GRU-CAP sentiment
analysis model. Subsequently, the matrix data of product online reviews
fused with the product feature index was obtained (Fig. 10).
Then, the GRU-CAP sentiment analysis model was applied to
accomplish the sentiment analysis at the product feature level. Some
critical parameter settings in the model are shown in Table 6. It should
be noted that the number of training rounds of the sentiment analysis
model is 20, which is less than the number of training rounds of the
feature extraction model. This is because the running process of the
4.2.4. Comparison
To demonstrate the advantages of the proposed method, the
following six models were selected for comparison using the same data
1
set Datasetvec , and the experimental results are shown in Fig. 12.
(i) Max-Entropy model: A classification model that selects the
maximum entropy in the data set satisfying the constraint con­
ditions through the principle of maximum entropy.

Fig. 11. Performance of GRU-CAP sentiment analysis model on the validation set. The total loss of the model is shown in Fig. a. and the performance on the
validation set is shown in Fig. b.

11
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Fig. 12. Evaluation index of each comparison model and the proposed GRU-CAP product feature extraction model.

Fig. 13. Evaluation index of each comparison model and the proposed GRU-CAP sentiment analysis model.

(ii) K Nearest Neighbor model: A model that does not display the (iv) Support Vector Machine model: A class of generalized linear
learning process and uses the training data set to partition the classifiers that classifies data in binary terms according to su­
feature vector space to achieve classification. pervised learning.
(iii) Naive Bayes model: A classification model based on Bayesian (v) Gate Recurrent Unit network
theorem and the assumption of feature conditional (vi) Capsule network
independence.
As for the performance of product feature extraction, the precision,
recall and the comprehensive index F1_Score reached 96.38%, which are

12
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Table 7 Table 8
Representative results of GRU-CAP feature extraction model including correct Representative results of GRU-CAP sentiment analysis model.
cases (1 ~ 7) and error cases (8 ~ 9). ID Content of review Predicted Real Product
ID Content of review Predicted Real feature Feature sentiment sentiment feature
feature label label index polarity polarity

1 手机(mobile phone)不错 Appearance Appearance 5 1 流畅度(fluency)很舒服 Positive Positive Performance

(not bad), 绿的(color (very comfortable), 苹果
green)通透自然 (iPhone)还是(still)比较
(transparent and natural) 快的(faster)！
2 照相清清楚楚(clearly Camera Camera 2 2 轻薄机身(thin body), 颜 Positive Positive Appearance
photographed), 聚焦特别快 色好看(nice color), 很清
(fast focus) 新(very fresh)
3 看屏幕(look at the screen) Screen Screen 0 3 充电插件(charging plug- Negative Negative Battery
眼睛会不舒服(eyes will be in)都没有, 真是绝了
uncomfortable) (unacceptable)！
4 苹果系统(IOS system)稳定 System System 3 4 苹果系统(IOS system)稳 Positive Positive System
流畅(stable and smooth), 定流畅(stable and
运行速度很快(run fast) smooth), 运行速度很快
5 很好(very good), 打电话语 Multimedia Multimedia 4 (run fast)
音通话(phone voice call)都 5 很好(very good), 打电话 Positive Positive Multimedia
没有杂音(no noise)出现！ 语音通话(phone voice
6 价格(price)很美丽(very Price Price 6 call)都没有杂音(no
cheap), 机器(the machine) noise)出现！
很给力(very powerful) 6 价格(price)很美丽(very Positive Positive Price
7 手机续航(battery life)不错, Battery Battery 1 cheap), 机器(the
一天够用(enough for a day) machine)很给力(very
8 手 机很 好 , 如 丝 般 顺 滑 Performance Appearance 7 powerful)
(silky smooth) 7 这 手 机 的 续 航能 力 Positive Negative Performance
9 手 机待 机 时 间 (standby Performance Battery 7 (battery life)我
我真 的 会 谢
time)很 很长(very long) 8 用了一个月了绿屏(green Negative Negative Screen
screen)严重(bad), 真是够
了(that is enough)
higher than other six purposed methods such as BERT (Lehečka et al., 9 新机面容锁屏(new plane Negative Negative Screen
lock screen)就失灵
2020), EM algorithm (Ashraf et al., 2022), LDA and SVM, etc. It shows
(failure), 呵呵(lol)
that the proposed product feature extraction model can be more accu­
rate than traditional model and machine learning model in the product
feature extraction task of product online reviews. Furthermore, although reason is that the training data does not cover all sentence patterns and
the proposed model may not have massive improvement than the model semantic usage, leading to the insufficient learning and misjudgment of
before fusion, it indeed shown the advantage of this fusion of the pro­ the model.
posed model (See Fig. 13). Some representative output results of the GRU-CAP sentiment anal­
As for the performance of sentiment analysis, it can be seen that if the ysis model are shown in Table 8. It can be seen from the table that most
accuracy of the validation set is taken as an indicator, among the first product online review texts, such as text 1, text 2 and text 5, not only
four classical classification models, the accuracy of Naive Bayes model is have explicit product feature objects, but also have sentiment image
the lowest, which is lower than that of KNN and SVM, while the accuracy expression words with obvious sentiment tendency. After simple mul­
of Max-Entropy model is higher than that of KNN and SVM, and the tiple rounds of training, they can be accurately analyzed by the model.
accuracy of GRU-CAP sentiment analysis model is 96.47%, which is Meanwhile, for some sentences with typical sarcastic semantic features,
significantly higher than the aforementioned models. It shows that the such as text 3 and text 8, which either express two completely different
proposed sentiment analysis model can be more accurate than tradi­ emotional tendencies in one sentence, or the described content conflicts
tional statistical analysis model and machine learning model in the with the expressed emotion, the proposed model can also successfully
sentiment analysis task of product online reviews. Furthermore, as a new analyze the correct emotional polarity. However, there are also sen­
model integrated with GRU model and CAP model, which are repre­ tences that fail to accurately analyze, such as text 7. Some latest Chinese
sentative deep learning models, the GRU-CAP sentiment analysis model Internet sarcastic expressions usually appear in such text, such as ‘服了
is at least 6% higher in accuracy than the two original models, the 你这个老6’ (used to describe a feeling of disappointment), ‘芭比Q’(used
similar conclusion is also valid on other evaluation index such as pre­ to describe something terrible), ‘真的会谢’ (used to describe a situation
cision, recall and F1-Score, which further reflects the rationality and in words speechless), etc. For text 7, it is not difficult to judge con­
effectiveness of the integration of the two structures. sumers’ negative emotions about mobile phone performance. However,
due to the limited amount of data used for model training and the rapid
update of Internet terms, the proposed model may misjudge due to the
4.3. Results & discussion inability to identify these new terms when processing such texts.

In this section, we further analyze and discuss the output of the
models and their influencing parameters.
4.3.1. Analysis of model output results
Some representative output results of the GRU-CAP feature extrac­
tion model are shown in Table 7. For texts with implicit product fea­
tures, such as text 1, the model can well associate content objects with
product features and accomplish the corresponding product feature
extraction. Notice that there are a few error cases in the model output,
such as text 8 and text 9. One of the reasons is that adjectives in sen­
tences can modify a variety of features at the same time. The second
4.3.2. Dimension of word vector
The dimension of the word vector also affects the performance of the
model to some extent, so we give a comparative experiment to deter­
mine the dimension of the word vector. This important parameter affects
the computational complexity of the model training, and different set­
tings present different results under the same experimental conditions.
As it can be seen, before the dimension reaches 256, the accuracy and
precision of the feature extraction model are both on the rise, and reach
the peak value at 256, and then decrease with the increasing dimension.
The same phenomenon can be observed in Fig. b, so this parameter is set

13
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Fig. 14. The effect of dimension of word vector on GRU-CAP model. The effect on GRU-CAP feature extraction model is shown in Fig. a. and the effect on GRU-CAP
sentiment analysis model is shown in Fig. b.

which means that the importance of channels in the model is different in

Table 9
different application scenarios. In the task of product feature extraction,
The change of GRU-CAP feature extraction model according to the change of
GRU channel plays more important role than CAP channel; in the task of
value of α and β.
sentiment analysis, CAP channel performs better.
ID α β Precision Recall F1_Score

1 0.16 0.84 57.16 57.74 57.45 4.3.4. Number of routing iterations

2 0.46 0.54 87.55 86.93 87.24 Dynamic routing algorithm is the essential content of capsule
3 0.5 0.5 88.47 86.97 87.71
network. According to the algorithm steps of Eq. (5) ~ (9), the
4 0.51 0.49 91.25 90.21 90.72
5 0.52 0.48 92.04 93.55 92.78 connection weights of the primary capsules and the connection methods
6 0.54 0.46 91.69 94.26 92.95 between different layers of capsules in the network are selectively
7 0.49 0.51 92.41 93.87 93.13 updated, which can overcome the limitations of the previous full
8 0.48 0.52 92.2 94.53 93.35 connection mode of neurons and quickly excavate the features condu­
cive to the extraction operation. The number of routing iterations is an
important parameter that affects the updating weight of dynamic rout­
Table 10 ing algorithm, so how to determine it is very important. Based on this,
The change of GRU-CAP sentiment analysis model according to the change of multiple experiments on GRU-CAP feature extraction model and GRU-
value of α and β. CAP sentiment analysis model were carried out to determine the most
ID α β Precision Recall F1_Score appropriate number of routing iterations. We set the number of itera­
1 0.16 0.84 69.83 68.42 69.12 tions of the routing algorithm as 1, 2 and 3 to observe the effect of this
2 0.33 0.67 80.17 81.61 80.88 parameter on the accuracy of the two proposed model.
3 0.41 0.59 87.59 86.37 86.98 As it can be seen, the GRU-CAP feature extraction model peak at a
4 0.44 0.56 90.11 89.45 89.78
slightly faster rate of accuracy when the number of routing iterations is 1
5 0.46 0.54 90.73 88.62 89.66
6 0.47 0.53 88.36 89.69 89.02
than when the value is 2 and 3 (Fig. 15(a)). At the same time, it is more
7 0.48 0.52 91.39 91.08 91.23 important that the model enters a stable state earlier when the value is 1,
8 0.53 0.47 96.38 96.38 96.38 which not only avoids the redundant fitting process of the model, but
also shortens the time of model training to a certain extent. The effect of
routing iteration on the GRU-CAP sentiment analysis model is shown in
to 256 when conducting the experiment in this paper (See Fig. 14).
Fig. 15(b). As the number of iterations increases, the accuracy of the
model on the test set decreases. Due to the complex spatial relations of
4.3.3. Value of α and β
text data, which includes not only word order and various demonstrative
In Equation (11), the performance of GRU-CAP in product feature
words, but also the hidden emotions in sentences, such as the sarcastic
extraction and sentiment analysis is dependent on the value of α and β,
semantics involved in this paper, the increase of the number of iterations
these two important values are constantly changing based on the loss1
aggravates the process of computation and the loss of repeated
and loss2 . To further elaborate the influence of these two variables on the
computation, resulting in increased time and fluctuations in stability.
performance of proposed model, we give 8 groups of α and β value to
Therefore, the number of routing algorithm iterations for the proposed
further analysis (Table 9 & Table 10). As it can be seen, the performance
model is set to 1.
of GRU-CAP is not satisfactory when the values of two variables differ
greatly. With the narrowing of the gap, the performance of the model
4.3.5. Analysis of sarcastic semantic factors
continues to improve and finally achieves the greatest performance
To further prove the influence of sarcastic semantics factors on the
when α is 0.48 and β is 0.52. The same situation occurs in sentiment
results of affective analysis, we further conducted a comparative
analysis when α is 0.53 and β is 0.47. From this, it can be seen that the
experiment on the GRU-CAP sentiment analysis model. In the experi­
weight of the two channels has changed from being equal at the
ment, while keeping the overall structure of the model unchanged, the
beginning to being slightly tilted towards one channel or the other
model was trained in the modes of considering sarcasm recognition and

14
Z. Wang et al. Expert Systems With Applications 241 (2024) 122512

Fig. 15. The effect of routing algorithm iterations on GRU-CAP model. The effect on GRU-CAP feature extraction model is shown in Fig. a. and the effect on GRU-CAP
sentiment analysis model is shown in Fig. b.

Fig. 16. The effect of sarcastic semantics on GRU-CAP sentiment analysis model.

Table 11 Table 12
Auto_home-dataset label construction. Performance on different dataset with proposed GRU-CAP feature extraction
Label_attribution Number
model.
Data set Precision(%) Recall(%) F1_Score(%)
Feature label 4010
Positive sentiment label 2726 JD-dataset 96.38 96.38 96.38
Negative sentiment label 1284 Auto_home-dataset 93.34 95.58 94.45
Sarcastic label 522
Non-sarcastic label 3488
analyze the sentiment polarity of product online review.

not considering sarcasm recognition respectively.

4.3.6. Generalization verification
The comparative results of sarcastic semantics experiment are shown
To verify the generalization of proposed GRU-CAP model, take
in Fig. 16, where Fig. a shows the performance of GRU-CAP without
automotive products as an example, we use another dataset Auto_home-
sarcasm recognition, Fig. b shows the performance of GRU-CAP with
dataset (Table 11) collected from Autohome website which is one of the
sarcasm recognition. It is easy to figure out that the measure index’Re­
call’ from Fig. a lower than Fig.b’s, which means that GRU-CAP with
sarcasm recognition can extract negative sentiment from online review Table 13
more efficient than the one without sarcasm recognition, and negative Performance on different dataset with proposed GRU-CAP sentiment model.
reviews with ironic meaning were also accurately analyzed. Therefore,
Data set Precision(%) Recall(%) F1_Score(%)
considering the sarcastic semantic factors, the product feature level
sentiment analysis based on the proposed model can more accurately JD-dataset 92.20 94.53 93.35
Auto_home-dataset 89.20 93.60 91.34

15
Z. Wang et al.
most influential automotive product information platform in China to
verify the model’s malleability. This dataset has been preprocessed as
described in section 4.1. The architecture of model and parameters are
also set as described above. As it can be seen in Table 12 and Table 13,
after the change of data set, the performance of both feature extraction
model and sentiment analysis model will be affected to some extent.
Although the performance is reduced, it still maintains a high level of
classification, which demonstrate the method’s ability to generalize
online review texts for different types of products.
5. Conclusion
Aiming at the problems and deficiencies in the current research on
sentiment analysis of product online reviews, a novel sentiment analysis
method combining gate recurrent units and capsule neural network and
considering sarcastic recognition has been proposed. It uses the GRU-
CAP feature extraction model to extract the product features con­
tained in online reviews and generate the corresponding product feature
index. After fusing the product feature index with the online review data
set, the GRU-CAP sentiment analysis model is innovatively used to
identify the sentiment polarity and sarcastic semantics at the same time
and integrate the output to achieve fine-grained sentiment analysis of
product features. Finally, taking a mobile phone from Jindong e-com­
merce platform as an example, the feasibility and effectiveness of the
proposed method are verified. All the experimental results show that the
method can better understand consumer emotions in all aspects of
products, provide more accurate and real consumer demand for subse­
quent product design improvements and further promote the design of
highly competitive products that are more in line with consumer
demand.
However, this study also has some shortcomings. First of all,
although the proposed method has completed the corresponding anal­
ysis tasks well, the training time of its models is longer than that of the
traditional analysis model. In the meantime, whether it is applicable to
complex review such as use allusions and complex narratives in the
social media platform remains to be further verified. Aiming at these
shortcomings, we consider using the multi-source data of customers’
reviews (including texts, graphs, videos, etc) and keyword extraction
technology to build a real-time updated online sarcastic expression
database to solve the defects mentioned above.
CRediT authorship contribution statement
Zeng Wang: Conceptualization, Funding acquisition, Validation,
Writing – original draft. Shi-jie Hu: Conceptualization, Investigation,
Methodology, Writing – original draft, Writing – review & editing. Wei-
dong Liu: Conceptualization, Funding acquisition, Supervision, Writing
– review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
Data will be made available on request.
Acknowledgements
This research is supported by the National Natural Science Founda­
tion of China (Grant No. 72071099 and Grant No. 71661023), Jiangxi
Province Humanities and Social Science Research Projects of Univer­
sities (Grant No. JC22131) and the first batch of new liberal arts
research and Reform practice projects of the Ministry of Education of
16
Expert Systems With Applications 241 (2024) 122512
China (Grant No. 2021160038).
References
Alamoudi, E. S., & Alghamdi, N. S. (2021). Sentiment classification and aspect-based
sentiment analysis on yelp reviews using deep learning and word embeddings.
Journal of Decision System, 30, 259–281. https://doi.org/10.1080/
12460125.2020.1864106
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., …
Asari, V. K. (2019). A State-of-the-Art Survey on Deep Learning Theory and
Architectures. Electronics, 8, 292. https://doi.org/10.3390/electronics8030292
Ashraf, N., Khan, L., Butt, S., Chang, H., Sidorov, G., & Gelbukh, A. (2022). Multi-label
emotion classification of Urdu tweets. PeerJ Computer Science, 8, e896.
Ben, M. A., Hlaoua, L., & Ben, R. L. (2021). A fuzzy approach for sarcasm detection in
social networks. In 25th KES International Conference on Knowledge-Based and
Intelligent Information & Engineering Systems (KES) (Vol. 192, pp. 602-611). https://
doi.org/10.1016/j.procs.2021.08.062.
Bora, N. N. (2012). Summarizing public opinions in tweets. International Journal of
Computational Linguistics and Applications, 3, 41–55.
Chen, J., Sun, L., Peng, Y., & Huang, Y. (2015). Context weight considered for implicit
feature extracting. In 2015 IEEE International Conference on Data Science and
Advanced Analytics (DSAA) (pp. 1-5). https://doi.org/10.1109/
DSAA.2015.7344860.
Dress, M. L., Kreuz, R. J., Link, K. E., & Caucci, G. M. (2008). Regional variation in the
use of sarcasm. Journal of Language and Social Psychology, 27, 71–85. https://doi.org/
10.1177/0261927X07309512
Filieri, R., Lin, Z. B., Pino, G., Alguezaui, S., & Inversini, A. (2021). The role of visual cues
in eWOM on consumers’ behavioral intention and decisions. Journal of Business
Research, 135, 663–675. https://doi.org/10.1016/j.jbusres.2021.06.055
Geng, Z. Q., Chen, G. F., Han, Y. M., Lu, G., & Li, F. (2020). Semantic relation extraction
using sequential and tree-structured LSTM with attention. Information Sciences, 509,
183–192. https://doi.org/10.1016/j.ins.2019.09.006
Hinton, G. E., Krizhevsky, A., & Wang, S. D. (2011). Transforming auto-encoders. In
Proceeding of 21st International Conference on Artificial Neural Networks (pp. 44-51).
Hu, M., & Bing, L. (2004). Mining and summarizing customer reviews. In Proceedings of
the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
(pp. 168-177). https://doi.org/10.1145/1014052.1014073.
Huang, X., Liu, Y., Wang, Y., & Wang, X. (2022). Feature extraction of search product
based on multi-feature fusion-oriented to Chinese online reviews. Data Science and
Management, 5, 57–65. https://doi.org/10.1016/j.dsm.2022.04.002
Jakob, N., & Gurevych, I. (2010). Extracting Opinion Targets in a Single- and Cross-
Domain Setting with Conditional Random Fields. In Conference on Empirical Methods
in Natural Language Processing (pp. 1035-1045).
Joung, J., & Kim, H. M. (2021). Automated keyword filtering in latent Dirichlet
allocation for identifying product attributes from online reviews. Journal of
Mechanical Design, 143(8). https://doi.org/10.1115/1.4048960
Joung, J., & Kim, H. M. (2022). Explainable neural network-based approach to Kano
categorisation of product features from online reviews. International Journal of
Production Research, 60(23), 7053–7073. https://doi.org/10.1080/
00207543.2021.2000656
Li, M. X., Huang, L. Q., Tan, C. H., & Wei, K. K. (2013). Helpfulness of online product
reviews as seen by consumers: Source and content features. International Journal of
Electronic Commerce, 17, 101–136. https://doi.org/10.2753/JEC1086-4415170404
Li, T. K., Xu, H. J., Liu, Z., Dong, Z., Liu, Q., Li, J., … Sun, X. J. (2022). A spatiotemporal
multi-feature extraction framework for opinion mining. Neurocomputing, 490,
337–346. https://doi.org/10.1016/j.neucom.2021.11.098
Liu, S., Li, F., Li, F., Cheng, X., & Shen, H. (2013). Adaptive co-training SVM for
sentiment classification on tweets. In Proceedings of the 22nd ACM international
conference on Conference on information & knowledge management (pp. 2079-2088).
https://doi.org/10.1145/2505515.2505569.
Liu, Y., Ji, L. X., Huang, R. Y., Ming, T. S. Y., Gao, C., & Zhang, J. P. (2019). An attention-
gated convolutional neural network for sentence classification. Intelligent Data
Analysis, 23, 1091–1107. https://doi.org/10.3233/IDA-184311
Lehečka, J., Švec, J., Ircing, P., Šmídl, L. (2020). Adjusting BERT’s pooling layer for
large-scale multi-label text classification. In International Conference on Text, Speech,
and Dialogue (pp. 214-221). https://doi.org/10.1007/978-3-030-58323-1_23.
Naseem, U., Razzak, I., Eklund, P., & Musial, K. (2020). Towards improved deep
contextual embedding for the identification of irony and sarcasm. In 2020
International joint conference on neural networks (IJCNN) (pp. 1-7). https://doi.org/
10.1109/IJCNN48605.2020.9207237.
Park, C., Jun, J., Lee, T., & Lee, H. (2018). Customer orientation or employee orientation:
Which matters more? The moderating role of firm size. Journal of Business &
Industrial Marketing, 33, 1001–1011. https://doi.org/10.1108/JBIM-05-2017-0119
Pilehvar, M. T., & Collier, N. (2016). Improved semantic representation for domain-
specific entities. In Proceedings of the 15th Workshop on Biomedical Natural Language
Processing (pp. 12-16).
Plepi, J., Flek, L. (2021). Perceived and intended sarcasm detection with graph attention
networks. arXiv preprint arXiv:2110.04001.
Quan, C. Q., & Ren, F. J. (2014). Unsupervised product feature extraction for feature-
oriented opinion determination. Information Sciences, 272, 16–28. https://doi.org/
10.1016/j.ins.2014.02.063
Rajadesingan, A., Zafarani, R., & Liu, H. (2015). Sarcasm detection on twitter: A
behavioral modeling approach. In Proceedings of the eighth ACM international
conference on web search and data mining (pp. 97–106). https://doi.org/10.1145/
2684822.2685316
Z. Wang et al.
Ray, A., Mishra, S., Nunna, A., Bhattacharyya, P. (2022). A multimodal corpus for
emotion recognition in sarcasm. arXiv preprint arXiv:2206.02119.
Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules.
Advances in neural information processing systems, 30. URL: https://proceedings.
neurips.cc/paper_files/paper/2017/hash/2cad8fa47bbef282badbb8de5374b894-Ab
stract.html.
Sarsam, S. M., Al-Samarraie, H., Alzahrani, A. I., & Wright, B. (2020). Sarcasm detection
using machine learning algorithms in Twitter: A systematic review. International
Journal of Market Research, 62, 578–598. https://doi.org/10.1177/
1470785320921779
Shen, C., Wang, A. N., Fang, Z., & Zhang, Q. (2021). Trend mining of product
requirements from online reviews. Chinese Journal of Management Science, 29,
211–220. https://doi.org/10.16381/j.cnki.issn1003-207x.2018.1508
Sherstinsky, A. (2020). Fundamentals of recurrent neural network (RNN) and long short-
term memory (LSTM) network. Physica D: Nonlinear Phenomena, 404, Article
132306. https://doi.org/10.1016/j.physd.2019.132306
Sun, L., Chen, J., Li, J., & Peng, Y. (2015). Joint topic-opinion model for implicit feature
extracting. In 2015 10th International Conference on Intelligent Systems and Knowledge
Engineering (ISKE) (pp. 208-213). https://doi.org/10.1109/ISKE.2015.17.
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011). Lexicon-based
methods for sentiment analysis. Computational linguistics, 37, 267–307. https://doi.
org/10.1162/COLI_a_00049
Tang, D., Qin, B., Feng, X., & Liu, T. (2015). Effective LSTMs for target-dependent
sentiment classification. Computer Science, arXiv preprint arXiv:1512.01100.
Tang, X., Lin, W., Du, Y., & Wang, T. (2019). Short text feature extraction and
classification based on serial-parallel convolutional gated recurrent neural network.
Advanced Engineering Sciences, 51, 125–132. https://doi.org/10.15961/j.
jsuese.201801160
Tubishat, M., Idris, N., & Abushariah, M. (2018). Implicit aspect extraction in sentiment
analysis: Review, taxonomy, opportunities, and open challenges. Information
Processing & Management, 54, 545–563. https://doi.org/10.1016/j.ipm.2018.03.008
Turney, P. D., & Littman, M. L. (2003). Measuring praise and criticism : Inference of
semantic orientation from association. Acm Transactions on Information Systems, 21,
315–346.
Wang, X., & Lu, Z. (2014). Extracting product features from online reviews based on two-
level HHMM. In Computer & Information Technology (pp. 1-4). https://doi.org/
10.1145/944012.944013.
17
Expert Systems With Applications 241 (2024) 122512
Wawre, S. V., & Deshmukh, S. N. (2016). Sentiment classification using machine learning
techniques. International Journal of Science and Research (IJSR), 5, 819–821.
Wang, W., Feng, Y., & Dai, W. (2018). Topic analysis of online reviews for two
competitive products using latent Dirichlet allocation. Electronic Commerce Research
and Applications, 29, 142–156. https://doi.org/10.1016/j.elerap.2018.04.003
Wang, R., Wang, Q., Liang, B., Liang, B., Chen, Y., Wen, Z., Qin, B., Xu, R. (2022).
Masking and generation: An unsupervised method for sarcasm detection. In
Proceedings of the 45th International ACM SIGIR Conference on Research and
Development in Information Retrieval (pp. 2172-2177). https://doi.org/10.1145/
3477495.3531825.
Yang, A. M., Zhou, Y. M., & Lin, J. H. (2013). A method of Chinese texts sentiment
classification based on Bayesian algorithm. In International Conference on Information
Technology and Management Innovation (ICITMI2012) (Vol. 263, pp. 2185-2190).
https://doi.org/10.4028/www.scientific.net/AMM.263-266.2185.
Zeng, Y. C., Ku, T., Wu, S. H., Chen, L. P., & Chen, G. D. (2014). Modeling the helpful
opinion mining of online consumer reviews as a classification problem. International
Journal of Computational Linguistics & Chinese Language Processing, 19, 17–31.
Zha, Z., Yu, J., Tang, J., Wang, M., & Chua, T. (2014). Product aspect ranking and its
applications. IEEE Transactions on Knowledge & Data Engineering, 26, 1211–1224.
https://doi.org/10.1109/TKDE.2013.136
Zhang, Y. B., Zhang, Z. F., Miao, D. Q., & Wang, J. Q. (2019). Three-way enhanced
convolutional neural networks for sentence-level sentiment classification.
Information Sciences, 477, 55–64. https://doi.org/10.1016/j.ins.2018.10.030
Zhang, Y. F., Shen, Y., & Jing, L. Y. (2021). Research on custom demand mining of short-
life-cycle electronic products. Software Guide, 20, 217–222.
Zhou, L., & Lin, J. (2018). Research on product feature extraction based on noderank
algorithm. Data Analytical Knowledge Discovery, 2, 90-98. https://doi.org/10.11925/
infotech.2096-3467.2017.1252.
Zhou, F., Ayoub, J., Xu, Q., & Jessie Yang, X. (2020). A machine learning approach to
customer needs analysis for product ecosystems. Journal of Mechanical Design, 142,
Article 011101. https://doi.org/10.1115/1.4044435
Zhang, Y., Liu, Y., Li, Q., Tiwari, P., Li, Y., Pandey, H., … Song, D. (2021). CFN: A
complex-valued fuzzy network for sarcasm detection in conversations. IEEE
Transactions on Fuzzy Systems, 29, 3696–3710. https://doi.org/10.1109/
TFUZZ.2021.3072492