European Journal of Operational Research 282 (2020) 957–971

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Decision Support

Adaptive consensus reaching process with hybrid strategies for

large-scale group decision making
Ming Tang a, Huchang Liao a,∗, Jiuping Xu a, Dalia Streimikiene b, Xiaosong Zheng c
a
Business School, Sichuan University, Chengdu 610064, China
b
Kaunas Faculty, Vilnius University, Muitines 8, Kaunas LT-44280, Lithuania
c
SHU-UTS SILC Business School, Shanghai University, Shanghai 201899, China

a r t i c l e i n f o a b s t r a c t

Article history: Large-scale group decision making, which involves dozens to hundreds of experts, is attracting increasing
Received 4 December 2018 attention and has become an important topic in the field of decision making. Because of the clustering
Accepted 2 October 2019
process, a large-scale group decision making problem can be divided into two levels: inter sub-group and
Available online 10 October 2019
intra sub-group. In existing consensus models under the large-scale group decision making environment,
Keywords: the degree of consensus within the intra sub-group is not truly taken into account. To deal with this
Decision support systems issue, this work develops an adaptive consensus model for the sub-groups composed of hybrid strategies,
Large-scale group decision making with or without a feedback mechanism, according to the different levels of inter and intra degrees of
Adaptive consensus consensus. These different levels of consensus are divided into four scenarios (high–high, high–low, low–
Hybrid strategies high, low–low), and different feedback suggestions are generated corresponding to different cases. This
Reciprocal comparison matrices hybrid mechanism can reduce the cost of supervision for the moderator. The fuzzy c-means clustering
algorithm is used to classify experts. A weight-determining method combining the degree of cohesion and
the size of a sub-group is introduced. Finally, an illustrative example is offered to verify the practicability
of the proposed model. Some discussions and comparisons are provided to reveal the advantages and
features of the proposed model.
© 2019 Elsevier B.V. All rights reserved.

1. Introduction unrealistic. Many scholars have softened this strict definition

With the rapid expansion of technological paradigms such as
e-democracy (Efremov, Insua & Lotov, 20 09; Kim, 20 08) and social
networks (Liang, Liao & Liu, 2017), a decision-making problem may
include a large number of experts. Consequently, large-scale group
decision making (LSGDM), which includes dozens to hundreds
of experts, is attracting increasing attention and has become an
important topic in the decision-making field. Unlike the traditional
group decision making (GDM) problems in which few experts
are involved, there are two mainly used standards to identify
the LSGDM problems regarding the number of experts: one is 11
experts (Xu, Du & Chen, 2015; Zhang, Dong & Herrera-Viedma,
2018) and the other is 20 experts (Wu & Xu, 2018). In this study,
we use the latter standard. That is to say, a GDM problem can be
considered as an LSGDM problem if at least 20 experts participate.
Consensus is an important issue in GDM. Strictly speaking,
consensus means a unanimous agreement among all partici-
pants concerning all alternatives. However, this concept is often
∗
Corresponding author.
E-mail address: liaohuchang@163.com (H. Liao).
(Wu & Xu, 2016, 2018; Xu, Du et al., 2015; Zhang et al., 2018).
Kacprzyk and Fedrizzi (1986, 1988, 1989) conducted a series of
work on soft consensus. A review on soft consensus can be found
in Herrera-Viedma, Cabrerizo, Kacprzyk and Pedrycz (2014). Never-
theless, most existing consensus models (Chen, Lee, Yang & Sheu,
2012; Dong & Cooper, 2016; Gupta, 2018; Labella, Liu, Rodríguez
& Martínez, 2018) only considered small-scale experts, and thus,
may not be appropriate for handling LSGDM problems because
of the high adjustment cost or time limitation. A comprehensive
analysis of the limitations of classical consensus models can be
found in Labella et al. (2018).
In practice, it is a challenge to reach an agreement with
all experts owing to their different attitudes, motivations, and
perceptions. Thus, the consensus reaching process (CRP) is es-
sential to obtain the best solution agreed by all experts in GDM.
Scholars have developed different CRP methods and structures
in different contexts, such as the CRP in a dynamic/web context
(Pérez, Cabrerizo & Herrera-Viedma, 2010, 2011), CRP in a social
network (Capuano, Chiclana, Fujita & Herrera-Viedma, 2018),
consensus with minimum cost (Cheng, Zhou, Cheng, Zhou & Xie,
2018), and adaptive CRP (Chen et al., 2012; Dong & Cooper, 2016;

https://doi.org/10.1016/j.ejor.2019.10.006
0377-2217/© 2019 Elsevier B.V. All rights reserved.
958 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Gupta, 2018; Mata, Martínez & Herrera-Viedma, 2009; Pang, Liang
& Song, 2017; Pérez, Cabrerizo, Alonso & Herrera-Viedma, 2014;
Rodríguez, Labella, Tré & Martínez, 2018; Zhang, Zhu, Liu, Chen &
Ma, 2017). Among these models, the adaptive CRP can generate
recommendations adaptively by considering dynamic parameters
such as levels of group consensus (Mata et al., 2009) and the
weights of experts (Pérez et al., 2014). For instance, Mata et al.
(2009) proposed a consensus model in which the amount of
recommendations and the number of experts involved in each
iteration were adapted to different levels of group consensus. At
the initial iteration stage, given that the level of consensus is
very low, all the experts are advised to modify their evaluation
information. With the level of consensus increases, both the num-
ber of changes and the number of experts who need to modify
their evaluations decrease. This adaptive CRP was based on four
categories of consensus levels: very low, low, medium, and high.
Pérez et al. (2014) introduced a CPR that adapted to the weights
of experts. The logic of this model is that the experts with higher
weights have greater expertise, and thus, they can be asked to
perform a smaller amount of modifications in the CRP.
The existing methods can be used to handle GDM problems
under different circumstances. However, there are still some
limitations:
(1) The clustering process plays a vital role in solving LSGDM
problems, and this is the main difference between general
GDM and LSGDM. Determining a cluster’s weight is critical
in LSGDM. Most of the existing studies (Wu & Xu, 2018; Xu,
Du et al., 2015; Zhang et al., 2018) simply used the num-
ber of experts in a cluster as the standard to determine the
weights of clusters. In this way, the clusters with the same
number of experts but possessing different inner character-
istics in terms of cohesiveness or diversity would still have
the same weights. Thus, we need to develop a method to de-
termine the weights of different clusters that considers both
the degree of cohesion and the size of a cluster.
(2) The use of the adaptive consensus model in an LSGDM con-
text is still at an initial exploratory stage, and thus, it faces
big challenges. In LSGDM, owing to the various backgrounds
and expertise of experts, it is more difficult to reach a con-
sensus result than in conventional GDM. Thus, it is necessary
to research the adaptive consensus model within a large-
scale context. Although Rodríguez et al. (2018) introduced an
adaptive consensus model under the LSGDM context similar
to the method of Mata et al. (2009), their model classified
the global consensus into three levels: low, high, and high
enough. Nevertheless, this model did not take into account
the intra consensus of a sub-group. In LSGDM, the cluster-
ing process should be used to cluster the experts into sev-
eral groups based on their preference information to sim-
plify the decision-making process. As a result, two levels of
consensus are generated, i.e., the level of consensus within
a sub-group and the level of consensus of a sub-group to
the global group. These two kinds of consensus are respec-
tively named as the intra consensus and inter consensus in
this study. The existing studies about LSGDM did not gen-
erate feedback suggestions considering these two kinds of
consensus at the same time.
To address these shortcomings, in this study, we develop a
novel adaptive consensus method under the LSGDM context.
This method first uses the fuzzy c-means (FCM) clustering al-
gorithm to classify the experts into several clusters. Then, a
weight-determining method for clusters is proposed. Afterwards,
the consensus measures corresponding to the intra consensus and
inter consensus are defined. Based on these consensus measures,
an adaptive feedback mechanism, composed of hybrid consensus
Table 1
List of notations.
Notations Meanings
X = {x1 , x2 , . . . , xn } A set of n alternatives
E = {e1 , e2 , . . . , em } A set of m experts
P k = ( pki j )n×n The reciprocal comparison matrix (RCM) of the kth expert
{C 1 , C 2 , . . . , C N } N clusters
ch Cluster center
μch,t (Pk ) Membership degree of P k to the selected centroid ch,t
d (P k , ch,t ) Distance between an RCM P k and the cluster center ch,t
#C h,t Number of experts in cluster C h,t
s(C h ) Cohesion degree of cluster C h
ω (C h ) Normalized priority vector of cluster C h
CL(P k ) Consensus level of expert ek
SCL Consensus level of a sub-group
SCL Threshold of SCL
ICL Inter consensus level of a sub-group to the global group
GCL Global consensus level
GCL Threshold of GCL
GSC Global/sub-group consensus
c pi j Degree of consensus on a position
c ai Degree of consensus on an alternative
strategies with or without a feedback mechanism according to the
different degrees of inter and intra agreements, is presented in
detail. Finally, a case study is provided to better understand the
theory of this study.
The primary contributions of this work are summarized as
follows:
(1) A weight-determining method for clusters is developed. This
method considers both the size and the cohesion degree of
a cluster, and thus improves the weight-determining method
that only considers the size of a cluster. Furthermore, a co-
hesive cluster will exert much influence in the global group,
and thus this method can accelerate the convergence speed
of the CRP objectively since the aim of the CRP is to make
the group information central. An aggregation formula is
proposed to integrate these two parameters. The degree of
cohesion of a cluster includes the silhouette coefficient.
(2) According to the different levels of inter and intra consensus
of a sub-group, this study develops an adaptive consensus
model with hybrid strategies for multiple groups. This model
can generate different recommendations according to the
degrees of consensus of two levels. These different strategies
can reduce the numbers of involved experts and the recom-
mendations for modifications in the CRP within the LSDGM
context. Thus, the proposed method can reduce the supervi-
sion and adjustment costs of the CRP.
(3) An illustrative example about the legislative amendment of
the International Trade Law within the United Nations sys-
tem is provided to explore and verify the applicability and
feasibility of the proposed model. Then, comparisons regard-
ing different parameters including the number of clusters
and the consensus threshold are provided. We also give a
comparison with Rodríguez et al. (2018)’s method to validate
the advantage of the adaptive mechanism.
To achieve the above goals, this paper is organized as follows:
Section 2 describes some concepts used in this study, including
the LSGDM, CRP, and FCM. Section 3 presents a novel adaptive
consensus model under the LSGDM context. In Section 4, an
illustrative example is included to verify the applicability and
feasibility of this model. Some comparisons and discussions are
also provided. Concluding remarks are presented in Section 5.
To facilitate the comprehension of the paper, Table 1 summa-
rizes some of the used notations.
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Problem
Problem Experts
Experts
Alternatives
Altern
r atives
Preference
Prefe
f rence
relations
relations
Fig. 1. General scheme of the CRP.
2. Preliminaries
In this section, we review the tools employed to build the
proposed model, that is, the concepts of LSGDM, CRP, and FCM.
2.1. Large-scale group decision making (LSGDM)
The concept of LSGDM, which is evolved from GDM, has been
researched by many scholars because of the increasing complexity
of the decision environment and the evolution of technological
paradigms. The LSGDM differs from the general GDM in the
number of experts. Formally, the elements of an LSGDM problem
consist of
1) a set of alternatives, X = {x1 , x2 , . . . , xn }, (n ≥ 2 ), which are as-
sessed as possible solutions for a problem.
2) a set of experts, E = {e1 , e2 , . . . , em }, (m ≥ 20 ), who express
their preferences on alternatives.
If a pairwise comparison matrix P k = ( pki j )n×n satisfies
pi j ∈ [0, 1], pi j + p ji = 1, ∀i, j ∈ 1, 2, . . . , n, then P k = ( pki j )n×n is
called a reciprocal comparison matrix (RCM) (or reciprocal pref-
erence relation) (Switalski, 1999). pi j in the RCM indicates the
preference degree of xi to x j . pi j > 0.5 denotes that xi is preferred
to x j ; pi j < 0.5 denotes that x j is preferred to xi ; pi j = 0.5 denotes
that xi is indifferent to x j . In the process of decision making,
preference relation is one of the most widely used representations
of information (Herrera-Viedma, Herrera, Chiclana & Luque, 2004;
Liu, Zhang & Zhang, 2014; Wu & Liao, 2019). If the number of
alternatives is small, this representation is efficient. In this study,
we use four alternatives in the illustrative example.
2.2. Consensus reaching process (CRP)
Considering its similar structure to general GDM problems, the
LSGDM problems can be solved by a selection process with an ag-
gregation phase and an exploitation phase. However, the result of
the selection process cannot guarantee that it can be supported by
all experts because their opinions may be not sufficiently consid-
ered. Under such circumstance, a CRP should be applied in LSGDM.
The CRP in GDM or LSGDM is an iterative process with discussion
rounds, in which experts can receive recommendations from a
moderator. The general scheme of the CRP is illustrated in Fig. 1.
1) Collecting preferences. The preferences evaluated by experts are
collected.
959
Moderator
Moderator
Feedback
Feedback recommendations
recomm
mendations
Low
ow consensus
consensus
level
level High
High consensus
consensus
level
level
Consensus
Consensuus
control
contro
ol
Selection
Selection
Calculating
alculating process
process
consensus
us degree
consensu degreee
2) Calculating the degrees of consensus. The moderator calculates
the degrees of consensus of the group and individuals by using
a consensus measure based on distance or similarity measures.
3) Controlling consensus. If the degree of consensus does not reach
an expected level, discussion rounds should be performed; oth-
erwise, go to the selection process.
4) Generating feedback recommendations. The moderator provides
modification suggestions to the experts (or sub-groups in LS-
GDM) who have low consensus levels.
The moderator in the CRP does not participate in the discussion
among the experts, but he/she knows the level of consensus of the
whole group and the individual level of agreement in each round
of iteration. A moderator should perform the following tasks: (1)
calculating the degrees of consensus; (2) verifying the degrees of
consensus; (3) providing suggestions for the experts to modify
their preferences if they need to do so. In LSGDM problems,
there could be a heavy workload for the moderator because the
supervision and time costs may be very high.
2.3. A clustering algorithm: FCM
Clustering is the process of dividing a set of data objects into
multiple groups which we call clusters. Data objects in a cluster
are similar to those in the same cluster but are different from
those in other clusters (Bezdek, Ehrlich & Wull, 1984). In LSGDM,
clustering is essential to manage and analyze experts’ preferences.
Some clustering algorithms have been used in LSGDM, such as
the k-means algorithm (Wu & Xu, 2018) and broad first searching
neighbors algorithm (Zhang et al., 2018). However, these clustering
algorithms are crisp methods, where a data object can only belong
to one cluster strictly. Fuzzy clustering, with the assumption
that a data object can belong to different clusters with different
membership degrees, has been proposed to overcome this limita-
tion (Bezdek et al., 1984). The most widely used fuzzy clustering
algorithm is the FCM algorithm. The FCM algorithm aims to find a
locally optimal solution in which the difference between member-
ship degrees of a cluster in two consecutive iterations tends to 0.
Inspired by Palomares, Martinez and Herrera (2014), we adopt the
FCM algorithm to the context of this study. Algorithm 1 presents
the steps of the FCM algorithm with RCMs as input data.
Notes:
1) The fuzziness parameter b is the weighting exponent in
FCM to control the extent of sharing among clusters
960 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Algorithm 1
FCM algorithm with RCMs as input data.
1. Set the number of clusters N (N ≥ 2 ) and the fuzziness degree of a cluster b.
2. Initialize the centroid of each cluster.
3. While the terminating condition is not reached, do
4. Calculate the membership degree of each expert (associated with the RCM
P k ) to the selected centroid ch,t , μch,t (P k ) ∈ [0, 1], where t is the number of
iteration.
5. Update the centroid ch,t .
6. End while.
(Pal & Bezdek, 1995). The larger b is, the higher the degree
of fuzziness of the cluster should be (Palomares et al., 2014).
There have been many studies that focused on the selection
of b. Bezdek (1976) suggested that b = 2 through studying the
physical interpretation of FCM. Furthermore, Pal and Bezdek
(1995) held that the value of b should be limited in [1.5, 2.5]
based on a heuristic rule. They thought that the median value
b = 2 can be chosen without any particular constraints. This
parameter can be selected subjectively in practice. The most
widely used value for b is 2 (Zhou, Fu & Yang, 2014). Thus, in
this study, we set b = 2.
2) In this study, we use the max–min method proposed by
Gonzalez (1984) to identify initial cluster centers. This method
will be explained in Section 3.1.
3) The membership degree μch,t (P k ) is calculated based on dis-
tance measures, which will be illustrated in Section 3.1 in de-
tail.
4) The terminating condition of this algorithm will be presented
in Section 3.1.
3. An adaptive CRP within the LSGDM context
In this section, an adaptive consensus model for LSGDM prob-
lems with hybrid strategies for multiple groups is proposed. The
main innovation of this model is that it generates feedback rec-
ommendations according to the different levels of intra consensus
(consensus in a sub-group) and inter consensus (consensus of a
sub-group to the global group). The consensus measure is divided
into two levels in LSGDM because of the clustering algorithm.
The intra consensus level can reflect the degree of agreement of
opinions within a sub-group, while the inter consensus level can
reflect the degree of agreements of opinions between a sub-group
and the global group. When the intra and inter consensus levels
are high or low, the cohesion of a sub-group and the distance
between the cluster center and the collective preference point
vary. Therefore, according to four different scenarios of inter and
intra consensus levels, namely high–high, high–low, low–high, and
low–low, this study introduces four strategies to reach consensus.
These strategies can improve the CRP performance and reduce the
costs of supervision. The consensus model consists of four phases,
as presented below:
1) Using the FCM to classify experts into multiple sub-groups. In this
phase, all experts are classified into different clusters with re-
spect to membership degrees based on their RCMs.
2) Determining clusters’ weights. After the clustering process, it is
necessary to compute the weight of each sub-group. Two indi-
cators are considered: the size and the degree of cohesion of a
sub-group.
3) Computing the degrees of consensus. Degrees of agreement in the
two levels are calculated based on distance measures. If the
degree of consensus of a group is sufficiently high, then
the group goes to the selection process; otherwise, more dis-
cussions should be conducted.
4) Adapting feedback mechanism. Different strategies for provid-
ing adjusted recommendations in a consensus round are ap-
plied based on the classification of consensus levels of the two
levels, i.e., high–high, high–low, low–high, and low–low. Then,
the corresponding recommendations are generated for different
situations.
The whole structure will be illustrated in detail in the following
sections.
3.1. Classifying experts
The FCM algorithm is used to classify experts based on their
RCMs. Once all experts’ RCMs are gathered, this algorithm should
be used to identify sub-groups with similar opinions.
1) In the FCM algorithm, the number of clusters can be prede-
termined randomly. In this work, we set N as 4 because we
want to test and verify four different strategies in the pro-
posed model. A discussion about this topic will be included in
Section 5.1.
2) One way to choose the initial centroids is to select them ran-
domly. However, this method is sensitive to the initial points
and is vulnerable to noisy points. In this regard, we use the idea
of the max–min method (Gonzalez, 1984) and extend it to our
context. In the max–min method, the first cluster centroid c1 is
selected randomly. Then, the point that is at the greatest dis-
tance from the first centroid is selected as the second centroid
c2 . Computing the distances of the remaining points to the cur-
rent two centroids, the point that has the nearest distance is se-
lected as the third centroid c3 . This procedure is continued un-
til N centroids are selected. To avoid randomness and inspired
by the idea of the clustering algorithm published in Science in
2014 (Rodríguez & Laio, 2014), we use the point that has the
highest density as the first cluster center.
3) The membership degree μch,t (P k ) of P k to the selected centroid
ch,t is calculated by
1/(b−1 )
(1/d (Pk , ch,t ))
μch,t (Pk ) = N
k u,t 1/(b−1 )
u=1‘ (1/d (P , c ))
where d (P k , ch,t ) is a measure of distance. Here, we use the Eu-
clidean distance (Rodríguez et al., 2018), the most widely used
distance function in the FCM algorithm (Palarea-Albaladejo,
Martín-Fernández & Soto, 2012). It measures the real distance
between two points in m-dimensional space and has simple
calculation process. d (P k , ch,t ) is defined as follows:
 1 / 2
1 
n 
n
d (P , c
k h,t
)= | pki j − cih,tj |2
n (n − 1 )
i=1 j=1
If there are two or more coincident points, we select one
point randomly as the centroid. Then, the remaining coincident
points will be assigned to the cluster with this centroid associ-
ated with a membership degree of 1.
4) The RCM P i provided by expert ei is assigned to cluster C h,t if
P i has the maximum membership degree to its centroid ch,t :
C h,t (P k ) = arg max μch,t (P k )
h
If Pihas the same membership degree to two or more cen-
troids, then P i is assigned to a cluster randomly.
5) The updated centroids are calculated based on the RCMs in-
cluded in corresponding clusters:
1 
cih,t+1 = pk , i, j ∈ {1, 2, . . . , n}
j
#C h,t k h,t i j
P ∈C
where #C h,t is the number of experts in cluster C h,t .
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971 961

6) The FCM algorithm executes the termination condition when all
clusters are stabilized. This occurs when the difference between
membership degrees in two successive iterations approaches to
0, i.e.,
m N
k=1 h=1 |μch,t+1 (Pk ) − μch,t (Pk )|
≤ε
m·N
where ε is a parameter whose value approaches 0. In this work,
we set ε as 0.001.
3.2. Determining the weights of clusters
After classifying the experts into multiple sub-groups, the next
task is to assign weights to each cluster. One principle is that
the larger cluster with more experts should be assigned a larger
weight. Another indicator that should be considered is the degree
of cohesion of a cluster. The higher the degree of cohesion of
a cluster is, the larger the weight that should be given to this
cluster. To measure the degree of cohesion of a cluster, we use the
silhouette coefficient introduced by Rousseeuw (1987) and adapt
it to the FCM algorithm.
Let A(P k ) be the average dissimilarity of an RCM P k to all other
data objects in cluster C a . Dissimilarity is a type of proximity
that measures how far away two objects are from each other
(Rousseeuw, 1987). It is contrary to another widely used type of
proximity: similarity. Here, we use the Euclidean distance to define
the dissimilarity degree. Let d (P k , C h ) be the average dissimilarity
degree of P k to all data objects in cluster C h , h = a. Then, the
smallest value of d (P k , C h ) is selected as B(P k ), where
B(P k ) = min d (P k , C h ), P k ∈ C a
h=a
The silhouette coefficient of an RCM P k is obtained by:
 0.9 (Tian, Zhang & Ha, 2018), 0.85 (Rodríguez et al., 2018), 0.8 (Wu
1 − A(P k )/B(P k ), if A(P k ) < B(P k )
s (P ) =
k
0, if A(P k ) = B(P k )
B(P k )/A(P k ) − 1, if A(P k ) > B(P k )
or
B (P k ) − A (P k )
s (P k ) =
max{A(P k ), B(P k )}
It is easy to find that −1 ≤ s(P k ) ≤ 1. The closer the value of
s (P k )
is to 1, the better P k attached to its cluster should be.
To measure the degree of cohesion of a sub-group, we define
the silhouette coefficient of a cluster in the FCM as:
3.3. Computing the degrees of consensus
Expert ek provides preferences on a set of alternatives by an
RCM P k = ( pki j )n×n with n(n − 1 )/2 pairwise comparisons. Then, a
moderator computes the degrees of consensus of the expert group
by distance measures.
Let P k = ( pki j )n×n be an RCM provided by expert ek , and
P c = ( pci j )n×n be the group preference relation, which can be
represented by its centroid ci j . Then, the consensus level (CL) of
expert ek is defined as:
CL(P k ) = 1 − d (P k , P c ) (3)
where d (P k , P c ) is a distance measured between P k and P c .
The consensus level of a sub-group C h (SC L(C h )) is computed
by
1 
SC L(C h ) = C L (P k )
#C h k h
P ∈C
where #C h
is the number of experts in the sub-group C h .
If SC L(C h ) = 1, all experts in the sub-group C h have a unified
opinion. A larger value of SC L(C h ) represents a higher consensus
level among the experts in the cluster. SCL is a predefined thresh-
old for SC L(C h ). If SC L(C h ) ≥ SCL, then the sub-group has an ac-
ceptable consensus; otherwise, the CRP should be applied to reach
an acceptable degree of consensus. Note that SCL should be set
in advance, which represents the expected standard that a group
needs to achieve in LSGDM. In a real-world decision-making prob-
lem, a high value of SCL may be difficult to achieve or needs many
iterations. Thus, we use a parameter (max _round) to set the max-
imum number of iterations. The value of SCL can be determined
according to specific decision-making problems. In classical GDM
problems, the value of the predefined threshold is usually set as
& Xu, 2016), or 0.75 (Herrera-Viedma, Martínez & Mata, 2005). For
some vital and major issues, the minimum value of consensus level
should be high (Herrera-Viedma et al., 2005). However, in other
extreme cases, such as emergency problems, the consensus thresh-
old should be low (Xu, Du et al., 2015) because of the time limit.
In LSGDM problems, a low threshold is also needed because there
are great differences in experts’ background, knowledge, and ex-
pertise. For the reason that SCL is the consensus threshold within a
sub-group, in this study, we set SCL as 0.9, a relatively high value.
The inter consensus level (IC L(C h )) between a cluster C h and
the global preference relation P G = ( pGi j )n×n is defined as:

1  IC L(C h ) = 1 − d (P G , C h ) (4)
s(C h ) = s (P k )
#C h k h
P ∈C where PG
is obtained by aggregating all sub-groups’ preference
relations, such that
Given that s(C h )
can be used to measure the degree of cohesion
of a cluster C h and the number of experts in the cluster C h , #C h , 
N

can also be used to reflect the importance of the cluster, we pGi j = ω (C u ) · pui j
aggregate these two indicators as: u=1

Then, the global consensus level (GCL) can be obtained as

es(C )
h
#C h
ϕ (C h ) = N · N (1)
1
s( ) Cu u N
u=1 e u=1 #C
GC L = IC L(C h )
In this sense, the weight of a cluster can be determined as:
N
h=1

ϕ (C h ) It is obvious that 0 ≤ GCL ≤ 1. Similarly, a threshold GCL for

ω (C h ) = N , ∀h ∈ {1 , 2 , . . . , N } (2) GCL should be set in advance. If GCL ≥ GCL, then the global group
u=1 ϕ (C )
u
has an acceptable degree of consensus; otherwise, the CRP should
Since the value of s(C h ) may be negative, we use the expo- be applied to improve the consensus level. Based on this idea, we
nential function here. Eq. (1) emphasizes the impact of the degree can further define the degree of consensus at an alternative level
of cohesion when calculating the weight of a cluster, because to facilitate the generation of the feedback mechanism. Based on
the degree of cohesion in the weight is identified by the natural the former discussions, in this study, we set GCL as 0.8, a value
exponential function. lower than SCL.
962 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971

The degree of consensus at a position (i, j ) for the expert ek

can be defined as:
cpki j = 1 − | pki j − cihj |, k ∈ {1, 2, . . . , m}; i, j ∈ {1, 2, . . . , n}; ek ∈ C h
It is easy to know that cpki j = cpkji .
The degree of consensus on an alternative xi for an expert ek
is
n k
j=1, j=i c pi j
caki = (5)
n−1
Note that the model proposed in this study contains a single
criterion. If an LSGDM problem contains more than one criterion,
it can be regarded as a multi-criteria LSGDM problem. Our model
can also be extended to solve multi-criteria LSGDM problems. The
difference is that the consensus measure in multi-criteria LSGDM
is based on decision matrices. The degree of consensus of an
alternative can be obtained by measuring the evaluation values
regarding all criteria.
Fig. 2. Graphic display of the four scenarios.
3.4. Adaptive feedback mechanism

If the levels of agreement among all sub-groups do not reach

the predefined threshold, then feedback strategies need to be
applied. Different consensus models have been introduced to
generate recommendations for experts. In LSGDM, some consensus
methods have also been proposed concerning different topics
such as non-cooperative behaviors (Palomares et al., 2014) and
heterogeneous preference structures (Zhang et al., 2018). However,
these models do not take into account the simultaneous consensus
of two levels (SCL and GCL). The first level is the inter consensus
between a sub-group and the global group, while the second level
is the intra consensus within a sub-group. To deal with this issue,
this section introduces an adaptive consensus model with hybrid
strategies for multiple groups. Specifically, we divide the CRP
into four categories: high–high, high–low, low–high, and low–low.
Two thresholds, GCL and SCL, whose values depend on different
decision-making problems, are determined before the CRP. To find
controversial preferences for multiple groups {C 1 , C 2 , . . . , C N }, the
following four categories are identified:
1) High–high: IC L(C h ) ≥ GCL and SC L(C h ) ≥ SCL, h ∈ {1, 2, . . . , N}; Fig. 3. Graphic display of S-HL.
2) High–low: IC L(C h ) ≥ GCL and SC L(C h ) < SCL, h ∈ {1, 2, . . . , N};
3) Low–high: IC L(C h ) < GCL and SC L(C h ) ≥ SCL, h ∈ {1, 2, . . . , N};
4) Low–low: IC L(C h ) < GCL and SC L(C h ) < SCL, h ∈ {1, 2, . . . , N}. To find out the set of preferences that should be changed
by the experts in a sub-group, C h ∈ S−HL, the following strategy
A simple graphic that displays these four scenarios is provided should be implemented:
in Fig. 2. The blue points in a red circle represent a cluster and
the green point is the cluster center. The biggest red point is the i) Initially, we need to identify the experts, EX P S−HL , that should
global collective point. make modifications in a sub-group. It is computed by
Next, four strategies corresponding to the four scenarios are
explained in detail: EX P S−HL = {ek |CL(P k ) < SCL} (6)
1) Strategy for high–high consensus (S-HH). After the clustering pro-
ii) For any expert ek ∈ EX P S−HL , we then identify the alternatives
cess, some sub-groups are cohesive. At the same time, their
that should be modified by
centroids are near the global collective point, that is, they have
high degrees of consensus with the global group. In this situa- ALT S−HL = {xi |cai < γ S−HL ∧ ek ∈ EX P S−HL } (7)
tion, there is no need for them to modify their preferences.
2) Strategy for high–low consensus (S-HL). If the SC L(C h ) of a sub- In this scenario, the alternatives ALT S−HL that have a smaller
group C h does not reach the threshold SCL while its inter con- degree of consensus than the predefined threshold γ S−HL are iden-
sensus level IC L(C h ) does, then this sub-group is diverse but its tified. That is, the rows of the preference matrices corresponding
centroid is near to the global collective point. In this scenario, to the sub-group C h ∈ S−HL that need to be changed are identified.
for the purpose of reducing adjustment costs, there is no need The value of γ S−HL may be fixed before the CRP or be dynamic
to require all the experts in the sub-group to make modifica- according to the specific level of consensus in each iterative
tions according to the global collective preference relation. The discussion. The selection of γ S−HL plays a vital role in the CRP. A
main task of the moderator in this situation is to make the sub- high value of γ S−HL may need a large quantity of modifications
group become coherent, that is, to make experts’ RCMs con- while a low value may result in a slow convergence. Therefore, a
verge to its centroid. The graphic display for S-HL is provided dynamic γ S−HL adapting the level of agreement of each iterative
in Fig. 3. round is appropriate. In this study, we use the average degree of
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Fig. 4. Graphic display of S-LH.
consensus at the alternative level as γ S−HL , such that

n
γ S−HL = cai /n
i=1
iii) For any alternative xi ∈ ALT S−HL , we then identify the positions
to be modified:
P OSS−HL = {(i, j )|c pi j < γ S−HL ∧ xi ∈ ALT S−HL }
Here, γ S−HL denotes the average degree of consensus
at the position level. Because the degree of consensus at
an alternative level cai is calculated based on all positions
associated with the alternative xi (see Eq. (5)), we have
n n n
i=1 ( j=1,i= j c pi j )/ (n − n ) =
2
i=1 cai /n in the same prefer-
ence relation. We use the same symbol to represent the average
degree of consensus at the position level as that in Eq. (8).
Generation of advice. This strategy provides suggestions for each ex-
pert in a sub-group. The direction rules are described as follows:
a) DR. 31: If pki j < cihj , then expert ek should increase the assess-
ments associated with pairs of alternatives (xi , x j );
b) DR. 32: If pki j > cihj , then expert ek should decrease the assess-
ments associated with pairs of alternatives (xi , x j ).
3) Strategy for low–high consensus (S-LH). If the SC L(C h ) of a sub-
group C h reaches the threshold SCL while its inter consensus
level IC L(C h ) does not, then this sub-group is cohesive but its
centroid is far from the global collective point. In this situation,
there is no need to provide personalized recommendations for
all of the experts in the sub-group. To reduce supervision and
adjustment costs, it is appropriate to provide a unified sugges-
tion for the sub-group. Then, this sub-group with its centroid
can converge toward the global center point. The graphic dis-
play for S-LH is included in Fig. 4.
To find out the set of preferences that should be changed by
each sub-group, C h ∈ S−LH, this strategy is developed as follows:
i) In this scenario, it is not necessary to identify the experts who
contribute less in reaching a high level of consensus. First, the
alternatives ALT S−LH that have a smaller degree of consensus
than the predefined threshold γ S−LH are identified as
ALT S−LH = {xi |cai < γ S−LH }
It is noted that the symbol γ S−LH
has the same calculation as
that in Eq. (8). The difference is that γ S−LH here is on the level
of a sub-group to the global group, while the former is on the
level of intra sub-group.
963
Fig. 5. Graphic display of S-LL.
ii) The positions P OSS−LH to be modified are identified as
P OSS−LH = {(i, j )|c pi j < γ S−LH ∧ xi ∈ ALT S−LH } (11)
(8)
Generation of advice. Once the positions that need to be mod-
ified are identified, the direction rules that are used to give
adjustment suggestions should be provided by the moderator. The
directions rules are recommended as follows:
(9)
a) DR. 21: If cihj < pGi j , then all experts in the sub-group Ch
should increase the assessments associated with pairs of al-
ternatives (xi , x j ).
b) DR. 22: If cihj > pGi j , then all experts in the sub-group Ch
should decrease the assessments associated with pairs of al-
ternatives (xi , x j ).
4) Strategy for low–low consensus (S-LL). If the SC L(C h ) of a sub-
group C h does not reach the threshold SCL, nor does its level of
consensus to the whole group IC L(C h ), then this sub-group is
diverse and its centroid is far from the global collective point.
In this scenario, the moderator needs to provide suggestions to
make the sub-group’s centroid converge to the global collective
point and also make the sub-group coherent. It seems reason-
able to use both the global collective point and the cluster’s
center point to control the adjustment directions. The graphic
display for S-LL is included in Fig. 5. To do this, we define a
new measure, called the global/sub-group consensus (GSC),
which is achieved by a control parameter δ :
GSC = (1 − δ )CL(P k ) + δ IC L(C h ) (12)
where CL(P k ) is the degree of consensus of expert to ek
the sub-group’s centroid, which can be obtained by Eq. (3);
IC LG (C h ) is the degree of consensus of sub-group C h to the
global collective point, which can be obtained by Eq. (4).
δ ∈ [0, 1] is a parameter to control the weight of two levels
of consensus. Usually δ > 0.5 can be used to assign more
importance to the global consensus level.
To find out the set of preferences that should be changed
by experts in a sub-group, C h ∈ S − LL, the following strategy
should be implemented:
i) Initially, we identify the experts, EX P S−LL , who should make
modifications in a sub-group. It is computed by
(10)
EX P S−LL = {ek |(1 − δ )C LSub (P k ) + δ IC LG (C h ) < (1 − δ )SCL
+ δ GCL} (13)
The consensus threshold here is dynamic. It takes into account
two levels of consensus at the same time.
964 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Algorithm 2
Adaptive CRP with hybrid strategies for LSGDM with RCMs.
Input: The original individual RCMs P k = ( pki j )n×n , the given consensus
thresholds, SCL and GCL, and the parameter δ .
Output: The ranking of alternatives.
Step 1: Classify experts into several clusters {C 1 , C 2 , . . . , C N } via the FCM
algorithm.
Step 2: Determine the weights of clusters by Eq. (2).
Step 3: Calculate the levels of consensus of the two levels, SCL and GCL.
Step 4: If GCL ≥ GCL, then go to the selection process; otherwise, identify
sub-groups’ categories and go to the next step.
Step 5: Generate hybrid feedback adjustment rules to help experts modify
their preferences.
Step 6: Derive priority vector from the global collective RCM and rank the
alternatives
Step 7: End.
ii) For any expert ek ∈ EX P S−LL , this step identifies the alternatives
that should be modified:
ALT S−LL = {xi |(1 − δ )caSub
i + δ caGi < γ S−LL ∧ ek ∈ EX P S−LL } (14)
 
where γ S−LL = ( (1 − δ ) ni=1 caSub
i
+ δ ni=1 caGi )/n.
iii) Finally, for any alternative xi ∈ ALT S−LL , this step identifies the
positions to be modified:
P OSS−LL = {(i, j )|(1 − δ )cpSub
i j + δ cpi j < γ
G S−LL
∧ xi ∈ ALT S−LL }
(15)
Generation of advice. In this step, the moderator provides per-
sonalized recommendations to help the experts to modify their
preferences. The new preference value on a pair of alternatives
(xi , x j ) that should be modified by the expert is generated as
r pi j = (1 − δ )ciSub
j + δ pi j
G
(16)
Finally, the direction rules are implemented as follows:
a) DR. 1: If pki j < r pki j , then expert ek should increase the assess-
ments associated with pairs of alternatives (xi , x j );
b) DR. 2: If pki j > r pki j , then expert ek should decrease the assess-
ments associated with pairs of alternatives (xi , x j ).
Note. The value of δ can reflect different degrees of emphasis
concerning the consensus of the two levels. As mentioned be-
fore, when δ > 0.5, the consensus model leads experts toward the
global collective point rather than toward the sub-group’s centroid.
If δ = 0, then GSC = CL(P k ). In this case, the global collective point
is not used to guide experts to modify their preferences. This case
goes into the third strategy S-HL. If δ = 1, then GSC = IC L(C h ) and
the cluster center point is not used to guide experts to modify
their preferences. This case goes into the second strategy S-LH.
Therefore, the second and third scenarios can be regarded as two
extreme cases of the fourth scenario. Furthermore, the parameter
δ can also work when generating advice in the fourth scenario
(see Eq. (16)). Different values of δ can reflect different emphases.
3.5. An adaptive CRP with hybrid strategies for LSGDM with RCMs
Based on the above analyses, we provide the following algo-
rithm for LSGDM with RCMs. The flow chart of the algorithm is
illustrated intuitively in Fig. 6.
4. Illustrative example
This section presents an illustrative example to verify the
applicability of the proposed adaptive CRP with hybrid strategies
for LSGDM problems with RCMs.
Table 2
Weights of clusters.
Cluster Degree of cohesion Size Weight
C1 0.6286 6 0.3422
C2 0.4700 3 0.1460
C3 0.4560 6 0.2879
C4 0.3867 5 0.2239
4.1. Case description
The United Nations Commission on International Trade Law
(UNCITRAL) is the only core legal institute in the field of inter-
national trade law within the United Nations system. Since the
end of the 20th century, trade tensions between countries around
the world are increasing, and trade disputes have intensified.
UNCITRAL has made a great contribution for the unification of leg-
islation on international trade law. However, with the rapid devel-
opment of economic globalization, a growing number of countries
are involved. This makes the development of the international
trade law more diversified and the conflicts between different
legal systems more acute. The task of unification of legislation on
international trade is facing big challenges and many problems
have arisen, such as fuzzy description, delayed legislative tools,
and vague terms. Therefore, UNCITRAL has taken some measures
to solve these problems. One way is to speed up the process of
democratic legislation. UNCITRAL absorbs recommendations from
countries all over the world in legislative planning, text drafting,
and draft reviewing. The main forms of democratic legislation
are international academic conferences, specialist symposiums,
feasibility study meetings, and non-government organizations.
Suppose that four problems existing in old laws need to be
amended:
x1 : Unified possibility of digital currency rules;
x2 : Unification of drug cross-border trading rules;
x3 : Feasibility of unification of the International Trade Contract
Law;
x4 : Legal principle of the confidentiality characteristics of Inter-
national Commercial Arbitration.
To select the alternative that needs to be solved firstly, UNCI-
TRAL invites 20 experts E = {e1 , e2 , . . . , e20 } from 20 different
countries to evaluate these alternatives. The experts provide their
preferences by RCMs. To save space, we put these 20 RCMs in the
Appendix.
As mentioned before, for vital and major issues, the consensus
level should be high. Therefore, in this case, the parameters are
established as GCI = 0.8, SCI = 0.9, and δ = 0.7.
4.2. Solving the problem by the large-scale adaptive CRP with hybrid
strategies
To obtain the problem’s final solution, the proposed large-scale
adaptive CRP with hybrid strategies is applied.
First round.
Step 1 Using the FCM algorithm, the 20 experts are clas-
sified into 4 sub-groups: C 1 = {e1 , e2 , e3 , e4 , e5 , e6 },
C 2 = {e7 , e8 , e9 }, C 3 = {e10 , e11 , e12 , e13 , e14 , e15 } and
C = {e , e , e , e , e20 }.
4 16 18 17 19
Step 2 Eq. (2) is used to obtain the weight of each cluster. Table
2 shows the degree of cohesion, size, and weight of each
cluster.
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971 965

Start

Formulate a LSGDM problem

Problem formulation and data
input process
Collect individual RCMs

Classify experts by FCM

Cohesion Size

Determining clusters weights by (2)

Computing consensus levels

Consensus reaching process

YES
GCL ≥ GCL

NO

High- High- Low- Low- GCL

high low high low SCL

Generating feedback suggestions

Deriving priority vector from the global

collective RCM
Selection process
Ranking alternatives

Finish

Fig. 6. Procedure of Algorithm 2.

⎧ ⎫
Step 2 The RCMs of the four subgroups and the global group are: ⎪
⎨0.50 0 0 0.4667 0.1667 0.2167 ⎪
⎬
0.5333 0.50 0 0 0.2667 0.2833
PC2 = ,
⎩ 0.8333
⎪ 0.7333 0.50 0 0 0.5333 ⎪
⎭
0.7833 0.7167 0.4667 0.50 0 0
⎧ ⎫ ⎧ ⎫
⎪
⎨0.50 0 0 0.6833 0.8083 0.7500⎪
⎬ ⎪
⎨0.50 0 0 0.2917 0.3417 0.5333 ⎪
⎬
0.3167 0.50 0 0 0.7167 0.6417 0.7083 0.50 0 0 0.5750 0.8083
PC1 = , PC3 = ,
⎩ 0.1917
⎪ 0.2833 0.50 0 0 0.4083⎪
⎭ ⎩ .6583
⎪ 0 0.4250 0.50 0 0 0.6500⎪
⎭
0.2500 0.3583 0.5917 0.50 0 0 0.4667 0.1917 0.3500 0.50 0 0
966 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971

Table 3
Degrees of consensus of the two levels.

GCL: 0.7889

C1 C2 C3 C4

ICL 0.8358 0.7361 0.8013 0.7825

SCL 0.9370 0.9044 0.9107 0.8860
CL e1 : 0.9400, e2 :0.9448, e3 :0.9274 e7 :0.9201, e10 : 0.8196, e11 :0.9431, e16 :0.9215, e17 :0.9173
e4 :0.8770, e5 :0.9799, e6 :0.9530 e8 :0.8798, e12 :0.9407, e13 :0.9320, e18 :0.8134,e19 : 0.9087
e9 :0.9134 e14 :0.9019, e15 : 0.9270 e20 :0.8690

⎧ ⎫
⎪
⎨0.50 0 0 0.7900 0.7400 0.4200⎪
⎬
Table 4
Identification rules for sub-group C 4 .
0.2100 0.50 0 0 0.4400 0.1800
PC4 = ,
⎪ 0.2600 0.5600 0.50 0 0 0.2300⎪ γ S−LL ALT S−LL POSS−LL
⎩ ⎭
0.5800 0.8200 0.7700 0.50 0 0 e16
0.8388 x2 , x4 (2,1), (2,4), (4,2)
⎧ ⎫ e17 0.8408 x4 (4,2), (4,3)
⎪
⎨0.50 0 0 0.5600 0.5613 0.5334 ⎪
⎬ e18 0.8048 x1 , x2 , x3 , x4 (1,2), (1,3), (2,1), (2,4), (3,2), (3,4), (4,2), (4,3)
0.4400 0.50 0 0 0.5465 0.5343 e19 0.8358 x2 , x4 (2,1), (2,4), (4,2)
PG =
⎩0.4387
⎪ 0.4535 0.50 0 0 0.4577 ⎪
⎭ e20 0.8218 x2 , x4 (2,1), (2,4), (4,2), (4,3)
0.4666 0.4657 0.5423 0.50 0 0
Step 3 Compute the level of consensus of the two levels. Table 3
The experts in sub-group C 4 are required to modify their
presents the degrees of consensus in the first round.
preference assessments as follows:
Step 4 Given that GCL = 0.7888 < 0.8, the feedback mechanism
Expert e16 should increase his assessments for positions (2,1)
should be applied. These four sub-groups’ categories are
and (2,4), and decrease his assessment for position (4,2).
identified as:
Expert e17 should decrease his assessments for positions (4,2)
Step 4 IC L(C 1 ) = 0.8358 > 0.8, SC L(C 1 ) = 0.9370 > 0.9, then C 1 ∈
and (4,3).
S − HH;
Expert e18 should increase his assessment for position (1, 2 ),
Step 4 IC L(C 2 ) = 0.7361 < 0.8, SC L(C 2 ) = 0.9044 > 0.9, then C 2 ∈
(2,4), (3,2), (3,4), and (4,2), and decrease his assessments for
S − LH;
positions (1,3), (2,1), and (4,3).
Step 4 IC L(C 3 ) = 0.8013 > 0.8, SC L(C 3 ) = 0.9107 > 0.9, then C 3 ∈
Expert e19 should increase his assessments for positions (2, 1 )
S − HH;
and (2, 4 ), and decrease his assessment for position (4, 2 ).
Step 4 IC L(C 4 ) = 0.7825 < 0.8, SC L(C 4 ) = 0.8860 < 0.9, then C 4 ∈
Expert e20 should increase his assessments for positions (2, 1 ),
S − LL.
Step 5 Hybrid strategies are used to help experts modify their
(2, 4 ), and (4,3), and decrease his assessment for position
preferences. Because C 1 , C 3 ∈ S−H H , the experts in sub-
( 4, 2 ).
groups C 1 and C 3 do not need to adjust their preferences. We should note that the adjustment suggestions may require
the experts to make many changes with respect to some pairs
1) Because C2
∈ S−LH, the feedback strategy for C2 is implemented
of alternatives. In these situations, the experts may completely
as follows:
refuse to follow the suggestions, and instead, make the modifi-
γ S−LH = 0.7648 {ca1 = 0.7318(c p12 = 0.9067, c p13 = 0.6054, cations that they like. This involves another important topic in
c p14 = 0.6832 ); LSGDM problems, that is, non-cooperative behavior management.
ca2 = 0.7920(c p21 = 0.9067, c p23 = 0.7201, c p24 = 0.7491 ); There are some studies that focused on detecting and managing
ca3 = 0.7500(c p31 = 0.6054, c p32 = 0.7201, c p34 = 0.9243 ); non-cooperative behaviors (Palomares et al., 2014; Xu, Du et al.,
ca4 = 0.7855(c p41 = 0.6832, c p42 = 0.7491, c p43 = 0.9243 )}. 2015; Xu, Du, Chen & Cai, 2019). Since this is not the emphasis
of our study, we assume that all experts agree to modify their
i) According to Eq. (10), the alternatives that should be adjusted preferences.
are x1 , x3 . According to the previous recommendations, experts provide
ii) According to Eq. (11), the positions that should be adjusted are the modified preferences as follows:
⎧ ⎫ ⎧ ⎫
(1,3), (1,4), (3,1), and (3,2). ⎪ 0.4⎪ ⎪ 0.2 ⎪
⎨0.5 0.4 0.2
⎬ ⎨0.5 0.4 0.4
⎬
0.6 0.5 0.3 0.4 0.6 0.5 0.5 0.25
Generation of advice: P7 = 8
,P = ,
All the experts in sub-group C 2 should increase their as- ⎩0.8
⎪ 0.7 0.5 0.6⎪
⎭ ⎩0.6
⎪ 0.5 0.5 0.4 ⎪
⎭
0.6 0.6 0.4 0.5 0.8 0.75 0.6 0.5
sessments for positions (1,3) and (1,4), and decrease their ⎧ ⎫ ⎧ ⎫
assessments for positions (3,2) and (3,2). ⎪
⎨0.5 0.6 0.2 0.4⎪
⎬ ⎪
⎨0.5 0.6 0.6 0.4⎪
⎬
0.4 0.5 0.3 0.2 0.4 0.5 0.4 0.2
P9 = 16
,P =
2) Because C 4 ∈ S − LL, the feedback strategy for C 4 is imple- ⎪0.8
⎩ 0.7 0.5 0.6⎪
⎭ ⎩0.4
⎪ 0.6 0.5 0.2⎪
⎭
mented as follows: 0.6 0.8 0.4 0.5 0.6 0.8 0.8 0.5
⎧ ⎫ ⎧ ⎫
i) According to Eq. (13), the experts who should make modifi- ⎪
⎨0.5 0.8 0.7 0.6⎪
⎬ ⎪
⎨0.5 0.7 0.8 0.2⎪
⎬
0.2 0.5 0.4 0.3 0.3 0.5 0.6 0.4
cations are EX P S−LL = {e16 , e18 , e17 , e19 , e20 }. P 17 = , P 18 = ,
ii) According to Eq. (14), the alternatives that should be ⎩0.3
⎪ 0.6 0.5 0.4⎪
⎭ ⎩0.2
⎪ 0.4 0.5 0.2⎪
⎭
0.4 0.7 0.6 0.5 0.8 0.6 0.8 0.5
adjusted are listed in Table 4. ⎧ ⎫ ⎧ ⎫
iii) According to Eq. (15), the positions that should be adjusted ⎪
⎨0.5 0.8 0.7 0.3 ⎪
⎬ ⎪
⎨0.5 0.8 0.8 0.6⎪
⎬
are listed in Table 4. 0.2 0.5 0.4 0.3 0.2 0.5 0.3 0.2
P 19 = 20
,P =
⎩0.3
⎪ 0.6 0.5 0.15⎪
⎭ ⎩0.2
⎪ 0.7 0.5 0.3⎪
⎭
Generation of advice: 0.7 0.7 0.85 0.5 0.4 0.8 0.7 0.5
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971 967

Table 5
Weights of clusters (second round).

Cluster Degree of cohesion Size Weight

C1 0.6733 6 0.3346
C2 0.5337 3 0.1455
C3 0.4238 6 0.2607
C4 0.6000 5 0.2592

Second round
Using the proposed FCM clustering algorithm, the clustering
results are unchanged.
Then, the weights of the clusters are calculated and presented
in Table 5.
Values of ICL for four subgroups are ICL(C1 ) = 0.8428,
ICL(C2 ) = 0.7706, ICL(C3 ) = 0.8200, ICL(C4 ) = 0.8097. We can
obtain GCL = 0.8108 > 0.8. Thus, the group reaches an acceptable
consensus level. The global collective RCM is
⎧ ⎫
⎪
⎨ 0.5 0.5450 0.5696 0.5542⎪
⎬
0.4550 0.5 0.5597 0.5649
G
P = . companions. There are some studies that focused on manag-
⎪0.4304 0.4403 0.5 0.4665⎪
⎩ ⎭
0.4458 0.4351 0.5335 0.5
Then, using the normalizing rank aggregation method (Xu & Da,
2005), we can obtain the final ranking of these four alternatives
as x1 > x2 > x4 > x3 .
5. Discussions and comparative analyses
In this section, we provide some discussions about setting
parameters for the proposed model and the comparisons with
related LSGDM methods.
5.1. Analysis of K
The clustering process plays a critical role in LSGDM. It can
reduce the dimension of large-scale experts, based on which the
complexity and cost can be reduced (Ma, Zhu, Ponnambalam &
Zhang, 2019). Furthermore, it can identify the spokesperson repre-
senting the subgroup through finding common opinion patterns
such as sub-groups with similar opinions (Palomares, 2018). In the
FCM algorithm, the initial number of clusters K should be set in
advance. To better reflect the impact of the number of clusters
on the decision results, we simulate the CRP and present the
results in Table 6. Based on Table 6, we can obtain the following
observations:
1) The value of initial GCL decreases as the number of clusters
increases. When the number of clusters is not greater than 3,
the initial value of GCL is larger than 0.8. Therefore, there is
no need to provide modification suggestions to experts. How-
ever, the silhouette coefficients are very low in these two situ-
ations. Fig. 7 presents the relationship between the silhouette
coefficient and the number of clusters. As mentioned before,
the silhouette coefficient is an indicator to measure the clus-
tering effect. A higher value of silhouette coefficient reflects a
better clustering effect. The silhouette coefficient is low when
the number of clusters is 2 or 3 because the partition is rough.
2) There is a rapid growth for the silhouette coefficient from 3
clusters to 4 clusters and small changes between 4 and 6. We
should note that there are clusters that contain only one ex-
pert when the number of clusters is greater than 4. In such a
case, the value of A(P k ) for ek is 1 and S(P k ) = 1. Furthermore,
we set the value of SCI for these clusters as 1. Therefore, those
Fig. 7. Relationship between the number of clusters and the silhouette coefficient.
experts who form a sub-group on their own (minority opin-
ions) will have a larger weight than the experts who have team
ing minority opinions in LSGDM problems (Xu, Du et al., 2015).
Considering all these factors, K = 4 is an appropriate choice.
3) The final alternative ranking is stable with the change of K.
However, there are some differences on the final group RCM.
The neutralization effect also contributes to the fact that these
differences are not particularly significant since some experts
increase the assessments and some experts decrease the assess-
ments.
Rodríguez et al. (2018) and Labella, Liu, Rodríguez and Martínez
(2019) also applied the idea of adaptive CRP to solve LSGDM prob-
lems. To demonstrate the generalizability and applicability of our
proposed model, we also provide the performance of Rodríguez
et al. (2018)’s and Labella et al. (2019)’s methods. Both of these
two studies set two thresholds regarding the level of consensus.
The first one is the level of consensus for advice generation, i.e.,
ς = 0.7: if the level of global consensus is lower than 0.7, then
all experts in sub-groups with their proximity degrees lower than
the average proximity degree are selected. The proximity degree
in Rodríguez et al. (2018) is equal to the inter consensus level
ICL in this paper. Another parameter is the consensus threshold,
i.e., θ = 0.85. If ς ≤ GCL < θ , then the level of consensus is high
but not sufficient. In this case, the selection of sub-groups is the
same as that in the above case. The difference from the above
case is that only those experts whose opinions differ most from
the collective one are selected. If GCL ≥ θ , then the degree of
consensus is sufficiently high.
Since Rodríguez et al. (2018) took the average proximity degree
as the sub-group selection criterion, nearly half of the sub-groups
were selected in each round regardless the degree of consensus. By
contrast, our method covers a wide range of clusters if the degree
of consensus is low, and a narrow range if the degree of consensus
is high. From Table 6, we can summarize two advantages of our
method:
√
First, when the number of clusters is 6 or 7 with a low initial
GCL, our method has a higher iterative efficiency since more ex-
perts are covered in the first iteration.
√
Second, when the number of clusters is 4 or 5, both of our
method and Rodríguez et al. (2018)’s method need only one it-
eration. However, our method requires less modifications since
less pairs of alternatives are covered according to the adaptive
feedback mechanism presented in Section 3.4. Therefore, our
method has a better performance in avoiding over-adjustment
968 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971

Table 6
Decision results associated with the number of clusters.
K Initial clustering results Initial NI(1) NP(1) NI(2) NP(2) Final Final global group RCM Alternative ranking
GCL GCL
⎧ ⎫
⎪0.50 0 0 0.5611 0.5575 0.5242⎪
⎨ ⎬
0.4389 0.50 0 0 0.5391 0.5237
2 C 1 = {e1 , e2 , e3 , e4 , e5 , e6 , e11 , e12 , 0.8660 0 0 0 0 0.8660 x1 > x2 > x4 > x3
e13 , e14 , e17 , e18 , e19 , e20 },
⎪
⎩0.4425 0.4609 0.50 0 0 0.4546⎪
⎭
0.4758 0.4763 0.5454 0.50 0 0
C 2 = {e7 , e8 , e9 , e10 , e15 , e16 }
⎧ ⎫
⎪0.50 0 0 0.5437 0.5459 0.5271 ⎪
⎨ ⎬
0.4563 0.50 0 0 0.5435 0.5447
3 C 1 = {e1 , e2 , e3 , e4 , e5 , e6 , 0.8137 0 0 0 0 0.8137 x1 > x2 > x4 > x3
e17 , e18 , e19 , e20 },
⎪
⎩ 0.4541 0.4565 0.50 0 0 0.4673 ⎪
⎭
0.4729 0.4553 0.5327 0.50 0 0
C 2 = {e7 , e8 , e9 , e16 },
C 3 = {e10 , e11 , e12 , e13 , e14 , e15 }
⎧ ⎫
⎪0.50 0 0 0.5450 0.5696 0.5542⎪
⎨ ⎬
0.4550 0.50 0 0 0.5597 0.5649
4 C1 = {e1 , e2 , e3 , e4 , e5 , e6 }, 0.7888 1 23 1 32 0.8108 x1 > x2 > x4 > x3
C 2 = {e7 , e8 , e9 },
⎪
⎩0.4304 0.4403 0.50 0 0 0.4665⎪
⎭
0.4458 0.4351 0.5335 0.50 0 0
C = {e10 , e11 , e12 , e13 , e14 , e15 },
3

C 4 = {e16 , e18 , e17 , e19 , e20 }.

⎧ ⎫
⎪0.50 0 0 0.5565 0.5841 0.5429⎪
⎨ ⎬
0.4435 0.50 0 0 0.5615 0.5235
5 C1 = {e1 , e2 , e3 , e4 , e5 , e6 }, 0.7773 1 39 1 45 0.8171 x1 > x2 > x4 > x3
C2 = {e7 , e8 , e9 },
⎪
⎩ 0.4159 0.4358 0.50 0 0 0.4693⎪
⎭
0.4571 0.4757 0.5307 0.50 0 0
C3 = {e10 , e11 , e12 , e13 , e14 , e15 },
C4 = {e16 , e17 , e19 , e20 },
C5 = {e18 }
⎧ ⎫
⎪0.50 0 0 0.5631 0.5737 0.5351 ⎪
⎨ ⎬
0.4369 0.50 0 0 0.5663 0.5302
6 C1 = {e1 , e2 , e3 , e4 , e5 , e6 }, 0.7686 1 36 2 58 0.8128 x1 > x2 > x4 > x3
C2 = {e7 , e9 },
⎪
⎩0.4263 0.4337 0.50 0 0 0.4366⎪
⎭
0.4649 0.4698 0.5634 0.50 0 0
C3 = {e10 , e11 , e12 , e13 , e14 , e15 },
C4 = {e16 , e17 , e19 , e20 },
C5 = {e18 }, C 6 = {e8 }
⎧ ⎫
⎪0.50 0 0 0.5560 0.5684 0.5208⎪
⎨ ⎬
0.4440 0.50 0 0 0.5631 0.5248
7 C1 = {e1 , e2 , e3 , e4 , e5 , e6 }, 0.7663 1 38 2 58 0.8077 x1 > x2 > x4 > x3
C2 = {e7 , e9 },
⎪
⎩ 0.4316 0.4369 0.50 0 0 0.4405⎪
⎭
0.4792 0.4752 0.5595 0.50 0 0
C3 = {e10 , e11 , e12 , e13 , e14 , e15 },
C4 = {e16 , e19 }, C 5 = {e18 },
C6 = {e8 }, C 7 = {e17 , e20 }

Note. NI(1): number of iterations by using our model; NP(1): number of positions in the upper triangle of RCM need to be modified by using our model; NI(2): number of
iterations by using Rodríguez et al. (2018)’s model; NP(2): number of positions in the upper triangle of RCM need to be modified by using Rodríguez et al. (2018)’s model.

Table 7
Decision results associated with different consensus thresholds.

Consensus NI(1) NP (NE)(1) NI(2) NP(NE)(2) Final global group RCM Alternative
threshold ranking
(GCI)
⎧ ⎫
⎪0.50 0 0 0.5600 0.5613 0.5334 ⎪
⎨ ⎬
0.4400 0.50 0 0 0.5465 0.5343
0.75 0 0 0 0 x1 > x2 > x4 >
⎪
⎩0.4387 0.4535 0.50 0 0 0.4577 ⎪⎭ x3
0.4666 0.4657 0.5423 0.50 0 0
⎧ ⎫
⎪0.50 0 0 0.5450 0.5696 0.5542⎪
⎨ ⎬
0.4550 0.50 0 0 0.5597 0.5649
0.8 1 23(8) 1 32(8) x1 > x2 > x4 >
⎪
⎩0.4304 0.4403 0.50 0 0 0.4665⎪ ⎭ x3
0.4458 0.4351 0.5335 0.50 0 0
⎧ ⎫
⎪0.50 0 0 0.5715 0.5905 0.5333 ⎪
⎨ ⎬
0.4285 0.50 0 0 0.5247 0.5581
0.85 2 59(20)+18(8) 4 32(8)+34(9) x1 > x2 > x4 >
+27(10)+26(10)
⎪
⎩0.4095 0.4753 0.50 0 0 0.4831 ⎪⎭ x3
0.4667 0.4419 0.5169 0.50 0 0
⎧ ⎫
⎪0.50 0 0 0.5491 0.5543 0.5171 ⎪
⎨ ⎬
0.4509 0.50 0 0 0.5346 0.5501
0.9 3 7 x1 > x2 > x4 >
59(20)+54(20)+24(9) 32(8)+34(9)+27(10)
⎪
⎩ 0.4457 0.4654 0.50 0 0 0.4425⎪ ⎭ x3
0.4829 0.4499 0.5575 0.50 0 0
+26(10)+26(9)
+22(8)+23(8)

Note. NE: number of experts that need to make modifications. NI(1): number of iterations by using our model; NP(NE)(1): number of positions (experts) in the upper triangle
of RCM need to be modified by using our model; NI(2): number of iterations by using Rodríguez et al. (2018)’s model; NP(NE)(2): number of positions (experts) in the upper
triangle of RCM need to be modified by using Rodríguez et al. (2018)’s model.

when the degree of consensus is close to the threshold. In this
sense, the supervision cost of our method is reduced.
Overall, our method can adapt to fine or rough partitioning
scenarios. Furthermore, if there are one or more minority opinion
subgroups, the CRP can also reach an expected consensus level
within one iteration. Hence, this method is robust to the partition
distribution.
5.2. Analysis of the consensus threshold
In this section, we analyze the effect of different consensus
thresholds on the decision results. Table 7 presents the results
including the number of iterations, number of adjusted positions,
final group RCM and alternative ranking. Based on Table 7, some
observations can be obtained as:
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
Fig. 8. The iterative process of four CRPs.
1) The number of iterations increases as the value of consensus
threshold increases, and the final alternative ranking is stable.
2) The closer the value of GCL to the consensus threshold is, the
smaller the quantity of positions and experts involved in the
feedback mechanism are. For instance, when GCI = 0.85, a total
of 59 positions (in the upper triangle of the RCM) for all 20 ex-
perts need to be modified in the first iteration round. In the
second iteration round, 8 experts should make modifications
in 18 positions altogether. Furthermore, in the same iteration
round, if the consensus thresholds are different, the numbers of
positions and experts involved in the feedback mechanism are
also different. For instance, when GCI = 0.8, 8 experts should
make modifications in 23 positions in the first iteration round.
However, when GCI = 0.85, all 20 experts should make modi-
fications in 59 positions in the first iteration round. With the
increase of GCI, some sub-groups belong to S-HH turn into S-
LH. These two perspectives can elaborate the word “adaptive”
in the title.
3) A higher consensus threshold GCI needs a larger quantity of
modifications, which distorts the original opinions. For instance,
when GCI = 0.9, all experts are identified to make modifications
in two rounds. This explains why an LSGDM problem should
not set up a high consensus threshold. The consensus thresh-
old should be set according to actual situations.
From Table 7, we can also find the two advantages summarized
in Section 5.1 compared with Rodríguez et al. (2018)’s method.
One is the higher iterative efficiency. When GCI is high, such as
GCI =0.9, Rodríguez et al. (2018)’s method needs more iterations.
The other is avoiding over-adjustment. As we can see from the
fifth column of Table 7, in each iterative round, nearly half of
all experts are involved. One big difference between Rodríguez
et al. (2018)’s method and our method is that when the degree
of consensus is far from the threshold, then a large number
of modifications are required to fasten the iteration process. In
the initial iterations, all experts will participate in the feedback
mechanism. In the last iterations, only a small number of experts
and pairs of alternatives are involved. If we use Rodríguez et al.
(2018)’s method, one problem is that some subgroups may need
to make adjustments of similar quantities in several successive
iterations, which may discourage their enthusiasm to participate
in the feedback mechanism. Additionally, our method has a better
performance with the change of GCI and is relatively insensitive to
the selection of GCI.
5.3. Comparison with different CRPs
Firstly, we use Rodríguez et al. (2018)’s method to solve the
problem. Because ς ≤ GCL = 0.7888 < θ , then the initial degree of
969
consensus in our illustrative example is high but not sufficient. Us-
ing the method in Rodríguez et al. (2018), two sub-groups (C 2 and
C 4 ) and all the four alternatives are selected. For sub-group C 2 , the
pairs of alternatives (x1 , x3 ), (x1 , x4 ), (x2 , x3 ), and (x2 , x4 ) are se-
lected; for sub-group C 4 , the pairs of alternatives (x1 , x2 ), (x1 , x3 ),
(x1 , x4 ), (x2 , x4 ), and (x3 , x4 ) are selected. If we focus on the cost
of supervision, the moderator needs to provide 3 × 4 + 4 × 5 = 32
suggestions for these seven experts. Our proposed model also
needs to provide 32 suggestions. However, these 32 suggestions
cover all 4 sub-groups and 20 experts. Because C 1 , C 2 , C 3 ∈ S−LH,
then all experts in these three sub-groups are required to change
following a unified direction. After the first iteration, our method
reaches a degree of consensus of 0.8412 and Rodríguez et al.
(2018)’s method reaches 0.8125.
Except these two adaptive CRPs, there are several other CRPs.
One widely used CRP is to choose subgroups whose degree of con-
sensus is lower than a predefined threshold in each iterative round
and requires them to make modifications (Zhang, Dong, Chiclana
& Yu, 2019) (CRP1). The other is to choose the furthest subgroup
in each iterative round (Xu, Du et al., 2015) (CRP2). When we use
these four CRPs to solve the problem, the iterative processes are
presented in Fig. 8. As we can see from Fig. 8, our method has a
high efficiency. CRP1 has three iterations. However, compared with
the adaptive CRP, CRP1 has the problem of over-adjustment. After
the second iteration, CRP1 reaches a degree of consensus that is
close to 0.85. After the third iteration, its degree of consensus is
the highest because many modifications have been made. CRP2 is
the slowest since only the farthest subgroups are required to make
modifications in each round.
Based on all above discussions, we can summarize the advan-
tages of the proposed model:
1) It has a high iterative efficiency, especially in the initial itera-
tions. The adaptive mechanism involves a large scope when the
degree of consensus GCL is far from the threshold GCI.
2) It can avoid over-adjustment when the degree of consensus GCL
is near to the threshold GCI because of the introduction of S−LH
and S−HL.
These two advantages make the proposed model have a good
performance with the change of K and GCI (see Tables 6 and
7).
3) The introduction of S−LH can reduce supervision cost since
only a unified suggestion is required for the low–high sub-
groups.
4) The clusters in this model are allowed to change. In the CRP,
experts make modifications on their preferences. Therefore, the
970 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971

clusters may change. The adoption of fixed clusters (Liu, Xu, moderator can be reduced. For the experts belonging to S-LH,
Montes, Ding & Herrera, 2019; Xu, Du et al., 2015) is a special a unified suggestion for the sub-group instead of specific sug-
case in the proposed model. gestion for each expert is enough. For the experts belonging to
5) The weight determination combines two indices, namely, the S-HL, the task of the moderator is to make these experts’ opin-
majority principle and cohesion of subgroups. The single index- ions more cohesive. In this situation, only a small number of
based method based on the majority principle (Xu, Du et al., changes are required since these experts have been classified
2015) or the degree of consistency of RCMs has some limita- into the same cluster.
tions. For instance, if we only use the majority principle, the
In this study, the preference structures that the experts used
subgroups with the same number of experts but possessing dif-
to express their opinions were assumed to be RCMs. In the future,
ferent inner characteristics will have the same weights. If we
it is necessary to investigate the LSGDM method in the case that
use only the degree of consistency, the importance of experts
the representation models are qualitative. In addition, this work
in a small subgroup with a high degree of consistency may be
mainly focused on the cost of supervision, i.e., the number of
several times higher than the importance of experts in large
suggestions provided by the moderator. The adjustment cost can
subgroups with a low degree of consistency. Therefore, the pro-
also be taken into account under the adaptive LSGDM context in
posed method is more applicable.
the future. The optimization of the time complexity of the LSGDM
There are many other consensus models for LSGDM problems consensus model is another interesting topic.
(Palomares et al., 2014; Wu & Xu, 2018; Xu, Du et al., 2015; Xu,
Zhong, Chen & Zhou, 2015; Zhang et al., 2018). Different models Acknowledgments
have their own purposes. Therefore, it is not fair to compare these
different consensus models. There is not a model that can be The work was supported by the National Natural Science Foun-
regarded as the best. dation of China (71571156 and 71971145), and the 2019 Sichuan
The proposed model also has limitations. Several parameters Planning Project of Social Science (No. SC18A007).
used in this study, including the consensus thresholds GCL and SCL,
the control parameter δ in Eq. (16), and the number of clusters K, Appendix
should be set in advance. For GCL and SCL, some discussions are
Twenty initial RCMs:
given in Sections 3.3 and 5.2. These two thresholds can be deter- ⎧ ⎫ ⎧ ⎫
mined by the moderator according to the specific decision-making ⎪
⎨0.5 0.7 0.9 0.8⎪
⎬ 0.7 0.85 ⎪
⎨ 0.5 0.8⎪
⎬
problem. The value of δ controls the degree of preference between 1 0.3 0.5 0.8 0.7 2 0.3 0.5 0.7 0.6
P = ,P = ,
⎩0.1 0.2 0.5 0.4⎪
⎪ ⎩0.15 0.3 0.5
⎪ 0.3⎪
the global collective point and the center point of a cluster. It can
⎭ ⎭
be determined by the degrees of deviation and diversity of the 0.2 0.3 0.6 0.5 0.2 0.4 0.7 0.5
group. The number of clusters, K, is also a parameter that should ⎧ ⎫ ⎧ ⎫
be considered. The number of clusters must be first defined for ⎪
⎨0.5 0.7 0.8 0.6⎪
⎬ 0.6 0.65 0.75⎪
⎬ ⎪
⎨ 0.5
3 0.3 0.5 0.8 0.6 4 0.4 0.5 0.6 0.8
many automatic clustering algorithms. In Section 5.1, we discuss P = ,P = ,
the appropriate number of clusters with the help of silhouette ⎩0.2 0.2 0.5 0.4⎪
⎪ ⎭ ⎩0.35 0.4 0.5 0.55⎪
⎪ ⎭
0.4 0.4 0.6 0.5 0.25 0.2 0.45 0.5
coefficient and minority opinions. There are also many studies that ⎧ ⎫ ⎧ ⎫
focused on this topic and some approaches have been developed, ⎪
⎨ 0.5
0.7 0.8 0.75⎪
⎬ ⎪
⎨ 0.5 0.7 0.85 0.8 ⎪
⎬
such as the validity indices (Wu & Yang, 2005) and the robust 5 0.3 0.5 0.7 0.6 6 0.3 0.5 0.7 0.55
P = ,P = ,
and adaptive clustering analysis method (Mok, Huang, Kwok &
⎩ 0.2 0.3
⎪ 0.5 0.4 ⎪
⎭ ⎩0.15 0.3 0.5 0.4 ⎪
⎪ ⎭
Au, 2012). The different methods have their own advantages. From 0.25 0.4 0.6 0.5 0.2 0.45 0.6 0.5
another perspective, these parameters provide great freedom for ⎧ ⎫ ⎧ ⎫
the moderator to control the decision-making process.
⎪
⎨0.5 0.4 0.1 0.3⎪
⎬ 0.4 ⎪
⎨ 0.5 0.3 0.05⎪
⎬
7 0.6 0.5 0.2 0.4 8 0.6 0.5 0.4 0.25
P = ,P = ,
6. Conclusions ⎩0.9 0.8 0.5 0.6⎪
⎪ ⎭ ⎩ 0.7 0.6
⎪ 0.5 0.4 ⎪
⎭
0.7 0.6 0.4 0.5 0.95 0.75 0.6 0.5
⎧ ⎫ ⎧ ⎫
This study developed an adaptive consensus model with hy- ⎪
⎨0.5 0.6 0.1 0.3⎪
⎬ ⎪
⎨0.5 0.3 0.1 0.4 ⎪
⎬
brid strategies within the LSGDM context. A key characteristic 0.4 0.5 0.2 0.2 0.7 0.5 0.3 0.6
P9 = , P 10 = ,
of this model is the design of a hybrid feedback mechanism to ⎩0.9 0.8 0.5 0.6⎪
⎪ ⎭ ⎩0.9 0.7 0.5 0.65⎪
⎪ ⎭
improve the consensus performance in LSGDM problems. This 0.7 0.8 0.4 0.5 0.6 0.4 0.35 0.5
model is composed of four parts: classifying experts, determining
⎧ ⎫ ⎧ ⎫
⎪
⎨0.5 0.3 0.4 0.6⎪⎬ ⎪
⎨ 0.5 0.25 0.4 0.55⎪
⎬
the weights of sub-groups, computing degrees of consensus, and 0.7 0.5 0.6 0.9 0.75 0.5 0.7 0.8
improving levels of consensus. An illustrative example about the P 11 = , P 12 = ,
legislative amendment in UNCITRAL was implemented to illus- ⎩0.6 0.4 0.5 0.7⎪
⎪ ⎭ ⎩ 0.6 0.3 0.5 0.65⎪
⎪ ⎭
0.4 0.1 0.3 0.5 0.45 0.2 0.35 0.5
trate the method. The main contributions of this study can be ⎧ ⎫ ⎧ ⎫
highlighted as follows: ⎪
⎨ 0.5 0.3 0.45 0.6⎪
⎬ ⎪
⎨0.50.4 0.4 0.6 ⎪
⎬
0.7 0.5 0.55 0.9 0.6 0.5 0.7 0.95
1) A weight-determining method for the sub-groups was devel- P 13 = , P 14 = ,
. .45 0.5 0.7⎪
oped. This method combines two parameters: cohesion and size
⎪
⎩0 55 0
⎭ ⎩0.6 0.3
⎪ 0.5 0.6 ⎪
⎭
0.4 0.1 0.3 0.5 0.4 0.05 0.4 0.5
of a sub-group. ⎧ ⎫ ⎧ ⎫
2) The proposed model truly takes into account the impact of the ⎪
⎨ 0.5 0.2 0.3 0.45⎪
⎬ ⎪
⎨0.5 0.7 0.6 0.4⎪
⎬
clustering process in LSGDM problems. Because of the cluster- 0.8 0.5 0.6 0.7 0.3 0.5 0.4 0.1
P 15 = , P 16 = ,
ing algorithm, an LSGDM problem was divided into two levels: ⎩ 0.7 0.4
⎪ 0.5 0.6 ⎪
⎭ ⎩0.4
⎪ 0.6 0.5 0.2⎪
⎭
inter sub-group and intra sub-group. According to the different 0.55 0.3 0.4 0.5 0.6 0.9 0.8 0.5
⎧ ⎫ ⎧ ⎫
levels of degree of consensus in the two levels, four scenarios ⎪
⎨0.5 0.8 0.7 0.6⎪
⎬ ⎪
⎨0.5 0.6 0.9 0.2⎪
⎬
were distinguished. Then, hybrid strategies were generated to 0.2 0.5 0.4 0.2 0.4 0.5 0.7 0.3
P 17 = , P 18 = ,
⎩0.3 0.6 0.5 0.3⎪ ⎩0.1 0.4 0.5 0.1⎪
provide suggestions for experts to modify their opinions. Be- ⎪ ⎪
⎭ ⎭
cause of this hybrid mechanism, the cost of supervision of the 0.4 0.8 0.7 0.5 0.8 0.3 0.9 0.5
 M. Tang, H. Liao and J. Xu et al. / European Journal of Operational Research 282 (2020) 957–971
⎧ ⎫ ⎧ ⎫
⎪
⎨ 0.5 0.95 0.7 0.3 ⎪
⎬ ⎪
⎨0.5 0.9 0.8 0.6⎪
⎬
0.05 0.5 0.4 0.2 0.1 0.5 0.3 0.1
P 19 = 20
,P =
0.3 0.6 0.5 0.15⎪
⎪
⎩ ⎭ ⎩0.2
⎪ 0.7 0.5 0.4⎪
⎭ Pal, N. R., & Bezdek, J. C. (1995). On cluster validity for the fuzzy c-mean model.
0.7 0.8 0.85 0.5 0.4 0.9 0.6 0.5
References
Bezdek, J. C., Ehrlich, R., & Wull, F. (1984). FCM – The fuzzy c-means clustering
algorithm. Computers & Geosciences, 10(2–3), 191–203.
Bezdekm, J. C. (1976). A physical interpretation of fuzzy ISODATA. IEEE Transactions
on Systems, Man and Cybernetics, 6, 387–390.
Capuano, N., Chiclana, F., Fujita, H., & Herrera-Viedma, E. (2018). Fuzzy group deci-
sion making with incomplete information guided by social influence. IEEE Trans-
action on Fuzzy Systems, 26(3), 1704–1718.
Chen, S. M., Lee, L. W., Yang, S. W., & Sheu, T. W. (2012). Adaptive consensus support
model for group decision making systems. Expert Systems with Applications, 39,
12580–12588.
Cheng, D., Zhou, Z. L., Cheng, F. X., Zhou, Y. F., & Xie, Y. J. (2018). Modeling the min-
imum cost consensus problem in an asymmetric costs context. European Journal
of Operational Research, 270, 1122–1137.
Dong, Q. X., & Cooper, O. (2016). A peer-to-peer dynamic adaptive consensus reach-
ing model for the group AHP decision making. European Journal of Operational
Research, 250, 521–530.
Efremov, R., Insua, D. R., & Lotov, A. (2009). A framework for participatory decision
support using Pareto frontier visualization, goal identification and arbitration.
European Journal of Operational Research, 199(2), 459–467.
Gonzalez, T. F. (1984). Clustering to minimize the maximum inter-cluster distance.
Theoretical Computer Science, 38(2–3), 293–306.
Gupta, M. (2018). Consensus building process in group decision making—An adap-
tive procedure based on group dynamics. IEEE Transaction on Fuzzy Systems,
26(4), 1923–1933.
Herrera-Viedma, E., Cabrerizo, F. J., Kacprzyk, J., & Pedrycz, W. (2014). A review of
soft consensus models in a fuzzy environment. Information Fusion, 17(1), 4–13.
Herrera-Viedma, E., Herrera, F., Chiclana, F., & Luque, M. (2004). Some issues on con-
sistency of fuzzy preference relations. European Journal of Operational Research,
154, 98–109.
Herrera-Viedma, E., Martínez, L., & Mata, F. (2005). A consensus support system
model for group decision-making problems with multigranular linguistic pref-
erence relations. IEEE Transaction on Fuzzy Systems, 13(5), 644–658.
Kacprzyk, J., & Fedrizzi, M. (1986). Soft consensus measures for monitoring real
consensus reaching processes under fuzzy preferences. Control and Cybernetics,
15(3–4), 309–323.
Kacprzyk, J., & Fedrizzi, M. (1988). A ‘soft’ measure of consensus in the setting
of partial (fuzzy) preferences. European Journal of Operational Research, 34(3),
316–325.
Kacprzyk, J., & Fedrizzi, M. (1989). A ‘human-consistent’ degree of consensus based
on fuzzy logic with linguistic quantifiers. Mathematical Social Sciences, 18(3),
275–290.
Kim, J. (2008). A model and case for supporting participatory public decision mak-
ing in e-democracy. Group Decision and Negotiation, 17(3), 179–192.
Labella, Á., Liu, Y., Rodríguez, R. M., & Martínez, L. (2018). Analyzing the perfor-
mance of classical consensus models in large scale group decision making: A
comparative study. Applied Soft Computing, 67, 677–690.
Labella, Á., Liu, Y., Rodríguez, R. M., & Martínez, L. (2019). An adaptive consensus
reaching process dealing with comparative linguistic expressions in large-scale
group decision making. In Proceedings of the 11th conference of the European so-
ciety for fuzzy logic and technology. Atlantis Press.
Liang, Q., Liao, X. W., & Liu, J. P. (2017). A social ties-based approach for group de-
cision-making problems with incomplete additive preference relations. Knowl-
edge-Based Systems, 119, 68–86.
Liu, F., Zhang, W. G., & Zhang, L. H. (2014). Consistency analysis of triangular fuzzy
reciprocal preference relations. European Journal of Operational Research, 235,
718–726.
Liu, X., Xu, Y. J., Montes, R., Ding, R. X., & Herrera, F. (2019). Alternative rank-
ing-based clustering and reliability index-based consensus reaching process for
hesitant fuzzy large scale group decision making. IEEE Transaction on Fuzzy Sys-
tems, 27(1), 159–171.
Ma, Z. Z., Zhu, J. J., Ponnambalam, K., & Zhang, S. T. (2019). A clustering method
for large-scale group decision-making with multi-stage hesitant fuzzy linguistic
terms. Information Fusion, 50, 231–250.
Mata, F., Martínez, L., & Herrera-Viedma, E. (2009). An adaptive consensus support
model for group decision making problems in a multi-granular fuzzy linguistic
context. IEEE Transaction on Fuzzy Systems, 17(2), 279–290.
971
Mok, P. Y., Huang, H. Q., Kwok, Y. L., & Au, J. S. (2012). A robust adaptive clustering
analysis method for automatic identification of clusters. Pattern Recognition, 45,
3017–3033.
IEEE Transaction on Fuzzy Systems, 3, 370–379.
Palarea-Albaladejo, J., Martín-Fernández, J. A., & Soto, J. A. (2012). Dealing with dis-
tances and transformations for fuzzy c-means clustering of compositional data.
Journal of Classification, 29(2), 144–169.
Palomares, I. (2018). LGDM approaches and models: A literature review. Large group
decision making. SpringerBriefs in computer science. Cham: Springer.
Palomares, I., Martinez, L., & Herrera, F. (2014). A consensus model to detect and
manage non-cooperative behaviors in large-scale group decision making. IEEE
Transaction on Fuzzy Systems, 22(3), 516–530.
Pang, J. F., Liang, J. Y., & Song, P. (2017). An adaptive consensus method for multi-at-
tribute group decision making under uncertain linguistic environment. Applied
Soft Computing, 58, 339–353.
Pérez, I. J., Cabrerizo, F. J., Alonso, S., & Herrera-Viedma, E. (2014). A new consen-
sus model for group decision making problems with non-homogeneous experts.
IEEE Transactions on Systems, Man and Cybernetics: Systems, 44(4), 494–498.
Pérez, I. J., Cabrerizo, F. J., & Herrera-Viedma, E. (2010). A mobile decision support
system for dynamic group decision-making problems. IEEE Transactions on Sys-
tems, Man, and Cybernetics – Part A: Systems and Humans, 40, 1244–1256.
Pérez, I. J., Cabrerizo, F. J., & Herrera-Viedma, E. (2011). Group decision making prob-
lems in a linguistic and dynamic context. Expert Systems with Applications, 38(3),
1675–1688.
Rodríguez, A., & Laio, A. (2014). Clustering by fast search and find of density peaks.
Science, 344(6191), 1492–1496.
Rodríguez, R. M., Labella, Á., Tré, G. D., & Martínez, L. (2018). A large scale consen-
sus reaching process managing group hesitation. Knowledge-Based Systems, 159,
86–97.
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and vali-
dation of cluster analysis. Journal of Computational and Applied Mathematics, 20,
53–65.
Switalski, Z. (1999). Rationality of fuzzy reciprocal preference relations. Fuzzy Sets
and Systems, 107, 187–190.
Tian, J. F., Zhang, Z. M., & Ha, M. H. (2018). An additive consistency and consensus-
based approach for uncertain group decision making with linguistic preference
relations. IEEE Transaction on Fuzzy Systems. doi:10.1109/TFUZZ.2018.2865132.
Wu, K. L., & Yang, M. S. (2005). A cluster validity index for fuzzy clustering. Pattern
Recognition Letters, 26(9), 1275–1291.
Wu, X. L., & Liao, H. C. (2019). A consensus-based probabilistic linguistic gained and
lost dominance score method. European Journal of Operational Research, 272(3),
1017–1027.
Wu, Z. B., & Xu, J. P. (2016). Managing consistency and consensus in group decision
making with hesitant fuzzy linguistic preference relations. Omega, 65(3), 28–
40.
Wu, Z. B., & Xu, J. P. (2018). A consensus model for large-scale group decision mak-
ing with hesitant fuzzy information and changeable clusters. Information Fusion,
41, 217–231.
Xu, X. H., Du, Z. J., & Chen, X. H. (2015). Consensus model for multi-criteria large–
group emergency decision making considering non-cooperative behaviors and
minority opinions. Decision Support Systems, 79, 150–160.
Xu, X. H., Du, Z. J., Chen, X. H., & Cai, C. G. (2019). Confidence consensus-based
model for large-scale group decision making: A novel approach to managing
non-cooperative behaviors. Information Sciences, 477, 410–427.
Xu, X. H., Zhong, X. Y., Chen, X. H., & Zhou, Y. J. (2015). A dynamical consensus
method based on exit-delegation mechanism for large group emergency deci-
sion making. Knowledge-Based Systems, 86, 237–249.
Xu, Z. S., & Da, Q. L. (2005). A least deviation method to obtain a priority vec-
tor of a fuzzy preference relation. European Journal of Operational Research, 164,
206–216.
Zhang, H. J., Dong, Y. C., Chiclana, F., & Yu, S. (2019). Consensus efficiency in group
decision making: A comprehensive comparative study and its optimal design.
European Journal of Operational Research, 275(2), 580–598.
Zhang, H. J., Dong, Y. C., & Herrera-Viedma, E. (2018). Consensus building for the
heterogeneous large-scale GDM with the individual concerns and satisfactions.
IEEE Transaction on Fuzzy Systems, 28(2), 884–898.
Zhang, S. T., Zhu, J. J., Liu, X. D., Chen, Y., & Ma, Z. Z. (2017). Adaptive consen-
sus model with multiplicative linguistic preferences based on fuzzy information
granulation. Applied Soft Computing, 60, 30–47.
Zhou, K. L., Fu, C., & Yang, S. L. (2014). Fuzziness parameter selection in fuzzy
c-means: The perspective of cluster validation. Science China Information Sci-
ences, 57(11), 1–8.