Information Sciences 545 (2021) 44–64

Contents lists available at ScienceDirect

Information Sciences
journal homepage: www.elsevier.com/locate/ins

Supplier’s goal setting considering sustainability: An uncertain

dynamic Data Envelopment Analysis based benchmarking
model
Xiaoyang Zhou a,b, Linzi Li a,⇑, Haoyu Wen a,⇑, Xin Tian c,d, Shouyang Wang b, Benjamin Lev e
a
School of Economics and Management, Xidian University, Xi’an 710126, China
b
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China
c
School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
d
Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100190, China
e
LeBow College of Business, Drexel University, Philadelphia, PA 19104, USA

a r t i c l e i n f o a b s t r a c t

Article history: Because of the growing awareness of balancing the economy, the society and the environ-
Received 10 September 2019 ment, there has been an increased focus on goal setting under sustainable context.
Received in revised form 22 July 2020 However, underperforming suppliers may set impractical goals that are difficult to achieve
Accepted 25 July 2020
in the short term due to the poor information analysis, which could cause benchmarking
Available online 5 August 2020
difficulties. Therefore, rather than focusing on a single ultimate goal, this paper developed
an implementable goal sequence for which a dynamic Data Envelopment Analysis based
Keywords:
benchmarking model was developed as a multi-period process. The indicators that could
Goal setting
Sustainability
not be directly measured using numerical values were incorporated into the model as
Uncertain dynamic DEA fuzzy numbers, and based on efficiencies, the suppliers were divided into several layers.
Fuzzy data. A method that identified supplier attractiveness and progress was then introduced to select
the most appropriate goal in each layer for the goal sequence. A numerical example is pro-
vided that illustrated the feasibility and efficacy of the proposed method to successfully
identify the inefficient suppliers and determine a goal sequence that had three attainable
goals. The sensitivity analysis and comparisons revealed that the proposed model avoided
selecting period-inefficient suppliers and provided a better benchmarking process.
Ó 2020 Elsevier Inc. All rights reserved.

1. Introduction

Sustainable Supply Chain Management (SSCM) has become a major global focus, with the incorporation of sustainability
into supply chain management indicating that involved companies are simultaneously responsible for activities related to
economic, social and environmental triple bottom line (TBL) benefits. Therefore, nowadays, any company that only pursues
economic growth without the regard for social security and environmental protection will not achieve long-term develop-
ment [38], that is, TBL performances need to be improved to ensure that inefficient suppliers meet the needs of downstream
members and gain competitiveness.
Sustainable supplier goal setting requires the weighing up of multiple criteria. Traditionally, decision makers have set
goals based on cost, quality, and delivery; however, given the importance of sustainability, supplier goal setting no longer

⇑ Corresponding authors.
E-mail addresses: linz.li@foxmail.com (L. Li), hywen@xidian.edu.cn (H. Wen).

https://doi.org/10.1016/j.ins.2020.07.074
0020-0255/Ó 2020 Elsevier Inc. All rights reserved.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 45

involves a simple compliance with the standards and norms defined by the purchasers or a unilateral improvement in eco-
nomic efficiency [24]. As companies that fail to focus on social and environmental goals are more likely to gain negative rep-
utations and be considered inefficient, to achieve sustainable development, it has become necessary for suppliers to include
social and environmental indicators when setting their goals. Therefore, the key research focus is determining how to extract
valuable information from complex, competitive environment to establish more sensible goals.
Data Envelopment Analysis (DEA), a non-parametric linear programming method used to assess the relative efficiencies
of a set of homogeneous Decision Making Units (DMUs) that have multiple inputs and outputs [18], has been extensively
used for goal setting to ensure that the evaluations are based on DMUs that represent best practice [26]. In this case, the goal
setting is similar to benchmarking, which is the process of comparing performance with industry best practice [8]. DMUs
that have poor performances can learn from target companies and more importantly, understand the specific competitive
advantages in these companies. Benchmarking is therefore a comparison and monitoring process to determine best practice
and future performance planning, which is consistent with the idea of goal setting. In addition to obtaining the relative effi-
ciencies, DEA can also reveal the efficient DMUs on the efficient frontier, which give guidance for the improvements needed
in the inefficient DMUs [17].
Decision makers in previous studies have usually directly taken the DMUs on the efficient frontier as the goals; however,
this assumption presents certain problems and difficulties. First, the goals on the efficient frontier may not be achievable
[26]. As traditional DEA methods use a convex combination of the DMUs on the efficient frontier to measure the relative effi-
ciencies [7], the goals for the inefficient DMUs are either a hypothetical composite DMU or an efficient DMU; therefore, when
the decision makers directly take the composite DMU as the goal, it may be unattainable as it does not actually exist. Second,
it is extremely challenging for suppliers with poor operations to achieve best practice within one operating cycle, especially
when they are far from the efficient frontier [22], that is, choosing a distant goal can be unrealistic for underachieving DMUs.
Third, it is impractical for an inefficient DMU to learn from multiple pioneers simultaneously as the best practitioners may
have different operational strategies. Some suppliers may fix their inputs and increase production capacity to maximize out-
puts, while others may improve their resource utilization to reduce their input consumption. Therefore, without proper
information processing and analysis, decision makers may directly take the efficiency evaluation results as the sole basis
for their goal setting.
To overcome these issues, a more practical method for appropriate goal setting is proposed in this paper that considers
sustainability and avoids the setting of unworkable goals. The proposed model incorporates a dynamic structure into the
goal setting process and regards improvements as a multi-period process. Therefore, this paper develops a sequence of
attainable goals for inefficient suppliers rather than a single ultimate goal. In addition to economic sustainability, this paper
also takes social responsibility and environmental protection into consideration. For example, companies that are seeking to
implement SSCM need to ensure safe working environments [35], and suppliers who want to maintain long-lasting cooper-
ation need to assess customer satisfaction from a social perspective. However, only degrees from strong satisfaction to strong
dissatisfaction can be used to assess linguistic indicators such as customer satisfaction. Due to the complexity and ambiguity
of linguistic information, the ability to provide more accurate values to express these types of degrees often exceed reality.
However, fuzzy set theory could be used to address the uncertainty in linguistic information [5,2,36]. First proposed by
Zadeh [40], fuzzy set theory has been used in combination with many different methods to reduce uncertainty. In addition
to dealing with imprecise, unclear, and subjective judgment, attitude, and human behavior data, fuzzy numbers have been
found to be particularly powerful for modeling real-world parameters that cannot be accurately measured [23]. Many stud-
ies have used triangular fuzzy numbers to describe uncertain information in different scenarios as it has been a commonly
used tool to address uncertainty [33]. Therefore, due to their relative simplicity and calculation convenience, this paper pro-
cessed the linguistic information using classical triangular fuzzy numbers, with the degrees of customer satisfaction
expressed in linguistic terms being converted into triangular fuzzy numbers.
Therefore, the supplier goal setting focus in this paper involves sustainable development, the dynamic structure and fuzzy
information. To fully characterize these concepts, a detailed information classification and analysis process was conducted,
as shown in Fig. 1. To the best of our knowledge, there has been no research that has considered both a dynamic structure
and uncertainty of supplier goal setting in a sustainable context. Therefore, this paper provides a viable goal sequence across
multiple periods in the presence of uncertain information. To set multiple incremental improvement goals, a context-
dependent DEA is adopted that evaluates the DMUs against specific contexts [28], which allows for the DMUs to be evaluated
against different contexts, thereby be divided into different layers through changing the evaluation context. The measure of
supplier attractiveness and progress is introduced to select the most appropriate goal in each layer. Therefore, to compre-
hensively consider economic, social and environmental sustainability, this paper incorporates the dynamic structure and
uncertain linguistic information into a context-dependent DEA model and employs classical triangular fuzzy numbers to
address the uncertainty.
The remainder of this paper is organized as follows. Section 2 provides a brief overview of the literature related to DEA
and benchmarking, Section 3 gives the problem statement and preliminaries, Section 4 develops the uncertain dynamic DEA-
based benchmarking models to layer suppliers and measure the attractiveness and progress, Section 5 gives a numerical
example, and Section 6 concludes the paper.
46 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Fig. 1. Information classification and analysis process.

2. Literature review

Supplier goal setting has been extensively studied with DEA from a benchmarking perspective, and previous studies have
provided many insights. DEA has been one of the most widely applied tools for performance evaluations and benchmarking
problems [37,20]. Charnes et al. [6] first proposed a CCR model that assumed a constant returns to scale (CRS), after which
Banker et al. [4] modified the CCR model to the BCC model that assumed a variable returns to scale (VRS). However, the basic
CCR and BCC models were both radial models that limited the input and output changes to the same proportion, and tradi-
tional DEA models are either input-oriented (minimize inputs without reducing outputs) or output-oriented (maximize out-
puts without increasing inputs). Therefore, in the past few decades, a variety of applications have been proposed with the
improvement of traditional DEA models [32,43,12].
Traditional DEA models are static that focus only on the initial inputs and final outputs and treat supply chain operations
as black boxes [41]. However, supply chain operational activities are usually continuous, with decisions in the current period
often having an effect on future performances. Further, it is necessary to consider the time effect under SSCM considerations,
that is, a dynamic structure is needed when formulating the goal setting model. Compared with the static DEA approach, the
dynamic DEA links the multiple operation cycles through carry-over activities and is able to reflect the impact of the current
decisions on future performances, which well meets the long-term requirement in the sustainable background [31]. Rashidi
and Saen [24] used a dynamic DEA to measure supplier sustainability over three consecutive periods, with the inefficient
suppliers being projected towards the efficient frontier step by step. Shabanpour et al. [29] combined a dynamic DEA with
artificial neural networks to forecast the future efficiencies of green suppliers. Piao et al. [21] used three DEA models to dis-
tinguish the weak and strong disposability of undesirable outputs to assess the environmental efficiencies and the dynamic
change trends in 30 Chinese provinces over several years. In line with these studies, to reflect the intertemporal effects of
operational decisions on neighboring periods, this paper developed a dynamic DEA model to reflect the time effects in sus-
tainable goal setting.
Supplier goal setting under sustainability has new characteristics and trends that require a comprehensive assessment of
economic, social and environmental indicators. However, many indicators such as customer satisfaction and environmental
pollution cannot be precisely evaluated and can only be reviewed based on best to worst degree parameters, which can give
rise to uncertainties when applied to mathematical models [2]. Fuzzy set theory proposed by Zadeh [40], therefore, has been
frequently employed to handle uncertainty in real-world problems, and due to its wide applicability and functionality, has
been combined with various methods in many fields. For example, Roudneshin and Azadeh [25] developed a unique multi-
objective fuzzy model to optimize oil sludge management, Liao et al. [15] proposed a hesitant fuzzy linguistic best worst
method for hospital performance evaluations, and Akman [1] combined fuzzy thought with the C-means method to deter-
mine green supplier performances and identify which supplier should be included in a green supplier development program.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 47

Some fuzzy DEA (FDEA) models have been developed by introducing uncertain indicators into the DEA model [14], which
have been adopted in several different settings because of the proven flexibility and reasonable depiction. The FDEA models
have been found to relax the indicator quantification requirements and broaden the DEA model applications. Therefore, this
paper adopted one of the most classical fuzzy numbers, the triangular fuzzy number, to express the linguistic information in
the goal setting problem as it has the relative simplicity of calculation while retaining the advantages of the FDEA model.
By solving the DEA models, the relative efficiencies can be determined and a set of efficient DMUs can be identified to
form the efficient frontier [17] onto which the inefficient DMUs can be projected to reveal their improvement directions.
However, both the CCR and BCC models are radial models that limit the input and output changes to the same proportion.
Based on the additive model proposed by Charnes et al. [7], Cooper et al. [9] proposed an additive model with a Range
Adjusted Measure (RAM) that overcame the above limitations. The RAM-DEA model with non-radial characteristics puts
the slack variables directly into the objective function, thereby limiting the slack variable coefficients to a range, which in
turn ensures that the efficiencies are not biased due to the different input and output improvement proportions [10]. Mika
et al. [19] included a technology input index into the RAM model, which revealed that technological innovation played an
important role in improving industrial efficiency. Tsolas [34] applied a RAM-DEA model to evaluate construction engineering
efficiencies, and designed composite evaluation indicators to study the factors that affected the efficiency. The main reason
behind the RAM-DEA model choice for this paper is that it unifies the input-oriented and output-oriented models into a sin-
gle framework and does not limit the variable change proportions.
Generally, decision makers directly select the DMUs on the efficient frontier as benchmarks for the future improvement of
the inefficient DMUs. However, the goals located on the efficient frontier may be impractical because the underachieving
DMUs may be too far from the efficient DMUs for the goal to be achieved in the short term or the goal is a linear combination
of all DMUs so does not exist [22]. Therefore, a more reasonable approach for inefficient DMUs is to set implementable goals
and design stepwise improvement plans.
To sum up, traditional radial DEA models limit the DMUs to improving their inputs and outputs in the same proportion,
are therefore do not allow for flexible improvements. Further, as basic static DEA models do not consider the time effect and
ignore the difficulty of achieving goals within one operating cycle, which runs counter to the need to achieve long-term sus-
tainable development. Therefore, to ensure that achievable goal sequences for inefficient suppliers can be set that consider
comprehensive economic-social-environmental sustainability, this paper developed a dynamic RAM-DEA model that
included uncertain information within a context-dependent DEA.

3. Problem statement and preliminaries

3.1. Problem statement

Traditionally, suppliers are the roles that provide raw materials to manufacturers; however, in this paper, the suppliers
involved in the goal setting issue are any supply chain member that provides products or materials to downstream mem-
bers, which means that the supplier performances affect many supply chain aspects, such as procurement costs, product
quality, prices, and customer satisfaction. To reflect comprehensive sustainability in the goal setting, based on the TBL, the
economic, social and environmental aspects are taken into account. The description of the variables is given in the
following.
As suppliers can achieve economic sustainability through technical research and financial management, their technical
and financial capabilities can be used to measure economic performance. SSCM also stresses social sustainability to address
the needs of laborers and customers, that is, ensuring a safe work place and a humane working environment and improving
customer satisfaction to establish longstanding cooperation are as significant as economic growth. Therefore, the cost of
work safety and the degree of customer satisfaction need to be incorporated to reflect the social care of the companies. Envi-
ronmental sustainability is achieved by reducing the exhaust gas, water and residue emissions; therefore, environmental
pollution is an undesirable output [13,42]. When there is production inefficiency, to improve efficiency, it is necessary to
distinguish the desirable outputs from the undesirable outputs to be reduced. Previous studies have used three main meth-
ods to deal with undesirable outputs: directly treat the undesirable outputs as inputs; apply data transformation to the
undesirable outputs and then use the adapted data as normal outputs [27]; and model the undesirable outputs in the
slack-based DEA. In this paper, environmental pollution is taken as an input indicator in the RAM-DEA model, which is devel-
oped based on the slack-based DEA. As sustainable development is a long-lasting process, current performances can affect
future decisions; therefore, it is necessary to include carry-overs to connect the neighboring periods. Account receivables are
payments that suppliers collect from downstream members for the supply of products in the previous period that have not
yet been collected and accounts payable are the commissions suppliers should pay but have not yet paid, and are treated as
expenses carrying over to the next period. Therefore, accounts receivables as desirable carry-overs and account payables as
undesirable carry-overs are incorporated in this paper.
In summary, the dynamic supplier operating process is shown in Fig. 2 and Table 1 lists the indicators to be used in the
developed model.
48 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Period t
Technical and financial Value of raw material
capacity

Cost of work safety

Degree of customer Environmental
satisfaction pollution

Accounts receivable Accounts payable

Technical and financial Value of raw material

capacity

Cost of work safety

Degree of customer Environmental
Period t+1
satisfaction pollution

Fig. 2. Dynamic supplier operation structure.

Table 1
Indicators.

Type Indicator Notation Explanation

Input Technical and financial capability xtaj Economic input of jth supplier at period t
Cost of work safety xtbj Social input of jth supplier at period t

Output Value of raw material ytaj Desirable output of jth supplier at period t
Environmental pollution ytbj Undesirable output of jth supplier at period t
Degree of customer satisfaction ~tj
y Fuzzy output of jth supplier at period t

Carry-over Accounts receivable t ðtþ1Þ Desirable carry-over of jth supplier at period t

cjgood
Accounts payable t ðtþ1Þ Undesirable carry-over of jth supplier at period t
cjbad

3.2. Preliminaries

Of the above-mentioned indicators, customer satisfaction has usually been described using degrees in linguistic term,
which contain uncertain information. Fuzzy numbers have been found to be useful for expressing uncertain parameters,
properties, geometry and initial conditions [30]. Therefore, one of the most classical fuzzy numbers, triangular fuzzy num-
bers, were employed in this paper to express the degree of customer satisfaction. Triangular fuzzy numbers are not crisp
values but a connected set of possible values, where the weight of each possible value is between 0 to 1, with the weight
being the triangular fuzzy number membership function [11].
In this problem, the degrees of customer satisfaction were obtained from a customer investigation that employed a 5-
point Likert language Scale to illustrate the degree of satisfaction, after which the results were converted into triangular
fuzzy numbers, as shown in Table 2. Based on the triangular fuzzy number addition and division, the opinions of different
customers were synthesized to obtain an integrated customer satisfaction degree for each supplier. Generally, two mathe-
matical approaches have been used for fuzzy arithmetic: the a
cut approach and the extension principle approach. As most
existing computational methods used the a
cut due to its simplicity, therefore, the a
cut was used in this paper to con-
vert the triangular fuzzy numbers into interval numbers.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 49

Table 2
Degrees of customer satisfaction.

The degree of customer satisfaction Triangular fuzzy number

Strong dissatisfaction (10, 20, 30)
Dissatisfaction (30, 40, 50)
Medium (50, 60, 70)
Satisfaction (70, 80, 90)
Strong satisfaction (80, 90, 100)


 
Definition 1. The space of fuzzy numbers is denoted by Rf . If a 2 Rf ¼ aL ; aM ; aU , where 0 6 aL 6 aM 6 aU , then a is a

triangular fuzzy number. When aL ¼ aM ¼ aU ; a is a crisp value. For 0 < a 6 1; a
cut of a 2 Rf is defined by
 
½aa ¼ fx 2 R j aðxÞ P ag with ½aa ¼ ua
; uaþ . Therefore, the membership function of the triangular fuzzy number is defined
as follows, and the a
cut for the triangular fuzzy number is shown in Fig. 3.

8 x
aL
< aM
aL ; a 6 x 6 a
> L M

la ðxÞ ¼ aaU
a
xM ; aM 6 x 6 aU

U
ð1Þ
>
:
0; otherwise

~ for which the addition and division operators to

~ and b,
Definition 2. Suppose there are two triangular fuzzy numbers a
obtain the degree of customer satisfaction are as follows.

   
L M U
Addition : a þ b ¼ aL þ b ; aM þ b ; aU þ b ;

 
 
Div ision : a = b ¼ aL =b ; aM =b ; aU =b :
L M U

In this study, the arithmetic mean value was adopted to obtain the integrated degree of customer satisfaction. For exam-
ple, after 10 customers expressed their opinions using Table 2 for Supplier A, 10 triangular fuzzy numbers a ~10 were
~1 to a
given. Then, the addition and division operators were employed to calculate the arithmetic mean value for a ~10 , with
~1 to a
the result being used to describe Supplier A’s customer satisfaction degree.

4. Modelling

In this section, an uncertain dynamic benchmarking model is developed that combines the RAM-DEA with context-
dependent DEA to address the sustainable supplier goal setting issue. As mentioned, rather than a single ultimate goal, an
inefficient DMU is provided with a sequence of implementable goals that gradually lead to the ultimate goal.

Fig. 3. The a
cut of triangular fuzzy number.

50 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Benchmarking often assumes that each supplier uses X inputs to get Y outputs, with the C carry-overs being considered
the inputs in the current period and outputs in the subsequent period. The production possible set
PPS ¼ fð X; Y; C ÞjXcanproduceY; C g satisfies the convexity and VRS.
Suppose that during period t ¼ 1; . . . ; T, the input, output and carry-over of the jth ð j ¼ 1; . . . ; nÞ supplier are
 
xtij ; ytrj ; cgood
hjt ; chjt , where yrj contains the undesirable output ybj that is inevitably generated when producing the desirable
bad t t

product ytj ; cgood

hjt refers to desirable carry-overs such as the accounts receivable, and cbad
hjt refers to the undesirable carry-
overs such as the accounts payable. Therefore, the PPS of an uncertain dynamic DEA model can be characterized as (2),
and Fig. 4 shows the dynamic operating process of supplier j from period t to t þ 1.
 8
 > X n
 >
> Xt P ktj xtij
 >
>
 >
>
 >
> j¼1
 >
>
 >
> X
 >
>
n
 >
> Yt 6 ktj ytj
 >
>
 >
>
 >
>
j¼1
 >
>
 >
> X n
 >
>
 >
> Y tb P ktj ytbj
 >
>
 >
> j¼1
 >
>
 >
> X
 >
>
n


 > t
ktj cgood
 > C good 6
>
> hjt h ¼ 1; . . . ; hgood
 < j¼1
PPS ¼ ð X; Y; C Þ ð2Þ
 >
> X n
 >
>
 >
> C tbad P ktj cbad
hjt ðh ¼ 1; . . . ; hbad Þ
 >
>
 >
> j¼1
 >
>
 >
>
 >
> Xn Xn
 >
> ktj cgood ¼ ktþ1 cgood
 >
> hjð t;tþ1 Þ j hjðt;tþ1Þ
 >
> j¼1
 >
>
j¼1

 >
>
 >X
> n Xn
 >
>
 >
> ktj cbad
hjðt;tþ1Þ ¼ ktþ1 cbad
 >
>
j hjðt;tþ1Þ
 >
> j¼1 j¼1
 >
>
 >
>
 >
> Xn
 >
> ktj ¼ 1; ktj P 0; i ¼ 1; . . . m; r ¼ 1; . . . ; s; t ¼ 1; . . . ; Tg
>
:
 j¼1

As the additive model with RAM allows the inputs and outputs to be adjusted in different proportions, the constraints in
the PPS can be modified to the following Eqs. (3), (4) to obtain model (6). The DMUs to be evaluated are accordingly substi-
tuted into the model and the relative efficiencies of all DMUs can be obtained by solving the model n times. Each DMU selects
its input and output weights to maximize efficiency [16], and a DMU is considered efficient if the efficiency is equal to 1;
however, if the efficiency is less than 1, the DMU is considered inefficient. In this paper, environmental pollution was taken
as the input to meet the performance assessment requirements for pollutant reduction. The constraints for the inputs, unde-
sirable outputs, and carry-overs are expressed as follows.
X
xtik ¼ ktj xtij þ s
it
j2F ðJ l Þ
X
ytbk ¼ ktj ytbj þ s
bt
ð3Þ
j2F ðJ
Þ l

X
hkt ¼
cbad ktj cbad
hjt þ sht
bad

j2F ðJl Þ

lþ1
where s


it ; sbt ; sht indicate the slacks for the inputs, undesirable outputs and carry-overs, J
bad
¼ J l
El ; J 1 ¼ DMU j ; j ¼ 1; . . . ; n is
the set of all n DMUs and El is the set of efficient DMUs.
The constraints for the desirable outputs and carry-overs are listed in the following equations.
X
ytk ¼ ktj ytj
sþrt
j2F ðJ
Þ l

X ð4Þ
cgood
hkt ¼ ktj cgood good
hjt
sht
j2F ðJ l Þ
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 51

Fig. 4. Dynamic supplier operating process.

 
good
where sþ
rt ; sht are the slacks denoting the shortfalls for the desirable outputs and carry-overs, and the definition for F J l is
the same as in Eq. (3).

Definition 3. The slacks-based efficiency for the evaluated supplier in period t is defined as follows.
Pm


Ps þ þ
Phgood good þ Phbad bad

i¼1 sit Rit þ þ h¼1
r¼1 srt Rrt sht Rht þ h¼1 sht Rht
hk ¼ 1
ð5Þ
m þ s þ hgood þ hbad
     n o n o

where R
it ¼ 1= max xij
min xij ; Rþ rt ¼ 1= max yrj
min yrj ; Rþ ht ¼ 1= max cgood
hjt
min cgood hjt ; and
j¼1;...;n j¼1;...;n j¼1;...;n j¼1;...;n j¼1;...;n j¼1;...;n
 n o n o 
R

ht ¼ 1= max cbad
hjt
min cbad hjt .
j¼1;...;n j¼1;...;n

In this paper, the objective function value is the efficiency loss value for the evaluated DMU, the larger the objective func-
tion value, the lower the efficiency level.

Definition 4. If and only if all slacks are equal to 0 will the objective function value be equal to 0, at which point hk ¼ 1,
which indicates that the supplier being evaluated fully utilizes the inputs to obtain the desirable outputs and is on the
efficient frontier. Otherwise, there is still a need for improvement.
Therefore, the uncertain dynamic RAM-DEA efficiency evaluation model is as follows.
Pm Ps Phgood Phbad
s
R
þ sþ Rþ þ
good þ
sht Rht þ sbad R

OV ¼ max xt i¼1 it it r¼1 rt rt h¼1
mþsþhgood þhbad
h¼1 ht ht

s; k
8
>
> Eqs:ð3Þ
ð4Þ
>
>
>
>
>
> Xn Xn
>
> ktj cgood ¼ ktþ1 cgood
>
> hj ð t;tþ1Þ j hjðt;tþ1Þ
>
>
>
>
j¼1 j¼1
>
>
>
>X X
>
>
n n
ð6Þ
< ktj cbad
hjðt;tþ1Þ ¼ ktþ1
j cbad
hjðt;tþ1Þ
s:t: j¼1 j¼1
>
>
>
> X
>
>
>
> ktj ¼ 1
>
>
>
> j2F
>
>
>
>
>
> ktj ; S
it ; Sþrt ; Sgood bad
>
> ht ; Sht P 0
>
>
:
i ¼ 1; . . . m; r ¼ 1; . . . s; t ¼ 1; . . . ; T
where k is the intensity variable affecting the objective function value and slacks in the constraints, xt is the weight in period
t. As higher weights should be given to recent periods, different weights are given to the periods based on the age of the data
in the numerical example.
Based on the extreme difference in all inputs, outputs and carry-overs, the adjustment ranges for all slacks are defined as
follows.
52 X. Zhou et al. / Information Sciences 545 (2021) 44–64

X
0 6 s

it ¼ xik

t
ktj xtij 6 R
it
j2F ðJ l
Þ
X
0 6 sþ
t ¼ ktj ytj
ytk 6 Rþt
j2F ðJ l Þ
X
0 6 s

bt ¼ ybk

t
ktj ytbj 6 R
bt
ð7Þ
j2F ðJ l Þ
X þ
0 6 sgood
ht ¼ ktj cgood good
hjt
c hkt 6 Rht
j2F ðJ l
Þ
X

06 sbad
ht ¼ cbad
hkt
ktj cbad
hkt 6 Rht
j2F ðJl Þ

 
where the definition of F J l is the same as in Eq. (3).
P t
When the model obtains the optimal solution, the sum of all intensity variables is equal to 1, that is, j2F ðJl Þ kj ¼ 1, and
ktj
combined with the constraint that is non-negative, it indicates that the production technology has VRS. Then, the objective
function value that maximizes efficiency is between 0 and 1 and the RAM efficiency of the jth ð j ¼ 0; . . . ; nÞ supplier in period t
is converted into the following inequality.
Pm


Ps þ þ
Phgood good þ Phbad bad

i¼1 sit Rit þ þ h¼1
r¼1 srt Rrt sht Rht þ h¼1 sht Rht
0 6 hk ¼ 1
61 ð8Þ
m þ s þ hgood þ hbad
As shown in Fig. 5, the horizontal axis represents the input and the vertical axis represents the output. The efficient pro-
jection direction of the jth ð j ¼ 0; . . . ; nÞ supplier is given by JF. The arc EFG constitutes the frontier for the RAM efficiency, JE
and JG are the traditional input-oriented and output-oriented DEA methods, and JH is an inefficient projection.

4.1. Layering procedure

As mentioned, the goal for an inefficient DMU may be a virtual DMU generated by a linear combination of DMUs, which
would be difficult to imitate. Another problem is that due to resource allocation difficulties, achieving the ultimate goal pro-
vided in traditional goal setting models is impossible in one operating cycle, especially when the inefficient DMU is far from
best practice. Therefore, a stepwise method to approach the final goal gradually is proposed for inefficient DMUs, which has a
goal sequence rather than a single ultimate goal. The inefficient DMU first selects a goal that is easier to achieve, and then
gradually learns from the higher-level goals to finally achieve the ultimate goal. In this way, the difficulty of implementing
resource scheduling in only one operating cycle can be avoided.
In summary, all DMUs are divided into several efficient frontiers based on the initial efficiencies. The inefficient DMUs in
the last layer gradually learn from the relatively efficient DMUs in the higher-levels, with the most appropriate DMU in each
layer being selected as the goal to form the goal sequence to gradually approach the final goal. Therefore, all DMUS are lay-
ered based on the context-dependent DEA proposed by Seiford and Zhu [28]. Context-dependent DEAs evaluate the DMUs
against different contexts that represent the efficient frontiers composed of the DMUs at a specific performance level. Mod-
ifications were made to the basic model to accommodate the additive model with RAM, the dynamic structure, and the fuzzy
data. To develop the goal sequence, J 1 ¼ DMU j ; j ¼ 1; . . . ; n is taken as the set of all n DMUs and J lþ1 ¼ J l
El , where
n o
El ¼ DMU k 2 J l jOV ¼ 0 is defined, and OV is the optimal value for the model.
More specifically, after sequentially substituting n DMUs into the model, the efficiencies of all DMUs are obtained, with
the DMUs with objective functions equal to 0 (the efficiency is equal to 1) forming the original efficient frontier. If the DMUs
in the original efficient frontier are deleted and the efficiencies for the remaining DMUs recalculated, the relatively efficient
DMUs in the remaining DMUs then form a new second-level efficient frontier. This process is continued until there are no
DMUs left. In this way, several efficient frontier levels are determined, with El being the lth -level efficient frontier.
When l ¼ 1, model (6) is the general RAM-DEA model, with the DMUs in El forming the first efficient frontier. When l ¼ 2,
model (6) generates the second-level efficient frontier after the efficient DMUs in the first layer are removed. By iteratively
performing the model until all DMUs are layered, all DMUs are divided into several layers. The following algorithm details
the layering process introduced in model (6).

Step 1: Set l ¼ 1, with J l referring to the set of all n DMUs;

Step 2: Calculate the relative efficiencies of all DMUs in J l using model (6), the lth -efficient frontier El is composed of the effi-
cient DMUs;
Step 3: Remove the efficient DMUs from J l , let J lþ1 ¼ J l
El . If J lþ1 ¼ /, then stop;
Step 4: Let l ¼ l þ 1, return to Step 2;
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 53

Fig. 5. RAM efficiency.

Stopping rule: If J lþ1 ¼ /, the algorithm stops.

When all DMUs are divided into several layers after the layering procedure, the goal sequence for the inefficient DMUs in
the last layer is determined by choosing one DMU in each of the higher levels (Fig. 6). For example, an inefficient DMU in
layer l can improve its performance by imitating the goal sequence consisting of efficient DMUs from layer l
1 to layer
1. This gradual improvement overcomes the limitations of the traditional method as it alleviates the difficulties of targeting
a single ultimate goal. As the DMUs in the same layer are all equally efficient, the most suitable DMU among the actual exist-
ing DMUs in each layer needs to be selected.

4.2. Measure of relative attractiveness

The layering process divides a group of DMUs into several layers, and the DMUs located in the same layer are equally
efficient. When there are multiple equally efficient DMUs in the current layer available for selection, it is necessary to iden-
tify the relatively better DMU. Suppose that DMU x and DMU y are on the same efficient frontier and it is difficult to determine
which is more appropriate in terms of the efficiency; therefore, an evaluation context is needed to distinguish the most suit-
able DMU to be the benchmark in each layer.
The context-dependent DEA evaluates the DMUs against different contexts, which are efficient frontiers consisting of the
DMUs with specific efficiency levels. If the DMUs from the original efficient frontier are removed, the relatively efficient
DMUs in the remaining DMUs form a new second-level efficient frontier, with second-level efficiencies being lower than
the DMUs on the original frontier. If the DMUs on the second-level frontier are removed, a third-level efficient frontier is
formed with the remaining DMUs. This continues until all DMUs are layered. When the DMUs with poor performances
are used as the evaluation context, a measurement called relative attractiveness is obtained for each higher level DMU
[28]; for example, when measuring the relative attractiveness of the DMUs on the first level, the second-level efficient fron-
tier can be selected as the evaluation context. Similarly, the third level can be used as the context for evaluating the relative
attractiveness of the DMUs in the second layer.

Fig. 6. Layering of DMUs.

54 X. Zhou et al. / Information Sciences 545 (2021) 44–64

The relative attractiveness of DMU k in the lth ð1 6 l 6 L
1Þ layer, El , is determined using the following model.
Pm

Ps þ þ Phgood good þ Phbad bad

s R þ s R þ sht Rht þ s Rht
Ak ¼ minxt i¼1 it it r¼1 rt rt h¼1
mþsþhgood þhbad
h¼1 ht
s; k
8 t X t
>
> xik P kj xtij
s
it
>
>
>
> j2F ðE Þ lþ1
>
> t X t
>
> 6 kj ytj þ sþt
>
> y
>
>
k
>
> j2F ðE Þ lþ1
>
> t X t
>
> P kj ytbj þ sþbt
>
> y
>
>
bk
>
> j2F ð E lþ1
Þ
>
> X t good
> good
>
>
> chkt 6 kj chjt þ sgood
>
>
ht
ð9Þ
< j2F ðElþ1 Þ
s:t: X t
> hkt P
cbad kj cbad
hjt
sit
bad
>
>
>
> j2F ðE Þ
>
>
lþ1

> X t good
>
>
X tþ1 good
>
> k j c hjðt;tþ1Þ ¼ kj chjðt;tþ1Þ
>
>
>
> j2F ðElþ1 Þ j2F ðElþ1 Þ
>
> X t X tþ1
>
>
>
> hjðt;tþ1Þ ¼
kj cbad kj cbad
>
> hjðt;tþ1Þ
>
> j2F ðElþ1 Þ j2F ðElþ1 Þ
>
>  
>
>
> ; ktj ; S
it ; Sþrt ; Sgood
> 8j 2 F E
lþ1 bad
> ht ; Sht P 0
>
:
i ¼ 1; . . . m; r ¼ 1; . . . s; t ¼ 1; . . . ; T
where l is the number of layers obtained through the layering procedure.
With a specific evaluation context, model (9) enables us to select the most attractive DMU from multiple equally efficient
DMUs. In model (9), the efficient frontier, Elþ1 , represents the evaluation context for measuring the relative attractiveness for
DMUs in El . As a larger Ak indicates that the DMU k is farther away from the evaluation context Elþ1 , which is composed of the
DMUs with lower efficiency than DMU k ; therefore, the larger the value of Ak , the more attractive the DMU k . The DMUs in El
can be ranked based on the relative attractiveness to determine the best.

4.3. Measure of relative progress

Contrary to the relative attractiveness, when an efficient frontier composed of better-performing DMUs is selected as the
evaluation context, the DMUs located on the lower level are able to obtain the relative progress, which represents the dis-
tance from the underperforming DMUs to the upper efficient frontier.
The relative progress for a specific DMU k 2 El ð2 6 l 6 LÞ is measured using the following model.
Pm

Ps þ þ Phgood good þ Phbad bad

s R þ s R þ sht Rht þ s Rht
Pk ¼ maxxt i¼1 it it r¼1 rt rt h¼1
mþsþhgood þhbad
h¼1 ht
s; k
8 t X t
>
> xik ¼ kj xtij þ s
it
>
>
>
> j2F ðE Þ
l
1
>
> X t
>
>
>
> y t
¼ kj ytj
sþt
>
> k
>
> j2F ðEl
1 Þ
>
> X t
>
>
> kj ytbj þ sþbt
> ybk ¼
> t
>
>
>
> j2F ðE Þ
l
1
>
> X t good
>
>
> good
> hkt
c ¼ kj chjt
sgood
>
> ht
>
< j2F ðEl
1 Þ ð10Þ
s:t: X t
> hkt ¼
cbad hjt þ sit
kj cbad bad
>
>
>
> j2F ðE Þ
>
>
l
1

>
> X t good X tþ1 good
>
> kj chjðt;tþ1Þ ¼ kj chjðt;tþ1Þ
>
>
>
>
>
> j2F ð Elþ1
Þ j2F ð E lþ1
Þ
>
> X t X tþ1
>
>
>
> kj cbad
hjðt;tþ1Þ ¼ kj cbad
>
>
hjðt;tþ1Þ
>
> j2F ðE Þ
lþ1
j2F ðE Þlþ1
>
>  
>
>
>
> 8j 2 F El
1 ; ktj ; S
it ; Sþrt ; Sgood bad
>
> ht ; Sht P 0
>
:
i ¼ 1; . . . m; r ¼ 1; . . . s; t ¼ 1; . . . ; T
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 55

Each evaluation context, El
1 , represents a direction heading the inefficient DMU in El to improve its performance, with a
larger P k indicating that the DMU k in El needs to make greater progress to catch up with the DMUs in El
1 , therefore, a smaller
P k is preferred. Similar to relative attractiveness, the DMUs in El can be sorted according to their relative progress to identify
the best.

4.4. Goal sequence selection

Using the relative attractiveness and relative progress measures, the following comprehensive index for the selection of
goal sequence is formulated. For DMU k in El , the next goal in El
1 is the one that has the largest comprehensive index value.
n o

Benchmarkk ¼ arg max Bkp ¼ x1 Ap
x2 Pp : p 2 El
1 ð11Þ

where x1 ; x2 are the weights given to the two measurements and can be determined based on the decision makers’
preference.
From a screening of the comprehensive index value Benchmarkk , the most appropriate DMU in each layer can be identified
as the goal and forms part of the goal sequence. Each improvement step, therefore, involves a single real operating goal, and
the aforementioned limitations can be compensated.

4.5. Model processing

Before solving the models, the degrees of customer satisfaction expressed by the triangular fuzzy numbers in models (6),
(9), (10) are transformed into interval numbers using a
cut. At a given level of confidence a 2 ½0; 1, there has


C a ¼ xjlc ðxÞ P a for a triangular fuzzy number C [3]. The left endpoint (LE) and the right endpoint (RE) are determined
using Eq. (12).

h    i
j þ a yj
yj
ytj ¼ ytL j
a yj
yj
; ytU ð12Þ
tM tL tU tM

where a is the confidence level given in advance, the closer a is to 1, the closer the variable is to the crisp value, and the
closer a is to 0, the stronger the uncertainty of the indicator and the larger the interval value of the cut.

 
After the fuzzy variable transformations, the degrees of consumer satisfaction are expressed as ytj ¼ ytLj ; yj
tR
. Because the
a
cut is used to reduce the triangular fuzzy numbers into interval numbers, the decision makers attitudes influence the
degree of customer satisfaction, that is, an optimistic decision maker believes that more desirable outputs can be produced,
RE, which is the larger value of the interval number, is used to obtain the optimistic efficiencies. Conversely, a pessimistic
decision maker is conservative about the desirable outputs, LE, which is the smaller value of the interval number, is used
to measure the pessimistic efficiencies. In the following section, the RE was first substituted into the model for the solution,
and then the LE was substituted to conduct sensitivity analysis.

4.6. Procedure summary

In this subsection, the entire goal setting process is summarized, and the roles of models (6), (9) and (10) and their rela-
tionships are explained.
The three models proposed in this paper are all based on the context-dependent DEA. As shown in Fig. 7, model (6) is first
used to conduct the efficiency evaluation and divide all the DMUs into several layers based on their efficiencies. The 10 DMUs
in Fig. 7 were divided into three layers, with the DMUs in each layer having the same efficiencies as they are located on the
same efficient frontier. The DMUs in the first layer have better performance than the DMUs in the second and the third lay-
ers, and the DMUs in the second layer have better performance than the DMUs in the third layer. Therefore, the DMUs in the
last layer select appropriate DMUs in the first two layers as their gradual improvement goals.
Then, models (9) and (10) are used to select the most appropriate DMUs on the higher-levels for the inefficient DMUs on
the last layer. More specifically, model (9) is used to determine the relative attractiveness of the DMUs located on the higher
levels against the DMUs on the lower levels. Therefore, the right-hand sides of the inequalities in the constraints correspond
to the sets of DMUs on the lower efficient frontier, while the left-hand sides correspond to the evaluated DMUs on the higher
level. Model (10) is used to obtain the relative progress of the DMUs located on the lower levels against the DMUs on the
higher levels. Therefore, the right-hand sides of the inequalities in the constraints correspond to the sets of DMUs on the
higher efficient frontier, and the left-hand sides correspond to the evaluated DMUs on the lower level.
The relative attractiveness obtained from model (9) and the relative progress obtained from model (10) are combined to
select the most appropriate DMU from the DMUs with equal performance in each layer. In this way, the goal sequence for
inefficient DMUs on the last layer is determined.
56 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Fig. 7. Roles and relationship of three models.

5. Numerical example

In this section, an illustrative numerical example is given to illustrate the model and determine: (1) How to apply the
proposed model to determine the goal sequence for an inefficient supplier?; (2) Which supplier is the most sensitive to
the attitude change of decision maker?; (3) What is the difference between the proposed model and the traditional goal set-
ting models?.

5.1. Data

The operational efficiencies of 20 selected suppliers were evaluated and a goal sequence was identified for one of the
most inefficient. The data for this numerical example were from the 20 Asian container terminals in Lim et al. [16]. As
the problem had been updated and there were incomplete data, these data could not be adopted directly into the calcula-
tions and needed to be properly processed before being employed. Due to difficulty in accessing actual industry data, it is
believed that these processed data closely resembled the real-world conditions. The span of data was three periods to better
reflect the operations, and higher weights are given to more recent periods. Therefore, the weights in each period were: first
period = 0.3; second period = 0.3; and third period = 0.4. The original data sets for all suppliers in the three periods are
shown in Appendix A. The fuzzy variable data in the three periods after the treatment with a ¼ 0:3 are shown in Appendix
B. After determining the efficiencies, a goal sequence was then selected for an inefficient DMU.

5.2. Results

The data set directly treating the undesirable outputs as inputs was first employed for the efficiency calculations. After
solving the proposed uncertain dynamic RAM-DEA model, eight suppliers were found to be efficient and 12 suppliers were
h i
bj ¼
ybj þ M, where M ¼ max ybj þ 1
inefficient. The data set transformed using a linear monotone decreasing function, yt t t

was then applied for efficiency evaluation to verify the effect of different processing methods on the efficiencies. This data
transformation function makes the undesirable output negative and then adds a proper positive to make all data greater than
0. Therefore, the smaller the undesirable outputs the better are converted into the larger the desirable outputs the better.
Specifically, the transformation functions applied to the undesirable output data in the three periods were
bj ¼
ybj þ 1601; ybj ¼
ybj þ 1800; ybj ¼
ybj þ 1529, respectively. The original data and the corresponding transformed
y1 1 2 2 3 3

undesirable output data in the three periods are shown in Appendix B. The recalculated results from the adapted data indi-
cated that there was no significant change in the efficiencies derived from the data processed using different methods. There-
fore, in the following analysis, the method that directly treated the undesirable outputs as inputs was adopted.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 57

The first efficient frontier was formed by eight efficient DMUs. Using the layering procedure introduced in Section 4, two
other efficient frontiers were determined. Then, the relative attractiveness of all suppliers in each layer except the last layer
and the relative progress for all suppliers in each layer except the first layer were determined, as summarized in Table 3.
The results indicated that a goal sequence for an inefficient DMU in the last layer needed to be determined. As an exam-
ple, the next goal for Supplier 6, which had the lowest efficiency value in the third layer, was selected from the DMUs in the
second layer based on the value of weighted sum. As Benchmark16 was the largest in the second layer, Supplier 16 was
selected as the first goal for Supplier 6. Similarly, the next goal was selected from the DMUs in the first layer using the same
method. Therefore, the goal sequence for Supplier 6 was Supplier 16 then Supplier 5.
The layering results for all 20 suppliers and the goal sequence for Supplier 6 are shown in Fig. 8, from which it can be seen
that the stepwise improvements determined using the comprehensive index make up an implementable plan.
To obtain a more detailed evaluation, it was necessary to calculate the period efficiencies for each supplier to compare
their internal performances. This paper took the RE of the degree of customer satisfaction into the model as an example.
As can be seen in Fig. 9, most DMUs efficiencies within the three periods had a certain fluctuation, with the period efficien-
cies of Supplier 7 fluctuating the most: significant progress in the second period and a sharp decrease in the third period. If
the goals were simply set using overall efficiency, then it would be likely that Supplier 7 would be selected as a goal even
though it performed badly in the third period. However, by taking advantage of the screening offered by the comprehensive
index, decision makers can select a better goal from the multiple equal-efficiency suppliers in each layer, and avoid choosing
a period-inefficient supplier.

5.3. Comparison and discussion

To prove the robustness and superiority of the developed model, a sensitivity analysis and three comparative analyses
were conducted.

5.3.1. Sensitivity analysis of suppliers to a decision maker attitude change

An attitude change in the decision maker needs to be considered when dealing with fuzzy variables. Triangular fuzzy
numbers can be converted into corresponding interval numbers using a
cut with a given confidence level. The results

Table 3
Evaluation results for 20 suppliers.

No: Layer Efficiencies Attractiveness Progress Weighted Sum

S1 1 1.0000 0.0242 – 0.0121
S2 3 0.6840 – 0.2833 0.1417
S3 3 0.5875 – 0.1882 0.0941
S4 2 0.7821 0.2810 0.2430 0.2620
S5 1 1.0000 0.4082 – 0.2041
S6 3 0.5397 – 0.4814 0.2407
S7 1 1.0000 0.0147 – 0.0074
S8 2 0.7129 0.4031 0.3116 0.3574
S9 2 0.7686 0.4723 0.2593 0.3658
S10 1 1.0000 0.0596 – 0.0298
S11 2 0.9652 0.6039 0.2630 0.4335
S12 1 1.0000 0.1778 – 0.0889
S13 3 0.6717 – 0.3057 0.1528
S14 2 0.8601 0.4094 0.1461 0.2777
S15 1 1.0000 0.0762 – 0.0381
S16 2 0.9832 0.4547 0.5046 0.4796
S17 2 0.6997 0.3005 0.3355 0.3180
S18 1 1.0000 0.1377 – 0.0688
S19 2 0.9414 0.4430 0.0363 0.2396
S20 1 1.0000 0.0159 – 0.0079

Fig. 8. Goal sequence for inefficient supplier.

58 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Fig. 9. Period efficiencies of 20 suppliers.

shown in Section 5 were obtained using RE as the degree of customer satisfaction; therefore, in the following, LE was sub-
stituted into the proposed model to determine the efficiency of each supplier when the decision maker had a relatively neg-
ative attitude towards the degree of customer satisfaction. With a slight change in the degree of customer satisfaction, the
efficiencies of some suppliers changed. Table 4 shows the efficiencies of each supplier when the decision maker held differ-
ent attitudes toward the degree of consumer satisfaction. As Supplier 3 appeared to be the most sensitive to the degree of
customer satisfaction, this reminds Supplier 3 to make a greater effort to maintain good relationships with customers, in that
when the attitude of the decision maker changes, the efficiency of the most sensitive supplier experiences greater fluctua-
tions than the other suppliers.
Then, the same layering procedure was applied to all 20 suppliers after the LE substitution, from which it was found that
the suppliers were still divided into three layers. Through the screening of the relative attractiveness and progress, the goal
sequences for the inefficient suppliers were determined. It was found that the goal sequence for Supplier 6 did not change
(Fig. 10). However, the relative attractiveness and progress of some suppliers changed, which indicated that a suppliers’ rel-
ative attractiveness to the DMUs in the lower layers and the distance to the DMUs in the upper layers varies with the attitude
of the decision maker; therefore, the importance of maintaining good relationships with partners to ensure sustainability
should not be ignored.

Table 4
Sensitivity analysis based on RE and LE of fuzzy variable.

No: LE RE No: LE RE
S1 1.0000 1.0000 S11 0.9652 0.9652
S2 0.6840 0.6840 S12 1.0000 1.0000
S3 0.5875 0.5745 S13 0.6717 0.6717
S4 0.7821 0.7821 S14 0.8601 0.8601
S5 1.0000 1.0000 S15 1.0000 1.0000
S6 0.5397 0.5397 S16 0.9832 0.9832
S7 1.0000 1.0000 S17 0.6997 0.6997
S8 0.7129 0.7129 S18 1.0000 1.0000
S9 0.7686 0.7686 S19 0.9414 0.9414
S10 1.0000 1.0000 S20 1.0000 1.0000

LE: Efficiencies obtained by substituting the left endpoint (LE) of the fuzzy variable into the model;
RE: Efficiencies obtained by substituting the right endpoint (RE) of the fuzzy variable into the model.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 59

Fig. 10. Goal sequence for inefficient supplier with the substitution of LE.

5.3.2. Comparison with a traditional goal setting model

To further demonstrate the superiority of the proposed goal setting model, the same data set was put into the traditional
model to obtain the goal for an inefficient supplier.
From the general goal setting idea, the inefficient DMU were projected to the efficient frontier to achieve improvement, with

þ

the goal for an inefficient DMUj being DMU j is xaj
s

þ
a ; xbj
sb ; yaj þ sa ; ybj
sb ; yj þ s
þ
, where xaj ; xbj were the inputs, yaj was

þ þ þ
the desirable output, ybj was the undesirable output, yj was the desirable fuzzy output, and s
a ; sb ; sa ; sb ; s were the input and
output slacks. Previous studies have always viewed the supply chain operation as a black box. Therefore, in the traditional goal
setting model, the input data from period 1 and the output data from period 3 were used for the calculate. For example, the
current production level for Supplier 6 was (6, 1600, 1600, 1090, [53,67]), and based on the slack for each variable, the final goal
for Supplier 6 from the traditional model was (1.76, 578.98, 1600, 432.19, [83,97]). Compared with the final goal obtained using
the proposed method, which was (5, 1550, 4560, 1490, [83,97]), it can be seen that the outputs could be greatly improved with a
slight change in the inputs using the proposed method, while the goal suggested by the traditional model was too distant to
achieve in a short term. Due to resource allocation difficulties, attaining the ultimate goal provided by the traditional goal set-
ting model is impossible in one operation cycle, especially when the inefficient DMU is far from best practice.
Further, the goal obtained by the traditional model is a linear combination of DMUs, which may be a hypothetical DMU.
Therefore, as the goal provided by the proposed model actually exists, it is more implementable by the inefficient supplier,
that is, the proposed method provides a goal sequence that well addresses sustainable goal setting issue.

5.3.3. Comparison with a model without the dynamic structure

Previous studies have tended to regard supply chain operations as a black box. However, as sustainable supply chain
operations require long-term development, a dynamic structure was introduced into the proposed model to reflect the time
effect. In this comparison, the efficiencies obtained using the proposed model were compared with the results of a model
without a dynamic structure.
The different efficiencies obtained using the two models are shown in Fig. 11, from which it can be seen that the efficien-
cies of most suppliers changed with the introduction of carry-overs and multiple periods, which indicated that carry-overs
had a significant impact on the overall performance evaluation and that inefficient suppliers are able to gain more detailed
information about their internal performance from the dynamic model.
The static model found that Supplier 3 had the lowest efficiency value, and that Suppliers 1, 5, 7, 10, 12, 15, 18, and 20
were efficient. The traditional goal setting method is to select one of these efficient suppliers to be the benchmark. However,
it is not easy for supplier 3 to choose the most suitable as the reference goal and it would be too difficult to reach the ultimate
goal in a short time.

1.1

1.0

0.9

0.8

0.7

0.6

0.5

0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

dynamic model static model

Fig. 11. Efficiencies compared with a model without dynamic structure.

60 X. Zhou et al. / Information Sciences 545 (2021) 44–64

As the layering procedure separates all DMUs into several layers, it can assist inefficient suppliers achieve stepwise
improvement. Besides, it is necessary to consider the time effect when setting goals to help supply chain members achieve
long-term development. As the actual supplier operations last more than one period, and the current period performance
affects the benefits of the next period, it is reasonable to see benchmarking as a dynamic procedure. Therefore, the layering
procedure combined with the dynamic structure is able to effectively determine the stepwise improvements for the ineffi-
cient suppliers.

5.3.4. Comparison with a model without considering environmental sustainability

To verify the importance of maintaining a balance between economic growth, social security, and environmental protec-
tion when setting goals, the performance index concept proposed in You and Jie [39] was followed, that is, the total-factor
energy performance index, the total-factor labor performance index, and the total-factor capital performance index were
used to measure the potential improvements of each index, each of which was defined as the ratio of actual efficiency to
potential efficiency. Therefore, a similar idea was adopted to calculate the performance index with the TBL variables and
the performance index without the environmental variables to verify the importance of considering comprehensive sustain-
ability. The performance index was defined as the ratio of the actual efficiency of the evaluated supplier to the efficiency of
the final goal selected using the proposed model, that is, the potential improvement level of the inefficient supplier. Taking
Supplier 6 as an example, the performance index obtained using the proposed model was 0.5387, and the performance index
obtained by considering only economic and social sustainability was 0.5193. Obviously, the greater the value of the perfor-
mance index, the more the inefficient supplier can improve. Therefore, overall economic growth, social security and environ-
mental protection development is of great importance when setting goals for inefficient suppliers.
Without considering the variables related to environmental sustainability, the efficiencies of most DMUs decreased to
varying extents rather than having irregular changes. Therefore, sacrificing environmental protection to increase economic
and social profit would be unwise and unethical from a long-term development perspective as it could lead to the selection
of suppliers who would be unable to achieve sustainability as the goals for inefficient suppliers. Therefore, the proposed
model that considers TBL variables is more reasonable for goal setting.

6. Conclusions

As more people realize that solely focusing on economic growth and ignoring social responsibility and environmental pro-
tection cannot achieve long-term sustainable development, practitioners and researchers have begun to pay more attention to
sustainable supplier goal setting, which requires a comprehensive consideration of indicators related to the TBL. Therefore,
decision makers need to conduct reasonable classifications and analyses of the complex information and make a sensible
benchmarking plans. To address this problem, this paper provided implementable goal sequences that considered
economic-social-environmental sustainability rather than a single ultimate goal. The first advantage of the proposed method
is that the layering procedure and the screening of the relative attractiveness and progress ensured that the goal sequences
across the periods consisted of multiple actual DMUs rather than a hypothetical composite DMU that is difficult to learn from.
This gradual improvement process avoids the infeasibility of achieving the ultimate goal in one operating cycle, especially
when the inefficient DMU is far from the efficient frontier. Second, based on the relative attractiveness and progress measure-
ments, decision makers can select the most appropriate DMU as the goal from several equal-performance DMUs. To reflect the
complexity of the data, the proposed model also incorporated accounts receivable and accounts payable as desirable and unde-
sirable carry-overs to connect the neighboring periods and the degree of customer satisfaction described as triangular fuzzy
numbers to express the linguistic information. Lastly, using the proposed model, the period efficiencies for each supplier were
obtained, which allowed the managers to identify the worst performing periods and take corresponding measures.
The proposed model was applied to 20 suppliers as a numerical example to demonstrate its applicability, from which the
following conclusions were drawn: (1) the evaluated suppliers were divided into three layers and a goal sequence for Supplier
6 in the last layer was identified based on the relative attractiveness and progress measurements; (2) the proposed model was
able to identify the underperforming periods for each supplier; (3) the sensitivity analysis showed that the attitude change of
the decision maker had an impact on the efficiencies, and Supplier 3 was found to be the most sensitive to the degree of con-
sumer satisfaction; and (4) the comparisons demonstrated that the proposed model had strong discriminative power.
In summary, the developed model, which incorporated dynamic structure and uncertain information into a DEA-based goal
setting model and accounted for economic-social-environmental sustainability, was better able to describe the problems that
exist in current operating environments and identify implementable future improvement goal sequences for underperforming
suppliers, which overcomes the infeasibility of achieving goals in one operating cycle. In future research, we plan to use a more
diverse data set to validate the proposed model and apply the proposed model to problems in different settings.

CRediT authorship contribution statement

Xiaoyang Zhou: Conceptualization, Methodology, Supervision. Linzi Li: Methodology, Writing - original draft. Haoyu
Wen: Software, Validation. Xin Tian: Writing - review & editing. Shouyang Wang: Project administration. Benjamin Lev:
Writing - review & editing.
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 61

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Acknowledgments

We thank those that have given constructive comments and feedback to help improve this paper. Support was provided
by the National Natural Science Foundation of China (Grant Nos. 71871175, 71401093, 71932002), the Innovative Talents
Promotion Plan-Youth Science and Technology Star Project of Shaanxi Province (Grant No. 2019KJXX-031), the China Post-
doctoral Science Foundation Funded Project (Grant No. 2018M640188), the Fundamental Research Funds for the Central
Universities, and the Innovation Fund of Xidian University.

Appendix A

This section presents the data sets for 20 suppliers used in Section 5. Tables A1, A2, A3 present the original data for 20
suppliers in the three periods.

Table A1
Original data for the 20 suppliers in period 1.

No: x1aj x1bj y1aj y1bj y1j c1jgood c1jbad

S1 3 784 1118 305 medium 1750 8

S2 4 857 300 700 medium 680 7
S3 3 1888 1450 1006 dissatisfaction 1890 7
S4 3 900 520 500 satisfaction 720 6
S5 5 1550 4300 1600 strong satisfaction 6730 15
S6 6 1600 1500 1230 dissatisfaction 1880 11
S7 5 1474 1826.28 647 medium 2858.67 13
S8 4 1200 1982.306 1039 medium 3102.4 12
S9 4 1290 1700 800 satisfaction 2670 10
S10 4 1090 1850 750 strong satisfaction 2888 8
S11 5 990 2005 690 strong satisfaction 3010 7
S12 1 350 847.839 182.75 strong satisfaction 1369.3 4
S13 3 826 279 859 medium 660 7
S14 3 500 548.021 184 medium 890 5
S15 2 600 547.766 148 strong satisfaction 857.78 4
S16 1 350 396.211 210 medium 620.18 3
S17 4 1580 1790 990 medium 2540 10
S18 1 198 210 75 strong satisfaction 298 2
S19 1 350 224 210 medium 350 3
S20 1 350 198 210 dissatisfaction 211 2

Table A2
Original data for the 20 suppliers in period 2.

No: x2aj x2bj y2aj y2bj y2j c2jgood c2jbad

S1 2 749 1022 297 dissatisfaction 1690 8

S2 3 830 398 660 medium 790 6
S3 4 1906 1420 1150 medium 1770 9
S4 4 890 550 512 satisfaction 790 6
S5 4 1560 4288 1799 strong satisfaction 6700 17
S6 5 1550 1696 1190 dissatisfaction 1906 11
S7 5 1432 1933 632 medium 2878 14
S8 3 1180 1997 1078 medium 3006 13
S9 5 1330 1880 850 satisfaction 2770 11
S10 3 1188 1790 730 strong satisfaction 2900 6
S11 4 1200 2190 710 strong satisfaction 2980 7
S12 1 350 862 182.75 strong satisfaction 1344 4
S13 4 854 310 877 medium 689 6
S14 4 512 545 199 medium 878 5
S15 2 609 556 143 strong satisfaction 885 4
S16 1 350 401 210 dissatisfaction 629 4
S17 3 1699 1880 870 medium 2499 12
S18 1 180 205 80 satisfaction 310 2
S19 1 350 244 210 medium 366 4
S20 1 350 206 210 dissatisfaction 205 3
62 X. Zhou et al. / Information Sciences 545 (2021) 44–64

Table A3
Original data for the 20 suppliers in period 3.

No: x3aj x3bj y3aj y3bj y3j c3jgood c3jbad

S1 3 790 1238 315 medium 1690 8

S2 4 800 378 690 medium 790 6
S3 4 1954 1409 1110 dissatisfaction 1770 9
S4 3 875 535 494 satisfaction 790 6
S5 5 1269 4560 1490 strong satisfaction 6700 17
S6 6 1570 1600 1090 medium 1906 11
S7 5 1474 1826.28 647 dissatisfaction 2878 14
S8 5 1115 1982.306 1528 medium 3006 13
S9 4 1350 1806 835 satisfaction 2770 11
S10 4 1224 1900 780 strong satisfaction 2900 6
S11 4 1006 1990 750 satisfaction 2980 7
S12 1 350 902 182.75 strong satisfaction 1344 4
S13 3 814 298 860 medium 689 6
S14 2 511 539 173 strong satisfaction 878 5
S15 2 597 535 152 strong satisfaction 885 4
S16 1 350 780 210 medium 629 4
S17 4 1550 1950 960 medium 2499 12
S18 2 345 390 110 satisfaction 310 2
S19 1 350 244 210 medium 366 4
S20 1 350 180 210 dissatisfaction 205 3

Appendix B

This section presents the transformed data for the degree of customer satisfaction and the undesirable outputs. The orig-
inal data for the degree of customer satisfaction were transformed into corresponding triangular fuzzy numbers, and the
reduced interval numbers were obtained using a
cut. The transformed data for undesirable outputs in the three periods
were obtained using a linear monotone decreasing function Tables B1, B2.

Table B1
Degree of customer satisfaction for each supplier.

No. Reduced interval numbers

Period 1 Period 2 Period 3
S1 (53, 67) (33, 47) (53, 67)
S2 (53, 67) (53, 67) (53, 67)
S3 (33, 47) (53, 67) (33, 47)
S4 (73, 87) (73, 87) (73, 87)
S5 (83, 97) (83, 97) (83, 97)
S6 (33, 47) (33, 47) (53, 67)
S7 (53, 67) (53, 67) (33, 47)
S8 (53, 67) (53, 67) (53, 67)
S9 (73, 87) (73, 87) (73, 87)
S10 (83, 97) (83, 97) (83, 97)
S11 (83, 97) (83, 97) (73, 87)
S12 (83, 97) (83, 97) (83, 97)
S13 (53, 67) (53, 67) (53, 67)
S14 (53, 67) (53, 67) (83, 97)
S15 (83, 97) (83, 97) (83, 97)
S16 (53, 67) (33, 47) (53, 67)
S17 (53, 67) (53, 67) (53, 67)
S18 (83, 97) (73, 87) (73, 87)
S19 (53, 67) (53, 67) (53, 67)
S20 (33, 47) (33, 47) (33, 47)
 X. Zhou et al. / Information Sciences 545 (2021) 44–64 63

Table B2
Original data and transformed data for the undesirable outputs

No. Period 1 Period 2 Period 3

original transformed original transformed original transformed
S1 305 1296 297 1503 315 1214
S2 700 901 660 1140 690 839
S3 1006 595 1150 650 1110 419
S4 500 1101 512 1288 494 1035
S5 1600 1 1799 1 1490 39
S6 1230 371 1190 610 1090 439
S7 647 954 632 1168 647 882
S8 1039 562 1078 722 1528 1
S9 800 801 850 950 835 694
S10 750 851 730 1070 780 749
S11 690 911 710 1090 750 779
S12 182.75 1418.25 182.75 1617.25 182.75 1346.25
S13 859 742 877 923 860 669
S14 184 1417 199 1601 173 1356
S15 148 1453 143 1657 152 1377
S16 210 1391 210 1590 210 1319
S17 990 611 870 930 960 569
S18 75 1526 80 1720 110 1419
S19 210 1391 210 1590 210 1319
S20 210 1391 210 1590 210 1319

References

[1] G. Akman, Evaluating suppliers to include green supplier development programs via fuzzy c-means and VIKOR methods, Computers Ind. Eng. 86 (2015)
69–82.
[2] H.V. Arani, S.A. Torabi, Integrated material-financial supply chain master planning under mixed uncertainty, Inf. Sci. 423 (2018) 96–114.
[3] Z. Ayag, R. Grcan Ozdemir, Evaluating machine tool alternatives through modified TOPSIS and alpha-cut based fuzzy ANP, Int. J. Prod. Econ. 140 (2012)
630–636.
[4] R.D. Banker, W.W. Cooper, L.M. Seiford, R.M. Thrall, J. Zhu, Returns to scale in different DEA models, Eur. J. Oper. Res. 154 (2004) 345–362.
_ Otay, A two-stage fuzzy approach for supplier evaluation and order allocation problem with quantity discounts and lead time, Inf. Sci. 339
[5] F. Çebi, I.
(2016) 143–157.
[6] A. Charnes, W. Cooper, E. Rhodes, Measuring the efficiency of decision making units, Eur. J. Oper. Res. 2 (1978) 429–444.
[7] A. Charnes, W.W. Cooper, B. Golany, L. Seiford, J. Stutz, Foundations of Data Envelopment Analysis for Pareto-Koopmans efficient empirical production
functions?, J. Econ. 30 (1985) 91–107.
[8] W.D. Cook, N. Ramon, J.L. Ruiz, I. Sirvent, J. Zhu, DEA-based benchmarking for performance evaluation in pay-for-performance incentive plans, Omega
84 (2019) 45–54.
[9] W.W. Cooper, K.S. Park, J.T. Pastor, RAM: A range adjusted measure of inefficiency for use with additive models, and relations to other models and
measures in DEA, J. Prod. Anal. 11 (1999) 5–42.
[10] A. Emrouznejad, G.L. Yang, CO2 emissions reduction of Chinese light manufacturing industries: A novel RAM-based global Malmquist-Luenberger
productivity index, Energy Policy 96 (2016) 397–410.
[11] Y. Han, Z. Geng, Q. Zhu, Y. Qu, Energy efficiency analysis method based on fuzzy DEA cross-model for ethylene production systems in chemical
industry, Energy 83 (2015) 685–695.
[12] A. Juan, K. Magdalena, Accounting for slacks to measure dynamic inefficiency in Data Envelopment Analysis, Eur. J. Oper. Res. 278 (2019) 463–471.
[13] A.N. Junior, M.C.D. Oliveira, A.L. Helleno, Sustainability evaluation model for manufacturing systems based on the correlation between triple bottom
line dimensions and balanced scorecard perspectives, J. Cleaner Prod. 190 (2018) 84–93.
[14] S. Lertworasirikul, S. Fang, J.A. Joines, H.L. Nuttle, Fuzzy Data Envelopment Analysis (DEA): a possibility approach, Fuzzy Sets Syst. 139 (2003) 379–394.
[15] H. Liao, X. Mi, Q. Yu, L. Luo, Hospital performance evaluation by a hesitant fuzzy linguistic best worst method with inconsistency repairing, J. Cleaner
Prod. 232 (2019) 657–671.
[16] S. Lim, H. Bae, L.H. Lee, A study on the selection of benchmarking paths in DEA, Expert Syst. Appl. 38 (2011) 7665–7673.
[17] B. Lin, D. Song, Z. Liu, A model of aircraft support concept evaluation based on DEA and PCA, J. Syst. Sci. Inform. 6 (2018) 85–98.
[18] M.K. Mehlawat, A. Kumar, S. Yadav, W. Chen, Data envelopment analysis based fuzzy multi-objective portfolio selection model involving higher
moments, Inf. Sci. 460–461 (2018) 128–150.
[19] G. Mika, O. Akihiro, S. Toshiyuki, DEA (Data Envelopment Analysis) assessment of operational and environmental efficiencies on Japanese regional
industries, Energy 66 (2014) 535–549.
[20] W. Pan, L. Huang, L. Zhao, An integrated DEA model allowing decomposition of eco-efficiency: a case study of china, J. Syst. Sci. Inform. 5 (2017) 91–
106.
[21] S.R. Piao, J. Li, C.J. Ting, Assessing regional environmental efficiency in China with distinguishing weak and strong disposability of undesirable outputs,
J. Cleaner Prod. 227 (2019) 748–759.
[22] N. Ramn, J.L. Ruiz, I. Sirvent, Two-step benchmarking: Setting more realistically achievable targets in DEA, Expert Syst. Appl. 92 (2018) 124–131.
[23] K. Rashidi, K. Cullinane, A comparison of Fuzzy DEA and Fuzzy TOPSIS in sustainable supplier selection: Implications for sourcing strategy, Expert Syst.
Appl. 121 (2019) 266–281.
[24] K. Rashidi, R.F. Saen, Incorporating dynamic concept into gradual efficiency: Improving suppliers in sustainable supplier development, J. Cleaner Prod.
202 (2018) 226–243.
[25] M. Roudneshin, A. Azadeh, A novel multi-objective fuzzy model for optimization of oil sludge management by considering health, safety and
environment (HSE) and resiliency indicators in a gas refinery, J. Cleaner Prod. 206 (2019) 559–571.
[26] J.L. Ruiz, I. Sirvent, Performance evaluation through DEA benchmarking adjusted to goals, Omega 87 (2018) 150–157.
[27] L.M. Seiford, J. Zhu, Modeling undesirable factors in efficiency evaluation, Eur. J. Oper. Res. 142 (2002) 16–20.
[28] L.M. Seiford, J. Zhu, Context-dependent data envelopment analysis: Measuring attractiveness and progress, Omega 31 (2003) 397–408.
64 X. Zhou et al. / Information Sciences 545 (2021) 44–64

[29] H. Shabanpour, S. Yousefi, R.F. Saen, Forecasting efficiency of green suppliers by dynamic Data Envelopment Analysis and Artificial Neural Networks, J.
Cleaner Prod. 142 (2016) 1098–1107.
[30] S.J. Sharahi, K. Khalilidamghani, Fuzzy type-II De-Novo programming for resource allocation and target setting in network data envelopment analysis:
A natural gas supply chain, Expert Systems Appl. 117 (2019) 312–329.
[31] V. Shokri Kahi, S. Yousefi, H. Shabanpour, R. Farzipoor Saen, How to evaluate sustainability of supply chains? A dynamic network DEA approach, Ind.
Manag. Data Syst. 117 (2017) 1866–1889.
[32] T.H. Tran, Y. Mao, P. Nathanail, P. Siebers, D. Robinson, Integrating slacks-based measure of efficiency and super-efficiency in Data Envelopment
Analysis, Omega 85 (2019) 156–165.
[33] M.L. Tseng, M. Lim, K.J. Wu, L. Zhou, D.T.D. Bui, A novel approach for enhancing green supply chain management using converged interval-valued
triangular fuzzy numbers-grey relation analysis, Resour. Conser. Recycling 128 (2018) 122–133.
[34] I.E. Tsolas, Construction project monitoring by means of RAM-based composite indicators, J. Oper. Res. Soc. 64 (2013) 1291–1297.
[35] B. Umar, E. Pervin, D. Robert, Achieving triple bottom line performance in manufacturer-customer supply chains: Evidence from an emerging economy,
J. Cleaner Prod. 197 (2018) 1307–1316.
[36] Q. Wu, L. Zhou, Y. Chen, H. Chen, An integrated approach to green supplier selection based on the interval type-2 fuzzy best-worst and extended VIKOR
methods, Inf. Sci. 502 (2019) 394–417.
[37] W. Xi, X. Cheng, The difference of capital input and productivity in service industries: Based on four stages Bootstrap-DEA model, J. Syst. Sci. Inform. 6
(2018) 320–335.
[38] Z. Xu, J. Qin, J. Liu, L. Martnez, Sustainable supplier selection based on AHPSort II in interval type-2 fuzzy environment, Inf. Sci. 483 (2019) 273–293.
[39] Y.Q. You, T. Jie, A study of the operation efficiency and cost performance indices of power-supply companies in China based on a dynamic network
slacks-based measure model, Omega 60 (2016) 85–97.
[40] L.A. Zadeh, Fuzzy sets, Inform. Control 8 (1965) 338–353.
[41] X. Zhou, Y. Wang, J. Chai, L. Wang, S. Wang, B. Lev, Sustainable supply chain evaluation: a dynamic double frontier network DEA model with interval
type-2 fuzzy data, Inf. Sci. 504 (2019) 394–421.
[42] X. Zhou, X. Wei, J. Lin, X. Tian, B. Lev, S. Wang, Supply chain management under carbon taxes: a review and bibliometric analysis, Omega (2020)
102295.
[43] X. Zhou, Z. Xu, J. Chai, L. Yao, S. Wang, B. Lev, Efficiency evaluation for banking systems under uncertainty: a multi-period three-stage DEA model,
Omega 85 (2019) 68–82.