Supply Chain Management: An International Journal

Determining and applying sustainable supplier key performance indicators

Chunguang Bai Joseph Sarkis
Article information:
To cite this document:
Chunguang Bai Joseph Sarkis , (2014),"Determining and applying sustainable supplier key performance indicators", Supply
Chain Management: An International Journal, Vol. 19 Iss 3 pp. 275 - 291
Permanent link to this document:
http://dx.doi.org/10.1108/SCM-12-2013-0441
Downloaded on: 10 December 2016, At: 00:34 (PT)
References: this document contains references to 74 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 3517 times since 2014*
Users who downloaded this article also downloaded:
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

(2014),"Squaring the circle: Management, measurement and performance of sustainability in supply chains", Supply Chain
Management: An International Journal, Vol. 19 Iss 3 pp. 292-305 http://dx.doi.org/10.1108/SCM-12-2013-0440
(2014),"Do supply management and global sourcing matter for firm sustainability performance?: An international study", Supply
Chain Management: An International Journal, Vol. 19 Iss 3 pp. 258-274 http://dx.doi.org/10.1108/SCM-11-2013-0430

Access to this document was granted through an Emerald subscription provided by emerald-srm:203778 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service
information about how to choose which publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company manages a portfolio of
more than 290 journals and over 2,350 books and book series volumes, as well as providing an extensive range of online
products and additional customer resources and services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics
(COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.

*Related content and download information correct at time of download.

 Determining and applying sustainable supplier
key performance indicators
Chunguang Bai
School of Management Science and Engineering, Dongbei University of Finance and Economics, Dalian, China, and
Joseph Sarkis
WPI School of Business, Worcester Polytechnic Institute, Worcester, MA, USA

Abstract
Purpose – The purpose of this paper is to introduce a methodology to identify sustainable supply chain key performance indicators (KPI) that can
then be used for sustainability performance evaluation for suppliers.
Design/methodology/approach – Initially the complexity of sustainable supply chain performance measurement is discussed. Then, a two-stage
method utilizing neighborhood rough set theory to identify KPI and data envelopment analysis (DEA) to benchmark and evaluate relative
performance using the KPI is completed. Additional analysis is performed to determine the sensitivity of the KPI set formation and performance
results.
Findings – The results show that KPI can be determined using neighborhood rough set, and DEA performance results provide insight into relative
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

performance of suppliers. The supply chain sustainability performance results from both the neighborhood rough set and DEA can be quite sensitive
parameters selected and sustainability KPI sets that were determined.
Research limitations/implications – The data utilized in this study are illustrative and simulated. Only one model for the neighborhood rough set
and DEA was utilized. Additional investigations using a variation of rough set and DEA models can be completed.
Practical implications – This tool set is valuable for managers to help identify sustainable supply chain KPI (from among hundreds of potential
measures) and evaluate sustainability performance of various units within supply chains, including supply chain partners, departments, projects and
programs.
Social implications – Sustainability incorporates many business, economic and social implications. The methods introduced in this paper can help
organizations and their supply chains become more strategically and operationally sustainable.
Originality/value – Few tools and techniques exist in the sustainable supply chain literature to help develop KPIs and evaluate sustainability
performance of suppliers and the supply chain. This paper is one of the first that integrates neighborhood rough set and DEA to address this
important sustainable supply chain performance measurement issue.
Keywords Key performance indicators, Performance measurement, Data envelopment analysis, Sustainable supply chains, Green supply
chains, Rough set theory
Paper type Research paper

Introduction sustainability. A critical aspect of sustainable supply chain

performance measurement systems is the identification of key
The supply chain management program evaluation and
performance indicators (KPI) (Bai et al., 2012; Bai and Sarkis,
monitoring is dependent on the development and application
2012; Chae, 2009). Hundreds of measures for traditional
of performance measures. This evaluation and the set of
business and operational supply chain evaluations may exist
performance dimensions become more complex when
(Gunasekaran and Kobu, 2007). This number of performance
considering supply chain sustainability as environmental and
measures increases greatly when additional environmental and
social responsibility dimensions are added (Ashby et al.,
social sustainability dimensions are included in supply chain
2012). Sustainable supply chain management performance
evaluations. Thus, the need to identify KPI becomes more
measurement can be used for multiple purposes such as
critical when such a large set of sustainable supply chain
supplier selection, performance monitoring and development
performance measures are used (Zhu et al., 2010; Dotoli et al.,
(Gimenez and Tachizawa, 2012; Hervani et al., 2005).
2006).
Investigation of performance measurement systems is
KPI may or may not provide similar amounts of information
needed for advancing supply chain management
when compared to the complete performance measure
indicator set. The use of information theory tools relying on
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1359-8546.htm
The authors thank the National Natural Science Foundation of China
Project (71102090) for supporting this work.

Supply Chain Management: An International Journal Received 6 December 2013

19/3 (2014) 275–291 Revised 15 February 2014
© Emerald Group Publishing Limited [ISSN 1359-8546] 28 February 2014
[DOI 10.1108/SCM-12-2013-0441] Accepted 8 March 2014

275
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

information entropy measures[1] such as the rough set theory measurement and KPI, an introduction into the rough set and
are examined to see if they are valuable for determining a neighborhood rough set technique, and also into the DEA
usable subset of KPI for sustainable supplier evaluation with multifactor performance evaluation approach.
minimal information loss. Using this reduced KPI set,
benchmarking tools, such as data envelopment analysis
Sustainable supply chain performance measurement
(DEA), can then be used to evaluate sustainable supplier
Sustainable supply chain performance measurement has its
performance (Talluri and Sarkis, 2002). DEA’s application to
foundations in the recently emergent work on supply chain
sustainable supplier evaluation is limited; the objective of this
performance measurement (Bai and Sarkis, 2012). The
paper is to further integrate DEA as a tool for performance
competitive nature of supply chains has caused a shift from
measurement for sustainability in supply chains after
individual organizations competing against each other to
reduction of the data sets using rough set approaches.
supply chains competing against each other (Antai, 2011;
DEA is dependent on systemic information, inputs and
Antai and Olson, 2013). Thus, evaluating the supply chain
outputs, related to a unit of analysis such as suppliers, where
and improving supply chain performance requires the
a primary concern of DEA applications is the effective
development of supply chain performance measurement
selection of the input and output performance measures
systems and performance measures. Even though many
(Sarkis, 2007). DEA results are sensitive to the input and
performance measurement system models and performance
output data used in the evaluation. The large set of potential
measures exist (Gunasekaran and Kobu, 2007; Kaplan
performance data that may be used for sustainable supply
and Norton, 1996; Neely et al., 2001), some have
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

chain evaluation makes identifying the appropriate input and

needed adjustment to address supply chain performance
output performance data for DEA a non-trivial exercise.
measurement. Integrating sustainability into supply chain
Information on theoretic approaches such as the rough set
performance measurement is part of this adjustment,
analysis can help identify a reduced, but effective, set of
beginning with the integration of environmental, social and
performance (input and output) data for DEA to evaluate.
business performance measurements into the supply chain
The objective of this paper is to introduce and investigate
(Olugu et al., 2011; Bai and Sarkis, 2012; Zhu et al., 2010).
the linkage of DEA and neighborhood rough set information
Traditional supply chain performance measures that focus on
theoretic approaches for sustainable supply chain performance
operational and business factors can add up to dozens or
evaluation. This paper is one of the first, within the field of
hundreds of measures, while adding sustainability dimensions
supply chain management, to investigate the joint application
increases the size and complexity of the performance measure
of these two tools. The integrative methodology first utilizes
set. Given this large set of potential sustainable supply chain
rough set approaches to identify sustainable supply chain KPI
performance measures, the argument has always been that
through the use of an attribute reduction algorithm based on
“less is better” (Chae, 2009) in terms of performance
information theory. This reduced set of attributes forms input
measures and indicators. Identifying key performance
and output factor data for DEA. DEA is then used to evaluate
indicators (KPI) is thus the best practice policy for operations
the sustainability performance of suppliers. A comparative
and supply chain management.
evaluation of results based on KPI will be completed and
It has been argued that in order for corporate social
implications are identified. Additional sensitivity analyses with
responsibility policy (including sustainable environmental
the rough set technique are evaluated. The results show that
policy) to deliver quantifiable benefits, KPIs are needed
even though information theoretic reduction of data sets using
(Manning, 2013). Most KPIs have focused on financial,
rough set may reduce the need for the many sustainable supply
business and market measures, until recently (Tsoulfas and
chain management performance measures, information loss
Pappis, 2008). For example, the balanced scorecard (Kaplan
should be expected when using DEA as an evaluation tool.
and Norton, 1996), which has been one of the more popular
Managerial implications associated with this observation are
performance measurement tools and focuses on key business
also discussed.
categories, did not originally incorporate social or
The contributions of this research include:
environmental sustainability performance measurements.
● identification of a general multistage approach for
Later, the development of social and environmental balanced
evaluating and selecting KPI for sustainable supply chain
scorecard measures was a way to help managers identify the
management (Bai et al., 2012; Bai and Sarkis, 2012; Chae,
key performance factors to a company’s strategic sustainability
2009) and
objectives (Epstein and Wisner, 2001; Figge et al., 2002;
● testing and evaluation of this new multistage approach as a
Möller and Schaltegger, 2005). In addition to linking KPI to
performance measurement tool such as benchmarking and
strategy and strategic planning, it can be integrated with
supplier evaluation in sustainable supply chains (Hervani
managerial decision making, which in complex supply chain
et al., 2005; Sarkis, 2010; Yakovleva et al., 2012).
situations such as sustainable supply chain management is
This paper contributes to performance measurement dependent on KPI identification (Cabral et al., 2012; Grimm
literature in sustainable supply chain management as well as in et al., 2013).
DEA and rough set application research. Critical (key) success factor theory helps to understand the
importance of KPI development. The theory of critical success
factors is founded in strategy research (Dinter, 2013; Daniel,
Background 1961; Sá, 1988; Vasconcellos e Sá, 1988) and is defined
As part of the background discussion, we briefly introduce as identifying the activities, functions and measures that
the importance of sustainable supply chain performance will ensure successful competitive performance for the

276
 Sustainable supplier key performance indicators
Chunguang Bai and Joseph Sarkis
organization (and extensively to the supply chain). Part of the
argument is that poorly aligned critical success factors (CSF)
will lead to less desirable results. CSF and related activities
should be consistently monitored for proper management
decisions. KPI measuring the CSF and their integration into
performance management systems are means for effective
management decisions in general and particularly and recently
critical for supply chains (Bai and Sarkis, 2012; Chae, 2009).
Besides focusing on CSF themselves, the necessary actions on
how to get there, i.e. process, should be considered as well. One
of the important steps in this process is the identification of KPI.
One of the more popular and extensive performance
evaluation systems is the supply chain operations reference
(SCOR) model (Huan et al., 2004). The SCOR model
categorizes the processes of five supply chain stages including
plan, source, make, deliver and return (Gulledge and
Chavusholu, 2008). We focus on the source function in our
study due to a focus on suppliers. The performance measures
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
within SCOR are categorized on cost, time, quality, flexibility
and innovation dimensions (Shepherd and Gunter, 2006;
Bai and Sarkis, 2011). These performance measures lack
social and environmental sustainability measures and should
be adjusted to incorporate sustainable supply chain
performance measures.
As stated, by adding both business and environmental
sustainability measures, the set of performance measures
becomes quite extensive. The number of performance
measures that can be used to evaluate just one set of
relationships (e.g. sourcing) within the supply chain is also
extensive. Using the SCOR model as its foundation, a listing
of example performance measures for a traditional supply
chain (left column) and environmentally sustainable supply
chain (right column) are shown in Table I. Even this listing is
not complete and sub-measures may also be derived from this
grouping.
Industry is always seeking to become more efficient. Having
cumbersome and complex supply chain performance
measures diminishes management efficiency of the supply
chain. Thus, the identification and application of KPI
becomes even more valuable to supply chain managers
especially with the increasing integration of sustainability
(Chae, 2009).
A major element within sustainable supply chain
management is concerned with inter-organizationally sharing
responsibility for various aspects of environmental and social
performance (Dotoli et al., 2006). Within the environmental
arena, several techniques exist to help managers map the
environmental impacts along supply chains, such as the
life-cycle assessment, product stewardship and design for
environment (DFE) principles (Sarkis, 2006). Performance
measurement, metrics and KPI are critical to all these
dimensions of life-cycle assessment. The rough set theory,
described in the next section, is a valuable resource in
identifying KPI.
Developing tools for performance management systems also
to help and guide benchmarking for sustainable supply chains
further improves the management and adoption of sustainable
supply chain practices (Hervani et al., 2005). Also, according
to Genovese et al. (2013), using KPI for actual performance
measurement and benchmarking of sustainable supply chains
277
Supply Chain Management: An International Journal
Volume 19 · Number 3 · 2014 · 275–291
is the most difficult stage in application of performance
measurement. Thus, models that can help in decision making
and management of sustainable supply chains, using the KPI,
can greatly benefit organizations seeking to manage the
sustainability of their supply chains.
Few modeling applications have been applied to
benchmarking and comparative analysis within sustainable
supply chain situations (Kannan et al., 2013; Seuring, 2013).
Data envelopment analysis (DEA) has been a valuable
analytical modeling tool that can be applied to complex,
multi-attribute benchmarking situations, in both sustainability
(Lee and Saen, 2012; La Rovere et al., 2010; Sarkis and
Talluri, 2004) and supply chain management (Liu et al., 2000;
Xu et al., 2009). But, its explicit application to sustainable
supply chain management has been limited in part because of
the excessive number of performance measures that are
characteristic of sustainable supply chain management
environments and the need to limit performance metrics for
discriminatory evaluation use by DEA. Only recently has DEA
been applied directly to sustainability-oriented supply chain
evaluation (Mirhedayatian et al., 2014).
Integration of DEA and rough set approaches is a relatively
recent development (Azadeh et al., 2011; Yeh et al., 2010),
with the linkage of rough set to KPI identification for input
into DEA models not yet investigated.
Thus, given this background of the complexity of
performance measurement in sustainable supply chains and
the scarcity of models to help aid managers working in this
environment, a practical two-stage model is introduced.
Rough set theory and neighborhood rough sets
To help identify the most information intensive performance
measures and reduce the number of performance measures,
we rely on the neighborhood rough set method. This method
fits within the family of the rough set theory. The rough set
theory is based on the information theory to help reduce the
number of factors with applications primarily within the data
mining realm.
The rough set theory (Pawlak, 1982) classifies objects into
similarity classes containing objects that are indiscernible with
respect to previous occurrences and knowledge (information).
The rough set method was developed as a non-parametric
data-mining approach that can effectively determine core
relationships among a variety of factors. It has been utilized for
diverse applications, including investigating marketing data
(Shyng et al., 2007), education, (Beynon et al., 2000), credit
risk for financial information (Ong et al., 2005) and
manufacturing process control (Sadoyan et al., 2006). The
supply chain literature’s application of the rough set is only
a recent occurrence. Recent application of the rough set
for sustainable and green supply chain and operations
management concerns has occurred (Bai and Sarkis, 2010).
The rough set theory is advantageous to other approaches
for data-mining that typically utilize multivariate statistics that
require specific parametric assumptions. It can integrate both
tangible and intangible information, which is characteristic of
sustainability data, especially data focusing on various supply
chain performance dimensions.
Neighborhood rough set models can deal directly with
continuous numeric data, whereas the traditional rough
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

Table I SCOR-based sourcing function business and environmental performance measures categorized on cost, time, quality, flexibility and
innovation dimensions
Business performance measures Environmental performance measures
Cost
Supplier cost-saving initiatives Environmental costs savings
Labor efficiency Energy Efficiency of systems
Cost variance from expected costs Environmental cost performance variance
Amount of environmental penalties
Time
Supplier lead time against industry norm Length to time to implement environmental programs
Supplier’s booking-in procedures Meeting environmental program implementation period
Purchase order cycle time Speed of acquiring environmental information
Percentage of late deliveries Communication speed on environmental issues to supplier’s suppliers
Information timeliness
Efficiency of purchase order cycle time
Quality
Buyer-supplier partnership level Environmental relationship and cooperation level
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

Level of supplier’s defect-free deliveries Supplier rejection rate

Supplier rejection rate Waste generated from products and materials
Delivery reliability Percentage recycled material
Percentage of wrong supplier delivery Mutual trust on environmental issues
Mutual trust Mutual planning for environmental improvements
Satisfaction with knowledge transfer Mutual assistance for environmental improvements
Satisfaction with supplier relationship Environmental information accuracy
Supplier assistance in solving technical problems Environmental information availability
Extent of mutual planning cooperation leading to improved quality
Extent of mutual assistance leading in problem-solving efforts
Distribution of decision competences between supplier and customer
Quality and frequency of exchange of logistics information between
supplier and customer
Quality of perspective taking in supply networks
Information accuracy
Information availability
Flexibility
Supplier ability to respond to quality problems Amount of environmentally safe alternatives
Response to product changes Response to environmental programs for suppliers
Materials variety (number of materials available) Response to environmental product requests
Product and service variety
Product volume variability capabilities
Product development time
Innovation
Satisfaction with knowledge transfer satisfaction Environmental Knowledge Transfer Satisfaction
Technological capability levels Environmental Technology Levels
Involvement in new product design New environmentally sound processes introduced
Introduction of new processes New environmentally sound product development
Source: Sarkis and Talluri, 2002; Shepherd and Gunter, 2006; Gunasekaran and Kobu, 2007

set typically requires discretization or categorization of where ⌬ is a distance function. For @xi, xj, xk 僆U, and satisfies
continuous data to be effective. Use of actual continuous the following four conditions:
numeric data lessens the likelihood of information loss due to
categorizing data. The definitions and methods presented in (1) ⌬(xi, xj) ⱖ 0;
this paper for neighborhood rough sets are based on the (2) ⌬(xi, xj) ⫽ 0 if and only if xi ⫽ xj;
(2)
developments of Hu et al., 2008a and 2008b. (3) ⌬(xi, xj) ⫽ ⌬(xj, xi);
Definition 1: Given an arbitrary xi 僆 U and B 債 C, the (4) ⌬(xi, xk) ⱕ ⌬(xi, xj) ⫹ ⌬(xj, xk);
neighborhood ␦B共xi兲 of xi in attribute space B is defined as:
Let vij denote the value of object i on attribute j, then a
␦B(xi) ⫽ 兵xj ⱍ xj 僆 U, ⌬B(xi, xj) ⱕ ␦其, (1) Minkowski distance function may be defined as:

278
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

冉兺 冊 xj ⫽ the vector of input factor values for supplier j,

N 1/p

⌬p(xi, xk) ⫽ |vij ⫺ vkj |p
j⫽1
when p ⫽ 1 it is defined as a Manhattan distance; p ⫽ 2 is a
Euclidean distance; and p ⫽ ⬁ is a Chebyshev distance
(Wilson and Martinez, 1997). The Chebyshev distance
equation is represented by equation (4):
⌬⬁(xi, xk) ⫽ max ⱍ vij ⫺ vkj
j
Given that we can identify the most salient performance
measures through the rough set and the neighborhood rough set
theory, we can now utilize a tool for performance evaluation of
multiple factors defined as data envelopment analysis (DEA).
Data envelopment analysis
DEA is a multifactor productivity measurement tool based on
multiple inputs and/or outputs for a given decision-making
unit (DMU). DEA-based models addressing a variety of
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
(3) yj⫽ full set of output factor values, for supplier j, xo ⫽ the
“test” supplier input factor values, yo ⫽ the “test” supplier
output factor values, m⫽ number of inputs, n ⫽ number of
DMUs (suppliers), ␭ ⫽ envelopment (facet) weight for a
specific supplier solution.
An illustrative application
(4)
Given the definitions in the previous section on neighborhood
rough set characteristics and expressions and DEA, we now
introduce a multi-step procedure within the context of an
illustrative application. There are five steps in the rough set
portion of the two-staged method to evaluate supplier
sustainability performance. To set the stage of the application,
we will focus on the supply chain activities associated with the
SCOR “sourcing” function. Further details of the illustrative
application will occur in the steps defined below (additional

information and detail on other, general rough set method can

problem environments have been proposed in the literature. be found in Bai and Sarkis, 2011).
For the purposes of our study, a DMU is defined as an
electricity generating plant.
Step 1: construct the original neighborhood
Tone (2002) provides an important distinction between two
information system
types of measures or approaches in DEA. These measures are
To be able to construct a decision table, we need to identify
termed radial and non-radial measures, respectively, based on
the various performance measures and factors utilized within
differences in the relationships of input or output items to each
the illustrative example[2]. The five SCOR performance
other and whether it is ratio- or slack-based modeling
categories are: cost, time, quality, flexibility and innovation.
approaches (also additive- versus ratio-based).
The exemplary measures and the metrics of the “raw” data are
The original DEA multifactor productivity models were
shown in Tables III and IV. Table III contains the input
introduced by Charnes et al. (1978) (CCR), also defined as
performance factors[3] of DEA data, while Table IV provides
the constant returns to scale ratio-based radial model, and by
examples of the output performance factors of DEA data.
Banker, (1984) (BCC), also defined as the variable returns to
Historical extreme or ideal ranges from best to worst are also
scale ratio-based radial model, effectively considered multiple
summarized in Table II. This table provides a good way of
input and multiple output factors in evaluating relative
structuring KPI for supply chain managers interested in
efficiencies of DMU. Even though there are advantages and
sustainable practices of their suppliers (Tables III and IV).
disadvantages with various forms of DEA models, we will focus
The objects in this neighborhood decision system are
on the slack-based models for evaluation in this paper (Cooper
represented by 120 suppliers. For this illustrative application,
et al., 2007). We describe one particular model in detail.
we randomly select and utilize 8 input performance measures
A general DEA super-efficiency slack-based model and 18 output performance measures for five SCOR
For our example application, to illustrate information performance categories from Table I. For the rough set model,
theoretic filtering relationships with DEA, we utilize the the “conditional attributes” include 8 input performance
super-efficiency slack-based input-oriented model with measures and 18 output performance measures for the
constant returns to scale (Super SBM-I-C) for evaluation of information table. When these two tables are integrated, we
sustainable supply chain performance (Tone, 2002). The have 26 total conditional attributes (made up of the 8 input
model is formulated as: and 18 output performance measures) and 120 objects[4] in
(SUPER SBM-I-C) the rough set “information system”. In this illustrative
m
example, we will use integrated input and output information

兺x
1 x៮ i tables, shrinking the number of performance measures
␦ ⴱ ⫽ min ␦ ⫽
m i⫽1 io simultaneously. We also run a separate input and output sets
subject to: of attributes filtration process later in this paper. The
n definitions of the measures and the metrics used to measure
x៮ ⱖ 兺 ␭x
j⫽1,⫽0
j j
these various sustainable supply chain performance measures
are shown in Table II.
n

y៮ ⱕ 兺 ␭y
j⫽1,⫽0
j j
Step 2: develop an adjusted neighborhood
x៮ ⱖ xo information system
y៮ ⫽ yo A normalization procedure is first introduced so that the
␭ⱖ0 techniques are using similar scales. This normalization will
adjust all the sustainable supply chain performance
where: measurement values (vij) to be 0 ⱕ vij ⱕ 1.

279
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

Table II Input and output performance measures and their definitions and metrics used
Input performance measures
Cost_IN: Environmental cost performance variance, cost variance as percentage of expected cost (0-50 per cent)
Time1_IN: Supplier’s booking-in procedures, number of booking-in procedures (range:1-10)
Time2_IN: Purchase order cycle time, purchase order cycle time in days (1 to 60 days)
Time3_IN: Percentage of late deliveries, percentage of late deliveries (0-50 per cent)
Quality1_IN: Supplier rejection rate, percentage of materials rejected from suppliers based on environmental norm (0.01 per cent to 10
per cent)
Quality2_IN: Percentage of wrong supplier delivery (0-10 per cent)
Flexibility_IN: Response to environmental product requests, responding time in days (0-60 days)
Innovation_IN: Product development time, product time in months (0-24 months)
Output performance measures
Cost
Cost1_OUT: Environmental costs savings initiatives (range: 0-10 initiatives)
Cost2_OUT: Number of innovations (range: 0-10 initiatives)
Cost3_OUT: Energy efficiency of systems, percentage of production compare Energy (Range: 0-50 per cent)
Time
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

Time1_OUT: Supplier lead time against industry norm, percentage over or under industry average lead time (ⴙ25 per cent to ⴚ25 per cent)
Time2_OUT: Speed of acquiring environmental information, speed-level based on following categories very high (VH), high (H),
moderate (M), low (L) and very low (L);
Time3_OUT: Efficiency of purchase order cycle time, percentage of purchase order cycle time (0-50 per cent)
Quality
Quality1_OUT: Level of supplier’s defect-free deliveries, defect-free deliveries level based on following categories very high (VH) to very
low (L)
Quality2_OUT: Delivery reliability, reliability level based on following categories very high (VH), high (H), moderate (M), low (L) and very
low (L)
Quality3_OUT: Extent of mutual planning cooperation leading to improved quality, cooperation level based on following categories very
high (VH) to very low (L)
Quality4_OUT: Mutual assistance for environmental improvements, Assistance level based on following categories very high (VH) to very
low (L)
Quality5_OUT: Environmental information accuracy, level based on following categories very high (VH), high (H), moderate (M), low (L)
and very low (L)
Flexibility
Flexibility1_OUT: Supplier ability to respond to quality problems, ordinal qualitative evaluation of response from VH to VL
Flexibility2_OUT: Product and service variety, number of product and service (1 to 50)
Flexibility3_OUT: Materials variety (number of materials available), number of materials (1 to 500)
Innovation
Innovation1_OUT: Environmental knowledge transfer satisfaction, level of satisfaction with knowledge transfer VH to VL
Innovation2_OUT: Technological capability levels, level of technological capability VH to VL
Innovation3_OUT: New environmentally sound product development, level of development VH to VL
Innovation4_OUT: Introduction of new processes, number of new processes introduced on an annual basis (0 to 4)

We will complete a simple linear normalization based on metric of 45 per cent; thus, the result is a normalized score of 0.9.
equation (5). The normalized results are shown in Table V.

ṽij ⫽
ⱍvij
⫺ vjmin ⱍ (5)
ⱍvjmax ⫺ vjmin ⱍ
where ṽij is the normalized value for supplier i, sustainability
performance measure j, vjmax is the maximum value of
sustainability performance measure j over all suppliers and
vjmin is the minimum value of sustainability performance measure
j over all suppliers. For example, for the Cost_IN metric, the
minimum value is cost variance as 0 per cent of the expected cost,
the maximum is 50 per cent; thus, the extreme range is 0-50 per
cent (appearing as a denominator). Supplier 1 has a performance
Step 3: compute neighborhood relational matrices for
the sustainable supply chain performance measures
(conditional attributes)
Neighborhood relations are a form of similarity relationship.
Neighborhood relations are used to group sets of suppliers
(objects) based on similarity or indistinguishability in terms of
distances using the neighborhood distance parameter. Our
first sub-step in this process is to calculate the distances
between each of the suppliers on each of the conditional
attributes. This will result in a 26 ⫻ n distance matrices for the
input measures. Because initially there is only 1 attribute, we
only consider the simple distance between two suppliers based

280
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

Table III Evaluation of suppliers on performance measurement by decision makers (DEA INPUT values)
Supplier ID Cost_IN Time1_IN Time2_IN Time3_IN Quality1_IN Quality2_IN Flexibility_IN Innovation_IN
Supplier1 45 per cent 6 45 days 35 per cent 9.00 per cent 1 per cent 2 days 19 months
Supplier2 21 per cent 9 31 days 1 per cent 7.00 per cent 9 per cent 53 days 13 months
Supplier3 16 per cent 4 17 days 39 per cent 5.00 per cent 4 per cent 41 days 14 months
Supplier4 7 per cent 9 52 days 35 per cent 4.00 per cent 8 per cent 54 days 8 months
Supplier5 32 per cent 6 39 days 30 per cent 4.00 per cent 10 per cent 11 days 14 months
Supplier6 23 per cent 3 50 days 21 per cent 3.00 per cent 0 per cent 26 days 1 month
Supplier7 42 per cent 2 37 days 9 per cent 7.00 per cent 4 per cent 41 days 17 months
Supplier8 37 per cent 7 39 days 29 per cent 6.00 per cent 10 per cent 16 days 23 months
Supplier9 44 per cent 8 45 days 31 per cent 3.00 per cent 4 per cent 16 days 7 months
Supplier10 22 per cent 9 5 days 41 per cent 4.00 per cent 7 per cent 20 days 12 months
...... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
Supplier116 28 per cent 5 51 days 4 per cent 10.00 per cent 4 per cent 43 days 18 months
Supplier117 33 per cent 8 59 days 38 per cent 4.00 per cent 1 per cent 46 days 5 months
Supplier118 8 per cent 2 12 days 6 per cent 9.00 per cent 10 per cent 22 days 15 months
Supplier119 5 per cent 10 40 days 25 per cent 6.00 per cent 1 per cent 19 days 13 months
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

Supplier120 6 per cent 7 6 days 43 per cent 7.00 per cent 5 per cent 42 days 18 months

on the attribute. If more than one attribute exists, we may wish to
use the Chebyshev distance calculation as shown in equation (4)
(Hu et al., 2008a). For example, for the Cost_IN sustainability
performance measure, the distance between Supplier1 and
Supplier2 is simply |0.9-0.42| ⫽ 0.48. For the sake of brevity,
we do not show the matrix with the distance scores.
The next sub-step determines the neighborhood relation for
each sustainable supply chain performance measure we need
to arrive at a relational matrix. The neighborhood relational
matrix is defined as:
MB(N) ⫽ (rij)n ⫻ n , where rij ⫽ 再1,0,⌬(xotherwise.
i , x ) ⱕ ␦,
j
(6)
For our illustrative example, we begin with a neighborhood
granule, where ␦ ⫽ 0.2.
Step 4: Determine the information significance of a
sustainability performance measure
The first sub-step in this process is to compute the
neighborhood information granule [␦B共xi兲; see Definition 1]
for each of the suppliers for a given sustainable supply chain
performance measure or set of sustainable supply chain
performance measures B.
The neighborhood rough set-based approach reduces the
number of sustainable supply chain performance measures
(inputs/outputs) to determine the sustainable KPI, which has
the same characterizing power as the whole sustainability
performance measures set. To calculate the information
significance of a sustainable performance measure j (aj) with
respect to a set B, we use equation (7).
ⱍUⱍ

All the diagonal elements of this matrix are equal to 1. That is

rii ⫽ 1 because ⌬B共xi, xi兲 ⫽ 0 ⱕ ␦.
Sig(aj, B)⫽1 ⫺
1
ⱍUⱍ2 兺 ⱍ␦ (x )ⱍ
i⫽1
B i (7)

In the initial example matrix, the set B is a reduct set that is

the union of the performance metric Cost_IN and the previous
reduct set Atr. Where Atr is the core performance measures
(reduct) set. The reduct set will be used as the input for the
DEA model (Tone, 2002) as inputs and outputs. In the
initialization step, the core sustainability performance
measurement in our reduct set is set Atr ⫽ A. Thus, set B is
just the individual sustainability performance metrics that we
are calculating.
For the sustainability performance metric Cost_IN, we
know that r12 ⫽ r21 ⫽ 0 because ⌬Cost_IN共x1, x2兲 ⫽ 0.420 ⱖ 0.2.
MCost_IN共120兲 ⫽ 共rij兲120 ⫻ 120 is shown in Table VI; an additional
25 relational matrices (not shown here) can be determined for
each of the sustainability performance measures separately.
In later steps, B ⫽ 兵aj 艛 Atrⱍaj 僆 Atr其., where Atr is the
sustainability KPI and aj is from the set of unselected
sustainability performance measures. Initially, Atr ⫽ A. We
return to Step 3 after Step 6 until no additional KPIs are
found [i.e. Sig1共aj, B兲 ⱕ ␧]. Each cycle of these steps will result
in only one additional sustainability KPI. The final Atr core
KPI is the reduced outcome.
In equation (7), Sig共aj, B兲 is the information significance[5]
over the aj conditional attribute, |U| is the cardinality of the
universe of suppliers (120 in our example). | ␦B共xi兲 | is the
number of suppliers with similar attribute levels across B
sustainability performance measure for a supplier i. It is also
defined as the number of members within the B supplier
sustainability performance measure elementary set for
supplier i.
As an example, supplier 1 is in the same neighborhood value
in the a1 sustainable supplier performance measure as
suppliers {7, 8, 9 . . ., 107}; thus |␦B共x1兲 | ⫽ 33. The value of
|␦B共xi兲 | is determined by summing the rows of MCost_IN共120兲.
Step 5: select and update sustainable KPI and reduct
This step of the methodology requires that we select the
sustainable performance measure ak that satisfies (8):
Sig(ak, B) ⫽ max j(Sig(aj, B)) (8)
where ak 僆 ∀aj, for aj 僆 Atr.

281
 Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

Table IV Evaluation of suppliers on performance measurement by decision makers (DEA OUTPUT)

Chunguang Bai and Joseph Sarkis

Cost1_ Cost2_ Cost3_ Time1_ Time2_ Time3_ Quality1_ Quality2_ Quality3_ Quality4_ Quality5_ Flexibility1_ Flexibility2_ Flexibility3_ Innovation1_ Innovation2_ Innovation3_ Innovation4_
Supplier ID OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT
Sustainable supplier key performance indicators

Supplier1 2 4 13 per cent ⫺7 per cent L 40 per cent H M L L L L 27 60 L VL L 3

Supplier2 7 9 11 per cent 24 per cent L 48 per cent H H M H M VH 28 257 H H L 0
Supplier3 3 0 41 per cent 5 per cent VL 18 per cent H H M L H H 18 478 L H M 2
Supplier4 5 8 27 per cent 12 per cent M 24 per cent M VL VL H VL H 20 189 H VL H 2
Supplier5 1 1 41 per cent 8 per cent L 11 per cent L VH L VL L L 19 26 L H H 2
Supplier6 6 5 1 per cent ⫺20 per cent H 9 per cent H VH M M M L 22 495 VH VL VH 2
Supplier7 1 8 22 per cent ⫺18 per cent VL 42 per cent VL M M M M VL 41 104 M M L 2

282
Supplier8 2 8 17 per cent ⫺1 per cent H 40 per cent H VH M M VH VH 40 204 H L L 3
Supplier9 5 7 6 per cent 7 per cent M 43 per cent L H H VL VH L 33 135 L VH VL 3
Supplier10 7 7 30 per cent 14 per cent M 38 per cent H VH M H H M 48 307 M L L 4
...... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
Supplier116 7 5 45 per cent 17 per cent L 30 per cent M VL H L M H 20 16 VH L L 0
Supplier117 4 5 35 per cent ⫺16 per cent H 43 per cent M H VL L L VL 28 468 L H L 1
Supplier118 2 9 33 per cent ⫺16 per cent H 16 per cent M H L H M L 43 358 H VH M 4
Supplier119 9 2 22 per cent ⫺23 per cent VL 18 per cent L VH L L M L 49 439 M H H 2
Supplier120 10 5 40 per cent ⫺4 per cent L 30 per cent M H L VL VH L 39 3 M VL VL 3
Volume 19 · Number 3 · 2014 · 275–291
Supply Chain Management: An International Journal
 Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

Table V Adjusted scores of suppliers on performance measurement

Chunguang Bai and Joseph Sarkis

Supplier Cost_ Time1_ Time2_ Time3_ Quality1_ Quality2_ Flexibility_ Innovation_ Cost1_ Cost2_ Cost3_ Time1_ Time2_ Time3_ Quality1_ Quality2_ Quality3_ Quality4_ Quality5_ Flexibility1_ Flexibility2_ Flexibility3_ Innovation1_ Innovation2_ Innovation3_ Innovation4_
ID IN IN IN IN IN IN IN IN OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT OUT
Sustainable supplier key performance indicators

Supplier1 0.9 0.56 0.75 0.7 0.9 0.1 0.03 0.79 0.2 0.4 0.26 0.36 0.4 0.8 0.8 0.6 0.4 0.4 0.4 0.4 0.53 0.12 0.4 0.2 0.4 0.75
Supplier2 0.42 0.89 0.51 0.02 0.7 0.9 0.88 0.54 0.7 0.9 0.22 0.98 0.4 0.96 0.8 0.8 0.6 0.8 0.6 1 0.55 0.51 0.8 0.8 0.4 0
Supplier3 0.32 0.33 0.27 0.78 0.5 0.4 0.68 0.58 0.3 0 0.82 0.6 0.2 0.36 0.8 0.8 0.6 0.4 0.8 0.8 0.35 0.96 0.4 0.8 0.6 0.5
Supplier4 0.14 0.89 0.86 0.7 0.4 0.8 0.9 0.33 0.5 0.8 0.54 0.74 0.6 0.48 0.6 0.2 0.2 0.8 0.2 0.8 0.39 0.38 0.8 0.2 0.8 0.5
Supplier5 0.64 0.56 0.64 0.6 0.4 1 0.18 0.58 0.1 0.1 0.82 0.66 0.4 0.22 0.4 1 0.4 0.2 0.4 0.4 0.37 0.05 0.4 0.8 0.8 0.5
Supplier6 0.46 0.22 0.83 0.42 0.3 0 0.43 0.04 0.6 0.5 0.02 0.1 0.8 0.18 0.8 1 0.6 0.6 0.6 0.4 0.43 0.99 1 0.2 1 0.5
Supplier7 0.84 0.11 0.61 0.18 0.7 0.4 0.68 0.71 0.1 0.8 0.44 0.14 0.2 0.84 0.2 0.6 0.6 0.6 0.6 0.2 0.82 0.21 0.6 0.6 0.4 0.5

283
Supplier8 0.74 0.67 0.64 0.58 0.6 1 0.27 0.96 0.2 0.8 0.34 0.48 0.8 0.8 0.8 1 0.6 0.6 1 1 0.8 0.41 0.8 0.4 0.4 0.75
Supplier9 0.88 0.78 0.75 0.62 0.3 0.4 0.27 0.29 0.5 0.7 0.12 0.64 0.6 0.86 0.4 0.8 0.8 0.2 1 0.4 0.65 0.27 0.4 1 0.2 0.75
Supplier10 0.44 0.89 0.07 0.82 0.4 0.7 0.33 0.5 0.7 0.7 0.6 0.78 0.6 0.76 0.8 1 0.6 0.8 0.8 0.6 0.96 0.61 0.6 0.4 0.4 1
...... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... . . .. . . ... ... ... ... ... ... ...
Supplier116 0.56 0.44 0.85 0.08 1 0.4 0.72 0.75 0.7 0.5 0.9 0.84 0.4 0.6 0.6 0.2 0.8 0.4 0.6 0.8 0.39 0.03 1 0.4 0.4 0
Supplier117 0.66 0.78 0.98 0.76 0.4 0.1 0.77 0.21 0.4 0.5 0.7 0.18 0.8 0.86 0.6 0.8 0.2 0.4 0.4 0.2 0.55 0.94 0.4 0.8 0.4 0.25
Supplier118 0.16 0.11 0.19 0.12 0.9 1 0.37 0.63 0.2 0.9 0.66 0.18 0.8 0.32 0.6 0.8 0.4 0.8 0.6 0.4 0.86 0.72 0.8 1 0.6 1
Supplier119 0.1 1 0.66 0.5 0.6 0.1 0.32 0.54 0.9 0.2 0.44 0.04 0.2 0.36 0.4 1 0.4 0.4 0.6 0.4 0.98 0.88 0.6 0.8 0.8 0.5
Supplier120 0.12 0.67 0.08 0.86 0.7 0.5 0.7 0.75 1 0.5 0.8 0.42 0.4 0.6 0.6 0.8 0.4 0.2 1 0.4 0.78 0 0.6 0.2 0.2 0.75
Volume 19 · Number 3 · 2014 · 275–291
Supply Chain Management: An International Journal
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

Table VI The relational matrix for the COST_IN performance measurement between suppliers’ normalized scores
No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 . . . 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120
1 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 ... 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0
2 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 1 1 0 1 0 1 1 0 0 0 0
3 0 1 1 1 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 0 1 0 1 1 1 0 0 1 0 0
4 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1
5 0 0 0 0 1 1 0 1 0 0 0 0 1 1 0 ... 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0
6 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 ... 0 0 1 0 1 1 0 1 0 1 1 0 0 0 0
7 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 ... 1 1 0 1 0 0 0 0 0 0 0 1 0 0 0
8 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 ... 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0
9 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 ... 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0
10 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 1 0 1 1 0 1 0 1 1 0 0 0 0
11 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 1 1 0 1 0 1 1 0 0 0 0
12 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1
13 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 ... 0 1 1 1 1 0 0 0 0 0 1 1 0 0 0
14 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 ... 0 1 1 1 0 0 0 0 0 0 0 1 0 0 0
15 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 1 1 0 1 0 1 1 0 0 0 0
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

. . . . . .. . .. . .. . .. . .. . .. . .. . .. . . ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
106 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 ... 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0
107 1 0 0 0 0 0 1 1 1 0 0 0 1 1 0 ... 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0
108 0 0 0 0 1 1 0 1 0 1 0 0 1 1 0 ... 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0
109 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 ... 1 1 1 1 0 0 0 0 0 0 0 1 0 0 0
110 0 1 0 0 1 1 0 1 0 1 1 0 1 0 1 ... 0 0 1 0 1 0 0 1 0 1 1 1 0 0 0
111 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 0 1 0 1 1 1 0 0 1 0 0
112 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1
113 0 1 1 0 0 1 0 0 0 1 1 0 0 0 1 ... 0 0 0 0 1 1 0 1 0 1 1 0 0 0 0
114 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 1 1 0 1 0 0 0 1 1 1
115 0 1 1 0 1 1 0 0 0 1 1 0 0 0 1 ... 0 0 1 0 1 1 0 1 0 1 1 1 0 0 0
116 0 1 0 0 1 1 0 1 0 1 1 0 1 0 1 ... 0 0 1 0 1 0 0 1 0 1 1 1 0 0 0
117 0 0 0 0 1 0 1 1 0 0 0 0 1 1 0 ... 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0
118 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 1 1 0 1 0 0 0 1 1 1
119 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1
120 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 ... 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1

The sustainability performance measure with the largest bolded in Table VII. We show other values of ␦ in Table VII
significance ak ⫽ Innovation4_OUT. to provide some idea of the sensitivity of the sustainable
In this method, we initialize the KPI set Atr to an empty set. KPI set.
We will add the remaining attributes’ importance in each cycle This initial illustrative analysis shows that only six
choosing to add the sustainability performance measure with sustainability performance measures are needed to provide a
the greatest importance to the KPI set Atr, until Sig共ak ⫹
Atr, B兲 ⫺ Sig共Atr兲 ⱕ ␧. That is, adding any new performance Table VII Overall sustainability KPI results of the neighborhood rough set
measures to the characterizing power of the KPI set Atr until methodology using various neighborhood distance values (␦)
very little additional information is achieved.
Neighborhood
To update the KPI set Atr, the following rule is applied:
distance (␦) Reduced set of factors
If Sig共ak ⫹ Atr, B兲 ⫺ Sig共Atr兲 ⬎ ␧, where ␧ is a positive
infinitesimal real number used to control the convergence, 0 22, 3
then Atr艛ak ⫺ ⬎ Atr. We then return to step 3 with a new 0.05 1, 7, 21
KPI set Atr. Otherwise, if Sig共ak ⫹ Atr, B兲 ⫺ Sig共Atr兲 ⱕ ␧, we 0.1 2, 7, 6, 8
stop, and the final sustainable supply chain KPI set is Atr. 0.15 13, 19, 24, 17, 3
For our illustrative example, we define ␧ ⫽ 0.001. We find 0.2 26, 2, 8, 21, 1, 19
that for ak ⫽ Innovation4_OUT, Sig1 (Innovation4_OUT, 0.25 6, 1, 9, 10, 7, 5, 11, 4
Innovation4_OUT) ⬎ 0.001, so initially Atr ⫽ 0.3 2, 22, 1, 11, 7, 21, 19, 15, 4
Atr 艛 Innovation4_OUT ⫽ 兵Innovation4_OUT其. 0.35 6, 25, 1, 19, 9, 5, 7, 22, 16, 15
We then return to Step 3 0.4 26, 21, 11, 1, 2, 7, 22, 12, 14, 3, 6, 19, 4
After a number of iterations, the final KPI set Atr for 0.45 26, 6, 7, 9, 11, 1, 3, 22, 21, 14, 12, 19, 8, 4, 15, 5
␦ ⫽ 0.2 is: { Innovation4_OUT, Time1_IN, Innovation_IN, 0.5 7, 21, 1, 22, 11, 14, 3, 19, 12, 6, 2, 9, 4, 15, 8, 5,
Flexibility2_OUT, Cost_IN, Quality5_OUT } or numerically, 25, 24, 18, 20
{26, 2, 8, 21, 1, 19}. This result is italicized and

284
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

threshold amount of information from the whole set of 26
performance measures. We now complete an evaluation of the
suppliers’ sustainability performance using the Super
SBM-I-C DEA model to determine what happens to the
resulting relative sustainability performance evaluation among
the various suppliers using the whole set of performance
metrics and the six sustainability KPIs we identified in this
illustrative example.
Evaluating supplier sustainability performance
using DEA
To complete this step of the process, we shall initially use the
original sustainable supply chain performance measures in the
DEA model. The overall DEA sustainability scores of the full
set of sustainability measures are shown in Table VIII. We see
that supplier 41 (bolded in Table VIII) is the highest-ranked
supplier in terms of overall sustainability performance, with a
DEA score of 7.510. This result is useful to managers who
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
information may be useful for more effective sustainability
benchmarking. Supplier 69 had the lowest ranking score.
The goal of this paper is to focus on the application of rough
set to sustainability KPIs that a manager would need to collect
and apply within a DEA model. Going through the five-step
neighborhood rough set process, we determined that the KPIs
were six performance measures. The KPIs included three
input and three output sustainability measures.
We now execute the DEA model with only the sustainability
KPIs (we do not report these scores in this paper). The
resulting sustainability rankings of the suppliers with the KPIs
are shown in Table VIII under column headings of “KPI
Rank”. Even for the KPI set of sustainability performance
measures, Supplier 41 still ranks as the top overall
sustainability performer. Supplier 69 still ranks very low using
the sustainability KPI.
Overall, the groupings of the suppliers into high, medium
and low performances are relatively consistent. The simple

wish to benchmark the processes that allow this supplier to
perform well on both sustainability dimensions. DEA also
provides other information such as “facet” sets that determine
which suppliers play constraining roles in the efficiency (DEA)
scores. We will not detail these in our paper, but this
correlation analysis shows that it is not a perfectly matched
ordering with only a 0.389 score using a Kendall’s Tau-b
non-parametric correlation statistic. That means there are
some significant differences. To investigate the prevalence of
these differences, we complete some sensitivity analysis.

Table VIII DEA scores and ranks of all suppliers including all sustainability performance measures (full rank) and the KPI (KPI rank)
Supplier DEA Full KPI Supplier DEA Full KPI Supplier DEA Full KPI Supplier DEA Full KPI
ID score rank rank ID score rank rank ID score rank rank ID score rank rank
Supplier1 1.919 29 104 Supplier31 2.241 13 8 Supplier61 2.009 23 12 Supplier91 1.096 106 97
Supplier2 1.626 49 99 Supplier32 5.491 3 51 Supplier62 1.116 105 108 Supplier92 1.695 41 73
Supplier3 1.157 102 90 Supplier33 1.225 91 111 Supplier63 1.463 64 48 Supplier93 1.627 48 62
Supplier4 1.184 98 66 Supplier34 1.577 54 86 Supplier64 1.464 63 82 Supplier94 1.663 45 84
Supplier5 0.843 119 113 Supplier35 1.432 66 77 Supplier65 1.794 35 46 Supplier95 1.551 56 63
Supplier6 4.432 4 27 Supplier36 1.626 50 18 Supplier66 1.190 97 117 Supplier96 1.023 113 118
Supplier7 1.095 107 68 Supplier37 1.634 47 34 Supplier67 1.190 96 110 Supplier97 1.466 62 7
Supplier8 0.869 117 96 Supplier38 1.519 60 69 Supplier68 1.648 46 39 Supplier98 2.648 7 30
Supplier9 1.252 90 64 Supplier39 1.762 37 47 Supplier69 0.729 120 112 Supplier99 1.125 104 89
Supplier10 1.360 74 50 Supplier40 1.428 67 92 Supplier70 1.368 72 16 Supplier100 1.139 103 107
Supplier11 1.598 51 76 Supplier41 7.510 1 1 Supplier71 1.298 85 106 Supplier101 0.911 115 100
Supplier12 1.357 75 9 Supplier42 2.120 17 41 Supplier72 1.956 26 54 Supplier102 1.351 77 28
Supplier13 1.384 70 75 Supplier43 1.182 99 79 Supplier73 1.892 32 22 Supplier103 2.225 14 55
Supplier14 1.585 53 91 Supplier44 1.211 93 98 Supplier74 1.369 71 42 Supplier104 1.170 100 119
Supplier15 1.027 112 70 Supplier45 2.065 21 38 Supplier75 1.495 61 14 Supplier105 1.288 86 26
Supplier16 1.774 36 45 Supplier46 1.586 52 25 Supplier76 1.282 87 94 Supplier106 1.917 30 44
Supplier17 0.864 118 103 Supplier47 2.082 20 72 Supplier77 1.327 82 87 Supplier107 1.331 80 61
Supplier18 2.451 11 116 Supplier48 1.091 108 67 Supplier78 1.068 110 120 Supplier108 1.303 84 83
Supplier19 3.217 5 101 Supplier49 1.921 28 4 Supplier79 2.167 15 17 Supplier109 1.325 83 65
Supplier20 1.993 24 13 Supplier50 1.190 95 78 Supplier80 0.895 116 102 Supplier110 1.202 94 32
Supplier21 2.544 9 60 Supplier51 2.156 16 6 Supplier81 1.757 38 5 Supplier111 1.331 81 93
Supplier22 1.437 65 49 Supplier52 1.907 31 37 Supplier82 1.028 111 105 Supplier112 1.694 42 24
Supplier23 1.548 57 52 Supplier53 1.575 55 10 Supplier83 2.113 18 3 Supplier113 2.347 12 20
Supplier24 1.356 76 59 Supplier54 1.389 69 43 Supplier84 2.568 8 21 Supplier114 1.809 33 57
Supplier25 6.856 2 53 Supplier55 1.693 43 31 Supplier85 1.361 73 56 Supplier115 2.869 6 71
Supplier26 1.221 92 95 Supplier56 2.064 22 11 Supplier86 1.261 89 85 Supplier116 1.070 109 115
Supplier27 1.332 79 58 Supplier57 1.395 68 35 Supplier87 2.480 10 29 Supplier117 1.272 88 114
Supplier28 2.088 19 2 Supplier58 1.963 25 74 Supplier88 1.807 34 80 Supplier118 1.922 27 15
Supplier29 1.528 59 88 Supplier59 1.165 101 40 Supplier89 1.696 40 19 Supplier119 1.663 44 33
Supplier30 1.754 39 109 Supplier60 1.005 114 81 Supplier90 1.544 58 23 Supplier120 1.337 78 36

285
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

Additional analyses
The evaluation in the illustration is precursory and exemplary.
To extend the evaluation and provide additional insight,
various sustainability KPIs using different neighborhood
values are completed. Table VII provides the sustainable KPI
over the range of neighborhood distances. We also evaluate
the situation when separate sustainable input KPI and
sustainable output KPI are determined. That is, first, the
neighborhood rough set technique is used to identify the input
KPI, and then the neighborhood rough set technique is
applied again to identify the output KPI. The sustainability
KPI results over the distance ranges for the separate
reductions are shown in Table IX.
We complete two sets of sensitivity analyses for each of the
KPI sets to evaluate how the rankings change and the
robustness of the solutions. The first evaluation will focus on
executing the DEA analysis over distance neighborhood
measures ranging from ␦ ⫽ 0 to ␦ ⫽ 0.5. We do this analysis
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
(Table IX) having very few factors when ␦ ⫽ 0. This situation
exemplifies the loss of potentially significant information, even
though rough set approaches are meant to minimize this
effect. Even though the rough set technique reduced the set to
only sustainability KPIs for the full set (sustainability
performance measures 3 and 22 remain), the DEA
correlation, although statistically significant, is 0.165. This
result shows DEA’s sensitivity. What could be considered little
information loss from rough-set and other information
theoretic approaches can cause different results using DEA.
Overall, a methodological warning here is that reduction in
factors should be completed with great care. A full set KPIs
and reduced set of input and output KPIs will include most of
the factors when ␦ ⬎ 0.4.
When determining the input and output KPIs, separating
these two factors in the rough set analysis produces a larger
final set to include in the DEA analysis. For example, at ␦ ⫽
0, the KPIs for the aggregated inputs and outputs (full set)

for the full set of KPIs and for the separate input and output
KPIs. Table X summarizes these results. In this table, the
results show that there is growth in correlation as the distance
neighborhood increases. The larger the neighborhood
distance value, the greater the number of KPIs. Over the range
of neighborhood distances, the level of correlation is not
monotonically increasing. For example, over the ␦ ⫽ 0.30 to
␦ ⫽ 0.35 range, correlation decreases. For both full set KPI
DEA scores and separate input/output KPI DEA scores, the
Kendall’s Tau-b correlation decreases from 0.404 to 0.392 for
the full set reduction and 0.828 to 0.793 for separate I/O
KPIs.
Looking at a slightly more detailed picture, we see the full
set of KPIs (Table VII) and separate input and output KPIs
contains only two remaining factors. The KPIs for separate
input and output performance measurement sets for this range
have two remaining input performance measures and two
output performance measures (Table IX).
An advantage of using the rough set for reduction of input
and output factors separately is that we guarantee at least one
input and one output KPI remain for the DEA analysis. For
the DEA models and software used in this paper, it is essential
that both an input and an output sustainability KPIs remain.
Another sensitivity evaluation is to consider how the
rankings change at the extremes when there is a change in
the distance neighborhood value for sustainable KPI
determination. This type of analysis is useful because we may
wish to group the DEA rankings into good and poor

Table IX Reduced set of input and output KPIs after application of rough set to input- and output-oriented sustainable supply chain performance measures
separately
Neighborhood distance (␦) Sustainability input KPI Sustainability output KPI
0 3, 1 12, 22
0.05 1, 7, 2 9, 11, 21
0.1 2, 7, 6, 8 9, 13, 21, 22
0.15 6, 1, 8, 4, 2 13, 17, 19, 24
0.2 2, 8, 7, 1, 4, 5, 6 9, 12, 13, 14, 15, 22, 26
0.25 6, 1, 5, 4, 8, 3, 7, 2 9, 10, 12, 14, 17, 21, 22, 25
0.3 2, 7, 1, 4, 6, 8, 5, 3 9, 11, 12, 14, 16, 21, 22, 23, 24, 25
0.35 6, 1, 7, 5, 4, 2, 8, 3 9, 13, 15, 16, 17, 19, 20, 22, 24, 25
0.4 1, 2, 7, 6, 4, 8, 5, 3 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 22, 23, 24, 25, 26
0.45 6, 1, 7, 2, 5, 8, 4, 3 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 21, 22, 23, 24, 25, 26
0.5 7, 1, 6, 2, 4, 3, 8, 5 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26

Table X Correlation values (Kendall’s tau) for spectrum of neighborhood distances, sustainability KPI sets
␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ ␦ⴝ
Neighborhood distance 0.0 0.05 0.10ⴱ 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50
Full KPI set DEA score 0.165 0.173 x 0.208 0.389 0.337 0.404 0.392 0.607 0.706 0.882
Separate KPI I/O DEA score 0.308 0.339 0.363 0.508 0.674 0.811 0.828 0.793 0.933 0.967 1.000
Full KPI ranks 0.165 0.173 x 0.209 0.389 0.337 0.404 0.391 0.608 0.706 0.882
Separate KPI I/O rank 0.309 0.338 0.362 0.508 0.674 0.811 0.828 0.792 0.933 0.967 1.000
ⴱ
Notes: For ␦ ⫽ 0.10 there were no input KPI and only output KPI existed. The DEA software and model used would not evaluate data with no out-
put KPI

286
 Sustainable supplier key performance indicators Supply Chain Management: An International Journal
Chunguang Bai and Joseph Sarkis Volume 19 · Number 3 · 2014 · 275–291

performing groups instead of looking at correlations over the
full ranges. In Table XI, we see the results of this analysis.
First, we observe that the KPIs for separate input and
output factors show a higher percentage of consistency with
the initial sustainability performance measurement set than
KPI from the full set reduction. To arrive at this comparison,
we take the top 10 (20) and bottom 10 (20) performing
suppliers and determine how much overlap exists in these
top and bottom performing suppliers using the full set of
performance factors and the reduced set. For example, at ␦ ⫽
0.0 in Table XI, 20 per cent of the top ten suppliers and
bottom ten suppliers overlap when compared to the rankings
for when the full set of factors are evaluated using DEA.
For the range (␦ ⫽ 0-0.5) the percentage overlap is not
monotonically increasing. This characteristic is most clear
when comparing the top ten sustainability performers when
the KPIs are from the separate input output set reduction.
From ␦ ⫽ 0.10 to ␦ ⫽ 0.30, there is an increase to 100 per cent
consistency. At ␦ ⫽ 0.35, the consistency of the initial set
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
because we realize that organizations find this an increasingly
important issue for managing their competitiveness. In
addition, supply chain sustainability performance evaluation
may require the extensive use and application of dozens, if
not hundreds, of additional performance measures. Critical
success factor theory stipulates that utilization of key
performance indicators (measures) that are carefully selected
are important for strategic and operational management of
organizations (Sá, 1988; Vasconcellos e Sá, 1988). Helping
to focus organizational strategy by focusing on core
competencies, capabilities and performance measures, can
improve an organization’s competitive stance. Sustainability is
one such strategy, and managing a sustainable supply chain
and its performance requires this focus. KPI development for
management can aid in performance evaluation of the
sustainable supply chain.
Sustainability performance evaluation can be eased for
managers by using KPIs, and then use these KPIs to develop

an easy and comparable performance measure. The

percentage drops to 70 per cent overlap and consistency. To
help identify what is occurring in this situation, we can go back
to Table IX, to evaluate the remaining sustainable KPIs. We
can see that when we compare ␦ ⫽ 0 and ␦ ⫽ 0.05,
sustainability KPIs to the ␦ ⫽ 0.1 sustainability KPIs (1, 11,
12) are missing, reducing the percentage of top ten supplier
overlap. This factor difference seems to be also occurring at
␦ ⫽ 0.3 when compared to the KPI set for ␦ ⫽ 0.35, which
does not include performance measures (11, 12, 14, 21, 23).
This quick analysis indicates that performance measures 11
and 12 are very significant for the top ten supplier selection.
This indirect evaluation may be valuable for managers and not
directly determined with the DEA analysis alone and without
some sensitivity over the ranges. When ␦ ⫽ 0.5, KPIs from the
separate input and output factors analysis is the same as the
initial set of performance measures (i.e. there is no reduction
in performance measures). Over the range ␦ ⫽ 0-0.5, some
suppliers always show up in the top 10/20 or bottom 10/20. It
is clear that these suppliers are continuously poorer or better
performers.
Discussion and conclusion
In this paper, we have introduced a two-staged method that
can be valuable for evaluating sustainability performance
of suppliers (or any other organizational entities) more
efficiently. We use sustainability evaluation of suppliers
two-staged method using the neighborhood rough set and
then DEA help us to accomplish this goal. The advantages of
this two-staged method are reduced data sets require less
extensive acquisition of data by managers and lessened
computational requirements, especially if an organization is
potentially considering thousands of suppliers. DEA allows for
the joint consideration of a number of performance measures
as well as a tool for comparative analysis for sustainability of
suppliers. The synergistic contributions of DEA and rough set
are valuable to managers and researchers. This paper
represents a first step in integrating these tools and also for the
purposes of evaluating sustainable supply chains.
We have shown that even though there are advantages of
minimizing the set of performance measures, careful
evaluation of the DEA results are also needed. Too tight a
“neighborhood distance” in the rough set may cause
significant information loss. We have found that this distance
measure alone may not tell us the information loss for DEA
because there is not always a direct relationship between the
size of the neighborhood and correlations/consistency among
the best and worst suppliers.
From a benchmarking perspective, DEA can be a valuable
tool. For example, identifying good and bad performers can be
completed with DEA, based on the performance measures
evaluated, and we did this in the example of ranking of
suppliers on the set of sustainability measures. Not only can

Table XI Percentage of similar top- and lowest-ranked supplier sets over spectrum of neighborhood distances, KPI
Neighborhood distance ␦ ⴝ 0.0 ␦ ⴝ 0.05 ␦ ⴝ 0.10 ␦ ⴝ 0.15 ␦ ⴝ 0.20 ␦ ⴝ 0.25 ␦ ⴝ 0.30 ␦ ⴝ 0.35 ␦ ⴝ 0.40 ␦ ⴝ 0.45 ␦ ⴝ 0.50

Full set reduction

Top 10 suppliers (per cent) 20 10 x 20 10 60 40 50 60 90 90
Bottom 10 suppliers (per cent) 20 0 x 10 30 30 30 20 30 10 0
Top 20 suppliers (per cent) 30 20 x 40 35 60 50 40 75 70 85
Bottom 20 suppliers (per cent) 20 20 x 10 55 40 40 40 55 65 80
Separate input output
set reduction
Top 10 suppliers (per cent) 40 40 20 40 60 90 100 70 90 90 100
Bottom 10 suppliers (per cent) 10 10 30 10 80 80 90 60 100 100 100
Top 20 suppliers (per cent) 30 35 35 65 65 65 85 80 95 95 100
Bottom 20 suppliers (per cent) 40 35 35 45 75 90 75 70 90 90 100

287
 Sustainable supplier key performance indicators
Chunguang Bai and Joseph Sarkis
relative ranking of suppliers based on sustainability measures
be completed, more nuanced benchmarking and performance
evaluation investigation (not completed in this study) can
be completed. For example, DEA can be valuable for
organizations identifying with whom suppliers can benchmark
(partners) to improve their performance. This type of
benchmarking can be completed by considering “facet” sets
within DEA (Charnes, 1994; Sarkis and Talluri, 2004).
Supply chain benchmarking requires models which can deal
with multiple performance measures and which can provide
an integrated measure. DEA seems to fit these requirements
well. The gap lies in the lack of valid measurement criteria and
adequate methodologies to aggregate individual performance
measures into a single index of overall performance (Wong
and Wong, 2008). The use of a single performance measure is
attractive because of its simplicity, such as the ease of
statistical process management tool usage for performance
monitoring (Beamon, 1999; Talluri and Sarkis, 2002).
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
Establishing an integrated measure (such as use by DEA)
allows managers to analyze and compare the performances of
different supply chain scenarios (Chen et al., 2007). In
addition, from a behavioral perspective, if individuals are
confronted with a single performance measure indicating
either high or low performance, there is no ambiguity, and
therefore, there is little room for self-enhancing interpretations
and for additional bias by self-selecting measures for the best
managerial scenario (Audia and Brion, 2007). In fact, many
techniques have been utilized to take multiple measures and
reduce them to a single unified measure including tools such
as DEA (e.g. Talluri and Sarkis, 2002) or the analytical
hierarchy process (Bentes et al., 2012).
But, if a single performance measure is utilized, this
measure must adequately describe the system performance
(Beamon, 1999). The critiques of a single integrated measure
are also self-evident. For example, the selection of a single
measure ignores the distribution of value created across
stakeholder groups (Richard et al., 2009), especially those that
incorporate sustainability measures derived from various
stakeholders. A tension still exists in that managers will still
require additional information from multiple measures and
that too much information and transparency may be lost with
a single measure. For example, performance measurement
tools such the balanced scorecard (Kaplan and Norton, 1996)
and benchmarking tools such as spider charts (Colicchia et al.,
2011), and strategic analysis tools such as key performance
indicators thrive on multiple dimensions (Bai et al., 2012;
Genovese et al., 2013). Practitioners also seem to prefer the
flexibility and need for multiple measures (Kleijnen and Smits,
2003).
This tension and evaluation of which scenario (multiple vs
single measures) is best from a practical and managerial
perspective still requires evaluation. Empirical and experimental
studies may provide additional insights into practical
performance measurement requirements, especially in
sustainable supply chain contexts.
Future research directions
The paper provides a significant methodological contribution,
but it also shows the usefulness of managing suppliers under
sustainability considerations. Even with the initial benefits of
288
Supply Chain Management: An International Journal
Volume 19 · Number 3 · 2014 · 275–291
this investigation, significant limitations and future research
opportunities exist.
We did not consider the possibility of using different
neighborhood granules (␦) to determine the sensitivity of the
solution. As seen in the various evaluations of the
neighborhood granules, the larger the granule value, the more
supplier performance measures that form the KPI. This result
means that the level of correlation between the DEA with a
reduced set of measures and the complete set of measures may
be higher. There may be some additional variations in
finer-tuned granules (e.g., instead of 0.05 changes in
neighborhood values, use 0.01 increments to determine
further sensitivity). Advances in rough set approaches such as
the use of multiple granulations (Qian et al., 2010) could also
be investigated.
For the DEA models, only generic input and output
measures are utilized. When investigating sustainability issues,
there are also concerns of bad outputs. That is, pollution may
be generated by a supplier as an output of its manufacturing
processes. There are more advanced techniques that
incorporate good/bad inputs and outputs (Sarkis and
Cordeiro, 2012). Investigating some of these additional tools
can provide valuable insights into what performance measures
should or should not be considered.
When eliminating the full set of factors, the resulting
solution may only contain input or output factors. This same
situation may also occur with respect to only business
(economic) sustainability inputs/outputs in the final
evaluation or only environmental sustainability factors. If that
situation results, then the managerial acceptance of the
technique may be at jeopardy. To address this potential issue,
separate reductions may be made for various groupings of
performance metrics (business/environmental), similar to the
separated reduction process for input and output measures.
Another issue that could also be investigated is a more
complete rough set evaluation. In the neighborhood rough set
technique described in this paper, we did not include
“decision attributes”, but typically the decision attribute is
very important in the set formation and reduction process. For
example, we could incorporate historical information on
supplier sustainability performance as decision attributes.
Such a historical performance level may provide a more
realistic guide on which metrics have traditionally contributed
to sustainability or economic performance among suppliers.
From a broader DEA variable selection research perspective
other techniques exist (Nataraja and Johnson, 2011), and
the evaluation of utilizing rough set with respect to these
other variable reduction/selection approaches should be
investigated. The stochastic frontier analysis (Van Passel et al.,
2009) integrated with the rough set KPI selection is one such
technique that can complete a relative evaluation to further
identify various exogenous factors that may influence
performance. The rough set does not utilize any preconceived
parametric assumptions in reducing the variable set such as
principal components analysis and regressions approaches and
thus may be advantageous. Also, it does not require as much
computational effort as techniques such as bootstrapping for
variable selection. But, it may lose some predictive power that
other techniques may portend. In addition, we provide a
relatively linear relationship between the techniques, an
 Sustainable supplier key performance indicators
Chunguang Bai and Joseph Sarkis
iterative approach with adjustments and feedback on the
results and managerial input may allow for further refinement
of the performance measures and input/output KPI. These
issues and comparisons provide fertile foundation for
additional investigation.
Notes
1 Information entropy is also defined as information content
of sets of data. See Liang and Shi (2004) for a discussion
of information entropy within the context of rough set
theory.
2 These data are developed by the authors for expository
purposes. It is not actual data from any case company or
field study.
3 An input performance measure is defined as a measure
that improves as its value decreases. An output
performance measure is defined as a measure that
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
improves as its value increases (Sarkis, 2000).
4 A rule of thumb in DEA, for helping to improve its
discriminatory power, is to have the number of objects
(decision-making units) be approximately equal to the
product of the number of inputs (8) and the number of
outputs (18), or three times the sum of the number of
inputs or outputs (Sarkis, 2007). We believed that 120
DMUs in the illustrative example, after identifying a
smaller set of KPI, would be enough reliably to cover most
input/output number scenarios. If all inputs and outputs
are kept, the current total needed at this time would be
144 objects if a product of the number of inputs and
outputs is used, or 78 objects if three times the sum is
used, 120 objects fall into the middle of this range.
5 This term has also been defined as information entropy of
a system (Liang and Shi (2004)).
References
Antai, I. (2011), “Supply chain vs supply chain competition: a
niche-based approach”, Management Research Review,
Vol. 34 No. 10, pp. 1107-1124.
Antai, I. and Olson, H. (2013), “Interaction: a new focus for
supply chain vs. supply chain competition”, International
Journal of Physical Distribution and Logistics Management,
Vol. 43 No. 7, pp. 511-528.
Ashby, A., Leat, M. and Hudson-Smith, M. (2012), “Making
connections: a review of supply chain management and
sustainability literature”, Supply Chain Management: An
International Journal, Vol. 17 No. 5, pp. 497-516.
Audia, P.G. and Brion, S. (2007), “Reluctant to change:
self-enhancing responses to diverging performance measures”,
Organizational Behavior and Human Decision Processes, Vol. 102
No. 2, pp. 255-269.
Azadeh, A., Saberi, M., Moghaddam, R.T. and
Javanmardi, L. (2011), “An integrated data envelopment
analysis – artificial neural network – rough set algorithm for
assessment of personnel efficiency”, Expert Systems with
Applications, Vol. 38 No. 3, pp. 1364-1373.
Bai, C. and Sarkis, J. (2010), “Integrating sustainability into
supplier selection with grey system and rough set
289
Supply Chain Management: An International Journal
Volume 19 · Number 3 · 2014 · 275–291
methodologies”, International Journal of Production
Economics, Vol. 124 No. 1, pp. 252-264.
Bai, C. and Sarkis, J. (2011), “Evaluating supplier
development programs with a grey based rough set
methodology”, Expert Systems with Applications, Vol. 38
No. 11, pp. 13505-13517.
Bai, C. and Sarkis, J. (2012), “Supply-chain
performance-measurement system management using
neighbourhood rough sets”, International Journal of
Production Research, Vol. 50 No. 9, pp. 2484-2500.
Bai, C., Sarkis, J., Wei, X. and Koh, L. (2012), “Evaluating
ecological sustainable performance measures for supply
chain management”, Supply Chain Management: An
International Journal, Vol. 17 No. 1, pp. 78-92.
Banker, R.D. (1984), “Estimating most productive scale size
using data envelopment analysis”, European Journal of
Operational Research, Vol. 17 No. 1, pp. 35-44.
Beamon, B.M. (1999), “Measuring supply chain
performance”, International Journal of Operations and
Production Management, Vol. 19 No. 3, pp. 275-292.
Bentes, A.V., Carneiro, J., da Silva, J.F. and Kimura, H.
(2012), “Multidimensional assessment of organizational
performance: integrating BSC and AHP”, Journal of
Business Research, Vol. 65 No. 12, pp. 1790-1799.
Beynon, M., Curry, B. and Morgan, P. (2000), “Classification
and rule induction using rough set theory”, Expert Systems,
Vol. 17 No. 3, pp. 136-148.
Cabral, I., Grilo, A. and Cruz-Machado, V. (2012), “A
decision-making model for lean, agile, resilient and green
supply chain management”, International Journal of
Production Research, Vol. 50 No. 17, pp. 4830-4845.
Chae, B. (2009), “Developing key performance indicators for
supply chain: an industry perspective”, Supply Chain
Management: An International Journal, Vol. 14 No. 6,
pp. 422-428.
Charnes, A. (Ed) (1994), Data Envelopment Analysis: Theory,
Methodology and Applications, Springer, Netherlands.
Charnes, A., Cooper, W.W. and Rhodes, E. (1978),
“Measurement in the efficiency of decision making units”,
European Journal of Operational Research, Vol. 2 No. 6,
pp. 429-444.
Chen, M.C., Yang, T. and Yen, C.T. (2007), “Investigating
the value of information sharing in multi-echelon supply
chains”, Quality and Quantity, Vol. 41 No. 3, pp. 497-511.
Colicchia, C., Melacini, M. and Perotti, S. (2011),
“Benchmarking supply chain sustainability: insights from a
field study”, Benchmarking: An International Journal, Vol. 18
No. 5, pp. 705-732.
Cooper, W.W., Seiford, L.M. and Tone, K. (2007), Data
Envelopment Analysis: A Comprehensive Text with Models,
Applications, References and DEA-Solver Software, 2nd ed.,
Springer, Heidelberg.
Daniel, D.R. (1961), “Management information crisis”,
Harvard Business Review, Vol. 39 No. 5, pp. 111-121.
Dinter, B. (2013), “Success factors for information logistics
strategy — An empirical investigation”, Decision Support
Systems, Vol. 54 No. 3, pp. 1207-1218.
Dotoli, M., Fanti, M.P., Meloni, C. and Zhou, M. (2006),
“Design and optimization of integrated E-supply chain for
agile and environmentally conscious manufacturing”, IEEE
 Sustainable supplier key performance indicators
Chunguang Bai and Joseph Sarkis
Transactions on Systems, Man and Cybernetics, Part A:
Systems and Humans, Vol. 36 No. 1, pp. 62-75.
Epstein, M.J. and Wisner, P.S. (2001), “Using a balanced
scorecard to implement sustainability”, Environmental
Quality Management, Vol. 11 No. 2, pp. 1-10.
Figge, F., Hahn, T., Schaltegger, S. and Wagner, M. (2002),
“The sustainability balanced scorecard – linking sustainability
management to business strategy”, Business Strategy and the
Environment, Vol. 11 No. 5, pp. 269-284.
Genovese, A., Lenny Koh, S.C., Kumar, N. and
Tripathi, P.K. (2013), “Exploring the challenges in
implementing supplier environmental performance
measurement models: a case study”, Production Planning
and Control, pp. 1-14.
Gimenez, C. and Tachizawa, E.M. (2012), “Extending
sustainability to suppliers: a systematic literature review”,
Supply Chain Management: An International Journal, Vol. 17
No. 5, pp. 531-543.
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
Grimm, J.H., Hofstetter, J.S. and Sarkis, J. (2013), Critical
factors for sub-supplier management: a sustainable food
supply chains perspective. International Journal of Production
Economics, Vol. 152, pp. 159-173.
Gulledge, T. and Chavusholu, T. (2008), “Automating the
construction of supply chain key performance indicators”,
Industrial Management and Data Systems, Vol. 108 No. 6,
pp. 750-774.
Gunasekaran, A. and Kobu, B. (2007), “Performance
measures and metrics in logistics and supply chain
management: a review of recent literature (1995-2004) for
research and applications”, International Journal of
Production Research, Vol. 45 No. 12, pp. 2819-2840.
Hervani, A.A., Helms, M.M. and Sarkis, J. (2005),
“Performance measurement for green supply chain
management”, Benchmarking: An International Journal,
Vol. 12 No. 4, pp. 330-353.
Hu, Q., Yu, D., Liu, J. and Wu, C. (2008a), “Neighborhood
rough set based heterogeneous feature subset selection”,
Information Sciences, Vol. 178 No. 18, pp. 3577-3594.
Hu, Q., Yu, D. and Xie, Z. (2008b), “Neighborhood
classifiers”, Expert Systems with Applications, Vol. 34 No. 2,
pp. 866-876.
Huan, S.H., Sheoran, S.K. and Wang, G. (2004), “A review
and analysis of supply chain operations reference (SCOR)
model”, Supply Chain Management: An International
Journal, Vol. 9 No. 1, pp. 23-29.
Kannan, G., Rajendran, S., Sarkis, J. and Murugesan, P.
(2013), “Multi criteria decision making approaches for
green supplier evaluation and selection: a literature review”,
Journal of Cleaner Production, available online 13 July 2013.
Kaplan, R.S. and Norton, D.P. (1996), The Balanced
Scorecard: Translating Strategy into Action, Harvard Business
School Press, Boston, MA.
Kleijnen, J.P. and Smits, M.T. (2003), “Performance metrics
in supply chain management”, Journal of the Operational
Research Society, Vol. 54 No. 5, pp. 507-514.
Lee, K.H. and Saen, R.F. (2012), “Measuring corporate
sustainability management: a data envelopment analysis
approach”, International Journal of Production Economics,
Vol. 140 No. 1, pp. 219-226.
290
Supply Chain Management: An International Journal
Volume 19 · Number 3 · 2014 · 275–291
liang, J.Y. and Shi, Z.Z. (2004), “The information entropy,
rough entropy and knowledge granulation in rough set
theory”, International Journal of Uncertainty, Fuzziness and
Knowledge-Based Systems, Vol. 12 No. 1, pp. 37-46.
Liu, J., Ding, F.Y. and Lall, V. (2000), “Using data
envelopment analysis to compare suppliers for supplier
selection and performance improvement”, Supply Chain
Management: An International Journal, Vol. 5 No. 4,
pp. 143-150.
Manning, L. (2013), “Corporate and consumer social
responsibility in the food supply chain”, British Food
Journal, Vol. 115 No. 1, pp. 9-29.
Mirhedayatian, S.M., Azadi, M. and Saen, R.F. (2014), “A
novel network data envelopment analysis model for
evaluating green supply chain management”, International
Journal of Production Economics, Vol. 147, pp. 544-554.
Möller, A. and Schaltegger, S. (2005), “The sustainability
balanced scorecard as a framework for eco-efficiency
analysis”, Journal of Industrial Ecology, Vol. 9 No. 4,
pp. 73-83.
Nataraja, N. and Johnson, A.L. (2011), “Guidelines for using
variable selection techniques in data envelopment analysis”,
European Journal of Operational Research, Vol. 215 No. 3,
pp. 662-669.
Neely, A., Adams, C. and Crowe, P. (2001), “The
performance prism in practice”, Measuring Business
Excellence, Vol. 5 No. 2, pp. 6-13.
Olugu, E.U., Wong, K.Y. and Shaharoun, A.M. (2011),
“Development of key performance measures for the
automobile green supply chain”, Resources, Conservation and
Recycling, Vol. 55 No. 6, pp. 567-579.
Ong, C.S., Huang, J.J. and Tzeng, G.H. (2005), “Building
credit scoring models using genetic programming”, Expert
Systems with Applications, Vol. 29 No. 1, pp. 41-47.
Pawlak, Z. (1982), “Rough sets”, International Journal of
Computer and Information Sciences, Vol. 11 No. 5,
pp. 341-356.
Qian, Y., Liang, J. and Dang, C. (2010), “Incomplete
multigranulation rough set”, IEEE Transactions on Systems,
Man and Cybernetics, Part A: Systems and Humans, Vol. 40
No. 2, pp. 420-431.
Richard, P.J., Devinney, T.M., Yip, G.S. and Johnson, G.
(2009), “Measuring organizational performance: towards
methodological best practice”, Journal of Management,
Vol. 35 No. 3, pp. 718-804.
La Rovere, E.L., Soares, J.B., Oliveira, L.B. and Lauria, T.
(2010), “Sustainable expansion of electricity sector:
sustainability indicators as an instrument to support
decision-making”, Renewable and Sustainable Energy
Reviews, Vol. 14 No. 1, pp. 422-429.
Sá, J.A. (1988), “Some empirical evidence on a contingency
theory of success factors”, European Management Journal,
Vol. 6 No. 3, pp. 236-249.
Sadoyan, H., Zakarian, A. and Mohanty, P. (2006), “Data
mining algorithm for manufacturing process control”, The
International Journal of Advanced Manufacturing Technology,
Vol. 28 No. 3, pp. 342-350.
Sarkis, J. (2000), “A comparative analysis of DEA as a discrete
alternative multiple criteria decision tool”, European Journal
of Operational Research, Vol. 123 No. 3, pp. 543-557.
 Sustainable supplier key performance indicators
Chunguang Bai and Joseph Sarkis
Sarkis, J. (2006), Greening the Supply Chain, Springer, Berlin.
Sarkis, J. (2007), “Preparing your data for DEA”, in Zhu, J.
and ve Cook, W.D. (Eds), Modelling Data Irregulaties
Structural Complexities in Data Development Analysis,
Springer, New York, NY, pp. 305-320.
Sarkis, J. (2010), “Benchmarking the greening of business”,
Benchmarking: An International Journal, Vol. 17 No. 3,
pp. 317-319.
Sarkis, J. and Cordeiro, J.J. (2012), “Ecological
modernization in the electrical utility industry: an
application of a bads – goods DEA model of ecological and
technical efficiency”, European Journal of Operational
Research, Vol. 219 No. 2, pp. 386-395.
Sarkis, J. and Talluri, S. (2002), “A model for strategic
supplier selection“, Journal of Supply Chain Management,
Vol. 38 No. 1, pp. 18-28.
Sarkis, J. and Talluri, S. (2004), “Ecoefficiency measurement
using data envelopment analysis: research and practitioner
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)
issues”, Journal of Environmental Assessment Policy and
Management, Vol. 6 No. 1, pp. 91-123.
Seuring, S. (2013), “A review of modeling approaches for
sustainable supply chain management”, Decision Support
Systems, Vol. 54 No. 4, pp. 1513-1520.
Shepherd, C. and Gunter, H. (2006), “Measuring supply
chain performance: current research and future directions”,
International Journal of Productivity and Performance
Management, Vol. 55 Nos 3/4, pp. 242-258.
Shyng, J.Y., Wang, F.K., Tzeng, G.H. and Wu, K.S. (2007),
“Rough set theory in analyzing the attributes of
combination values for the insurance market”, Expert
Systems with Applications, Vol. 32 No. 1, pp. 56-64.
Talluri, S. and Sarkis, J. (2002), “A model for performance
monitoring of suppliers”, International Journal of Production
Research, Vol. 40 No. 16, pp. 4257-4269.
Tone, K. (2002), “A slacks-based measure of super-efficiency
in data envelopment analysis”, European Journal of
Operational Research, Vol. 143 No. 1, pp. 32-41.
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints
291
Supply Chain Management: An International Journal
Volume 19 · Number 3 · 2014 · 275–291
Tsoulfas, G.T. and Pappis, C.P. (2008), “A model for supply
chains environmental performance analysis and decision
making”, Journal of Cleaner Production, Vol. 16 No. 15,
pp. 1647-1657.
Van Passel, S., Van Huylenbroeck, G., Lauwers, L. and
Mathijs, E. (2009), “Sustainable value assessment of farms
using frontier efficiency benchmarks”, Journal of
Environmental Management, Vol. 90 No. 10, pp. 3057-3069.
Vasconcellos e Sá, J. (1988), “The impact of key success
factors on company performance”, Long Range Planning,
Vol. 21 No. 6, pp. 56-64.
Wilson, D.R. and Martinez, T.R. (1997), “Improved
heterogeneous distance functions”, Journal of Artificial
Intelligence Research, Vol. 6 No. 1, pp. 34.
Wong, W.P. and Wong, K.Y. (2008), “A review on
benchmarking of supply chain performance measures”,
Benchmarking: An International Journal, Vol. 15 No. 1,
pp. 25-51.
Xu, J., Li, B. and Wu, D. (2009), “Rough data envelopment
analysis and its application to supply chain performance
evaluation”, International Journal of Production Economics,
Vol. 122 No. 2, pp. 628-638.
Yakovleva, N., Sarkis, J. and Sloan, T. (2012), “Sustainable
benchmarking of supply chains: the case of the food
industry”, International Journal of Production Research,
Vol. 50 No. 5, pp. 1297-1317.
Yeh, C.C., Chi, D.J. and Hsu, M.F. (2010), “A hybrid
approach of DEA, rough set and support vector machines
for business failure prediction”, Expert Systems with
Applications, Vol. 37 No. 2, pp. 1535-1541.
Zhu, Q., Dou, Y. and Sarkis, J. (2010), “A portfolio-based
analysis for green supplier management using the analytical
network process”, Supply Chain Management: An
International Journal, Vol. 15 No. 4, pp. 306-319.
Corresponding author
Joseph Sarkis can be contacted at: jsarkis@wpi.edu
 This article has been cited by:

1. Sunil Luthra, Kannan Govindan, Devika Kannan, Sachin Kumar Mangla, Chandra Prakash Garg. 2017. An integrated framework
for sustainable supplier selection and evaluation in supply chains. Journal of Cleaner Production 140, 1686-1698. [CrossRef]
2. Morgane M.C. Fritz, Josef-Peter Schöggl, Rupert J. Baumgartner. 2017. Selected sustainability aspects for supply chain data
exchange: Towards a supply chain-wide sustainability assessment. Journal of Cleaner Production 141, 587-607. [CrossRef]
3. Zulfiquar N. Ansari, Ravi Kant. 2016. A state-of-art literature review reflecting 15 years of focus on sustainable supply chain
management. Journal of Cleaner Production . [CrossRef]
4. Mengfeng Gong, Andrew Simpson, Lenny Koh, Kim Hua Tan. 2016. Inside out: The interrelationships of sustainable
performance metrics and its effect on business decision making: Theory and practice. Resources, Conservation and Recycling .
[CrossRef]
5. Toshiyuki Sueyoshi, Yan Yuan, Mika Goto. 2016. A Literature Study for DEA Applied to Energy and Environment. Energy
Economics . [CrossRef]
6. Madjid Tavana, Morteza Yazdani, Debora Di Caprio. 2016. An application of an integrated ANP–QFD framework for sustainable
supplier selection. International Journal of Logistics Research and Applications 1-22. [CrossRef]
7. Mouna Rekik, Khouloud Boukadi, Hanene Ben-AbdallahA Comprehensive Framework for Business Process Outsourcing to the
Cloud 179-186. [CrossRef]
Downloaded by UNIVERSITY OF OTAGO At 00:34 10 December 2016 (PT)

8. Anthony SwaimJames James Anthony Swaim MaloniMichael J. Michael J. Maloni HenleyAmy Amy Henley CampbellStacy Stacy
Campbell Kennesaw State University, Kennesaw, Georgia, USA Department of Management and Entrepreneurship, Kennesaw
State University, Kennesaw, Georgia, USA Kennesaw State University, Kennesaw, Georgia, USA . 2016. Motivational influences
on supply manager environmental sustainability behavior. Supply Chain Management: An International Journal 21:3, 305-320.
[Abstract] [Full Text] [PDF]
9. Konrad Zimmer, Magnus Fröhling, Frank Schultmann. 2016. Sustainable supplier management – a review of models supporting
sustainable supplier selection, monitoring and development. International Journal of Production Research 54:5, 1412-1442.
[CrossRef]
10. Chandra Prakash, M.K. Barua. 2016. An analysis of integrated robust hybrid model for third-party reverse logistics partner
selection under fuzzy environment. Resources, Conservation and Recycling 108, 63-81. [CrossRef]
11. Joseph SarkisCorporate Environmental Sustainability and DEA 483-498. [CrossRef]
12. S. M. Lo. 2015. Impact of greening attitude and buyer power on supplier environmental management strategy. International
Journal of Environmental Science and Technology 12:10, 3145-3160. [CrossRef]
13. Philip Beske-Janssen Centre for Sustainability Management (CSM), Leuphana Universität Lüneburg, Lüneburg, Germany.
Matthew Phillip Johnson Centre for Sustainability Management (CSM), Leuphana Universität Lüneburg, Lüneburg, Germany.
Stefan Schaltegger Centre for Sustainability Management (CSM), Leuphana Universität Lüneburg, Lüneburg, Germany. .
2015. 20 years of performance measurement in sustainable supply chain management – what has been achieved?. Supply Chain
Management: An International Journal 20:6, 664-680. [Abstract] [Full Text] [PDF]
14. Árni Halldórsson Department of Technology Management and Economics, Chalmers University of Technology, Gothenburg,
Sweden. Juliana Hsuan Department of Operations Management, Copenhagen Business School, Copenhagen, Denmark. Herbert
Kotzab Department of Logistics Management, University of Bremen, Bremen, Germany. . 2015. Complementary theories to
supply chain management revisited – from borrowing theories to theorizing. Supply Chain Management: An International Journal
20:6, 574-586. [Abstract] [Full Text] [PDF]
15. Stefan Gold International Centre for Corporate Social Responsibility, Nottingham University Business School, Nottingham, UK
Alexander Trautrims Operations Management and Information Systems, Nottingham University Business School, Nottingham,
UK Zoe Trodd American and Canadian Studies, University of Nottingham, Nottingham, UK . 2015. Modern slavery challenges
to supply chain management. Supply Chain Management: An International Journal 20:5, 485-494. [Abstract] [Full Text] [PDF]
16. Chunguang Bai School of Management Science and Engineering, Dongbei University of Finance and Economics, Dalian, China
Joseph Sarkis School of Business, Worcester Polytechnic Institute, Worcester, MA, USA Yijie Dou Center for Industrial and
Business Organization, Dongbei University of Finance & Economics, Dalian, China . 2015. Corporate sustainability development
in China: review and analysis. Industrial Management & Data Systems 115:1, 5-40. [Abstract] [Full Text] [PDF]
17. Roger Burritt, Stefan Schaltegger. 2014. Accounting towards sustainability in production and supply chains. The British Accounting
Review 46:4, 327-343. [CrossRef]
18. Kijpokin KasemsapThe Role of Sustainable Performance Measurement System in Global Supply Chain 331-348. [CrossRef]