Journal Pre-proof

Supplier sustainability performance evaluation using the analytic network process

Mihalis Giannakis, Rameshwar Dubey, Ilias Vlachos, Yanbing Ju

PII: S0959-6526(19)34309-4
DOI: https://doi.org/10.1016/j.jclepro.2019.119439
Reference: JCLP 119439

To appear in: Journal of Cleaner Production

Received Date: 7 March 2019

Revised Date: 31 October 2019
Accepted Date: 23 November 2019

Please cite this article as: Giannakis M, Dubey R, Vlachos I, Ju Y, Supplier sustainability performance
evaluation using the analytic network process, Journal of Cleaner Production (2019), doi: https://
doi.org/10.1016/j.jclepro.2019.119439.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2019 Published by Elsevier Ltd.

 Supplier Sustainability Performance Evaluation using the Analytical
Network Process
Mihalis Giannakis1*, Rameshwar Dubey2 Ilias Vlachos3, Yanbing Ju4

1*
Audencia Business School, Nantes, France (corresponding author)
Email: mgiannakis@audencia.com
2
Montpellier Business School, Montpellier, France
Email: ramewshwardubey@gmail.com
3
La Rochelle Business School, La Rochelle, France
Email: ivlachos@gmail.com
4
Beijing Institute of Technology, Beijing, China
Email: juyb@bit.edu.cn

Abstract
We develop a sustainability performance measurement framework for supplier evaluation and selection, using
the Analytic Network Process (ANP) method. Even though the literature is rife with studies that deal with the
supplier selection problem, companies that actively pursue sustainability strategies may need to add metrics
that show suppliers’ sustainability performance. Existing models for measuring sustainability performance are
limited in that they either evaluate the environmental and social performance separately, do not consider the
inter-relationships between metrics across the three dimensions of sustainability, or utilize metrics that are
difficult to obtain and evaluate accurately. To overcome this deficiency, we use the ANP method, that takes
into account the interrelations between quantifiable and easy to obtain sustainability-related evaluation
metrics. First, through an extensive literature review and feedback from an experts’ panel, we select and
classify salient sustainability performance metrics related to supplier evaluation. With data collected through
an extensive survey amongst 144 supply chain professionals in the UK and France, we develop the
interdependencies between several sustainability metrics and determine the most critical metrics by
calculating their relative weights. Results show that the selected socio-economic metrics carry the most
relatively important role in supplier selection. Based on the findings of the study, we discuss implications for
theory and practice. The proposed evaluation system can provide details on observing sustainable supply chain
performance. It can also help to get a clearer insight into sustainability with a well-established quantitative
decision-making process so that business strategies can be developed with more concerns on supply chain
sustainability.

Keywords: Sustainable Supplier Selection, Sustainability Performance, Analytical Network

Process
 Supplier Sustainability Performance Evaluation using the Analytic
Network Process
Abstract
We develop a sustainability performance measurement framework for supplier evaluation and selection,
using the Analytic Network Process (ANP) method. Even though the literature is rife with studies that deal
with the supplier selection problem, companies that actively pursue sustainability strategies may need to
add metrics that show suppliers’ sustainability performance. Existing models for measuring sustainability
performance are limited in that they either evaluate the environmental and social performance separately,
do not consider the inter-relationships between metrics across the three dimensions of sustainability, or
utilize metrics that are difficult to obtain and evaluate accurately. To overcome this deficiency, we use the
ANP method, that takes into account the interrelations between quantifiable and easy to obtain
sustainability-related evaluation metrics. First, through an extensive literature review and feedback from
an experts’ panel, we select and classify salient sustainability performance metrics related to supplier
evaluation. With data collected through an extensive survey amongst 144 supply chain professionals in the
UK and France, we develop the interdependencies between several sustainability metrics and determine
the most critical metrics by calculating their relative weights. Results show that the selected socio-
economic metrics carry the most relatively important role in supplier selection. Based on the findings of
the study, we discuss implications for theory and practice. The proposed evaluation system can provide
details on observing sustainable supply chain performance. It can also help to get a clearer insight into
sustainability with a well-established quantitative decision-making process so that business strategies can
be developed with more concerns on supply chain sustainability.

Keywords: Sustainable Supplier Selection, Sustainability Performance, Analytical Network

Process

1. Introduction
Over the past decade, a large number of companies have adopted (or claim to have adopted) sustainable

management strategies (Hou et al. 2019). A growing body of literature has been established that developed

useful sustainability metrics and sustainable performance management frameworks (Hassini et al., 2012,

Morioka et al., 2016; Baxter et al. 2018, Osiro et al. 2018). The majority of studies consider sustainable

performance management using environmental, social, as well as economic/financial measures (see, for

example, Ahi and Searcy, 2015, Kuo et al. 2015, Dubey et al. 2017, dos Santos et al., 2019, Pislaru et al.

-1-
2019). Amongst the developed methods for measurement of sustainability in corporations, sectors and

even nations, notable contributions include the Global Reporting Initiative (2015), the world business

council for sustainable development (2015), the development of standards (OECD 2004) and the Dow

Jones sustainability index (Searcy et al., 2012).

Sustainability has been integrated into the corporate strategy of many organisations. Yet, one of the main

challenges is to develop models to guide decisions towards achieving the sustainability goals as they were

originally defined by the Brundtland Commission report,that emphasizes the interrelationships among

society, the environment, and economic development. In this direction, as organisations are increasingly

dependent upon the performance of their suppliers, it is imperative for companies to monitor and evaluate

not only the sustainable performance of their operations but also to extend this evaluation to their suppliers

and other stakeholders. Therefore, problems such as the selection of sustainable suppliers (defined as

suppliers with good sustainability performance) (Haeri and Rezaei, 2019), or the investment of a venture

capital fund on a business based on its perceived sustainable performance, are of paramount importance

for organisations in their pursuit of sustainable performance (Bai and Sarkis, 2010, Rashidi and Cullinane,

2019).

The selection of sustainable suppliers constitutes one of the most important supply chain decisions

towards a company’s sustainable performance (Chen et al., 2006, Chai et al. 2013), as the suppliers’

performance is directly correlated with vendors’ performance. For example, the Rana Plaza disaster in

Bangladesh in 2013, which involved perilous working conditions at a garment supplier that lead to the

death of more than 1100 people, tarnished the reputation of a large number of companies in the fashion

industry Ehrgott et al. (2011). It turned out that the due diligence and supplier selection processes for the

majority of vendors from that supplier were inadequate. As a result, the apparel companies were obliged to

sign a legislation to be held accountable for monitoring their Bangladeshi suppliers (Jacobs and Singhal,

2017).

There is a plethora of studies that deal with the supplier selection problem. A broad spectrum of different

methods and techniques have been applied. They include multi-attribute decision analysis methods such as

data envelopment analysis, total cost of ownership (Bhutta and Huq, 2002) weighted linear model

approaches and weighted sum models, linear and goal programming models, case-based reasoning,

-2-
clustering methods, human judgment models, or statistical analysis (de Boer et al., 2001; Ho et al., 2010).

However, companies that actively incorporate sustainability into their strategy may need to add to their

selection criteria, metrics that show suppliers’ sustainability performance (Govindan et al., 2015).

The existing models for measuring sustainability performance are limited in that they focus on evaluating

the environmental and social performance separately and do not consider the inter-relationships between

different sustainability performance metrics (Singh et al., 2009, Bai and Sarkis 2010, Hassini et al, 2012)

and more importantly the relative prioritization of these metrics (Luthra et al., 2017). For example, in the

case where a global supplier has a low carbon footprint, but at the same its performance in terms of the

working conditions for its employees or the tax contributions to the local economy is poor, it is essential to

consider the vendor’s priorities in each of these dimensions and then determine a score for the overall

sustainable performance of the supplier (Price and Sun, 2017). Similarly, a precise method for evaluating

the sustainability performance based on prioritization of sustainability metrics is useful in the case where a

supplier may have the process of ensuring financial transparency and providing value to local

communities, but at the same time having an opaque relationship with regional governments which may

undermine competition. Which of the sustainability indicators are more important for vendors, so that they

can identify the best supplier based on their corporate strategy and vision for sustainability? In light of

these observations, we set out two objectives:

(1) To identify and select appropriate evaluation criteria for sustainable supplier selection, and

(2) To develop a sustainability performance evaluation framework for sustainable supplier selection

As the sustainability-related performance dimensions may be closely correlated, decision-makers would

benefit from a method which: (i) guides them in selecting the most appropriate performance indicators by

identifying which attributes of suppliers’ performance they need (or not need) to utilize, and (ii) assesses

the perceived importance of each performance indicator so that the strategic objectives of the firm can be

achieved (Ho et al., 2010).

The paper proceeds as follows. In the 2nd section, we review pertinent literature on sustainable supplier

evaluation and selection with a particular focus on the use of MCDM methods. We also discuss the ANP

method and justify its usefulness in the sustainable supplier evaluation and problem. In the 3rd section, we

present our proposed method for the development of a sustainability multi-criteria framework for supplier
-3-
selection. Through a comprehensive literature review on the use of sustainability metrics, we provide the

foundations for the selection of appropriate sustainability-related indicators. We also present the data

collection process for the empirical research to operationalise and validate our proposed approach. In the

4th section, we present the findings of the study, and in the final 5th section we conclude our paper with a

discussion of the main contributions of the study, its limitations and an agenda for future research.

2. Literature Review
Sustainable supply chain management can be defined as “the management of material, information and

capital flows as well as cooperation among companies along the supply chain while taking goals from all

three dimensions of sustainable development, i.e., economic, environmental and social, into account which

are derived from customer and stakeholder requirements” (Seuring and Müller, 2008). It involves

managerial decisions across several areas towards achieving sustainability, such as changing the

organisational culture, improving transparency and effective supply chain risk management (Carter and

Rogers, 2008). We use these definitions as a precursor

An indispensable part of sustainable supply chain management is the selection of sustainable suppliers and

the evaluation of their performance (Luthra et al., 2017). There is a growing literature on the selection of

sustainable suppliers that incorporate factors related to the triple bottom line effect of sustainability

(Govindan et al., 2013; Govindan et al., 2015). Although economic and environmental performance have

long existed as parameters in the sustainable supplier selection process, the systematic consideration of

social and ethical outlook of companies has been developed recently, including issues such as child labor,

human right abuses, and corruption indexes (Awasthi et al., 2018).

The sustainable supplier selection problem can be considered as a multiple-criteria decision-making

(MCDM) process. The MCDM discipline is very useful for disentangling the decision-making processes

of complex issues but also enables decision-makers to take into account and balance the trade-offs

amongst a wide range of criteria that can affect a decision (Saaty, 1996; Chai et al., 2013). This is

particularly evident in the selection of sustainable suppliers where the tradeoffs between economic and

either social or environmental issues are prevalent.

Several MCDM evaluation methods have been applied, with Analytic Hierarchy Process, Analytic

Network Process and fuzzy-based approaches dominating the literature (Zimmer et al., 2016). Among the

-4-
first researchers that incorporated sustainability into the supplier selection problem are Lu et al.

(2007) who explored environmental principles applicable to green supplier evaluation by using a multi-

objective decision analysis, Bai and Sarkis (2010) who used a grey system and rough set theory to develop

a sustainability framework for their supplier selection process and Buyukozkan and Çifçi (2011) who

used a fuzzy multi-criteria decision framework for sustainable supplier selection with incomplete

information. In Table 1 we provide a short list of well-cited studies over the past 10 years related to

sustainable supplier selection that use MDCM methods to illustrate the growing interest in the topic.

As sustainability is becoming a central strategic objective for many companies, the proposed MCDM

models are becoming more elaborate and focused on specific sustainability and operational related issues.

For example, the ANP/AHP methods have been combined with rough set and fuzzy set theories or with

several MCDM methods. Chen et al. (2019) use a rough-fuzzy DEMATEL-ANP method for evaluating

sustainable requirements. Notable recent studies in this domain are also the work the work of Parkouhi et

al. (2019) who use Grey DEMATEL technique to examine the importance of the criteria used for supplier

selection, and the work Bai et al. (2019) who use a group decision support approach to select suppliers

using social sustainable criteria.

Table 1:
Summary of literature on the use of MDCM for sustainable supplier evaluation and selection
Authors Method Topic / relevance to this study
A green supplier selection model that considers the vagueness
Lee et al. (2009) Fuzzy extended AHP
of experts’ opinions in evaluating criteria.
Fuzzy entropy and fuzzy A sustainability performance model that utilizes an alert
Erol et al. (2011)
multi-attribute utility management system.
Amindoust et al. A mathematical sustainable supplier ranking model that
Fuzzy Inference System
(2012) considers the subjectivity of decision-makers’ perceptions
A sustainability performance measurement tool that uses
Govindan et al.
Fuzzy TOPSIS triangular fuzzy numbers to express linguistic values of experts'
(2013)
subjective preferences.
A model that rates green suppliers and allocates optimum order
Kannan et al. (2013) Fuzzy AHP / MOLP
quantities based on various constraints.
Fuzzy AHP / multi-objective A mathematical model that allocates the optimal quantities of
Azadina et al. (2015)
mathematical programming orders to suppliers considering lot-sizing problems.
Govindan et al. A systematic literature review to identify the most prevalent
Literature review
(2015) techniques and topics for green supplier evaluation.
ANP / Grey Relational A green supplier selection model that enables decision-makers
Hashemi et al. (2015)
Analysis to use linguistic evaluation.
Bayesian framework / Monte A model to rank and select suppliers using specific selection
Sarkis et al. (2015)
Carlo Markov chain simulation objectives, appropriate for small or missing data sets.
A framework for evaluating sustainable supplier selection using
Luthra. et al. (2017) AHP / VIKOR
22 evaluation criteria.
Haeri & Rezaei “Best-worst” / fuzzy grey
A weight assignment model for green supplier selection.
(2019) cognitive maps

-5-
 A method for supplier selection that takes into account the
Kellner & Utz (2019) Markowitz portfolio theory trade-offs between the supplier sustainability, the purchasing
costs, and the overall supply risk.
A method to weigh sustainable supply chain management
Li et al. (2019) Rough cloud TOPSIS
practices under conditions of decision method uncertainty.

2.1 The ANP method

In our pursuit for a sustainability evaluation framework for supplier selection, we utilize the ANP method

to identify the relative importance and prioritization of specific sustainability metrics, which can assist the

decision of sustainable suppliers. The ANP is a generalization of the AHP method and is useful in solving

complex decision-making problems (Saaty and Vargas, 2004). Our choice is primarily grounded in the

fact that the ANP method can deal with complex multi-criteria issues and can generate evidence of inter-

relationships and dependencies among several performance metrics and therefore, determine a more

accurate ranking of each metric (Saaty, 1996).

The ANP method has been combined with fuzzy or rough sets models and/or with other MCDM methods

such as TOPSIS. However, the main limitation of TOPSIS is that it assumes that all the criteria are

independent (Aguaron-Joven et al., 2015), which is not always the case in sustainability. Saaty and Tran

(2007) posit that good judgement produces valid answers in AHP, and fuzzy AHP simply produces

perturbation without producing any better outcomes. If the pairwise comparison is consistent, then fuzzy

AHP would yield similar priority vectors as the classical AHP (Csutora and Buckley, 2011). Similarly,

fuzzy set theory and its application in combination with MCDM methods is appropriate when dealing with

the uncertainty of measurement of qualitative evaluation metrics (Ferrero and Salicone, 2004; Lun and

Leng, 2007; Rahmanita et al., 2018).

The majority of existing studies that deal with the sustainable supplier selection problem utilise metrics

that cover the triple bottom line of sustainability. However, a critical limitation is that they use evaluation

criteria that are either generic, difficult to quantify and entail a certain level of ambiguity. For example,

despite the usefulness of such criteria such as the presence of “environmental management systems”

(Govindan et al., 2015), “environmental costs” (Luthra et al., 2019), or “fight for fair-trading” (Li et al.,

2019), their evaluation is not easy and requires subjective measures. In our study, we select metrics that

can be obtained accurately and are easily quantifiable, as most of them are in the form of ratios. Therefore,

-6-
the use of a method that combines the ANP method with fuzzy or rough set theories would not add

substantial benefits to our model.

Saaty and Vargas (2006) outline four steps for the utilization of the ANP method.

Step 1 - Define the problem and construct the network model in a logical way including appropriate and

parsimonious criteria: Unlike the AHP method where the problem is structured as a hierarchy, in ANP,

the problem is structured in the form of a network, that consists of decision clusters and elements which

are connected through network links. The links represent the inter-dependencies amongst the clusters and

the elements (Saaty, 2004). The network is built by considering all potential inter-dependencies amongst

the elements of the system. The inter-dependencies among elements can be in the form of outer

dependencies (when an element or elements in a cluster have a dependency with an element or elements in

other clusters), or inner dependencies (when an element or elements in the same cluster have an influence

on each other) (Saaty, 2004). The objective of the decision (in our case the selection of sustainable

suppliers) is defined as the goal of the hierarchy of the selected criteria (the sustainability metrics).

Step 2 - Conduct pairwise comparisons of the clusters and elements and obtain priority vectors: For a
given set of sustainability metrics, the ANP method conducts pairwise comparisons for each combination
of pairs of these metrics and attempts to identify how much more does a metric (of a single pair of metrics)
influence the decision for selecting a supplier than the other metric of the pair, to finally rank the
alternatives in the selection of the supplier (Saaty and Vargas, 2006). During this step, the potential
decision-makers are asked to evaluate the relative importance clusters and elements. For every pairwise
comparison matrix, the consistency index and ratio should be calculated to check for the consistency of the
matrix. The concept of consistency is that if metric a is more important than is metric b, and metric b is
more important than is metric c, then metric a must be more important than metric c. The consistency ratio
Consistenc y Index
(C.R.) can be calculated by the formula Consistenc y Ratio =
Random Consistenc y Index
λ −n
and the consistency index (C.I) as follows C.I. = n − 1 , with n representing the number of criteria. The
max

random consistency index (R.I) as proposed by Saaty and Vargas (2007). For n=12 elements the RI=1.48.

If C.R≤0.1, the pairwise comparison matrix is consistent.

The pairwise comparisons are conducted using the following ranking scale (Saaty, 1996).

-7-
 Metric 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Metric
ai aj (1)

Where 1= equally important; 3 = moderately more important; 5= strongly more important; 7= very strongly more important; 9=
extremely more important; 2-4-6-8 = mean intermediate values (Saaty, 1996).

Step 3 – Form the supermatrix: The ANP problem is represented by a supermatrix, which is a partitioned

matrix in which each block matrix represents a relationship between two clusters (Peykarjou & Safavi,

2015). The structure of the supermatrix at the element level is shown below. If denotes the Nth cluster

and denotes the nth element in the Nth cluster, is a block matrix that denotes the priority vectors,

which represent the relative importance of the elements in the ith cluster with regard to the jth cluster

(Bottero & Mondini, 2008), i.e. the inter-dependencies between the elements (metrics). All relations are

assessed through pairwise comparisons and the priority vectors are used to construct the supermatrix. If

there is no relationship between clusters and within elements in the same cluster, will be zero. This

analytical process yields an unweighted supermatrix of the following form (Saaty, 2008).

C1 C2 CN

e e .. e e e .. e e e m2 .. e
11 12 1n1 21 22 2n2 m1 mnm

e 11

C1 e 12 W 11 W 12 W 1m
.

e 1n1

. e 21
W 21 W 22 W 2m
W= e 22 : : :
C2 .

e 2n2 (2)
.
e m1
e m2
Cm .
e mnm W m1 W m2 W mm

The supermatrix needs to be stochastic so that we can derive meaningful limiting priorities. Hence, there is

a need to form a weighted supermatrix. In our approach, we do this by using the priority vector from the

cluster matrix obtained in the previous step, weighted by each block matrix that falls into the column

under the given cluster (Peykarjou & Safavi, 2015). For instance, the first entry of the priority vector

derived from the cluster matrix is multiplied by all elements in the first block matrix of the unweighted

supermatrix, and so on. Based on the priority vectors obtained from the pairwise comparison matrices, the

unweighted and weighted supermatrices are constructed.

-8-
Step 4 - Identify the weighting attributes of the elements towards the goal and select the best alternatives:

The final step of the ANP is to obtain a priority ranking for each element in the network structure. In the

weighted supermatrix, there is only the direct influence of an element on any other element. However, an

element that directly influences a second element can indirectly influence a third element that is directly

influenced by a second element. Therefore, the weighted supermatrix obtained from the previous step is

transformed into a limiting supermatrix by raising itself to a limiting power to converge and to obtain a

long-term, stable set of weights, which is the global priority vector for each element (Abastante et al.,

2011). The reason for this calculation is to capture the transmission of all influence paths within the

network (Peykarjou & Safavi, 2015).

3. Research Method

3.1 Our ANP approach

Following the steps for the application of the ANP method (Saaty and Vargas, 2006), our proposed

approach consists of the following steps.

Identify dependencies between
Selection of Sustainability
the selected metrics and construct
performance metrics
the network model
Conduct a survey amongst Calculate the weights of the
professionals for pairwise criteria and rank them using a
comparisons of the criteria decision supermatrix

Selection of sustainability metrics: The selection of the sustainability-related metrics was done through a

thorough review of the literature. To triangulate our choice of the selected metrics, we also consulted a

panel of 5 senior academic experts what work in the field of sustainable supply chain management through

structured interviews. During the interviews, the experts were asked to provide their opinion on whether

selected metrics from the literature were relevant or not and to provide alternatives and conditions

whereby metrics should be selected. Based on the literature review and the experts’ consensus on the

process and selected criteria, a final list of metrics was established.

A large number of sustainability metrics have been proposed in studies from bodies of literature spanning

economics, management, agriculture, and politics. Typical metrics cover environment factors (e.g., energy

consumption, greenhouse gas emissions, waste generation), social issues, (e.g., total number of employees,

-9-
gender equality) and economic factors (e.g., economic value added, net income and revenue) (Szekely and

Knirsch, 2005). For the design of our proposed sustainability performance framework, initially, we

selected sustainability metrics based on the review of existing literature related to sustainability

performance indicators. Based on the literature review and the experts’ interview, we formulated and

applied the following four main qualifying conditions (Table 2).

Table 2:
Qualifying conditions for the selection sustainability performance indicators
Criteria Definition
Relevance to Promote sustainability (Guy & Kibert, 1998); Critically activity-related (Keeble et al., 2003);
corporate and Relevant to key internal and external concerns (Keeble et al., 2003); Complement existing
public policy regulatory programs (Tanzil & Beloff, 2006); Have social appeal, measure what is important to
stakeholders (Reed et al., 2006); Involve several community stakeholders, Useful for decision-
making (Tanzil & Beloff, 2006); Relevant to the local system (Nordheim & Barrasso, 2007
Comparability Capable of comparison with other values reported elsewhere (Holland, 1997); Potentially
benchmarkable (Keeble et al., 2003); Have a target level and baseline with regard to which to
measure (Reed et al., 2006).
Measurability Measureable and verifiable (Keeble et al., 2003); Be easily measured, make use of available data
(Reed et al., 2006); Quantitatively measurable and/or qualitatively descriptive, Data available or
possible to generate (Nordheim & Barrasso, 2007);.
Cost Effective Cost-effective in term of data collection (Tanzil & Beloff, 2006); Be cost effective to measure
(Reed et al., 2006).

We then applied the following additional set of conditions to filter the abundance of metrics that are

available in the literature. Based on the definitions of Seuring and Muller (2008) and of Carter and Rogers

(2008), we included metrics that cover all the three commonly used dimensions of sustainability, that are

simple to develop, relevant to business and useful to decision-makers, cost-effective to collect and

measure, easy to understand to a wide variety of people, assure the results can be reproducible, consistent

and comparable, and prevent the calculation of confidential or private information (Schwarz et al., 2002).

Finally, we selected metrics based on their ability to be constructed as a ratio that meets the needs of

industries and their numerators and denominators can be related to one product unit. This final qualifying

condition was deemed critical for the practical application of our proposed method, as the use of

quantitative metrics is easier to measure and obtain more objectively. Based on these conditions, we

selected the following sustainability-related performance metrics (Table 3).

- 10 -
Table 3:
Selected Sustainability metrics based on those commonly used in the literature
Metrics Authors
En1 - Greenhouse gas emissions Veleva & Ellenbecker (2001), Székely & Knirsch (2005), Singh
(tons/unit of production) et al. (2007), Santos et al. (2013), Erol et al. (2009)
En2 - Water consumption Veleva & Ellenbecker (2001), Singh et al. (2007), Erol et al.
(m3/unit of production) (2009), Hubbard (2009), Weber et al. (2010), Santos et al. (2013)
Environmental
En3 - Energy consumption Veleva & Ellenbecker (2001), Székely & Knirsch (2005), Singh
(kWh/unit of production) et al. (2007), Hubbard (2009), Santos et al. (2013)
En4 - Amount of waste generated Veleva & Ellenbecker (2001), Erol et al. (2009), Weber et al.
(tons/unit of production) (2010)
So1 – (Social) Investment in Epstein & Roy (2001), Linton et al. (2007), Székely & Knirsch
community (2005), Searcy et al. (2007), Erol et al. (2009), Hubbard (2009),
(% proportion of revenues) Bai and Sarkis (2010)
So2 - Customer/Community Veleva & Ellenbecker (2001), Linton et al. (2007), Erol et al.
Social complaints (rate) (2009)
So3 - Health and safety incident Veleva & Ellenbecker (2001), Singh et al. (2007), Santos et al.
rate (rate e.g. number/hour) (2013)
So4 - Average hour of employee
Veleva & Ellenbecker (2001), Székely & Knirsch (2005)
training (hours/ employee)
Ec1 - Productivity Székely & Knirsch (2005), Linton et al. (2007), Singh et al.
(turnover / employee) (2007)
Ec2 - Return on equity
Hubbard (2009), Weber et al. (2010), Santos et al. (2013)
(percentage)
Economic Ec3 - Economic value added
Epstein & Roy (2001), Picazo-Tadeo et al. (2011)
(in currency used, e.g. USD)
Ec4 - Investment in sustainable
processes/products (proportion Searcy, et al. (2007), Singh et al. (2007), Erol et al. (2009)
from annual revenues)

Their definition and major proponents are provided in the Appendix of the paper. It should be noticed that

the objective of our study is not to come up with an exhaustive list of sustainability metrics, but to point

out the breadth of different sustainability issues that need to be considered for the development of

sustainable supplier performance framework. Further classifications and detailed metrics could be

proposed for specific industries. Depending upon the strategy of organizations, the perspectives of their

stakeholders, the preferred indicators may be unique for each company.

Identify dependencies and design the network structure: To create the network structure of our MCDM

problem, first we classified the criteria into clusters (sustainability pillars) that comprise various elements

- 11 -
(sustainability metrics) (Bottero & Mondini, 2008). Subsequently, we consulted the panel of experts that

helped us in the selection of the sustainability metrics for a second round of interviews to define the inter-

dependencies between the clusters and between each element of the network. The experts were asked to

provide their opinion on potential dependencies between the selected criteria by inserting in a relationship

matrix the number 1 if they considered there was a dependency between a pair of metrics, and 0 if they

considered there was not any relationship. We determined that a relationship between two criteria is

present if the majority (i.e., more than 3 experts) agreed on it. Based on our analysis and the experts’

opinions, we identified the following key relationships between elements within the network (Figure 1).

Inner dependencies Explanation

It is possible that companies can minimize the community complaints if they can
So1 So2 engage the communities in which they operate.
So4 So3 Providing training to employees, might influence the health and safety incident
rate.
Ec1 Investment in new processes and products might influence productivity, return
Ec4 Ec2 on equity and economic value added.
Ec3

Outer dependencies Explanation

Ec1 The level of investment in the community might be dependent on economic
performance. For instance, if companies can generate higher revenue and profits,
Ec2 So1
they will have adequate budgets to create the initiative to invest in the
Ec3 community in which they operate.
En1 If companies can invest in developing better processes for reducing negative
En2 impacts on the environment, it is likely that they will achieve better
Ec4 environmental performance in the four metrics.
En3
En4
En1 Greenhouse gas emission and the amount of waste generated might influence
So2 community and customer complaints.
En4

Productivity might be dependent on the average number of hours of employee

So4 Ec1 training. The productivity of turnover per employee can potentially increase by
providing training and education for employees.
Figure 1: Inner and outer dependencies

Goal
Selection of sustainable supplier /
prioritization of sustainability metrics

H A B C
I

Social responsibility D Economic/Financial

performance (So) G performance (Ec)

F E
Environmental
performance (En)
- 12 -
 Outer Dependency
Inner Dependency

Figure 2: The network model at the cluster level

Based on the proposed inner and outer dependencies among all the elements, the network model at the

cluster level is shown in Figure 2. The nodes of the network show the components of the system and the

arcs represent the interactions between them. In Figure 3 we demonstrate the detailed network structure at

the element level, showing the inter-relations and dependencies within the network. In our analysis, we

considered all the interactions amongst the elements. It can be seen that all the metrics are linked.

Goal
Selection of sustainable supplier /
prioritization of sustainability metrics

Social Responsibility Performance

Economic/Financial Performance
Investment in community
(So1) Productivity or Turnover per
Customer / Community employee (Ec1)
complaints (So2)
Return on Equity (Ec2)
Health and safety incident
rate (So3)
Economic Value Added
Average number of hours of
(Ec3)
employee training (So4)
Investment in Sustainable
Processes and Products (Ec4)

Outer Dependency
Greenhouse gas Energy Water Consumption Amount of waste
Inner Dependency emissions (En1) Consumption (En2) (En3) generated (En4)

Environmental Performance

Figure 3: The network model at element level and the interaction among clusters and elements

Pairwise comparisons: Based on the structure of the network, we built the structure of the unweighted

supermatrix. The unweighted supermatrix helped us identify the number of pairwise comparisons at the

cluster as well as the element levels that are needed in our model. Table 4 shows the structure of the

unweighted supermatrix at the cluster level. It consists of the sub-matrices (A, B, C, D, E, F) that were

obtained from the priorities vectors derived from the pairwise comparisons, as defined by our network model.

If there is no relationship between clusters, the sub-matrices are zero.

Table 4: The unweighted supermatrix

Goal Environmental Social Economic
Goal 0 0 0 0
Environmental B 0 0 E
Social A F H D
Economic C 0 G I

- 13 -
Empirical study: We conducted a broad survey to collect data for the pairwise comparisons of the metrics.

The survey involved manufacturing companies in the UK and France with more than 100 employees in the

following sectors: aerospace, automotive, chemicals, electronics, steel and shipbuilding. We developed an

online questionnaire of the selected metrics using the software qualtrics©. We used stratified sampling

from the national databases of purchasing professionals: The Chartered Institute of Procurement and

Supply (CIPS) in the UK, and the Conseil National des Achats (CNA) in France. We filtered the original

list of professionals according to the following criteria:

• Professionals with career experience of at least 15 years as supply chain and procurement managers.
• Using their profiles, evidence of experience in supplier selection and supplier development processes.
• There was no filter in terms of race, sex, religion, or specific age groups.

The respondents were given the list of the selected sustainability metrics (along with their definition) and a

question that asked them to rate the relative importance of each metric in their decision to evaluate the

performance of a potential supplier was posed. We sent the questionnaire to 550 senior supply chain

professionals and we received 144 complete replies.

4. Data Analysis and Results

4.1 Pairwise Comparison

The data analysis involved three main steps: pairwise comparison of the selected metrics, the formulation

of a supermatrix and calculation of final priorities (Bottero & Mondini, 2008). Pairwise comparisons are

needed when at least one element in the source cluster is connected to at least two elements in the target

cluster (Saaty, 2008). We conducted pairwise comparisons at two levels: at the cluster and element levels.

We included the value obtained from each pairwise comparison in a pairwise comparison matrix that we

then used to calculate a priority vector (weight of the metric). Subsequently, we incorporated the priority

vectors in each pairwise comparison matrix into a supermatrix. We constructed the cluster matrices by

obtaining the priority vectors that were derived from the pairwise comparison matrices at cluster levels.

The analysis of the data collected was done using the Super Decisions® software package (“Introduction

to the ANP”, n.d.), using the simple geometric mean to aggregate the data to get a consensus opinion

(Adams, 2001; Shao et al., 2016). Based on the ANP network structure, the three required pairwise

comparison matrices at the cluster level and the five required pairwise comparison matrices at the element

- 14 -
level were developed using data collected from the survey. The pairwise comparison matrices with priority

vectors and the respective consistency ratios are shown in Table 5. We conducted consistency tests that

yielded consistency ratios of less than 0.10; this confirms that our data are consistent. The last columns of

each pairwise comparison matrix are the priority vectors for each element in the pairwise comparison

matrices. The priority vectors derived from these pairwise comparison matrices form the elements that

were included in the supermatrix.

Table 5 – Pairwise comparison Matrices

Pairwise comparison matrices at the cluster level
With respect to Goal En So Ec Priority vector
Environmental (En) 1.0000 1.4114 1.1336 0.3863
Social (So) 0.7085 1.0000 0.8314 0.2769
Economic (Ec) 0.8821 1.2028 1.0000 0.3369
Consistency Ratio = 0.0001

With respect to Social aspect En So Ec Priority vector

Social (So) 1.0000 0.8314 0.3107
Economic (Ec) 1.2028 1.0000 0.1744
Consistency Ratio 0.0000

With respect to Economic aspect En So Ec Priority vector

Environmental (En) 1.0000 1.4114 1.1336 0.3863
Social (So) 0.7085 1.0000 0.8314 0.2769
Economic (Ec) 0.8821 1.2028 1.0000 0.3369
Consistency Ratio = 0.0001
Pairwise comparison matrices at the element level
With respect to Goal En En2 En2 En3 Priority vector
En1-Greenhouse gas emissions 1.0000 0.9119 1.0214 0.8796 0.2371
En2-Energy consumption 1.0966 1.0000 1.3258 1.1319 0.2822
En3-Water consumption 0.9790 0.7543 1.0000 0.9116 0.2256
En4-The amount of waste generated 1.1369 0.8835 1.0970 1.0000 0.2551
Consistency Ratio = 0.0014

With respect to Goal So1 So2 So3 So4 Priority vector

So1-Social Investment in community 1.0000 0.7207 0.4155 1.1199 0.1693
So2- Customer/Community complaints 1.3875 1.0000 0.4089 1.7797 0.2231
So3-Health and safety incident rate 2.4067 2.4456 1.0000 3.3466 0.4694
So4-Average number of hours of employee training 0.8929 0.5619 0.2988 1.0000 0.1382
Consistency Ratio = 0.0059

With respect to Goal Ec1 Ec2 Ec3 Ec4 Priority vector

Ec1-Productivity or turnover per employee 1.0000 0.8844 0.7447 1.0438 0.2263
Ec2-Return on equity 1.1307 1.0000 1.1005 1.2387 0.2770
Ec3-Economic value added 1.3428 0.9087 1.0000 1.2877 0.2783
Ec4-Investment in sustainable processes and products 0.9580 0.8073 0.7766 1.0000 0.2184
Consistency Ratio = 0.0023

With respect to Ec4 En1 En2 En3 En4 Priority vector

En1-Greenhouse gas emissions 1.0000 0.9119 1.0214 0.8796 0.2371
En2-Energy consumption 1.0966 1.0000 1.3258 1.1319 0.2822
En3-Water consumption 0.9790 0.7543 1.0000 0.9116 0.2256
En4-The amount of waste generated 1.1369 0.8835 1.0970 1.0000 0.2551
Consistency Ratio = 0.0014

- 15 -
With respect to Ec4 Ec1 Ec2 Ec3 Priority vector
Ec1-Productivity or turnover per employee 1.0000 0.8844 0.7447 0.2886
Ec2-Return on equity 1.1307 1.0000 1.1005 0.3568
Ec3-Economic value added 1.3428 0.9087 1.0000 0.3545
Consistency Ratio = 0.0077

The priority vectors derived from the pairwise comparison matrices at the cluster level were used to form

the cluster matrix in Table 6. The cluster matrix was used to develop the weighted supermatrix. According

to Saaty (2008), pairwise comparisons must be conducted if at least one element in the cluster is connected

to at least two elements in the target cluster. It can be seen that some priority vectors in the cluster matrix

are given the value 1 because the influencing element is only connected to one element. For example, So1

is directly connected only to So2; therefore, its priority vector has a value of 1, while the priority vectors

for others are obtained from the pairwise comparison between elements with regard to the controlling

criterion that is at the top of the column of each table.

Table 6:
The cluster matrix for the ANP model
Goal Environmental Social Economic
Goal 0.0000 0.0000 0.0000 0.0000
Environmental 0.3863 0.0000 0.0000 0.3863
Social 0.2769 1.0000 0.4540 0.2769
Economic 0.3369 0.0000 0.5460 0.3369

The final step of the ANP method is to obtain a priority ranking for each element in the network structure.

In the weighted supermatrix, there is only the direct influence of one element (metric) on any other

element. However, if an element a directly influences element b, then it can also indirectly influence a

third element c that is directly influenced by element b. Therefore, we transformed the weighted

supermatrix obtained from the previous step into a limiting supermatrix by raising itself to a limiting

power in order to converge and to obtain a long-term, stable set of weights, which is the global priority

vector for each element (Abastante et al., 2011). The reason for this calculation is to capture the

transmission of all influence paths within the network (Peykarjou & Safavi, 2015).

In Table 7 we show the detailed limited supermatrix and the final priority ranking of the selected metrics.

We also provide a column showing the goal for the weighted and unweighted supermatrices. The

priority ranking of each metric for the objective of evaluating the most important metric for sustainability

- 16 -
performance is shown in the “Limiting Supermatrix Goal” column and the column “Priority vector

normalized by cluster” shows the local weight within each cluster.

- 17 -
Table 7.
Unweighted, Weighted and Limiting Supermatrices
Unweighted Weighted Limiting En So Ec
Supermatrix Supermatrix Supermatrix Priority
Goal Goal Goal En1 En2 En3 En4 So1 So2 So3 So4 Ec1 Ec2 Ec3 Ec4 Vector
Rank
Goal Normalised
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 by cluster
En1 -
Greenhouse 0.2371 0.0916 0.0503 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0577 0.2371 9
gas emissions
En2 - Energy
Consumption
0.2822 0.1090 0.0599 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0687 0.2822 4
En
En3 - Water
consumption
0.2256 0.0871 0.0479 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0549 0.2256 10
En4 - Amount
of waste 0.2551 0.0986 0.0542 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0621 0.2551 5
generated
So1 - Social
Investment in 0.1693 0.0469 0.1822 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2610 0.5000 0.5000 0.5000 0.2123 0.3077 2
community
So2 –
Customer /
Community
0.2231 0.0618 0.3175 1.0000 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.2610 0.5000 0.5000 0.5000 0.3321 0.5361 1
Complaints
So So3 - Health
0.1300
and safety 0.4694 0.0735 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2170 0.0000 0.0000 0.0000 0.0000 0.1241 3
incident rate
So4 - Average
number of
hours of 0.1382 0.0383 0.0191 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0322 12
employee
training
Ec1 -
Productivity
or turnover
0.2263 0.0762 0.0534 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2610 0.0000 0.0000 0.0000 0.0613 0.2730 6
per employee
Ec2 - Return
of equity
0.2770 0.0933 0.0526 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0757 0.2692 8
Ec Ec3 -
Economic 0.2784 0.0938 0.0528 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0753 0.2702 7
value added
Ec4 -
Investment in 0.0736
sust. process /
0.2184 0.0367 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1877 11
products
Final priority ranking of each metric 9 4 10 5 2 1 3 12 6 8 7 11

- 18 -
Figure 4 shows the inner and outer dependencies between the metrics in the ANP model. It can be seen

that most of the influence paths within the network, directly or indirectly point to customer/community

complaints (So2) and social investment in community (So1). In addition, the metric customer/community

complaints is the last destination of the influence path.

Figure 4: The outer and inner dependency in the network of ANP

5. Discussion of results
The empirical study was conducted in several manufacturing sectors and the results suggest that social

sustainability metrics are perceived to have a relatively higher priority in the selection of sustainable

suppliers. Similarly, for suppliers, efficiency, economic value-added and greenhouse gas emissions are

also critical areas. We should emphasize that we would expect that the results of the prioritization will be

dependent on the opinions of the professionals that participate the decision-making process for the

selection of sustainable supplier in terms of their pairwise comparisons. Similarly, we would expect the

ranking to change, if a different set of criteria is selected. However, the main focus of the study is not to

come up with an authoritative ranking of sustainability metrics but to generate a novel approach of

selecting sustainable suppliers. Our proposed approach has certain theoretical, managerial, and policy-

making implications that we discuss in this section.

From a theoretical point of view, our study moves beyond the normative prescription that a company

would typically seek excellence on behalf of its suppliers across all the three dimensions of sustainability.

Our results show that the cost and resource constraints that many companies face, forces decision-makers

to consider dimensions with short term effects, such as assessing and dealing with potential risks of poor

- 19 -
social responsibility performance of suppliers as well as short term economic dimensions such as turnover

per employee or the operational costs associated with waste generation.

From a methodological point of view, we consider that the dependencies among the various sustainability

metrics are pivotal in determining the choice of sustainable suppliers and propose the utilization of the

ANP, a well-established decision-making method of MCDM to take this into account. We posit that the

evaluation of sustainable performance should take into account the dependencies amongst several

sustainability performance indicators. The literature is rife of studies of sustainable supplier selection and

sustainable performance evaluation. However, the majority of the studies either evaluate the criteria

separately or use either a limiting, vague and in some cases arbitrary sustainability metrics without a

comprehensive objective. Our study attempts to close this gap in the literature by proposing an analytical

evaluation method for weighting sustainability metrics that are widely used across several industries and

are easily obtainable and quantified. As a result of the proposed method, the weights and prioritization for

each sustainability metric are calculated in a systematic and straightforward way.

Our study also has several managerial implications. Many companies and policy-makers often perceive

sustainability and sustainable supplier selection in defined silos. The environment in one box, child labour,

or racial injustice in another, economic growth or health are in different compartments. And there are

many stakeholders within and outside companies, each competing with one another for resources, leverage

in decision-making and priorities. This compartmentalisation generates a culture of inaction for supply

chain sustainability and prevents decision-makers from exploring and identifying dependencies between

sustainability issues. For instance, very often, companies that are active in reducing greenhouse gas

emissions, are lagging in social investment, even though the effects of climate change will primarily affect

communities with low income. As a result, their selection of suppliers neglects these issues.

To overcome this division and the lack of a holistic approach to sustainable suppliers’ evaluation, our

proposed method can assist decision-makers in appreciating which aspect or sustainability they should

focus on in the selection of appropriate suppliers; as it provides clear insight of suppliers’ sustainability

performance with quantitative analysis. Its main contribution is that it is based on authoritative experts’

support using a broad spectrum of standardized, easily quantifiable and measurable sustainability-related

metrics, thereby proving a clear weighting factor for each metric.

- 20 -
From an internal auditing point of view, a company can also use the method to compare at which metrics

it performs better than others. By calculating the perceived importance of each sustainability-related

metric derived from the proposed model, the company’s sustainability performance can be monitored and

managers can identify the critical operations or processes related to the performance metrics that need to

be improved to achieve the desired level of performance. As the notion of sustainable supply chain

management is concerned with ensuring that every stage and activity in the supply chain contributes to a

positive impact on society, the environment and the economy, companies can align their strategies and

operations, such as evaluating the sustainable performance of suppliers, based on the priority of their

sustainability performance metrics. As such, the proposed method can be used to aggregate multiple

decisions in cases when group decision-making is required.

The results of the study are important for policy-makers for sustainable development that can design

legislations and industrial policies based on issues that are important to companies. The comprehensive list

of metrics can help companies to improve competitiveness and supply chains to improve their sustainable

performance.

6. Conclusion
The growing trend for global sourcing and offshoring has increased the complexity of the selection of the

most appropriate partners. Parkouhi et al. (2019) argue that the selection of suppliers in turbulent

environments is one of the most challenging and complex tasks. Short-term, cost-focused sourcing and

development of arm’s length relationships are no longer considered an effective option. There is a growing

pressure for vendors in the developed world to focus on social as well as environmental and financial

factors in their selection of suppliers and for companies to opt for holistic approaches to suppliers’

evaluation that incorporate several dimensions (operational, financial, organizational) (Kraft et al., 2018).

Our study contributes to the existing literature by introducing a novel decision-making evaluation

framework for selecting sustainable suppliers using the ANP method. Through a comprehensive literature

review and interviews with sustainability experts, we incorporated salient environmental, social and

economic performance metrics in a framework that are quantifiable, easy to obtain and can be used across

a variety of different industries. Contrary to fuzzy methods that use criteria that are difficult to assess, our

- 21 -
model presents a clear method of well-established quantitative decision-making process, which can be

used to shape an evaluation platform for the sustainability performance of potential suppliers. It can also

be used as a diagnostic tool for a firm to evaluate and align its sustainability performance with its supply

chain and corporate strategy.

The application of this method is bound to some implementation challenges. Firstly, it is difficult to

determine with mathematical accuracy the total appropriate number of metrics, as too many measures will

result in heavy work, but too few may not fit well. Also, data against these metrics may not be available,

in which case projections need to be made. Our selected metrics may cover different areas, which requires

a lot of background knowledge to carry out the method accurately. The time to collect data and analyses

information may be too long, and thereby it may be too late to realize what should be changed in measures

or implementation.

Limitations of the study and future research

As with any proposed model, our model also has some limitations. The implementation of ANP requires

pairwise group comparisons. As the ideal minimum and maximum acceptable values for each metric are

challenging to measure or obtain, the utilization of the fuzzy numbers is limited in this model. The

selected method also represents a subjective bias of a researcher/decision maker. There is always going to

be a bounded rationality bias in this respect. Finally, for some metrics, there is an inherent difficulty in

deciding where and how to measure and apply them. Due to value-laden statements, place and time

dependency, the results for societal metrics cannot be assured to be as objective. As some metrics are

tracked by humans, calculations may not be always correct and provided on time (Lee and Amaril, 2002).

These challenges and limitations can be potentially overcome, with the following suggestions for future

research. As the data for the empirical study were collected from a variety of manufacturing companies,

further research could be conducted by studying specific companies in order to incorporate the influence

of certain selected suppliers (model alternatives). Finally, further research could use our proposed method

as the starting point of developing a sustainable supply chain performance measurement system by

integrating the ANP with an aggregation method such as a TOPSIS, the multi-attribute utility (MAUT), or

the ELECTRE methods.

- 22 -
References
Adams, W., (2001). Super Decisions Limit Matrix Calculations, Creative Decisions Foundation, White Paper, pp.
1–22 http://www.creativedecisions.net/papers/papers_etc/calc-white-paper.pdf
Ahi, P., & Searcy, C. (2015). An analysis of metrics used to measure performance in green and sustainable
supply chains. Journal of Cleaner Production, 86, pp.360-377.
Amindoust, A., Ahmed, S., Saghafinia, A., & Bahreininejad, A. (2012). Sustainable supplier selection: A ranking
model based on fuzzy inference system. Applied soft computing, 12(6), 1668-1677.
Awasthi, A., Govindan, K., & Gold, S. (2018). Multi-tier sustainable global supplier selection using a fuzzy
AHP-VIKOR based approach. International Journal of Production Economics, 195, 106-117.
Azadnia, A. H., Saman, M. Z. M., & Wong, K. Y. (2015). Sustainable supplier selection and order lot-sizing: an
integrated multi-objective decision-making process. International Journal of Production Research, 53(2), 383-408
Bai, C. & Sarkis, J. (2010). Integrating sustainability into supplier selection with grey system and rough set
methodologies. International Journal of Production Economics, Volume 124, pp. 252-264.
Bai, C., Kusi-Sarpong, S., Badri Ahmadi, H., & Sarkis, J. (2019). Social sustainable supplier evaluation and
selection: a group decision-support approach. International Journal of Production Research, pp. 1-22. DOI:
10.1080/00207543.2019.1574042.
Baxter, J., Gram-Hanssen, I., Askham, C., Hansen, I. J. B., & Rubach, S. (2018). Exploring sustainability metrics
for redesigned consumer products. Journal of Cleaner Production, 190, pp.128-136.
Bottero, M. & Mondini, G. (2008). An appraisal of analytic network process and its role in sustainability
assessment in Northern Italy. Management of Environmental Quality: An International Journal, 19(2), pp. 642-660.
Bhutta, K. S., & Huq, F. (2002). Supplier selection problem: a comparison of the total cost of ownership and
analytic hierarchy process approaches. Supply Chain Management: an international Journal, 7(3), 126-135.
Büyüközkan, G., & Çifçi, G. (2011). A novel fuzzy multi-criteria decision framework for sustainable supplier
selection with incomplete information. Computers in industry, 62(2), 164-174.
Carter, C. R. & Rogers, D. S. (2008). A framework of sustainable supply chain management: moving toward
new theory. International Journal of Physical Distribution & Logistics Management, 38(5), pp. 360-387.
Chai, J., Liu, J. N., & Ngai, E. W. (2013). Application of decision-making techniques in supplier selection: A
systematic review of literature. Expert Systems with Applications, 40(10), pp. 3872-3885.
Csutora, R., & Buckley, J. J. (2001). Fuzzy hierarchical analysis: The Lambda-Max method. Fuzzy sets and
Systems, 120(2), 181-195.
De Boer, L., Labro, E. and Morlacchi, P. (2001). A review of methods supporting supplier selection. European
Journal of Purchasing and Supply Management, 7 (2), pp. 75–89.
Dubey, R., Gunasekaran, A., Childe, S. J., Papadopoulos, T., Hazen, B., Giannakis, M., & Roubaud, D. (2017).
Examining the effect of external pressures and organizational culture on shaping performance measurement systems
(PMS) for sustainability benchmarking: Some empirical findings. International Journal of Production
Economics, 193, pp. 63-76.
dos Santos, B. M., Godoy, L. P., & Campos, L. M. (2019). Performance evaluation of green suppliers using
entropy-TOPSIS-F. Journal of Cleaner Production, 207, pp. 498-509.
Ehrgott, M., Reimann, F., Kaufmann, L., & Carter, C. R. (2011). Social sustainability in selecting emerging
economy suppliers. Journal of business ethics, 98(1), 99-119.
Epstein, M. J. & Roy, M. J. (2001). Sustainability in action: Identifying and measuring the key performance
drivers. Long Range Planning, 34, pp. 585-604.
Erol, I., Cakar, N., Erel, D. & Sari, R. (2009). Sustainability in the Turkish retailing industry. Sustainable
Development, 17, pp. 49-67.

- 23 -
 Erol, I., Sencer, S. & Sari, R. (2011). A new fuzzy multi-criteria framework for measuring sustainability
performance of a supply chain. Ecological Economics, 70, pp. 1088-1100.
Ferrero, A., & Salicone, S. (2004). The random-fuzzy variables: A new approach to the expression of uncertainty
in measurement. IEEE Transactions on Instrumentation and Measurement, 53(5), 1370-1377.
Global Reporting Initiative, 2015. Sustainability Reporting. Available at: www.globalreporting.org [Accessed 15
05 2016].
Govindan, K., Khodaverdi, R. & Jafarian, A. (2013). A fuzzy multi criteria approach for measuring sustainability
performance of a supplier based on triple bottom line approach. Journal of Cleaner Production, 47, pp. 345-354.
Govindan, K., Rajendran, S., Sarkis, J., & Murugesan, P. (2015). Multi criteria decision-making approaches for
green supplier evaluation and selection: a literature review. Journal of Cleaner Production, 98, 66-83.
Haeri, S. A. S., & Rezaei, J. (2019). A grey-based green supplier selection model for uncertain
environments. Journal of Cleaner Production, 221, pp. 768-784.
Hashemi, S. H., Karimi, A., & Tavana, M. (2015). An integrated green supplier selection approach with analytic
network process and improved Grey relational analysis. International Journal of Production Economics, 159, pp.
178-191.
Hasheminasab, H., Gholipour, Y., Kharrazi, M., & Streimikiene, D. (2018). A novel Metric of Sustainability for
petroleum refinery projects. Journal of Cleaner Production, 171, pp.1215-1224.
Hassini, E., Surti, C. & Searcy, C. (2012). A literature review and a case study of sustainable supply chains with
a focus on metrics. International Journal of Production Economics, 140(1), pp. 69-82.
Ho, W., Xu, X., & Dey, P. K. (2010). Multi-criteria decision-making approaches for supplier evaluation and
selection: A literature review. European Journal of Operational Research, 202(1), pp.16-24.
Hou, G., Wang, Y., & Xin, B. (2019). A coordinated strategy for sustainable supply chain management with
product sustainability, environmental effect and social reputation. Journal of Cleaner Production, 228, pp.1143-
1156.
Hubbard, G. (2009). Measuring organizational performance: Beyond the triple bottom line. Business Strategy
and the Environment, 18, pp. 177-191.
Introduction to the ANP and its Supermatrices for SuperDecisions v2.X. (n.d.). Retrieved from
https://superdecisions.com/manuals/index.php?section=2_X
Jacobs, B. W., & Singhal, V. R. (2017). The effect of the Rana Plaza disaster on shareholder wealth of retailers:
Implications for sourcing strategies and supply chain governance. Journal of Operations Management, 49, 52-66.
Kannan, D., Khodaverdi, R., Olfat, L., Jafarian, A., & Diabat, A. (2013). Integrated fuzzy multi criteria decision-
making method and multi-objective programming approach for supplier selection and order allocation in a green
supply chain. Journal of Cleaner Production, 47, 355-367.
Kellner, F., & Utz, S. (2019). Sustainability in supplier selection and order allocation: Combining integer
variables with Markowitz portfolio theory. Journal of Cleaner Production, 214, 462-474.
Kraft, T., Valdés, L., & Zheng, Y. (2018). Supply chain visibility and social responsibility: Investigating
consumers’ behaviors and motives. Manufacturing & Service Operations Management, 20(4), 617-636.
Kuo, T., Hsu, C. W., & Li, J. Y. (2015). Developing a green supplier selection model by using the DANP with
VIKOR. Sustainability, 7(2), pp.1661-1689.
Lee, A. H., Kang, H. Y., Hsu, C. F., & Hung, H. C. (2009). A green supplier selection model for high-tech
industry. Expert systems with applications, 36(4), 7917-7927
Li, J., Fang, H., & Song, W. (2019). Sustainable supplier selection based on SSCM practices: A rough cloud
TOPSIS approach. Journal of Cleaner Production.
Linton, J. D., Klassen, R. & Jayaraman, V. (2007). Sustainable supply chains: An introduction. Journal of
Operations Management, 25, pp. 1075-1082.
Lu, L. Y., Wu, C. H., & Kuo, T. C. (2007). Environmental principles applicable to green supplier evaluation by
using multi-objective decision analysis. International Journal of Production Research, 45(18-19), 4317-4331.

- 24 -
 Luthra, S., Govindan, K., Kannan, D., Mangla, S. K., & Garg, C. P. (2017). An integrated framework for
sustainable supplier selection and evaluation in supply chains. Journal of Cleaner Production, 140, 1686-1698.
Lun, L., & Leng, P. K. (2007). Does “Fuzzifying” AHP Improve the Quality of Multi-Attribute Decision
Making? Online]. Available: http://nus.edu.sg
Morioka, S. N., & de Carvalho, M. M. (2016). A systematic literature review towards a conceptual framework
for integrating sustainability performance into business. Journal of Cleaner Production, 136, 134-146.
Osiro, L., Lima-Junior, F. R., & Carpinetti, L. C. R. (2018). A group decision model based on quality function
deployment and hesitant fuzzy for selecting supply chain sustainability metrics. Journal of cleaner production, 183,
pp.964-978.
Parkouhi, S. V., Ghadikolaei, A. S., & Lajimi, H. F. (2019). Resilient supplier selection and segmentation in grey
environment. Journal of Cleaner Production, 207, pp.1123-1137.
Peykarjou, K. & Safavi, N. N. (2015). Using Analytic Network Process (ANP) in evaluation and prioritization
the barriers of credit rating insurance companies in Iran. European Online Journal of Natural and Social Sciences,
4(1), pp. 219-229.
Picazo-Tadeo, A. J., Gómez-Limón, J. a. & Reig-Martínez, E. (2011). Assessing farming eco-efficiency: A
Data Envelopment Analysis approach. Journal of Environmental Management, 92(4), pp. 1154-1164.
Pislaru, M., Herghiligiu, I., & Robu, I. B. (2019). Corporate sustainable performance assessment based on
fuzzy logic. Journal of Cleaner Production, 223, pp. 998-1013.
Price, J. M., & Sun, W. (2017). Doing good and doing bad: The impact of corporate social responsibility and
irresponsibility on firm performance. Journal of Business Research, 80, 82-97.
Rahmanita, E., Widyaningrum, V. T., Kustiyahningsih, Y., & Purnama, J. (2018, April). Model Multi Criteria
Decision-making with Fuzzy ANP Method for Performance Measurement Small Medium Enterprise (SME).
In IOP Conference Series: Materials Science and Engineering (Vol. 336, No. 1, p. 012023). IOP Publishing.
Rashidi, K., & Cullinane, K. (2019). A comparison of fuzzy DEA and fuzzy TOPSIS in sustainable supplier
selection: Implications for sourcing strategy. Expert Systems with Applications, 121, pp. 266-281.
Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of
Operational Research, 48, pp. 9-26.
Saaty T.L. (1996). Decision-making with dependence and feedback: the analytic network process. RWS
Publications, Pittsburgh
Saaty, T. L. (2004). Fundamentals of the analytic network process — Dependence and feedback in decision-
making with a single network. Journal of Systems Science and Systems Engineering, 13(2), pp. 129-157.
Saaty, T. L. (2008). The analytic network process. Iranian Journal of Operations Research, 1(1), pp. 1-27.
Saaty, T. L., & Tran, L. T. (2007). On the invalidity of fuzzifying numerical judgments in the Analytic
Hierarchy Process. Mathematical and Computer Modelling, 46(7-8), 962-975.
Saaty, T. L., & Vargas, L. G. (2006). Decision-making with the analytic network process (Vol. 282).
Springer Science+ Business Media, LLC.
Santos, M. a. D., Svensson, G. & Padin, C. (2013). Indicators of sustainable business practices: Woolworths in
South Africa. Supply Chain Management: An International Journal, 18(1), pp. 104-108.
Sarkis, J., & Dhavale, D. G. (2015). Supplier selection for sustainable operations: A triple-bottom-line approach
using a Bayesian framework. International Journal of Production Economics, 166, 177-191.
Searcy, C., Mccartney, D. & Karapetrovic, S. (2007). Sustainable Development Indicators for the Transmission
System of an Electric Utility. Corporate Social Responsibility and Environmental Management, 14, pp. 135-151.
Searcy, C., & Elkhawas, D. (2012). Corporate sustainability ratings: an investigation into how corporations use
the Dow Jones Sustainability Index. Journal of Cleaner Production, 35, 79-92.
Seuring, S., & Müller, M. (2008). From a literature review to a conceptual framework for sustainable supply
chain management. Journal of cleaner production, 16(15), 1699-1710.

- 25 -
 Shao, J., Taisch, M., & Mier, M. O. (2016). A study on a configuration model for facilitating sustainable
consumption: A case involving the automobile industry in Italy. Journal of cleaner production, 137, 507-515.
Singh, R. K., Murty, H. R., Gupta, S. K. & Dikshit, a. K. (2007). Development of composite sustainability
performance index for steel industry. Ecological Indicators, 7, pp. 565588.
Singh, R. K., Murty, H. R., Gupta, S. K. & Dikshit, a. K. (2012). An overview of sustainability assessment
methodologies. Ecological Indicators, 15(1), pp. 281-299.
Stern Stewart & Co., 2000. EVA and Strategy. [Online] Available at:
http://www.sternstewart.com.br/publicacoes/pdfs/EVA_and_strategy.pdf [Accessed 15 05 2016].
Székely, F. & Knirsch, M. (2005). Responsible leadership and corporate social responsibility: Metrics for
sustainable performance. European Management Journal, 23(6), pp. 628-647.
Veleva, V. & Ellenbecker, M. (2001). Indicators of sustainable production: framework and methodology.
Journal of Cleaner Production, Volume 9, p. 519–549.
Weber, O., Scholz, R. W. & Michalik, G. (2010). Incorporating sustainability criteria into credit risk
management. Business Strategy and the Environment, Volume 19, pp. 39-50.
Zimmer, K., Fröhling, M., & Schultmann, F. (2016). Sustainable supplier management–a review of models
supporting sustainable supplier selection, monitoring and development. International Journal of Production
Research, 54(5), 1412-1442.

- 26 -
 APPENDIX - The Definition of Selected Sustainability-related Metrics

Metrics Definitions References

Greenhouse gas Tons of CO2 equivalent emitted by an Veleva &
emissions organization per a unit of production Ellenbecker
(2001)
Energy consumption Energy used by organization’s activities per a Veleva &
Environmental

unit of production. Ellenbecker

(2001)
Water consumption Total fresh water consumed by an organization Veleva &
per a unit of production Ellenbecker
(2001)
Amount of waste generated to land, water, Veleva &
Amount of waste generated and air before recycling per a unit of Ellenbecker
production per a unit of production. (2001)

Social Share of distributed revenues provided Lodhia &

Investment in to the community such as donations of Martin (2014);
community money and products, and investment in Veleva &
community development. Ellenbecker
(2001)

Rate of customer or community complaints Lodhia &

Customer / Community regarding firm's operation on generated Martin (2014)
complaints revenue.
Social

Health and safety Rate of reportable lost workday injuries and Veleva &
incident rate illness case per total number of hour worked Ellenbecker
by all employees. (2001)

Average number of hours of employees Veleva &

Average number of
training in every field which provided by an Ellenbecker
hours of employee
organization per an employee. (2001)
training

Company's turnover or total sales per Székely &

Productivity (Turnover employee to assess sales value generated by Knirsch (2005)
per employee) an employee.

Return on equity Return or net income on shareholders' fund Santos et al.

Economic

on an annual basis. (2013)

Economic value added Net operating profit after taxes less the cost Stern Stewart
of all capital employed. & Co. (2000)

Percentage of investment in new and Singh et al.

Investment in sustainable sustainable processes and products on the (2007)
processes and products total revenues.

- 27 -
Highlights

• We develop a sustainability performance measurement framework for supplier evaluation.

• We select and classify salient sustainability performance metrics related to sustainable
supplier evaluation across all three dimensions of sustainability.
• We utilize the Analytic Network Process method to prioritize sustainability metrics and
incorporate the inter-relationships between them.
• We generate insights about aligning the assessment of the sustainability performance of
suppliers with a company’s sustainability strategic goals.
The authors declare that there is no conflict of interest.