Accepted Manuscript

Supply Chain Performance Measurement and Evaluation: A Mixed Sustaina-

bility and Resilience Approach

M.J. Ramezankhani, S. Ali Torabi, F. Vahidi

PII: S0360-8352(18)30469-8
DOI: https://doi.org/10.1016/j.cie.2018.09.054
Reference: CAIE 5439

To appear in: Computers & Industrial Engineering

Received Date: 1 June 2018

Revised Date: 16 August 2018
Accepted Date: 30 September 2018

Please cite this article as: Ramezankhani, M.J., Ali Torabi, S., Vahidi, F., Supply Chain Performance Measurement
and Evaluation: A Mixed Sustainability and Resilience Approach, Computers & Industrial Engineering (2018), doi:
https://doi.org/10.1016/j.cie.2018.09.054

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers
we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting proof before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
 Supply Chain Performance Measurement and Evaluation: A Mixed Sustainability and
Resilience Approach

M.J. Ramezankhani, S. Ali Torabi1, F. Vahidi

School of Industrial Engineering, College of Engineering, University of Tehran, Tehran, Iran

Abstract

In recent years, governments and social bodies have imposed mandates to tackle environmental
and social issues. Furthermore, responding effectively to supply chain disruptions prevented
many businesses from bankruptcy and complete shutdown. Therefore, adopting sustainability
and resilience concepts at an operational level have recently become an inevitable necessity for
business supply chains in order to survive in the tough rapid-changing competitive market
environment. This paper proposes a novel dynamic network data envelopment analysis
framework as a comprehensive performance management system to dynamically assess a supply
chain’s performance from both sustainability and resilience viewpoints over the course of time.
The proposed model is also associated with a hybrid method using Quality Function Deployment
(QFD) together with Decision Making Trial and Evaluation Laboratory (DEMATEL) to
systematically select the best sustainability and resilience factors, which are then used in the data
envelopment analysis model. The proposed framework is applied to an automotive
manufacturing sector to demonstrate its capabilities and effectiveness.

Keywords: Supply chain performance management; Sustainability; Supply chain resilience;

Dynamic network data envelopment analysis; QFD; DEMATEL.

1
Corresponding author Tel.: +98 21 61114267; fax: +98 21 88013102;

E-mail addresses: ramezankhani@ut.ac.ir (M.J. Ramezankhani); satorabi@ut.ac.ir (S. Ali Torabi);

f.vahidi@ut.ac.ir (F. Vahidi);

1
1. Introduction

Supply chain performance management (SCPM) has become an important issue for
organizations in order to pave the way to obtain and sustain competitiveness. Different
performance measurement and analysis tools have been developed in recent decades to evaluate
supply chain performance from different perspectives. Nowadays, supply chains try to find a
way to eradicate the insurmountable challenges, which are associated with their supply chains,
by implementing innovative ideas, policies and strategies. Therefore, an effective performance
measurement system is in high demand to help organizations in achieving their business goals by
monitoring the effectiveness of the deployment of their new strategies (Jayaram, Dixit, &
Motwani, 2014).

Recent global markets’ developments have made companies and organizations pay more
attention to new concepts in the realm of supply chain in a way that they label the new
approaches as critical to their survival and progress (Min & Kim, 2012). In recent years,
following the Great Recession and worldwide disturbances caused by natural disasters (e.g.
Hurricane Katrina) and rising global unrest, new concepts have been introduced in the field of
supply chain risk management in order to encounter the supply chain problems caused by these
disruptions (Schmitt & Singh, 2012). Supply chain resilience is defined as the ability of a supply
chain to recoil from a disruption in order to deliver on its promises to its customers in a
reasonable amount of time (Ponomarov & Holcomb, 2009). The ramifications caused by
disruptions could reduce a business market share and to a greater extent could lead to a business
shutdown. In order to be able to bounce back to the normal state in a reasonable amount of time,
supply chains should be designed (or redesigned) in a way to become capable of responding
swiftly and effectively to disruptions (Ponomarov & Holcomb, 2009).

Supply chain sustainability is another important topic in the field of supply chain management
which has attracted a lot of attention in the recent decades (Hassini, Surti, & Searcy, 2012).
Traditional approaches, which set goals for a supply chain, merely consider economic factors
such as reducing costs, reducing transportation time, delivering goods on time, etc. (Jiang, Sha,
Zhuge, & Wu, 2017). Contrary to the traditional approaches, market demands have changed in a
way that environmental and social performances of a firm have also become as equally important
as its economic performance from the customers’ viewpoint. Governments and administrations

2
all over the world have paid heed to calls for not sacrificing the quality of living for exploiting
natural resources and polluting the air. At the same price, green products are more welcomed
than non-green ones (Min & Kim, 2012). On the other hand, as the workforce has become more
educated than ever, respecting people and their rights, whether final customers or employees, has
become a main ingredient in the lifetime of a company. These have resulted in the development
of environmental and social goals to become as important as economic goals; because in the
long-term, walking down the path to improve socially and produce environment-friendly
products will certainly bolster a business economically.

Considering the fact that how application of sustainability and resilience concepts in supply
chains has influenced their performance, it is inevitable to apply these new concepts/paradigms
to evaluate a supply chain’s performance. The evaluation methods should be capable of
identifying the complex structures of a supply chain as well as assessing them thoroughly. Data
envelopment analysis (DEA), as a powerful method, suitably addresses the need for both
sweeping and partial evaluation of various supply chains (Hassini et al., 2012). It can cover both
sustainability and resilience aspects containing both qualitative and quantitative data within
complicated network structures.

Supply chains’ strategies are changing drastically and they are more concentrated on improving
their sustainability and resilience performances than ever. Evidently, the increasing trend in the
related academic publications perfectly reflects the importance of the topic. However, the
complexity associated with such concepts brings out the following questions:

- Which factors of sustainability and resilience should be considered for a comprehensive

while tractable performance evaluation?
- How is it possible to take into account the unconventional irregularities associated with
sustainability and resilience (e.g. qualitative data associated with the social aspect)?
- How is it possible to reflect the internal interactions between the supply chain’s parties in
its performance evaluation?
- How is it possible to evaluate implications of a strategy on different echelons of a supply
chain over several periods from different viewpoints (i.e. economic, social,
environmental and resilience)?

3
Questions above drove us to scour through SCPM literature but we came out almost empty-
handed because it seems that issues mentioned above have went unheeded in the literature. A
systematic framework that can cope with sustainability and resilience dimensions concurrently in
the context of SCPM is nowhere to be found in the literature. This research gap has given us a
strong motivation to develop a new and general performance measurement and analysis
framework to comprehensively assess the performance of a supply chain and compare its
efficiency with its peers by devising a new DEA model. In Section 2.4, we elaborate on the
research gaps, our motivation and our contributions in this study to address those gaps. Overall,
all the contributions in the present paper are briefly listed below.

- For the first time in the context of SCPM, a new dynamic network DEA (DNDEA) model
is presented to evaluate the performance of a supply chain and comparing its efficiency
with its peers by taking into account the sustainability and resilience factors concurrently.
This DNDEA model is a general approach, which is able to deal with modeling
complexities such as undesirable factors and data irregularities including interval data,
negative data and qualitative data.
- A new QFD-DEMATEL method is developed to determine the most influential criteria to
be used as input/output factors in the DNDEA model. To the best of authors’ knowledge,
such systematic factor selection is an unprecedented practice for the application of DEA
models in the literature of SCPM.
- For the first time, a DNDEA model is applied for performance evaluation in the
automotive industry while a comprehensive set of different performance indexes
including economic, social, environmental and resilience ones are used for this
assessment.

The rest of this paper is organized as follows. Section 2 reviews the relevant literature. Section 3
proposes a new efficiency evaluation framework including a hybrid QFD-DEMATEL method
for factor selection and a DNDEA model for performance evaluation. Section 4 provides a case
study in the automotive industry. Section 5 suggests several managerial implications extracted
from the case study’s numerical results. Finally, Section 6 presents the conclusions.

2. Literature Review

4
A supply chain comprises all the entities which play a role in fulfilling customers’ orders. So,
supply chain management means controlling all the activities, resources, information and capital
in order to raise the profitability of a business supply chain (Hassini et al., 2012). Aside from
profitability, sustainable business directs an organization to set long-term economic,
environmental and social goals. Moreover, a resilient supply chain would be more resistant to
potential risks. The literature is reviewed in three relevant streams, which include sustainable
supply chains, resilient supply chains and dynamic network data envelopment analysis.

2.1. Sustainable Supply Chains

Sustainable supply chain management is defined as managing the activities, resources,
information and capital related to a supply chain which aims at maximizing the supply chain’s
profitability and at the same time minimizing the environmental effects as well as raising the
social welfare (Hassini et al., 2012). Numerous studies have addressed the supply chain
performance evaluation problem from the sustainability viewpoint. Chang et al. (2013)
introduced a combinational criterion to measure companies’ sustainability performance. The
criterion, in fact, measures the efficiency changes over a three-year period based on a Malmquist
index. In addition to the applied social and environmental criteria, the sustainable approach to the
economic criteria is one of the main points of their research. In another research, a DEA model is
introduced for measuring the efficiency of energy consumption in Iran’s herbal oil industry in
which the environmental criteria are applied to assess the efficiency of industrial units (Nouri et
al., 2013). Moreover, a network data envelopment analysis model is developed to take into
account the intermediate relations in a four-level supply chain while a number of social and
environmental criteria are used as the sustainability criteria (Tajbakhsh & Hassini, 2015). In
addition to a multi-stage network model for evaluation of units’ efficiency, it also introduces a
centralized model which estimates the efficiency of each decision-making unit (DMU) requiring
less calculations. In another study that focuses on the three aspects of sustainability, the
performance of a resin producing supply chain with imprecise input and output data is studied by
Azadi et al. (2015) using a fuzzy DEA model enhanced with Russel measure. A set of
sustainability criteria extracted from the literature are shown in Table 1.

5
 2.2. Resilient Supply Chains
Over the past three decades, risk management has been under sweeping scrutiny in a way that
over fifty different frameworks and tools have been presented for evaluating and managing risks
(Cha, Juo, Liu, & Chen, 2008). Supply chain risks are typically classified into the operational
risks and disruption risks (e.g. see Tang (2006) and Sawik (2011)). Operational risks (i.e.
originating from business-as-usual events) are the inherent uncertainties stemming from the
difficulties existing in the coordination of supply and demand sides (e.g. uncertainty caused by
fluctuations in the customers’ order, resources and costs). These kinds of risks are captured by
considering the uncertainty in key input data usually in the form of random or/and fuzzy
numbers (Sawik, 2011). On the other hand, disruption risks are the major disturbances occurred
during the ordinary procedures of a supply chain. They might happen because of natural disasters
(flood, hurricane, and earthquake, etc.), human-related incidents (terrorist attacks, strikes, etc.)
and technical glitches (e.g. disorder of production equipment or disruptions in information
systems, and so on). In recent years, global disturbing incidents have increased. Tsunamis,
earthquakes, and abrupt disasters such as oil spill in Mexican Gulf 2010 and back-to-back
recessions are among them (Schmitt & Singh, 2012). Disturbances drastically affect any supply
chain since they impact the flow of information, substances and money in a supply chain;
therefore, hinder supply chains’ regular operations which reduce their efficiency and
competitiveness. Moreover, such disruptions have certain characteristics which makes it difficult
to predict the time and intensity of their eruption. Therefore, in order to make a supply chain
resistant in face of disruptions, it is imperative to understand the supply chain network,
comprehensively (Schmitt & Singh, 2012).

Although disruption risks might occur in any stage of a supply chain (i.e. raw materials and parts
procurement, storage, production, distribution and transportation) and blockade it at any random
time resulting in fatal economic and social impact, there has been very few studies conducted on
the subject. It is noteworthy that considerable delays in service/goods delivery to customers,
stock deficit, losing market share and personnel layoff are some of the ruinous impacts of supply
chain disturbances (Peng, Snyder, Lim, & Liu, 2011).
One of the most efficient and powerful methods to deal with disruption risks is designing
resilient networks. Supply chain resilience has been defined in various ways. Christopher and

6
Peck (2004) defined the resilience as the ability of a system or structure to return to its initial
state (or even a better state) after a disruption.
Recently, some extensive studies have been conducted on supply chain disruptions and the
concept of resilience. In order to evaluate the greenness and resilience of automakers’ supply
chains, a multi-criteria model was presented to make up a composite index named as GResilient
(Azevedo, Govindan, Carvalho, & Cruz-Machado, 2011). In another study, a three-stage supply
chain corresponding to a real supply chain is examined and simulated along with six supply
chain scenarios (Carvalho, Barroso, Machado, Azevedo, & Cruz-Machado, 2012). In order to
evaluate the six defined scenarios from the resilience viewpoint, the total cost and delivery time
criteria are used for each entity of the simulation study. In another research, it is claimed that
supply chain resilience can be enhanced by inclusion of various uncertainty factors such as
goods’ demands, machine availability time and different costs (Mitra, Gudi, Patwardhan, &
Sardar, 2009). A multi-product and multi-depot supply chain is formulated over a multi-period
horizon to increase its resilience. Moreover, the uncertainty effect of each factor on the cost
function is analyzed by applying a fuzzy approach.
In many of the studies on supply chain resilience risk management and resilience are
strategically viewed. Nevertheless, form the operational viewpoint, a handful of researches
include operational factors in their studies (see Table 1). Accordingly, this paper tries to
concentrate more on operational factors of supply chain resilience as they are measured easier
than strategic ones and when it comes to a yearly evaluation of some similarly-structured supply
chains, such assessment using operational factors is more sensible than using strategic factors
which usually have a long-term impact and cannot be modified on a yearly basis.

Table 1 – Sustainability and Resilience Factors

Factor Type Reference
Chaabane et. al (2011); Büyüközkan & Berkol
Operational cost Economic
(2011)

Transportation cost Economic Ahi & Searcy (2015)

Ordering cost Economic Varsei et al. (2014); Govindan et al. (2013)

Defective rate Economic Dai & Blackhurst (2012)

Environmental- Tseng et al. (2014); Hassini et al. (2012)
Minimum energy consumption
Economic
Environmental- Tseng et al. (2014)
Minimum (raw) material usage
Economic

7
 Büyüközkan & Berkol (2011); Boukherroub et. al
Environmental-
Water usage
Economic (2015)
Capability of using green
Environmental- Azadnia et al. (2015)
technologies Economic
Azadnia et. al (2015); Dai & Blackhurst (2012);
Environmental-
Technology level
Economic Wilhelm et al. (2016)
Environmental- Haldar et al. (2012); Azadnia et al. (2015)
Capability of R&D
Economic-Social
Pollution production Environmental Hassini et al. (2012); Wilhelm et al. (2016)
Büyüközkan & Berkol (2011); Boukherroub et al.
Usage of toxic substances Environmental
(2015)
Environmental management Varsei et al. (2014); Azadnia et al. (2015);
Environmental
systems Nakamichi et al. (2016)
Chaabane et al. (2011); Büyüközkan & Berkol
Reduction of solid waste Environmental
(2011)
Dai & Blackhurst (2012); Ghadimi & Heavey
Customer satisfaction Social
(2014)

ISO 14001 certification Social Büyüközkan & Berkol (2011); Azadnia et al. (2015)

Job security Social D. Kannan et al. (2014)

Occupational injury and illness Social G. Kannan et al. (2009)

Training education and Community
Social Varsei et al. (2014)
Development
Azadi et al. (2015); Varsei et al. (2014); Türkay et
Worker safety and labour health Social
al. (2016)

Job opportunity Social Hsu et al. (2011)

Funding special projects (school,
Social Chaabane et al. (2011)
hospital, etc.)
Economic- Büyüközkan & Berkol (2011)
Delivery lead time
Resilience
Institutional distance between the
Resilience Amindoust et al. (2012)
buying firm and its supply base

Buffer capacity Resilience Bairamzadeh et al. (2015)

It is worth mentioning that there are some well-known network-based resilience factors including
node complexity, network complexity, flow complexity, density, node criticality, redundancy
and network centralization but they will be meaningful when different network structures are
addressed (see for instance Kim et. al (2011) and Cardoso et. al (2015)); which is not the case in
this paper. In this paper, performance evaluation of several supply chains with similar structure

8
are taken into account. Notably, when it comes to using DEA as a performance management
tool, it is essential for the DMUs under study (in our case, a number of supply chains), to have
similar structures. Aforementioned strategic resilience factors only change when there is a
change in the network’s structure.

2.3. Dynamic Network Data Envelopment Analysis

Recently, the subject of developing new tools or adopting extant ones for measuring supply
chain efficiency has drawn many researchers’ attention (You & Jie, 2016). Any tool for
evaluation of a supply chain’s efficiency needs to consider the multi-dimensional and multi-
echelon structure of the supply chain under study to suitably calculate the efficiency of every
unit (i.e. each supply chain member). The data envelopment analysis (DEA) method is regarded
as a decision-making tool with several applications in the area of supply chain management (e.g.
to evaluate the relative efficiency of some DMUs or peer units and estimating the weight vector
of some criteria). When using a DEA model as an efficiency measurement and benchmarking
tool (which is its main application), each decision-making unit (DMU) has a set of processes by
which a set of outputs are produced using a set of inputs.

Supply chains as decision-making units might have complicated relations. That is, a supply chain
is made up of several echelons in a way that the output of some echelons is the input of another.
Therefore, all the echelons consist a unified network. Network DEA (NDEA) models are used
for evaluating the efficiency of such decision-making units and supply chains (Kao, 2014).
Various network DEA models and applications have also been proposed to assess the internal
relations among the members of a supply chain or a number of homogeneous/comparable supply
chains from the efficiency perspective. For instance, in a research, DEA is applied to evaluate the
efficiency of distributors of a supply chain (Wu & Olson, 2008). Färe et al. (2007) proposed the
first NDEA model. Since then, hundreds of papers covered the network data envelopment
analysis topic (Kao, 2014). Some NDEA models measure the efficiency of the whole network
and its internal divisions under certain conditions, and some other concentrate on solving the
real-world problems. Cook et al. (2010) studied some of the two-stage networks in which one
DMU consists of two divisions. In one of the important studies, Tone and Tsutsui (2009)
introduced a slacks-based method (SBM) model considering a network structure that is the basis
for the present research. They presented their model using an electricity supply network. An in-

9
depth review on Network DEA is provided, which broke ground on many new researches that
cover an extensive range of subjects (Kao, 2014). The performance of China’s electricity supply
network is evaluated by a model similar to that of enhanced Russel measure in which a detailed
performance analysis of power plants and electricity supply networks is provided (You & Jie,
2016). DMUs were also classified according to the geographical location, and three performance
measurement indexes are also introduced for better evaluation which distinguishes this research
from others. Furthermore, a model is introduced based on SBM model which consists of
inefficiencies on different sections of a network (Lozano, 2016). The paper studies all the
networks and their topologies used in the evaluation of banks in various researches. In the
proposed model, undesirable inputs/outputs in the banking industry are taken into account as
well.

Performance evaluation of a supply chain over the course of time has always been very
important. The reason is that merely identifying a solution to boost the performance of a supply
chain at a specific time period is not enough for enhancing its performance at all. In other words,
the positive/negative impacts of any decision are clarified in the course of time. Therefore, the
continuous performance evaluation of a supply chain is critical to guarantee performance
enhancement over time. Accordingly, a number of researchers have recently applied dynamic
DEA models along with transition models to move from one period of time to another. The book
published by Fare and Grosskopf (2012) was the first one exploring the basic concepts and
methods in dynamic DEA models.

The close relations between NDEA models and dynamic DEA ones have attracted the attention
of many researchers to work on developing dynamic network DEA models leading to more
comprehensive and practical DEA models in the context of SCM. In this regard, Tone and
Tsutsui (2010) were among those pioneers who developed a dynamic DEA model using a SBM
method. Then, they published their paper on dynamic and network DEA (Tone & Tsutsui, 2014).
Afterwards, a centralized model was introduced in which a method for weighing the valuation of
Australian universities’ performances over a seven-year period is introduced and the results are
compared to a non-centralized model (Lee & Worthington, 2016). An envelopment model
emphasizing on inefficiency ratio is also designed (Yu, Chen, & Hsiao, 2016). Moreover, several
factors such as undesirable outputs and shared inputs are applied to evaluate the efficiency of bus

10
transformation companies in a nine-year time horizon. To evaluate Chinese banks’ performance,
a two-stage framework based on SBM model is developed to assess Chinese banks’ efficiency
over a five-year horizon (Zha, Liang, Wu, & Bian, 2016).

Some studies account for data irregularities (such as undesirable inputs/outputs) along with the
network structure. One of these studies which included the undesirable factors in a network DEA
model, discussed a multi-stage system and presented a multiplier model to reduce the undesirable
outputs and increase the favorable ones (Kordrostami & Amirteimoori, 2005). It also included
undesirable inputs together with the normal inputs in order to calculate the permissible inputs. In
another study, a directional distance function was used for a two-stage system to evaluate the
efficiency of airports in which delayed flights were taken into account as undesirable outputs
(Lozano, Gutiérrez, & Moreno, 2013). Motevali Haghighi et al. (2016) used the concept of
Balanced Scorecard Card (BSC) and combined it with a network multiplier model and
introduced a unique DEA model. Their model uses undesirable factors along with qualitative
data. They also presented an application of the model in a supply chain of recycling plastics.
Valuable studies of Tajbakhsh and Hassini (2015), and Motevali Haghighi et al. (2016) are
among the most important research works which are conducted in the field of sustainable supply
chain evaluation using the network DEA approach. In another study, various kinds of sustainable
factors are presented and a model is designed to discuss different conditions of return to scale
(Faramarzi, Khodakarami, Shabani, Farzipoor Saen, & Azad, 2015). The authors applied their
model to evaluate the efficiency of Iranian combined fuel power plants. Song et al. (2014) used a
performance evaluation criterion to assess the environmental efficiency of China’s provinces
regarding their sewage output.

2.4. Research Gap Analysis

The above literature review shows that in the recent years, supply chain sustainability (SCS) and
supply chain resilience (SCR) have attracted researchers’ interest in large part as supply chains’
complexity have significantly increased. The increased complexity calls for more powerful tools
to be brought in for performance measurement and evaluation. Consequently, DEA is a suitable
tool which is able to incorporate newly-emerged complexities caused by SCS and SCR. That is,
despite several valuable researches done so far, there is still no systematic framework to
dynamically evaluate a supply chain’s performance in comparison with its peers over time from

11
both the sustainability and resilience viewpoints while considering their network structures.
Details of each specific aspect of the aforementioned research gap, our motivation and our
contributions to address such research gaps are listed below:
- In the SCPM literature, researches did not come up with a framework capable of
simultaneously taking into account sustainability and resilience factors. In this paper, we
propose a new and generalized DNDEA model to address such a gap.
- In the SCPM literature, a few researchers addressed modelling complexities such as
undesirable factors, interval data, negative data and qualitative data. In those who studied
such difficulties, no comprehensive framework included all these irregularities. In this
paper, the proposed generalized DNDEA model is capable of dealing with such
difficulties.
- In the SCPM literature associated with DEA models, DEA factors were presumed to be
predefined by the researchers according to the context of their study. To the best of our
knowledge, no one proposed a framework to systematically define, select and use factors
as inputs/outputs of a DEA model. In this paper, we propose a new QFD-DEMATEL
method to select the most influential criteria to be used as factors in the proposed
DNDEA model.
- In the literature focused on performance management applications in the automotive
industry, it is unprecedented for a research to evaluate a supply chain in several
consecutive periods while taking into account the network structure of such a supply
chain. In this paper, we have set precedent and used a novel DNDEA model in a case
study in the automotive industry. We have also proposed performance indexes to assess a
supply chain’s performance from sustainability and resilience viewpoints in comparison
with its peers.
Considering the fact that we have focused on network DEA models as a suitable tool for
performance evaluation in the SCPM literature, Table 2 illustrates the differences between our
paper and the relevant ones in the literature.

Table 2 – Research gap analysis among studies focused on NDEA models in SCPM
literature

NDEA Systematic Including Including Special Qualitative

Ref. Dynamic
Model Factor resilience sustainability Factors Factors

12
 Selection factors factors (e.g.
undesirable
factors)
(Tone &
SBM ✔ ✔
Tsutsui, 2014)
(Tajbakhsh &
Envelopment ✔
Hassini, 2015)
(Motevali
Haghighi et al., Multiplier ✔ ✔ ✔
2016)
(Faramarzi et
Multiplier ✔
al., 2015)
(Mirhedayatian,
Azadi, &
SBM ✔ ✔
Farzipoor Saen,
2014)
This Paper SBM ✔ ✔ ✔ ✔ ✔ ✔

As a result, we found it incumbent upon ourselves to develop a new DNDEA model in order to
come up with a novel systematic framework for performance evaluation of peer supply chains
from sustainability and resilience viewpoints. In the next section, we present a model which is
capable of coping with both qualitative and quantitative data. We also propose a new systematic
framework to choose the best factors to be used as inputs, intermediates and outputs of the DEA
model.

3. The Proposed Efficiency Evaluation Framework

This paper proposes a new framework to concurrently assess the performance of some
competing supply chains over the course of time from a mixed sustainability and resilience view.
The suggested model is an extension of the model developed by Tone and Tsutsui (Tone &
Tsutsui, 2014). The proposed model in this paper has the capability to handle data irregularities
such as qualitative data, interval data and negative data along with desirable and undesirable
inputs/outputs/intermediates. Furthermore, in order to systematically select the most appropriate
inputs/outputs/intermediates at each stage, the proposed model suggests a hybrid QFD-
DEMATEL method, which ensures selecting the best factors to be used in the DEA model.

13
It is worth mentioning that there are several systematic factor selection procedures in other
contexts. However, our proposed hybrid QFD-DEMATEL method is a new factor selection
method, which is associated with a DNDEA model aiming at performance measurement and
evaluation of a number of peer supply chains considering their network and dynamic structure.
To the best of authors’ knowledge, such factor selection method and its integration with the
developed DNDEA model is unprecedented and novel at least in the SCPM literature. More
specifically, the most common practice regarding the factor selection is that researchers usually
use experts’ opinions to decide what factors are the best ones to be used in a DEA model or they
just use those factors that suit their research based on their own opinion. Nevertheless, this paper
provides a systematic structure to select the most suitable factors to be fed into the DEA model
in order to get more accurate results.

Accordingly, the proposed hybrid QFD-DEMATEL method is first used to choose the most
important factors from a set of influential inputs, outputs and intermediates while considering the
importance weight of each factor from sustainability and resilience viewpoints using the experts’
opinions. Then, by considering the irregularities imposed by the selected factors, the proposed
dynamic network DEA is employed in order to provide a framework for performance evaluation
of peer supply chains under study. In this way, the proposed DNDEA model provides an
efficiency score for each echelon of each peer supply chain at each period along with a reference
set for each supply chain as a whole and its echelons. It also provides a total efficiency score for
each supply chain under evaluation at each period. Based on the slacks calculated by DNDEA
model, some aggregated performance indexes are also developed (see Table 8) to show the
efficiency scores of each supply chain from sustainability and resilience points of view at each
period which can be used as key performance indicators (KPIs) to assess the evolution of a
supply chain from sustainability and resilience aspects over the course of time. For more clarity,
Fig. 1 outlines the schema of the proposed performance measurement and evaluation framework
in this paper.

{Please insert Fig. 1 around here.}

3.1. Hybrid QFD-DEMATEL method for factor selection

Typically, different quantitative and qualitative factors are used in the literature of performance
evaluation and management in different contexts. Nevertheless, in the DEA literature,

14
surprisingly there is no systematic approach to select the most important factors from a list of
candidate factors, to our knowledge. Hence, this research firstly aims to develop a systematic
approach to tackle this gap.

In general, any effort done in the area of supply chain performance management is conducted in
order to finally come up with a number of performance improvement plans. In order to walk
down the improvement path, it is of great importance for the top managers to identify those
areas, which need improvement. Experts from different departments of an organization have
different mentalities and priorities regarding performance improvement plans. For instance, one
expert might believe that improvement of some environmental aspects are more important than
those of economic factors and another expert believes vice versa (Vahidi, Torabi, &
Ramezankhani, 2018). Therefore, a systematic framework is required to choose a set of factors
that partially satisfy different experts’ viewpoints. Fig. 2 depicts a flowchart showing main steps
of the proposed hybrid QFD-DEMATEL method in four stages.

Firstly, it capitalizes on DEMATEL method to weigh sustainability and resilience criteria using
experts’ subjective opinions. This very first step itself includes four sub-steps which are
determining the relation between different criteria using paired comparisons, calculating the
average matrix, forming the initial direct influence matrix and finally constructing the full
direct/indirect influence matrix. Secondly, appealing factors have to be identified and composed
in order to be able to provide a platform for assessing a supply chain’s performance from both
sustainability and resilience points of view. Thirdly, QFD is called in to determine weights
associated with different factors with taking sustainability and resilience criteria into account.
Finally, the factors are sorted in a descending order by their weights and the most important ones
are chosen.

Due to space limitation, full explanation for each stage and the related calculations are presented
in the electronic supplementary material associated with this paper.

{Please insert Fig. 2 around here.}

3.2. A novel DNDEA model with data irregularities

Data Envelopment Analysis (DEA) was first introduced by Charnes, Rhodes and Copper (1978)
to evaluate performance efficiency in US’s agricultural sector. It is a non-parametric method

15
which is pivoted on linear programming and since its introduction in 1978 many researchers
have resorted to it in order to build up performance evaluation frameworks in different contexts
(e.g. Nouri et al. (2013), You and Jie (2016) and Lee and Worthington (2016)). The main
concept behind it is the fact that efficiency is maximized only when the ratio of weighted outputs
to weighted inputs is maximized. Many research works have studied the problem of supply chain
performance assessment in which the multi-echelon structure of concerned supply chains is
captured (Kao, 2014). As it was clarified in the literature review Section, recent publications are
indicative of the fact that performance evaluation should be associated with dynamic aspects in
order to dynamically monitor the performances and choose appropriate performance
improvement strategies and plans. Hence, it is not strange that for instance Tone and Tsutsui
(2010) just after a year from presenting their SBM-based network DEA model (Tone & Tsutsui,
2009), introduced an equivalent model with dynamic structure and consequently the combination
of these two concepts paved the way for a dynamic network SBM model to be developed later in
2014 by them (Tone & Tsutsui, 2014). Based on this inspiration, the DEA model presented in
this paper is a generalized DNDEA model which is capable of handling modelling complexities
such as undesirable factors, interval data, negative data and qualitative data. In the realm of DEA
models, such a model which can handle these complexities in a network and dynamic structure is
unparalleled.

3.2.1. Undesirable factors

In order to maximize efficiency, the ratio of outputs to inputs needs to be maximized if it is
desirable to have more produced outputs with less consumed inputs. However, this case could
not be applied if any of inputs/outputs behave in an undesirable way. For instance, if in a
production process, some scraps in addition to the final products are generated, then scrap plays
as an undesirable factor in the production process where by producing less scrap the efficiency
would increase. Based on what Seiford and Zhu (2002) proposed, the data translation method
which is presented below, supports classification invariance and can be used to deal with
undesirable factors in the model.

Inputs and outputs fall into two separate groups as it is shown in Eq. (1).

16
  X ig 
 b
 Xi   Xi 
Y    g  (1)
 r  Yr 
Yrb 

where X ig and X ib represent the desirable and undesirable inputs, respectively, while Yrg and Yrb

represent the desirable and undesirable outputs, respectively. In order to be able to cope with
undesirable factors, the following transformation is used.

b
X i   X ib  Wi
(2)
b
Y  Y  Wr
r r
b

where the undesirable output and input vectors for the jth DMU are as follows, respectively.

xijb   xijb  wi
(3)
yrjb   yrjb  wr
b
In Eq. (2), the translation vectors Wr and Wi need to be chosen in a way that ensure Y r  0 and
b
X i  0 . It is worth mentioning that different sets of translation vectors produce almost the same
results and the difference is insignificant unless the translation vector approaches infinity which
would lead to elimination of the corresponding factor. Different sets of translation vectors only
affect the hundred thousandth range of the efficiency variable which does not lead to change in
major ranking variance.

3.2.2. Interval data

In many real-world cases, quality of data is not accurate due to many different issues. For
example, when it comes to financial revenues, costs, etc. companies refuse to share their exact
data and they prefer to only put a lower bound and an upper bound to such data on the table. In
this case, the interval data needs to be transformed into a set of exact data to be put in the DEA
model (Zhu & Cook, 2007). Consider the following interval inputs and outputs:

Xi  Xi  Xi i  BI
(4)
Yr  Yr  Yr r  BO

17
where X i and Yr are the lower bounds and X i and Yr are the upper bounds. The associated sets
for interval inputs and outputs are respectively represented by BI and BO. In order to convert the
interval data into a set of exact data, the settings shown in Eq. (5) will provide the optimal
solution for efficiency evaluation of DMU o (for more information see Zhu & Cook, 2007).

 X io  X io i  BI  X ij  X ij i  BI

DMU o  , DMU j  (5)
Yro  Yro r  BO j o Yrj  Yrj r  BO


Eq. (5) is indicative of the fact that increase in outputs (and decrease in inputs) of DMU o and
decrease in outputs (and increase in inputs) of other DMUs, does not result in deterioration of the
efficiency of DMU o under evaluation (Zhu & Cook, 2007).

3.2.3. Qualitative factors

In many real situations for some factors such as management quality, it is usual to see the data
measured in ordinal scale or measured based on Likert scale. In the literature, such qualitative
data are usually converted into quantified data but this quantification most of the time takes place
due to the fact that modeling convenience calls for this conversion and it imposes some sort of
superficiality to the model (Cook & Zhu, 2006). In order to prevent this issue, Cook and Zhu
(2006) proposed a structure to embed rank order (or ordinal scaled data) or Likert scaled data
into the DEA framework which is capable of handling both radial and non-radial model
formulations.

We assume that when it comes to quantitative and qualitative factors, inputs fall into two
different categories. Let I1 and I 2 show the sets of quantitative and qualitative inputs,

respectively. It is also assumed that R1 and R2 show the sets of quantitative and qualitative
outputs, respectively. For each qualitative input/output, where there is a set of N decision making
units, DMU j can be assigned to one of L ordinal positions where L  N . Based on that, for the

   
qualitative factors, the L-dimensional vectors of  ij   ij  l  and  rj   rj  l  are defined as

follows:

1 if DMU j is ranked in l th position on input i

 ij  l    i  I2 (6)
0 otherwise

18
 1 if DMU j is ranked in l th position on output r
 rj  l    r  R2 (7)
0 otherwise
For example, if a 9-point scale is used, for the ordinal input i ( i  I 2 ), if DMU j is ranked in

l=4th place, then  ij  4   1 and  ij  l   0 for all other l rank positions. In this case, it is

translated as  ij   0,0,0,1,0,0,0,0,0 . As Cook and Zhu (2006) thoroughly discussed this

matter, for DMU j , two parameters need to be introduced in order to apply worth ordering

principle to the model and define worth vectors for inputs and outputs. These two parameters for
DMU j are as follows

L
 ij  l    ij  L    ij  L  1    ij  l    ij  n  i  I 2 (8)
n l

l
 rj  l    rj 1   rj  2     rj  l    rj  n  r  R2 (9)
n 1

where  ij  l  and  rj  l  are associated with qualitative inputs and qualitative outputs,

respectively. With the use of these parameters, qualitative factors can be embedded into the
model which will be discussed in coming sections.

3.2.4. The proposed DNDEA model

In this paper, we consider a general structure of a supply chain network whose performance is
targeted to be assessed over a pre-defined time horizon (see Fig. 3). By doing so, the proposed
model could be viewed as a generalized DNDEA model which comprises general types of links
and intermediate relations, which can be adjusted for specific cases. The proposed DNDEA
model is able to find out inefficiencies of all the echelons/units/divisions within the peer supply
chain networks under study at each period.

{Please insert Fig. 3 around here.}

It is worth mentioning that each division in Fig. 3 is the representative of an echelon in the
supply chain. For example in Fig. 3, division K-1 can be a supplier, division K can be a
manufacturer and division K+1 can be a distributor. Fig. 3 is a partial image of a supply chain’s
network in a specific period (T-1 and T). There is no limit for the number of divisions or
echelons of a supply chain. Furthermore, subdivisions like production planning and quality

19
control sections can also be included if needed in a manufacturer echelon. Also, there is no limit
for the number of performance evaluation periods. A more complex network only needs more
calculations and the needful computing resources.

There are two assumptions considered in the proposed general DNDEA model. First, it has been
assumed that all inputs and those intermediate links or carryovers that are treated as input are
discretionary. However, the possibility of treating the intermediates and carryovers as non-
discretionary data has been included in the model and the decision on how they will be treated is
at discretion of the user. Second, each division may employ its own unique inputs/outputs that
could be either quantitative or qualitative. Also, intermediate links are used between divisions in
each period and carryovers are used between two periods for each division.

It is worth mentioning that the proposed DNDEA model uses transformed data for undesirable
inputs/outputs whose transformation method was discussed in Section 3.2.1. Furthermore, any
interval quantitative input/output data has to be translated using the method discussed in Section
3.2.2. We also propose a generalized bundle of constraints in order to provide flexibility for
treating the intermediate links. Intermediate links can be formulated in four different structures
which can be viewed as input, output, free links or fixed links. More or less, it is also applied for
carryovers. Carryovers fall into four categories which are desirable, undesirable, free links or
fixed links. A desirable carryover is a link that transfers an asset (e.g. profit) and an undesirable
carryover is a link that conveys a liability (e.g. a bad debt) from a period to the next one. If a
carryover is completely at the hands of the DMU then it is discretionary and presented as a free
link. Meanwhile, if it is beyond the control of the DMU, then it is non-discretionary and
presented as a fixed link. It is also at discretion of the user who can decide how to treat
carryovers in a specific case framework. The objective is to evaluate a supply chain’s
performance in comparison with its peers, as a whole and its echelons as well. Noteworthy, all
the performance calculations are carried out over the planning horizon in order to dynamically
analyze the application of those employed performance improvement policies/action plans
scoped for each echelon’s or/and the whole chain. It is also worth mentioning that the model
provides importance weights associated with each echelon and each period which are determined
based on experts’ opinions.

In the following, the notations used for formulating the proposed DNDEA model are defined:

20
Sets:

N Set of DMUs  jN

K Set of supply chain echelons (i.e. suppliers, manufacturers, distributors, etc.)  k  K 

T Set of time periods over the considered planning horizon  t  T 

I1kg Set of quantitative and desirable inputs for division k

I1kb Set of quantitative and undesirable inputs for echelon k

I 2k Set of qualitative inputs for echelon k

R1kg Set of quantitative and desirable outputs for echelon k

R1kb Set of quantitative and undesirable outputs for echelon k

R2k Set of qualitative outputs for echelon k

Dk ,k 1 Set of intermediate links between echelons k and k  1

Ck Set of carryovers for echelon k

L Set of ordinal positions for qualitative inputs/outputs.

Parameters:
tg
xijk Quantitative and desirable input i to DMU j for echelon k in period t

xijktb Quantitative and undesirable input i to DMU j for echelon k in period t

tg
yrjk Quantitative and desirable output r to DMU j for echelon k in period t
tb
yrjk Quantitative and undesirable output r to DMU j for echelon k in period t
 ijkt  l  Qualitative parameter associated with qualitative input i for ordinal position l to DMU j

for echelon k in period t

 rjk
t
l  Qualitative parameter associated with qualitative output r for ordinal position l to
DMU j for echelon k in period t

z tj ( k ,k 1)d Intermediate link d to DMU j between echelons k and k  1 in period t

z jkt ,ct 1 Carryover c to DMU j for echelon k between periods t and t  1

wk Importance weight for echelon k

Wt Importance weight for period t

21
Decision variables:

tg 
siok Slack associated with quantitative and desirable input i to DMU o for echelon k in period
t
tb 
siok Slack associated with quantitative and undesirable input i to DMU o for echelon k in

period t
tg 
srok Slack associated with quantitative and desirable output r to DMU o for echelon k in

period t
tb 
srok Slack associated with quantitative and undesirable output r to DMU o for echelon k in

period t

sot ,k ,k 1 Slack associated with intermediate link d to DMU o between echelons k and k  1 in
d

period t which is treated as  (input/output/free/fixed)

sokt ,ct 1, Slack associated with carryover c to DMU o for echelon k between periods t and t  1

which is treated as  (desirable/undesirable/free/fixed)

 tjk Intensity variable and reference set indicator for DMU j for echelon k in period t

 rkl
t1
Permissible worth vector for qualitative output r for ordinal position l to DMU j for

echelon k in period t
 ikl
t2
Permissible worth vector for qualitative input i for ordinal position l to DMU j for

echelon k in period t

Model formulation:
In this section, we elaborate the proposed DNDEA model using the aforementioned notations
and assumptions.

The overall efficiency of the supply chain network is calculated by the objective function (10):

22
      ikl t2
  
   tg 
siok tb 
siok  lLi  sot ,input
k ,k 1d sokt ,ct 1,undes   
   kg x tg  kb x tb  k   t l    z t    
   iI1 ijk iI1 iok iI 2   ijk     k , k 1 o ( k ,k 1)
d Dinput cCundes
k zokt ,ct 1   
 lLi 
d

 W t   wk 1    
tT 

kK 

kg kb

I1  I1  I 2  Dinputk  k ,k 1
 Cundes k
 

  
  
   (10)
o  min
      rkl  t 1
 
  
tg 
srok tb 
srok   sot ,output
k ,k 1d sokt ,ct 1,des   
  kg y tg  kb y tb  k   t  l       t ,t 1   
lL

 rR1 rok rR1 rok rR2   rok
t
 k , k 1 zo ( k ,k 1)
   dDoutput cCdesk zokc  
  lL   
d

 t
  k
  
 
W w 1
 k ,k 1
tT  kK  R1  R1  R2  Doutput  Cdes
kg kb k k

  
  
   
k t
where w is the weight of echelon k and W is the weight of period t. These weights are
provided exogenously and in our case they are provided by experts. Naturally, those periods
which are closer to the current time period should possess higher weights than the ones which are
more far away. The numerator comprises those terms related to both quantitative and qualitative
input slacks while the denominator comprises those terms which impart the influence imposed
by both quantitative and qualitative output slacks to the objective function. In general, overall
efficiency would be less than one (i.e. o*  1 ), and for an efficient DMU o*  1 if and only if all
the slacks are zero.

Constraints regarding quantitative inputs and outputs, which also comprise desirable and
undesirable categories, are as follows:


j 1
t tg
jk ijk
tg 
x  siok  xiok
tg
i  I1kg , k  K , t  T (11)


j 1
t tb
jk ijk
tb
x  siok  xiok
tb
i  I1kb , k  K , t  T (12)


j 1
t
jk
tg
yrjk tg 
 srok  yrok
tg
r  R1kg , k  K , t  T (13)


j 1
t
jk
tb
yrjk tb
 srok  yrok
tb
r  R1kb , k  K , t  T (14)

Eqs. (11) and (12) address desirable and undesirable quantitative inputs respectively, and Eqs.
(13) and (14) take desirable and undesirable quantitative outputs into consideration, respectively.

23
Also, constraints concerning qualitative inputs and outputs are listed below.


j 1
t
t
jk ijk  l   iklt 2  ijkt  l  i  I 2k , l  Li , k  K , t  T (15)


j 1
t
t
jk rjk  l   rklt1   rokt  l  r  R2k , l  Lr , k  K , t  T (16)

Eqs. (15) and (16) account for the effect of qualitative inputs and outputs, respectively.

The bundle of constraints (17)-

(20) present the divisional intermediate link relations which can be viewed as input-oriented,
output-oriented, free links or fixed links.

 n



j 1
t t
z
jk j ( k ,k 1)d  sot ,input
k ,k 1
d
 zot ( k ,k 1)d  k ,k 1
d  Dinput , k  K , t  T (input) (17)

 n




j 1
t t
z
jk j ( k ,k 1)d  sot ,output
k ,k 1
d
 zot ( k ,k 1)d  k ,k 1
d  Doutput , k  K , t  T (output) (18)

 n



j 1
t t
z
jk j ( k ,k 1)d  sot , kfree
,k 1
d
 zot ( k ,k 1)d d  D free
k ,k 1
, k  K , t  T (free) (19)
 n



 t t
z
jk j ( k ,k 1)d  zot ( k ,k 1)d d  D fixk ,k 1 , k  K , t  T (fixed) (20)
 j 1

If the link between two divisions is treated as input and slacks are considered as the related
surpluses, then the input-oriented constraint (i.e. Eq. (17)) has to be used. If it is treated as output
and slacks are dimmed as deficits, then the output-oriented constraint (i.e. Eq. (18)) has to be
considered in the model. Furthermore, if the link between consecutive divisions are
discretionary, then a free link in the form of Eq. (19) is used and if it is non-discretionary, then a
fixed link in the form of Eq.

(20) is used.

Similar to the bundle constraints presented for divisional intermediate links, the bundle
constraints (21)-(24) are used for the carryover links between consecutive periods. If a carryover
is supposed to be treated as output, then the desirable carryover Eq. (21) should be taken into
consideration. If a carryover is treated as input, then the undesirable carryover Eq. (22) should be
taken into account. Also, for free and fixed carryovers, Eqs. (23) and Eq. (24) should be

24
respectively incorporated into the model. Eq. (25) is the continuity condition between the
periods, which is a mandatory constraint for dynamic DEA models.

 n


 j 1
t
jk z jkt ,ct 1  sokt ,ct 1,des  zokt ,ct 1 c  Cdes
k
, k  K , t  T (desirable) (21)

 n

  j 1
t
jk z jkt ,ct 1  sokt ,ct 1,undes  zokt ,ct 1 c  Cundes
k
, k  K , t  T (undesirable) (22)


 n


 j 1
t
jk z jkt ,ct 1  sokt ,ct 1, free  zokt ,ct 1 c  C kfree , k  K , t  T (free) (23)

 n

  t
jk z jkt ,ct 1  zokt ,ct 1 c  C kfixed , k  K , t  T (fixed) (24)
 j 1

n n

  tjk z jkt ,t1    tjk1z jkt ,t 1

j 1
c
j 1
c
c  C k , k  K , t  1,2, ,T  1 (25)

Other required constraints completing the DNDEA model are as follows:


j 1
t
jk 1 k  K , t  T (26)

 t ,t 1 ,des  t ,t 1 ,undes

tg 
siok tb 
,siok tg 
,srok tb 
,srok ,sot ,input
k ,k 1
,sot ,output
k ,k 1
,sok c
,sok c
, tjk  0
d d

 t ,t 1 , free
sot ,free
k ,k 1
,sok c
 free (27)
d

 rkl
t1
,ikl
t2
 int eger
Eq. (26) is related to the intensity vector corresponding to each division in each period which
determines the return-to-scale feature of the model. Notably, the presented model considers the
variable return-to-scale (VRS) throughout the efficiency calculation process because it provides
a framework in which the DMUs are evaluated more fairly and each DMU’s performance is
compared to those of its league which are assessed using their specific frontier. Omitting this
constraint results in a constant return-to-scale (CRS) model formulation. Finally, Eq. (27)
determines the type of decision variables used in the model.

The proposed DNDEA model above is not a linear one because of its objective function.
However, it can be easily transformed to a linear programming model similar to the method
presented by Tone (2001).

25
 3.2.5. Divisional and periodical efficiencies

By adjusting the objective function (10), it is possible to calculate a division’s efficiency over
several periods, a period’s overall efficiency for the whole network or a division’s efficiency in a
given period. The divisional efficiency for DMU o and division k is defined by Eq. (28):

    ikl t2
 
  tg 
siok tb 
siok  lLi  sot ,input
k ,k 1d sokt ,ct 1,undes  
  kg x tg  kb x tb  k   t l    z t   
  iI1 ijk iI1 iok iI 2   ijk     k , k 1 o ( k ,k 1)
d Dinput cCundes
k zokt ,ct 1  
 lLi 
d

 W t 1   
tT 

kg kb

I1  I1  I 2  Dinputk  k ,k 1
 Cundes k
 

 
 
 ok    (28)
 
 
tg 
srok tb 
srok


  t1
rkl


t ,output
so k ,k 1 sok  t ,t 1,des 


  kg y tg  kb y tb  k   t  l       c t ,t 1  
lL d

  rR1 rok rR1 rok rR2  

t
 k , k 1 zo ( k ,k 1)
rok  dDoutput d Cdes
k zokc 
 
d

 t
 
lL

 
W 1
 k ,k 1
tT  R1  R1  R2  Doutput  Cdes
kg kb k k

 
 
 

Similarly, the periodical efficiency for DMU o in period t is defined by Eq. (29):

    ikl t2
 
  tg 
siok tb 
siok  lLi  sot ,input
k ,k 1d sokt ,ct 1,undes  
  kg x tg  kb x tb  k   t l    z t   
  iI1 ijk iI1 iok iI 2   ijk     k , k 1 o ( k ,k 1)
d Dinput d Cundes
k zokt ,ct 1  
 lLi 
d

 wk 1   
kK 

kg kb

I1  I1  I 2  Dinputk  k ,k 1
 Cundes k
 

 
 
 ot    (29)
     rkl  t1
 
 
tg 
srok tb 
srok  lL  sot ,output
k ,k 1d sokt ,ct 1,des  
  kg y tg   tb   t  k ,k 1 z t
  k z t ,t 1  
  rR1 rok rR1kb yrok rR2k    rok  l   dDoutput o ( k ,k 1) d cCdes okc 
wk 1     
 l L

kK 

R kg
1  R 1 
kb
 R k
2  D  k ,k 1
output  C k
des  

 
 

And the periodical-divisional efficiency for DMU j for division k in period t is defined by Eq.

(30):

26
    iklt 2  
 tg 
siok tb 
siok  lLi  sot ,input
k ,k 1d sokt ,ct 1,undes 
 kg x tg  kb x tb  k   t l      
 1
iI ijk iI1 iok iI 2   ijk    d  D
t
 k , k 1 zo ( k ,k 1) cCundes
k zokt ,ct 1 
 lLi 
d

1  
input

 okt 
 kb k  k ,k 1
I1  I1  I 2  Dinput  Cundes
kg k
 (30)
    rklt1
 t ,t 1 ,des 
 s tg 
s tb 
  sot ,output
k ,k 1d sok c  
 kg y tg  kb y tb  k   t  l       t ,t 1 
rok rok lL

 rR1 rok rR1 rok rR2   rok

t
 k , k 1 zo ( k ,k 1)
 dDoutput cCdes
k zokc 
   
d
lL
1
kg
kb k  k ,k 1
R1  R1  R2  Doutput  Cdes k

The objective function Eq. (10) does not involve the slack variables for the free intermediate
links and carryovers. Therefore, their influences on the model have been considered indirectly by
having effect on the intensity vector variables. However, they can be included in the objective
function using one of the two methods suggested by Tone and Tsutsui (2010). Further
information regarding the initial conditions for carryover links, uniqueness of periodical
efficiencies and model linearization is discussed by Tone and Tsutsui (2014) at length which can
be applied to the proposed DNDEA model as well.

4. A Case Study in automotive supply chains

In this section, the proposed performance evaluation framework is applied in order to assess
sixty automotive part supply chains active in Iranian automotive industry over the course of four
years from 2013 to 2016. In order to find the most influential factors to be used in the proposed
model considering the specifications that needed to be addressed in our case, we have consulted
with five general managers with more than 10 years experience in Iranian automotive market.
Experts’ positions and automotive-related experience in years are Production and Planning
General Manager with 10 years experience, Purchasing General Manager with 8 years
experience, Supply Chain Director with 15 years experience in automotive supply chain,
Industrialization Project Manager with 9 years experience and Human Resources Manager with 7
years experience. Based on that, a network of sustainability and resilience factors is provided for
the evaluation of these supply chains. In order to thoroughly evaluate sustainability and
resilience aspects, several performance indexes have also been developed.

27
 4.1. Elected factors, dataset and the proposed network for automotive supply chains

In Iran, automotive industry is conquered by two giant automobile manufacturers. Supply chains
that have been subjected to this study are in contact with these two carmakers at their
manufacturing’s echelon. As Fig. 4 shows, supply chains used in this study are of three-echelon
ones each of which including a Tier2 supplier, a main supplier and the manufacturer. It should be
noted that some of these supply chains differ only in the manufacturer echelon.

{Please insert Fig. 4 around here.}

Using the experts’ opinions and the hybrid QFD-DEMATEL method for factor selection
proposed in Section 3.1, Table 3 shows the selected factors to be used in the proposed network. It
is worth mentioning that one should consider the number of DMUs (peer supply chains in this
case) while determining the number of factors to be selected for the model. Evidently, large
number of inputs and outputs compared to the number of DMUs will undermine the
discrimination power of DEA model and will result in too many efficient DMUs. There is a
prominent rule in the DEA literature which states that the number of DMUs should be at least
three times of the number of inputs and outputs combined in order to produce meaningful results
(Cook, Tone, & Zhu, 2014). This rule does not have any statistical basis but an experimental
ground. However, this rule does not seem to be applicable for a network DEA model.
Nevertheless, throughout this study, based on the trial and error approach, we have realized that
in the network structures, considering the complexity of the network including intermediate links
and carryovers, the number of DMUs should be at least three to five times the number of factors
used in the model in order to obtain meaningful results.

Table 3 – Elected factors using the Hybrid QFD-DEMATEL method

Sustainability/Res
Echelon Factor Factor Type Units
ilience Dimension
Cost_1 Input Economic Billion IR_Rials*
Number_of_Employees_1 Input Social Persons
Tier2 Supplier

Number of days
Average_Inventory_1 Carryover Resilience
production secured
Waste_1 Undesirable Output Environmental Tons
Employee_Satisfaction_1 Qualitative Output Social Likert Scale

28
 Part_Unit_Profit_1 Intermediate Link Economic Thousand IR_Rials
Recyclable_Waste_2 Intermediate link Environmental Equivalent Parts
Cost_2 Input Economic Billion IR_Rials
Number_of_Employees_2 Input Social Persons
Number of days
Main Supplier

Average_Inventory_2 Carryover Resilience

production secured
Waste_2 Undesirable Output Environmental Tons
Employee_Satisfaction_2 Qualitative Output Social Likert Scale
Part_Unit_Profit_2 Intermediate Link Economic Thousand IR_Rials
Recyclable_Waste_3 Intermediate link Environmental Equivalent Parts
Cost_3 Input Economic Billion IR_Rials
Number_of_Employees_3 Input Social Persons
Manufacturer

Number of days
Average_Part_Inventory Carryover Resilience
production secured
Car_Delivery_Delay Undesirable Output Social Days
Employee_Satisfaction_3 Qualitative Output Social Likert Scale
Prorated_Part_Unit_Profit Output Economic Thousand IR_Rials
* Iranian Rial is the official unit of Iranian currency.

Factors Cost_1, Cost_2 and Cost_3 are provided as interval data because of the data
classification level rule in companies. Cost, in all the echelons, is indicative of the collective
direct cost including raw material cost, maintenance cost of the equipment and machinery used
for production of that specific part, cost associated with the personnel and several other factors.
Number_of_Employees_1, Number_of_Employees_2 and Number_of_Employees_3 are those
factors taking into account the number of employees in different echelons who are directly
involved in the process. Factor Average_Inventory_1 is the daily average of the final product
which is going to be used as an input for the main supplier over the past year. The amount of
inventory is divided to the main supplier’s production plan which will result in the number of
days during which the production of main supplier is secured based on the inventory of Tier2
supplier. Average of daily figures over a year would determine the value of this factor for Tier2
supplier. Similarly, factor Average_Inventory_2 is the same for the main supplier and the
manufacturer. Furthermore, factor Average_Part_Inventory contains a similar concept for the
manufacturer. The amount of supplied part’s inventory by the main supplier is divided to the
manufacturer’s production plan every day, and the average of this daily figure over a year would
determine the value of this factor for the manufacturer. Factors Waste_1 and Waste_2 indicate

29
the amounts of non-recyclable waste in Tier2 supplier and main supplier, respectively. Factors
Recyclable_Waste_2 and Recyclable_Waste_3 represent the amounts of recyclable waste
produced in the main supplier and the manufacturer, respectively. The recyclable waste produced
in the main supplier is sent back to the Tier2 supplier and the recyclable waste produced in the
manufacturer is sent back to the main supplier for rework and recycling. It is measured with the
equivalent parts unit which means that how many parts a certain amount of waste would produce
after recycling and it does not matter how much the waste is. Factors Employee_Satisfaction_1,
Employee_Satisfaction_2 and Employee_Satisfaction_3, which are measured based on a 9-point
Likert Scale, are indicators of the global employee satisfaction in Tier2 supplier, the main
supplier and the manufacturer echelons, respectively. They are determined by the annual
employee surveys which are conducted throughout the companies. It is also worth mentioning
that in our study, the higher Likert Scale value is indicative of the higher satisfaction level.
Factors Part_Unit_Profit_1 and Part_Unit_Profit_2 indicate the amount of profit each part will
contribute to the Tier2 Supplier and Main Supplier respectively. Similarly,
Prorated_Part_Unit_Profit is a factor which intends to suggest the total profit of selling a car
prorated on the supplier part from the Main Supplier. Factor Car_Delivery_Delay is a social
factor indicating the customer satisfaction regarding the vehicle delivery time and if it is delayed
or not compared to the obligated delivery date. It is the deviation of real car delivery time to the
customer and the obligated delivery time which was set when the customer paid the first
instalment. Finally, the network used for this study is shown in Fig. 5.

{Please insert Fig. 5 around here.}

4.2. Performance evaluation and analysis

We selected 60 supply chains active in the Iranian automotive market and the collected raw data
over four consecutive years from 2013 to 2016 whose accuracy has been validated by the
managers. It is worth mentioning that each supply chain is considered as a DMU which has its
own inputs, outputs, intermediate links and carryovers illustrated in Fig 5. Statistics of the
dataset for each factor and each echelon are shown in Table S1 of the online Supplementary
Material.

30
Based on the experts’ proposal, importance weights for Tier2 Supplier, main supplier and
Manufacturer are set to 0.2, 0.5 and 0.3, respectively. It is also worth mentioning that a
comprehensive sensitivity analysis on the importance weights of echelons is conducted which is
available in the online supplementary material which supports the weights proposed by experts.
Furthermore, importance weights for years from 2013 to 2016 are set to 0.1, 0.2, 0.3 and 0.4,
respectively. All the carryovers and intermediate links that are used in this study are assumed
free based on their features which are all supposed to be discretionary. Using the proposed
DNDEA model, a concise summary of the results of overall supply chain efficiency, and their
average divisional and average periodical efficiencies are shown in Table 5. Full results can be
found in Table S2 of the online supplementary material.

Table 5 – A concise summary of results of the proposed DNDEA model

Average Divisional Efficiency Average Periodical Efficiency
Supply Overall
Rank Tier2 Main
Chain Efficiency Manufacturer 2013 2014 2015 2016
Supplier Supplier
SC01 0.8961 9 0.9657 1 0.6441 0.8288 0.8689 0.8645 0.9176
SC02 0.7742 38 0.3592 1 0.6377 0.6387 0.6491 0.6435 0.7313
SC03 0.7847 35 0.5404 0.8238 1 0.8239 0.8430 0.7836 0.7017
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
SC58 0.5798 50 0.7875 0.4882 0.6346 0.5985 0.6938 0.7043 0.5505
SC59 0.7609 39 0.6052 0.8436 0.5911 0.5726 0.6699 0.7128 0.7645
SC60 0.7114 45 1 0.5664 0.9089 0.8527 0.9796 0.8027 0.6653
Average 0.7551 - 0.7017 0.8019 0.7168 0.7238 0.7549 0.7482 0.7337

It turns out that SC19 is the best supply chain from the overall performance viewpoint in which
its main supplier echelon has performed the best over the four years. Moreover, the whole chain
has performed very well in 2014 and 2015 by getting full score for the periodical efficiencies in
those years. Fig 6 compares all the supply chains’ overall performances with the average.

{Please insert Fig. 6 around here.}

31
Using the intensity variable  tjk will help to determine a reference set for DMU j for division k

in period t. For instance, if we consider SC58, it is possible to determine the reference set in each
period for each division in order to establish a guideline to walk down the path for efficiency
improvement throughout the supply chain’s network and enhance the overall performance. Table
6 shows the reference set for SC58.
Table 6 – Reference set for SC58
Division Tier2 Supplier Main Supplier Manufacturer
Year Reference  value Reference  value Reference  value
SC22 0.439
SC25 0.697 SC31 0.916
2014 SC28 0.288
SC38 0.303 SC38 0.084
SC57 0.273
SC10 0.082
SC31 0.916
2015 SC58 1 SC25 0.579
SC38 0.084
SC28 0.339
SC19 0.143 SC31 0.984
2016 SC58 1
SC25 0.857 SC32 0.016
SC14 0.893 SC22 0.5
2017 SC03 1
SC56 0.107 SC25 0.5

Similarly, slacks can be determined in order to give guidance for efficiency improvement in each
divisional and periodical performance. Using these slacks could be a helping hand for managers
in order to come up with new strategies to reduce inefficiency sources. For SC58, Table 7
indicates the slack for each factor in each division over the course of four years.
Table 7 – Slacks for SC58 over 2013-2016
Echelon Factor 2013 2014 2015 2016

Cost_1 314.53 - - 299.27

Number_of_Employees_1 89 - - 631
Tier2 Supplier

Average_Inventory_1 - - - -
Waste_1 0.044 - - 0.284
Employee_Satisfaction_1 - - - 1
Part_Unit_Profit_1 5.77 - - 0.68
Recyclable_Waste_2 174.25 214 506.71 467.5
Cost_2 231.91 276.24 141.07 281.7
Supplier

Number_of_Employees_2 85 112 144 93

Main

Average_Inventory_2 - - 0.8 -
Waste_2 0.067 0.170 0.614 0.522

32
 Employee_Satisfaction_2 1 2 4 2
Part_Unit_Profit_2 - 6.33 7.85 3.42
Recyclable_Waste_3 1923 1843.5 1828 1853
Cost_3 - - - 1976
Number_of_Employees_3 - - - 2280
Manufacturer

Average_Part_Inventory - 1.55 1.61 1.4

Car_Delivery_Delay - - - 2
Employee_Satisfaction_3 - - - 3
Prorated_Part_Unit_Profit - 102.8 96.4 121.5

Table 7 provides rough input data for the strategic planning for the top management. Managers
usually need some indicators for decision-making for which they use some raw data as inputs
and deliver processed information as output. Using slacks provided by Table 7, it is possible to
use performance indexes to evaluate a DMU’s performance from economic, environmental,
social and resilience points of view. In our case, we have developed the indexes shown in Table
8 to address this issue.
Table 8 – Performance indexes
Division
Tier2 Supplier Main Supplier Manufacturer
Aspect

art Unit rofit art Unit rofit 2 rorated art Unit rofit
Cost Cost 2 Cost
Economic art Unit rofit Slack art Unit rofit 2 Slack rorated art Unit rofit Slack
Cost Surpluss Cost 2 Surpluss Cost Surpluss

umber of Employees umber of Employees 2 Car Delivery Delay

Social umber of Employees Surpluss umber of Employees 2 Surpluss Car Delivery Delay Surpluss
aste aste 2
Recyclable aste 2 Recyclable aste
Environmental aste Slack aste 2 Slack
-
Recyclable aste 2 Surpluss Recyclable aste Surpluss

Average Inventory Average Inventory 2 Average art Inventory

Resilience Average Inventory Surpluss Average Inventory 2 Surpluss Average art Inventory Surpluss

Obviously, these indexes will take a value between 0 and 1. The higher value is indicative of a
better performance in that aspect. For instance, if we consider SC58, these indexes can be
calculated based on Table 9 for 2016.
Table 9 – Performance indexes for SC58
Division
Aspect Tier2 Supplier Main Supplier Manufacturer Average

Economic 0.487 0.528 0.625 0.546

33
 Social 0.740 0.819 0.891 0.816
Environmental 0.282 0.021 - 0.151
Resilience 1 1 0.682 0.894

The results of Table 9 would determine how good a supply chain is when it comes to
sustainability and resilience aspects. For instance, for SC58, the economic performance index
shows that this supply chain has not performed in an acceptable level and it needs to be
improved in this aspect by adopting strategies such as cost reduction and austerity plans. The
social performance index is indicative of the fact that the supply chain’s social performance has
been acceptable and it can be even improved if Tier2 supplier improves employee satisfaction
throughout its company. From the environmental viewpoint, the associated performance index
shows that the supply chain has performed at a disastrous level and it needs to put environmental
plans in the top of its agenda in order to increase its competency in the long-term. Furthermore,
the resilience performance index is indicative of the fact that the supply chain’s performance
from the resilience viewpoint in Tier2 supplier and main supplier echelons has been phenomenal
and by increasing the level of inventory in the manufacturer echelon, it can become a benchmark
for other supply chains.

4.3. Model validation

In order to validate the proposed model, we have conducted a case study analysis in order to find
out whether the proposed DNDEA model could produce legitimate results or not. For this, we
have made a comparative analysis between the proposed DNDEA model and those models
without dynamic and without network structure separately. Fig 7 shows the network structure
used for the model without dynamic structure; which is called NDEA model hereafter. NDEA
model neglects the carryover links and there is no relation between time periods. In fact,
efficiency is calculated as the weighted average of periodical efficiencies.

{Please insert Fig. 7 around here.}

Table 10 compares the overall efficiency obtained from both the NDEA and the proposed
DNDEA model in a concise style. However, a full comparison is available in Table S3 of the
online supplementary material. For instance, the NDEA model sets a lower overall efficiency for
SC58 because Average_Inventory_1, Average_Inventory_2 and Average_Part_Inventory
carryovers are eliminated from the model. Average_Inventory_1 and Average_Inventory_2 are

34
the factors which are the sources of efficiency for SC58 due to the fact that slacks of
Average_Inventory_1 have been zero throughout the analysis span and also
Average_Inventory_2 has the same situation except for 2015. It is worth mentioning that
Average_Part_Inventory is partially the source of inefficiency for SC58 but the combination of
Average_Inventory_1 and Average_Inventory_2 carryovers outweigh that factor and contributes
more to the overall efficiency.

For the model without network structure, which is called DDEA hereafter, Fig. 8 depicts the
dynamic structure as well as inputs, outputs and carryovers which are used in this model in order
to make a comparison with the proposed DNDEA model. As it is illustrated in Fig. 8, the
network structure in the supply chain as a DMU is now eliminated and the whole supply chain is
considered as a black box. Therefore, intermediate links including Part_Unit_Profit_1,
Part_Unit_Profit_2, Recyclable_Waste_2 and Recyclable_Waste_3 are eliminated from the
model. Furthermore, some factors are aggregated into a single factor in the DDEA model.
Total_Cost is a combination of Cost_1, Cost_2 and Cost_3 and also Total_Employees is an
aggregation of factors Number_of_Employees_1, Number_of_Employees_2 and
Number_of_Employees_3. Similarly, Total_Waste is formed by the summation of Waste_1 and
Waste_2. Furthermore, Average_Inventory is the divisional average of two related carryovers
namely Average_Inventory_1 and Average_Inventory_2. Similarly,
Average_Employee_Satisfaction is the divisional average of Employee_Satisfaction_1,
Employee_Satisfaction_2 and Employee_Satisfaction_3.

{Please insert Fig. 8 around here.}

Table 10 includes the overall efficiency obtained from both the DDEA and the proposed
DNDEA models which sets the stage for deeper analysis in a concise summary. The full table is
available in Table S3 of the online supplementary material.

Being unmindful of the internal network structure for supply chain performance evaluation and
aggregation of divisional inputs and outputs could lead to biased and misguiding results. For
SC58, intermediate links in the proposed DNDEA model are mainly the sources of inefficiency,
especially Recyclable_Waste_2 and Recyclable_Waste_3 and when they are eliminated in the
DDEA model, the overall efficiency is consequently increased.

Table 10 – Concise comparison between the results of NDEA and DNDEA models

35
 Supply Proposed DNDEA model NDEA model DDEA model

Chain Overall Efficiency Rank Overall Efficiency Rank Overall Efficiency Rank

SC01 0.8961 9 0.9143 3 0.7464 38

SC02 0.7742 38 0.8172 15 0.8551 21
SC03 0.7847 35 0.5734 44 0.7400 41
. . . . . . .
. . . . . . .
. . . . . . .

SC58 0.5798 50 0.5446 49 0.6690 50

SC59 0.7609 39 0.7907 20 0.8389 23

SC60 0.7114 45 0.5235 52 0.8297 26

It is now clear that without taking the dynamic and network structure into account in the context
of supply chain performance evaluation, results would be biased and negligence in this matter
leads to wrong decisions for managers who wish to improve the efficiency of the whole supply
chain.
A comparative sensitivity analysis between the proposed DNDEA model, the NDEA and DDEA
models from the echelons’ importance weights viewpoint is also included in Section S2 of the
online supplementary material which bolsters the validation of DNDEA model.

5. Managerial implications
The proposed performance measurement and evaluation framework which mainly relies on the
developed DNDEA model, enables top managers to evaluate their strategic decisions and their
impacts on the performance of their supply chains in the short and long-term horizons. They can
use divisional and periodical efficiencies provided by the proposed DNDEA model for each
DMU in order to analyze where they have to put their concentration in a supply chain network to
improve its performance. The model also provides a set of benchmark DMUs for each division at
each period which can help managers to put their focus and energy on and try to provide the
same conditions existing in the benchmark DMUs to be applied for the underperforming DMU at
the echelon level. For instance, the reference set for SC58 is provided in Table 6. It is also worth
mentioning that in our case study there was not a single DMU which performed perfectly in all
its echelons in all periods. It provides more options and more opportunities for the managers to

36
make improvements in more than one possible way as there will be enough number of DMUs to
be used as benchmarks.

On the other hand, the proposed DNDEA model is categorized under the SBM classification of
DEA models. Therefore, it provides slacks that show to what extent a DMU in each of its
input/output/intermediate/carryover links shows inefficiency. Using these slacks, we have
developed different performance indexes in order to measure and evaluate a supply chain’s
performance from different sustainability and resilience viewpoints at an overall and divisional
levels. Those indexes are shown in Table 8 for our case study which can be applied in any other
context with minor modification. For instance, as it is shown in Table 9, the way SC58
performed from economic, social, environmental and resilience aspects is clearly suggesting that
despite having an acceptable performance level on the social and resilience fronts, on the
environmental front its performance is nothing near acceptable and most of the concentration
should be directed at improving this aspect of SC58.

6. Conclusions
This paper proposes a systematic framework for supply chain performance evaluation by
focusing on internal divisions and their related performances over a period of time. In order to
evaluate a supply chain’s performance comprehensively, it is now inevitable to look beyond the
economic aspects and include the sustainability and resilience factors in such a study.
Deployment of sustainability and resilience concepts in SCPM context motivated the authors to
develop a generalized dynamic network DEA model. It is worth noting that this paper develops a
DNDEA model which can cope with modelling complexities including the undesirable factors
and data irregularities (i.e. interval data, negative data and qualitative data) associated with the
complexities pertaining to the sustainable and resilient supply chain performance evaluation. The
proposed model is capable of handling both quantitative and qualitative data at the same time in
a DNDEA model which is unprecedented in the literature of SCPM. Also, for the first time in the
literature, a hybrid QFD-DEMATEL method is linked with the proposed DNDEA model to
select the best sustainability and resilience inputs/outputs in a systematic manner, which then are
used in the DNDEA model. Moreover, the proposed DNDEA model is used for the first time in a
case study to evaluate sixty supply chains active in Iran’s automotive industry in order to assess
their divisional, periodical and overall performances from 2013 to 2016. Internal structure and

37
relations between consecutive periods have been taken into consideration by using intermediate
and carryover links in the network structure. Sources of inefficiencies are then determined using
the model which are very useful for the managers to adopt suitable performance improvement
policies. The proposed DNDEA model can provide the reference set whose performance would
be a benchmark for inefficient DMUs in order to walk down the path for efficiency
improvement. Moreover, in order to showcase the comprehensive results of the model, some
performance indexes are designed to summarize the supply chain’s performance from
sustainability and resilience viewpoints. At last, model’s validation is examined by making
comparisons between the proposed DNDEA model with two models constructed by eliminating
the dynamic or network structure feature from the DNDEA model.

For further research, a common-weight DEA model can be used in order to decrease the
computational complexity. Another possible future research direction is to use the proposed
DNDEA model for performance evaluation of supply chains after an incident eruption especially
from the resilience viewpoint. Researchers can also conduct specific performance evaluation
studies based on the geographical locations of supply chains and their divisional distances for
future research. Moreover, a Malmquist index can be used to analyze the overall and divisional
efficiencies in a more detailed way in several consecutive periods. Another interesting future
research proposal can include proposing a new DNDEA model to cope with a larger number of
inputs/outputs, intermediates and carryovers without losing its discrimination power. As the final
future research direction, we propose that supply chains with different network structures to be
incorporated in the performance evaluation process while the strategic resilience factors (e.g.
complexity, density and network centralization) are taken into account in the study.

38
References

Ahi, P., & Searcy, C. (2015). An analysis of metrics used to measure performance in green and
sustainable supply chains. Journal of Cleaner Production, 86, 360–377.
https://doi.org/http://dx.doi.org/10.1016/j.jclepro.2014.08.005
Amindoust, A., Ahmed, S., Saghafinia, A., & Bahreininejad, A. (2012). Sustainable supplier
selection: A ranking model based on fuzzy inference system. Applied Soft Computing,
12(6), 1668–1677. https://doi.org/http://dx.doi.org/10.1016/j.asoc.2012.01.023
Azadi, M., Jafarian, M., Farzipoor Saen, R., & Mirhedayatian, S. M. (2015). A new fuzzy DEA
model for evaluation of efficiency and effectiveness of suppliers in sustainable supply chain
management context. Computers & Operations Research, 54, 274–285.
https://doi.org/http://dx.doi.org/10.1016/j.cor.2014.03.002
Azadnia, A. H., Saman, M. Z. M., & Wong, K. Y. (2015). Sustainable supplier selection and
order lot-sizing: an integrated multi-objective decision-making process. International
Journal of Production Research, 53(2), 383–408.
https://doi.org/10.1080/00207543.2014.935827
Azevedo, S. G., Govindan, K., Carvalho, H., & Cruz-Machado, V. (2011). GResilient index to
assess the greenness and resilience of the automotive supply chain. Discussion Papers of
Business and Economics, 8.
Bairamzadeh, S., Pishvaee, M. S., & Saidi-Mehrabad, M. (2015). Multiobjective Robust
Possibilistic Programming Approach to Sustainable Bioethanol Supply Chain Design under
Multiple Uncertainties. Industrial & Engineering Chemistry Research, 55(1), 237–256.
Boukherroub, T., Ruiz, A., Guinet, A., & Fondrevelle, J. (2015). An integrated approach for
sustainable supply chain planning. Computers & Operations Research, 54, 180–194.
https://doi.org/http://dx.doi.org/10.1016/j.cor.2014.09.002
Büyüközkan, G., & Berkol, Ç. (2011). Designing a sustainable supply chain using an integrated
analytic network process and goal programming approach in quality function deployment.
Expert Systems with Applications, 38(11), 13731–13748.
https://doi.org/http://dx.doi.org/10.1016/j.eswa.2011.04.171
Cardoso, S. R., Paula Barbosa-Póvoa, A., Relvas, S., & Novais, A. Q. (2015). Resilience metrics
in the assessment of complex supply-chains performance operating under demand
uncertainty. Omega, 56(Supplement C), 53–73.
https://doi.org/https://doi.org/10.1016/j.omega.2015.03.008
Carvalho, H., Barroso, A. P., Machado, V. H., Azevedo, S., & Cruz-Machado, V. (2012). Supply
chain redesign for resilience using simulation. Computers & Industrial Engineering, 62(1),
329–341. https://doi.org/http://dx.doi.org/10.1016/j.cie.2011.10.003
Cha, S.-C., Juo, P.-W., Liu, L.-T., & Chen, W.-N. (2008). Riskpatrol: A risk management system
considering the integration risk management with business continuity processes. In
Intelligence and Security Informatics, 2008. ISI 2008. IEEE International Conference on
(pp. 110–115). IEEE.

39
Chaabane, A., Ramudhin, A., & Paquet, M. (2011). Designing supply chains with sustainability
considerations. Production Planning & Control, 22(8), 727–741.
https://doi.org/10.1080/09537287.2010.543554
Chang, D.-S., Kuo, L. R., & Chen, Y. (2013). Industrial changes in corporate sustainability
performance – an empirical overview using data envelopment analysis. Journal of Cleaner
Production, 56, 147–155. https://doi.org/http://dx.doi.org/10.1016/j.jclepro.2011.09.015
Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making
units. European Journal of Operational Research, 2(6), 429–444.
https://doi.org/10.1016/0377-2217(78)90138-8
Christopher, M., & Peck, H. (2004). Building the resilient supply chain. The International
Journal of Logistics Management, 15(2), 1–14.
Cook, W. D., Liang, L., & Zhu, J. (2010). Measuring performance of two-stage network
structures by DEA: A review and future perspective. Omega, 38(6), 423–430.
https://doi.org/http://dx.doi.org/10.1016/j.omega.2009.12.001
Cook, W. D., Tone, K., & Zhu, J. (2014). Data envelopment analysis: Prior to choosing a model.
Omega (United Kingdom), 44, 1–4. https://doi.org/10.1016/j.omega.2013.09.004
Cook, W. D., & Zhu, J. (2006). Rank order data in DEA: A general framework. European
Journal of Operational Research, 174(2), 1021–1038.
https://doi.org/http://dx.doi.org/10.1016/j.ejor.2005.01.063
Dai, J., & Blackhurst, J. (2012). A four-phase AHP–QFD approach for supplier assessment: a
sustainability perspective. International Journal of Production Research, 50(19), 5474–
5490. https://doi.org/10.1080/00207543.2011.639396
Faramarzi, G. R., Khodakarami, M., Shabani, A., Farzipoor Saen, R., & Azad, F. (2015). New
network data envelopment analysis approaches: an application in measuring sustainable
operation of combined cycle power plants. Journal of Cleaner Production, 108, 232–246.
https://doi.org/10.1016/j.jclepro.2015.06.065
Färe, R., & Grosskopf, S. (2012). Intertemporal production frontiers: with dynamic DEA.
Springer Science & Business Media.
Färe, R., Grosskopf, S., & Whittaker, G. (2007). Network dea. In Modeling data irregularities
and structural complexities in data envelopment analysis (pp. 209–240). Springer.
Ghadimi, P., & Heavey, C. (2014). Sustainable Supplier Selection in Medical Device Industry:
Toward Sustainable Manufacturing. Procedia CIRP, 15, 165–170.
https://doi.org/http://dx.doi.org/10.1016/j.procir.2014.06.096
Govindan, K., Khodaverdi, R., & Jafarian, A. (2013). A fuzzy multi criteria approach for
measuring sustainability performance of a supplier based on triple bottom line approach.
Journal of Cleaner Production, 47, 345–354.
https://doi.org/http://dx.doi.org/10.1016/j.jclepro.2012.04.014
Haldar, A., Ray, A., Banerjee, D., & Ghosh, S. (2012). A hybrid MCDM model for resilient
supplier selection. International Journal of Management Science and Engineering

40
 Management, 7(4), 284–292. https://doi.org/10.1080/17509653.2012.10671234
Hassini, E., Surti, C., & Searcy, C. (2012). A literature review and a case study of sustainable
supply chains with a focus on metrics. International Journal of Production Economics,
140(1), 69–82. https://doi.org/http://dx.doi.org/10.1016/j.ijpe.2012.01.042
Hsu, C.-W., Hu, A. H., Chiou, C.-Y., & Chen, T.-C. (2011). Using the FDM and ANP to
construct a sustainability balanced scorecard for the semiconductor industry. Expert Systems
with Applications, 38(10), 12891–12899.
https://doi.org/http://dx.doi.org/10.1016/j.eswa.2011.04.082
Jayaram, J., Dixit, M., & Motwani, J. (2014). Supply chain management capability of small and
medium sized family businesses in India: A multiple case study approach. International
Journal of Production Economics, 147, 472–485.
https://doi.org/https://doi.org/10.1016/j.ijpe.2013.08.016
Jiang, W., Sha, E. H.-M., Zhuge, Q., & Wu, L. (2017). Efficient assignment algorithms to
minimize operation cost for supply chain networks in agile manufacturing. Computers &
Industrial Engineering, 108, 225–239.
https://doi.org/https://doi.org/10.1016/j.cie.2017.04.014
Kannan, D., Jabbour, A. B. L. de S., & Jabbour, C. J. C. (2014). Selecting green suppliers based
on GSCM practices: Using fuzzy TOPSIS applied to a Brazilian electronics company.
European Journal of Operational Research, 233(2), 432–447.
https://doi.org/http://dx.doi.org/10.1016/j.ejor.2013.07.023
Kannan, G., Pokharel, S., & Sasi Kumar, P. (2009). A hybrid approach using ISM and fuzzy
TOPSIS for the selection of reverse logistics provider. Resources, Conservation and
Recycling, 54(1), 28–36. https://doi.org/http://dx.doi.org/10.1016/j.resconrec.2009.06.004
Kao, C. (2014). Network data envelopment analysis: A review. European Journal of Operational
Research, 239(1), 1–16. https://doi.org/10.1016/j.ejor.2014.02.039
Kim, Y., Choi, T. Y., Yan, T., & Dooley, K. (2011). Structural investigation of supply networks:
A social network analysis approach. Journal of Operations Management, 29(3), 194–211.
https://doi.org/https://doi.org/10.1016/j.jom.2010.11.001
Kordrostami, S., & Amirteimoori, A. (2005). Un-desirable factors in multi-component
performance measurement. Applied Mathematics and Computation, 171(2), 721–729.
https://doi.org/http://dx.doi.org/10.1016/j.amc.2005.01.081
Lee, B. L., & Worthington, A. C. (2016). A network DEA quantity and quality-orientated
production model: An application to Australian university research services. Omega (United
Kingdom), 60, 26–33. https://doi.org/10.1016/j.omega.2015.05.014
Lozano, S. (2016). Slacks-based inefficiency approach for general networks with bad outputs:
An application to the banking sector. Omega (United Kingdom), 60, 73–84.
https://doi.org/10.1016/j.omega.2015.02.012
Lozano, S., Gutiérrez, E., & Moreno, P. (2013). Network DEA approach to airports performance
assessment considering undesirable outputs. Applied Mathematical Modelling, 37(4), 1665–
1676. https://doi.org/http://dx.doi.org/10.1016/j.apm.2012.04.041
41
Min, H., & Kim, I. (2012). Green supply chain research: past, present, and future. Logistics
Research, 4(1), 39–47. https://doi.org/10.1007/s12159-012-0071-3
Mirhedayatian, S. M., Azadi, M., & Farzipoor Saen, R. (2014). A novel network data
envelopment analysis model for evaluating green supply chain management. International
Journal of Production Economics, 147(PART B), 544–554.
https://doi.org/10.1016/j.ijpe.2013.02.009
Mitra, K., Gudi, R. D., Patwardhan, S. C., & Sardar, G. (2009). Towards resilient supply chains:
Uncertainty analysis using fuzzy mathematical programming. Chemical Engineering
Research and Design, 87(7), 967–981.
https://doi.org/http://dx.doi.org/10.1016/j.cherd.2008.12.025
Motevali Haghighi, S., Torabi, S. A., & Ghasemi, R. (2016). An integrated approach for
performance evaluation in sustainable supply chain networks (with a case study). Journal of
Cleaner Production, 137, 579–597.
https://doi.org/http://dx.doi.org/10.1016/j.jclepro.2016.07.119
Nakamichi, K., Hanaoka, S., & Kawahara, Y. (2016). Estimation of cost and CO2 emissions
with a sustainable cross-border supply chain in the automobile industry: A case study of
Thailand and neighboring countries. Transportation Research Part D: Transport and
Environment, 43, 158–168. https://doi.org/http://dx.doi.org/10.1016/j.trd.2015.12.018
Nouri, J., Hosseinzadeh Lotfi, F., Borgheipour, H., Atabi, F., Sadeghzadeh, S. M., &
Moghaddas, Z. (2013). An analysis of the implementation of energy efficiency measures in
the vegetable oil industry of Iran: a data envelopment analysis approach. Journal of Cleaner
Production, 52, 84–93. https://doi.org/http://dx.doi.org/10.1016/j.jclepro.2013.02.021
Peng, P., Snyder, L. V, Lim, A., & Liu, Z. (2011). Reliable logistics networks design with
facility disruptions. Transportation Research Part B: Methodological, 45(8), 1190–1211.
https://doi.org/http://dx.doi.org/10.1016/j.trb.2011.05.022
Ponomarov, S. Y., & Holcomb, M. C. (2009). Understanding the concept of supply chain
resilience. The International Journal of Logistics Management, 20(1), 124–143.
https://doi.org/10.1108/09574090910954873
Sawik, T. (2011). Selection of supply portfolio under disruption risks. Omega, 39(2), 194–208.
https://doi.org/http://dx.doi.org/10.1016/j.omega.2010.06.007
Schmitt, A. J., & Singh, M. (2012). A quantitative analysis of disruption risk in a multi-echelon
supply chain. International Journal of Production Economics, 139(1), 22–32.
https://doi.org/http://dx.doi.org/10.1016/j.ijpe.2012.01.004
Seiford, L. M., & Zhu, J. (2002). Modeling undesirable factors in efficiency evaluation.
European Journal of Operational Research, 142(1), 16–20.
https://doi.org/http://dx.doi.org/10.1016/S0377-2217(01)00293-4
Song, M., Wang, S., & Liu, W. (2014). A two-stage DEA approach for environmental efficiency
measurement. Environmental Monitoring and Assessment, 186(5), 3041–3051.
Tajbakhsh, A., & Hassini, E. (2015). A data envelopment analysis approach to evaluate
sustainability in supply chain networks. Journal of Cleaner Production, 105, 74–85.
42
 https://doi.org/10.1016/j.jclepro.2014.07.054
Tang, C. S. (2006). Perspectives in supply chain risk management. International Journal of
Production Economics, 103(2), 451–488.
https://doi.org/http://dx.doi.org/10.1016/j.ijpe.2005.12.006
Tone, K. (2001). A slacks-based measure of efficiency in data envelopment analysis. European
Journal of Operational Research, 130(3), 498–509. https://doi.org/10.1016/S0377-
2217(99)00407-5
Tone, K., & Tsutsui, M. (2009). Network DEA: A slacks-based measure approach. European
Journal of Operational Research, 197(1), 243–252.
https://doi.org/http://dx.doi.org/10.1016/j.ejor.2008.05.027
Tone, K., & Tsutsui, M. (2010). Dynamic DEA: a slacks-based measure approach. OMEGA The
International Journal of Management Science, 38(3-4), 145–156.
https://doi.org/10.1016/j.omega.2009.07.003
Tone, K., & Tsutsui, M. (2014). Dynamic DEA with network structure: A slacks-based measure
approach. Omega (United Kingdom), 42(1), 124–131.
https://doi.org/10.1016/j.omega.2013.04.002
Tseng, M.-L., Lin, R.-J., Lin, Y.-H., Chen, R.-H., & Tan, K. (2014). Close-loop or open
hierarchical structures in green supply chain management under uncertainty. Expert Systems
with Applications, 41(7), 3250–3260.
https://doi.org/http://dx.doi.org/10.1016/j.eswa.2013.10.062
Türkay, M., Saraçoğlu, Ö., & Arslan, M. C. (20 6). Sustainability in Supply Chain Management:
Aggregate Planning from Sustainability Perspective. PLOS ONE, 11(1), e0147502.
Retrieved from http://dx.doi.org/10.1371%2Fjournal.pone.0147502
Vahidi, F., Torabi, S. A., & Ramezankhani, M. J. (2018). Sustainable supplier selection and
order allocation under operational and disruption risks. Journal of Cleaner Production, 174,
1351–1365. https://doi.org/https://doi.org/10.1016/j.jclepro.2017.11.012
Varsei, M., Soosay, C., Fahimnia, B., & Sarkis, J. (2014). Framing sustainability performance of
supply chains with multidimensional indicators. Supply Chain Management: An
International Journal, 19(3), 242–257. https://doi.org/10.1108/SCM-12-2013-0436
Wilhelm, M., Blome, C., Wieck, E., & Xiao, C. Y. (2016). Implementing sustainability in multi-
tier supply chains: Strategies and contingencies in managing sub-suppliers. International
Journal of Production Economics, 182, 196–212.
https://doi.org/http://dx.doi.org/10.1016/j.ijpe.2016.08.006
Wu, D., & Olson, D. L. (2008). A comparison of stochastic dominance and stochastic DEA for
vendor evaluation. International Journal of Production Research, 46(8), 2313–2327.
You, Y. Q., & Jie, T. (2016). A study of the operation efficiency and cost performance indices of
power-supply companies in China based on a dynamic network slacks-based measure
model. Omega (United Kingdom), 60, 85–97. https://doi.org/10.1016/j.omega.2014.11.011
Yu, M. M., Chen, L. H., & Hsiao, B. (2016). Dynamic performance assessment of bus transit

43
 with the multi-activity network structure. Omega (United Kingdom), 60, 15–25.
https://doi.org/10.1016/j.omega.2015.06.003
Zha, Y., Liang, N., Wu, M., & Bian, Y. (2016). Efficiency evaluation of banks in China: A
dynamic two-stage slacks-based measure approach. Omega (United Kingdom), 60, 60–72.
https://doi.org/10.1016/j.omega.2014.12.008
Zhu, J., & Cook, W. D. (2007). Modeling data irregularities and structural complexities in data
envelopment analysis. Springer Science & Business Media.

44
Highlights

- A generalized DNDEA model is presented for supply chain performance assessment.

- Performance assessment is conducted from both sustainability and resilience viewpoints.
- A QFD-DEMATEL method is devised to select the most influential factors of DNDEA
model.
- A case study in the automotive industry is provided.

45
 Use hybrid QFD-
Construct the network of
DEMATEL method to
peer supply chains to be
choose best factors to be
evaluated
used in the DNDEA model
Perform analysis on each
Run DNDEA model and
supply chain using
calculate efficiencies for
reference set indicators
each supply chain
and performance indexes

Fig 1 – Outlined flowchart of the proposed performance measurement and evaluation framework

Weigh Sustainability Identify and Compose Sort Factors in

Determine Factor
and Resilience Criteria Factors for Sustainability Descending Order and
Weights Using QFD
with DEMATEL and Resilience Criteria Selecting The Best Ones

Determine the relation Calculate the full

Calculate the average Calculate the initial direct
between different criteria direct/indirect influence
matrix influence matrix
using paired comparisons matrix.

Fig 2 – The proposed QFD-DEMATEL method flowchart

Fig 3 – General supply chain dynamic network structure

46
 Tier2 Supplier Main Supplier Manufacturer

Fig 4 – The general three-echelon supply chain in Iranian automotive industry

Period T Number_of_ Number_of_ Number_of_

Cost_1 Employees_1 Cost_2 Employees_2 Cost_3 Employees_3
Average_Inventory_1 Average_Inventory_2
Average_Part
_Inventory

Part_Unit_Profit_1 Part_Unit_Profit_2

Tier2 Supplier Recyclable_Waste_2

Main Supplier Recyclable_Waste_3
Manufacturer

Prorated_Part
_Unit_Profit

Employee_Satis Employee_Satis Car_Delivery Employee_Satis

Waste_1 Waste_2
faction_1 faction_2 _Delay faction_3
Average_Inventory_1 Average_Inventory_2 Average_Part
_Inventory

Period T+1 Number_of_ Number_of_

Cost_1 Number_of_ Cost_2 Employees_2 Cost_3 Employees_3
Employees_1

Part_Unit_Profit_1 Part_Unit_Profit_2

Tier2 Supplier Recyclable_Waste_2

Main Supplier Recyclable_Waste_3
Manufacturer

Prorated_Part
_Unit_Profit
Average_Part
Average_Inventory_1 Average_Inventory_2
_Inventory
Employee_Satis Employee_Satis Car_Delivery Employee_Satis
Waste_1 Waste_2
faction_1 faction_2 _Delay faction_3

Fig 5 – Supply Chain network designed for part supply performance evaluation in Iranian automotive industry

47
 Efficiency

0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9

0
1

SC01
SC03
SC05
SC07
SC09
SC11
SC13
SC15
SC17
SC19
SC21
SC23
SC25
SC27

48
SC29
SC31
SC33
SC35
SC37
SC39
SC41
SC43
SC45
SC47
SC49
Fig 6 – Comparison between supply chains’ overall efficiencies and the average

SC51
SC53
SC55
SC57
SC59
 Period T Number_of_ Number_of_ Number_of_
Cost_1 Employees_1 Cost_2 Employees_2 Cost_3 Employees_3

Part_Unit_Profit_1 Part_Unit_Profit_2

Tier2 Supplier Recyclable_Waste_2

Main Supplier Recyclable_Waste_3
Manufacturer

Prorated_Part
_Unit_Profit

Employee_Satis Employee_Satis Car_Delivery Employee_Satis

Waste_1 Waste_2
faction_1 faction_2 _Delay faction_3

Period T+1 Number_of_ Number_of_

Cost_1 Number_of_ Cost_2 Employees_2 Cost_3 Employees_3
Employees_1

Part_Unit_Profit_1 Part_Unit_Profit_2

Tier2 Supplier Recyclable_Waste_2

Main Supplier Recyclable_Waste_3
Manufacturer

Prorated_Part
_Unit_Profit

Employee_Satis Employee_Satis Car_Delivery Employee_Satis

Waste_1 Waste_2
faction_1 faction_2 _Delay faction_3

Fig 7 – Supply Chain network structure used for the model without dynamic structure (NDEA)

49
Period T
Total_Cost Average_Inventory Average_Part_Inventory

Supply Chain

Car_Delivery_Delay Total_Waste Average_Employee_Satisfaction Prorated_Part_Unit_Profit

Average_Inventory Average_Part_Inventory

Period T+1
Total_Cost

Supply Chain

Car_Delivery_Delay Total_Waste Average_Employee_Satisfaction Prorated_Part_Unit_Profit

Average_Inventory Average_Part_Inventory

Fig 8 – Supply chain network structure used for the model without network structure (DDEA)

50