sustainability

Review
Performance Evaluation of Reverse Logistics:
Opportunities for Future Research
Derek Voigt * , Nelson Casarotto Filho, Mayara Atherino Macedo , Thais Guerra Braga and
Rodrigo Ulisses Garbin da Rocha
Department of Production Engineering, Federal University of Santa Catarina; Florianópolis 88040-900, Brazil;
ncasarottofilho@gmail.com (N.C.F.); m.atherinomacedo@gmail.com (M.A.M.); thais.guerra@gmail.com (T.G.B.);
eng_garbin@yahoo.com.br (R.U.G.d.R)
* Correspondence: derekvoigt@gmail.com




Received: 6 August 2019; Accepted: 20 September 2019; Published: 26 September 2019


Abstract: This study aimed to analyze the characteristics of the scientific research that approaches
Reverse Logistics (RL) from the perspective of Performance Evaluation (PE). For this purpose,
ProKnow-C was used to select 21 articles to compose a Bibliographical Portfolio (BP). Among the
results, we mention Govindan, Sarkis, Zhu and Lai as prominent authors. By analyzing the articles,
it was perceived that most of them perform the RL and PE fields separately and present tools that
consider the alignment of the indicators with the strategy. Also, the articles lack of a structured PE
process that could serve as a subsidy to the practice of logistics management. The results present that
there is a theoretical gap in the literature of a PE model of RL.

Keywords: reverse logistics; performance evaluation; literature review

1. Introduction
Reverse logistics is an increasingly discussed subject both under supply chain management
literature and the environmental theme [1]. The reverse logistics process consists of a series of activities
required to collect an obsolete, unused, or disposal product, in order to generate revenue through
recycling or resale in the secondary market, but also to guarantee environmentally correct disposal [2].
In the last decade, environmental awareness has become a liability for many companies in the supply
chain and is mostly fostered by government regulations and under client perspectives on social and
environmental issues [3,4].
The dissemination of knowledge of the practices of reverse logistics management has made
companies aware not only of the impact they are causing but also the loss of new business
opportunities [5]. With the generation of RL (Reverse Logistics) knowledge, combined with
the high level of information and indicators on which decision-making is based, the Reverse
Logistics Performance Assessment can be seen as a crucial factor in guiding strategic, tactical and
operational decisions.
Developing metrics for measuring the performance of a supply chain is a difficult problem and it
becomes more complex when taking the reverse logistics chain in account, given the difficulties in
operating and coordinating the flow of materials and information [6]. Among the questions that need
to be addressed for the evaluation of performance: What has to be measured? How often does it have
to be measured? How to integrate several individual measurements into a measurement system [7]?
Neely [8] has reported the difficulty of establishing performance metrics due to the singularities of each
organization and that, even when using unique metrics adequate for a given scenario, they should be
periodically reviewed and, when necessary, new ones should be incorporated into measuring systems.

Sustainability 2019, 11, 5291; doi:10.3390/su11195291 www.mdpi.com/journal/sustainability

Sustainability 2019, 11, 5291 2 of 17

Effective performance management is an important aspect of the reverse logistics initiative

in corporations [9]. Organizations are dealing with many uncertainties when facing their market,
characterized with a high rate of return due to the expansion of portfolios and shorter life cycles of
products, which generates a decisive impact on the strategies of companies [6]. There are two main
issues in this context: (i) Evaluation of various strategies, processes and resources to achieve the
objectives and develop measures; and (ii) how the organization should prioritize the initiatives and
returns of stakeholders that directly impact them.
In addition to the need for awareness and sustainable practices, reverse logistics operators still
suffer from a lack of knowledge of the process involved, which leads to various problems such as
loss-allocation processes, little information exchange between different providers (members of sector),
lack of knowledge diffusion, and a small participation of the academic community in the development
of the sector [10].
Based on these arguments, it is evident the need for knowledge of PE of RL in organizations,
providing support and robustness for decision making. Therefore, the question that guides this
study: What are the opportunities of research on the subject of reverse logistics management from the
perspective of performance?
As a way of addressing this question, the objective of this study is to analyze the characteristics of
the scientific research that approach the fragment of literature related to reverse logistics under the
perspective of performance evaluation. This analysis is guided by the purpose of generating knowledge
on the studied subject and to point out possibilities of future contributions to this area. In order to guide
the defined objective, the methodology carried out includes the selection of a bibliographic portfolio (BP)
on the subject and a critical analysis of the articles of the selected BP, based on the theoretical constructs
of a performance evaluation, in order to identify future research opportunities. To systematize
this methodology, we used the intervention tool Knowledge Development Process-Constructivist
(ProKnow-C) [11,12].
This research is justified by the analysis of the fragment of the selected literature, from a
constructivist perspective of performance evaluation, to encompass substantial advances in this area of
knowledge and in the improvement of reverse logistics performance management methods. In the
literature, no substantial works that were willing to analyze research opportunities for the fragment of
the scientific literature on the performance evaluation of reverse logistics were found, which validates
the originality of this study. Finally, its feasibility is conferred by access to scientific articles in the
CAPES journal portal (Brazilian Higher Education Personnel Improvement Coordination).
This article, in addition to this introductory section, presents in Section 2 the methodology
and the instrument selected to guide the achievement of the research objective. In Section 3 is the
theoretical reference, describing performance evaluation and reverse logistics from the perspective of
its Performance Evaluation. Section 4 presents and discuss the results of the research in terms of the
selected bibliographical, critical analysis of the BP and research opportunities identified. The final
considerations will be presented in Section 5; and, finally, the article references.

2. Literature and Analyzing Methods

2.1. Methodology and Materials and Methods

This research is classified exploratory-descriptive, since it is based on the formation of a
bibliographic portfolio and explores a fragment of the literature (performance evaluation of reverse
logistics). The articles of the BP were extracted in the selected databases, according to the delimitations
imposed by researchers in order to identify and highlight research opportunities in the academe
and management fields of reverse logistics. It is descriptive as it deals with the interpretation and
analysis of BP data and disclosure of research opportunities and practical challenges to promote the
advancement of this area [13].
Sustainability 2019, 11, 5291 3 of 17

Sustainability
As for the 2019, 11, x FOR procedures,
technical PEER REVIEWthe bibliographic research was used as it involved the analysis 3 ofof17
articles published in the selected databases [14].
As Asforfor
thethe
data data collection,
collection, they they
were were obtained
obtained from primary
from primary and secondary
and secondary sources.
sources. The primaryThe
primary data were obtained in a selection of the bibliographic portfolio in
data were obtained in a selection of the bibliographic portfolio in the Portal of CAPES Journals. the Portal of CAPES
InJournals. In a subsequent
a subsequent step, the secondary
step, the secondary data were data were present
present in the bibliometric
in the bibliometric analysisanalysis step,
step, so thatso
the interpreted data was obtained through the formed portfolio of articles. Thus, the approach isis
that the interpreted data was obtained through the formed portfolio of articles. Thus, the approach
qualitativebecause
qualitative becausethe theanalysis
analysissteps
stepsofofthethearticles
articlesBPBPwere
werebased
basedononthe
theknowledge,
knowledge,judgments,
judgments,
andinterpretations
and interpretationsofofthe theauthors
authorsofofthis
thisresearch
research[14].
[14].

2.2.Instrument
2.2. InstrumentofofIntervention:
Intervention:Knowledge
KnowledgeDevelopment
DevelopmentProcess-Constructivist
Process-Constructivist(Proknow-C)
(Proknow-C)
TheKnowledge
The KnowledgeDevelopment
DevelopmentProcess-Constructivist
Process-Constructivist(Proknow-C)
(Proknow-C)methodmethodisisthe
theinstrument
instrumentofof
intervention selected by the authors, since its objective is the generation
intervention selected by the authors, since its objective is the generation of knowledge, made possible of knowledge, made
bypossible by the constructivist
the constructivist action demandedaction todemanded
operationalize to operationalize
its stages. Theits stages. Theofconstruction
construction knowledge isof
knowledge
based is based onreflection
on the researchers’ the researchers’
that leadsreflection that leads
them to establish them to establish
delimitations and make delimitations
choices during and
make choices during a sequence of activities
a sequence of activities that make up the ProKnow-C [11,15]. that make up the ProKnow-C [11,15].
TheProKnow-C
The ProKnow-C consists
consists of of four
four steps:
steps: (i) (i) Selection
Selection of the
of the bibliographic
bibliographic portfolio
portfolio (BP)(BP) of articles
of articles on
on the subject of research; (ii) bibliometric analysis of BP; (iii) systemic analysis
the subject of research; (ii) bibliometric analysis of BP; (iii) systemic analysis of BP; and (iv) identificationof BP; and (iv)
ofidentification of the research
the research question question and
and objective(s) objective(s)
for future for[16–20].
research future research
Figure 1[16–20].
shows theFigure 1 shows
summary viewthe
ofsummary
the four viewsteps ofof the four steps It
ProKnow-C. ofshould
ProKnow-C. It should be
be emphasized emphasized
that thatonly
in this article, in this
thearticle,
steps only the
(i), (ii),
steps (i), (ii), and (iv) of
and (iv) of ProKnow-C are presented.ProKnow-C are presented.

Figure1.1.Stages
Figure Stagesofofthe
theKnowledge
KnowledgeDevelopment
DevelopmentProcess-Constructivist
Process-Constructivist(Proknow-C)
(Proknow-C)intervention
intervention
instrument. Source: Translated from Valmorbida and Ensslin [15].
instrument. Source: Translated from Valmorbida and Ensslin [15].
2.2.1. Procedures for Data Collection: Selection of the Bibliographic Portfolio
2.2.1. Procedures for Data Collection: Selection of the Bibliographic Portfolio
The first step proposed by ProKnow-C is with the selection of the articles that compose the
The first step proposed by ProKnow-C is with the selection of the articles that compose the
bibliographic portfolio (BP) and, thus, the formation of a gross bank of articles. Based on the definition
bibliographic portfolio (BP) and, thus, the formation of a gross bank of articles. Based on the
of the research axes, keywords, and their delimited combination, the articles were searched for
definition of the research axes, keywords, and their delimited combination, the articles were
in six selected databases available in the Portal of CAPES (Brazilian Higher Education Personnel
searched for in six selected databases available in the Portal of CAPES (Brazilian Higher Education
Improvement Coordination) Journals. The search in the databases was held on 30 September and
Personnel Improvement Coordination) Journals. The search in the databases was held on 30
1 October 2017. The filters delimited in each database considered only the articles published in English
September and 1 October 2017. The filters delimited in each database considered only the articles
published in English language, between the years of 2000 and 2017, published in journals. Figure 2
illustrates the procedures adopted from the selection of the raw articles bank to BP formation.
 Sustainability 2019, 11, 5291 4 of 17

language, between the years of 2000 and 2017, published in journals. Figure 2 illustrates the procedures
adopted from the selection of the raw articles bank to BP formation.
Sustainability 2019, 11, x FOR PEER REVIEW 4 of 17

Figure
Figure2.2.Procedures
Proceduresadopted
adoptedfor
forbibliographic
bibliographicportfolio
portfolio(BP)
(BP)selection.
selection.Source:
Source:Developed
Developedbybythe
the
authors based on the procedures adopted in Proknow-C.
authors based on the procedures adopted in Proknow-C.

With
Withthe
theBPBPformed,
formed,the
theselected
selectedpublications
publicationswere
wereanalyzed
analyzedwith
withthe
theaim
aimofofsearching
searchingfor
for
opportunities
opportunitiesfor
forimprovement
improvementtotoreverse
reverselogistics
logisticsmanagement.
management.

2.2.2.Procedures
2.2.2. Proceduresfor
forData
DataAnalysis:
Analysis:Basic
Basicand
andAdvanced
AdvancedBibliometric
BibliometricAnalysis
Analysis
Continuingwith
Continuing with thethe subsequent
subsequent steps,steps, Proknow-C
Proknow-C first presents
first presents a bibliometrics
a bibliometrics analysis,analysis,
which
which
aims to aims to identify
identify and highlight
and highlight the prominent
the prominent characteristics
characteristics of the of the analyzed
analyzed BP, based BP, based
on two on
two different approaches: (i) Basic bibliometrics; and (ii)
different approaches: (i) Basic bibliometrics; and (ii) advanced bibliometrics. advanced bibliometrics.
Forthe
For thebasic
basicvariables,
variables,the theresearchers
researcherscarried
carriedout outa anumerical
numericalcount countofofoccurrences
occurrencesand/or and/or
elementsthat
elements that could allow allow totoincrease
increasetheir
theirknowledge
knowledge on on
the the
subject. It was
subject. evidenced
It was that the
evidenced thatsimple
the
simple numerical measurement and quantitative compilation of the data are not enough the
numerical measurement and quantitative compilation of the data are not enough to guarantee to
generation
guarantee theofgeneration
knowledgeofinknowledge
the researchers
in the[12]. In this way,
researchers [12]. it
Inisthis
necessary
way, it is tonecessary
interpret the obtained
to interpret
data,
the its origin
obtained data,and itsother
origincomplementary data, to justify
and other complementary data,their occurrences.
to justify Only in thisOnly
their occurrences. way in can the
this
researcher support his arguments and choices [11,12,21]. For this research,
way can the researcher support his arguments and choices [11,12,21]. For this research, the following the following basic variables
werevariables
basic verified: were(i) Researchers
verified: (i) with trajectory with
Researchers in thistrajectory
theme; (ii) in dynamics
this theme; of (ii)
the dynamics
authors networks
of the
formation;
authors (iii) institutions
networks formation; to which authors are affiliated;
(iii) institutions to whichand (iv) BP are
authors classification
affiliated;inand empirical
(iv) BP and
theoretical studies.
classification in empirical and theoretical studies.
Forthe
For the next
next step,
step, ofofthe
theadvanced
advanced variables, thethe
variables, analyzes
analyzeswerewereperformed
performed only in empirical
only articles.
in empirical
The analysis of the variables in empirical articles has the purpose of verifying
articles. The analysis of the variables in empirical articles has the purpose of verifying the presence the presence or absence
orofabsence
the characteristics pointed out pointed
of the characteristics as necessary in the
out as theoretical
necessary researches,
in the theoretical when the fundamentals
researches, when theof
the performance
fundamentals of theevaluation
performance are analyzed.
evaluation Thus, the theoretical
are analyzed. Thus,research that makes
the theoretical up the
research thatBPmakes
of this
research are not analyzed on the advanced analysis variables.
up the BP of this research are not analyzed on the advanced analysis variables.
Theadvanced
The advanced variables
variablesused used in the present
in the research
present are derived
research from the from
are derived characteristics pointed out
the characteristics
by Neely,
pointed outGregory,
by Neely, andGregory,
Platts [8] and
and by Carneiro-da-Cunha,
Platts Hourneaux, andHourneaux,
[8] and by Carneiro-da-Cunha, Corrêa [22]. and The Corrêa
features
are: (i) Organization of performance measures (single, whole system);
[22]. The features are: (i) Organization of performance measures (single, whole system); (ii) (ii) definition of performance
indicators;
definition of (iii) alignmentindicators;
performance of the indicators with the
(iii) alignment of strategy;
the indicators(iv) purpose
with theofstrategy;
the presented model
(iv) purpose
of the presented model (measurement, diagnosis, comparison, management); (v) interaction with the
environment (internal, external, or both) [8]; (vi) focus (measurement, management); (vii)
perspective (operational, strategic); (viii) scope (reduced, broad); (ix) amplitude (internal, external);
(x) nature (transactional, relational); and (xi) accountability (executives, stakeholders) [22].
Sustainability 2019, 11, 5291 5 of 17

(measurement, diagnosis, comparison, management); (v) interaction with the environment (internal,
external, or both) [8]; (vi) focus (measurement, management); (vii) perspective (operational, strategic);
(viii) scope (reduced, broad); (ix) amplitude (internal, external); (x) nature (transactional, relational);
and (xi) accountability (executives, stakeholders) [22].

3. Theoretical Framework

3.1. Performance Evaluation

Organizations have undergone several changes in recent years and among them, are the form of
control and measurement of the indicators. Performance evaluation should contribute to analyze and
measure all deliverables of a given organizational context and not only input and output in isolation [2].
Neely et al. [8] categorizes performance evaluation as a complex activity, which consists of quantifying
the efficiency and effectiveness of the actions, through a set of individual indicators, called Performance
Evaluation Systems.
The evaluation system must have indicators defined internally and grouped in a set, linked to
the strategic context of the company that assist in decision making, since the isolated metrics are not
enough to define what should be measured [8]. Once indicators and the way in which they will be
measured are defined, performance evaluation is an ongoing task and aims to increase the likelihood
of an organization achieving the desired results [23].
However, several authors portray the difficulty of establishing performance metrics because of the
individualities of each organization and how, even when using unique metrics for context, they should
be reviewed periodically. This means that, when there is a need, new variables should be incorporated
into the measurement systems [8].
The constructivist perspective of performance evaluation occurs in several environments, and also
in the Brazilian context. The importance of a structured process is presented by Ensslin et al. [16] and
Ensslin et al. [21], who mention that in the area of business knowledge, the performance measurement
dimension provides input for management activity. Shaik and Abdul-Kader [24] show that, in reverse
logistics, many indicators are raised and discussed; however, in most cases, they have no connection
with the company’s strategy.
Besides not presenting a connection with the company’s strategy, many of the indicators have no
relationship to each other, which creates difficulty in decision making, not allowing the organization
process to be more effective and efficient. This means that the indicators are raised and measured,
but they are not managed in order to achieve the organization’s goal. Ensslin et al. [21] present
the importance of performance evaluation (PE) to ensure management processes are linked to the
organization’s strategy. In this case, managers can make decisions based on data and secure and
reliable information. In the view of more reliable processes, Carneiro-da-Cunha et al. [22] portray
a new demand that arises in the area—the general tendency to have more and better transparency,
control, and efficiency in the operations of organizations.
From the organization’s strategy point of view, strategic value is established by the characteristics
a resource must possess in order to be considered a source of competitive advantage. Thus, strategic
value depends on the ability of resources to help the corporation gain competitive advantage,
reduce competitive disadvantage, or achieve other goals [25].
In conclusion, the reverse logistics sector faces the challenge of deploying its strategies into
indicators with the objective of managing and creating opportunities for efficiency and operational
effectiveness with a clear and focused view of the actions’ results [26].

3.2. Reverse Logistics

Rogers and Tibben-Lembke [10] defined reverse logistics (RL) as “the process of planning,
application, control of operation, cost, flow of raw materials, the inventory process, finished products,
Sustainability 2019, 11, 5291 6 of 17

the information related, from the point of consumption to the point of origin, in order to recover or
create value or proper disposal”.
The holistic view detailed by Shaik and Abdul-Kader [24], defines RL as: Product return
management, real-time inventory and workflow, tracking warranties, ordering and parts exchange,
flow of materials and information with suppliers, data analysis, execution of repairs, customer
notification, and all the logistics flow and return. Wang and Sun [27] presented distinct characteristics
of RL, such as high uncertainties of supply in time, quantity and quality, and complexities of the
operations, among other things.
Fleischmann et al. [28] proposed a framework based on the reverse flow of distribution from
the producer to the user and back to the producer, with definite types of motivations (governance
legislation, economic value, and ecological image) and types of disposal (reuse, repair, recycling, and
remanufacturing), and discussed the issue of management separator counter network flows.
De Brito and Dekker [29] proposed a framework for RL that depends on five dimensions:
(i) The return reasons; (ii) reception structure; (iii) the type of products and their characteristics;
(iv) recovery processes and options; and (v) the actors involved and their roles. The authors
looked at the characteristics of RL systems according to drivers (economics, legislation and green
citizenship), reasons (manufacturing returns, distribution returns, and customer returns), recovery
processes (collection, inspection, classification, selection and types of products returned, composition,
deterioration, and use) and agents (to the front of the supply chain, logistics reverse, and central
and specialized). Rogers, Melamed and Lembke [30] considered a number of key reverse logistics
problems by modeling techniques that can be helpful in understanding problems and developing
solutions, considering a high number of variables. Dowlatshahi [31] cited five strategic factors that
are important for the RL: Costs, quality, customer service, environmental concerns, and political/legal
concerns. Meade et al. [32] presented a framework composed of environmental factors (regulation
and respect for the environment) and business factors (returns and customer satisfaction), and also
observed that there were a large number of models generalized in the literature. Similarly, Rubio et
al. [33] confirmed the need for further research on strategic aspects and organizational frameworks.
Companies also view the RL process at strategic, tactical, and operational levels [34]. A good
RL process is necessary to deal with the return of products and to monetize the process involved in
some way. Companies now recognize that the management of the RL is important and the use of
performance indicators to measure its effectiveness can bring differential and competitive advantage [35].
In increasing competitive advantage, the study by Espino-Rodríguez and Rodríguez-Días [36] revealed
the internal and relational capacities for value creation in consumer perception, considering essential
and non-essential activities and how they relate in the generation of highest value.
Table 1 outlines the aspects related to reverse logistics based on the articles used to structure
the bibliographic review. It is noticed that the authors of the BP raise several aspects related to the
subject, but for the most part they converge to environmental, legal aspects, and of the operational
restrictions. As an exception, De Brito and Dekke [29] takes a more holistic approach on the subject in
the article: “The Framework for Reverse Logistics”. Subsequently, in the article published in 2011 by
Shaik and Abdul-Kader [24], new aspects are discussed, such as tracking warranties, orders and parts
replacement, and performing repairs. In this case, there is a clear evolution in the factors discussed,
including new aspects as the studies and proposals evolve. Given this information, it’s validated
the need for specific indicators for reverse logistics and not only the replication of indicators already
known in the literature for the supply chain.
Sustainability 2019, 11, 5291 7 of 17

Table 1. Aspects of reverse logistics. Source: Research Data (2017).

Aspects of RL [10] [24] [27] [28] [29] [31] [32]

Planning and management X X X
Application X
Operation control X X
Cost X X X X
Flow of raw materials X X X
Inventories X X
Finished products X
Flow of information X X X
Tracking warranties X
Orders and parts exchange X
Data analysis X X
Performing repairs X
Time supply uncertainties X X
Quality X X
Complexity of operations X X
Barriers X X
Legislation X X X X
Ecological image X X X X
3 R’s X X
Types and characteristics of
X
products
Services X
Customer satisfaction X
Returns X

Based on the various points raised by the authors, it is evident that reverse logistics is an issue
that is highly discussed regarding the management of its operations. Some corporations aim at the
strategic differential, others look at marketing through trustworthiness with customers, and others
are more concerned with complying with legislation. Despite having several aspects that can be seen
as objectives in the operation of reverse logistics, all seek a better result, considering efficiency and
effectiveness in the return of their products.

4. Results

4.1. Bibliometric Analysis: Basic Variables

The first concern regarding the literature on the PE of RL consists of the identification of the most
prolific authors. Prolific authors, within the framework of the constructivist perspective adopted in this
research, consist of the authors who stand out in the area, not only by the number of works identified
in the BP, but as effective researchers in the area during their academic trajectory.
The first of the basic selected variables is to check how many BP items are from the same authors,
in order to identify researchers with a track record in this area.
In Figure 3, with five articles, the most prominent author is Kannan Govindan, who is an
expert researcher in the areas of operations management and sustainability at the University of
Southern Denmark, with several published articles of high impact on logistics, recycling, resources,
Sustainability 2019, 11, 5291 8 of 17

and conservation. With three articles, there are Mohamed N. Shaik and Walid Abdul-Kader,
researchers of the Department of Manufacturing and Industrial Engineering of University Systems of
Windsor, Canada, who have many articles on reverse logistics, warehousing, logistics performance,
Sustainability 2019, 11, x FOR PEER REVIEW 8 of 17
and management.

Sustainability 2019, 11, x FOR PEER REVIEW 8 of 17

Results of
Figure 3. Results ofarticles
articlesby
byauthor
authorfor
forBP.
BP.Source:
Source:Research
ResearchData
Data(2017).
(2017).

Following
The second theconcern
analysiswhenof Figure 3, having
analyzing thetwo articleson
literature in the
the BP, there are Ezutah
performance Udoncy
evaluation Olugu,
of reverse
Hamed
logisticsSoleimani,
consists Joseph Sarkis, Kuan
of identifying Yew Wong,
the author Qasim
articles Murtaza,
aligned with Rajesh K. Singh, and
BP references. Saurabh
Under the
Agrawal; more than authors with one item in the BP, for a total of 21 articles.
constructivist perspective adopted in this research, this process consists of measuring articles Of these authors, Joseph
that
Sarkis
presentis highly recognized
identification to theforproposal
his high impactful
of reversepublications on the greenevaluation.
logistics performance supply chain (GSC),
When with a
analyzed
high
from the perspective of its authors, it allows us to verify the researchers with recognized activity in
volume of publications and a high contribution to the area. Ezutah Udoncy Olugu excels in
publications Figure
on the 3. Results
green supply of articles
chain andby author for BP.
performance Source:
and, thus, Research
stands Data
out (2017).
with publications on the
this area. Figure 4 shows the quantification of author-aligned articles in BP references.
reverse logistics performance theme.
The second
The second concern
concern when when analyzing
analyzing the the literature
literature on on the
the performance
performance evaluation
evaluation ofof reverse
reverse
logistics consists of identifying the author articles aligned with BP references.
logistics consists of identifying the author articles aligned with BP references. Under the constructivist Under the
constructivist
perspective perspective
adopted adopted this
in this research, in this research,
process consists this process consists
of measuring articlesofthat
measuring articles that
present identification
present identification to the proposal of reverse logistics performance
to the proposal of reverse logistics performance evaluation. When analyzed from the perspective evaluation. When analyzed
of its
from the perspective of its authors, it allows us to verify the researchers with recognized
authors, it allows us to verify the researchers with recognized activity in this area. Figure 4 shows the activity in
this area. Figure 4 shows the quantification of
quantification of author-aligned articles in BP references. author-aligned articles in BP references.

Figure 4. Results of articles by authors in references. Source: Research Data (2017).

The author most prominently identified in the references is Joseph Sarkis, with a total of 16
articles. As portrayed previously, Sarkis is currently the most active researcher in the environmental
and sustainable area, being an influence for other researchers in the area. In the quantification
carried out in the sequence, there is Qinghua Zhu with 14 articles, Kee-hung Lai with 10, Kannan
Govindan with 7, Shad Dowlatshahi, Moritz Fleischmann, Ezutah Udoncy Olugu and Kuan Yew
Figure 4.
Figure Results of
4. Results ofarticles
articlesby
byauthors
authorsin
inreferences.
references.Source:
Source:Research
ResearchData
Data(2017).
(2017).
Wong with 3 articles and other authors, presented in Figure 5, with an article each. Zhu Qinghua
and Kee-hung Lai are distinguished researchers in the GSC area, who also have important
The author most prominently identified in the references is Joseph Sarkis, with a total of 16
contributions in the area of sustainable logistics.
articles. As portrayed previously, Sarkis is currently the most active researcher in the environmental
With the focus of analyzing the prominent authors of BP references, it can be seen that some of
and sustainable area, being an influence for other researchers in the area. In the quantification
the main authors of the references, with an expressive quantity of articles, are not in the list of BP
carried out in the sequence, there is Qinghua Zhu with 14 articles, Kee-hung Lai with 10, Kannan
authors. One of the factors may be due to the fact that the works of these authors have been
Sustainability 2019, 11, 5291 9 of 17
Sustainability 2019, 11, x FOR PEER REVIEW 9 of 17

in theThe
BPauthor mosttwo
with only prominently identified
articles, now appear inwith
the references is Joseph
a significant numberSarkis, with a total
of articles in theofreferences,
16 articles.
As
as in the case of Joseph Sarkis. Kannan Govindan is the author who has great representation inand
portrayed previously, Sarkis is currently the most active researcher in the environmental the
sustainable
BP and also area,
in thebeing an influence
references for other
of the articles, researchers
which denotesingreat
the area. In of
affinity theresearch
quantification carried
to the topic of
out in the sequence,
the PE of RL. there is Qinghua Zhu with 14 articles, Kee-hung Lai with 10, Kannan Govindan
with Figure
7, Shad5Dowlatshahi, Moritz
presents a four Fleischmann,
quadrants graph of Ezutah Udoncy
the cross Oluguthe
between and Kuan Yew
number Wong with
of articles of the3
articles
authorsand other
in the BP authors, presented
and the number of in Figureof5,the
articles with an article
author in theeach. Zhu Qinghua
references of the BPand Kee-hung
(cross Lai
of Figure
are distinguished researchers in the GSC area, who also have important contributions
3; Figure 4), in order to make a cut of which authors should be highlighted in the studies concerning in the area of
sustainable
the topic of logistics.
reverse logistics performance evaluation.

Figure
Figure 5. 5. Crossing
Crossing of the
of the number
number of articles
of articles of authors
of BP BP authors
in BPinreferences.
BP references. Source:
Source: Research
Research Data Data
(2017).
(2017).
With the focus of analyzing the prominent authors of BP references, it can be seen that some of
the main
Fromauthors
Figure of5,the it references,
is possible with an expressive
to identify quantity
four groups of ofauthors
articles,referred
are not in to the
BP.list
Forof the
BP
authors. One of
distribution of the
thefactors
research mayin be the
due quadrants,
to the fact that the works
author of these authors
prominence in BP and have references
been publishedwas
in bases not In
considered. included
the upper in the
rightliterature analyzed.
block, Kannan In addition,
Govindan other authors
is identified with fivethat appeared in
publications in BP
theand
BP
with only two articles, now appear with a significant number of articles
seven publications in the references. The second group of authors is identified in the lower rightin the references, as in the case
of Joseph Sarkis.
quadrant, prominent Kannan authors Govindan
in the isBPthe author
such who has
as Walid great representation
Abdur-Kader and Mohammed in the BP and also
Shaik. in
In the
the
thirdreferences
quadrant, of prominent
the articles, authors
which denotes great affinity
are identified in theofBPresearch to the located
references, topic of in thethe
PE upper
of RL. left
FigureIn
quadrant. 5 presents
this quadranta four quadrants
Joseph Sarkis, graphfollowed
of the cross
by between
Qinghuathe number
Zhu, of articles
Kee-hung Lai,ofand
the Kishore
authors
in the BP and the
Pochampally, who number
are authorsof articles
withofsignificant
the authornumber
in the references of the BP
of publications (cross
listed in of
theFigure 3; Figure
references, but4),
a
in order to make a cut of which authors should be highlighted in the studies
low number of articles in the BP. In the fourth and last quadrant analyzed, in the lower left corner, concerning the topic of
reverse
we havelogistics performance
the relevant authors evaluation.
for the research theme: Kuan Yew Wong, Ezutah Olugu, Moritz
From Figure
Fleischmann, Samir 5, itSrivastava,
is possible Ru to identify
Oqi Geng, fourZelda
groups of authorsTheresa
Zabinsky, referredBarker,
to BP. For the distribution
Hamed Soleimani,
of the research in the quadrants,
Rajesh Singh, Qasim Murtaza, and Saurab Agrawal.author prominence in BP and references was considered. In the upper
rightInblock, Kannanwith
continuity Govindan is identified
the analysis of thewith
basicfive publications
variables, in BP and
17 articles of theseven
BP publications
were identified in the
in
references.
magazines,The secondingroup
as shown Figure of 6.
authors is identified in the lower right quadrant, prominent authors in
the BP such
The as Walid
journal withAbdur-Kader
the most articles and isMohammed
the JournalShaik. In theProduction,
of Cleaner third quadrant, prominent
followed by the authors
Omega
are identified
Magazine andin the
the BP references,
Journal located inConservation
of Resources, the upper left andquadrant. In thisThe
Recycling. quadrant
Journal Joseph Sarkis,
of Cleaner
followed by Qinghua Zhu, Kee-hung Lai, and Kishore Pochampally,
Production is one of the most recognized magazines in high impact publications in the theme of who are authors with significant
number
sustainableof publications
logistics, reverselisted in logistics,
the references,
and but
GSC. a low
Thenumber
Omega of articles
magazine in thepresents
BP. In thea fourth
focus andon
last quadrant analyzed,
management, which generatesin the lower left corner,for
opportunities we publications
have the relevant authorsareas
in several for the researchreverse
including theme:
Kuan Yew
logistics Wong, Ezutah
management and Olugu, Moritz Fleischmann,
its performance. The JournalSamirof Srivastava,
Resources,Ru Oqi Geng, Zelda
Conservation Zabinsky,
and Recycling
Theresa
makes clearBarker, Hamed
in its name Soleimani,
that it focuses Rajesh Singh,
on the themeQasim Murtaza, conservation,
of resources, and Saurab Agrawal.and recycling, which
in turn facilitates the discussions of the theme related to the environment including reverse logistics,
which is one of the focuses of the same publication.
 Sustainability 2019, 11, 5291 10 of 17

In continuity with the analysis of the basic variables, 17 articles of the BP were identified in
Sustainability
magazines, 2019,
as11,shown
x FOR PEER REVIEW
in Figure 6. 10 of 17

Sustainability 2019, 11, x FOR PEER REVIEW 10 of 17

Figure
Figure 6. 6. Publications
Publications byby
BPBP journals.
journals. Source:
Source: Research
Research Data
Data (2017).
(2017).

InThe journal
addition to with the mostperformed
the analysis articles is of
thethe
Journal
main of Cleaner
authors of Production,
the BP and offollowed by the Omega
the references, and
Magazine
also of their and the Journal
cross-section, of Resources,
details of ResearchConservation
Collaborationand Recycling.
Networks The Journal
are presented of Cleaner
according to
Production is
Figures 7 and 8. one of the most recognized magazines in high impact publications in the theme of
sustainable logistics, reverse logistics, and GSC. The Omega magazine presents a focus on management,
which generates opportunities for publications in several areas including reverse logistics management
and its performance. The Journal of Resources, Conservation and Recycling makes clear in its name
that it focuses on the theme of resources, conservation, and recycling, which in turn facilitates the
discussions of theFigure
theme6.related
Publications
to thebyenvironment
BP journals. Source: Research
including Datalogistics,
reverse (2017). which is one of the
focuses of the same publication.
InIn
addition to the
addition analysis
to the performed
analysis performed of the
of main authors
the main of theofBP
authors and
the BPofand
the of
references, and
the references,
also of their cross-section, details of Research Collaboration Networks are presented according
and also of their cross-section, details of Research Collaboration Networks are presented according to to
Figures 7 and 8.
Figures 7 and 8.

Figure 7. Analysis of collaboration networks (the prominence of the authors Govindan and Sarkis).
Source: Research Data (2017).

Figure 7 shows the prominence of the authors Govindan (red cluster) and Sarkis (green cluster),
with an expressive number of publications and with it the formation of a denser network of
publications between them and other authors. We can see a sum of forces of knowledge, where
Sarkis also publishes with renowned and recognized authors in the analysis of the BP, such as Zhu
and Lai. This figure graphically shows that there is a clear collaboration network in the field of
research of performance
Figure evaluation
7. Analysis of of the
collaboration logistics,
networks (the as well as inofthe
prominence thethemes
authorsthat pervade
Govindan andit, such as
Sarkis).
Figure 7. Analysis of collaboration networks (the prominence of the authors Govindan and Sarkis).
the green supply
Source: chain,
Research Datasustainable
(2017). supply chain, and management. The figure also expresses
Source: Research Data (2017).
another nine groups that collaborate in the research of this theme, however the publications occur in
isolation.
Figure 7 shows the prominence of the authors Govindan (red cluster) and Sarkis (green cluster),
with an expressive number of publications and with it the formation of a denser network of
publications between them and other authors. We can see a sum of forces of knowledge, where
Sarkis also publishes with renowned and recognized authors in the analysis of the BP, such as Zhu
and Lai. This figure graphically shows that there is a clear collaboration network in the field of
Sustainability 2019, 11, 5291 11 of 17
Sustainability 2019, 11, x FOR PEER REVIEW 11 of 17

Figure 8. Analysis of collaboration networks (the analysis of collaboration networks in a heatmap).

Figure 8. Analysis of collaboration networks (the analysis of collaboration networks in a heatmap).
Source: Research Data (2017).
Source: Research Data (2017).

Figure 7 shows the prominence of the authors Govindan (red cluster) and Sarkis (green cluster),
Figure 8 presents the analysis of collaboration networks in a heatmap, that allows us to analyze
with an expressive number of publications and with it the formation of a denser network of publications
the authors and networks that present the highest concentration of publications. Thus, the redder the
between them and other authors. We can see a sum of forces of knowledge, where Sarkis also publishes
cluster is, larger is the quantity of publications in the area of reverse logistics and stronger is the
with renowned and recognized authors in the analysis of the BP, such as Zhu and Lai. This figure
collaboration network.
graphically shows that there is a clear collaboration network in the field of research of performance
evaluation of the logistics, as well as in the themes that pervade it, such as the green supply chain,
4.2. Advanced Variable Analysis
sustainable supply chain, and management. The figure also expresses another nine groups that
In the advanced
collaborate bibliometric
in the research analysis
of this theme, of the the
however 21 initial articlesoccur
publications of BP,in13 empirical articles were
isolation.
analyzed from the perspective of the performance evaluation bases presented
Figure 8 presents the analysis of collaboration networks in a heatmap, that allows by Neely,
us toGregory
analyze
and Platts [8]
the authors andand Carneiro-da-Cunha,
networks that present theHourneaux JR and Corrêa
highest concentration [22] A preliminary
of publications. Thus, theanalysis
redder
consisted
the cluster is, larger is the quantity of publications in the area of reverse logistics and stronger in
of verifying the existence or absence of a tool, method, approach, or PE system cited the
is the
article. From this
collaboration network.perspective, it is verified that only one article did not present a set of measures, and
it was excluded from the basis. In this way, 12 articles that present measures are the basis for the next
4.2. Advanced Variable Analysis
analysis.
Figure 9 presentsbibliometric
In the advanced the first analysis of how
analysis performance
of the measures
21 initial articles are13organized,
of BP, empirical as individual
articles were
measures or a set of them. With the formation of the BP, it can be seen that
analyzed from the perspective of the performance evaluation bases presented by Neely, Gregory andof the 12 articles, 10
present joint measures of performance.
Platts [8] and Carneiro-da-Cunha, Hourneaux JR and Corrêa [22] A preliminary analysis consisted
of verifying the existence or absence of a tool, method, approach, or PE system cited in the article.
From this perspective, it is verified that only one article did not present a set of measures, and it was
excluded from the basis. In this way, 12 articles that present measures are the basis for the next analysis.
Figure 9 presents the first analysis of how performance measures are organized, as individual
measures or a set of them. With the formation of the BP, it can be seen that of the 12 articles, 10 present
joint measures of performance.

Figure 9. The performance evaluation system. Advanced variable 1. Source: Research Data (2017).

For the second analysis, presented in Figure 10, considers the relationship between PE and the
environment. In this perspective, 10 articles considered this interaction, since they involved the
stakeholders for strategy, the measurement of some indicator, or decision making.
 it was excluded from the basis. In this way, 12 articles that present measures are the basis for the next
analysis.
Figure 9 presents the first analysis of how performance measures are organized, as individual
measures or a set of them. With the formation of the BP, it can be seen that of the 12 articles, 10
present joint
Sustainability measures
2019, 11, 5291 of performance. 12 of 17

Figure 9. The performance evaluation system. Advanced variable 1. Source: Research Data (2017).
Figure 9. The performance evaluation system. Advanced variable 1. Source: Research Data (2017).
SustainabilityFor
2019,the
11, xsecond analysis,
FOR PEER REVIEWpresented in Figure 10, considers the relationship between12PE
of and
17 the
For the second analysis, presented in Figure 10, considers the relationship between
environment. In this perspective, 10 articles considered this interaction, since they involved PE andthethe
Sustainability
environment. 2019,In
11, this
x FORperspective,
PEER REVIEW10 articles considered this interaction, since they involved 12 ofthe
17
stakeholders for strategy, the measurement of some indicator, or decision making.
stakeholders for strategy, the measurement of some indicator, or decision making.

Figure 10. Performance evaluation and the environment. Advanced variable 2. Source: Research
Figure 10. Performance evaluation and the environment. Advanced variable 2. Source: Research Data
Data (2017).
Figure 10. Performance evaluation and the environment. Advanced variable 2. Source: Research
(2017).
Data (2017).
The third analysis consists of verifying the alignment of the indicators with the strategy and if
The third analysis consists of verifying the alignment of the indicators with the strategy and if this
this is evident
The in the study ofconsists
third the analyzed article. In alignment
Figure 11, of thethe 12 articles,with
10 ofthethem have and if
is evident in theanalysis of verifying
study of the analyzed article. the
In Figure 11, ofof the 12indicators
articles, 10 of them strategy
have tools that
tools that
this consider
is evident an
in alignment
the study indicator
of the to strategy.
analyzed article.ItInisFigure
critical11,
that
of PE systems
the 12 reinforce
articles, 10 of the have
them
consider an alignment indicator to strategy. It is critical that PE systems reinforce the organization’s
organization’s
tools thatstrategy to emphasize critical elements that can influence behavior, motivate, monitor
strategy to consider
emphasize ancritical
alignment indicator
elements to strategy.
that can influence It is critical
behavior, that PEmonitor
motivate, systemsand reinforce the
stimulate
and stimulate learning
organization’s [8].
learning [8]. strategy to emphasize critical elements that can influence behavior, motivate, monitor
and stimulate learning [8].

Figure 11. Alignment of indicators with company strategy and/or objectives. Advanced variable 3.
FigureSource: ResearchofData
11. Alignment (2017). with company strategy and/or objectives. Advanced variable 3.
indicators
Source: Research Data (2017).
Figure
The 11. Alignment
fourth of indicators
variable selected with company
analyzes whether strategy
the tooland/or objectives.
measures Advancedthe
and compares variable
result3.with
Source: Research Data (2017).
another performance or not. In Figure 12, we have the result of the analysis, that 11 articles have a
The fourth variable selected analyzes whether the tool measures and compares the result with
process
another of comparison
performance not.between performances.
The fourth or
variableInselected
Figure 12, we have
analyzes the result
whether of the
the tool analysis,and
measures thatcompares
11 articles
thehave a with
result
process of comparison
another between
performance performances.
or not. In Figure 12, we have the result of the analysis, that 11 articles have a
process of comparison between performances.
 Figure 11. Alignment of indicators with company strategy and/or objectives. Advanced variable 3.
Source: Research Data (2017).

The fourth variable selected analyzes whether the tool measures and compares the result with
another performance or not. In Figure 12, we have the result of the analysis, that 11 articles have a
process ofSustainability
comparison 2019, 11, 5291 performances.
between 13 of 17

Figure 12. Evaluation and comparison of measurement tools. Advanced variable 4. Source: Research
Figure 12. Data
Evaluation
(2017).and comparison of measurement tools. Advanced variable 4. Source: Research
Data (2017).
The fifth selected variable considers the PE system interaction. Of the 12 articles, 11 present
Sustainability
interaction
The fifth selected 2019,
with 11,
thex FOR
variable PEERand
internal REVIEW
considers external means tointeraction.
the PE system the organization,
Of theand only one11
12 articles, presents 13 of 17
presentinteraction
Sustainability 2019,
only 11, x
with FOR PEER REVIEW
theinternal
internal andenvironment, 13 of 17
interaction with the external as shown
means tointhe
Figure 13.
organization, and only one presents
interaction only with the internal environment, as shown in Figure 13.

Figure 13. Performance evaluation system

system interactions.
interactions. Advanced variable 5.
5. Source: Research Data
(2017).
Figure 13. Performance evaluation system interactions. Advanced variable 5. Source: Research Data
(2017).
The
The sixth
sixth variable
variable isis based
based onon the
the evolution
evolution ofof the
the organizational performance measurement.
organizational performance measurement.
According
The sixth
According totoCarneiro-da-Cunha
variable is Carneiro-da-Cunha [22],
based on the evolution performance
[22], measurement
ofperformance
the organizational has evolved
performance
measurement over time andtime
measurement.
has evolved over is currently
and is
measured
According currently in six
to Carneiro-da-Cunha main
measured in[22],topics: Focus,
six performancePerspective,
main topics: measurement Scope, Amplitude,
has evolved
Focus, Perspective, Nature,
Scope,over and Accountability.
time and Nature,
Amplitude, is and
In Figure 14,
currently measured inthe
Accountability. sixresults
In mainoftopics:
Figure thethe
14, BPresults
articles
Focus,ofarethesummarized
Perspective, in these
Scope,
BP articles are six topics.
Amplitude,
summarized inNature,
these sixand
topics.
Accountability. In Figure 14, the results of the BP articles are summarized in these six topics.

14. Evolution of organizational

Figure 14. organizational performance
performance measurement.
measurement. Advanced variable 6.
6. Source:
Figure 14. Evolution of organizational
Research Data (2017). performance measurement. Advanced variable 6. Source:
Research Data (2017).
4.3. Research Opportunities and Contributions to the Advancement of the Area
4.3. Research Opportunities and Contributions to the Advancement of the Area
From the theoretical contribution of the PE of RL, several aspects are cited in an isolated way in
From the
the theoretical contribution
field of operation of and
control the PE of RL, several
management. Theaspects are discussion
constant cited in an of
isolated way in
the theme is advancing
the field oftowards
operationthecontrol andconsolidated
already management.theory
The constant discussionevaluation.
of performance of the theme is advancing
However, of the role of
towards the alreadythat
elements consolidated theory
bases PE, some haveofnot
performance evaluation.
yet been addressed in theHowever,
literature of reverse
the rolelogistics
of and/or
elements that bases
have not PE,
beensome haveinnot
applied yetstudies
case been addressed
to validateinthe
thetheoretical
literature of reverse logistics and/or
ones.
Sustainability 2019, 11, 5291 14 of 17

4.3. Research Opportunities and Contributions to the Advancement of the Area

From the theoretical contribution of the PE of RL, several aspects are cited in an isolated way in
the field of operation control and management. The constant discussion of the theme is advancing
towards the already consolidated theory of performance evaluation. However, of the role of elements
that bases PE, some have not yet been addressed in the literature of reverse logistics and/or have not
been applied in case studies to validate the theoretical ones.
In this way, Figure 15 aims to summarize the elements of the theoretical reference and of the
bibliometric analysis that should be presented in the reverse logistics performance evaluation.14 of 17
Sustainability 2019, 11, x FOR PEER REVIEW

Figure
Figure 15. Elements
15. Elements of performance
of performance evaluation
evaluation for reverse
for reverse logistics.
logistics. Source:
Source: Prepared
Prepared by authors
by the the authors
(2018).
(2018).
In the outer loop of the figure, six key factors that should be addressed on the studies of RL
In the outer
are exposed. loop of
Among thethe
keyfigure,
factorssixfound
key factors
in thisthat should
paper be addressed
as necessary on the
for the studies of RL
development of are
PE
exposed.inAmong
metrics RL, we the key factors
highlight found in factor
the “process” this paper as necessary
in which for the development
the operational aspects of RL of arePE metrics
included.
in RL, operational
These we highlightaspects
the “process”
were also factor in whichinthe
highlighted theoperational
work of [6]aspects of RL for
as necessary are further
included. These
research.
operational
Indeed, aspectsare
operations werethealso highlighted
constituent basisin
ofthe
RL,work of [6]
but need toas
benecessary
analyzed in fortheir
further research.
broadest Indeed,
possibilities
operations
and also in are the constituent
relation basis of RL, as
to their performance, butproposed
need to be inanalyzed
this paper. in their broadest
The other key possibilities
factors listedandin
also in 15
Figure relation to their
represent performance,in
a breakthrough asthe
proposed
field ofin RLthis paper.
study by The otherimportant
unifying key factorsand listed in Figure
measurable
15 represent
aspects a breakthrough
for analyzing in the field
RL performance ofone
into RL study
single by unifying important and measurable aspects
framework.
for analyzing RL performance
The internal elements of Figureinto one15single
present framework.
eight measurement aspects that should guide the
The internal
evaluation elementsinoforder
of performance Figure 15 present
to satisfy eight measurement
the aspects aspects that
of RL. It is noteworthy thatshould guide the
many studies on
evaluation
RL of performance
that address the issue of in order to
strategy satisfy [6]
involved thein aspects of RL.the
evaluating It RL
is noteworthy that many
practices. Based on thestudies
theory on
of
RL that
PE, address
a broader theshows
view issue of strategy
other aspectinvolved [6] in evaluating
to be measured the RL practices.
in RL performance, such as Based on the theory
the satisfaction of
of PE, a broader
stakeholders, view
also shows
pointed outother aspect
in [37] basedto of
beconsumer
measuredsatisfaction,
in RL performance, such as
the evolution of the satisfaction
organizational
of stakeholders,
learning, also pointed
and the possibility out in [37]
of adaptation to thebased of and
internal consumer
externalsatisfaction,
environments. the evolution of
organizational learning, and the possibility of adaptation to the internal and external environments.
5. Conclusions
5. Conclusions
Facing the challenges presented to reverse logistics managers in recent years, as government
regulations, the pursuit
Facing the for innovation
challenges presented to and new forms
reverse of business,
logistics managersthe in environmental
recent years, asawareness
governmentfor
management of generated
regulations, the waste,
pursuit for and theand
innovation increasing needof
new forms forbusiness,
the reusethe
of materials, these managers
environmental awarenesshave
for
begun to seek improvement in their forms of management. At this point, the performance
management of generated waste, and the increasing need for the reuse of materials, these managersevaluation is
have begun to seek improvement in their forms of management. At this point, the performance
evaluation is configured as a tool that provides information to support management activities.
However, the literature still presents a low volume of studies in this subject.
The selection of the BP, performed according to the delimitations imposed by the researchers,
and the methodology proposed by ProKnow-C, allowed the identification of 21 pieces of research
Sustainability 2019, 11, 5291 15 of 17

configured as a tool that provides information to support management activities. However, the literature
still presents a low volume of studies in this subject.
The selection of the BP, performed according to the delimitations imposed by the researchers,
and the methodology proposed by ProKnow-C, allowed the identification of 21 pieces of research that
correspond to the searched literature: “Evaluation of Performance of Reverse Logistics”. The analysis
of the BP allowed the verification of how the reverse logistics area is related to performance evaluation.
Based on the basic and advanced analyses, prominent authors are highlighted as Sarkis, Zhu
and Lai, who corroborate each other in publications regarding the issue of logistics management.
Their publications have high impact, given by a high number of citations in the references of articles
in the BP. Another prominent author is Govindan, with five articles in the BP and seven citations
in references in the BP, which presents a research focus on reverse logistics performance based on
mathematical models in order to establish references and control metrics.
In general, it was noted that the articles performed isolated logistics performance evaluations,
which did not follow a standard of what was being measured, making it clear that there is no
consolidated theoretical contribution of the indicators. The authors did not observe a structured
process of PE that would serve as a subsidy to the practice of management by logistics managers.
On the other hand, it can be seen that the articles which are in line with the evolution of the performance
evaluation proposed by Carneiro-da-Cunha [22] and are largely aligned with Neely [8], have metric
results that relate to the environment and are compared to other performances.
Finally, it should be emphasized that the most used approaches were based on specific case studies
and case studies for the composition of the indicators, so that they do not follow a pattern of related
metrics in order to clearly obtain the result of actions within an organization, which emphasizes a
theoretical contribution gap for a reverse logistics performance evaluation model.
The theoretical framework of elements that should base the PE of RL is a contribution of this paper,
but has to be detailed in measurable metrics to be applied in the real case of RL. Also, the authors point
out the need in future research of some empirical and quantitative research in the field of RL based on
the theories that have already been developed in the academic field.
This study, however, presents some limitations: (i) The search procedure for BP formation
considered only the periodicals made available in the CAPES portal that were in English; (ii) only
six databases were searched, so that the entire academic universe was not considered; and (iii) the
work identifies studies that are references in the area, but did not find a model of PE of RL. Based
on the analysis performed in the literature, the development of a structured performance evaluation
tool that integrates the indicators and supports the management of reverse logistics is suggested for
future investigation.

Author Contributions: Conceptualization, D.V.; Data analysis, T.G.B. and M.A.M.; Formal analysis, D.V.
and R.U.G.d.R.; Methodology, T.G.B. and N.C.F.; Writing, D.V. and R.U.G.d.R.; Writing—review & editing,
N.C.F. and T.G.B.
Funding: This research is partially funded by CAPES (Brazilian Higher Education Personnel Improvement
Coordination). and CNPQ (Brazilian National Council for Scientific and Technological Development).
Acknowledgments: Acknowledgment to the Post Graduate Program in Production Engineering at the Federal
University of Santa Catarina (UFSC).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Nikolau, I.E.; Evangelinos, K.I.; Allan, S. A reverse logistics social responsibility evaluation framework based
on the triple bottom line approach. J. Clean. Prod. 2011, 56, 173–184. [CrossRef]
2. Olugu, E.U.; Wong, K.Y.; Shaharoun, A.M. development of key performance measures for the automobile
green supply chain. Resour. Conserv. Recycl. 2010, 55, 567–579. [CrossRef]
3. Fleischmann, M. Quantitative Models for Reverse Logistics: Lecture Notes in Economics and Mathematical Systems;
Springer: Berlin, Germany, 2001.
Sustainability 2019, 11, 5291 16 of 17

4. Gungor, A.; Gupta, S.M. Issues in environmentally conscious manufacturing and product recovery: A survey.
Comput. Ind. Eng. 1999, 36, 811–853. [CrossRef]
5. Björklund, M.; Martinsen, U.; Abrahamsson, M. Performance measurements in the greening of supply chains.
Supply Chain Manag. 2012, 17, 29–39. [CrossRef]
6. Russo, I.; Confente, I.; Gligor, D.; Cobelli, N. A roadmap for applying qualitative comparative analysis in
supply chain research: The reverse supply chain case. Int. J. Phys. Distrib. Logist. Manag. 2019, 49, 99–120.
[CrossRef]
7. Beamon, M.B. Measuring supply chain performance. Int. J. Oper. Prod. Manag. 1999, 19, 275–292. [CrossRef]
8. Neely, A.; Gregory, M.; Platts, K. Performance measurement system design—A literature review and research
agenda. Int. J. Oper. Prod. Manag. 1995, 15, 80–116. [CrossRef]
9. Govindan, K.; Sarkis, J.; Palaniappan, M. An analytic network process-based multicriteria decision making
model for a reverse supply chain. Int. J. Adv. Manuf. Technol. 2013, 68, 863–880. [CrossRef]
10. Rogers, D.S.; Tibben-Lembke, R.S. Going Backwards: Reverse Logistics Trends and Practices; Logistics Executive
Council: Pittsburgh, PA, USA, 1998.
11. Dutra, A.; Ripool-Feliu, V.M.R.; Fillol, A.G.; Ensslin, S.R.; Ensslin, L. The construction of knowledge from the
scientific literature about the theme seaport performance evaluation. Int. J. Product. Perform. Manag. 2015,
64, 243–269. [CrossRef]
12. Tasca, J.E.; Ensslin, L.; Ensslin, S.R.; Alves, M.B.M. An approach for selecting a theoretical framework for the
evaluation of training programs. J. Eur. Ind. Train. 2010, 34, 631–655. [CrossRef]
13. Richardson, R.J. Pesquisa Social: Métodos e Técnicas, 3rd ed.; São Paulo: Atlas, France, 1999.
14. Creswell, J.W.; Creswell, J.D. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches;
Sage publications: thousand oaks, CA, USA, 2017.
15. Valmorbida, S.M.I.; Ensslin, L. Construção de conhecimento sobre avaliação de desempenho para gestão
organizacional: Uma investigação nas pesquisas científicas internacionais. Rev. Contemp. Contab. 2016, 13,
123–148. [CrossRef]
16. Ensslin, L.; Ensslin, S.R.; Pinto, H.M. Processo de investigação e análise bibliométrica: Avaliação da qualidade
dos serviços bancários. Rev. Adm. Contemp. 2013, 17, 325–349. [CrossRef]
17. Lacerda, R.T.O.; Ensslin, L.; Ensslin, S.R. Uma análise bibliométrica da literatura sobre estratégia e avaliação
de desempenho. Gestão Produção 2012, 19, 59–78. [CrossRef]
18. Rosa, F.S.; Ensslin, S.R.; Ensslin, L.; Lunkes, R.J. Management environmental disclosure: A constructivist
case. Manag. Decis. 2012, 50, 1117–1136. [CrossRef]
19. Shaik, M.; Abdul-Kader, W. Transportation in reverse logistics enterprise: A comprehensive performance
measurement methodology. Prod. Plan. Control 2013, 24, 495–510. [CrossRef]
20. Silva, R.V.; Ensslin, S.R.; Ripoll-Feliu, V.M.; Soler, C.C. E-government and public accounting information:
Bibliometric and systemic analysis. Int. Res. J. Financ. Econ. 2014, 1, 76–91.
21. Ensslin, S.R.; Ensslin, L.; Imlau, J.M.; Chaves, L.C. Processo de mapeamento das publicações Científicas de
um tema: Portfólio bibliográfico e análise bibliométrica sobre avaliação de desempenho de cooperativas de
produção agropecuária. Rev. Econ. Sociol. Rural 2014, 52, 587–608. [CrossRef]
22. Carneiro-Da-Cunha, J.A.; Hourneaux, F.J.; Corrêa, H.L. Evolution and chronology of the organisational
performance measurement field. Int. J. Bus. Perform. Manag. 2016, 17, 223–240. [CrossRef]
23. Lacerda, R.T.O.; Ensslin, L.; Ensslin, S.R.; Dutra, A. A Constructivist Approach to Manage Business Process
as a Dynamic Capability. Knowl. Process. Manag. 2014, 21, 54–66. [CrossRef]
24. Shaik, M.; Abdul-Kader, W. A Comprehensive Performance Measurement Framework for Reverse Logistics
Enterprise. Available online: https://www.researchgate.net/publication/292836790_A_comprehensive_
performance_measurement_framework_for_reverse_logistics_enterprise (accessed on 19 September 2019).
25. Espino-Rodríguez, T.F.; Gil-Padilla, A.M. The impact of outsourcing strategies on information systems
capabilities in the hotel industry. Serv. Ind. J. 2007, 27, 757–777. [CrossRef]
26. Barker, T.J.; Zabinsky, Z.B. A multicriteria decision making model for reverse logistics using analytical
hierarchy process. Ómega 2010, 39, 558–573. [CrossRef]
27. Wang, B.; Sun, L. A review of reverse logistics. Appl. Sci. 2005, 7, 16–29.
28. Fleischmann, M.; Bloemhof-Ruwaard, J.M.; Dekker, R.; van der Laan, A.E.; van Nunen, J.A.E.E.;
van Wassenhove, L.N. Quantitative models for reverse logistics: A review. Eur. J. Oper. Res. 1997,
103, 1–17. [CrossRef]
Sustainability 2019, 11, 5291 17 of 17

29. De Brito, M.P.; Dekker, R. A Framework for Reverse Logistics, ERIM Report Series Research in Management;
Erasmus University Rotterdam: Rotterdam, The Netherlands, 2003.
30. Rogers, D.S.; Melamed, B.; Lembke, R.S. Modeling and analysis of reverse logistics. J. Bus. Logist. 2012, 33,
107–117. [CrossRef]
31. Dowlatshahi, S. A strategic framework for the design and implementation of remanufacturing operations in
reverse logistics. Int. J. Prod. Res. 2005, 43, 3455–3480. [CrossRef]
32. Meade, L.; Sarkis, J.; Presley, A. The theory and practice of reverse logistics. Int. J. Logist. Syst. Manag. 2007,
3, 56–84. [CrossRef]
33. Rubio, S.; Chamorro, A.; Miranda, F.J. Characteristics of the research on reverse logistics (1995–2005). Int. J.
Prod. Res. 2008, 46, 1099–1120. [CrossRef]
34. Agrawal, S.; Singh, R.K.; Murtaza, Q. Triple bottom line performance evaluation of reverse logistics. Compet.
Rev. 2016, 26, 289–310. [CrossRef]
35. Govindan, K.; Paam, P.; Abtahi, A.R. A fuzzy multi-objective optimization model for sustainable reverse
logistics network design. Ecol. Indic. 2016, 67, 753–768. [CrossRef]
36. Espino-Rodríguez, T.F.; Rodríguez-Díaz, M. Determining the core activities in the order fulfillment process:
An empirical application. Bus. Process Manag. J. 2014, 20, 2–24. [CrossRef]
37. Wang, J.J.; Chen, H.; Rogers, D.S.; Ellram, L.M.; Grawe, S.J. A bibliometric analysis of reverse logistics
research (1992–2015) and opportunities for future research. Int. J. Phys. Distrib. Logist. Manag. 2017, 47,
666–687. [CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).