Journal of Humanitarian Logistics and Supply Chain Management

Measuring while moving (humanitarian supply chain performance measurement –

status of research and current practice)
Charles D'Haene Sara Verlinde Cathy Macharis
Article information:
To cite this document:
Charles D'Haene Sara Verlinde Cathy Macharis , (2015),"Measuring while moving (humanitarian
supply chain performance measurement – status of research and current practice)", Journal of
Humanitarian Logistics and Supply Chain Management, Vol. 5 Iss 2 pp. 146 - 161
Permanent link to this document:
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

http://dx.doi.org/10.1108/JHLSCM-04-2013-0016
Downloaded on: 24 February 2016, At: 02:13 (PT)
References: this document contains references to 41 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 280 times since 2015*
Users who downloaded this article also downloaded:
Umberto Peretti, Peter Tatham, Yong Wu, Fabio Sgarbossa, (2015),"Reverse logistics in humanitarian
operations: challenges and opportunities", Journal of Humanitarian Logistics and Supply Chain
Management, Vol. 5 Iss 2 pp. 253-274 http://dx.doi.org/10.1108/JHLSCM-07-2014-0026
Cécile L'Hermitte, Marcus Bowles, Peter Tatham, Ben Brooks, (2015),"An integrated approach to
agility in humanitarian logistics", Journal of Humanitarian Logistics and Supply Chain Management,
Vol. 5 Iss 2 pp. 209-233 http://dx.doi.org/10.1108/JHLSCM-04-2014-0016
Jae-Dong Hong, Ki-Young Jeong, Keli Feng, (2015),"Emergency relief supply chain design and trade-
off analysis", Journal of Humanitarian Logistics and Supply Chain Management, Vol. 5 Iss 2 pp.
162-187 http://dx.doi.org/10.1108/JHLSCM-05-2014-0019

Access to this document was granted through an Emerald subscription provided by emerald-
srm:126209 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

*Related content and download information correct at time of download.


JHLSCM
5,2
146
Received 22 April 2013
Revised 4 September 2013
30 January 2014
11 April 2014
Accepted 14 April 2014
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)
Journal of Humanitarian Logistics
and Supply Chain Management
Vol. 5 No. 2, 2015
pp. 146-161
© Emerald Group Publishing Limited
2042-6747
DOI 10.1108/JHLSCM-04-2013-0016
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/2042-6747.htm
Measuring while moving
(humanitarian supply chain
performance measurement
– status of research and
current practice)
Charles D’Haene
Médecins Sans Frontières, Brussels, Belgium, and
Sara Verlinde and Cathy Macharis
Research Group MOBI (Mobility, Logistics and Automotive Technology),
Vrije Universiteit Brussel, Brussels, Belgium
Abstract
Purpose – Raising standards are leading humanitarians to become increasingly interested in measuring
the performance of their supply chain. A few researchers have addressed this topic, building on
classical measurement theories and trying to identify the salient features of the humanitarian sector.
This young body of literature must now be tested against current practice. The paper aims to discuss
these issues.
Design/methodology/approach – The practitioners’ perspective is explored through a multiple
case study, based on qualitative evidence, within three major humanitarian organizations. Results are
discussed using a situation-actor-process – learning-action-performance model of inquiry. Preceding
this investigation, a literature review delves into research on humanitarian supply chain performance
measurement and presents the most relevant pieces in a general framework.
Findings – Humanitarians have elaborated measures whose essential focus is laid on service level.
The more comprehensive approach advocated by researchers has received some consideration but
is still poorly implemented. An issue that is given more priority by humanitarians is the accelerated
supply chain integration they are going through.
Originality/value – In addition to an unprecedented literature review, this paper offers a comparative
study of humanitarian organizations’ practices in the field of supply chain performance measurement.
Humanitarian procurement centres, virtually ignored in literature until now, are scrutinized at a turning
point of their evolution.
Keywords Performance measurement, Supply chain integration, Humanitarian logistics,
Humanitarian supply chain
Paper type Research paper
1. Introduction
Various loose estimations of the recent growth in the humanitarian sector exist.
Kent (2004) considered that the number of humanitarian organizations, including
governmental, intergovernmental (IGO) and non-governmental (NGO) ones, increased
from 280 in the mid-1980s to four times this figure 20 years later. Similarly, some
experts estimated in 2009 that the number of aid workers increased by approximately
6 percent per year over ten years (ALNAP, 2010). And the Financial Tracking Service
of the United Nations Office for the Coordination of humanitarian Affairs (OCHA)
showed that humanitarian contributions rose from approximately 2 billion dollars
in 2000 to 16 billion dollars in 2010 (OCHA, 2011).
 Spurred by this growth, many humanitarian organizations began to seek more Humanitarian
professionalism in their actions. Standards detailed in guidelines started to spread; supply chain
initiatives aiming at unifying good practices across the profession were launched; projects
to enhance accountability surfaced. Among the most notable, the Active Learning
performance
Network for Accountability and Performance in Humanitarian Action (ALNAP), People in
Aid, Humanitarian Accountability Partnership (HAP) and The Humanitarian Charter and
Minimum Standards in Disaster Response, commonly referred to as the Sphere Handbook. 147
As another sign of professionalization, the interest for humanitarian studies strengthened
significantly, leading many universities to launch programs in this field. On the Global
Humanitarian Studies Index of The University of Columbia, 81 institutions can be found,
among which the most famous, offering master degrees, doctoral study and short courses
in humanitarian studies (Salomons, 2012).
In the context of accelerated development that the humanitarian sector has been facing
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

during the last decades, one of the areas considerably pressed by the professionalization
urge is supply chain. Prerequisite for humanitarians to professionalize in this critical field
was first to extend once and for all their perspective from the strict logistics view applied
in the past to an advanced supply chain management approach. But not even ten years
ago, academics still identified many obstacles on the way. Among them: the little
consideration for this activity, the lack of professional staff and of institutional learning,
inadequate technology and limited collaboration between organizations (Thomas and
Kopczak, 2005).
The comparison with developments in the corporate world is natural. But humanitarian
supply chain is a distinct field of study, even though the final goal is still about getting the
right item at the right time at the right place for the right price, that is the supply chain four
R’s (Tomasini and Van Wassenhove, 2009). Distinctive features of humanitarian operations
have been identified in various areas such as the nature of their strategic goals and demand
characteristics (Beamon and Balcik, 2008). Typical constraints they are submitted to
are also often put forward, such as the implication of multiple stakeholders, the limited
resources available, the impossibility to act in total independence and most notably the
high levels of uncertainty and urgency prevailing in the humanitarian environment.
Among the three A’s conceptualized by Lee in 2004 (agility, adaptability and alignment),
the fundamental notion that applies to humanitarian supply chains is definitely its
agility, the faculty to quickly respond to short-term changes in demand or supply and
to handle external disruptions smoothly, as many articles have suggested (Tomasini
and Van Wassenhove, 2009; Oloruntoba and Gray, 2006; Scholten et al., 2010; Kovács
and Tatham, 2009).
Today, as humanitarians are professionalizing, they are increasingly confronted
with the need to assess the performance of their operations. Besides, as in the rest of
the non-profit sector, greater accountability is now expected from them (Walker and
Russ, 2011). The implementation of performance measurement systems is starting to
be seen as a necessity by many organizations whose survival will often depend on
the extent to which they are willing and able to show that they are “doing good while
doing well,” as formulated by Kanter and Summers (1987).
In summary, change has been steadily blowing on the structures responsible for
the provision of goods to humanitarian missions on the field since the early 2000s.
During a necessary process of professionalization, humanitarian organizations have
come to familiarize with supply chain management, a discipline that the private sector
embraced before them. In the same move, humanitarians have also increasingly felt
the urge to measure the performance of their supply activities. Rather timidly, the
 JHLSCM former evolution has received an increasing coverage during the last decade. Interest
5,2 in the latter is even more scant and the body of literature on the subject is limited to a
very few pieces of research. Today, before designing new measurement frameworks
for the humanitarian context, there is a need to examine performance measurement
as it is actually handled in real life by the different types of organizations. This
investigation is critical to ensure a proper orientation of future research.
148
2. Literature review
As few as they are, performance measurement theories elaborated by academia
for the humanitarian supply chain are certainly worth being examined. When it
comes to informing policy and practice in any discipline, reviewing the related
literature is essential for both the academic and the practitioner communities
(Tranfield et al., 2003). As a first step, material was collected by questioning
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

scientific databases. After examination of the content, a classification system was

elaborated. As a third step, a framework was designed and used to categorize the
literature collected. And finally, the literature review was developed, based on
the categorization.

2.1 Available material

Material collection was performed by questioning scientific databases. Emerald, Google
Scholar and Web of Knowledge electronic databases were consulted, using the following
keywords and related terms in combination: humanitarian, supply chain, performance
and measurement. No restriction was made on the time period of publication.

2.2 Categories selection

Literature on humanitarian supply chain performance measurement is still in its infancy.
The first source is found in 2002 and it is only in the second half of its existence that
studies start to accumulate slowly. It is therefore not surprising that no chronological
trend can yet be identified to classify literature on the topic.
To categorize the available pieces of research, other aspects must be considered.
The first characteristic that stands out is that researchers have focussed until now on
actual measurement frameworks. Other issues such as attitudes toward performance
measurement in the humanitarian sector have been approached but only in a secondary
way. Also striking is that the majority of researchers have opted for single notorious
solutions used in the profit sector such as the balanced scorecard, the SCOR model
or Beamon’s framework. The other few authors who have addressed the subject
have instead chosen not to concentrate on a single system but rather to derive their
contribution from multiple sources. It can be inferred from these first observations that
referential framework use is a first rational base to classify literature on humanitarian
supply chain performance.
The other notable distinction that can be made when comparing available papers lies
in the methodology applied. While most authors include case studies in their research,
approaches can be divided in two categories. The first group of researchers adopt a
purely descriptive approach. They typically choose one humanitarian organization; they
analyze its structure and processes; and they describe their performance measurement
practices in a most detailed manner. Some recommendations to practitioners are
sometimes included, but they do not constitute the core of this research type. On the
contrary, the second group clearly seeks to offer original solutions to humanitarian
 professionals. These researchers choose to address the topic more conceptually Humanitarian
and they develop actual frameworks, using case studies to support their model. supply chain
This methodology-based distinction was selected as a second axis to categorize the
available literature.
performance

2.3 Literature classification

In the following table, the collected material is classified according to the categories 149
identified in the previous section Table I.

2.4 Literature review

As previously noted, many papers on humanitarian supply chain performance
measurement take as starting point well-known frameworks initially developed for the
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

business sector. For now, there is no unanimity on a single approach for humanitarian
supply chain. But it must be noted that a particular tool is selected more often than any
other: the balanced scorecard. As a reminder, Kaplan and Norton remarked 20 years
ago that traditional financial performance measures were not as satisfactory as they
were during the industrial era. In the wake of this observation, they developed
the balanced scorecard, a set of measures giving managers a fast and comprehensive
view of the business, putting strategy and vision, not control at the center. Financial
measures, reporting the results of actions already taken, were complemented with
operational measures considered as the drivers of future financial performance:
customer satisfaction, internal processes and innovation and improvement activities
(Kaplan and Norton, 1992). In the oldest piece of research that was found on
humanitarian supply chain performance measurement, the master thesis of Guyoton
and Muirheid (2002), the utilization of the balanced scorecard is recommended further
to an investigation carried out at the International Federation of Red Cross and Red
Crescent Societies (IFRC). In a paper examining the relevance of business managerial
tools for humanitarian supply chain professionals, McLachlin et al. (2009) develop a
case study within the Mennonite Central Committee (MCC). They chose the balanced
scorecard as an example of a performance measurement tool that could be transposed
in the humanitarian context. Moe et al. (2007) select the same tool but they approach
it in a specific way. After analyzing its relevance for project management, they adapt
it to natural disaster project management and they apply it to a real flood disaster in
Hat Yai Municipality, Thailand. Also in favor of the balanced scorecard, Schulz and
Heigh (2009) describe its successful implementation, under the name of “Development

Approach
Referential framework Descriptive Conceptual

Yes Balanced scorecard Guyoton and Muirheid (2002), Moe et al. (2007), De Leeuw
Kaplan and Norton (1992) McLachlin et al. (2009), Schulz (2010)
and Heigh (2009)
SCOR Model Bölsche (2012)
Stewart (1997)
Beamon’s framework Beamon and Balcik (2008),
Beamon (1999) Blecken et al. (2009) Table I.
No Van der Laan et al. (2009) Davidson (2006), Tatham and Literature review
Hughes (2011) classification
 JHLSCM Indicator Tool,” at the IFRC, winner of the coveted European Supply Chain Excellence
5,2 Award in 2006. Building further on the application of the balanced scorecard in the
humanitarian sector, De Leeuw (2010) takes into account the criticism it received and the
consequent update that was made by Kaplan and Norton (2000), namely the introduction of
the “strategy map” to better link intangibles with strategy and performance. Transferring
this concept from the business to the humanitarian environment, the author develops the
150 “mission map,” meant to incite humanitarian organizations to define measurable and
mission-oriented goals and to assess progress toward these goals (De Leeuw, 2010).
The balanced scorecard is the most frequently tested tool but other perspectives
worth noting have been put forward. One of these solutions is the SCOR model,
originally inspired by the work of Stewart (1997). This tool, organized around five
basic business processes of an integrated supply chain (plan, source, make, deliver
and return), is essentially conceived as a supply chain process reference model.
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

But it also includes a performance measurement pillar which is found relevant

by some authors interested in humanitarian supply chains. This is the case of
Bölsche (2012). The author, supportive of the process-oriented perspective of the
model, examines how an organization such as the World Food Programme (WFP)
can be assessed against the different levels of business processes provided by the
framework.
Another solution is chosen by Beamon and Balcik (2008) who adapt to the humanitarian
context the framework initially elaborated by Beamon (1999) for commercial supply
chains and based on the distinction between resource, output and flexibility performance
metrics. Each type being considered as critical both in the humanitarian and the profit
sector, the framework is concluded to be fully adaptable. The approach of Beamon (1999)
is also adopted by Blecken et al. (2009). After developing a process reference model
for humanitarian supply chains, largely inspired by the SCOR model, this conceptual paper
focusses on performance measurement. The authors select Beamon’s framework which
they slightly adapt in two ways. First, the category flexibility is extended to responsiveness,
since it is assumed the latter can be used generically to include both flexibility and
adaptability measures. And second, a set of indicators is introduced to differentiate disaster
response phases.
Rather than focussing on a single reference, some authors propose original ideas
stemming from a large variety of sources. This is the case of Van der Laan et al. (2009)
who identify, through a literature review, the necessary conditions for a successful
implementation of a performance measurement system in the humanitarian context.
The fulfillment of these conditions is then evaluated with the Dutch section of Médecins
Sans Frontières (MSF). This is also the case of Davidson (2006), who discards two
common solutions before developing a new framework, which is consequently applied to
a set of operations performed by the IFRC. The SCOR model is described as appropriate
for businesses operating with pre-defined supply chains and distribution networks but as
being too rigid to adapt to the variability of humanitarian operations. The balanced
scorecard is rejected because of the difficulty to fit the IFRC’s priorities within its four
quadrants and because it is considered to be best used for change management purposes
rather than to simply measure performance. Eventually, other authors consider that
measures of effectiveness of humanitarian supply chains have been sufficiently covered
by literature and that now investigations need to be primarily turned toward measures
of impact. According to their view, the real challenge for humanitarian professionals
will be in the future to adopt metrics capable of capturing the viewpoint of the “final
customer,” the relief aid recipient (Tatham and Hughes, 2011).
 3. Methodology Humanitarian
The approach chosen to address the practitioners’ perspective was to develop a multiple supply chain
case study within three major humanitarian organizations. The case study appeared
indeed to be the most relevant research strategy. Case studies attempt to examine
performance
contemporary phenomena in their real-life context, particularly when the boundaries
between phenomenon and context are not clearly evident (Yin, 1981). To support the case
study, it was chosen to use qualitative evidence. As opposed to quantitative methods, this 151
type of evidence is particularly effective in providing the peripheral vision needed at
the early stages of inquiries (Sofaer, 1999). Considered as a core discipline of the
qualitative researcher (Patton, 1980), the open-ended questions approach was selected to
explore as overtly as possible a largely unknown territory and to let unexpected material
surface. The partial results obtained shall be seen as a necessary step that will help paving
the way for further systematic research.
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

The study was based on face-to-face interviews with three humanitarian top managers.
Actual meetings were complemented by remote exchanges to address specific questions
that needed to be examined in more depth. The interviewees were first asked to provide
an update on supply chain management developments recently undergone by the
humanitarian sector. They were then invited to express themselves on themes that
appeared to be frequently covered either in general or humanitarian-oriented literature on
performance measurement. Among the major subjects addressed, the following should
be highlighted: the interest of the three organizations for established frameworks, the
comparison with other sectors’ practices, the definition of measure levels and categories,
the balance between different types of measures, the process of data collection, behavioral
aspects of performance measurement, performance measurement in the context of supply
chain integration and the delimitation of the measurement system’s perimeter. After the
introductory update, the interviews’ account is structured around three sections: the first
one focusses on current practice, the second on the main difficulties encountered and the
third on the critical question of prioritization.
This account is followed by a discussion based on the situation-actor-process-
learning-action-performance (SAP-LAP) model of inquiry developed by Sushil (2000).
This model was created in answer to the increasing need for flexible and systemic
modes of managerial analysis. Used to generate a wide variety of inquiry models,
the framework has been used in the field of supply chain by various researchers such
as Arshinder and Desmukh (2007), Charan (2012) and Lijo and Ramesh (2012).
Basically, the model is composed of three dimensions: a situation to be managed,
an actor or group of actors who have to deal with it and a process or set of processes
related to the situation. Adapted to the topic of humanitarian supply chain performance
measurement, questions have been organized as follows:
(1) SAP
• Situation. How did performance measurement progress to reach the
current situation, how is it developed today, and what is expected to
happen in the future?
• Actors. How is performance measurement perceived by the different actors
involved; what capabilities are exhibited and in which domain is freedom of
choice available?
• Process. Which performance measurement processes are in place? Why are
those processes implemented? What can be changed?
 JHLSCM (2) LAP
5,2 • Learning. What are the key issues related to the situation, the actors
and the processes?
• Action. What should be done to improve the situation, the actors
and the processes?
152 • Performance. What will be the impact on the situation, the actors
and the processes?
The case studies, which started in the course of 2012, were conducted in structures known
to be evolving toward greater end-to-end responsibility and showing a comparatively high
level of maturity in the area of performance measurement. They were also set within the
three main types of humanitarian organizations: NGO, IGO and hybrid (private but whose
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

activities are mandated by governments and possessing international legal personality).

Interviews of top managers were carried out in the European supply centres of those three
types of organizations.

4. Case study
4.1 Update
The selected institutions are amongst the prominent humanitarian organizations
which have been addressing supply chain management very actively in recent
years. Before discovering their performance measurement practices, it will be shown
how they have been dealing with the obstacles mentioned earlier (Thomas and
Kopczak, 2005).
According to the three organizations, there has been a clear improvement in the
way supply chain is perceived by humanitarians, even though there is still some room
for progression. With regards to the lack of professional staff, they reckon that
the expansion of academic programs in the field of supply chain during the 2000s has
had a significant impact on recruitment quality.
When it comes to inadequate technology as an obstacle to development, it is a more
complex and critical issue. The hybrid organization, for example is struggling to find
systems able to operate in low connectivity environment or to support the large panel
of activities they manage. For several years, the organization has been tendering for an
integrated IT system but it still has not found a convenient tool. The ones on the market
are deemed either too vertical or not adapted. From the organization’s viewpoint,
the referential tool for the management of humanitarian supply chains is not born yet.
It is all the more problematic in face of another important parameter put forward by
the organization: the affordability of such a system. In the NGO, the feeling is that,
in general terms, supply chain technology has evolved tremendously over the years,
while basics remain unchanged: it is still about information, financial and physical
flows aiming at the satisfaction of clients or beneficiaries. Distinction must, however, be
made between the technologies used by the NGO at mission level and at headquarters
or hubs, even though connectivity has progressed a lot on the field. But it is clear
for the organization that new IT systems will someday have to replace the ones they
currently use, which can absorb loads of data but are extremely weak when it comes
to processing this data into intelligent reporting needed for high reactivity. In the
IGO, a big step has been made recently: since the January 1, 2012, the IT system
which has been used at headquarters for ten years is also operational at country
level. This implementation is seen as a great challenge by the organization. It is
 indeed no small change to have all supply staff starting to work globally on a Humanitarian
single standard system. Unfortunately, the organization must recognize that further supply chain
improvements need to be made in the area of training to address this challenge in the
most appropriate manner.
performance
Regarding the last obstacle identified by researchers, namely the lack of collaboration
between organizations, opinions are mixed. According to the hybrid organization,
for example, there is less collaboration in the business world and this sector is ahead 153
as far as supply chain management is concerned. The idea of partnerships with external
players, mainly the private sector and universities, seems to stir up more interest. In the
IGO, these kinds of initiatives are favored as a matter of fact. Just to name a few
examples, a famous American university had researchers working on supply chain
design for its supply centre and a leading express logistics company was helping
to implement improvement initiatives. In the hybrid organization, partnerships are also
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

common practice. They are mainly carried out with notorious European management
universities. Few contacts were also made with the private sector for benchmarking
purposes, but no real partnerships were deliberately launched. In the NGO, the main
collaboration is held with a European university and in-kind donations consulting
services are links with the private world that are currently not operated but that will be
investigated in the future.

4.2 Current practice

While removing obstacles toward advanced supply chain management, the three
organizations have been putting increasing efforts to measure their performance.
The hybrid organization has elaborated metrics that are found adequate by the
management, but that are not centralized for the time being. A unified system
is currently being developed internally and expected to be operational in the course of
2015. It shall not only consolidate the supply centre’s indicators, but it will actually
encompass the whole organization’s supply chain, from suppliers to beneficiaries.
It will be constructed on the basis of three existing systems (hubs and headquarters’
purchasing systems, the organization’s international transport system and the one
used to manage activities on the field). Like the two other organizations, the hybrid
organization handles a large variety of measures reflecting the volume of its activities.
They can range from the number of processed order lines to the overall warehouse
surface, or the number of import and export files treated. But in the field of performance
assessment, the hybrid organization keeps track of three main indicators:
on-time deliveries, adequacy of product with respect to demand, and price. It should
also be noted that satisfaction surveys are used on a regular basis to evaluate
the quality of service provided by the organization to its country offices.
The IGO has built its own system, with a dozen major indicators compiled into a
common framework that classifies them and gives details such as their owner,
source, update frequency, target and baseline. But there are performance measures that
receive particular attention. As part of an internal survey, 31 supply professionals
of the organization were asked to rank the importance of suggested metrics.
The survey revealed that the metrics they used, or would have liked to use, focussed on
inventory quality and accuracy and, more importantly, on service level. Four measures
of the latter category are monitored with special care. The first one is emergency
response, expressed as the percentage of rapid response orders shipped within 72 hours
of sales order release. The second is delivery performance, expressed as the percentage
of all orders delivered at port of entry within agreed upon target arrival date. The third
 JHLSCM also measures delivery performance, but from the supplier side, as the percentage of
5,2 on-time delivery. And the fourth one, again, reflects the delivery performance of
the organization, as the percentage of all orders picked by freight forwarders
being delivered on-time. Along with measures related to delivery time, the following
KPIs are included in the annual management plan: strategic-essential supplies
throughput, forecast accuracy to industry, results of audit reports at mission level,
154 number of joint procurement initiatives with country and regional offices, percentage of
procurement services to governments and partners concerning strategic-essential
supplies, degree of collaboration with sister agencies, number of technical support
requests received, use of market analysis and product innovation.
In the NGO, focus is also laid on service level. It is essentially monitored
through measures related to delivery time and registered claims. With regards to the
first type of measure, the management team follows and sets targets for the respect
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

of ready-to-ship dates announced by the procurement centre to the country offices.

In the same area, it also controls the duration of all processes from the reception of the
demand to the expedition of the goods. After two years of development, process
time can now be calculated accurately, but measures extending to the actual date
of delivery to the field can still not be obtained. This is due to the fact that the IT
infrastructure of the NGO’s supply centre is limited to its physical location and is
not interfaced with country offices. The second type of measure concentrates on the
number and nature of claims received from the field. These two types of performance
indicators are expected to be complemented by others to eventually constitute a general
dashboard. As service level monitoring is now considered to be running, cost control
is next on the agenda.
It must be noted that within the three organizations, virtually no connection is
made with scientific research in the field of performance measurement. No recognized
system such as the balanced scorecard or the SCOR model is currently used. Frameworks
have been, or are planned to be, developed internally without any reference to existing
models. The only application was found in a department of the IGO which used
the SCOR model in the past. This department, mainly in charge of warehouse
operations, stock management, production of emergency kits and international transport,
found this model convenient given the similarity of its own activities with the
manufacturing industry.

4.3 Issues
When it comes to the main difficulties linked to the development of performance
indicators, data collection appears to be the main issue for the three organizations.
It took the NGO several years, only to define the data set that is now used to measure
the respect of lead time. For purchase and financial performance, it will expectedly
also take a long time to obtain a clean data set and to fill in the existing gaps with
readable data. The situation is very similar for the hybrid organization where finding
the right information is considered to be the main problem. In general terms, everybody
in this organization agrees on what needs to be measured. But how indicators need
to be computed is a much more debated question. The IGO’s view on data collection
is much the same. It is assumed that the integrated IT system should make the
process easier. But then, the cultural factor should not be underestimated, according to
the organization. Measuring performance is not a task that is being done instinctively.
The shift from doing to analyzing is not an obvious one. At department level, indicators
that have been aligned with the ones of the IGO as a whole are supposed to serve
 as basis for reporting to the direction, but this procedure is not systematically Humanitarian
observed. According to one of its managers, the organization may not yet be culturally supply chain
ready to have them followed with absolute rigor and discipline.
In the hybrid organization, personnel adhesion to the practice of performance
performance
monitoring is also seen as a sensible question. The opinion is that this can only be
approached through constant awareness-raising. Stress is put internally on the fact
that performance indicators are aimed mainly at finding ways of improvement; that 155
they are meant to be translated into new means or methods and are not solely used
for rewarding high-performing staff members. In this regards, it can be noted that the
organization grants small performance bonuses to best employees. For the NGO’s part,
more than reluctance of individuals to be confronted with their bad performance, it is
the lack of understanding that is striking and that needs to be addressed through
uninterrupted communication.
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

4.4 Priority
Eventually comes the question of prioritization of performance measurement
development in the long term.
In the IGO, traditional focus has been on the definition of needs, budgeting
and planning, and procurement. Current improvements are being made in the fields of
delivery and clearance, inspection, warehousing, distribution and reordering. And the
next area of work that the organization plans to tackle is the utilization by beneficiaries
and monitoring and evaluation. But according to the IGO, if its supply teams succeeded
to link with the last mile distribution, they would enter a zone which is unknown.
This is felt as a big challenge because trying to reach people that are off the radar
would put the organization off the radar too, which makes monitoring a very tough
issue. Until recently, the IGO’s supply centre was accountable up to the port of entry
where country offices were taking over to route the goods to implementing partners.
But the current process of IT integration means that, although delivery accountability
rests with the country office, the supply centre’s oversight will be soon extended over
the whole supply chain, bringing it closer to end-users. This enlargement of the scope
may also entail a shift in the performance indicators of the supply centre, although
enhanced visibility does not mean it will be possible to objectively analyze everything.
It is expected that some crucial aspects will remain in the hands of country offices.
In the NGO, the priority is clearly set on the implementation of an end-to-end supply
chain. In the view of the organization, this integration process, that shall synchronize
main supply chain stakeholders and leverage competencies, will bring the new supply
unit to the doors of projects on the field. Particular attention will be paid to organizing
information flows, recruiting, communicating on the project and most importantly
to designing the supply chain. But methodology is still lacking in this area, according
to the management. IT systems are seen as important parts, but also as nothing more
than tools. Supply chain design means much more for the NGO which recognizes that
it is obviously easier to implement an integrated supply chain when systems
are integrated, although this is not an indispensable condition. Once end-to-end supply
chain is implemented, the organization plans to have performance indicators
thoroughly developed.
For the hybrid organization, performance indicators development is among
the major long-term objectives. First of all, the organization wants to conclude the
strategic project mentioned before which should lead to the adoption of an IT
system able to manage the organization processes. Then, attention will be paid to the
 JHLSCM implementation of the best organizational structure at central, regional and local level.
5,2 And finally, efforts will be put in the construction of performance dashboards
able to monitor the whole supply chain.

5. Discussion
5.1 SAP
156 The last decades’ growth provoked a strong impetus toward professionalism in the
humanitarian world, bringing about a decisive break-through in the field of supply
chain. As regards comments that were made a few years ago about the development
level of humanitarian supply chains (Thomas and Kopczak, 2005), managers from the
three organizations unanimously feel that progress was made on major pain points.
This evolution induced some significant progresses in the practice of supply chain
performance measurement. It must, however, be observed that there is still a lot of room
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

for improvement within the three organizations. Performance measurement systems

are in place and used on a daily basis at various organizational levels. But these
systems, developed with no connection to scientific research, show some weaknesses
that humanitarian actors need to overcome.
When considering these actors, it must be noted that not all of the organizations’
members equally adhere to the practice of performance measurement. Some of them
are indeed showing some cultural hindrance. The interviewees acknowledge that it
will not disappear naturally and that communication efforts must be made to produce
a shift in behavior. It can be assumed that their task is made especially difficult
considering the relative youth of performance measurement in the humanitarian sector.
A lucrative connotation stemming from early conceptions is probably still felt by some
people who find it hard to combine the idea of performance measurement with their
philanthropic mission. But minds are evolving. Recognition has definitely matured
for supply chain practices and roles. And in the three case organizations, managers
are pleased to note that they can now recruit staff with expertise. For certain, these new
employees will play a key role in disseminating new ways of thinking.
In the matter of capabilities exhibited to address performance measurement,
they are varying from one organization to another. Part of this difference is certainly
to be explained by budget considerations. There are indeed financial differences
between the three organizations which cannot be ignored when assessing the resources
invested. The IGO’s budget is notably three times higher than the ones of its two
counterparts. In all cases, as far as priorities are concerned, the three organizations
all tend to relegate performance measurement to a secondary place in favor of supply
chain integration.
Turning to processes, it can be observed that the humanitarian supply chain
professionals who were interviewed are utterly embarked into the four processes
identified by Neely in relation to performance measurement (system design,
implementation, managing through measurement and “refreshing” the measurement
system; Neely and Powell, 2004). First of all, the three organizations tend to have
system design focussed on a single set of metrics. The balance is tilting in the opposite
direction of the private one: non-financial dimensions are prioritized over financial
ones. The structuring of individual metrics into more balanced systems is planned,
but current efforts seem to be concentrated toward refining the monitoring of
service to operations on the field. Second, the implementation process appears to be
particularly difficult. Defining and accessing the data sets that will be used to measure
performance in the most accurate way is seen as a complex and lengthy task. Deciding
 on the performance to be measured does not seem to be much of an issue, whereas Humanitarian
determining the way to measure it precisely requires considerable efforts. In the area of supply chain
management, performance measurement is increasingly used, but as seen above,
collective adhesion still needs to progress to make it fully effective. What appears to be
performance
the biggest challenge of all, in relation with the fourth process, is keeping measurement
systems up-to-date in a changing environment. As we have seen, humanitarian
professionals are decisively heading toward enhanced integration of their supply chain. 157
Due to this evolution, the scope of the entities in charge of supply chain management,
within the three organizations, is extending to the point of progressively erasing the
boundaries that separated them from the field. Piece by piece, the fragmented
accountability of the past is being placed in their hands, inducing the need for a shift
in their indicators. It is, however, still unknown whether it will lead them to seek the
measures of impact suggested by Tatham and Hughes (2011).
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

5.2 LAP
As we have seen, performance measurement is partially developed within the three
organizations. In a nutshell, it can be said that systems are lacking balance and
comprehensiveness; that they can hardly keep the pace with the ongoing supply chain
integration; that adhesion to the practice among actors is not unanimous but
improving; and most importantly that capabilities are dedicated primarily to the
implementation of an integrated supply chain.
To address this situation, a series of actions can be taken. First, new measures will
need to complement the existing service level monitoring, in order to build a more
balanced approach such as, for example the one recommended by Kaplan and Norton
(1992). Among them, measures of efficiency, neglected until now, will certainly have to
be addressed as a priority. Following the same idea, a more holistic approach should
materialize into a general framework including new parts of the supply chain
being implemented. While seeking this more comprehensive and proportionated view,
efforts should not be spared to reinforce collective adhesion to performance
measurement, a critical success factor for further developments. Eventually, instead
of giving end-to-end integration absolute priority over performance measurement,
humanitarians should consider the strong links between the two. If they are properly
interconnected, they can become part of a virtuous cycle bringing significant benefits.
This connection is depicted in Figure 1 which also includes issues mentioned earlier
such as internal support, capabilities, service to field operations and accountability.
An explanation of the reasoning supporting the model follows.
The positive effects of supply chain integration on performance are now
commonly accepted (Kim, 2013). It can therefore be expected that the end-to-end
integration humanitarian organizations are currently implementing will result
in higher performance of their supply chain and of their field operations. Similarly,
there is evidence that performance measurement can enhance performance, provided
it is supported by appropriate performance management (Bourne, et al., 2005).
Communication is another important function of performance measurement that
particularly relates to the integration of the humanitarian supply chain. Identified
by CBP (2004) as one of the three main roles of supply chain management systems,
together with the strategic and motivational functions, the communication role
materializes in two ways in the humanitarian end-to-end perspective. On the one hand,
it contributes to show accountability toward increasingly demanding donors (Greiling,
2011). On the other hand, it serves as the “voice” that will increase the internal support
 JHLSCM
5,2
INTERNAL
SUPPORT FOR
VOICE
SUPPLY CHAIN
ATTITUDES AND MANAGEMENT
CAPABILITIES
158 FOR
SUPPLY CHAIN
SUPPLY CHAIN DONOR
IMPROVEMENT PERFORMANCE ACCOUNTABILITY
SATISFACTION
MEASUREMENT

Figure 1.
The virtuous cycle of FIELD
RESOURCES END-TO-END SUPPLY CHAIN
OPERATIONS
humanitarian supply FOR INTEGRATION PERFORMANCE
PERFORMANCE
chain performance SUPPLY CHAIN
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

IMPROVEMENT
measurement and
end-to-end
integration

that Thomas and Kopczak (2005) identified as highly needed by humanitarians in the
field of supply chain management. In turn, internal support for supply chain management,
donor satisfaction and field operations performance result in the development of attitudes
and capabilities and increase the resources available for supply chain improvement,
a set of benefits which can eventually be used to promote end-to-end integration and
performance measurement.

6. Conclusion
At the origin of this investigation, the confidence was strong that new factors would
emerge, bringing significant implications for future research on humanitarian supply
chain performance measurement. These new factors emerged as a matter of fact. Among
the major ones, two must be highlighted: the current prioritization of supply chain
integration and the prevalence of service-level metrics in the humanitarian world.
As necessary as it is, the disclosure of uncovered elements is nevertheless not enough to
make a legitimate, value-added contribution to theory development. In line with Whetten
(1989), efforts were also made to describe the significance of these new factors in relation
with aspects that had already been studied in previous work. Some of these aspects
were presented in our literature review, one of the first being performed on the topic.
To conclude and encapsulate our findings, it can be said that far from being
static, the supply structure of humanitarian organizations is in high motion today, with
emerging entities at the core of a large-scale integration process that should become
a subject of interest for future research. Although this evolution can only be considered
as desirable, one can hope that humanitarian organizations’ focus on supply chain
integration will not dampen concerns for performance measurement. These two
processes are indeed part of a virtuous cycle that should be preserved. Hopefully,
humanitarian managers will also find ways to overcome the issues they have been
facing such as data collection, cultural hindrance or technological difficulties and they
will be able to build more holistic and balanced performance systems. In this latter
effort, they should certainly benefit from the young body of research now available,
as they should more generally from all future material that will be brought to light by
humanitarian supply chain studies.
 References Humanitarian
ALNAP (2010), The State of the Humanitarian System: Assessing Performance and Progress, supply chain
a Pilot Study, Overseas Development Institute, London. performance
Arshinder, K.A. and Desmukh, S.G. (2007), “Supply chain coordination issues: an
SAP-LAP framework”, Asia Pacific Journal of Marketing and Logistics, Vol. 19 No. 3,
pp. 240-264.
Beamon, B.M. (1999), “Measuring supply chain performance”, International Journal of Operations 159
and Production Management, Vol. 19 No. 3, pp. 275-292.
Beamon, B.M. and Balcik, B. (2008), “Performance measurement in humanitarian relief chains”,
International Journal of Public Sector, Vol. 21 No. 1, pp. 4-25.
Blecken, A., Hellingrath, B., Dangelmaier, W. and Schulz, S.F. (2009), “A humanitarian supply
chain process reference model”, International Journal of Services Technology and
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

Management, Vol. 12 No. 4, pp. 391-413.

Bölsche, D. (2012), “Performance measurement in humanitarian logistics – a process-oriented
perspective”, Proceedings from the 2nd International HumLog Workshop, Essen,
September 24.
Bourne, M., Kennerley, M. and Franco-Santos, M. (2005), “Managing through measures: a study of
impact on performance”, Journal of Manufacturing Technology Management, Vol. 16 No. 4,
pp. 373-395.
CBP (2004), “Literature review on performance measurement and management”, prepared for
The IDeA and Audit Commission Performance Management, Measurement and
Information (PMMI) Project, CBP, Cranfield University, Cranfield.
Charan, P. (2012), “Supply chain performance issue in an automobile company: a SAP-LAP
analysis”, Measuring Business Excellence, Vol. 16 No. 1, pp. 67-86.
Davidson, A.L. (2006), “Key performance indicators in humanitarian logistics”, master of
engineering in logistics thesis, Massachusetts Institute of Technology, Cambridge, MA.
De Leeuw, S. (2010), “Towards a reference mission map for performance measurement in
humanitarian supply chains”, International Federation for Information Processing,
Advances in Information and Communication Technology Series, Vol. 336, pp. 181-188.
Greiling, D. (2011), “Performance measurement systems in non-profit organizations as management
tool or an option for strategic responses?”, presented at the 13th Biennial CIGAR Conference,
Bridging Public Sector and Non-profit Sector Accounting, Ghent University, Ghent,
Performance Measurement Systems in Non-profit Organisations.pdf, June 9-10, available
at: www.feb.ugent.be/accoeco/Papers_Cigar2011/72. (accessed March 29, 2013).
Guyoton, S. and Muirheid, B. (2002), “Building a balanced scorecard for the IFRC”, unpublished
master thesis, INSEAD Business School, Fontainebleau.
Kanter, R.M. and Summers, D.S. (1987), “Doing good while doing well: dilemmas in performance
measurement in nonprofit organizations and the need for a multiple constituency
approach”, in Powell, W.W. (Ed.), The Nonprofit Sector: A Research Handbook, Yale
University Press, New Haven, CT, pp. 154-166.
Kaplan, R.S. and Norton, D.P. (2000), “Having trouble with your strategy? Then map it”,
Harvard Business Review, September-October, pp. 167-176.
Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard: measures that drive performance”,
Harvard Business Review, January-February, pp. 71-79.
Kent, R.C. (2004), “International humanitarian crises: two decades before and two decades
beyond”, International Affairs (Royal Institute of International Affairs 1944-), Vol. 80 No. 5,
pp. 851-869.
 JHLSCM Kim, D.Y. (2013), “Relationship between supply chain integration and performance”,
Operations Management Research, Vol. 6 Nos 1-2, pp. 74-90.
5,2
Kovács, G. and Tatham, P. (2009), “Responding to disruptions in the supply network – from
dormant to action”, Journal of Business Logistics, Vol. 30 No. 2, pp. 215-229.
Lee, H.L. (2004), “The triple-a supply chain”, Harvard Business Review, October, pp. 102-112.
160 Lijo, J. and Ramesh, A. (2012), “Humanitarian supply chain management in India:
a SAP-LAP framework”, Journal of Advances in Management Research, Vol. 9 No. 2,
pp. 217-235.
McLachlin, R., Larson, P.D. and Khan, S. (2009), “Not-for-profit supply chains in interrupted
environments. the case of a faith-based humanitarian relief organization”, Management
Research News, Vol. 32 No. 11, pp. 1050-1064.
Moe, T.L., Gehbauer, F., Senitz, S. and Mueller, M. (2007), “Balanced scorecard for natural
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

disaster management projects”, Disaster Prevention and Management, Vol. 16 No. 5,

pp. 785-806.
Neely, A.D. and Powell, S. (2004), “The challenges of performance measurement: Andy Neely
in conversation with Sarah Powell”, Management Decision, Vol. 42 Nos 7-8,
pp. 1017-1023.
OCHA (2011), “Funding trend analysis”, available at: http://fts.unocha.org/pageloader.aspx?
page¼Trend-TrendAnalysis (accessed March 29, 2013).
Oloruntoba, R. and Gray, R. (2006), “Humanitarian aid: an agile supply chain?”, Supply Chain
Management: An International Journal, Vol. 11 No. 2, pp. 115-120.
Patton, M.Q. (1980), Qualitative Evaluation Methods, Sage Publications, Beverly Hills, CA.
Salomons, D. (2012), “Global Humanitarian Studies Index”, available at: www.sipa.columbia.edu/
academics/concentrations/ha/ghsi/index.html (accessed March 29, 2013).
Scholten, K., Sharkey-Scott, P. and Fynes, B. (2010), “(Le)agility in humanitarian aid (NGO) supply
chains”, International Journal of Physical Distribution and Logistics Management, Vol. 40
Nos 8-9, pp. 623-635.
Schulz, S.F. and Heigh, I. (2009), “Logistics performance management in action within a
humanitarian organization”, Management Research News, Vol. 32 No. 11, pp. 1038-1049.
Sofaer, S. (1999), “Qualitative research methods: what are they and why use them?”, Health
Services Research, Vol. 34 No. 5, pp. 1101-1118.
Stewart, G. (1997), “Supply-chain operations reference model (SCOR): the first cross-industry
framework for integrated supply-chain management”, Logistics Information Management,
Vol. 10 No. 2, pp. 62-67.
Sushil (2000), “SAP-LAP models of inquiry”, Management Decision, Vol. 38 No. 5, pp. 347-353.
Tatham, P.H. and Hughes, K. (2011), “Humanitarian logistic metrics: where we are, and how we
might improve”, in Christopher, M. and Tatham, P. (Eds), Humanitarian Logistics: Meeting
the Challenge of Preparing for and Responding to Disasters, Kogan Page, London, pp. 65-84.
Thomas, A.S. and Kopczak, L.R. (2005), From Logistics to Supply Chain Management: The Path
Forward in the Humanitarian Sector, Fritz Institute, San Francisco, CA.
Tomasini, R. and Van Wassenhove, L. (2009), Humanitarian Logistics, Palgrave Macmillan,
Basingstoke.
Tranfield, D., Denyer, D. and Smart, P. (2003), “Towards a methodology for developing evidence-
informed management knowledge by means of systematic review”, British Journal
of Management, Vol. 14 No. 3, pp. 207-222.
 Van der Laan, E.A., de Brito, M.P. and Vergunst, D.A. (2009), “Performance measurement in Humanitarian
humanitarian supply chains”, International Journal of Risk Assessment and Management,
Vol. 13 No. 1, pp. 22-45.
supply chain
Walker, P. and Russ, C. (2011), “Fit for purpose: the role of modern professionalism in evolving
performance
the humanitarian endeavour”, International Review of the Red Cross, Vol. 93 No. 884,
pp. 1193-1210.
Whetten, D.A. (1989), “What constitutes a theoretical contribution?”, Academy of Management 161
Review, Vol. 14 No. 4, pp. 490-495.
Yin, R.K. (1981), “The case study crisis: some answers”, Administrative Science Quarterly, Vol. 26
No. 1, pp. 58-65.
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com
 This article has been cited by:

1. Marianne Jahre, Nathalie Fabbe-Costes. 2015. How standards and modularity can improve
humanitarian supply chain responsiveness. Journal of Humanitarian Logistics and Supply Chain
Management 5:3, 348-386. [Abstract] [Full Text] [PDF]
Downloaded by Florida Atlantic University At 02:13 24 February 2016 (PT)