Article

Exploring the Divide between Output Journal of Health Management

19(4) 1–16
and Outcome Measures in Health Care: © 2017 Indian Institute of
Health Management Research
Conceptual and Empirical Insights from SAGE Publications
sagepub.in/home.nav
a Literature Review DOI: 10.1177/0972063417727622
http://jhm.sagepub.com

Rocco Palumbo1

Abstract
Performance assessment plays a crucial role in providing policymakers and governing bodies with
timely and relevant information to monitor the quality of health care systems. However, scholars are
not consistent in discussing the attributes and the effects of performance assessment in health care.
This article aims at providing a synthesis of scientific evidence in the field of performance measurement
in the health sector to explore the paradigm shift from output to outcome. The transition to outcome
measurement turns out to be an unresolved issue. Nonetheless, it encourages the reconceptualization
of performance measurement as a crucial communication tool between health care organizations and
their stakeholders.

Keywords
Performance measurement, health care, health outcomes, outcome assessment, performance
management

Introduction
As compared with other managerial innovations which were introduced in the public sector in the last 30
years of the twentieth century as a result of the managerialization process (Ferlie, Lynn & Pollitt, 2005),
performance management systems arose more or less at the same time in the public and in the private
realms (Adair et al., 2006). This circumstance encouraged scholars to identify in the public sector a
prolific research area to achieve a leading edge on key issues in the field of performance management
(Lapsley, 1996). However, to the to the author’s knowledge, there is still poor agreement on the ultimate
meaning of performance measurement in the public sector, with some definitions showing a focus on
efficiency and effectiveness of past actions (Neely, 2005) and others paying attention to the purpose of

1
Research Fellow in Organizational Studies, University of Salerno, Fisciano, Italy.

Corresponding author:
Rocco Palumbo, PhD, Research Fellow in Organizational Studies, University of Salerno, 84084, Via Giovanni Paolo II, No. 132,
Fisciano (SA), Italy.
E-mail: rpalumbo@unisa.it
2 Journal of Health Management 19(4)

performance measurement tools to evaluate the ability of health care organizations to deliver meaningful
value for their stakeholders (Moullin, 2007).
This consideration is especially true dealing with the health care service system, where two contextual
concerns affect the characteristics and the development of performance measurement tools. First of
all, different stakeholders are variously interested in the value produced and delivered by health care
organizations, including actual and potential patients, carers, health care professionals, third-party
payers, the scientific community, and governing bodies (Campbell et al., 2002). The balance of power
between these agents influences the priorities and objectives of performance assessment tools, thus
affecting the inner attributes of the latter. Moreover, the evolution of performance management systems
is conditioned by the prevailing practice paradigms and delivery approaches which inspire the provision
of health services and define the overarching aims of the health care service system (McIntyre, Rogers
& Heier, 2001).
The scientific literature has pointed out that performance measurement tools play a crucial role in
providing policymakers and governing bodies with timely and relevant information to monitor the over-
all quality of health care systems (Van der Wees et al., 2014a). Besides, they allow patients, carers, health
care professionals, researchers, and scientists to get useful information about the ability of health care
organizations to create value and to meet the health needs of the population served (Smith et al., 2009).
Nonetheless, scholars and practitioners are not consistent in discussing the ultimate effects of performance
measurement systems on the improvement of health care practices (Cassel et al., 2014).
On the one hand, performance measurement systems have been considered to be effective in making
health care organizations accountable—especially in financial and institutional terms—to their external
stakeholders (Kromm et al., 2014). On the other hand, several barriers have been found to prevent
performance measurement tools to drive significant improvements in everyday health care practices
(Elg, Palmberg & Kollberg, 2013). In most of the cases, these barriers are related to the traditional focus
of performance measurement systems on cost containment and outputs (Li & Benton, 1996). Indeed,
performance measurement tools of health care organizations are largely compliant with the prevailing
bio-medical approach to care, which is illness-centred and neglects health outcomes to concentrate the
attention on delivery processes (Kaplan, 1997).
In spite of the growing attention paid to health outcomes both in theory and in practice, it seems
that a paradigm shift from output to outcome is still far from being achieved. Several studies have
shown that the hybridization of traditional performance management systems with outcome measures
contributes in quality improvement of health services, but it is difficult to realize (Van Der Wees et al.,
2014b). In fact, performance measurement systems produce several unintended consequences on the
behaviour of health care organizations, including measurement fixation, tunnelling, myopia, and
quantification privileging, which encourage to sacrifice health outcomes on the altar of efficiency
(Mannion & Braithwaite, 2012).
There is a desperate need for studies aimed at critically discussing the paradigm shift of performance
management systems in the health sector, stressing the role played by outcome measures. Current
literature reviews in the field of performance measurement systems of health care organizations are
characterized by a focus either on specific health care settings (see, e.g., Lauriks et al., 2012; Worth,
Hammersley, Nurmatov & Sheikh, 2014) or on particular types of illnesses (see, e.g., Brown et al.,
2015). This situation prevents full-fledged representations of the issues which which affect the meas-
urement and the management of performance in health care organizations. To contribute in filling the
extant gaps in the scientific knowledge, this manuscript is aimed at providing a synthesis of timely
evidence in the field of performance measurement in the health care sector. The attention is focused
Palumbo 3

on the quest for the transition from output to outcome measurement. It echoes the shift from the classic
bio-medical approach to care, which is illness-centred and adopts a find-it/fix-it approach, to a
patient-centred approach to care, which puts the needs of the users first. In line with the arguments
discussed above, it is assumed that the attributes of performance management systems are influenced
by the evolution of prevailing practice paradigms in health care. Therefore, the transition to patient-
centred care is expected to put greater emphasis on health outcomes, as compared with health care
costs and outputs.
The following research questions inspired this article:

R.Q. 1: What are the attributes and the consequences of the paradigm shift from output to outcome
in performance measurement of health care organizations?
R.Q. 2: What kind of barriers prevent or slow down the transition from output to outcome?
R.Q. 3: And, last but not least, what interventions could encourage the implementation of outcome
measures in the health care system?

This manuscript is organized as follows. The second section provides some details on the method-
ology of the study and describes the research strategy which was used to collect and analyse the
contributions included in this literature review. For this purpose, a systematic approach was adopted
through multiple electronic database queries. The third section presents the main findings of the research.
They are organized in three subsections, according to the three research questions at the basis of this
study. The first subsection delves into the characteristics of the paradigm shift from cure to care on
performance measurement tools. The second subsection points out the barriers which prevent the
inclusion of outcome measures in performance management instruments, while the third subsection
synthesizes the managerial interventions which could assist the implementation of outcome assess-
ment in health care organizations. Discussion critically examines the findings of the literature review,
taking into consideration the limitations which affected this study. The concluding sections summarize
the twofold relevance of this manuscript, which, on the one hand, contributes in better contextualizing
outcome measurement in health care organizations and, on the other hand, provides health care practi-
tioners with interesting insights about the role of outcome assessment to enhance the effectiveness of
health care organizations.

Methodology and Research Strategy

A specific research protocol was designed and implemented for the purpose of this study. On the one
hand, a systematic approach to collect and analyse the contributions of the extant scientific literature
was adopted. On the other hand, a narrative approach to synthesize the current evidence on the evolv-
ing attributes and role of performance measurement systems in health care organizations was used.
This research design paved the way for a thorough and critical analysis of relevant conceptual and
empirical evidence about the topic being examined (Hart, 1998), thus allowing a full-fledged answer
to the research questions.
A clear search and selection strategy is the first step to an effective literature review (Coughlan,
Cronin & Ryan, 2013). The eligibility criteria, that is to say the rules which informed the preliminary
identification of potentially relevant contributions, are at the basis of the research strategy. In order to
avoid biases in the collection of published evidences, no temporal limitations were included in the
4 Journal of Health Management 19(4)

research strategy. All the scientific contributions published until 2015 were taken into consideration.
Moreover, to improve the replicability of the literature review and to ensure that all the records included
in the analysis had an international audience, a strict linguistic criterion was established. Going more into
details, only English-written contributions were contemplated. To enhance the consistency of this study,
both formally published literature and grey literature were recognized as pertinent. Therefore, beyond
regular articles published in international peer-reviewed journal, also editorials, letters to the editors,
commentaries, books, book’s chapters, books’ reviews, research reports, proceedings, and original
researches published in grey literature sources were taken into consideration. Lastly, to catch all the
shades of the topic being studied, both conceptual and empirical contributions were allowed within the
research criteria.
To materially collect the items which informed this literature review, two citation databases were
queried: Scopus-Elsevier and Web of Science. The former is the largest abstract and citation database
of peer-reviewed literature in the fields of science, technology, medicine, social sciences, and arts
and humanities, indexing more than 57 million records. The latter indexes more than 200 million
different sources, which concern 55 different disciplinary areas. Considering that these two citation
databases allowed to access a wide spectrum of current scientific literature, other sources were not
queried.
Since Scopus-Elsevier and Web of Science showed similar graphical interfaces, the same research
strategy was used for both of them. In particular, the algorithm Performance Measurement OR
Outcome Measurement was employed as the primary key of research. These keywords allowed to
largely delve into the topics of this study. The algorithm was run in the field Article Title, Abstract,
Keywords of Scopus-Elsevier and in the field Topic of Web of Science. In both of the cases, this
algorithm was associated with a secondary key of research through the Boolean operator AND.
Either ‘health care’ or ‘healthcare’ were employed as secondary key of research, which was queried
in the field Article Title, Abstract, Keywords of Scopus-Elsevier and in the field Topic of Web of
Science. In light of the specific aims of this manuscript, the attention was focused on five subject
areas, namely: (i) business, management and accounting, (ii) social sciences, (iii) economics,
econometric and finance, (iv) decision sciences and (v) public administration. The last query was
run on 23 January 2016.
This research strategy gave back 1.034 records (719 within ‘Scopus-Elsevier’ and 315 within ‘Web of
Science’). The items retrieved were organized by publication year in an electronic worksheet. A total of
48 records were immediately removed, since they did not meet the language criterion. In addition, 83
items were duplicated, that is to say retrieved twice due to their compliance with more than one research
strategy. As a result, the attention was focused on 903 items, which were duly screened on both their
titles and abstract.
The preliminary screening activities were performed by two independent researchers, in order to
minimize the risks of removal of potentially relevant contributions. Only items showing a focus on per-
formance measurement systems in health care organizations were considered as relevant to be included
in this literature review. Otherwise, articles dealing with topics only indirectly related to performance
management systems were excluded. As well, manuscripts which concentrated their attention on health
outcomes and did not discuss the inclusion of outcome measures in performance management systems
were not included in the analysis. The two reports arranged by the researchers involved in the research
were duly confronted, in order to identify and settle potential inconsistencies.
As a result of this preliminary screening, 546 items were removed, since they were found to be not
consistent with the specific purposes of this study. Therefore, 357 records were selected for potential
Palumbo 5

inclusion in this systematic review. Their full-texts were retrieved from either Scopus-Elsevier or
Web of Science. Where not available, the full texts were collected from external sources, including
Google Scholars, research repositories and social networking sites for scientists and researchers. When
needed, manuscripts’ authors were approached via e-mail to ask a courtesy copy of their articles. As a
result of this retrieval process, 43 items were discarded, since their full texts were not available.
The manuscripts were examined by the author, in order to identify the articles which contributed to
provide an answer to the research questions. Records which did not take into consideration the inner
attributes of performance measurement systems or did not discuss the paradigm shift from output to
outcome were not taken into consideration. At the end of this second screening, 128 papers were selected
as relevant and thus included in this literature review. Their publication period ranged between 2015 and
1982, with most of them (56 out of 128) being published between 2015 and 2010. Only 20 items were
published on 1999 or before. Most of the manuscripts included in this literature review consisted of
original articles published in peer reviewed journals (94 out of 128). Books’ chapters (12 items)
proceedings (8 items), reports (7 items) and editorials (3 items) were also included. Figure 1 depicts a
flow-diagram which synthetizes the collection and selection process above described. The following
section provides a narrative synthesis of the research report, focusing on the paper which better contrib-
uted to provide an answer to the research questions.

Records collected from Scopus- Records collected from Web of

Elsevier® Science™
n = 719 n = 315

Total number of records collected through

database query
n = 1.034
Records non-compliant with the Duplications
language criterion

n = 48 n = 83

Records submitted to preliminary

screening titles, abstracts, and/or extracts
n = 903
Records removed due to poor
relevance to the study purposes
n = 546

Records in-depth screened on their full-

texts
n = 357
Records excluded due to lack of
Records excluded due to non focus or poor consistency with the
availability of their full-texts research aims
n = 43
n = 186
Records included in the analysis

n = 128

Figure 1. Flow Diagram Depicting the Research Strategy

Source: Author’s elaboration.
6 Journal of Health Management 19(4)

Findings of the Research

From Output to Outcome: A Paradigm Shift or a Temporary Fad?

Health care organizations are complex systems, which have to deal with a large and dynamic set of needs
and priorities, to which different evaluation criteria are attached. In an attempt to simplify this issue,
it has been claimed that performance measurement systems of health care organizations should pay
attention to three contextual areas of concern: (i) efficiency, (ii) effectiveness and (iii) flexibility (Purbey,
Mukherjee & Bhar, 2007). Efficiency ultimately involves the ability of health care organizations to meet
the financial constraints of the health care service system. Alternatively, effectiveness deals with the ability
of health care organizations to satisfy the health needs of the population served, while flexibility
concerns the need for adaptation of current structures and practices to environmental changes and
external pressures.
From this point of view, the concept of performance in the health care sector has been conceived as a
multifaceted construct, which consists of a combination of costs of care, quality of services, access to
care and users’ satisfaction (Lied & Kazandjian, 1999). Beyond providing information on costs and
revenues of past actions, performance measurement systems allow to gain knowledge about what works
and what does not work in health care organizations, paving the way for a better management of organi-
zational structures and procedures (O’Leary, 1998).
In spite of these arguments, cost containment (efficiency) and quality of care (effectiveness) have
been usually depicted as conflicting goals, rather than as concurring aims of health care organizations
(Seetharaman, Raj & Saravanan, 2010). In turn, the perceived clash between these two purposes affected
the development of performance measurement systems. As suggested by Spath (2007, p. 3), health care
organizations ‘...are being challenged to meet the data demands of a growing number of mandatory and
voluntary measurement projects’, which are addressed to different concerns. To properly deal with these
conflicting demands, health care organizations are called to tame the monster of measurement (Spath,
2007), avoiding the misuse of resources available to meet different information needs.
Coercive isomorphism has played a significant—even though tacit—role in the attempt of health care
organizations to tame the monster of measurement, sacrificing the meaningfulness of performance
management systems on the altar of institutional legitimacy (Vendramini, Lecci & Filannino, 2014).
Efficiency has been widely assumed to be the gold standard of performance measurement systems, with
the eventual purpose of containing health care costs. However, institutional pressures aimed at cost con-
tainment are expected to produce a sort of ‘measurement bias’ among health care organizations (Pettersen
& Nyland, 2006, p. 146). In fact, they incite to focus on input and output measures, which are easier to
operationalize and quantify as compared with outcome indicators. This quantification bias engenders a
deleterious process of organizational gaming, which is aimed at manipulating performance measures in
order to minimize the risks of sanctions for negative evaluations (Mears & Webley, 2010). It follows a
decline in the meaningfulness of performance assessment tools of health care organizations, which turn
out to not be able to fully satisfy the information needs of internal or external stakeholders.
The focus on efficiency implies that health services are deconstructed in their elementary compo-
nents, which are easy to be measured and managed. This deconstruction process does not allow to iden-
tify and deal with systemic problems, producing unintended drawbacks on the ability of health care
organizations to improve the quality of care (Fryer, Antony & Ogden, 2009). In addition, it incites a
perverse reasoning by the side of health care providers, which are more likely to pay attention to
economic issues and to overlook health outcomes, since the former are more immediately related to the
elementary components of health services as compared with the latter (Lega & Vendramini, 2008).
Palumbo 7

Also, it paves the way for a proliferation of performance measures and indicators (Arah et al., 2003),
which are difficult to be handled and do not provide with timely information to inspire future policies
and decisions aimed at improving health care practices (Ginsberg & Sheridan, 2001).
The transition from a bio-medical to a patient-centred approach to care produced a greater emphasis
on outcome measures to enhance the meaningfulness of performance management systems. However,
scholars have pointed out that different problems prevent the use of outcome measures in the health care
service system: among others, probability factors, low frequency, long delays and inadequate compre-
hensibility of health outcomes have been presented as relevant barriers to the introduction of perfor-
mance measures based on health outcomes (Eddy, 1998). The difficulties related to the attribution and
causality of health outcomes compound these problems, discouraging the use of this kind of measures
in performance measurement tools (Terris & Aron, 2009).
In addition, the enlargement of measurement activities lead to additional costs on the health care
service system (Loeb, 2004). Actually, the adoption of an outcome concern implies greater demands for
data and information and additional processing activities, which are expected to produce spiralling costs.
This point is especially relevant in light of the financial constraints which are currently faced by most of
health care systems. Several studies have found that health care providers are likely to perceive little
utility from outcome measurement tools, which are considered to be not critical to inform decision-
making activities (Garland, Kruse & Aarons, 2003). Echoing these findings, it has been shown that less
accurate performance measurements systems provide unexpected advantages to their users. Going more
into details, simple measurement tools perform as coordination mechanisms between individual and
policy decisions, reducing the risks of either misunderstanding or misuse of more accurate, but more
complex performance measures (De Bont & Grit, 2012).
In sum, it could be argued that the transition to patient-centred care produced a greater attention to
health outcomes, which are key to assess the overall effectiveness of health care organizations.
Notwithstanding, the introduction of health outcomes measures in the health care service system
seems to be a widely unresolved issue. Even though outcome-based measures should not be conceived
as a temporary fad, several barriers prevent the integration of health outcome in performance measure-
ment systems. From this point of view, the accountability of health care organizations is at risk of
being impaired. As shown in the following section, beyond financial constraints and poor perceived
utility of outcome measures, other factors have been found to prevent the transition from output to
outcome in health care organizations, thus impoverishing the overall meaningfulness of performance
measurement systems.

The Barriers to Outcome Measurement in the Health Care Sector

As anticipated, there is still poor agreement on the characteristics and role of outcome measures in health
care. It is necessary to identify the main attributes and the purposes of outcome measurement tools to
discuss the barriers which prevent their inclusion in performance evaluation systems of health care
organizations. Scholars have argued that outcome measures should not solely focus on the actual or
potential impacts of health services on personal ability. Rather, they should consider how health-related
services help people in improving their functioning in everyday life (Forder & Caiels, 2011).
In fact, outcome measures which concentrate on personal health conditions are generally affected by
problems of attribution, evaluation biases and equity issues (Lorgelly, Lawson, Fenwick & Briggs,
2010), which do not allow to draw adequate information from them. This is especially true when
integrated health and social services are concerned, where issues related to attribution and evaluation are
8 Journal of Health Management 19(4)

difficult to face (Petch, Cook & Miller, 2013). Alternatively, models which take into consideration the
social functioning of patients—such as the capability approach—are expected to produce greater insights
on the effectiveness of health interventions, by providing decision makers with a richer evaluative space
(Lorgelly, Lorimer, Fenwick, Briggs & Anand, 2015). However, these models pave the way for further
points of discussion, which should be resolved to encourage their use in the health care service system.
In particular, unresolved questions include: (i) the identification of a comprehensive set of capabilities
to properly assess health outcomes, (ii) the differentiation between evidence of individual functioning
and capabilities, (iii) the measurement of objective versus perceived capabilities, (iv) the objective
evaluation of capability sets for interpersonal and cross-contextual comparisons, and (v) the provision of
useful information on efficiency and equity to inform decision-making activities at both the policy and
the organizational levels (Simon et al., 2013).
Summarizing these considerations, reliability, validity, responsiveness, precision, interpretability,
acceptability, and feasibility issues have been presented as barriers to the implementation of outcome
measures in the health care sector (Fitzpatrick, et al., 1998). Besides, Williams (2013) suggested that
social factors—such as regional subcultures—are expected to deeply affect the assessment of health
outcomes. As a consequence, the effects of these contingent variables should be contemplated, in order
to improve the reliability of outcome measures. Obviously, the lack of validated outcome indicators in
the health care service system exacerbates these problems, discouraging the inclusion of an outcome
concern in performance measurement systems (Payne et al., 2008).
In light of these concerns, the scientific literature has argued that outcome measures range from
extremely useful to worse than no information at all, providing decision makers with misleading, rather
than with helpful information (McGlynn, 1998). To address this issue, different outcome measures
should be conceived as complements rather than as substitutes (Johannesson, Jönsson & Karlsson,
1996). In fact, multifaceted outcome measures shed light on different aspects which are useful to
adequately assess the effectiveness of health interventions. Therefore, they should be included in an
integrated performance measurement framework to effectively assist the decision-making process. On
the other hand, outcome measurement tools are expensive and the need for their contextual use increases
the current financial pressures on health care organizations.
As an alternative, the use of specific weights could help in improving the meaningfulness of health
outcome measures, without requiring the contextual use of different assessment tools. In fact, they
allow to enhance the interpretability and the acceptability of outcome measures, by enlarging their
evaluative space. For the sake of the argument, formal equity weighting systems help in improving the
meaningfulness of outcome measures, by taking into consideration the consequences of health inter-
vention on equity issues and allowing the users of performance measurement systems to catch more
insightful information on the effectiveness of health services provided (Norman et al., 2013). In a
quite similar way, risk adjustment factors are crucial to improve the ability of outcome assessment
tools to depict the real performance of health care organizations, by correcting outcome measures with
relevant confounding variables, including illness severity, age, and social support (Moran & Jacobs,
2015).
To properly identify the weighting and the risk adjustment factors that should be attached to outcome
measurement systems, it is necessary to consider the different perspectives of the agents involved or inter-
ested in the health care service system. Among others, health care professionals are argued to play a critical
role in designing balanced performance measurement systems (Blanchette et al., 2014). Knowledge and
resource deficits have been presented as significant organizational barriers to the involvement of health care
professionals to the development of meaningful outcome measurement systems (Fitzpatrick, 2009). On the
one hand, knowledge barriers concern the inclination of health care professionals to be sceptical about
Palumbo 9

the value added of outcome measurement systems and about their utility to inform decision-making pro-
cesses. On the other hand, resource barriers concern the lack of time of health care providers to participate
in the development of outcome measures and the significant economic costs which are associated with the
full-fledged implementation of outcome measurement systems.
Nevertheless, a stronger involvement of health care professionals in designing and implementing
health outcome measures could pave the way for a greater awareness of the role played by outcome
measurement tools in the improvement of health services and, consequently, for their greater use.
From this point of view, scholars have pointed out that the adaptation of health outcome measures to
the local characteristics of health care organizations and the clarification of the importance of outcome
data to the enhancement of health care practices are likely to encourage strategies of ‘tailoring’ or
‘accommodating’ outcome assessment tools (Skeat & Perry, 2008, p. 117), which are aimed at changing
usual routines to allow time and resources for performance measurement activities by the side of
health care professionals.

The Requisites to the Implementation of Effective Performance Management Systems in

Health Care Organizations
The scientific literature has discussed several determinants which are argued to contribute in the success
of performance management systems in health care organizations. Among others, Nuti, Seghieri and
Vainieri (2013) identified five critical success factors to the implementation of performance management
systems, namely: (i) the political commitment to the introduction of performance measurement systems,
(ii) the involvement of internal and external stakeholders in measurement activities, (iii) the linkage
between performance measurement systems and rewarding systems, (iv) the public disclosure of data
and information collected, and (v) the design of visual reporting tools which are aimed at improving the
meaningfulness of performance evaluations.
In particular, the engagement of internal stakeholders—such as administrative staff, health care pro-
fessionals and caregivers—has been pointed out to be essential to promote the effective implementation
of performance measurement systems (Mauro et al., 2014). This is especially true dealing with health
services networks, where a stronger involvement of internal stakeholders allows a common understanding
of performance measurement systems and reduces the barriers to their use (Hammerschmidt, Falk &
Staat, 2012). Of course, to enhance the involvement of stakeholders, the perceived utility of performance
measurement systems should be stressed (Mauro et al., 2014).
To catch the different information needs of relevant stakeholders and to encourage their commitment
to the implementation of performance management systems, it is necessary to build multidimensional
measurement tools, which are aimed at holistically assessing the ability of health care organizations to
meet the expectations of the population served (Dey, Hariharan & Despic, 2008). This consideration is
consistent with the performance assessment framework for hospitals suggested by the World Health
Organization, which identifies six critical dimensions to assess organizational performance, that is to
say: (i) clinical effectiveness, (ii) safety, (iii) patient centredness, (iv) production efficiency, (v) staff
orientation and (vi) responsive governance. These dimensions include both output and outcome measures,
thus providing a full-fledged representation of performance of health care organizations (Veillard et al.,
2005). Nonetheless, since the assessment of a large number of performance and outcome indicators
implies a significant investment of resources, multidimensionality should not result in an excessively
large panel of performance indicators, which could pave the way for technical difficulties, financial
burdens and implementation problems (Blank, Koch & Burkett, 2004).
10 Journal of Health Management 19(4)

In other words, an adequate balance between variety and focus of performance measures should be
sought for, in order to enhance the overall meaningfulness of performance management systems
(Thier & Gelijns, 1998). On the one hand, variety is aimed at differentiating performance measure in
order to catch the different shades of health care organizations’ performance. On the other hand, focus
is necessary to minimize the risks of information overload, which harms rather than enhancing decision-
making activities. From this standpoint, a stronger link of performance measures with strategy (Perrin,
2002) and structures and processes (Friedman, Kokia & Shemer, 2003) is crucial to balance the
competing needs for variety and focus. In fact, it allows to establish a more direct and meaningful
relationship between performance indicators and strategic, organizational and managerial variables,
providing with useful information to assist decision-making activities and to support the improvement
of health care practices.
In line with these arguments, Moullin (2004) included the use of a balanced set of measures among
the essentials of performance measurement in the health care service system. In fact, balanced perfor-
mance measurement systems—including balanced scorecards—are expected to improve the variability
of performance indicators, thus enhancing their ability to meet the various information needs of the dif-
ferent categories of external stakeholders. Besides, scholars have emphasized that balanced measure-
ment tools allow an equilibrium between the conflicting purposes of cost control and quality improvement
(Yuen & Ng, 2012). Moreover, they have been found to contribute in preventing short-term behaviours
by the side of health care professionals that are induced by financial rewards based on outputs. Balanced
measurement tools are also argued to enhance health care professionals’ commitment and individual
satisfaction with performance evaluation (Lin, Yu & Zhang, 2014). Actually, they provide health care
organization with a better understanding of performance, thus contributing in improvements at both the
strategic (Grigoroudis, Orfanoudaki & Zopounidis, 2012) and the organizational levels (Kollberg & Elg,
2011). In sum, balanced tools perform as quality management instruments, rather than as sheer measure-
ment techniques.
Last but not least, Berta, Seghieri and Vittadini (2013) suggested that the adoption of a systemic
approach in assessing the performance of health care organizations could help in raising the awareness
of the role and the importance of performance measurement tools for the purpose of health care services’
improvement. In particular, comparisons of organizational performance and dissemination of best prac-
tices pave the way for a culture of evaluation, which encourages health care organizations to conceive
performance management systems as key to the enhancement of health services’ quality. Equity issues
have been claimed to be particularly important in evaluation systems (Nakaima, Sridharan & Gardner,
2013). In fact, the assessment of the ability of health care organizations to meet the health needs of the
population served by complying with ethical requirements is key to properly gauge the effectiveness of
health care organizations, minimizing the risks of opportunistic behaviours in the health care service
system.

Discussion and Limitations

The paradigm shift from output to outcome in performance measurement systems of health care organi-
zations is strictly related to the transition from the traditional bio-medical model to care, which focuses
on illnesses and adopts a find-it fix-it approach, to a patient-centred philosophy, which incites health care
providers to put the needs of the patients first in designing and delivering health-related services.
As suggested in the introductory section, the prevailing practice paradigms which inspire the provision
Palumbo 11

of health care is able to affect the attributes and the concerns of performance measurement systems,
thus influencing their inner functioning.
The traditional bio-medical approach implies a strong emphasis on outputs and costs. In this circum-
stance, performance measurement systems are ultimately aimed at providing information on past actions,
summarizing their results in terms of quantity of services provided and related costs. Alternatively,
patient-centred care involves a wider and deeper evaluation of health care organizations’ performance.
Rather than focusing on costs of health care practices, performance measurement systems are revised to
consider quality of care, access to care and users’ satisfaction, which are eventually understood as crucial
areas of concern to assess the ability of health care organizations to meet the expectations of the popula-
tion served.
Hence, the shift from output to outcome in performance measurement systems of health care organi-
zations should not be dealt with as a temporary fad or a needless innovation. Quite the opposite, this shift
is consistent with the underlying evolution of the prevailing philosophy of care. Patient centredness
emphasizes the importance of health outcomes. Besides, it has been claimed that outcome measures
should not merely focus on personal health conditions; rather, they should take into consideration the
overall social functioning of patients. In fact, capability measures allow to get more adequate and insight-
ful information on the ability of health care organizations to improve the quality of life of their patients,
thus providing decision makers with better information to inspire significant improvements in health
care practices.
However, several barriers prevent the use of outcome measures in performance management tools of
health care organizations. Drawing on the findings of this literature review, it could be argued that four
kinds of issues are likely to slow down the transition from output to outcome in the health care environ-
ment: (i) reliability issues, (ii) institutional issues, (iii) organizational issues and (iv) professional issues.
First of all, the inclusion of an outcome concern in performance measurement systems of health care
organizations is affected by the lack of homogeneous, reliable and concise outcome indicators, which
allow to get prompt and useful information to gauge organizational performance, minimizing at the
same time the risks of poor consistency of the data collected. To fill this gap, scholars and practitioners
are called to make a joint effort, which should be aimed at designing valid, generalizable and comparable
outcome measures. The introduction of balanced set of indicators and the attachment of specific weights
to outcome measures could help in improving the reliability of measurement tools. In this way, it is
possible to take into consideration the different variables which are critical to properly assess the performance
of health care organizations, including equity and accessibility to health services.
As discussed above, current institutional arrangements in most of health care systems show a strong
focus on outputs, rather than on health outcomes. This prevailing situation produced several side effects
on performance management systems of health care organizations, which are widely affected by quanti-
fication privileging, measurement fixation and organizational gaming. In line with these consideration,
coercive isomorphism by the side of health care organizations has been pointed out to affect the mean-
ingfulness of performance measurement systems, linking their functioning to the pursuit for institutional
legitimacy. Therefore, a process of institutional change is a necessary step for the transition from output
to outcome, encouraging health care organizations to pay greater attention to health services’ quality
rather than to outputs and costs.
Organizational constraints play a significant role in limiting the use of outcome indicators and in
preserving the primacy of output measures. In particular, cost-related and time-related concerns have
been discussed as the main organizational barriers to the introduction of outcome measures in performance
management systems of health care organizations. On the one hand, the design and the assessment of
outcome measures imply additional costs for health care organizations, which contribute in impairing
12 Journal of Health Management 19(4)

their sustainability in the current period of financial distress. On the other hand, the involvement of
internal stakeholder is assumed to be essential to pave the way for the arrangement of meaningful
and relevant performance indicators based on health outcomes. However, health care professionals,
caregivers and administrative staff are likely to perceive poor time availability to participate in an
ancillary activity, such as the construction of valid and reliable outcome measurement tools. Therefore,
more time and resources should be allowed for the purpose of outcome assessment in the health care
environment, in order to overcome the organizational constraints, which prevent the transition from
output to outcome.
Lastly, health care professionals are still likely to ascribe poor utility to outcome measures, considering
them not crucial to drive timely and significant improvements in every day health care practices. This
perception is mainly produced by the longstanding loyalty of health care professionals to the traditional
bio-medical approach to care, which incites to focus on objective and quantifiable measures. To enhance
the commitment of internal stakeholders to outcome measures, the latter should be more strictly linked
to incentive systems at the strategic, the organizational, and the operation levels, thus emphasizing the
role played by health outcomes in informing the future policies of health care organizations.
The findings of this manuscript should be read in light of its main limitations. First of all, the focus
on only two databases, that is to say Scopus-Elsevier and Web of Science, allowed to enhance the effi-
ciency of this literature review, but it affected its depth of analysis. Nonetheless, the use of two generic
citation databases for the purpose of literature review is not uncommon among scholars (see, among
others, Schmidt et al., 2015; Yeager et al., 2014). Moreover, the two citation databases queried in this
study allowed to widely search for scientific contributions in the field of performance measurement of
health care organizations, thus minimizing the risks of exclusion of potentially relevant manuscripts.
The design of the collection and the selection processes at the basis of this literature review represents
another potential weakness. Indeed, the research strategy was arranged to solely include in this evidence
synthesis the scientific contributions which focused their attention on performance measurement systems
of health care organizations. In line with this purpose, only five disciplinary areas were taken into
consideration. As a consequence, there is a significant risk of exclusion of potentially relevant contri-
butions. Nevertheless, the use of generic research criteria contributed in minimizing this risk, thus
improving the reliability and the consistency of the findings discussed.

Conclusions
The paradigm shift from output to outcome in performance measurement systems of health care organi-
zations is an unresolved issue. The growing emphasis ascribed to patient centred models of care incite to
pay greater attention to health outcomes, rather than to costs and outputs. However, several barriers—at
both the institutional, the organizational and the operational levels—prevent the use of outcome meas-
ures for the purpose of performance management of health care organizations. A stronger policy commit-
ment, a more direct link between health outcomes measures and incentive systems, and the inclusion of
outcome assessment among the organizational priorities of health care organizations are crucial to foster
the transition from output to outcome in the health care service system.
The relevance of this manuscript is twofold. First of all, it contributes in raising the awareness of the
hurdles which prevent the introduction of an outcome concern in performance measurement systems of
health care organizations. From this point of view, it provides both scholars and practitioners with
intriguing trajectories for further developments, with the eventual purpose of encouraging the adoption
of an outcome perspective in assessing the performance of health care organizations.
Palumbo 13

Scholars should pay greater attention to the arrangement of meaningful and reliable outcome meas-
ures, which have to consider the ultimate effects of health interventions on the social functioning of
patients, rather than focusing on the individual health status. The participation of relevant internal and
external stakeholders is crucial for this purpose. There is a desperate need for studies aimed at examining
the process of stakeholder involvement and engagement in the design and implementation of perfor-
mance measurement systems based on health outcomes. To enhance the commitment of relevant stake-
holders, institutional barriers should be removed, by refocusing the attention of incentive systems from
outputs to outcomes. This process of institutional change will allow to overcome the problems of tunnel-
ling, myopia and fixation, which affect the measurement systems of most of health care organizations.
Health care organizations should devote more time and resources to the establishment and the func-
tioning of effective performance measurement systems. Indeed, time and resource constraints are widely
recognized as the most relevant barriers to the involvement of health care professionals in the arrangement
and use of valid outcome measures. When time and resources are lacking, health care professionals are
likely to adopt either tailoring or accommodating strategies, which—on the one hand—acknowledge the
role of health outcomes in performance measurement systems but—on the other hand—impoverish their
meaningfulness and relevance.

References
Adair, C. E., Simpson, E., Casebeer, A. L., Birdsell, J. M., Hayden, K. A., & Lewis, S. (2006). Performance
measurement in healthcare: Part I—Concepts and trends from a state of the science review. Healthcare Policy,
1(4), 85–104.
Arah, O. A., Klazinga, N. S., Delnoij, D. M., ten Asbroek, A. H., & Custers, T. (2003). Conceptual frameworks for
health systems performance: A quest for effectiveness, quality, and improvement. International Journal for
Quality in Health Care, 15(5), 377–398.
Berta, P., Seghieri, C., & Vittadini, G. (2013). Comparing health outcomes among hospitals: The experience of the
Lombardy Region. Health Care Management Science, 16(3), 245–257.
Blanchette, V., O’Mahony, B., McJames, L., & Mahlangu, J. N. (2014). Assessment of outcomes. Haemophilia,
20(4), 114–120.
Blank, M. B., Koch, J. R., & Burkett, B. J. (2004). Less is more: Virginia’s performance outcomes measurement
system. Psychiatric Services, 55(6), 643–645.
Brown, E. C., Robicsek, A., Billings, L. K., Barrios, B., Konchak, C., Paramasivan, A. M., & Masi, C. M. (2015).
Evaluating primary care physician performance in diabetes glucose control. American Journal of Medical
Quality, 31(5), 392–399. DOI: 10.1177/1062860615585138.
Campbell, S. M., Braspenning, J., Hutchinson, A., & Marshall, M. (2002). Research methods used in developing and
applying quality indicators in primary care. BMJ Quality & Safety in Health Care, 11(4), 358–364.
Cassel, C. K., Conway, P., Delbanco, S. F., Jha, A. K., Saunders, R. S., & Lee, T. H. (2014). Getting more performance
from performance measurement. New England Journal of Medicine, 371(23), 2145–2147.
Coughlan, M., Cronin, P., & Ryan, F. (2013). Doing a literature review in nursing, health and social care. London:
SAGE Publications.
De Bont, A., & Grit, K. (2012). Unexpected advantages of less accurate performance measurements: How simple
prescription data works in a complex setting regarding the use of medications. Public Administration, 90(2),
497–510.
Dey, P. K., Hariharan, S., & Despic, O. (2008). Managing healthcare performance in analytical framework.
Benchmarking: An International Journal, 15(4), 444–468.
Eddy, D. M. (1998). Performance measurement: Problems and solutions. Health Affairs, 17(4), 7–25.
Elg, M., Palmberg, K., & Kollberg, B. B. (2013). Performance measurement to drive improvements in healthcare
practice. International Journal of Operations & Production Management, 33(11/12), 1623–1651.
14 Journal of Health Management 19(4)

Ferlie, E., Lynn, L. E., & Pollitt, C. (2005). The Oxford handbook of public management. New York, NY: Oxford
University Press.
Fitzpatrick, R. (2009). Patient-reported outcome measures and performance measurement. In P. C. Smith, E.
Mossialos, I. Papanicolas, & S. Leatherman (Eds), Performance measurement for health system improvement:
Experiences, challenges and prospects (pp. 63–86). Cambridge: Cambridge University Press.
Fitzpatrick, R., Davey, C., Buxton, M., & Jones, D. (1998). Evaluating patient based outcome measures for use in
clinical trials. Health Technology Assessment, 2(14), 1–74.
Forder, J. E., & Caiels, J. (2011). Measuring the outcomes of long-term care. Social Science & Medicine, 73(12),
1766–1774.
Friedman, N. L., Kokia, E., & Shemer, J. (2003). Health value added (HVA): Linking strategy, performance, and
measurement in healthcare organizations. The Israel Medical Association Journal, 5(1), 3–8.
Fryer, K., Antony, J., & Ogden, S. (2009). Performance management in the public sector. International Journal of
Public Sector Management, 22(6), 478–498.
Garland, A. F., Kruse, M., & Aarons, G. A. (2003). Clinicians and outcome measurement: What’s the use? Journal
of Behavioral Health Services & Research, 30(4), 393–405.
Ginsberg, C., & Sheridan, S. (2001). Limitations of and barriers to using performance measurement: Purchasers’
perspectives. Health Care Financing Review, 22(3), 49–57.
Grigoroudis, E., Orfanoudaki, E., & Zopounidis, C. (2012). Strategic performance measurement in a healthcare
organisation: A multiple criteria approach based on balanced scorecard. Omega, 40(1), 104–119.
Hammerschmidt, M., Falk, T., & Staat, M. (2012). Measuring and improving the performance of health service
networks. Journal of Service Research, 15(3), 343–357.
Hart, C. (1998). Doing a literature review: Releasing the social. London: SAGE Publications.
Johannesson, M., Jönsson, B., & Karlsson, G. (1996). Outcome measurement in economic evaluation. Health
Economics, 5(4), 279–296.
Kaplan, R. M. (1997). Health outcomes and communication research. Health Communication, 9(1), 75–82.
Kollberg, B., & Elg, M. (2011). The practice of the balanced scorecard in health care services. International Journal
of Productivity and Performance Management, 60(5), 427–445.
Kromm, S. K., Ross Baker, G., Wodchis, W. P., & Deber, R. B. (2014). Acute care hospitals’ accountability to
provincial funders. Healthcare Policy, 10(SP), 25–35.
Lapsley, I. (1996). Reflections on performance measurement in the public sector. In I. Lapsley & F. Mitchell (Eds),
Accounting and performance measurement: Issues in the private and public sectors (pp. 109–128). London:
Paul Chapman Publishing.
Lauriks, S., Buster, M. C., de Wit, M. A., Onyebuchi, A. A., & Klazinga, N. S. (2012). Performance indicators for
public mental healthcare: A systematic international inventory. BMC Public Health, 12(214), 214–237.
Lega, F., & Vendramini, E. (2008). Budgeting and performance management in the Italian National Health System
(INHS). Journal of Health Organization and Management, 22(1), 11–22.
Li, L. X., & Benton, W. C. (1996). Performance measurement criteria in health care organizations: Review and
future research directions. European Journal of Operational Research, 93(3), 449–468.
Lied, T. R., & Kazandjian, V. A. (1999). Performance: A multi-disciplinary and conceptual model. Journal of
Evaluation in Clinical Practice, 5(4), 393–400.
Lin, Z., Yu, Z., & Zhang, L. (2014). Performance outcomes of balanced scorecard application in hospital
administration in China. China Economic Review, 30(3), 1–15.
Loeb, J. M. (2004). The current state of performance measurement in health care. International Journal for Quality
in Health Care, 16(1), 5–9.
Lorgelly, P. K., Lawson, K. D., Fenwick, E. A., & Briggs, A. H. (2010). Outcome measurement in economic
evaluations of public health interventions: A role for the capability approach? International Journal of
Environmental Research and Public Health, 7(5), 2274–2289.
Lorgelly, P. K., Lorimer, K., Fenwick, E. A., Briggs, A. H., & Anand, P. (2015). Operationalising the capability
approach as an outcome measure in public health: The development of the OCAP-18. Social Science &
Medicine, 142(2), 68–81.
Palumbo 15

Mannion, R., & Braithwaite, J. (2012). Unintended consequences of performance measurement in healthcare: 20
salutary lessons from the English National Health Service. Internal Medicine Journal, 42(5), 569–574.
Mauro, M., Cardamone, E., Cavallaro, G., Minvielle, E., Rania, F., Sicotte, C., & Trotta, A. (2014). Teaching hospital
performance: Towards a community of shared values? Scoail Science & Medicine, 101(1), 107–112.
McGlynn, E. A. (1998). The outcomes utility index: Will outcomes data tell us what we want to know? International
Journal for Quality in Health Care, 10(6), 485–490.
McIntyre, D., Rogers, L., & Heier, E. J. (2001). Overview, history, and objectives of performance measurement.
Health Care Financing Review, 22(3), 7–21.
Mears, A., & Webley, P. (2010). Gaming of performance measurement in health care: Parallels with tax compliance.
Journal of Health Services Research & Policy, 15(4), 236–242.
Moran, V., & Jacobs, R. (2015). Comparing the performance of English mental health providers in achieving patient
outcomes. Social Science & Medicine, 140(9), 127–135.
Moullin, M. (2004). Eight essentials of performance measurement. International Journal of Health Care Quality
Assurance, 17(3), 110–112.
———. (2007). Performance measurement definitions. International Journal of Health Care Quality Assurance,
20(3), 181–183.
Nakaima, A., Sridharan, S., & Gardner, B. (2013). Towards a performance measurement system for health equity in
a local health integration network. Evaluation and Program Planning, 36(1), 204–212.
Neely, A. (2005). Defining performance measurement: Adding to the debate. Perspectives on Performance, 4(2),
14–15.
Norman, R., Hall, J., Street, D., & Viney, R. (2013). Efficiency and equity: A stated preference approach. Health
Economics, 22(5), 568–581.
Nuti, S., Seghieri, C., & Vainieri, M. (2013). Assessing the effectiveness of a performance evaluation system in the
public health care sector: Some novel evidence from the Tuscany region experience. Journal of Management &
Governance, 17(1), 59–69.
O’Leary, D. S. (1998). Perspective: Reordering performance measurement priorities. Health Affairs, 17(4),
38–39.
Payne, K., Nicholls, S., McAllister, M., MacLeod, R., Donnai, D., & Davies, L. M. (2008). Outcome measurement
in clinical genetics services: A systematic review of validated measures. Value in Health, 11(3), 49–508.
Perrin, E. B. (2002). Some thoughts on outcomes research, quality improvement, and performance measurement.
Medical Care, 40(6), 89–91.
Petch, A., Cook, A., & Miller, E. (2013). Partnership working and outcomes: Do health and social care partnerships
deliver for users and carers? Health and Social Care in the Community, 21(6), 623–633.
Pettersen, I. J., & Nyland, K. (2006). Management and control of public hospitals—The use of performance
measures in Norwegian hospitals: A case-study. International Journal of Health Planning and Management,
21(2), 133–149.
Purbey, S., Mukherjee, K., & Bhar, C. (2007). Performance measurement system for healthcare processes.
International Journal of Productivity and Performance Management, 56(3), 241–251.
Schmidt, K., Aumann, I., Hollander, I., Damm, K., & Graf von der Schulenburg, J. M. (2015). Applying the analytic
hierarchy process in healthcare research: A systematic literature review and evaluation of reporting, 15(112),
DOI: 10.1186/s12911-015-0234-7.
Seetharaman, A., Raj, J. R., & Saravanan, A. (2010). The changing role of accounting in the health care industry.
Research Journal of Business Management, 4(2), 91–102.
Simon, J., Anand, P., Gray, A., Rugkåsa, J., Yeeles, K., & Burns, T. (2013). Operationalising the capability approach
for outcome measurement in mental health research. Social Science & Medicine, 91(8), 187–196.
Skeat, J., & Perry, A. (2008). Exploring the implementation and use of outcome measurement in practice: A
qualitative study. International Journal of Language & Communication Disorders, 43(2), 110–125.
Smith, P. C., Mossialos, E., Papanicolas, I., & Leatherman, S. (2009). Performance measurement for health system
improvement: Experiences, challenges and prospects. Cambridge: Cambridge University Press.
16 Journal of Health Management 19(4)

Spath, P. L. (2007). Taming the measurement monster. Frontiers of Health Services Management, 23(4), 3–14.
Terris, D. D., & Aron, D. C. (2009). Attribution and causality in health-care performance measurement. In P. C.
Smith, E. Mossialos, I. Papanicolas & S. Leatherman (Eds), Performance measurement for health system
improvement: Experiences, challenges and prospects (pp. 311–338). Cambridge: Cambridge University Press.
Thier, S.O., & Gelijns, A.C. (1998). Perspective: Improving health: The reason performance measurement matters.
Health Affairs, 17(4), 26–28.
Van der Wees, P. J., Nijhuis-van der Sanden, M. W., van Ginneken, E., Ayanian, J. Z., Schneider, E. C., & Westert,
G. P. (2014a). Governing health care through performance measurement in Massachusetts and the Netherlands.
Health Policy, 116(1), 18–26.
Van Der Wees, P., Nijhuis-Van Der Sanden, M. W., Ayanian, J. Z., Black, N., Westert, G. P., & Schneider, E. C. (2014b).
Integrating the use of patient–reported outcomes for both clinical practice and performance measurement: Views
of experts from 3 countries. Milbank Quarterly, 92(4), 754–775.
Veillard, J., Champagne, F., Klazinga, N., Kazandjian, V., Arah, O. A., & Guisset, A. L. (2005). A performance
assessment framework for hospitals: The WHO regional office for Europe PATH project. International Journal
for Quality in Health Care, 17(6), 487–496.
Vendramini, E., Lecci, F., & Filannino, C. (2014). When the rubber meets the road: Isomorphism, rhetoric, and
(mis)management of interinstitutional performance. International Journal of Public Administration, 37(13),
980–986.
Williams, J. (2013). Regional cultures and health outcomes: Implications for performance measurement, public
health and policy. The Social Science Journal, 50(4), 461–470.
Worth, A., Hammersley, V. S., Nurmatov, U., & Sheikh, A. (2014). Assisting allied health in performance evaluation:
A systematic review. BMC Health Services Research, 21(4), 572–585.
Yeager, V. A. Menachemi, N., Savage, G. T., Ginter, P. M., Sen, B. P., & Beitsch, L. M. (2014). Using resource
dependency theory to measure the environment in health care organizational studies: A systematic review of the
literature. Health Care Management Review, 39(1), 50–65.
Yuen, P. P., & Ng, A. W. (2012). Towards a balanced performance measurement system in a public health care
organization. International Journal of Health Care Quality Assurance, 25(5), 421–430.