Professional Documents
Culture Documents
net/publication/312032295
CITATIONS READS
43 48,597
4 authors:
Some of the authors of this publication are also working on these related projects:
Identifying system model of project supply chain risks using Fuzzy Cognitive Mapping View project
All content following this page was uploaded by Hamed Rahimi on 03 January 2017.
ABSTRACT
Introduction: Performance measurement is receiving increasing verification all over the world. Nowadays in a lot of organizations,
irrespective of their type or size, performance evaluation is the main concern and a key issue for top administrators. The purpose of this
study is to organize suitable key performance indicators (KPIs) for hospitals’ performance evaluation based on the balanced scorecard
(BSC).
Method: This is a mixed method study. In order to identify the hospital’s performance indicators (HPI), first related literature was
reviewed and then the experts’ panel and Delphi method were used. In this study, two rounds were needed for the desired level of
consensus. The experts rated the importance of the indicators, on a five-point Likert scale. In the consensus calculation, the consensus
percentage was calculated by classifying the values 1-3 as not important (0) and 4-5 to (1) as important. Simple additive weighting
technique was used to rank the indicators and select hospital’s KPIs. The data were analyzed by Excel 2010 software.
Results: About 218 indicators were obtained from a review of selected literature. Through internal expert panel, 77 indicators were
selected. Finally, 22 were selected for KPIs of hospitals. Ten indicators were selected in internal process perspective and 5, 4, and 3
indicators in finance, learning and growth, and customer, respectively.
Conclusion: This model can be a useful tool for evaluating and comparing the performance of hospitals. However, this model is flexible
and can be adjusted according to differences in the target hospitals. This study can be beneficial for hospital administrators and it can
help them to change their perspective about performance evaluation.
1
School of Management and Medical Information, Shiraz University of Medical Sciences, Shiraz, Iran
2
Health Human Resources Research Center, School of Management & Information Sciences, Shiraz University of Medical Sciences, Shiraz, Iran
3
Department of Management, Shiraz University, Shiraz, Iran
*Corresponding Author: Z Kavosi, Health Human Resources Research Center, School of Management & Information Sciences, Shiraz University of Medical Sciences,
Shiraz, Iran. Email: zhr.kavosi@gmail.com.
Rahimi H et al.
equip the managers with the information they need for (KPIs) for hospitals’ performance evaluation based on the
the evaluating, controlling and monitoring of hospital’s BSC perspectives using SAW.
current situation and effective technique to assess
and supervise hospital activities (6, 12). Evaluation of Methods
hospital performance is beneficial to payment systems, This is a mixed method study in which, depending on
policymakers, hospitals, and physicians. The assessment various phases of the investigation, various tools have
also assists the managers in promoting the quality of been used (2013-2014). In order to identify the hospital’s
performance (13). performance indicators (HPIs), first related literature was
The main property of a perfect performance evaluation reviewed and then the experts’ panel and Delphi method
system is the accuracy of its outcomes. Thus, it is important were used. The SAW technique was used to rank the
to choose the proper methods and reasonable indicators indicators and select hospital’s KPIs.
that reflect the purpose of the performance evaluation (14).
A set of indicators that properly reflect the organizational Literature review
performance should be set up to fully utilize the function The literature review was carried out to identify relevant
of performance measurement. (15). studies by searching electronic databases. We conducted
Organizational performance is not a simple phenomenon; a search of the literature for articles on PubMed, Scopus,
rather, it is a complex and multidimensional concept Sciences direct, Google scholar, DOJA, SID, Iranmedex,
(16). Various models have been used to manage the Magiran, and Medlib databases.
organizational activity and performance, including total Keywords used for searching was a combination of
production analysis, Analytical Hierarchical Process “hospital,” “performance,” “assessment,” “evaluation,”
(AHP), ratio analysis, Delphi analysis, Data Envelopment “measurement,” “indicator,” and “criteria.” The Boolean
Analysis (DEA), Six Sigma, Total Quality Management OR AND operators were placed between keywords in the
(TQM), and Balanced Scorecard (BSC), (1, 15, 17). searches.
The balanced scorecard is seen as a powerful tool for First, the titles of all articles were reviewed by two of the
organizational change and as an effective performance authors, and then abstracts of the selected articles were
measurement system. reviewed. Then, the full text of the selected articles was
Balanced scorecard developed in 1992 as a management studied and articles in the area of HPIs, which were most
accounting tool that translates an organization’s mission, relevant to our study’s objective, were selected. The quality
strategies, and goals into performance measuring. The of the selected articles was assessed by the Strengthening
BSC consisted of four perspectives such as financial, the Reporting of Observation Studies in Epidemiology
internal process, learning and growth and customer (5, (STROBE) instrument. STROBE instrument was
15, 18). Balanced scorecard perceived as the most proper produced in 2007 to improve the quality of observational
framework is able to provide remarkable information study reporting and allow for critical assessment by others
relevant to the organizational internal and external factors of the strengths and weaknesses in study design, conduct,
that will subsequently contribute to the organization’s and analysis (28). STROBE addresses the three main
success (1). The BSC is used increasingly for measuring types of observational studies: cohort, case-control and
and reporting health system performance. In this method, cross-sectional studies (29).
indicators are located in different perspectives, which
provides a balanced view of performance and guides Expert panel
strategic decisions at the hospitals (19-21). This is for We asked an experts’ panel with five experts in the field
setting up a complete performance evaluation system of health services administration from “health care
and forming a whole set of performance indices to management” and “community medicine” departments of
assess strategies so that the strategies and prospect of Shiraz University of Medical Sciences (SUMS) to identify
organizations could be achieved (11). HIPs from the extracted indicators of literature review
The BSC model indicates that a single performance based on four BSC perspectives. Seventy-seven indicators
indicator could not show the performance of a complex were selected during the internal expert panel.
organization such as hospitals (9, 22). The advantages of the
BSC are classified into three aspects, i.e. communication Delphi methods
and teamwork, commitment, and feedback and learning. We used a Delphi method to reach a consensus among a
The BSC enables the senior management to clarify panel of experts. With Delphi method, the consensus is
vision, develop teamwork, expand strategy, and foster the achieved through a series of rounds. In this study, two
obligation to customer focus in the organization (23, 24). rounds were needed for the desired level of consensus.
Thus, organization performance is a multi-criteria We selected the experts for the Delphi survey to assess
decision-making (MCDM) issue. MCDM is relatively new the validity of the indicators. The experts comprised 17
to be used for the evaluation of performance. MCDM aims professionals from industry and academia. All of the
at using a set of criteria for decision problem (10). One of experts had more than five years of related experience and
the MCDM methods is simple additive weighting (SAW) seven of them had more than fifteen years. In addition,
which is known as a simple and most often used multi- twelve of experts from industry had been managers/
attribute decision technique (25-27). The purpose of this directors of hospitals, and five from academia were
study was to organize suitable key performance indicators professors/researchers with healthcare management
background. We guaranteed anonymity by assigning a Where xij is the score of the ith indicator according to the
code number to each participant. All of the participants jth criteria, Cj is the weighted criteria.
were anonymous volunteers.
The data were collected using paper based survey. So, Results
each expert completed two rounds of the Delphi survey. Results of literature review and expert panel
The first round was developed from indicators compiled A total of 39 studies were identified for inclusion in the
as a result of the expert panel. Experts were given five review. The search of databases provided a total of 14,842
days to complete the surveys. From the results of the first citations. Initially, the title of all the articles was checked
round and comments listed by respondents, the second out and 13,985 were excluded owing to contradiction with
round was developed. During the second round, individual the study objectives. After adjusting for duplicates, 309
and group results from the first round were presented to articles were remained. Of this number, 173 articles were
the panel members and asked to indicate if they agreed omitted because after reviewing the abstracts it emerged
or disagreed with each of remained indicators. All items that these articles clearly did not meet the criteria and
which did not receive agreement from 80% of the panel there was a lack of indication to hospitals’ performance
respondents were removed from the list of indicators. indicators in their results. Sixteen articles were rejected
In Delphi method, data were analyzed by descriptive because the full text of the article was not accessible, the
statistics using Excel software. The experts rated the paper could not be feasibly translated into English, the
importance of the indicators, on a five-point Likert paper was not in the English language, or the type of the
scale (“Not important,” “Somewhat important,” article was not original. The full texts of the remaining
“Relative important,” “Very important,” and “Extremely 120 articles were studied in more detail. It emerged that
important,” respectively). The required level of consensus 64 article did not meet the inclusion criteria. Then, using
was defined. Two necessary conditions had to be fulfilled: the STROBE instrument, we assessed the quality of 56
(1) a mean value of at least 4 (“Very important” and remained articles and 39 of them had the quality needed.
“Extremely important”) and (2) a consensus percentage of Finally, 39 studies (23 English and 16 Persian) met the
at least 80%. In the consensus calculation, the consensus inclusion criteria and were included in the literature
percentage was calculated by classifying the values 1-3 as review (Figure 1). Two hundred ninety-eight indicators
not important (0) and 4-5 to (1) as important. were obtained from a review of selected literature.
Finally, throughout the internal expert panel, 77 indicators
Simple Additive Weighting (SAW) were selected. These performance indicators were
The SAW, which is known as a weighted linear classified into four perspectives of BSC: finance (20),
combination or scoring methods, is a simple and most the customer (5), internal process (37), and learning and
often used multi-attribute decision technique. The SAW growth (15).
is based on the weighted average. The advantage of
simple additive weighting is that it is a proportional linear Evaluation criteria by Delphi method
transformation of the raw data (25-27). The method of data Twenty criteria were selected by the first Delphi round and
collection applied in this phase was a questionnaire, using 30 criteria by the second Delphi round. Results of the two
the Likert scale from 1 to 5 (“Not important,” “Somewhat rounds are shown in Table 1.
important,” “Relative important,” “Very important,” and
“Extremely important,” respectively). The questionnaires Simple Additive Weighting
were completed by the participants of the Delphi method. First, the weights of the criteria were computed by using
The process of SAW consisted of the following steps (25- five experts’ opinion with Interview: Necessity (C1) 0.4,
27): Specificity (C2) 0.15, Relatively (C3) 0.15, and Measurable
Step 1: (C4) 0.3.
Determining and computing the weight of the criteria Then, we calculated the normalized decision matrix for
were used as a reference in decision-making, namely Ci. the criteria used in this study; the criteria were positive,
In this study, the four selected criteria were necessity, so we used formula 1, and the result is shown in Table 2.
specificity, relatively, and measurable. The SAW method evaluates each alternative (Ai) by
Step 2: formula 3. In this step, indicators of each BSC perspective
Making a decision matrix (m × n) that includes m indicators are ranked separately. Finally, in SAW method, the best
and n criteria. Calculating the normalized decision matrix indicators were P1, F9, G3, and C2 in the internal process,
for positive criteria (1) and negative criteria (2): finance, learning and growth and customer perspective,
respectively. The results of SAW method are shown in
Table 3.
C1 C2 C3 C4 C1 C2 C3 C4
Finance (F) F1 0.9807 0.9744 0.9956 1 F6 0.6959 0.8148 0.7239 0.8063
F2 0.9464 0.9404 1 0.9473 F7 0.6959 0.8510 0.7782 0.9115
F3 0.8565 0.8680 0.8695 0.9305 F8 0.8565 0.9574 0.8869 0.9642
F4 0.8736 0.8680 0.8152 0.8947 F9 1 1 0.9782 0.9831
F5 0.9271 0.9936 0.9608 0.9473 F10 0.7494 0.8340 0.6891 0.7536
Internal P1 0.9737 0.9938 0.9917 1 P15 0.9576 0.9259 0.9404 0.9
Process (P)
P2 0.8407 0.8909 0.8562 0.9 P16 0.9415 0.8909 0.8377 0.884
P3 0.9415 1 0.9917 0.984 P17 0.9415 0.8909 0.8891 0.75
P4 0.9677 0.9094 0.8377 0.95 P18 0.9737 0.9423 0.9589 0.916
P5 0.8729 0.9773 0.9589 0.984 P19 0.9415 0.9094 0.9589 0.9
P6 0.8911 0.9094 0.9075 0.934 P20 0.8064 0.7037 0.8726 0.684
P7 0.8407 0.9094 0.9075 0.9 P21 0.8064 0.7037 0.7864 0.716
P8 0.9072 0.9259 1 0.934 P22 0.7721 0.8230 0.8562 0.766
P9 0.9737 0.8580 0.9589 0.85 P23 0.8407 0.8395 0.8562 0.884
P10 1 0.7716 0.9240 0.7 P24 0.8911 0.8909 0.9075 0.9
P11 0.8225 0.8744 0.8213 0.884 P25 0.8729 0.8395 0.8377 0.834
P12 0.7217 0.7037 0.7864 0.75 P26 0.9919 0.9259 0.9240 0.9
P13 0.7721 0.7366 0.7864 0.766 P27 0.9576 0.9259 0.9075 0.9
P14 0.9415 0.7716 0.9404 0.65 P28 0.9737 0.8744 0.8726 0.834
Learning and G1 1 0.8687 1 0.8201 G6 0.9115 0.7979 0.8800 0.8565
Growth (G)
G2 0.8589 1 0.9184 0.8565 G7 0.8252 0.7979 0.8201 0.9100
G3 0.9305 0.9020 0.8800 1 G8 0.8947 0.8166 0.8800 0.9464
G4 0.7536 0.7125 0.8201 0.7494 G9 0.8063 0.8166 0.8393 0.9464
G5 0.5957 0.6937 0.6402 0.7665
Customer (C) C1 0.8163 0.8423 0.8155 0.8047 C4 0.7306 0.8047 0.8511 0.7835
C2 1 1 1 1 C5 0.8326 0.7247 0.7777 0.6282
C3 0.9857 0.9411 0.9622 0.9011
Table 3. The ranked indicators of hospital performance
21. Chiu WT, Yang CM, Lin HW, Chu TB. Development and 31. Raeisi AR, Yarmohammadian MH, Bakhsh RM, Gangi H.
implementation of a nationwide health care quality indicator Performance evaluation of Al-Zahra academic medical center
system in Taiwan. International journal for quality in health care based on Iran balanced scorecard model. Journal of education
: journal of the International Society for Quality in Health Care and health promotion. 2012;1:1-7.
/ ISQua. 2007;19(1):21-8. 32. Olden PC, Smith CM. HOSPITALS, COMMUNITY HEALTH,
22. Urrutia I, Eriksen SD. Application of the balanced scorecard in AND BALANCED SCORECARDS. Academy of Health Care
Spanish private health-care management. Measuring Business Management Journal. 2008;4(1 %@ 1559-7628):39-56.
Excellence. 2005;9(4):16-26. 33. Niven PR. Balanced scorecard step-by-step: maximizing
23. Banker RD, Chang H, Pizzini MJ. The balanced scorecard: performance and maintaining results: John Wiley & Sons; 2002.
Judgmental effects of performance measures linked to strategy. 34. Iravani Tabrizipour AP, Fazli S, Alvandi M. Using combined
The Accounting Review. 2004;79(1):1-23 %@ 0001-4826. approach of FAHP-BSC to evaluate the performance of Hashemi
24. Aranda C, Arellano J. Consensus and link structure in strategic Nejad Hospital Tehran. Health Information Management.
performance measurement systems: A field study. Journal of 2012;9(3):327- 38.
Management Accounting Research. 2010;22(1):271-99 %@ 35. Kocakulah MC, Austill AD. Balanced scorecard application
1049-2127. in the health care industry: a case study. Journal of health care
25. Afshari A, Mojahed M, Yusuff RM. Simple additive weighting finance. 2007;34(1):72-99.
approach to personnel selection problem. International Journal 36. Arab M, zarei a, rahimi a, rezaeian f, akbari f. Analysis of
of Innovation, Management and Technology. 2010;1(5):511- 5. Factors Affecting Length of stay in Public Hospitals in Lorestan
26. Amiri M, Darestani Farahani A. Multi criteria decision making. Province, Iran. Hakim Research Journal. 2010;12(4):27-32.
1 ed. Iran: Kian; 2013. 248 p. 37. Kaplan RS, Norton DP. The balanced scorecard: translating
27. Asgharpour MJ. Multiple Criteria Decision Making. 10 ed. Iran: strategy into action: Harvard Business Press; 1996.
University of Tehran; 2011. 232 p. 38. barati a, maleki m, golestani m, imani a. Evaluation of Staff
28. Sorensen AA, Wojahn RD, Manske MC, Calfee RP. Using Performance Management System for Semnan Amir Al-
the Strengthening the Reporting of Observational Studies in Momenin Hospital using Balanced Scorecard. Journal of Health
Epidemiology (STROBE) Statement to Assess Reporting of Management. 2006;9(25):47-54.
Observational Trials in Hand Surgery. The Journal of Hand 39. Peters DH, Noor AA, Singh LP, Kakar FK, Hansen PM, Burnham
Surgery. 2013;38(8):1584-9.e2. G. A balanced scorecard for health services in Afghanistan.
29. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Bulletin of the World Health Organization. 2007;85(2):146-51.
Mulrow CD, Pocock SJ, et al. Strengthening the Reporting
of Observational Studies in Epidemiology (STROBE):
Explanation and elaboration. International Journal of Surgery.
2014;12(12):1500-24.
30. Nikjoo RG, Beyrami HJ, Jannati A. Selecting Hospital’s
Key Performance Indicators, Using Analytic Hierarchy
Process Technique. Journal of Community Health Research.
2013;2(1):30-8.