THE INTERNATIONAL JOURNAL OF HEALTH PLANNING AND MANAGEMENT

Int J Health Plann Mgmt 2014; 29: e187–e206.

Published online 28 January 2014 in Wiley Online Library
([Link]) DOI: 10.1002/hpm.2237

Developing a national framework of quality

indicators for public hospitals
Effie Simou*, Paraskevi Pliatsika, Eleni Koutsogeorgou
and Anastasia Roumeliotou
Department of Epidemiology and Biostatistics, National School of Public Health, Athens, Greece

SUMMARY
Background The current study describes the development of a preliminary set of quality indi-
cators for public Greek National Health System (GNHS) hospitals, which were used in the
“Health Monitoring Indicators System: Health Map” (Ygeionomikos Chartis) project, with the
purpose that these quality indicators would assess the quality of all the aspects relevant to public
hospital healthcare workforce and services provided.
Methods A literature review was conducted in the MEDLINE database to identify articles
referring to international and national hospital quality assessment projects, together with an online
search for relevant projects. Studies were included if they were published in English, from 1980 to
2010. A consensus panel took place afterwards with 40 experts in the field and tele-voting
procedure.
Results Twenty relevant projects and their 1698 indicators were selected through the literature
search, and after the consensus panel process, a list of 67 indicators were selected to be
implemented for the assessment of the public hospitals categorized under six distinct dimensions:
Quality, Responsiveness, Efficiency, Utilization, Timeliness, and Resources and Capacity.
Conclusion Data gathered and analyzed in this manner provided a novel evaluation and moni-
toring system for Greece, which can assist decision-makers, healthcare professionals, and patients
in Greece to retrieve relevant information, with the long-term goal to improve quality in care in
the GNHS hospital sector. Copyright © 2014 John Wiley & Sons, Ltd.

KEY WORDS: quality indicators; hospital sector; evaluation; healthcare services

INTRODUCTION

The evaluation of quality in healthcare has been a growing field in the last couple of
decades, as the need for evidence-based decision-making, quantifiable improvement,
and information useful for benchmarking has been manifested in many aspects of
caregiving (Campbell et al., 2000). Quality of performance in healthcare services is
of uttermost importance for stakeholders related to the healthcare industry, whether
regarding health professionals, policy makers, or service users. Given the fact that the
majority of annual health expenditure is invested on hospital healthcare at national

*Correspondence to: E. Simou, Department of Epidemiology and Biostatistics, National School of Public
Health (ESDY), 196, Leoforos Alexandras, 11521, Athens, Greece. E-mail: esimou@[Link]

Copyright © 2014 John Wiley & Sons, Ltd.

e188 E. SIMOU ET AL.

level, mostly because of the high costs associated with hospital care and the profile that
such organizations hold to the public eye (Marshall et al., 2006), projects regarding
quality in hospital performance have been developed and quality indicators have been
employed in a variety of settings and facing different goals. According to Ham et al.
(2012), future demand of health services can be predicted through the monitoring of
data on the utilization of services, and at population level, epidemiological data not only
can help monitor health profiles/lifestyles but also can contribute to the delivery of
better health outcomes and care for “at risk” groups at community and household
levels. Additionally, it is important to put in the hands of both clinicians and patients
data and information on health and healthcare, because only then can the health and
social care system fully meet the demands of its population (Ham et al., 2012).
In line with the already widespread trend of healthcare quality assessment, there has
recently been intense debate on evaluating performance of the public Greek National
Health System (GNHS), with major service providers and policy-sensitive receivers
in Greece being the GNHS hospitals. Specialized inpatient care in Greece is provided
either by public hospitals, mainly GNHS hospitals, or by private clinics. GNHS hospi-
tals are characterized as general or as specialized, the latter signifying referral healthcare
centers for a single or a specific number of specialties (Economou, 2010). General
GNHS hospitals, which are the main providers of hospital care, with at least one such
facility available at each district, offer a variety of diagnostic and therapeutic healthcare
services on outpatient or inpatient basis, while during each hospital’s on-call day, out-
patient departments receive patients requiring emergency care. At its total, personnel
and infrastructure costs within public, GNHS hospitals are financed by the state, under
regulations of the Ministry of Health (MoH), whereas healthcare services provided to
patients are either covered through arrangements with social insurance funds, by private
insurance, or by out-of-pocket money by patients.
Although operation of public GNHS hospitals is governed by the MoH at a central
basis, the volume of resources needed at each district has not been carefully determined
or updated according to a national scheme; administrative and capacity parameters had
not been standardized and reliably updated in the past for public hospitals, and this was
a need that the MoH was called upon to fulfill. Moreover, while data have been
collected from public hospitals, either through routine internal recording, national
census, or hospital-based surveys, only few efforts of utilizing the data to the patients’
advantage or for evaluation purposes had been made in the past. Most efforts related to
attempts of intra-organizational quality assessment have been made with no in-depth
analysis on possible improvement of procedures. The arising needs for rational and
efficient distribution of healthcare resources, improvement of population’s general
health status, and progress in the field of specialized healthcare posed the challenge
to provide reliable and updated information on available services and quality of public
hospitals, and to collect benchmarking information at national level, which could lead,
in the near future, to renovation, a needs-based strategically planned public healthcare
system and reduction of disparities in healthcare usage.
Greece, lying in the jurisdictional territory of organizational initiatives such as
WHO’s and Organisation for Economic Co-operation and Development’s (OECD’s)
quality projects, has been providing public hospital data for the purposes of the afore-
mentioned quality frameworks; however, those initiatives are international and not

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e189
specifically sensitized to current healthcare demands of the Greek population, plus they
mainly cover matters of clinical quality aspects of healthcare—not of capacity or
management of hospital care services. The national health resources monitoring project
“Health Monitoring Indicators System: Health Map” (Ygeionomikos Chartis) is a
pioneer project at national level in Greece, recently engaged at developing standards
for healthcare services, while quality indicators useful for assessment of health system’s
performance have been carefully selected according to major topics at hand. Partici-
pation in data collection and provision was mandatory for all public GNHS hospitals,
to record the full extent of information, which would be valuable for improvement
decisions and for benchmarking procedures among hospitals.
The current study describes the development of a preliminary set of quality indicators
for public GNHS hospitals, which were used in the “Health Map” project, with the
purpose that these quality indicators would assess the quality of all the aspects relevant
to public hospital healthcare workforce and services provided.

METHODS

The development of the quality indicators that were used for the preliminary version of
the Greek “Health Map” project regarding the public hospitals’ sector derived from a
process including, firstly, a literature review and, secondly, a consensus procedure.

Literature review
When handling quality measurements in the form of quality indicators, one expects
them to have been developed and validated following scientific soundness and useful-
ness, and to cover all challenging dimensions of health caregiving. Indicator develop-
ment and realization is not always governed by hierarchical listing of information
necessity but can sometimes be regulated by availability of records, making the scope
of healthcare assessment rather insufficient and the inferential power of the reports rather
limited, and possibly further obscuring the consumers’ decision-making processes
(Freeman, 2002; Copnell et al., 2009; Evans et al., 2009). Therefore, preceding a com-
prehensive approach to healthcare monitoring, a systematic review of previous evidence
is essential to collect information on usefulness of measures that could be utilized.
On the basis of the aforementioned items, before developing a national set for quality
assessment in hospitals, a review of quality indicator projects on hospital performance
already employed in other countries at national or international level was carried out.
Electronic literature search was conducted in the MEDLINE database to identify arti-
cles referring to international and national hospital quality assessment projects. Studies
were included if they were published in English, from 1980 to 2010. The search was
performed using various combinations of the following search terms/keywords: quality
indicators, hospital, measure, assess, and/or evaluate. The reference lists of the
selected studies were reviewed as well to trace relevant information on the projects
selected, and internet search (i.e., via Google) was also performed, with the same key-
words, to retrieve information for more existing projects and/or relevant gray literature.
At the end, the official internet website of the selected papers was traced and

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e190 E. SIMOU ET AL.

investigated for updated description or technical information on the indicators included

in the corresponding projects.
No publication types were excluded from the current review, apart from reviews,
editorials, and letters, because these types of publication were considered to provide
inadequate information on the design and implementation of the projects in question.
The studies that referred to projects implemented at hospitals were analyzed further.
The literature review and the selection and eligibility of the initial list of indicators
that met the inclusion criteria were performed independently by two reviewers
(E. S. and P. P.), and in case of disagreement, a third reviewer was consulted to reach
final decision (A. R.). There was an 80–90% agreement between the initial two
reviewers, regarding the selected list of indicators.

Consensus procedure
Selection of quality standards required to reduce the amount of the indicators that
would emerge through the literature search; therefore, the initial list of quality indi-
cators had to be reduced to a practical and balanced amount of standards/indicators.
For this reason, following the step of assembling information on existing indicators,
40 experts on hospital care—independent, specialized, and/or recognized experts
who covered all the aspects of the topic under scrutiny (such as policy makers, health
professionals, researchers, and healthcare managers)—were asked to deliver assistance
in successfully selecting useful standards for Greek public hospitals. The consensus
panel evaluation of the indicators took place in Athens in November 2010. Because a
relative quality-evaluating framework or previous scientific research on quality was
practically nonexistent in the Greek territory, it was decided that a consensus procedure
for the experts invited should be conducted (Jones and Hunter, 1995; Campbell et al.,
2002). A consensus panel was organized, with a preparatory round performed on postal
level and the final round performed through a tele-voting procedure. The preliminary
list of quality indicators identified through literature review was initially posted to the
experts participating in the consensus procedure, in questionnaire form, to gain famil-
iarity. The same group of experts was, within one month, called to participate in a
tele-voting panel; all of the experts summoned accepted the invitation.
Experts were asked to rate, through tele-voting, the quality indicators in terms of im-
portance (impact on health status, policy relevance, and susceptibility to being
influenced by the public health system performance) and feasibility (data availability,
reporting burden), on a 5-point Likert-type scale ranging from “1—Not important” to
“5—Very important” and from “1—Not feasible” to “5—Very feasible”. Experts were
also encouraged to propose additional standards/indicators and to propose rephrasing of
indicators that were unclear at initial presentation. Through this procedure, only quality
indicators that were both evaluated as highly important and highly feasible by the
expert panel were included in the preliminary “Health Map” set of quality indicators
for hospital care.

Data analysis
The indicators selected for use in public hospitals by the consensus panel were catego-
rized by the authors under dimensions. The indicators varied in terms of features and

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e191
domains they covered; thus, a descriptive approach was preferred to analyze the se-
lected projects. The reviewers compared each project’s volume and dimensional com-
position with the overall trend for quality assessment in hospital care services, while
considering the objectives and data collection and analysis potentials of each project.
Finally, after the selection of standards that would be used, technical specifica-
tions and data collection tools were also finalized and prepared for distribution. In
addition to the standardized collection tools proposed, some quality indicators would
require the additional use of surveys and data from other sources (e.g., census data),
which were also proposed for the future by the experts of the panel. The standards
and tools suggested for evaluation of quality in hospital care were selected keeping
in mind the need for thriftiness in requirements and personnel burden.

The Health Map project

In Greece, no standardized tool had ever systematically been used for recording health-
care data in specific; apart from the healthcare data being collected by the Hellenic
Statistical Authority ([Link].)—among other types of data. The Health Map project
was designed on the basis of data collection tools that have been implemented in other
countries for health and healthcare monitoring and evaluation purposes. The result of
the selection process was the development of a web-based information platform that
would be updated regularly with raw data on health and healthcare from public and pri-
vate providers around Greece. The MoH urged all healthcare providers to participate in
the data collection process. The data collected concerned the recording of a variety of
information (quantitative and qualitative variables/items) on the healthcare workforce
and the health services provided by the public GNHS (Ethniko Systima Ygeias).
The information database of the “Health Map” has been grounded on four parame-
ters: (i) the insertion, updating, coding procedure, and archiving of data; (ii) the
geographical–topical classification and organization of indicators; (iii) the publication
of customizable reports; and (iv) the availability of data to the users through a friendly
environment that supports the access via the worldwide web (WWW) and electronic
mail (e-mail). All “Health Map” online forms included a unified set of structured or
semi-structured questionnaires all requiring information on predefined categories. Data
sources have been documented with their variables and procurement methods. The
Health Map project’s information database is currently available to the public via the
official website ([Link]

RESULTS

Quality indicators that could be used for evaluation of hospital performance were iden-
tified through the literature review, and after removal of duplicate standards, an initial
list was selected with 1698 quality indicators (Table 1) that could potentially be used
for monitoring hospital care services. More specifically, through the literature search,
we managed to retrieve several articles on assessment of quality in specialized inpatient
healthcare services, including two reviews of such projects and indicator frameworks
(Groene et al., 2008; Copnell et al., 2009), with the result of 20 main projects being

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 Table 1. Hospital care quality-assessing projects and their corresponding dimensions and number of indicators (total N = 1698) selected through the
e192

literature review

Country and year Number of

of launching the project Organization: Project title Source Dimensions of indicators indicators*
International, 2003 WHO: Performance Assessment Tool Veillard et al., 2005 Clinical effectiveness, 41 (17 core)
for Quality Improvement in safety, efficiency, patient-
Hospitals (PATH) centeredness, staff
orientation, responsive
governance
International, 2002 OECD: Health Care Quality Arah et al., 2006 Clinical effectiveness, 15
Indicators (HCQI) project— safety, patient-

Copyright © 2014 John Wiley & Sons, Ltd.

Patient Safety centeredness, timeliness
USA, 1984/ Maryland Hospital Association Kazandjian et al., 2003 Clinical effectiveness, 47
International, 1991 (MHA): Quality Indicator Project safety, efficiency, patient-
(QIP)/Press Ganey Associates: centeredness, timeliness
International QIP (IQIP)
Europe, 2004 The European Commission: Torre et al., 2007 Clinical effectiveness, 45
E. SIMOU ET AL.

European Public Health efficiency

Outcome Research and
Indicators Collection (EUPHORIC)
Europe, 2007 The European Commission: [Link] Clinical effectiveness, 42
Safety Improvement for [Accessed on safety, patient-
Patients in Europe (SImPatIE) (Safety 15 December 2011] centeredness
Improvement for Patients in Europe
(SImPatIE), Europe, 2011)
Australia, 1989 The Australian Council Collopsy et al., 2000 Clinical effectiveness, 338
on Healthcare Standards safety, efficiency,
(ACHS): Clinical timeliness
Indicators project
Australia, 2004 Australian Institute of Health and [Link] Clinical effectiveness, 8 for sentinel
Welfare (AIHW): Sentinel events in au/publications/index. safety, efficiency, patient- events/10

DOI: 10.1002/hpm
Int J Health Plann Mgmt 2014; 29: e187–e206.
 Table 1. (Continued)

Country and year Number of

of launching the project Organization: Project title Source Dimensions of indicators indicators*

Hospitals project/National Health cfm/title/10353 [Accessed on centeredness, staff hospital-

Performance Committee (NHPC) 15 December 2011]; orientation, timeliness related in
(Australian Institute of Health and and [Link] NHPC
Welfare (AIHW), 2011a; Australian indicators/[Link] [Accessed on 15
Institute of Health and Welfare December 2011]
(AIHW), 2011b)
Canada, 2001 Ontario Hospital Association (OHA)/ [Link] Clinical effectiveness, 158
Hospital Report Research [Accessed on 15 December 2011] safety, efficiency, patient-
Collaborative (HRRC) (Hospital centeredness, responsive

Copyright © 2014 John Wiley & Sons, Ltd.

Report Research Collaborative, governance
Canada, 2011) /Health System
Performance Research Network
(HSPRN): Hospital Reports
Canada, 2010 Canadian Institute for Health [Link] [Accessed on 15 Clinical effectiveness, 32
Information (CIHI): Canadian Hospital December 2011] safety, efficiency, patient-
Reporting Project (CHRP) (Canadian centeredness, staff
Institute for Health Information, 2011) orientation, timeliness
Denmark, 2000 Danish Ministry of Health/Danish [Link] [Accessed on 15 Clinical effectiveness, 87
National Board of Health (et al.): December 2011] safety, efficiency, patient-
The National Indicator Project (NIP) centeredness, timeliness
(The Danish National Indicator
Project, Denmark, 2011)
France, 2003 French Ministry of Health: Grenier-Sennelier et al., Clinical effectiveness, 43
A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS

COMPAQH 2005 [in French] patient-centeredness, staff

orientation
Germany, 2000 BQS—Bund The Gemeinsamer Bundesgeschaftsstelle Clinical effectiveness 194 (26 core)
Bundesausschuss (GBA) agency: Qualitatssicherung.
e193

DOI: 10.1002/hpm
Int J Health Plann Mgmt 2014; 29: e187–e206.
(Continues)
 Table 1. (Continued)
e194

Country and year Number of

of launching the project Organization: Project title Source Dimensions of indicators indicators*

Bundesgeschaftsstelle (Bundesgeschaftsstelle
Qualitatssicherung (BQS) Qualitatssicherung, 2007) Qualitat
(Bundesgeschaftsstelle sichtbar machen: BQS—
Qualitatssicherung, 2007) Qualitatsreport 2006. Dusseldorf:
BQS, 2007 [in German]
The Netherlands, 2003 The Dutch National Institute for Public Berg et al., 2005 Clinical effectiveness, 39
health and the Environment (RIVM): safety, efficiency, patient-
Dutch Healthcare Performance Reports centeredness, timeliness
(hospitals)

Copyright © 2014 John Wiley & Sons, Ltd.

United Kingdom, 2000 NHS Quality Improvement (Scotland): [Link] Clinical effectiveness 64
Clinical Indicators Support Team [Link] [Accessed on 15
(Clinical Indicator Support Team, December 2011]
2011)
United States of Joint Commission on Accreditation of [Link] Clinical effectiveness, 70
E. SIMOU ET AL.

America, 1997 Health Care Organizations (JCAHO): PerformanceMeasurement/ safety, efficiency, patient-
Hospital Accreditation Standards [Accessed on 15 December 2011] centeredness, timeliness
(Joint Commission on Accreditation of
Health Care Organizations (JCAHO),
2011)
United States of National Quality Forum (NQF): [Link] Clinical effectiveness, 113
America, 1999 Endorsed Standards (ES) for [Accessed on 15 December 2011] safety, efficiency, patient-
Hospital care (National Quality Forum, centeredness, timeliness
USA, 2011)
United States of Agency for Healthcare Research and [Link] Clinical effectiveness, 27 for patient
America, 2000 Quality (AHRQ): Quality Indicators gov/ [Accessed on 15 December safety Safety, 34 for
(Agency for Healthcare Research and 2011] inpatient care,
Quality (AHRQ): Quality Indicators 18 pediatric
project, 2011)

DOI: 10.1002/hpm
Int J Health Plann Mgmt 2014; 29: e187–e206.
 Table 1. (Continued)

Country and year Number of

of launching the project Organization: Project title Source Dimensions of indicators indicators*
United States of The Leapfrog Group: Hospital [Link] Clinical effectiveness, 54
America, 2000 Quality and Safety Survey (Leapfrog [Accessed on 15 December 2011] safety, efficiency, patient-
Hospital Quality and Safety Survey, centeredness, timeliness
USA, 2011)
United States of Centres for Medicaid and Medicare [Link] Clinical effectiveness, 101 (38 core)

Copyright © 2014 John Wiley & Sons, Ltd.

America, 2004 Services (CMMS)/Hospital Quality HospitalQualityInits/ [Accessed on safety, efficiency, patient-
Alliance (HQA): Hospital Compare 15 December 2011] centeredness, timeliness
(Centres for Medicaid and Medicare
Services (CMMS)/Hospital Quality
Alliance (HQA): Hospital Compare,
USA, 2011)
Switzerland, 2000 The Canton of Zurich: Verein Luthi et al., 2002 Clinical effectiveness, 118
Outcome safety, efficiency, patient-
centeredness, responsive
governance
“Number of indicators” refers to hospital-specific indicators of a more general spectrum found in respective projects
A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS
e195

DOI: 10.1002/hpm
Int J Health Plann Mgmt 2014; 29: e187–e206.
e196 E. SIMOU ET AL.

identified through our literature review as relevant to the aim of the study. Description
of the projects selected is presented in Table 1.
Most existing frameworks were launched in the last decade, varied in number of
used dimensions and indicators, and were employed at national level. However, five
international frameworks were also identified: Performance Assessment Tool for
Quality Improvement in Hospitals (PATH), which was originally implemented by
WHO and is perhaps the most global project to present (Veillard et al., 2005); Inter-
national Quality Indicator Project (IQIP), which is the international modification of
the Quality Indicator Project (QIP) originally developed in the USA (Kazandjian
et al., 2003); the European Public Health Outcome Research and Indicators Collection
(EUPHORIC) (Torre et al., 2007) and the Safety Improvement for Patients in Europe
(SImPatIE) (Safety Improvement for Patients in Europe (SImPatIE), Europe, 2011)
projects, both developed and funded by the European Commission (EC) and mainly
focused on health outcomes and patient safety, respectively, throughout the European
region; plus, OECD’s Health Care Quality Indicators (HCQI) project’s indicators fo-
cusing on patient safety (Arah et al., 2006). Among frameworks implemented at na-
tional level, most existing quality indicator sets originated from the USA, serving a
variety of objectives, while Canada, Australia, and many European countries also
presented hospital-assessing quality projects. Mainly depending on the voluntary or
mandatory nature of each framework, the results of the data analysis can be disclosed
to the broader public, used for internal quality purposes only, or dispensed at a central-
ized, benchmarking organization.
Performance Assessment Tool for Quality Improvement in Hospitals—sometimes
compared with other quality frameworks as the golden standard (Groene et al., 2008)
—was originally launched in 2003 and is run by the WHO Regional Office in Eu-
rope. PATH addresses a variety of national hospital settings in a comprehensive
manner, including six essential dimensions of quality in hospital performance in
four main domains (clinical effectiveness, efficiency, staff orientation, and respon-
sive governance) and two transversal perspectives (safety and patient-centeredness)
(Arah et al., 2003). Its main purpose of development was to assist in a
multidimensional assessment and improvement of quality in hospital care, at the
local, national, or international level, making use of data previously remaining
unused. Four of the dimensions used in PATH are also suggested by the Institute of
Medicine (IOM), namely safety, effectiveness, efficiency, and patient-centeredness
(Vanselow et al., 1995). IOM also suggests the use of indicators measuring timeliness
and equity in evaluating healthcare quality, therefore empowering indicators related to
accessibility and, in the case of equity, indicators with a broad field of implementation
(Copnell et al., 2009). All the dimensions mentioned earlier, namely those mentioned
in PATH and by the IOM, were evaluated while developing the “Health Map” set of
quality indicators. Finally, from the selected projects, as mentioned earlier, a list of
1698 indicators was assessed for eligibility in the current study to be evaluated by
the consensus panel.
The consensus panel process resulted in the selection of 67 indicators (Table 2),
which comprised the preliminary set of indicators of the Greek “Health Map” project
and were used in the pilot study of the project—being the basis of the ongoing web-
based information system developed for the Greek “Health Map” study—which was

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e197
Table 2. The NHIF quality indicators (N = 67) selected for the assessment of hospital care, by
dimensions and sub-dimensions/features
QUALITY
Effectiveness
• Inpatient mortality from selected causes (AMI*, stroke, pneumonia, CABG*, hip fracture,
pneumonia)
• Readmission rate for selected causes (AMI, stroke, pneumonia, CABG, hip fracture,
pneumonia, asthma, diabetes mellitus)
• Unscheduled readmission to ICU*
• Perioperative mortality
• Perinatal mortality due to complications (mother, child)
• Cancer patients successfully surviving surgery/chemotherapy/transplant
Safety
• In-hospital avoidable VTE*
• Hospital-acquired infections (VAP*, urinary catheter associated UTI*, central line associated
blood stream infection, surgical site infection, infections in neonates)
• Medical errors per sector (following surgery, improper or delayed treatment, iatrogenic
complications)
• Obstetric trauma
• Staff injury
• Staff needle puncture incidents
RESPONSIVENESS
Patient centeredness
• Patient feedback management
• Pain control
• Satisfaction from personnel
• Explanation of procedures, treatment and discharge information
• Satisfaction from hospital environment (cleanliness, quietness, privacy)
Staff orientation
• Staff burnout
• Staff absenteeism
• Staff working overtime
• Satisfaction from working environment
• Clearly defined responsibilities in staff
• Continuous education for health professionals
EFFICIENCY
• Length of stay
• ICU length of stay
• Hospital bed coverage
• Admission/discharge rate
• Cost of inpatient services per patient day
• Exams ordered at the ER*, per patient
• Laparoscopic/open surgery rate
• Single-day stay for selected surgeries
• Caesarian section rate
• Surgery postponed or canceled
UTILIZATION
• Patients visiting the ER department
• Admissions for acute conditions
• Usage of equipment/facilities
• Usage of laboratory exams
• Surgical Theater use
(Continues)

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e198 E. SIMOU ET AL.

Table 2. (Continued)
TIMELINESS
• Time needed for initial clinical examination at the ER after arrival
• Time needed for admission after arrival at the ER
• Time needed for selective surgical treatment
• Patients leaving without being examined
RESOURCES and CAPACITY
Human resources
• Permanent personnel (per discipline)
• Detached personnel (per discipline)
• Temporary personnel (per discipline)
• Personnel educational level (per discipline)
• Intra-sector nurses to physicians ratio
Information technology
• Computers for the personnel
• Computers with Internet access
• Computers with modern applications
• Use of electronic medical records
• Hospitals having a webpage
• Telephone center
Infrastructure and facilities
• Surgical theaters
• Beds per sector
• Beds per room
• Short-term stay beds
• Space for patient baggage
• Toilet in patients’ rooms
• Intra-communication facilities in patients’ rooms
• Oxygen facilities in patients’ rooms
• Air-conditioning facilities in patients’ rooms
• Telephone facilities in wards
• Imaging facilities (radiography, ultrasound, CT*, MRI*, mammography, gamma-camera,
DSA*, PET*)
• ICU and HCU* unit(s)
• Hemodialysis facilities
• Management of hospital waste
*AMI, acute myocardial infarction; CABG, coronary artery bypass graft; ICU, intensive care
unit; VTE, venous thromboembolism; VAP, ventilator-associated pneumonia; UTI, urinary
tract infection; ER, emergency room; CT, computerized tomography; MRI, magnetic reso-
nance imaging; DSA, digital subtraction angiography; PET, positive emission tomography;
HCU, high care unit.

firstly implemented in its current form in 2011. The indicators selected for use in public
hospitals by the consensus panel were categorized under six distinct dimensions
(N = number of indicators included): Quality (N = 12), Responsiveness (N = 11),
Efficiency (N = 10), Utilization (N = 5), Timeliness (N = 4), and Resources and Capacity
(N = 25).
Because of the early stage of quality-assessing efforts in the sector of hospitals in
Greece and taking into consideration the lack of nationwide information on hospital
personnel, equipment, facilities, and infrastructure, an extensive variety of quality in-
dicators was dedicated to administrative subjects, meaning the dimension of

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e199
resources and capacity. All administrative indicators proposed were expected to be
measured by raw data that directly originated from the “Health Map” questionnaires
and that can usually be recorded by organizational staff in hospitals as routine infor-
mation, but are usually not collected centrally, standardized, or coded; these measures
are aligned with previous knowledge of the managerial status in Greek public hospi-
tals, and indicators were carefully proposed according to this knowledge (or therefore
lack of).
On the other hand, the remaining indicators of the “Health Map” set were balanced
among all other dimensions, most indicators dealing with quality, including clinical
effectiveness and safety standards, which are areas highly prioritized in current quality
indicator sets for hospitals and reflect evidence-based medical applications. Indicators
on clinical effectiveness and safety in the “Health Map” framework’s quality set
covered a wide range of in-hospital outcomes and were mostly selected from larger
sets used abroad or globally, first and foremost being the set of PATH indicators, while
other quality frameworks such as the Joint Commission on Accreditation of Health
Care Organizations (JCAHO), (Joint Commission on Accreditation of Health Care Or-
ganizations (JCAHO), 2011) IQIP, the EC-funded projects (EUPHORIC, and
SImPatIE) and OECD’s HCQI were also carefully considered for additional quality
indicators; indicators in these sets were found to be extremely important though feasi-
bility in the GNHS was not always their strength, on the basis of data collection so far,
while standardized data collection tools were expected to moderate this difficulty.
Responsiveness, meaning patient-centeredness and staff orientation, and timeliness
indicators, based on similar measurements in global frameworks such as PATH and
the Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project,
(Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project,
2011) were also rated high in terms of importance, while valuable sources of data
relevant to these areas, especially for measures regarding patient feedback, were
surveys among in-hospital patients. Lastly, indicators related to efficiency and uti-
lization, most of which would make use of administrative data collected by the
standardized questionnaires, were two dimensions that counterbalanced the set of
quality indicators and were found to be widely used abroad as well as highly appre-
ciated by the consensus panel.

DISCUSSION

Hospital care, being the health sector that receives the largest volume of financial
support and massive policy-related interest, has generally been in the epicenter of
healthcare evaluation, especially during the last decade that most of the internationally
utilized healthcare quality projects were launched. In our study, to proceed to the devel-
opment of a quality indicator framework for Greek public hospitals, we attempted to
depict the current global status of hospital evaluation by reviewing existing sources
of quality indicators for hospitals; we also tried to identify possible areas that would
require additional attention in the GNHS and, specifically, in public hospitals, which
in turn would denote subjects and dimensions that would need surplus caution in their
evaluation approach.

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e200 E. SIMOU ET AL.

In our review, we identified several quality projects regarding hospital performance,

most of which were employed at national level, while five projects expanded to an in-
ternational fit for hospital evaluation (i.e., PATH, IQIP, OECD’s HCQI, EUPHORIC,
and SImPatIE). Differences on volume and final selection of dimensions of quality
measurements were noted, and were also expected to a point, as each project appealed
to a specific accommodation and was implemented through a distinct strategy, with
some projects being compulsory, while others were voluntary and served as a guide
to improving quality (Groene et al., 2008; Copnell et al., 2009), and as projects were
based on different methodological backgrounds, although the basic quality dimensions
are usually covered by the IOM’s framework; these differences make an attempt of
crosscheck of indicators difficult between projects. Differences in definitions and
methods of data collection and analysis of quality indicators do not allow comparisons
of findings across different frameworks, so in our case, this would present problems
regarding the parallel operation of alternative quality-assessing projects in the same
geographical area (e.g., of Europe or Greece) where internationally implemented
projects operate in parallel with national projects. This situation, if not accounted
for, might result into multiple schemes exploiting the same funds, creating an unnec-
essary burden of data gathering, discouraging participation, and creating competing
interests, while it could also lead to biased duplication of results from the same area
(Groene et al., 2008); therefore, it is extremely useful to map quality indicators
according to project and aims, and to acknowledge issues regarding similar frame-
works in the same areas, especially before developing a novel evaluation agenda. In
the “Health Map” project, caution was applied so that indicators selected converged
to measurements previously proposed and charted internationally, with priority given
to projects funded by WHO, the OECD, and the EC (i.e., OECD’s HCQI, PATH,
EUPHORIC, SImPatIE); therefore, the expected results will generally be in accordance
with other foreign quality frameworks, while the cost of data selection will minimize.
In the set of quality indicators finally selected by the “Health Map”, it is important to
denote that all data gathered and analyzed centrally were later on redistributed to
healthcare providers, suggesting sectors that could progress, but, in addition, dissemi-
nated to consumers of healthcare services, that is, patients, who never before had the
quality-assessing or benchmarking information for public healthcare facilities around
Greece; public disclosure is a trait common among quality projects that are based on
non-voluntary participation (Groene et al., 2008). This feature is very important, as par-
ticipation in the evaluation “Health Map” framework is mandatory for all Greek public
healthcare facilities; therefore, information would universally cover public healthcare in
the Greek territory and, most probably, provide organizational motives in the direction
of investing on quality improvement (McGuckin et al., 2006).
In general, most indicators used in the identified quality frameworks abroad were
process indicators, probably easier and more feasible to measurement (Donabedian,
2005) as patients’ stay in contemporary hospital facilities has been reduced to a mini-
mum, due to modern medical care. Therefore, rendering outcome measures has been
found to be inadequate (Mant, 2001; Lilford et al., 2004) in quality assessment during
recovery stage (e.g., postoperatively, recovering from infectious disease) as patients
tend to leave the hospital environment relatively early (Groene et al., 2008), and
structural measurements have often been made redundant because of previously

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e201
recorded information in most countries, mostly as a result of accrediting standardization
(Copnell et al., 2009). However, taking into account the lack of a previous national
standardization framework in hospital care in Greece and the need for updating
information on capacity and healthcare volume support by public hospitals according
to region, led into a rather extensive utilization of structural indicators in the “Health
Map” quality indicator set, comprising the additional dimension of resources and
capacity, including infrastructure, facilities, personnel, and technology indicators; the
use of structural indicators was also prioritized in the “Health Map” because of relative
easiness of data collection at this early stage of a national quality-assessing attempt.
In our literature review, it was noted that the most focused-upon dimension of quality
in hospital care has been clinical effectiveness (Groene et al., 2008; Copnell et al.,
2009), being evaluated at some extent in all current frameworks; this might reflect
the modern trend of following and appreciating clinical guidelines and evidence-based
medicine and might also reflect the need for cost-effectiveness practices and strategies
(Mainz, 2003). The second most valued and emphasized dimension in hospital care was
patient safety, with several projects clearly focusing on this particular sector (SImPatIE,
HCQI, AHRQ, (Agency for Healthcare Research and Quality (AHRQ): Quality Indica-
tors project, 2011) Leapfrog, (Leapfrog Hospital Quality and Safety Survey, USA,
2011) AIHW (Australian Institute of Health and Welfare (AIHW), 2011a; Australian
Institute of Health and Welfare (AIHW), 2011b) patient safety is thought to be the
distinguishing value in healthcare quality between providers and systems and might
also reflect the current hierarchy in healthcare priorities (Kazandjian et al., 2005;
Copnell et al., 2009). These dimensions also received adequate attention in the “Health
Map” set of hospital indicators, as many indicators referred to results in following effec-
tive clinical care and at the same time ensuring safety, primarily for patients; as already
stated, priorities in healthcare, in Greece as much as abroad, include following
guidelined clinical decision-making and avoiding patient discomfort and possible legal
proceedings. However, at this point, outcome indicators were used more than process
standards, as administrative data recorded at the hospital level (and not patient or sector
specific), including hospital registries, were considered the primary source of raw data
for clinical effectiveness and safety issues.
Responsiveness, mainly patient-centeredness and also staff orientation and satisfac-
tion, are also valued in many quality frameworks of our literature review. The anthro-
pocentric ideals governing the inclusion of quality indicators assessing patient-
centeredness were much appreciated in the “Health Map” quality framework (Wylie
and Wagenfeld-Heintz, 2004; Clark et al., 2006); therefore, a number of indicators
assigned to this dimension were included in the final “Health Map” set. As mentioned
before, data regarding patient-centeredness indicators would not be collected through
the web-based form of “Health Map” questionnaires, but would rather require the
assistance of patient surveys; a number of Greek hospitals already use forms of pa-
tient-oriented questionnaires to handle patient satisfaction and complaints; however,
the results are usually left unused; therefore, utilization of such information might prove
to be valuable in the field of enhancing satisfaction and quality of life. Staff orientation
also qualified as important in the “Health Map” set, in the direction of efficacy of the
healthcare workforce, considering that satisfied health professionals are the most pro-
ductive ones (Cleary et al., 2010; Schulte and Vainio, 2010); hospital administrative

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e202 E. SIMOU ET AL.

data, such as records of leaves or overtime duty, standardized for the “Health Map”
questionnaires but, also, in-hospital health workers’ surveys would help in acquiring
data fit for use in such standards.
Efficiency indicators, similar to other quality projects, were also highlighted in the
“Health Map” framework’s public hospitals set, as it is of great importance and
policy-making significance that, in the current financially challenging period,
orthological use of existing resources is made and that accounting for expenditure
in a cost-effectiveness guided manner takes place. Standards assessing patient stay,
charges related to in-hospital procedures, and proper and adequate use of surgical
techniques were highly valued in the direction of assessing and improving profitabil-
ity, productivity, and thus, efficiency of services provided by public hospitals
(Clarke, 1996; Martinussen and Midttun, 2004; Huerta et al., 2008). Utilization
indicators, such as surgical theater use, admissions, and usage of equipment and
exams, were also considered important and ended up comprising a discrete dimen-
sion in “Health Map” quality standards; most of these measurements can also reflect
accessibility matters; however, because of lack of previous systematic information
on volume and allocation of services provided per facility and per district, this di-
mension received considerable attention in the “Health Map” set of quality standards
in the direction of acceptable delivery of hospital services and reasonable use of
resources, next to efficiency standards (Barrett et al., 2005; Kraus et al., 2005).
Although utilization standards were through of as more related to accountability
and justification matters, another theoretical component of accessibility, timeliness,
was included, also as a distinct dimension, receiving a number of indicators, all of
which encompassed in other quality projects’ sets. Timeliness standards emphasize
on the possible difficulties in managing the workload in hospitals (Bernstein et al.,
2009), as these facilities sometimes receive responsibility for health needs left
unfulfilled (e.g., patients leaving without being examined, patients admitted with
delay) because of considerable preference in public hospital care versus private
healthcare, with financial as well as qualitative criteria on the part of patients, and
because of perhaps a less proactive role of primary public healthcare services in
Greece (Economou, 2010).
On the other hand, matters of responsiveness in hospital governance were not
considered as most important or as easy to measure at the current starting phase of
the “Health Map” framework; therefore, no indicators specifically dedicated to this
dimension were included. Also, linking hospital care to the community and to public
health is much valued; however, because of the need to be parsimonious in our
“Health Map” quality measures, such matters were given priority in assessment of
public primary healthcare, a different parallel project of the “Health Map”, rather than
within the hospital environment. Lastly, equity is perhaps the most important factor of
ensuring a solid and functional healthcare system, to improve the health status of the
general population and reduce health disparities; indicators relating to this dimension
might not notably be developed in the existing frameworks, but equity may be the
driving force beneath any comprehensive quality-assessing system (Hebb et al.,
2003; Coffey et al., 2005). In the “Health Map” quality framework, similar to the
global trend, indicators specifically dedicated to equity were not included; however,
many of the multi-level indicators used to estimate the overall quality and accreditation

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e203
in performance of public hospitals (e.g., space in rooms, timeliness in meeting acute
healthcare needs, understanding information) address the subject of equity (Copnell
et al., 2009).
It is perhaps useful to note some limitations of the current study regarding the
selected “Health Map” set of quality indicators for public hospitals. First, indica-
tors selected generally pertained to a hospital-wide range rather than being sector
or disease specific, as this was a starting effort to assess hospital quality in Greece.
Therefore, some more clinically focused indicators, on specific subjects (e.g.,
cardiac and stroke care, treatment options, and mobility and rehabilitation), were
not included. These subjects are traditionally addressed in quality indicator pro-
jects (e.g., EUPHORIC, JCAHO, IQIP, and PATH), and although they might be
relevant to a distinct specialty (e.g., cardiology and infection control), they also
reflect severe conditions that inflict massive workload, costs, and focus on part
of the healthcare system (Copnell et al., 2009; Evans et al., 2009) and should there-
fore be included in a generalized hospital assessment quality set; this was not the
case in “Health Map” as main hospital data sources could not cover collection of
such detailed information. The addition of omitted, but important, measurements
in future versions of the “Health Map” indicators set was highly recommended
by the experts of the consensus panel and will be given serious attention in upcom-
ing selections.
Furthermore, the measurements that were selected referred to acute illness care
only, while aspects of outpatient healthcare provision, such as health promotion,
chronic disease management and palliative care, were not addressed in the current
set of hospital indicators; such subjects are more systematically considered and ful-
filled in the primary care setting (Vanselow et al., 1995; Starfield, 1998); therefore,
standards regarding these subjects were left to primary healthcare quality assess-
ment, consisting a parallel project of “Health Map” as already mentioned. Also, in a
traditional manner and to converge with global quality frameworks (Copnell et al.,
2009), quality and effectiveness of care in the national hospital indicator set majorly
involved physicians’ performance, although all health-related disciplines were
accounted for in recording the current GNHS resources and capacity; perhaps in the
future, additional quality measures may need to be indorsed to cover the need of a
universal appraisal in effectiveness and improvement of healthcare provision. Finally,
it should also be noted that, at present, the GNHS is under parallel reformation of the
coding system related to diagnosis-related groups and hospitalization-related costs,
endeavoring to adopt international classification and to update information tech-
nology utilized (Polyzos et al., 2013); therefore, success of the concurrent projects
of reorganization and of evaluation in hospital services cannot be guaranteed yet,
because realization and usefulness of performance measures remains to be vali-
dated in the future.

CONCLUSION

In the current study, we investigated contemporary quality indicator frameworks al-

ready implemented abroad or internationally, as the first step to advance to a national

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e204 E. SIMOU ET AL.

quality framework for public GNHS hospitals. We also attempted to isolate specific de-
ficiencies originating from the existing systems of data recording and updating regard-
ing performance of hospitals in Greece, and specific needs rising from contemporary
challenges in the Greek society. With the use of evidence from the international litera-
ture and electronic sources, along with information on shortages in sectors of quality in
healthcare in the Greek territory, the “Health Map” quality indicator set was developed,
following, but not restricted to, standards of global quality projects, while all dimen-
sions of quality in hospital care were discussed at its developing point, each dimension
receiving an essential number of dedicated quality indicators. Data gathered and ana-
lyzed in this manner have been expected to provide useful and novel information to de-
cision-makers, healthcare professionals, and patients in Greece and to assist in a future
overall estimation and improvement of quality in care in the GNHS hospital sector.

ACKNOWLEDGEMENT

This work was part of the “Health Monitoring Indicators System: Health Map” project
funded by the European Social Fund in the framework of the Axis 5.1, 5.2, 5.3 of the
European Project “Development of Human Resources” (2007–2013). The contents of
this paper are solely the responsibility of the authors and do not necessarily represent
the official views of the sponsor.

REFERENCES

Agency for Healthcare Research and Quality (AHRQ): Qual- Berg M, Meijerink Y, Gras M, et al. 2005. Feasibility first
ity Indicators project, USA: [Link] developing public performance indicators on patient
[Link]/ [Accessed on 15 December 2011]. safety and clinical effectiveness for Dutch hospitals.
Arah OA, Custers T, Klazinga NS. 2003. Updating the Health Policy 75: 59–73.
key dimensions of hospital performance: the move Bernstein SL, Aronsky D, Duseja R, et al. 2009. The
towards a theoretical framework. Third WHO Workshop effect of emergency department crowding on
on Hospital Performance Measurement, Barcelona, clinically oriented outcomes. Acad Emerg Med
13–14 June 2003. 16: 1–10.
Arah OA, Westert GP, Hurst J, Klazinga NS. 2006. Con- Bundesgeschaftsstelle Qualitatssicherung. 2007. Qualitat
ceptual framework for the OECD Health Care Quality sichtbar machen: BQS—Qualitatsreport 2006. BQS:
Indicators project. Int J Qual Health Care 18: 5–13. Dusseldorf. [in German].
Australian Institute of Health and Welfare (AIHW). Senti- Campbell SM, Roland MO, Buetow SA. 2000. Defining
nel events in Hospitals project, Australia. [Link] quality of care. Soc Sci Med 51: 1611–25.
[Link]/publications/[Link]/title/10353 Campbell SM, Braspenning J, Hutchinson A, Marshall M.
[Accessed on 15 December 2011a]. 2002. Research methods used in developing and
Australian Institute of Health and Welfare (AIHW). applying quality indicators in primary care. Qual Saf
National Health Performance Committee, Australia. Health Care 11: 358–64.
[Link] Canadian Institute for Health Information. Canadian
[Accessed on 15 December 2011b]. Hospital Reporting Project, Canada. [Link]
Barrett B, Way C, McDonald J, Parfrey P. 2005. ca/ [Accessed on 15 December 2011].
Hospital utilization, efficiency and access to care Centres for Medicaid and Medicare Services (CMMS)/
during and shortly after restructuring acute care in Hospital Quality Alliance (HQA): Hospital Compare,
Newfoundland and Labrador. J Health Serv Res Policy USA. [Link]
10(Suppl.2): 31–7. [Accessed on 15 December 2011].

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
 A FRAMEWORK OF QUALITY INDICATORS FOR HOSPITALS e205
Clark PA, Wolosin RJ, Gavran G. 2006. Customer Huerta TR, Ford EW, Peterson LT, Brigham KH. 2008.
convergence: patients, physicians, and employees Testing the hospital value proposition: an empirical
share in the experience and evaluation of healthcare analysis of efficiency and quality. Health Care
quality. Health Mark Q 23: 79–99. Manage Rev 33: 341–9.
Clarke A. 1996. Why are we trying to reduce length of Joint Commission on Accreditation of Health Care Orga-
stay? Evaluation of the costs and benefits of reducing nizations (JCAHO). Hospital Accreditation Standards.
time in hospital must start from the objectives that Research and Performance Measurement. The Joint
govern change. Qual Health Care 5: 172–9. Commission: USA. [Link]
Cleary M, Hunt GE, Horsfall J. 2010. Identifying and PerformanceMeasurement/ [Accessed on 15 December
addressing bullying in nursing. Issues Ment Health 2011].
Nurs 31: 331–5. Kazandjian VA, Matthes N, Wicker KG. 2003. Are
Clinical Indicator Support Team. NHS Quality Improve- performance indicators generic? The international
ment Scotland, Information Services Division (ISD): experience of the Quality Indicator Project. J Eral Clin
Scotland. [Link] Pract 9: 265–76.
[Accessed on 15 December 2011]. Kazandjian VA, Wicker K, Ogunbo S, Silverman N.
Coffey RM, Andrews RM, Moy E. 2005. Racial, 2005. Understanding safer practices in health care: a
ethnic, and socioeconomic disparities in estimates prologue for the role of indicators. J Eval Clin Pract
of AHRQ patient safety indicators. Med Care 43: 11: 161–70.
I48–57. Kraus TW, Büchler MW, Herfarth C. 2005. Relationships
Collopsy BT, William J, Rodgers L. 2000. The ACHS between volume, efficiency, and quality in surgery—a
Care Evaluation Program: a decade of achievement. J delicate balance from managerial perspectives. World
Qual Clin Practice 20: 36–41. J Surg 29: 1234–40.
Copnell B, Hagger V, Wilson SG, Evans SM, Sprivulis Leapfrog Hospital Quality and Safety Survey, USA.
PC, Cameron PA. 2009. Measuring the quality of [Link] [Accessed on 15
hospital care: an inventory of indicators. Intern Med December 2011].
J 39: 352–60. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R.
Donabedian A. 2005. Evaluating the quality of medical 2004. Use and misuse of process and outcome data in
care. Milbank Q 83: 691–729. managing performance of acute medical care: avoiding
Economou C. 2010. Greece: health system review. Health institutional stigma. Lancet 363: 1147–54.
Syst Transit 12: 101–108. Luthi JC, McClellan WM, Flanders WD, Pitts S, Burnand
Evans S, Lowinger J, Sprivulis P, Hagger V, Copnell B, B. 2002. Quality of health care surveillance systems:
Cameron P. 2009. Prioritising quality indicator review and implementation in the Swiss setting. Swiss
development across the healthcare system: identifying Med WKLY 132: 461–9.
what to measure. Intern Med J 39: 648–54. Mainz J. 2003. Defining and classifying clinical indicators
Freeman T. 2002. Using performance indicators to for quality improvement. Int J Qual Health Care 15:
improve health care quality in the public sector: a 523–30.
review of the literature. Health Serv Manage Res Mant J. 2001. Process versus outcome indicators in the
15: 126–37. assessment of quality of health care. Int J Qual Health
Grenier-Sennelier C, Vorriol C, Daucourt V, Michel P, Care 13: 475–80.
Minvielle E. 2005. Developing quality indicators in Marshall M, Klazinga N, Leatherman S, Hardy C,
hospitals: the COMPAQH project. Rev Epidemiol Bergmann E, Pisco L, Mattke S, Mainz J. 2006. OECD
Santle Publiq 53: IS22–IS33. [in French]. Health Care Quality Indicator Project. The expert
Groene O, Skau JK, Frolich A. 2008. An international panel on primary care prevention and health promo-
review of projects on hospital performance assessment. tion. Int J Qual Health Care 18 Suppl 1: 21–5.
Int J Qual Health Care 20: 162–71. Martinussen PE, Midttun L. 2004. Day surgery and
Ham C, Dixon A, Brooke B. 2012. Transforming the hospital efficiency: empirical analysis of Norwegian
Delivery of Health and Social Care: the Case for hospitals, 1999–2001. Health Policy 68: 183–96.
Fundamental Change. The King’s Fund: London. McGuckin M, Waterman R, Shubin A. 2006. Consumer
Hebb JH, Fitzgerald D, Fan W. 2003. Health care attitudes about health care-acquired infections and
disparities in disadvantaged Medicare beneficiaries: a hand hygiene. Am J Med Qual 21: 342–46.
national project review. J Health Hum Serv Adm 26: National Quality Forum, USA. [Link]
153–73. org/ [Accessed on 15 December 2011].
Hospital Report Research Collaborative, Canada. http:// Polyzos N, Karanikas H, Thireos E, Kastanioti C,
[Link]/ [Accessed on 15 December Kontodimopoulos N. 2013. Reforming reimbursement
2011]. of public hospitals in Greece during the economic

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm
e206 E. SIMOU ET AL.

crisis: implementation of a DRG system. Health Policy Torre M, Morciano C, D’Errigo P, et al. 2007. The
109: 14–22. Euphoric project: outcome indicators collection in
Safety Improvement for Patients in Europe (SImPatIE), Europe. Results of the first phase. Eur J Public Health
Europe. [Link] [Accessed on 17(Suppl.2): 213.
15 December 2011]. Vanselow NA, Donaldson MS, Yordy KD. 1995. From
Schulte P, Vainio H. 2010. Well-being at work— the Institute of Medicine. JAMA 273: 192.
overview and perspective. Scand J Work Environ Veillard J, Champagne F, Klazinga N, Kazandjian V,
Health 36: 422–9. Arah OA, Guisset AL. 2005. A performance assess-
Starfield B. 1998. Primary Care: Balancing Health ment framework for hospitals: the WHO regional
Needs, Services, and Technology. Oxford University office for Europe PATH project. Int J Qual Health
Press: New York. Care 17: 487–96.
The Danish National Indicator Project, Denmark. http:// Wylie JL, Wagenfeld-Heintz E. 2004. Development of
[Link]/ [Accessed on 15 December 2011]. relationship-centered care. J Healthc Qual 26: 14–21.

Copyright © 2014 John Wiley & Sons, Ltd. Int J Health Plann Mgmt 2014; 29: e187–e206.
DOI: 10.1002/hpm