Accepted Manuscript

Changes in technical efficiency after quality management

certification: A DEA approach using difference-in-difference
estimation with genetic matching in the hospital industry

Ivonne Lindlbauer , Jonas Schreyogg , Vera Winter

PII: S0377-2217(15)00955-8
DOI: 10.1016/j.ejor.2015.10.029
Reference: EOR 13313

To appear in: European Journal of Operational Research

Received date: 16 July 2015

Revised date: 12 October 2015
Accepted date: 15 October 2015

Please cite this article as: Ivonne Lindlbauer , Jonas Schreyogg , Vera Winter , Changes in techni-
cal efficiency after quality management certification: A DEA approach using difference-in-difference
estimation with genetic matching in the hospital industry, European Journal of Operational Research
(2015), doi: 10.1016/j.ejor.2015.10.029

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service
to our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and
all legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRPT

Highlights

We analyze changes in hospital efficiency after quality management

certification.

The study is based on German acute care hospitals over a ten-year period.

We employ a novel approach to control for self-selection bias or

f certification.

The international ISO 9001 certification has a negative mpact

i on efficiency.

The hospital-specific KTQ certification has a positive impact on efficiency.

1
 ACCEPTED MANUSCRPT

Changes in technical efficiency after quality management certification:

A DEA approach using difference-in-difference estimation with genetic
matching in the hospital industry

Keywords: OR in health services; Quality management; Data envelopment analysis;

Certification; ISO 9001

Authors:

Ivonne Lindlbauer
Hamburg Center for Health Economics
University Medical-Centre Hamburg-Eppendorf
Martinistr. 52, D-20246 Hamburg
Phone: +49 40 74 10-54 172
Fax: +49 40 74 10
-40 261
E-mail:i.lindlbauer@uke.de

Jonas Schreygg*
Hamburg Center for Health Economics
University of Hamburg
Esplanade 36, 20354 H
amburg, Germany
Phone: +49 40 42838-8041
Fax: +49 40 42838-8043
E-mail jonas.schreyoegg@wiso.uni-hamburg.de

Vera Winter
Hamburg Center for Health Economics
University of Hamburg
Esplanade 36, 20354 H
amburg, Germany
E-mail: vera.winter@uni-hamburg.de

2
 ACCEPTED MANUSCRPT

This paper is our original, unpublished work, and there are no submissions or previous reports
that might be regarded as a redundant or duplicate publication of the same or very similar
work. The paper has not been submitted previously to any journal.

Conflicts of Interests: None

Acknowledgments

This study was supported by a research grant from the German Federal Ministry of Education
and Research (BMBF) in Germany (Grant number: 01FL10055). The sponsor had no role in
the study design, collection and analysis of data, the writing of the report or the submission of
the paper for publication.

We thank three anonymous reviewers for their valuable comments and suggestions.

*
Corresponding author

3
 ACCEPTED MANUSCRPT

Changes in technical efficiency after quality management certification:

A DEA approach using difference-in-difference estimation with genetic
matching in the hospital industry

Abstract

Hospitals in Germany have been required to have an internal quality management system
(QMS) since 2000. Although formal certification of such systems is voluntary, the number of
certifications has increased steadily. The most common standards in Germany are ISO 9001,
which is also widely used internationally, and KTQ (Kooperation fr Transparenz und
Qualitt im Gesundheitswesen), which was developed specifically for the German health care
sector. While a large body of literature has investigated the impact of QMS certification on
performance in many industries, there is only scarce evidence on the causal link between
QMS certification and technical efficiency. In the present study, we seek to elucidate this
relationship using administrative data from all German hospitals from 2000 through 2010
combined with information on certification. Our analysis has three steps: First, we calculated
efficiency scores for each hospital using a bootstrapped data envelopment analysis. Second,
we used genetic matching to ensure that any differences observed could be attributed to
certification and were not due to differences in sample characteristics between the
intervention and control groups. Third, we employed a difference-in-difference specification
within a truncated regression to examine whether certification had an impact on hospital
efficiency. To shed light on a potential time lag between certification and efficiency gains, we
used various periods for comparison. Our results indicate that hospital efficiency was
negatively related to ISO 9001 certification and positively related to KTQ certification.
Moreover, coefficients were always larger in the period between first certification and
recertification.

Keywords: OR in health services; Quality management; Data envelopment analysis;

Certification; ISO 9001

4
 ACCEPTED MANUSCRPT

1 Introduction

A quality management system (QMS) consists of a set of fixed business procedures and rules,
which shall ensure that a product, process or service meets a set of standards (Tzelepis,
Tsekouras, Skuras, & Dimara, 2006). Since quality is difficult to measure and to observe, in
particular from outside the organization, quality standards ought to be certified. For that
purpose, national and international, comprehensive and industry-specific QMS standards have
been established. The most prominent QMS certification is the ISO 9000 series, which has
been established in 1987. At the end of 2006, over 870,000 certifications had been granted in
140 countries worldwide (Heras-Saizarbitoria, Casadess, & Marimn, 2011), demonstrating
its persistence.

While research generally agrees on the reasons for which a firm strives for certification
(Prajogo, 2011), understanding the effects of these standards on performance has always been
a topic of great controversy (Benner & Veloso, 2008; Dick, Heras, & Casadess, 2008;
Terziovski, Samson, & Dow, 1997; Terziovski, Power, & Sohal, 2003). Several researchers
have invested in the consolidation of prior findings (Chatzoglou, Chatzoudes, & Kipraios,
2015; Corbett, Montes-Sancho, & Kirsch, 2005; Dick et al., 2008; Sampaio, Saraiva, &
Guimares Rodrigues, 2009; Sampaio, Saraiva, & Monteiro, 2012), yet without conclusive
insights. Several of these reviews categorize prior studies by the nature of its data. Most
studies on the certification-performance link are based on cross-sectional data. From a
methodological point of view, a longitudinal study design is necessary to draw causal
conclusions. Therefore, researchers advocate a before-and-after comparison of intervention
and matched control groups (Corbett et al., 2005; Levine & Toffel, 2010; Lo, Pagell, Fan,
Wiengarten, & Yeung, 2014; Sharma, 2005; Yeung, Lo, & Cheng, 2011). This
methodological approach helps to disentangle the changes in performance as a result of
certification from changes in performance over time, but has been scarcely applied to analyze
the effects of ecrtification.

The inconsistency of prior findings might also be partially explained by the quite distant
relationship between certification and financial performance, the latter being most frequently
used as dependent variable. Providing empirical evidence that the choice of performance
measure significantly affects the results, Naveh and Marcus (2005) showed that implementing
the ISO 9000 standard led to improved operating performance, but did not directly lead to
improved business performance. Lima, Resende, and Hasenclever (2000) questioned the use
5
 ACCEPTED MANUSCRPT

of accounting indicators and called for studies analyzing the productive efficiency of certified
and non-certified firms. Levine and Toffel (2010) pointed out that they found no prior
research that examined how the ISO 9001 quality standard affects product or process quality
or employees. Hence, analyzing the link between QMS certification and efficiency remains a
research gap.

Additionally, in comparison to manufacturing industries, research is scarce in the service

industry, and very few studies have been undertaken in the hospital industry. The paucity of
studies in the hospital industry stands in contrast to its high relevance for a nations economy,
and the increasing number of certifications issued within this sector. In particular, hospitals
account for the greatest share of health expenditure in most EU Member States (European
Commission eurostat, 2015), and strategies to increase efficiency and reduce costs in the
hospital sector have become central to efforts at health system reform in many countries. At
the same time, stakeholders continue to expect high-quality treatment and increased
transparency. To this end, policymakers in a number of European countries have encouraged
or even required the use of QMS in hospitals, leading to the adaptation of standards from
other industries, such as the ISO 9000 series, or to the development of national QMS systems
specifically for the health care sector (Shaw, Bruneau, Kutryba, Jongh, & Sunol, 2010).
Hospitals in Germany have been required to have an internal QMS since 2000, but are free to
choose which internal system they wish to implement. Formal certification of the QMS
through costly third-party audits is not legally required. Although the certification of such
systems is voluntary, the number of certifications has increased each year. The most common
types of certification in the German hospital sector are ISO 9001 and KTQ (Kooperation fr
Transparenz und Qualitt im Gesundheitswesen; Cooperation for Transparency and Quality
in the Health System), which was developed for theGerman health care sector.

The purpose of the present study is to examine the relationship between QMS certification
and technical efficiency, estimated via data envelopment analysis (DEA). It thereby
differentiates between an international, comprehensive QMS standard (ISO 9001:2000) and a
national, industry-specific standard (KTQ). It uses the hospital industry as a research setting
and draws on panel data from the years 2000 through 2010, allowing certified organizations
to be contrasted with non-certified organizations using a difference-in-difference approach.
Data analysis comprises a sample of 374 certified and 374 matched non-certified hospitals in

6
 ACCEPTED MANUSCRPT

Germany applying the genetic matching method. The results are subsequently discussed
alongside implications for practice and research.

Section 2 of this study provides an overview on ISO 9001 and KTQ certification and reviews
the relevant literature on the impact of certification on performance. Section 3 presents the
data and the estimation strategy we used to explore this relationship. Section 4 presents and
discusses the results of our analysis, and Section 5 gives a summary and suggests topics for
future research.

2 Background and literature review

2.1 ISO and KTQ certification

As a process-oriented, international standard originally developed for the manufacturing

industry, ISO 9001 belongs to the ISO 9000 series of standards and is used to assess QMS in
whole hospitals or, more frequently, in parts of them, such as hospital departments (Shaw,
Kutryba, Braithwaite, Bedlicki, & Warunek, 2010). In Germany, the number of whole
hospitals certified according to ISO 9001 increased from 26 in 2000 to 211 in 2010. ISO 9001
provides QMS standards based on eight principles: customer focus, leadership, involvement
of people, process approach, system approach to management, continual improvement, factual
approach to decision making, and mutually beneficial supplier relationships. To obtain ISO
9001 certification, a QMS must not only be established, implemented, maintained, and
continually improved. It must also be documented, including documented statements of a
quality policy and quality objectives; a quality manual; documented procedures required by
the ISO 9001 standard; documents needed by the hospital to ensure the effective planning,
operation, and control of its processes; and records required by the ISO 9001 standard
(International Organization for Standardization, 2008). Internal audits are also required at
regular intervals to check for evidence of conformity and the effectiveness of the QMS.
Implementation of the standard is reviewed by official external auditors, and a certificate can
be awarded if there is no major nonconformity. The certification is valid for a maximum of
three years, after which time recertification is necessary. As the standard is continually being
revised by standing technical committees and advisory groups, there exist several versions:
ISO 9000:1987, ISO 9000:1994, ISO 9001:2000, ISO 9001:2008, and ISO 9001:2015
forthcoming. Between the ISO 9000:1994 version and the ISO 9001:2000, significant changes
with respect to its focus on process optimization have occurred. Additionally, the 2000
version required a higher involvement of upper executives, the measurement of the
7
 ACCEPTED MANUSCRPT

effectiveness of tasks and activities, expectations of continual process improvement and a

tracking of customer satisfaction (Tamayo-Torres, Gutierrez-Gutierrez, & Ruiz-Moreno,
2014; Tzelepis et al., 2006). Finally, it also aims at better meeting service sector
organizations needs (Terziovski et la., 2003).

Available only at the whole hospital level, KTQ certification is based on a range of standards
and assessment procedures that are carried out by franchised organizations (Shaw et al.,
2010). The number of KTQ-certified hospitals has increased from nine in 2002, when it was
introduced, to 462 in 2010, making this the most common form of standards-based
assessment in Germany. KTQ certification is based on a catalogue containing the categories
patient-centeredness, employee-centeredness, safety in the organization, information and
communication, leadership, and quality management (Kooperation fr Transparenz und
Qualitt im Gesundheitswesen, 2015). To obtain KTQ certification, a hospital must produce a
comprehensive report assessing the quality of its patient-care processes according to the
criteria in the catalogue. Although required, a QMS is therefore only part of the KTQ
standard. Subsequently, a team of external auditors evaluates this self-assessment and can
award certification if at least 55% of the points in each category are achieved (Kooperation
fr Transparenz und Qualitt im Gesundheitswesen, 2015). The auditors evaluate the QMS, as
well, but by spot checks. Like ISO 9001 certification, KTQ certification is valid for three
years, after which time recertification isnecessary.

Both ISO 9001 and KTQ aim at implementing QMS standards and comprise several
dimensions. They differ with respect to the emphasis, which is predominantly targeted at
process standardization for ISO 9001, while KTQ has a somewhat broader approach and also
includes structure- und outcome-related aspects. Furthermore, ISO 9001s requirements must
be met without exception, are partially difficult to be translated to the hospital context, and a
high documentary workload is associated with certification. KTQ seems a little bit more lean,
directly adapted to the hospital context, and leaves some discretionary space by not
demanding tofulfill all points.

2.2 Prior literature

While the majority of certification research has relied on cross-sectional data, calls for a
longitudinal study design have been increasing (Dick et al., 2008; Sampaio et al., 2012).
Among the studies with a longitudinal design, most studies analyze the impact of QMS
8
 ACCEPTED MANUSCRPT

certification on financial performance, based on indicators such as return on assets (ROA),

return on sales (ROS), sales growth, earnings per share, and stock prices. Some of the most
rigorous studies are present below.

Using an event study method with 27 certified Spanish stock firms and the complete market
as control group for the 19931999 period, Nicolau and Sellers (2002) only find a positive
stock price effect on the day of certification itself. Applying an event study method as well,
Corbett et al. (2005) compare changes in ROA, ROS, and the cost of goods sold between 554
certified and 554 matched non-certified manufacturing firms for the 19871997 period. They
find that ISO adoption is followed by significant modest abnormal improvements in financial
performance, though the effect depends on the specification of the control group. Benner and
Veloso (2008) use a sample of 75 automotive supplier companies in the U.S. with data for the
years 1988 to 1997 and apply a longitudinal panel data design with firm fixed effects and year
controls. They observe that performance benefits of ISO adoption are moderated by timing
and firm-specific technological coherence. Dick et al. (2008) compare sales revenues and
ROA of 400 ISO 9001 certified companies and 400 non-certified companies matched by
industrial sector in the 19941998 period applying an attribution testing method. They detect
that there is some evidence to indicate that QMS certification has some causal influence on
business performance, while there is also evide
nce for the existence of everse
r causality, i.e., a
self-selection of better performing firms to certification. Yeung et al. (2011) use a sample of
185 certified and 185 matched non-certified U.S. manufacturing firms within the 19942006
period and analyze differences in the changes in CEO compensation and ROA with the event
study method approach. They found a significantly positive net effect of certification on CEO
compensation, and a significantly negative net effect of ecrtification on ROA.

The longitudinal studies which have analyzed productivity changes after certification are
heterogeneous with regard to the productivity measures used, the data analysis methods, and
the findings. Naveh and Marcus (2005) analyzed, among others, the impact if certification on
perceived operating performance improvement (incl. defect rates, cost of quality,
productivity, on-time delivery, and customer satisfaction) using a sample of 313 US firms and
309 matched control firms between 1990 and 2000. Using hierarchical linear models (HLM)
and validating results via the event study method, they partially observe significantly positive
effects of certification on operating performance. Martnez-Costa, Martnez-Lorente, and
Choi (2008) measured productivity as income per employee and conducted t-tests for

9
 ACCEPTED MANUSCRPT

differences of ISO 9000 and non-ISO 9000 companies in a new variable created as the
difference between the 2-year average productivity after and before ISO 9000
implementation. They found that certification had no significant impact on productivity.
Levine and Toffel (2010) analyzed the effect of ISO 9000 certification on inputs (i.e.,
employment) and outputs (i.e., sales) separately. In the sales model, they estimated
regressions with and without employment as an independent variable. For data analysis, they
used a difference-in-difference approach in which they compared changes in the dependent
variables for certified hospitals in comparison to changes in the dependent variables for a
matched group of non-certified hospitals. Matching was performed based on propensity
scores, which allows taking into account a variety of observable factors. They found that the
significantly positive effect of certification on sales became insignificant when employment
was introduced as explanatory variable, indicating that certification is not related to an
increase in labor productivity. Lo et al. (2014) analyzed the effect of OHSAS 18000
certification, which is an international occupational health and safety management system
specification, on the ratio of operating income to number of employees as indicator of labor
productivity. Following the event study method, they applied paired-sample t-tests (Wilcoxon
sign-rank and significance tests). They found a positive effect of OHSAS 18000 certification
on labor productivity. Using a more holistic productivity measurement, Tzelepis et al. (2006)
analyzed the relationship between technical efficiency and ISO 9000 certification using
stochastic frontier analysis (SFA) and found that adopting ISO 9001 reduced managerial
inefficiency. However, they collapsed the panel structure of their data set to a cross-sectional
structure, limiting possible causal conclusions about the relationship. Hence, results on the
impact of certification on productivity are inconclusive, and research analyzing the causal link
between certification and efficiency with a lon
gitudinal design is csarce.

Further, to our knowledge, there exists only one quantitative study that tackles the research
gap related to the hospital industry. Makai, Klazinga, Wagner, Boncz, and Gulacsi (2009)
assess the impact of QMS certification (ISO 9000 and Hungarian Hospital Care Standards,
HHCS, certification) on patient safety with data collected from 102 Hungarian hospitals in
2005. In a cross-sectional analysis using linear regressions, they find no significant
relationship between ISO or HHCS certification and the number of patient safety activities.

Although these studies have generated valuable initial insights into the effects of certification,
the generalizability of their findings remains limited. The majority of studies relies on data of

10
 ACCEPTED MANUSCRPT

the 90s and assesses the benefits of ISO 9000:1994 certification. Empirical evidence suggests
that the ISO 9000:1994 and the ISO 9001:2000 version differ in their impact on internal
processes (e.g., Total quality management (TQM) implementation, Martnez-Costa, Choi,
Martnez, & Martnez-Lorente, 2009) and on external reactions (e.g., stock prices, McGuire &
Dilts, 2008). While some studies compared changes in performance of certified firms to
changes of performance of none-certified firms, only few studies ensured that baseline
characteristics were balanced across the corporatized and non-corporatized firms. Those
studies which applied matching strategies only matched on individual characteristics, such as
industry, firm size, and pre-certification ROA. The only exception to that is Levine and Toffel
(2010), who used propensity score matching. Only one study (Tzelepis et al. 2006) estimated
efficiency as a combination of multiple inputs and outputs using SFA, and there is no
longitudinal study using DEA for efficiency estimation. In general, DEA is the most
frequently used approach to measuring efficiency in health care (Hollingsworth, 2008).
Within the hospital industry, there is no study assessing the impact of ISO certification on
performance with a longitudinal study design and quantitative data. Further research in this
field is therefore needed to confirm these preliminary findings.

Our study has a number of strengths compared to previous investigations of certification.

First, it is, to our knowledge, the only study so far to examine the effects of certification on
hospitals technical efficiency, and to assess the impact of two different types of QMS
certification (ISO and KTQ) with a longitudinal study design. Second, we used the novel
genetic matching method, which is superior to existing methods, to generate a control group
for the certified hospitals. Third, we apply the difference-in-difference method, which allows
estimating the net effect of certification while controlling for a variety of time-variant and
time-invariant factors. This method has only been applied by Levine and Toffel (2010) yet,
but not in combination with genetic matching. Our sample is extracted of all German
hospitals, comprising 273 hospitals with KTQ certification, 101 hospitals with ISO
certification, and 456 non-certified hospitals for the 2000-2010 period.

11
 ACCEPTED MANUSCRPT

3 Methodology

3.1 Sample

The data used in this study were drawn from two sources. The first of these consisted of the
hospital reports submitted annually to the Research Data Centre of the Federal Statistical
Office of Germany (Federal Statistical Office of Germany, [Destatis], 2013). This extensive
dataset covers all hospitals (n=2,390) in Germany from the years 2000 through 2010 and
contains hospital-level information on cost and hospital infrastructure, as well as patient-level
information on age, gender, main diagnosis and length of stay. The second source was the
structured quality reports that hospitals in Germany have had to publish online every two
years since 2006. We manually extracted the following details on QMS certification from the
structured quality reports from 2010: whether a hospital was certified; type of certification;
certification coverage (partly versus fully certified hospitals); year in which certification was
first obtained; and duration of certification. In cases where information was not available from
the structured quality report, we obtained it directly from hospital web sites. The most
commonly used types of certification for whole hospitals (i.e., as opposed to hospital
departments) were ISO 9001 (abbreviated henceforth as ISO) and the KTQ standard. QMS
information was available only for existing hospitals and could not be collected for hospitals
that had closed. The two datasets were merged by the Research Data Centre of the Federal
Statistical Office of Germany and were provided for statistical analysis through remote
access. To ensure the comparability of the hospitals included in the analysis, the following
hospitals were excluded: hospitals with ISO certification only at the level of individual
departments; hospitals with other QMS certifications; ISO or KTQ certified hospitals without
at least two pre-certification and one post-certification period; hospitals providing mainly
psychiatric care; university and military hospitals; day and night clinics; and hospitals with
fewer than 50 beds or more than 2000 beds. Furthermore, we conducted systematic
plausibility checks to identify hospitals with input data error. The unbalanced sample
ultimately used in the analysis contained 830 acute care hospitals, including 273 hospitals that
had obtained their first KTQ certification, 101 hospitals that had obtained their first ISO
certification, and 456 hospitals that had no ertification
c between 2002 and 2009.

12
 ACCEPTED MANUSCRPT

3.2 Data analysis

The methodological approach in this study has been similarly applied in other settings before
(Lindlbauer, Winter, & Schreygg, 2015; Tiemann & Schreygg, 2012). It consists of three
steps: (1) calculating efficiency scores for all hospitals in the sample, (2) finding an
appropriate control group consisting of non-certified hospitals, and (3) comparing changes in
efficiency between certified and non-certified hospitals using a difference-in-difference
approach. Table 1 provides a comprehensive outline of the statistical analysis and each of
these steps isexplicitly described in order below.

------------------------------------------
Insert Table 1 about here
------------------------------------------

3.3 Data envelopment analysis

DEA is a nonparametric frontier method that uses linear programming to evaluate the relative
technical efficiency of a decision-making unit, such as an individual hospital, based on
observational data. When selecting inputs and outputs, we followed the example of other
studies that have developed DEA frameworks for measuring hospital efficiency (Dyson et al.,
2001; Tiemann & Schreygg, 2012). For our purposes, we chose seven inputs and one output,
which are described in he
t following.

The relevant inputs comprise material and labor. The first input variable, (Supplies),
represents the amount spent on supplies each year, including operational expenses but
excluding payroll, capital, and depreciation expenses. It is used as a proxy for material
resources. To account for labor input, we used five variables separated according to labor type
and counted as full-time equivalents: number of physicians (Physician), nurses (Nurse), other
clinical staff (Clinical), administrative staff (Admin), and other nonclinical staff members
(Nonclinical). The last input is the number of beds (Beds), which can be seen, at least over the
medium term, as an exogenous factor that is not under the direct control of hospital
management but rather to planning authorities in Germany (Steinmann, Dittrich, Karmann, &
Zweifel, 2004). We therefore included it as a non-discretionary input to he
t model.

13
 ACCEPTED MANUSCRPT

As the output of hospital services, we used the number of treated inpatient cases weighted by
mean predicted resource intensity (Weighted cases). Weighting ensures the comparability of
hospitals with different patient heterogeneity (i.e., case-mix), as proposed by Lindlbauer and
Schreygg (2014). Using output variables without weights can be problematic in efficiency
analysis because hospitals with a more complex case-mix are likely to obtain lower efficiency
scores (Tiemann & Schreygg, 2012). ). The prediction of resource intensity was based on
length of stay modeled as a function of the main diagnoses, a dummy for surgery, patient age,
gender, and year dummies of treatment, and followed the approach described in Lindlbauer
and Schreygg (2014). Weights greater than one indicate an above-average predicted resource
intensity, and vice versa. The sum of the input cases multiplied with their respective weights
generated the output ofach
e hospital.
Table 2 reports descriptive statistics for the inputs and output used for DEA stratified
according to hospitals without certification and hospitals with KTQ or ISO certification for
the years 2000, 2005and 2010 ofthe unbalanced panel.
------------------------------------------
Insert Table 2 about here
------------------------------------------

DEA identifies observations with the highest input-to-output ratios, which then determine the
efficiency frontier, whereas other observations lying below the frontier are relatively less
efficient. In the early DEA paper by Charnes, Cooper and Rhodes (CCR) (1978), efficiency
was measured assuming constant returns to scale (CRS), whereas Banker, Charnes and
Cooper (BCC) (1984) extended this to accommodate a more flexible model with variable
returns to scale (VRS). VRS, and hence the BCC model, are recommended when it is
impossible to assume that all observed units are operating at an optimal scale. Because
organizations in the health care sector often operate at an inefficient scale due to factors such
as imperfect competition and constraints on finance, we adopt the VRS assumption and opt
for the BCC model. The BCC approach requires choosing whether to assume an input or an
output orientation. Traditionally, studies evaluating hospital efficiency suppose a
minimization of inputs for a given level of outputs (input orientation) rather than a
maximization of outputs for a given level of inputs (output orientation). The input orientation
is justified based on the assumption that hospitals take public demand for healthcare as given,
and have to manage their inputs so as to meet these demands (Rego, Nunes, & Costa, 2010),
14
 ACCEPTED MANUSCRPT

and was applied in this study. Furthermore, Banker and Morey (1986) modified the BCC
model to account for non-discretionary inputs, which are factors that affect production but
may be outside of the assessed hospitals influence, such as in our case hospital beds.
Accordingly, we estimated the following objective function (Banker & Morey, 1986;
Syrjnen, 2004):

min
subject to ,
+ ,
+ ,
= ,
,
where subscript = 1, , represents the hospital; matrix refers to observed
inputs of compared hospitals; and matrix refers to the observed outputs of
compared hospitals. Vectors and are column vectors of inputs and outputs
of hospital (i.e., the th columns of matrix and , respectively). Similarly, matrix
refers to the observed non-discretionary (i.e., exogenously fixed) inputs of
compared hospitals, and refers to those of hospital . is a column vector of
constants, and is a scalar representing the efficiency score of hospital . Furthermore,
refers to a column vector of ones with a suitable dimension. The objective of this linear
program is to seek the minimum that reduces the input vector to while guaranteeing
at least the output level .

After estimating our DEA model, we applied the bootstrapping procedure developed by Simar
and Wilson (2007) to obtain the bias-corrected efficiency scores. The bias-corrected scores
were derived from 1,000 bootstrap iterations, allowing us to improve the statistical efficiency
in the second-stage truncated regression. For the sake of brevity, the term bias-corrected
efficiency is abbreviated as efficiency below. Due to legal restrictions or other changes
between years affecting hospital efficiency, we estimated the DEA model per year. The
analysis was performed in R 2.14.2 (R Development Core Team, 2015) using the FEAR
package (Wilson, 2008).

15
 ACCEPTED MANUSCRPT

3.4 Genetic matching

The absence of selection bias, i.e., meeting the assumption that the intervention is quasi
randomly or exogenously assigned, is a fundamental concern in health care studies with
observational data (Jones, 2007). The random sampling assumption implies that certified
hospitals (i.e., the intervention group) and non-certified hospitals (control group) do not differ
with respect to baseline characteristics, such as size or pre-certification performance. A
baseline imbalance may result in the over- or underestimation of the difference-in-difference
effect attributed to entry into a health system. For instance, if the probability of being certified
was higher for larger hospitals and if larger hospitals have greater performance gains from
certification than smaller hospitals, the results of the difference-indifference approach will
overestimate the effect of certification if no baseline balance is created. To address the
problem of baseline imbalance between the intervention and control groups, different
matching methods are available. Besides matching on single variables, the best-known
methods are propensity score matching (Rosenbaum & Rubin, 1983) and multivariate
matching based on Mahalanobis distance (Rubin, 1980). A relatively novel generalization of
both methods, genetic matching, was proposed by Diamond and Sekhon (2013) and Sekhon
and Grieve (2012). In this approach, the propensity score is included in the model as one of
the covariates, and the Mahalanobis distance metric between the covariates for two units
and is generalized by including an additional weight parameter, as follows:
12
( , , ) = {( ) ( 12 ) 12 ( )}

where is a positive definite weight matrix; 12 is the Cholesky decomposition of

the sample covariance matrix of ; and is the transpose of the matrix . Genetic
matching is a multivariate matching method that uses an evolutionary search algorithm to
iteratively check and maximize covariate balance across matched intervention and control
units by minimizing the largest observed covariate discrepancy at every step. A major
advantage of this method is that not only the variables means in the intervention and the
control group are aligned, but that both groups have the same joint distribution of observed
covariates after matching.

The method is nonparametric and does not depend on knowing or estimating the propensity
score, but the method is improved when a propensity score is incorporated (Sekhon, 2011).
Thus, we first perform a bootstrapped logistic regression by using defined covariates to derive
propensity scores, i.e., the probability that each hospital will be certified during the
16
 ACCEPTED MANUSCRPT

observation period. To do so, the intervention period in our case, the period during which a
QMS based on ISO or KTQ is implemented must be specified. According to Corbett et al.
(2005), implementing a QMS can take anywhere from six to 18 months. We thus defined our
intervention period as the year in which the certification was obtained ( ) and the year
immediately preceding certification ( 1). The period two years before certification ( 2)
was then used to determine the control group by applying genetic matching. We differentiate
between KTQ and ISO certification and perform two separate regressions.
The covariates used to predict the propensity scores represent hospital and environmental
characteristics, which may affect the likelihood of certification during the period observed.
The first covariate is hospital efficiency (Efficiency), which was obtained from the DEA
model. As the number of beds (Beds) in German hospitals has the nature of being exogenous,
it is also included in the model. We additionally included the proportion of all hospital beds
that had been leased (Leased beds) as another covariate, because hospitals in Germany are
allowed to lease hospital beds to self-employed private-practice physicians, and the cases
referred to these physicians are counted as hospital output (i.e., inpatient cases), whereas the
corresponding resource use in terms of physicians is not considered on the input side
(Tiemann & Schreygg, 2012). Further, we controlled for ownership (dummy variables
Private for-profit, Private nonprofit and reference category Public); competition, using the
Herfindahl-Hirschman index at the county level as a measure for market concentration
(Market concentration); and technological, medical or governmental changes (dummy
variables for the year of observation). Subsequently, we calculated the predicted probabilities
and linear predictors ofintervention assignment for each observation.
In a next step, we aimed to achieve balance among potential confounding variables. We
therefore used the same covariates as in the propensity score model and, moreover, the linear
predictor from the logistic regression (as the propensity score is bounded between zero and
one, Sekhon (2011) recommends using the linear predictor rather than the predicted
probability to avoid compression of propensity scores near zero and one), as potential
confounding variables for genetic matching. We applied one-to-one matching with
replacement i.e., each hospital in the control group could be drawn more than once, and the
total distance between matched pairs was also minimized. The appropriateness of the
matching was first assessed with a paired t-test comparing differences in means between
control and intervention group. For discrete variables, however, tests of the difference in
means are of limited informatory value. Matching can decrease the mean discrepancies but
17
 ACCEPTED MANUSCRPT

may also worsen the balance of other moments of a distribution, such as maximum, skew or
kurtosis. Therefore, the Kolmogorov-Smirnov (KS) test, which assesses the difference
between the empirical distribution functions, is more appropriate for assessing matching
quality of continuous variables (Sekhon & Grieve, 2012). The KS test provides the -statistic
for two samples (i.e., control and intervention group), which is defined as the supremum of
the absolute value of the difference between the empirical distribution functions 1, () and
2, ():
, = sup |1, () 2, ()|

for (control group) and accordingly (intervention group) independently and identically
distributed observations of covariate . Bootstrapping with 1,000 iterations was applied to
examine the significance of deviations between the distribution functions.

Logistic regression, genetic matching and the assessment of balance in the matched samples
were performed in R 3.0.1 (R Development Core Team, 2013) using the Matching package
(Sekhon, 2011).

3.5 Difference-in-difference estimates

In the third step of our analysis, the effect of certification on hospital efficiency was
estimated. Certification can be interpreted as an event that alters the course of an organization
and potentially its performance, too. The difference-in-difference approach was used to
identify changes in efficiency of certified hospitals relative to changes in efficiency of non-
certified hospitals, while controlling for time-variant variables. A basic assumption of this
approach is that all other temporal factors affecting hospital efficiency had the same impact
on hospitals in the intervention group as they did on hospitals in the control group. We thus
assume that all hospitals were affected in the same way by any changes over time that we did
not control for. To address the problem of serial correlation in our difference-in-difference
panel analysis, we applied individual regressions for each of the six comparison periods
(Bertrand, Duflo, & Mullainathan, 2004). This allowed us to investigate the impact of
certification not only for one fixed post-period, but as a progressing effect over time.
Following Simar and Wilson (2007), we applied a truncated regression model to take into
account that efficiency is truncated at 1, i.e., cannot take larger values than 1. The difference-
in-difference model was as follows:

18
 ACCEPTED MANUSCRPT

= 0 + 1 + 2 + 3 + 4 + +

where is the efficiency of the th hospital at period = 1 , 2 with 1 = 2

as the pre-certification period used for matching and 2 { 1, , + 1, + 2, + 3, + 4}
depending on which comparison period was analyzed; is a dummy variable with value 1
for hospitals in the intervention group and with value 0 for hospitals in the control group to
control for time-invariant differences between intervention and control group; is
assigned a value of 0 in period 1 and 1 in period 2 ; and are observable factors affecting
the efficiency of hospital at period (i.e., the same time-variant hospital and environmental
characteristics used for genetic matching except for the linear predictor). Finally, is the
hospitals fixed effect, and is the random error.

The coefficient of interest, 3 , is the interaction between and . It identifies

changes in efficiency in the intervention group relative to changes in efficiency in the control
group. Regressions are estimated separately for KTQ and ISO certification.

4 Findings

Table 3 shows the baseline statistics in measured covariates before and after matching.
Overall, the quality of the matching is very high, as all significant differences in the means of
the indicator variables and the distributions of the continuous variables were eliminated. The
contrast between hospitals with KTQ and ISO can be seen in the mean values of the variables.
For instance, a typical KTQ certified hospital had 304 beds, whereas a typical ISO certified
hospital had 251 beds. This emphasizes the need to investigate the effects of both types of
certification independently of one another. Mean efficiencies prior to certification are
comparable for KTQ and ISO certified hospitals (72.5 % and 72.4 %, respectively), but
significantly differ from the mean efficiency of non-certified hospitals (73.6%). This
underlines the relevance of the matching and difference-in-difference approach to allow any
causal conclusions.

------------------------------------------
Insert Table 3 about here
------------------------------------------

19
 ACCEPTED MANUSCRPT

The logistic regressions conducted to obtain the propensity scores (described in subsection
3.4) also reveal which variables significantly affect the likelihoods of being KTQ or ISO
certified. The results of these regressions are shown in Table 4, separately for KTQ and ISO.
As can be seen, the estimates from the KTQ and ISO models alternate in sign and
significance: whereas the likelihood to be KTQ certified significantly increased with hospital
size and decreased with higher baseline efficiency, the likelihood to be ISO certified
significantly depended on lower market concentration and being a public hospital.
------------------------------------------
Insert Table 4 about here
------------------------------------------

Table 5 contains the estimators for each difference-in-difference regression for the interaction
term , which represents the effect of certification on efficiency. Regarding KTQ
certification, the estimators were always positive, which means that KTQ certification has a
positive net effect on efficiency compared to the matching period 2 and to the control
group of comparable non-certified hospitals. The positive effect of certification was
significant in the year of certification and in the two years between the first and second
certification, and increased over this time period. Afterwards, the effect was still positive,
although it was not significant anymore, while the coefficient was lower in the year of
recertification and increased again one year later.

The estimators in the ISO models were always negative, which means that KTQ certification
has a negative net effect on efficiency compared to the matching period 2 and to the
control group of comparable non-certified hospitals. The coefficients in the ISO models were
significantly negative in the year before the first certification and in the year of certification.
The coefficients for the following two years were still negative, though insignificant. In the
year of recertification and in the subsequent year, the estimator showed largely significant
negative effects. Thus, ISO certification consistently showed negative effects onefficiency.

To examine whether our assumption of beds as fixed input might be too strong, we performed
a sensitivity analysis, removing beds as a fixed input, re-estimating the DEA, and performing
the same procedure as before based on the new efficiency scores. Moreover, the decision to
seek QMS certification may have resulted from privatization, which could also be interpreted
as an intervention that might have had an impact on efficiency (Tiemann & Schreygg, 2012).

20
 ACCEPTED MANUSCRPT

We therefore added a dummy variable for ownership change to the models and re-calculated
the difference-in-difference regressions. In a further sensitivity analysis, we first eliminated
hospitals with ownership change from the dataset and then proceeded as before. All of our
sensitivity analyses confirmed thedirections and significances of our original model.

------------------------------------------
Insert Table 5 about here
------------------------------------------

5 Discussion and limitations

The significant effects of certification in both the KTQ and ISO models indicate that different
certifications result in differences in efficiency changes relative to non-certified hospitals.
There are several potential explanations for the differences in the effects of the two types of
certifications. To start with, these may be explained by the more comprehensive requirements
and presumably higher workloads linked to the implementation of ISO. Although the self-
assessment needed for KTQ certification requires a comprehensive and time-consuming
report to be submitted shortly before the external assessment, ISO entails continuous
documentation of process results and thus a continuous use of resources. Even though the
effects of ISO certification were negative and not significant in the period between the two
certifications, it is remarkable that the estimators here were as in the KTQ models higher
than in the other periods. It is conceivable that the process of certification ties up resources,
which are subsequently released between certification and recertification. It may also be
possible that obtaining certification increases employee motivation, leading to greater
technical efficiency.

Considering that the KTQ system allows nonconformities when awarding certification, it is
also conceivable that ISO is more effective in facilitating the implementation of routines that
increase process quality or the quality of care in general. There is increasing evidence in
health care, especially hospital care, of a trade-off between the quality of care and costs (as a
proxy for efficiency). Greater increases in the quality of care resulting from a certain type of
QMS certification might therefore hinder increases in efficiency (Carey & Burgess Jr., 1999;
Stargardt, Schreygg, & Kondofersky, 2014). Another possible explanation may be
differences in the marketing effect. Weston (1995) found that ISO 9000 certification led to

21
 ACCEPTED MANUSCRPT

competitive marketing advantages in only 35% of the questioned manufacturing firms. This
finding may also apply to the hospital market, as patients likely associate the ISO standard
with industry. Another explanation could be that hospitals may not have achieved the
expected marketing benefits because they had not adequately marketed their certification
(Stevenson & Barnes, 2002). On the contrary, KTQ certification, having been developed for
the health care sector, may be easier to commercialize and therefore have a positive marketing
effect on patients, for example by attracting cases that fit well into a hospitals existing,
technically efficient processes.

Our study has several limitations worth mentioning. To start with, as we were unable to
obtain information about certification from hospitals that had closed, our regression results
may be subject to survivorship bias. For instance, if hospital lcosures tended to be theresult of
poor efficiency, and certified hospitals with poor efficiency were dropped from the sample as
they closed, we may have overestimated the effects of certification. Second, the effects of
certification could have been underestimated due to our having classified hospitals as certified
only when this had been confirmed by third-party audit. Hospitals can base their QMS on a
standard such as ISO or KTQ without certification and had been included in the control
group, even though their QMS might have been as effective as that in certified hospitals.
Finally, we were unable to control for changes in the quality of care resulting from
certification, although the principle aim of introducing a QMS is to improve the quality of
care. Hence our results only give hint to the advantageousness of certification with respect to
efficiency and do not allo
w any conclusions about quality of acre aspects.

Despite its limitations, the study also makes some valuable contributions. To start with, it
adds to existing insights on the antecedents and effects of certification by drawing on a large
and rich data set, including data from approximately 750 hospitals over a ten-year period and
a broad range of inputs and outputs, as well as environmental and organizational
characteristics. Moreover, the research setting, namely the hospital industry, is amenable to
quantitative longitudinal analysis of the effects of two different types of QMS certification.
The methodological approach is comprehensive and builds on recent developments in
relevant methodology. Finally, the sensitivity analyses underscore the validity and robustness
of the findings. Together, these represent an improvement over previous certification studies
and an extension of theevidence base on hospital certification in particular.

22
 ACCEPTED MANUSCRPT

As a result, the study provides some valuable insights for practice and future research. To
start with, our results indicate that from an efficiency point of view, the KTQ certification
seems more preferable than the ISO certification. Hospital managers hence can take into
account the varying effects of KTQ and ISO certification on efficiency when deciding upon
which QMS certification to aim for. Additionally, our results suggest that KTQ QMS might
bind fewer resources for installation and maintenance. Thus, also hospital managers who do
not intend to have their QMS externally certified could draw on these results if they intend to
base their QMS on standards such as ISO or KTQ without undergoing certification. However,
they should also bear in mind that there might exist a tradeoff between quality and efficiency,
and that this study does not provide any evidence for which QMS standard is more
appropriate in achieving high quality. As the negative effect of ISO certification on efficiency
may be due to the high documentation requirements for ISO certification, the technical
committees and advisory groups could attempt further reducing red tape and minimizing
documentation loads. However, as long as no sound empirical evidence on the impact of
certification standards on quality is provided, changes should be made carefully and
conservatively, because lowering requirements might affect quality outcomes.

Regarding future research, studies should examine the impact of QMS certification on
financial performance and, more importantly, the quality of care. In particular in settings
where quality is difficult to observe and assess which is frequently the case in service
industries , a higher quality does not directly translate into improvements in financial
performance. With respect to efficiency outcomes, it might even be oppositely related. Hence,
it is of utmost importance to provide sound empirical evidence on the certification-quality
link. Additionally, as our study shows that the effects of certification on efficiency differ with
different types of QMS certification, further comparing comprehensive and industry-specific
standards within one country or comparing national standards across countries would be
illuminating.

6 Conclusion

In the present study, we investigated the effect of QMS certification on the technical
efficiency of hospitals in Germany. The study analyzed the most common standards in
Germany, ISO 9001 and KTQ. Our findings indicate that the national standard KTQ has
significant positive effects on efficiency in the year of certification and in the following two
23
 ACCEPTED MANUSCRPT

years. The international ISO 9001 standard, however, which was originally developed for
industry, has a significant negative impact on efficiency in the year before and in the year of
certification, as well as in the year of recertification and the year after that. Although our
results suggest preferring the national KTQ standard in order to increase technical efficiency,
it is important to keep in mind that technical efficiency is only one aim associated with the
introduction of QMS. Further research is needed to quantify the impact of QMS certification
on financial performance and, more importantly, the quality of care.

24
 ACCEPTED MANUSCRPT

7 References

Banker, R. D.,& Morey, R. C. (1986). Efficiency Analysis for Exogenously Fixed Inputs and
Outputs. Operations Research, 34(4), 513521. doi:10.1287/opre.34.4.513
Banker, R. D., Cha rnes, A., &Cooper, W. W. (1984). Some Models for Estimating Technical
and Scale Inefficiencies in Data Envelopment Analysis. Management Science, 30(9),
10781092. doi:10.1287/mnsc.30.9.1078
Benner, M. .J, & Veloso, F. M. (2008). ISO 9000 practices and financial performance: A
technology coherence perspective. Journal of Operations Management, 26(5), 611629.
Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How Much Should We Trust Differences-
in-Differences Estimates? The Quarterly Journal of Economics, 119(1), 249275.
doi:10.1162/003355304772839588
Carey, K., & Burgess Jr., J. F. (1999). On measuring the hospital cost/quality trade-off.
Health Economics, 8(6), 509520. doi:10.1002/(SICI)1099-1050(199909)8:6<509::AID-
HEC460>3.0.CO;2-0
Charnes, A., Cooper, W. W., & Rhodes, E. 1978).( Measuring the feficiency of decision
making units. European Journal of Operational Research, 2(6), 429444.
doi:10.1016/0377-2217(78)90138-8
Chatzoglou, P., Chatzoudes, D., &Kipraios, N. (2015). The impact of ISO 9000 ecrtification
on firms financial performance. Int Jrnl of Op & Prod Mnagemnt (International Journal
of Operations & Production Management), 35(1), 145174. doi:10.1108/IJOPM-07-2012-
0387
Corbett, C. .J, Montes-Sancho, M. J., & Kirsch, D. A. 2005).
( TheFinancial Impact of ISO
9000 Certification in he t United States: An Empirical Analysis. Management Science,
51(7), 10461059. doi:10.1287/mnsc.1040.0358
Diamond, A., &Sekhon, J. S. (2013). Genetic Matching for Estimating Causal Effects: A
General Multivariate Matching Method for Achieving Balance in Observational Studies.
Review of Economics and Statistics, 95(3), 932945.
Dick, G. P. M., Her as, I., & Casadess, M. (2008). Shedding light on causation between ISO
9001 and improved business performance. Int Jrnl of Op & Prod Mnagemnt (International
Journal of Operations & Production Management), 28(7), 687708.
doi:10.1108/01443570810881811
Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sar rico, C.S., & Shale, E. A.
(2001). Pitfalls and protocols in DEA. European Journal of Operational Research, (132),
245259. doi:10.1016/S0377-2217(00)00149-1
European Commission eurostat. (2015). Healthcare statistics - Statistics Explained. Retrieved
from http://ec.europa.eu/eurostat/statistics-explained/index.php/Healthcare_statistics
Federal Statistical Office of Germany, [Destatis]. (2013). Krankenhausstatistik der Jahre
20002010: Hospital statistic of the years 2000-2010. Controlled remote data processing.
Heras-Saizarbitoria, I., Casadess, M., & Marim n, F. (2011). The impact of ISO 9001
standard and the EFQM model: The view of theassessors. Total Quality Management &
Business Excellence, 22(2), 197218.
Hollingsworth, B. (2008). The measurement of efficiency and productivity of health care
delivery. Health Economics, 17(10), 11071128. doi:10.1002/hec.1391
International Organization for Standardization. (2008). ISO 9000 Introduction and Support
Package: Guidance on the Documentation Requirements of ISO 9001:2008. Retrieved
25
 ACCEPTED MANUSCRPT

from http://www.iso.org/iso/02_guidance_on_the_documentation_requirements_of_iso_
9001_2008.pdf
Jones, A. M. (2 007). Identification of treatment effects in Health Economics. Health
Economics, 16, 11271131. doi:10.1002/hec.1302
Kooperation fr Transparenz und Qualitt im Gesundheitswesen. (2015). Das KTQ-Verfahren
(The process of KTQ certification). Retrieved from http://www.ktq.de/
Levine, D. I., & Toffel, M. W. (2010). Quality Management and Job Quality: How theISO
9001 Standard for Quality Management Systems Affects Employees and Employers.
Management Science, 56(6), 978996. doi:10.1287/mnsc.1100.1159
Lima, M. A. M., esende,
R M., &Hasenclever, L. (2000). Quality certification and
performance of Brazilian firms: An m e pirical study. International Journal of Production
Economics, 66, 143147.
Lindlbauer, I., & Schreygg, J. (2014). The relationship between hospital specialization and
hospital efficiency: do different measures of specialization lead to different results? Health
Care Management Science, 17(4), 365378. doi:10.1007/s10729-014-9275-1
Lindlbauer, I., Winter, V., & Schreygg, J. (2015). Antecedents and oc nsequences of
corporatization: An m e pirical analysis of German public hospitals. Journal of Public
Administration Research and Theory, accepted.
Lo, C.K. Y., Pagell, M., Fan, D., Wiengarten, F., & Yeung, A. C. L. (2014). OHSAS 18001
certification and operating performance: The role of complexity and coupling. Journal of
Operations Management, 32(5), 268280. doi:10.1016/j.jom.2014.04.004
Makai, P., Klazinga, N., Wagner, C., Boncz, I., & Gulacsi, L. (2009). Quality management
and patient safety: Survey results from 102 Hungarian hospitals. Health Policy, 90(2-3),
175180. doi:10.1016/j.healthpol.2008.09.009
Marazzi, A.,Paccaud, F., Ruffieux, C., &Beguin, C. (1998). Fitting the Distributions of
Length of Stay by Parametric Models. Medical Care, 36(6), 915927.
doi:10.1097/00005650-199806000-00014
Martnez-Costa, M., Choi, T. Y., Ma rtnez, J. A.,& Martnez-Lorente, A. R. (2009). ISO
9000/1994, ISO 9001/2000 and TQM: The performance debate revisited. Journal of
Operations Management, 27(6), 495511. doi:10.1016/j.jom.2009.04.002
Martnez-Costa, M., Martnez-Lorente, A. R., &Choi, T. Y. 2008). ( Simultaneous
consideration of TQM and ISO 9000 on erformance
p and motivation: An me pirical study
of Spanish companies. International Journal of Production Economics, 113(1), 2339.
doi:10.1016/j.ijpe.2007.02.046
McGuire, S. J., & Dilts, D. M. (2 008). Thefinancial impact of standard stringency: An ve ent
study of successive generations of theISO 9000 standard. International Journal of
Production Economics, 113, 322. doi:10.1016/j.ijpe.2007.02.045
Naveh, E., & Marcus, A. (2005). Achieving competitive advantage through implementing a
replicable management standard: installing and using ISO 9000. Journal of Operations
Management, 24, 126. Retrieved from http://ac.els-cdn.com/S0272696305000525/1-s2.0-
S0272696305000525-main.pdf?_tid=28cc7af0-e443-11e4-b167-
00000aacb35d&acdnat=1429193962_1a478f331e91cfee7f55fd53926939df
Nicolau, J. L., & Sellers, R. (2002). The stock markets reaction to quality certification:
Empirical evidence from Spain. European Journal of Operational Research, 142, 634
642.

26
 ACCEPTED MANUSCRPT

Prajogo, D. I. (2011). The roles offirms' motives in affecting the outcomes of ISO 9000
adoption. Int Jrnl of Op & Prod Mnagemnt (International Journal of Operations &
Production Management), 31(1), 78100.
R Development Core Team. (2015). R: A Language and Environment for Statistical
Computing (Version 3.0.0). Vienna, Austria: R Foundation for Statistical Computing.
Retrieved from www.r-project.org
Rego, G., Nunes, R., &Costa, J. (2010). The Challenge of Corporatisation: The Experience of
Portuguese Public Hospitals. European Journal of Health Economics, 11(4), 367381.
Rosenbaum, P. R., &Rubin, D. B. (1983). The central role of thepropensity score in
observational studies for causal effects. Biometrika, 10(1), 4155.
doi:10.1093/biomet/70.1.41
Rubin, D. B. (1980). Bias Reduction Using Mahalanobis-Metric Matching. Biometrics, 36(2),
293298. doi:10.2307/2529981
Sampaio, P., Saraiva, P., &Guimares Rodrigues, A. (2009). ISO 9001 ecrtification research:
questions, answers and approaches. International Journal of Quality & Reliability
Management, 26(1), 3858. doi:10.1108/02656710910924161
Sampaio, P., Saraiva, P., &Monteiro, A. (2012). ISO 9001 ecrtification payoff: myth
versus reality. International Journal of Quality & Reliability Management, 29(8), 891914.
doi:10.1108/02656711211270351
Sekhon, J. S. (2011). Multivariate and Propensity Score Matching Software with Automated
Balance Optimization: The Matching package for R. Journal of Statistical Software, 42(7),
152.
Sekhon, J. S., & Grieve, R. D. (2012) . A matching method orf improving covariate balance in
cost-effectiveness analyses. Health Economics, 21(6), 695714. doi:10.1002/hec.1748
Sharma, D. S. (2005) . The association between ISO 9000 ecrtification and financial
performance. The International Journal of Accounting, 40(2), 151172.
doi:10.1016/j.intacc.2005.01.011
Shaw, C. D., rBuneau, C., Kutryba, B., Jongh, G. de, & Sunol, R. (2010). Towards hospital
standardization in Europe. International Journal for Quality in Health Care, 22(4), 244
249. doi:10.1093/intqhc/mzq030
Shaw, C. D., K utryba, B., Braithwaite, J., Bedlicki, M., & Warunek, A. (2010). Sustainable
healthcare accreditation: messages from Europe in 2009. International Journal for Quality
in Health Care, 22(5), 341350. doi:10.1093/intqhc/mzq043
Simar, L., & Wilson, P. W. (2007). Estimation and inference in wo-stage,
t semi-parametric
models of production processes. Journal of Econometrics, 136(1), 3164.
doi:10.1016/j.jeconom.2005.07.009
Stargardt, T., Schreygg, J., & Kondofersky, I. (2014). Measuring the Relationship Between
Costs and Outcomes: The Example of Acute Myocardial Infarction in German Hospitals.
Health Economics, 23(6), 653669. doi:10.1002/hec.2941
Steinmann, L., Dittrich, G., Karmann, A., &Zweifel, P. (2004). Measuring and comparing the
(in)efficiency of German and Swiss hospitals. The European Journal of Health Economics,
formerly: HEPAC, 5(3), 216226. doi:10.1007/s10198-004-0227-4
Stevenson, T. H., &Barnes, F. C. (2002). What industrial marketers need to know nowabout
ISO 9000 certification. Industrial Marketing Management, 31(8), 695703.
doi:10.1016/S0019-8501(01)00180-8

27
 ACCEPTED MANUSCRPT

Syrjnen, M. .J (2004). Non-discretionary and discretionary factors and scale in data

envelopment analysis. European Journal of Operational Research, 158(1), 2033.
http://doi.org/10.1016/S0377-2217(03)00362-X
Tamayo-Torres, J., Gutierrez-Gutierrez, L., & Ruiz-Moreno, A. 2014).
( The relationship
between exploration and exploitation strategies, manufacturing flexibility and
organizational learning: An m e pirical comparison between Non-ISO and ISO certified
firms. European Journal of Operational Research, 232(1), 7286.
doi:10.1016/j.ejor.2013.06.040
Terziovski, M., Power, D., & Sohal, A. S. (2003). The longitudinal effects of theISO 9000
certification process on business performance. European Journal of Operational Research,
146, 580595.
Terziovski, M., Samson, D., & Dow, D. (1997). The business value of quality management
systems certification. Evidence from Australia and New Zealand. Journal of Operations
Management, 15, 118.
Tiemann, O., &Schreygg, J. (2012). Changes in hospital efficiency after privatization.
Health Care Management Science, 15(4), 310326. doi:10.1007/s10729-012-9193-z
Tzelepis, D., Tsekouras, K., Skuras, D., &Dimara, E. 2006).
( The effects of ISO 9001 on
firms' productive efficiency. Int Jrnl of Op & Prod Mnagemnt (International Journal of
Operations & Production Management), 26(10), 11461165.
doi:10.1108/01443570610691111
van Walraven, C., Augustin, P. C.,Jennings, A., Quan, H., &Forster, A. J. (1985). A
Modification of the Elixhauser Comorbidity Measures into a Point System for Hospital
Death Using Administrative Data. American Statistician, 39(1), 626633.
doi:10.1080/00031305.1985.10479383
Weston, F. C. (1995). What do managers really think of theISO 9000 registration process?
Quality Progress, 28(10), 6774.
Wilson, P. W. (2008). FEAR: A software package for frontier efficiency analysis with R.
Socio-Economic Planning Sciences, 42(4), 247254. doi:10.1016/j.seps.2007.02.001
Yeung, A. C.L., Lo, C.K. Y., & Cheng, T. C. E.(2011). Behind the Iron Cage: An
Institutional Perspective on ISO 9000 Adoption and CEO Compensation. Organization
Science, 22(6), 16001612. doi:10.1287/orsc.1110.0652

28

Tables
Table 1
Outline of statistical analysis
Step
1. Collecting data from two sources
and conducting data mining
(subsection Sample)
2. Combining data from both sources
and identifying hospitals with
treatment (i.e. KTQ or ISO
certification) and hospitals without
certification
(subsection Sample)
3. Applying data envelopment analysis
(DEA) and estimating bootstrapped
efficiency scores
(subsection DEA)
ACCEPTED MANUSCRPT
Implementation and sample size
Hospital data from the annual hospital reports Data collection
Data from 2,390 German acute care hospitals were obtained between Data collection was originally based on a list of all 2,008 German acute
2000 and 2010 containing care hospitals existing in 2009. To ensure comparability of the hospitals
a) hospital-level information on cost and hospital infrastructure in the sample we excluded
b) patient-level information on age, gender, main diagnosis and - hospitals providing only psychiatric or child
length of stay leading to the number of weighted cases per care
hospital - university and military hospitals
(subsection DEA). - day clinics
To ensure comparability of the hospitals in the sample we excluded hospitals with fewer than 50 beds and more than 2,000 beds
- hospitals providing only psychiatric or child care A total of 1,453 hospitals remained in the sample. The structural quality
- university and military hospitals report 2010 of almost all hospitals was online available and following
- day clinics information have been extracted
- hospitals with fewer than 50 beds and more than 2,000 beds - whether a certification is reached actually
- the type, first year and duration of certification
A total of 1,602 hospitals remained in the sample. A total of 1,424 hospitals remained in the sample.
After merging both datasets, we excluded hospitals with
- only partly ISO certification
- EFQM level
- JCI accreditation
- with interrupted or double certifications
- missing information from 2000 to 2010
- hospitals without at least two pre-certification and one post-certification period
A total of 830 hospitals remained in the sample:
273 hospitals which reached a KTQ certification in the years 2002 through 2009
101 hospitals which reached an ISO certification in the same period
456 hospitals without any certification
Application of an input-oriented variable returns to scale model with non-discretionary inputs for all hospitals in the dataset per year. Deriving
bias-corrected DEA efficiency scores from 1,000 bootstrap iterations.
29

Table 1: oc ntinued
Step
4. Conducting Genetic Matching
(subsection Genetic Matching)
5. Matching diagnostics
(subsection Genetic Matching)
6. Regression analysis
(subsection Difference-in-
difference estimate)
ACCEPTED MANUSCRPT
Implementation and sample size
a) Estimate conditional probability for the dummy variable assignment to treatment based on our defined
covariates by logistic regression.
b) Match hospitals of each treatment group to hospitals without certification on defined covariates by using
one-to-one matching with replacement.
Paired t-tests and Kolmogorov-Smirnov tests.
Use of a truncated regression model with bias-corrected DEA efficiency scores asdependent variable.
Difference-in-difference interaction was used to examine effect of certification on efficiency compared to hospitals without certification while
controlling for time-variant hospital organizational and environmental characteristics.
30
 ACCEPTED MANUSCRPT

Table 2

Descriptive statistics for inputs and output stratified according to certification before genetic
matching

Hospitals Hospitals with Hospitals with

without certification KTQ certification ISO certification
mean stand. dev. mean stand. dev. mean stand. dev.

Year 2000
Number of hospitals n=400 n=254 n=95
Cases (per 1,000) 7.2 5.5 9.4 6.5 7.9 5.3
Weighted cases (per 1,000) 7.0 5.2 9.1 6.2 7.6 4.9
Beds 240.9 169.1 312.2 205.2 258.1 150.6
Physician 38.1 40.2 57.0 54.5 43.0 34.6
Nurse 129.3 110.0 175.6 135.4 143.3 90.0
Clinical 71.7 71.9 101.4 89.7 77.6 58.4
Nonclinical 45.6 39.5 61.5 57.8 53.9 41.2
Admin 21.2 17.1 28.6 21.1 23.0 14.1
Supplies (per 100,000 EUR) 67.7 75.1 96.5 86.2 72.4 53.5

Year 2005
Number of hospitals n=444 n=269 n=101
Cases (per 1,000) 7.7 6.5 10.1 6.8 8.2 5.8
Weighted cases (per 1,000) 7.5 6.2 9.7 6.3 7.7 5.2
Beds 235.2 180.3 304.2 191.5 252.4 152.6
Physician 45.1 50.2 65.7 58.2 49.0 43.3
Nurse 122.7 111.9 166.4 122.1 130.4 87.8
Clinical 76.1 81.2 103.3 85.4 78.2 62.4
Nonclinical 40.4 38.3 52.2 45.9 41.3 33.2
Admin 21.7 18.6 28.8 20.4 23.3 15.5
Supplies (per 100,000 EUR) 88.3 97.9 124.5 104.4 95.4 78.0

Year 2010
Number of hospitals n=434 n=266 n=99
Cases (per 1,000) 8.5 7.5 11.3 7.7 9.4 7.4
Weighted cases (per 1,000) 8.2 7.2 10.8 7.0 8.9 6.8
Beds 230.7 190.0 298.2 190.6 248.2 173.6
Physician 51.1 58.6 74.3 65.3 59.5 58.0
Nurse 125.7 116.3 173.5 127.3 144.3 110.1
Clinical 82.0 87.1 111.9 88.1 92.6 83.3
Nonclinical 32.8 32.9 44.4 37.8 39.4 31.0
Admin 22.0 20.5 29.0 19.2 25.9 21.2
Supplies (per 100,000 EUR) 121.9 135.6 168.0 136.1 133.3 119.4

31
 ACCEPTED MANUSCRPT

Table 3

Balance statistics in measured covariates before and after genetic matching

KTQ certification
Hospitals Hospitals Bootstrapped
t-test
t-2 without with D-statistic KS-test
(p-value)
certification certification (p-value)

Number of hospitals
before Genetic Matching 3,423 273
after Genetic Matching 273 273
Mean Beds
before Genetic Matching 239 304 0.000 0.215 0.000
after Genetic Matching 301 304 0.356 0.059 0.677
Mean Efficiency
before Genetic Matching 0.736 0.725 0.147 0.051 0.529
after Genetic Matching 0.726 0.725 0.325 0.033 1.000
Mean market concentration
before Genetic Matching 0.190 0.184 0.503 0.093 0.021
after Genetic Matching 0.181 0.184 0.560 0.055 0.810
Mean Leased Beds
before Genetic Matching 0.078 0.056 0.003 0.081 0.047
after Genetic Matching 0.054 0.056 0.327 0.066 0.428
Mean Pscore
before Genetic Matching 0.071 0.110 0.000 0.309 0.000
after Genetic Matching 0.110 0.110 0.786 0.037 0.988
Mean Linear predictor
before Genetic Matching -2.839 -2.254 0.000 0.309 0.000
after Genetic Matching -2.250 -2.254 0.672 0.037 0.988
Private
before Genetic Matching 0.198 0.154 0.053 - -
after Genetic Matching 0.154 0.154 1.000 - -
Non-profit
before Genetic Matching 0.470 0.509 0.212 - -
after Genetic Matching 0.509 0.509 1.000 - -
Year 2001
before Genetic Matching 0.121 0.040 0.000 - -
after Genetic Matching 0.040 0.040 1.000 - -
Year 2002
before Genetic Matching 0.124 0.154 0.181 - -
after Genetic Matching 0.158 0.154 0.564 - -
Year 2003
before Genetic Matching 0.125 0.282 0.000 - -
after Genetic Matching 0.282 0.282 1.000 - -
Year 2004
before Genetic Matching 0.126 0.169 0.068 - -
after Genetic Matching 0.169 0.169 1.000 - -
Year 2005
before Genetic Matching 0.130 0.106 0.230 - -
after Genetic Matching 0.106 0.106 1.000 - -
Year 2006
before Genetic Matching 0.130 0.125 0.804 - -
after Genetic Matching 0.132 0.125 0.157 - -
Year 2007
before Genetic Matching 0.129 0.103 0.178 - -
after Genetic Matching 0.095 0.103 0.317 - -

32
 ACCEPTED MANUSCRPT

Table 3:continued

ISO certification
Hospitals Hospitals Bootstrapped
t-test
t-2 without with D-statistic KS-test
(p-value)
certification certification (p-value)

Number of hospitals
before Genetic Matching 3,423 101
after Genetic Matching 101 101
Mean Beds
before Genetic Matching 239 251 0.432 0.099 0.258
after Genetic Matching 249 251 0.625 0.069 0.952
Mean Efficiency
before Genetic Matching 0.736 0.724 0.338 0.088 0.422
after Genetic Matching 0.725 0.724 0.965 0.099 0.676
Mean market concentration
before Genetic Matching 0.190 0.173 0.184 0.119 0.109
after Genetic Matching 0.173 0.173 0.821 0.089 0.797
Mean Leased Beds
before Genetic Matching 0.078 0.072 0.641 0.073 0.499
after Genetic Matching 0.071 0.072 0.592 0.099 0.586
Mean Pscore
before Genetic Matching 0.029 0.033 0.000 0.202 0.001
after Genetic Matching 0.033 0.033 0.961 0.059 0.989
Mean Linear predictor
before Genetic Matching -3.618 -3.438 0.000 0.202 0.001
after Genetic Matching -3.441 -3.438 0.762 0.059 0.989
Private
before Genetic Matching 0.198 0.178 0.605 - -
after Genetic Matching 0.178 0.178 1.000 - -
Non-profit
before Genetic Matching 0.470 0.396 0.141 - -
after Genetic Matching 0.396 0.396 1.000 - -
Year 2001
before Genetic Matching 0.121 0.129 0.826 - -
after Genetic Matching 0.129 0.129 1.000 - -
Year 2002
before Genetic Matching 0.124 0.050 0.001 - -
after Genetic Matching 0.050 0.050 1.000 - -
Year 2003
before Genetic Matching 0.125 0.149 0.511 - -
after Genetic Matching 0.139 0.149 0.317 - -
Year 2004
before Genetic Matching 0.126 0.139 0.711 - -
after Genetic Matching 0.158 0.139 0.156 - -
Year 2005
before Genetic Matching 0.130 0.139 0.800 - -
after Genetic Matching 0.129 0.139 0.317 - -
Year 2006
before Genetic Matching 0.130 0.129 0.977 - -
after Genetic Matching 0.129 0.129 1.000 - -
Year 2007
before Genetic Matching 0.129 0.158 0.421 - -
after Genetic Matching 0.158 0.158 1.000 - -
KS-test is based on 1,000 bootstrap iterations; post matching the paired t-test is provided

33
 ACCEPTED MANUSCRPT

Table 4

Logistic regression on binary variable certification

KTQ certification ISO certification

Intercept -3.276 *** (0.5822) -2.4188 *** (0.7222)

Beds 0.0019 *** (0.0003) 0.0006 (0.0006)

Leased beds -0.5775 (0.5758) 0.0192 (0.7391)

Market concentration -0.6563 (0.5120) -1.6002 * (0.8926)

Pubic (Reference) (Reference)

Private -0.2921 (0.2015) -0.5101 * (0.2994)

Non-profit 0.1067 (0.1496) -0.5179 ** (0.2333)

Efficiency -1.7869 *** (0.6049) -1.0049 (0.9360)
Year 2000 (Reference) (Reference)
Year 2001 0.5925 (0.5135) 0.1412 (0.4163)

Year 2002 1.8506 *** (0.4433) -0.8548 (0.5451)

Year 2003 2.5307 *** (0.4309) 0.2825 (0.4041)

Year 2004 2.0505 *** (0.4417) 0.2269 (0.4111)

Year 2005 1.5616 *** (0.4556) 0.1984 (0.4106)
Year 2006 1.6787 *** (0.4497) 0.0943 (0.4174)

Year 2007 1.4902 *** (0.4568) 0.3154 (0.4002)

*p<0.1; **p<0.05; ***p<0.01; SEin parentheses

34
 ACCEPTED MANUSCRPT

Table 5

Truncated regression results for difference-in-differences models on efficiency

KTQ certification ISO certification

Period compared to t-2
n N Interaction term n N Interaction term

t-1 One year before 540 1,080 0.001 (0.004) 201 402 -0.016 *** (0.006)
certification

t Year of 540 1,080 0.008 * (0.005) 200 400 -0.020 ** (0.008)

certification
t+1 One year after 539 1,078 0.017 *** (0.005) 200 400 -0.012 (0.009)
certification
t+2 Two years after 485 970 0.019 *** (0.006) 167 334 -0.015 (0.010)
certification
t+3 Year of 419 838 0.004 (0.007) 143 286 -0.042 *** (0.011)
recertification
t+4 One year after 358 716 0.006 (0.007) 117 234 -0.032 ** (0.013)
recertification
*p<0.1; **p<0.05; ***p<0.01; SEin parentheses; n number of hospitals; N number of observations

35