Professional Documents
Culture Documents
12252
ISSN 1059-1478|EISSN 1937-5956|15|2403|0369 © 2014 Production and Operations Management Society
Gopesh Anand
Department of Business Administration, University of Illinois at Urbana-Champaign, 350 Wohlers Hall, 1206 S. Sixth Street, Champaign,
Illinois 61820, USA, gopesh@illinois.edu
Aleda V. Roth
Department of Management, College of Business and Behavioral Sciences, Clemson University, 132E Sirrine Hall, Clemson,
South Carolina 29634-1305, USA, aroth@clemson.edu
his article examines the influence of ISO 9000 certification on plant-level process compliance, which arguably is its
T first-order, targeted performance dimension. The empirical setting is the medical device manufacturing industry. Pro-
cess compliance is measured through Food and Drug Administration inspections of manufacturing plants. We control for
several observable factors that possibly affect process compliance by matching certified plants with non-certified plants.
Using longitudinal data, we find plants that obtained certification in the earlier diffusion period (early-certified plants)
tend to have significantly better process compliance than a matched, non-certified control group of plants. The compliance
difference between early-certified plants and their matched control group is greater than the compliance difference
between late-certified plants and their matched control group. We also find deterioration in process compliance over time
after certification. Because we capture longitudinally the first-order effects of ISO 9000 on process compliance, this study
provides a useful baseline for assessing causality in ISO 9000-performance linkages. Also, we explain, in part, the inconsis-
tencies observed in related ISO 9000 literature examining the performance effects of certification. Further, this research
offers managerial insights on the dynamics of certification and process compliance with time, and highlights the need for
continued vigilance post certification.
Key words: certifications; decay; diffusion; management standards; quality control
History: Received: July 2012; Accepted: March 2014 by Kalyan Singhal, after 3 revisions.
the production processes reside, which may intro- of ISO 9000 have produced inconsistent results.
duce attribution errors. In contrast, this study is the Because a cross-sectional study cannot account for
first to longitudinally assess the first-order, direct these two temporal factors individually, the date of
effect of ISO 9000 certification at the plant level. certification (earlier is better) and the time since certi-
Therefore, we can more precisely assess its effective- fication (more recent is better) may counteract each
ness regarding the intended process compliance out- other in cross-sectional research.
come over time. In so doing, we help to reconcile
some contradictions in the literature.
To gauge the certification-compliance dynamics,
2. Literature
this study uses a 13-year panel of data on manufactur- We classify previous empirical studies of ISO 9000
ing plants in the medical device industry. The mea- certification into two categories based on the principal
sure of process compliance is based on the results of phenomena they examine: (1) the diffusion of ISO
inspections conducted by the Food and Drug Admin- 9000 and (2) performance effects. Studies in the first
istration (FDA). We consider two time-related aspects category focus on the adoption process of ISO 9000
of certification: the timing of ISO 9000 certification and its diffusion in industries and countries, address-
and the passage of time after certification. Regarding ing why and how organizations choose to obtain certi-
the timing of initial certification, we compare process fication (e.g., Anderson et al. 1999, Boiral 2003,
compliance of certified plants to control groups of Corbett 2006, Ivanova et al. 2014). These studies
non-certified plants and contrast plants certified early inform our arguments relating the timing of adoption
in the diffusion of the certification with those certified to its performance effects.
later. Our results indicate that early certification In this study, the main focus of the ISO 9000 litera-
relates largely to superior process compliance. Next, ture review is on studies that associate certification
considering the post-certification period, our study with performance. We organize this literature in
reveals deterioration in process compliance among all Table 1 along two dimensions: (1) type of performance
plants over time after certification. Delving further assessed, which is divided between performance
into this result, we find it is primarily driven by dete- related to ISO 9000’s effect on process compliance-
rioration among early-certified plants in the ISO 9000 related outcomes and to other outcomes; and (2)
diffusion process. Thus, our empirical analyses pro- research design, which is bifurcated between cross-
vide plausible explanations for why existing studies sectional and longitudinal. Within each cell, we
Table 1 Classification of the Quantitative Empirical Literature Examining the Performance Consequences of ISO 9000 Certification: Type of
Performance Measure by Time Frame*
Performance measure
Time frame Other than process compliance/quality performance Process compliance/quality performance
Cross-sectional Perceptual measures: Bernardo et al. (2012), Prajogo (2011), Perceptual measures: Singh et al. (2011), Karapetrovic et al. (2010)†,
Martınez-Costa et al. (2009), Boiral and Roy (2007), Nair and Prajogo (2009), Singh (2008), Casadesus and
Naveh et al. (2004), Chow-Chua et al. (2003), Karapetrovic (2005), Naveh and Marcus (2005)‡,
Terziovski et al. (2003), Jones et al. (1997), Naveh and Erez (2004)§, Naveh and Marcus (2004),
Terziovski et al. (1997) Dreyfus et al. (2004), Singels et al. (2001), Brown et al. (1998),
Archival measures: Simmons and White (1999) Curkovic and Handfield (1996)
Longitudinal Archival measures: Lo et al. (2013), Iyer et al. (2013),
Yeung et al. (2011), Levine and Toffel (2010), Benner (2009),
Lo et al. (2009), Benner and Veloso (2008), Dick et al. (2008),
Naveh and Marcus (2007), Terlaak and King (2006), The “gap” in the ISO 9000 literature addressed by this study
Corbett et al. (2005), Sharma (2005), Naveh and
Marcus (2005)‡, Benner and Tushman (2002),
Heras et al. (2002), Wayhan et al. (2002), H€aversj€o (2000),
Docking and Dowen (1999)
identify whether the research is based on archival examined (Boiral 2012, Boyer 1999). Furthermore,
measures (secondary data) or perceptual measures cross-sectional research cannot disentangle two
(primary data). Table 1 is representative of a more dynamic aspects related to plant level certification:
exhaustive literature review. namely, the timing of ISO 9000 certification in the
Table 1 illustrates two key aspects of the extant diffusion process and the passage of time after plant
literature. First, all the research that uses secondary- certification. Perhaps due to these temporal factors,
source archival performance measures has empha- some studies found ISO 9000 certification to be related
sized outcomes other than process compliance to improved process compliance performance (e.g.,
quality. Examples include productivity (Iyer et al. Naveh and Erez 2004, Singh et al. 2011); others did
2013), return on assets (Lo et al. 2013), Tobin’s q not (e.g., Curkovic and Handfield 1996, Karapetrovic
(Benner and Veloso 2008), sales and profit margins et al. 2010).
(Naveh and Marcus 2005), and stock prices (Docking This study targets the void in the ISO 9000 literature
and Dowen 1999). Moreover, most of this research is on longitudinal studies of process compliance repre-
at level of analysis of the firm, as explained by Levine sented in the lower right quadrant of Table 1. Levine
and Toffel (2010, p. 980): “ISO 9000 is typically and Toffel (2010, p. 979) noted that there is “no prior
adopted at the plant level, whereas outcome data are [longitudinal] research that examines how the ISO
typically available only at the firm level.” Certification 9000 quality standard affects product or process qual-
may certainly affect non-process compliance-related ity, or employees.” Levine and Toffel’s seminal work
performance outcomes. However, such outcomes aimed to fill the gap in employee-related outcomes
often are second-order effects that materialize from through a longitudinal study of ISO 9000 certification.
improved operational outcomes. Furthermore, sec- As the first to focus exclusively on the first-order pro-
ond-order performance improvements also may be cess compliance effects of ISO 9000 certification at the
attributed to other non-operational factors, such as manufacturing plant level longitudinally, this study
“signaling,” in which the certified company is treated complements their work.
more favorably by its customers and by the stock mar-
ket, regardless of any realized, process-level benefits
of certification (Terlaak and King 2006). Perhaps
3. Theory and Hypotheses
because of the varied causal chains from certification 3.1. Influence of the Date of Certification
to such non-process compliance performance dimen- We first address the issue of whether a manufacturing
sions, these studies have had mixed results (Boiral plant obtains ISO 9000 “early” or “late,” referring to
2012), most finding benefits (e.g., Corbett et al. 2005, the broader period of diffusion of this certification.
Levine and Toffel 2010, Lo et al. 2009, Naveh and Institutional theory suggests that first-mover organi-
Marcus 2007) and others failing to do so (e.g., Heras zations obtaining certification during the earlier diffu-
et al. 2002, Terziovski et al. 1997, Yeung et al. 2011). sion of a standard are more likely to be internally
Table 1 also shows that all the research on the first- motivated to adopt its underlying practices (Westphal
order effects of ISO 9000 certification on process com- and Zajac 1994). Early adopters may be more likely to
pliance/quality performance (as well as some deliberately scrutinize existing values, reframe strate-
research examining other performance dimensions) gies, and take corrective actions to enable effective
used perceptual outcome measures and was cross- adoption of the standard. In other words, early
sectional. The self-administered surveys consisting of adopters may be more likely to pursue double-loop
psychometric scales used in these studies allowed learning, which “involves modifications of an organi-
researchers to consider a rich set of explanatory fac- zation’s underlying norms, policies and objectives”
tors related to ISO 9000 adoption. These factors (Argyris and Sch€ on 1978, p. 3).
include adoption costs (Chow-Chua et al. 2003), In contrast, organizations obtaining certification
quality management experience (Naveh et al. 2004), relatively later in the diffusion of the certification
integration of quality management practices standard are more apt to do so in response to an exter-
(Martınez-Costa et al. 2009, Naveh and Marcus 2005), nal motivation, such as copying competitors or
and motivation for adoption (e.g., Naveh et al. 2004, appeasing customers (Barratt and Choi 2007, Lee and
Prajogo 2011). However, perceptual measures of per- Klassen 2008). Such externally motivated seekers of
formance, especially when collected using the same certifications are, on average, more likely to ignore
method as independent variables, can be prone to the certification’s stated intentions, and adopt rituals
common methods bias (Podsakoff et al. 2003, Siemsen superficially to achieve symbolic certification (Keto-
et al. 2010). More importantly, the absence of longitu- kivi and Schroeder 2004). Adopting any management
dinal studies of the first-order effect of certification on system can create a mismatch between intended out-
process compliance limits the extent to which causal- comes and organizational goals, and disrupt existing
ity of certification–performance relationships can be systems (Abrahamson 1991, Westphal et al. 1997). In
Gray, Anand, and Roth: IS0 9000 and Process Compliance
372 Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society
other words, later adopters may be more prone to compliance decay (Anand et al. 2012, Katz and Kahn
pursue single-loop learning (Argyris and Sch€
on 1978), 1978, Knott 2001). This deterioration is consistent with
making reactionary adjustments to their processes to the observed declines in implementing coherent oper-
fulfill the standard’s requirements. Based on the ational strategies in manufacturing plants as priorities
above arguments, we posit two hypotheses: evolve (Giffi et al. 1990, Hill and Duke-Woolley 1983).
In light of these two plausible opposing effects
HYPOTHESIS 1 (H1). Early ISO 9000-certified plants drawn from organizational learning theories, the
exhibit greater post-certification quality-related process directional relationship between the passage of time
compliance than non-certified matched plants, on average. after certification and process compliance is unclear.
Therefore, we propose two competing hypotheses:
HYPOTHESIS 2 (H2). There is a greater difference in
post-certification quality-related process compliance HYPOTHESIS 3A[B] (H3A[B]). Quality-related process
between early ISO 9000-certified plants and their compliance performance at the plant level increases
matched non-certified plants than there is between late- [decreases] as time passes after ISO 9000 certification, on
certified plants and their matched non-certified plants. average.
0 when the plant was in the highest observed state of We discussed these concerns with the database
compliance (i.e., a Form 483 was not issued and a owners at Quality Digest, who stated: “The data
district decision of “No Action” was recorded); and provided by the registrars got progressively worse,
NonComply equals 3.5 when the plant was severely and the updates became very few and far between,
non-compliant (i.e., a Form 483 was issued and a dis- to the point where we felt it wasn’t really serving its
trict decision indicated “Official Action” was taken). intended purpose, which is why we discontinued it
The Gray et al. (2011) scoring procedure of FDA [sometime after 2007]” (D. Dusharme, Editor-in-
inspections was developed following an iterative Del- Chief, Quality Digest, pers. comm., April 21, 2010).
phi process, employing a panel of industry experts. Additional communication with Mary Benner (Assis-
As noted in Anand et al. (2012), in pharmaceuticals, tant Professor, Wharton School of Management, pers.
and observed in this study’s medical device data, the comm., June 23, 2010), who has multiple publications
FDA seems to be issuing fewer “Official Action” out- based on the McGraw-Hill data, indicated that the
comes over time. Anand et al. (2012, p. 1710) stated certification data collected through 2000, and used in
“. . .the year dummies. . .seem to indicate that inspec- this study, were indeed accurate. To reduce the
tion outcomes have been tending to improve through possibility of misclassifying a certified plant as non-
the years. . ..Because it is our understanding that there certified because it (1) became certified later than
were no substantial changes to the inspection protocol 2000, (2) was certified but inadvertently omitted
during our study time frame, this result could also from the McGraw-Hill database, and/or (3) obtained
imply that the FDA’s district managers in the 2000s an industry-specific ISO quality-related certification,
began to make it easier for plants to obtain better we maintained the unreliable information about
inspection outcomes.” To address this apparent eas- certification from Quality Digest’s ISO certification
ing of FDA inspection scores over time, in the current database collected in 2007 in our database. There
study, we normalized our inspection scores to reflect were 198 plants for which evidence existed of a
the plant’s performance in relation to the entire indus- quality-related certification not included in the
try in the given year. Appendix A describes this McGraw-Hill data. As these plants apparently have
normalization process in detail. Note that higher been certified at some point, they were deleted from
scores indicate lower levels of process compliance. We the database so they would not be erroneously
employ the variable label “NonComply” for an individ- included as a non-certified plant.
ual inspection (used in the panel analysis for H3) and
“AvgNonComply” for a plant’s average compliance 4.1.3. National Establishment Time Series
over all inspections in a given period (H1–H2). For Database. The NETS database captures annual,
the analysis of H1–H2, we average the NonComply establishment-level data from Dun and Bradstreet
scores for the 0–24 month period after certification to (D&B). It contains longitudinal data on all US estab-
compute AvgNonComply. lishments covered by D&B since 1990. It was the
source of the following variables: industry (i.e., four-
4.1.2. ISO 9000 Certification Database. The digit SIC code), two proxies capturing distinct aspects
McGraw-Hill database of ISO 9000 certifications was of plant size (i.e., number of employees [NumEmp]
used to determine plant certifications up to the year and sales [PltSales]), Public (i.e., 1 = plant part of a
2000. The database, which contains the name, public company, 0 = otherwise), and Standalone (i.e.,
address, and initial certification date for all establish- 1 = the plant is the only establishment of its firm,
ments included, has been used in multiple studies, 0 = otherwise).
including Benner (2009), Lenox and King (2004), and
Naveh and Marcus (2004, 2007). It is widely believed 4.1.4. Merged Data for Analysis. To merge the
to be an accurate representation of ISO 9000 certifica- three above mentioned databases to generate a sam-
tion. We intended to supplement this database, ple, first, the NETS database was manually searched
which only includes certifications through 2000, with to find corresponding plants in the FDA database
other publicly available listings of ISO 9000 certifica- and, where found, the unique DUNS identification
tions. In our search, the most extensive data set number was added to the FDA database. This step
found was the Quality Digest Database. While sup- located 2459 plants that were in both the NETS and
plementing our database with the Quality Digest FDA databases. Then, using a manual search direc-
data, we cross-checked the Quality Digest certifica- ted by the nearest geocoded address, 479 of these
tion information with the McGraw-Hill database and plants were located in the McGraw-Hill ISO 9000
other sources. This process raised concerns about the certification database. These plants appeared in the
reliability of the Quality Digest data (as of 2007) to FDA and NETS databases and were certified per the
the point that these certifications could not be McGraw-Hill database. The candidate set of control
viewed as having sufficient validity for the analysis. non-ISO 9000 facilities for matched comparisons with
Gray, Anand, and Roth: IS0 9000 and Process Compliance
374 Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society
ISO 9000 facilities included all plants that appeared Figure 1 Cumulative ISO 9000 Certifications through 2000 (Naveh
in both the FDA and the NETS databases and had no et al. 2004, p. 1845)
evidence of ISO 9000 certification based on the
McGraw-Hill data or any other source, as discussed
in section 4.1.2.
*The number of matched, non-certified plants is less than the number of ISO-certified plants because we matched “with replacement,” as described in
section 4.3. When matching with replacement, some matched non-certified plants were matched to more than one certified plant.
†
Although perfectly matched, Public and Standalone are slightly different here because we matched with replacement and only count a matched non-
certified plant once, even if it was the best match more than once.
‡
These variables may be time varying; we report them during the year in which the match was made (i.e., the same year that a plant became certified).
§
All years, all inspections are the plant-level average of the NonComply inspection scores. Individual inspection scores range from 0 (best process
compliance) to 3.5 (worst process compliance), as discussed in section 4.1.1.
Table 3 Pearson Bivariate Correlation Coefficients: Plant-Level Variables, Total Sample (n = 552)‡
Figure 2 Average NonComply for Non-Certified, Early-Certified, and Late-Certified Plants, Relative to Time of ISO 9000 Adoption
1.4
1.2
1
0.8 Non-CerƟfied
0.6 Late CerƟfied
0.4 Early CerƟfied
0.2
0
Pre-CerƟficaƟon 0-24 Months Post- 24-48 Months Post- >48 Months Post-
Cert Cert Cert
Gray, Anand, and Roth: IS0 9000 and Process Compliance
376 Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society
Table 4 Process Compliance (AvgNonComply): Early Certified [Early = 1996 or Earlier; H1]*,†
*Not all plants had an inspection during the time period; if not, they were excluded from this analysis.
†
Plant-level non-compliance scores (AvgNonComply) are based on FDA inspection outcomes. Individual inspection scores range from 0 to 3.5, with
higher scores indicating lower process compliance. See section 4.1.1.
performance measure is AvgNonComply, which is the Table 5 Difference-in-Differences, DV = AvgNonComply 0–24 Months
average of all inspections performed in the 0– After Certification (n = 300 Plants, R2 = 2.2%, F(3,
24 month period post-certification. The statistically 296) = 2.18†) [H2]
significant difference (p < 0.01) provides support for Variable Coefficient (SE)
H1 (Table 4). Constant 1.10 (0.11)**
Though not presented, the same analysis with all ISO9K 0.08 (0.16)
plants (both early and late-certified) is also statisti- Early 0.06 (0.17)
cally significant (p < 0.05). The analysis of only late- Early × ISO9K 0.34 (0.24)†
certified plants with their matched non-certified
Control variables not included due to matching.
plants did not show a statistically significant differ- **p < 0.01, †p < 0.10.
ence in process compliance (p > 0.10), indicating that
the significant result including all plants is driven by
the early-certified plants. We further analyze this dif- matched. Nonetheless, we ran the regression with
ference in our tests of H2. control variables, and as expected, it did not affect the
result. This regression has a low R-squared, which is
5.2. Difference between Early and Late-Certified common in difference-in-differences analyses
Plants in Relation to Control Groups (H2) (Wooldridge 2009, p. 455). Figure 3 depicts the “dif-
To test H2, we performed a “difference-in-differ- ference in difference” graphically, comparing the
ences” test in Stata (Wooldridge 2009) to determine mean predicted levels of AvgNonComply of early and
whether a statistically significant difference exists late-certified plants and their matches.
between (1) the difference in AvgNonComply between
early-certified plants and their matched counterparts 5.3. Examination of Process Compliance after
and (2) the difference in AvgNonComply between late- Certification (H3)
certified plants and their matches. Specifically, we ran To test H3a and H3b, we establish whether non-
the following regression: compliance increases or decreases after certification.
As there are multiple post-certification inspections,
AvgNonComply ¼ b0 þ d0 Early þ b1 ISO9K we tested these hypotheses with a panel analysis,
ð1Þ
þ d1 Early ISO9K þ u; using Stata 12’s “xtreg” procedure. For the main
analysis, a random effects regression model was
where ISO9K is defined as = 1 for all certified employed with the normalized inspection score
plants, and 0 for all non-certified matched plants; (NonComply) as the dependent variable and the
and, Early is defined as = 1 if the plant is either an natural log of the number of days since certification
early-certified plant or a non-certified match to an (Ln_DaysSinceISO) as the key independent variable.
early-certified plant, and = 0 if the plant is either a The change in NonComply, as posited by H3a and
late-certified plant or a non-certified match to a late- H3b, is indicated by the sign of the coefficient of
certified plant. Ln_DaysSinceISO. To employ robust standard errors
From Equation (1), a significant and negative d1 and simultaneously account for the fact that some
would indicate that the difference between the early- participating plants belong to the same firm, we
certified plants (Early = 1 and ISO9K = 1) and their employed the cluster option in Stata (Rogers 1993).
matched controls (Early = 1 and ISO9K = 0) is greater Since this analysis only involved certified plants, and
than the difference between late-certified plants therefore, was no longer a matched sample, we
(Early = 0 and ISO9K = 1) and their matched controls included control variables. Table 6 presents results of
(Early = 0 and ISO9K = 0). Table 5 presents the the analysis for all certified plants and the subset of
results of this regression, which shows moderate early-certified plants. The empirical results support
significance (p = 0.09). No control variables are H3b, indicating that time since certification has a
included in the regression because the plants are significant detrimental effect on process compliance
Gray, Anand, and Roth: IS0 9000 and Process Compliance
Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society 377
Figure 3 Comparison of Early and Late-Certified Plants (AvgNonCom- to different early/late classifications, except for the
ply, 0–24 months after certification) vs. Their Non-Certified, earliest cutoff dates. Table 7 shows the results of this
Paired Matches (Match), Based on Difference-in-Differences
Regression in Table 5
supplemental analysis.
We also performed a supplemental analysis related
1.4
to the post-certification period. For the main analysis
1.2 presented in sections 3.1–3.2, we used AvgNonComply
1
as our dependent variable in the 0–24 month post-
certification period. Here, to understand the dynamic
0.8
ISO9K
differences between certified plants and matched
0.6 Match
plants, the results are reported in different, discrete
time buckets. Specifically, we examined AvgNonCom-
0.4
plyt for the following post-certification time buckets:
0.2 0–12 months, 0–18 months, 0–36 months, and all
0
inspections post certification (see Table 8). Consider-
Early CerƟfied Plants Late CerƟfied Plants ing H1, for both the 0–18 (p < 0.05) month and 0–36
(p = 0.051) month periods, the early adopters signifi-
cantly outperformed their matched plants. There was
for early-certified plants (p = 0.03) and a moderate no such difference for 0–12 months or when all
effect for all certified plants (p = 0.06). Though not inspections post certification are included. Consider-
presented, there is no significant effect (p > 0.10) ing H2, moderate significance (p < 0.10) was main-
when only considering the late-certified plants. tained in two of the four supplemental analyses. The
0–12 month time period was not significant, nor was
5.4. Supplemental Analyses the 0–36 month time period (p = 0.13). Taken
First, we tested the robustness of the results to the dis- together, H1 and H2 tend not to be supported when
tinction between “early” and “late.” The classification the post-certification period in which AvgNonComplyt
for all the main analyses in the study is that the early- is calculated is too short, both hindering sample size
certified plants were certified before January 1, 1997, and possibly implying a period of adjustment to the
and late-certified plants were certified on or after that new system (Pil and MacDuffie 1996, Sterman et al.
date. Our FDA inspection data began in 1994 and our 1997). In the longer post-certification periods, results
ISO 9000 certification data ended in 2000. Figure 1 become less consistent as the effects of decay begin to
indicates this range of dates should cover any reason- be realized.
able early/late cutoff for ISO 9000 diffusion. We The next supplemental analyses consider pre-cer-
performed five supplementary analyses for each tification performance in analyzing H1. Our sparsely
hypothesis, testing the early/late cutoff dates before populated process compliance measure, combined
January 1, 1995, 1996, 1998, 1999, and 2000. H1, H2, with a lack of process compliance data prior to
and the analysis of H3 for only early-certified plants 1994, prevented matching based on pre-certification
are all robust (p < 0.10) to the later cutoff dates (Janu- process compliance. Nonetheless, there is no sig-
ary 1, 1998, 1999, and 2000). H1 and H3 are also robust nificant difference (p > 0.50) in pre-certification
(p < 0.10) to a cutoff of January 1, 1996, but H2 is not. AvgNonComply between certified plants and their
None are robust to the January 1, 1995 cutoff, but by matches. We incorporate pre-certification by
this classification there were only 23 early-certified employing a “difference-in-differences” methodol-
plants. Taken together, our results are mostly robust ogy (Wooldridge 2009), similar to our analyses of
Note: NumEmp was removed due to high correlation with PltSales; H3 result is unaffected if included.
**p < 0.01 *p < 0.05, †p < 0.10.
Gray, Anand, and Roth: IS0 9000 and Process Compliance
378 Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society
Table 7 Results of Supplemental Analysis Related to Early/Late Cutoff post certification (indicating that, overall, certification
#Early #Late
helps, when controlling for time) and (2) decay is
Early adopter cert cert #Non-cert H1 H2 H3‡ more substantial after than before certification.
†
Before January 1, 2000 139 14 147 * *
Before January 1, 1999 124 29 147 * ** * 6. Discussion
† †
Before January 1, 1998 92 61 147 **
Before January 1, 1997 68 85 147 ** †
*
Supporting H1, our empirical results indicate that
†
early-certified plants had consistently better process
Before January 1, 1996 47 106 147 n.s. * compliance than their matched non-certified plants
Before January 1, 1995 23 130 147 n.s. n.s. n.s.
(p < 0.01). Furthermore, in support of H2, moderate
Grey-shaded cells are used in main analysis. evidence suggests that early-certified plants outper-
**p < 0.01, *p < 0.05, †p < 0.10 H1 and H2 one-tailed tests; H3 two- formed their matches more than late-certified plants
tailed. did (p < 0.10). Combined with insights from the ISO
‡
H3 test is for early-certified plants only; analogous to right-side column
of Table 6. Of course, test of all plants (left-side column of Table 6) is
9000 adoption literature (Anderson et al. 1999,
unchanged by early/late designation. Corbett 2006), our results suggest any industry-wide
learning advantages related to process compliance for
late-certified plants (Ansari et al. 2010, Delmas and
H2, but examining a different “difference.” Here, Montiel 2008) were outweighed by the fact that early-
the objective was to determine if there is a statisti- certified plants are more likely to have been internally
cally significant difference between (1) the difference motivated to obtain certification.
in AvgNonComply before and after certification for Further, using longitudinal panel data, we found
certified plants and (2) the difference in AvgNon- decay in process compliance after certification (H3b),
Comply before and after their match’s certification especially among early-certified plants. This finding
for non-certified plants. This analysis casts certifica- is contrary to any expectations that changes made to
tion as a treatment and assesses the effect of this practices at the time of certification would prompt
treatment relative to an untreated matched plant. continual improvement of certification-related perfor-
We operationalized before-certification as the aver- mance over time (Ansari et al. 2010, Sun 1999). In
age of all inspections before certification and after aggregate, deterioration in process compliance
certification as in H1 and H2. Results of this analysis appears to dominate any experiential and deliberate
indicate moderate significance (p < 0.10). learning effects in these plants over time. Our empiri-
H3, which found support for deterioration in com- cal results demonstrate that the managerial difficulty
pliance over time, was tested considering only time and challenges of sustaining the operationally ori-
after certification. This approach does not assess ented gains of ISO 9000 certification over time are
whether certification affects the rate of decay. To widespread.
study this, we performed a supplemental analysis We now consider the placement of our research in
with the new independent variables of Days (days the context of existing literature that has examined
since December 31, 1993), Pre-Cert (whether the the effects of ISO 9000 on performance dimensions
inspection occurred before or after certification), and related to process compliance. As previously dis-
the interaction Days*Pre-Cert. As the start date is arbi- cussed, our study design (longitudinal, plant level,
trary from the standpoint of decay, it did not make matched sample, and focused on plant-level process
sense to take the log of Days. As expected, the coeffi- compliance) is unique to this literature (Boiral 2012).
cient for Pre-Cert is positive and significant, and the Our results indicate that previous cross-sectional
interaction of the two is negative and significant. studies, even when they consider time since certifica-
Taken together, these results indicate that (1) Non- tion, may face conflicting time-related effects leading
Comply, on average, is higher pre-certification than to inconsistent results. Specifically, cross-sectional
Post-certification time period #Early cert #Early matches H1 #Late cert #Late matches n‡ H2
0–12 months 41 37 n.s. 56 53 187 n.s.
†
0–18 months 57 49 * 71 69 246
†
0–24 months 68 66 ** 85 81 300
†
0–36 months 79 80 114 102 375 n.s.
†
All post-certification inspections 107 109 n.s. 191 176 583
‡
n = sample size for difference-in-difference regression to test H2. It is the sum of all plants with data during the time period studied.
Gray, Anand, and Roth: IS0 9000 and Process Compliance
Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society 379
studies are unable to disentangle the countervailing Another limitation is the coarse and noisy depen-
positive performance effects of early certification with dent variable assessed infrequently for each plant. We
the increased time to deteriorate since the earlier certi- assert that our measure of process compliance is the
fication. For example, while two previous studies best widely available measure of the performance
included the year of certification as an independent dimension with which ISO 9000 is concerned. How-
variable, they did not find it significantly related to ever, the FDA may sometimes examine more than just
performance (Nair and Prajogo 2009, Prajogo 2011). compliance with procedures or focus only on part of a
Their results may be attributed to any performance facility. However, this “noise” should not bias our
benefits related to early certification decaying over results. A practical limitation discussed earlier is that
time post-certification. Two other studies explicitly we were only able to obtain reliable data on the ISO
examined temporal issues and found significant 9000:1994 certifications because conversations with
results. In a cross-sectional study, Naveh and Marcus registrars and database owners indicated that the
(2005) found that time since certification related to databases became unreliable soon after the 2000 stan-
better performance, which is opposite to our finding dard was created. Further, our capacity to categorize
related to H3b, but consistent with H1 and H2. As our plants as being component manufacturers and
their performance measurement was designed to final assemblers in the medical devices industry was
capture improvement in no more than five years after also limited by the data. This information could have
certification, we infer that the decay effects in that allowed us to study whether the certified and non-
time frame (0–5 years since certification) were likely certified plants differed based on the positions they
dominated by the benefits of early certification. In occupied in the supply chain.
contrast, Karapetrovic et al. (2010) examined three This research has led to several salient opportuni-
different samples at four-year intervals and found ties for future research. If reliable data on certifica-
declining benefits in each subsequent sample. As tions of plants from later ISO 9000 standards (2000
Karapetrovic et al. (2010) examined independent and 2008) could be found, it would be worthwhile to
samples, and did not specifically include time since determine whether these updated standards have
certification, their observed decline in performance succeeded in enabling long-term improvements in
is consistent with our two findings about late compliance. It would also be interesting to examine
certification and time since certification. Specifically, longitudinally the long-term implications of other
each later sample would contain more late certifica- management standard certifications on their primary
tions as well as, presumably, greater average time intended outcomes. Another potentially fruitful ave-
since certification. nue would be to combine more detailed plant-level
characteristics, such as factors related to the imple-
7. Limitations, Future Research, and mentation process and/or managerial efforts post cer-
tification, and tie these to process compliance and its
Conclusion sustainability.
As with all empirical research, this study has limita- In the context of medical device manufacturing,
tions, some of which present opportunities for future our analysis of longitudinal data provides an assess-
research. For example, the matching algorithm only ment of the effectiveness of ISO 9000 on quality-
ensures that the pairs of plants are similar or identical related process compliance. While the results are not
in publicly observable characteristics (and not encouraging for practitioners, they do provide
less-observable factors such as internal motivation, insights about the reliance on ISO 9000 certification.
top-management leadership, and competence of Our research shows plants that obtained ISO 9000
employees). It is possible that plants seeking certifica- certification, on average, outperformed similar plants
tion and those not seeking certification differ in some that did not under two conditions. First, the plants
unobservable characteristics that affect process com- were early seekers of certification. Second, the plants
pliance. There are two reasons to believe this is not were recently certified, as their heightened level of
the case in our study. First, the certified and non-certi- process compliance appears to be rather short-lived,
fied plants performed similarly in process compliance on average. This research provides insights that have
before certification. Second, two phenomena were implications for buyers’ sourcing strategies. Buyers
observed, especially among early adopters: an must be aware of the conditions under which certifi-
improvement in process compliance performance cation serves as a reliable “signal” of compliance
(relative to matched plants) and then a decay in such performance in their potential suppliers. Moreover,
performance post certification. It is difficult to con- to ensure compliance, buyers may need to incur
struct an unobserved managerial characteristic that monitoring costs. Executives considering plant certi-
explains both improvement and decay unrelated to fication must be aware any improvement tends to
certification. decay without renewal interventions. Thus, it is
Gray, Anand, and Roth: IS0 9000 and Process Compliance
380 Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society
important that operations managers maintain vigi- scoring system from Gray et al. (2011). Table A1 also
lance in their focus, resources, and systems to sustain provides subsequent adjustments made to normalize
(and possibly grow) process compliance in their the scores. Note that the grand mean of the entire
plants. To the literature, our research design pro- medical devices industry is 1.1014. The normalization
vides insights and empirical evidence that help factor, depicted in the last column of Table A1, is
explain the inconsistent results from cross-sectional computed as the grand mean [1.1014] divided by the
studies of ISO 9000 and provides a springboard for average inspection outcome of a given year. Thus,
future studies on management standard certification multiplying each observed inspection outcome in our
in other industries and contexts. sample by the given year’s normalization factor stan-
dardizes year-over-year inspections to the average
level of stringency in the FDA inspections of the given
Acknowledgments year. These normalized scores are the only scores
The authors would like to acknowledge the Fisher College used in all analyses and are labeled NonComply for a
of Business at The Ohio State University for providing fund- single inspection and AvgNonComply for the average
ing to enable part of the data collection effort. We are very of a set of inspections for a given time period.
grateful to Brett Massimino for his extensive work as a We now discuss robustness of our main results to
research assistant on this manuscript; and Sriram Venkatar- the use of non-normalized scores. The results for H1
aman at Clemson University for his research assistance. We
and H2 are substantively the same, whether we use
are also in debt to Mary Benner and Dirk Dusharme for
their assistance as discussed in section 4.1.2. We thank Mike
normalized or non-normalized FDA inspection out-
Toffel for providing a careful friendly read of an earlier come scores, as would be expected since both analyses
draft of this study. Finally, we thank several student work- compare approximately contemporaneous inspection
ers at Ohio State (Amber Hurley, Eric Nash, Xiao Yao, and outcomes of plants and their matched control plants.
Erika Randolph) for manual data entry and auditing. H3b was not robust (p = 0.12) using the AvgNonCom-
ply_nonadj dependent variable. It is not surprising that
Appendix: Normalization of NonComply evidence of decay was difficult to detect in the face of
the observed industry-wide downward trend in
Scores adverse inspection outcomes shown in Table A1,
Careful observation of the FDA inspection results especially when we consider the hypothesized rela-
indicate they became “better” (i.e., lower levels of tionship in H3b tracks decay from the date of certifica-
detected process non-compliance) over time, a finding tion to early 2007, possibly over 10 or more years.
also observed by Anand et al. (2012) in a pharmaceu-
tical industry study. In Table A1 we show the average
NonComply_nonadj, which is the average of all
non-normalized inspections in that year based on the References
Abadie, A., G. W. Imbens. 2006. Large sample properties of
matching estimators for average treatment effects. Econometrica
Table A1 Average Inspection Compliance and Normalization Factors 74(1): 235–267.
by Year Abrahamson, E. 1991. Managerial fads and fashions: The diffusion
and rejection of innovations. Acad. Manag. Rev. 16(3): 586–612.
Average of Normalization
* Adler, P. S., K. B. Clark. 1991. Behind the learning curve: A sketch
Year NonComply_nonadj factor† of the learning process. Manage. Sci. 37(3): 267–281.
1994 1.338 0.823 Anand, G., J. Gray, E. Siemsen. 2012. Decay, shock, and renewal:
1995 1.196 0.921 Operational routines and process entropy in the pharmaceuti-
1996 1.026 1.073 cal industry. Organ. Sci. 23(6): 1700–1716.
1997 1.117 0.986 Anderson, S. W., J. D. Daly, M. F. Johnson. 1999. Why firms seek
1998 1.191 0.925 ISO 9000 certification: Regulatory compliance or competitive
1999 1.174 0.938 advantage? Prod. Oper. Manag. 8(1): 28–43.
2000 1.197 0.920
Ansari, S. M., P. C. Fiss, E. J. Zajac. 2010. Made to fit: How prac-
2001 1.112 0.990 tices vary as they diffuse. Acad. Manag. Rev. 35(1): 67–92.
2002 1.068 1.032
Argyris, C., D. Sch€on. 1978. Organizational Learning. A Theory of
2003 0.968 1.138
Action Perspective. Addison Wesley, Reading, MA.
2004 1.023 1.076
2005 1.020 1.080 Barber, B. M., J. D. Lyon. 1996. Detecting abnormal operating per-
2006 0.999 1.103 formance: The empirical power and specification of test statis-
2007 1.010 1.090 tics. J. Financ. Econ. 41(3): 359–399.
Barratt, M., T. Choi. 2007. Mandated RFID and institutional
*NonComply_nonadj in a given year is based on the complete medical responses: Cases of decentralized business units. Prod. Oper.
device inspection data set (n = 25,373 inspections), not just plants used Manag. 16(5): 569–585.
in this study’s analysis. Benner, M. J. 2009. Dynamic or static capabilities? Process man-
†
Each inspection score in the given year is multiplied by this number to agement practices and response to technological change. J.
create the AvgNonComply used in all analyses. Prod. Innov. Manage 26(5): 473–486.
Gray, Anand, and Roth: IS0 9000 and Process Compliance
Production and Operations Management 24(3), pp. 369–382, © 2014 Production and Operations Management Society 381
The performance debate revisited. J. Oper. Manag. 27(6): Simon, H. 1957. A behavioral model of rational choice. H.A.
495–511. Simon, ed. Models of Man, Social and Rational Mathematical
Nair, A., D. Prajogo. 2009. Internalisation of ISO 9000 standards: Essays on Rational Human Behavior in a Social Setting. Wiley,
The antecedent role of functionalist and institutionalist driv- New York, NY, 241–260.
ers and performance implications. Int. J. Prod. Res. 47(16): Singels, J., G. Ruel, H. van de Water. 2001. ISO 9000 series – Certifi-
4545–4568. cation and performance. Int. J. Qual. Reliab. Manag. 18(1): 62–75.
Naveh, E., M. Erez. 2004. Innovation and attention to detail in Singh, P. J. 2008. Empirical assessment of ISO 9000 related man-
the quality improvement paradigm. Manage. Sci. 50(11): agement practices and performance relationships. Int. J. Prod.
1576–1586. Econ. 113(1): 40–59.
Naveh, E., A. A. Marcus. 2004. When does the ISO 9000 quality assur- Singh, P. J., D. Power, S. C. Chuong. 2011. A resource dependence
ance standard lead to performance improvement? Assimilation theory perspective of ISO 9000 in managing organizational
and going beyond. IEEE Trans. Eng. Manage. 51(3): 352–363. environment. J. Oper. Manag. 29(1): 49–64.
Naveh, E., A. Marcus. 2005. Achieving competitive advantage Sterman, J. D., N. P. Repenning, F. Kofman. 1997. Unanticipated
through implementing a replicable management standard: side effects of successful quality programs: Exploring a
Installing and using ISO 9000. J. Oper. Manag. 24(1): 1–26. paradox of organizational improvement. Manage. Sci. 43(4):
Naveh, E., A. Marcus. 2007. Financial performance, ISO 9000 stan- 503–521.
dard and safe driving practices effects on accident rate in the Sun, H. 1999. Diffusion and contribution of total quality manage-
U.S. motor carrier industry. Accid. Anal. Prev. 39(4): 731–742. ment: An empirical study in Norway. Total Qual. Manag. 10(6):
Naveh, E., A. Marcus, H. K. Moon. 2004. Implementing ISO 9000: 901–914.
Performance improvement by first or second movers. Int. J. Terlaak, A., A. A. King. 2006. The effect of certification with the
Prod. Res. 42(9): 1843–1863. ISO 9000 quality management standard: A signaling
Nelson, R. R., S. G. Winter. 1982. An Evolutionary Theory of Eco- approach. J. Econ. Behav. Organ. 60(4): 579–602.
nomic Change. Harvard University Press, Cambridge, MA. Terziovski, M., D. Samson, D. Dow. 1997. The business value of
Pil, F. K., J. P. MacDuffie. 1996. The adoption of high-involvement quality management systems certification. Evidence from
work practices. Industrial Relations 35(3): 423–455. Australia and New Zealand. J. Oper. Manag. 15(1): 1–18.
Podsakoff, P. M., S. B. MacKenzie, J. Y. Lee, N. P. Podsakoff. Terziovski, M., D. Power, A. S. Sohal. 2003. The longitudinal
2003. Common method biases in behavioral research: A criti- effects of the ISO 9000 certification process on business per-
cal review of the literature and recommended remedies. J. formance. Eur. J. Oper. Res. 146(3): 580–595.
Appl. Psychol. 88(5): 879–903. Uzumeri, M. V. 1997. ISO 9000 and other metastandards: Princi-
Prajogo, D. I. 2011. The roles of firms’ motives in affecting the ples for management practice? Acad. Manage. Executive 11(1):
outcomes of ISO 9000 adoption. Int. J Oper. Prod. Manag. 21–36.
31(1): 78–100. Wayhan, V. B., E. T. Kirche, B. M. Khumawala. 2002. ISO 9000
Rogers, W. H. 1993. Regression standard errors in clustered sam- certification: The financial performance implications. Total
ples. Stata Tech. Bull. 13: 19–23. Qual. Manag. 13(2): 217–231.
Rosenbaum, P. R., D. B. Rubin. 1983. The central role of the Westphal, J. D., E. J. Zajac. 1994. Substance and symbolism in
propensity score in observational studies for causal effects. CEOs’ long-term incentive plans. Adm. Sci. Q. 39(3): 367–390.
Biometrika 70(1): 41–55. Westphal, J. D., R. Gulati, S. M. Shortell. 1997. Customization of
Schroeder, R. G., K. Linderman, D. Zhang. 2005. Evolution of conformity? An institutional and network perspective on the
quality: First fifty issues of Production and Operations Man- content and consequences of TQM adoption. Adm. Sci. Q.
agement. Prod. Oper. Manag. 14(4): 468–481. 42(2): 366–394.
Sharma, D. S. 2005. The association between ISO 9000 certification Wooldridge, J. M. 2009. Introductory Econometrics: A Modern
and financial performance. Int. J. Account. 40(2): 151–172. Approach, 4th edn. South-Western, Mason, OH.
Siemsen, E., A. Roth, P. Oliveira. 2010. Common method bias in Yeung, A. C. L., C. K. Y. Lo, T. C. E. Cheng. 2011. Behind the iron
regression models with curvilinear, quadratic and interaction cage: An institutional perspective on ISO 9000 adoption and
effects. Organ. Res. Methods 13(3): 456–476. CEO compensation. Organ. Sci. 22(6): 1600–1612.
Simmons, B. L., M. A. White. 1999. The relationship between ISO Zhang, G. P., Y. Xia. 2013. Does quality still pay? A reexamination
9000 and business performance: Does registration really of the relationship between effective quality management and
matter? J. Manag. Issues 11(3): 330–343. firm performance. Prod. Oper. Manag. 22(1): 120–136.