CAP Laboratory Improvement Programs
Using Outlier Events to Monitor Test Turnaround Time
A College of American Pathologists Q-Probes Study in 496 Laboratories
Steven J. Steindel, PhD; David A. Novis, MD
● Objectives.—To determine the causes of excessive test
turnaround time (TAT) and to identify methods of improve-
ment by studying reasons for those tests reported in excess
of 70 minutes from the time the test was ordered (ie, out-
liers).
Design.—Self-directed data-gathering of stat outlier TAT
events from intensive care units and emergency depart-
ments, with descriptive parameters associated with each
event and additional descriptive parameters associated
with the participant.
Participants.—Laboratories enrolled in the 1996 College
of American Pathologists Q-Probes program.
Main Outcome Measures.—Components associated with
outlier TAT events and outlier TAT rates.
Results.—Four hundred ninety-six hospital laboratories
returned data on 218 551 stat tests, of which 10.6% had
TATs in excess of 70 minutes. Ten percent of stat emergen-
cy department tests and 14.7% of stat intensive care unit
tests were outliers. Major areas in which delays occurred
were test ordering, 29.9%; within-laboratory (analytic)
phase, 28.2%; collection of the specimen, 27.4%; postan-
alytic phase, 1.9%; and undetermined, 12.5%. The type of
O ne of the most visible and talked about areas of lab-
oratory service is how fast a test result is returned
to the caregiver. A recent review of laboratory test turn-
around times (TATs)1 indicated that the most common way
to monitor test TAT is by recording some starting point
(test order or collection time) and ending point (test result
reporting or verification time), and then analyzing the dif-
ference between the two. Many laboratorians also analyze
the multiple possible intervals of test TAT, such as test
order to specimen collection time, collection to laboratory
specimen receipt time, and receipt time to reporting of test
results or verification time. Analyzing time intervals helps
From the Centers for Disease Control and Prevention, Public Health
Practice Program Office, Division of Laboratory Systems, Laboratory
Performance Assessment Branch, Chamblee, Ga (Dr Steindel); and the
Reprints: Steven J. Steindel, PhD, Centers for Disease Control and
Prevention, Public Health Practice Program Office, Division of Labo-
ratory Systems, Laboratory Performance Assessment Branch, MS G-23,
4770 Buford Hwy, Chamblee, GA 30341.
Arch Pathol Lab Med—Vol 123, July 1999
test performed was a significant factor and was indepen-
dent of location: Chemistry–Multiple Test appeared most
frequently (;40%), followed closely by Hematology–Com-
plete Blood Count (;20%) and Chemistry–Single Test
(;18%). Factors of outlier TAT components for intensive
care unit specimens were identified using statistical mod-
eling and included hour of day, type of health care per-
sonnel collecting specimen, performing the test in a stat
laboratory, and reason for delay. Outlier rates were not
associated with any identified factors. The practice param-
eters of laboratories with outlier rates in the lowest 10th
percentile significantly differed from those with rates in the
top 10th percentile in test request computerization, report
methods, and ordering methods.
Conclusions.—We observed that outlier analysis yields
new information, such as type of test and reason for delay,
concerning test delays when compared with TAT determi-
nation alone. Laboratories experiencing stat test TAT prob-
lems should use this tool as an adjunct to routine TAT mon-
itoring for identifying unique causes of delay.
(Arch Pathol Lab Med. 1999;123:607–614)
locate areas in which delays may occur in the total testing
process.
The College of American Pathologists Q-Probes pro-
gram has studied various aspects of test TATs and has
related specific practice parameters with improving TATs.
These studies have included evaluating intralaboratory
TATs for cerebrospinal fluid analytes,2 routine chemistry
and hematology tests,3 and stat tests performed in emer-
gency departments (EDs) in large4,5 and small6 hospitals.
Our previous studies on ED TATs7 indicated shorter TATs
correlated with decreased transportation times and with
laboratory control of specimen collection. Specimen prep-
aration, when required, prolonged TATs. Overall, moni-
toring stat test TAT appeared to have no impact, but our
comparison of 1990 and 1993 ED TATs indicated the im-
portance of active (ie, results are continually reviewed) as
opposed to passive (ie, results are obtained but not acted
upon) monitoring to control and improve TATs.5 Using
computer systems and stat laboratories did not result in
decreased TATs. Furthermore, we observed that TATs var-
ied for different analytes.2,7
The inability to use monitoring as a means to improve
TATs may be an artifact of how the data are analyzed.
Turnaround Time Outliers—Steindel & Novis 607
Valenstein and Emancipator8 noted that the distribution of
TAT data is non-Gaussian and should be described using
nonparametric statistics, such as median, frequency dis-
tributions, and percentile rankings. These values are sel-
dom available in the simple computer programs provided
with most laboratory information systems. The skewed
nature of TAT data distribution may result in specimens
with excessively long TATs (outliers). We hypothesized
that analyzing outlier TAT events would help determine
the causes of delays in the TAT process for stat tests from
EDs and intensive care units (ICUs).
Events that occur at extremes are thought to have
unique properties. Service companies, such as those pro-
viding fast food or automobile rental, strive to deliver ser-
vice that is consistent, reliable, and predictable. To refine
their performance, they investigate incidents which may
reveal idiosyncratic processes due to random events or
systematic and repetitive causes. In our service area of
medical laboratory testing, detecting and understanding
these outlier events may improve laboratory performance
and user perceptions of laboratory service.
Outlier analysis is commonly used in industrial quality
control as a method of determining reasons for system
failure.9 Outliers are an acceptable part of the industrial
process and are even referenced in case law with regard
to generic drug manufacturing.10 Other industrial uses ap-
pear, such as spurring improvements in manufacturing
processes in response to an accident; these can be called
catastrophic outliers. Disasters such as those at Union Car-
bide’s plant in Bhopal, India (1984), the Phillip’s 66 chem-
ical complex in Houston, Tex (1989), and the Arco Chem-
ical plant in Channelview, Tex (1990), were national news
stories. They resulted in subsequent process improve-
ments designed to prevent such outliers from occurring
again.11 Studying these events has resulted in a decrease
in the number of industrial accidents during the last de-
cade. Analogous use of outlier analysis by members of the
service industry is proprietary and difficult to document.
Excessive TATs are minor events compared with the de-
struction of a major chemical plant, but the same analyt-
ical techniques apply.
Several previous Q-Probes studies have shown that pa-
tient and physician satisfaction levels for TATs are depen-
dent only on exceeding certain limits. One study found
that patient satisfaction with phlebotomy services deteri-
orated only after waiting times exceeded 30 minutes.12 An-
other showed that surgeons’ satisfaction with biopsy re-
ports deteriorated after the report TAT exceeded 4 days.13
Events exceeding established or tolerable limits can be
viewed as outliers and can be investigated in a fashion
similar to that described here. Both the solutions and lim-
its can be viewed as unique and subject to the general
health care quality assurance practices as they exist within
institutions.
We instituted this study to see if detailed information
from specimens showing excessive TATs would reveal
new insights into the duration of the testing process.
Based on previous Q-Probes studies, we expected about
10% of stat tests to have TATs in excess of 70 minutes from
the time the test was ordered; we arbitrarily defined tests
in excess of 70 minutes to be outliers.4 We wanted to de-
termine if the practice variables and testing factors studied
in previous Q-Probes studies of TATs appeared more fre-
quently in outlier TAT events. We also investigated the role
of ordering location and test type in testing delays, since
608 Arch Pathol Lab Med—Vol 123, July 1999
Table 1. Study Test Types
Chemistry
Single test
.1 test, including panels/profiles
Hematology*
CBC or any CBC component
Other than CBC
Coagulation
Single test
.1 test
Immunology/serology
Single test
.1 test
*CBC indicates complete blood count.
Table 2. Health Care Worker Collecting Stat
Specimen
Phlebotomy/laboratory personnel
Attending physician/house officer
Medical student
Nursing staff, nurse practitioner, physician’s assistant
Table 3. Specific Reasons for Delay in Ordering of
Test to Collection of Specimen
Emergency department/intensive care unit delay in transmitting
test order to laboratory or to phlebotomist
Laboratory delay in transmitting test order to phlebotomist
Phlebotomy delay
Patient: patient not available for specimen collection
Documentation/identification: paperwork incomplete,
patient identification incomplete, etc.
Personnel: phlebotomist not immediately available to draw,
other patients to draw, laboratory staff shortage, etc.
these factors were not well studied previously. A prelim-
inary report of these findings was sent to participants as
a Q-Probes study critique.14
MATERIALS AND METHODS
Participants in the 1996 College of American Pathologists Q-
Probes program were provided with forms to record the follow-
ing information on up to 60 stat test specimens for each day of
the week (total of up to 420 stat tests): the testing date; test origin
location (ED or ICU); whether the test was performed in a des-
ignated stat laboratory; and the time of day that tests were or-
dered, specimens were collected and received in the laboratory,
and test results were verified. These forms were used as internal
log sheets. All stat tests were eligible for the study with the ex-
ception of those for urinalysis, arterial blood gas determinations,
and tests on body fluids other than blood. Tests were also ex-
cluded from the study for various procedural reasons, including
point-of-care tests, tests collected by the patient, timed or pro-
tocol tests, tests having verification times that normally exceed 1
hour, tests ordered stat but not routinely available on a stat basis
in the participant’s institution, add-on or change-in-priority tests,
canceled tests, and any test with associated amended or corrected
reports. All 60 tests need not have come from the same date, but
must have come from the same day of the week. Participants
were instructed to provide only those tests with order-to-verifi-
cation times exceeding 70 minutes.
Participants completed a second form on which they recorded
and submitted the above and the following information on each
of the up to 60 specimens: day of week, type of test (Table 1),
category of health care worker collecting the specimen (Table 2),
Turnaround Time Outliers—Steindel & Novis
 Table 4. Specific Reasons for Delay in the Collection
of Specimen to Receipt in Laboratory
Phlebotomy delay–technical
Difficult venipuncture
Specimen transit delay
Emergency department/intensive care unit to laboratory–per-
sonnel: specimen transport personnel unavailable, staff
shortage, etc
Emergency department/intensive care unit to laboratory–mechan-
ical: tube transport system malfunction, etc.
Clerical delay
Specimen log-in data entry, computer malfunction/down, etc
Table 5. Specific Reasons for Delay in the Receipt of
Specimen, Analytic Phase
Intralaboratory communication delay, specimen transport delay, etc
Technical delays
Specimen processing/preparation
Difficulty with instruments (eg, malfunction, out of service),
methods, standards, controls, etc
Specimen delays
Abnormal results and specimen integrity requiring additional
follow-up, verification, retest required by laboratory policy, etc
Personnel delays
Staff shortage, staff busy with other tests/duties
Laboratory accident
Table 6. Specific Reasons for Delay in the Receipt of
Specimen, Postanalytic Phase
Clerical delays
Technical: data entry, equipment malfunction/down (eg,
computer, telephone, tube, fax), etc
Personnel: staff shortage, staff busy with duties, etc
Procedural/protocol: supervisory approval needed for
release, waiting for other results before reporting
Emergency department/intensive care unit: problems in
reaching clinician, nurse, secretary, etc, with the report result
(eg, telephone/fax line busy, difficulties identifying to whom
or where report should be sent)
and their assessment of the single most important reason for the
testing delay. Participants selected these reasons from a supplied
list of specific reasons within broad categories of delays. Each
category also included a provision to select Other or Undeter-
mined. For guidance in helping to locate the causes of testing
delay, it was suggested that reasonable component TATs (based
on prior Q-Probes studies4,5) were 15 minutes for order-to-collec-
tion and collection-to-receipt times, and 30 minutes for receipt-
to-verification interval. Specimens with component TATs exceed-
ing these established times suggested delays in that specific area.
The specific reasons for delay within each broad category are
found in Tables 3 through 6. In addition, an overall grouping of
Other or Undetermined was also calculated.
Participants provided responses to 12 questions concerning
their general practice parameters. These questions covered test
ordering, specimen transport, specimen collection, result report-
ing, computerization, and TAT monitoring. These multiple-choice
questions were asked independently for the ED and the ICU. Par-
ticipants were also asked to record the time of shift change and
stat test volume during the study period. Responses to these
questions allowed identification of effects of various practice pa-
rameters on outlier TAT rates.
Data received were analyzed using SAS (SAS Inc, Cary, NC).
Arch Pathol Lab Med—Vol 123, July 1999
Descriptive statistics, including mean, median, range, standard
deviation, and frequency, enumerated the TAT process. Modeling
to find significant parameters used multivariate analysis of vari-
ance (MANOVA).15 For modeling, the specimen collection-to-lab-
oratory receipt, receipt-to-result verification, collection-to-verifi-
cation, test order-to-collection, and order-to-verification times
(overall TATs) were the dependant variables. We selected the fol-
lowing as independent variables: the hour of order, collection,
receipt, and verification; personnel collecting the specimen; the
day of the week; if the test was performed in a stat laboratory;
the type of test; and the broad delay classification. Models were
considered adequate to describe the data if the probability of fit
was significant (P # .05) and the correlation coefficient was ad-
equate (r 2 . 0.2). Since the chosen practice parameters may not
have fully described the process, they were considered significant
when P # .10, so that most of the correlating parameters could
be identified.16
Outlier rates for both the ED and ICU were calculated by di-
viding the number of submitted outliers by the reported total
number of stat tests. Identification of statistically significant prac-
tice parameters was performed by comparing those of the group
having outlier rates in the lowest 10th percentile versus those
having the highest 10th percentile using MANOVA in both the
ICU and ED separately. The model used an indicator to show
whether the participant’s percentage of outliers was in either the
lowest or highest percentile rank as the dependent variable. As
the dependent variable in our model, an indicator represented
whether the participant’s outlier rate was in the lowest or highest
percentile. The independent variables consisted of the partici-
pant’s responses to their various practice parameters. As stated,
practice parameters were considered significant if P # .10.
RESULTS
Four hundred ninety-six hospital laboratories provided
data from 218 551 stat tests (Table 7). Of the 466 partici-
pants who provided information, 30.3% were in teaching
hospitals. Of the 468 who provided their location, 57.7%
were in urban, 21.2% were in suburban, and 19.7% were
in rural areas. All participants provided bedsize infor-
mation, and most (30.5%) were located in hospitals with
between 151 and 300 beds; hospitals with 301 to 450 beds
constituted the next largest group (26.0%). Of the remain-
der, 22.0% had more than 450 beds, and 21.8% had be-
tween 1 and 150 beds.
Figure 1 shows the percent distribution of the broad
delay categories. The leading cause (37.8%) for delay in
the order-to-collection phase (Table 3) was attributed to
lack of staff to perform the phlebotomy. Specimen trans-
port problems accounted for 56.3% (51.4% due to person-
nel; 4.9% due to breakdowns in mechanical transport sys-
tems) of the delays in the collection-to-receipt phase (Table
4). The largest single cause of delays in the analytic phase
(Table 5) was attributed to personnel shortages (33.7%),
but 38.8% of the delays were caused by the 3 technical
reasons listed in this category (specimen processing/prep-
aration, difficulty with instrument, and specimen-associ-
ated delays). Delays in the postanalytic phase (Table 6)
were few, with the leading cause (54.8%) attributed to the
category of clerical delays–technical. Undetermined was a
major reason for delays both overall (12.5%) and within
each broad category (ordering of test, 24.7%; collection of
specimen, 21.2%; receipt of specimen–analytic phase,
18.3%; and receipt of specimen–postanalytic phase, 9.4%).
Distribution of practice parameters determined for out-
lier specimens by ordering location is shown in Table 8,
and Table 9 shows the distribution of the same practice
parameters by broad delay categories. The distribution of
test type by broad delay category (Table 9) was statistically
Turnaround Time Outliers—Steindel & Novis 609
Figure 1. Percentage distribution of broad
delay categories.
Table 7. Overall Stat Test Turnaround Time (TAT) Outlier Rates by Patient Care Location
No. of
Location Institutions
Emergency department 484
Intensive care unit 380
Total 496
significant (x2, P , .001). In the order-to-collection delay
category (Table 3), Hematology–Complete Blood Count
(CBC) appeared as the test 5% more frequently than the
overall average for the category, whereas Hematology–
Other Than CBC appeared 5% less frequently than the
overall average. Similarly, for the collection-to-receipt in-
terval (Table 4), Immunology/Serology–Single Test ap-
peared 5% less frequently than the average, and no test
occurred 5% more frequently than the average. For delays
occurring in the analytic phase (Table 5), Chemistry–Sin-
gle Test and Immunology/Serology–Single Test appeared
5% more frequently than the average, and Hematology–
CBC and Other appeared 5% less frequently than the av-
erage. Postanalytic (Table 6) and Undetermined appeared
too infrequently (3.3% and 1.1%, respectively) by test type
to classify in detail.
For outlier specimens obtained in the ED, MANOVA
analysis resulted in statistically significant models with r 2
# 0.10 for all independent variables. Time of day (order-
ing, collection, and receipt hour) (Figure 2) and broad de-
lay category factors (Table 9) were significant for all in-
dependent variables. Day of the week was a significant
factor for collection-to-receipt and receipt-to-verification
times only. Performing the test in a stat laboratory and the
type of test (Table 1) were significant factors for all time
intervals except ordering to collection.
Analysis of outlier factors for ICU specimens by MAN-
OVA also produced statistically significant models for all
dependent variables. The correlation coefficient for the de-
pendent variables ranged from 0.20 for the order-to-col-
lection time interval to 0.38 for the collection-to-receipt in-
610 Arch Pathol Lab Med—Vol 123, July 1999
Total No. of Stat Test With Excessive
Stat Tests TAT, n (%)
189 148 18 924 (10.0)
29 403 4317 (14.7)
218 551 23 241 (10.6)
terval. The hours of order, collection, receipt, and verifi-
cation were significant factors for all dependent variables
(Figure 2). The person collecting the specimen (Table 8),
the broad reason for delay, and type of test (Table 1) were
also significant factors. Day of the week was a significant
factor only for the time intervals of order to verification
and collection to verification. Testing performed in a stat
laboratory was a significant factor for all time intervals
except receipt to verification.
The distribution of outlier rates for those hospitals in
the lowest 10th percentile (highest performers), highest
10th percentile (lowest performers), and those between the
highest and the lowest 10th percentiles is shown in Table
10. Modeling of outlier rates using MANOVA produced
excellent correlation coefficients (r 2: ED 5 0.64, ICU 5
0.74). Although the correlation coefficients were excellent,
the probabilities found for the models were not statisti-
cally significant (P , .05). The extent of computerization
was the only significant factor found in the ICU model. A
similar highest-lowest analysis of the ED outlier rate data
found laboratory monitoring of TATs and how the speci-
mens were transported to be significant factors. Table 11
shows the number of participants in each group (1st–10th,
10th–90th, and 90th–100th percentiles) for each significant
practice parameter.
COMMENT
This study attempted to find common causes of outlier
TAT events (arbitrarily defined as an event having a total
TAT from the time of test order to the time of test result
verification in excess of 70 minutes) resulting from stat
Turnaround Time Outliers—Steindel & Novis
 Table 8. Practice Parameters by Ordering Location studies generally produces a sufficient number of data
points to allow the use of modeling.2,4 Modeling also re-
Emergency Intensive
quires homogeneity within the process, such that the mod-
Department, Care
Practice Parameter % Unit, % el describes the data adequately (and by inference the pro-
cess), as measured by the degree of correlation (r 2). One
Testing done in a stat laboratory difficulty in modeling data is that modeling is sensitive to
n 19 488 4380 diversity among the participants with respect to the se-
Yes 14.8 26.9
No 85.2 73.1
lected factors, as observed by the low correlation coeffi-
cients and few significant parameters. Our study popu-
Test type lation was quite diverse, but the final models identified a
n 19 528 4375 few common significant practice parameters with a high
Chemistry degree of statistical confidence.
Single test 18.0 21.0 Intensive care units were found to have a 4.7% higher
Multiple tests 45.7 34.9 outlier rate than EDs (Table 7). These rates were not found
Hematology to be associated with any practice variables from either
Complete blood count 21.2 20.3 location. Lack of common associations for differences in
Not complete blood count 1.5 1.3 outlier rates indicates the institutional uniqueness of these
Coagulation events, a recognized attribute of health care quality as-
Single Test 4.3 10.8 surance practices. Indeed, the history of present health
Multiple Tests 7.1 10.7 care quality assurance documents many instances in
Immunology/serology which laboratories monitor their performance, compare
Single test 1.1 0.4
that performance with benchmark levels, and then develop
Multiple tests 0.1 0.1 their own methods of improvement.17,18
Our attempt to find significant reasons why one group
Other 1.0 0.5
of participants had extremely low outlier rates while oth-
Collection Personnel ers had extremely high rates produced models describing
n 19 428 4299 the data well (r 2 . 0.6), but the probability of fit for the
Laboratory personnel 38.3 29.1 models was not statistically significant. For these data sets,
Physician/house officer 2.5 1.5
Medical student 0.0 0.0 the number of participants in the highest and lowest
Nursing services 50.1 63.2 groups was too small (Table 10) to confidently attribute
Other 3.3 0.6 practice variables to indications of improved performance.
Do not know 5.0 5.6 Although the statistical fit of the models was not adequate,
Day of the week we did observe significant factors relating to practice pa-
n 19 538 4380 rameters that were consistent with those from previous
Sunday 15.9 16.4 studies (Table 11). More of the lowest 10th percentile per-
Monday 14.8 15.3 formers from the ICU received requests from a laboratory
Tuesday 15.0 17.1 information system than from a hospital information sys-
Wednesday 14.1 15.3 tem, a pattern found previously in ED4,5 and routine test3
Thursday 14.4 12.9
Friday 13.2 11.9 TAT studies. From the ED, that same group did not mon-
Saturday 12.8 11.0 itor TAT and fewer monitored it manually, both factors
found important in maintaining or improving ED TAT.5
The factor not found in prior studies was the order trans-
mission method, where more of those among laboratories
testing. Based on prior studies,4,5 this definition was pre- in the highest 10th percentile used the telephone for ED
dicted to yield an outlier rate of about 10%. Our actual stat test orders. From these statistically identified factors,
results yielded an outlier rate of 10.6% (Table 7), indicating we can conclude that to control outlier rates, it is impor-
that the underlying distribution of TATs found in 1990 and tant for the laboratory to receive and act on an order in a
1993 had not changed significantly at the time this study timely fashion. It must then control the TAT process
was conducted (1996). through monitoring.
This study also investigated the differences in the rate Most individual outlier events were due to causes un-
of outlier events among participants. As with all Q-Probes related to the actual test determination (Figure 1). Person-
studies, conclusions are confounded by self-reported and nel problems (primarily staff shortages) were a major
unverified data. Individual laboratories will interpret the cause of delays and occurred predominantly in the test
supplied instructions in unique fashions that are unantic- ordering (37.8%), collection (51.4%), and analytic (33.7%)
ipated by the study designers. In addition, each laboratory phases of the process. Problems that related to performing
has unique solutions to providing optimum TATs based tests were responsible for only 10.9% of the overall delays.
on their own observations. To overcome these obstacles in Ross and Boone19 first reported on the low percentage of
study design and to provide unbiased identification of fac- errors (,20%) associated with the analytic phase of the
tors significant in outlier events, we used statistical mod- total testing process; other studies have confirmed this
eling of the data. finding.20,21 Our outlier analysis only reinforces the recur-
Such modeling is successful in identifying common fac- ring theme that many laboratory errors occur before the
tors causing outliers only when sufficient data points for specimen arrives.
the statistical tools, as measured by the model’s P values, Many outlier events (12.5%) did not have an identified
are available. Past experience with TAT data indicated that reason for delay and within the major delay categories, Un-
the large number of laboratories participating in Q-Probes determined accounted for approximately 20% of responses.
Arch Pathol Lab Med—Vol 123, July 1999 Turnaround Time Outliers—Steindel & Novis 611
 Table 9. Practice Parameters by Broad Delay Category
Broad Delay Category, %
Receipt:
Ordering of Collection of Receipt: Analytic Postanalytic
Practice Parameter Test Specimen Phase Phase Undetermined
Test performed in a stat laboratory
n 5918 5490 11 038 751 264
Yes 14.7 24.8 14.8 8.7 36.4
No 85.3 75.2 85.2 91.3 63.6
Test type
n 5493 5513 11 046 770 264
Chemistry
Single test 16.1 15.0 21.3 19.2 28.4
Multiple tests 43.1 45.1 44.1 43.1 33.3
Hematology
Complete blood count 25.2 23.9 17.2 20.0 17.4
Not complete blood count 1.1 25.7 1.5 2.9 2.3
Coagulation
Single test 5.7 23.9 5.2 3.9 10.2
Multiple tests 6.7 7.3 8.6 8.0 6.1
Immunology/serology
Single test 0.8 0.6 1.3 0.8 1.9
Multiple tests 0.1 0.1 0.1 0.1 0.0
Other 1.0 0.8 0.7 2.0 0.4
Collection personnel
n 5927 5483 11 001 770 179
Laboratory personnel 42.0 18.8 41.3 52.9 27.4
Physician/house officer 0.2 4.2 2.5 5.4 0.0
Medical student 0.0 0.0 0.0 0.0 0.0
Nursing services 53.2 64.5 48.8 38.7 33.0
Other 3.0 2.4 2.9 0.4 16.2
Do not know 1.7 10.1 4.5 2.6 22.9

Figure 2. Distribution of ordering, collec-

tion, and reporting time of day. The upper line
terminates at the maximum value, and the
lower line terminates at the minimum value.
The rectangle encloses the area between the
75th and 25th percentiles. The line inside the
rectangle is at the median of the distribution.
ED indicates emergency department; ICU, in-
tensive care unit.

Knowledge of where outliers arise is the first step in de-
termining how to correct them. For example, about 38% of
the outlier delays were attributed to the 4 categories relating
to possible personnel shortages (Table 1). Staff shortages
may be related to an inability of existing staff to meet de-
612 Arch Pathol Lab Med—Vol 123, July 1999
mands during periods of high workloads. Analyzing work
assignments during these times may resolve apparent staff
shortages without the need for additional staff; for example,
cross-training or redistribution of work tasks, such as pre-
ventive maintenance, may alleviate such staffing problems.
Turnaround Time Outliers—Steindel & Novis
 Table 10. Frequency Distribution of Outlier Rates as tified using the good ICU models were similar to those
Percent of Testing from the weak ED models. Two common factors were as-
sociated with time, namely, day of the week and time of
Emergency Intensive
Department Care Unit day that events occurred. It is possible that institution ad-
mission patterns tend to cluster testing naturally around
Lowest 10th percentile (least percentage of outliers) certain days (Table 8), primarily weekdays, and certain
n 44 22 times (Figure 2), primarily during the daylight hours.
Maximum 2.0 2.0
75th percentile 1.6 1.8
These observations may be useful in helping to arrange
50th percentile (median) 1.0 1.0 staff assignments to cope with personnel shortages. Two
25th percentile 0.5 0.6 other common significant factors resulting in outliers were
Minimum 0.2 0.2 type of health care worker collecting the specimen and
10th to 90th percentile whether the test was performed in a stat laboratory. These
n 372 196 factors were previously found to be significantly related
Maximum 26.9 40.0 to overall ED TAT.5 The last 2 significant factors identified
75th percentile 13.1 22.1 were the broad reason for delay and the test type. Delays
50th percentile (median) 8.5 11.7 that occurred with chemistry tests (Table 9) were most
25th percentile 4.8 6.1 likely due to the time-consuming specimen processing
Minimum 2.1 2.4
that is generally required.4
Highest 10th percentile (greatest percentage of outliers) The most frequently delayed test varied with the broad
n 47 25 delay categories. In the analytic phase, Chemistry–Single
Maximum 187.5* 100.0 Test and Immunology/Serology–Single Test were delayed
75th percentile 46.2 75.0
50th percentile (median) 32.4 50.0
most frequently, perhaps due to analytic processing time,
25th percentile 29.5 42.9 a conclusion supported by noting that Hematology–CBC,
Minimum 28.0 40.4 which has a short analytic processing time, rarely had de-
* Data were self-reported and unverified, resulting in logical incon- lays attributable to this delay group. The other 2 broad
sistencies, such as an emergency department outlier rate greater than delay categories that had significant test groups were in
100% reported by one participant. Four participants reported emergen- the preanalytic phases, order-to-collection time and collec-
cy department outlier rates of 100%. tion-to-receipt time. There is no apparent reason that the
tests observed, Hematology–Other Than CBC and Im-
munology/Serology–Single Test, should stand out.
Table 11. Participant Responses to Significant Factors The homogeneity of individual ICU outlier data allowed
from Highest/Lowest Outlier Rate Analysis (No. of us to make observations concerning each TAT interval.
Participants) Some factors were not significant in all TAT intervals. Day
90th– of the week was significant only in the initial order-to-
0–10th 100th verification and overall collection-to-verification phases,
Percentile 10th–90th Percentile
but its effect was sufficiently large to also appear as a
Factor (Highest) Percentile (Lowest)
factor when all data were combined, perhaps owing to
Intensive care unit: extent of computerization patient and workload distributions. This factor may not be
Requests by hospital amenable to laboratory control. Testing in a stat laboratory
information systems 15 146 18 appeared as a statistically significant factor in all phases
Requests by laboratory
information systems 6 6 36
except the receipt-to-verification phase. It has been hy-
No computer 0 13 1 pothesized that stat laboratories are not equipped to han-
dle high-volume workloads and delays occur when these
Emergency department: turnaround time monitoring
tests form a queue.5 The delays attributed to stat labora-
Monitor by computer 28 270 28
Monitor manually 8 38 6
tory use found in this study do not occur in the testing
No monitor 8 62 13 phase, but rather in those phases that would have a queue,
supporting the earlier observation. It appears that once a
Emergency department: stat order transmission method
laboratory enters the analytic phase, testing is timely.
Messenger 0 1 1
Telephone 16 48 1
Our study found that outlier investigation is a useful
Intercom 1 2 0 tool for general stat TAT monitoring. Information concern-
Fax 0 2 0 ing the causes of extreme TATs was similar to those of
Computer 22 206 24 overall TATs, but some new factors did emerge. Lack of
Pager strong statistical models, however, indicates the local na-
Care area to ture of many factors influencing outlier events. We rec-
phlebotomist 1 13 0 ommend that laboratories add outlier investigation to their
Laboratory area to TAT quality assurance programs both on a periodic basis
phlebotomist 0 2 0 and when they suspect developing TAT problems.
A system to monitor outlier TATs is easy to establish.
The first step is to set your criteria for outliers. This study
Causes of ED outlier events were too diverse to produce assumed a 10% outlier rate and used a criterion of 70 min-
good modeling (low r 2). Intensive care unit outlier events utes from order to reporting time. Each laboratory should
were better related among participants, and strong models determine the distribution of outlier TATs in its own in-
could be developed for each of the TAT components, al- stitution and set a rate at the TAT seen when a sufficient
lowing identification of statistically significant practice pa- volume of outlier specimens (recommendation, 10%) is ob-
rameters. Statistically significant practice parameters iden- served. When investigating outliers, which may occur as
Arch Pathol Lab Med—Vol 123, July 1999 Turnaround Time Outliers—Steindel & Novis 613
a continual process or in response to a specific event, de-
termine the cause of delay for all specimens exceeding the
established outlier criteria so that there are as few un-
knowns as possible. This study indicates most causes will
be unique to an individual laboratory, but several common
causes of delay were observed. If an institution has certain
admission or ordering patterns centered around times of
day or day of week, the laboratory may see outliers fol-
lowing those patterns. Adjusting staffing schedules may
help that situation. Certain test types may be delayed
more often than others. If chemistry tests are frequently
delayed, consider using methods that do not require spec-
imen processing or optimize the specimen processing
area. If delays are in the preanalytic phase, which is the
most common source of delays, study the collection and
transport processes used. Even if nonlaboratory personnel
perform these tasks, the laboratory should control them;
lack of laboratory control in the preanalytic phase is a
common cause of delay.22 Finally, start to monitor TATs
and react to outliers before others notice significant prob-
lems. Continual investigation of outliers will help achieve
this last goal.
This study did not demonstrate clear reasons for outli-
ers, as first hypothesized. It does demonstrate, however,
that investigation of outliers adds new details not found
when monitoring overall TATs. These details include such
items as test type, day of the week, and time of day and
are most likely specific to each institution. Studying out-
liers can help laboratories find TAT bottlenecks in the
same way the process is used in many other service in-
dustries.
References
1. Valenstein P. Laboratory turnaround time. Am J Clin Pathol. 1996;105:676–
688.
2. Howanitz PJ, Steindel SJ. Intralaboratory performance and laboratorians’ ex-
pectations for stat turnaround times: a College of American Pathologists Q-Probes
study of four cerebrospinal fluid determinations. Arch Pathol Lab Med. 1991;115:
977–983.
3. Steindel SJ, Jones BA, Howanitz PJ. Timeliness of automated routine labo-
614 Arch Pathol Lab Med—Vol 123, July 1999
ratory tests: a College of American Pathologists Q-Probes study of 653 institutions.
Clin Chim Acta. 1996;251:25–40.
4. Howanitz PJ, Steindel SJ, Cembrowski GS, Long TA. Emergency department
stat test turnaround times: a College of American Pathologists’ Q-Probes study for
potassium and hemoglobin. Arch Pathol Lab Med. 1992;116:122–128.
5. Steindel SJ, Howanitz PJ. Changes in emergency department turnaround
time performance from 1990 to 1993: a comparison of two College of American
Pathologists Q-probes studies. Arch Pathol Lab Med. 1997;121:1031–1041.
6. Novis D, Steindel SJ. Small Hospital Emergency Department Turnaround
Time: Data Analysis and Critique. Q-Probes 93-12. Northfield, Ill: College of
American Pathologists; 1993.
7. Steindel SJ. Timeliness of clinical laboratory tests: a discussion based on five
College of American Pathologists Q-Probe studies. Arch Pathol Lab Med. 1995;
119:918–923.
8. Valenstein PN, Emancipator K. Sensitivity, specificity, and reproducibility of
four measures of laboratory turnaround time. Am J Clin Pathol. 1989;91:452–
457.
9. Grant EL, Leavenworth RS. Statisical Quality Control. New York, NY: Mc-
Graw-Hill Inc; 1980.
10. United States v Barr Laboratories, Inc, Civil Action No. 92–1744. United
States v Barr Laboratories, 812 F Supp 458; 1998. 1993 US Dist Lexis 1932, US
District Court for the District of New Jersey, as amended 30 March 1993.
11. Thayer AM. Process safety is no accident. Chem Engineering News. 1998;
76:15–17.
12. Dale JC, Howanitz PJ. Patient Satisfaction and Phlebotomy Service: Data
Analysis and Critique. Northfield, Ill: College of American Pathologists; 1994.
13. Novis DA, Zarbo RJ, Saladino AJ. Interinstitutional comparison of surgical
biopsy diagnosis turnaround time: a College of American Pathologists Q-Probes
study of 5384 surgical biopsies in 157 small hospitals. Arch Pathol Lab Med.
1998;122:951–956.
14. Novis D, Steindel SJ. Stat Test Turnaround Time Outliers: Data Analysis and
Critique. Q-Probes 96-02. Northfield, Ill: College of American Pathologists; 1997.
15. GLM-VARCOMP. In: SAS/STAT User’s Guide. Version 6, vol 2. Cary, NC:
SAS Institute Inc; 1990:891–996.
16. Neter J, Wasserman W, Kutner MH. Applied Linear Regression Methods.
Boston, Mass: Richard D Irwin Inc; 1989.
17. Bigelow B, Arndt M. Total quality management: field of dreams? Health
Care Manage Rev. 1995;20:15–25.
18. Luce JM, Bindman AB, Lee PR. A brief history of health care quality as-
sessment and improvement in the United States. West J Med. 1994;160:263–268.
19. Ross JW, Boone DJ. Assessing the effect of mistakes in the total testing
process on the quality of patient care [abstract 102]. In: Martin ML, Addision BV,
Wagner WM, Essien JDK, eds. 1989 Institute of Critical Issues in Health Labo-
ratory Practice. Minneapolis, Minn: Dupont Press; 1991:173–173.
20. Boone DJ, Steindel SJ, Herron R, et al. Transfusion medicine monitoring
practices: a study of the College of American Pathologists/Centers for Disease
Control and Prevention Outcomes Working Group. Arch Pathol Lab Med. 1995;
119:999–1006.
21. Plebani M, Carraro P. Mistakes in a stat laboratory: types and frequency.
Clin Chem. 1997;43:1348–1351.
22. Steindel SJ, Howanitz PJ. Emergency Department Turnaround Time: Data
Analysis and Critique. 93-04. Northfield, Ill: College of American Pathologists;
1993.
Turnaround Time Outliers—Steindel & Novis