CUN. CHEM.

40/10, 1909-1914 (1994) Laboratory

#{149} Management and Utilization

Allowable Imprecision for Laboratory Tests Based on Clinical and Analytical Test
Outcome Criteria
James 0. Westgard’, Julie J. Seehafer, and Patricia L. Barry

The allowable imprecision for laboratory tests has been
estimated from criteria based on clinical and analytical
test outcome. The analytical outcome criteria studied are
the Clinical Laboratory Improvement Amendments (CLIA)
criteria for proficiencytesting. The clinical outcome criteria
be used in judging the acceptability of laboratory per-
formance for -80 tests that are regulated in the US (1 )2
Given that the origins of the CLLA PT criteria are not
well-documented, we consider it of interest to under-
stand how they compare with clinical outcome criteria.

Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020

are estimates of medically significant changes in test re- Clinical outcome criteria can be formulated in terms
sultstaken from a study in the literature. The estimates of of the total variation in a test result that would cause
allowable imprecision were obtained from quality-plan- the medical interpretation to change. Almost 10 years
fling models that relate test outcome criteria to the allow- ago, Skendzel et al. (2) surveyed physicians by use of
able amount of imprecision and inaccuracy and to the clinical vignettes that focused on test interpretation in
quality controlthat is necessary to assure achievement of various situations, such as patients who are healthy and
the desired outcome criteria in routine operation. These undergoing routine screening, patients having a variety
operating specifications for imprecision are consistently of disorders, and patients being monitored to detect drug
more demanding (require lower CV5) than the medically toxicity. The authors determined the average changes
useful CVs originally recommended in the literature be- in laboratory test results that would be judged by phy-
cause the latter do not properly consider within-subject sicians to be medically significant. These estimates re-
biological variation. In comparing estimates of allowable flected physicians’ opinions of changes in test results
imprecision,the CLIA outcome criteria are more demand- that would cause them to”. - take action such as order-
.

ing than the clinical outcome criteria for aspartate ami- ing more tests, changing therapy, or considering an-
notransferase (asymptomatic patients), cholesterol, cre- other diagnosis.”
atinine (asymptomatic patients), glucose, thyroxine, total The usefulness of clinical outcome criteria based on
protein, urea nitrogen, hematocrit, and prothrombin time. physicians’ opinions vs the quality goals for imprecision
The clinical outcome criteria are more demanding for derived from biological variation has been discussed (3),
bilirubin (acute illness), iron, potassium, urea nitrogen as has the use of analytical outcome criteria for PT vs
(acute illness), and leukocyte count. The estimates of quality goals for imprecision (4). Unfortunately, direct
allowable imprecisionfrom analytical and clinicaloutcome comparisons are not possible between clinical outcome
criteria overlap for aspartate aminotransferase (acute ill- criteria, analytical outcome criteria, and goals for im-
ness), bilirubin (asymptomatic patients), calcium, creati- precision and inaccuracy because each set of criteria
nine (acute illness), sodium, triglyceride, and hemoglobin. describes limits for different factors that affect the vari-
ation in a test result. Clinical outcome criteria describe
IndexIngTerms: laborafolymanagement/qualitycontrol/proficiency a limit for both preanalytical and analytical components
testing of variation in the testing process. Analytical outcome
criteria describe a limit for only components of the an-
Test outcome criteria provide consumer-oriented re- alytical process, e.g., imprecision, inaccuracy, and qual-
quirements for quality, in contrast to performance goals ity control (QC). Goals for imprecision and inaccuracy
for imprecision and inaccuracy, which provide labora- are operating specifications for individual components.
tory specifications for operating a testing process. Out- Comparisons can be made when clinical and analyti-
come criteria define a limit for the total variation that is cal outcome criteria are used to derive operating speci-
allowable in a laboratory test result, rather than setting fications for the imprecision and inaccuracy that are
a limit for an individual factor or component that con- allowable and for the QC that is necessary in routine
tributes to the variation of a test result. Such outcome operation of an analytical testing process (5, 6). Previ-
criteria can be formulated to reflect either analytical or ously, we illustrated how proposed European specifica-
clinical requirements for test performance. tions for imprecision and inaccuracy could be compared
Analytical outcome criteria 4escribe the total analyt- with US CLLA proficiency testing criteria (7). Here, we
ical errors that would cause a test result to be judged as determine the maximum imprecision that is allowable
analytically unacceptable. For example, the 1988 Clin- based on the clinical outcome criteria of Skendzel et al.
ical Laboratory Improvement Amendments (CLIA) de- (2) and compare these estimates with the maximum
fine the total error criteria for proficiency testing (PT) to

Department of Pathology and Laboratory Medicine, Medical 2Nonstandard abbreviations: PT, proficiency testing CLIA,
School, University of Wisconsin, 600 Highland Ave., Madison, WI Clinical Laboratory Improvement Amendments (1988); QC, qual-
53792. ity control; and NCEP, National Cholesterol Education Program.
‘Author for correspondence. Fax 608-263-0910. Received April 4, 1994; accepted May 24, 1994.

CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994 1909

imprecision that is allowable based on CLIA analytical chemistry tests were based on Fraser’s (10, 11) summa-
outcome criteria. ries of studies in the literature. The particular values we
used represent the average of all values tabulated for
Materials and Methods studies that lasted >1 week. Results listed in the orig-
Quality-Planning Models inal summaries as “Neg” were excluded, as was one
The formulation of clinical and analytical quality value for cilkaline phosphatase that did not seem con-
planning models has been described earlier (8, 9). To sistent with others. Estimates of within-subject varia-
apply the clinical model, we estimated the clinical deci- tion for hematology tests were obtained from a single
sion interval as the difference between the “from” study by Fraser (12), and the estimate for prothrombin
and “to” values of physicians’ opinions of a significant time was the mean value for the group from a study by
change in test results shown in Table 1 of Skendzel eta!. Dot eta!. (13).
(2). was then related to preanalytical and analyti- Estimates of the maximum allowable imprecision
cal factors by the following equation: were determined from the x-intercepts of the lines de-

Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020

scribing the allowable limits of imprecision and inaccu-
D1 = bias + biasm + SEcontSmeas racy on charts of operating specifications (OPSpecs
charts, 6), which were prepared with the QC Validator
/4b S?,pec (iREconiSmea8)2 program (WesTgard QC, Ogunquit, ME). These esti-
+z’j/-+ +
V test testspec testfl8pecflsamp mates represent the imprecision that would be allow-
able with commonly used QC procedures, such as 1
where bias is the sampling bias, biasme,, is the stable with N = 2, 1258 with N = 2 and 4, 139 with N = 2 and
measurement bias or analytical inaccuracy, lSEa,nt is 4, 1I2/R4,8 with N = 2, and 139/2R/4 with N = 4.
the sensitivity of the QC procedure for detecting system- OPSpecs charts for 90% analytical quality assurance
atic error, Sm is the stable measurement standard were utilized to specilr an error detection of 0.90 or 90%
deviation or analytical imprecision, z is related to the for critical systematic errors that would cause measure-
maximum defect rate allowable before stopping the ment performance to exceed the defined outcome crite-
process, 8b is the within-subject biological variation, ria. The relative demands of clinical and analytical out-
s is the between-specimen sampling variation, come criteria are evaluated by comparing these
RECOflt is the sensitivity of the QC procedure for detect- estimates of the clinical and analytical maximum allow-
ing random error or changes in imprecision, n is the able imprecision.
number of tests performed, n8 is the number of spec-
imens drawn, and n1, is the number of samples mea- Results
sured for each specimen. Figure 1 shows the inaccuracy and imprecision that
When the preanalytical factors in the clinical quality- are allowable for a calcium method at a decision level of
planning model are set toO, only analytical factors re- 85 mg/L when the medically significant change is 9
main, so that the outcome criterion becomes the total mgIL, or 10.58%. The top line on this chart describes the
allowable analytical error, TEa. For the condition that maximum limits that would be allowable for a perfectly
n, n, and are 1 (i.e., a single test with a stable measurement procedure that requires no QC. The
single specimen drawn and a single aliquot tested), TEa lower lines describe the operating limits that are neces-
is related to the analytical factors as follows: sary to assure that the desired quality will be achieved
in routine operation with use of the QC procedures that
TEa = biS.Bmea8 + LSEcontSmeas + ZLREoontSmeas are commonly employed in clinical laboratories today.
The key area at the right of the chart identifies the
In applying the analytical model, the CLLA PT criteria control rules and number of control measurements in
for acceptable performance provide values for TEa, the order of the lines from top to bottom. The operating
which we indicate by the symbol TEp1.. point, which is seen to be offscale on this chart, repre-
sents the medically useful CV of 4.8% from Table 3 of
Estimation of Allowable Imprecision Skendzel et a!. (2). By comparison, when within-subject
Outcome criteria were applied at the medical decision biological variation and the sensitivity of the QC proce-
levels or target values recommended by Skendzel et a!. dures are accounted for, the clinical maximum allow-
(2), which correspond to clinical situations for asymp- able CV is 1.7-2.4%, as determined from the x-inter-
tomatic patients, acute illness, and drug monitoring (as cepts of the operating lines for the different QC
identified in Table 1 of ref. 2). procedures.
Sampling bias (bias) and between-specimen varia- Figure 2 shows similar information for calcium for the
tion (8,) were assumed to be 0, analytical perfor- CLIA PT criterion of 10 mgfL, which corresponds to
mance was evaluated for a single test (n = n8 = 11.76% at a decision level of 85 mg/L. The analytical
= 1), z was 1.65 (to set a one-tailed or one-sided maximum allowable imprecision is 2.3-3.0%, as esti-
maximum defect rate of 0.05 or 5%), and control perfor- mated from the x-intercepts of the operating lines for
mance was optimized for detection of systematic error the different QC procedures. The operating point again
(LRECOflt = 1.0). shows that the medically useful CV of 4.8% exceeds the
Estimates of within-subject biological variation for allowable limits.

1910 CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994

 OPSpecs Chart D11g10.58% wIth 90% AQA(SE)

N
1/2,./R441
0.03 4 1
-bIe 1255

0.04 4 Fig.1. Calcium operating specifications for

1,q
providing 90% analytical quality assurance
0.09 2 1 (AQA) that increases in systematic errors
1
____________________ will not cause a clinical decision interval of
0.01 4 10.58% to be exceeded: allowable inaccu-
__________________ racy (y) vs allowable imprecision (x).
\\ 1,,/2,JR4
0.03 2 The differentlinesdescribe the operatinglimits
for the different control rules and differentnum-
bers of control measurements shown in the key
0.01 2 1

Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020

1.0 area on the nght. The orderof lines in the key
1.
from top to bottomrepresents the order of the
u.U
\\\J Coeratina cointoff scale
.
0.00 2 lines on the chail fromtop to bottom. The oper-
ating point represents the medically useful CV
0.00 1.00 2.00 3.00 4.00 fromSkendzelet al. (Table3 in ref. 2). Pfr, prob-
Allowab4s lniqx.ci.ion (s,,..,as%) abilityfor false rejection;R, run.

OPSpecs Chart 1176% wIth 90% AQA(SE)

12.C #{149}

2 1 Fig. 2. Calcium operating specifications for

, ; eratin Point 0.00
1’ providing 90% assurance that increases in
0.00 1.00 2.00 3.00 4.00 5.00 6.00 systematic errors will not cause an analyt-
AlIowabl knpr.clslcn (s.%)
ical total error of 11.76% to be exceeded.

Table 1 summarizes the allowable imprecision for for bilirubin (acute illness), iron, potassium, urea nitro-
those tests for which clinical outcome criteria are de- gen (acute illness), and leukocyte count. The estimates
fined by Skendzel et a!. (2) and analytical outcome cri- of allowable imprecision overlap for aspartate ami-
teria are defined by CLIA PT criteria (1). Column 4 notransferase (acute illness), bilirubin (asymptomatic
shows the clinical outcome criteria in the form of the patients), calcium, creatinine (acute illness), sodium,
medically significant changes from Table 1 by Skendzel triglyceride, and hemoglobin. All the operating specifi-
et al. (2). Column 5 shows the estimates of within- cations for allowable imprecision are more demanding
subject biological variation that were used to derive the than the medically useful CVs recommended by Skend-
clinical maximum allowable imprecision shown in col- zel et al. (2).
umn 6. These estimates of imprecision are to be com-
pared with the values in column 7 for the medically Discussion
useful CVs [taken from Table 3 of Skendzel et a!. (2)] The difficulty in comparing different requirements for
and with the values for the analytical maximum allow- analytical performance is ifiustrated in Table 1 by the
able imprecision (column 8), derived from the CLIA iron test, where the medically significant change or clin-
analytical outcome criteria shown in column 9. ical decision interval of 33% is almost entirely con-
These estimates of the maximum allowable impreci- sumed by the within-subject biological variation of
sion show that the CLIA analytical outcome criteria are 19.8%, requiring analytical imprecision of <1% when
more demanding than the clinical outcome criteria for biological variation is taken into account, rather than
aspartate aminotransferase (asymptomatic patients), the medically allowable CV of 17.2% recommended by
cholesterol, creatinine (asymptomatic patients), glucose, Skendzel et a!. (2). Similar situations are seen for bill-
thyroxine, total protein, urea nitrogen, hemathcrit, and rubin (acute illness) and urea nitrogen (acute illness),
prothrombin time. The clinical outcome criteria are where the within-subject biological variation by itself
more demanding than the analytical outcome criteria will cause changes that exceed the intervals for medi-

CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994 1911

 Table 1. EstImates of allowable imprecision from clinical and analytical outcome criteria.
ClIn. Analytical
Medically maximum Skendz&’s maximum
sIgnificant Within-subject allowable medIcally allowable
Decision change, blol. varIation, ImprecIsion, useful CV Imprecision, US CLIA PT
Test level Units DIN (%) (%) (%) (%) s,... (%) Criteria, TEPT
Aspartate aminotrans. 30 U/L 13 (43.3) 10.8 6.5-9.0 26.3 3.9-5.1 20%
70 20 (28.6) 2.9-4.2 14.3 3.9-5.1
Bilirubin, total 8 mg/L 6(75.0) 18.1 11.2-15.7 23.4 9.7-13.0 20%or 4 mg/L5
150 20 (13.3) o.oa 5.4 3.9-5.1
Calcium 85 mg/L 9(10.6) 2.0 1.7-2.4 4.8 2.3-3.0 10.0 mg/L
Cholesterol 2100 mg/L 700 (33.3) 6.1 5.6-7.6 12.3 1.9-2.6 10%
Creatinine 10 mg/L 5 (50.0) 4.9 9.1-12.4 17.2 5.7-7.7 15% or 3 mg/Lb
30 5 (16.7) 2.2-3.1 10.1 2.9-3.8

Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020

Glucose 1000 mg/L 300 (30.0) 12.2 2.7-3.9 11.2 1.9-2.6 10% or 60 mg/li’
Iron 1500 1g/L 500 (33.3) 19.8 0.2-0.3 17.2 3.9-5.1 20%
Potassium 3.8 mmol/L 0.4 (10.5) 4.9 0.7-1.0 4.8 2.6-3.4 0.5 mmol/L
Sodium 125 mmol/L 5 (4.0) 0.8 0.7-0.9 1.7 0.6-0.8 4 mmol/L
135 6 (4.4) 0.7-1.0 2.7 0.6-0.8
Thyroxine, total 60 pg/L 20 (33.3) 5.8 5.7-7.6 17.2 3.9-5.1 20% or 10 Mg/Lb
Total protein 70 g/L 15(21.4) 3.3 3.8-5.1 8.3 1.9-2.6 10%
Triglyceride 1300 mg/L 600 (46.2) 19.8 3.4-5.5 16.1 4.8-6.4 25%
Urea N 180 mg/L 100 (56.6) 12.2 8.7-12.0 18.7 2.2-2.8 9% or 20 mg/Lb
400 80(20.0) *0.0 12.2 1.8-2.3
Hematocrit 420 mL/L 50 (11.9) 2.5 1.9-2.6 5.4 1.2-1.5 6%
Hemoglobin 150 g/L 12 (8.0) 2.4 1.1-1.5 3.6 1.4-1.8 7%
Leukocyte count 5 x io cell/L 1.6 x io (32) 15.6 1.7-2.6 16.4 2.9-3.8 15%
25 x 10#{176} 7 x io#{176}
(28) 0.6-1.0 14.0 2.9-3.8
Prothrombin time 12.0 s 3 (25.0) 2.3 5.1-7.1 15.2 2.9-3.8 15%
20.0 5 (25.0) 2.3 5.1-7.1 15.2 2.9-3.8
aPreanalytical variationexceedsthe medically significant change.
Whichever is greater.

cally significant change. These examples illustrate why sion, whereas the earlier estimates assumed all of the
there has been considerable skepticism about the valid- variation could be analytical imprecision. Linnet recog-
ity of these earlier estimates of medically useful CVs nized these difficulties and accounted for both preana-
(14) and may also help explain why there has been little lytical and biological variation when deriving the max-
ongoing work to determine clinical outcome criteria in imum allowable errors to be used in selecting QC
the form of medically significant changes in test results. procedures (15).
Our purpose here is to consider the difficulties in trans- The rigorous demands for analytical imprecision re-
lating medically significant changes into specifications sult from the design objective to achieve 90% detection
for operating laboratory testing processes, rather than of critical-sized systematic errors by using only the low
the difficulties in estimating medically significant numbers of control measurements (N = 2-4) that are
changes. However, we think that there is merit in de- deemed acceptable by professional practice and regula.
termining how customers actually use laboratory tests, tory guidelines. The 90% objective is based on the desire
that guidelines are needed for obtaining reliable esti- to detect medically important analytical errors in the
mates of medically significant changes, and that knowl- first run in which they occur (expected average run
edge of customer requirements should be valuable when length ARL = 1/Pedor 1/0.90 = 1.1). Use oflowNvalues
establishing outcome criteria and operating specifica- limits the power of statistical QC procedures and makes
tions. it difficult to detect critical systematic errors that are
The operating specifications for imprecision that we <-3 CV of the method. Improving precision (reducing
derived in this study are consistently more demanding the CV) has the effect of changing those critical system-
than earlier estimates of medically useful CVs because atic errors to higher multiples of the CV, enabling
the additional factors of within-subject biological varia- greater error detection with commonly used QC proce-
tion and the sensitivity of QC procedures are included in dures.
the quality-planning models that were applied. These These rigorous demands for analytical performance
quality-planning models provide error budgets for the are achievable in routine operation, as demonstrated in
various factors that may affect the variation of a test an earlier study (16), where 90% error detection was
result. Factors such as within-subject biological varia- obtained for 15 out of 18 tests on a multitest chemistry
tion and the sensitivity of the QC procedures reduce the analyzer with use of N = 2 and single-rule QC proce-
amount of variation that is left for analytical impreci- dures. Lower probabffities for error detection may be

1912 CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994

acceptable for very stable analyzers that have a low interval from 2000 to 2400 mg/L as the medically sig-
frequency of problems (17), but it is generally advisable nificant change, whereas it is the change from 2100 to
to first attempt to achieve 90% detection and then, if 2800 mg/L that represents medical practice according to
necessary, optimize performance for the observed or ex- the study by Skendzel et al. (2). The NCEP reduction of
pected frequency of problems. When appropriate, such the earlier decision interval of 700 mgfL to 400 mg/L
optimization can be aided by OPSpecs charts with 50% places considerably greater demands on test perfor-
or 25% analytical quality assurance specifications mance.
(available from the QC Validator program) or by QC Wiebe and Westgard (20) also identify cholesterol as
selection grids (18). a model system for relating medical needs to analytical
In summarizing recommendations for allowable im- performance and demonstrate that current NCEP clin-
precision, our use of the x-intercepts from the OPSpecs ical requirements for data interpretation and CLIA PT
curves assumes that inaccuracy is zero (biasm =
criteria are not consistent. The NCEP guidelines for
0.0%). The effects of non-zero biases can be determined interpretation of a single cholesterol test correspond to a

Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020

from the OPSpecs curves themselves and would lead to decision interval of 20%, which leads to an estimate of
the need for smaller CVs. Even smaller CVs would be 2.4% to 3.5% for the maximum allowable imprecision,
required if other preanalytical components were signif- compared with the 1.9% to 2.6% that is allowable for a
icant. Except for within-subject biological variation, we 10% CLIA PT criterion. A CLIA PT criterion of 13%
assumed other preanalytical variables were zero be- would provide operating specifications that are consis-
cause of the difficulty of obtaining this information for tent with the NCEP clinical requirement for data inter-
the many analytes considered here; however, the qual- pretation. Thus, clinical guidelines for the interpreta-
ity-planning models provide the capability of dealing tion of a cholesterol test could be used to establish
with additional variables, such as sample variation and operating specifications for imprecision, inaccuracy, and
specimen bias. QC, which in turn could be related to an analytical
When within-subject biological variation is accounted outcome criterion that requires consistent operating
for in the clinical model and QC sensitivity is included specifications.
in both the clinical and analytical models, the estimates This cholesterol model ifiustrates the importance of
of allowable imprecision from the clinical and analytical understanding the hierarchy between clinical and ana-
outcome criteria are actually in good agreement for lytical outcome criteria, the relationships between out-
about one-third of the tests studied, the analytical out- come criteria and operating specifications, and the need
come criteria are more demanding than the clinical for to include the performance of the QC procedures in
another half, and the clinical are more demanding than establishing operating specifications. All of these char-
the analytical for the remaining sixth. These findings acteristics are important in managing the quality of
suggest that it should be possible to establish a coherent laboratory testing processes, but each has a particular
set of clinical and analytical outcome criteria that would role to play in a system for total quality management.
provide consistent operating specifications for the im-
precision, inaccuracy, and QC needed by laboratory test- In summary, this study demonstrates how the impre-
ing processes. cision that is allowable for a laboratory test can be
Boone discussed the advantages of using analytical determined from either clinical or analytical outcome
outcome criteria of the total error format for PT pur- criteria and suggests that medically relevant criteria for
poses but also stressed the need for PT criteria to reflect proficiency testing can be established by using coherent
medical usefulness (4). This would seem to be achiev- clinical and analytical quality-planning models. In con-
able by relating clinical and analytical outcome criteria junction with our earlier study that demonstrated how
through the operating specifications required to assure analytical goals for imprecision and inaccuracy can be
the desired outcome. Estimates of medically important compared with analytical outcome criteria (7), it should
changes and within-subject biological variation are the also be apparent that those analytical goals can simi-
primary information from which the total analytical larly be compared with operating specifications derived
error would be determined. Other preanalytical compo- from clinical outcome criteria. Together, these studies
nents of error could be considered when necessary. PT ifiustrate an approach for relating, comparing, and
criteria in the format of an allowable total analytical making some sense of the many different quality re-
error could then be established that would provide con- quirements, recommendations, goals, and specffications
sistent operating specifications for imprecision, inaccu- for laboratory tests.
racy, and QC. This approach would permit PT criteria to
reflect medical needs and would provide a rationale for
References
deriving, explaining, and verifying such criteria.
1. US Dept. of Health and Human Services. Medicare, Medicaid,
Experience with cholesterol testing in the US is cited and CLIA Programs: regulations implementing the Clinical Lab-
by Boone (4) as evidence that medically relevant goals oratory Improvement Amendments of 1988 (CLIA). Final rule. Fed
for analytical performance are helpful in setting realis- Regist 1992;57:7002-186.
tic criteria for PT and QC. National Cholesterol Educa- 2. Skendzel LP, Barnett RN, Platt R. Medically useful criteria for
analytic performance of laboratory tests. Am J Clin Pathol 1986;
tion Program (NCEP) guidelines (19) for use and inter- 83:200-5.
pretation of a cholesterol test have established an 3. Fraser CG, Hyltoft Petersen P. Desirable standards for labors-

CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994 1913

tory tests if they are to fulfill medical needs. Clin Chem 1993;39:
1447-53.
4. Boone DJ. Governmental perspectives on evaluating laboratory
performance. Cliii Chem 1993;39:1461-5.
5. Westgard JO. Assuring analytical quality through process
planning and quality control. Arch Pathol Lab Med 1992;116:
765-9.
6. Weetgard JO. Charts of operational process specifications (“OP-
Specs charts”) for assessing the precision, accuracy, and quality
control needed to satisfy proficiency testing performance criteria.
Cliii Chem 1992;38:1226-33.
7. Westgard JO, Seehafer JJ, Barry PL. European specifications
for imprecision and inaccuracy compared with operating specifica-
tions that assure the quality required by US CLIA proficiency-
testing criteria. Clin Chem 1994;40:1228-32.
8. Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory pro-
cess specifications for assuring the quality in the U.S. National
Cholesterol Education Program. Cliii Chem 1991;37:656-61.
9. Westgard JO, Wiebe DA. Cholesterol operational process spec-
ifications for assuring the quality required by CLIA proficiency
testing. Cliii Chem 1991;37:1938-44.
10. Fraser CG. Biological variation in clinical chemistry. An
update: collated data, 1988-1991. Arch Pathol Lab Med 1992;116:
916-23.
11. Fraser CG. The application of theoretical goals based on
biological variation data in clinical chemistry. Arch Pathol Lab
Med 1988;112:404-15.
1914 CLINICAL CHEMISTRY, Vol. 40, No. 10, 1994
12. Fraser CG. Desirable standards for hematology tests: a pro-
posal. Am J Clin Pathol 1987;88:667-9.
13. Dot D, Miro J, Fuentes-Arderiu X. Within-subject and be-
tween-subject biological variation of prothrombin time and acti-
vated partial thromboplastun time. Ann Clin Biochem 1992;29:
422-5.
14. Fraser CG. Better criteria for desirable laboratory perfor.
mance exist [Letter]. Am J Cliii Pathol 1986;85:251.
15. Linnet K. Choosing quality-control systems to detect maxi-
mum clinically allowable analytical errors. Cliii Chem 1989;35:
284.-S.
16. Koch DD, Ojyall JJ, Quam EF, Feldbruegge DH, Dowd DE,
Barry PL, Westgard JO. Selection of medically useful quality-
control procedures for individual tests done in a multitest analyt-
ical system. Clin Chem 1990;36:230-3.
17. Westgard JO, Oryall JJ, Koch DD. Predicting effects of qual-
ity-control practices on the cost-effective operation of a stable,
Downloaded from https://academic.oup.com/clinchem/article/40/10/1909/5647719 by guest on 21 December 2020
multitest analytical system. Clin Chem 1990;36:1760-4.
18. Westgard JO, Quam EF, Barry PL. Selection grids for plan.
nung quality control procedures. Clin Lab Sci 1990;3:271-8.
19. National Cholesterol Education Program Laboratory Stan-
dardization Panel. Current status of blood cholesterol measure-
ment in clinical laboratories in the United States. Clin Chem
1988;34:193-201.
20. Wiebe DA, Westgard JO. Cholesterol-a model system to
relate medical needs with analytical performance. Cliii Chem
1993;39:1504-13.