22/07/2021

Clinical Application of
Measurement Uncertainty
Tony Badrick

July 2021

1
 22/07/2021

5.5.1.4
The laboratory shall determine
measurement uncertainty for each
measurement procedure in the
examination phase used to report
measured quantity values on patients’
samples. The laboratory shall
define the performance requirements for
the measurement uncertainty of each
measurement procedure and regularly
review estimates of measurement
uncertainty.

What is different?
• Numbers
− 70
− 71
− 71.2
− 71.24

• Change in Numbers
− 142 to 149
− 142 to 147
− 142 to 144

• Numbers compared
− 150 compared to 150.1
− 148 compared to 148.5
− 146 compared to 147
− 143 compared to 145

2
 22/07/2021

1. “Does This Change in Result Reflect a Pathological Process?”

Reference Change Value

2. Choice of Reporting Unit Interval

Significant Figures

3. The Problem of Using Sharply Defined Cut points

Confidence Intervals

3
 22/07/2021

1. “Does This Change in Result Reflect a Pathological Process?”

Reference Change Value

When are two results different?

4
 22/07/2021

• CVA = analytical imprecision CVI = intra-individual biological variation

10

5
 22/07/2021

Reference Change Value (Critical Difference)

• RCV = √2 x z x √{(CVI)2 + (CVA)2}

• Where z = factor of statistical significance (two-sided 1.96)

• CVI = intra-individual biological variation
• CVA = analytical imprecision
• If there is no BV, then this collapses to MU

11

https://www.rcpa.edu.au/getattachment/463e837a-6674-41ca-9284-19b5d1096ad3/Measurement-Uncertainty.aspx

12

6
 22/07/2021

Mr P is a 56-year-old man who has been having serum PSA measurements

annually by his GP. Mr P has a PSA result of 4.2µg/L; 12 months ago, the
result by the same laboratory and measurement procedure was 3.8µg/L.
(Reference Range < 3.8ug/L). Are these results different?

Answer: When the laboratory was asked, they provided the MU for PSA at a
concentration of 2.9µg/L to be 0.15µg/L (uncertainty = 5.0%). The PSA has
increased from 3.8 to 4.2. This is 0.4/3.8 x 100 = 10.5%
For the above example: We will ignore biological variation and just consider
analytical variation (MU). Significant difference is √2 x 1.96 x CVA = 14%, namely
the two results should differ by >14% (>0.53µg/L) for there to be 95%
confidence that they are in fact different.
In this case, the laboratory has used its MU data in several ways to assist the
referring practitioner with an interpretative comment, ‘Taking account of
measurement variability, this result is not significantly different at a confidence
level of 95%’.
13

Question: Is the latest PSA value significantly above the age-related clinical
decision value of 4.0µg/L?

Answer: The clinical decision value of 4.0µg/L does not have a known
uncertainty associated with it. The MU of the result of 5.0% = 0.21µg/L at a
level of 4.2µg/L.

The lab can assist with calculating the confidence interval for the patient
result, which was found to be +/- 0.4µg/L.

Thus the 95% confidence interval for the latest result = 3.8 - 4.4µg/L.

So the result is not significantly different to 4.0 µg/L

14

7
 22/07/2021

1. “Does This Change in Result Reflect a Pathological Process?”

Reference Change Value

2. Choice of Reporting Unit Interval

Significant Figures

15

Dilutions

16

8
 22/07/2021

17

Using MU estimates to determine the reporting interval for quantitative

laboratory
results ensures reporting practices match local analytical performance and
recognises the inherent error of the measurement process.

18

9
 22/07/2021

19

1. “Does This Change in Result Reflect a Pathological Process?”

Reference Change Value

2. Choice of Reporting Unit Interval

Significant Figures

3. The Problem of Using Sharply Defined Cut points

Confidence Intervals

20

10
 22/07/2021

Figure shows the proportion of patients having annual health checks who cross a
threshold for diabetes of HbA1c of 6.5%, stratified by the HbA1c of the first test.

Patients who are initially close to the threshold (with a baseline HbA1c of 6.0 to 6.4)
have a high probability of crossing the threshold on subsequent testing.
21 [Footer]

21

A patient with an initial measurement of HbA1c <6.0% is unlikely to cross the disease
threshold over a three-year period, and so annual measurement is not needed.

This is partly as a result of the regression to the mean effect described above
(which explains the large jump in the proportion crossing the threshold in the
second measurement and fewer in the third and fourth measurement), but also
22 [Footer]
because amongst these patients, the true value is more likely to cross the threshold.
22

11
 22/07/2021

1) Avoid the ‘ping-pong’ effect which arises from adjusting treatment in response to
small changes, or from over- adjusting. This may be helped by providing appropriate
nomograms or algorithms.

2) Don’t read single measurements in isolation, but interpret results from the
longer-term trends and variation, preferably from a graph of results (which the
laboratory might supply).

3) Don’t re-measure until there is a chance of a real change, which means

either the patient’s clinical picture has changed or there has been a long enough
interval for ‘drift’ beyond a threshold.

The latter interval is usually longer than clinicians’ intuition suggests, as that
has been coloured by the test imprecision.

4) Encourage patients to use the same laboratory when monitoring, due to the
potential for changes in measures between laboratories.

23

What is good enough?

• EQA - performance

• Method evaluation – MU calculation

• Sigma metric (Capability)

• QC strategy – Frequency and QC rules

24

12
 22/07/2021

What is good enough?

• What is the ‘error’ in the new/existing method?
• Total error concept
− Bias and imprecision
− Reference material for bias?
• MU
− QC data
− ?bias

• What is ‘good enough’?

− Total Allowable Error
− Analytical Performance Goals
− Stockholm/Milan Hierarchy
− Biological Variation

25

26

13
 22/07/2021

Model 1. Based on the effect of analytical performance on clinical outcomes

1. Direct outcome studies – investigating the impact of analytical performance of the test on clinical
outcomes;

2. Indirect outcome studies – investigating the impact of analytical performance of the test on clinical
classifications or decisions and thereby on the probability of patient outcomes, e.g., by simulation or
decision analysis.

Model 2. Based on components of biological variation of the measurand.

This attempts to minimize the ratio of ‘analytical noise’ to the biological signal.

Model 3. Based on state-of-the-art.

This relates to the highest level of analytical performance technically achievable. Alternatively, it could be
defined as the analytical performance achieved by a certain percentage of laboratories. It could be
manufacturer’s insert.

27

Analytical Performance Specifications

EQA providers use analytical performance goals to assess the quality of results

• These can include one or more of the following:

- Z-scores / SD Index (e.g. RIQAS, BioRad, RCPAQAP)
- Analytical Performance Specifications (e.g. RCPAQAP, WEQAS UK)

Desirable versus Observed

28

14
 22/07/2021

Analytical Performance Specifications (APS)

Jones et al Clin Chem Lab Med 2017;55(7)

29

RCPAQAP Analytical Performance Specifications

• Analytical performance goal
• Based on biological and analytical
variation and clinical purpose
• Reviewed by an expert panel of
pathologists and scientists
• Ideally, it is set to be attainable by 80% of
users, but only if it meeting fitness for Analytical
purpose Performance
• It is the acceptable range around the Specifications
central value

30

15
 22/07/2021

Basis of RCPAQAP APS

• Imprecision - Monitoring
−Can monitor patient between laboratories

• Total Error – Diagnosis

−Includes a component of bias + imprecision
−Labs can share reference interval

31

CVa = 0
9

8
VALUE

6
+0% more dispersion
5
12:00 AM 6:00 AM 12:00 PM 6:00 PM 12:00 AM
TIME

4. What is the clinical value of MU?

32

16
 22/07/2021

CVa = 0.25 CVb

9

8
VALUE

+3% more dispersion

5
12:00 AM 6:00 AM 12:00 PM 6:00 PM 12:00 AM
TIME

4. What is the clinical value of MU?

33

CVa = 0.5 CVb

9

8
VALUE

6
+12% more dispersion
5
12:00 AM 6:00 AM 12:00 PM 6:00 PM 12:00 AM
TIME

4. What is the clinical value of MU?

34

17
 22/07/2021

CVa = 0.75 CVb

9

8
VALUE

6
+25% more dispersion
5
12:00 AM 6:00 AM 12:00 PM 6:00 PM 12:00 AM
TIME

4. What is the clinical value of MU?

35

CVa = CVb
9

8
VALUE

6
+41% more dispersion
5
12:00 AM 6:00 AM 12:00 PM 6:00 PM 12:00 AM
TIME

4. What is the clinical value of MU?

36

18
 22/07/2021

37

What is good enough?

38

19
 22/07/2021

39

40

20
 22/07/2021

From: Establishing Evidence-Based Statistical Quality Control Practices

Am J Clin Pathol. Published online December 05, 2018. doi:10.1093/ajcp/aqy158
Am J Clin Pathol | © American Society for Clinical Pathology, 2018. All rights reserved. For permissions, please e-mail:
journals.permissions@oup.comThis article is published and distributed under the terms of the Oxford University Press, Standard
Journals Publication Model
(https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

41

Z-scores
Assumes that the measurement procedure is ‘fit for purpose”
“Imprecise method peer groups will have a large acceptance interval whereas
precise method peer groups will have a small interval, independent of what is
required for clinical needs”

42

21
 22/07/2021

Z-scores
Z-Scores often track APS’s, but will flag differently

43

Summary
• MU describes laboratory-specific error based on real data – Evidence based

• It is used each time we interpret a result – importance of significant figures, cut-

offs and reference intervals

• We need to understand, based on the MU of the method, what we should report

and how to advise the interpretation of that result

• We usually use MU together with BV to determine if the method is good enough

• Understand what is the basis for EQA acceptability

44

22