Diagnostic Tests

Jason Ryan, MD, MPH

Diagnostic
Special Topics
Tests
• Accuracy/Precision
• ROC Curves
• Likelihood ratios

Shutterstock
Accuracy and
Precision
• Describe quality of measurements used as part
of diagnostic test
• Accuracy (validity): how closely data matches
reality
• Precision (reliability): how closely repeated
measurements match each other
• Can have accuracy without precision (or vice versa)

Precise and accurate Precise not accurate Accurate not precise Not accurate or precise
Accuracy and
Precision
• More precise tests have smaller standard
deviations
• Less precise tests have larger standard deviations
Number or tests with result

Test B

Test A

10mg/dl
Accuracy and
Precision
• Random measurement errors: reduce precision
• Random error: some measurements okay, others bad
• Accuracy may be maintained but lots of data scatter
• Systemic errors reduce accuracy
• Imagine every BP measurement is off by 10 mmHg due to wrong
cuff size
• This Will créate a systemic error in data set (non-random error)
• Precision maintained because repeated measurements are going to
match one another but accuracy is poor
 ROC
Receiver Operating Characteristic
Curves
• Cutoff value for positive tests determines
sensitivity/specificity
• Which cutoff value maximizes sensitivity/specificity?
• ROC curves answer this question
ROC Curve

The point here is that for lost of different

values of cutoffs for blood sugar levels, we can
generate an associated sensitivity and
specificity for our test. So ROC is simply a plot
of all those cutoff values
Cutoff (mg/dL) Sensitivity Specificity
(%) (%)

100 80 20

110 65 42

120 54 63

130 42 75
 This lines tell us that every time we try to operate higher on the Y
axis, wich means higer sensitivity, , we pay an equal Price in the false

ROC positive rate . So there is no real ideal operating point here that
would give us a high sensitivity and a low false positive rate

Curves High sensitivity – low false positive rate

Sensitivity (%) (is proportional

to True Positive Rate)

1-Specificity (%) (Is

proprotional to False positive rate)
 ROC
Curves
• Straight line from bottom left to top right is a bad
test
•• Straight
Closer the curve
line fromis bottom
to a right angle,
left to the better the
test right is a bad test
top
• Closer the curve is to a right angle,
the better the test
• Point closest to top left corner is
the best cutoff to maximize
sensitivity/specificity

• Useless test has 0.5 (50%) area under curve

• Perfect test has 1.0 (100%) area under curve
• More area under curve = better test
• More ability to discriminate healthy from disease
Likelihood
Ratios
Post-test Post-test
Probability Probability
(-) Test (+) Test

0% 100%

Pretest
Probability Likelihood ratios tell us how much
probability of having the disease
shifts with (+) or (-) test
Likelihood
Ratios
LR + = Sensitivity
1 - Specificity

LR - = 1 - Sensitivity
Specificity

Characteristics of test like sensitivity/specificity

Do not vary with prevalence of disease

LR Interpretation
> 10 Large increase in probability associvted with the test result
1 No change in probability of disease with a + or – test result
< 0.1 Large decrease in probability
Likelihood
Ratios
• Serum ferritin as a screening test for iron deficiency
in children

Serum Ferritin (ng/dL) Likelihood Ratio

< 15 51.8

15 to 24 8.8

25-34 2.5

45-100 0.5

>100 0.08

Gets smaller because we are getting closer to

the normal range
Screening for Iron Deficiency in Early Childhood Using Serum Ferritin
in the Primary Care Setting Hannah Oatley et al. Pediatrics Dec 2018, 142 (6)
Term:
“Likelihood”
• What is likelihood of disease in a person with (+)
test?
• Positive predictive value
• What is likelihood of disease in a person with (-)
test?
• Negative predictive value
• What is the positive likelihood ratio?
• Calculated from sensitivity/specificity
• What is the negative likelihood ratio?
• Calculated from sensitivity/specificity

Shutterstock
Likelihood
Ratios
LR Interpretation
> 10 Large increase in probability associvted with the test result
1 No change in probability of disease with a + or – test result
< 0.1 Large decrease in probability