Diagnostic tests

evaluation & uses

Charles Caraguel
Centre for Veterinary Epidemiological Research
Centre for Aquatic Health Sciences
Atlantic Veterinary College
University of Prince Edward Island
 Why is it so exciting?
“Test performance is
critical to start a new life.” Me (2008)

1
 Basics (1):
Variation & errors in estimation
Sources
• Sampling error (selection bias)
• Biological variation (time & space)
• Measurement error (diagnostics)

Semantics
DISEASED (D+)
Health status (Binary outcome)
NON-DISEASED (D-)

POSITIVE (T+)
Test result (Binary outcome)
NEGATIVE (T-)

2
 Validation Objectives
“evaluation of a process to determine its fitness for a particular use”

Phase 1 - ”Laboratory“ validation

• Optimisation & standardisation of reagents
• Analytical sensitivity (ASe) & specificity (ASp)

Dilution

3
 Validation Objectives
“evaluation of a process to determine its fitness for a particular use”

Phase 1 - ”Laboratory“ validation

• Optimisation & standardisation of reagents
• Analytical sensitivity (ASe) & specificity (ASp)
Phase2 - “Field” validation
• “Precision” (agreement sensus stricto)
Repeatability (overall intra-lab)
Reproducibility (overall inter-lab)
• “Accuracy” (performances)
Degree of mistakes
Diagnostic Sensitivity (DSe)
made by the test
Diagnostic Specificity (DSp)

4
 Basics (2):
Conditional Probability
Pr(A/B) = given B, the probability of A

i. First set the primary information or condition (B)

Health status (Se&Sp); Test result (Predictive values)
ii. Then defined the component of interest (A)
Health status (Predictive values); Test result (Se&Sp)

5
 Tested positive
“True “False
Negative” Positive”

Positive & Diseased

Se =
Diseased

“Non diseased” “Diseased”

Negative & Non dis.

Sp =
Non diseased “False “True
Negative” Positive”

Tested negative
6
 Accuracy Definition
Se & Sp
Diagnostic Sensitivity (true positive)
Pr(T+/D+) = given D+, the probability of T+

Diagnostic Specificity (true negative)

Pr(T-/D-) = given D-, the probability of T-

7
 Legend of the
2X2 TABLE
Test Results

T+ T-
TP FN

Health
D+ a b a+b
Status
D- FP
c d c+d
TN
a+c b+d n = a+b+c+d

Sensitivity (true positive fraction) Specificity (true negative fraction)

Pr(T+/D+) = a / (a + b) Pr(T- /D-) = d / (c + d)

8
Real applications in the “field”
“farmers & veterinarians”
1. Prevalence estimate
“Apparent” Se / Sp “True”
prevalence (APr) prevalence (TPr)

Prevalence definition
# Positive # Diseased
APr = TPr =
# Total (n) # Total (n)

True prevalence formulae Apparent prevalence formulae

APr + Sp - 1 APr = Pr Se + (1 - TPr) (1 – Sp)
TPr = True positive False positive
Se + Sp - 1 among D+ among D-

9
 Exercise (1):
True prevalence estimation

During a surveillance screening, 18 cows tested positive

for Staphylococcus aureus out of 100 tested
(Test performances: Se = 72 % & Sp = 98 %)

Compute the following value:

Apparent Prevalence =

True Prevalence =

10
Real applications in the “field”
“farmers & veterinarians”
1. Prevalence estimate
“Apparent” Se / Sp “True”
prevalence (APr) prevalence (TPr)

2. Decision making
Positive Predictive Value (PPV)
Given test is positive, what probability to be diseased?
Negative Predictive Value (NPV)
Given test is negative, what probability to be non-diseased?

Primary Information: TEST RESULTS!

(condition)

11
 Diseased
Neg “Neg
True Neg Neg Neg Pos Pos Pos Pos Pos
“False
Negative” Negative”

Neg Neg Neg Neg Neg Pos Positive

Pos Pos& Diseased
Pos Pos
PPV =
Positive

Neg Negative
Neg Neg Neg Neg Pos Positive
Pos Pos Pos Pos

Negative & Non dis.

NPV
Neg =Neg Neg Neg Neg Pos Pos Pos Pos Pos
Negative
“False “True
Positive” Positive”

Neg Neg Neg Neg Neg Pos Pos Pos Pos Pos
Non-diseased
12
 Predictive Values Definitions
Positive Predictive value (PPV) Strongly influenced by Sp

Pr(D+/T+) = Given T+, the probability of D+

Negative Predictive value (NPV) Strongly influenced by Se

Pr(D-/T-) = Given T-, the probability of D-

13
 Predictive Values Formulae
Test Results

T+ T-
TP FN
D+ a b a+b

D- FP
c d c+d
TN
a+c b+d n = a+b+c+d

Positive Predictive Value Negative Predictive Value

Pr(D+/T+) = a / (a + c) Pr(D- /T-) = d / (b + d)

14
 Exercise (2):
2X2 table and more
You test 100 cows for Staph. aureus infection. Sixty-five cows tested
negative by PCR and 35 tested positive. However, using culture as
a gold standard, 5/35 of the PCR-positives were disease-free and
14/97 of the seronegatives were diseased.
Fill the 2X2 table and compute AP, TP, Se, Sp, PPV, NPV:

Se = T+ T-
Sp = D+
D-
PPV =
NPV =
15
 When no a, b, c or d?
but Se, Sp and Prevalence
Bayesian Theorem: Pr(A/B) = Pr(A&B) / Pr(B) Pr(A&B) = Pr(B) *Pr(A/B)

n take any value even 1

Pr(D+) = Prevalence(Pr)
Pr(D-) = 1 - Pr
Pr(T+/D+) = Se T+ T-
Pr(T- /D-) = Sp

D+ PrPr
n*n*
Pr( a
*Se
+&T+)
D*Se n* b
PrPr( +&T-)
*(1D-Se)

D- n* Pr)c
(1-Pr( - +
D &-Sp)
*(1 T) n* d
(1Pr( -&T+)
D*Sp
-Pr)

a+c b+d 1
n = a+b+c+d

Positive Predictive Value Negative Predictive Value

Pr*Se (1-Pr)*Sp
Pr(D+/T+) = Pr(D- /T-) =
Pr*Se + (1-Pr)*(1-Sp) (1-Pr)*Sp + Pr*(1-Se)
16
 Real applications in the “field”
“farmers & veterinarians”
1. Prevalence estimate (risk analysis)
“Apparent” Se / Sp “True”
prevalence (APr) prevalence (TPr)

2. Decision making
Positive Predictive Value (PPV)
DSe
DSp Given test is positive, what probability to be diseased?
Prevalence Negative Predictive Value (NPV)
Given test is negative, what probability to be non-diseased?

17
 Exercise (3):
Predictive value
Clarice is a 3 year-old cow showing
clinical signs of mastitis. The farmer
and the vet are interested
in determining if she is infected with Staph. aureus. The PCR test
used assures 99 % accuracy (assumed 99% of DSe and 99% DSp).
The prevalence of cows infected with Staph. aureus and
presenting with clinical signs of mastitis of is 25%.
Define and compute the following values:
PPV =

NPV =

18
 Real applications in the “field”
“farmers & veterinarians”
1. Prevalence estimate (risk analysis)
“Apparent” Se / Sp “True”
prevalence (APr) prevalence (TPr)

2. Decision making
Positive Predictive Value (PPV)
DSe
DSp Given test is positive, what probability to be diseased?
Prevalence Negative Predictive Value (NPV)
Given test is negative, what probability to be non-diseased?
3. Surveillance Program
Computation of sample size to declare a population ‘free of disease’
n / Se

19
QUESTIONS ?

20
21