Diagnosis overview-RCE Primer - Part 1

A Primer on the Precision and
Accuracy of Diagnostic Tests
The Rational Clinical Examination

Copyright © American Medical Association. All rights reserved. | JAMA | The McGraw-Hill Companies, Inc.
Overview
• Decision Threshold • Guides for
• Key Concepts Reading Articles
• Precision on Clinical Exam
• Accuracy • Quality of the
• The Posterior Odds Evidence
Equation
• Prior Probability • The Bottom Line
• Likelihood Ratios
• Confidence
Intervals
• Meta-analysis
Why Conduct a Diagnostic Test?
• We perform a diagnostic test to affect
what we do, think, or say
• The decision threshold

• A starting point for decisions
• How certain we need to be about a
diagnosis before we act
• Involves incorporation of values and
preferences

Is a Diagnostic Test Valid?
• Precision
• Do findings remain consistent with repeat
examinations?
• Accuracy
• Do findings correctly predict diagnosis of
disease?
• Methodologic standards for study design

• Critical appraisal guides

Precision and Accuracy

X
X
X
X
X X
X
X X

XX X
X
X X
X X X
X

XX
XXXXX
XX
X

XX
XXXXX
XX
X

Precision
Observer Variation
• Intraobserver agreement
Does the same clinician get the same result when
repeating a symptom or sign on a patient who is
clinically unchanged?
• Interobserver agreement
Do 2 or more observers agree on the presence or
absence of a finding in a patient who experienced no
change in condition?
• Kappa ()
Agreement beyond chance and can be used to describe
both intra- and interobserver agreement

Kappa: Agreement Beyond Chance
• Kappa () values

• Actual agreement beyond chance / potential
agreement beyond chance
• Ranges for agreement

• 0.0-0.2 slight
• 0.21-0.4 fair
• 0.41-0.6 moderate
• 0.61-0.8 substantial
• 0.81-1.0 perfect

Maximum potential agreement = 100%
0 20 40 60 80 100
40%
Possible agreement beyond chance
Chance agreement
60% 40%
20%
Observed agreement beyond chance
Observed agreement: 80%
60% 20%
0 20 40 60 80 100
20% Observed agreement beyond chance

Kappa = = 0.5
40% Possible agreement beyond chance
Moderate
agreement
40%
Agreement beyond chance
20%
60% Chance agreement

Quantifying Test Performance
• Sensitivity and Specificity
• Likelihood Ratio
• Positive and Negative Predictive Value
• Accuracy or Diagnostic Odds Ratio

Test Characteristics
• Sensitivity
• T+/D+
• SnN(–)ow: Negative result lowers post-test
probability (Does Not rule out)
• Specificity
• T-/D-
• SpP(+)in: Positive result increases post-
test probability (does Not rule in)

Why Focus on Likelihood Ratio?
• Can be applied clinically to individual patient
circumstance
• Can be used to change pretest probability to

posttest probability
• Calculated for each test result as the ratio of

those with disease to those without
• Can be calculated for multilevel outcomes

Test Characteristics
• Likelihood Ratio
• Determined for a particular test result
• Conceptually related to how much more

(LR > 1) or less (LR < 1) likely the disease
is, given a particular test result
• LR = 1 will not change your assessment of

the likelihood of disease, given a particular
test result

LR for positive test result
LR+ = T+/D+//T+/D-
Likelihood of having a positive test in those who
have the disease versus having a positive test
in those without the disease
Ideally:
T+/D+ = 100%
T+/D- = 0
Perfect LR+ = infinity
LR for negative test result
LR- = T-/D+//T-/D-
Likelihood of having a negative test result in
those who have the disease versus having a
negative test result in those without the disease
Ideally:
T-/D+ = 0
T-/D- = 100%
Perfect LR- = 0
Using Likelihood Ratios
Values of Likelihood Ratio: How much do they
affect probability of disease?
• LR = 1 No effect on likelihood
• LR = 3-10 Disease More Likely

• LR = 0.3-0.1 Disease Less Likely
• LR > 10 Disease More Likely

• LR < 0.10 Disease Less Likely

LR: Impact on Likelihood of Disease
LR = 0.01 LR = 100
LR = 0.1 LR = 10
Less Less
LR = 0.2 LR = 5
More More
Less LR = 0.3 LR = 3 More
Likely Likely Likely Likely
Likely Less More Likely
0 Likely Likely 
Increasing impa ct increasi ng impac t
LR = 1
No
Impact on
Likelihood of
Disease
LR: Impact on Likelihood of Disease
LR = 0.01 LR = 100
LR = 0.1 LR = 10
More More
LR = 0.2 LR = 5
More More
More LR = 0.3 LR = 3 More
Impact Impact LR = 1 Impact Impact
Impact More More Impact
No
0 Impact Impact 
Increasing impact
Impact
increasi ng impac t

The Posterior Odds Equation
Prior odds × LR = Posterior odds

Pretest Probability
• Prevalence: Probability of disease
before we apply a screening test, in this
case clinical examination.
• Disease prevalence can be found when

existing studies describe their study
populations.

Pretest Probability vs Odds
• Pretest Probability: The proportion of individuals in
the starting population with disease
# With disease/total number in the population
• Pretest Odds: The proportion of people with disease

compared to the proportion of people without disease
Proportion with disease/proportion without disease

Pretest Probability vs Odds
For those who prefer formulas
Step 1: Begin with probability and convert to odds

Odds = probability/(1 − probability)
Step 2: Take odds and multiply by LR

Pretest odds × LR = posttest odds
Step 3: Convert odds back to probability

Probability = odds/(odds + 1)

Probability vs Odds
25% • Odds 0.25/0.75 = 0.33

• Probability 25%
75% Disease
No disease

Posterior Probability vs Odds
• LR of 3 • Odds = 3 × 0.33 = 1
• Probability 50%
25%
75% Disease
No disease

Nomogram for Interpreting LR
• Plot patient’s pretest

probability on left
• Draw straight line
through LR for given
test result
• Line points to
posttest probability


Diagnosis overview-RCE Primer - Part 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Diagnosis overview-RCE Primer - Part 1

Uploaded by

Copyright:

Available Formats

A Primer on the Precision and

Accuracy of Diagnostic Tests

The Rational Clinical Examination

• The decision threshold

The Rational Clinical Examination

• Methodologic standards for study design

The Rational Clinical Examination

The Rational Clinical Examination

The Rational Clinical Examination

The Rational Clinical Examination

The Rational Clinical Examination

The Rational Clinical Examination

The Rational Clinical Examination

• Kappa () values

• Ranges for agreement

The Rational Clinical Examination

20% Observed agreement beyond chance

Agreement beyond chance

The Rational Clinical Examination

• Sensitivity and Specificity

• Positive and Negative Predictive Value

• Accuracy or Diagnostic Odds Ratio

The Rational Clinical Examination

The Rational Clinical Examination

• Can be used to change pretest probability to

• Calculated for each test result as the ratio of

• Can be calculated for multilevel outcomes

The Rational Clinical Examination

• Conceptually related to how much more

• LR = 1 will not change your assessment of

The Rational Clinical Examination

• LR = 3-10 Disease More Likely

• LR > 10 Disease More Likely

The Rational Clinical Examination

The Rational Clinical Examination

Prior odds × LR = Posterior odds

The Rational Clinical Examination

• Disease prevalence can be found when

The Rational Clinical Examination

# With disease/total number in the population

• Pretest Odds: The proportion of people with disease

Proportion with disease/proportion without disease

The Rational Clinical Examination

Step 1: Begin with probability and convert to odds

Step 2: Take odds and multiply by LR

Step 3: Convert odds back to probability

The Rational Clinical Examination

25% • Odds 0.25/0.75 = 0.33

The Rational Clinical Examination

The Rational Clinical Examination

• Plot patient’s pretest

The Rational Clinical Examination

You might also like