You are on page 1of 68

Diagnosis: Highlights

Cita Rosita Sigit Prakoeswa


Department of Dermato Venereology Dr Soetomo Hospital –
Faculty of Medicine, Airlangga University, Surabaya
Tropical Disease Center, Airlangga University, Surabaya
What is diagnosis ?
• Increase certainty about
presence/absence of
disease
• Disease severity
• Monitor clinical course
• Assess prognosis –
risk/stage within diagnosis
• Plan treatment
• Screening
• Epidemiology
Knottnerus, BMJ 2002
By the end of this session,
you should be able to….

• describe and illustrate key measures of


diagnostic test performance
• represent diagnostic test performance
EBM Process
Drawing conclusion Patient Diagnosis
That impact on practice Therapy
•DOEs
Encounter Prognosis
•POEMs Etiology

Appraising the Formulating the


Evidence Clinical Question

•Patient
•Hierarchy of evidence •Intervention
•Pre appraised resources •Comparison
Searching the •Outcome
Evidence
(Lang, 2000) 4
What should I do
about this condition INTERVENTION
or problem?

What cause PROGNOSIS/RISK FACTORS


the problem?

Does this person


have the condition DIAGNOSIS
CLINICAL or problem?

QUESTION Who will get


PROGNOSIS FACTORS
the condition
or problem?

How common FREQUENCY & RATE


is the problem?

What are the PHENOMENA / THOUGHTS


type of problem? 5
ACQ Diagnosis (PICO)

Patient / Intervention
Problem / Comparison Outcome
Population (Index)

In an otherwise how does the compare to in diagnosing


healthy 7-year- clinical exam throat culture GAS infection?
old boy with
sore throat
Controlled?
Randomized?

Longitudinal

Cross-sectional

Research Research Researcher


Approach 3 2 Goal 4 Involvement
1
Research 7
Clinical Manifestation / Diagnosis / Prognosis / Therapy / Review
Focus
Hierarchy of study designs

8
Basic Principles (1)
• Ideal diagnostic tests – right answers:
– (+) results in everyone with the disease and
– ( - ) results in everyone else
• Usual clinical practice:
– The test be studied in the same way it would
be used in the clinical setting
• Observational study, and consists of:
– Predictor variable (test result)
– Outcome variable (presence / absence of the
disease)
Basic Principles (2)
• Sensitivity, specificity
• Prevalence, prior probability, predictive values
• Likelihood ratios
• Dichotomous scale, cutoff points (continuous
scale)
• Positive (true and false), negative (true & false)
• ROC (receiver operator characteristic) curve
What is the reason that there are
many parameters in diagnostic test?
Prevalence Disease Disease
Total
Sensitivity (%) (+) (-)
Specificity (%) False
Test True pos
LR+ (+) a
pos a+b
b
LR- False
PPV (%) Test neg
True neg
c+d
(-) d
NPV (%) c

Pre-test Odds Total a+c b+d


a+b+
Post-test Odds c+d
Pre-test Probability (%)
Post-test Probability (%)
METHOD 1:
NATURAL FREQUENCIES TREE
Population
1.000
IN EVERY 1.000 PEOPLE, 200 WILL HAVE THE DISEASE

Population
1.000

Disease + Disease -
200 800

If these 1000 people are representative of the population at


risk, the assessed rate of those with the disease (20%)
represents the PREVALENCE of the disease – it can also be
considered the PRE-TEST PROBABILITY of having the disease
Sensitivity

– The proportion of people who truly


have a designated disorder who are
so identified by the test.
– Sensitive tests have few false
negatives.
– When a test with a high Sensitivity is
Negative, it effectively rules out the
diagnosis of disease. SnNout
Sensitivity
In other words, the Population
sensitivity is
190/200=95% 1.000

Disease + Disease -
200 800

Test + Test -
190 10

Test Alergi dengan Uji Kulit


Sensitivitas 95 %, artinya:
“SnNout”: bila hasil uji kulitnya (-): 95% “out” (dia bukan penderita alergi )
Specificity

– The proportion of people who are


truly free of a designated disorder
who are so identified by the test.
– Specific tests have few false positives
– When a test is highly specific, a
positive result can rule in the
diagnosis. SpPin
Specificity
Population In other words, the
specificity is 768/800
1000 = 96%

Disease + Disease -
200 800

Test + Test - Test + Test -


190 10 32 768

Test Alergi dengan Uji Kulit


Spesifitas 96 % artinya:
“SpPin”: bila hasil uji kulitnya (+): 96% “in” (dia penderita alergi)
“Sensitivity & Specificity”
NON-CASES CASES
FALSE
NEGATIVES Test cut-off
FALSE
POSITIVES
% of Group

NON-DISEASED

DISEASED

Negative Positive
Degree of ‘positivity’ on test

Numeric? (complex)
“Sensitivity & Specificity”
• Sensitivity and Specificity are usually
considered properties of the test rather
than the setting, and are therefore
usually considered to remain constant.

• However, sensitivity and specificity are


likely to be influenced by complexity of
differential diagnoses and a multitude of
other factors (spectrum bias).
“Sensitivity & Specificity”
“Positive & Negative Predictive Value”

• For sensitivity and specificity, the


reference variable (‘denominator’) is the
DISEASE
• For predictive value, the reference
variable (‘denominator’) is the TEST
Pre Test & Post Test Probability
• Pre-test Probability
– The probability of the target condition
being present before the results of a
diagnostic test are available. (prevalence)
• Post-test Probability
– The probability of the target condition
being present after the results of a
diagnostic test are available.
(Positive Predictive Value)
“Positive Predictive Value”
Population
This is also the POST-
1000 TEST PROBABILITY of
having the disease

Disease + Disease -
200 800

POSITIVE
Test + Test + PREDICTIVE
VALUE = 190/222
190 32
=86 %

Test - Test -
10 768

Test Alergi dengan Uji Kulit


PPV 86 % artinya bila hasil uji kulitnya (+): kemungkinan dia
menderita alergi adalah 86%
“Negative Predictive Value”
Population
1000

Disease + Disease -
200 800

Test + Test +
190 32
NEGATIVE
PREDICTIVE Test - Test -
VALUE = 768/778
=99% 10 768

Test Alergi dengan Uji Kulit


NPV 99 % artinya bila hasil uji kulitnya (-): kemungkinan dia
tidak menderita alergi adalah 99 %
“Positive & Negative
Predictive Value”

• The Positive Predictive Value of a test


will vary (according to the prevalence
of the condition in the chosen setting)
Predictive value & changing prevalence

Population
10.000

Disease + Disease -
200 9.800

Prevalence reduced by an order


of magnitude from 20% to 2%
Predictive value & changing prevalence

Sensitivity and Population


Specificity 10.000
unchanged

Disease + Disease -
200 9.800

Test + Test +
190 392

Test - Test -
10 9.408
Positive predictive value
at low prevalence
Population
Previously, PPV
10.000
was 86%

Disease + Disease -
200 9.800

Test + Test + POSITIVE


PREDICTIVE
190 392 VALUE = 33%

Test - Test -
10 9.408
Negative predictive value
at low prevalence
Population
Previously, NPV
10.000
was 99%

Disease + Disease -
200 9.800

Test + Test +
190 392

NEGATIVE Test - Test -


PREDICTIVE
VALUE >99% 10 9.408
Prediction of low prevalence events

• Even highly specific tests, when applied


to low prevalence events, yield a high
number of false positive results
• Because of this, under such
circumstances, the Positive Predictive
Value of a test is low
• However, this has much less influence
on the Negative Predictive Value
Likelihood Ratio
– Relative likelihood that a given test would be
expected in a patient with (as opposed to one
without) a disorder of interest.

probability (%) of a test result in patients with disease


LR=
probability (%) of the test result in patients without disease
Likelihood
Population
1000

Disease +
200 The likelihood that
someone with the
disease will have a
Test +
positive test is
190 190/200 or 95%
This is the same as
Test - the sensitivity
10
Likelihood
Population
1000

Disease -
800
The likelihood that
someone without
the disease will Test +
have a positive test 32
is 32/800 or 4%
This is the same as Test -
the (1-specificity)
768
Likelihood Ratio
LIKELIHOOD OF POSITIVE TEST
LIKELIHOOD = GIVEN THE DISEASE
RATIO + (LR+)
LIKELIHOOD OF POSITIVE TEST
IN THE ABSENCE OF THE DISEASE
SENSITIVITY 0.95
= = = 23.8
1- SPECIFICITY 0.04
A Likelihood Ratio (LR) of 1.0
indicates an uninformative test (occurs when sensitivity and specificity
are both 50%)
The higher the Likelihood Ratio
the better the test (other factors being equal)

Test Alergi dengan Uji Kulit


LR+=23,8, artinya bila hasil uji kulitnya (+): hasil (+) ini dapat terjadi 23,8
kali lebih besar terjadi pada “penderita alergi” dibandingkan dengan yang
“bukan penderita alergi”
The diagonal line (representing Sensitivity=0.5
and Specificity=0.5) represents performance no
better than chance

RECEIVER OPERATING CHARACTERISTIC CURVE


Overall shape is
predicted by the
reciprocal relationship
between sensitivity and
specificity

The closer the curve gets


to Sensitivity=1 and
Specificity=1, the better
the overall performance
of the test

Hence the area under the


curve gives a measure of
the test’s performance

FALSE POSITIVE RATE (1-Specificity)


AREA UNDER ROC CURVES

100
Sensitivity

Sensitivity and specificity both


100% - TEST PERFECT
AREA=1.0

0 Sensitivity and specificity both


1-Specificity 50% - TEST USELESS

100
Sensitivity

The area under a ROC


AREA=0.5
curve will be between
0
0.5 and 1.0
1-Specificity
AREA UNDER ROC CURVES

100
Area = 0.7 (between
0.5 and 1.0)
Sensitivity

0
1-Specificity

• Consider (hypothetically) two patients drawn


randomly from the DISEASE+ and DISEASE- groups
respectively
• If the test is used to guess which patient is from the
DISEASE+ group, it will be right 70% of the time
RECEIVER OPERATING
CHARACTERISTIC (ROC) CURVE
100 This study compared
90 the performance of a
80 dementia screening test
70 in a community sample
Sensitivity

60 (ACAT) and a memory


50 clinic sample (MC)
40
30
20 ACAT
Flicker L, Loguidice D, Carlin
10 MC JB, Ames D. The predictive
0 value of dementia screening
0 20 40 60 instruments in clinical
populations. International
1-Specificity Journal of Geriatric
Psychiatry 1997 ; 12 : 203-
209
Diagnostic tests …
Is not about finding absolute truth, but
about limiting uncertainty
establishes both the necessity and the
logical base for introducing probabilities,
pragmatic test-treatment thresholds ..\

Start thinking about


• what you’re going to do with the results of the
diagnostic test, and
• whether doing the test will help your patients
Interpreting Diagnostic Studies

VIA - RaMMbo
Validity
Selection? VALIDITY
QUESTION:

Representative?
Participants

Index group (IG) & I C


Gold standard G G Reproducible
Comparison Group (CG) Maintain?

+ -
Outcome
+ A B
Measurements
- C D blind subjective? OR
objective?
Diagnostic Accuracy Study:
Basic Design

Series of patients

Index test

Reference standard

Blinded cross-classification
Recruitment:
Was diagnostic test evaluated is representative
spectrum of patient?

Series of patients

Index test

Reference standard

Blinded cross-classification
Maintenance:
Was the endpoint of the reference standard
obtained for all subjects?

Series of patients

Index test

Reference standard

Blinded cross-classification
Measurement:
Were the assesors kept blind to the results of each
test and/or were the reference standard endpoint
objective

Series of patients

Index test

Reference standard

Blinded cross-classification
Spectrum Bias

Selected Patients

Index test

Reference standard

Blinded cross-classification
Verification Bias

Series of patients

Index test

Reference standard

Blinded cross-classification
Differential Reference Bias

Series of patients

Index test

Ref. Std A Ref. Std. B

Blinded cross-classification
Observer Bias

Series of patients

Index test

Reference standard

Unblinded cross-classification
Importance
What should I do
I INTERVENTION about this condition
or problem?
M What cause
ETIOLOGY/RISK FACTORS
P the problem?

O DIAGNOSIS
Does this person
have the condition
R or problem?
Who will get
T PROGNOSIS & PREDICTION the condition
or problem?
A
N FREQUENCY & RATE
How common
is the problem?
C
What are the
E PHENOMENA / THOUGHTS
type of problem? 53
I RRR, ARR, NNT
CLINICAL TRIAL
M p & CI
P
O
Survival curve
R PROGNOSIS RR / OR
T p & CI
A
N
C DIAGNOSTIC
Sn,Sp,LH,PPV,NPV
p & CI
E 54
Applicability
PICO & Applicability

Applicability
Your question
(PICO)
Study What do the
Result mean?
Importance

How well was


study done?
56
Validity
CRITICAL
APPRAISAL
DIAGNOSTIC
TEST
Critical appraisal diagnostic test

• Use worksheet (VIA; RAMMbo)


• STARD
• Use supporting softwares
– CAT Maker
Validity (1)
Apakah penelitian uji diagnostik dilakukan secara tersamar dengan baku
emas yang benar ?

Validity (2)
Apakah uji diagnostik dilakukan terhadap pasien dengan spektrum
penyakit atau kelainan yang memadai sehingga dapat diterapkan dalam
praktek sehari-hari?

Validity (3)
Apakah pemeriksaan dengan baku emas dilakukan tanpa memandang
hasil pemeriksaan dengan uji diagnostik ?
Important
Berapa Sn, Sp, LR+, LR-, PPV, NPV, Pre-test probability, Post-test
probability, Pre-test Odds, Post-test Odds ?
Applicable (1)
Apakah uji diagnostik tersebut tersedia, terjangkau dan akurat?

Applicable (2)
Apakah kita bisa memperkirakan pre-test probability (prevalens)
penyakit pada pasien kita ?

Applicable (3)
Apakah post-test probability yang dihitung akan mengubah tatalaksana
pasien kita?

Applicable (4)
Apakah secara keseluruhan uji diagnostik tersebut bermanfaat bagi
pasien ?
STARD initiative (25 items)
Standards for Reporting of Diagnostic Accuracy

• Section and and  Results


topic ◦ Participants
◦ Test results
– Title, abstract, and ◦ Estimates
keywords ◦ Discussions
– Introduction
• Methods
– Participants
– Test methods
– Statistical methods
Bossuyt PM, Reitsma JB, Bruns, DE, Gatsonis CA, Glasziou PP et al. BMJ 2003,326:41-6
1st component of STARD
2nd component of STARD
Guides for deciding whether a screening or
early diagnostic maneuver does more good
than harm:
 Does early diagnosis really lead to improved
survival, or quality of life, or both?
 Are the early diagnosed patients willing partners in
the treatment strategy?
 Is the time and energy it will take us to confirm the
diagnosis and provide (lifelong) care well spent?
 Do the frequency and severity of the target disorder
warrant this degree of effort and expenditure?