Diagnostic Test Design

MS Anam

Child Health Department

Diponegoro University/Kariadi Hospital
Semarang
 Why we need a diagnostic test?

 We need “information” to make a decision

 “Information” is usually a result from a test
 Medical tests:
To screen for a risk factor (screen test)
To diagnosse a disease (diagnostic test)
To estimate a patient’s prognosis (pronostic test)
 When and in whom, a test should be done?
When “information” from test result have a value.
 Value of a diagnostic test

 The ideal diagnostic test:

Always give the right answer:
Positive result in everyone with the disease
Negative result in everyone else
Be quick, safe, simple, painless, reliable & inexpensive
 But few, if any, tests are ideal.
 Thus there is a need for clinically useful
substitutes
 Is the test useful ?

 Reproducibility (Precision)
 Accuracy (compare to “gold standard”)
 Feasibility
 Effects on clinical decisions
 Effects on Outcomes
 Determining Usefulness
of a Medical Test

Question Possible Designs Statistics for

Results
1. How Studies of: Proportion
reproducible - intra- and inter agreement,
is the test? observer & kappa, coefficient
- intra- and inter of variance, mean
laboratory & distribution of
differences (avoid
variability correlation
coefficient)
 Determining Usefulness
of a Medical Test

Question Possible Designs Statistics for

Results
2. How Cross-sectional, case- Sensitivity,
accurate is control, cohort-type specificity,
the test? designs in which test PV+, PV-,
result is compared ROC curves,
with a “gold standard” LRs
 Determining Usefulness
of a Medical Test

Question Possible Statistics for Results

Designs
3. How Diagnostic Proportion abnormal,
often do yield studies, proportion with
test results studies of pre- discordant results,
affect & post test proportion of tests
clinical clinical leading to changes in
decisions? decision clinical decisions; cost
making per abnormal result or
per decision change
 Determining Usefulness
of a Medical Test

Question Possible Statistics for Results

Designs
4. What are Prospective or Mean cost, proportions
the costs, retrospective experiencing adverse
risks, & studies effects, proportions
acceptability willing to undergo the
of the test? test
 Determining Usefulness
of a Medical Test

Question Possible Designs Statistics for

Results
5. Does Randomized trials, cohort Risk ratios, odd
doing the or case-control studies in ratios, hazard
test which the predictor ratios, number
improve variable is receiving the needed to treat,
clinical test & the outcome rates and ratios
outcome, includes morbidity, of desirable
or having mortality, or costs related and
adverse either to the disease or to undesirable
effects? its treatment outcomes
 Common Issues for
Studies of Medical Tests
 Spectrum of Disease Severity and Test Results:
Difference between Sample and Population?
Almost tests do well on very sick and very well
people.
The most difficulty is distinguishing Healthy & early,
presymtomatic disease.
 Subjects should have a spectrum of disease
that reflects the clinical use of the test.
 Common Issues for
Studies of Medical Tests
 Sources of Variation:
Between patients
Observers’ skill
Equipments
=> Should sample several different institutions to
obtain a generalizable result.
 Common Issues for
Studies of Medical Tests
 Importance of Blinding: (if possible)
Minimize observer bias
Ex. Ultrasound to diagnose appendicitis
(It is different to clinical practice)
 Studies of Diagnostic tests

 Studies of Test Reproducibility

 Studies of The Accuracy of Tests
 Studies of The Effect of Test Results on Clinical
Decisions
 Studies of Feasibility, Costs, and Risks of Tests
 Studies of The Effect of Testing on Outcomes
 Studies of Test Reproducibility

 The test is to test the precision

Intra-observer variability
Inter-observer variability
 Design:
Cross-sectional design
Categorical variables: Kappa
Continuous variables: coefficient of variance
 Compare to it-self (“gold standard” is not
required)
Studies of the Accuracy of Tests

 Does the test give the right answer?

 “Tests” in clinical practice:
Symptoms
Signs
Laboratory tests
Imagine tests
To find the right answer.
“Gold standard” is required
 How accurate is the test?

 Validating tests against a gold standard:

 New tests should be validated by comparison
against an established gold standard in an
appropriate subjects
 Diagnostic tests are seldom 100% accurate
(false positives and false negatives will occur)
 Validating tests against a gold
standard
 A test is valid if:
It detects most people with disorder (high Sen)
It excludes most people without disorder (high Sp)
a positive test usually indicates that the disorder is
present (high PV+)

 The best measure of the usefulness of a test is

the LR: how much more likely a positive test is
to be found in someone with, as opposed to
without, the disorder
A Pitfall of Diagnostic test

A test can separate the very sick from the very

healthy does not mean that it will be useful in
distinguish patients with mild cases of the
disease from others with similar symptoms
 Sampling

 The spectrum of patients should be

representative of patients in real practice.
 Example: Which is better? What is the limits?
Chest X-ray to diagnose aortic aneurism (AA).
Sample are 100 patients with and 100 without AA
that ascertained by CT scan or MRI.
FNA to diagnose thyroid cancer. 100 patients with
nodule > 3cm and had indication to thyroidectomy
(biopsy was the gold standard).
 “Gold standard”

 “Gold standard” test: often confirm the

presence or absence of the disease : D(+) or
D(-).
 Properties of “Gold standard”:
Ruling in the disease (often doing well)
Ruling out the disease (maybe not doing well)
Feasible & ethical ? (ex. Biopsy of breast mass)
Widely acceptable.
 The test result

 Categorical variable:
Result: Positive or Negative
Ex. FNA cytology
 Continuous variable:
Next step is: find out “cut-off point” by ROC curve
Ex. almost biochemical test: pro-BNP, TR-Ab,..
 Analysis of Diagnostic Tests

How accurate is the test?

 Sensitivity & Specificity
 Likelihood ratio: LR (+), LR (-)
 Posterior probability (Post-test probability) /
Positive, Negative Predictive value (PPV, NPV);
given Prior probability (Pre-test probability)
 Sensitivity and Specificity

a
Sens  Disease D
ac “Gold standard”
Test
Result + -
d + a b
Spec 
bd - c d
Positive & Negative Predictive Value

 PV (+): positive
predictive value Test Disease D
 PV (-): negative Result + -
predictive value + a b
a - c d
PV () 
ab
a /(a  c)
LikelihoodRatio( LR) 
d b /(b  d )
PV () 
cd
 Posterior odds

When combined with information on the prior

probability of a disease*, LRs can be used to
determine the predictive value of a particular test
result:
Posterior odds = Prior odds x Likelihood ratio

*expressing the prior probability [p] of a disease as the prior odds [p/(1-p)] of
that disease. Conversely, if the odds of a disease are x/y, the probability of the
disease is x / (x + y)
 Choice of a cut-off point
for continuous results

Consider the implications of the two possible

errors:
 If false-positive results must be avoided (such as
the test result being used to determine whether
a patient undergoes dangerous surgery), then
the cutoff point might be set to maximize the
test's specificity
 If false-negative results must be avoided (as
with screening for neonatal phenylketonuria),
then the cutoff should be set to ensure a high
test sensitivity
 Choice of a cut-off point
for continuous results

 Using receiver operator characteristic (ROC)

curves:
Selects several cut-off points, and determines the
sensitivity and specificity at each point
Then, graphs sensitivity (true-positive rate) as a
function of 1-specificity (false-positive rate)

 Usually, the best cut-off point is where the ROC

curve "turns the corner”
 RECEIVER OPERATING
CHARACTERISTIC (ROC) curve

 ROC curves (Receiver

Operator Characteristic)
 Ex. SGPT and Hepatitis Sensitivity

SGPT D+ D- Sum 1

< 50 10 190 200

50-99 15 135 150
100-149 25 65 90
150-199 30 30 60
200-249 35 15 50
250-299 120 10 130
>300 65 5 70 1
1-Specificity
Sum 300 450 750
 Self Learning

 Search one article, the best you can get, about

diagnostic test
 Make the PICO
 What is “Gold Standards”
 What is diagnostic table
 Sensitivity ?
 Specificity ?
 NPV
 PPV
 LR (+)
 LR (-)
THANK YOU