What is EBM?

“The conscientious, explicit and judicious

use of current best evidence in
making decisions about the care of
individual patients”

Prof. David L. Sackett, 1997

Why EBM?
 Primum non nocere
“First do no harm”

Hippocrates, Epidemics
 Sources of Evidence in Medicine

• Traditional Teaching
• Textbooks
• Basic sciences
• Observational studies
• Computer simulation
• Decision Analysis
• Case-Control Studies
• Randomised Controlled Trials (RCT)0
• Meta-analyses
 RCOG Classification of Evidence Levels

1++ High quality meta-analyses of RCT's

1+ Meta-a. Or RCT's at low risk of bias

1- Meta-a. Or RCT's at high risk of bias

2++ High quality meta-analyses of CC's

2+ Well-conducted cc or cohort studies

2- Case-control or cohort studies with ? bias

3 Case reports

4 Expert opinion
Archie Cochrane (1909-88)

“Effectiveness and
Efficiency: Random
Reflections of Health
Services “, 1971
Randomized Controlled Trials

“Gold standard” in evaluating new

therapies or surgical techniques

May also be applied to new diagnostic
tests
 Randomized Controlled Trials
Objectives of RCT:

Minimize bias by randomisation

Achieve statistical power through
adequate sample size

“Blinding “ - single or double

Analysis by intention to treat
 Randomized Controlled Trials

Randomisation

Several techniques available

Computer software linked to central
monitoring station

“Block “ randomisation

Sealed envelope method
 RCT’s and Observational Studies

• Two studies published in the NEJM in 2000 suggested that

RCTs and observational studies overall produced similar
results

• JAMA 2001: “discrepancies beyond chance do occur and

differences in estimated magnitude of treatment effect are
very common”

• RCTs may be unnecessary for treatments that have

dramatic and rapid effects relative to the expected
 RCT’s and Industry Funding

•RCT’s funded by industry are significantly more likely to

report positive results

•Possibly due to publication bias

•RCTs may be unnecessary for treatments that have

dramatic and rapid effects relative to the expected
 RCT’s and Statistical Error

• Type I error – “false positive”

• Type II error – “false negative”

• Sample size calculations often inaccurate

Diagnostic Tests
 2 X 2 Table

Disease Disease
present absent
Test a b
Positive
Test c d
Negative
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Sensitivity = TP =
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Sensitivity = TP = a/(a+c)
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

False Positive Rate = FP =

 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

FP = b/(d+b)
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Specificity = 1 - FP =
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Specificity = 1 - FP = d/(d+b)
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Positive predictive value (PPV) =

 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

PPV = a/(a+b)
 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

Negative predictive value (NPV) =

 2 X 2 Table
Disease Disease
present absent
Test a b
Positive
Test c d
Negative

NPV = d/(c+d)
 PPV and Prevalence

Steep drop in positive predictive

value as disease prevalence
decreases
 The Likelihood Ratio

Single value to indicate the

clinical utility of a test

Independent of prevalence

LR = Sensitivity/(1- Spec.)

LR >8 : tests usually clinically
useful
 The Likelihood Ratio

LR is an odds modifier:

Posterior odds =
prior odds x LR
Odds and Probability

Inter-convertible:

Odds = p/(1-p)
Can tests be combined ?

Rare conditions: high rates of

false positives

Lead to excessive unnecessary
intervention

Can be reduced by combining
tests e.g. intrapartum
monitoring
 Impact of new diagnostic
test on clinical outcomes:

RCT

Cohort study

Case-control study

Before and after study
SYSTEMATIC REVIEWS
"It is surely a great criticism of our
profession that we have not organised
a critical summary, by specialty or
subspecialty, adapted periodically, of
all relevant randomized controlled
trials."
Archie Cochrane, 1972
 Role of systematic reviews
 Before commencing a new project: to determine
whether further studies are really indicated: ‘state-
of-the-art’ literature review.
 Gain in statistical power for average estimates.
 'Cumulative' meta-analysis can determine when
further studies are no longer indicated.
 Design of subsequent studies.
 Setting policy for treatment and health care –
making the best use of the data available.
 Can Studies be Combined?

 Identification of optimal inclusion criteria can be

difficult.
 The most critical step is choosing the appropriate
research question.
 A fairly general question is more preferable to a very
specific one.
 Tukey : "...far better an approximate answer to the
right question, than an exact answer to the wrong
question.."
 Publication Bias

 Entire research studies may fail to reach publication

because of the nature of the results.

 Identification of unpublished trials can be very

difficult - in one study it accounted for 22% of the
papers included in the meta-analysis.

 Failure to publish rests with the investigators rather

than editors.
 PREDICTIVE ABILITY OF META-ANALYSES
Villar et al, Lancet 1995

 Comparison of the meta-analyses of smaller studies

with the corresponding result of the largest study.
 30 meta-analyses including a total of 185 randomised
controlled studies (RCT) obtained from the Cochrane
pregnancy and childbirth database. The meta-
analyses were only included if they had at least one
trial with a total sample size of over 1000.
 Calculations differ from the Cochrane database in
that the largest trial was excluded, this being used as
the 'gold standard' for outcome
The Cochrane Collaboration

• Established in 1993 by Sir Iain Chalmers

• International: 100 countries
• Independent
• Not-for-profit
• Over 27000 contributors
 RANZCOG and EBM

“RANZCOG endorses the principles of Evidence-

based Medicine and recognizes the NHMRC levels of
evidence and grades of recommendations”

College Statement C-Gen 15, Nov. 2009

…a scientific idea can never be proven true,
because no matter how many observations seem to
agree with it, it may still be wrong. On the other
hand, a single contrary experiment can prove a
theory forever false

Sir Karl Popper