How to appraise

Treatment Study

MUHAMMAD ALI
FM USU
1
 Principles in critical appraisal
Critical appraisal of primary research (clinical trial) involves
three overall questions:

• Question 1: What is the PICO of the study, and

is it close enough to your PICO?

• Question 2: How well was the study done?

• Question 3: What do the results mean and could they

have been due to chance?

2
 MAN IN EBP…

Sir Archie Cochrane

Dave Sackett
?
3
 Steps in EBP
1. Formulate clinical problems in answerable questions

2. Search the best evidence: use internet or other on-

line database for current evidence
VIA
3. Critically appraise the evidence for
 Validity (was the study valid?)
 Importance (were the results clinically important?)
 Applicability (could we apply to our patient?)

4. Apply the evidence to patient

4
Treatment
 Is this evidence about treatment valid?

1. Was the assignment of patients to treatment randomized?

2. Was the randomization conclealed?
3. Were the groups similiar at the start of the trial?
4. Was follow-up of patients sufficiently long and complete?
5. Were all patients analyzed in the groups to which they
were randomized?

Some finer points :

6. Were patients, clinicians, and study personnel kept blind to
treatment?
7. Were groups treated equally, apart from the experimental
therapy?
Is this valid evidence about treatment important?

1. What is the magnitude of the treatment effect?

2. How precise the estimate of the treatment

effect?
Is this valid and important evidence applicable to
our patient?

1. Is our patient so different from those in thr study that

its results cannot apply?
2. Is the treatment feasible in our setting?
What are our patient’s potential benefits and harms
from the therapy?
3. What are our patient’s potential benefits and harms
from the therapy?
4. What are our patient’s values and explectations for
both the outcome we are trying to prevent and the
treatment we are offering?
9
 Internal Validity: RAMMbo
•R Recruitment : were the subjects representative?
(See Methods)

•A Allocation : was the allocation randomised and

concealed? (See Methods)

•M Maintenance : did the groups have equal co-

intervention and adequate follow up?
( See Methods & Results)

•Mbo Measurement, blinding, objectivity : were the subjects and

assessors kept ‘blind’ to which treatment
was being received and/or were the
measures ‘objective’ (See Methods)
10
•R Recruitment : Were the subjects representative of
the target population?

* The best way to ensure that study groups are representative is to:

@ Recruit potential subject sequentially and clearly describe the source

of patients
@ Only apply exclusion criteria that are relevant to study methods
@ See the size of study groups

* Where do I find the information?

Early in the Methods should tell you how patients were selected for
the study

11
•A Allocation : was the treatment allocation concealed before
randomisation and were the groups comparable at the
start of trial?

@ Match groups as closely as possible in every way except for the

intervention
@ Allocation concealment used for randomisation :
* centralised computer randomisation (the best one)
* sealed envelopes: not as good because the allocation is not as
well concealed

*Where do I find the information?

@ Methods: should tell you how patient were allocated and
whether or not randomisation was concealed
@ Results : see table of Baseline characteristics

12
•M Maintenance : Was the comparable status of the
study groups maintained through equal
management and adequate follow up?

Equal management:
@ the only difference between the groups is factor
being tested
@ use an identical measurement strategy for everyone

Adequate follow-up:
@ subjects at the start = subjects at the end
@ subject are analysed in the groups that they started

* Where do I find the information?

@ look in the Methods for the precise protocol
@ look in Results for any further information and flowchart study

13
•Mbo Measurement, blinding, objectivity :
Were the outcomes measured with blinded subjects and assessors
and/or objective measures?
‘Blinding’
Best -- Double blind trial : subject and investigators (outcome
assessors) both unware of group allocation
Moderate -- Single blind trial: either the subject or investigators are
unware of group allocation
Worst -- Not blinded: subject and investigators both aware of group
allocation

* Where I do find the information?

Methods section should describe how the outcome was assessed
and whether the assessor/s were aware of the patient’s treatment

14
 Question 3: What do the results mean and could
they have been due to chance?

* Outcome measures
1. Binary outcomes:
Relative Risk/RR, Absolute Risk Reduction/ARR
Relative Risk Reduction/RRR
Number Needed to Treat/NNT

2. Continuous outcomes

* Are the results real and relevant?

P-values ( hypothesis testing)
Confidence Interval/CI ( estimation)

15
16
THANK
YOU

17