CRITICAL APPRAISAL OF AN ARTICLE ABOUT THERAPY

CLINICAL SCENARIO You are working at the out-pa ent clinic in your ins tu on when a 55 year old
male bank execu ves came to your clinic for constric ng chest pain located at
the mid- sternum precipitated by exer on and relieved by rest. His blood
pressure was 130/80, heart was 85/minute and the respiratory rate was 20
beats/ minute. An ECG was done revealing lateral wall ischemia. You sent the
pa ent home on oral nitrates and aspirin. However, further work-ups revealed
hypercholesterolemia with a level of 6.5 mmol/liter. You gave dietary advice but
the pa ent claimed that he has been on a high ber diet with low fat intake for
the past 6 months.
(dilemma) He is now inquiring if he should take drug for his elevated
cholesterol.
SEARCH You decided to translate this clinical dilemma to an answerable ques on. You
decide that the ar cle should include a popula on of pa ents that has that has
an elevated cholesterol who have already undergone dietary therapy for at least
6 months to be consistent with your pa ent’s pro le. The drug interven on
should be compared with placebo. The ar cle must report clinically important
outcomes such as reduc on in cardiovascular deaths. Finally, you wanted an
ar cle that employed randomiza on.
The ar cle you nally retrieved included 4444 pa ents to either simvasta n or
placebo. However, before applying the results to your pa ent, you decided to
appraise the ar cle using the following guides.
CRITICAL APPRAISAL
PRIMARY VALIDITY 1. Was the assignment of pa ents to treatment randomized?
GUIDES In order to answer this ques on, the reader is advised to look into the ar cle’s
Abstract or methodology sec on. For the 4S study, randomiza on was done and
was wri en both in the abstract and method sec on.
The beauty of randomiza on is that if the sample size is su ciently large, it
assures that both the known and unknown determinants of outcome are evenly
distributed between the interven on and control groups. In the absence of
randomiza on, these factors might be di cult to control and might be the one
strongly in uencing the outcome rather than the treatment itself.

At mes though because of the rarity of the disease and small pa ent sample
size, randomiza on might not be feasible. In these cases, a clinician must rely on
weaker studies but should be aware of its poten als for errors.

2. Were all pa ents who entered the trial properly accounted for and a ributed
at its conclusion?
ti
ti
ti
tt
fl
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
fi
ffi
ti
ti
fi
ti
ti
ti
fi
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ffi
ti
ti
ti
tt
ti
ti
 a) Was follow-up complete?
This is best checked by looking at the number of pa ents enrolled at the outset
and comparing this with the number of pa ents reported in the result table.
Every pa ent who entered the trial should be accounted for at its conclusion. If
substan al numbers are” lost to follow -up”, the validity of conclusions are open
to ques on. A drop-out rate of 20% or more is frequently declared as
substan al. If the number lost to follow up is less than this, the reader can
decide if this e ects the conclusion by assuming a “worst case scenario”. This
means that the number lost in the treatment group are assumed to have bad
outcome and the number lost in the control group are assumed to have been
cured and if the conclusions di er, a substan al number was lost to follow-up.

Another way of deciding whether follow-up was complete is to check whether

an inten on to treat analysis was done. If this is reported one can safely assume
that follow-up was complete.
b) Were pa ents analyzed in the groups to which they were
randomized?
It simply means that those belonging to the control group or treatment group
are analyzed from the beginning to the end in the same grouping including
those who were dropped or withdrawn or change treatment. No crossing over
modali es were done as this would lead to biased results.
Excluding non-compliant pa ents from the analysis leaves behind those who
maybe des ned to have a be er outcome and destroys the unbiased
comparison provided by randomiza on. This principle of a ribu ng all pa ents
to the group to which they were randomized results in an “inten on- to- treat
analysis”. This strategy preserves the value of randomiza on: prognos cs factor
that we know about, will be, on average, equally distributed in the two groups
and the e ect will see will be just due to the treatment assigned.

SECONDARY VALIDITY 3. Were pa ents, their clinicians and study personnel “blind to treatment”?
GUIDES To answer this ques on the reader is again advised to look into the abstract or
methodology sec on.
Blinding is the process by which the interven on being given is concealed from
the pa ent, the clinicians and the one who analyzes the data. Pa ents, clinicians
and analyst are likely to have an opinion regarding the experimental treatment.
These opinions whether op mis c or pessimis c, can systema cally distort
repor ng of outcomes. As to avoid these, blinding is necessary.
4. Were the groups similar at the start of the trial?
ti
ti
ti
ti
ti
ti
ti
ff
ti
ff
ti
ti
tt
ff
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
tt
ti
ti
ti
ti
ti
ti
ti
ti
 To answer this ques on, one should look for the report of the comparison of the
baseline characteris cs of the experimental and control group.. For most
studies, this is labeled as table 1.
For reassurance about the study’s validity, readers would like to be informed
that the treatment and control groups were similar for all the factors that
determine the clinical outcomes of interest save for the experimental therapy.
The greater the similarity between known prognos c factors for the control and
experimental group, the more likely that the results can be a ributed to the
interven on rather than due to the di erences in these factors.
5. Aside from the experimental interven on, were the groups treated equally?
Interven ons other than the treatment under study, when di eren ally applied
to the treatment and control groups are called “co-interven ons”. This might
distort the results since they in themselves might cause changes in reported
outcomes.
OVERALL IS THE STUDY If you want to be strict about it you should answer yes in all 5
VALID ques ons. However, you as the user of the journal can make the decision. A
simple rule might be to answer yes to at least one primary guide and two
secondary guides.
WHAT ARE THE 1. How large was the treatment e ect?
RESULTS? Most randomized control trials report outcomes either as treatment success or
treatment failure and adverse events. Examples of outcomes include cure rates,
side e ects or deaths. Pa ents either do or do not su er these events and the
ar cle frequently reports the propor on of pa ents who develop such events. In
the 4S study, 11.5% died in the placebo group and 8.2% in the simvasta n group.
By eyeballing these gures, simvasta n seems be er in reducing events. But
how else could this gures be compared? The following simple computa ons
will help.
Risk in control (RC)=Death in control/N pa ents in the control group.
Risk in treatment (RT)=Death in the treatment group/N pa ents in the treatment
group.
Rela ve Risk (RR)=RT/RC=0.082/0.115=0.71
Rela ve Risk Reduc on (RRR)= 1-RR= 1-0.71= 0.29 (29%)
Absolute Risk reduc on (ARR)= RC-RT=0.115-0.082=0.033
Number Needed to Treat(NNT)=1/ARR=1/0.033=30.30
Absolute Risk Reduc on is the absolute di erence between the propor on who
died in the placebo group compared to the simvasta n group.
ti
ff
ti
fi
fi
ti
ti
ti
ti
ti
ti
ti
ti
ff
ti
ti
ti
ti
ti
tt
ti
ff
ti
tt
ti
ff
ti
ff
ti
ti
ti
ti
ff
ti
ti
 Rela ve Risk is the risk of events in the treatment group (simvasta n) rela ve to
the placebo group. The most useful measure to use in explaining the bene t of
treatment to pa ents is the rela ve risk reduc on. In this case, simvasta n
reduces death by 29%.
2. How precise was es mate of treatment e ect?
To decide regarding precision, you should look at the 95% con dence interval.
The closer these values the more precise your es mates. If this is not reported
check the p-value, anywhere from </=.10 is acceptable.
CAN RESULTS HELP 1. Can the results be applied to my pa ent care?
ME IN CARING FOR If your pa ent meets all of the inclusion criteria and none of the exclusion
MY PATIENT criteria, the applicability of the study’s results to your pa ent is without
ques on. It is rare however that we get a pa ent who conforms to all the
characteris cs of the study subjects. In these cases, we should decide if the
reason is compelling enough not to apply the results of our study to our
par cular pa ent.
2. Were all clinically important outcomes considered?
Clinically important outcomes may range from decreasing mortality, morbidity
and improving quality of life. These are outcomes that are important to the
pa ents and will lead directly to reducing symptoms and decreasing death.
Some studies might report improvement in cholesterol level, improvement in
LFT but these are what might be labeled as surrogate endpoints. That is, the
researchers have subs tuted these physiologic measures for important
outcomes we have just men oned. For reduc on in these laboratory parameters
does not always translate in decrease mortality and morbidity.
A drama c example of the danger of subs tute endpoints was found in the
evalua on of the usefulness of clo brate as an -cholesterol drug. It shown to
decrease cholesterol but shown to increase all mortality.
3. Are the likely treatment bene ts worth thee poten al harm and
costs?
Computa on of the might be done to check cost e ec veness and checking for
side e ects might be done to check if the treatment bene ts are worth the
poten al harm and costs.

RESOLUTION OF THE The ar cle was valid and giving simvasta n reduces all deaths by 29%
ti
ti
ti
ff
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
ti
fi
ti
ti
ti
ti
ti
ti
fi
fi
ti
ti
ff
ti
ti
fi
ff
ti
ti
ti
fi
ti
ti
 PROBLEM compared to placebo. However, when we look into costs, giving this drug is fairly
expensive. However, due to the cardiac risks for this pa ent you decide to give
simvasta n since your pa ent can also a ord drug treatment.
ti
ti
ff
ti