Aetiology or harm critical appraisal guide

Validity Where do I look?

Are the objectives of the study clearly stated?

• Introduction
The main question being addressed should be clearly stated. (title, abstract or
final paragraph)
The question will often be expressed in terms of a simple relationship but may not fit a
PICO question.

Were exposures and outcomes measured in the same way in the patient groups?
• Methods
The Methods section should define the outcomes and describe how exposures were • Statistical
assessed. In cohort studies and RCT there should be clear descriptions and definitions of methods
outcomes, and patients in each group should be assessed equally carefully for these
outcomes.

In case-control studies, beware of:

• recall bias - increased chance that cases will carefully examine and recall their
exposure.
• interview bias – more probing by interviewer in cases than in controls.

Look at whether the opportunity for exposure was the same in cases and controls, or
whether the groups were dissimilar in this way.

Were the patient groups clearly defined and similar in prognosis other than exposure
to the treatment or aetiological factor? • Patients
• Methods
How was the patient sample selected? A randomised trial is the strongest design for
studies of harm or aetiology, however it may not always be ethical to do a randomised
trial to answer these types of questions.

If it is not feasible or ethical to do a randomised trial, a cohort study may be the best
design. Have the characteristics of the different cohorts been well described?

Was follow-up sufficiently complete and was it long enough?

• Methods
A flow diagram may show loss to follow up. Check whether the follow up time was long • Results
enough for the outcome/harm of interest to occur. Loss to follow up may not be random
and may relate to either a good or bad outcome.

A ‘sensitivity analysis’ can investigate the effects of loss to follow up. For example, the
investigators may re-analyse the data assuming that all patients lost to follow up died.
This can show how the ‘worst case scenario’ would affect the results.

Did the study have a sufficiently large sample size?

• Statistical
Larger samples usually mean more precise results. methods

Faculty of Medicine, Dentistry and Health Sciences UWA Medical and Dental Library
An online sample size calculator is available at:
http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSize.

Was there statistical adjustment for important differences between patient groups?
• Statistical
In a study of aetiology or harm, the clinical characteristics of the groups should be similar methods
at baseline, or the analysis should make statistical adjustments for prognostic variables.
This is simpler in a randomised controlled trial but more challenging for a cohort study or
case-control study.

Is the temporal relationship plausible?

• Results
An exposure must happen first to cause an outcome! • Discussion

Check whether enough time passed for the outcome to have plausibly been caused by
the exposure.

Is there evidence of a dose-response?

• Results
If the risk of a bad outcome increases with the duration or amount of exposure, this is • Discussion
stronger evidence of aetiology.

Is the biological relationship plausible?

• Introduction
Does an association between cause and effect make biological sense? • Discussion

Is there evidence of causation from a withdrawal-rechallenge study?

• Results
If an outcome resolved on withdrawal of the exposure, and reoccurred on re-exposure, • Discussion
this is stronger evidence that the exposure caused the outcome.

Clinical Importance Where do I look?

How strong is the association between exposure and outcome (harm, disease)?
• Results
Often expressed as a relative risk (RR) or odds ratio (OR). Relative risk cannot be used for
case-control studies, as the number of exposed people per case (denominator) is
unknown.

In case-control studies, an odds ratio is used. Low OR or RR are hard to interpret from
weaker study designs such as case-control or cohort studies, while very high OR or RR can
be convincing even when the study design is not randomised.

More information: Relative Risk, Absolute Risk Reduction, and Number Need to Treat
(harm).

How precise are the estimates of risk?

• Results
A 95% confidence interval should indicate the precision of the result.

More information: Relative Risk, Absolute Risk Reduction, and Number Need to Treat

Faculty of Medicine, Dentistry and Health Sciences UWA Medical and Dental Library
(harm).

Are the results discussed in relation to existing knowledge and is the discussion biased?
• Discussion
The discussion should put the results into a clinical context and the authors conclusions
should be justified by the study results.

What level of evidence does this paper give?

• Methods
Try to assign a level of evidence using the Oxford CEBM Levels of Evidence Hierarchy.

Applicability Where do I look?

Were the study patients and their management similar to those in my practice?
• Methods
Can you generalise the study population to your patient? Check whether your patient has (inclusions and
the same exposure, and whether their other risk factors are similar to the study group. exclusions
criteria)

Are the results useful in your patient?

• Results
How great is the risk in your patient? The OR and HR tell us about risk relative to an
unexposed group.

What is the baseline risk in your patient if he/she is not exposed? How does that change
on exposure? Try to calculate a NNH.

More information about Relative Risk, Absolute Risk Reduction, and Number Need to
Treat (harm).

EBP Calculations
Adverse outcome
Present (case) Absent (control)
Exposed Yes (Cohort) a b a+b

a+c b+d a+b+c+d

RCT, cohort study: Relative risk (RR) = [a/(a+b)]/[c/(c+d)]

Case-control study: Odds ratio (OR) = ad/bc

Calculations (2) for studies of aetiology/harm

Number needed to Harm (NNH)

Individualise for your patient by estimating their Event Rate for the adverse event if they were not exposed
to the causative factor (PEER = Patient’s Expected Event Rate)

NNH = PEER (OR – 1) + 1

PEER (OR – 1) x (1 – PEER)

Faculty of Medicine, Dentistry and Health Sciences UWA Medical and Dental Library