CRITICAL APPRAISAL TRIAL (CAT)

Z0BIA NASEEM
LECTURER
CRITICAL APPRAISAL
TOOL

The process that has been developed by biostatisticians and

clinical epidemiologist for assessing trial.
CAT

Vast and expanding

literature.
Limited time to read.
STAGES
1. Clarify your reasons for reading.
2. Specify your information need.
3. Identify relevant literature.
4. Critically appraise what you read.
REASONS FOR READING

Different reasons to read

 Keeping up to date.
 Answering specific clinical questions.
 Pursuing a research interest.
 SPECIFY YOUR
INFORMATION NEED.
What kind of reports do I want?
How much detail do I need?
How comprehensive do I need to be?
How far back should I search?
The answers to these questions should flow from the reasons for
reading.
 IDENTIFY RELEVANT LITERATURE.

There are many ways of finding literature.

Use of ‘and’ ‘or’ to broaden or narrow the research.
Selectivity is the key to successful critical appraisal.
QUESTIONS TO ASK

Is it of interest?
 Title, abstract, source.

Why was it done?

 Introduction.
 Should end with a clear statement of the purpose of the study.
QUESTIONS TO ASK
How was it done?
 Methods.
 Brief but should include enough detail to enable one to judge quality.
 Must include who was studied and how they were recruited.
 Basic demographics must be there.
QUESTIONS TO ASK
What has it found?
 Results.
 The data should be there – not just statistics.
 Are the aims in the introduction addressed in the results?
 Flaws and inconsistencies.
 All research has some flaws
QUESTIONS TO ASK
What are the implications?
 Abstract / discussion.
 The whole use of research is how far the results can be generalised.
 All authors will tend to think their work is more important than the rest of us!
 What is new here?
 What does it mean for health care?
 Is it relevant to my patients?
QUESTIONS TO ASK
What else is of interest?
 Introduction / discussion.
 Useful references?
 Important ideas?
NOT JUST A FAULT FINDING
EXERCISE!!
To illustrate this, we will consider the issue of deep vein
thrombosis (DVT) on long-haul flights and whether it is
possible to prevent it by wearing elastic stockings.
For this search, we obtained one study, a randomized
controlled trial, and the authors concluded that wearing
elastic stockings helps to prevent symptomless DVT on long
haul flights. But how do we know that the results are valid
and real?
Setting up representative and comparable
groups
• If the subjects in a study are not representative of
population to be studied ----- outcomes may not be
applicable for that population.
Best way to ensure that the study groups are
representative is to
• initially select subjects randomly from the whole
population of interest,
• apply relevant inclusion/ exclusion criteria and
• ensure that the study groups are large enough to provide
a representative sample (and hence statistically
meaningful results).
•For comparative studies, once
subjects have been recruited
for a trial, they must be
allocated to either the control
or treatment group.
Examples of ways in which groups could differ include
(CONFOUNDERS):
• age
• sex
• socioeconomic group
• smoker/nonsmoker
• disease status
• past exposure to risk factors
• …… (there are many more)
The best way to ensure that groups are matched is
to allocate subjects to them randomly.
MAINTAINING REPRESENTATIVE AND
COMPARABLE GROUPS: EQUAL
TREATMENT
2. WAS THE OUTCOME MEASUREMENT
ACCURATE?
Even if the study groups have been randomly
selected and allocated, the results obtained may
not reflect the true effect if the outcomes have
not been measured accurately. The two most
important factors that can affect outcome
measurements are:
• measurement bias
• measurement error
Measurement bias
Measurement bias reflects the human tendency to inadvertently
‘nudge’ results in the direction that they are predicted to go.
Measurement error
A second problem that can arise for outcome measurements is
measurement error. This occurs if outcomes are not measured in
the same way for all subjects. It is therefore important to use
exactly the same measurement strategy and methods for everyone
(both the treatment and control groups).
3. WAS A PLACEBO USED?
 4. COULD THE RESULTS HAVE
BEEN DUE TO CHANCE?

Statistics provides two methods of assessing chance:

• P-values (hypothesis testing)
• confidence intervals (estimation)
P-values are a measure of probability.
if the P-value is less than 0.05, it means that the probability that
the result was due to chance is less than 5%. This is called a
statistically significant result.
DIFFERENT TYPES
CAT for Diagnosis.
CAT for Prognosis.
CAT for evaluating risk and harm in case-Control study.
CAT for Evaluating risk and harm in a cohort Study.
CAT for treatment, prevention and Screening.