Critical Appraisal of the

Literature
Sarah-Jo Sinnott MPharm MPSI
Dept of Epidemiology and Public Health UCC
s.sinnott@ucc.ie

Overview
Critical Appraisal as part of evidence based
dentistry
Why do critical appraisal?
How to do critical appraisal
First steps in deciding whether to read a paper
properly or not
Steps when reading the paper properly
Tools to help you

Scenario
You are working as a dentist/hygienist in a busy
private practice

You are visited by a sales rep for a dental supply

company who urges you to buy their latest stateof-the art technology for treating dental caries
HealOzone

 No pain, no drilling, no side-effects.

Didn't you always wish you could say
these three catchphrases to your
patients before their treatment?
This is now possible with HealOzone,
a new therapy concept for caries
removal

How
does
it
work?
Ozone (O ) is a naturally occurring compound and a
3

powerful oxidising agent

Toxic to living systems in high concentrations (damages cell

membranes)

Applying ozone to carious lesions reduces the number

of cariogenic bacteria
Arrests or reverses lesions and prevents the need to restore
the tooth surface

A very attractive alternative to conventional drilling

and filling
The rep advises you to check their website for full
details and promises to return in a week to discuss
further

 Do you buy a HealOzone machine for

your dental practice?

What further information do you

need to help to make your decision?

Critical Appraisal as part of Evidence

Based Medicine/Dentistry
Definition of EBM
Evidence based Medicine (dentistry) is the
enhancement of a clinicians traditional skills in
diagnosis, treatment, prevention and relayed
areas through the systematic framing of relevant
and answerable questions and the use of
mathematical estimates of probability and risk
Greenhalgh 2001

The three elements of evidence-based dentistry (EBD)

The needs and preferences
of patients

Evidence-based
Dentistry

The best available

evidence

The professionals
expertise, skills and
judgement

Critical Appraisal as part of Evidence

based medicine/dentistry
1. Convert information needs into an
answerable question
2. Track down, EFFICENTLY, the best
evidence to answer question
3. Appraise the evidence critically, assess
validity and usefulness
4. Implement results into clinical practice
5. Evaluate performance

Convert your question into an

answerable question: PICO format

P
I
C
O
(T)

Population
Intervention
Comparison
Outcome
Time

Convert information needs into an

answerable question
PICO format
For an adult population with caries, does Ozone therapy
compared to conventional therapy for caries result in a
more effective treatment, as measured by the reversal of,
or arrest of, caries?
For a patient with tooth discoloration due to coffee and
tobacco, will Crest Whitestrips, as compared to custom
trays for use with an at-home whitening/bleaching
system, be as effective in whitening his teeth within 1
week

Critical Appraisal as part of Evidence

based medicine
1. Convert information needs into an
answerable question
2. Track down, EFFICENTLY, the best
evidence to answer question
3. Appraise the evidence critically, assess
validity and usefulness
4. Implement results into clinical practice
5. Evaluate performance

Track down, EFFICENTLY, the best

evidence to answer question
Key word

How to search
Has anyone else searched before you and
summarised the evidence?
The Cochrane Library

Pubmed, Embase, Web of Science, CINAHL

Google Scholar
Use PICO
tooth discolouration AND coffee OR
tobacco AND whitening

How to search
Has anyone else searched before you and
summarised the evidence?
The Cochrane Library

Pubmed, Embase, Web of Science, CINAHL

Google Scholar
Use PICO
tooth discolouration AND coffee OR
tobacco AND whitening

Critical Appraisal as part of Evidence

based medicine
1. Convert information needs into an
answerable question
2. Track down, EFFICENTLY, the best
evidence to answer question
3. Appraise the evidence critically, assess
validity and usefulness
4. Implement results into clinical practice
5. Evaluate performance

What is Critical Appraisal

Balanced assessment of strengths of research
against its weaknesses
Assessment of research process and results
To be undertaken by all health professionals as
part of their work
Both dentist and hygienist

What Critical Appraisal isnt

Negative dismissal of research
Assessment of results alone

P<0.05

To be undertaken only by experts

Why do Critical Appraisal

Published research is not always reliable
Only an estimated 2% is judged clinically
relevant
Patients come pre-read

Critical Appraisal on first glance

Why was the study done and what hypothesis
were the authors testing?
Bottom of intro
Beginning of methods
Discussion

Critical Appraisal on first glance

What type of study was done?
Secondary Research
(also Economic analyses,
decision analyses,
guidelines)

Primary
Research

Critical Appraisal on first glance

Was this design appropriate to the question addressed?
Research
Concern

Description

Preferred design

Therapy

Testing efficacy of method,

approach, equipment

RCT

Diagnosis

Is a test valid (can we trust it) or

reliable (? same result if repeated)

Cross-sectional survey of
new test and gold standard

Screening

Demonstrating the value of tests to

pick up a defect at a presymptomatic stage

Cross-sectional survey

Prognosis

What happens to someone whos

illness is picked up at an early
stage

Longitudinal cohort study

Causation

Does harmful agent lead to a

particular outcome

Cohort or case-control

Choosing the best research design for each question Sackett, 1997, BMJ

On first glance
Why?
What type?
Design?
Can now proceed

Going through the methods..

1.
2.
3.
4.
5.

Was the study original?

Who was it about?
Was it well designed?
Was systematic bias avoided?
Statistically sound?

Was the study original?

Does this new research add to the literature in
any way?
Longer duration
More rigorous
Different population

Who was it about?

How were subjects recruited?
Who was included in the study?
If inclusion criteria very narrow, results may not be
generalisable
Who was excluded from the study
Focus on moderate/severe forms of disease could lead to
false conclusion about treatment of mild disease
Were subjects studies in real life circumstances?
Applicability of the results to your practice

Was it well designed?

Identify
Intervention
Comparison

-We measured how often dentists ask patients about

family history of oral cancer
-We looked in dental records and counted how many
people had family hx recorded

-Assumption that dental records are 100% accurate

What was the outcome and how was it measured?

Valid (e.g. Index)
Reliable calibrated examiners and measure of reliability
(Kappa)
Surrogate outcomes

Was systematic bias avoided?

Any error in the design or conduct of a study
that results in a conclusion that is different
from the truth
Groups being compared should be as similar
as possible except for the difference being
examined

Bias
Selection Bias
Systematic errors are introduced by the selection of
study participants or allocation of individuals to
different study groups
Comparison groups no longer comparable
RANDOMISATION

Measurement Bias
Inaccurate measurement of classification of an
outcome or exposure
BLINDING

Statistically Sound
1. Size of sample
o
o

Power
Clinical difference

2. Duration of follow up
o
o

Survival
Painkillers

3. Completeness of follow up
o
o
o
o
o

Death?
Non-adherent patients?
Withdrawal by clinician?
Adverse reactions?
INTENTION TO TREAT

Ignoring these people will bias

results in favour of intervention

Statistically Sound
Tests
Categorical vs continuous
Are the data normal?

Parametric vs non-parametric
Repeated measures on patients?

Categorical:
Caries; Yes or No
Continuous:
Height: ~30cm to ~200cm

Paired tests

Tests of difference?
Correlation?
Causalitybeware associations
E.g., coffee drinkers found to be associated with higher
rates of cancer.

Four possible outcomes from any study

1. Difference is clinically and statistically
significant i.e. important and real.
2. Of clinical significance but not statistically
so. ?sample size too small.
3. Statistically significance but not clinically
i.e. not clinically meaningful.
4. Neither clinically nor statistically
significant.

Checklists
Help with assessing quality of research can be
found at
http://www.casp-uk.net/

And
http://www.cebm.net/index.aspx?o=1157

Checklists GRADE (google scholar this

to get pdf)

 Do you buy a HealOzone machine for

your dental practice?

What further information do you

need to help to make your decision?

In summary what did we cover?

How critical appraisal is essential in EBD

How to formulate your research question
How to go about searching for evidence
How to decide whether to read something or
not..time is precious
How to read it properly
Tools to help you read the paper and make a
decision on whether you should take it
seriously or not.

References
Greenhalgh, T. (2001). How to read a
paper: Assessing the methodological
quality of published papers. London:
BMJ Publishing Group
Users guides to the medical
literature 1990s, JAMA (about 20
articles)
Tip
USE THE GUIDELINES!