Critical Appraisal

A brief overview of the critical appraisal process in

relation to evidence based practice (medicine)

Raden Tina Dewi Judistiani

Lecture for year 3
Block : Research Method
Dr. R Tina Dewi J dr., SpOG 1
Why learn EBP and CA
Become a professional and responsible , to the best of 
your ability, to help client, assuring their safety
Apply current knowledge, technology and skills in 
solving medical and health issues, at individual-
family-community level
Gain ability to implement relevant scientific approach 
in problem solving
Incorporate socio-cultural value in the 
implementation , from individual-family and
community level
Dr. R Tina Dewi J dr., SpOG 2
The values of EBP : how they relate
In daily clinical decision making, GAPS are inevitable.
Your knowledge + skill vs making the best decision for
a situation
Figure 1
A decision-making process to
establish a clinical treatment plan
according to medical evidence that is
compatible with national health policy
and patient factors.

EBM= evidence-based medicine

EBP= evidence-based practice
Healt care
providers Picture modified from :
https://www.researchgate.net/publication/282481889_Promoting_e
vidence-based_practice_in_pharmacies/figures?lo=1

Dr. R Tina Dewi J dr., SpOG 3

The Gaps = Problems to solve
In daily clinical decision making, GAPS are inevitable.
(Your knowledge + skill) vs (making the best
decision)
YOU NEED EVIDENCE The GAPS should be formulated in an
ANSWERABLE QUESTION , comprised
of 4 section : PICO
Population
Intervention/ Risk/Exposure
Comparison of I/R/E
Outcome (event )

Healt care
providers EBM= evidence-based medicine.
Picture modified from :
https://www.researchgate.net/publication/282481889_Promoti
ng_evidence-based_practice_in_pharmacies/figures?lo=1 .

Dr. R Tina Dewi J dr., SpOG 4

PICO : starting point
Assess the ‘gap’ situation (of your problem)
 Population : general – women – pregnant women ?
 Intervention or issue : old vs new, better vs worse,
risk vs protective exposure
 Comparison to the issue in the gap
 Outcome (of your interest, depending on the ‘gap’

Dr. R Tina Dewi J dr., SpOG 5

The situation and the gap
A women was admitted to your clinic with recurrent 
colicky and burning sensation during micturation
The medicine ‘A’ given previously showed no effect 
She found out that she is pregnant now 

Dr. R Tina Dewi J dr., SpOG 6

A women was admitted to your clinic with recurrent 
colicky and burning sensation during micturation
The medicine ‘A’ given previously showed no effect 
She found out that she is pregnant now

Define the situation

– complete anamnesis, physical exams and
necessary lab tests ...
Hypothetical situation : probable recurrent cystitis

Population : women or pregnant women ?

– with recurrent cystitis
Intervention : Medicine A
Comparison : Medicine X
Outcome : complete cure – exhibit by PE and Labs
Dr. R Tina Dewi J dr., SpOG 7
PICO : starting point
Act as keywords , for searching research evidence

What needs to be solved / what do you compare

treatment : Best find RCT
exposure : cohort
risk : case control

When the situation is not yet well understood :

cross sectional studies are acceptable
with rigorous statistical analysis , they can be of
high value
Dr. R Tina Dewi J dr., SpOG 8
Evidence found  critically
appraise it
Remember :
methodologically, does the conclusion answers the
research problem (PICO) in the article correctly ?
No discharge

Yes simply put,

the conclusions of this CA process :
the article worth reading further

Dr. R Tina Dewi J dr., SpOG 9

Some articles MAY have conclusion which do not
answer the research problem correctly.
Is it worth skipping/ discharging ?
Example : if there is methodological flaw

Aim : To compare treatment regiments for recurrent

UTI
Method : retrospective cohort
Conclusion : Regimen X is superior to regimen A in the
treatment of recurent UTI
Pitfall : it is best to find a clinical trial
You need to understand the “GCP”
– good clinical practice in conducting a clinical trial
Picture source free clip art : http://www.clker.com/clipart-trash-bin-full.html
Dr. R Tina Dewi J dr., SpOG 10
So , in the previous case example : Metodological flaw
Aim : To compare treatment regiments for recurrent UTI
Method : retrospective cohort
Conclusion : Regimen X is superior to regimen Y in the treatment of recurent UTI

Is it possible that the article might be useful if you

cannot find a clinical trial article ?
Read the method :
Did they do recruitment correctly for cohort ?
Did the article provide data that you can re-analyze ?
If so : Can you calculate the “relative risk” (from the
presented data)
Make the PROPER conclusion on the RR !

Dr. R Tina Dewi J dr., SpOG 11

Know the ‘measures’ of a
research outcome

Number Needed to Treat,

Number Needed to Harm
Compliance
Cost , etc
Dr. R Tina Dewi J dr., SpOG 12
Critical appraisal and EB practice
 Critical appraisal is an essential part of evidence-based P.

 Critical appraisal allows us to make sense of research

evidence and it may begin to close the ‘gap’

Dr. R Tina Dewi J dr., SpOG 13

Critical Appraisal
Remember :
methodologically, does the conclusion answers the
research problem in the article in correctly ?
Yes , the article worth reading

Next step : Putting into practice :

Assess the research results ,
does it really answer your own PICO ?
will the results alter your action/ decision making?

Dr. R Tina Dewi J dr., SpOG 14

What are the benefit of doing
Critically Appraisal of an Article ?
• Conducting literature reviews for research proposals.

• Evaluating the effectiveness, costs, and benefits of

health services

• Public health decision making, especially at the senior

management level :
to establish new health programs , or
to implement health policies (health economics)

Dr. R Tina Dewi J dr., SpOG 16

Consider the following questions
before selecting an article
 What kind of reports do I want?
biomolecular mechanism ?
– animal study,tissue or cell
culture
looking for epidemiologic studies ?
prevalence ? – Cross sectional study
cohort, CC, disease survival , trend, prediction , and
many more (still growing) studies.
Combined topic : health economics , etc
Picture source :https://pubmed.ncbi.nlm.nih.gov/23124115/17
Dr. R Tina Dewi J dr., SpOG
Consider the following questions
before selecting an article
 How much detail do I need?
 How comprehensive do I need to be?
 How far back (in time/stage) should I search
 Is this article relevant to my issue and setting?

Dr. R Tina Dewi J dr., SpOG 18

To Critically Appraise What You
Read.
You need : Simple TOOL (S)
in checklists format

To identify useful informations

Comprised of 3 stages questions

 Basic questions
 Essential appraisal
 Detailed appraisal
Dr. R Tina Dewi J dr., SpOG 19
 First stage readers would mostly screen
the abstract

 Are your issues discussed there?

 Does it address a relevant question (to your need, fulfil the gap) ?
Even research that covers your issue indirectly, can be useful.
 Was the research done in a similar setting to yours ? introduction
 What are the main findings of the research? Conclusion

Then :

 Do you want to know more after reading the abstract?

 Are there reasons to doubt the findings without reading the
whole article? (such as methodological flaw in previous example)

Dr. R Tina Dewi J dr., SpOG 20

 One may conclude that the study is relevant or
irrelevant -- merely by reading the abstract.

 But, the correct practice is by using the checklist

Dr. R Tina Dewi J dr., SpOG 21

Baby step
Know your question : the reason why you need to
perform CA in the first place .

PICO
Population
Intervention/Risk/Exposure
Comparison to I/R/E
Outcome (expected)/Event

Dr. R Tina Dewi J dr., SpOG 22

Baby step
Example 1 PICO
Population : pregnant women with UTI
Intervention/Risk/Exposure : Amoxycillin clavulenic
acid (known as safest medication in pregnancy)
Comparison to I/R/E : fosfomycin (new medicine , new
indication)
Outcome (expected)/Event : (choices) recurrency,
compliance, side effects , or hazard effects

Dr. R Tina Dewi J dr., SpOG 23

Baby step
Example 2 PICO
Population : pregnant women
Intervention/Risk/Exposure : normal BMI
Comparison to I/R/E : high BMI /overweight or obese
Outcome (expected)/Event : (choices) UTI occurrence
or recurrence

Dr. R Tina Dewi J dr., SpOG 24

Three broad issues in CA

 A Are the results of the study valid?

 B What are the results?
 C Will the results help locally (to your need) ?

Dr. R Tina Dewi J dr., SpOG 25

Introduction or Background

Does it contain a clear statement on the purpose

of the study ? (which is relevant to your need)

 Focus on the time, place and circumstances of the

population and setting studied.

 How similar or different is the study population or

setting to yours? And if there is any difference(s),
would they likely to matter for the issue at hand?

Dr. R Tina Dewi J dr., SpOG 26

 Methods
• Study type ? _ is it the best design to answer your PICO
questions
 Does it include enough detail to enable quality
assessment ?
 Must include : sample/population , recruitment

 What basic demographics data available ?

 How the data was collected (interviews, focus

groups, questionnaires, surveys, observations, etc)
 Does it use appropriate method for analysis ?

Dr. R Tina Dewi J dr., SpOG 27

How good are the measures?
• Do the measures accurately reflect what the researcher
was trying to measure (validity)?
• How clear and appropriate are these measures? Too
broad? Too narrow?
• Are they actual measures or proxy measures?
• Are the measures well established in either prior
research or through pilot testing?

Dr. R Tina Dewi J dr., SpOG 28

Proxy measures
= surrogate, substitute, intermediate .
example :
reduction of pain , instead of cure .
Blood pressure change instead of getting a “lowest risk
group” in hypertension classification

Dr. R Tina Dewi J dr., SpOG 29

RESULTS : What are the findings?

Descriptive study : do we need statistics ??

Statistics scrutiny in quantitative studies :

sample calculation method
– relevant to research question
the sample/ subject recruitment
the power of association
(OR, RR, NNT, NNH, LR – include the CI )
significance of findings (p value)

Dr. R Tina Dewi J dr., SpOG 30

RESULTS : What are the findings?

Deeper quetions , such as

Is the effect size clinically or operationally relevant?

Keep for later ... (statistic lectures)

Dr. R Tina Dewi J dr., SpOG 31

RESULTS : What are the findings?
 Are the aims in the introduction addressed in the
results?
 Look for illogical sequences, too bald statements of
results.
 Flaws and inconsistencies.

All research has some flaws , the impact of the flaws

need to assessed

Dr. R Tina Dewi J dr., SpOG 32

Discussion
 The discussion section is more detailed and precise
than the abstract, and will explain:
 how the results of “this” study compare or contrast
with previous research(es) ?
 Did “this” study find something different ?
 Does it explain the limitations of the research and the
possible implications ? (which are usually not
mentioned in the abstract)

Dr. R Tina Dewi J dr., SpOG 33

 Level of evidence
systematic reviews
meta analyses
RCTs,
cohort studies,
case-control studies,
case report
Expert opinion

What is a
diagnostic study?
Where is its place ?
Dr. R Tina Dewi J dr., SpOG 34
 Recommended Reading

https://www.cebm.ox.ac.uk/resources/ebm-tools/critical-appraisal-tools

https://drive.google.com/file/d/10xRBhchn05AZsGdihv3
uMAf_9POVFg1d/view?usp=sharing

Sign in to unpad account to access the book

Dr. R Tina Dewi J dr., SpOG 35

 Situation 1 : on bed side teaching session

Mr Grey, 78 years old , was admitted to ER for

fever, sore throat and general malaise. That
morning his urine was orange-red, someting
he noticed for the first time.
In the previous day he had been tested
positive for Covid-19 antigen, prior to his
travel plan and it was later confirmed by a
PCR result, presented at the ER.
His son asked about the prognosis of Mr Grey
.
Baby step
PICO
Population : elderly man with covid infection
Intervention/Risk/Exposure :
renal involvement
Comparison to I/R/E : no renal involvement
Outcome /Event : (choices of prognosis )
Sepsis
ICU admission
Death

Dr. R Tina Dewi J dr., SpOG 37

PICO : starting point
Act as keywords , for searching research evidence

An article was found during quick google scholar search

in break out room

Dr. R Tina Dewi J dr., SpOG 38

What type of research method is best to predict 
prognosis ?
Select the right tool to critically appraise the 
article from

https://www.cebm.ox.ac.uk/resources/ebm-
tools/critical-appraisal-tools