Critical Appraisal

CARA EFEKTIF BELAJAR SEPANJANG HAYAT

DR. ARIANI RD.,SPM. FK UNISMA MALANG.

2011.
 The Problem
1

• Vast and expanding literature.

• Limited time to read.

• Different reasons to read :

– Keeping up to date.

– Answering specific clinical questions.

– Pursuing a research interest.

 mean different strategies

2
 Stages
2

Clarify your reasons for reading.

Specify your information need.

Identify relevant literature.

Critically appraise what you read.

 Clarify Your Reasons for Reading
3

Keeping up to date.
 Skimming the main journals and summary bulletins.

Answering specific clinical questions.

 Finding good quality literature on subject.

Pursuing a research interest.

 Extensive literature searching.
 Specify Your Information Need
4

What kind of reports do I want?

How much detail do I need?

How far back should I search?

The answers to these questions should flow from

the reasons for reading.
 Kind of Reports
5

Study Types for Question Types

What causes this disease?
How good is a diagnostic test? good quality validation against
Does this treatment work?
How is the prognosis?
Randomized controlled clinical
trial (RCT), Prospective Cohort
Study, Case Control Study (Rare
Diseases, probably retrospective)
Prospective cohort study with
“Gold Standard”
RCT
Cohort Study, Cross-sectional
Survey, Qualitative Study
...Specify Your Information Need
 .
Identify Relevant Literature
6

There are many ways of finding literature.

 Selectivity is the key to successful critical

appraisal.
 Ways of Finding Literature
7

Source of Medical Information

 Colleagues
 Conferences

 Drug Reps
 Textbooks

 Journals

 Internet/Patients

...Identify Relevant Literature

 Keeping Up to Date
8

Daily InfoPOEMs http://www.infopoems.com/

bmjupdates+
http://bmjupdates.mcmaster.ca/index.asp
Tables of contents of journals
(e.g. JAMA)
 From journal itself
 My NCBI from PUBMED

...Identify Relevant Literature

 Clinical Question - Research
9

EBM Databases
 MEDLINE
 Free online access thru PubMed
 www.ncbi.nlm.nih.gov/entrez/
 Search by MESH terms or free text
 EBM Filters

...Identify Relevant Literature

 Selectivity
10

Filters for Medline Search

 Diagnosis
 Sensitivity and Specificity
 Treatment
 Randomized controlled trial
 Blind or Double Blind

...Identify Relevant Literature

Critically Appraisal What You Read
11

EBM
Evidence-Based Medicine
12

What is EBM ?

Why do it ?

...Critically Appraisal What You Read

 What is EBM ?
13

"...the conscientious, explicit, and judicious

use of current best evidence in making
decisions about the care of individual
patients.“

from Sackett, DL, et al. "Evidence based medicine:

What it is and what it isn't." (BMJ 1996; 312: 71-2)

...Critically Appraisal What You Read

 What is EBM ?
14

An approach to clinical decision-making

in which one searches the literature,

critically evaluates the research evidence, and

then chooses the most appropriate intervention

or course of action to take.

...Critically Appraisal What You Read

 Why do EBM ?
15

APPLICATION TO PRACTICE

Increase patient trust

Provide best possible care

Reduce liability risk

...Critically Appraisal What You Read

 What a Physician can be Faced With
on a Daily
16
Basis?

~1 - 2 questions per patient (clinics)

15/patient/day (wards)

40% easy to answer

30% tough to answer
30% cannot answer

Source: Dawes M, Sampson U. Knowledge management in clinical practice: a systematic

review of information seeking behaviour in physicians. Int J Med Inform. 2003 Aug;
71(1):9-15. Review. PMID: 12909153

...Critically Appraisal What You Read

 Critical Appraisal of Literature
17

Intended to enhance the clinician’s skill to

determine whether the results reported in
an article were likely to be . . . .

. . . true
. . . important
. . . applicable to their patients!

...Critically Appraisal What You Read

 Why Should we Critically Appraise?
18

 Published research is not always reliable

 Published research is not always relevant

 To improve clinical effectiveness, we need a

systematic framework to interpret research

...Critically Appraisal What You Read

 Difficulties with Critical Appraisal
19

o Can be time consuming initially

o Doesn’t provide an “easy” answer

o It could show a lack of good evidence in a

particular topic

...Critically Appraisal What You Read

 How can we do Critical Appraisal?
21

Use common sense

Use simple checklists

Use different checklists depending on the

different types of studies (i.e., RCTS, systematic
reviews etc)
Checklists help you focus on the important
parts of the article

...Critically Appraisal What You Read

Checklists
22

Quick and easy

3 stages:
 Basic questions.
 Essential appraisal.
 Detailed appraisal.

22What You Read

...Critically Appraisal
1st. Basic Questions.

...Critically Appraisal What You Read

 How to Read a Research Article? – First Glance.
24
1. Purpose of reading the paper.
2. Do not read the abstract.
3. Read the title, find out who the authors are and where they work,
look for sources of funding and conflicts of interest
4. Look at the tables and figures
5. Is there a diagram to show the flow of participants through each
stage of the study
6. Why was the study done and what hypothesis were the authors
testing?
7. What is broadly the topic of research?
8. Therapy, Diagnosis, Screening, Prognosis, Causation?
9. What type of study was done?
10. Is this a primary (experimental, clinical trial, survey) or a secondary
paper (review, meta-analysis, guideline)?

...1st. Basic Questions.

 PICO Structure
25

Patient or problem
Intervention
Comparison
Outcome

...1st. Basic Questions.

2nd. Essential Appraisal.

...Critically Appraisal What You Read

 EBM “simplified” approach
27

 What are the results?

 Are the results valid?

 Will the results help me in patient care?

...2nd. Essential Appraisal.

 KEY QUALITY PARAMETERS
28

 VALIDITY

 RELIABILITY

 IMPORTANCE

...2nd. Essential Appraisal.

 VALIDITY
29

 INTERNAL
Is the study designed in such a way that I
can trust the findings?

 EXTERNAL
Is the study designed in such a way that I
can generalize the findings?

...2nd. Essential Appraisal.

 RELIABILITY
30

If the study was conducted again,

would the results be the same?

Usually interpreted as the accuracy of measurement.

...2nd. Essential Appraisal.

 Validity and Reliability
31

A test is valid when it measures what it’s

supposed to.

If a test is reliable, it yields consistent results.

A test can be both reliable and valid, one or

the other, or neither.

...2nd. Essential Appraisal.

Reliable, but Not Valid!

...2nd. Essential Appraisal.

Not Reliable, Not Valid!

...2nd. Essential Appraisal.

Reliable and Valid

...2nd. Essential Appraisal.

 IMPORTANCE
35

What was the effect size

or magnitude of effect?

Clinical vs. statistical significance.

...2nd. Essential Appraisal.

3rd. Detailed Appraisal.

...Critically Appraisal What You Read

 Dasar-Dasar
Metodologi
Penelitian Klinis
Edisi ke-2

Prof. DR. Dr. Sudigdo Sastroasmoro, Sp.A (K)

Prof. Dr. Sofyan Ismael, Sp.A (K)

Jakarta 2002
 Deskripsi Umum
38

Desain apakah yang digunakan

Manakah populasi target, populasi terjangkau,
sampel
Bagaimanakah cara pemilihan sampel
Manakah variabel bebas
Manakah variabel tergantung
Apakah hasil utama penelitian

Tabel 21-2. Telaah Kritis Makalah Kedokteran

: Hal-hal yang dinilai pada studi hubungan sebab-akibat.

Sastroasmoro, Sudigdo (2002)

Validitas Interna, hubungan non-kausal
39

Apakah hasil dipengaruhi bias

Apakah hasil dipengaruhi faktor peluang
Apakah observasi dipengaruhi perancu

Tabel 21-2. Telaah Kritis Makalah Kedokteran

: Hal-hal yang dinilai pada studi hubungan sebab-akibat.

Sastroasmoro, Sudigdo (2002)

 Validitas Interna, hubungan kausal
40

Apakah hubungan waktu benar

Apakah asosiasi kuat
Apakah ada hubungan dosis
Apakah hasil konsisten dalam penelitian ini
Apakah hubungan bersifat spesifik
Apakah ada koherensi
Apakah hasil biologically plausible

Tabel 21-2. Telaah Kritis Makalah Kedokteran

: Hal-hal yang dinilai pada studi hubungan sebab-akibat.

Sastroasmoro, Sudigdo (2002)

 Validitas Eksterna
41

Apakah hasil dapat diterapkan pada subjek

terpilih
Apakah hasil dapat diterapkan pada populasi
terjangkau
Apakah hasil dapat diterapkan pada populasi
yang lebih luas

Tabel 21-2. Telaah Kritis Makalah Kedokteran

: Hal-hal yang dinilai pada studi hubungan sebab-akibat.

Sastroasmoro, Sudigdo (2002)

 Desain Penelitian
42

Diagnosis Prospective cohort study.

Therapy Randomized controlled clinical trial (RCT)
Etiology/ RCT.
Harm Cohort.
Case-control study.
Prognosis Prospective cohort study

Deskripsi Umum
Desain Penelitian
Desain Penelitian
Desain Penelitian
Desain Penelitian
Desain Penelitian
 Populasi
48

TARGET
TERJANGKAU/SUMBER
SAMPEL

Deskripsi Umum
Populasi TARGET
Populasi
49

Deskripsi Umum
 Sampel
50

Karakteristik Subjek Penelitian


= pasien kita ?

Dapat kita terapkan ?

Deskripsi Umum
 Cara Pemilihan Sampel
51

Probability Sampling
 Simple Random Sampling
 Systematic Sampling
 Stratified Random Sampling
 Cluster Sampling

Non Probability Sampling

 Consecutive Sampling
 Convenient Sampling
 Judgmental / purposive sampling

Deskripsi Umum
 Variabel
52

BEBAS
: faktor resiko, perlakuan (pemeriksaan / terapi
medikamentosa / tindakan)

TERGANTUNG
: dipengaruhi oleh variabel bebas
Faktor resiko  Kejadian penyakit
Pemeriksaan  Terdiagnosis
Obat / Tindakan  Kesembuhan

Deskripsi Umum
 Bias Seleksi Subjek
53

1. Neyman’s Bias
Bias prevalens / insiden
Penyakit dengan mortalitas tinggi di awal

2. Berkson’s fallacy
Admission rate bias
Case-control : perbedaan indikasi rawat kontrol

Validitas Interna, non-kausal

 Bias Seleksi Subjek
54

3. Bias non-respons / bias relawan

Latar belakang kesediaan / penolakan sebagai relawan

4. Membership bias
Multifaktorial

5. Procedure selection bias

Non randomized  tidak setara

Validitas Interna, non-kausal

 Bias Pengukuran
55

1. Bias prosedur
Perlakuan tidak setara antar kelompok
Ketersamaran (blind / double-blind)

2. Recall bias
Case-control : kelompok kontrol < ingat

3. Insensitive measurement bias

Alat ukur < sensitif

Validitas Interna, non-kausal

 Bias Pengukuran
56

4. Detection bias
Deteksi > dini (alat ukur > sensitif ?)  masa sakit >>

5. Compliance bias
Case-control : frekuensi pemakaian obat berbeda

Validitas Interna, non-kausal

 Perancu
57

( Confounding factors )
Variabel bebas Variabel tergantung
Penyakit
Minum Kopi Jantung
Koroner

Merokok

Perancu
Validitas Interna, hubungan kausal
 Peluang
58

Kemungkinan hasil tersebut disebabkan

peluang bila hipotesis nol (tidak ada/tidak)

P value
Nilai P ≤ 0,05

Confidence interval
CI ≥ 95%

Validitas Interna, hubungan kausal

 Hubungan Waktu
59

Variabel bebas
(sebab, faktor resiko, perlakuan)

HARUS MENDAHULUI
Variabel tergantung
(akibat, penyakit, kesembuhan)

Validitas Interna, hubungan kausal

 Asosiasi
60

Kemungkinan
Sebab A menghasilkan Akibat B
lebih besar bila
RR (Relative Risk)
Odds ratio BESAR
Prevalens ratio

Atau
Nilai P (peluang / probability)  Kecil
Interval kepercayaan (CI : Confidence Interval)  Sempit

Validitas Interna, hubungan kausal

 hubungan lain
61

Hubungan Dosis
Dose dependent  Studi menetapkan dosis optimal

Konsisten
Beberapa kelompok subjek  multicenter

Koherensi
Fenomena / kenyataan dalam masyarakat

Validitas Interna, hubungan kausal

 …hubungan lain
62

Biological plausibility
Hipotetis – spekulatif

Perbandingan dengan hasil penelitian lain

Beda  Mengapa ?

Validitas Interna, hubungan kausal

 Validitas Eksterna
63

pada Sampel Terpilih

Drop out : ≤ 15%

pada Populasi Terjangkau

Random sampling / Consecutive sampling

pada Populasi Target

Logika dalam bab Diskusi
Karakteristik subjek ~ pasien sehari-hari

Validitas Eksterna
Stay up to date with the current literature
64

A clinician needs to read 17 peer reviewed

articles per day, every day of the year, to stay
current (Haynes 1993).

Haynes, R. (1993) Where's the meat in clinical journals?

ACP Journal Club, 119: A23-4.

There are 20 million pieces of 'evidence' of varying quality

and sometime of contradictory conclusions.
 Basic elements of clinical decision making

Haynes RB. Loose connections between peer-reviewed clinical journals and

clinical practice. Ann Intern Med 1990;113:724-8.
 Clinical Expertise & Critical Appraisal
67

"Without clinical expertise , practice risks

becoming tyrannized by external evidence,
for even excellent external evidence may be
inapplicable to or inappropriate for an
individual patient. 
Without current best external evidence ,
practice risks becoming rapidly out of date, to
the detriment of patients."

Haynes RB. Loose connections between peer-reviewed clinical journals and

clinical practice. Ann Intern Med 1990;113:724-8.
 TERIMA KASIH

Critical Appraisal. Cara Efektif Belajar Sepanjang Hayat.

Dr. Ariani RD.,SpM. FK Unisma Malang. 2011.