1

Systematic Review. Y Nugraha

A Brief Introduction of

Systematic Review

Yudhi Nugraha
Departemen Biologi Molekuler
Stem Cell and Regenerative Medicine
Fakultas Kedokteran UPN Veteran Jakarta
 2
COVID-19 research adaptation

Systematic Review. Y Nugraha

 3
Term of Literature review

Systematic Review. Y Nugraha

The general term for all attempts to
synthesize the results and conclusions of
several publications on a given topic.

A review may or may not be systematic.

Level of Literature Review 4

Systematic Review. Y Nugraha

Narrative Meta- Systematic
Review analysis Review
Levels of literature review 5

Narrative review

Systematic Review. Y Nugraha


 Traditional expert review; usually subjective in nature
 Meta-analysis
 Quantitative evidence
 Use of statistical methods to combine the results of various
independent, similar studies
 More precise calculation of one estimate of treatment effect than
could be achieved by any of the individual, contributing studies
 Systematic review
 A review of a clearly formulated question that uses systematic and
explicit methods to identify, select and critically appraise relevant
research, and to collect and analyze data from the studies that are
included in the review
 6
Systematic review – brief history

Systematic Review. Y Nugraha

 Astronomers claim to be the
first users of this method
 Explosion of SRs in health sciences
in mid 1980s 9000

811
Term “systematic review” was coined

709 3
8000


6538

6
7000
by health care researchers 6000

9 39304850
5000
 SR became a significant tool 4000

8 323
6277
5259
for “evidence-based medicine”
3000

3233
4 206
9 159
2000

9128
8107
or “evidence-based practice”

994
184
974
1000

663
259

52
948
642
338
2 37
33
32

6
1
4
27
1
1
1
0

Growth of SRs in health sciences

 7
SR – another definition
 A systematic review “attempts
 to collate all empirical evidence that
fits pre- specified eligibility criteria
 in order to answer a specific research
question.
 It uses explicit, systematic methods
that are selected with a view to
minimizing bias,
 thus providing more reliable findings
from which conclusions can be drawn
and decisions made.”
Systematic Review. Y Nugraha
 8

SysRev vs. narrative review

Systematic Review. Y Nugraha

 9
Why we need
systematic reviews

Systematic Review. Y Nugraha

 Minimize the impact of bias/errors
 Can help to end confusion
 Highlight where there is not
sufficient evidence
 Combining findings from different
studies can highlight new findings
 Can mitigate the need for further
trials
 10
Characteristics of an SR

Systematic Review. Y Nugraha

 Clearly stated set of objectives with pre-defined
eligibility criteria for studies
 Explicit, reproducible methodology
 Systematic search that attempts to identify all
studies that would meet the eligibility criteria
 Assessment of the validity of the findings of the
included studies
 Systematic presentation, and synthesis, of the
characteristics and findings of the included
studies
Stages of a systematic review 11

Develop a focused research question

Define inclusion and exclusion criteria
Search the literature
Select studies
Assess study quality
Extract data
Analyze and present results
Interpret results and draw
conclusions
Update as necessary
 12
Review team

Systematic Review. Y Nugraha

 Normally a team work
 Key skills:
 Managing research projects
 Leading, coordinating
Expertise in the topic


 Methodological expertise
 Planning, searching, managing information,
coding, analyzing, synthesizing, writing
 13
Protocol

Systematic Review. Y Nugraha

 What is the title?
 What is the context and what are the conceptual issues?
 What is the aim?
 What is the research question?
 What is the search strategy?
 What are the inclusion / exclusion criteria?
 How will the data be extracted and analyzed?
 How will the quality of studies be assessed?
Research Question 14

Systematic Review. Y Nugraha

• A clearly defined, focused systematic review
begins with a well formulated research question.

 The research question guides the author in working through many

stages of the systematic review process
 Defining inclusion and exclusion criteria
 Searching the literature
 Selecting studies
 Extracting data
 Analyzing and presenting results
PICO – question 15
components in medicine

Systematic Review. Y Nugraha

 P – Population
 Patients (Demographic factors, socioeconomic
factors, setting, etc.)
 I – Intervention
 Drug, procedure, etc.
 C – Comparison
 Alternative to compare with intervention (placebo or
active)
 O – Outcome
 Improvement, effect, measure, etc.
 16

Systematic Review. Y Nugraha

PICO
 Patients  Patients undergoing upper abdominal surgery
 Intervention  Prophylactic physical therapy
 Comparison  No prophylactic physical therapy
 Outcome  Prevent post operative pulmonary complications
PICO – example
Clinical question: "In children with headache, is paracetamol more effective 17
than placebo against pain?"
•Population = Children with headaches; keywords = children + headache
•Intervention = Paracetamol; keyword = paracetamol

Systematic Review. Y Nugraha

•Compared with = Placebo; keyword = placebo
•Outcome of interest = Pain; keyword = pain
Pubmed (health research database) search strategy:
children headache paracetamol placebo pain
18

Systematic Review. Y Nugraha

PICO – example
 19
Formulate Sample questions

Systematic Review. Y Nugraha

 In undergraduate medical education, does the use
of tutorial system improve learning outcomes?
 Are antiseptic washes more effective than non
antiseptic washes at preventing nosocomial
infections in patients undergoing surgery?
 Are mass media (or school-based or community-
based) interventions effective in preventing smoking in
young people?
Inclusion and exclusion criteria 20

One of the features that distinguish a systematic review from a

Systematic Review. Y Nugraha


traditional review is the pre-specification of inclusion and exclusion
criteria.
 Inclusion criteria are a combination of
• Aspects of the research question
 Population
 Intervention
 Comparison
 Outcome

• Study type
 Randomized controlled trials
 Observational studies
 21
Searching literature

Systematic Review. Y Nugraha

 Identifying major concepts
 Keywords, synonyms, controlled vocabulary
 Identifying where to search
 Search strategy
 Varies in different databases
 Export citations to a reference management
software
 EndNote, etc.
 Documenting your search
 Database name, date of searching, number of results
 22
Sources of literature

Systematic Review. Y Nugraha

 Electronic databases
 General vs. subject
 Grey literature
 Conference proceedings, theses,
reports, Websites
 Browsing issues of topical journals
 Backward and forward citations of the
most relevant articles
 Conversation with experts in the field
 23
Sources of literature

Systematic Review. Y Nugraha

 The goal of the literature search is to discover all studies that meet
the inclusion criteria
 The Cochrane Collaboration recommends searching the following
databases (at minimum):
 PubMed
 EMBASE
 Cochrane Central Register of Controlled Trials
 Interdisciplinary databases
 Scopus
 Web of Science
Compile search results 24

Systematic Review. Y Nugraha

 Compile
search results using reference
management software (EndNote or
Mendeley)
 Remove duplicate records
Select studies 25

Systematic Review. Y Nugraha

 Identification of studies meeting inclusion criteria
should be done independently by two review
authors.
 Review titles and abstracts of retrieved citations.
 Review full text of studies which are found to meet
the inclusion criteria.
 Keep a record of reasons for inclusion or exclusion.
Tools for sorting 26

Systematic Review. Y Nugraha

 Tools for sorting
 Reference management software
 EndNote

 New web application

 Rayyan

 Excel
PRISMA Protocol 27

Preferred Reporting items for Systematic reviews and Meta-Analyses:

Systematic Review. Y Nugraha

The PRISMA Statement

• Is essential reading before starting a systematic review.

• Editors of journals increasingly expect systematic reviews
to use PRISMA or similar guidelines.
• The PRISMA Checklist will guide you on how to develop
a systematic review protocol and what to include
when writing up your review.
• http://prisma-statement.org/
28

Systematic Review. Y Nugraha

 29

Systematic Review. Y Nugraha

http://www.prisma-statement.org/documents/PRISMA_2020_expanded_checklist.pdf
30

Systematic Review. Y Nugraha

PRISMA flow diagram
 31

Systematic Review. Y Nugraha

PRISMA diagram – example
Quality assessment criteria 32

Systematic Review. Y Nugraha

 33
Quality assessment results

Systematic Review. Y Nugraha

 34
Data extraction

Systematic Review. Y Nugraha

 Think about what data you need to extract
from included studies to answer the
questions
 Pilot a draft data extraction form
Data extraction 35

What data needs to be extracted?

Systematic Review. Y Nugraha

Intervention: e.g. number of groups, specific intervention for each
(giving enough detail for replication)
Outcomes: e.g. measurement time points, outcome definition, units of
source/s, authors conclusions, references to measurements
Results: e.g. number of participants in each intervention group,
sample size, missing participants, summary data
Miscellaneous: Funding other relevant studies

Ref: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated
March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
Data extraction 36

Systematic Review. Y Nugraha

Two reviewers extract the data
Why:
- Data is not always found in consistent places in articles
- It can be easy to miss or misinterpret information

Before data extraction occurs:

- Decide on the level of detail that needs to be extracted
- Develop and / or trial a data extraction tool: paper or
electronic?
37

Systematic Review. Y Nugraha

Data extraction form
38

Systematic Review. Y Nugraha

Summary table of evidence
 39
Data synthesis

Systematic Review. Y Nugraha

 Will results be pooled? How?
 How will differences between studies be
taken into account?
 Can subgroups of data be made?
 How will results be displayed?
Summary of findings table 40

 Summary of findings table

Systematic Review. Y Nugraha

 Provides key information concerning the quality of evidence
 Depicts the magnitude of effect of the interventions
 Illustrates the sum of available data on all important
outcomes for a given comparison

 Interpret results and draw conclusions

 41

Interesting
Source
 42

How they
manage
 43

Estimation
time

Systematic Review. Y Nugraha

 44

Tools
they use

Systematic Review. Y Nugraha

45

Systematic Review. Y Nugraha

Thank you
Registries, tools and archives 46

Cochrane Collaboration

Systematic Review. Y Nugraha


 Cochrane Library
 Cochrane protocols, systematic reviews, other reviews and trials
 Guides and handbooks
 Cochrane Handbook for Systematic Reviews of Interventions
 Cochrane Handbook for Systematic Reviews of Diagnostic Test
Accuracy
 Cochrane Standards for conduct and reporting of new reviews of
interventions
 GRADE Handbook
 Cochrane tools for assessing risk of bias
 ROBIS: Risk of Bias in Systematic Reviews
 ROBINS-I tool: Risk of Bias in Non-randomized studies of Interventions
 RoB 2.0 tool (revised tool for Risk of Bias in randomized trials
 Methodological Expectations of Cochrane Intervention Reviews
Registries, tools and archives 47

Systematic Review. Y Nugraha

 GRADE Working Group
 GRADE (Grading of Recommendations, Assessment, Development and
Evaluation)
 Method of grading quality of evidence and strength of recommendations in
guidelines
 GRADEpro software can be used to create Summary of Findings tables
Registries, tools and archives 48

Systematic Review. Y Nugraha

 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-
Analyses)
 PRISMA checklist
 27 items for inclusion in systemic reviews and meta-analyses
 PRISMA flow diagram
 Diagram depicting flow of information and number of records through
phases of a systematic review

 Prospero
 International prospective register of health-related systematic reviews
Resources to learn more 49
 Margaret Foster’s Webinar Series:
https://nnlm.gov/scr/professional-development/systematic-

Systematic Review. Y Nugraha

review-series

 John’s Hopkins Coursera Course: Intro to Systematic

Review and Meta-Analysis:
https://www.coursera.org/learn/systematic-review

 Campbell Collaboration Training Resources:

https://www.campbellcollaboration.org/research-
resources/training-courses.html

 Cochrane Interactive Learning(free access to income-

eligible countries):
 https://training.cochrane.org/interactivelearning
Software for systematic reviews 50

Systematic Review. Y Nugraha

 Software
 Software for Systematic Reviewing
 Software Programs for Preparing and Maintaining Systematic
Reviews
 Systematic Review Toolbox
 51

Systematic Review. Y Nugraha

Resources to learn more