Systematic Review:

Concept and Process

A/P Patraporn Bhatarasakoon, PhD, MSN,

APPMH, RN, FAAN
Director of The Thailand Centre for Evidence Based
Health Care: a JBI collaborating Centre
Faculty of Nursing, CMU
“a 21st centuey clinician who cannot
criticalX$ read a study is as
-.unprepared as one who canxot tahe a

cardiovascular spstem”
BMj 2008;337:704-705
 Contents

Systematic review Types and questions of How to interpret/use

concept and process systematic review systematic review
 What is Systematic Review?
• “Research on research”
• “Research synthesis”
• “Secondary research”
• But not meta-analysis- it is only the statistic
procedure
• SR and Meta-analysis began to use in the
health field in 1970s and 1980s (Bastian et al.
2010).
• 1990s confusion arose between these 2 terms
 What is Systematic Review?
• Systematic reviews aim to provide a comprehensive,
unbiased synthesis of many relevant studies in a
single document using rigorous and transparent
methods.
• A systematic review does not seek to create new
knowledge but rather to synthesize and summarize existing
knowledge-however it creates a valid evidence whether to
confirm/refute/expand the existing knowledge-could be new
in another point of view
• It attempts to uncover “all” of the evidence relevant to a
question and to focus on research that reports data
rather than concepts or theory.
 The History of Systematic
Reviews
• Archie Cochrane (1909-1988)
• Widely known for writing ‘Effectiveness and Efficiency: Random
Reflections on Health Services’ in 1972
“It is surely a great criticism of our profession that
we have not organised a critical summary, by
specialty or subspecialty, adapted periodically, of
all relevant randomised controlled trials.” (Cochrane,
1979)
 How SR differs from Traditional Review
• SR
– Protocol driven which indicated
the comprehensive process of
methodology which need to
follow
– Aim to decrease the bias
– Use meta-analysis to evaluate
the effectiveness of treatment or
intervention for quantitative
evidence and meta-synthesis for
qualitative evidence
– Aim is to provide the best
available evidence for decision
making at the point to care
• Traditional Review
– No need of protocol
– No need of rigorous methodology
and normally done with the ad
hoc fashion ( Mr. A said this and
Ms. B said that…)
– No aim to evaluate the
effectiveness of the intervention or
treatment but
just show what selected study
found
– Aim is to describe the state of the
art of the phenomena of interest
(subjective selected)
 How SR differs from Traditional Review
• SR reproducible method
– Cover all relevant literature
both nationally and
internationally
– Search systematically and
comprehensively
– Systematic report
– Recommendation to
practice/policy
making/and further
research
– Transparency and
• Traditional Review
– No need to cover
all relevant
literatures
(subjective
selected)
– No need to do meta-
analysis or meta-
synthesis
– No
reproducible
and not
transparency
 Why do SR?
• 1. To cover all international literature- seamless world
• 2. To Confirm current practice or address any variation
• 3. To identify area of future research
– Do the primary research still need to conduct?
– Or the certainty of effect is strong enough to utilize and integrate
them to develop the recommendation for practice and policy
making
• 4. To Investigating conflicting results
• 5. To produce statement/recommendation to guide
decision making for practice and policy making
 Characteristics
of high-quality SRs
• Protocol driven
– Clearly state of objectives and questions
– Clearly state of inclusion and exclusion criteria
• Comprehensive search to identified all relevant
studies (both published and unpublished)
• Critical appraise of the included study; assessment of
the validity of their results, and of any exclusions based
on quality
 Characteristics
of high-quality SRs
• Analysis of data extracted from the included research
• Presentation and synthesis of the findings extracted
• Rating certainty of the evidence included in the review
• Transparent reporting of the methodology and
methods used to conduct the review
Standard Tools to Guide the Review Report

1. PRISMA (Preferred Reporting Items for

Systematic Reviews and Meta-Analyses)
2. ENTREQ (Enhancing transparency in reporting
the synthesis of qualitative research)
3. JBI Reviewer’s Manual and JBIES Author guidelines
4. Cochrane Handbook and MECIR
(Methodological Expectations of Cochrane
Intervention Reviews)
 Benefits of SR
• Fill the gap of research to practice
– There is 15 years taking from the research into practice
• Recommend the best available evidence to decision
point of care
– Summarize the effective intervention or treatment for patients
– Evidence based recommendation for practice /policy making/
future research
 Benefits of SR
• In term of conduction- Do on your own pace but in
a timely manner
– No need to conduct interview or questionnaire directly to the
subjects or informants
– You can do wherever/when internet available and
whenever/when you are available
– No need to worry about the REC process since we not conduct
RCT with direct patients/population
 Types of SR
1. Effectiveness Reviews
2. Qualitative Reviews
3. Costs/Economics Reviews
4. Prevalence or Incidence Reviews
5. Diagnostic Test Accuracy Reviews
6. Etiology and Risk Reviews
7. Textual Synthesis Reviews
8. Mixed Methods Reviews
9. Umbrella Reviews and;
10. Scoping Reviews
 Effectiveness Reviews
Effectiveness can be defined as “the extent to which an
intervention, when used appropriately, achieves the
intended effect”
Aim - To evaluate the effectiveness of a certain
treatment/practice in terms of its impact on outcomes.
Study Design - Experimental studies; quasi-
experimental studies and observational
studies
 Qualitative Reviews
Aim - To investigate the experience or
meaningfulness of a particular phenomenon.
Focusing on analyzing human experiences and
cultural and social phenomena

Study Design – Qualitative research informed via a

qualitative philosophy; methodology and
appropriate data collection method
 Costs/Economics Reviews
Aim - To determine the costs associated with a
particular approach/treatment strategy, particularly in
terms of cost effectiveness or benefit.
Particularly useful to inform health policy decisions
attempting to achieve equality in healthcare provision to
all members of society
Study Design – Experimental studies; quasi-
experimental studies with focus on those including
both costs and outcomes of strategies of interest.
 Prevalence or Incidence Reviews
Aim - To determine the prevalence and/or incidence of a
certain condition (measure of disease burden). Enable
governments, policy makers, health professionals and the
general population to inform the development and delivery of
health services

Study Design – Mainly observational study designs,

including prospective and retrospective cohort, case-control
and cross- sectional studies

Prevalence (the proportion of a population who have a certain disease)

Incidence (how often a disease occurs)
 Diagnostic Test Accuracy Reviews
Aim - To determine how well a diagnostic test works
in terms of its sensitivity and specificity for a particular
diagnosis. Important for clinicians to determine the
accuracy of the diagnostic tests they use

Study Design – The two main types of studies

of diagnostic test accuracy are case–control and cross-
sectional.
 Etiology and Risk Reviews
Aim - To determine the association between particular
exposures/risk factors and outcomes. e.g. Who is getting
the disease? Where is the disease occurring? What factors
are associated with the disease?
Study Design – Commonly, epidemiological or
observational studies are utilized to investigate etiology
and risk
Not able to determine causality; rather they are only able to
infer correlations or relationships between variables
 Textual Synthesis Reviews
Aim - To review and synthesize current expert opinion, narrative
or policy on a certain phenomena

Study Design – Drawn from expert opinions, consensus, current

discourse, comments, assumptions or assertions that appear in
various formats including journals, magazines, newspapers,
blogs, internet sites, monographs and reports
 Mixed Methods Reviews
Aim - To combine quantitative and qualitative data or
integrate quantitative evidence and qualitative evidence
to produce more informative conclusions than those
derived from evidence presented in effectiveness or
qualitative reviews alone.
Study Design – Can either be a quantitative
synthesis or a qualitative synthesis of data from
primary studies.
 Umbrella Reviews

Aim - To provide an overview and synthesis of existing

systematic reviews. Compare and contrast published
reviews and provide an overall examination of a body of
information that is available for a given topic

Study Design – Systematic Reviews

 Scoping Reviews
(Mapping Reviews or Scoping Studies)

Aim - To determine the size/scope of a body of literature.

To underpin the research area, clarify definitions and/or the
conceptual boundaries of a topic and to identify gaps in the
knowledge.

Unlike other review types, the question is intentionally less

precise.

Study Design – Broad scope and correspondingly

less restrictive in the types of studies to be synthesized.
 How to do
SR??
• Because SR is the research synthesis or
Secondary research
• Following Scientific Method
– Strong methodology
–Transparency/Auditability
–Reproducible
 How to do
SR??
• There are rigorous method/process of
doing SR
• You need to be TRAINED!!! the same as to
conduct primary research!!

• The TCEBHC will offer training 3/year

 Steps in Conducting SR
• 8 steps
– 1. Formulating the review question
– 2. Defining inclusion and exclusion criteria
– 3. Locating and selecting the study
– 4. Appraising the quality of the studies
– 5. Extracting data
 Steps in Conducting SR
• 8 steps
– 6. Analyzing data
• Meta-analysis for quantitative data
• Meta-synthesis for qualitative data
– 7. Presenting result- narrative with forest plot/meta-aggregation
– 8. Interpreting and making recommendations
• To practice
• To policy making
• To further research
Patraporn Tungpunkom (2015 a; b). Editorial: Steps in Conducting Systematic Review:
It is Simple but Not That Easy. Pacific Rim International Journal of Nursing Research,
19 (1), Jan-March 2015.
Question Development
 Effectiveness Reviews
Question Development:
PICO
–Population
–Intervention
–Comparator/control
–Outcome
 Effectiveness Reviews
Example:
• Are art therapy compared with exercise effective
in reducing depressive symptoms of older
adults with depressive disorders?

Population Intervention Comparison Outcome

 Qualitative Reviews
Question Development
PICo
– Population
– Phenomena of Interest
– Context
 Qualitative Reviews
Example:
• What is the experience of caregiving in
Persons with schizophrenia during their stay
at home?

Population Phenomena of Interest Context

 Qualitative Reviews
Example:
• What is the experience of relations in
Persons with dementia during their stay at
home?

Population Phenomena of Interest Context

 Scoping Reviews
Example:
• What types of neurological reactions to the
COVID-19 vaccination have been reported
in people who have received the COVID-
19 vaccine?

Population Concept Context

 Planning a Systematic Review
• Human and technical resource
– You need at least 2 reviewers
• 1 is expert on the process: need to be trained how to conduct SR
• 1 is expert in the area
• Review panel is recommended-expert in method/area/and consumer
representatives-depend of topic
– Library support
– Statistician support
– Methodologist support
 Planning a Systematic Review
• Check whether this topic is already existed or not!!!—
the most important one
– Cochrane
– JBI library
– Prospero
• Is there any relevant studies to the topic of interest
– At least 2 primary studies-RCT is easy to start with
 Planning a Systematic Review
• Registration your topic
– This enables other reviewers to identify topics that are
currently in development and avoids accidental
duplication of topics
– For titles registered through JBI, once registered, a title is valid
for six months from the date of entry in the database.

• Published Protocol in JBIES; or PROSPERO

JBIM

JBI
 Protocol Structure
• Title – Assessment of
methodological quality
• Review objective/question(s) – Data extraction
• Introduction – Data synthesis
• Keywords – Assessing confidence
• Acknowledgements
• Inclusion criteria
• Funding
(Framework)
• Conflicts of interest
• Methods: • References
– Search strategy
• Appendices:
– Study selection – Search strategy example
 PRISMA-P
• PRISMA-P statement provides guidance on
reporting for protocols
• Checklist contains 17 numbered items that
they recommend should be described at a
minimum
• Grouped into: administrative information,
introduction, and methods
 How to interpret/use SR

• “A 21st century clinician who cannot critically read

a study is as unprepared as one who cannot take
a blood pressure or examine the cardiovascular
system”
BMJ; 2008:337,704-705.
 SR Report Structure
• Background
• Objectives/Question • Results
• Criteria:
• Discussion and
– PICO/PICo
• Search strategy
conclusion
• Methods: • Appendices
– Methodological quality –Included studies
– Data extraction
–Excluded studies
– Data synthesis
– References
–Forms etc
 Forest Plot example: Dichotomous Effect

Data
Intervention
size
Model
Method
Control
Weights in meta-
analysis
Study authors

Confidence
Interval
Point Estimate

Test for Meta-analysis

heterogenei summary result
ExperimentalLine
Label
of no effect Control Label
Forest Plot example: Continuous Data
Meta-synthesis (aggregative) Flowchart
The Evidences are out there!!!

Find out & Utilize them!!

 Example of SR Report

50
KAltGElt
Experiences of medication adherence among people with
schizophrenia: a systematic review protocol of
qualitative evidence Exercise programs for the management of people
Nkakorn Pochimas' • PaOaj>>rn Tungpunkom” • Thidarat Kanungpiarn'
with hand osteoarthritis: a scoping review protocol
Beatrice E.A. San kah "“ • Maria Stokes'•“ • Jo Adams' ’”

Ftea4•n• question/•>bje<zlwe: she objecs•e of Azs quaGtatt•e re•ñe•v is O sy=
medicabon adherence azu>ng peo$ e wsth s::hb«ophzen+a. The
chJ opkxeoia ie a chs xdc axtd a<riozzs zaeatal
’Centre for innointron end Leodersfiip in Health Sciences Fchcol of Health Sciences, Oniwem*ty of •outfiomprori, 5outfiomptorb United firipdom
°Artfirir/s feseorcfi HR Cenare of f«ce//eri<e for Sport fxerc*se end Osteoarthritis /\Jottirigfiom, Ortrted Kingdom
=e be I+•ed ices oF
fic New quesñm Is: what are the es‹periezx-es
Rauiaw questions: The review quenions for this scoping review are:
i) What are the available pubI1shecI exercise programs implementecl for people with han<J osteoanhrit\s?
ii} Are the available exercises deveIope<I following theory-basecl treatment approaches, Le. revlew of ex1stlng
Ilterature, consultation w1th experts, and consultation wlth patients?
iii} Are these exerclses prescribed following cIIn1caI practlce guldellne recornrnendatlons wlth regards to
frequency, Intensity, type and tlrne?
xaozzthe or year‘Szgaa axul symptoazs, aunts aa ao<>af lv) Is patlent adherence to these exerclses reported, ance what are the exerclse adherence strategies usecl?
wizlsdcawal, deteri<sxaQ<sxz <sf• orfe fuzwti<sxMxzg axed Keywords Aclults; exercise program; hancl osteoarthritis, scoping revlew

•naBk* doooom gRo‹ms) z•x) pobéooe wSA

Introduction supports che use ofqh$and exercises in che manage-
artd osreoarrhriris (OA) is a common adutr h

after the prodxo-al pericid. This pbaae in character- EE ]oinr diorder with a prevalence in
wvornert (44% ) rhact men (38 % Age is a commozt
risk facror for developing hand OA. According re
che Cencre for Disc use Control and Prewenrion,
for wo-souod 23 year Mr -eo/T1e s.' Finally, in the xeaidiml plaaae, people With
OA a Elects adults of all ages with an increase at US
moAao years.‘ Wich che glo ba1 aging population, che p ze
IiJ<tizae pcevaleace f•<u peraoa*e wece 4.O per va- lence of OA is expected to rise. Pe ople with
hand OA ofren experience pain, joinr sriffness,
poor grip strength and reduced hand function,
which can influence daily funcciona I casks and
Whezeaa the ability a› fummicin ziornmlly -by
resulc in associ- ated socioeconomic burden to
both patients and society.^•^ Considering che high
prevalence rate and socioeconomic impacr, OA is
a recognized global pobI i z healrh concern, which
-ig•a°t versus -ative bQxn was I .84. warrants the need co era I uace curre nc and
promising cvidence- based management
poaitive syxoptooxs» xsegatzve syotptoaxs aod cogoi-
interventions to improve the quality c›f life c›f
Furthermore, individuals living with hand OA.
capita ixxroxae showed 1owez- p<evateace t•Izazs Ake Many pharmacologic and non-pharmacologic
trearment incervenrions are recomme rided in the
manage rrienc of hand OA.‘ Among rhese, hand
d-ie alch
a d Cdre cellenhe°' ' che Az er!can College
of Rheu nacolcigy '‘ a nd the Eiirc›pe an Leagiie
Against Rheumacism (EULAR) ' recommend che use
of lOw-impacc physical activity, self-manage- me nc
and joinc-proceccion scra cegi es, among ochers, as a
core part of hand OA management, despite limited
research evide rice. The EUfLA R ' r ecom- mended
hand range-of-morion and srrengchening exercises
based on level IV evidence (expert opin- ion)' ^ due
re rhe pauciry of qua liry research evi- dence. The
EWLAR' also reported the lack of an exhaustive
licerarure review in che guideline devel- opment and
acknowle dged that pez h o pn less com- monly used
hand OA manageme nt ince rvencions may have been
missed. The need ro scope rhe tire r- a ture fOr all
available research evidence On hand exercise in hand
OA management is rherefore rimely and reasona ble.
Several crearr cue incervencions have been crici-
cized aslackin robusc evidence-based developme
rtc and reportng. The dcvelcipment of an
evidence-

three p , •a°i‹dy, the p odre-at phase, the exer- cises are frequently recommended. Despite
Exclude beBavzora aru3x aa bafiuoow4xooa, deJoaJooa
mixed
and -avuo•uxu dBorderY” T4ugauve sy•opwmoa am

ace o£ s kardex to detect.’ Negatñre syzaptoozs t

include b1uat<d a£féut o< a/féctive £tatteab*g»
 ici•i." Cogx•iaue ayxcaptozna axe relatively ditf•mitt a› reports of irs beneficial effects,* currenr evidence
forneapondenre BeaiñcefA.Sankah, B.2A.Sankahgsotonacuk
OOt: IO.i 1124 'JBt5RtR-TO I Z-003806
based exercise pzogce rri should be based on synthe-
sized evidence from quality research evidence, clini-
cal expertise and clienr evidence. “‘ T-However,
current literature highlights the lack of consensus
regarding che design of such programs fOr people wich
hand OA.'* Due to this uncertainty, there is a
need to

2Od9VBJOANMA BKGGSlMSWTOTE

SIB Josnne Briggs Institute. Uneuthorizad reproduction of this article is prohibited. 019 Joanna Briggs Institute. Unauthorized reproduction of this article la prohibited.
 Thank you for your attention!!

Questions or any thoughts are welcome

patraporn.t@cmu.ac.th