SYSTEMATIC REVIEW

& META-ANALYSIS

MARK R. MANZANO, MD, FPAFP

FAMILY AND COMMUNITY MEDICINE
LECTURE OUTLINE
01 SYSTEMATIC REVIEW

02 SYSTEMATIC VS TRADITIONAL
REVIEW

03 META-ANALYSIS

04 HOW TO WRITE SYSTEMATIC

REVIEW AND META-ANALYSIS
 OBJECTIVES

To discuss the systematic review

To understand the uses of systematic review
and meta-analysis
To be able to write a systematic review or
meta-analysis research paper
WHAT IS SYSTEMATIC REVIEW?
 Systematic Review
 form of research that provides a summary
of medical reports on a specific clinical
question, using explicit methods to search,
critically appraise, and synthesize the world
literature systematically
 useful in bringing together a number of
separately conducted studies 
synthesizing the results
 take account of the whole range of relevant
findings from research on a particular topic,
and not just the results of one or two
studies
WHAT IS SYSTEMATIC REVIEW?
 Systematic Review
 Review of a clearly formulated question
that uses systematic and explicit methods
to identify, select, and critically appraise
relevant research, and to collect and
analyze data from the studies that are
included in the review
 SYSTEMATIC REVIEW
 Advantages
 used to establish whether scientific findings
are consistent and generalizable across
populations, settings, and treatment
variations, or whether findings vary
significantly by particular subgroups
 Limit bias
 Improve reliability and accuracy of
conclusions  more POWER
 systematic reviews of RCTs are considered
to be evidence of the highest level in the
hierarchy of research designs evaluating
effectiveness of interventions.
 WHY THE NEED FOR
SYSTEMATIC REVIEW?
 Health care professionals are increasingly
required to base their practice on the best
available evidence
 It is therefore important that health care
decisions are not based solely on one or two
studies without account being taken of the
whole range of research information available
on that topic
 WHY THE NEED FOR
SYSTEMATIC REVIEW?
 Review articles in the medical literature have
traditionally been in the form of ‘‘narrative
reviews’’ where experts in a particular field
provide what is supposed to be a ‘‘summary of
evidence’’ in that field
 Narrative reviews have been criticized because
of the high risk of bias
 Systematic reviews apply scientific strategies in
ways that limit bias to the assembly, a critical
appraisal, and synthesis of relevant studies
that address a specific clinical question.
PROBLEM WITH TRADITIONAL
REVIEWS
 validity of a review article depends on its
methodological quality
 can be useful when conducted properly, there
is evidence that they are usually of poor
quality
 tend to be selective in citing reports that
reinforce their preconceived ideas or promote
their own views on a topic
 often biased, and the recommendations made
may be inappropriate
 SYSTEMATIC REVIEW VS
TRADITIONAL REVIEWS
SYSTEMATIC REVIEW TRADITIONAL/ NARRATIVE
REVIEW
EVIDENCE High evidence Low evidence
METHOD Systematic steps Expert opinion “no protocol
according to a detailed and no systematic steps”
protocol
BIAS Low bias High Bias
REPRODUCIBILITY Yes Not reproducible
SCOPE Narrow scope Wide scope
 METHODOLOGY OF
SYSTEMATIC REVIEW
 This includes a comprehensive, exhaustive
search for primary studies on a focused clinical
question, selection of studies using clear and
reproducible eligibility criteria, critical
appraisal of primary studies for quality, and
synthesis of results according to a
predetermined and explicit method
 METHODOLOGY OF SYSTEMATIC REVIEW
State objectives and Outline eligibility assemble the most comprehensive dataset
criteria feasible

COMPREHENSIVELY search for trials Analyze results of eligible RCT's

Tabulate characteristics of each trial and Compare alternative analysis if appropriate

assess its methodological criteria and possible

Prepare a critical summary of the review,

apply eligibility criteria and justify
stating aims, describing materials and
exclusions
reporting results
 META-ANALYSIS
 Following a systematic review, data from
individual studies may be pooled quantitatively
and reanalyzed using established statistical
methods
 Rationale  by combining the samples of the
individual studies, the overall sample size is
increased, thereby improving the statistical
power of the analysis as well as the precision
of the estimates of treatment effects
 META-ANALYSIS
 Use of statistical techniques in a Systematic
Review to integrate the results of included
studies to conduct statistical inference
 Process of combining multiple studies together
to give one effect size with larger power (larger
sample)
 META-ANALYSIS
 2 step process
 first stage involves the calculation of a
measure of treatment effect with its 95%
confidence intervals (CI) for each individual
study
 The summary statistics that are usually
used to measure treatment effect
include odds ratios (OR), relative risks
(RR), and risk differences
 META-ANALYSIS
 2 step process
 second stage of meta-analysis, an overall
treatment effect is calculated as a weighted
average of the individual summary statistics
 Greater weights are given to the results
from studies that provide more
information closer to the ‘‘true
effect’’ (estimate of effect)
 The weights are often the inverse of the
variance of the treatment effect, which
relates closely to sample size
 The typical graph for results of a meta-
analysis ‘‘forest plot’’.
 META-ANALYSIS
 Advantages
 Used when there are conflicting studies
 Produce precise effect estimate
 Large power so a small but clinically
significant effect size will be apparent
 META-ANALYSIS
 Forest Plot
 information from the individual studies that
went into the meta-analysis, and an
estimate of the overall results
 allows a visual assessment of the amount
of variation between the results of the
studies (heterogeneity)
 META-ANALYSIS

Figure 1: Example of Forest Plot

 META-ANALYSIS
 Description of Forest Plot (Look at Figure 1)
 Branches  names on the left of the plot
are the first authors of the primary studies
included
 black squares  represent the odds ratios
of the individual studies
 horizontal lines  represent 95%
confidence intervals
 Size of black squares  reflects the weight
each trial contributes in the meta-analysis
 META-ANALYSIS
 Description of Forest Plot (Look at Figure 1)
 95% confidence intervals true underlying
effect in 95% of the occasions if the study
was repeated again and again
 solid vertical line  corresponds to no
effect of treatment (OR = 1.0)
 If the CI includes 1, then the difference
in the effect of experimental and control
treatment is not significant at
conventional levels (p.0.05)
 META-ANALYSIS
 Description of Forest Plot (Look at Figure 1)
 Diamond  overall treatment effect from
the meta-analysis and its Confidence
Interval (CI)
 center of the diamond  represents the
combined treatment effect
 horizontal tips  represent the 95% CI
 KEY POINTS
 Meta-analysis should not be used as synonym
for systematic review
 A Meta-analysis should be done in the context
of systematic review
 A Meta-analysis should not be assumed to
always be an appropriate step in a systematic
review

MA
SR
 STEPS OF A SYSTEMATIC
REVIEW
1. Develop a focused research question
2. Define inclusion/exclusion criteria
3. Select the outcomes for your review
4. Find the studies
5. Abstract the data
6. Assess quality of the data
7. Explore data (heterogeneity)
8. Synthesize the data descriptively and
inferentially via meta-analysis if appropriate
9. Summarize the findings
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
1. Defining the review question
 defining the problem to be addressed in
the form of a clear, unambiguous, and
structured question
 must initially choose an interesting broad
area and then expand his knowledge about
it by reading, discussing, and exploring
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
1. Defining the review question
 Then must narrow down to a specific
problem within the broad area
 Finally, must convert it into a review
question
 Make use of Patient/population,
Intervention, Comparison, Outcome format
(PICO)
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
1. Defining the review question
 Patients/Population  Defining the subject
group like; age, sex, race, and other patient
characteristics
 Intervention  Consider the intervention
of interest
 Comparison  Group with whom the
initially defined population and
intervention would be compared to
 Outcome  The item you hope to
accomplish, measure, or define
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
1. Defining the review question
 Once the review question has been defined
 there should be no alterations to be
made
 Must ensure that the question that is
formulated is Feasible, Interesting, Novel,
Ethical, Significant, Time-bound (FINEST)
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
2. Defining the inclusion and exclusion criteria
 very essential that the authors explicitly
define the studies which they would select
and those they would exclude
 also important to decide the language
whose article would be included
 also a good idea to define the time frame
of publication of the included studies
 should also be defined if only human
studies would be included or both human
and animal studies would be included
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
3. Systematic literature search
 literature search strategy must be defined
 must identify the main themes within the
review question and find as many
keywords/Medical Subject Headings
(MeSH) terms for each theme
 keywords need to be connected using
appropriate Boolean operators
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
3. Systematic literature search
 All possible relevant electronic data basis
must be searched
 Some of the data basis which should be
included are PubMed, MEDLINE through
PubMed, Embase, Web of Science, Scopus,
and Cochrane Controlled Trials Register
 Appropriate search filters such as duration,
type of studies (animal or human),
language etc. must be used at this stage
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
4. Selection of the studies based on the previous
inclusion and exclusion criteria
 Duplicates should be removed
 titles of each article must be read to
removed the irrelevant ones
 should usually be done by two reviewers
 If consensus cannot be made, a third
reviewer will resolve the conflict
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
5. The assessment of the quality of the included
studies
 Once the studies that are shortlisted for
final inclusion in the systematic review
have been identified, their quality analysis
must be done
 The design and level of evidence of the
included studies must be ascertained
 The internal and external validity of
included studies must be assessed
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
5. The assessment of the quality of the included
studies
 Any bias such as description bias, selection
bias, measurement bias, analytic bias, and
interpretation bias must be assessed
 Available quality scales or checklists like
critical appraisal skills program (CASP)
checklists can be useful tools, but their
strengths and weaknesses must be known
and described
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
6. Extraction of data from the included studies
 relevant data from every study, based on
the review question needs to be extracted
 data that is extracted should be
meticulously filled in a well designed
spreadsheet
 Initially as much data as possible should be
extracted so that anything important is not
missed because that would require going
through all the manuscripts again
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
7. Summarizing the evidence
 extracted data need to be summarized to
draw valid and logical conclusions
 When the data extracted for the
interventions and outcomes being studied
in the review question, is similar enough
that it can be pooled together using
statistical tools
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
7. Summarizing the evidence
 Review Manager (Rev Man 5) is a very
useful tool for preparing and maintaining
cochrane reviews
 If that is not possible then a qualitative
review should be produced
 When a qualitative summation is done
each outcome that has been mentioned
must be reported separately and hence
named systematic review
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
8. Discussing the review findings and drawing
conclusion
 discussion should include the key findings
about each of the main outcomes
 The strength of evidence about each
outcome measure must also be discussed
 The limitations and strengths of the
included studies and the authors own
reviews must also be included
HOW TO WRITE SYSTEMATIC
REVIEW OR META-ANALYSIS
8. Discussing the review findings and drawing
conclusion
 The results obtained must be compared to
those of the other studies
 Sometimes, the results may be
inconclusive, but even then they should be
reported because it enlightens researchers
to conduct research in that area
THANK
YOU!