[ evidence in practice ]

STEVEN J. KAMPER, BSc, BAppSc, PhD1,2

Systematic Reviews 1 – Gathering the

Evidence: Linking Evidence to Practice
Downloaded from www.jospt.org at University of Otago on August 8, 2023. For personal use only. No other uses without permission.

J Orthop Sports Phys Ther 2023;53(8):490-491. doi:10.2519/jospt.2023.0701

R
arely does a single study definitively answer an important tifies a small number, be wary that relevant
research question. Compelling evidence is usually accumulated studies may have been missed.
from several studies. Systematic review is an approach to The studies identified in searches need
to be screened for inclusion. Screening in-
collecting all available evidence on a question and making
volves 2 stages: (1) assessing all the titles
sense of it. Before methods for systematic reviews were established and abstracts and excluding those clearly
(the first systematic review in physical therapy was published in not eligible, and (2) assessing the full text
Copyright © 2023 Journal of Orthopaedic & Sports Physical Therapy®. All rights reserved.

1982), reviews were done differently. Typically, they were written
by an eminent professor who gathered
the papers in his filing cabinet and sum-
marized them. The problem was that the
conclusions were highly subject to the
biases of the author.
Systematic review is a methodological ap-
proach (FIGURE) that includes the following
Journal of Orthopaedic & Sports Physical Therapy®
Components of a Systematic Review
A systematic review is useless unless it
starts with a specific, clear, and answer-
able question.1 The requirements of a sys-
tematic review question are no different to
any other study, if the research question
for the review is not clear to you, do not
bother reading on. For treatment effective-
of the articles that were not excluded in
stage 1 against the inclusion criteria. Ide-
ally, both stages are done by 2 people inde-
pendently, and disagreements are resolved
by discussion or a third person to make
sure nothing is missed. The screening pro-
cess ends with the list of included studies.
Relevant data are extracted from all
the included studies, and each study is

key features which aim to reduce the risk that
the authors’ biases will influence the findings.
1. The question, search, inclusion, and
analysis methods are clearly defined be-
fore review starts.
2. All the evidence relevant to the re-
search question is included.
3. Only the evidence relevant to the re-
search question is included.
The protocol for a systematic review
should be published and available before
the review is undertaken—typically on
PROSPERO8 or Open Science Frame-
work10—occasionally, in a peer-reviewed
journal. Readers should place more trust
in a systematic review that has an a prio-
ri protocol (registered before the system-
atic review started) than one without.
ness reviews, remember that defining the
comparison group is a key part of the re-
search question.2 The question determines
the inclusion and exclusion criteria for the
studies included in the systematic review.
A comprehensive, sensitive search that
locates all the available research relevant
to the question is critical. Search keywords
map to the inclusion criteria, joining them
with the appropriate connectors (AND/
OR). Good-quality searches are usually
informed by an information specialist or
follow the format set out in PRISMA-S12 or
PRESS7 guidelines. The search should be
run on several databases such as PubMed,
PEDro, Embase, and CINAHL. Authors
are also encouraged to search the reference
lists of included studies and similar reviews.
Typically a sensitive search will find more
than 1000 articles. If the initial search iden-
assessed for risk of bias. Data include de-
tails that describe the study participants
and methods, and the outcomes that an-
swer the systematic review question. Risk
of bias is assessed to help readers decide
how much confidence they should have in
the results of the systematic review. The
appropriate risk of bias tool will depend
on the design of studies included in the
review, for treatment effectiveness, this is
usually the Cochrane Risk of Bias tool or
PEDro scale. These processes should also
be done by 2 authors independently.
Synthesis means bringing the results of
all the included studies together to answer
the review question. Synthesis is the key
to producing clinically relevant informa-
tion from several primary studies, so the
methods matter. Synthesis methods will
be discussed in Systematic Reviews 2.5

1
School of Health Sciences, University of Sydney, Camperdown, Australia. 2Nepean Blue Mountains Local Health District, Penrith, Australia. ORCID: Kamper, 0000-0003-3012-5499.
The author certifies that he has no affiliations with or financial involvement in any organization or entity with a direct financial interest in the subject matter or materials discussed
in the article. Address correspondence to Steven Kamper, Nepean Clinical School, 62 Derby St, Kingswood, NSW 2747, Australia. E-mail: steven.kamper@sydney.edu.au t Copyright
©2023 JOSPT®, Inc

490 | august 2023 | volume 53 | number 8 | journal of orthopaedic & sports physical therapy

Downloaded from www.jospt.org at University of Otago on August 8, 2023. For personal use only. No other uses without permission.
FIGURE. Steps in a systematic review of quantitative
studies.
Types of Research Question
Systematic reviews can address any of
the 3 types of research question (descrip-
tive, predictive, or causal3); they can in-
Copyright © 2023 Journal of Orthopaedic & Sports Physical Therapy®. All rights reserved.
clude quantitative or qualitative studies.6
Examples include systematic reviews of
qualitative studies investigating patients’
experiences with shoulder pain (descrip-
tive),11 models predicting outcome in low
back pain (predictive),13 effectiveness of
nonsurgical treatments for patellofemo-
ral pain (causal).9
Journal of Orthopaedic & Sports Physical Therapy®
Quality of Systematic Reviews
Systematic reviews are at the top of most
evidence hierarchies regarding treatment
effectiveness because, when conducted
well, they ensure all available evidence on
a question is considered and treated in an
even-handed way. However, a systematic
review can be poorly conceived, performed,
or reported and at high risk of bias just like
any other study. There are quality check-
lists for systematic reviews, such as AM-
STAR 2 or ROBIS, which are equivalent
to the study quality tools for other study
types.4 Note that the quality of a system-
atic review is separate to the quality of the
evidence in the review. It is possible to have
a well-conducted (high-quality) systematic
review that finds low-certainty (quality)
evidence because the included studies are
at high risk of bias, but a low-quality sys-
tematic review cannot provide high-quality
evidence.
Cochrane systematic reviews generally
provide the most reliable evidence. This
journal of orthopaedic & sports physical therapy | volume 53 | number 8 | august 2023 |
is because they are closely scrutinized by
methods and content experts throughout
the process from protocol development
to publication. The downside is that Co-
chrane reviews are almost exclusively
related to treatment effectiveness and
diagnostic accuracy questions, and they
are often not updated frequently.
Other Types of Reviews
Scoping reviews address a broader (ex-
ploratory) research question than sys-
tematic reviews. They map the available
evidence on a topic rather than inform
a specific clinical decision. Done well,
scoping reviews still have a clearly stated
question and design, conduct a sensitive
search, and systematically extract and
synthesize the data. Umbrella reviews
(reviews of reviews) are becoming more
common as increasing numbers of sys-
tematic reviews are published. They aim
to synthesize evidence from a large num-
ber of systematic reviews or bring togeth-
er evidence on a set of related questions.
Living reviews are systematic reviews that
are updated constantly as new primary
studies are published. This ensures that
the systematic review remains current.
There are also newer methods of
analysis. Independent participant data
meta-analysis uses data from the indi-
viduals in the included studies, rather
than the summary data (means, propor-
tions) reported in the published articles.
This depends on authors of the included
studies providing their dataset to the
review authors; the main advantage is
in enabling subgroup analyses. Network
meta-analysis can compare multiple in-
terventions simultaneously and rank the
effectiveness of the interventions. Note
that this analysis method relies on criti-
cal assumptions that are often not met in
the rehabilitation field, which can make
interpretation challenging. t
STUDY DETAILS
AUTHOR CONTRIBUTIONS: Steven J. Kamper
drafted and revised the manuscript.
DATA SHARING: There are no data associ-
ated with this article.
PATIENT AND PUBLIC INVOLVEMENT: There
was no patient consultation involved in
this article.
REFERENCES
1. Kamper SJ. Asking a question: linking evidence
with practice. J Orthop Sports Phys Ther.
2018;48:596-​597. https://doi.org/10.2519/
jospt.2018.0702
2. Kamper SJ. Control groups: linking evidence to
practice. J Orthop Sports Phys Ther. 2018;48:905-​
906. https://doi.org/10.2519/jospt.2018.0706
3. Kamper SJ. Types of research questions:
descriptive, predictive, or causal. J Orthop
Sports Phys Ther. 2020;50:468-​469. https://doi.
org/10.2519/jospt.2020.0703
4. Kamper SJ. Risk of bias and study quality as-
sessment: linking evidence to practice. J Orthop
Sports Phys Ther. 2020;50:277-​279. https://doi.
org/10.2519/jospt.2020.0702
5. Kamper SJ. Systematic reviews 2 synthesizing
the evidence: linking evidence with practice. J
Orthop Sports Phys Ther. 2023;53XXX-​XXX.
6. Kamper SJ, Thompson BL. Qualitative research:
linking evidence to practice. J Orthop Sports
Phys Ther. 2022;52:408-​409. https://doi.
org/10.2519/jospt.2022.0701
7. McGowan J, Sampson M, Salzwedel DM, Cogo E,
Foerster V, Lefebvre C. PRESS – Peer review of
electronic search strategies: 2015 guideline.
J Clin Epidemiol. 2016;75:40-​46. https://doi.
org/10.1016/j.jclinepi.2016.01.021
8. National Institute for Health and Care Research.
PROSPERO. York, UK: University of York. https://
www.crd.york.ac.uk/prospero/
9. Neal BS, Bartholomew C, Barton CJ,
Morrissey D, Lack SD. Six treatments have
positive effects at 3 months for people with
patellofemoral pain: a systematic review with
meta-analysis. J Orthop Sports Phys Ther.
2022 52:750-​768. https://doi.org/10.2519/
jospt.2022.11359
10. Center for Open Science. Charlottesville, VA,
USA. Open Science Framework. https://osf.io/
11. Page MJ, O’Connor DA, Malek M, et al. Patients’
experience of shoulder disorders: A systematic
review of qualitative studies for the OMERACT
Shoulder Core Domain Set. Rheumatology
(Oxford). 2019;58:1410-​1421. https://doi.
org/10.1093/rheumatology/kez046
12. Rethlefsen ML, Kirtley S, Waffenschmidt S.
PRISMA-S: an extension to the PRISMA state-
ment for reporting literature searches in sys-
tematic reviews. Syst Rev. 2021;10:39. https://
doi.org/10.1186/s13643-020-01542-z
13. Silva FG, Costa LOP, Hancock MJ, Palomo GA,
Costa LCM, da Silva T. No prognostic model
for people with recent-onset low back pain
has yet been demonstrated to be suitable for
use in clinical practice: a systematic review.
J Physiother. 2022;68:99-​109. https://doi.
org/10.1016/j.jphys.2022.03.009
491