Why systematic reviews matter

elsevier.com/connect/authors-update/why-systematic-reviews-matter

This article was written as part of the Cochrane International Mobility Programme.

Introduction
The vast scale of scholarly literature occasions various problems. One is how to
comprehensively record and assess the state of knowledge on a particular topic. A
potent means of doing so is that of systematic reviews. The continuous growth of
research, coupled with the demand to systematically summarize the available evidence
to inform decisions from consumers and stakeholders, led to the formal development of
systematic reviews (SRs) in the late 20th century1-3.

Systematic reviews search, appraise and collate all relevant empirical evidence in order
to provide a complete interpretation of research results. Although conventional SRs are
typically used in clinical research and social sciences, they have found application in
various other subject areas for example in advertising, education, international
development, public policy, ecology, environmental sciences, engineering and basic
science research4-7.

A brief history of systematic reviews

The first example of a systematic review was conducted in 1753 by James Lind, who
published a paper that aimed to provide a concise and unbiased summary of evidence
on scurvy8 9. However, it was not until the 1970’s and 1980’s when more attention was
paid to the growing need to improve the state of evidence synthesis.

1/4
In 1972, Archie Cochrane published a textbook titled “Effectiveness and Efficiency: Random
Reflections on Health Service”10 . Cochrane drew attention to the vital importance of
randomized control trials in determining the effectiveness of health treatments. This led
to a greater international emphasis on the need to improve research synthesis by policy
makers, academics, and clinicians3. Gradually, topic areas outside of healthcare also
adopted SRs as a way of comprehensively and systemically summarizing existing
research.

How to conduct a systematic review

If you are considering embarking on a systematic review, there are several issues you
need to contemplate if you wish to conduct one. In healthcare, for example, the first step
would be to define an explicit research question by using the PICOTS (Population,
Intervention, Comparator, Outcome, Timing, Setting) framework11 , and also register the
protocol for the review on PROSPERO (https://www.crd.york.ac.uk/prospero/), the
international database of prospectively registered systematic reviews. Protocols provide
a complete detailed description of the process by which the review will be conducted.
Registering the protocol reduces research bias, duplication of effort, resource waste, and
provides greater transparency12 . Outside of medical sciences, protocols can be uploaded
to Open Science Framework (https://osf.io).

You must adopt a comprehensive, objective and reproducible search strategy to capture
all relevant sources of evidence. In doing so, you can be confident of having incorporated
all the appropriate material for the topic at hand. A thorough search strategy should
involve multiple databases, registries, sources of grey literature
(https://onlinelibrary.london.ac.uk/resources/databases/opengrey)13 , conference
proceedings and abstracts. Following the predefined eligibility criteria, you then need to
analyze the screened search results to extract data from those publications that meet
the inclusion criteria.

Don’t forget to assess the risk of bias when applicable (i.e., in clinical research). Ideally,
these methodological steps should preferably be performed by two authors
independently, one of which is a methodologist and the other a content area expert.
Summarizing the results of the included studies and interpreting their findings in the
light of certainty of evidence and their applicability are the final steps of completing a
systematic review. You can also include a meta-analysis if applicable.

In order to assess the methodological quality of systematic reviews in biomedical

sciences, checklists like the AMSTAR – “A MeaSurement Tool to Assess systematic
Reviews”14 (https://amstar.ca) can be utilized. Lastly, the Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA)16 (http://prisma-statement.org/)
checklist, is a minimum set of items for reporting in systematic reviews: include one with
your full article. The flow diagram can also be adopted to use for non-medical research
(http://prisma-statement.org/PRISMAStatement/FlowDiagram).

2/4
Why are systematic reviews important?
Systematic reviews offer a number of benefits. For starters, they deliver a clear and
comprehensive overview of available evidence on a given topic. Moreover, SRs also help
identify research gaps in our current understanding of a field. They can highlight
methodological concerns in research studies that can be used to improve future work in
the topic area17 . Lastly, they can be used to identify questions for which the available
evidence provide clear answers and thus for which further research is not necessary 18 .

The process of conducting systematic reviews, especially for new authors, will prove to
be a worthwhile endeavour. Authors refine their knowledge on the subject area of
interest, develop new research ideas, and gain critical skills in synthesising existing
literature.

We hope that you have found this introduction to systematic reviews helpful. Additional
information about SRs can be found on the Cochrane website. If you have any questions
or observations, please feel free to comment below.

References

1. Meerpohl JJ, Herrle F, Reinders S, et al. Scientific value of systematic reviews:

survey of editors of core clinical journals. PLoS One 2012;7(5):e35732. doi:
10.1371/journal.pone.0035732 [published Online First: 2012/05/09]
2. Higgins JPT, Green S, Cochrane Collaboration. Cochrane handbook for systematic
reviews of interventions Version 5.1.0 [updated March 2011]. London: The
Cochrane Collaboration,, 2011:1 online resource. doi: 10.1002/9780470712184
3. Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval
Health Prof 2002;25(1):12-37. doi: 10.1177/0163278702025001003 [published
Online First: 2002/03/01]
4. O’Hagan EC, Matalon S, Riesenberg LA. Systematic reviews of the literature: a
better way of addressing basic science controversies: American Physiological
Society Bethesda, MD, 2018. doi: 10.1152/ajplung.00544.2017
5. Gilbody S, Wilson P, Watt I. Benefits and harms of direct to consumer advertising:
a systematic review. BMJ Quality & Safety 2005;14(4):246-50. doi:
10.1136/qshc.2004.012781
6. Pullin AS, Stewart GB. Guidelines for systematic review in conservation and
environmental management. Conservation biology 2006;20(6):1647-56. doi:
10.1111/j.1523-1739.2006.00485.x
7. Petticrew M. Systematic reviews from astronomy to zoology: myths and
misconceptions. Bmj 2001;322(7278):98-101. doi: 10.1136/bmj.322.7278.98
8. Lind J. A treatise on the scurvy. In three parts. Containing an inquiry into the
nature, causes, and cure, of that disease. London,: A. Millar 1753.
doi:10.1136/bmj.330.7482.92-a
9. Clarke M, Chalmers I. Reflections on the history of systematic reviews. BMJ Evid
Based Med 2018;23(4):121-22. doi: 10.1136/bmjebm-2018-110968 [published
3/4
Online First: 2018/06/21]
10. Cochrane AL. Effectiveness and efficiency: random reflections on health
services. London: Nuffield Provincial Hospitals Trust 1972. doi:
10.1017/cbo9781107256644
11. Santos CMdC, Pimenta CAdM, Nobre MRC. The PICO strategy for the research
question construction and evidence search. Revista latino-americana de enfermagem
2007;15(3):508-11. doi: 10.1590/s0104-11692007000300023
12. Stewart L, Moher D, Shekelle P. Why prospective registration of systematic
reviews makes sense. Syst Rev 2012;1:7. doi: 10.1186/2046-4053-1-7 [published
Online First: 2012/05/17]
13. Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic
reviews: challenges and benefits. Research synthesis methods 2014;5(3):221-34. doi:
10.1002/jrsm.1106
14. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for
systematic reviews that include randomised or non-randomised studies of
healthcare interventions, or both. Bmj 2017;358:j4008. doi: 10.1136/bmj.j4008
15. Chandler J, Churchill R, Higgins J, et al. Methodological standards for the
conduct of new Cochrane Intervention Reviews. The Cochrane Library 2013 doi:
10.4073/cpg.2016.3
16. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic
reviews and meta-analyses: the PRISMA statement. Annals of internal medicine
2009;151(4):264-69. doi: 10.7326/0003-4819-151-4-200908180-00135
17. Eagly AH, Wood W. Using research syntheses to plan future research. The
Handbook of Research Synthesis: Russell Sage Foundation 1994:485-500. doi:
10.1002/(SICI)1097-0258(19970330)16:6<713::AID-SIM430>3.0.CO;2-4
18. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of
research evidence. Obstet Gynecol 2009;114(6):1341-5. doi:
10.1097/AOG.0b013e3181c3020d [published Online First: 2009/11/26]

4/4