Learning Module: Systematic

Literature Review
Academic Writing
What is it?
A review of a clearly formulated question that uses

systematic and explicit methods to:

 Identify
 Select
 Critically Appraise relevant research
 Analyze studies that have been included in the review

Methods used to analyze data:

 Meta-Analysis (Quantitative Studies)
 Thematic Analysis (Qualitative Studies)
The Purpose
Research Systematic Review:
Question • Identification of current Literature
• Assessment of quality and
limitations of current literature
• Assessment of research potential on
the topic

Outcome:
• Answer to the research question
• Guidance for planning of research
• Actual value of novel research
The Process
Scoping Planning

Screening Searching

Defining
Eligibility
Methods

Presenting
The Process: Scoping

●
Formulating ●
Estimating
1 Research 2 3 ●
Involvement
Stage
Questions
The Process: Scoping
Formulating Research Questions:
 Patient/Person: who does this relate to?
 Intervention (or cause, prognosis): what is the intervention or
cause?
 Comparison (Is there something to compare the intervention
to?)
 Outcome (What outcome are you interested in?).
The Process: Scoping
Estimating:
Establish whether this question has

 already been answered in the published literature,

 is registered as an ongoing review (e.g. search in DARE,
which contains abstracts of quality assessed systematic
reviews and details of all Cochrane reviews and protocols,
or NHS EED which contains abstracts of quality assessed
economic evaluations: http://www.crd.york.ac.uk/crdweb/)
The Process: Scoping
Determine who all will be involved:
 Experts from the subject being reviewed
 Experts in systematic review methodology
 Experts in information retrieval
 Experts in statistics
 Experts in other aspects e.g. health economics if required
The Process: Planning

●
Creating Search ●
Inclusion and
1 2
terms Exclusion Criteria
The Process: Planning
Search Terms

Specific searches using a well thought out collection of keywords

(or ‘search string’) is required to whittle down hundreds, if not

thousands, of published studies in any one paradigm.
For the example, a search for investigating the efficacy of a

neuroprotective drug: (drug-x OR drug-x-alternative-name(s) OR

drug-xalternative-spelling(s)) AND (stroke OR isch(a)emia OR
isch(a)emic OR cerebrovascular accident OR encephalic vascular
accident).
The Process: Planning
Inclusion Criteria
 Scope of the research
 Key measurement variables

Exclusion Criteria
 Participants
 Time of Publication
 Language of Publication
 Research Design
 Full-text availability
The Process: Inspecting

1 ●
Searching 2 ●
Inspecting
The Process: Inspecting
Searching:
 Select databases that are relevant to your topic area (e.g. Medline, EMBASE, ISI Web of

Knowledge)
 Consider which parts of articles you want to search (e.g. abstract, full text, title)
 Consider using limits and filters within particular databases to search by article type (e.g.

review articles and research syntheses, empirical articles), subject categories, sub-headings, etc
 Consider using Boolean search operators to broaden or narrow your search: ‘AND’ (a search for

all of your search terms; e.g., ‘heart AND lung’), ‘OR’ (a search for at least one of your search

terms; e.g., ‘heart OR lung’), ‘NOT’ (a search to exclude certain search terms; e.g., ‘heart not

lung’)
 Consider using a truncation symbol to look for all words starting with a particular combination

of letters (use ‘$’ or ‘*’ depending on the database) (e.g. ‘dep$’ or ‘dep*’ will retrieve ‘depression’,

‘depressive’, ‘depressed’, etc)

The Process: Inspecting
Searching:
 Consider using a wildcard symbol to stand-in for one character, if that character exists
(‘#’ or ‘?’ or ‘$’ depending on the database) (e.g. ‘wom#n’ will find ‘women’ or ‘woman’)
 Consider using the truncation symbol (*) between words to match words (e.g.
‘midsummer * dream’ will return results that contain the exact phrase, ‘a midsummer
night’s dream’);  Consider using parentheses because commands within these run
first (e.g. ‘(smok* OR tobacco)’)
 Consider searching by proximity, to search for one word within n number of words of
another word (‘adjn’ or ‘NEAR/n’ depending on the database) (e.g. ‘patient adj3
anxiety’ will retrieve records where ‘patient’ and ‘anxiety’ appear within three words of
each other)
 Consider excluding unwanted meanings (e.g. ‘jaguar NOT(car OR cars)’, if you want
the animal);
The Process: Inspecting
Inspect:
 Do the search results suggest that your inclusion and
exclusion criteria are reliable and effective in identifying
potentially relevant articles and balancing specificity and
sensitivity?
 If not, do you need to revise your inclusion and exclusion
criteria, or search terms?
 Do the search results reveal new search terms that would
make a useful addition to your existing search terms?
The Process: Screening

●
Reading titles and
1 ●
Exporting References 2
abstracts
The Process: Screening
Exporting:

Export citations to: EndNote/ Refworks

This will
 Save time
 Organize reference lists in required format
 Help remove duplicates from the reference list
The Process: Screening
Obtain and Read May Pass the selection
Abstract criteria

Yes No

Obtain Full Text Reject and Move on

The Process: Eligibility

●
Quantitative vs.
1 ●
Deciding Eligibility 2
Qualitative
The Process: Eligibility
Obtaining Full Texts Matching against
for potentially inclusion and
relevant studies exclusion criteria

Focusing on
methods
adopted

Focusing on Assessing reliability

Results and quality of
obtained evidence obtained
The Process: Eligibility (Defining Methods)
Quantitative vs. Qualitative
 Meta-analysis would be appropriate when a collection of studies:
 Report quantitative results, rather than qualitative findings, or theory
 Examine the same or similar constructs/relationships and are therefore meaningfully comparable
 Derived from similar research designs (e.g., it would not be appropriate to combine experimental

trials of treatments for depression, with observational studies in which the level of depression was

correlated with the level of service received – although, in this case, separate meta-analyses might

be appropriate)
 Results are bivariate/zero order relationships/single-degree of freedom contrasts (i.e. the

relationship between two variables), rather than relationships that have been adjusted for the

effect of additional variables (e.g., partial or multivariate effects)

 Have results that can be configured as standardized effect sizes
The Process: Eligibility (Defining Methods)

Quantitative vs. Qualitative

 A group of studies are so methodologically diverse as to make
meta-analytic aggregation impractical
 Conceptual and methodological approaches to research on a topic
have changed over time and you argue that there is benefit in
reviewing all of this research
 When developing a new theory or critiquing one or more existing
theories
 When reviewing measurement approaches in a particular literature
The Process: Presenting
Section 1: Introduction
Background

 Explanation of Key terms

 Definitions and concepts

Aims of the Study

Objectives of the study

Gaps in Knowledge
Background Sample
Background Sample
The Process: Presenting
Section 2: Method
 Need for structured Approach
 Selection Criteria
 Exclusion Criteria
 Search Methods
 Data Extraction
 Data Synthesis
Method Sample
Method Sample
The Process: Presenting
Section 3: Results
 Characteristics of Included studies
 PRISMA chart
 Screening Methods
 Quality Assessment of Included Studies
 Overview of Included Studies
 Data Extraction
Result Sample
Result Sample
Result Sample
Result Sample
Result Sample
The Process: Presenting
Section 4: Discussion

Summarize and discuss the findings and conclusions of the review in a balanced and impartial

way, in the context of previous theory, evidence and practice

Explicitly and intuitively link your conclusions to the evidence reviewed

Discuss the strengths and limitations of the literature and, by implication, the review, including

considering the scientific quality of included studies and methodological problems in the

literature (e.g., methodological rigor or lack thereof, the amount of evidence, its consistency, and

its methodological diversity)

Conclusions should be tempered by the flaws and weaknesses in the evidence. Perhaps propose a

new conceptualization or theory which accounts for inconsistencies.

Establish to what extent existing research has progressed towards clarifying a particular

problem/formulate general statements or an overarching conceptualization.

The Process: Presenting
Quantitative or qualitative reviews may conclude that the available evidence

suggests one of four possibilities

 1) A hypothesis is correct, at least based on the present evidence
 2) A hypothesis, although not proven, is currently the best guess and should be assumed
to be true until contrary evidence emerges
 3) It is not clear whether a hypothesis is true or false
 4) A hypothesis is false

Comment on, evaluate, extend, or develop theory

Draw conclusions and make recommendations for practice

Describe directions for future theory, evidence and practice by pointing out

remaining unresolved issues

THANKYOU!