WHATS, HOWS, AND

WHYS OF QUALITATIVE
SYSTEMATIC REVIEWS
Outline

1. Systematic Reviews
2. Qualitative Systematic Review Process
3. Examples of Qualitative Systematic Reviews
4. Qualitative Systematic Review Designs
5. Deliberations and Considerations
6. Questions
What is a systematic review?

Empirical research
that synthesizes,
analyses, and/or
interprets findings
across studies to
answer a research
question.
 Systematic Review Literature Review Critical Review
Definition Synthesis of primary research that Summary of literature on a topic using Critique of literature on a topic using informal
identifies, screens, synthesis, and informal (non-systematic) methods to collect (non-systematic) methods to collect and
appraises empirical literature to answer a and summarize themes across studies. critically analyze studies.
research question.

Purpose Answer a focused research question. Provide a summary or overview of a topic. Provide a critique of literature on a topic.
Question Focused and answered through analysis Can be general or focused. Can be answered Critical and answered by identifying
of primary studies. through summarizing. weaknesses, gaps, inconsistencies in the
literature.

Example What is the impact of high-stakes testing What research has been conducted on What is the scope and quality of the literature
qualitative on curriculum? (Au, 2007) sustainability in post-secondary education? on student stress in higher education?
(Vaughter, et al., 2013) (Robotham & Julian, 2006)
question
Process Systematic and predetermined Iterative or systematic Can be iterative or systematic
Inclusion/Exclusion criteria No predetermined inclusion/exclusion criteria Search a few relevant databases
Systematic search strategy Search in a few relevant databases Appraisal, but often no tool
Search in ALL relevant databases Little or no appraisal Critique of the scope, focus, methods, and/or
Appraisal tool Summary of topics discussed weaknesses of the literature.
Analysis of findings

Authors Usually three or more Usually one or two Usually one or two
Timeline Usually one to two years Usually three to nine months Usually three to nine months
Analysis Statistical or qualitative data analysis Summary Summary and evaluation
methods used
Write-Up Usually follow PRISMA guidelines Variety of options Variety of options
Adapted from: Kysh, Lynn (2013): Difference between a systematic review and a literature review. [figshare]. Available at: http://dx.doi.org/10.6084/m9.figshare.766364
Types of Systematic Reviews

Systematic review: broad term for qualitative,

quantitative, and mixed methods reviews
Quantitative systematic review or meta-
analysis: synthesis and analysis of quantitative
studies
Mixed methods systematic review: synthesis
of quant, qual, and mixed methods studies
Qualitative systematic review: aggregates,
integrates, and/or interprets the findings from
(usually) qualitative studies (Sandelowski &
Barroso, 2007)
What are the effects of
high-stakes testing on
curriculum?
Overview of the Qualitative Systematic Review Process

Search
Write-
Protocol and Analyze
Up
Screen
Research
Run Search Analyze Data Make Tables
Question(s)

Inclusion/ Synthesize, Describe and

Screen
Exclusion Interpret Interpret
Abstracts
Criteria Findings findings

Search Screen Full-

Strategy Texts

Extraction
and Appraisal
Tools
Au (2007), Example Inclusion and Exclusion Criteria

Inclusion Exclusion
 Original research  Relationship between high-stakes testing
and retention
 Qualitative methods
 Focused on student teachers
 United States
 Address relationship between high-
stakes testing and either curriculum or
instruction or both.
Source: Duke, T.S.,
(2011): Lesbian, Gay,
Bisexual, and
Transgender Youth with
Disabilities: A Meta-
Synthesis, Journal of
LGBT Youth, 8:1, 1-52
 PRISMA Flow Diagram
Source: Braun, A., Trivedi, D. P.,
Dickinson, A., Hamilton, L., Goodman, C.,
Gage, H., … Manthorpe, J. (2019).
Managing behavioural and psychological
symptoms in community dwelling older
people with dementia: A systematic review
of qualitative studies. Dementia, 18(7–8),
2950–2970.
“Figure. A thematic analysis of young
people’s views on the role of education,
training, employment and careers, and
financial circumstances in unintended
teenage pregnancy”

Source: Harden A, Brunton G, Fletcher

A, Oakley A, Burchett H, Backhans M
(2006). Young people, pregnancy and
social exclusion: A systematic synthesis
of research evidence to identify
effective, appropriate and promising
approaches for prevention and support.
London: EPPI-Centre, Social Science
Research Unit, Institute of Education,
University of London
DESIGNS AND DELIBERATIONS
Some Types of Qualitative Systematic Reviews

Qualitative Systematic Reviews

Meta- Critical Critical

ethnography Meta-study Interpretive Construct
Synthesis Synthesis

For others see: Barnett-Page, E., Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical
review. BMC Medical Resesearch Methodolology, 9(59). doi:10.1186/1471-2288-9-59
Deliberations and Considerations
To what extent should qualitative research be ‘systematic?’ What is systematic? (e.g.
Campbell et al, 2011).

How to choose sources? Does the search need to be as comprehensive as quantitative

systematic reviews? Should sampling of studies be purposive or exhaustive? (e.g.,
Booth, 2016).

What types of studies should be included? What is the role of the appraisal tool? How
do we appraise qualitative research? (e.g., Dixon-Woods et al., 2004).

To what extent do journal articles convey rich contexts and personal experience?

What is the role of reflexivity in qualitative systematic reviews?

Philosophy of inquiry? (e.g., Gordon, 2016)

References
Au, W. (2007). High-Stakes Testing and Curricular Control: A Qualitative Metasynthesis. Educational Researcher, 36(5), 258–267.

Barnett-Page, E., Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9, 59.

Booth, A. (2016). Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev 5, 74.

Braun, A., Trivedi, D. P., Dickinson, A., Hamilton, L., Goodman, C., Gage, H., … Manthorpe, J. (2019). Managing behavioural and
psychological symptoms in community dwelling older people with dementia: A systematic review of qualitative studies. Dementia, 18(7–8),
2950–2970.

Campbell R, Pound P, Morgan M, et al. (2011) Evaluating meta-ethnography: systematic analysis and synthesis of qualitative
research. Health Technol Assess,15: 1–164.

Dixon-Woods M, Shaw RL, Agarwal S, Smith JA: The problem of appraising qualitative research. Qual Saf Health Care. 2004, 13: 223-225.

Duke, T.S., (2011): Lesbian, Gay, Bisexual, and Transgender Youth with Disabilities: A Meta-Synthesis, Journal of LGBT Youth, 8:1, 1-52

Gordon M. (2016). Are we talking the same paradigm? Considering methodological choices in health education systematic review. Medical
teacher, 38(7), 746–750.

Moher D., Liberati A., Tetzlaff J., Altman D.G., The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-
Analyses: The PRISMA Statement. BMJ 2009;339:b2535,

Wolgemuth, J.R., Agosto, V., Yu Hin Lam, G., Riley, M.W., Jones, R., & Hicks, T. (2016) Storying transition-to-work for/and youth on the
autism spectrum in the United States: a critical construct synthesis of academic literature, Disability & Society, 31:6, 777-797
Resources
Atkins, S., Lewin, S., Smith, H. et al. Conducting a meta-ethnography of qualitative literature: Lessons learnt. BMC Med
Res Methodol 8, 21 (2008) doi:10.1186/1471-2288-8-21

Dixon-Woods, M., Cavers, D., Agarwal, S. et al. Conducting a critical interpretive synthesis of the literature on access to
healthcare by vulnerable groups. BMC Med Res Methodol 6, 35 (2006) doi:10.1186/1471-2288-6-35

Noblit, G. W., Hare, R. W., Hare, R. D. (1988). Meta-Ethnography: Synthesizing Qualitative Studies. India: SAGE
Publications.

Paterson, B.L., Thorne, S.E., Canam, C., & Jillings, C. (2001). Meta-Study of Qualitative Health Research: A Practical
Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, Sage.

Saini, M., & Schlonsky, A. (2007). Systematic Synthesis of Qualitative Research. Social Work Research Methods. New
York: Springer.

Sandelowski, M. and Barroso, J. (2007) Handbook for Synthesizing Qualitative Research. Springer Publishing Company,
New York.

Wolgemuth, J. R., Hicks, T., & Agosto, V. (2017). Unpacking Assumptions in Research Synthesis: A Critical Construct
Synthesis Approach. Educational Researcher, 46(3), 131–139. https://doi.org/10.3102/0013189X17703946
CONTACT ME
Jennifer Wolgemuth, Ph.D.
Associate Professor of Qualitative Research
University of South Florida
4202 E. Fowler Ave
Tampa, FL 33620

jrwolgemuth@usf.edu