by John W. Creswell, Ph.D.

Department of Educational Psychology,

University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research

University of Pretoria, October 21, 2008

© Please do not duplicate or use these slides without the express permission of the author.
1
Topics

Introductions
Positioning myself
A definition of mixed methods research
Steps in the process of designing a mixed methods
study

2
Positioning myself

View research as set of interactive components; not

always linear
Focus on rigorous data collection and analysis
Work as an applied research methodogist
Trained in quantitative, self-trained in qualitative,
lst generation mixed methods writer
Serve as a consultant on mixed methods on projects
Work on projects in an order not in proposal format;
then I reassemble into proper format
A few more thoughts about myself…

Published by Sage Publications, Pearson Education (Merrill Educ.)

A Definition of Mixed Methods
Research
How would you combine two types of data?

Qualitative Quantitative Numeric Data

Text Data
2 3 4 2 5 2 3 1 1 2 3 4 2 3 3 2 1
 This is a sample of a text file of 11534123144554121
words that might be collected 43351423155221535
on interview transcripts, 13153225132443124
observation fieldnotes, or
2241554215
optically-scanned documents.
Quantitative Data Qualitative Data

Mixed Methods

Method
Methodology
Paradigm
Perspective Use of mixed methods
in other designs
7
A mixed methods researcher…

Collects both quantitative and qualitative data

“Mixes” them
“Mixes” them at the same time (concurrently) or one
after the other (sequentially)
Emphasizes both equally or unequally
A definition
• Mixed methods research is both a method and
methodology for conducting research that involves
collecting, analyzing, and integrating quantitative and
qualitative research in a single study or a longitudinal
program of inquiry.
• The purpose of this form of research is that both
qualitative and quantitative research, in combination,
provide a better understanding of a research problem
or issue than either research approach alone.

9
Collecting both quantitative and
qualitative data
Quantitative data Qualitative data
Interviews
Instruments
Observations
Checklists
Documents
Records Audio-visual materials

10
Quantitative and qualitative data
analysis

Quantitative analysis Qualitative analysis

Use statistical Use text and
analysis, images,
For description For coding
For comparing For theme
groups development
For relating For relating themes
variables

11
Mixing or linking the data

Converge data:

Qual Results Quan

Connect data:

Qual Quan Results

Embed the data:

Quan data
Qual data
12
Typical situations in which mixed methods
is used…

To compare results from quantitative and qualitative

research
 To use qualitative research to help explain
quantitative findings
 To explore using qualitative research and then to
generalize findings to a large population using
quantitative research
 To develop an instrument because none are available
or useful
To augment an experiment with qualitative data
 What is the reason for using mixed methods?

 The insufficient argument – either quantitative or qualitative may be

insufficient by itself
 Multiple angles argument – quantitative and qualitative approaches
provide different “pictures”
 The more-evidence-the-better argument – combined quantitative and
qualitative provides more evidence
 Community of practice argument – mixed methods may be the
preferred approach within a scholarly community
 Eager-to-learn argument – it is the latest methodology
 “Its intuitive” argument – it mirrors “real life”
Designing a Mixed Methods Study

Preliminary considerations
Creating a title
Posing a general question
Listing the types of data collection and analysis
Making explicit your worldview
Identifying your research design
Drawing a figure of your design
Writing a purpose statement
Writing research questions
Completing a research plan
• Research problem
• Content – any topics
• Fit the problem to mixed methods (arguments)

•Access to both qualitative and quantitative data

•Background and resources

•Receptive audience

16
Create a working title

Writing the title

Short
Topic
Participants
Include the words “Mixed methods”
Neutral –neither quan or qual

17
Pose the general question to be answered

Write it as a question
Look to see how it is phrased
Make sure that it is specific enough and focused (an
answerable question)
Ask yourself, “when I end the study, what question
would like to have answered?”

18
 List your types of data collection (a
review)
Quantitative data Qualitative data (open-
(closed-ended) ended)
Interviews
Instruments
Observations
Behavioral Documents
checklists Audio-visual materials
Records

19
Activity – List your sources of data

Quantitative Sources of Qualitative Sources of

Data Data

20
List your approach to data analysis (a
review)

Quantitative analysis Qualitative analysis

Use statistical Use text and
analysis, images,
For description For coding
For comparing For theme
groups development
For relating For relating themes
variables Design-type
Design-type

21
Determine your worldview…
Postpositivism
Determination
Reductionism
Empirical observation and
measurement
Theory verification
Advocacy/Participatory
Political
Empowerment issue-oriented
Collaborative
Change-oriented
Constructivism
Understanding
Multiple participant meanings
Social and historical
construction
Theory generation
Pragmatism
Consequences of actions
Problem-centered
Pluralistic
Real-world practice oriented
22
Several stances on philosophy in mixed
methods…

One paradigm (pragmatism, transformative) (Tashakkori

& Teddlie, 2003; Mertens, 2003)
Multiple paradigms (dialectic perspective) (Greene, 2007)
Linking paradigms to design features) (Creswell & Plano
Clark, 2007)
Epistemological stance (ontology, epistemology, axiology,
methodology) (Guba & Lincoln, 2005)
Shared beliefs in a research field (Morgan, 2007)

What it is
How it informs your study

23
Make explicit your interpretive lens (theory)

Advocacy lens (feminist, racial, ethnic, disability,

sexual orientation) (Mertens, 2003)
Social science lens (social science theory)

Components:
What is it
Who has used it in your field
How it will shape your study (rephrase your guiding
research question, if a lens applies)

24
State your procedures (methods)

Procedures for handling your qualitative and quantitative

data

Sequence – concurrent or sequential or both

Emphasis – emphasis on qualitative or quantitative
Sometimes both concurrent and sequential phases are
used
Designs may include more than two phases
Think about using a simple, elegant design

25
Parsimonious designs (Creswell & Plano Clark, 2007)
Concurrent Mixed Methods Designs
Triangulation Design

QUAN QUAL
Data & Data &
Interpretation
Results Results

Embedded Design

QUAN Intervention QUAN

Pre-test Post-test
Interpretation
Data & Data &
qual
Results Results
Process

26
 Sequential designs
Explanatory Design
QUAN qual
Data & Data &
Results Following up
Results Interpretation

Exploratory Design
QUAL quan
Data & Data &
Interpretation
Results Building to
Results

Sequential Embedded Design

Before-
QUAN After-
Intervention
intervention
Trial intervention
qual qual Interpretation

27
 Design options

Design Name Equal priority QUAN emphasis QUAL emphasis

Concurrent, triangulation QUAL+QUAN QUAN+qual QUAL+quan

Concurrent, embedded n/a QUAN(qual) QUAL(quan)

Explanatory, sequential, quan first QUANQUAL QUANqual quanQUAL

Exploratory sequential, qual first QUALQUAN qualQUAN QUALquan

Sequential, embedded n/a (qual) QUAN (quan) QUAL

QUAN (qual) QUAL (quan)

28
 Explanatory sequential design
Quantitative Case Selection Qualitative Interpretation –
Quantitative
Data Collection Data Analysis Data Analysis based on quan
(quan) (QUAL) ad QUAL results

+
Qualitative
Data Collection

Quantitative Data* Quantitative Analysis Case Selection Qualitative Analysis Interpretation

Number of cigarettes Graphic plot of CES- Selected 5 cases Description of each Why did changes in
CES-D6 D6 scores over time maximally varying case smoking occur?
for each participant Identified critical Identification of life
Qualitative Data* Graphic plot of months in which events occurring
cigarettes/day values smoking varied during critical
Semi-structured
over time for each months where
interviews, audio
participant smoking increased or
recorded and
decreased
transcribed
Thematic analysis of
life events for each
* Data collected 10 times
case
over the course of a
calendar year for 40 Cross-case thematic
participants analysis

Source: Creswell, Plano Clark, Shope, McVea. (in progress)

 Phase I Qualitative Research - Year 1

Qualitative Data Collection Unstructured Interviews -

50 participants
8 observations at the site
16 documents

Qualitative Data Analysis Text Analysis: Using QSR N6

Development of codes and themes

Qualitative Findings for each site

Phase II Quantitative Research - Year 2

Create approximately a 80-item
Quantitative Instrument Development instrument plus demographics

Administer survey to 500 individuals

Quantitative Test of the Instrument
Determine factor structure of items and
conduct reliability analysis for scales

Quantitative Results
Determine how groups differ
using ANOVA test

Exploratory sequential design

 Embedded research design

Experiment
Intervention
QUAN QUAN
Data collection Data collection
Pre-test Post-test

Process – collection
and analysis of qualitative
data
(before, during, after trial)
32
Mixing the quan and qual data

Type of Mixing Type of Design Why Mixing Where Mixing

Occurs Occurs in
Research Process
Connecting Sequential One phase builds Between data
on the other analysis (Phase 1)
and data
collection (Phase
2)
Merging Concurrent Bring results After analysis of
together both quan and
qual – typically in
discussion
Embedding Sequential or Either building or Either between
Concurrent bringing results phases or in
together discussion after
analysis 33
Methodological issues
Concurrent Designs
 Use strategies to explore
contradictory findings
 Use parallel questions
 Select sub-sample of
quantitative for
qualitative
 Be sensitive to bias from
one data collection to the
other
Sequential designs
 In Explanatory Design,
select qual sub-sample
from quan sample
 In Explanatory Design,
consider alternatives for
followup qual sampling
 In Exploratory Design,
samples can differ
 In Exploratory Instrument
Design, consider qual data
analysis approaches for
developing instrument
34
Write a purpose statement for a
triangulation design
This mixed methods study will address _________________ (overall
content-aim of the study). A triangulation mixed methods design will
be used, and it is a type of design in which different but
complementary data will be collected on the same topic. In this study,
_________________ (quantitative instruments) will be used to test the
theory of _____________ (the theory) that predicts that
__________________ (independent variables) will influence
________________ (positively, negatively) the __________ (dependent
variables or outcomes) for ____________ (participants) at __________
(the research site). Concurrent with this data collection, qualitative
_______________ (type of qualitative data, such as interviews) will
explore ____________ (the central phenomenon) for _________________
(participants) at _____________(site). The reason for collecting both
quantitative and qualitative data are to bring together the strengths of
both forms of research to ___________________ (e.g., compare results,
validate results, corroborate results).

35
Write a purpose statement for
a embedded design
This mixed methods study will address _________________ (overall
content aim of the study). An embedded mixed method design will be
used, and it is a design in which one data set provides a supportive,
secondary role in a study based primarily on the other data set. The
primary purpose of this study will use ________________ (quantitative
instruments) to test the theory of _____________ (the theory) that
predicts that __________________ (independent variables) will
influence ________________ (positively, negatively) the __________
(dependent variables or outcomes) for ____________ (participants) at
__________ (the research site). A secondary purpose will be to gather
qualitative data _______________ (type of qualitative data, such as
interviews) that will explore ____________ (the central phenomenon)
for _________________ (participants) at _____________(site). The reason
for collecting the secondary database is ________________ (e.g., to
address different question, to provide support for the primary
purpose).

36
Write a purpose statement for
an explanatory design
This study will address _______________ (content-aim of the study). An
explanatory mixed methods design will be used, and it will involve
collecting qualitative data after a quantitative phase in order to
explain or follow up on the quantitative data in more depth. In the
first quantitative phase of the study, ______________ instrument data
to be collected from _______________ (participants) at ___________
(research site) to test _______ (the theory) that explains why
______________ independent variables) relate to the ____________
(dependent variables). The second qualitative phase will be
conducted because ______________ (intent of the qualitative phase). In
this exploratory follow-up, the ______________ (central phenomenon)
will be tentatively explored with ___________ (participants) at
_____________ (the research site). The reason for the exploratory
follow-up is to _______________ (e.g., to help explain or build upon
initial quantitative results).

37
Write a purpose statement for
an exploratory design
This study addresses _________________ (content-area of the study). The
purpose of this exploratory sequential design will be to __________________
(e.g., develop an test an instrument, generate a taxonomy). The first phase
of the study will be a qualitative exploration of ________________ (the
central phenomenon) by collecting ___________________ (types of data)
from _________________ (participants) at _____________ (the research site).
The second quantitative phase will follow up on the qualitative phase for
the purpose of __________(intent of this followup). In the quantitative
phase, ___________ (instrument data) will be collected from ____________
(participants) at ______________ (research site). Quantitative research
questions/hypotheses will be formulated after the completion of the initial
qualitative phase. The reason for collecting qualitative data initially is that
_____________________ (e.g., instruments are not available, variables are
not known, there is little guiding theory).

38
Qualitative research questions

Qualitative central question

Begin with “what” or “how”
Focus on single phenomenon
Use exploratory verbs (discover, understand, explore)
Non-directional language
A general question (allowing participants’ perspectives
to emerge)

39
Quantitative research questions

Can be hypotheses or questions

State variables – independent, dependent, mediating,
covariates
Develop from theory
Use distinct measures for independent and
dependent variables
Order variables from independent to dependent

40
Writing research questions/hypotheses in
mixed methods research

Write qualitative research questions and write quantitative

research questions/hypotheses

Also write a mixed methods research question

Write these questions separately

41
A new type of research question: A mixed
methods question
Three ways to write this question:

Methodologically-focused:

 To what extent do the qualitative results confirm the quantitative results?

Content-focused:

 How do the interviews with adolescent boys support the quantitative

results that their self-esteem changes during the middle school years?

Hybrid of quantitative and qualitative elements:

 What results emerge from comparing the exploratory qualitative data

about boy’s self-esteem with outcome quantitative instrument data
measured on a self-esteem instrument?
Order the topics for your plan

 Title
 Introduction
 (Problem)
 Worldview/theoretical lens
 Audience
 Purpose Statement
 Research Questions
 (Literature Review)
 Methods
 Type of Mixed Methods Design (also add definition of mixed methods)
 Types of Data Collection
 Types of Data Analysis
 Sequence/Emphasis/ Mixing Procedures
 Figure of procedures
 Anticipated methodological issues
 Ethical issues anticipated
 Validity issues
 Researcher resources and skills
 References, Appendices

43
Additional resources
Books:
 Creswell, J. W., & Plano Clark, V. L. (2007). Designing and
conducting mixed methods research. Thousand Oaks, CA: Sage.
 Creswell, J. W. (2008). Research design: Qualitative,
quantitative, and mixed methods approaches (3rd ed.). Thousand
Oaks, CA: Sage Publications.
 Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco: Jossey-Bass.
 Mertens, D. M. (2005). Research methods in education and
psychology: Integrating diversity with quantitative and qualitative
approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
 Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods
reader. Thousand Oaks, CA: Sage.
 Tashakkori, A. & Teddlie, C. (1998). Mixed methodology:
Combining qualitative and quantitative approaches. Thousand
Oaks, CA: Sage Publications.
 Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed
methods in social and behavioral research. Thousand Oaks, CA:
Sage Publications.

44
Additional resources
Articles and Chapters:

 Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method
evaluation designs. Educational Evaluation and Policy Analysis, 15 (2), 195-207.
 Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced
mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.),
Handbook of mixed methods in social and behavioral research (pp. 209-240).
Thousand Oaks, CA: Sage.
 Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues
in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in
mixed methods research. London: Sage.
 Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual
framework for mixed-method evaluation designs. Educational Evaluation and
Policy Analysis, 11 (3), 255-274.
 Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods
sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-
20.
 Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed
Methods Research, 1, 48-76.
 Morse, J. M. (1991). Approaches to qualitative-quantitative methodological
triangulation. Nursing Research, 40, 120-123.

45
 by John W. Creswell, Ph.D.
Department of Educational Psychology,
University of Nebraska-Lincoln, and Founding Co-Editor, Journal of
Mixed Methods Research

University of Pretoria, October 21, 2008

© Please do not duplicate or use these slides without the express permission of the author.
46