CHAPTER

LOCATII{G A]\D REVIE\NI{G

MIXED METHODS STTJDIES

fter developing an undersunding of mixed methods research and

recognizing some preliminary considerations, the next step is to
read some examples of pubrished studies in the literature, to
apply
the definidon given in Chapter 1 and gain an overall perspective about
designing and conducting mixed methods research. The first task is
to locate
potential mixed methods studies and. review them to learn how
they'are
composed. To this end, we begin by providing some suggestions for
search-
ing the scholarly literature. Once mixed methods studies are found, judg-
a
rnent needs to be made as ro whether they are good examples
of mixed
methods research. We provide a checklist to use and tools: a notation
sysrem
and visual diagrams that we use to help us understand the. procedures
of
potential mixed methods studies. Finally, we present four
complete pub_
lished studies and review them for their mixed merhods .o*pon.nrs,
using
the features of the checklist.
This chapter addresses

a How to locate mixed methods studies

a A notation sys[em and visual diagrams that depict the methods,
procedures, and products of mixed methods studies
o A checklist for reviewing mixed methods studies
o The main methodological features of the sample mixed methods
studies
a similarities and differences among the four studies

3B
 Locating and Reuiewing Mixed Methods Studies t 39

SEARCHING FOR AND REVIE\rING O

MDGD METHODS STUDIES

Researchers benefit from knowing how to identify mixed methods studies

published in the literature. These studies can serve as concrete illusrations
for how mixed methods studies are designed, conducted, and reported'
Examples of published studies can be used to inform others (such as gradu-
ate committee members) as to what this rype of research looks like. Research-
ers looking for mixed method studies need strategies for searching databases
and criteria with which to iudge whether a study is a good example of mixed
methods research.

Use Search Terms

Mixed methods srudies can be difficult to locate in the literature

becauSe many researchers have not used the term mixed metbods to
describe their research when they used a mixed methods design. It is
simply too new a concept. Also, different disciplines may use different
terms for naming this research approach. Based on our extensive search of
the literature, we have developed a short list of terms that is useful for
searching for mixed methods studies within electronic databases and iour-
nal archives. These terms include

mixed merhod* (where * is a wildcard that will allow hits for "mixed
method," "mixed methods," and "mixed methodology")
o quantitative AND qualitative
o multimethod

Note that the second search term uses the logic operator AND (quantita-
tive AND qualitative). This requires that both words appear in the document
so it will satisry the search criteria, If too many articles are found, try limiting
the search so that the terms must appear within the abstract or restricting it
to recent years. If not enough articles result, try searching for combinations of
Common data collection techniques, such aS "survey AND interview'"

Are the Studies Mixed Methods Research?

Once we identify potential studies in the literature, we must still

review the studies to determine whether they are mixed methods studies.
40 o DESIGNING AND CONDUCTING MDGD METHODS RESEARCH

Certainly, the definition provided in Chapter 1 offers guidance to help make

this determination. Our approach is to use the definition and then focus on
key elements that characterize mixed methods research. We first look
through an article to see if the author mentions the term mixed metbods
'We
researcb or a comparable term. especially examine the title and the
methods section for this term. Next we review the article to see if the
aurhor has collected both quantitative and qualitative data. A quick read
through rhe methods section of the article will help make this determina-
tion. Following this, we look for specific qualitative and quantitative data
analysis approaches (e.g., coding or theme development, use of descriptive
or inferenrial statistics). I|fe review the methods section again to make this
determination. Then we search through the article to see if the author pre-
sents a discussion about the "reasons" for using mixed methods research or
the statement of a research problem that is best suited for mixed methods
research.'W'e would find such a passage in the introduction, in the methods
section, or in the discussion at the end of the article. Also, we would look
for references to the mixed methods literature by scanning the reference
list at the end of the article or the references cited within the text of the
study. Finally, recognizing the basic elements of both quantitative and qual-
itative research (from Chapter 2), we would review the entire article for
elements of both approaches. lVe might ask, are multiple worldviews used?
Are both qualitative and quantitative research questions raised? Are con-
clusions drawn from both the qualitative and quantitative data? Finding
mixed methods components through all of these reviews certainly points
toward a mixed methods study. Finding fewer than all elements indicates
one of those mixed methods studies that may be in the "gray area," as we
discussed in Chapter 1. For these gray area studies, we need to determine
whether the study is close enough to be considered mixed methods.

o A MDGD METHODS NOTATION SYSTEM

AND VISUAL DTAGMMS

Before turning our attention to reviewing published mixed methods studies,

two important mixed methods research tools will assist in reviewing the stud-
ies. These tools are a notation system and visual diagrams for describing the
procedures, methods, and products of mixed methods studies. The notation
system and visual diagrams have a substantial history of use in the mixed
methods literature. They are usefi:l tools for designing and communicating
the complexiry inherent in mixed methods designs. Because of their value for
understanding mixed methods approaches, we will briefly introduce these
two tools before presenting the sample studies.
 Locating and Reuiewing Mixed Metbods studies o 4I

To facilitate the discussion of mixed methods design features, a notation

the
system, first used by Morse (1991), now appears extensively throughout
mixed methods literature. Her notation uses pluses (+) to indicate methods
that occur at the same dme and arrows ()) to indicate methods that occur
in a sequence. Plano Clark (2005) added the use of parentheses to indicate
methods that are embedded within other methods. In addition, Morse's
notation system designates the relative importance of the methods within
the study. (This topic will be discussed further in Chapter 4') The primary
method is indicated with uppercase letters (i.e., QUAN or QUAI) and the
secondary method with lowercase letters (i.e., quan or qual)' The shorthand
.,quan,, and "qual" is used to illustrate the equal starure of the two methods
(ic., both abbreviations have the same number of letters and same format)'
This shorthand notation can be very helpful for describing the overall design
of a study. Consider the following examples of using this notatioft sfstelTl:

QUAN + QUAI: This notation indicares that both the quantitative and
qualitative methods were used at the same time during the research,
and both have equal emphasis in the study'
methods were used in
QUAI ) quan: This nomdon indicates that the
r.q.r..,.e, with (in this example) the qualitative methods being used
"
before the quantitative metho'ds and the qualitative methods empha-
sized in the studY.
qualitative methods are
QUAN(qual): This notation indicates that the
embedded within a quantitative design'

Building from this notation system, visual diagrams have been used to
convey the complexiry of mixed methods designs. Such diagrams were intro-
duced by Sreckler er al. (1992) and have been adopted by many authors
(for
use
example, Creswell, 2002; Thshakkori & Teddlie, 2003b)' These diagrams
geomerric shapes (boxes and ovals) to illustrate the steps in the research
process (i.e., data collection , dataanalysis) and arrows made with solid lines
and
i+) ,o show the progression through these steps' Ivankova, Creswell,
stick (2005) studied the use of visual diagrams and suggested 10 guidelines
for drawing visual diagrams for mixed methods designs so that they can be
and
easily and conveniently drawn. These guidelines are listed in Figure 3.1
were applied in the visual diagrams that appear in this chapter'

FOUR EXAMPLES OF MIXED METHODS STUDIES

o

To facilitate our discussion of mixed methods, we have included

four com-
plete studies in this book (see Appendixes A, B, C, and D). These studies
42 o DESIGNING AND CONDUCTING MIXED METHODS RESEARCH

1. Give a title to the visual diagram.

2. Choose either a horizontal or a vertical layout for the visual diagram.

3. Draw boxes for the quantitative and qualitative stages of data collection, data analysis, and interpretation ol
the study results.
4. Use upper-case or lower-case letters to designate the relative priority of the quantitative and qualitative data
collection and analysis.
5. Use single-headed arrows to show the llow of procedures in the design.

6. Specify procedures for each stage of quantitative and qualitative data collection and analysis.
7. Specify expected products or outcomes of each procedure in quantitative and qualitative data collection and
analysis.
8. Use concise language for describing the procedures and products.
9. Make your visual diagram simple.
10. Size your visual diagram to a single page.

Figure 3.1 Ten Guidelines for Drawing Visual Diagrams for Mixed Methods Studies
SOURCE: lvankova et al. (2006).

represent examples of mixed methods research as applied across different

disciplines. In addition, each study reports a different approach to mixed
methods research, and together they help illustrate the different possible
mixed methods design options that will be discussed further in Chapter 4.
The four articles included in the appendixes are

Jenkins,J. E. (2001). Rural adolescenr perceprions of alcohol and other

drug resistance . Cbild Study Journal, 31 (4),2tL-224. (See appendix A.)
Rogers, A., Day, J., Randall, F., & Bentall, R. P (2003). Patienrs' under-
standing and participation in a trial designed to improve the manage-
ment of anti-psychotic medication. Social Psycbiatry and Psycbiatric
Epidemiolog,t, 38, 720-727. (See appendix B.)
Aldridge,J. M., Fraset 8.J., & Huang, T.I. (1999). Investigating classroom
environments in Thiwan and Australia with multiple research methods.
Journal of Educational Researcb, 93(1),4&42. (See appendix C.)
Myers, K. K., & Oetzel, J. G. QA$). Exploring the dimensions of
organizational assimilation: Creating and validating a measure. Com-
munication Quarterly, 51(4),438457. (See appendix D.)

To begin studying the application of mixed methods research, read each

of the articles and note the implementation of the research. Artention should
focus on the methods'decisions that occurred during the studies, in addition
to the actual topic and findings. To review these studies, we have developed
a checklist, shown in Figure 3.2, of the major components.
 Iocating and. Reuiewing Mixed Metbods Studies o 43

-Assess tbe study's topic. Consider the general issue being studied
when assessing the topic. It can usually be identified by examining the
study's title and reading the abstract.
Identifu tbe study's purpose by locating tbe purpose statemenf. The
purpose statement is the passage in which the author smtes the spe-
cific intent of the study. It is generally found within the introducdon
section of the article and often at the very end of this secdon. It often
includes phrases such aS "the purpose of this study is" or "the primary
aim of this study was."
, Identif! tbe ways by wbich tbe quantitatiue and qualitatiue data
were collected. Datacollection is described in the methods section of
an article, and the quandtative and qualitative data collection proce-
dures are often discussed in separate paragraphs.
, Identzf! tbe tttays by wbicb tbe quantitatiue and qualitatiue data
u)ere analyzed. Data analysis procedures are also discussed in the
methods secrion of an article and, like data collection, are often dis-
cussed separately for each data rype (quantitadve and qualitative).
r Assess tbe autbor's reAson for using mixed rnetbods researcb. The
reason for using a mixed methods approach may be found in one of
several places. It may be discussed in close proximity to the study's
purpose srarement in the introduction or in the description of the
methods. Some aurhors may include it in the final section as part of
the discussion of the study's findings.
t Deter-rnine bow the two 4,pes of data were mixed- There are three over-
all procedures for mixing qualitadve and quandative data (see Figure
1.2): merging the rwo data sets into one interpretation, embedding one
data rype within the otheq or connecting from the results of one rype of
data to rhe other. (The concept of mixing will be further developed in
Chapter 4.) Ideally the author will discuss in the methods section how
the data were mixed, but in many studies, this process must be inferred
by how the quantitative and qualitative results relate to each other.
t ldenttf! tbe ouerall nzixed metbods approacb using tbe mixed.
metbods notation systenx. Examine how the different methods were
implemented within rhe study. Consider whether the methods were
implemented at the same time or in a sequence and if one was
emphasized more than the other. Use the notation system to describe
the overall mixed methods approach.
o Draw a one-page picture of theflow of actiuities tbat occurred in tbe
study. Consider the main activities of data collection, data analysis,
and interpretarion of results for both the quantitative and qualitative
methods. Sketch out how these activities occurred in the study. Refer
to Figure 3.1 for guidelines about drawing this picture.
44 o DESIGNING AND CONDUCTING MDGD METHODS RESEARCH

Assess the study's topic.

Look for the purpose statement.
Determine whether author collected both quantitative and qualitative data.
Determine whether the author analyzed both quantitative and qualitative data.
Locate the reasons for collecting both quantitative and qualitative data.
Determine how the author mixed the two types of data. Use (a) merged, (b) embedded, or (c) connected.
ldentify the overall mixed methods approach, using the mixed methods notation system.
Draw a picture of the flow of activities that occurred during the study.

Figure 3.2 Checklist for Reviewing the Features of a Mixed Methods Study

After reading each of the four articles and identifying these eight fea-
tures, read the commentary provided in the following secdons. This com-
mentary analyzes and reviews the important mixed methods features
reported in each of the sample studies. In addition to this commentary
we have developed and included visual diagrams of the procedures reported
in each of the articles.

study A: collecting Both Quanritarive and Qualitative Data

at the Same Time to Undersrand a Problem ()enkins, 2001)

Jenkins' (2001) study fits within the broad discipline of adolescent devel-
opment. In particular, her study is on the topic of how adolescents resist
offers to use drugs. She documented previous research indicating rhat ado-
lescents in the United States have high levels of illegal substance use and that
a better understanding of adolescents' perceptions of drug resistance diffi-
culties is needed to inform prevention efforts.
The purpose of this study was stated in the last line of the introduction
section. Jenkins wrote, "the purpose of this study was to examine rural ado-
lescen$' perceptions of factors interfering with the ability to refuse offers of
beer, mariiuana, and hard drugs" (p. 272).
This study was carried out in one phase, with both quandtarive data
and qualitative data collected during the same time period. The researcher
collected the quantitative data from 361 participants, using a structured
questionnaire about the level of drug use for three drug types. The qualita-
tive data included three semistructured questionnaire items, completed by
the same 361 adolescents who completed the quantitative items. In addition,
four focus group interviews were completed with 29 adolescenr parricipants
to both validate and elaborate on the qualitative written responses.
 Locating and Reuiewing Mixed Metbods studies o45

The quantitative data were analyzed and used to classify each partici-
pant as being a nonuser, low-frequency user, or high-frequency user for each
of the drug types.Jenkins initially analyzed the qualitative data using content
analysis procedures, resulting in 15 categories that emerged from the ado-
lescents' responses. These categories were described and the qualitative
dam were also transformed into quantitative counts by determining the per-
centage of parricipants whose responses included each of the categories.
Jenkins o<pected that adolescents' perceptions would differbased on drug
type and level of drug use. Therefore, she needed both qualitative data about
adolescents' perceptions and quantitative data about adolescents' drug use.
That is, her reason for using a mixed methods approach was that both quanti-
tative data and qualitative data were needed to best understand this problem.
Jenkins merged her quantitative and qualitative results in two interesting
ways. First, she included three tables that brought together the quantitadve
variables (drug rype and level of use) with the qualitative findings (response
categories). These cross-tab tables interrelated the findings, with the numer-
ical values presented in the cells of the tables representing the frequency of
the qualitative responses. In addition,Jenkins combined the quantitative vari-
ables with the qualitative findings when she discussed the results and in the
interpretations she made in the discussion section.
The notation for this study can be written as QUAN + QUAL. Each of
the methods was used during the same timeframe, and they appeared to be
equally imporrant for addressing the overall purpose of the study. This
approach is an example of what is called a Triangulation'Design in Chapter 4.
'We
summafizedthe procedures of this studyin the visual diagram depicted
in Figure 3.3. The quantitadve data collection and analysis appear on the left
side of the figure and the qualiudve data collection and analysis appear on the
right side. As shown in this diagram, both the quantitative and qualitative data
were collected and analyzed during the same phase of the research process
and had an equal emphasis within the study. These rwo data rypes were then
merged into one overall interpretation, as depicted in the final box.

Study B: Using QualitatveData in an

Experiment (Rogers, D^y, Randall, & Bentall, 2003)

Rogers et al. conduct research in the area of mental health services. The
topic of their 2003 study was the management of antipsychotic medication
by people diagnosed as suffering from schizophrenia. They stated that random-
Ded controlled trials (i.e., true experiments) are the "gold standard" of research
(p. 720). Therefore, they designed and carried out an experimental trial to
test the effectiveness of rwo different trea[ments designed to improve patient
 U
E
(6
a
o
-
.;o)a rd
(J
I0)
(J

s e e€ € Ff; o
o
=og
E;
O...
$ifl E
(L..
e8 U
OJ
L
o)

6
(d

rO c o
q)
j68 >o 't
\,,
tal L) :
o -o>
o --,^ocSi
..:96i.eEoqE .:
rd

(!
9
E ;'R 243
oti:6)-;.=o oJ

57i=O-O,()=(5 -o
F 3E€3TE
E.g + .,,
E
O-..
c(g
o
'{=

$ ; e 6 3 $o € E a= c
.d
.=
c
g,F3ef;€egSHEs _E€
E (6

O..O-.. E(Ug
6gE of
c 6-a * -o
o o 0) .l: co
69 .F
(U
Lc
u.9R to-S2 .9
6 06 -.c
=Ec = x-6
E Y(6 €
E:E
O..
g oE
O.
o>-o)
o=(D;
o'=
(l) (u (l)=
.5
! s)
i EEE€gg
Y{ o X (5'c
(gE
E
J

at,
E 6 E e s e€
O. l
(o

-a .=
?sg
f, gE
rtt

o"E
OJ
L

q.)
u.-
o
I ig
OCJ
iic9
o (I) =^ iio o
o
.Eq

frEfif
\€
ra -:l
;Ha tr(u c

fi ;i I (o
g E $ ;oo
O-r(L. tu?
;\ 0J
t-l 6
fg
G-o
)-F U
\/)C:
t-
.?l-oo
-6
.1'
m-
.o
c.)
o
/
\-
L LLJ

oor)
t!z

46
 Locating and' Reuiewing Mixed Metbods Studies o 4/

to this quantita-
adherence to antipsychotic medication regimens. In addition
qualitative data within this
tive purpose, they stated two purposes for including
actuallyo<perienced
orperiment. These purposes were to o<plore what padents
during the intervention (primary qualitadve purpose) and
to ortend their
better design the
understanding of the treatment mechanisms to help them
qualitadve purpose)'
reatment condidons used in the orperiment (secondary
This arperimental trial incorporated many of the features of a rigorous
control groups' ran-
intervention design, including the use of treatment and
of quantitative
dom assignment to reatment conditions, and the collection
Rogers et d. collected quantiEdve
data before and after the intervendon.
the intervention' at the
data using the Drug Attitude Inventory @AI) befiore
These researchers
completion of the intervention, and at a !-year follow-up'
Before the trial began, they inter-
also collected two sets of qualitarive data.
the ffeatment procedures
viewed 25 patients to inform the development of
patients to participate
(secondaryqualitative purpose). Theyalso selected 1'6
after the
in semistructured one-on-one interviews about their experiences
patients were
rrial was complete (primary qualitative purpose)' These
and they scored
selected because they experienced the two ffeatments
either positively or negatively on the outcome measure'
analyzed
Although not e<plicitly discussed in this article' the researchers
in the change scores among
the quantitative data for significant diffierences
was provided about the
the treatment and conUol groups. Little information
the trial to address the sec-
analysis of the qualitative data collected before
of the primary qualitative data set'
ondary qualirative purpose. In terms
the interview transcripts' The
the authors conducted a thematic analysis of
as themes and sub-
results of the qualitative data analysis were discussed
themes, and g(ensive illustrative quotes wefe
provided' The researchers
described and interpreted the themes within the context of the experimen-
participants (for example, dis-
tal trial and in rerms of the outcome of the
positively to the reatment)'
cussing the responses of those who responded
study, the
Alrhough,h. qu".rritative dagwere sufficient forthe experimental
by the quantitative out-
authors had addirional questions thatwere not answered
the participants who experi-
comes. They wanted to study the perspecdves of
enced rhe different reatments to understand
the meaning and processes behind
the outcome measures. In addition, they wanted
to use qualitative information
the experiment began'
to help develop the intervention treatments before
Rogers et al. embedded qualitative components within their larger, quan-
mixes at a different level than
titative, experimental methodology. This stucly
the previous study. The previous example mixed the quantitative and quali-
tative daa and findings at the dam level. In
this study, the qualitative data
were embedded within a latger quantitative design. That is, the mixing
48 o DESIGNING AND CONDUCTING MIXED METHODS RESEARCH

occurred at the design level instead of at the data level. Mixing also occurred
in different ways throughout the procedures-from developing the inter-
vention, to selecting the cases for the qualitative follow-up, to providing a
broader perspective on the quantitative outcomes.
The embedded nature of this approach can be indicated with the nota-
tion QUAN(qual). The qualitative data played a subservient role within the
larger quantitative methodology. These qualitative data were included so that
the researchers would be able to understand the experiences and perspec-
tives of the participants who experienced the treatment conditions. Ve will
discuss Embedded Designs, such as this approach, later in Chapter 4.
We depicted the embedded nature of this design in the visual diagram in
Figure 3.4.The supplemental qualitadve aspect of the study is shown within
the context of the quantimdve ecperimental methodology and occurs before
and after the intervention.

Study C: Explaining Quantitative Results

With Qualitative Data (Aldridge, Fraser, & Huang, t999)

Aldridge et al. (1999) are educational researchers who reported their

study about cross-national differences in classroom learning environments.
They noted that such comparative studies are limited but that they are
important for identiffing and understanding the important variables and
assumptions within national educational settings.
The purpose of this study is discussed in the introduction to the article.
It was to identify differences between classrooms in Thiwan and Australia and
to explore sociocultural influences that explain these differences.
Unlike Jenkins' single-phase study, the authors implemented this study
in two distinct phases. The first phase was quantitative. Aldridge et al. admin-
istered the \ilhat Is Happening in This Class? MHIC) instrument, with nine
subscales, to a total of 2950 students divided between 25 Australian and 25
Thiwanese schools. Next, the authors collected extensive qualitative data in
the second phase of this study. Their qualitative data included observations,
student interviews, and teacher interviews
The quantitative data in this study were analyzed primarily in two ways.
The properties of the instrument s/ere evaluated using factor analysis to val-
idate the subscales and reliability analysis to assess the internal consistency
of the subscale items. Group comparisons were tested to identifli significant
differences between the responses of the participants from Australia and
those from Taiwan. The analysis of the qualitative data resulted in narrative
stories describing the rypical classroom environment for each country as well
 E
oo
E9q,
H
g
o
Ee-E A; E 8EI H C

=:r=o
5E-fr fiE$eic:sg
.E g o6o.c.s to E I
O...O..

'6 cc)
E E
o(!
e5 F ,E L
q)

ss*e sEi [,
o-
g x
llj
c(d
g € HEf i;f s =
5
c

G
(g
CL
o
:t o
*- .z
zOO)
a:
<o: \
8. p
8.9 Qr
E
-F
(o

=(6
t'
.9
a; H gE 96 c)
g J

o o bo
E
o
E
TL
aE EE s
9'i o.€ o E :)
c
r^

o
=(!
co
..1
ol
cl
g.E
e fr
EIe
F; E *4E * 3e o
E
-o
J
tt,
6)I
E zl ASEHfigEF ?, rc
.=
o
CL
x
0)l
-l
CI aPsEaie H E8 8
r/r
o
-ou
=i.
ru
oo
Uc!
2E os
<d;
I ^- fiio
o*
tcE5Et3Fg€sgE
o...r4... g6
€g
LE

h
b8"o
EM,
'6 G'

E bbb
(!
,g q i5g dt
E-o
.eh
=C
-bo .E
tEEEeFFEE' tf.e
E
:a
cn
o .::
bou
lJ- Z-

49
50 o DESIGNING AND CONDUCTING MXED METHODS RESEARCH

as the identification of themes that transcended the rwo cases and high-
lighted the differences between the nvo settings.
The authors chose to collect both types of data because they needed
to follow up with qualitative information to interprer and explain the initial
quantitative results. This reason was summanzed with the following state-
ment: "The stories and their subsequent commentaries provide a culturally
sensitive basis [from] which the researcher was able to explain differences
and similarities between the learning environmenrc in each counrry" 1p. 53).
Aldridge et al. mixed the two data rypes by connecting their quantitative
phase to a second qualitative phase: They identified new questions based on
the initial quantitative results and selected cases for qualitative study to answer
these questions. The results of the quantitative analysis led rhem ro identi$r
significant differences between the two countries and "interesdng anomalies"
(p. 50) in the quandtative data. To explain these results, they selected four
classes from each of the two countries for a qualitative follow-up.
This study can be described using the following notation: QUAN ) qual.
This notation depicts the fact that the two rnethods were implemented in a
definite sequence, with the quantitative occurring first. In Chapter 4, we will
discuss this mixed methods approach as an Explanatory Design. Due to rhe
extensive nature of the quantitative data collection and analysis and the
authors' discussion of their wanting to explain the quantitative results, this
study appears to give grearer emphasis to the quantitative phase.
As shown in our visual diagram in Figure 3.5, this study's methods were
implemented in two distinct ph.ases. The srudy started with the quantirative
approach (the three left boxes of the diagram) and then followed wirh the
qualitative approach (the three right boxes). The center box of Figure 3.5
shows how the researchers connected the quantitative results ro the qualita-
tive data collection. These qualitative data were used to explain the results
obtained in the inirial quanritarive phase.

Study D: Exploring Qualitatively ro Develop

a Quantitative Instrumenr (Myers & Oetzel, 2003)

Myers and Oetzel (2003) are researchers in the discipline of communi-

cations. The topic of their study was the assimilation of new employees
within organizational settings. They stated that organizational assimilation is
an important problem to study because it leads to improved productivity and
employee persistence.
Myers and Oeuel (2003) argued that current measures of organizational
assimilation were inaclequate. Therefore, the overall purpose of their study was
to create and validate an instrument for measuring organizational assimilation.
 o
a
!
9 s.E ..
3N
.E e,
CT' C
-69oo
€.H.E E d'P 59Eo.=
II -gEgEE
6€ n
=
E TE
O..'O = O-.

Ein)
ttt
0)
e. ;
q)
s)
E

teuEea
C; . iE;gE
o.. '
(g

crd
J
@
.g

q) 6
(!

c
'6 o
-2
o- F
x
lJ-l
o
.;
.g
tsg*Eg'EgEBEsE
O.... o
G'
(€
cc,
(!
o I
(g ra
(!
]=
F .C

g:f
bD so
t=*'ga€riFEBts$€
[3oO..'
.g
tn
:) go
.9
-lJ
(6

E
) .9
tt>
(6 II
c
5
lo,
lL
Itt
.:
O)
IJ Or

tgEEegEEi'seEE O...
IE
ld)
l(J
lo
I L
O)

-i(!
lo-
lo,
l-c
a)

t:
o)
Ebo
l;aE.c.9
g --5 E l; qf .E
lo
lE
l(d
IL -
gfeEeEe ;ggesse;
d...L"'
loo o
l(tt
lo
l-
rto
6
l(6 -o
l= E
l(^
l5 (0
bo
.(!
g -o
]E

nt iEE
rn a
6
ca

t*=,i
o..o. lL
q)
L
BO roFz
di

5I
52 o DESIGNING AND CONDUCTING MIXED METHODS RESEARCH

To accomplish the study's purpose, Myers and Oetzel reported that their
study "proceeded in rwo srages" (p. 439). They began their srudy with a qual-
itative er<ploration of the dimensions of organizational assimilation. During
this phase, theyconducted one-on-one semistructured interviews with L3 par-
ticipants. These interviews generated two types of qualitative data: interviewer
field notes and transcripts of the interviews. After creating the organ izational
assimilation instrument from the qualitative findings, the study moved into
its second, quandtative phase. The authors administered their Organizational
Assimilation Indsr (OAI) instrument along with additional measures hypoth-
esized as being related to the dimensions of organuational assimilation. This
survey was administered to 342 employees across diverse industries.
Myers and Oeuel analyzed their qualitative data during the firsr phase.
They used qualitative analytic techniques, including coding, themaric devel-
opment' and comparison of the themes to existing theory The result of this
qualitative phase was the identification of six dimensions of organizational
assimilation from the perspecdves and experiences of the participants.
The quantitative data were analyzed during the second phase. The survey
responses were analyzed in three different ways: analysis of the scale relia-
bility, confirmatory factor analysis to validare subscales, and correlational
hypothesis testing to establish construct ralidiry.
The authors explained that the different dimensions of organizational
assimilation were unknown and that they needed to first explore this phe-
nomenon with qualitative data before they could measure it quantitatively.
Thereficre, they needed both types of data ro creare and subsequently vali-
date an instrumenl The researchers collected both qualitative and quantita-
tive data to, respectively, explore a phenomenon and measure it.
The authors connected their initial qualimtive phase to the quantirative
phase by developing an insrument to measure organ izational assimilation.
Building from their qualitative findings, rhe authors develope d 6l survey
items to represent the six dimensions of organizational assimilation. This
instrument was then implemented in the second phase.
The notation for this study can be writren as qual +
eUAN.'We will dis-
cuss this two-phase approach, which begins with qualimtive methods, as an
Exploratory Design in chapter 4. This study seemed to emphasize the quan-
titative findings and instrument validation, thus demonstrating the overall
importance of the quantitative data in this study.
The authors used two distinct phases to implement this study's methods.
As depicted in Figure 3.6, it began with qualitacive data collection
and analysis
to explore a phenomenon (the first three boxes of the visual diagram). From
this initial phase, an instrument was developed (note the "develop instru-
ment" box in Figure 3.6). This instrument was then used to collect quantita-
tive data in a second phase (the last chree boxes in rhe diagram).
 (d
c
o
ooc rt
.N
-ocud (!
bs9 o
oo

6EE
t5EEEe€F€EaE
d..O"
(d q
o ox'
(d

0) 0)
E
(6
;o
@ c
?
j s'9,
gE
C' s
q
-egggr € ssF€ElE
J

o"F tr
'a
td...G.." -r- att

GL

=
i:
t) o
2r,_ ft
oA-:Z o\f
J
v,
q)
o
.!
-?bL'=ma d, (g
C) oo
g SEqcg* € g g o
3cfi
o "E z (g

g 5E6EEE E 5g G o
tr.4.. g:t c)
(!
(J
lt)
o
aE (o o
o l)
(n
gE E.
x .,
oL
>J
ll.l
v\
(6

o)E 0)
a2 €cEggec€*gc3 )(! I
L

vt
!
n
d..O'. C)
q)
(!
.9
o E
o6 a, ;(rr (u
E'AL
o .. .9 {.e iJ1
-'(o
at,
9ro f ^()ci (t)
(t'c
-ol
eE
5'so
=-o(' €EE6 .h
o)
L
Por
UB
g 8E
O-.
E
O..
$;5 :l
(u
u
6J
o fre
L
L
o
-o .1.
o !3

c a' =c)
'ti gi
otsI ..XEE LV
o
(t o
a s'9, ! o,E * g EE E E
(d Eu;
e€ -s(g €Eg rr x x.E
L
bo 6o-
E
H ()FO I6it9 .g oo-=iu'
O.. O... o ns
:E
:J
-oE
tro
r/|
g;
hoo
(df
co €g
s EEA,
EF
=
iE
rd'E
g I iE \o
(n
>3
o
o SIEEg
g.oG..
€EF o
bo
di=
--
O6
u- ZP

53
54 o DESIGNING AND CONDUCTING MDGD METHODS RESEARCH

similarities and Differences Among the sample studies

The main features of the four mixed methods studies discussed in this
chapter are summarized in Table 3.1. The similarities and differences among
the information within this table highlighr many of the important features of
mixed methods research and the different approaches for applying mixed
methods research.
First, it is interesting to note that these four studies represent four
diverse disciplines and examine four different research topics. Their diversiry
is also reflected in the fact that they were conducted for different purposes.
Jenkins (2001) o<amined adolescent drug resistance, using both qualirative
and quantitative information. Rogers et al. (2003) wanted ro orplore patient
o<periences as part of their experiment. Aldridge er al. (1999) identified
and explained cross-national diffierences in classroom environments. Finally,
Myers and Oeuel (2003) created and validated an instrumenr to measure
organaational assimilation.
Each of these studies collected at least one form of quantitative data
and at least one form of qualitative data. The quancitative data were collected
using some rype of structured questionnaire or measurement instrument in
all four studies. The forms of qualitative data collected across these studies
included written responses, focus group interviews, one-on-one interviews,
observations, and researcher field notes.
Each study also included procedures for analyzing the quantitative and
qualitative data. The quantitative procedures included descriptive analyses,
group comparisons, reliabiliry checks, confirmatory factor analyses, and cor-
relational analyses. Qualimtive analytic procedures included contenr and the-
madc analyses, narradve story developmenr, and quantifying the frequency
of qualitative responses.
The authors of each study offered their reasons for collecting both quan-
titative and qualitative forms of data. Jenkins (2001) needed borh types of
data to best understand the problem. Rogers et al. (2003) needed qualitative
data to understand participant perspectives as part of their experimenml rial.
Aldridge et al. (1999) needed to collect qualitative data to explain their inirial
quantitative results. Myers and Oetzel (2003) wanted ro creare and validate a
new instrument and therefore needed to begin with qualitative data to
explore their topic before they tried ro measure it.
These studies all mixed their quantitative and qualitative data, but they
did so in different ways. Jenkins (2001) merged the two dam sets by interre-
lating the two sets of findings and combining them in a table. Rogers et al.
Q}A, mixed by embedding qualitative data within their experimenml rrial ro
develop the intervention and to select cases for qualitative followup about
 Locating and Reuiewing Mixed Metbods Studies o 55

Table 3.1 Features of the Four Sample Mixed Methods Studies

Rogers, Day,
Randall, and Aldridge, Fraser, and Myers and Oetzel
Huang (1999) (2003)
Feature Jenkins (2001) Bentall Q003)

Adolescent resistance Management of Classroom learning The assimilation of

Topic
to drug offerings antipsychotic environments new employees
medication into organizational
seftings

To examine To explore patient To identify and To create and

Primary study
perceptions of drug experiences during explain cross- validate an
purpose
resistance d ifficulties the experimental national classroom instrument for
by drug type and treatments differences measuring
level of drug use organizational
assimilation

o Structured a Outcome o Measure r Questionnaire

Quantitative including
data collection questionnaire measures including
(preintervention, multiple multiple
postintervention, subscales subscales
1-year follow-up)

Qualitative a Semistructured . One-on-one a Classroom a One-on-one

questionnaire items semistructured observations semistructured
data collection
a Focus group interviews a Student interviews interviews
interviews a Teacher interviews a Field notes

Quantitative . Classify by group o Outcome scores a Scale reliability o Scale reliability

data analysis o Croup a Confirmatory a Confirmatory

comparisons factor analysis factor analysis
a Croup a Correlational
comparisons tests

Qualitative a Content analysis . Thematic a Narrative story a Coding

a Percentages for analysis development a Thematic
data analysis
each category a Thematic analysis development
a Comparison of
themes to theory

Need both Need qualitative Need qualitative Need quantitative

Reason for
quantitative and information as part data to explain data to measure
collecting both
qualitative data to of an experimental quantitative findings qualitative findings
types of data
understand the trial
problem
The two types of The qualitative data The qualitative data The quantitative
How the two tyPes
data were merged were embedded were connected to data were
of data were mixed
within an the quantitative connected to the
experiment results qualitative results

QUAN -+ qual qual -+ QUAN

Notation QUAN + QUAL QUAN(qual)
Figure 3.3 Figure 3.4 Figure 3.5 Figure 3.6
Visual diagram
56 o DESIGNING AND CONDUCTING MDGD METHODS RESEARCH

parricipant experiences during the intervention. Aldridge et al. (1999) and

Myers and Oeael (2003) both mixed by connecting two sequential phases in
which the second phase built offthe first phase. Aldridge et al. identified key
results from their quantitative data and used them to direct their qualitative
phase. Myers and Oetzel developed an instrument based on their qualitative
findings, which they then validated quantitatively in their second phase.
'Wb can identiff methodological differences in these studies by e><amin-

ing the different patterns that emerge from the stated notations and the
visual diagrams pictured in Figures 3.3 through 3.6. Notice how the designs
differ in their timing. Figure 3.3 depicts both methods being implemented
concurrently in one phase, whereas Figures 3.5 and 3.6 depict the methods
being implemented in a definite sequence. Figure 3.4has both a sequential
and a concurrenr aspect embedded within the design. The designs also dif-
fer in rerms of the relative emphasis given to the different forms of data, as
shown by rhe QUAN and QUAL letters appearing in uppercase or lowercase
letters. Aldridge et al. (1999), Myers and Oetzel (2003) and Rogers et al.
(2003) had more of a quantitative emphasis, andJenkins (2001) emphasized
the two forms of data equally.
These four studies illustrate many important features of mixed methods
research, including the collection of cwo rypes of data, the analysis of both
data sets, reasons for collecting the two rypes of data, how the mo rypes of
dataare mixed, and the timing and relative emphasis between the two forms
of data.It is important to pay attendon to these features when o<amining
published arricles, and it can be very helpful to use the shorthand notation
and to draw a visual diagram to present and organize the procedures.

Summary

Mixed methods studies can be located in the literature and identified based
on the definition of mixed methods research. Reading published mixed
methods studies requires that key features be examined. These features
include assessing the topic, locating the purpose statement, identifying the
rypes of qualitative and quantitative data collection and the types of analysis
procedures, noting the reason given for collecting both types of data, and
recognizing how the two data types were mixed. The complexity of mixed
methods designs can be communicated through a notation system and visual
diagrams that are drawn using standard conventions and that identi$r the
specific procedures and products for each stage of the research process.
These diagrams also convey the timing and relative emphasis of the two
 'Locating and Reuiewing Mixed Metbods Studies o 57

methods. Researchers wanting to read and design mixed methods studies

will benefit from having the skills to locare key features within study reports
and the abiliry to draw visual diagrams that communicate a study's methods.

Activities

1. Locate a mixed methods study published in the literature. Using the

definition of mixed methods research, explain why this study is a
mixed methods study.

2. Carefully read the mixed methods study you located. Using the check-
list in Figure 1.2, review this study by considering the items included in
this list.

3. Using the mixed methods notation system, the rules for drawing
visual diagrams (see Figure 3.1), and the sample figures in this chapter,
draw a diagram representing the mixed methods features of the arti-
cle you located.

Additional Resources to Examine

For additional discussions on drawing visual diagrams for mixed methods

studies, see
Irrankova, N. V, Creswell, J.\(, & Stick, S. (2006). Using mixed-methods sequential
explanatory design: From theory to practice. Field Metbods, 18(7),3-2O.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangula-
tion. Nursing Researcb, 40, 12V123.
Tashakkori, A, & Teddlie, C. (2003b). The past and future of mixed methods research:
From data triangulation to mixed model designs. In A Tashakkori & C. Teddlie
(Eds.), Handbook of mixed metbods in social and bebauioral researcb
@p.671-701). Thousand Oaks, CA: Sage.