See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/274694952

Mixing quantitative and qualitative research

Chapter · November 2015

CITATIONS READS
20 10,121

1 author:

Sarah Kaplan
University of Toronto
62 PUBLICATIONS   5,920 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Sarah Kaplan on 08 April 2015.

The user has requested enhancement of the downloaded file.

 Mixing quantitative and qualitative research

Sarah Kaplan
Rotman School, University of Toronto
105 St. George St., Room 7068,
Toronto, ON, M5S 3E6
sarah.kaplan@rotman.utoronto.ca

Forthcoming
Handbook of Innovative Qualitative Research Methods:
Pathways to Cool Ideas and Interesting Papers
K. Elsbach and R. Kramer, eds.
October 1, 2014

The author would like to acknowledge useful comments from Valentina Assenova, Jillian Chown, Kim
Elsbach, Rod Kramer, Michael Lounsbury, Chad McPherson, Matthew Regele, Michael Sauder, Amy
Wrzesniewski and the slump_management research group. All errors and omissions remain my own.
 Abstract: This chapter offers several models of ways that scholars can usefully integrate
qualitative and quantitative research in both single works and broader research programs: using
qualitative methods to identify constructs or hypotheses that can subsequently be examined in quantitative
studies; using quantitative studies to identify patterns that can be explored in qualitative work; and,
quantification of qualitative data. Through this analysis, I call into question existing characterizations of
the relationships among the two types of research and offer a more nuanced portrayal of the links between
them. I suggest that the advantage of mixed methods studies is that they recognize the value of iterating
between that which can be counted and that which cannot in order to generate richer insights about the
phenomena of interest.

In the field of management, there is often a gulf between qualitative and quantitative research.

Despite a wealth of literature on mixing qualitative and quantitative methods of research (Bryman, 2006;

Creswell & Clark, 2007; Jick, 1979; Small, 2011), rarely do individual scholars conduct both forms of

research and even more rarely do they present them together in a single scholarly work. Quantitative

researchers may view qualitative research as suspect, or only as a means to generate hypotheses to test in

quantitative analyses. According to this view, qualitative research is but an intermediate output supporting

“empirical” regression analysis. On the other hand, qualitative researchers often view quantitative work as

too reductive. They may only include quantitative data for rhetorical purposes, recognizing that they are

outnumbered (no pun intended) in a world dominated by scholars untrained in qualitative methods.

There are many understandings of the words “qualitative” and “quantitative” within various

research traditions. For the purposes of this chapter, I take the broadest of definitions of each, where

qualitative research is any analysis based on words (e.g., texts, interviews, or field notes from

observations), and quantitative research as any analysis using numbers, either descriptively or in

regression analysis.

In this chapter, I offer several models of ways that scholars can usefully integrate qualitative and

quantitative research with the hopes of expanding the perspectives of scholars in both camps. In doing so,

I call existing characterizations of the relationships among the two types of research into question and

offer a more nuanced portrayal of the relationship between them. That is, qualitative methods can be used

to understand mechanisms that underlie relationships identified in quantitative research as much as they

can be sources of hypotheses or constructs for “large n” analyses (see Figure). Further, quantifications of

Mixing quantitative and qualitative research -1-

qualitative data or supplemental quantitative analyses may sometimes serve a useful purpose in validating

findings. Small (2011) and Creswell and Clark (2007) among others provide very detailed analyses of

mixed methods research and the various controversies within it. I have a much narrower goal in this

chapter: to identify several models for integrating qualitative and quantitative data and analyses into a

research paper (or in a research program) (see Table), highlighting the potential benefits and pitfalls of the

various approaches. Through this analysis, I demonstrate that the line in the sand between qualitative and

quantitative research may get in the way of interesting and compelling research projects and programs.

Figure: Relationship between qualitative and quantitative research

Use*qualita've*methods*to*understand*
mechanisms*underlying*rela'onships*
iden'fied*in*quan'ta've*regression*analysis*

Qualita've*research* Quan'ta've*research*

Use*qualita've*findings*to*generate*
hypotheses*to*be*tested*with*a*“large*n”*
sample*in*quan'ta've*regression*analysis*
*
Quan'fy*qualita've*data*to*validate*
observed*pa>erns*

Small (2011) makes important distinctions between complementary and confirmatory purposes

for integrating qualitative and quantitative methods, and also between sequential and concurrent research

designs. When planning research, it is certainly important to think about these choices. However, my own

experience and that of many of the scholars’ whose work I describe below (as told to me in interviews), is

that these purposes evolve over time as the research unfolds. Research intended to be confirmatory ends

up offering new, complementary insights. Research that was intended only to be quantitative becomes

multimethod as reviewers ask for evidence to elucidate the proposed mechanisms. The published articles

or books may not reflect these vagaries of the research process, but researchers engaging in mixed

Mixing quantitative and qualitative research -2-

methods studies may want to be prepared for or attentive to these potential shifts. I will return to these

insights in the conclusion of this chapter.

Examples of mixed methods papers and research programs

Mixing  qualitative  and  quantitative  methods  in  a   Mixing  qualitative  and  quantitative  methods  in  a  
single  research  paper   research  program  
   
Qualitative  before  quantitative:  Qualitative  data  to   Lounsbury  (1997,  1998,  2001):  qualitative  fieldwork  
justify  the  development  and  testing  of  a  construct…   used  to  develop  a  rich  understanding  of  the  context  
• in  archival  regression  analysis  (Tripsas  1997;  Briscoe   which  is  presented  in  a  series  of  papers  taking  
2007;  Canales  2014;  Doering  2014)   different  theoretical  angles.  The  qualitative  evidence  
• in  a  survey  (Ely  1994;  Edmondson  1999;  Lounsbury   is  combined  with  a  subsequent  survey  for  quantitative  
2001;  Bresman  2010)   analysis  in  on  paper.  
• in  an  experiment  (Elsbach  1994;  Ranganathan  2014)    
  Kaplan  (2008  a,  b):  concurrent  qualitative  and  
Quantitative  before  qualitative:  Survey  or  archival   quantitative  studies,  iterating  between  a  quantitative  
data  based  regressions  to  establish  phenomenon  and   analysis  of  relationships  and  a  qualitative  analysis  of  
fieldwork  to  understand  the  mechanisms  (Fernandez-­‐ mechanisms.  Published  as  two  separate  papers,  but  
Mateo  2009,  Bidwell  2009,  Edmondson  1999)   the  emerging  insights  in  each  shaped  the  research  
  strategies  and  analysis  of  the  other.  
Quantifying  qualitative  data  to  enrich  analysis,    
validate  qualitative  insights  (Barley  1986,  Kellogg   Mische  (2008):  rich  ethnographic  detail  combined  
2009)  or  to  generate  constructs  to  be  tested  in   with  social  network  analysis  (based  on  network  data  
regressions  (Wageman  2001)   collected  as  part  of  the  fieldwork).  Presented  in  a  
book  format  in  order  to  accommodate  the  richness  of  
the  two  forms  of  analysis.  

Qualitative Data in Support of Quantitative Analyses

Amongst the most traditional views of the relationship between qualitative and quantitative research is the

use of the former to generate hypotheses and constructs to be measured and tested in the latter. That is,

there is an outcome that is puzzling (such as heterogeneity in incumbent organizations’ responses to

technological change or differences in team learning), existing explanations are not satisfactory, and

scholars use qualitative evidence to identify potential alternative explanations that are more complete or

resolve tensions that exist. Researchers then translate these insights into hypotheses that they then test in a

subsequent quantitative, regression analysis. The quantitative component can take multiple forms, from

archival data, to surveys to experiments.

Mixing quantitative and qualitative research -3-

Using Qualitative Data for the Development of a Construct and/or Model, which is then Tested in
Archival Research.

Tripsas’ (1997) study of creative destruction in the typesetter industry starts with a puzzle: if a new

generation of technology is competence-destroying for incumbent firms, why do some survive and even

thrive? She uses a detailed historical analysis of the companies in the typesetter industry – using company

and trade association archives, interviews, personal records of retired employees, industry trade journals,

and other sources – to uncover the reason. She found that the differences are due to the degree to which a

firm’s specialized complementary assets are aligned with the new technological regime. She

demonstrated this in a series of tables identifying the technological competencies and complementary

assets required for each generation of technology. In the second half of the paper, she used these

qualitative insights to generate measures of competencies for each of the firms in a complete dataset of all

of the firms and products in the industry over time (note that she developed this quantitative dataset using

many of the same resources that provided the qualitative insights). In the regression analysis, she

demonstrated that incumbents whose complementary assets were not devalued had higher market shares

in new technological regimes than those whose complementary assets lost value. In this study, the

qualitative and quantitative analyses were both complementary and confirmatory. The analysis of the

qualitative data uncovered relationships and helped to generate constructs for a quantitative analysis that

then confirmed the qualitative insights.

Briscoe’s (2007) study of professional service workers follows a similar approach. He used a

field study of primary care physicians to develop a model of how organizations influence the temporal

flexibility of their work. The fieldwork – observations, archival data collection and interviews with 55

informants – allowed him to identify the primary source of inflexibility – inability to hand off patients at

the end of a shift – and locate specific practices associated with bureaucratic formalization that might

create more flexibility. This first study uncovered the mechanisms but did not allow Briscoe to rule out

other potential factors shaping temporal flexibility. Thus, in the second study, he used survey data on

physicians and their organizations in order to evaluate the effect of bureaucratic formalization on three

measures associated with temporal flexibility. He did not design or conduct this survey of over 6,000

Mixing quantitative and qualitative research -4-

physicians himself but rather was able to obtain it from another researcher who had collected these survey

data for other purposes. Matching his qualitative insights with these more comprehensive data enabled

him to confirm that bureaucratization was indeed associated with greater temporal flexibility, controlling

for other organizational characteristics such as size, type of ownership and age.

Doering’s (2014) study of a microfinance institution explored the ways that escalation of

commitment can actually be productive for organizational outcomes. She used ethnographic observations

and interviews to provide insights into loan officers’ behaviors as they interacted with clients, from which

she developed a series of hypotheses. As she noted, such data are “are poorly suited to address the

frequency of these processes or their long-term, broader consequences for the organization” (pp. 12-13).

So, she used a database from the organization on lending and repayment behaviors over time in order to

show that delinquent clients who work with original loan officers (who are presumed to escalate their

commitment if they keep these clients on the books rather than sending them to collections) are more

likely to repay loans on time and take out subsequent loans (evidence of higher levels of performance).

This regression analysis enabled her to connect escalation of commitment to performance as well as to

account for alternative explanations and selection bias not possible in the field study. In this study,

Doering iterated between complementary and confirmatory goals and between qualitative and

quantitative data collection and analysis over time. (See Canales (2014) for a similar style study of

microfinance using ethnography and archival evidence from loan databases to understand the tensions

between standardization and flexibility in loan management).

In these examples, qualitative research enabled the researchers to identify a potential explanation

not previously explored in the literature and measures that could then be tested in a regression. However,

the ways that the scholars made the connections between their qualitative and quantitative data differ. In

Tripsas’ (1997) study, the sources of data for the qualitative and quantitative studies were highly

overlapping. Her comprehensive database of firms and product models was not available off-the-shelf, so

she necessarily had to dig into many forms of archival evidence to construct the quantitative variables for

testing the hypothesis she generated from a qualitative analysis of these same materials. For Briscoe

(2007), he was able to match insights from his own fieldwork with a previously conducted survey that he

Mixing quantitative and qualitative research -5-

could adapt for his purposes. For Doering (2014) and Canales (2014), the ethnographic work was

complemented by a database from the same organization.

Using Qualitative Data for the Development of a Construct and/or Model, which is then Tested in a
Survey Designed Based on those Constructs.

Edmondson’s (1999) paper on team psychological safety started with observations and interviews of 8

teams to verify that the constructs of team psychological safety and team learning behavior could be

operationalized in her research site. She then administered a survey measuring these constructs to

members of 53 teams and a separate survey to recipients of the teams’ work in order to measure team

performance. As a final step, she then turned back to more fieldwork to conduct a comparison of extreme

cases (four high- and three low-learning teams) to the relationships between the two focal constructs. The

goal, as she stated, “was to learn more about how they functioned as teams rather than to confirm or

disconfirm a model” (1999: 370). The most powerful part of this third phase of qualitative research was

her use of outliers to show how the lack of psychological safety hindered team learning even in contexts

that should have otherwise enabled it.

Similarly, Bresman’s (2010) study of external learning by pharmaceutical teams used qualitative

data from 94 interviews of members of 6 teams to develop measures of different forms of external

learning – contextual and vicarious – that he introduced subsequently in a complementary survey of 62

additional teams. The survey data then allowed him to examine the performance implications of these

external learning activities, showing particularly that vicarious learning only leads to higher performance

when coupled with internal learning.

Ely’s (1994) study worked slightly differently in that she interviewed and surveyed the exact

same sample of people (women in law firms) to understand the impact of the presence of women in upper

echelons of management on hierarchical and peer relationships among professional women. She first

conducted 4-5 hours of interviews for each of 30 women professionals from 4 matched pairs of law firms

(matched according to whether the upper management was gender inclusive or not). She then conducted

content analysis of the interviews (with inter-rater reliability tests) to identify the key dimensions of the

theoretical mechanisms of interest. She subsequently used the natural language of the respondents to

Mixing quantitative and qualitative research -6-

design questions in a survey that each informant completed. This “empathic” approach to survey design

increased construct validity. Based on the survey results, she conducted a series of t-tests on the average

scores by type of firm in order to confirm that women in firms with few women leaders did not

experience gender as a positive basis for identification, did not find senior women to be positive role

models, and were more likely to compete with their peers. She complemented these statistical tests with

rich quotes from the interviews that showed the grounding of these identified relationships in the lived

experience of the women.

Lounsbury’s (2001) study of the introduction of recycling programs at universities started with

interviews with program leaders across 60 different universities. In this exploratory study, he found that

the implementation of recycling came in two forms, either programs staffed by a full-time recycling

coordinator or ones where the responsibilities were added to existing work roles. The fieldwork (in

iteration with the literature) suggested several reasons why there might be variation in staffing models for

recycling programs. To test these hypotheses, he then conducted a survey of 154 schools in order to

confirm the role that status and social activism had in shaping adoption patterns.

Thus, qualitative fieldwork and survey studies can be useful complements (Sieber, 1973, provides

an excellent description of these complementarities). Qualitative research can be seen as a precursor to a

survey; or a survey can be seen as a useful supplement to fieldwork. Surveys are very tricky to get right.

Assuring the validity of the questions and the scales is essential for generating insight. For those scholars

whose initial intention is to do survey research, conducting in-depth fieldwork in order to develop a

survey may lead to a more valid set of measures than if they had just pre-tested a survey designed in the

absence of these insights. Fieldwork can also help in the interpretation and theorizing of the survey data.

For scholars starting with qualitative fieldwork, such as Lounsbury (2001), the idea to conduct a survey

might only emerge during the research process at a point when it becomes beneficial to collect data over a

larger number of sites. Surveys can contribute to the analysis of qualitative data from the field by

confirming or contextualizing observations derived from a smaller sample.

Mixing quantitative and qualitative research -7-

Using Qualitative Data for the Development of a Construct and/or Model, Which is then Tested in
an Experiment.

Elsbach (1994) studied the California beef industry in order to understand how organizations can manage

perceptions of negative events in order to maintain their legitimacy. Her paper presents three studies

sequentially. In the first two, she conducted interviews of those attempting to shape perceptions (15

executives in beef producer organizations and industry associations) and the intended audiences (15

members of the press, nutritionists, and politicians). The analysis of these interviews (as well as of

supplemental press articles and press releases) allowed her to produce a typology of different responses to

negative events based on the form and content of the communications and to construct three propositions

of the relationships between the types of negative events, the types of responses and their effectiveness.

In the third study, she translated these insights into a vignette-based lab experiment in which she

tested the effects on various measures of organizational legitimacy of each of the different types of

accounts. The experimental results largely confirm the findings from the two qualitative studies. This

combination of studies proves to be effective in a way that none would be on its own. With only 15

interviewees for each of Study 1 and 2, the reader might not be convinced of the findings; but

complementing these insights with a lab experiment validates the qualitative findings.

A recent study by Ranganathan (2014) on craftsmen and traders of lacquer-work in India follows

a similar template, using ethnographic evidence to show how different orientations towards work shape

prices charged and then conducting a field experiment to validate the findings and hone in on the

mechanism (with the counterintuitive insight that the craftsmen will charge less if they think the customer

will appreciate the product more).

Coupling fieldwork and experiments is not frequently found in the literature but offers great

potential. As Fine and Elsbach (2000) point out, qualitative field data can contribute a richer,

contextualized set of theories to test in experiments where lab work alone is a poor proxy for the real

world. On the other hand, they argue, qualitative data alone may produce theories that are highly complex

and too specific to the case or cases studied. They suggest, as the examples above illustrate, that coupling

fieldwork and experiments in an iterative process can lead to more nuanced analyses and more rigorous

Mixing quantitative and qualitative research -8-

understandings of the mechanisms at play.

Starting with Quantitative Data

An alternative model of mixing qualitative and quantitative methods is to start with a statistical regression

and then follow with qualitative evidence. This approach can have two effects. The first is to lend face

validity to regressions. If the relationships identified using necessarily abstracted quantitative measures

make sense in the context of the lived experience of people in those contexts, one can feel more

comfortable that the measures capture the constructs they are meant to capture and that the associations

(perhaps even causal relationships) identified are not spurious. Second, qualitative analysis may also

allow the researcher to explain the underlying mechanisms behind a relationship identified in a

regression, ruling out some and offering in-depth elaborations of others.

One example is Fernandez-Mateo’s (2009) study of gender disadvantage in contract employment.

This study is based on data from the placement records of a temporary staffing agency showing how the

gender wage gap evolves over time. She used the quantitative data in a series of analyses to show that

men accrue benefits from tenure at a greater rate than do women. She also ruled out a number of possible

mechanisms – such as firm-specific skills that develop over time – previously suggested by the literature.

In the second part of the paper, she identifies the two remaining possible mechanisms (the supply-side

explanation of general skills acquisition and the demand-side explanation of implicit biases). She then

used information from observations and 49 interviews at the staffing agency to adjudicate between them,

suggesting that implicit bias on the part of employers is the most plausible explanation for the cumulative

disadvantages of women in this context.

In an interesting twist on the use of surveys, in Bidwell’s (2009) effort to collect survey data

using in-person surveys in IT outsourcing, he ended up accumulating a good deal of rich qualitative

evidence that emerged in the discussions associated with collecting the quantitative survey data. He used

the qualitative data to give richness to the explanations about the differences between contractors and

employees and when each would be deployed on a project. And, as illustrated above, Edmondson’s

(1999) study came full circle, having used qualitative evidence to generate a survey and then returning to

fieldwork to understand the mechanisms underlying the effects from the survey-based quantitative

Mixing quantitative and qualitative research -9-

analysis.

While it is certainly true that not all quantitative, regression analyses demand a qualitative

component in the published version of the paper, it is also true that nearly any quantitative scholar would

benefit from entering into the field. Even the most high-tech, sophisticated regression analysis can benefit

from some fieldwork to assure the regression results make sense and are grounded in reality. This is

important because variables used in quantitative studies are always simplifications or approximations of

the underlying constructs and the sanity check with the field helps assure the scholar of the validity of the

measures. Unfortunately, few quantitative scholars receive training in qualitative methods and therefore

can feel ill-equipped to engage in this complementary work.

Quantifying Qualitative Data

A further model for the relationship of qualitative and quantitative research is in the quantification of

qualitative data (see McPherson & Sauder, this volume, for further detail on this subject). Scholars may

undertake this for the purposes of conducting subsequent statistical analyses to confirm findings.

Quantification may also help to represent qualitative evidence compactly, potentially revealing patterns

that might not be visible otherwise.

Barley’s (1986) study of the introduction of CT scanners into two hospitals is largely an

ethnographic study of the different organizational forms that can emerge from the adoption of the same

technology, but he introduced quantitative elements to reinforce or validate his analyses. He based the

bulk of the story on his observations of the negotiations about who makes the decisions in the scanning

process in which he documents a series of scripts (different by hospital) that were enacted over time. To

demonstrate the differences in the two hospitals, he counted the occurrences of these various scripts in his

field notes and showed that different patterns emerged in the different contexts.

Over the course of these observations, he recorded 400 radiological examinations of which 91

were CT scans sufficiently documented to understand whether the radiologist or the technologist was the

primary decision maker in the various decisions required by each scan. He calculated the percent of

decisions made by the radiologist for each scan (49 scans in one hospital and 42 in the other) and used the

plots of these ratios over time to demonstrate not only the shifts towards the technologist over time in

Mixing quantitative and qualitative research - 10 -

each, but also the different rates and phases associated with the different settings. He then offered

correlations established in two very simple regressions to support his qualitative conclusions about the

differences in rates between the two hospitals and also to validate his identification of the different phases

of structuration in each.

In a similar study of organizational change in two hospitals, Kellogg (2009) examined the degree

to which new practices associated with work hour reductions (such as handing off patients to doctors

coming on in the next shift) were implemented. She counted the different kinds of change mobilization

activities in 202 different meetings and the percent of handoffs that occurred in 101 sign-out encounters

she observed. She used a comparison of these counts to demonstrate that doctors in the successful hospital

engaged in a set of practices that those in the unsuccessful hospital did not.

In Wageman’s (2001) study of how leaders foster team effectiveness, she transformed interview

responses into quantitative variables by using multiple coders to assess the presence of various team

characteristics – several dimensions of team design, self-managing behaviors and leadership coaching –

as being either high, medium or low. She then matches these quantified data with survey responses and

archival information on team performance in a regression analysis in order to conclude that the

effectiveness of leaders’ coaching depending on how well teams were designed to begin with. In this

study, the qualitative data are not presented in a separate narrative but rather only as they are quantified.

Because of the rhetorical power of numbers, scholars are often tempted to generate counts from

qualitative, even ethnographic, data. As the Barley (1986) and Kellogg (2009) studies show, such

numbers can provide useful confirmation of the patterns developed through the qualitative analysis. In

each of these studies, counting is appropriate because the researchers were able to observe repeated

equivalent events collected in a systematic way. Other forms of quantification might be more problematic.

For example, generating a quantitative rating of degree (of importance, of intensity, etc.) using a Likert

scale should be seen as suspect especially if it is generated by the researcher herself. Better versions of

this approach might involve multiple coders not informed of the research question with tests for inter-

rater reliability (as in Wageman 2001). Alternatively, one could get the informants themselves to provide

a numerical rating through a survey instrument (perhaps used in the context of an interview, e.g.,

Mixing quantitative and qualitative research - 11 -

Eisenhardt & Bourgeois, 1988). But, it should be emphasized that good qualitative research does not have

to offer quantifications, and indeed, such strategies should only be deployed when it is appropriate and

useful in extending the analysis.

Recent trends in quantification of texts, from simple word counts to more sophisticated computer

science-based methods such as topic modeling or feature extraction, are another form of quantification of

qualitative data (e.g., Kaplan & Vakili, Forthcoming). With the increasing availability of electronic texts,

these approaches are only likely to grow over time (increasingly blurring the distinctions between

qualitative and quantitative evidence and analysis). Some of the early approaches have usefully mixed

these quantifications with qualitative evidence, such as in DiMaggio, Nag and Blei’s (2013) study of

controversies in arts funding in the US where they use topic modeling to identify topics emerging from

over 8,000 newspaper articles and then mobilize these topics in a rich historically informed explanation of

how these controversies played out.

Mixing Methods in Research Programs Rather than Single Articles

These same principles can also be applied within a research program rather than in a single paper. As

those who have attempted to shepherd a mixed method paper through the review process have discovered,

it is very difficult to introduce the required detail for each method in a journal-length article. The rewards

for succeeding in this endeavor are high, but there are also times when it is more appropriate to iterate

between quantitative and qualitative work, focusing single articles on specific data sets and

methodologies. The accumulation of these studies over the course of a research program can lead to rich

insights that might not always be possible to generate in a single mixed-method article.

An example would be Lounsbury’s stream of research on recycling programs. The survey-based

study reported above included only some of the rich detail he gained through the fieldwork that preceded

it. In other papers, he took advantage of the more than 160 interviews he conducted to report on how

collective entrepreneurship contributed to the creation of a new occupational category (Lounsbury, 1998)

and to illustrate different forms of institutional analysis (Lounsbury, 1997). Kaplan conducted an

ethnography of strategy making in one organization in the communications industry (Kaplan, 2008b)

concurrently with a quantitative study of more than 70 firms in the industry over 20 years (Kaplan,

Mixing quantitative and qualitative research - 12 -

2008a). Each study looked at a different aspect of the role of cognition in shaping how organizations

respond to radical shifts in the industry – the first looking at one firm’s response to the crash of 2001-

2002 and the other looking at the response to the emergence of fiber-optics in an industry previously

dominated by copper wiring. Insights from the field study that cognition and incentives were tightly

intertwined in how decision-making unfolded (in process she called “framing contests”) led her to explore

how variables measuring CEO cognition and organizational incentives might interact in the large-sample

study. As a result, she was able to demonstrate that CEO attention can compensate for poorly aligned

organizational incentives. The ethnographic study of the decision making processes in one organization

could then explain why this might be the case.

While in both of these examples the papers were published separately, the research process itself

was highly iterative. Rather than splitting up these results into multiple papers, another alternative (taken

less frequently in management scholarship) is to write a book that collects these insights together.

Mische’s (2008) sociological study of mobilization of Brazilian youths combining ethnography and social

network analysis is a powerful example of this approach.

Observations about Mixed Methods Research

Across these examples, the mixing of methods takes two forms: one is confirmation and the other is

complementarity (Small 2011). (This is the simplest typology. Greene, Caracelli and Graham (1989) and

Bryman (2006) offer more granular categorizations for those who are interested.) Sometimes the

quantitative analysis confirms a pattern observed in the qualitative data, either through quantification of

that data or through triangulation in a subsequent study. Or, the qualitative data can offer confirmation of

the validity of constructs or mechanisms identified in quantitative research. In complementary studies,

qualitative analyses can get at mechanisms underlying relationships identified in regression analyses, or

quantitative studies can establish connections to performance of the constructs identified in qualitative

studies.

Thus, the popular dichotomy – that quantitative research is deductive and qualitative is inductive

or exploratory – is less useful when we consider the research discussed above. Sometimes qualitative

evidence is used to confirm quantitative findings, sometimes it is used to explore a phenomenon in order

Mixing quantitative and qualitative research - 13 -

to develop hypotheses for quantitative testing. Sometimes quantitative research can be exploratory – e.g.,

is there a relationship between these two variables? – and qualitative research can explain why. In other

words, once we mixed qualitative and quantitative methods, we see that each type of research plays a

variety of roles. These relationships between quantitative and qualitative data and analysis can be built

into the initial research design (e.g., Kaplan’s studies which were designed as multi-method from the

outset) or may emerge over time (e.g., Lounsbury’s survey work which only emerged as he was analyzing

his interview data or Fernandez-Mateo’s fieldwork which was instigated by the desire to sort through

alternative explanations of her quantitative results). Where one starts may be as much a matter of taste as

of any hard and fast rules about matching methods to inductive or deductive research problems.

Sometimes mixing qualitative and quantitative methods is based on the same data and involves

different forms of analysis, as in Tripsas’ (1997) historical study of the typesetter industry or Barley’s

(1986) study of CT scanners in hospitals. Other times, the mix requires two different data collection

strategies, as in Edmondson’s (1999) interviews and surveys or Elsbach’s (1994) interviews and lab

experiments. Small (2011) defines the former case as “mixed data analysis” studies and the latter as

“mixed data collection” studies. The challenge with either case is that the researcher must give adequate

treatment to each method. The risk, in the constraining form of a journal article, is in glossing over some

of the methodological details in order to save space. But, for a multi-method paper to be convincing, the

reader must be privy to the methodological details in the same way she would be in a single method

paper. Elsbach’s (1994) study is a model here as she presents each of her three studies sequentially,

including very rich detail on the data sources and analytical methods: she does not give short shrift to any

of the three methodological approaches.

As Edmondson and McManus (2007) have pointed out, mixing qualitative and quantitative

evidence does not always lead to better research or a stronger paper. Mixing methods is often worthwhile

because it is a useful route to a “full” understanding of a problem. However, in undertaking multi-method

research, the scholar risks being jack-of-all-trades and master of none. In the presentation of research

results, if the researcher gives each dataset inadequate treatment, the evidence may remain superficial or

unconvincing. Adding qualitative data to a quantitative analysis must serve a purpose other than

Mixing quantitative and qualitative research - 14 -

lengthening the paper. Including quantitative analyses in a rich qualitative paper must be done sufficiently

rigorously and add enough insight so as not to appear to be a transparent rhetorical strategy to convince

readers who would be presumed to be unconvinced by qualitative data only.

In Western societies, we privilege quantitative knowledge as being somehow more “objective”

than qualitative evidence (Denis, Langley, & Rouleau, 2006; Porter 2005). Mixing methods is often a

strategy that qualitative researchers are tempted to deploy for rhetorical reasons in a field (management)

where qualitative work is still in the minority. That is, because we are socialized to be convinced by

quantitative evidence, many scholars are likely to pursue quantitative research strategies. But, as Cameron

(1963) points out, “not everything that can be counted counts, and not everything that counts can be

counted.” The advantage of the mixed methods studies highlighted here is that they recognize the value of

iterating between that which can be counted and that which cannot in order to generate richer insights

about the phenomena of interest.

References

Barley, S. R. (1986). Technology as an Occasion for Structuring - Evidence from Observations of CT

Scanners and the Social-Order of Radiology Departments. Administrative Science Quarterly,

31(1), 78-108.

Bidwell, M. (2009). Do Peripheral Workers Do Peripheral Work? Comparing the Use of Highly Skilled

Contractors and Regular Employees. Industrial & Labor Relations Review, 62(2), 200-225.

Bresman, H. (2010). External Learning Activities and Team Performance: A Multimethod Field Study.

Organization Science, 21(1), 81-96. .

Briscoe, F. (2007). From iron cage to iron shield? How bureaucracy enables temporal flexibility for

professional service workers. Organization Science, 18(2), 297-314.

Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative

Research, 6(1), 97-113.

Mixing quantitative and qualitative research - 15 -

Cameron, W. B. (1963). Informal sociology, a casual introduction to sociological thinking. New York,:

Random House.

Canales, R. (2014). Weaving Straw into Gold: Managing Organizational Tensions Between

Standardization and Flexibility in Microfinance. Organization Science, 25(1), 1-28.

Creswell, J. W., & Clark, V. L. P. (2007). Designing and conducting mixed methods research. Thousand

Oaks, Calif.: SAGE Publications.

Denis, J. L., Langley, A., & Rouleau, G. A. (2006). The power of numbers in strategizing. Strategic

Organization, 4(4), 349-377.

DiMaggio, P., Nag, M., & Blei, D. (2013). Exploiting affinities between topic modeling and the

sociological perspective on culture: Application to newspaper coverage of US government arts

funding. Poetics, 41(6), 570-606.

Doering, L. (2014). Rethinking Escalation of Commitment: Relational Lending in Microfinance. Working

Paper.

Edmondson, A. (1999, Jun). Psychological safety and learning behavior in work teams. Administrative

Science Quarterly, pp. 350-383

Edmondson, A. C., & Mcmanus, S. E. (2007). Methodological fit in management field research. Academy

of Management Review, 32(4), 1155-1179.

Eisenhardt, K. M., & Bourgeois, L. J., III. (1988). Politics of Strategic Decision Making in High-Velocity

Environments: Toward a Midrange Theory. Academy of Management Journal, 31(4), 737-770.

Elsbach, K. D. (1994). Managing Organizational Legitimacy in the California Cattle Industry - the

Construction and Effectiveness of Verbal Accounts. Administrative Science Quarterly, 39(1), 57-

88.

Ely, R. J. (1994). The Effects of Organizational Demographics and Social Identity on Relationships

among Professional Women. Administrative Science Quarterly, 39(2), 203-238.

Fernandez-Mateo, I. (2009). Cumulative Gender Disadvantage in Contract Employment. American

Journal of Sociology, 114(4), 871-923.

Mixing quantitative and qualitative research - 16 -

Fine, G. A., & Elsbach, K. D. (2000). Ethnography and experiment in social psychological theory

building: Tactics for integrating qualitative field data with quantitative lab data. Journal of

Experimental Social Psychology, 36(1), 51-76.

Greene, J. C., Caracelli, V. J., & Graham, W. (1989). Toward a conceptual framework for mixed-method

evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274.

Jick, T. D. (1979). Mixing Qualitative and Quantitative Methods: Triangulation in Action. Administrative

Science Quarterly, 24(4), 602-611.

Kaplan, S. (2008a). Cognition, capabilities, and incentives: Assessing firm response to the fiber-optic

revolution. Academy of Management Journal, 51(4), 672-695.

Kaplan, S. (2008b). Framing contests: strategy making under uncertainty. Organization Science, 19(5),

729-752.

Kaplan, S., & Vakili, K. (Forthcoming). The double-edged sword of recombination in breakthrough

innovation. Strategic Management Journal.

Kellogg, K. C. (2009). Operating Room: Relational Spaces and Micro-Institutional Change in Two

Surgical Teaching Hospitals. American Journal of Sociology, 115(3), 657-711.

Lounsbury, M. (1997). Exploring the institutional tool kit - The rise of recycling in the US solid waste

field. American Behavioral Scientist, 40(4), 465-477.

Lounsbury, M. (1998). Collective entrepreneurship: the mobilization of college and university recycling

coordinators. Journal of Organizational Change Management, 11(1), 50-69.

Lounsbury, M. (2001). Institutional sources of practice variation: Staffing college and university

recycling programs. Administrative Science Quarterly, 46(1), 29-56.

McPherson, C.M., M. Sauder. 2015. The Quantification of Qualitative Data, Handbook of Innovative

Qualitative Research Methods: Pathways to Cool Ideas and Interesting Papers, K. Elsbach and

R. Kramer, eds.

Mische, A. (2008). Partisan publics: communication and contention across Brazilian youth activist

networks. Princeton: Princeton University Press.

Mixing quantitative and qualitative research - 17 -

 Porter, T. M. (1995) Trust in Numbers: The Pursuit of Objectivity in Science and Public Life. Princeton,

NJ: Princeton University Press.

Ranganathan, A. (2014). Choosing Meaning over Money? Evidence from a Field Audit Study with

Handicraft Artisans in Southern India. Working Paper.

Sieber, S. D. (1973). Integration of Fieldwork and Survey Methods. American Journal of Sociology,

78(6), 1335-1359. .

Small, M. L. (2011). How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing

Literature. Annual Review of Sociology, Vol 37, 37, 57-86. .

Tripsas, M. (1997). Unraveling the process of creative destruction: Complementary assets and incumbent

survival in the typesetter industry. Strategic Management Journal, 18, 119-142.

Wageman, R. (2001). How leaders foster self-managing team effectiveness: Design choices versus hands-

on coaching. Organization Science, 12(5), 559-577.

Author bio: Sarah Kaplan is Associate Professor of Strategic Management at the Rotman School of

Management, University of Toronto. She is a multimethod researcher exploring how organizations

participate in and respond to the emergence of new technologies and fields, with a particular focus on the

interpretive processes that shape choice and action. Her studies examine the biotechnology, fiber optics,

personal digital assistant, and nanotechnology fields. Most recently, she has turned her attention to the

new field emerging at the nexus of gender and finance. She received her PhD from MIT’s Sloan School

of Management and previously served on the faculty of the Wharton School, University of Pennsylvania

where she remains a Senior Fellow. She is Senior Editor at Organization Science, Guest Editor for the

Special Issue on New Research Methods for the Strategic Management Journal and formerly Associate

Editor for the Academy of Management Annals.

Mixing quantitative and qualitative research - 18 -

View publication stats