857093

research-article2019
QIXXXX10.1177/1077800419857093Qualitative InquiryMaxwell

Research Article
Qualitative Inquiry

The Value of Qualitative Inquiry

2020, Vol. 26(2) 177­–186
© The Author(s) 2019
Article reuse guidelines:
for Public Policy sagepub.com/journals-permissions
DOI: 10.1177/1077800419857093
https://doi.org/10.1177/1077800419857093
journals.sagepub.com/home/qix

Joseph A. Maxwell1

Abstract
This article focuses on public policies and programs as a major component of the “public sphere,” and argues that
qualitative inquiry can make uniquely valuable contributions to their development and evaluation. These contributions
include understanding (a) how people interpret and respond to such policies, (b) contextual variability and its effects on the
implementation and consequences of these policies, and (c) the processes through which policies achieve their results. The
movement for “evidence-based” policy and practice has largely ignored these issues, but they are critical for developing
policies that actually achieve their goals and avoid unintended and damaging consequences.

Keywords
public policy, qualitative inquiry, public sphere, implementation

In this article, I address the development and evaluation of
public policies and programs as a major part of the “public
sphere,” and the roles that qualitative research can play in
this endeavor. To do this, however, I want to begin by con-
sidering why the value of qualitative research for these
tasks has even been questioned, or seen as problematic. In
my doctoral training in anthropology, in the 1970s and
1980s, I never encountered this prejudice; “applied anthro-
pology” (which was basically applied qualitative research)
was assumed to be practice- and policy-relevant. Michael
Quinn Patton’s (1980) Qualitative evaluation and research
methods and David Fetterman’s (1984) Ethnography in
educational evaluation were widely used in qualitative
methods courses, and qualitative researchers were a signifi-
cant part of the evaluation community, as recognized in
Cook and Reichardt’s (1979) important work Qualitative
and quantitative methods in evaluation research.
This is not to say that there was no controversy about
qualitative methods. Mary Anne Pitman reported an inci-
dent from her own experience, with two applied anthropol-
ogist colleagues, at the annual meeting of the Evaluation
Research Society in 1977:
We were in a standing-room-only crowd listening to an
evaluator, an organizational psychologist by training,
presenting his case for the state licensing of evaluators . . . The
speaker’s proposed licensing criteria would have mandated a
gate-keeping structure that would “keep the charlatans out of
the field.” It soon became clear to the three of us, however, that
we were the charlatans. Qualitative researchers, according to
the speaker, were threatening the integrity of evaluation,
particularly the rigor of its procedures and the validity of its
findings. (Pitman & Maxwell, 1992, p. 730)
Such conflicts were temporarily lessened by the merger
of the Evaluation Research Society (largely quantitative)
and the Evaluation Network (largely qualitative) as the
American Evaluation Association in 1986, but were not
eliminated even within that organization, and reemerged
more broadly with the movement for “evidence-based
policy” in the 1990s, which promoted a hierarchy of meth-
ods for establishing evidence, with randomized controlled
trials (RCTs) as the “gold standard” and qualitative meth-
ods relegated to the bottom (e.g., National Research
Council, 2002).
The Origins of Quantitative
Dominance in the Social Sciences
What was the basis (aside from competition for jobs) for
this dismissal of qualitative research? To understand this, I
believe that it’s necessary to go back several centuries, to
the spectacular successes of Newton’s laws in explaining
the physical universe. Thomas Kuhn (1957) stated, in The
Copernican revolution “These mathematical derivations
were without precedent in the history of science. They
transcend all the other achievements that stem from the
new perspective introduced by Copernicus” (p. 256). They
were unprecedented in two ways that have been profoundly
important for the relationship between quantitative and
1
George Mason University, Fairfax, VA, USA
Corresponding Author:
Joseph A. Maxwell, College of Education and Human Development,
George Mason University, 44 Endicott Road, Stoneham, MA 02180, USA.
Email: jmaxwell@gmu.edu
178 Qualitative Inquiry 26(2)
qualitative research. First, Newton’s laws were entirely
mathematical; there were no underlying theories, pro-
cesses, or mechanisms on which they depended. (The latter
feature was later incorporated in David Hume’s “constant
conjunction” theory of causation, which held that causa-
tion simply is the regular association of phenomena, with
nothing “behind” these that explains this conjunction. This
later came to be known as the “regularity” theory of causa-
tion, which has been influential in quantitative research up
to the present.) Second, they were universal, applying
without exception to all physical phenomena.
The influence of these two ideas was profound and long-
lasting, and strongly shaped the conceptions of what “sci-
ence” appropriately meant for the sciences generally. Giere
(1999) argued that
The theories about science prominent in Europe and North
America until around 1960 derived ultimately from the
Enlightenment of the eighteenth century, a period in which the
achievements of the seventeenth century scientific revolution
became incorporated into a more general cultural worldview. A
number of concepts came into prominence in particular ways
during that time, including those of laws of nature, scientific
truth, and scientific rationality. (pp. 3-4)
As a consequence,
Many people on both sides [of the ‘science wars”] seem so to
have internalized the Enlightenment view of science that, for
them, to challenge aspects of that view is to challenge science
itself, and, conversely, to defend science is to defend it in its
Enlightenment form. (Giere, 1999, p. 5)
Giere noted elsewhere that, ironically,
The notion of a law of nature did not arise out of the practice
of science itself. Sometime in the seventeenth century it was
imported into discourse about science from Christian
theology, both directly and indirectly through mathematics.
Originally, laws of nature were understood as God’s laws for
nature. Thus, behind the laws of nature was a lawgiver, from
whence came the universality and necessity of such laws . . .
It also seems pretty clear that this theological notion of laws
of nature became prominent through the writings of Descartes.
Newton then picked up the idea from Descartes, though he
hardly needed Descartes to encourage his theological
inclinations. How the notion got stripped of its theological
implications throughout the eighteenth and nineteenth
centuries seems a story yet to be told. Twentieth century
philosophers struggled to understand the nature of natural
laws, especially features such as universality and necessity, in
completely secular terms. (Giere, 2006, p. 70, referencing
Weinert, 1995)
In this article, I argue that, to counter arguments for the
inherent superiority of quantitative research, it is essential
for qualitative researchers to understand the assumptions,
explicit and implicit, that lie behind these arguments, and
how to effectively challenge these assumptions. A full
account of the origins and impacts of these assumptions is
beyond the scope of this article, and an overview, with a few
examples, will have to suffice.
The two fundamental concepts discussed above, of gen-
eral laws and their mathematical formulation, were central
to prominent philosophies of science that emerged during
the next two centuries, which assumed that the goal of sci-
ence was to establish such laws, not just for physics, but for
the social sciences as well. Friedrich Hayek (1952), in The
Counterrevolution of Science, described in detail how this
happened. In doing so, Hayek (1952) introduced into
English the term “scientism” to refer to
an attitude which is decidedly unscientific, in the true sense
of the word, since it involves a mechanical and uncritical
application of habits of thought to different fields from those
in which they have been formed . . . The scientistic as opposed
to the scientific view is not an unprejudiced but a very
prejudiced approach which, before it has considered its
subject, claims to know what is the most appropriate way of
investigating it. (p. 24).
The influence of these ideas on the philosophy of science
eventually led to the positivism of Auguste Comte (1798-
1857), who stated that “the fundamental character of all
positive philosophy is to regard all phenomena as subject to
invariable natural laws, whose precise discovery and reduc-
tion to the smallest number possible is the aim of all our
efforts” (Comte, quoted by Hayek, 1952, p. 328).
Although Comte was a major force in establishing the
view of science as inherently dependent on “natural laws,”
the mathematicization of these laws, and the incorporation
of this into later positivist thinking, was influenced more by
his contemporary, the astronomer and statistician Adolphe
Quetelet (1796-1874). For Quetelet,
the more advanced the sciences have become, the more they
have tended to enter the domain of mathematics . . . We can
judge of the perfection to which a science has come by the
facility, more or less great, with which it may be approached by
calculation. (Hayek, 1952, p. 357)
A key aspect of Quetelet’s impact on statistics was his
expanded use of the concept of “average.” Astronomers had
realized that when multiple measurements of a single phe-
nomenon (e.g., the observed time that it took for a planet to
move a given distance) varied, the best estimate of the
“true” value was the arithmetic mean (average) of these
measurements; variations from this mean were treated as
“error.” This technique, which is quite legitimate for this
purpose, was then transferred by Quetelet to single mea-
surements of multiple individuals or events, treating
Maxwell 179
individual variation as “error” (Mayr, 1982, p. 47; Rose,
2015, pp. 23-28). This was key to Quetelet’s and others’
dream of “‘automating’ the human sciences by substituting
calculation for intuition” (Weisberg, 2014, p. 4).
Quetelet’s emphasis on averages as the true measure of
some phenomenon was far-reaching. “scholars and thinkers
in every field hailed Quetelet as a genius for uncovering the
hidden laws of society” (Rose, 2015, p. 31). The father of
experimental psychology, Wilhelm Wundt, invoked
Quetelet in claiming that “more psychology can be learned
from averages than from all philosophers, except Aristotle”
(Rose, 2015, p. 31). Weisberg (2014) argued that these
influences led to “willful ignorance” of the actual variabil-
ity of phenomena and a profound misunderstanding and
mismeasure of uncertainty, and have distorted the use of
statistics for understanding variability.
Logical positivism, which was the dominant philoso-
phy of science during the first half of the 20th century,
differed in important respects from Comte’s positivism,
but it retained these two elements: the focus on mathe-
maticization and the goal of general laws. The most
influential presentation of the latter was Carl Hempel’s
“deductive-nomological” or “covering law” model of
science. For Hempel, there was “no difference in princi-
ple between human sciences like history and natural sci-
ences like physics. To the extent that both offer
explanations, both appeal to general, predictive laws”
(Bohman, 1991, p. 19).
However, logical positivism was shown in the 1950s to
be largely unworkable, and many of its tenets were aban-
doned by philosophers. A. J. Ayer (1936), whose book
Language, Truth, and Logic basically introduced logical
positivism to the English-speaking world, famously said
much later, when asked what the problems were with logi-
cal positivism, “I suppose the greatest defect is that nearly
all of it was false” (Ayer, 1978).
Despite this, positivist ideas have continued to influence
the thinking of many social researchers. Steinmetz (2005)
wryly noted that “Despite repeated attempts by social theo-
rists and researchers to drive a stake through the heart of the
vampire, the [social science] disciplines continue to experi-
ence a positivistic haunting” (p. 3). Contemporary versions
of positivism have typically held
that there is a unity of method between the natural and the
social sciences; the notion that the social sciences ought to
search for eternal, lawlike generalizations; . . . a rejection of
explanations which refer to subjective states of individuals
such as motives or purposes; [and] a predilection toward
quantification. (Baert, 1998, p. 140)
Problems With the Positivist Agenda
These goals have proved elusive even in many of the natu-
ral sciences, aside from physics and those fields, such as
astronomy, that rely substantially on physics (although
Cartwright [1983, 1999] has argued that they are problem-
atic even for physics). In genetics, the discovery of the
structure of DNA in 1953 led many researchers to believe
that they were on the verge of a Newtonian transformation
of their field. These researchers, some trained in physics,
sought the simplest possible organisms for study, aiming to
establish general laws analogous to those of physics (Keller,
1983, pp. 4-6, 159-165). This goal was eventually undercut
by (among other things) the work of Barbara McClintock
on the genetics of maize, for which she belatedly received a
Nobel Prize. McClintock’s approach was essentially quali-
tative, grounded in the naturalist tradition of field biology,
and relying on detailed microscopic study of the cytology of
her maize plants. This was guided by what she called “a
feeling for the organism,” an intimate understanding of the
entire plant and of the diversity of genetic features that her
maize plants exhibited (Keller, 1983, pp. 8-9, 197-201).
For evolutionary biology, Stephen Jay Gould (1989)
similarly argued that
The stereotype of the “scientific method” has no place for
irreducible history. Nature’s laws are defined by their invariance
in space and time . . . [and] presuppose that all times can be
treated alike . . . But the restricted techniques of the “scientific
method” cannot get to the heart of [a] singular event involving
creatures long dead on an earth with climates and continental
positions markedly different from today’s. The resolution of
history must be rooted in the reconstruction of past events
themselves—in their own terms—based on narrative evidence
of their own unique phenomena . . .
Historical science is not worse, more restricted, or less capable
of achieving firm conclusions because experiment, prediction,
and subsumption under invariant laws of nature do not
represent its usual working methods. The sciences of history
use a different mode of explanation, rooted in the comparative
and observational richness of our data. (pp. 277-279)
The eminent biologist Ernst Mayr (1976, pp. 408-415;
1982, 1988, pp. 9-19) has provided a detailed critique of
what he termed “essentalist” approaches that ignored indi-
vidual variability and sought to establish general laws for
biology. He argued that biology in general does not seek or
rely on such laws, but develops theories that have limited
applicability. He stated that “every student in biology is
impressed by the fact that there is hardly a theory in biology
for which some exceptions are not known” (Mayr, 1988, p.
19), and that “as is characteristic for so much of the biologi-
cal sciences, the answers to many of these questions are of
a qualitative, rather than a quantitative, nature” (Mayr,
1976, p. 410).
In the social sciences, efforts to establish causal laws, in
anything like the sense of Newton’s laws, have been a strik-
ing failure. Donmoyer (2008) stated that “For a variety of
reasons, including the complexity of social phenomena and
180 Qualitative Inquiry 26(2)
the ever-changing cultural dimension of social life, quanti-
tative researchers have had difficulty producing even prob-
abilistic findings that can be generalized to even delimited
populations” (p. 372). Mohr (1996), in a work on the causes
of human behavior, similarly argued that “few practitioners
in the social and behavioral disciplines would claim today
that there are universal laws governing human behavior” (p.
1), and that such laws are unnecessary for the causal expla-
nation of such behavior (an issue discussed in detail below).
Some of the problems with attempts to develop causal
laws in the social sciences are illustrated by the work of the
anthropologist George Peter Murdock (1949), who com-
piled in the 1940s a massive inventory of data, which he
later called the Human Relations Area Files, on all human
societies for which such data were available, and quantita-
tively analyzed these data to discover correlations among
features of social structure. In his book Social Structure, he
presented a unified general theory of how changes in social
structure occur, supported by voluminous correlations,
many with highly significant chi-square values, among fea-
tures of social structure in 250 societies. On this basis, he
asserted that “cultural forms in the field of social organiza-
tion reveal a degree of regularity and of conformity to sci-
entific law not significantly inferior to that found in the
so-called natural sciences” (p. 259).
However, many of the actual correlations that Murdock
reported were quite modest. In addition, for at least one of
Murdock’s culture areas (the indigenous tribes of the North
American Plains), one that was studied specifically to test
Murdock’s (and others’) theories, the data that he used were
often incorrect. This investigation showed that, although
there were some commonalities across these tribes, there
were also substantial differences, and that neither the com-
monalities nor the differences could be understood without
a detailed grasp of the specific natural and social environ-
ments that each tribe had encountered, its history of adapta-
tions to these environments, and an adequate theory
(including an understanding of the meanings and uses of
kinship terms) of how the changes in their kinship systems
and social structures occurred (Eggan & Maxwell, 2001;
Maxwell, 1978). The methods for establishing this were
fundamentally qualitative rather than quantitative (Maxwell,
2013, Chapter 9), and cast serious doubt on any claims
about “scientific laws” of social organization.
The failure of the positivist agenda to adequately repre-
sent or guide the social sciences, in combination with the
increasing dominance of quantitative methods in many
social science fields, was one factor leading to what have
been called the “paradigm wars.” Many qualitative research-
ers, following the mandate of Egon Guba and Yvonna
Lincoln (1989; Lincoln & Guba, 1985), adopted a radical
constructivism that rejected not only the goals of mathema-
ticization and general laws, but also the concepts of “real-
ity,” validity, causation, and even of generalization (Denzin,
1983, p. 133, cited by Schofield, 1990, p. 202). The philoso-
pher Susan Haack (2003) has trenchantly observed that
In the agenda inherited from logical positivism it was often
assumed that we must choose either methodological monism,
according to which the social sciences seek explanatory, causal
theories and use the scientific method just as physics and
chemistry do, or methodological dualism, according to which
the social sciences seek understanding rather than causal
explanation, and use a method of empathy rather than objective
observation. This makes me feel like the legendary Irishman
who asked for directions to a distant village: “sure and
begorrah, I wouldn’t start from here.” (pp. 165-166)
However, although qualitative researchers largely
rejected the relevance of mathematicization and natural
laws for their own approach, they have often accepted
these, at least implicitly, as valid for quantitative research
in the social sciences. A recent example, from the
American Psychological Association Publications and
Communications Board Task Force (2018), is found in
their statement on Reporting Standards for Qualitative
Primary, Qualitative Meta-Analytic, and Mixed Methods
Research in Psychology (Levitt et al. 2018):
Because their work tends to focus on human experiences,
actions, and social processes, which fluctuate, qualitative
researchers do not aim to seek natural laws that extend across
time, place, and culture, but to develop findings that are bound
to their contexts. (p. 29)
This statement appears to accept (or at least fails to chal-
lenge) the claim that discovering such laws is a legitimate
goal of quantitative research in the social sciences. However,
this claim, if valid, inherently justifies quantitative research-
ers’ arguments for their methods’ superiority in generalizing
their findings across times and settings, a key issue for pro-
grams and policies. Even works that ostensibly seek a broad
consensus across qualitative and quantitative research, such
as the U.S. National Research Council’s (2002) report
Scientific Research in Education, often uncritically incor-
porate these arguments, leading to the dismissal or subordi-
nation of qualitative research. (For a detailed critique of this
report, see Maxwell, 2004.)
The Strengths of Qualitative Research
for Developing and Evaluating Public
Policies and Programs
Having criticized some influential beliefs that have often
marginalized qualitative researchers in addressing public
policies and programs, I want to now turn to the particular
strengths of qualitative research and how these are relevant
to public policy. I will discuss three main strengths that I see
in qualitative research, strengths that are widely recognized
Maxwell 181
in the qualitative methods literature, but that are often
ignored in defending the value of qualitative research. I’ve
dealt with these strengths in more detail elsewhere
(Maxwell, 2004, 2012).
The message of this article can be summarized in three
statements about what matters in planning and evaluating
such policies and programs: that meaning matters, context
matters, and process matters. Meaning, context, and pro-
cess are three key issues for, and strengths of, qualitative
research. What I hope to do here is to show how these are
critically relevant for developing and evaluating public
policies and programs, and to challenge quantitative
researchers’ claims for the greater value of their methods
in this task.
Meaning Matters
Qualitative research is essential for understanding the
meanings that a particular policy or program has, both for
those creating or implementing this and for those affected
by it. By “meaning,” I refer broadly to people’s beliefs, val-
ues, theories, understandings, and other “mental” phenom-
ena, what is often encompassed by the phrase “participants’
perspectives”; understanding these is the primary goal of
what is often termed “interpretive inquiry.” The importance
of such beliefs, values, and understandings in creating and
evaluating policies and programs is basically just common
sense; it should be obvious that people’s understanding of
and attitudes toward a policy or program impact its effec-
tiveness and can create unintended consequences.
However, “evidence-based” approaches typically
ignore these issues, or address them only through “thin”
surveys of people’s “attitudes,” which fail to provide an
in-depth understanding of these issues and are almost
always analyzed to discover “average” values rather than
to explore individual differences and how these interact
with contextual influences. An understanding of the com-
plexity and diversity of people’s views of a policy or pro-
gram, how these influence their responses to this policy
or program, and how these might be addressed in devel-
oping and explaining the policy or program are essential
to its effective development and implementation.
Problems with the development and rollout of the
Affordable Care Act in the United States were due in part
to a failure to adequately understand how the policy was
perceived by a large portion of the electorate—how they
understood the policy, the assumptions and values that
shaped these perceptions, and the actions to which these
perceptions and beliefs led.
This emphasis on meaning does not imply or require a
constructivist or relativist ontology; any philosophical posi-
tion other than extreme positivism or radical relativism
assumes that meanings are ontologically real—that a per-
son’s meanings exist independently of the researcher’s
perceptions, and that a researcher might be mistaken about
these meanings. Thomas Schwandt (2007), in his Dictionary
of Qualitative Research, argued that
On a daily basis, most of us probably behave as garden-variety
empirical realists—that is, we act as if the objects in the world
(things, events, structures, people, meanings, etc.) exist as
independent in some way from our experience with them. We
also regard society, institutions, feelings, intelligence, poverty,
disability, and so on as being just as real as the toes on our feet
and the sun in the sky. (p. 256)
This is implicitly assumed by much of the actual practice
of qualitative research, which is concerned with how to
avoid misunderstanding these meanings or imposing the
researcher’s own theories and biases. This practice implic-
itly (and sometimes explicitly) combines an ontological
realism about meanings (these meanings exist indepen-
dently of the researcher’s constructions) with an epistemo-
logical constructivism that recognizes that our interpretation
of these meanings is our own construction, and that reality
is more complex than any single construction can ade-
quately capture. Our understanding of people’s meanings,
as with our understanding of things in general, is necessar-
ily partial, incomplete, and fallible; we are inherently inca-
pable of “objectively” capturing these. A particularly clear
statement of this position is by the anthropologist Fredrik
Barth (1987):
Like most of us, I assume that there is a real world out there—
but that our representations of that world are constructions.
People create and apply these constructions in a struggle to
grasp the world, relate to it, and to manipulate it through
concepts, knowledge, and acts. In the process, reality impinges,
and the events that occur subsequently are not predicated on
the cultural system of representations employed by the people,
although they may be largely interpretable within it. A people’s
way of life is thus not a closed system, contained within their
own cultural constructions. That part of the real world on
which we as anthropologists need to focus is comprised of this
widest compass: a natural world, a human population with all
its collective and statistical social features, and a set of cultural
ideas in terms of which these people try to understand and cope
with themselves and their habitat. (p. 87)
For a more detailed account of a realist understanding of
meaning and culture, see Maxwell (2013, Chapter 2).
This argument implies that reasons can legitimately be
seen as causes of people’s behavior, a view that has been
advanced by numerous philosophers, including Bhaskar
(1989), Davidson (1980, 1993), McGinn (1991/1999), and
Putnam (1999), as well as by psychologists and social sci-
entists (Bandura, 1986; House, 1991; Huberman & Miles,
1985, p. 354; Menzel, 1978; Mohr, 1996; Pawson, 2006, pp.
27-29; Sayer, 1992, pp. 110-112). Thus, Mohr (1996) con-
cluded that
182 Qualitative Inquiry 26(2)
people’s motives or reasons for undertaking certain behaviors
could not form part of general laws governing those behaviors,
but . . . those same motives or reasons could, in the individual
case, cause the behaviors to be carried out (p. vi).
The psychologist Albert Bandura (1986) similarly argued
that
What people think, believe, and feel affects how they behave.
The natural and extrinsic effects of their actions, in turn, partly
determine their thought patterns and affective reactions. (p. 25)
This position is generally accepted in both our everyday
explanations of people’s behavior and in psychology as a
science. However, the sharp distinction between “meaning”
and “cause” explanations by Dilthey, Weber, Winch, and
others has been far more influential in qualitative research.
The view of meanings as causes becomes less problematic
when the dependence of causality on laws is abandoned
(Putnam, 1999, pp. 73-91, p. 200, fn. 11, 14; Sayer, 1992,
pp. 126-127). The issue of causality is discussed in more
detail below, under “process.”
Context Matters
The importance of context is a key feature of qualitative
research; qualitative researchers inherently incorporate the
local context of the phenomena they study in their analyses
and findings, and reject the idea of “context-independent”
conclusions about the effects of a policy or program.
Unfortunately, most approaches to “evidence-based policy”
fail to incorporate this context-dependence, assuming that
the results of “rigorous” evaluations will be inherently gen-
eralizable. Thus, RCTs, which are seen as the “gold stan-
dard” for understanding the effect of some intervention, are
treated as the “best evidence” for generalizing the effects of
a given policy.
Nancy Cartwright and Jeremy Hardie, in their important
book “Evidence-based Policy: A Practical Guide to Doing it
Better,” stated at the very outset that
You are told: use policies that work. And you are told: RCTs–
randomized controlled trials–will show you what these are.
That’s not so. RCTs are great, but they do not do that for you.
They cannot alone support the expectation that a policy will
work for you . . . For that, you need to know a lot more.
(Cartwright & Hardie, 2012, p. ix)
The “lot more” that their book explains is primarily an
understanding of context: why context is important, and
how to assess its implications for implementing a particular
policy in your own setting. This includes identifying the rel-
evant features of the setting in which the policy is imple-
mented, which can support or impede the goals of the
policy, and how these interact with each other and with the
policy to shape its outcomes. Although they never use the
term “qualitative” in presenting these issues, their book is to
a substantial extent a detailed argument for the importance
of qualitative understandings in implementing any policy in
a specific context.
The “external validity” of experimental research is usu-
ally assumed to be primarily a matter of the similarity of
the intended context to that in which the research was
done. For example, Shadish, Cook, and Campbell (2002)
stated, as their first “principle of generalized causal infer-
ence,” that “scientists generalize by judging the apparent
similarities between the things they studied and the targets
of generalization” (p. 353). Cartwright and Hardie (2012)
countered that
For us, the key question is how good a job this advice does in
getting you from “it worked there” to “it will work here.” The
answer: you are lucky if it gets you anywhere. (p. 46)
“Similarity” is both too vague (what features must be simi-
lar?) and too demanding (almost no two contexts are similar
in all important respects), and provides no guidance in
understanding how your specific context will affect the out-
come of the policy (Cartwright & Hardie, 2012, pp. 46-49).
Although internal (causal) validity in a given experimental
study is clearly relevant to the generalizability of the result,
the latter is an entirely different issue from the former, and
conceptualizing the two as both being forms of “validity” (a
view that seems inherently dependent on the idea of “gen-
eral laws”) can lead to lack of attention to what is critical for
the transferability of the results to other contexts. Shadish
et al. (2002) acknowledged that “the embeddedness of
experimental results in a particular local context seems to
provide little basis for generalizing results beyond that con-
text” (p. 341). However, in their extensive discussion of
external validity/generalization (pp. 83-93, 341-455), they
paid little attention to a key issue for generalizability: how
will the process that resulted in the experimental outcome
in that specific context operate in different contexts?
The qualitative concept of “transferability” is a far more
useful tool, for both qualitative and quantitative research,
than is “external validity” in understanding how a given
policy or program may work in a new context. The transfer-
ability of either quantitative or qualitative results is not a
straightforward extension of the findings to “similar” con-
texts, but involves developing a theory of the processes that
led to a particular outcome, and of how these processes
might operate in a different context (Donmoyer, 2008;
Maxwell & Chmiel, 2014, pp. 240-241). Becker (1992)
showed that, in a qualitative study of social organization in
men’s and women’s prisons, a similar process (that prison-
ers developed a social organization that provided things that
they felt deprived of in prison) resulted in very different
organizational structures in the two prisons, both because
Maxwell 183
the administrative policies in the two prisons (context) were
different, and because the deprivations the women felt
(meanings) were very different from those of the men.
Process Matters
This last point leads to the third issue: process.
Anticipating the likely consequences of implementing a
particular policy or program in a specific setting depends
fundamentally on understanding the actual processes by
which the policy or program could achieve its results,
which may result in unintended as well as intended out-
comes. These processes incorporate the meanings that
stakeholders and recipients have toward the policy and
how these influence their actions, as well as the many
features of the particular setting that influence the imple-
mentation and outcomes of the policy.
By “processes,” I’m explicitly referring to causal pro-
cesses. Causality has been a deeply contested concept in
qualitative research, largely due to the prevalence of the
positivist understanding of causality as simply the regular
association of events or variables (Maxwell, 2012). As
described above, both sides in this debate have largely
assumed the positivist view of causation, and differed pri-
marily on whether it applied to qualitative research.
Quantitative researchers generally argued that it did apply,
and that qualitative research was therefore inherently infe-
rior to quantitative research in establishing the regularities
that this conception required. Some qualitative researchers
accepted (explicitly or implicitly) this understanding of
causation; Patton (1990) warned that
One of the biggest dangers for qualitative researchers is that,
when they begin to make interpretations about causes,
consequences, and relationships, they fall back on the linear
assumptions of quantitative analysis and begin to specify
isolated variables that are mechanically linked together out of
context. (p. 423)
Other qualitative researchers either argued that this concep-
tion of causation didn’t apply to qualitative research because
its assumptions and goals were different from those of
quantitative research, or that the entire idea of causation
was so seriously flawed that it needed to be completely
rejected (e.g., Guba & Lincoln, 1989, pp. 44, 86; Lincoln &
Guba, 1985, p. 141).
All of these positions ignored the development of an
alternative, realist, process-oriented conception of causa-
tion, both by philosophers (e.g., Salmon, 1984, 1998) and
by researchers (e.g., Huberman & Miles, 1985; Pawson,
2006; Pawson & Tilley, 1997; Sayer, 1992, 2000; for a
detailed discussion of how this conception is relevant for
qualitative research, see Maxwell, 2012, 2013, chapter 3).
Sayer (1992) argued that
social science has been singularly unsuccessful in discovering
law-like regularities. One of the main achievements of recent
realist philosophy has been to show that this is an inevitable
consequence of an erroneous view of causation. Realism
replaces the regularity model with one in which objects and
social relations have causal powers which may or may not
produce regularities, and which can be explained independently
of them. In view of this, less weight is put on quantitative
methods for discovering and assessing regularities and more on
methods of establishing the qualitative nature of social objects
and relations on which causal mechanisms depend (pp. 2-3).
In particular, causal processes are not well addressed by
experimental research. Shadish et al. (2002) stated that
the unique strength of experimentation is in describing the
consequences attributable to deliberately varying a treatment.
We call this causal description. In contrast, experiments do
less well in clarifying the mechanisms through which and the
conditions under which that causal relationship holds—what
we call causal explanation. (p. 9; emphasis in original)
Qualitative research, with its unique ability to identify
the actual processes operating in a specific event or case,
and the contextual influences on these, is thus particularly
relevant to understanding how policies and programs lead
to particular (sometimes unexpected) outcomes. This was
well understood by many qualitative sociologists prior to
the 1950s, who argued against the positivist emphasis on
quantitative associations between variables and for a pro-
cess understanding of causality (Abbott, 2001, pp. 107-
108). Even Shadish et al. (2002) acknowledged that
“qualitative methods provide an important avenue for dis-
covering and exploring causal explanations” (p. 389), and
described how, in one experiment, the actual cause of the
result was “revealed only by the qualitative examination of
the actual operation of groups” (p. 391).
Miles and Huberman (1994), in their influential book
Qualitative Data Analysis, coined the phrase “local causal-
ity” to refer to process-oriented, context-specific interpreta-
tions of causation, stating that
Qualitative analysis, with its close-up look, can identify
mechanisms, going beyond sheer association. It is unrelentingly
local, and deals well with the complex network of events and
processes in a situation. It can sort out the temporal dimension,
showing clearly what preceded what, either through direct
observation or retrospection. (p. 147)
Ray Pawson and Nick Tilley (1997; Pawson, 2006,
2013) have provided a detailed critique of quantitative/
experimental strategies for program and policy evaluation,
and present a realist strategy for evaluation that signifi-
cantly incorporates qualitative methods. They use the acro-
nym “CMOs” to refer to the complex system of contexts,
184 Qualitative Inquiry 26(2)
mechanisms, and outcomes that they see as central to under-
standing how programs and policies achieve their results.
Cartwright and Hardie (2012) similarly use the concept of
“causal cakes,” diagrams showing the “ingredients” that
must be present for the policy to achieve its intended out-
comes in your situation (pp. 62-63).
The value of qualitative methods for evaluating pro-
grams and policies was eventually recognized by the major
figure in the experimental evaluation of social programs,
Donald Campbell, who stated that
without competence at the qualitative level, one’s computer
printout is misleading or meaningless. We failed in our thinking
about programme evaluation to emphasize the need for a
qualitative context that could be depended on . . . The lack of
this knowledge . . . makes us incompetent estimators of
programme impacts, turning out conclusions that are not only
wrong, but often wrong in socially destructive ways. (Campbell,
1984, p. 36, quoted by Pawson, 2006, p. 50)
The key issue for qualitative researchers who accept
these arguments is, how do I do this? The development
and evaluation of public programs and policies is cur-
rently dominated by quantitative researchers. However,
there is an extensive body of literature, by qualitative
evaluators and others, on how to understand the pro-
cesses involved in developing, implementing, and
assessing programs and policies (e.g., Bennett &
Chekel, 2015; Cartwright & Hardie, 2012; Donaldson,
Christie, & Mark, 2009; Pawson & Tilley, 1997; Weller
& Barnes, 2014.)
In addition, many quantitative researchers have now
recognized the value of integrating qualitative and quanti-
tative approaches—a practice that has become known as
“mixed methods” research, although its use long predated
the invention of the term (LeVine, 2016; Maxwell, 2016).
Case reports of such “mixed methods” evaluations go
back at least to a study by Trend (1979), and more recent
examples can be found in books edited by Thomas Weisner
(2005) and M. Cameron Hay (2016). (See the index in the
Hay volume for policy-relevant examples.) The U.S.
Department of Education, the Spencer Foundation, and
the W. T. Grant Foundation have provided significant
funding for such integrative efforts, and the fifth edition of
the Handbook of Research on Teaching contains a major
paper by Pamela Moss and Edward Haertel (2016) on
“Engaging Methodological Pluralism.”
This integration of qualitative and quantitative
approaches is clearly not just “positivism in drag”
(Giddings, 2006). In many of these studies, qualitative
methods were essential to revising the initial research
questions as a result of unexpected findings, and made
major contributions to the conclusions; see, for example,
the studies discussed by Maxwell and Loomis (2003) and
Maxwell, Chmiel, and Rogers (2015). Greene (2007) has
argued persuasively for the value of integrating different
“mental models” for research; the compatibility and com-
plementarity of qualitative and quantitative approaches is
clearly recognized, both within the mixed methods com-
munity and in other disciplines that have developed such
integration independently of the “mixed methods” move-
ment (Maxwell, 2016). Maxwell (1993) presented some
ideas on how qualitative researchers can effectively inter-
act with quantitative researchers to gain acceptance for
their methods and ideas.
Conclusion
The development and evaluation of public policies and pro-
grams has mostly been dominated by quantitative research-
ers, aided by a long history of quantitatively oriented views
in the philosophy of science. However, these views have
been strikingly unsuccessful in guiding the social sciences
(and even some of the natural sciences), and have been
challenged by more recent developments in the philosophy
of science. These developments have led to increased rec-
ognition of the strengths of qualitative research for develop-
ing and evaluating policies and programs, and of the
complementarity and compatibility of qualitative and quan-
titative approaches. I encourage qualitative researchers to
engage with these efforts.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect
to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
References
Abbott, A. (2001). Chaos of disciplines. Chicago, IL: University
of Chicago Press.
Ayer, A. J. (1936). Language, truth, and logic. London, England:
Victor Gollancz Ltd.
Ayer, A. J. (1978). Interview (this contains a link to a video of the
television interview in which Ayer said this). Retrieved from
https://www.basicincome.com/bp/ratherlike.htm
Bandura, A. (1986). Social foundations of thought and action: A
social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Barth, F. (1987). Cosmologies in the making: A generative
approach to cultural variation in New Guinea. Cambridge,
UK: Cambridge University Press.
Becker, H. S. (1992). Generalizing from case studies. In E. Eisner
& A. Peshkin (Eds.), Qualitative inquiry in education: The
continuing debate (pp. 233-242). New York, NY: Teachers
College Press.
Bennett, A., & Chekel, J. T. (2015). Process tracing: From meta-
phor to analytic tool. Cambridge, UK: Cambridge University
Press.
Maxwell 185
Bhaskar, R. (1989). Reclaiming reality: A critical introduction to
contemporary philosophy. London, England: Verso.
Bohman, J. (1991). New philosophy of social science. Cambridge,
MA: MIT Press.
Campbell, D. C. (1984). Can we be scientific in applied science? In
R. F. Connor, D. G. Altman, & C. Jackson (Eds.), Evaluation
studies review annual (pp. 24-48). Thousand Oaks, CA: Sage.
Cartwright, N. (1983). How the laws of physics lie. Oxford, UK:
Oxford University Press.
Cartwright, N. (1999). The dappled world: A study of the boundar-
ies of science. Cambridge, UK: Cambridge University Press.
Cartwright, N., & Hardie, J. (2012). Evidence-based policy:
A practical guide to doing it better. Oxford, UK: Oxford
University Press.
Cook, T. D., & Reichardt, C. S. (1979). Qualitative and quantita-
tive methods in evaluation research. Beverly Hills, CA: Sage.
Davidson, D. (1980). Essays on actions and events. Oxford, UK:
Oxford University Press.
Davidson, D. (1993). Thinking causes. In J. Heil & A. Mele
(Eds.), Mental causation (pp. 3-17). Oxford, UK: Clarendon
Press.
Denzin, N. K. (1983). Interpretive interactionism. In G. Morgan
(Ed.), Beyond method: Strategies for social research (pp.
129-144). Beverly Hills, CA: Sage.
Donaldson, S. I., Christie, C. A., & Mark, M. M. (2009). What
counts as credible evidence in applied research and evalua-
tion practice? Thousand Oaks, CA: Sage.
Donmoyer, R. (2008). Generalizability. In L. Given (Ed.), The
Sage encyclopedia of qualitative research methods (pp. 171-
117). Thousand Oaks, CA: Sage.
Eggan, F., & Maxwell, J. A. (2001). Kinship and social organi-
zation. In R. DeMallie (Ed.), Handbook of North American
Indians, volume 13: Plains (pp. 974-982). Washington, DC:
Smithsonian Institution.
Fetterman, D. (1984). Ethnography in educational evaluation.
Beverly Hills, CA: Sage.
Giddings, L. S. (2006). Mixed-methods research: Positivism
dressed in drag? Journal of Research in Nursing, 11, 195-203.
Giere, R. N. (1999). Science without laws. Chicago, IL: University
of Chicago Press.
Giere, R. N. (2006). Scientific perspectivism. Chicago, IL:
University of Chicago Press.
Gould, S. J. (1989). Wonderful life: The Burgess Shale and the
nature of history. New York, NY: W.W. Norton.
Greene, J. C. (2007). Mixed methods in social inquiry. San
Francisco, CA: Jossey-Bass.
Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evalua-
tion. Newbury Park, CA: Sage.
Haack, S. (2003). Defending science within reason: Between sci-
entism and cynicism. New York, NY: Prometheus Books.
Hay, M. C. (Ed.). (2016). Methods that matter. Chicago, IL:
University of Chicago Press.
Hayek, F. A. (1952). The counterrevolution of science: Studies on
the abuse of reason. Glencoe, IL: The Free Press.
House, E. (1991). Realism in research. Educational Researcher,
20(6), 2-9+25.
Huberman, A. M., & Miles, M. B. (1985). Assessing local causal-
ity in qualitative research. In D. N. Berg & K. K. Smith (Eds.),
Exploring clinical methods for social research (pp. 351-382).
Beverly Hills, CA: Sage.
Keller, E. F. (1983). A feeling for the organism: The life and work
of Barbara McClintock. New York, NY: W. H. Freeman.
Kuhn, T. S. (1957). The Copernican revolution: Planetary astron-
omy in the development of Western thought. Cambridge, MA:
Harvard University Press.
LeVine, R. A. (2016). Repairing the fractured social sciences: An
introduction from a historical point of view. In M. C. Hay
(Ed.), Methods that matter: Integrating mixed methods for
more effective social science research (pp. 3-12). Chicago,
IL: University of Chicago Press.
Levitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M.,
Josselson, R., & Suárez-Orozco, C. (2018), Journal Article
Reporting Standards for Qualitative Primary, Qualitative
Meta-Analytic, and Mixed Methods Research in Psychology:
The APA Publications and Communications Board Task
Force Report. American Psychologist, 73(1), 26-46.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic evaluation.
Thousand Oaks, CA: Sage.
Maxwell, J. A. (1978). The evolution of Plains Indian kin termi-
nologies: A non-reflectionist account. Plains Anthropologist,
23, 13-29.
Maxwell, J. A. (1993). Gaining acceptance for qualitative meth-
ods from clients, policy-makers, and participants. In D. M.
Fetterman (Ed.), Speaking the language of power (pp. 105-115).
Washington, DC: Falmer Press.
Maxwell, J. A. (2004). Causal explanation, qualitative research,
and scientific inquiry in education. Educational Researcher,
33(2), 3-11.
Maxwell, J. A. (2012). The importance of qualitative research for
causal explanation in education. Qualitative Inquiry, 18, 649-
655.
Maxwell, J. A. (2013). A realist approach for qualitative research.
Thousand Oaks, CA: Sage.
Maxwell, J. A. (2016). Expanding the history and range of mixed
methods research. Journal of Mixed Methods Research,
10(1), 12-27.
Maxwell, J. A., & Chmiel, M. (2014). Generalization in and from
qualitative analysis (with Margaret Chmiel). In U. Flick (Ed.),
The Sage handbook of qualitative data analysis (pp. 540-
553). London, England: Sage.
Maxwell, J. A., Chmiel, M., & Rogers, S. (2015). Designing
integration in mixed method and multi-method research. In
S. Hesse-Biber & R. B. Johnson (Eds.), Oxford handbook of
multimethod and mixed methods research inquiry (pp. 223-239).
Oxford, UK: Oxford University Press.
Maxwell, J. A., & Loomis, D. (2003). Mixed methods design:
An alternative approach. In A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral
research (pp. 241-271). Thousand Oaks, CA: Sage.
Mayr, E. (1976). Evolution and the diversity of life. Cambridge,
MA: Harvard University Press.
Mayr, E. (1982). The growth of biological thought: Diversity, evo-
lution, and inheritance. Cambridge, MA: Harvard University
Press.
Mayr, E. (1988). Toward a new philosophy of biology. Cambridge,
MA: Harvard University Press.
McGinn, C. (1991). Conceptual causation: Some elementary
reflections. Mind, 100, 573-586. (Reprinted in C. McGinn,
Knowledge and reality: Selected essays, pp. 152-167, by C.
McGinn, Oxford, UK: Clarendon Press, 1999)
186 Qualitative Inquiry 26(2)
Menzel, H. (1978). Meaning: Who needs it? In M. Brenner, P.
Marsh, & M. Brenner (Eds.), The social contexts of method
(pp. 140-171). New York, NY: St. Martin’s Press.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data
analysis: An expanded sourcebook. Thousand Oaks, CA:
Sage.
Mohr, L. B. (1996). The causes of human behavior: Implications
for theory and method in the social sciences. Ann Arbor:
University of Michigan Press.
Moss, P. A., & Haertel, E. H. (2016). Engaging methodologi-
cal pluralism. In D. Gitomer & C. Bell (Eds.), Handbook of
research on teaching (5th ed., pp. 127-247). Washington, DC:
American Educational Research Association.
Murdock, G. P. (1949). Social structure. New York, NY: The Free
Press.
National Research Council. (2002). Scientific research in educa-
tion. Washington, DC: National Academy Press.
Patrick, B. (1998). Social Theory in the Twentieth Century.
Washington Square, NY: New York University Press.
Patton, M. Q. (1980). Qualitative evaluation and research meth-
ods. Thousand Oaks, CA: Sage.
Patton, M. Q. (1990). Qualitative evaluation and research meth-
ods (2nd ed.). Thousand Oaks, CA: Sage.
Pawson, R. (2006). Evidence-based policy: A realist perspective.
London, England: Sage.
Pawson, R. (2013). The science of evaluation: A realist manifesto.
London, England: Sage.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. London,
England: Sage.
Pitman, M. A., & Maxwell, J. A. (1992). Qualitative approaches to
evaluation. In M. D. LeCompte, W. L. Millroy, & J. Preissle
(Eds.), The handbook of qualitative research in education
(pp. 729-770). San Diego, CA: Academic Press.
Putnam, H. (1999). The threefold cord: Mind, body, and world.
New York, NY: Columbia University Press.
Rose, T. (2015). The end of average. New York, NY:
HarperCollins.
Salmon, W. C. (1984). Scientific explanation and the causal struc-
ture of the world. Princeton, NJ: Princeton University Press.
Salmon, W. C. (1998). Causality and explanation. New York,
NY: Oxford University Press.
Sayer, A. (1992). Method in social science: A realist approach
(2nd ed.). London, England: Routledge.
Sayer, A. (2000). Realism and social science. London, England:
Sage.
Schofield, J. W. (1990). Increasing the generalizability of qualita-
tive research. In E. W. Eisner & A. Peshkin (Eds.), Qualitative
inquiry in education: The continuing debate (pp. 201-232).
New York, NY: Teachers College Press.
Schwandt, T. A. (2007). The Sage dictionary of qualitative inquiry
(3rd ed.). Thousand Oaks, CA: Sage.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002).
Experimental and quasi-experimental designs for generalized
causal inference. Boston, MA: Houghton Mifflin.
Steinmetz, G. (2005). Introduction: Positivism and its others in
the social sciences. In G. Steinmetz (Ed.), The politics of
method in the human sciences (pp. 1-58). Durham, NC: Duke
University Press.
Trend, M. (1979). On the reconciliation of qualitative and quanti-
tative analyses: A case study. In T. D. Cook & C. S. Reichardt
(Eds.), Qualitative & quantitative methods in program evalu-
ation (pp. 68-86). Thousand Oaks, CA: Sage.
Weinert, F. (Ed.). (1995). Laws of nature: Essays on the philo-
sophical, scientific, and historical dimensions. New York,
NY: Walter de Gruyter.
Weisberg, H. I. (2014). Willful ignorance: The mismeasure of
uncertainty. Hoboken, NJ: John Wiley.
Weisner, T. S. (Ed.). (2005). Discovering successful pathways in
children’s development: Mixed methods in the study of child-
hood and family life. Chicago, IL: University of Chicago Press.
Weller, N., & Barnes, J. (2014). Finding pathways: Mixed meth-
ods research for studying causal mechanisms. Cambridge,
UK: Cambridge University Press.
Author Biography
Joseph A. Maxwell is a professor (Emeritus) in the Research
Methods program in the College of Education and Human
Development at George Mason University. His doctoral degree is
in anthropology, but his research and teaching have mainly been in
education, with a focus on methodology. He is the author of
Qualitative Research Design: An Interactive Approach, 3rd edi-
tion (Thousand Oaks, Sage: 2013) and A Realist Approach for
Qualitative Research (Thousand Oaks, Sage: 2012), as well as
many papers on qualitative and mixed methods research. His cur-
rent research deals with using qualitative methods for causal
explanation, validity in qualitative and quantitative research, the
history and breadth of mixed methods research, the value of philo-
sophic realism for research, and the importance of diversity and
dialogue across research paradigms and methods.