A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research

Authors: Mahoney & Goertz (2006)

The authors think of the two traditions as alternative cultures, which however complement
each other, so none of them should be excluded. The AIM of the authors is indeed to
overcome the view of the 2 cultures as contrasting, and instead enhancing cross-tradition
communication.
 Each has its own values, beliefs and norms.

King, Keohane, Verba (KKV) wrote a book in 1994 explicitly about qualitative research,
but containing the assumption that quantitative researchers have the best tools for
making scientific inferences, and hence qualitative researchers should attempt to
emulate quantitative tools to the degree possible.

The authors thus present a stylized view of both quantitative and qualitative research,
confronting both of them on 10 different criteria/areas (these are certainly not the only
areas in which the 2 traditions are different, but surely those that generate most of the
misunderstandings and miscommunication).

1. Approaches to explanation

A core goal of qualitative researchers is the explanation of outcomes (Y) in individual

cases, so look for the specific causes (A, B, C) that brought to a certain result (Y).
 They want to do so for each and every case that falls within the scope of the theory
that they are investigating.

1
  They basically move back from cases/outcomes to causes, and thus adopt a
“causes-of-effects” approach to explanation.
o Good theories should ideally be able to explain the outcome in all the
cases within the population: research goal is to explain particular outcomes
o Example: attempt to identify causes for the end of the Cold War, i.e what
were the causes that led to the end of the Cold War?
o There´s skepticism on this approach
On the contrary, statistical approaches to explanation usually use the “paradigm of the
controlled experiment”, i.e. with a controlled experiment one doesn´t know the outcome
until the treatment has been applied.
 Statistical approaches try to reproduce this paradigm in the context of an
observational (and not experimental) study, so they follow the “effects-of-causes”
approach – usually employed in experimental research – trying to estimate the
average effect of one or more causes across a population of cases.
 Quantitative researchers ask questions like: what is the effect of economic
development on democracy?
o Mostly questions like “what happens if…?”
Much misunderstanding between the 2 cultures seems to derive from these approaches to
explanation

2. Conceptions of causation
3. Multivariate explanations

4. Equifinality

Equifinality is also often referred to as “multiple causation” and the concept is strongly
associated with Ragin´s developed QCA approach (1987).
The term expresses the idea that there are multiple causal paths possible to the get to
the same outcome.
While equifinality plays a key-role in qualitative research when scholars think about causal
relationships, discussions on this topic are ABSENT in the quantitative world.
In the qualitative world, there are only a few causal paths possible to get to a specific
outcome, and each path is a specific conjunction of factors (which are not that many).
Since the scope conditions of QL work is narrower, the goal is to identify all causal paths
– i.e. for each case – that are present in the population (being the overall goal the one of
explaining cases).
On the contrary, for statistical (QN) models, there are thousands/millions of potential
causal paths to arrive to a particular outcome.
 This is basically due to the fact that: the right-hand side of the equation is a weighted
sum, and as long as the weighted sum is greater than the threshold (there are
countless ways in which this can be possible), the outcome should (on average)
occur, so one has equifinality in abundance.
 In QN research once again the goal is not to explain any particular cases but
rather to generalize about individual causal effects.

5. Scope and causal generalization

6. Case selection practices

2
7. Weighting observations

From the point of view of QL researchers, not all pieces of evidence count equally in order
to build an explanation.
 Comparison QL researchers and criminal detectives: they solve puzzles and explain
particular outcomes, by focusing on detailed fact gathering, working with similar cases
and knowledge of general causal principles.
o QL people DO NOT approach observations in a theoretically neutral way,
rather they ask themselves: ‘‘Given my prior theoretical beliefs, how much does
this observation affect these beliefs?’’
 Personal biases are important
 A single piece of evidence can affect very radically posterior beliefs
 E.g. theory holds that China performed better economically than India,
before 1980, because of the higher level of GDP per capita, but when a
new measure of economic development was introduced, which addressed
problems with the previous GDP per capita estimate and showed a similar
level of development in the 2 countries, the whole theory was called into
question and rejected.
 By contrast QN people generally make NO assumptions that some particular
observations could count more heavily than others.
o They usually weight A PRIORI all observations as EQUAL.
 With this approach, a single observation cannot affect that much the result (in
both positive and negative terms), but only a pattern of many observations can
bolster or call into question a theory.
 No importance of the individual factor, but importance given to the group of
observations.
Brady and Collier elaborate based on these differences, a distinction on the use of data:
a) Data-set observation: it is simply a row in a standard rectangular data-set and it´s
ordinarily what statistical researchers call a case/observation.
- Provide analytic leverage because they show (or not) patterns of association
between variables, and allow for estimation of the size of the effect.
- Especially helpful when one wishes to generalize about average causal
effects for a large population.
b) Causal-process observation: it is an insight/observation that provides info about
context or mechanism and contributes a diff. kind of leverage in causal inference.
- Causal-process observations are crucial for theory testing in a QL setting
because one sorts through the data with preexisting theoretical beliefs.
- Especially useful when one seeks to explain specific outcomes in particular
cases.

8. Substantively important cases

9. Lack of fit

In QL research, the researcher normally knows quite well each case under
investigation, so in this approach a particular case that is not conform to the
researcher´s causal model is NOT simply ignored: the researcher will instead try to

3
understand why this case follows a distinct causal patterns, and what are the distinctive
factors that lead to this.
 The QL researcher will thus try to understand WHY the particular/different case
didn´t conform to the theoretical expectation (Ragin).
In QN research instead the failure of a theoretical model to explain particular cases is
NOT A PROBLEM as long as the model provides good estimates of parameters for
the population as a whole. The exclusion of specific factors does not bias the parameter
estimates of the model, given that these factors are often not systematically correlated with
error terms specified in the model.
 The lack of fit of a theoretical model maybe due not only to omitted variables but
also to randomness and nonsystematic measurement error (which again however
don´t bias the results)
In sum, QL researchers believe that prediction error ‘‘should be explained, rather than
simply acknowledged’’ (Ragin 2004), so they may be troubled by statistical models that
explain only a small portion of the variation of interest, leaving the rest to the error term.
On the other side, QN researchers may be perplexed when qualitative researchers
spend a great deal of energy attempting to identify factors at work in non-conforming
cases. They (QN) may wonder, ‘‘Why use up valuable time on research that does not lead
to generalizable findings?’’

10. Concepts and measurement

QL: it is common to spend much time & energies developing clear and precise definitions
for concepts that are central in research.
 They´re concerned with conceptual validity
o Failure to address their concern is probable to result in measurement error
o Error comes at the concept level
 In qualitative research, in short, measurement error needs to be addressed and
eliminated completely, if possible.
QN: the focus is less on measurement error deriving from the definition & structure of
concepts;
 They are more concerned with OPERATIONALIZATION and the use of
indicators.
 concentrate on modeling measurement error and modifying indicators with little
concern for concept revision.
o Some quantitativists would go so far as to say that that a concept is
defined by the indicators used to measure it.
 For quantitative researchers, by contrast, measurement error is something that is
unavoidable but not devastating so long as it can be adequately modeled.

In sum, it is appropriate to speak of two separate strands in the methodological

literature on measurement error in political science: a QL strand that focuses on
concepts and conceptual validity and that is centrally concerned with eliminating
measurement error, and a QN strand that focuses on indicators and measurement
validity and that seeks to model measurement error and avoid systematic error.
 Cross-cultural communication is relatively rare.

4
Misunderstanding among representatives of the 2 cultures is enhanced by the fact that
the labels ‘‘quantitative’’ and ‘‘qualitative’’ do a poor job capturing the real differences
between the traditions.
 Better options could be: statistics vs. logic, effect estimation vs. outcome
explanation, population-oriented vs. case-oriented approaches.
Q2: 3 criteria that differentiate qualitative & quantitative research
designs (explain how thesemanifest both in qual.& quant.
approaches). [table 1.page 229 FUNDAMENTAL]
-> 3 criteria to take into consideration among 10:

1) Approaches to explanation: "causes of-effects" approach (QL) vs.

"effects-of causes" approach (QN)
2) Conceptions of causation: mathematical logic (QL) vs.
probabilistic/statistical logic (QN)
3) Multivariate explanations: INUS causation and occasional individual
effects (QL) vs. additive causation and occasional interaction terms (QN)
4) Equifinality: core concept and few causal paths (QL) VS. absent concept,
implicitly large number of causal paths (QN)
5) Scope & causal generalization: narrow scope (QL) to avoid causal
heterogeneity vs. broad scope (QM) analysts can omit minor variables to
say + general about large numbers of cases and the broader population.
6) Case selection practices: QL people are oriented toward positive cases
on dependent variable, while QN people select RANDOMLY on
independent variables, so that all cases are analyzed
7) Weighting observations: causal process observations (QL) and data-set
observations (QM): no assumptions that some piece of evidence should
count + heavily than others.
8) Substantively important cases: for QL these must be explained, while for
QM these shouldn't be given much attention. (If many other cases
conform,the exceptional case is not a special problem).
9) Lack of fit: QL people analyze closely the non-conforming cases, while QN
people consider the non-conforming cases as errors.
10) Concepts & measurement: QL analysts often spend time and energies
to develop clear and precise definitions x central concepts in their research.
Furthemore, the failure to address conceptual validity is a major source of
error. QN people instead are more focused on operationalization and the
use of indicators