Bailey Sellers

DUE: September 11, 2020

Qualitative Methods
Word Count: 2228

Short Paper 1: KKV and RSI

In the social science discipline, there is an ongoing debate over the advantages

and disadvantages of quantitative and qualitative methods for cultivating research. This

dialogue has sparked a number of scholars to examine these methods and distinguish

their respective strengths and weaknesses. The merits of each approach are strongly

contested topics with robust perspectives on each side of the dispute. King, Keohane,

and Verba’s work, Designing Social Inquiry, is argued as one of the most influential

pieces of literature regarding the topic of quantitative and qualitative methods. KKV’s

publication concerns suggestions for better qualitative research, yet the majority of their

writing is focused on how qualitative researchers should implement quantitative

measures to perform research. While Designing Social Inquiry is undeniably significant

for the discipline and dialogue between methods, this piece of literature has attracted

considerable criticism. In a direct response to Designing Social Inquiry, Brady and

Collier (along with others) wrote Rethinking Social Inquiry (RSI). The goal of

Rethinking Social Inquiry is not to simply critique KKV’s shortcomings but to

ultimately broaden the methodological dialogue. Both KKV and RSI discuss a number of

important topics. The duration of this paper will consider keys points that were echoed

in both texts and ultimately focus on causal inference, increasing the number of

observations and the how the authors view the relationship between quantitative and

qualitative methods. The ultimate aim of this paper is to cultivate a meaningful dialogue

between the two works.

 An important topic that thoroughly discussed in Designing Social Inquiry is

causal inference. Throughout the totality of this book, the authors stress the importance

of causal inference and that it should be sought after more than descriptive inference.

KKV’s perspective of causal inference is pulled from Holland, who established a,

“counterfactual understanding of causation,” (Brady and Collier, 2010, pp. 37). In very

simple terms, this view of causal inference can be explained as A causes B. If A causes B,

then the natural question that follows is: “What would have happened if A would not

have been included in the analysis?” Therefore, this view deals with something that

happens and something that does not happen. For social science research, this can be

problematic. As the authors in Rethinking Social Inquiry state, this view implies that

there are essentially two parallel universes, where all things are equal except for one

(Brady and Collier, 2010, pp. 38). Consider this example: a political scientist seeks to

better understand the causes of insurgent recruitment in countries where the

government is considered repressive. This political scientist hypothesizes that increased

state repression will lead to increased insurgent recruitment. In order to achieve causal

inference, the political scientist would have to choose two very homogenous villages,

where one village would be the treatment group that experienced state repression and

where the other village would be the control group that experienced no state repression.

Obviously, this research experiment would be an extremely difficult task since political

science researchers are not dictators and cannot orchestrate an experiment like this.

This example demonstrates what KKV calls the “fundamental problem of causal

inference” (King et al., 1994, pp. 82).

KKV acknowledges this pressing issue and offers a number of statistical

techniques that will, in their opinion, mitigate the problem. Two primary prescriptions
KKV offer is regression analysis and increasing the number of observations, both of

which are empirical tools of analysis. In Rethinking Social Inquiry, the authors

highlight the issues that arise from depending upon regression analysis and increasing

the number of observations. In terms of regression analysis, RSI suggests that in order

for a regression analysis to produce accurate results, the model must also be correct. RSI

goes on to write that if a researcher, “estimates a regression method with empirical data,

then the model cannot be fully tested,” (Brady and Collier, 2010, pp. 18). Another

assumption that is made when using linear regression models is that the research

assumes that the relationship between the independent and dependent variable are

linear and also assumes unit homogeneity. An additional prescription that KKV offers

concerns increasing the number of observations to gain more leverage, this too can

cause issues and will be discussed further in detail in following sections. The main point

here is that the authors of RSI disagree with KKV’s view on the adoption of quantitative

methods for qualitative research as it is a very narrow approach. In addition,

quantitative tools have their shortcomings and KKV does not do an adequate job

addressing the potential pitfalls. KKV ultimately dismiss any unique contribution of

qualitative research. This outlook can make one question why have qualitative research

methods when quantitative methods is superior.

As previously stated, regression analysis and increasing the number of

observations is mainly used for quantitative analysis and KKV suggests that qualitative

researchers should adopt these tools to explain causal inference. An important

assumption posed by KKV is unit homogeneity. KKV ultimately argues that unit

homogeneity assists in alleviating the fundamental problem of causal inference (King et

al., 1994, pp. 92). Unit homogeneity happens when the independent variables have the
same value for a certain independent variable. It is important to note that unit

homogeneity rarely ever happens within the social sciences. For example, consider

coups and dictators. Just because coups typically occur when there is a dictator in power

does not mean it will have every time. Not every dictator will experience a coup. KKV

also suggests that researchers should, “make the substantive implications of this weak

spot in their research designs extremely clear and visible to readers,” (King et al., 1994,

pp. 91). This quote is important because assumptions and inferences are closely

associated and dependent on one another. In order to effectively display an inference,

one must be clear on the assumptions they are making.

An important area of departure between Rethinking Social Inquiry and

Designing Social Inquiry regards causal inference. In Rethinking Social Inquiry, the

authors point out how KKV stresses an abundance of significance on causal inference

but they argue that KKV does not offer researchers enough assistance on how to choose

the correct variables (Brady and Collier, 2010, pp. 59). For example, KKV discusses

omitted variable bias and claims that researchers should incorporate all applicable

omitted variables and discard any extraneous explanatory variables (King et al., 1994,

pp. 172 and pp. 184). The authors of RSI view these suggestions as problematic because

they do not believe that KKV offers enough guidance the to researcher. A common

concern is what constitutes a relevant or an irrelevant variable? How do researchers

distinguish between the two? (Brady and Collier, 2010, pp. 59) KKV suggests that

common issues concerning causal inference have been rectified in the quantitative

approach and qualitative researchers should implement these resolutions. RSI disagrees

with this viewpoint and argue that causal inference in both approaches can be

problematic. For instance, Brady’s chapter in RSI argues that KKV’s view of causal
inference does not pay enough attention to causal mechanisms (Brady and Collier, 2010,

pp. 58). While causal inference may show cause and effect, it lacks the mechanisms

necessary for explaining the why something occurred. In general, the authors of RSI

argue that KKV overstates the use and effectiveness of casual inference and disregard

how difficult it is to achieve in both quantitative and qualitative approaches.

A key topic that is discussed in both KKV and RSI is the number of observations

within an analysis. In simplistic terms, KKV argues that the more observations the

better the analysis. Much of this opinion comes from the assumption that increasing the

N (number of observations) leads to increased leverage. KKV states, “the most reliable

practice is to collect data randomly on a large number of units,” (King et al., 1994, pp.

199). Again, the authors in RSI disagree with this notion and argue that this assumption

is far too narrow and does not place enough attention on the potential problems that can

occur when increasing the N. A primary issue with increasing the number of

observations concerns the level of generality and the loss of conceptual knowledge

(Brady and Collier, 2010, pp. 19). When a researcher increases the number of

observations, they run the risk of including cases that may not be comparable or

equivalent. Overall, RSI suggests that increasing the number of observations does not

necessarily make one’s analysis better because some cases will not be similar.

An instance of how increasing the number of observations can be problematic is

the example of the United Nations: Women in Politics. The United Nations created a

map that displays every country in the world and the percentage of women that are in

parliament. This data shows that women make up 20% of Saudi Arabia’s “parliament”

(Women in Politics: 2019, 2019). This is very interesting since Saudi Arabia does not

have a parliament. While this dataset has a large number of observations, the case of
Saudi Arabia demonstrates that the data is not necessarily accurate. This example

highlights that increasing the N does not necessarily make analyses better.

To mitigate this issue, the authors of RSI suggest that researchers should

acknowledge additional alternatives to employ. In RSI, the authors write about data set

observations and causal process observations. CPOs are closely associated with process

tracing. Process tracing assesses, “diagnostic pieces of evidence… with the goal of

supporting or overturning alternative explanatory hypothesis,” (Brady and Collier, 2010,

pp. 201) and the diagnostic pieces of evidence are the CPOs (Brady and Collier, 2010,

pp. 201). In other words, CPOs are data that sheds light on the mechanisms. RSI offers

the example of Snow and Cholera; this instance demonstrates a successful CPO. Before

Snow’s experiment, the said cause of cholera was miasma, or bad air supply. Snow

investigated two different water supply companies (small n) to investigate if

contaminated water causes this disease. Through the use of CPOs, Snow discovered that

Cholera was not transmitted by miasma but through contaminated water supplies

(Brady and Collier, 2010, pp. 225).

CPOs are contrasted with DSOs. Data set observations deal with variables instead

of mechanisms. DSOs utilize regression analysis and are frequently used in quantitative

methods. The example featured in RSI regards the 2000 presidential election in Florida

and the debate over loss votes due to different time zones (Brady and Collier, 2010, pp.

240). Social scientist Lott conducted research on this controversy and argued that there

were at least 10,000 lost votes in Florida’s panhandle (they are in the Central Time

Zone). The alleged lost votes were caused by media outlets in the Eastern Time Zone

releasing statements that Al Gore was the winner of Florida when their polls closed on

election day. To conduct his research, Lott used a “difference-in-differences” analysis

that are systems of regression analysis and data set observations (Brady and Collier,

2010, pp. 40). Brady in RSI challenged this research through the use of CPOs and found

the numbers to be incorrect. Brady argues that since the media released the statement

around 7pm (Eastern Time), approximately 4,200 people would have been influenced, if

they even heard the statement (Brady and Collier, 2010, pp. 240). The main question

proposed by Brady is, how many people actually heard the media statement that Al Gore

had won? Brady ultimately concludes that at most, 224 people would have been affected

by these statements (Brady and Collier, 2010, pp. 241). This example demonstrates that

while CPOs may appear as less complex, there are instances, such as this, where they are

deemed more effective in finding answers.

The aim of RSI is not to simply compose a critique of KKV, the authors of RSI

wish to broaden the methodological dialogue and offer qualitative suggestions that

researchers should employ. Weaved throughout Designed Social Inquiry is the notion

that quantitative methods are better tools for analysis and superior to qualitative

methods. A central concern with this argument is that if quantitative methods are

superior and should be implemented in qualitative research, should there even be

quantitative research methods? RSI authors vehemently disagree with this notion and

offer ample alternatives to quantitative methods. The authors within Rethinking Social

Inquiry also point out the often-ignored pitfalls of the quantitative approach and offer

solutions to mitigate key issues. Their view offers an expanded outlook and displays key

advantages to conducting qualitative research.

In closing, this paper’s aim was to construct a thoughtful dialogue between

Designing Social Inquiry and Rethinking Social Inquiry. KKV’s contribution to the

discipline is undeniable and has given researchers within the social science discipline a
number of recommendations to create better research. However, RSI’s response is

necessary and significant. Instead of qualitative researchers feeling pressure to

implement quantitative methods into their analysis, the authors of RSI give

recommendations that embrace the uniqueness of qualitative research. Both of these

works attempt to help researchers conduct more effective research and cover numerous

topics. For this paper, I focused on the key points of casual inference, increasing the

number of observations, and how the authors view the relationship between

quantitative and qualitative methods.

 Works Cited:

King, G., Keohane, R. O., & Verba, S. (1994). Designing Social Inquiry: Scientific
Inference in Qualitative Research. Princeton University Press.

Brady, H. E., & Collier, D. (2010). Rethinking Social Inquiry: Diverse Tools, Shared
Standards(Second ed.). Rowman & Littlefield Publishers

Women in politics: 2019. (2019). UN Women. https://www.unwomen.org/en/digital-

library/publications/2019/03/women-in-politics-2019-map