28 _Straregies for Comparative Research in Political Science the consequences oftheir choices otha they can make good st Be decisions abou shoe chores. “ ‘The one thing that shouldbe universal in studying comparative poliss, whether in one country oF in many, and whether wing Statistical or qualitative methods ia constousatention o exp "tin nd cere dig Wa the hing the rsneh nts {Sexplain~ what is the dependent variable? When the problem lear the question then becomes: What isthe presumed “cre” of the phenomenon in question? Ie is better forthe development ofthe dscipline of political selence if that presumed cause te reited to some brosdet theoretial concer, but there may be interesting ‘esearch in which tat inital connection is cenvous at best Healy What evidence is needed to ‘prove’ the connection berween exse and effect and how ean that evidence be mustered? How ean we be ‘ss sure as possible about the quality of ete evidence? These rather Straightforward eesearch questions sce she foundations of rescareh desig, just a they might he the faces of hatriography ia weigh boring dseipine. This book ts intended tobe the reader answer ‘hese questions lchough the styl of answering tem may belie ‘nt from those tsually encountered in many pars oF he line chapter 2 The Logic of Comparison Le have alveady argued in bret that the logic of comparative analy i diferent from the logie of orher forms of social science fevenech, especially From the statstcal method that fends so domi ‘search (ace Ragin, 2006) This difference evident in terms ofthe tendency ofthe uni of analysis For stastical research fo be the individual as opposed to the groups ot politcal units ‘hore commonly encountered in conpacative research. I 8 se, {here importentgsin germs of che manner in whieh the reo methods ‘Seal withthe crucial question of controlling sourees of extrancous Variance, Inthe complex socal world, shere age any number of Fhetors associated with the variance obseeved in any autaber of ther vavables, Finding sigaificane statistieal corelations among those fncors ts eavely a. problem for social researches. What 1 tmore ofa problen is determining whether those correlations are Epicealy and chcoretically meaningtsl, that is, an we say coni= Alen that X causes Yor whether th reraly the product oF ‘then unmeasured, variables affetn red, ln other words, the real difieulty for the socal seiences is making convincing statements about the causation of political phenomena, given the complexity of interactions among the whole fnge of social phenomena and the ruumber of extemal sources of ‘Over a century and a hol ago, Joka Stuaet Mill (1846) was ginning to geapple wih some ofthe same iesues although he did ‘ot eall his efforts comparative polities. Instead, he was concerned ‘with de fandaonental logical and philosophical problem of proving ‘ausatin a itellectusl problem dha persist o this day (Blalock, 1964, King, Keohane and Verba, 1994; 75-114; Geeting, 2005). Many socal scientists remain very skeptical about any claims of ‘usation, believing that the complexity ofthe issues and the many 2930 _ Strategies for Comparative Research in Plitial Science | - interactions among variables make claims of eaueation suspect at bres. tn his analysis of causation, Mill presented theee conditions thae could be satisfied inorder ta demonsteate a eatsal connection between “variables 1. The Method of Agreement. To paraphrase Mil, thi method argues that, ifthe several observations of dhe phenamencn nder investigation (dependent variable) have only one of several possible causal circumstances in common, then the ‘irenmstance i which all he instances agree the cause ofthe phenomenon, 2. ‘The Method of Difference. Again paraphrasigs Mi argument that, if there Isa occutrence and nos ‘of a phenomenon ithe dependent variable), “tances in which these arcobacrved (ey the same i all circumstances save one, teanae ofthe occurrence 3. The Method of Concomitant Variation. Wtwo vatiables ted to vary inthe same pattern, then they ate somehow linked, either antally or through seme other pattern of conneetion (inked tosome tied variable). nd he circu vent variables) ace a that one ie the "Those three methods for coping with the problem of cavsution were devised long before contemporary social sience was invented, And indeed Mill did not consider that his methods were appropriate Tor the socialsciences. He (like many scholars now) believed t the complexity of the causal relationships encountered in social enquiry limited the possibility of discovering mesningfal caveal ‘lations. Despite Mis own cautions his three approaches to solv- ing the problem of eassation wil be echoed rather strongly in some of the research methods diseuased Inger in the book (ece Chapter 9), 4s wel arin the dizcussion of fundamental problems of compara: tive analysis inthis chapter “The basic problem for Millas wells for the contemporary sox researches is (0 isolate, a8 far ae possbley a factor or a limite fhumber of factors thar appear to produce (ar atleast are strongly Associated with) changes inthe dependent variable. This problems, inturn,a problem of identifying pssble ways to exclude the rarer ‘ous possible confounding factors in the relationship between ark ables, a problem of the greatest theoretical importance. The iMentiRcation ofthe confounding influences is also part a fonction | ‘The Logic of Comparison 34 afhovng equate theoretical guidance about whee 10 10k fo the i tice Chapter #) There ave any momber of possible 2 ef Sasa plal con ce theory iy the noe Het sures of conf ig variance sed by Nill are im marked conteast with iis buile on statistical swuch of contempor Sra nde eration of mall * eta on finding singe cause and damising all thers ~ some oar again he deemed wally inthe social world. That may et ery strong level of telatiostip eo expect to ind im te aeanstive esearch That said, this method ean be use 0 dismiss Soe causal factors, and cam help to Bid clear relationships Pg limited number of cases and variables, When expanded ime Ghapeer 7) this method becomes the foundation for much of [Sscmporary qualitative analysis arid can be ised to find mote fale interactions among explanatory variables. Comparative research design [A mote contemporary approach to the same problem addressed by Dill'ean be expressed im terms of the variance observed inthe pendent variable in comparative research, The fundamental Yor social eaearch, which will kep appearing in the course of this volume i ‘Maximize experimental variance, minimize error variance, and Control extraneous variance “This is a relatively casy phrase to repeat, but a substantially mote dict ask to implement in comparative research designs, In each dependent yartable that we utilize for comparative analysis, cere is Siwy some of each of these thre types of variance he question in how much, Empirically, t may be impossible to parcel out vari ‘Dhocino then theee categories. This ispossbiiryintrue because oF the umerous possible sources of extraneous variance, an the ‘umrouy soutes af ror in any aearement andthe site “Tifictey in knowing when any particular obscrvation is ffected by ‘roe Despite the impossibly of Knowing how eo atebte all che ‘hnerved Varian, the researcher must sil examine his or her ‘eethods with this basic question of researc design in mind.32 Strategies for Comparative Research in Poliieal Science Experimental vaviance isthe observed differences or changes in the dependent variable that are aFanetion ofthe independent va ables identified as central to the analysis For social acientific analy sis, even research a «qualitative nature, ecre must bea dependent vatiable “something we are attempting to explain In soine esearch situations, the dependent variable may be a simple dichotomy: evolutions oceur or they do not (Wickhhams-Crowley, 1991); people ote or they do not (Gray and Cauly 2000). In ether eases, the dlependent variable may be limited range of responses of gover ‘ments 10 changing, economic forunes (Gourevteh, 19865 Taylor, yt may bea continuous variable, such as per capita ies for social welfare ina range of countries (Hicks and Swank, 1992: Gupea, 2001), thae appears sich more ike what we usualy thiok of as a variable, For qualitative case studies, the Alependene variable may be changes im observed configurations Of political powee, oF the mature of the decisions taken by political lites, with those differences often being very subtle ‘Whatever the natuee ofthe variable nthe researc, dhe inestiga- ‘or i attempting to ensure thst Gwo characteristics exist for tae dependent variable. One is to be eectain that it doce rary. Same Published politcal science research does nor, in fact, utilize a ‘dependent variable hat varies at least in the data collected for the purposes of the particular research. For example, number of the fecent studies of democratization have examined only those cases Uhat have democratized successfully, however tha sucess defined ova, 1991; Ciprut, 2008). There is always the assumption that the other end ofthe continuum was diferene, but thae assumption, ‘as not eeally tested. Ir may be thatthe variables wed to expla? successful democratization may actually have roughly the same values for those countries that did not demoeratie successfully “This practice of sclecting cases on theie values on the depen variablchas been argued to be one ofthe moreal sins that can afflict comparative rexearch (King etal, 1994; Geddes, 2003), but ike, Imost sins, it sone which is practiced frequently, appears to make sense fo examine cases that satisfy some eriterion of relevance, but in face this siplycliminates crucial variance inthe phenomenon bbe explained. Rven wete relationships to be discovered between successful democratization and any presumed independent vas ables, we would not knovr if they were any different from the Fel ‘lonships that mig be found for unsuccessfal democratinations, ar for authoritarian regimes tha did not attempt to democraize stall, 2 Bthoteea ‘The Logie of Comparison 33 ese may simply be features of systems that have been authoritr Tees She pitas point i ine. Abus the bee raul ttuinable ian tony cases with ie same value on she dependent variables 10 SERS mae thot sore iccpndene vacate appears to be = ces serrrialin for the oecureenc ofthe depennt variable (Most hd Sears 1989:49-50), ‘rtiough the admonitions about sampling on the dependent vanablelve become the conventaous wiony, these are also are ‘Gant te ius wsing samples with lide or no variance of th {pnden variable. The mou abou is to explore the charaters= {eof Uependers vaciable about which relatively ate known. Fries sing sarap wih nar valves on he dependent eat {ele helps dita possible cues, something that may be as STaportane it longa a substantiating posible causes, oc even Ton dentiying apparent necessary conditions The social elences omuake public mostly postive findings bat ejecting plas ticenplanaton for abserved phenomena ean kelp simplify farther fereneh and help 0 foctson more mmporant explanatory relation Spe "The logic of not sampling on the dependent vasiable makes pood sense for guantative research, cope thar bas om fenresion, ii'my not be as relevent for the qualrative research that is Common in. comparative police (sce Collen, Mahoney and SEwwght, 2004). In comparcaive policy when cage selection It encrally mote purposive than ia other areas ofthe disipine, Enoosing pariclar coves for theoretical selevances more permis bles or even desirable. Even there sao variance onthe dependent ey hore may be poor theoretical reason for choosing ese: ‘hs wll as vary, oe good to have dependent variables that say alot. While thisssue ay tol be largely a part ofthe general {uestion of measurement (ose Chapter) fs thos question for {esearch design and case selection. I we’ want to develop sobst {cored statements that are menaingal serosa range OF eile ent types of cicumtances (and we certainly do) then ee Wise 10 fan the research with that wide stray of eae Researchers would lite to develop, for example, propositions thet work nov nly for the wealthy indastalsed world but aso for at lease the evely ‘ndatiatising countries (NICs ~ Singapore; South Korea, TatW2, Sind 30 on), and if possible Fr theless developed count “Atiempting to develop thsorses witha wide rangeot applicability is ‘ita ee pp. 11820}, bor we should begin withthe 959 20a34_ Strategies for Comparative Research i Political Science snd therefore be concerned in most research situations with select: tng as wide a range of cases as possible. The ‘as possible" in that entence then rises the ise of the judgment of the esearch the existing knowledge about variance with rent. Inthe end, there may be no realistic substiete for searcher's ood judgment "Minimizing error variance i ess of « concers for cate selection, Error variance is that portion of the variance observed im the dependent variable dha is function of randoms occurences an t¢rors in measurement. Its the slip of the pen of the data recorder, fo by the data entry technician, of sloppy fieldwork on the par af interviewer ora host of other occurrences that ean produce dita posts thar are just inaceurate. I addition, no teasurement Of 3 focial phenomenon is a perfect jadieator of that underly ie, so. certain amount of Inevitable. For qualitative research, 1 from faulty observations bythe researches, oF ‘of behaviors in differen cultural settings, or et lagy oF aga ‘whole range of things that ean produce inaccurate observations ‘Statistically eror variance is assumed to have @ team of 1or0, since iis random and hence is sssumned nor eo bias most satiate ‘manipulations. The comparative researcher eannar afford to be 50 sanguine about the effects of error Ifthe cases are selected inappro- priately, or the observations made by the researcher are faulty, there Js little or no way to recover from the errors. Any tesuts arising from research with suc a poor design and execution aelikely tobe sleaing, and thetefore in some ways worse chan no results at all ‘This potential for eeror simply argues for even greater attention the selection of eases, and'a very searching consideration of the comparisons that are planned at che begitaing of the research, Further, there need to be extremely careful weighing of the modes cof data collection to.ensure that they ate indeed compatible in the range of sociedes within which they are being applied, and that the researcher is capable of doing the research in those cosines Tinally, thee isthe question of controlling extraneous vaviance. “This is even morea concern for the comparative researcher, because this is systematic rather than random erzor. With extraneous vari lance, there are one or more vatiables that have a syst tionship with the dependent variable and perhaps also with Independent variables in an analysis. A researcher may Bind a ala- tionship (Stastically or less quantitatively) berween X and ¥. "The ‘The Logic of Comparison 35 sblemis thar there variable Z tha is systematically related co Pretend ¥, For example, we may find dhat democratic politcal pet de spend more money’ on social welfare than do nondemo- Pre eatnes The problem) that levels of economic development ce othe elated rar-both democracy and welfare spending GeSuhiee counties find iveasce co maintain democracy and also Weta money for social programs. Thus, assuming that democracy Mehuces the welfare state may be only partially correct fall Pesttetatonship should rather be ested in the presence of a meas- Ate ofthe level of economic development. ‘comparative eeearch, there are an almost inf ‘opportunities for extrancous variance to ereep into the analysis, evearehers tend a be Focusing upon whole counttes or on large Nitnational governments, all of which come as ‘data bundles Sehtaining huge number of variables and characeristics. John ‘Stuart Mil referred to the nation a8 a ‘permanent case’, which 10 Sinoune of aldiionat variables would permit one 10 eliminate ‘Whom scleting a country for islusion ita comparative invest Tine the tesemeher Wil seceive all of those factors at once. ‘Anempting to determine, therefore, whether the rue cause” of the Shserved state of the dependent variable was the independent vat bles or not ieextremely problemari, Statistically, se {ble tht elferntiates cares at the level of the cou {eopoitical entity) fs a8 likely to cause che outcome as any other evlable having the same effect In the conduct of comparative polities! rescareh, minimising andlor detecting extrancous variance Ba dihcult probiem, bue nota totally insurmountable one. “One ofthe fist defenses against extraneous variance is theory. AS Faure (1994: 313) points out, ‘comparative method is therefore tntely dependent tpom preexisting criteria of relevance such ts Sonceptsy propositions and theories, These can be tested by the tmethod but na discovered by the method." If there is yood theory inthe rescacch area then # should help the investigator desermmine ‘whether the right control variables ave being, considered for a sats ‘consi infor concept ‘seagchrs ave dependent upon existing conceptions ofthe politics! worl36 _ Strategies far Comparative Research in Palitieal Science “This reliance on theory may bea eateh-22 forthe research ~iF the theory is already so good, why do we have to do the addtional esearch? ‘cam the theory ever be rally tested if we assume ein the selection of cause and contval variables i ie? Thus if the research in question i to remain within the ‘normal science” mode (Kuhn, 1970}, and assume chat there is 3 reliable body of accepted and verified evcence anda paradij that an guide research, then existing theory te 8 good guide Ifthe esearch is more exploratory,or is intended to challenge the existing. theories, then the researcher is more alone inthe hostile world of fexteaneous variance. Again, the discovery of ‘antomalis” (Kuh {word} may lead to che questioning and then improvement of the theory. [A sccond possible defense against the problems of extrancous variance isthe use of tine series analysis, or sing subnational nits from within a single country. Neither af these approsche 0 aly fis is perfect prophylactic against extraneous variance, but each Aloes provide some limited protection. Using time reies analysis will do two useful chings im the detection and potential elimination ‘Of extrancont variance, One ec allow forthe detection of systen tie relationships other than those being tested through he analy: Sis. Suppose, for example, a researcher i using time series Fegeession and finds a significant corselation between the depend! tent and independent variables. That is very nize, but ies only the hreginning ofthe research exercise. There are a variety of statistical ‘methods to test whether the residuals (the difference between the Actual and the predicted values ofeach case) ol the repression analy "bare themselves systematically corelated nerosstime, an outcome Called autocortelation. If there i significant mutocorrelaion then this systematic vor in the felaionship eaa be siminated, ether through removing i€ by means of statistical manipulations, oF bY identifying che variables causing the relationship and inclading, them directly inthe analysis, ‘Examining residuals inthis manner is most commonly dome in me series analysis, but a similar logic could be uted t0 went systematic error in other types of regression analysis I the observa {dons can be ordered by any variable iwaddition tothose included in the analysis, and if the residuals are then examined and sre fund 0 hhave a discernible pattern, then theres an obvious danger of extra. ‘neous variance contaminating the finding. For example, consider & egrestion sty of German Lander tha elated per eapiva spending The Logie of © narison ¥7 cn education to income, The opus of that a iTcrvars and these might be ordered by a potentall in fst or eamply she, pscenge a Rom Whol in'the population: (Wieraky, 1974 Soper and Fens, $002), Of course, amore wreful statistical result could be Schieved chro mcluding the prenumed confounding varie as Share of the regression equation, aod determining, Mow It fefloums tclative to other explanatory vatiabler The ewe of Examining the residual directly, however that they may present STpwucin thet cam be diacerned visually quite easly, but ehich ‘eould be more dithcule to detect satsically. For example, & Krvilincor pesternanight not be ns enry to detect statically a8 Shh a simple plor of seid w whore inthe ime Series change in se auracorvlaion acces may suggest the source Sian atta : j help elimomte some problems of extrancous variance. Rs has bec Sigued above hese units wil tend to be more sini on a mamber aici ra denon han yl thc com usonnl sescach, Therefore, some of the potential sources 0 xtrancous variance will have been eiinaed by selection ofthat focus of research. The problems with this strategy sre, mom ‘thers, it selevance and the potential for 4 false sense of secur Firse many theoretical relaionships in which researchers Comparative politics interested will not be relevant in sob Stonal contest. For example, legitimacy i largely 4 question for Satonal goveroments not for subnational overmments. In mst governments, howeven, subnational units 4 proup may have nore legitimacy than the national government. The inereasing Uisdain for the federal government in the United States may reflect fone notableae. ‘also, ascuming that che problom of extrancous variance ix solved by sing subnational aie within «single county ea ll, the reseascher into false sense of security. There are afer ll Sificanteiferences among subnational its even in cownties {hat may appese tothe casual observer to be reasonably homoge tous pitialy~ for example, the diferences between Silly and Lombardy in aly. Therelore extraneous relationships may well be present in subnational date without the researcher bein, a8 seaive o them ts he or she might ins erose national analy, hd therefore unecessary etvor can creep into the analysis.