Seventh Edition

W. Phillips Shively
University of Minnesota

PEARSON

Upper Saddle River, NJ 07458

Library of Congress Cataloging-in-Publication Data Shively.W. Phillips, 1942The craft of political research / W. Phillips Shively. — 7th ed. p. cm. Includes bibliographical references and index. ISBN-13: 978-0-13-602948-9 (alk. paper) ISBN-10: 0-13-602948-5 (alk. paper) 1. Political science—Methodology. JA71.S45 2009 320.072—dc22 2007047170 Executive Editor: Dickson Musslewhite Editor-in-Chief: Yolanda de Rooy Associate Editor: Rob De George Editorial Assistant: Synamin Ballatt Senior Marketing Manager: Kate Mitchell Marketing Assistant: Jennifer Lang Managing Editor: Joanne Riker Manufacturing Buyer: Wanda Rockwell Creative Director: Christy Mahon Cover Design Director: Jayne Conte Cover Design: Jon Boylan Cover Illustration/Photo: Getty Images, Inc. Composition: Integra Full-Service Project Management: Integra Printer/Binder: Courier-Stoughton Credits and acknowledgments borrowed from other sources and reproduced, with permission, in this textbook appear on appropriate page within text. 2. Political science—Research. I. Title.

To

BARBARA

Copyright © 2009, 2005, 2 0 0 2 , 1 9 9 8 , 1 9 9 0 , 1 9 8 0 by Pearson Education, Inc., Upper Saddle River, New Jersey, 07458. All rights reserved. Printed in the United States of America. This publication is protected by Copyright and permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permission(s), write to: Rights and Permissions Department. Pearson Prentice Hall™ is a trademark of Pearson Education, Inc. Pearson® is a registered trademark of Pearson pic Prentice Hall® is a registered trademark of Pearson Education, Inc. Pearson Pearson Pearson Pearson Education Ltd. Education Singapore, Pte. Ltd. Education, Canada, Ltd. Education-Japan Pearson Pearson Pearson Pearson Education Australia PTY, Ltd. Education North Asia Ltd. Educacion de Mexico, S.A. de C.V. Education Malaysia, Pte. Ltd.

ISBN-13: ISBN-ID:

c

10 9 8 7 6 5 4 3 2 1 l7fl-D-13-bD5T4fl- 1 D-ig-bDSTMfl-S
c

L

Preface

xi 1
2

Doing Research
Social Research

Types of Political Research Research Mix 8

4

Evaluating Different Types of Research
E t h i c s of P o l i t i c a l R e s e a r c h 11

10

Political Theories and Research Topics
C a u s a l i t y and P o l i t i c a l T h e o r y 14 15

What Does Good Theory L o o k L i k e ?

Example of Elegant Research: Philip Converse To Quantify or Not
C h o i c e of a T o p i c 22

20

Engineering Research Theory-Oriented Research

22 23
23

D e v e l o p m e n t of a R e s e a r c h D e s i g n

Observations, Puzzles, and the Construction of Theories 25

C a s e Studies S e l e c t e d for the R e l a t i o n s h i p B e t w e e n the Independent and D e p e n d e n t V a r i a b l e s Further D i s c u s s i o n 110 109 Enrichment of the Level of Precision in Measurement Examples of Enrichment 67 72 Quantifiers and Nonquantifiers Again Further D i s c u s s i o n 73 8 74 Causal Thinking and Design of Research C a u s a l i t y : A n Interpretation 75 Introduction to Statistics: Measuring Relationships for Interval Data 111 Statistics 112 112 E l i m i n a t i o n o f A l t e r n a t i v e C a u s a l Interpretations 76 Importance of L e v e l s of Measurement W o r k i n g w i t h Interval D a t a 113 Summary 77 . H o w e v e r : S i n g l e .Contents Machiavellian Guide to Developing R e s e a r c h Topics 28 31 Contents A F e w B a s i c s of Research Design 78 ix Designs Without a Control Group Use of a Control Group 82 79 Further D i s c u s s i o n Importance of Dimensional Thinking E n g l i s h as a L a n g u a g e for R e s e a r c h 33 32 True Experiment 83 84 D e s i g n s for P o l i t i c a l R e s e a r c h Ordinary Language 34 37 Special Design Problem for Policy Analysis: Regression to the Mean 88 U s e o f V a r i e d D e s i g n s and M e a s u r e s 90 Proper U s e of Multidimensional Words Example of Dimensional 40 Analysis 38 Example of Varied Designs and Measures Conclusion 93 94 92 Further D i s c u s s i o n Problems of Measurement: Accuracy Reliability 45 41 46 Holding a Variable Constant Further D i s c u s s i o n 96 Reliability as a Characteristic of Concepts Testing the Reliability of a Measure Validity 48 47 7 Selection of Observations for Study Random Sampling 99 101 102 103 97 99 S a m p l i n g f r o m a Population of Potential O b s e r v a t i o n s Some Examples Checks for Validity 49 51 53 Quasi-random Sampling Purposive Sampling I m p a c t o f R a n d o m and N o n r a n d o m E r r o r s Importance of A c c u r a c y Further D i s c u s s i o n 56 54 Selection of Cases for Case Studies Censored Data 103 When Scholars Pick the Cases They 're Interested In 104 Problems of Measurement: Precision P r e c i s i o n in M e a s u r e s Precision in Measurement 58 62 57 When Nature Censors Data 106 S e l e c t i o n A l o n g the D e p e n d e n t V a r i a b l e : D o n ' t D o It! 106 108 S e l e c t i o n o f C a s e s for C a s e Studies ( A g a i n ) Innate Nature of Levels of Measurement The Sin of Wasting Information 64 63 A n o t h e r Strategy.

I must a l s o admit to a recent c h a n g e of heart (or. and it h a s been a thrill w h e n students have told me that they have benefited f r o m it. but I a l w a y s regretted s a c r i f i c i n g C o n v e r s e ' s article. Null Hypothesis Example: % 2 152 153 155 158 Independent Observations 161 162 Sampling Distribution Importance of'N Problem of It has f o l l o w e d me through the rest of my career so far. a return to old l o v e ) . A n d so I wrote this book. W h i l e the general p r i n c i p l e s of good argument and investigation don't c h a n g e . In teaching a n u m b e r of sections of Introduction to R e s e a r c h to undergraduates there. It is still the most xi . P u t n a m ' s b o o k a n d the resultant r e s e a r c h p r o g r a m on s o c i a l c a p i t a l are of c o u r s e splendid. I c h a n g e d m y e x a m p l e o f elegant r e s e a r c h f r o m P h i l i p C o n v e r s e ' s "Of Time and Partisan Stability" to R o b e r t P u t n a m ' s Making Democracy Work.Contents Regression Analysis Correlation Analysis 113 122 127 130 Correlation and Regression Compared Problem of Measurement 131 Error Further D i s c u s s i o n Introduction to Statistics: Further Topics on Measurement of Relationships 132 M e a s u r e s o f R e l a t i o n s h i p for O r d i n a l D a t a M e a s u r e s o f R e l a t i o n s h i p for N o m i n a l D a t a 132 136 Dichotomies and Regression Analysis L o g i t and Probit A n a l y s i s Multivariate A n a l y s i s Conclusion 146 141 139 136 Introduction to Statistics: Inference. I had f o u n d that the students benefited f r o m an introduction that e m p h a s i z e d the internal l o g i c of r e s e a r c h methods and the c o l l e c t i v e . I have updated a n u m 163 ber of e x a m p l e s . In this seventh edition. or How to Gamble on Your Research 148 L o g i c of Measuring Significance E x a m p l e o f Statistical I n f e r e n c e Hypothesis Testing 151 149 150 I wrote this little b o o k in 1970. I have also c h a n g e d significantly the position I take on the question of selecting c a s e s for a n a l y s i s . I h a v e i m p r o v e d on my treatment of the use of n o m i n a l variables in r e g r e s s i o n a n a l y s i s . and I have added a n e w d i s c u s s i o n of the p r o b l e m of c o m p a r i n g U s e s and L i m i t a t i o n s of Statistical T e s t s Conclusion 164 164 units in r e g r e s s i o n a n a l y s i s . w h e n I w a s an assistant p r o f e s s o r at Y a l e U n i v e r s i t y . I c o u l d not find a b o o k that presented things in this w a y at a sufficiently elementary l e v e l to be r e a d i l y a c c e s s i b l e by undergraduates. It has a l w a y s s e e m e d to me that it fills a needed n i c h e . Further D i s c u s s i o n Where Do Theories Come From? Selected Bibliography Index 174 170 166 R e g u l a r users of this text m a y r e c a l l that a c o u p l e of editions a g o . perhaps. I have m a d e a n u m b e r of additions and deletions over the last c o u p l e of editions to Significance Test: A l w a y s N e c e s s a r y ? P o l l i n g and S i g n i f i c a n c e T e s t s 163 reflect n e w p o s s i b i l i t i e s in technique. under prodding f r o m r e v i e w e r s . cooperative nature of the r e s e a r c h p r o c e s s . and has g i v e n me enormous p l e a s u r e . I am pleased that it still s e e m s to be w o r k i n g for them.

this is a book for w h i c h I have great affection." and i n c l u d e w i t h it the s a m e e x p l i c a t i o n of M a r k o v c h a i n s that I u s e d in those earlier editions. h i s account of their w o r k . and P a t r i c k J a m e s . indirectly. As y o u c a n no doubt tell f r o m the tone of this preface. The first afternoon following the discovery that A . even though they lacked the casual sense of understatement known to be the correct way to behave in Cambridge. the approval of others. W a t s o n d e s c r i b e s his and F r a n c i s C r i c k ' s s e a r c h for the structure of the D N A m o l e c u l e . It is m o r e gripping than most mystery novels. e a c h of these anxieties m a y a l s o be present for 'Reprinted with permission from Watson. for professors or for students w h o c a n involve t h e m s e l v e s in a l o n g . The Double Helix. ACKNOWLEDGMENTS T h a n k s to the f o l l o w i n g r e v i e w e r s for their helpful suggestions and c o m m e n t s : Terri S u s a n F i n e .xii Preface Chapter I beautiful p i e c e of p o l i t i c a l s c i e n c e r e s e a r c h I know. Garrett G l a s g o w . F i r s t of a l l . Francis's preoccupation with D N A quickly became full-time. U n i v e r s i t y o f C a l i f o r n i a a t S a n t a B a r b a r a . c o u r s e papers are tied to all sorts of r e w a r d s and p u n i s h m e n t s — y o u r future e a r n i n g s . B u t for people w h o c a n carry on r e s e a r c h in a m o r e r e l a x e d w a y . worriedly look at the cardboard models. (Watson.r a n g e project. Constantly he would pop up from his chair. to lodge on the paper. 1 . 1968). A l l o f the anxiety a s s o c i a t e d w i t h these vulnerabilities c o m e s . look satisfied and tell me how important our work was. W. Phillips Shively S c h o l a r l y r e s e a r c h is e x c i t i n g and is f u n to do. he went back to his thesis measurements. fiddle with other combinations. S o m e students. Y e t this is probably a l e s s e r c a u s e for frustration in student r e s e a r c h . gives a good picture of the excitement of r e s e a r c h .T and G . A n d so I returned in the sixth e d i t i o n — a n d continue i n this seventh e d i t i o n — t o " O f T i m e and P a r t i s a n Stability. James D. p. but his effort was ineffectual. A l t h o u g h r e s e a r c h c a n be e x c i t i n g in this w a y . The Double Helix (New York: Atheneum.C base pairs had similar shapes. and then. U n i v e r s i t y o f C e n t r a l F l o r i d a . r e s e a r c h m a y be a source of fascination and great satisfaction. 198) 1 T h i s i s the w a y J a m e s D . caught in the grind of daily and term a s s i g n m e n t s . the period of momentary uncertainty over. 1968. m a y not see it this way. the s a d fact is that w r i t i n g papers for c o u r s e s is too often something of a drag. and s o o n . U n i v e r s i t y o f Southern C a l i f o r n i a . I enjoyed Francis's words. A f t e r a l l . I hope y o u w i l l e n j o y it as m u c h as I have e n j o y e d it.

there w o u l d be four distinct political positions in the country: the C a t h o l i c w o r k e r position. by both teacher and student. we n e e d only think of a single pattern w i t h s o m e variations. the electoral l a w forces people of d i v e r s e positions to c o n s o l i d a t e into two large political parties so as SOCIAL RESEARCH to gain an electoral advantage. S o m e t h i n g i s a l w a y s lost w h e n w e s i m p l i f y reality i n this way. and w h e r e the groups of people i n v o l v e d in these taneously with a great number of idiosyncratic party systems. c o n f l i c t s o v e r l a p . s o m e b i l l s are p a s s e d by C o n g r e s s . Instead of h a v i n g to think about a large n u m b e r of disparate h a p p e n i n g s . and he c o u l d have then incorporated that explanation into the original theory to produce a larger theory. To have any hope of controlling w h a t h a p p e n s . s u c h as whether any one of the parties w a s revolutionary. as a practice r u n . there w i l l be m o r e than two distinct p o l i t i c a l positions in the c o u n try. that a student w i l l learn f r o m d o i n g the thing w r o n g . and the n o n . going through the motions of s c h o l a r s h i p . D u v e r g e r no longer h a d to concern h i m s e l f s i m u l - S o c i a l r e s e a r c h is an attempt by s o c i a l scientists to develop and sharpen theories that give us a handle on the universe. H i s i d e a w a s that w h e r e there is m o r e than one sort of political c o n f l i c t going on s i m u l t a n e o u s l y in a country. Students must have the c h a n c e to learn f r o m their o w n m i s t a k e s . S u c h a p r o c e s s of c o n s o l i d a t i o n l o g i c a l l y w o u l d c u l m i n a t e in a two-party s y s t e m . and of h o w important i n v e n t i v e n e s s . I a l s o hope to m a k e y o u a w a r e of w h a t a c h a l l e n g i n g g a m e it c a n be. we m u s t understand w h y these things h a p p e n . in others there are not. w h i l e s o m e o f the n o n . the country w i l l develop a two-party s y s t e m . the n o n . pp. To s u m m a r i z e the theory: A country w i l l develop a two-party s y s t e m (1) if there are only two distinct p o l i t i c a l positions in the country. H a v i n g formulated this theory. if these groups overlapped s o that s o m e o f the C a t h o l i c s w e r e w o r k e r s and s o m e w e r e m i d d l e c l a s s . o t h e r w i s e . U s u a l l y . of the f e e l i n g of creating s o m e t h i n g that no one ever s a w before. W i t h o u t theories. originality. e c o n o m i c development p r o g r a m s s u c c e e d i n s o m e c o u n t r i e s . we a l w a y s have to strike a b a l a n c e b e t w e e n the s i m p l i c i t y of a theory and the n u m b e r of exceptions we are w i l l i n g to tolerate. e s p e c i a l l y w i t h other papers c o m p e t i n g for attention. and b o l d n e s s are to g o o d r e s e a r c h . w i t h v a r y i n g n u m b e r s and types o f parties present at different times in their histories. We do not r e a l l y h a v e any c h o i c e .2 Chapter 1 Doing Research 3 p r o f e s s i o n a l s c h o l a r s . As y o u c a n see. of votes in a district.C a t h o l i c m i d d l e . too. A n d t o have any h o p e o f understanding w h y they h a p p e n . there u s u a l l y is a feeling on both sides that this is " j u s t a student p a p e r " — t h a t it d o e s n ' t r e a l l y matter h o w g o o d it i s . D u v e r g e r thought that this tendency c o u l d be s h o r t .C a t h o l i c w o r k e r p o s i t i o n . It w a s necessary for h i m to do this in order to keep the theory simple and to the point. Instead.c i r c u i t e d if the electoral s y s t e m w e r e set up in s u c h a w a y as to p e n a l i z e s m a l l p a r t i e s — b y requiring that a candidate have a majority. B u t I c a n introduce y o u to s o m e selected p r o b l e m s y o u s h o u l d be a w a r e of if y o u want to do good r e s e a r c h y o u r s e l f or to evaluate the w o r k of others. H o w e v e r . and (2) if the electoral s y s t e m of the country does not p e n a l i z e s m a l l parties. T h e r e is probably no w a y out of this d i l e m m a . but fail i n others. in s o m e elections there are m a j o r shifts. or h o w m a n y of the parties h a d any c h a n c e of getting a majority of the votes. for instance. In a b o o k s u c h as this. the C a t h o l i c m i d d l e . D u v e r g e r d e v i s e d the theory that (1) if s o c i a l c o n f l i c t s o v e r l a p . for instance. a c o n f l i c t b e t w e e n w o r k e r s and the m i d d l e c l a s s might o c c u r at the s a m e time as a c o n f l i c t b e t w e e n C a t h o l i c s and n o n . W h y w a s this s o ? D u v e r g e r might have cast around to find an explanation for the exception to his theory. but this attitude toward the r e s e a r c h w o r k cheats them of the p l e a s u r e and excitement that r e s e a r c h c a n b r i n g . I cannot give y o u the d r a m a and excitement of o r i g i n a l r e s e a r c h . w i t h one party c o r r e s p o n d i n g to e a c h distinct p o s i t i o n . A theory takes a set of s i m i l a r things that h a p p e n — s a y . s o m e t i m e s w a r c o m e s . he c h o s e to accept them as accidents. for e x a m p l e . A n d e v e n w h e n adequate time is a l l o w e d .c l a s s position. As theorists. that a country for w h i c h his theory had predicted a two-party s y s t e m developed a multiparty s y s t e m instead. He needed to think only about a single developmental p r o c e s s . B y restricting his attention to the n u m b e r of parties c o m p e t i n g in the s y s t e m . as one for w h o m r e s e a r c h is v e r y e x c i t i n g . the country w i l l develop a multiparty s y s t e m . . D u v e r g e r h a d to forget about m a n y other potentially interesting things. it a l s o generally requires that we both narrow the range of reality we l o o k at and o v e r s i m p l i f y e v e n the portion of reality that falls w i t h i n that n a r r o w e d range. or (2) if despite the p r e s e n c e of m o r e than two distinct p o l i t i c a l p o s i t i o n s .c l a s s p o s i t i o n . it might have grown as c o m p l e x as the reality that it sought to simplify. T h i s requirement w o u l d f o r c e s o m e of the distinct groups to c o m p r o m i s e their positions and merge into larger parties that w o u l d h a v e a better c h a n c e of w i n n i n g e l e c t i o n s . A more important r e a s o n for the student's l a c k of e n t h u s i a s m is the s i m p l e fact that a paper is g e n e r a l l y regarded. others are not. of w h i c h t i l those party systems were examples. that D u v e r g e r restricted h i m s e l f in more than j u s t his c h o i c e of a theme: He c h o s e deliberately to play d o w n exceptions to his theory. B e c a u s e the theory s i m p l i f i e s reality for u s . O t h e r w i s e . S u p p o s e .C a t h o l i c s w e r e w o r k e r s and s o m e w e r e m i d d l e c l a s s . rather than a plurality. s o m e t i m e s it does not. R e a l i t y unrefined by theory is too c h a o t i c for us to absorb. w h e n faced with exceptions s u c h as these. 2 0 6 . F o r e x a m p l e . S o c i a l scientists c a r r y out this s i m p l i f i c a t i o n by d e v e l o p i n g theories. T h e appropriate n u m b e r o f parties w o u l d then tend to r i s e . F o r e x a m p l e . I c a n o n l y give my o w n testimony. s c o r e s o f countries w e r e i n v o l v e d . i n his book o n political parties. T h e n . T h e initial reality w a s c h a o t i c .C a t h o l i c s . although these exceptions might have provided interesting additional information. w e must s i m p l i f y our perceptions o f reality.2 8 0 ) . there are costs in setting up a theory. S o m e people vote and others do not. M a u r i c e D u v e r g e r w a s c o n c e r n e d w i t h the question of w h y s o m e countries develop two-party s y s t e m s and others develop multiparty s y s t e m s ( 1 9 6 3 . not e n o u g h time is a l l o w e d for the student to think l o n g and s e r i o u s l y about the subject. we are f a c e d w i t h the unreadable c h a o s of reality. Note. the development of party s y s t e m s in d e m o c r a c i e s — a n d finds a c o m m o n pattern a m o n g them that a l l o w s us to treat e a c h of these different o c c u r r e n c e s as a repeated e x a m p l e of the s a m e thing.

a n d d e s i g n i n g methods o f riot c o n t r o l . H o w e v e r . that i s . a n d she w o u l d not even k n o w w h i c h w a y to m o v e . or a s s u m p t i o n s . (2) it m a y be rational for the voter to r e m a i n uninformed. students of society are trapped. T h i s type of research is r e a l l y not as flippant as the c h o i c e of the term recreational might m a k e it s e e m . It t y p i c a l l y takes certain p o l i t i c a l facts as given and c o m b i n e s t h e m w i t h m o r a l arguments to p r e s c r i b e p o l i t i c a l action. they posit facts as e m p i r i c a l conditions rather than as the foundation for m o r a l arguments. to add to our general understanding of politics. but this c a r r i e s the u n p l e a s ant implication that applied r e s e a r c h is either i m p u r e or of l i m i t e d v a l u e . or interpreting. P o l i t i c a l scientists pursue this type of r e s e a r c h for the t w i n p l e a s u r e s of e x e r c i s i n g their m i n d s a n d i n c r e a s i n g their understanding of things. T h e i r e n d goal is to develop reasonably b r o a d a n d general theories b a s e d on a s m a l l n u m b e r of agreed-upon assumptions. a n d others. Thus. may be thought of as applied versus basic research. A good e x a m p l e T A B L E 1-1 Types of Political R e s e a r c h Applied Recreational Formal theory Theory-oriented research theory f r o m a set of a s s u m p t i o n s s u c h as: (1) voters a n d parties b e h a v e rationally. or it may be carried on largely for its own sake. in w h i c h he urges the adoption of d e m o c r a t i c representative g o v e r n m e n t b e c a u s e (1) the c h i e f end of g o v e r n m e n t s h o u l d be to facilitate the d e v e l o p m e n t in e a c h citizen of h i s full potential ( m o r a l a r g u m e n t ) . At the other e n d of the c o n t i n u u m f r o m applied r e s e a r c h is recreational 1. we must look largely outside a c a d e m i c life to such sources as the R A N D C o r p o r a t i o n a n d the Weekly Standard. and (3) democratic governments tend to redistribute income. engineering research is geared to s o l v i n g prob- l e m s . and r e s e a r c h is carried on under that n a m e . not the active formulation of normative arguments. A n d they distinctively operate by d e r i v i n g further i m p l i c a t i o n s of the posited conditions by p r e c i s e l o g i c a l and m a t h e m a t i c a l operations. P o l i t i c a l theory. S o c i a l scientists m e r e l y interpret reality in a m o r e s y s t e m a t i c a n d explicit way. T h e y are r e d u c e d to m e r e l y observing events. c o n sists o f a l l theories that have been d e v e l o p e d to e x p l a i n political behavior. P a u l K r u g m a n . w h e r e a s Nonempirical Empirical Normative philosophy Engineering research in a multiparty s y s t e m . Downs builds a wide-ranging arguments about what should be in p o l i t i c s . S o m e e x a m p l e s w o u l d be m e a s u r i n g the effects of various reapportionment m e t h o d s . a subset of s o c i a l theory. R e s e a r c h m a y be c l a s s i f i e d a c c o r d i n g to these two criteria. for this is r e s e a r c h c a r r i e d on for its o w n s a k e . T h i s m e a n s that it is not intended so m u c h to develop p o l i t i c a l theory as to u s e what political theory tells us about society and p o l i t i c s as a b a s i s for m a k i n g p o l i t i c a l d e c i s i o n s . L i k e normative p h i l o s o p h e r s . I m a g i n e a p h y s i c i s t — o r a fruit p i c k e r for that m a t t e r — operating in the a b s e n c e of theory." Normative philosophy is taught extensively. is the m o s t r e c e n t l y by the extent to which it seeks to provide new factual information (empirical versus nonempirical). It is a p p l i e d r e s e a r c h . they w i l l not. is the s u m total of a l l those theories d e v e l o p e d by s o c i a l scientists to e x p l a i n h u m a n behavior. This distinction. f o r m a l theorists posit certain facts about p o l i t i c s . . our environment. but generally this m e a n s the history of normative philosophy a n d its development. A y n R a n d . or it may be intended to provide new ways of looking at old facts. its goal is p r o b l e m s o l v i n g . S o c i a l theory. S o m e of the predictions generated f r o m h i s theory are (1) in a two-party s y s t e m . Normative philosophy c o n s i s t s of introduced f o r m o f political r e s e a r c h . P r o b a b l y the oldest f o r m of p o l i t i c a l r e s e a r c h . it is "recreation. w h a t s o c i a l scientists do in developing theories is not different f r o m w h a t we n o r m a l l y do every day in perceiving. T h e i r c o n c e r n is to take the posited facts. then. a n d derive theories f r o m them. (2) p o l i t i c a l c o n f l i c t o c c u r s on o n l y one i s s u e at a t i m e . it is c o n c e r n e d w i t h a s c e r t a i n i n g the facts needed to s o l v e p o l i t i c a l p r o b l e m s . F o r both forms of applied r e s e a r c h . A l l she c o u l d do if she s a w an apple falling f r o m a tree w o u l d be to d u c k ." Formal theory.4 Chapter 1 is John Stuart Mill's argument in Considerations Doing Research on Representative 5 Government. T h e two m a i n w a y s b y w h i c h t o d i s t i n g u i s h o n e p i e c e o f r e s e a r c h f r o m another are: political s c i e n c e . based on the uses for which research is designed. by giving the people responsibility. T h e s e two forms of applied r e s e a r c h exist in s o m e estrangement f r o m a c a d e m i c Types of Political R e s e a r c h T h e w a y a particular p o l i t i c a l scientist c o n d u c t s r e s e a r c h w i l l depend both o n the u s e s that she v i s u a l i z e s for the project a n d on the w a y she m a r s h a l s e v i d e n c e . In a h i g h s e n s e of the w o r d . i t i n c l u d e s a m o n g its practitioners P l a t o . w i l l do this (factual a s s u m p t i o n ) . trying to d e s i g n a d i p l o m a t i c strategy to effect d i s a r m a m e n t p r o c e d u r e s . Like normative philosophy. its stance is e m p i r i c a l . but in contrast to normative p h i l o s o p h e r s . A g l a n c e at T a b l e 1-1 s h o w s us the four types of p o l i t i c a l r e s e a r c h b a s e d on different combinations of these two dimensions. Research may be directed toward providing the answer to a particular problem. A c t u a l l y . a n d (2) d e m o c r a t i c government. It is u s u a l l y c a l l e d " p u r e " or " b a s i c " r e s e a r c h . l a r g e l y a p o s t . Political engineering is a thriving industry a n d m a n y courses relevant to it are taught in political s c i e n c e departments.W o r l d W a r II p h e n o m e n o n . A g o o d e x a m p l e of f o r m a l t h e o r y — i n d e e d . but research in it is often relegated to a separate institute or " s c h o o l of public policy. It is a l s o n o n e m p i r i c a l in that it does not c o n s i s t p r i m a r i l y of investigating matters of fact. a w o r k by w h i c h m a n y w o u l d date the e m e r g e n c e of f o r m a l theory as a distinct field in p o l i t i c a l s c i e n c e — i s A n t h o n y Downs' An Economic Theory of Democracy (1957). to i m p r o v e p o l i t i c a l theory. Without theories. Research may also be intended primarily to discover new facts. political research can be characterized r e s e a r c h . G e o r g e W i l l . K a r l M a r x . a n d (3) p o l i t i c a l events are not perfectly predictable. without c o m m e n t . 2. parties w i l l tend to agree very c l o s e l y on i s s u e s .

6 Chapter 1 Doing Research 7 ( O f c o u r s e . participation in the group is u s u a l l y nonrational. a f o r m a l theory is u s e d to construct a set of conditions f r o m w h i c h the thing we w i s h to e x p l a i n w o u l d have l o g i c a l l y f l o w e d . if e a c h of us contributes $ 100 to the c a u s e . T h e rational c h o i c e a s s u m p t i o n pointed O l s o n to a question no one h a d a s k e d before. what they l o g i c a l l y i m p l y — i t is a l s o u s e f u l for d e v e l o p i n g and a n a l y z i n g strategies for p o l i t i c a l action. H o w e v e r . it is irrational for t h e m to do s o .3 . my i n d i v i d u a l contribution to an interest group p u r s u i n g preservation w i l l not m a k e a m e a s u r a b l e difference. To the organization this a m o u n t w o u l d be trivially s m a l l . B e f o r e his b o o k . and at the end they m a y c o m p a r e their final m o d e l s w i t h this body of k n o w l e d g e . then ( c ) c a n we d e d u c e what sort of taxes in the context of the a s s u m p t i o n s of (b) w o u l d best a c h i e v e ( a ) ? L i k e normative p h i l o s o p h y . one m u s t r e c o g n i z e that excerpts s u c h as these do e v e n m o r e than the u s u a l v i o l e n c e to a r i c h net of theories. O l s o n r e a s o n e d that there w a s nothing natural about organization at a l l . and (b) if we a s s u m e that governmental investment is inefficient and that i n d i v i d u a l taxpayers act so as to m a x i m i z e their i n c o m e .e n c o m p a s s i n g . T h e virtue of O l s o n ' s theory in this c a s e is that instead of v i e w i n g s u c h contributions as " n a t u r a l " and therefore i gnor i ng t h e m . However. If I am a p e r s o n c o n c e r n e d w i t h h i s t o r i c preservation. W h a t is rational. c a n we d e v i s e a set of r e a s o n a b l y true a s s u m p t i o n s and an action w h i c h . . s c h o l a r s had a s s u m e d that w h e n For example. he c o n c l u d e d . Duverger (1963) assumed this in the theory I described on pp. B u t b e c a u s e f o r m a l theory consists of taking a set of a s s u m p t i o n s and w o r k i n g out w h e r e they l e a d — t h a t i s . as in the e x a m p l e above. b u s i n e s s e s . S u c h explanatory f o r m a l theories are then often tested e m p i r i c a l l y through theory-oriented r e s e a r c h . O l s o n laid out several conditions under w i i i c h organizations might nonetheless arise. the m a r v e l s h o u l d be that any interest or gani zations exist at a l l . p r o f e s s i o n s . is to be a free rider. he w o u l d go b a c k and r e e x a m i n e them. ( I n e c o n o m i c s the v a l u e d object is gene r a l l y taken to be m o n e y . w a s organized ar ound the expectation that most of the time.) It is important to e m p h a s i z e that this sort o f w o r k i s a l m o s t s o l e l y a n e x e r c i s e i n deduction. the large store cannot get its v a l u e d g o o d u n l e s s it takes the lead in setting up the a s s o c i a t i o n . O l s o n pointed out.t a x p r o p o s a l s are a g o o d e x a m p l e : T h e y originated in argument of the f o l l o w i n g f o r m : (a) I f w e want t o m a x i m i z e investment and e c o n o m i c growth. If y o u j o i n K X X X . T h e m a i n u s e of f o r m a l theory. in a w i d e array of settings O l s o n ' s theory predicts behavior rather w e l l . k n o w s that a D o w n t o w n M e r c h a n t s ' A s s o c i a t i o n cannot function i f i t does not j o i n and contribute. 0 0 0 people around the country w h o share my interest. the difference if I do or do not contribute is a budget of $ 2 9 . but to me $ 1 0 0 m a k e s a real difference. and in fact one function of theory m a y be that it highlights exceptions for c l o s e r e x a m i n a t i o n . T h e reader must then either accept the surprising conclusion or reexamine the assumptions that had seemed plausible. U n d e r these c i r c u m s t a n c e s . groups w i t h special c o n c e r n s s u c h as historic p r e s e r v a t i o n — p o l i t i c a l organizations c o u l d be expected to emerge naturally to represent those i n t e r e s t s . formal theory provides insights by logical argument. and a l l o w e d h i m to stand r e c e i v e d w i s d o m o n its h e a d . F l a t . w h i l e m i n i m i z i n g the c o s t expended in a c h i e v i n g it. w i l l l o g i c a l l y lead to XI F o r m a l theory is u s e d in this w a y . B a s e d on the rational c h o i c e a s s u m p t i o n . for e x a m p l e . not by a direct examination of political facts. L e t us say there are 3 0 0 . 0 0 0 . we are f o r c e d to treat the contributions as a p u z z l e requiring further investigation. F r o m the standpoint of any i n d i v i d u a l in a group w i t h a shared object. P r e s u m a b l y . B u t they are not t h e m s e l v e s c o n c e r n e d w i t h turning up n e w factual i n f o r m a t i o n . G o o d work in formal theory w i l l take a set of seemingly reasonable assumptions and w i l l show by logical deduction that those assumptions lead inescapably to c o n clusions that surprise the reader. we c a n u s e f o r m a l theory to construct a n a l y s e s of the f o l l o w i n g f o r m : If we want to a c h i e v e X. is not in that situation. 0 0 0 . T h u s . S o m e t i m e s the object m a y e v e n be left u n s p e c i f i e d in the theory. w h i c h is not rational. instead. a great deal of f o r m a l theory in political s c i e n c e has b a s e d itself on the e c o n o m i s t s ' c o r e a s s u m p t i o n of interests existed i n s o c i e t y — r a c i a l m i n o r i t i e s . m o s t of s o c i e t y ' s interests w o u l d be actively o r g a n i z e d . T h e excruciating efforts of public television stations to get their viewers to j o i n rather than be free riders ("Please! O n l y one in ten of our viewers is a member. T h e B i j o u x T e e . we c a n count on an organization b e i n g set up. in political s c i e n c e it m a y be m o n e y — a s in theories of w h y and h o w c o m m u n i t i e s s e e k pork-barrel s p e n d i n g — b u t theories m a y also posit that the v a l u e d object is a nonmonetary p o l i c y s u c h as abortion. s i n c e every potential m e m b e r of s u c h an organization is in this s a m e situation.S h i r t S h o p on the corner. If I contribute. F o l l o w i n g f r o m D o w n s . T h e largest department store in town. or p o l i t i c a l p o w e r itself. O n e s u c h condition is that one potential m e m b e r might h a v e s u c h large resources that she k n o w s no organization is p o s s i b l e without her participation. A l l o f the c o n c l u s i o n s derive l o g i c a l l y f r o m a limited set of e x p l i c i t a s s u m p t i o n s . as O l s o n has p r o v e d . R e m e m b e r that the rational c h o i c e a s s u m p t i o n states that i n d i v i d u a l s choose their actions in order to m a x i m i z e a v a l u e d object. the 2 pluralist school. We k n o w that m a n y people do contribute to political organizations even though. In the next chapter y o u w i l l see that it m a y also help to explain w h y s m a l l nations typically do not pull their "fair" weight in international alliances. We s h o u l d thus expect to see a w i d e range of parties and interest groups engaged in p o l i t i c s . O l s o n wrote o n the v e r y b a s i c question o f 2 political organization in society.T V today we w i l l send y o u this beautiful coffee m u g ! " ) bears testimony to the power of O l s o n ' s logic. D o w n s ' p u r p o s e in this is s i m p l y to see w h e r e the a s s u m p t i o n s he started w i t h w i l l lead h i m . A w h o l e s c h o o l of political s c i e n c e . rational choice: the a s s u m p t i o n that i n d i v i d u a l s c h o o s e their actions in order to m a x i m i z e s o m e v a l u e d object. if the a s s u m p t i o n s p r o d u c e d an untenable result. I w i l l h a v e e x p e n d e d a significant cost without getting any m o r e of my v a l u e d good. I k n o w that u n l e s s I have v e r y u n u s u a l r e s o u r c e s . and m i n i m i z e the cost e x p e n d e d in a c h i e v i n g it. 9 0 0 v e r s u s $ 3 0 . f o r m a l theory interacts w i t h e m p i r i c a l r e s e a r c h . or for v a r i o u s w a y s to arrange taxes so as to get the o u t c o m e s we want. b e c a u s e rationally. No theory c a n ever be a l l . to argue for v a r i o u s w a y s to set up e l e c t i o n s . 2 . F o r m a l theorists u s u a l l y try to start w i t h a s s u m p t i o n s that are in a c c o r d w i t h existing k n o w l e d g e about p o l i t i c s . T h a t i s . for i n s t a n c e . 9 9 9 . though. in the context of those a s s u m p t i o n s . is explanation. h o w e v e r . and let all those other people m a k e the contributions.) A good e x a m p l e of f o r m a l theory that illustrates the rational c h o i c e a s s u m p tion is M a n c u r O l s o n ' s The Logic of Collective Action ( 1 9 6 5 ) .

I n d e e d . this r e s e a r c h deals w i t h them to develop n e w p o l i t i c a l theories or to c h a n g e or c o n f i r m old ones. and those w h o w e r e promoted felt h o n o r e d . I n this r e s p e c t n o r m a t i v e p h i l o s o p h y differs f r o m the e m p i r i c a l types o f p o l i t i c a l r e s e a r c h . T h e normative p h i l o s o p h e r m u s t be free to i m a g i n e realities that have never existed before. promotion w a s sufficiently infrequent that not being promoted w a s seen as the n o r m . let us l o o k at a c a s e in w h i c h r e s e a r c h e r s w o r k i n g on a p r i m a r i l y e n g i neering project f o u n d they h a d to develop a theory to m a k e s e n s e out of their w o r k . u s i n g K a r l M a r x ' s w o r k as an e x a m p l e . t o c o n d u c t a . l i k e M a r x . they often n e e d to a s s u m e facts that cannot p o s s i b l y be tested against reality. any s p e c i f i c p i e c e of w o r k is a m i x of m o r e than one of the types. facts are sought out if they are n e e d e d to s o l v e a p r o b l e m . S u c h a c t i v i t y w i l l . we s h o u l d strive to c h a n g e the state and the e c o n o m y in w h i c h w i l l eliminate the b a d effects.o f f a p e r s o n is objectively. p a r a d o x i c a l l y . T h e y c o n c l u d e d by e x p l o r i n g the i m p l i c a t i o n s of this finding. and those w h o w e r e p r o m o t e d d i d not feel particularly h o n o r e d . a m i x is so m u c h the u s u a l situation that w h e n I tried to m a k e a r o u g h h e a d count of the f r e q u e n c y of the P r a c t i c a l l y no r e s e a r c h is a pure e x a m p l e of any of the types I h a v e presented here. o f c o u r s e . and these. T h i s sort of paradox occurred a number of times in their study. w h o w e r e l e s s l i k e l y to be p r o m o t e d . but of whether her c o n d i t i o n c o m p a r e s favorably or unfavorably w i t h a standard that she p e r c e i v e s as n o r m a l . A c c o r d i n g l y . F o r instance. T h e fact that s o m a n y air c o r p s m e n w e r e N C O s apparently m a d e the c o r p s m e n feel that p r o m o t i o n w a s the n o r m a l thing. T h e s e are abstract distinctions. o f c o u r s e . T h e distinction is an important one. O f Stouffer's sample o f M P s . cannot b e "tested. T h e y e x a m i n e d c h a n g e s i n military e x p e n diture by m a j o r p o w e r s f r o m 1816 to 1976 to see w h e t h e r i n c r e a s e s tended to be f o l l o w e d b y i n v o l v e m e n t i n war. N o matter h o w they adjusted t h i n g s — l o o k i n g for d e l a y e d r e a c t i o n s . the air c o r p s m e n . A n o t h e r g o o d e x a m p l e is a test by D i e h l and K i n g s t o n ( 1 9 8 7 ) of the theory that arms buildups l e a d t o m i l i t a r y confrontations. and the M P s . all U t o p i a n dreams w o u l d h a v e to be thrown out. A s a s e c o n d e x a m p l e o f the w a y i n w h i c h types o f r e s e a r c h are m i x e d i n any one w o r k . on the other h a n d . I t i s c h a r a c t e r i s t i c o f n o r m a t i v e p h i l o s o p h y . M o r e important. L i k e p o l i t i c a l e n g i n e e r i n g . theory-oriented research. A l t h o u g h one m e t h o d w i l l u s u a l l y predominate. that the r e s e a r c h e r n e e d not f e e l required to p r o d u c e the f u l l factual b a s i s for h i s argument. it is c o n c e r n e d w i t h d i s c o v e r i n g facts about p o l i t i c s . I n s t e a d ." I f normative p h i l o s o p h e r s w e r e h e l d to the s a m e standards of factual e v i d e n c e as e m p i r i c a l r e s e a r c h e r s . felt that c h a n c e s for p r o m o t i o n in the A r m y w e r e poor.8 Chapter 1 Doing Research 9 A l t h o u g h f o r m a l theory is the fastest g r o w i n g type of political r e s e a r c h . G e n e r a l l y . there w i l l a l m o s t a l w a y s b e s o m e interaction b e t w e e n the different types i n any g i v e n w o r k . A m o n g the M P s . B u t u n l i k e e n g i n e e r i n g . the m o s t important activity in this r e s e a r c h is the development of theories l i n k i n g o b s e r v e d facts about politics. c o m p a r e d with 47 percent of the air c o r p s m e n . A group of s o c i o l o g i s t s l e d by S a m u e l Stouffer w a s e m p l o y e d by the A r m y to study the m o r a l e o f A m e r i c a n soldiers during W o r l d W a r I I (Stouffer and others. a s M i l l w a s . Research Mix with ability h a d a good chance to advance in the A r m y . l o o k i n g only a t " a r m s r a c e s " i n w h i c h two rivals s i m u l t a n e o u s l y i n c r e a s e d their military f o r c e s . In e n g i n e e r i n g . 2 4 percent were n o n c o m m i s sioned officers ( N C O s ) . h e m a y w i s h . H i s argument takes the s a m e general f o r m as that in M i l l ' s e s s a y on representative government: " B e c a u s e aspects of the h u m a n condition today are w a y s to p r o d u c e these ways. T w o e x a m p l e s m a y help illustrate this point. F i r s t . A researcher m a y be relatively more w i l l i n g t o a s s u m e these facts f r o m g e n e r a l e x p e r i e n c e a n d / o r f r o m the r e s e a r c h o f others. the fact that normative p h i l o s o p h e r s are not required to p r o v i d e e v i d e n c e for all their a s s u m p t i o n s l e a v e s t h e m free to devote m o r e energy to other parts of the r e s e a r c h task. T h o s e w h o w e r e not p r o m o t e d w e r e d i s a p pointed. w h o w e r e m o r e l i k e l y to be p r o m o t e d . M P s were objectively less likely to be promoted than were members o f the A r m y A i r C o r p s . the M P s w e r e m u c h more likely than the air c o r p s m e n to think that soldiers e x p a n d i n g our k n o w l e d g e of w h a t happens in p o l i t i c s and of w h y it happens as it does. i n v o l v e h i m t o s o m e degree i n e n g i n e e r i n g r e s e a r c h . satisfaction w i t h o n e ' s c o n d i t i o n is not a function of h o w w e l l . D u v e r g e r ' s study of p o l i t i c a l parties is an e x a m p l e of theory-oriented r e s e a r c h . and the researchers felt they had to m a k e some sense of it if their efforts were to help the A r m y improve morale. one of w h i c h is that a r m s expenditure m u s t therefore be determined m o r e by d o m e s t i c p o l i t i c a l c o n s i d e r a t i o n s than by the international situation. he spent y e a r s of r e s e a r c h trying to w o r k out the p r e c i s e e c o n o m i c effects of c a p i t a l i s m . M i l i t a r y engagements w e r e no m o r e or l e s s l i k e l y to o c c u r f o l l o w i n g military buildups than under other c i r c u m s t a n c e s . F o r one thing. Stouffer and his c o w o r k e r s w e r e p u z z l e d by the fact that often a s o l d i e r ' s m o r a l e h a d little to do w i t h h i s objective situation. however. M a r x ' s theory of the d i a l e c t i c is p r i m a r i l y a w o r k in normative philosophy. let us l o o k a bit m o r e c l o s e l y at normative p h i l o s o p h y . w h i c h deals w i t h facts o n l y for their u s e f u l n e s s in s p e c i f i c p o l i t i c a l p r o b l e m s . most r e s e a r c h and teaching in p o l i t i c a l s c i e n c e is still of the fourth type suggested in Table 1-1. and b e c a u s e the state and the e c o n o m y function in b a d effects. o n the other h a n d . b a d . 1949)." B u t M a r x w a s less w i l l i n g than M i l l t o s i m p l y assume the factual portions of his argument. T h u s . This type of research is concerned with p e r s o n a l i n v e s t i g a t i o n o f this f a c t u a l b a s i s . types o f e m p h a s i s found i n particular p i e c e s o f r e s e a r c h . h o w e v e r . here they are sought out if they w i l l be u s e f u l in d e v e l o p i n g theories. felt that c h a n c e s for promotion i n the A r m y w e r e g o o d ! I h a v e m e n t i o n e d these two e x a m p l e s to illustrate my point that m o s t r e s e a r c h w o r k i n v o l v e s s o m e m i x of the four types of r e s e a r c h . and so o n — t h e y found no relationship. it is e m p i r i c a l . A c c o r d i n g to this theory. It s h o u l d be e v i d e n t that a n y o n e d e v e l o p i n g n o r m a t i v e theories about p o l i t i c s must begin with some factual assumptions. T h e y d i d this by d e v e l o p i n g the theory of relative deprivation to a c c o u n t for their s e e m i n g l y contradictory findings. T h o s e w h o w e r e not p r o m o t e d w e r e not disappointed. Paradoxically.

r e s e a r c h s h o u l d be evaluated subjectively. In either f o r m of n o n e m p i r i c a l r e s e a r c h . only to learn later that it w a s u s e d by a military j u n t a in a L a t i n A m e r i c a n country to figure out h o w to p r o d u c e a controllable "democracy. difficult though these e t h i c a l questions m a y be. in that w h a t s e e m s " d e h u m a n i z i n g " to a p e r s o n d e p e n d s o n w h a t the p e r s o n t h i n k s " h u m a n " m e a n s — t h a t i s . H o w to c h o o s e an interesting p r o b l e m is one of the most difficult and c h a l l e n g i n g parts of e m p i r i c a l r e s e a r c h . however. applied r e s e a r c h s h o u l d be relevant. but still our results i n v o l v e i n d i v i d u a l c h o i c e s and j u d g m e n t by u s . F i r s t . In the first p l a c e . T h e s e goals have a lot to do w i t h the w a y a study s h o u l d be set up and the criteria a c c o r d i n g to w h i c h it should be judged. engineering. R e s e a r c h on r a c i a l or ethnic groups is particularly s e n s i t i v e . a c c o r d i n g to i n f o r m a l and rather flexible criteria. S i m i l a r l y . or a theory-oriented e m p i r i c a l researcher. a f e w h u n d r e d y e a r s ago the reverse w o u l d probably h a v e been the c a s e . We operate w i t h i n rules of e v i d e n c e in interpreting reality. the r e s e a r c h e r s h o u l d be of political p e r s u a s i o n . depending on the goals of the researcher. o c c u r r i n g j u s t after the extension of the vote to w o m e n . the t h e o r y o f e v o l u t i o n a p p e a r s d e h u m a n i z i n g to m a n y f u n d a m e n t a l i s t C h r i s t i a n s but d o e s not a p p e a r so to m a n y other p e o p l e . in either f o r m of e m p i r i c a l r e s e a r c h ." A l s o . a f o r m a l theorist. this b o o k i s c o n c e r n e d w i t h e m p i r i c a l r e s e a r c h . T o d a y . T h a t most r e s e a r c h i n v o l v e s a m i x of the types does not p r e c l u d e the i m p o r tance of the d i s t i n c t i o n s . the s o c i a l scientist rarely deals i n u n q u e s t i o n e d truths. w a s upset w h e n h e l e a r n e d that h i s r e s e a r c h o n f r o g s ' e y e s t u r n e d out t o h a v e a p p l i c a t i o n s i n the d e s i g n o f g u i d a n c e s y s t e m s for m i s s i l e s ! I n the c a s e o f d e m e a n i n g o r d e h u m a n i z i n g r e s e a r c h . W i t h i n e m p i r i c a l r e s e a r c h . We are not s i m p l y neutral agents of truth. good r e s e a r c h of any sort s h o u l d be directed to an interesting problem. arguing that b e c a u s e it is so hard to j u d g e the results of k n o w l e d g e a n y w a y . G e n e r a l l y . In other w o r d s . we s h o u l d let the c h i p s fall w h e r e they may. m a k e s a m o r e interesting p r o b l e m in n o r m a tive theory than a p r o b l e m dealing w i t h an argument about d y n a s t i c s u c c e s s i o n . j u s t as y o u are w i t h all of y o u r a c t i o n s . the results of a r e s e a r c h c a n be d e m e a n i n g or d e h u m a n i z i n g . R e c r e a t i o n a l r e s e a r c h . I w i l l d i s c u s s this in s o m e detail in Chapter 2. if y o u study t e c h n i q u e s Evaluating Different Types of Research It is dangerous to set d o w n s i m p l e standards for good r e s e a r c h . W e w o r k under sufficient difficulties. we must take personal r e s p o n s i b i l i t y for the results of our r e s e a r c h . As we w i l l see throughout this book. requires p r o b l e m s that w i l l have a substantial i m p a c t on existing bodies of theory. e s p e c i a l l y the fact that we u s u a l l y cannot operate by experimentation (see C h a p t e r 6 ) . for i n s t a n c e . In the s e c o n d p l a c e . I n g e n e r a l . T h e s e types interact in the w o r k of every political scientist. d e p e n d i n g o n o n e ' s v i e w o f h u m a n n e s s . T h e r e are two r e a s o n s for this: (1) It is the m o r e c o m m o n k i n d of research in p o l i t i c a l s c i e n c e . We s h o u l d s i m p l y s e e k truth and not w o r r y about its effects. on the other h a n d . A s a n o t h e r e x a m p l e . It is j u s t not often the c a s e that a r e s e a r c h e r c a n e a s i l y be labeled a normative philosopher. M a n y topics that are of c o n s i d e r a b l e importance to an engineer show little promise for theory-oriented research. R e c e n t results in p s y c h o l o g y suggesting that a w i d e range of b e h a v i o r s are genetically c o n trolled go against our prevailing d i s p o s i t i o n to think of h u m a n s as free agents in w h a t they do. Y o u m u s t therefore be c o n c e r n e d about the ethics of y o u r r e s e a r c h . might be important for theories of h o w voting patterns b e c o m e established a m o n g new voters. this is not n e c e s s a r y . B u t these voting patterns. F o r i n s t a n c e . i t i s v e r y much a matter of personal beliefs and cultural context. The psychological held responsible for demonstrating the factual basis of his conclusions. A c o l l e a g u e o n c e p u b l i s h e d a study of the effects of electoral s y s t e m s on representation. an engineer. and (2) it poses rather m o r e difficult instructional problems than does e n g i n e e r i n g . I devote s o m e w h a t m o r e attention to theory-oriented r e s e a r c h than to . O n e r e s p o n s e to s u c h difficulties might be to take a "pure s c i e n c e " a p p r o a c h . although a normative argument m a y b e m a d e m o r e c o n v i n c i n g . F o r e x a m p l e . h o w e v e r . I w a s s i m p l y u n w i l l i n g to a s s i g n most articles to one or another of the categories. p r o b l e m s s h o u l d be c h o s e n w h i c h are of r e a l i m p o r t a n c e for c o n t e m p o r a r y p o l i c y . o r a n e x e r c i s e i n f o r m a l theory m a y b e m a d e m o r e interesting. A n o t h e r c o l l e a g u e .10 Chapter 1 Doing Research 11 different types of r e s e a r c h in p o l i t i c a l s c i e n c e j o u r n a l s . F o r either sort of a p p l i e d r e s e a r c h . ETHICS OF POLITICAL RESEARCH C o n d u c t i n g r e s e a r c h is an act by y o u . so we are c o n s t r a i n e d in w h a t we c a n assert and cannot s i m p l y p u l l findings out of a hat. B u t w h a t sort of p r o b l e m is "interesting" depends largely on the motivation of the study. as we m a y fear that innocent r e s e a r c h results might reinforce preexisting stereotypes. i n b i o l o g y . E t h i c a l q u e s t i o n s o f this sort are e s p e c i a l l y d i f f i c u l t b e c a u s e the r e s u l t s o f our r e s e a r c h are s o h a r d t o p r e d i c t . for e x a m p l e . w e m u s t c o n c e r n o u r s e l v e s w i t h the effects on society of what we discover. I w a s u n a b l e to do s o . r e s e a r c h noted a b o v e m a y s e e m w h o l l y a p p r o p r i a t e . a further p r o b l e m a r i s e s . by p r o v i d i n g e v i d e n c e for the factual b a s i s on w h i c h its a s s u m p t i o n s rest. T h e r e are two broad c l a s s e s o f e t h i c a l questions regarding our r e s e a r c h . an argument about c i v i l d i s o b e d i e n c e . one type of r e s e a r c h is dominant in any given p i e c e of w o r k . in the c o m m o n usage of the w o r d . it is p o s s i b l e that w h a t y o u learn c o u l d be u s e d by a political charlatan to do b a d things. B u t I w i l l r i s k suggesting t w o standards for r e s e a r c h that w i l l s e r v e as e x a m p l e s of the w a y in w h i c h the type (or types) of r e s e a r c h we are doing dictates the w a y we s h o u l d c o n d u c t that r e s e a r c h . m a n y p r o m i s i n g topics for recreational r e s e a r c h are not directly relevant. that our results are to s o m e extent a subjective interpretation of reality. L i k e any creative w o r k . r e s e a r c h o n the difference b e t w e e n m e n ' s and w o m e n ' s voting i n I c e l a n d in the 1920s and 1930s w o u l d s o u n d absurd f r o m the standpoint of an engineer.

as I pointed out in C h a p t e r 1. more specific than the ones described earlier but not necessarily easier to answer. has h a d to c o n c e r n itself w i t h d e v e l o p i n g theories to e x p l a i n w h y one c h i l d learns things m o r e q u i c k l y than another. To effect a c h a n g e in s o m e given p h e n o m e n o n . B u t w h a t if it puts subjects in danger of death. Harming the subjects of your study. T h i s w a s n e c e s s a r y if they w e r e to devise w a y s to raise the m o r a l e of A r m y p e r s o n n e l i n general. 13 . you should make certain that your subjects know exactly what they will be doing and what use you will make of them. Is it ethically right. Imposition. Particular problems arise in the f o l l o w i n g w a y s : 1. In this chapter we look more c l o s e l y at the nature of political theories and at the f a c tors that i n f l u e n c e the d e c i s i o n to do r e s e a r c h on a particular theory. w h i c h w a s p u r c h a s e d at great h u m a n p a i n . Fooling or misleading the subjects. we w o u l d g e n e r a l l y s a y y e s . As you will see in Chapter 6. female. You are asking your subjects to help you. the k e y to s o l v i n g m a n y engineering p r o b l e m s m a y be a political theory of s o m e sort. but it is also an annoying time if you have 15 minutes' worth of questions to ask. Confidentiality.12 Chapter 1 A s e c o n d c l a s s of questions. If the costs to s u b j e c t s are slight ( i n c o n v e n i e n c e . should generally be avoided. D o e s u s i n g the results o f the r e s e a r c h j u s t i f y it? I f s o . in evaluating the effects of a program to get people off the welfare rolls and into jobs. for e x a m p l e . s h o u l d it be d o n e ? T h e r e is no c l e a r a n s w e r . for me at least. p a i n of w h i c h they are i n f o r m e d in a d v a n c e ) and the social benefits great. or submitting them to embarrassing situations. it is an e n d in itself in the other. T h e Stouffer study. Stouffer and his collaborators h a d t o e x p l a i n w h y M P s h a d higher m o r a l e than air c o r p s m e n . they s i m p l y i n v o l v e m e a s u r i n g things that need to be m e a s u r e d . everyone has the right not to be fooled and not to be used without his or her consent. to withhold the program from some deserving people while administering it to others. Generally. I n d e e d . y o u m a y need to develop a theory that a c c o u n t s for several factors and a l l o w s y o u to T h e p r o b l e m s I have noted here p o s e difficult ethical questions of the " e n d s and m e a n s " sort. 4. On the other h a n d . As an overall rule. In that c a s e . If a r e s e a r c h that w i l l benefit society c a n be c o n d u c t e d only by mistreating s u b j e c t s . it is w o r t h taking a c l o s e r l o o k at the nature of theory. However. A p a i n f u l ethical question today is w h e t h e r e v e n to u s e the results of that r e s e a r c h . M u c h a p p l i e d r e s e a r c h on the probl e m of e n r i c h i n g the education of underprivileged c h i l d r e n . S . perhaps w e s h o u l d destroy the results. Harm to subjects. Dinnertime is a good time to reach people by phone. but w h i c h m a y potentially help i n s a v i n g lives a n d — w e h o p e — w i l l never b e available again f r o m any s o u r c e . You should avoid shaming people into participating in your study. deals with our treatment of the people we are studying. for example. is that people s h o u l d never be c o e r c e d or tricked into participation and s h o u l d a l w a y s be fully i n f o r m e d before they agree to participate. Relevant details you include in your report might make it easy to identify the subject (a member of Congress. keep yours short. studies m e a s u r i n g the malapportionment of state legislatures. A l t h o u g h this chapter deals w i t h p o l i t i c a l theories. T a k i n g the U . 3. in order to see how effective it is? 2. B u t no matter w h i c h type of r e s e a r c h one is currently e n g a g e d i n . the subjects of your study will wish to have their privacy protected. from the South. T h e o r y is a tool in one type of r e s e a r c h . c e n s u s i s one e x a m p l e o f s u c h engineering r e s e a r c h . We are responsible to treat the subjects of our study fairly and decently. O t h e r s i n c l u d e the G a l l u p P o l l . as m a y be true of political r e s e a r c h that d e l v e s into racketeering or c o r r u p t i o n ? T h e most horrible historic e x a m p l e of s c i e n c e gone bad is that of the N a z i doctors w h o k i l l e d prisoners b y i m m e r s i n g t h e m i n i c e water t o see h o w long people c o u l d s u r v i v e in f r e e z i n g water. Public officials may get a hundred questionnaires a year. c i t e d in C h a p t e r 1. m a n y e n g i n e e r i n g studies do not require that a theory be developed. and c o m parisons of the relative military strength of v a r i o u s countries. y o u s h o u l d not a s s u m e that it is important only for w h a t I h a v e c a l l e d theory-oriented r e s e a r c h . Don't demand more of them than is reasonable. It is not enough just to withhold publishing their names. 5. You should take care to truly mask the subjects who have helped you. either by doing harmful things to them or by withholding good things from them. the results of your study might well be more valid if the people you study are unaware that they are being studied. the senior member of her committee). A l o n g the w a y I w i l l d i s c u s s s o m e standards to u s e in d e c i d i n g w h e t h e r a theory is w e a k or strong. is another e x a m p l e of an engineering study in w h i c h it w a s n e c e s s a r y to develop a theory. B u t might that not lead to greater h u m a n pain for v i c t i m s o f f r e e z i n g and exposure w h o m w e might have h e l p e d ? T h e one f i r m rule. I n s u m . Embarrassment or psychological stress. manipulate them to produce the d e s i r e d c h a n g e . it s h o u l d be done. e n g i n e e r i n g r e s e a r c h m a y o r m a y not i n v o l v e the development o f political theories. theory-oriented r e s e a r c h a l w a y s does.

B u t i n a s m u c h as the borough presidents h a v e little power. the authors tested a theory that if a country i n c r e a s e s its m i l i t a r y expenditure. then the n u m b e r of political parties in the c o u n t r y c a n be e x p e c t e d to grow or s h r i n k to a certain number. T h u s . F o r i n s t a n c e . they are not taken as determined by any other factor u s e d in this particular theory. 2. It is e a s y to develop a trivial theory. that i s . of c o u r s e . T h i s is the dependent variable." T h e e x a m p l e s we l o o k e d at in C h a p t e r 1 w e r e all of this f o r m . p e r h a p s the w a y the p e r s o n v o t e s . but an i m a g i n a t i v e r e s e a r c h e r w h o sits d o w n a n d thinks about it for a w h i l e m a y be able to point up additional. If one or the other of h i s independent v a r i a b l e s h a d not v a r i e d . A theory should make accurate predictions. A l l of these factors are c a l l e d " v a r i a b l e s " s i m p l y b e c a u s e it is the variation of e a c h that m a k e s it of interest to u s . Simplicity. ories. let alone the c h a o s of p o l i t i c s in general. but before y o u begin y o u r r e s e a r c h . the theory s h o u l d give a h a n d l e on as big a portion of the u n i v e r s e as p o s s i b l e . but it w o u l d a l s o shed light on the w i d e r u n i v e r s e dealt w i t h by the other theories. if all countries h a d h a d the s a m e electoral s y s t e m . A theory c a n attain great generality rather e c o n o m i c a l l y if it h e l p s to recast older theTheory 1: Theory 2: Democracies do not tend to initiate wars. In theory 1. if e a c h country h a d had e x a c t l y the s a m e n u m b e r o f p a r t i e s — t h e r e w o u l d h a v e b e e n nothing for D u v e r g e r to e x p l a i n . T h e dependent v a r i a b l e is so n a m e d b e c a u s e in terms of the particular theory used it is thought to be the result of other factors (the independent v a r i a b l e s ) . whether or not a country is l i k e l y to be a d e m o c r a c y depends on its per capita i n c o m e . y o u s h o u l d try to j u s t i f y y o u r c h o i c e of topic. a theory s h o u l d address a p r o b l e m that is currently p r e s s i n g . A n o t h e r theory might u s e the p e r s o n ' s s o c i a l status as an independent v a r i a b l e to e x p l a i n s o m e t h i n g e l s e . e a c h o f w h i c h i n v o l v e s its o w n portion o f reality. i n a s m u c h a s there w e r e n o differences. it w o u l d p e r f o r m two s i m p l i f y i n g f u n c t i o n s : It w o u l d not o n l y give us a handle on the rather l i m i t e d portion of our environment that it sought to e x p l a i n directly. In the D i e h l a n d K i n g s t o n study. the dependent T h r e e things are important if we are to develop g o o d . b e c a u s e "the applications are o b v i o u s . then Y w i l l f o l l o w as a result. It m a y s e e m u n n e c e s s a r y to point this out. S i m i l a r l y . Such a theory would be about as chaotic and as difficult to absorb as the reality it sought to simplify. no variable is innately either independent or dependent. it w o u l d not help us very m u c h to r e d u c e the c h a o s of N e w Y o r k C i t y p o l i t i c s . " T r u e . In D u v e r g e r ' s theory." we are referring not o n l y to h o w large a selection of items f r o m reality the theory deals w i t h . might predict quite a c c u r a t e l y for that s p e c i f i c situation. but it w o u l d be a m o r e v a l u a b l e tool if it c o u l d be s h o w n to h a v e significant i m p l i c a t i o n s for other areas of s o c i a l t h e o r y — d e m o c r a t i c theory.14 Chapter 2 POLITICAL THEORY Political Theories and Research Topics W H A T D O E S G O O D T H E O R Y L O O K LIKE? 15 CAUSALITY AND In the s o c i a l s c i e n c e s . effective theories: 1. Students have been k n o w n . so we shall consider them separately. F o r i n s t a n c e . a n d if the country adopts a certain electoral law. W h e n we say that a theory s h o u l d apply " b r o a d l y " a n d "generally. 3. v a r i a b l e w a s the n u m b e r of parties. or whatever. T h e s a m e v a r i a b l e m a y b e a n i n d e p e n d e n t v a r i a b l e i n o n e theory a n d a d e p e n d e n t v a r i a b l e i n another. o n e theory m i g h t u s e the s o c i a l status of a p e r s o n ' s father (the i n d e p e n d e n t v a r i a b l e ) to e x p l a i n the p e r s o n ' s s o c i a l status (the dependent v a r i a b l e ) . that factor w o u l d have b e e n u s e l e s s i n e x p l a i n i n g the dependent variable. to explain why people vote the way they do. I n d e p e n d e n c e and d e p e n d e n c e are the two roles a variable m a y p l a y in a c a u s a l theory. Importance. In theory 2. broad theory which gives predictions that are not much better than one could get by guessing. A theory of the organization of borough p r e s i d e n c i e s in N e w Y o r k C i t y . s i m p l y to turn in a c o m p u t e r printout as a paper. but it is an important part of the e n g i n e e r i n g r e s e a r c h project. A c a u s a l theory a l w a y s i n c l u d e s s o m e p h e n o m e n o n that is to be e x p l a i n e d or a c c o u n t e d for. democracy functions as a dependent variable. It all depends on the theory: In engineering r e s e a r c h . D u v e r g e r u s e d t w o independent v a r i a b l e s in h i s theory: the nature of s o c i a l c o n f l i c t s in a country a n d the c o u n t r y ' s electoral s y s t e m . Predictive accuracy. if a certain configuration of political c o n f l i c t s e x i s t s . but also to h o w great a variety of preexisting theories are affected by the n e w theory. democracy functions as an independent variable. in intricate combinations. Y o u r r e s e a r c h report s h o u l d i n c l u d e s o m e d i s c u s s i o n of the i m p o r tance of the p r o b l e m a n d of p o s s i b l e applications for y o u r findings. In the D u v e r g e r e x a m p l e . It does not help to have a simple. the obvious a p p l i c a t i o n s are o b v i o u s . for e x a m p l e . it s h o u l d apply broadly a n d generally. a theory o f electoral change might take on importance partly f r o m the p h e n o m e n a it e x p l a i n e d d i r e c t l y — c h a n g e s in p e o p l e ' s votes. Countries with high per capita incomes are more likely to be democracies than poor countries are. In theory-oriented r e s e a r c h . what makes a theory important is different in engineering research than in theory-oriented research. T h e s e are c a l l e d the independent variables. the variations in party s y s t e m s that p u z z l e d h i m c o u l d not h a v e b e e n due to differences in the c o u n t r i e s ' electoral s y s t e m s . If party s y s t e m s h a d not v a r i e d — t h a t i s . general theories of attitude c h a n g e . the tendency to w a g e w a r depends on whether or not a country is a democracy. T h i s is a subjective j u d g m e n t . A theory should be important. In effect. However. theories g e n e r a l l y are stated in a c a u s a l m o d e : " I f X h a p p e n s . then it might be e x p e c t e d to go to war. It would not be very useful to develop a theory that used 30 variables. m o r e v a r i e d w a y s in w h i c h the results c a n be u s e d . T h e shape it takes " d e p e n d s " on the configuration of the other factors. not o n l y to y o u r s e l f but a l s o to y o u r a u d i e n c e . a n d it is not s o m e t h i n g about the variable itself. for e x a m p l e . the independent v a r i a b l e s are thus d e s i g n a t e d b e c a u s e in terms of the particular theory. It should use no more than a few independent variables. one that is often c a r r i e d out s l o p p i l y a n d in an i n c o m p l e t e way. T h u s . A theory should give us as simple a handle on the universe as possible. A c a u s a l theory a l s o i n c l u d e s one or m o r e factors that are thought to affect the dependent variable. .

P e r h a p s one day the old s a w w i l l b e c o m e . an attempt to m a k e a theory m o r e general often w i l l cost us s o m e t h i n g in either the s i m p l i c i t y of the theory or the a c c u r a c y of its predictions. D o u g l a s S . C o n v e r s e wanted to k n o w w h y the level of party identification varied as it did from country to country. A s i d e f r o m its utility and simplicity. I m m i g r a t i o n and N a t u r a l i z a t i o n S e r v i c e added extra guards and i m p o s e d p u n i s h m e n t s on e m p l o y e r s found to be h i r i n g i l l e g a l i m m i g r a n t s . C o n v e r s e took as his dependent variable the strength of the "party identification" of i n d i v i d u a l s — t h e i r sense that they are supporters of one or another of the political parties. it is important to try to develop theories about it. 2 p r e s i d e n c i e s i n N e w Y o r k . it 1 T h e fact that most social s c i e n c e theory is not very elegant does not m e a n that it is not good. and predictive a c c u r a c y — i t w i l l be u s e f u l as a tool for s i m p l i f y i n g reality. This problem is addressed in Chapter 3. E s p i n o s a ( 1 9 9 7 ) found that s i n c e the border c r o s s i n g h a d been m a d e tougher. B e c a u s e the overall extent to w h i c h citizens of a particular country felt bound to the existing parties s e e m e d likely to have something to do w i t h h o w stable politics in that country w o u l d be. O n e difficulty in creating an elegant theory is that trying to meet any one of the three b a s i c criteria tends to m a k e it harder to meet the other two. but this w o u l d have r e d u c e d the s i m p l i c i t y of the theory. . S i m i l a r l y . Other studies h a d s h o w n high levels of party identification in B r i t a i n and N o r w a y . If it is important to understand h u m a n k i n d ' s behavior. he and G e o r g e s D u p e u x h a d found that. that the study w o u l d borr o w i m p o r t a n c e f r o m this u n d e r l y i n g p h e n o m e n o n . he a n d D u p e u x h a d f o u n d that the difference in percentage of party identifiers b e t w e e n F r a n c e a n d the U n i t e d States s e e m e d to b e e x p l a i n e d a l m o s t w h o l l y b y the fact that m o r e A m e r i c a n s than F r e n c h h a d s o m e i d e a o f w h a t party their fathers h a d identified w i t h . If a t h e o r y ' s p r e d i c t i o n s are r e a s o n a b l y a c c u r a t e . At any rate. It was not long before further work showed similar effects for mothers! 3 'The choice of this word typifies the aesthetic pleasure—and the vanity—with which researchers approach their work. N o t to take a w a y f r o m the difficulty of rocket s c i e n c e . Remember that this was an early study.M e x i c a n border. 2 A b o u t 80 p e r c e n t of those w h o did k n o w their father's Another reason for the difficulty of attaining elegance in social research is simply that most social science terms are imprecise and ambiguous. i t m a y b e that h u m a n b e h a v i o r c a n be u n d e r s t o o d . but that we h a v e not y e t c o m e up w i t h a s o c i a l theory that c o u l d s h o w the true potential of our f i e l d . w h a t the h e c k . In both countries about 50 percent of those who did not know 3 their father's party expressed identification with s o m e party t h e m s e l v e s . s o m e t h i n k i t m a y p e r h a p s b e l a r g e l y b e y o n d e x p l a n a t i o n . it's not rocket s c i e n c e " — i m p l y i n g that rocket s c i e n c e is the e s s e n c e of difficulty and complexity. even if things do not fall as neatly into p l a c e as we w o u l d like. T h e e n d result w a s that the n u m b e r of illegal i m m i g r a n t s present at any g i v e n time w a s i n c r e a s e d . and l o w e r levels of party identification in G e r m a n y and Italy. M a s s e y and K r i s t i n E . O n e thousand extra border patrol officers w e r e then added e a c h y e a r for several y e a r s afterward. H u m a n b e h a v i o r i s m o r e c o m p l e x than the b e h a v i o r o f p h y s i c a l o b j e c t s — i n fact. T h e b o r o u g h p r e s i d e n c i e s m i g h t . whereas about 75 percent of A m e r i c a n s w h o were polled identified with some political party. we s a w that he might h a v e i m p r o v e d the a c c u r a c y of h i s t h e o r y ' s predictions by bringing in additional explanatory v a r i a b l e s . it's not s o c i a l s c i e n c e . A s w e c a n see i n T a b l e 2 . a s i m i lar poll conducted in F r a n c e s h o w e d that less than 45 percent of the respondents did so ( C o n v e r s e & D u p e u x . If a theory c a n s u c c e e d r e a s o n a b l y w e l l at meeting these three c r i t e r i a — i m p o r t a n c e . given that subject matter. " Example of Elegant R e s e a r c h : Philip C o n v e r s e I n his article " O f T i m e and P a r t i s a n S t a b i l i t y " ( 1 9 6 9 ) . At the time of their e a r l i e r study. B u t i t m i g h t b e p o s s i b l e . the theory a c c r u e s s o little i m p o r t a n c e d i r e c t l y a s t o l o o k a b s u r d . it is rare for theory in the s o c i a l s c i e n c e s to a c h i e v e e l e g a n c e . 1 am a l w a y s a m u s e d w h e n people s a y of a question that is b e i n g m a d e to look more difficult than it r e a l l y i s . T h e r e a l test of a theory's v a l u e is whether its subject matter is important and h o w c l o s e it has c o m e to elegance. H i s study is worth l o o k i n g at in s o m e detail. P o l i t i c a l scientists often j o k i n g l y refer to this element as the "interocular subjectivity test" of r e s e a r c h — D o e s it hit us b e t w e e n the e y e s ? A g o o d e x a m p l e of r e s e a r c h w i t h beautiful surprise is a study of the i m p a c t of "get-tough" p o l i c i e s against i l l e g a l i m m i g r a t i o n a c r o s s the U n i t e d S t a t e s . i f the b o r o u g h p r e s i d e n c i e s w e r e taken a s e x a m p l e s o f s o m e b r o a d e r c o n c e p t i n u r b a n p o l i t i c s . S . a theory to e x p l a i n the o r g a n i z a t i o n of b o r o u g h Political Theories and Research Topics 17 i s u s u a l l y b e c a u s e the s c o p e o f the theory i s r e s t r i c t e d o r b e c a u s e m a n y o f the e x c e p t i o n s to the theory h a v e b e e n a b s o r b e d into it as a d d i t i o n a l v a r i a b l e s . In an earlier study. In the e x a m p l e of D u v e r g e r ' s theory. w h a t the h e c k . I t a p p e a r s t o b e p a r t i c u l a r l y h a r d t o a c h i e v e elegant r e s e a r c h i n the s o c i a l s c i e n c e s . s i m p l i c i t y . that m a k e s us rethink our w o r l d . illegal i m m i grants w h o o r i g i n a l l y w o u l d h a v e c o m e to the U n i t e d States for only a f e w months of s e a s o n a l labor n o w stayed y e a r . c o m p a r e d w i t h other s c i e n t i f i c a r e a s . for e x a m p l e . there is also an element of "beautiful s u r p r i s e " to elegant r e s e a r c h . within each row there was practically no difference between the French and A m e r i c a n l e v e l s of party identification. not d e c r e a s e d .r o u n d b e c a u s e they k n e w it w o u l d be h a r d to get b a c k into the U n i t e d States if they w e n t h o m e to M e x i c o . but plotting the trajectory of an object in a v a c u u m is far s i m p l e r than understanding the motivation of a h u m a n b e i n g . s e r v e as a u s e f u l m i c r o c o s m for s t u d y i n g the w o r k i n g s of patronage. by the stepped-up enforcement. done in 1962. making it very c o m p l e x . S u c h a theory is s o m e t i m e s d e s c r i b e d as elegant.1 . 1962). In the late 1980s and early 1 9 9 0 s . " T h i s s h o u l d n ' t be that h a r d . the U . " T h i s s h o u l d n ' t be that hard.16 Chapter 2 In the e x a m p l e j u s t c i t e d . P h i l i p C o n v e r s e c a m e about as c l o s e to d e v e l o p i n g an "elegant" theory as one c a n c o m m o n l y do in the s o c i a l s c i e n c e s . gives us a s p e c i a l k i n d of pleasure. A p i e c e of r e s e a r c h that goes against our expectations. O n the other h a n d .

J. h e r e a s o n e d that the d i f f e r e n c e b e t w e e n F r a n c e a n d the U n i t e d States c o u l d b e e x p l a i n e d e a s i l y i f the p r e v i o u s g e n e r a t i o n i n F r a n c e h a d i n d e e d i n c l u d e d v e r y f e w voters w h o i d e n t i f i e d w i t h a party. M o r e o v e r . In the next generation after that. 1 9 6 0 . T h e first of these derived f r o m the earlier r e s e a r c h on i n d i v i d u a l development." is described in J.7% of country A having an identification. and h o w l o n g e l e c t i o n s have b e e n h e l d in h i s / h e r c o u n t r y ) . the Markov Chains Converse's reasoning is based on some cute. C o n v e r s e a n d D u p e u x a c c e p t e d this a s a n interesting finding and d i d not elaborate o n it. Inc.) T h u s .6 50. g r a d u a l l y party a d h e r e n c e w o u l d r e a c h a stable l e v e l .: Prentice-Hall.1% of country B having an identification. pp. that the d i f f e r e n c e w a s due to the fact that the F r e n c h d i d not talk to their c h i l d r e n about p o l i t i c s as m u c h as the A m e r i c a n s d i d . ) T h u s . W o r k i n g f r o m these two a n g l e s . 171-178. F r a n c e m u s t b e m o v i n g t o w a r d the l e v e l o f party i d e n t i f i c a t i o n f o u n d i n the U n i t e d S t a t e s . Introduction to Finite Mathematics (Englewood Cliffs. B u t for the p u r p o s e s o f a r g u m e n t . T h e s e c o n d strand o f theory c a m e into play w h e n C o n v e r s e tied h i s theory o f national d e v e l o p m e n t to s o m e older findings on i n d i v i d u a l voters in the A m e r i c a n electorate. i f the a s s u m p t i o n w e r e true. C o n v e r s e c h o s e t o a s s u m e that this w a s not the c a s e . called a "Markov chain. and assuming that party identifiers have the same number of children as nonidentifiers. In the first e l e c t i o n . rather than of h i s age itself ( s e e . " on p. The process involved here. Converse and Dupeux estimated for France and the United States that about 80 percent of those whose fathers had identified with a party developed an identification of their own. we would expect them to converge rapidly. and 90 percent of the population of country B presently identify with a party.) If this w e r e s o . Kemeny.8 X 30%) + (0. for e x a m p l e . in the next generation we would expect to see (0. C o n v e r s e d e v e l o p e d a s i m p l e theory that predicts the strength of a voter's party identification f r o m j u s t two things: (1) the number of y e a r s the p e r s o n has b e e n eligible to vote ( w h i c h is a d u a l f u n c t i o n of age K n o w father's party Do not know father's party 79. H e then s h o w e d that i f h i s a s s u m p t i o n about the p r e v i o u s g e n e r a t i o n ' s l o w l e v e l o f party i d e n t i f i c a t i o n w e r e true. A c c o r d i n g to this s c h e m e . Snell.8 X 59%) + (0. of those whose fathers had not identified with a party. ( H e knew o n l y that they h e l d for F r a n c e a n d the U n i t e d States. we would expect to see (0. but 50 percent of s e c o n d . then if 30 percent of the population of country A presently identify with a party. and (0.7 party w e r e p o l i t i c a l party adherents.18 Chapter 2 Political Theories and Research Topics 19 T A B L E 2-1 Percent Having Same Sort of Party Identification France USA 81. and G. F i r s t . then even though the countries differ greatly in the level of identification at present. 5 0 percent d e v e l o p e d a n identification o f their o w n . simple mathematics that you can play with for yourself.8 X 77%) + (0. . about 50 percent developed an identification of their own. B r i t a i n . A t the time.5 X 70%) = 59% of country A having an identification. T h u s . o n e c o u l d e x p e c t the next g e n e r a t i o n i n F r a n c e t o b e m u c h m o r e l i k e the A m e r i c a n s . It c o u l d h a v e b e e n . of c o u r s e . 1957). ( F r e n c h w o m e n .g e n e r a t i o n voters w o u l d e x p r e s s i d e n t i f i c a tion. ( S i n c e o f those w h o s e fathers h a d not identified w i t h any party. in two generations the two countries. C a m p b e l l e t a l . N. For example. pp. and (0. w e r e f i r s t given the vote in 1945. Thus. s c a r c e l y any voters in a g i v e n c o u n t r y w o u l d identify w i t h a party." C o n v e r s e u s e d the earlier f i n d i n g to suggest a general theory of the p r o c e s s by w h i c h countries developed stable patterns of party preference.4 47. F r a n c e m u s t be at an e a r l i e r stage of the p r o c e s s than America. If the rates of transferring identifications are in fact the same in two countries.8 X 9 0 % ) + (0. . which had started out being quite different.J. a n d N o r w a y . " M a r k o v c h a i n s . Given these figures.5 X 41%) = 67. and that.. Voting studies c o m m o n l y h a d s h o w n that w i t h i n an i n d i v i d u a l ' s life s p a n . a n d (2) the l i k e l i h o o d that the i n d i v i d u a l ' s father h a d identified w i t h a party ( w h i c h in turn depends on w h a t portion of the father's adult life e l e c t i o n s w e r e h e l d in w h i c h he w a s e l i g i b l e to vote). for one thing. then both F r a n c e a n d the U n i t e d States m i g h t s i m p l y b e e x a m p l e s o f a g e n e r a l p r o c e s s that a l l c o u n t r i e s undergo w h e n their c i t i z e n s are first g i v e n the vote. the r e l a t i v e l y l o w l e v e l of party i d e n tification i n F r a n c e m u s t h a v e r e s u l t e d b e c a u s e the vote w a s extended later a n d l e s s c o m p l e t e l y there than i n A m e r i c a .5 X 23%) = 73. would have moved to similar levels of party identification. A l s o . the difference b e t w e e n the two c o u n t r i e s w a s a result of the fact that the A m e r i c a n s k n e w their father's party so m u c h m o r e frequently than the F r e n c h d i d . ( T h i s d e v e l o p m e n t c a n b e seen i n the b o x . 1 6 1 .7 the older he w a s the m o r e l i k e l y he w a s to identify strongly w i t h a party. I n d o i n g s o h e b r o u g h t t w o strands o f theory together. this h a d b e e n s h o w n to be a result of h o w long he h a d b e e n e x p o s e d to the party by being able to vote for it. 19) C o n v e r s e further r e a s o n e d that the 80 percent a n d 50 percent figures might be u n i v e r s a l l y true. Thompson.1 6 4 ) .5 X 10%) = 77% of country B having an identification. B u t i n " O f T i m e and P a r t i s a n Stability.

b e c a u s e s o m e k i n d s of data by their very nature are m o r e a m e n a b l e to p r e c i s e f o r m u l a t i o n . H i s f i n d i n g s s e r v e d a s a s s u m p t i o n s for f o r m a l theoretic w o r k ( P r z e w o r s k i . h o w little the U . " T h e first f o r m of the statement g i v e s m o r e p r e c i s e i n f o r m a t i o n . S . less quantitative r e s e a r c h provides greater breadth. O v e r the y e a r s after it appeared. T y p i c a l l y . S . the quantitative r e s e a r c h e r is a b l e to do a great d e a l . i t s e e m e d to predict fairly accurately. s o u r c e s o f quantitative data are quite r e s t r i c t e d . numerical m e a s u r e s of things. the c h a n g e p r o c e e d e d a bit m o r e r a p i d l y in the latter h a l f of that p e r i o d . T h u s . a n d m o s t r e s e a r c h m u s t b e r e l a t i v e l y n o n quantitative. then party identification levels in a l l electoral d e m o c r a c i e s s h o u l d converge over the next f e w generations toward a s i n g l e v a l u e of about 72 percent. quantitative m e a n s . or the U n i t e d States. the pattern we t y p i c a l l y o b s e r v e in E u r o p e a n d A m e r i c a (the y o u n g b e i n g w e a k l y identified. C o n v e r s e f o u n d that the theory predicted quite w e l l for a l l five c o u n t r i e s . however. the p a r t i c u l a r s u b j e c t w e are s t u d y i n g affects the extent t o w h i c h i t i s p o s s i b l e for u s t o quantify. are m o r e l i k e l y to p r o d u c e s i m p l e . but rapidly decreasing. the important thing is to see w h a t relationship degrees of quantification bear to e l e g a n c e in r e s e a r c h . a n d i t w a s b r o a d l y a p p l i c a b l e . although C o n v e r s e ' s theory w a s quite simple. the U n i t e d States. and b e c o m e s e n m e s h e d i n . J a m e s Prothro c a l l e d the dispute "the nonsense fight over scientific method. T h i s oversight m a y w e l l be due to the difficulty of getting " h a r d " data on w h a t goes on in that office. H o u s e i n c r e a s e d f r o m 1880 t o 2 0 0 5 o n the a v e r a g e b y 0. potential for electoral instability in a new electorate. a l l other things b e i n g e q u a l . . F i r s t o f a l l . is c o n s i d e r e d relatively less quantitative. a n d mathematical statements about them. I n e l e c t i o n s t u d i e s . w h i c h i s w h a t one w o u l d expect o f elegant w o r k . " U s i n g data f r o m B r i t a i n . T i l l e y . the o l d strongly) s h o u l d not h o l d . It is s t r i k i n g .20 Chapter 2 Political Theories and Research Topics 21 A n y t h i n g i n p o l i t i c a l s c i e n c e c a n b e stated w i t h v a r y i n g degrees o f q u a n t i f i c a t i o n . and they are certainly m o r e l i k e l y to give us a c l e a r i d e a of h o w accurate a theory's predictions are. On the other h a n d . a n d tends to make verbal statements about t h e m . the r e s u l t s o f m a n y attitude s u r v e y s are a l s o a v a i l a b l e . essentially. and a greater a w a r e n e s s of the c o m p l e x i t y of s o c i a l p h e n o m ena. e a c h a p p r o a c h has its o w n costs a n d benefits. and M e x i c o to test the theory. T h u s . S . a n d it is w e l l to r e m e m b e r that no particular degree of quantification h a s a corner on e l e g a n c e . I n v i r t u a l l y every f i e l d o f political r e s e a r c h . the rate o f i n c r e a s e w a s 0. (3) (/the transition rates for a l l countries w e r e r o u g h l y the s a m e as for F r a n c e a n d the U n i t e d States. p. w o r k c a n b e c o n d u c t e d in either a p r i m a r i l y quantitative or a p r i m a r i l y nonquantitative m o d e . o r i n s t u d i e s d e a l i n g w i t h the U . A f e w e x a m p l e s of predictions f r o m his theory are (1) at the time e l e c t i o n s are first h e l d in a country. thus f u l f i l l i n g the third criterion for " e l e g a n c e . It simultaneously explained individual behavior and characteristics of political s y s t e m s . all s h o u l d identify at the s a m e l o w l e v e l s . but generally. T h u s . greater openness to totally n e w theories. It is a bit hard to pin d o w n w h a t the term r e s e a r c h that p a y s a g o o d d e a l of attention to tends to m a k e best that studies w i t h varying degrees of quantification be c a r r i e d on in any g i v e n field of p o l i t i c a l r e s e a r c h . r e p r e s e n tatives s e r v e d a s t e a d i l y i n c r e a s i n g length of time in the H o u s e . N o r w a y . E a c h a p p r o a c h i n v o l v e s c o s t s a n d benefits for r e s e a r c h . B u t it m a y be that the time a n d attention spent in gathering p r e c i s e data m a k e it difficult for the r e s e a r c h e r to appreciate the larger a s p e c t s of a theory. and to test w h e t h e r n e w electorates actually b e h a v e as C o n v e r s e ' s theory predicts they w o u l d ( S h i v e l y . " s a y s a p p r o x i m a t e l y the s a m e thing a s " F r o m 1 8 8 0 t o 2 0 0 5 . S t u d i e s e m p l o y i n g m o r e quantitative data. It i m p l i e d a more or less universal f o r m of political development at the m a s s l e v e l — w i t h a prediction of initial. It is in this w a y that a good p i e c e of theoretical w o r k f e e d s . 1 9 7 5 ) . there is the danger that o v e r c o n c e r n w i t h p r e c i s i o n m a y restrict our c h o i c e of r e s e a r c h to those v a r i a b l e s we c a n m o r e e a s i l y quantify. 1997. p r e s i d e n c y . D a l t o n & W e l d o n . T h e theory w a s s i m p l e . L e i t n e r . (2) if e l e c t i o n s w e r e interrupted in a country (as in G e r m a n y f r o m 1933 to 1 9 4 5 ) . W h a t w a s m o r e . is c o n s i d e r e d quantitative. a n d m o s t voters d o not regard their a c t i o n s a s s o m e t h i n g about w h i c h they n e e d t o m a i n t a i n s e c r e c y . To Quantify or Not A side issue in the question of h o w to develop elegant theory is the o l d chestnut: S h o u l d political s c i e n c e be "quantitative" or not? T h e r e has been m u c h rhetoric spilled over this. R e s e a r c h that is less c o n c e r n e d w i t h m e a s u r i n g things n u m e r i c a l l y . i n C h i n e s e s t u d i e s . I deal w i t h the i s s u e of proper l e v e l s of p r e c i s i o n in C h a p t e r 5. T h u s . 1972. It is p r o b a b l y s e c o n d f r o m h i s a n d D u p e u x ' s c o m p a r a t i v e study o f F r a n c e and the U n i t e d States. 2 0 0 7 ) . there i s c o n s i d e r a b l e s c o p e for q u a n t i f i c a t i o n ." but it has not c o o l e d sufficiently in the intervening y e a r s . 2 0 0 3 . for the different levels of quantification c o m p l e m e n t e a c h other. As long ago as 1956. a n d 0 . T o g i v e a c r u d e e x a m p l e : " T h e length o f s e r v i c e o f m e m b e r s o f the U . p r e s i d e n c y i s studied b y p o l i t i c a l s c i e n t i s t s . but the s e n s e of the two statements is the s a m e . the C o n v e r s e article stimulated a great deal of further r e s e a r c h . Italy. M o s t people w o u l d agree that p r e c i s e i n f o r m a t i o n is m o r e u s e f u l than i m p r e c i s e i n f o r m a t i o n .78 y e a r s e v e r y d e c a d e . l e v e l s of party identification s h o u l d d e c l i n e at a predictable rate. T h e y a l s o s t i m ulated r e s e a r c h e r s to investigate w h e t h e r in fact the transition probabilities on w h i c h the M a r k o v c h a i n is b a s e d are the s a m e in all i n d u s t r i a l i z e d countries ( B u t l e r & Stokes. it w a s applicable to a w i d e variety of questions. the w h o l e b o d y o f theoretical exploration. 5 3 ) . 8 6 y e a r s p e r d e c a d e after 1 9 2 2 . however. u s a b l e theories. 1 9 6 9 . for i n s t a n c e . F o r our p u r p o s es here. A n d it included the startling suggestion of a convergence of "mature" electorates to a c o m m o n level of party identification approximately equal to that of B r i t a i n . A l s o .68 y e a r s p e r d e c a d e before 1 9 2 2 . party identification is predicted f r o m the i n d i v i d u a l ' s age and the length of time that the country h a s been h o l d i n g e l e c t i o n s . G e r m a n y . R e c o r d s f r o m e a r l i e r e l e c t i o n s are u s u a l l y kept i n f a i r l y g o o d order.

that c h o o s i n g a topic is not s o m e t h i n g for w h i c h rules c a n be laid d o w n . Y o u m a y decide that for the p r o b l e m nearest y o u r heart. it will look as if the unique aspects are indeed general. At the s a m e time. but b e c a u s e y o u h a v e seen s o m e r e s e a r c h on it that y o u think w o u l d be rather e a s y to correct. o r o n s i m i l a r p r o b l e m s . there s i m p l y is not e n o u g h material a v a i l a b l e to let y o u study it satisfactorily. for example. s o that y o u w i l l s e e w h e r e y o u are m o s t l i k e l y t o p r o d u c e r e s u l t s that are new. and importance. T h e m a i n thing t o d o i n l o o k i n g for a t o p i c i s t o r e a d . T h e r e s e a r c h e r s h o u l d then gather fresh data . T h i s is something for w h i c h no rules c a n be laid d o w n . T h u s . or at least point up w h e r e that w o r k p r o d u c e d m i s t a k e n results. then uses that evidence to affirm the theory. In selecting a topic. theories about e x e c u t i v e c o n g r e s s i o n a l relations. DEVELOPMENT OF A RESEARCH DESIGN It m a y be true. It is an art. c a n affect theories about p o w e r in C o n g r e s s . and then carried out a study of the House Appropriations Committee to test the theory. By c h o o s i n g to study political s c i e n c e y o u h a v e already begun to n a r r o w the field. p i c k i n g a particular topic to r e s e a r c h is difficult. The danger in this is that any given situation has certain unique aspects. To y i e l d elegant results. whereas if a different test situation had been used. M a n y topics relating to defense or to the executive are of this sort. T h e s e h y p o t h e s e s p r e s u m a b l y are b a s e d on w o r k others h a v e done in the past.22 Chapter 2 Theory-Oriented Research Political Theories and Research Topics 23 CHOICE OF ATOPIC T h e c h o i c e of a r e s e a r c h topic is intimately b o u n d up w i t h the elegance of what c o m e s out of the r e s e a r c h effort. It is at this s t e p — s e e i n g that a p r o b l e m exists and that there is a good c h a n c e y o u c a n provide n e w insight into i t — that originality and talent are most c r i t i c a l . F i n a l l y . B e c a u s e of an exaggerated fear of ex post facto argument. to attain the three criteria for e l e g a n c e : s i m p l i c i t y . T h e r e is no difficulty in this. and y o u certainly w i l l narrow things m o r e before y o u are ready to begin. Y o u must j u g g l e all of these d e c i s i o n s around so as to get the best m i x — a topic that w i l l produce results that are as new and as elegant as p o s s i b l e . and (2) y o u want y o u r results. the r e s e a r c h e r should first frame a theory. T h i s is the c r i t i c a l d e c i s i o n in doing r e s e a r c h . Or it m a y be that a topic interests y o u not b e c a u s e it deals w i t h the most p r e s s i n g p r o b l e m y o u c a n think of. Y o u s h o u l d r e a d s o that y o u are c e r t a i n y o u are p i c k i n g a n i m p o r t a n t p r o b l e m . Ex post facto argument results when an investigator forms a theory on the basis of certain evidence. as I say. but in theory-oriented r e s e a r c h it is harder to d e c i d e h o w important the results of a study are l i k e l y to be. y o u s i m p l y f o l l o w y o u r interests. theories about elite political behavior. those parts of the theory would have been found wanting. If the same situation is then used to test the theory. Y o u w i l l r e c a l l that if theory-oriented r e s e a r c h is to be important. the topic s h o u l d be one that deals w i t h a p r e s s i n g p r o b l e m and one on w h i c h y o u think y o u are l i k e l y to c o m e up w i t h findings that are both accurate and s i m p l e enough to be u s e f u l . T h i s task requires not o n l y that she be f a m i l i a r w i t h as broad a range of existing theories as p o s s i b l e . to be "new." the r e s e a r c h results must either p r o d u c e totally n e w theories or lead to s o m e c h a n g e in the status of older theories. C h o o s i n g a topic that w i l l p r o d u c e important results for theory is undoubtedly m o r e difficult than formulating a question that m a y y i e l d important practical applications. D e c i d i n g w h e r e y o u are l i k e l y to produce theoretical results that are s i m p l e Engineering Research C h o o s i n g a topic is s o m e w h a t s i m p l e r in e n g i n e e r i n g r e s e a r c h than it is in theoryoriented r e s e a r c h . T h e important thing in c h o o s i n g a topic is to p i c k one that s h o w s p r o m i s e of g i v i n g y o u n e w and elegant results. 4 and predict a c c u r a t e l y requires the s a m e sort of g u e s s i n g as in engineering r e s e a r c h . T h i s m e a n s that in f r a m i n g any topic for r e s e a r c h . y o u s h o u l d r e a d t o see w h a t other w o r k h a s b e e n d o n e o n the p r o b l e m . it is p r i m a r i l y a question of u s i n g y o u r time and talents efficiently. or indirectly through the variety of other theories it affects. for e x a m p l e . predictive a c c u r a c y . B u t o n c e y o u h a v e c h o s e n a general area to w o r k i n . B u t it is certainly something for w h i c h rules have been l a i d d o w n . T h i s i m p l i e s two things: (1) Y o u want t o f o r m u late y o u r topic question so that y o u r results w i l l be l i k e l y to alter existing o p i n i o n on a subject. of c o u r s e . 4 A c c o r d i n g to this procedure. and these are likely to be included in any theory based on it. T h e r e is no s e n s e in w a s t i n g y o u r time r u n n i n g over ground that has already been w o r k e d u n l e s s y o u think y o u are l i k e l y to d i s c o v e r d i s c r e p a n c i e s . and so o n . y o u are i n v o l v e d at o n c e in the full body of political s c i e n c e theory. it s h o u l d have a broad and general effect on theory. the first step is to c h o o s e a general area that is interesting and significant for y o u . as m u c h as p o s s i b l e . but that she also have s o m e i d e a of w h e r e an existing body of r e s e a r c h is weakest and most needs to be supported or c h a n g e d . If a political scientist formed a theory of congressional committees on the basis of intimate experience with the House Appropriations Committee. stating it in the f o r m of a set of h y p o t h e s e s to be tested. this would be ex post facto argument. R e s e a r c h on h o w a c o n g r e s s i o n a l c o m m i t t e e reaches its d e c i s i o n s . a n d y o u s h o u l d r e a d t o f i n d out h o w l i k e l y i t i s that y o u r t o p i c w i l l y i e l d u s e f u l r e s u l t s . T h i s effect c a n be a c h i e v e d either directly through the p h e n o m e n a it e x p l a i n s . y o u w i l l w a n t to state y o u r thesis so that y o u r results w i l l not duplicate an earlier study. H e r e . S i m i l a r l y . It is a l s o the most difficult aspect of r e s e a r c h to teach a n y o n e . for a single p i e c e of r e s e a r c h m a y s i m u l t a n e o u s l y affect m a n y different theories. what the r e s e a r c h e r in this area must do is to decide w h i c h r e s e a r c h topic is going to p r o d u c e the greatest c h a n g e in the status of e x i s t i n g theories. O n e difficulty in c h o o s i n g the topic is that y o u probably w i l l have to c o m p r o m i s e a m o n g y o u r g o a l s . general theories about c o m m i t t e e s and organizations. s o m e s o c i a l scientists h a v e developed a very restrictive p r o c e d u r e to serve as a standard in c a r r y i n g on r e s e a r c h .

S . T h e y p l a y w i t h data. 2 0 0 5 (billions of $) Defense Spending as Percentage of G r o s s National Product. and test them w i t h s o m e data.2 1. 3. general category of causal relationships (that is. E v e n in c a s e s w h e r e a variety of previously existing evidence favors a particular theory. C o m p a r e d with the Seven Smallest M e m b e r s of N A T O G r o s s National Product. F i n a l l y . then gradually l o o s e n i n g up as e x p e r i e n c e is gained. and we try to think of an explanation to account for it. w h i c h historically have armed heavily because of their c o n f l i c t s w i t h e a c h other. contribute a n y t h i n g n e a r as h i g h a p e r c e n t a g e of their g r o s s n a t i o n a l product t o the c o m m o n a l l i a n c e a s the U n i t e d States d o e s . In doing s o . F u r t h e r m o r e .24 Chapter 2 Political Theories and Research Topics 25 w i t h w h i c h to test the theory. argue w i t h c o l l e a g u e s . O n e of the betterkept secrets in p o l i t i c a l s c i e n c e is that g o o d p o l i t i c a l scientists g e n e r a l l y do not d r a w up r e s e a r c h d e s i g n s before they start to w o r k on a topic. To test whether the broader theory is valid as an explanation.6 1. c o n s i d e r the p u z z l e that the U n i t e d States h a s a l w a y s c o n tributed p r o p o r t i o n a l l y m o r e than a l m o s t all other m e m b e r s o f the N A T O m i l i t a r y a l l i a n c e . e v e n the e v i d e n c e that suggested a theory to t h e m in the first p l a c e . we put it in a broader. It is true that this procedure erects f o r m i d a b l e barriers to protect us f r o m ex post facto argument. He needed to find a topic for his dissertation and he thought that a good w a y to do this w o u l d be to look through a b o o k l a y i n g out theories of elections. it m a y m a k e sense for t h e m to w o r k w i t h s p e c i f i c goals in m i n d so that they c a n estimate a c c u r a t e l y at the b e g i n n i n g of a project w h e n it w i l l be c o m p l e t e d . theories that are elegant and give a n e w slant to things. we draw other specific predictions from the theory and test these to see whether the theory's predictions are generally true. A l s o . Something in our lives puzzles us. The Military Balance. To account for it. I hasten to add that it s h o u l d not be rejected completely. p i c k a f e w propositions about voting behavior. 2.4 3. but m o s t of us h a v e better sense than to f o l l o w our o w n precepts.0 1. this approach encourages the researcher to function as a clerk. T h i s is h o w we train p e o p l e to do r e s e a r c h . e s p e c i a l l y a s c o m p a r e d w i t h the s m a l l e r m e m b e r s o f the a l l i a n c e . that evidence p r e s u m a b l y is to be ignored if the new test gives contradictory results. a s s e e n in Table 2-2. and the C o n s t r u c t i o n of Theories O n e w a y to l o o k at c h o o s i n g topics and developing theories is w i t h the realization that they are at heart v e r y c o m m o n s e n s i c a l p r o c e s s e s . T h e y are m e r e l y supposed to react to old theories and concepts rather than to think up e n tirely new problems for explanation. it lends an exaggerated significance to the results of the new study. A s a n e x a m p l e . in w h i c h s c h o l a r s frequently deal w i t h abstractions (and w i t h e a c h other's rival abstractions). B e c a u s e students u s u a l l y operate under stricter deadlines than other r e s e a r c h e r s . It offers researchers no encouragement to think about their theories once research has begun. . it is m o r e m e t h o d i c a l and easier to apply for the b e g i n n i n g researcher. P u z z l e s . learning more about the subject. B e c a u s e it requires that hypotheses be fixed firmly at the beginning of the research p r o c e s s . although they u s u a l l y h a v e s o m e operational h u n c h e s about w h a t they e x p e c t t o f i n d . In the first p l a c e .6 0. Defense Spending. they grope for interesting theories. O n l y G r e e c e and Turkey. B u t at heart all theory-oriented research in the s o c i a l s c i e n c e s is of the f o l l o w i n g sort: 1. O n e of its advantages is safeguarding against ex post facto argument. 2006.7 United States Turkey Belgium Norway Denmark Greece Portugal Luxembourg 12. 2 0 0 5 4. N o r do they u s u a l l y " f r a m e h y p o t h e s e s " in any f o r m a l s e n s e before they start to w o r k . P e r h a p s a good w a y to learn is by starting w i t h the m o r e c l e a r and obvious p r o c e d u r e s . it effectively reduces the research task to a selection of obvious hypotheses. and think.6 1. T h e epitome of this type of thinking is the research design—a c o m m o n student exercise in w h i c h students are instructed to frame s o m e hypotheses (presumably based on their reading) and s h o w h o w they might gather data to test those hypotheses. the r e s e a r c h e r s h o u l d either reject it or e n s h r i n e it solely on the basis of those new data. S e c o n d and m o r e important. A s e a r c h of articles in p o l i t i c a l s c i e n c e j o u r n a l s w i l l turn up o n l y a f e w that report r e s e a r c h that f o l l o w s the r u l e s . the u s u a l procedure deters researchers f r o m casting about creatively for research topics and theories. A l t h o u g h I h a v e c o n d e m n e d the f o r m a l p r o c e d u r e for d e s i g n i n g r e s e a r c h . O b s e r v a t i o n s . T h i s is often lost in the forest of s c h o l a r s h i p .0 3. but it has a n u m b e r of serious d r a w b a c k s . A n d they m o s t c e r t a i n l y do not i g n o r e older e v i d e n c e . a theory). the theory qualifies as a plausible explanation of the thing we are trying to explain. it is h a r d to teach s o m e o n e to grope for interesting topics and theories. If they are.500 363 297 295 258 226 184 37 Source: International Institute for Strategic Studies. A doctoral candidate w h o m I o n c e talked w i t h s e e m e d to me the perfect e x a m p l e of repeated exposure to e x e r c i s e s s u c h as these. i m m e r s e t h e m s e l v e s in w h a t other people h a v e written. In short. T h e other s m a l l a l l i e s all appear to v a r y i n g degrees to ride on the coattails of the T A B L E 2-2 U . T h e i r procedure is m u c h l e s s f o r m a l than the one they p r e s c r i b e for students. h a v i n g tested the theory. b a s e d on our daily e x p e r i e n c e s . R e s e a r c h e r s are not supposed to r e m o l d theory as they go along. e v e n though the r e s e a r c h d e s i g n undoubtedly stifles initiative and creativity.

In this example we might examine chambers of commerce to see whether the biggest merchants in town usually carry most of the freight. These characteristics. Since the stakes are higher for democratic leaders. most political scientists consider the democratic peace to be an enduring relationship of quite unusual regularity. Given reasonably good information about each other. let me review for you a puzzle currently in real play. The structure of argument discussed in this section—see a puzzle. it is unlikely that either of two democrats involved in a conflict will think that he or she has a preponderant chance of winning a war. a relatively insignificant member will see that the group would do about equally well whether or not he contributes. If the theory holds up well across a variety of such tests. he was ousted only later by direct action of American troops in 2003. which was discussed in pp. testable predictions from the theory to see whether it is generally valid. Accordingly. as the diffuse scientific "conversation" has proceeded. devise other unrelated predictions from the general principle in order to test it—is presented skillfully by Charles A. and in democracies the wishes of the broad populace count for more than in nondemocracies." Lest working out a theory or a puzzle should seem too easy or too pat from these examples. Lave and James G. That is. . the alliance languishes. if the United States does not contribute vigorously. In the late nineteenth century.] above. A lot of the political scientist's creativity will then come into play in devising other. is low. but little Denmark hardly makes a difference one way or another. As one example. so it is not surprising that this has attracted the attention of some very good minds. In the NATO example. These were not wars waged on democracies. It is extremely rare in the social sciences for any relationship to be as invariant as this one. and will tend to sit back and be a free rider. because the democratic leader has to reward more people. a member who sees that the group would fail without her contribution will come through strongly. 3. such as respect for individual liberties and a high value on competition. (1999). A great deal of attention has been directed over the past decade to the so-called democratic peace—the observation that no democracy has ever been observed to initiate war with another democracy. a number of scholars have questioned whether the observation is as infallible as it is claimed to be. a leader must satisfy a fairly large number of people in order to stay in power. and the relationship is also of great importance in the assessment of democracy. They also share an international norm of compromise and bounded resolutions to conflicts. One way to explain this would be to treat it as a specific instance of a more general relationship—that in any voluntary cooperative group the member with the greatest resources always tends to make disproportionate contributions. many subtly nuanced theories have been developed. was that a case of democracies attacking another democracy? Similarly. if in both of them the people's concerns count for a good deal. Since no country is a pure democracy. People generally do not want to fight wars. did not prevent these countries from entering into warfare rather happily. A democratic leader needs a higher ratio of potential benefits to costs to justify going to war than an autocrat does. a number of scholars point to the institutional structure of democracies as the explanation. many scholars argue that democracies share a number of values in common. This observation.S. "war" is not always a black-and-white concept. Both "democracy" and "war" are subject to a certain amount of interpretation. especially in the first three chapters.) Therefore. democratic leaders will The theory used in this example derives from the broader theoretical structure of Mancur Olson's The Logic of Collective Action (1965). make democracies unlikely to wage war on those with whom they agree in so many areas.) It is hard to imagine a democratic leader losing a war and not being voted out of office. how much democracy is enough to have the country count as democratic? When the NATO alliance attacked the Yugoslavian government of Slobodan Milosevic in 1998 (a government which had been elected. for instance—or a single party. poses the puzzle: Why is it that democracies do not attack one another? There have been two primary explanations offered. it will be a plausible potential explanation for the "Denmark problem. 2. they will be more careful about entering wars than will autocrats. but also benefits if won.26 Chapter 2 Political Theories and Research Topics 27 United States. (See [1. the stakes are higher for democratic leaders. 6 fight them more determinedly. leaders typically need satisfy only a small number—the army. Central Intelligence Agency gave covert support to a military coup against democratically elected President Allende of Chile. for instance. and each will know that the other (because it is a democracy) will fight very determinedly if there is a war. When the U. A war involves costs. to show the uncertainty and the progression of steps by which scientific discussion of a puzzle usually proceeds in real life. Therefore. T h e discussion that follows draws heavily on the useful review of the democratic peace literature in Bueno de Mesquita et al. the chance that two democracies will enter a war. Bueno de Mesquita and his coauthors propose instead a slightly complicated explanation based on rational choice theory. the countries can be expected to try to negotiate their disputes rather than go to war over them. and once involved in wars. then. March in Introduction to Models in the Social Sciences (1975). but it was certainly the case that democratic values. 4. and the influence of "the neople" in these democracies. in an autocracy. continuing in office after he lost the first Gulf War in 1991. but in an election without much real competition). it is said. And in the early twentieth century. Alternatively. 6-7. 1. was that war? 5 6 Granting the gray areas of definition. Or we might look at trade union-supported political parties such as the Labour Party of Great Britain to see whether the largest unions carry a disproportionate share of the burden. First of all. (1999) point out that both of the broad explanations are hard to reconcile with the fairly strong willingness of democracies to initiate wars of conquest or empire. But beyond these two broad positions. An autocrat might lose a war yet stay in power (Saddam Hussein of Iraq was an example. democratic France and Great Britain frequently sent armed forces to help acquire additions to their empires. however. Bueno de Mesquita et al. First. the democratic United States enthusiastically engaged Spain in a war intended to lift some parts of the Spanish Empire such as Puerto Rico and the Philippines. In a democracy. frame an explanation based on a more general principle.

Most researchers can expect to carry on any or all activities at any time. . or Emile Durkheim. presidency. a study of presidents' political personalities (Barber. how simple they are (remember. But also. fitting these theories to the real world to see how accurately they predict things. that this rule is not something absolute.28 Chapter 2 Political Theories and Research Topics 29 This is a good explanation. 8 'Needless to say. One person may be hit on the head by an apple and form a theory of falling apples. Scholars involved in developing theory form a kind of priesthood—admittedly sometimes run less on faith and more according to the laws of laissez-faire and caveat emptor—focused on the common goal of perfecting elegant theories. your work will again be more significant than if the theory already had a good deal of confirming evidence. original theories" that are also elegant are hard to come up with. A broader range of theory is aimed at in studies of the U. 8 MACHIAVELLIAN GUIDE TO DEVELOPING RESEARCH TOPICS There are really no guidelines that I can give you for developing a research topic other than to remind you once again that you are working toward results that are both new and elegant. the greater your contribution will be. but it certainly has not represented the last word in political scientists' collective effort to work out an explanation of the puzzle. them. In this case. the sources of executive power. for instance. or an analysis of presidents' leadership (Greenstein. and modifies the meaning of their work. so as to resolve the apparent contradiction. and so on. for any phenomenon can be examined at different levels of generality. This is mixed. and feeding the results of such research back into the body of theory. Remember. Implicit in this chapter is the view that scholarly research represents a loose cooperative effort among many people. For one thing. A still broader range of theory is seen in studies that use the U. a study of the nature of the presidential office (such as Koenig. presidency as an example of sovereigns in general and seek to explain the sources and limitations of sovereign power. either by confirming them or by working changes in 7 2. though. the way in which presidents' personalities can influence their behavior in office. It then becomes the task of the scholar to decide among them on the basis of how broadly they apply. however.S. His conclusions led to the rejection of the traditional "interest in politics" measure. If empirical research is motivated by a desire to affect the state of theories. Perhaps if we view the development task from the perspective of political research in general. original theory yourself. which may analyze the nature of the office. 1968). and a third may relate the new evidence to the body of older theory. derived in just the way that I have outlined here. Perhaps the best way to use the strategy of picking a weak theory is to state a new. another may do a descriptive study of a partcular case. you have a greater probability of refuting a weak theory. The physical activity of the "study" is the same in both cases. The researcher in such a biography generally is concerned only with explaining what happened during a particular president's life. if your research does confirm the theory. though. another may have the same experience and form a theory of universal gravitation. Richard Neustadt's Presidential Power and the Modern Presidents (1991).S. Note. This anomaly led Tarrow to probe more deeply into what political involvement means to the French. This is basically a restatement of the first criterion for elegant research. with a sense of pride in being part of an ongoing tradition. 2000). or Robert Putnam. and so on. I have said that what is produced is a plausible potential explanation. Tarrow was struck by the fact that although French peasants regularly responded to surveys by stating that they were not very interested in politics.) Note that throughout this discussion of puzzles and explanations. you must: Maximize the generality of the theory you intend to examine. As an example from political science research. Another way to follow this strategy is to pick an anomaly—that is. The weaker the previous confirmations of a theory have been. a question on which previous research has been contradictory. I mentioned earlier the pleasure that researchers feel in creating something that no one has seen before. and any evidence you can buttress them with will be important. consider the variety of studies done on the presidency. we will gain some clue as to its place in the entire scheme. however. especially during his term in office. /. A good example of research stimulated by an anomaly is the study by Sidney Tarrow (1971) of the political participation of French peasants. One person may work simply at clarifying theories. you will be doing your best job when you maximize your effect on theory with a given investment of time and money. Of course. the celebrants carry on this process by developing new theories and adapting old ones. One's work is something brand new. But the basic building blocks of political explanation are plausible potential explanations. but it also draws on Karl Marx. The narrowest range of theory is found in biographies of particular presidents. often operates at this level of generality. a theory that is as complicated as the reality it is meant to explain has not gotten you very far). From this description of the process of research. the difference lies solely in the level at which the researcher works. To do this. your hypotheses are necessarily in need of proof. it is not quite as neat as this. Pick a weak theory to work on. they also regularly turned out to vote in greater numbers than did the urban French. we can derive a set of rules to guide the individual researcher. 1972). a given person usually does not handle all these aspects of a particular problem. As we have seen. that "new. (A good exercise for you would be to frame rebuttals of the rational choice explanation from the standpoint of the first two explanations—the "shared values" explanation and the "institutional role of the people" explanation. They also shed new light on F o r example. It typically is the case that more than one plausible potential explanation can be produced for anything needing explanation. how accurately they predict.

W h a t sort of theory might y o u d e v i s e to e x p l a i n t h i s ? B e s i d e s anomalies. Arthur A. work in normative philosophy or in formal theory could be evaluated in terms of elegance. Presumably. 4. I r e c o m m e n d W i l l i a m K n o w l t o n Zinsser's On Writing Well: The Classic Guide to Writing Nonfiction ( N e w Y o r k : 1998).k n o w n theory. the m o r e l i k e l y they w e r e to vote for G e o r g e W. On this hinged the apparent contradiction. H o w to write w e l l and d e s i g n g r a p h i c d i s p l a y s w e l l are b e y o n d the scope of this book. 1983). Fortunately. Although this is the usual procedure. is that the context is different. A n o m a l i e s s u c h as this are hard to c o m e by. T h i s m e a n s . : G r a p h i c s P r e s s . S o m e questions y o u might c o n s i d e r are: 1. these guidelines s h o u l d remain flexible enough to allow different m i x e s of research strategy. If y o u c a n find an anomaly having to do with a significant area of political theory. broader.30 Chapter 2 Political Theories and Research Topics 31 O n e person m a y find a tool that measures a variable better than had been done before and then s i m p l y apply it to sharpen previously e x a m i n e d relationships. Present your theory as clearly and vividly as possible. perhaps one in w h i c h all variables have not been covered. N e e d l e s s to say. A Machiavellian researcher wants to i n f l u e n c e as m a n y people as p o s s i b l e . just as empirical research is. rather c o n t r o v e r s i a l ideas about appropriate w a y s to build and test theory is R o n a l d B r u n n e r ' s " A n ' I n t e n t i o n a l ' Alternative i n P u b l i c O p i n i o n R e s e a r c h " ( 1 9 7 7 ) . A n o t h e r e x a m p l e of a n i c e a n o m a l y j u s t waiting to be a n a l y z e d is the fact that in the 2 0 0 0 e l e c t i o n . elegant theories is found in the first three chapters of L a v e and M a r c h ( 1 9 7 5 ) . People often think how y o u say s o m e t h i n g is separable f r o m what y o u say. y o u might replicate a study in a different context from the original one. G o l d s m i t h found that examination of these c o u n tries did not support O l s o n ' s theory. so it m a k e s sense to m a k e y o u r reader's life e a s i e r and y o u r m e s s a g e more c o m p e l l i n g . I f the purpose o f theory is to c h a n g e p e o p l e ' s understanding of the w o r l d . T u f t e ' s The Visual Display of Quantitative Information ( C h e s h i r e . What drawbacks might it involve? In what alternative ways might one develop a theory? 3. A n o t h e r perspective i s offered persuasively by J. it is neither the only nor necessarily the best approach. B u t if y o u look at the " r e d s t a t e s " — t h e states he c a r r i e d — they are on the w h o l e poor states w i t h low average i n c o m e s . E = mc . Make the connection between the general theory and your specific operations as clear as possible. FURTHER DISCUSSION A n intriguing b o o k relevant to this chapter is A r t h u r K o e s t l e r ' s The Act of Creation ( 1 9 6 9 ) . though he hastened to add that his measure of "instability" w a s questionable and might have been a source of the disparity. for e x a m p l e . B u t e a c h o f the r u l e s . because earlier investigators generally have noticed them already and have tried to resolve them. An article that presents s o m e interesting. or simpler interpretation on them. at least to my taste. T h u s . Y o u m a y have noticed that these three rules r e s e m b l e the criteria for elegance fairly c l o s e l y . M u c h of the rest of this b o o k f o c u s e s on s u c h p r o b l e m s . Donald Moon (1975). T h e most likely reason. A n o t h e r m a y note an a n o m a l y in a theory and organize an experiment to resolve the problem. that prolonged political stability produces rigid social and e c o n o m i c structures that retard e c o n o m i c growth. y o u c a n be certain that any plausible efforts at resolution w i l l be interesting. also has a b e n e f i c i a l effect on the field as a w h o l e . What would a noncausal theory look like? Under what circumstances would it be likely to be used? {Hint: Consider Einstein's famous theory. F o r w r i t i n g . w a s based mostly on the experiences of the West and Japan. the entire field w i l l benefit f r o m an e x a m i n a t i o n of those theories m o s t in need of investigation. but that is s i m p l y not true. O l s o n ' s theory relies on the negative e c o n o m i c effects of organized interest groups. I c a n suggest two truly good b o o k s that w i l l help y o u . In fact. If i n d i v i d uals c h o o s e those p r o b l e m s of theory that have so far h a d the w e a k e s t v e r i f i c a t i o n . then the w a y the theory is c o m m u nicated to them is an integral part of the development of the theory. G o l d s m i t h ' s negative results m a y thus point up the importance of organized interest groups in O l s o n ' s theory. however. It i n v o l v e s s u c h things as the a c c u r a c y of y o u r deductions f r o m the theory to the s p e c i f i c situation. I stated in this chapter that most social science theories are causal. the a c c u r a c y with w h i c h y o u have m e a s u r e d things. e a c h r e a l l y requires a b o o k in its o w n right. A n e x c e l l e n t introduction t o b u i l d i n g 3. y o u might choose a problem y o u believe has just not been sufficiently researched. G o l d s m i t h (1987) noted that M a n c u r O l s o n ' s (1982) w e l l . T h i s really j u s t boils d o w n to m a k i n g sure y o u say w h a t y o u think y o u are s a y i n g . This chapter has implied that the usual way to come up with a theory is to focus on a body of observations and look for regular patterns in them.) . A third m a y look over previous research findings and place a new. B u s h . T h o u g h they were in fact interested in politics. but not in the T h i r d World context that G o l d s m i t h studied. What changes would this require in the definition of elegance? 2. An excellent introduction to good graphic presentation HarperCollins. is E d w a r d R. the h i g h e r voters' i n c o m e s w e r e . since F r e n c h peasants were fiercely independent and understood "interest in politics" to i m p l y some sort of approval of the existing parties. write w e l l and present any g r a p h i c information w e l l . C o n n . and so o n . S u c h groups are strong and active in the countries O l s o n studied. d e r i v e d f r o m m y u n d e r l y i n g M a c h i a v e l l i a n outlook. Y o u also m a y have noticed that the b a s i c p h i l o s o p h y b e h i n d t h e m — " D o r e s e a r c h that m a k e s as big a s p l a s h as p o s s i b l e " — r e a d s like a guide for ruthless and h u n g r y assistant p r o f e s s o r s . T h e r e is no one "scientific method" involved here. A l l are following the rule of m a x i m i z i n g their impact on theory. T h e T h i r d W o r l d w o u l d provide a laboratory of less-developed countries with a wider range of cultures and e c o n o m i c circumstances. the nature of F r e n c h voters' attachment to political parties. they denied this because of their antipathy to the parties.

that is. W h e n r e s e a r c h e r s write about "the political party. but failed to get it p a s s e d — a n d t h e w r i t e r is in t h e o p p o s i n g party. T h i s c o n c e p t c a n be b r o k e n d o w n into at least six c o m p o n e n t d i m e n s i o n s : those people w h o vote for the party in a given election. the " g o o d n e s s " of weather varies. Just to look at one of its dimensions. the failure c o u l d have resulted f r o m any n u m b e r of other events. they m a y m e r e l y s u c c e e d in creating c o n f u s i o n . T h e w o r d liberal. some of w h i c h are probably even inconsistent. In this c h a p t e r I s h a l l stress the n e e d to state t h e o r i e s i n a c l e a r . for instance. but not as hard as the writer would have done. T h e a d m i n i s t r a t i o n w a n t e d t h e bill." "campaign for the bill. for instance. the E n g l i s h language is b a d l y d e s i g n e d to function as a m e d i u m for stating r e s e a r c h results c l e a r l y and without ambiguity. is a m u l t i d i m e n s i o n a l concept that involves a m o n g other things temperature. If the theory stated that " b a d weather" c a u s e d the spotted ibex to stop breeding. those people w h o are registered with the party. T h e r e f o r e . in contrast to concepts w h o s e meaning m a y vary f r o m reader to reader and even from time to time. In particular. In C h a p t e r 2. the late Senator Paul Wellstone of Minnesota 32 .Chapter 3 Importance of Dimensional Thinking ENGLISH AS A L A N G U A G E FOR RESEARCH 33 I have a s s u m e d so far that if p o l i t i c a l r e s e a r c h is to be u s e f u l . Is it rain that discourages the i b e x ' s ardor? Or do high w i n d s m a k e h i m think of other things? Is it the heat? T h e h u m i d i t y ? As we c a n see f r o m this e x a m p l e . " G o o d weather." S i n c e liberal has generally unpopular connotations in the United States and is multidimensional in complex w a y s . M o s t E n g l i s h w o r d s c a n take on a variety of m e a n i n g s . In this chapter we shall look at the E n g l i s h language in general and at the varied theories that m a y be hidden in ordinary language. c o n s i d e r political party. 4 . f r o m hot to c o l d . In the election of 1996. A w e l f a r e bill w a s p a s s e d . humidity. If there is a variation along any one of these separate d i m e n s i o n s . 2. conservative. Temperature c a n vary along only one d i m e n s i o n ." might m e a n : 1. 6. T h e a d m i n i s t r a t i o n p r i v a t e l y p a s s e d t h e w o r d t o its s u p p o r t e r s t o u n d e r c u t the bill. u n a m b i g u o u s w a y — a f o r m o f s e l f . T h e E n g l i s h language does not a l w a y s m a k e this easy. 5. multidimensional concepts. but n o t t h e o n e the w r i t e r w a n t e d . we shall be concerned with framing theories in terms of concepts that cannot easily be subdivided into further distinct meanings. it is natural to use this flexible w o r d as a club in describing candidates. economic development. the officers of the party. T h u s . and those w h o are r u n n i n g or elected as candidates under the party l abel . No w e l f a r e bill p a s s e d at all. very simple concepts w h o s e meaning is unequivocal. those people w h o do voluntary w o r k for the party at s o m e g i v e n time. however. f r e e d o m — a r e i m p o r t a n t in d o i n g g o o d r e s e a r c h . m mkM Unfortunately. w i n d velocity. if a theory states that a temperature b e l o w x degrees c a u s e s the spotted ibex to stop breeding. C o n s i d e r the f o l l o w i n g e x a m p l e : As a description of c l i m a t e s . Note that in these statements there are many additional phrases that are ambiguous: "pass the word. T h e a d m i n i s t r a t i o n c a m p a i g n e d as h a r d as it u s u a l l y d o e s for bills it w a n t s . that statement w i l l mean the s a m e thing to every reader." they m a y be referring to any or all of these v a r i o u s d i m e n s i o n s of the term. either denotative or connotative—in other words. amount of precipitation. I shall c a l l such simple concepts unidimensional. u n l e s s p o l i t i c a l scientists s p e c i f y w h i c h d i m e n s i o n s apply. the statement. those people w h o identify t h e m s e l v e s in their m i n d s w i t h the party. and degree of c l o u d cover. has many dimensions." "go all-out." although in its original usage." "want the bill. " T h e w e l f a r e b i l l failed in the H o u s e because the administration did not fight for it. w h i c h is still fairly current in E u r o p e . a m i n i m a l requirement is that its results s h o u l d m e a n the s a m e thing to any two different readers. "temperature" is a u n i d i m e n s i o n a l concept.d i s c i p l i n e that m u s t a c c o m pany such freedom. but did not c a m p a i g n a s h a r d a s i t u s u a l l y d o e s for bills it w a n t s . Debates in political science sometimes resolve themselves into the fact that participants are using the same term in different w a y s . the m e a n i n g of that theory w o u l d be left to the reader's judgment. A f e w further e x a m p l e s of m u l t i d i m e n s i o n a l terms are: power. it is preferable to frame t h e o r i e s i n terms of u n i d i m e n s i o n a l rather than multidimensional concepts. intelligent." on the other h a n d ." As another e x a m p l e of a m u l t i d i m e n s i o n a l concept. I a r g u e d that f l e x i b i l i t y a n d o r i g i n a l i t y — i n a w o r d ." "undercut the bill. and love. for e x a m p l e : 3 . To use an e v e r y d a y e x a m p l e . it means "favoring a limited role for government. enjoyment. T h e a d m i n i s t r a t i o n w e n t all-out for t h e bill. W h i c h e v e r of these is the c a s e . depending on the context and on the m o o d of the reader. it often connotes "free-spending.

For example. it is a p r o c e s s that g i v e s us little c o n f i d e n c e in the n e w t h e o r i e s that are p r o d u c e d . framing theories in terms of unidimensional concepts. s e r v e t o s l o w d o w n the p r o c e s s of c h a n g e . the better one understood things. it has implied the theory that the older (and thus the more experienced) one got. r e f l e c t i n g different v i e w s o f the w o r l d . Difficulty with measurement. it is the poet's j o b to create a rich confusion of varied nuances." I t i s i n j u s t s u c h a w a y that w o r d u s a g e g r a d u a l l y c h a n g e s . 2. however. India. and a p e r s o n ' s i n f l u e n c e is m o r e or l e s s p r o p o r t i o n a l to the n u m b e r of p e o p l e w h o h e a r her u s e the w o r d . T h u s . we pay a high price for this benefit. (2) a highly developed understanding of relationships. exchanges of mail. M u l t i d i m e n s i o n a l concepts are valuable and useful in ordinary language. A n o t h e r p r o b l e m w i t h the p r o c e s s is that these c h a n g e s often p r o c e e d at a s n a i l ' s p a c e . "interaction" involves a number of dimensions: alliances. Among other things. the t e r m liberal c a m e to denote their p r e s e n c e a n d conservative their a b s e n c e . for e x a m p l e . B e c a u s e s u c h attitudes as a d e s i r e for e c o n o m i c a c t i v i s m o n the part o f g o v e r n m e n t . ) T h e i m p l i e d t h e o r y — t h a t these attitudes tended to c o i n c i d e — w a s n e v e r v e r y a c c u r a t e as a d e s c r i p t i o n e v e n of the elite a n d 'Today. Ambiguous associations. they confront an impossible task unless they have defined the variable in such a way that it consists of a single dimension. T h e j o b of the social analyst. change as c o m m o n u s a g e r e f l e c t s c h a n g e d m o o d s a n d n e w e x p e r i e n c e s . w h i c h are so important to art. B. T h e r e is no p r o v i s i o n m a d e for greater i n f l u e n c e o n the p r o c e s s b y those w h o k n o w m o r e about the s u b j e c t . ( T h e w o r d s w e r e b o r r o w e d f r o m a E u r o p e a n c o n t e x t . Ordinary Language A s w e have j u s t seen. b y c o d i f y i n g w o r d u s a g e . Poor communication. B and C exchange a great deal of mail but do not trade together as much as A and B. D i c t i o n a r i e s . it is a poorly articulated and poorly controlled theory. A poet is pleased if she chooses a w o r d w h o s e meanings give the reader pause for thought. and these component d i m e n s i o n s are rarely specified in ordinary language. We return to such problems of measurement in Chapter 4. but they do not affect the quality of the c h a n g e . followed by A and B and by B and C. It m a y be an u n a r t i c u l a t e d d i s l i k e for the theory i m p l i e d by the w o r d wise. T h e polite substitution of the t e r m developing nations for underdeveloped countries in the 1 9 5 0 s a n d 1 9 6 0 s i s another e x a m p l e . Suppose that this political scientist discovered the following things about countries A. p a c i f i s m . Importance of Dimensional ininmng ->3 w a s described by his opponent as "embarrassingly liberal" for his vote against a bill reducing welfare payments. tourism. reality has caught up with rhetoric in the explosive economic growth of countries such as China. Clearly. B u t in the social s c i e n c e s . mail flow. for instance. T h e connotations accruing to a w o r d that means many different things simultaneously add richness to our language. But unless its component dimensions are made explicit. a d d e d t o the s u b s e q u e n t c o n f u s i o n . T h e w o r d s Alliance Yes Yes No liberal a n d conservative are a c a s e in point. He responded that his vote w a s actually "deeply conservative" because it reflected the moral values of Minnesotans. B u t the p r o c e s s b y w h i c h they c h a n g e is not a v e r y s a t i s f a c t o r y o n e . but their m e a n i n g s w e r e c o n s i d e r a b l y c h a n g e d . (3) a contemplative bent. By associating several distinct things with a single term. w h i c h h a s l e d to d i m i n i s h e d u s e o f the w o r d i n e g a l i t a r i a n A m e r i c a . and (4) a background of long practical experience. 1 1. a n d its d e g r a d a t i o n i n s u c h s l a n g a s " D o n ' t get w i s e w i t h m e ! " a n d " w i s e guy. R i c h connotations. volume of trade. E v e r y o n e w h o u s e s a p a r t i c u l a r w o r d takes part in the p r o c e s s . T h e strategy I am introducing here. of course. Because a reader can never be certain what meaning an author Whenever social scientists want to measure a variable. m a n y E n g l i s h words involve more than one d i m e n s i o n . A r t and rhetoric c o u l d not be restricted to unidimensional vocabularies. as is indicated by their popularity. another fault in multidimensional words is that each such word is itself a theory of sorts. can help us to keep clear the meanings of the words we use and to avoid this kind of confusion. embedded in multidimensional words. M u l t i d i m e n s i o n a l w o r d s h a m p e r s o c i a l scientists in at least three w a y s : T h e o r i e s such as this. and so on. W a s Wellstone's vote "liberal" or "conservative"? T h e argument proceeded almost irrespective of its merits. t h e o r i e s that are e m b e d d e d i n m u l t i d i m e n s i o n a l w o r d s s u r v i v e intended. if I use the word wise rather than intelligent. that A and C share the greatest trade. w h i c h o n l y Between which two countries is there greater interaction? A and C trade together a great deal but are not allies. and C: Between Countries A and B B and C A and C Volume Trade/Year $10 billion $5 billion $20 billion Items of Mail/Year 20 million 30 million 10 million p r a c t i c a l refutation m u c h l o n g e r than i f they w e r e m a d e e x p l i c i t . c o n c e r n for the legal rights o f i n d i v i d u a l s . a multidimensional word implies the theory that these things go together. consider a political scientist who wanted to measure the "amount of interaction" between various nations. . "amount of interaction" is not a unidimensional variable. We can confidently state. For example. and South Korea. and alliance—there would have been no problem in comparing the countries. Now. my reader picks up this little theory subliminally embedded in my statement. In a s e n s e it is v e r y d e m o c r a t i c — p r o b a b l y m o r e d e m o c r a t i c than w e w o u l d l i k e . the word wise has long connoted (1) broad practical knowledge. get in the w a y of analytic thinking. So. From the standpoint of the social scientist. T h i s c h a n g e i n u s a g e r e f l e c t e d the c h a n g e d status o f the T h i r d W o r l d a n d a t least s o m e h o p e f u l t h i n k i n g about w h a t w a s happening there. and i n t e r n a t i o n a l i s m often s e e m e d t o g o together w h e n o n e l o o k e d at the p o l i t i c a l elite in the U n i t e d S t a t e s . T h i s p r o c e s s of theory d e v e l o p m e n t is u n c o n t r o l l e d . is to bring order out of social chaos by means of simple theories. multidimensional words in effect hamper communication.34 Chapter 3 3. Had the investigator chosen three separate measures—trade.

" the mind does rebel. At the s a m e time. writers in any analytic field suffer the same handicap. but there does appear to be a need for s u c h a s u m m a r y term. B. T h i s p r o c e s s might not be at all e a s y to d i s c e r n if we were f o r c e d to w o r k s i m u l t a n e o u s l y w i t h the large n u m b e r of u n i d i m e n s i o n a l c o n cepts i n v o l v e d in it. they must invent the w o r d s t h e m s e l v e s . we are disturbed. Actually. T h i s is because the social scientist deals with people. but it w o u l d be a w k w a r d . Variables c a n n o t be m e a s u r e d u n a m b i g u o u s l y if they h a v e b e e n defined in a m u l t i d i m e n s i o n a l way. Of c o u r s e . T h e physicist m a y describe a body's motion in terms of " m a s s . s o that even today there are still many people who use 2 "hurtles through space. The secret is that centuries ago to "prove" meant to submit to a test. B u t w h e n the political scientist describes politics as consisting of " s y s t e m inputs. as c o m p a r e d w i t h ordinary l a n g u a g e . a m o n g the nation's people. T h e r e are a great n u m b e r of d i m e n s i o n s i n v o l v e d . T h e y m a y add clarity. 3. A reader cannot tell. with " p r o o f included in it." and "feedback loops. 2. I t t u r n s out. W h a t most people m e a n b y " s o c i a l s c i e n c e j a r g o n " i s the u n i d i m e n sional v o c a b u l a r y invented by s o c i a l scientists for their use in analytic r e s e a r c h . I n c l u s i o n of m u l t i d i m e n s i o n a l w o r d s in a t h e o r y c o n f u s e s that t h e o r y w i t h a d d i t i o n a l t h e o r i e s that are i m p l i e d b y t h e e x i s t e n c e o f the m u l t i d i m e n s i o n a l w o r d s t h e m s e l v e s . T h i s is one r e a s o n that s o c i a l s c i e n c e writing so often strikes readers as flat and c o l d . and w o r k directly with these d i m e n s i o n s . T h i s is s i m p l y one of the costs we must pay to write a n a l y t i c a l l y . . too. even though the changed meaning of the word has rendered the phrase meaningless. " N a t i o n a l integration" m a y provide an e x a m p l e of s u c h a concept. w i d e spread c o m m u n i c a t i o n and personal interchanges w i t h i n the nation. the family. An exception does not prove a rule. for e x a m p l e . S u c h explicit c o m b i n a t i o n s of d i m e n s i o n s have the advantage. that p e o p l e w h o are l i b e r a l o n i n d i v i d u a l s ' rights are often c o n s e r v a tive o n e c o n o m i c i s s u e s . or whatever. there is a w i d e s p r e a d feeling a m o n g political scientists that these things tend to go together to constitute a general p r o c e s s . w h i l e c h a n g e s in usage do p r o d u c e c h a n g e s in the theories i m p l i e d by m u l t i d i m e n s i o n a l w o r d s . Natural scientists must create their o w n vocabularies. A l l t h i s w e n t u n n o t i c e d u n t i l a t l e a s t the late 1 9 4 0 s . but w h e n social scientists try to simplify the c o m p l e x reality of politics. these contrived concepts retain m a n y of the disadvantages "Another example of meanings of words being slow to change. T h e m e a n i n g of a theory is not u n a m b i g u o u s l y clear if it is c o u c h e d in m u l t i d i m e n s i o n a l words. S o c i a l scientists s h o u l d be uncomfortable if hidden a w a y in their statement of a theory are additional little theories that are implicit a n d uncontrolled. T h e use of an explicit c o m b i n a t i o n of separate d i m e n s i o n s s u c h as this has s o m e of the advantages of both " o r d i n a r y l a n g u a g e " and u n i d i m e n s i o n a l l a n g u a g e . or that B w a s h i g h and A and C w e r e low. a f e e l i n g . and so on. for e x a m p l e . w h e n f a c e d with a h i g h score on a variable that c o m b i n e s d i m e n s i o n s A. or of protests. A l t h o u g h it is a l w a y s n e c e s s a r y to think and w o r k w i t h u n i d i m e n s i o n a l c o n c e p t s in the s o c i a l s c i e n c e s .36 Chapter 3 Importance of Dimensional Thinking 37 i t h a s p r o v e d t o b e quite i n a c c u r a t e i n d e s c r i b i n g p e o p l e i n g e n e r a l . H o w e v e r . A n d they help t o keep y o u r theory p a r s i m o n i o u s . W r i t i n g f r o m w h i c h the r i c h n e s s o f v a r i e d connotations h a s been r e m o v e d i s flat. and C." The phrase about exceptions has continued. the words liberal a n d conservative in this w a y . If ordinary language does not provide u n i d i m e n sional w o r d s for the things s o c i a l scientists want to say. the thing we care most passionately about. the p r o c e s s by w h i c h these c h a n g e s are made is c a p r i c i o u s . A l t h o u g h e x p l i c i t m u l t i d i m e n s i o n a l w o r d s are more u s e f u l than ordinary language. B u t somehow the loss of richness seems more painful in the social sciences than in other fields. of ordinary l a n g u a g e . and they m a y c o m b i n e in odd w a y s . It D e s p i t e our prescription. but it need not prevent them f r o m w r i t i n g c l e a r and graceful prose. w h e t h e r this means that A and B w e r e h i g h and C low. the word has the same root as "prove. in fact." " s y s t e m outputs. if y o u are w o r k i n g w i t h so m a n y d i m e n s i o n s s i m u l t a n e o u s l y that readers w o u l d have difficulty k e e p i n g track o f t h e m . T h e result h a s not been fully satisfactory. A c c o r d i n g l y . as if they h a d g e n e r a l v a l i d i t y . y o u s h o u l d not use them c a s u a l l y . L e t me s u m up the argument thus far: S o c i a l scientists s h o u l d use u n i d i m e n sional language for three r e a s o n s : PROPER USE OF MULTIDIMENSIONAL W O R D S I have a r g u e d that we s h o u l d try to use o n l y i n d i v i s i b l e d i m e n s i o n s as c o n c e p t s in p o l i t i c a l s c i e n c e . " "velocity. T h e s e are to be v a l u e d in their o w n right. it m a y be c o n v e n i e n t to put separate d i m e n s i o n s together in explicit m u l t i d i m e n s i o n a l c o m b i n a t i o n s . is offered by the old saying "the exception that proves the rule. too. T h e r e are many d i m e n s i o n s i m p l i e d in the term: political c o n s e n s u s w i t h i n the nation. It does not bother us w h e n a p h y s i c i s t tries to reduce the complex motion of a particle to unidimensional concepts. this is not meant to e x c u s e poor writing in the s o c i a l s c i e n c e s or a n y w h e r e e l s e . political scientists o v e r the last f e w d e c a d e s have explicitly developed the term national integration." Taken at face value—as it usually is—this phrase is idiotic. and s o m e of the disadvantages of e a c h . T h u s . we must a c k n o w l e d g e the fact that the E n g l i s h language contains many words that hold a rich variety of meanings and w o u l d be p o s s i b l e to do without the term national integration. B u t there is another side to the q u e s t i o n . to refer to the general p r o c e s s . however." and was used in phrases such as "submit to a proof by fire. but should cause us to reconsider the rule. that they all b e l o n g together." and "acceleration" rather than saying that it d i m e n s i o n s . that y o u have built y o u r o w n j u x t a p o s i t i o n o f d i m e n s i o n s into the w o r d . w h i c h l e a d s me to 1. legal integration." and the m i n d does not rebel. S u c h s u m m a r y constructs m a y add grace and interest to the presentation of y o u r results and ideas. an amusing one. this is p r e s u m a b l y not the w a y we want to d e v e l o p s o c i a l theory. temper that s t a n c e s o m e w h a t . U n i d i m e n s i o n a l language is a m i n o r h a n d i c a p under w h i c h s o c i a l scientists operate. composed of these various connotations. yet s o c i a l scientists often must create their o w n vocabulary.

for i n s t a n c e . If one c o u l d c o n v i n c e white w o r k e r s in N e w H a v e n that the m o s t important thing to them w a s education for their c h i l d r e n rather than defending w h i t e s ' prerogatives against b l a c k s — o r i f y o u d i d the reverse o f t h i s — y o u w o u l d h a v e already pretty w e l l determined the p o l i t i c a l o u t c o m e s . H o w e v e r . for i n s t a n c e . K r o e b e r a n d K l u c k h o h n ( 1 9 5 2 ) counted several h u n d r e d different w a y s i n w h i c h the w o r d culture w a s u s e d by anthropologists. that determines a l l that f o l l o w s . T h e p o w e r any person holds in a particular political d e c i s i o n m a y be a n a l y z e d in terms of her position on the four d i m e n s i o n s . as I have i n d i c a t e d . A s a f i n a l h o o k . P e r h a p s the most frequent f a i l i n g in r e s e a r c h is that this part of the w o r k is done c a s u a l l y a n d sloppily. c a n help us to c l a r i f y the m e a n i n g of the w h o l e body of work and may point u p new directions for theoretical development. E a c h of these conceptualizations actually emphasizes a different d i m e n s i o n of power: D a h l e m p h a s i z e s whether one has participated in a decision. whether one has influenced people's political identities.38 Chapter 3 Importance of Dimensional Thinking 39 I h a v e u s e d as my " b a d " e x a m p l e s w o r d s f r o m ordinary language rather than w o r d s s p e c i f i c a l l y d e s i g n e d for u s e by s c h o l a r s . whether one has influenced people's understanding of their interests. i t m i g h t b e that a s m a l l g r o u p o f p o w e r f u l f i g u r e s h a d d e t e r m i n e d w h a t i s s u e s w o u l d b e o p e n for d i s c u s s i o n . In one study. S t e v e n L u k e s a d d e d a further c o m p l i c a t i o n . Rather. nor at that t i m e h a d there b e e n a n y interest in the r e l a t i v e s i t u a t i o n of women vis-a-vis men. T h e f o l l o w i n g a n a l y s i s of different w a y s of u s i n g the c o n c e p t power is an extended e x a m p l e o f h o w s u c h d i m e n s i o n a l a n a l y s i s might p r o c e e d . for instance. If the N e w H a v e n w o r k e r s c a n be l e d to identify t h e m s e l v e s as w h i t e s rather than as w o r k e r s . and in determining the agenda of i s s u e s f a c i n g the city but m a y not C o n c e p t u a l a n a l y s i s of this sort is terribly important to r e s e a r c h . they m a y p r o d u c e c o n f u s i o n as great as that r e s u l t i n g f r o m ordinary language. different sorts o f p e o p l e w e r e i n v o l v e d . Major Dimension Four D i m e n s i o n s of Power A Political Decision debates o f the last f e w d e c a d e s c a n b e c l a r i f i e d b y d i m e n s i o n a l a n a l y s i s . e v e n i f it is already at a quite abstract l e v e l . A c c o r d i n g l y . dimensional analysis c a n show h o w they all w o r k together. On a question of park policy. interests but rather in determining w h a t leads t h e m to define their v e r y identity. 1 9 6 1 ) . R o b e r t D a h l l e d off the m o d e r n debate o v e r the nature o f p o w e r w i t h a study o f p o w e r i n l o c a l p o l i t i c s i n N e w H a v e n . Peter D i g e s e r ( 1 9 9 2 ) drew o n the w o r k o f M i c h e l F o u c a u l t t o suggest that real p o w e r c o n s i s t s not of determining w h a t people w i l l see as in their Identity influenced did did did did did did did did did did did people's not not not not not not definition of their identities did did did did not not Interests influenced people's understanding of interests Participation participated did not did not did did not did did not did did not . Figure 3-1 Example of Dimensional Analysis T h e b a s i c b u i l d i n g b l o c k o f all p o l i t i c a l s c i e n c e a n a l y s i s i s the c o n c e p t o f power. D e s p i t e the fact that a w i d e r a n g e of p e o p l e h a d p a r t i c i p a t e d i n N e w H a v e n d i s c u s s i o n s . C o n n e c t i c u t ( D a h l . s o h e c o n c l u d e d that l o c a l p o l i t i c a l p o w e r w a s not c o n c e n t r a t e d i n a n elite but w a s w i d e l y d i s t r i b u t e d . in i n f l u e n c i n g p e o p l e ' s understanding of their interests. e a c h set up for one particular r e s e a r c h u s e . e x p l i c i t d i m e n s i o n a l a n a l y s i s o f a body o f s c h o l a r l y w o r k . B a c h r a c h a n d B a r a t z ( 1 9 6 2 ) r e s p o n d e d that the true e s s e n c e o f p o w e r l a y not i n p a r t i c i p a t i o n i n d e c i s i o n s but i n b e i n g a b l e t o l a y out the a g e n d a for w h a t q u e s t i o n s w e r e to be d i s c u s s e d . e v e n s c h o l a r l y v o c a b u l a r y is often not as tight as it c o u l d be. T r u e p o w e r lay not in the m e c h a n i c s of d e c i s i o n m a k i n g . that several s c h o l a r s w o r k i n g independently of e a c h other d e v i s e related u n i d i m e n s i o n a l w o r d s . A l t h o u g h the scholars involved in these arguments have generally v i e w e d their o w n version of power as ruling out one or more of the others. H e noted that o n different i s s u e s . It m a y h a p p e n . T h e q u e s t i o n o f the c i t y g o v e r n m e n t t a k i n g o v e r the p u b l i c utilities h a d not b e e n o n the t a b l e . whether one has helped set the agenda for public decisions. a n e w s p a p e r editor m a y have b e e n i n v o l v e d in the setting of identities. w h e n s c h o l a r s d e s i g n t e r m i n o l o g y it is a good deal tighter than ordinary language. true p o w e r l a y in the ability to define for people w h a t their interests w e r e . W h e n these related w o r d s are taken as a group. either in the d e c i s i o n itself or in setting the agenda. In 1974. L u k e s . B u t this c o n c e p t is m u l t i d i m e n s i o n a l and notoriously difficult to pin d o w n . and Digeser. F i g u r e 3-1 l a y s these out as four d i m e n s i o n s of power. B a c h r a c h and Baratz. In g e n e r a l .

40

Chapter

3

Chapter 4

have been i n v o l v e d in the actual d e c i s i o n s . A m e m b e r of the city park board m a y have been i n v o l v e d in the d e c i s i o n but m a y not have e x e r c i s e d p o w e r in any of its other d i m e n s i o n s . A n elementary s c h o o l teacher m a y have been i n v o l v e d i n the p r o c e s s o f f o r m i n g identities but m a y not have e x e r c i s e d p o w e r in its other d i m e n s i o n s ; another s c h o o l teacher, interested and active on park questions, m a y have been i n v o l v e d in the p r o c e s s of f o r m i n g identities, and m a y have participated in the d e c i s i o n s , but not have e x e r c i s e d p o w e r in the other two d i m e n s i o n s . D i m e n s i o n a l a n a l y s i s of this sort a l l o w s us to see the interrelations of v a r i o u s a p p r o a c h e s to a question and c a n a l s o give us a r i c h f r a m e w o r k w i t h w h i c h to apply a m u l t i d i m e n s i o n a l c o n c e p t . It a l s o a l l o w s us to c o m p a r e the i m p o r t a n c e of the d i m e n s i o n s . I m p l i c i t in e a c h later s c h o l a r in the d i s c u s s i o n s k e t c h e d above w a s the i d e a that h i s notion of p o w e r w a s deeper and more b a s i c than those that went before. A d i m e n s i o n a l a n a l y s i s g i v e s us a structure w i t h i n w h i c h we c a n address this q u e s t i o n .

WClLTTTOClLTl] Accuracy

FURTHER DISCUSSION

F o r m a l treatment o f d i m e n s i o n a l a n a l y s i s w a s i n t r o d u c e d b y A l l a n H . B a r t o n , " T h e Concept of Property-Space in Social R e s e a r c h " (1955). Philip E. Jacob's "A Multid i m e n s i o n a l C l a s s i f i c a t i o n o f A t r o c i t y S t o r i e s " ( 1 9 5 5 ) f u r n i s h e s a good e x a m p l e o f dimensional analysis in practice. S o m e e x a m p l e s f r o m p o l i t i c a l s c i e n c e are C h a p t e r Political Oppositions in evant dimensions Western Democracies ( 1 9 6 6 ) , 11 of R o b e r t D a h l ' s

In this chapter I e x p l o r e p r o b l e m s of accurate m e a s u r e m e n t . T h e s e are p r o b l e m s that arise w h e n we try to relate the a c t u a l operations in a p i e c e of r e s e a r c h — t h a t i s , m e a s u r e s of t h i n g s — t o the c o n c e p t s that f o r m the b a s i s of our u n d e r l y i n g theory. C o n c e p t s , o f c o u r s e , exist o n l y i n the m i n d . O n e n e c e s s a r y a s s u m p t i o n , i f w e are t o c l a i m that a p i e c e of r e s e a r c h h a s tested a given theory, is that the things m e a s u r e d in the r e s e a r c h c o r r e s p o n d to the things in the theorist's m i n d . T h i s is often a difficult a s s u m p t i o n to m a k e . In the p r e c e d i n g chapter, y o u s a w one k i n d o f p r o b l e m that c a n stand i n the w a y o f it. T h e p o l i t i c a l scientist w h o w a n t e d to m e a s u r e the a m o u n t of interaction b e t w e e n nations f o u n d that there w a s no s i n g l e satisfactory indicator of "interaction." A n u m b e r of t h i n g s — t r a d e , m a i l e x c h a n g e s , a l l i a n c e , and so o n — p a r t o o k of "interaction," but no one of t h e m alone w a s s y n o n y m o u s w i t h the mental construct. In the social s c i e n c e s , only rarely are we able to measure our concepts directly. C o n s i d e r , for example, the concepts " s o c i a l c l a s s , " "respect for the presidency," and "power in the community." A n y variables we w o u l d c h o o s e to measure these concepts correspond only indirectly or in part to our mental constructs. T h i s is the basic problem of measurement in the social s c i e n c e s . C o n s i d e r the concept " s o c i a l status." A m o n g s o c i a l scientists there are two popular v e r s i o n s of this concept: " s u b j e c t i v e s o c i a l status," the c l a s s that i n d i v i d u a l s c o n s i d e r t h e m s e l v e s as b e l o n g i n g to; and "objective s o c i a l status," an i n d i v i d u a l ' s rank w i t h regard to prestige along s o c i a l h i e r a r c h i e s s u c h as education, i n c o m e , and o c c u p a t i o n . N e i t h e r v e r s i o n of the c o n c e p t c a n be m e a s u r e d directly. I n the c a s e o f " s u b j e c t i v e s o c i a l status," w e c a n n o t m e a s u r e d i r e c t l y w h a t i n d i v i d u a l s f e e l about their status. W e k n o w w h a t they report, but their r e p l i e s t o our i n q u i r i e s m a y not b e w h a t w e are l o o k i n g for. T h e y m a y not k n o w w h a t they " r e a l l y " f e e l , for i n s t a n c e ; o r they m a y m i s u n d e r s t a n d the q u e s t i o n a n d g i v e a

a first-rate a n a l y s i s of the r e l Harry Eckstein's Pressure Group

for c l a s s i f y i n g

"opposition";

Politics ( 1 9 6 0 ) , e s p e c i a l l y pp. 1 5 - 4 0 , in w h i c h he c l a s s i f i e s pressure group activities; and the third chapter of h i s Division and Cohesion in Democracy ( 1 9 6 6 ) , an e x c e l lent dimensional analysis of "political division"; Hanna Pitkin's Representation

( 1 9 6 9 ) , a c o l l e c t i o n of v a r i o u s w r i t i n g s on the c o n c e p t of representation, a m o n g w h i c h P i t k i n ' s o w n e s s a y is p a r t i c u l a r l y insightful; Representation ( 1 9 6 7 ) ; also P i t k i n ' s The Concept of

and G i o v a n n i S a r t o r i ' s Parties and Party Systems ( 1 9 7 6 ) .

O n e b r a n c h o f p o l i t i c a l p h i l o s o p h y i s the " a n a l y t i c p o l i t i c a l p h i l o s o p h y " a p p r o a c h , w h i c h s e e k s t o s t u d y p o l i t i c a l i d e a s b y a c l o s e e x a m i n a t i o n o f the meaning of concepts used to describe politics. T h i s approach is reviewed in

Richard

B e r n s t e i n ' s Restructuring
(1975).

of Social and Political

Theory

(1978)

and

in

Oppenheim

F i n a l l y , a s a n e x e r c i s e , y o u m i g h t c o n s i d e r the c o n c e p t u a l p r o b l e m s i n v o l v e d i n the w e l l - w o r n a p h o r i s m o f L o r d A c t o n : " P o w e r tends t o corrupt a n d absolute p o w e r c o r r u p t s a b s o l u t e l y . " H o w w o u l d y o u a n a l y z e this c o n c e p t u a l l y ? Can i t b e analyzed?

41

42

Chapter 4

Problems of Measurement

43

m i s l e a d i n g a n s w e r . T h e n a g a i n , a p e r s o n m a y f e e l d i f f e r e n t l y f r o m one d a y t o the next, i n w h i c h c a s e the m e a s u r e o f h i s status w i l l d e p e n d o n our rather arbitrary c h o i c e o f day. In the c a s e of "objective s o c i a l status," again we cannot m e a s u r e the variable directly. " O b j e c t i v e status" has something to do w i t h i n c o m e , something to do w i t h e d u c a t i o n , s o m e t h i n g to do w i t h o c c u p a t i o n , and s o m e t h i n g to do w i t h various other h i e r a r c h i e s , s o m e of w h i c h we m a y not k n o w about. N o n e of these provides a sufficient m e a s u r e in itself. F o r e x a m p l e , if we tried to u s e o c c u p a t i o n alone as a m e a s u r e of s o c i a l status, we w o u l d be f a c e d w i t h the difficult question of whether a p l u m b e r w h o m a d e $ 4 0 , 0 0 0 a y e a r w a s r e a l l y of l o w e r s o c i a l status than a bank teller w h o made $ 2 5 , 0 0 0 a year. S i m i l a r l y , if we tried to use i n c o m e alone as a m e a s u r e , we w o u l d be f a c e d w i t h the p r o b l e m of what to do w i t h a retired teacher, w h o s e i n c o m e might be b e l o w the poverty l e v e l . " S o c i a l status" in this c a s e is a c o n c e p t that is related to a n u m b e r of m e a s u r a b l e things but is related only i m p e r f e c t l y to e a c h of them. T h e best we c a n do in m e a s u r i n g it is to c o m b i n e the v a r i o u s m e a s u r a b l e indicators into a p o o l e d m e a s u r e that is r o u g h l y related to the c o n c e p t "objective s o c i a l status." We encounter s i m i l a r p r o b l e m s in m e a s u r i n g the other concepts I have cited as e x a m p l e s . L i k e m a n y other v a r i a b l e s i n p o l i t i c a l s c i e n c e , these concepts are o f c o n siderable interest and use in theories but are by their nature i m p o s s i b l e to measure directly. T h e general p r o b l e m p o s e d b y s u c h v a r i a b l e s i s presented s c h e m a t i c a l l y i n Figure 4-1. A s y o u s a w i n C h a p t e r 2 , i n p o l i t i c a l r e s e a r c h w e are c o m m o n l y interested i n relating c o n c e p t s through a theory. T h i s is a l w a y s true in theory-oriented r e s e a r c h , and it is often true in e n g i n e e r i n g r e s e a r c h as w e l l . If we cannot measure directly the concepts w e w i s h t o u s e , w e find o u r s e l v e s i n the position depicted i n F i g u r e 4 - 1 . We w a n t to say, " C o n c e p t A bears a relationship of t h u s - a n d - s o f o r m to c o n c e p t B . " B u t all that we c a n o b s e r v e is the relationship between m e a s u r e A and m e a s u r e B.

W h e t h e r w h a t we say about the m e a s u r e s is an accurate statement of the relationship b e t w e e n the c o n c e p t s depends on the u n o b s e r v e d relationships b e t w e e n c o n c e p t A and m e a s u r e A and between c o n c e p t B and m e a s u r e B. We c a n only a s s u m e what these relationships are. L i k e the theory itself, the relationships cannot be o b s e r v e d . A s a n e x a m p l e , suppose y o u want t o a s s e s s the theory tested b y D i e h l and K i n g s t o n , that countries that are i n c r e a s i n g their a r m a m e n t s tend to engage in aggressive international p o l i c i e s (see above, p. 8). Y o u might be f a c e d w i t h the f o l l o w i n g relationships:

1. Relationship between the concept "increasing armaments " and the measure of "increasing
armaments." You clearly cannot measure increases in a nation's armaments directly; only a national intelligence apparatus has the facilities to do that, and even then, the result is imperfect. Therefore, you might take as your measure the country's reported expenditures on armaments. Now, a country that is not preparing to launch an aggressive military venture would have less reason to lie about an arms buildup than would a country (such as Germany in 1933) that is consciously preparing for aggression. Therefore, the relationship between concept and measure in this case might be: When a country is not building up its armaments, or when it is building them up in order to launch an aggressive action, its reported expenditures on arms will not increase.

2. Relationship between the concept "increasing armaments" and the concept "aggressive
international policies." Let us assume, for this example, that countries that are increasing their armaments do tend to engage in aggressive international policies.

3. Relationship between the concept "aggressive international policies" and the measure
of "aggressive international policies." Let us assume, for this example, that we are able to develop a measure that corresponds almost perfectly to the concept "aggressive international policies." (In practice, of course, this would be a difficult variable to measure, and it certainly would be necessary first to analyze the varied dimensions involved in "aggression" and "policies" in order to state more clearly just what was meant by the concept.)

W e n o w find o u r s e l v e s i n the position depicted i n F i g u r e 4 - 2 . H e r e , b e c a u s e o f Figure 4-1 The Problem of Measurement peculiarities in the relationships b e t w e e n the concepts a n d the m e a s u r e s of these c o n c e p t s , the relationship y o u c a n o b s e r v e between the m e a s u r e s turns out to be the opposite of the true relationship b e t w e e n the c o n c e p t s . W o r s e yet, i n a s m u c h as the Theory Concept A -« >Concept B two m e a s u r e s and the c o n n e c t i o n between them are all that y o u c a n o b s e r v e , y o u w o u l d h a v e no w a y of k n o w i n g that this w a s h a p p e n i n g . T h i s is w h y I h a v e c a l l e d indirect m e a s u r e m e n t of concepts the p r o b l e m of m e a s u r e m e n t . O n e solution to the p r o b l e m might be to m e a s u r e v a r i a b l e s only directly. S o m e 9 I I 9 concepts are directly m e a s u r a b l e . A few e x a m p l e s are p e o p l e ' s votes if an election is nonsecret and y o u tabulate the result y o u r s e l f ; statements y o u hear m a d e on the Senate floor; a b o m b y o u see dropped or t h r o w n .
1

Relationship Measure A •* *Measure B

"it is reasonable, also, to include here reliable observers' accounts of such events. Even though the measurement of these things is technically indirect, if you accept another observer's account of them, you should be able to achieve a very tight fit between the concept "event happens" and the measure "reliable observer says that event happens."

44

Chapter

4

Problems of Measurement

45

Concept Increases in Armaments Increases in armaments tend to be followed by aggression

Concept Aggressive International Policies

elections are held with a secret ballot). Therefore, most of the time we must work with variables that are indirect measures of the concepts in which we are interested. This means that there are interposed, between our (concrete) operations and the (abstract) theory we want to work on, the relationships between our concrete concepts and their abstract measures. This is the situation illustrated graphically in Figures 4-1 and 4-2.
2

As concept increases, measure increases except when policies are aggressive

The chief problem of measurement is to ensure, as much as possible, that the relationships between concepts and measures are such that the relationship between the measures mirrors the relationship between the concepts.
As concept increases, measure increases

Problems we may encounter in trying to achieve this correspondence between measures and concepts fall under two headings: problems of measure reliability and problems of measure validity.

Measure Increases in Reported Arms Expenditures Increased reported expenditures are never followed by aggression

Measure Aggressive International Policies

RELIABILITY A measure is reliable to the extent that it gives the same result again and again if the measurement is repeated. For example, if people are asked several days in a row whether they are married, and their answers vary from one day to the next, the measure of their marital state is unreliable. If their answers are stable from one time to the next, the measure is reliable. The analogy of measuring with a yardstick may help make the meaning of reliability clear. If an object is measured a number of times with an ordinary wooden yardstick, it will give approximately the same length each time. If the yardstick were made of an elastic material, its results would not be so reliable. It might say that a chair was 20 inches high one day, 16 the next. Similarly, if it were made of a material that expanded or contracted greatly with changes in temperature, its results would not be reliable. On hot days it would say that the chair was shorter than on cold days. In fact, the choice of wood as a material for yardsticks is in part a response to the problem of reliability in measurement, a problem certainly not confined to the social sciences. Wood is cheap, rigid, and relatively unresponsive to changes in temperature. There are many sources of unreliability in social science data. The sources vary, depending on what kinds of data are used. Official statistics, for example, may be unreliable because of an unusual number of clerical errors or because of variability in how categories are defined from one time to the next. ("Votes cast in an election," for instance, may mean "all votes, including spoiled ballots" at one time, "all valid votes" at another.) Attitude measures may be unreliable because a question is hard for respondents to understand, and they interpret it one way at one time, another way the next. Or the people entering their responses into the computer may make mistakes.

Figure 4 - 2

Example of the Problem of Measurement

The difficulty with such a solution is that concepts that can be measured directly are usually trivial in and of themselves. They are too idiosyncratic to use in general, interesting theories. I would hate to say that particular statements by U.S. senators lead nowhere, but as far as political theory is concerned, it is true. Any given statement can apply only to itself. It takes on a general meaning only if it is placed in a category, so that it can be compared with other statements. For instance, Senator 's statement, "The president's policies are bankrupting the people of my state," is not intrinsically of theoretical interest. It can be placed in various categories, however: "statements of opposition to the president," "statements of concern for constituents' needs," "bombastic statements." Placing it in one of these categories allows us to compare it with other senatorial statements and to develop theories about the causes and effects of such statements. Note, though, that by placing it into a category, we have used the statement as an indirect measure of the concept which the category represents. No given statement is a perfect case of the "bombastic statement" or of the "statement of opposition to the president." Rather, a number of statements approximate each category, and we choose to use these statements as indirect measures of the abstract concept we cannot measure. To sum up the argument thus far: For a concept to be useful in building theories, it usually must be an abstraction, which cannot be measured directly. Further, of those interesting concepts that are in principle directly measurable (how individuals vote in an election, for example), many cannot be measured directly for practical reasons (the

t e c h n i c a l l y , these relationships are called epistemic correlations. "Correlation" means relationship, and "epistemic" has the same root as "epistemology"—the study of how we know.

trying out the questionnaire on a small pilot study in order to catch and correct unclear questions. It is particularly useful whenever a measure is multidimensional—for instance. such as "What do you like about candidate X?" "What social class do you consider yourself to be a member of?" and "Do you feel people generally can be trusted?" A few examples will help give you a sense of the magnitude of this problem. and he went on to draw interesting comparisons between elite and mass opinion. either by the interviewer on the spot or by the person w h o transcribes the interviewers' copy into a computer. On the average. from one time to the next. Testing the Reliability of a Measure Although unreliability may sometimes spur on further theoretical research. would show even more unreliability. Having first eliminated these sources of unpredictability in the relationship between concept and measure. Converse had to assume that he had effectively eliminated other sources of unreliability. because it would show up as a recognizable pattern of stable answers up to a certain point. 6. followed by changed. people's attitudes appeared to vary randomly across time. and so on. a measure of "social status. education. The example itself is a bit strained. inasmuch as straightforward informational items like this one can usually be measured with reasonable reliability. however." which is made by combining such items as an individual's income. or " n o " w h e n t h e y h a d h a d a b a d day. let us list the various sources of unreliability that might be involved in our example of tabulating responses to a simple question about marital status: 1 . .46 Chapter 4 Problems of Measurement 47 In our previous example." comprising such disparate items as welfare payments. questions that permit a considerable degree of interpretation. the As an illustration. P e r h a p s t h e y w o u l d a n s w e r " y e s " w h e n t h e y h a d h a d a g o o d day. In the split-half test. at least in American survey research. 3 . allowing for a suitable interval of time between the two measurements. as it did in this case. but once again stable. hospital subsidies. Note that to reach this conclusion. Presumably. the split-half check for reliability. such as gender and race. An additional source of unreliability in the measure is variability in the "true value" of the concept. perhaps some people got married. D i s h o n e s t i n t e r v i e w e r s m i g h t b e p l a y i n g g a m e s w i t h t h e r e s e a r c h e r b y filling o u t t h e f o r m s t h e m s e l v e s i n s t e a d o f g o i n g t h r o u g h t h e t r o u b l e o f getting the r e s p o n d e n t s t o answer them. Admittedly. answers. But the same sorts of conditions affect the reliability of less straightforward survey questions. he could then treat the unreliability in his measure as a reflection of unreliability in the concept. Christopher Achen (1975) later challenged Converse's conclusions on just these grounds. based on that conclusion. is that there is no way to distinguish instability that stems from "real" unreliability in the concept being measured from instability due to problems in the measurement process. R e s p o n d e n t s ' a n s w e r s m i g h t d e p e n d o n their m o o d . The test-retest check for reliability simply consists of repeating the measurement a second time. 5. unemployment relief. 3 These are drawn from Asher (1974). and school lunch programs. Because all of the items are taken to be measures of the same thing. s o that r e s p o n d e n t s s o m e t i m e s interpreted i t t o m e a n . We often wish to know how successfully we have reduced unreliability. " H a v e y o u ever b e e n m a r r i e d ? " I t m i g h t not b e c l e a r h o w p e o p l e w h o w e r e s e p a r a t e d from their s p o u s e s . He concluded that on certain issues. A more interesting case is presented when the true value itself varies randomly. If the second measure strongly resembles the first—that is. One problem with this test. Even on attributes that should be relatively easy to measure reliably. T h e q u e s t i o n m i g h t b e p h r a s e d badly. This situation sometimes provides the basis for interesting theories. a n s w e r i n g questions randomly. some errors appeared when the same respondents were interviewed at two-year intervals by a well-administered survey. Careful work is the best way to achieve reasonable reliability—double-checki rig all clerical work. I shall describe two of them briefly. whereas race did not change at all. T h e r e m i g h t be simple clerical errors in copying d o w n the answers. the researcher randomly divides these assorted items into two groups and then composes a summary "measure" out of each of the groups. such as interviewer error and confusion about the meaning of questions. the reported gender of respondents changed 0. but not d i v o r c e d . occupation. In one study. Converse (1964) noted that on many standard questions of political policy. ' A r e y o u n o w m a r r i e d ? " a n d s o m e t i m e s . A p e r s o n m i g h t s a y " n o " t o a n attractive i n t e r v i e w e r a n d " y e s " t o e v e r y o n e e l s e . or a measure of "welfare policy expenditures. some of these possibilities are farfetched. 4 . the report of respondents' educational background showed lower education two years later (which is logically impossible) an average of 13 percent of the time.5 percent of the time from one interview to the next. Another test. A number of tests have been developed to help researchers check the reliability of a measure. of course. should answer. R e s p o n d e n t s ' a n s w e r s m i g h t d e p e n d on the c o n t e x t of t h e interview. Characteristics on which it is somewhat easier to be vague or mistaken showed substantial unreliability. the mass public simply did not form stable opinions. such as attitudinal questions. or divorced. 2. and neighborhood into a single summary measure. house size. For example. 3 Reliability as a Characteristic of Concepts Thus far I have treated the unreliability of a measure as if it were a result of unpredictability in the relationship between the concept and its measure. This source of unreliability would be easily distinguishable from others. avoids this problem. R e s p o n d e n t s m i g h t be playing g a m e s with the interviewer. if the measure is stable over the elapsed time—it is considered relatively reliable. it is usually a barrier we want to elm inate.

h o w e v e r . T h u s . this m e a s u r e w o u l d not give an accurate estimate of the true v a l u e . T h e s e two c h e c k s for reliability c o m p l e m e n t e a c h other. A c c o r d i n g l y . but it c o u l d distort the c o n c e p t in the s a m e w a y e a c h of these t i m e s . " i n c r e a s e s in reported arms expenditures" m a y be . people tended to report their l e v e l of e d u cation differently f r o m one time to the next. E v e n in the l o n g r u n . we c a n a s s u m e that any error i n the m e a s u r e i s r a n d o m .48 Chapter 4 Problems of Measurement 49 either high or low. but this often i n c l u d e s c h a n g e s in the true v a l u e of the c o n c e p t rather than o n l y the instability that is due to the m e a s u r e m e n t p r o c e s s . In effect. U s i n g letters to the editor as a n indicator o f p u b l i c o p i n i o n w o u l d b e u n w i s e . for i n s t a n c e — v a r i e s greatly f r o m state to state a n d f r o m one y e a r to the next in any o n e state. T h e test-retest c h e c k is appropriate for any sort of m e a s u r e that c a n be replicated. that i s . for i n s t a n c e . 4 6 ) . If it g i v e s the s a m e result repeatedly. to distort a g i v e n m e a s u r e of a c o n c e p t . on the average. B u t it c a n be r e l i a b l e a n d yet not be v a l i d . A m e a s u r e of h o w c l o s e they are to e a c h other p r o v i d e s a c h e c k on h o w r e l i a b l e the total s u m m a r y m e a s u r e i s . Q u i t e s i m p l y . the m e a s u r e is r e l i a b l e . T h e s p l i t . In thinking about reliability. It a l s o s h o u l d c a u s e the s p l i t . as people s y s t e m a t i c a l l y tried to suppress their p r i s o n r e c o r d s . T h e relationship between validity and reliability c a n b e c l a r i f i e d b y introducing the notion of two s u m m a r y m e a s u r e s s h o u l d tend to be the s a m e . A m e a s u r e c a n n o t be v a l i d a n d yet not be r e l i a b l e . i f there is in p r i n c i p l e a relationship of equivalence b e t w e e n a m e a s u r e and Some Examples A f e w e x a m p l e s are in order. A s a n e x a m p l e . for instance. a c c i d e n t a l reporting errors w o u l d c a n c e l e a c h other out and the average of m a n y s u c h reports w o u l d give a true m e a s u r e of the c o n c e p t for a particular group of people. the m e a s u r e is v a l i d . A m e a s u r e is reliable to the extent that it is free of r a n d o m error alone. We a s s u m e d . " its c o n c e p t . a n d the a r c h e r h a s not y e t l e a r n e d to c o r r e c t for i t ) . s h o u l d be l i k e l y to report low increases the next y e a r a l s o ) . the average of those m e a s u r e s w o u l d reflect our " i d e a l " c o n c e p t . random error a n d nonrandom error. O n e c o m m o n s o u r c e of i n v a l i d m e a s u r e s is extrapolation f r o m a s a m p l e to a population that is not r e a l l y represented by that s a m p l e . s o w e w o u l d expect that in the long r u n . we w i l l have a m u c h better i d e a of any instability due to the m e a s u r e m e n t p r o c e s s .3 . VALIDITY R e l i a b i l i t y has to do w i t h h o w d e p e n d a b l y a m e a s u r e m i r r o r s its c o n c e p t . the m e a s u r e s of a c o n c e p t tend to be true. T h e i r o p i n i o n s w o u l d not be a v a l i d m e a s u r e of " p u b l i c o p i n i o n . so that it does not tend to m i r r o r it faithfully. it is l i k e l y that there w o u l d be a g o o d deal of n o n r a n d o m error in the m e a s u r e . T h e a r c h e r a c h i e v e s " r e l i a b i l i t y " on b o t h targets B " v a l i d i t y " o n l y o n target D . w h i c h w i l l also be important w h e n we l o o k at m e a s u r i n g the strength of relationships in C h a p t e r 8. c o u l d result if they lack reliability is a n e c e s s a r y but not a sufficient c o n d i t i o n of validity. I n m e a s u r i n g education. s o w e w i l l c o n f i n e ourselves here to e x a m p l e s of n o n r a n d o m error. in another there m i g h t be floods or a h u r r i c a n e . a n d D but a c h i e v e s validity. b e c a u s e the m e a s u r e does not mirror the concept accurately. but it w o u l d still be invalid. n o n r a n d o m error is s y s t e m a t i c error that tends in the l o n g r u n . T h a t p a r t i c u l a r i t e m w o u l d be a s o u r c e of u n r e l i a b i l i t y in the overall m e a s u r e . on the average. any one of them c o u l d be rather far f r o m the true v a l u e of the concept. It c h e c k s for all s o u r c e s of unreliability. or both.h a l f c h e c k is appropriate for m e a s u r e s c o m p r i s i n g a group of subitems." It m i g h t be that one p a r t i c u l a r i t e m — d i s a s t e r relief. however. By contrast. for its erratic variation w o u l d m a k e the s c o r e b a s e d o n the group i n w h i c h i t w a s i n c l u d e d l e s s l i k e l y to e q u a l the s c o r e b a s e d on the group that d i d not i n c l u d e it. i t c a n m i s s s o m e s o u r c e s o f instability i n the m e a s u r e m e n t p r o c e s s . In one y e a r there m i g h t be no natural d i s a s t e r s . B u t this is actually an important benefit: If we are able to s c r e e n out true c h a n g e over time in the c o n c e p t . If in the l o n g r u n . a m e a s u r e is v a l i d to the extent that it is free of both sorts of error. that if the c o n c e p t w e r e m e a s u r e d a large n u m b e r of t i m e s . it is " r e l i a b l y i n v a l i d . for e x a m p l e (see p . we a s s u m e d i m p l i c i t l y that the m e a s u r e tended to m i r r o r the c o n c e p t faithfully a n d that the p r o b l e m of reliability w a s s i m p l y that this tendency m a y be a rather loose one.2 is an example of invalid measurement. if we a s k e d people w h e t h e r they h a d a p r i s o n r e c o r d . T h i n k o f a n a r c h e r w h o is t r y i n g to p r o d u c e " v a l i d " s h o t s . T h u s . s u c h as the effect of the length of time s i n c e p a y d a y or of c h a n g e s in the weather. on respondents' a n s w e r s to an i n t e r v i e w question. w e encountered a good deal of r a n d o m error. T h e r e w a s no r e a s o n to expect that people w e r e deliberately m i s r e p r e s e n t i n g their educational l e v e l . We have already d i s c u s s e d s e v e r a l i n s t a n c e s o f r a n d o m error. b e c a u s e the people w h o write s u c h letters are not an accurate c r o s s s e c t i o n of the p u b l i c as a w h o l e . T h a t i s . T h e relationship between the concept and the m e a s u r e is s u c h that w h e n "increases in a r m a m e n t s " are h i g h . It c h e c k s o n l y for those s o u r c e s of unreliability that do not operate over time. i n a s m u c h as a l l of the subitems p r e s u m a b l y are m e a s u r e d at the s a m e time. T h e t w o sorts o f error are p r e s e n t e d v i s u a l l y i n F i g u r e 4 . in other w o r d s . c o n s i d e r a state-by-state m e a s u r e o f " w e l f a r e p o l i c y e x p e n d i tures. T h e s e t w o sorts o f error r e s u l t i n the f o u r p o s s i b l e c o m b i n a t i o n s s h o w n i n F i g u r e 4 . T h e p r o b l e m in reliability is that s i n c e the m e a s u r e s vary. T h e m e a s u r e might be reliable (a country that reported l o w increases in one year. A m o r e s e r i o u s f a i l i n g of our m e a s u r e m e n t s . A m e a s u r e is v a l i d i f it a c t u a l l y m e a s u r e s w h a t it purports to m e a s u r e . depending on the reason for the arms buildup. on the average. T h e s h o o t i n g m a y s u f f e r f r o m either r a n d o m error (the a r c h e r i s e r r a t i c ) o r n o n r a n d o m error (the a r c h e r h a s s o m e s y s t e m a t i c p r o b l e m — p e r h a p s there i s a w i n d f r o m the left.3 . shots at the c e n t e r of a target.h a l f test to s h o w a r e l a t i v e l y l o w reliability. R a n d o m error is the sort of error we have a d d r e s s e d in d i s c u s s i n g reliability. T h e r e are m a n y w a y s that a m e a s u r e c a n be i n v a l i d . " T h e relationship between the measure " i n c r e a s e s in reported arms expenditures" and the concept " i n c r e a s e s in a r m a m e n t s " in F i g u r e 4 .

however. for instance) guards against a fiasco like that which destroyed the Literary Digest poll. For example. depicted in Figure 4-1. its sympathies were a valid measure of the way the country was going to vote in those elections. A questionnaire item may also result in an invalid measure when respondents attach a significantly different meaning to the question than was intended by the researcher. of course. We cannot A comic case of sampling problems from the early days of opinion polls is the Literary Digest poll. Between 1932 and 1936. you should ask a few people to answer your questions and then to relate their understanding of the questions themselves. How. it sampled only those who were interested enough and energetic enough to return the sample ballot. "Not at all. we now know that drawing a sample in certain ways (drawing a random sample. Thus it was a shock when the poll's prediction of a landslide victory for Landon in 1936 was gainsaid by FDR's decisive victory. In 1924. and again in 1932. can we assess the validity of the measure? The answer. . which was sent out to a mailing list obtained from telephone directories and state automobile registration lists. Part of the "craft" in the craft of political research is cleverness and care in developing measures that appear likely to be valid. in determining the final form of a questionnaire that you hope to use in a study. 1928. The Digest sample distorted the U. Some techniques are available to help in developing valid measures. Accordingly. Virtually everyone who owned a car or a telephone was reached by the poll. coming within a few percentage points of the actual outcome in each of those elections. it was attacked in the Congressional Record and featured in New Yorker cartoons. it was striking that voting participation was higher among farmers than among most groups in the French population. Our problem in checking the validity of a measure is similar to the general problem of measurement. with many of them vehemently rejecting political parties. it essentially sampled only the upper and middle classes. We say that a measure is valid if it is a true measure of its concept. that French farmers were essentially apolitical.50 Chapter 4 Problems of Measurement 51 Random Error High Low Figure 4-3 R n o and Nonrandom Er r a dm ro Apparently. This may alert you to questions that mean something different to your respondents than they mean to you." Checks for Validity Taking precautions. but even at that. why did they vote? In a study designed to explore this paradox (see p. middle-class sample the Digest drew upon did not vote much differently from the rest of the country from 1924 through 1932. Roosevelt initiated the New Deal. the Digest had over 2 million responses each time. many farmers who were interested in politics but considered themselves independents responded "Not at all" to this invalid measure of "interest in politics. the Digest poll was very successful. When asked in surveys "How interested are you in politics?" they had generally responded. Thus. When the Literary Digest went out of business the next year. These deceptively simple strategies consist of taking various precautions against invalidity during the construction of a measure. If they were not interested in politics. population in two ways. the Digest ran an ambi- tious poll in presidential election years. After 1936. The most important thing is to think through the measurement process carefully and to be on guard against any way in which the relationship between concept and measure might be distorted. First. Sidney Tarrow discovered that the innocent question about political interest had been spreading confusion. Starting in 1924. Only about 20 percent of the sample ballots mailed out were returned. French farmers apparently interpreted "interest in politics" to mean commitment to some particular party. Also. inasmuch as those who did not have a car or a telephone—at a time when cars and telephones were far less universally owned than today—did not get onto the mailing list. this seems to have been a rather select sample. know what the relationship between a concept and its measure is. then. for example. The Literary Digest was a giant magazine in the United States in the early part of the twentieth century.S. the interested. By 1932 the poll had become an institution. which broadened his support among the poor and drove the middle class to the Republicans. Because only 20 percent of those who received the ballot returned it. the sympathies of the middle class were no longer a valid measure of the way the country would vote. 29). is that there is no pat way to do so. Also. It had always been thought. it was thought that the shock and loss of reputation from having called the election so badly was a factor in the magazine's demise." At the same time. But the general problem of measurement is precisely the fact that usually all we can observe is the measures.

At the m o s t general l e v e l .1 4.3 0. If a m e a s u r e is s y s t e m a t i c a l l y i n v a l i d . A s i t h a p p e n s .5 Seniority 12. he c o n c l u d e d that the conventional m e a s u r e s .0 3. this test is not a l w a y s . s u c h as might o c c u r f r o m clerical errors or other sources of unreliability. r a n d o m error. In the fifth and sixth c o l u m n s .0 30. the relationship at the m e a s u r e l e v e l w i l l tend to be looser and w e a k e r than the true relationship. If the m e a s u r e s are sufficiently u n r e l i a b l e . If there is s o m e m e a s u r e /3 that we are certain is strongly related to c o n c e p t A. T h u s far our strategies have not a c t u a l l y p r o v i d e d a test of the It s h o u l d be o b v i o u s that the o n l y g o o d m e a s u r e is one that is v a l i d . T h e m o s t general test of validity is w h a t is c a l l e d face validity. then a c o u l d not be a v a l i d m e a s u r e of A.8 4.6 0.0 10. T h e s e techniques s i m p l y require that the investigator think ahead to p r o b l e m s that c o u l d o c c u r in the relationship b e t w e e n c o n c e p t and m e a s u r e and act either to prevent these or to c h e c k to see whether they are present.8 12. that i s .2 13. how broadly they are intended to apply.0 12. T h a t k i n d of certainty is u n c o m m o n and m a y not be shared e q u a l l y by every observer. Is it v a l i d "on its f a c e " ? A f t e r a l l .0 19. Using Simulated D a t a Less Reliable Measures Very Unreliable Measures validity of the m e a s u r e . i n a s m u c h as representatives f r o m safe districts tend to have greater seniority than those f r o m marginal districts. the b a s i c relationship c a n be so w e a k e n e d that it w i l l appear. w h i c h had s h o w e d l o w political interest c o i n c i d i n g w i t h h i g h participation.4 Margin of Victory (%) 21. S u c h an indirect test of validity is p o s s i b l e only w h e n a r e s e a r c h e r is quite certain that /3 must go along w i t h A. has been added to the original m e a s u r e s . E a c h set of two c o l u m n s tabulates the c l o s e n e s s of elections in ten congressional districts. In the third and fourth c o l u m n s . an even greater degree of TABLE 4-1 Safe Districts Related to Seniority. a n d our a s s u m p t i o n of a relationship between /3 and A is true. what the terms m e a n .5 22. Test of validity. T h e effect of n o n r a n d o m error is s i m p l e and severe.6 6. L e t us say that we want to decide whether m e a s u r e a is a v a l i d m e a s u r e of c o n c e p t A. T h e relationship is also quite strong. It is apparent from the figures in the first set of c o l u m n s (True V a l u e s ) that there is a relationship between the two. however.2 Margin of Victory (%) 9. it is worth c o n s i d e r i n g what happens under those c i r c u m s t a n c e s .0 3.6 8. B e c a u s e s o c i a l scientists often operate w i t h m e a s u r e s that suffer f r o m one or the other sort of error.8 6. seniority increases without exception as the representatives' margins of victory increase. we c a n c h e c k to see whether m e a s u r e ji is related to m e a s u r e a. T a r r o w ' s initial c o n c l u s i o n that the u s u a l question " H o w interested are y o u in p o l i t i c s ? " w a s p r o v i d i n g a n i n v a l i d m e a s u r e o f "political interest" a m o n g F r e n c h farmers c a m e f r o m his observation that farmers h a d in fact one of the highest levels of electoral participation a m o n g F r e n c h c i t i z e n s . y o u s h o u l d go thoroughly into the b a c k g r o u n d of the things reported t h e r e — h o w they w e r e d e v e l o p e d .6 12. m a k i n g them less reliable. although it is n e c e s s a r i l y subjective and o p e n to v a r y i n g interpretations. S u c h a test c a n be m a d e . To the extent that m e a s u r e s are unreliable.0 14. a n d so o n . as w e l l as their representatives' seniority. If it is not." S i m i l a r l y . f r o m w h a t we c a n o b s e r v e . one that has in it neither r a n d o m nor n o n r a n d o m error. w h e n it c a n be u s e d .5 13. p o s s i b l e . w i l l greatly strengthen y o u r findings.5 12. 4 As another e x a m p l e of this k i n d of test.6 12. must not have b e e n m e a s u r i n g political interest v a l i d l y . "political interest" corresponds to A.8 21.8 1. the t w o sorts o f error have different effects on the development of theory. the strategy I have suggested here requires o n l y that the investigator c o n s i d e r c a r e fully h o w p l a u s i b l e it is that the m e a s u r e mirrors the concept.6 4. If y o u think it does (and people w h o read y o u r w o r k agree w i t h y o u ) . T h i s is illustrated in Table 4 . It w i l l p a r a l l e l the true relationship but w i l l appear w e a k e r than is actually the c a s e .2 22. T h e effect of unreliability ( i f the m e a s u r e s are o t h e r w i s e valid) is m o r e subtle. as if there is no relationship at a l l . farmers' responses to the question on political interest correspond to a. B e c a u s e h e c o u l d not c o n c e i v e o f high electoral participation o c c u r r i n g in the a b s e n c e of h i g h political interest. T h e study b y T a r r o w cited earlier provides a n e x a m p l e o f this l o g i c .0 12.52 Chapter 4 Problems of Measurement IMPACT O F R A N D O M A N D N O N R A N D O M ERRORS 53 S u c h p r e l i m i n a r y testing of o n e ' s m e a s u r e s is c a l l e d a "pilot study.6 0. and farmers' electoral participation corresponds to B. we w o u l d be s u s p i c i o u s of the validity of the m e a s u r e s . B u t a s s e s s i n g the validity of m e a s u r e s is so important that an indirect test.4 21. there is no r e a s o n for us to expect any c o r r e s p o n d e n c e b e t w e e n the relationship w e actually o b s e r v e f r o m our m e a s u r e s and the i d e a l i z e d relationship we w i s h to investigate.1 2. y o u have c o n s i d e r a b l e e x p e r i e n c e w i t h politics and m u s t j u d g e for y o u r s e l f whether the m e a s u r e does w h a t y o u want it to do.0 11.9 In this example.0 9. 4 2 . and it is a l w a y s rather subjective.2 11.1 2.4 5.5 2. in u s i n g o f f i c i a l d o c u m e n t s . If we found that two of the nations went to w a r against e a c h other (/3) yet the measure did not show an a c c o m p a n y i n g increase in feelings of hostility between the two.1 . True Values Seniority 32 24 23 20 14 11 10 6 5 Margin of Victory (%) 18 12 11 11 8 6 6 4 3 1 Seniority 32. T h u s . or even usually. c o n s i d e r a measure of nations' hostility to e a c h o t h e r — b a s e d on content a n a l y s i s of the nations' n e w s p a p e r s ( a ) . T h i s is j u s t a f a n c y term for whether a m e a s u r e l o o k s right to y o u .0 3. i t has face validity.

recurrent n a g g i n g i n a d e q u a c i e s in the c h o s e n m e a s u r e s m a y debilitate a theory so that it b e c o m e s a l m o s t a trivial e x e r c i s e . with a population of 9. Jr. "Understanding of politics" indicated by years of education. A n s o l a b e h e r e a n d h i s c o a u t h o r s pointed out. A l a n G e r b e r . the question of how precisely a measure should be calibrated. C o n s i d e r a test for the s i m p l e theory: " T o the extent that they understand p o l i t i c s . Jefferson County. in Florida. "Need for public services" indicated by the size of the person's family. " T h i s statement is r i d i c u l o u s . Test your measures for possible inadequacies. M u c h of survey r e s e a r c h tends to fall into this category. U n d e r the b a d . had had one seat in the state senate and one seat in the state house of representatives. at least. T h e s e W h e n they f o u n d that s p e n d i n g l e v e l s after reapportionment w e r e no higher than they h a d b e e n before reapportionment. using " n e w s p a p e r s read per 1. scholars c o n c l u d e d that c h a n g i n g the electoral r u l e s h a d h a d n o i m p a c t o n p u b l i c p o l i c y . T h e c u r e for this p r o b l e m is s i m p l y to u s e c a r e and imagination i n d e v e l o p i n g m e a s u r e s . Carr 369 US 186 (1962). O f t e n . In this decision the Supreme Court ruled that it was unconstitutional to have state legislative districts with widely varying populations. but w h e t h e r c o u n t i e s that g a i n e d fair IMPORTANCE OF ACCURACY representation thereafter got a m o r e equal share of state e x p e n d i t u r e s . the extent to w h i c h u n i m a g i n a t i v e s c h o l a r s w i l l a l l o w moderate errors of m e a s u r e m e n t to a c c u m u l a t e in a statement until the statement loses m u c h of its m e a n i n g .e n o u g h understood by political scientists. i f p e o p l e ' s n e e d for p u b l i c s e r v i c e s i s r e l a t i v e l y great. it would seem that the more dependents a person had. T h i s simple fact has tended to be forgotten in a general ethos of " m a k i n g d o " with poor measures. S o r e d r e s s i n g the i m b a l a n c e s w a s a p p r o x i m a t e l y a w a s h in terms of party strength. This would seem reasonable. you wish to examine. depending on whether they stem from flux in the measure (the problem of reliability) or from a basic lack of correspondence between measure and concept (the problem of nonrandom error). "Liberalism" indicated by voting Democratic. Fortunately. To the extent that our measures are not valid. what we do with them is irrelevant. In the next chapter we look at another aspect of measurement. S t e p h e n A n s o l a b e h e r e . 3. before the Court's decision. s o i f they lost i n f l u e n c e o n p o l i c y . 2.543. 5 problems turn out to be of two basic types. an interesting question is lost through m i s t a k e s in setting up e m p i r i c a l operations to parallel the theoretical question. L e t me r e e m p h a s i z e the important pitfalls to be c a r e f u l of in m e a s u r e m e n t : w h e t h e r the l e v e l o f s p e n d i n g r o s e — i t w a s pretty o b v i o u s w h y i t d i d n o t — b u t w h e t h e r it w a s distributed m o r e e q u a l l y as a result of m o r e e q u a l r e p r e s e n t a t i o n . " It m i g h t w e l l be that a p o l i t i c a l s c i e n t i s t w o u l d o p e r a t i o n a l i z e the three v a r i a b l e s of this theory in a w a y s u c h as the f o l l o w i n g : /. old s y s t e m both R e p u b l i c a n suburbs and D e m o c r a t i c c i t i e s h a d been d i s e n f r a n c h i s e d . I have exaggerated here slightly.54 Chapter 4 Problems of Measurement 55 random error has been added. the more that person would depend on a variety of public services. r e a c h e d e x a c t l y the w r o n g c o n c l u s i o n . In this chapter I have d i s c u s s e d problems in the a c c u r a c y of measurement. Note that the relationship b e c o m e s w e a k e r in the s e c o n d set of data (there are more exceptions to the general tendency) and virtually disappears in the last. population 935. Prior to the decision many states had done no legislative redistricting for half a century. . If we w e r e to test the relationship between safe districts and seniority and had only the data f r o m the last two c o l u m n s in hand. or on the o v e r a l l s p e n d i n g l e v e l of the state. In other w o r d s . 2. Carr d e c i s i o n . w e r e not a l l that c l e a r l y either D e m o c r a t i c o r R e p u b l i c a n . S n y d e r .000 population" as a measure of the political awareness of various electorates) is another e x a m p l e of this problem. but only slightly. had had one seat in the state senate and three seats in the state house of representatives. the stronger the relationship between the s i z e of that p e r s o n ' s f a m i l y and the probability that the person w i l l vote D e m o c r a t i c . and J a m e s M . as a result. they f o u n d d r a m a t i c p o l i c y effects f r o m r eappor t i onm ent .047. the e m p i r i c a l a n a l o g of the theory b e c o m e s : " T h e more educated a person i s . For instance. but its importance must be constantly underscored. In the w a k e of the Baker v. t h o u g h . w h i c h had benefited. W h e n A n s o l a b e h e r e a n d h i s c o a u t h o r s tested the effect i n this w a y . ( 2 0 0 2 ) d e m o n strated this p r o b l e m w e l l in a paper s h o w i n g that three d e c a d e s of r e s e a r c h on whether electoral reapportionment affected public policy had been subtly m i s d i r e c t e d and h a d . T h e loose acceptance of cross-nationa l indicators (for instance. T h r e e d e c a d e s o f r e s e a r c h and c o m m e n t a r y h a d m i s s e d the point. 5 N o w . we have in m a n y c a s e s c o m e to accept measures that we k n o w full w e l l are inadequate. s p e n d i n g s h o u l d h a v e r i s e n . w h i l e r u r a l a r e a s . political scientists are now b e c o m i n g more aware of the problem of measurement. Miami's Dade County. we w o u l d probably c o n c l u d e that there w a s no relationship. that the true test of w h e t h e r e l e c t o r a l institutions affected p o l i c y w a s not the effect on the t w o parties. B e c a u s e the variables we w o r k w i t h are difficult to m e a s u r e . understanding should be fairly closely related to education. Even when a measurement p r o b l e m is not so central as to n u l l i f y the results of a study. Again. Be sure that the measures you choose fit the relationship among concepts that 1. s c h o l a r s had l o o k e d to see w h e t h e r taking a w a y the unfair overrepresentation of rural c o u n t i e s h a d led to an e x p a n s i o n of state s p e n d i n g . the true test of w h e t h e r e q u a l i t y of representation affected p o l i c y w a s not I n a c c u r a c y in measurement is a critical p r o b l e m w h o s e potential for m i s c h i e f does not yet s e e m w e l l . although this is a rough measure. T h e r e a s o n w a s pretty c l e a r : Baker v. so backward rural areas that had not experienced much growth in their populations were hugely advantaged relative to rapidly growing cities and suburbs. T h e i m p l i c i t a s s u m p t i o n w a s that rural areas w o u l d h a v e preferred t o k e e p s p e n d i n g d o w n . they w i l l be m o r e l i b e r a l . In Chapter 8 we w i l l return to tackle the problem of measuring relationships w h e n random or nonrandom error is present.

I conceive of all social behavior as a vast "data bank. and " s e n s e o f c o m m u n i t y " ) rather than p r e c i s e n u m e r i c a l quantities. In a study of N o r w e g i a n p o l i t i c s . S c h w a r t z . More often. is w e l l worth reading. Division and Cohesion in Democracy: A Study of Norway (Copyright 1966 by Princeton University Press). Frederick Mosteller's article " E r r o r s . Herbert A s h e r (1974) presents some good examples of reliability problems. 7 9 . if used circumspectly. " M e a s u r i n g Partisanship. T h e ideas presented in the book are both creative and sound. In overemphasizing quantities we sometimes miss the most telling data—in any case. I do use such quantities in what follows. with a rather technical d i s c u s s i o n of w a y s to handle them. C a m p b e l l . footnote pp. and C o n v e r s e and Pierce ( 1 9 8 5 ) . 57 . Harry." A u s e f u l e x e r c i s e w o u l d be to list as m a n y factors as y o u c a n that w o u l d l e a d t o r a n d o m o r n o n r a n d o m error for e a c h o f the f o l l o w i n g m e a s u r e s : i n t e n d e d vote (in s u r v e y s ) . or h o w finely calibrated. and personal income. " S t u d y i n g Presidential Performance in C o n g r e s s " : F e l d m a n ( 1 9 8 3 ) . " T h e M e a s u r e m e n t and M e a n i n g of Trust in G o v e r n m e n t " . strength o f a r m e d f o r c e s . data that may be reliable in their own right or used as checks on the inferences drawn from quantitative data. u n e m p l o y m e n t . In this chapter we deal w i t h the " q u a l i t y " of the m e a s u r e i t s e l f — h o w p r e c i s e it s h o u l d be. " T h e Quality of S u r v e y i n g R e s p o n s e s A b o u t Parents and the F a m i l y " . " in the International Encyclopedia of the Social Sciences (1968)." only some of which is quantitatively aggregated in yearbooks and the like. [M]any of the indicators used in the text may not be readily recognized as such by contemporary social scientists. Reliability and Validity in Qualitative Research (1985)." for i n s t a n c e . h i e r a r c h i c a l c o n t r o l i n a n a g e n c y . for quantitative indicators are not always as "indicative" of what one wants to know as other observations. t r i b a l i s m . By an indicator we usually mean nowadays a precisely ascertainable quantity that stands for some imprecise quantity (as G N P may indicate level of economic development. nor always obtainable. a g r e e m e n t w i t h the p r e s i d e n t (in c o n g r e s s i o n a l v o t i n g ) . The preceding chapter dealt Precision with problems of the reliability and validity of m e a s u r e s . H a m m o n d and F r a s e r (1984). S e e also K i r k and Miller. by E u g e n e J. and L e e Sechrist (1966). D o n a l d T. This strikes me as both defensible and desirable. T h o s e p r o b l e m s c o n c e r n e d the relationship b e t w e e n a m e a s u r e and the c o n c e p t that m e a s u r e is intended to mirror. H a r r y E c k s t e i n felt that it w a s n e c e s s a r y to a p o l o g i z e for the fact that s o m e m e a s u r e s he w o u l d u s e w e r e subjective intangibles ( " w a r m t h i n s o c i a l relations. and the text itself is filled w i t h interesting and highly unusual examples. Mosteller details a variety of possible sources of invalidity and unreliability. Reprinted by permission of Princeton University Press. 1966.56 Chapter 4 Chapter 5 D FURTHER DISCUSSION A delightful book with a unique approach to handling certain problems of validity is Unobtrusive Measures. and much of the rest of which may speak volumes to our purposes. or as the number of casualties in revolutionary violence may indicate its intensity). readily observable "qualities" are used as indicators of not-so-readily observable qualities. R i c h a r d D. if it is to be u s e f u l . S e v e r a l good examples of specific measurement problems are N i e m i and K r e h b i e l ( 1 9 8 4 ) . (Eckstein. Webb.8 0 ) 1 'From Eckstein. however.

Reducing precision in measures is thus u n n e c e s s a r y when one is analyzing data. I n s t e a d . T h e C a r d i n a l R u l e of P r e c i s i o n might read: Use measurements that are as precise as possible. presidents. National Election Study. If it is true that s o m e t i m e s we m a y be better off w i t h l e s s p r e c i s i o n in our m e a s u r e s . " and so on is m o r e p r e c i s e than reporting it as "Protestant. given the subject you are studying. T h i s has been done in F i g u r e 5-2. T h e figure is so chaotic that it is h a r d to d e c i p h e r a relationship. reporting a p e r s o n ' s i n c o m e in dollars is m o r e p r e c i s e than rounding it off to the nearest thousand dollars. o r e v e n largely. Data provided by the Interuniversity Consortium for Political and Social Research. If we w e r e registering voters. " and "other. i n the e n s u i n g . the measures of percent voting fluctuate less and the f o r m of the relationship between age and participation becomes clearer. I must first distinguish between two kinds of precision with w h i c h we shall be concerned. S i m i l a r l y . creating a smaller number of categories with a larger number of cases in e a c h . and the second.1 . (See Chapter 7 for a description of regression analysis. O n e theme of this chapter w i l l be that this rule is as susceptible to violation by "quantifiers" as by "nonquantifiers. I f w e study U .) 2 18 30 40 50 Age 60 70 80 85 Figure 5-1 Age and Participation in 2 0 0 2 Election: Age Measured by Years Source: 2002 Congressional-Election Survey. but unfortunately. precise measurement has become enough of a fetish in political science that it is no doubt a l w a y s possible to find someone w h o w i l l d i s m i s s a piece of w o r k because it is not "quantitative. O n e w a y to handle this p r o b l e m ." " C a t h o l i c . we are interested in l o o k i n g at the entire range of variation and have no particular cutoff point in m i n d . F i r s t . reporting a p e r s o n ' s r e l i g i o n as " P r e s b y t e r i a n . PRECISION IN MEASURES P r e c i s i o n in m e a s u r e s c o r r e s p o n d s r o u g h l y to our c o l l o q u i a l u s e of the w o r d precision—that i s . w o u l d be to e x p a n d the study by c o n s i d e r i n g m o r e i n d i v i d u a l s i n e a c h category. " p . with age measured to the nearest half-decade rather than to the nearest year. and it may cause such techniques to give systematically inaccurate results. p r e c i s i o n in m e a s u r e s is o b v i o u s l y a g o o d thing. for i n s t a n c e ." In this chapter I sort out j u s t what kinds of precise quantification are helpful in political research. I c a l l the first of these precision in measures. then it appears l i k e l y that this sort of p r e c i s i o n is not so important that r e s e a r c h s h o u l d b e j u d g e d s o l e l y . For example." A l t h o u g h as a general rule. ( S e e the b o x " L a w o f L a r g e N u m b e r s . k e e p i n g the units of m e a s u r e m e n t relatively fine.300) c o n s u l t e d i n the p o l l . though. the degree of p r e c i s i o n we n e e d is determined by what we w i s h to do w i t h the data. B u t this u s u a l l y i s neither p r a c t i c a l Reducing precision in this way to eliminate random noise in the data is appropriate only for chartmaking and visual presentation. for e x a m p l e . too m u c h p r e c i s i o n i n m e a s u r e s c a n be a n u i s a n c e . nor i n m a n y c a s e s p o s s i b l e . there i s o n l y a s m a l l n u m b e r o f r e s p o n d e n t s for e a c h p a r t i c u l a r age. a l t h o u g h w e c a n see that participation g e n e r a l l y r i s e s w i t h age. 6 1 ) T h u s . o n h o w p r e c i s e its m e a s u r e s are. S o m e t i m e s . W i t h the larger number of cases in e a c h age c l a s s . C o n s i d e r F i g u r e 5 . A n o t h e r w a y to handle the problem is to decrease the precision of the measure." To discuss it further. it is difficult to p i c k out a s y s t e m a t i c pattern in the r e l a t i o n s h i p b e t w e e n age a n d p a r t i c i p a t i o n i n e l e c t i o n s . T h i s m e a n s that the p e r c e n t voting fluctuates a g o o d d e a l f r o m one age group to the next. in theory-oriented r e s e a r c h . do not waste information by imprecise measurement. S . T h i s should have been an unnecessary justification. we are not able to limit the n e c e s s a r y l e v e l of p r e c i s i o n in this way. Data analysis techniques such as regression analysis handle random noise in their own way. precision in measurement. w h i c h s h o w s the relationship b e t w e e n age and participation in the 2 0 0 2 c o n g r e s s i o n a l election. 2 . l a r g e l y r a n d o m fluctuation of p e r c e n t v o t i n g . and r o u n d i n g i n c o m e off to the nearest ten thousand dollars is still l e s s p r e c i s e . G e n e r a l l y . w e are limited to a population of 4 5 . its importance c a n be overrated. of c o u r s e . and a drop-off among the very aged. G i v e n the l i m i t e d n u m b e r o f i n d i v i d u a l s (about 1. " " G r e e k O r t h o d o x . any p r e c i s i o n in m e a s u r i n g age that w e n t b e y o n d l a b e l i n g people "under 18" and " o v e r 18" w o u l d be u n n e c e s s a r y and p o s s i b l y a n u i s a n c e as w e l l . even i n theory-oriented r e s e a r c h . " " R e f o r m e d J e w i s h . with the most rapid increases c o m i n g in the first couple of decades.58 Chapter 5 Problems of Measurement 59 It is wrong that E c k s t e i n should have felt the need to apologize.

5 0 ) v o t e d . w e c o u l d a g a i n l o s e s o m e o f t h e pattern that a p p e a r e d there. w e m u s t be careful not to overemphasize the importance of precision in measures. It w o u l d be foolish to give up such studies on this account. w h i c h he d e f e n d e d in the quotation given at the start of this chapter.60 100 Chapter 5 Problems of Measurement 61 L a w of Large Numbers The fact that the smaller the group of individuals sampled. do not group the informat i o n y o u h a v e u n t i l y o u h a v e f i n i s h e d g a t h e r i n g it. M a n y questions in political science simply do not admit of great precision of this type. For e x a m p l e . ." " 3 0 . but not indispensable.) The fact is intuitively obvious. if y o u r study c o n c e r n s a specific group within the population. one should not conclude that precision is not helpful or important. (This law forms the basis for a great deal of statistics. a l t h o u g h t h e r e a r e s i t u a t i o n s i n w h i c h w e m a y w a n t t o b a c k off from precise measures.g a t h e r i n g s t a g e is c a s u a l l y t h r o w i n g a w a y precision that m i g h t later p r o v e useful. given the subject w e are studying.4 0 . and Chicanos. you probably would have drawn too few Puerto Ricans and Chicanos. O n t h e o t h e r h a n d . w e w o u l d find that 5 4 p e r c e n t o f t h o s e i n t h e first unit ( 1 8 . they would expect to get very nearly the national figure in all of them. as the Cardinal R u l e states. No one should stop studying the m e a s u r e d still less p r e c i s e l y t h a n i n F i g u r e 5-2. not " u n d e r 30. m a k e sure that you include as large a n u m b e r of cases as is practical. for e x a m p l e . even if the resultant data are not "typical" of the entire population. a n d that 7 7 p e r c e n t o f t h o s e i n t h e s e c o n d u n i t ( 5 1 . the more any measure based on that group will deviate from its norm is one part of the L a w of Large Numbers. W h e n asking people their age. With groups of 100. that it is important to be as precise as possible and not to waste information. Had you simply drawn a random sample of the population. A n u m b e r of p r e c a u t i o n s are in order." or "Jewish ' In short. Precision in measures is important. B e c a u s e o f t h e l i m i t a t i o n s i m p o s e d b y a s p e c i f i c r e s e a r c h t o p i c ." "Catholic. If age w e r e m e a s u r e d in units of 33 years. (It would not be unlikely for there to be 60 males in 100 people.000 people randomly and calculated the percentage male. To continue with the example of age and participation. That c o n c e p t d o e s not lend itself to finely calibrated m e a s u r e s . there would be wild fluctuation." and so o n — n o t "Protestant. B u t t h e r u l e a l s o s t a t e s t h a t w e s h o u l d b e a s p r e c i s e a s w e c a n . if y o u w a n t e d to study the effects of different k i n d s presidency because the n u m b e r of cases is limited. First. A n o t h e r pitfall to a v o i d at t h e d a t a . If a research team selected groups of 1. or stop studying political corruption because many facts are hidden or grouped together. or stop studying past historical periods b e c a u s e data are limited and m a n y kinds are " p r e g r o u p e d " in inconvenient ways. for example. Puerto Ricans. " and the like. it m a k e s sense to get an adequate n u m b e r of the group into your study.D e c a d e s 60 70 80 8 t h e d a t a later i f y o u w a n t t o . You can always g r o u p 30 40 50 Age Figure 5-2 Age and Participation in 2 0 0 2 Election: Age Measured by H a l f .) With groups of 10. He wanted to study the degree of " c o m m u n i t y " in Norway. so that you will not have to r e d u c e the precision m o r e than you w o u l d like. A l t h o u g h w e w o u l d still s e e a strong relationship in these figures. and with groups of 1 (the smallest possible "group") all groups would be either 0 or 100 percent male. Thus. they would get increasingly erratic measurement. If y o u ask people their religion. you might draw up a sample consisting of equal numbers of 'WASP is the acronym for white. record the n u m b e r of years old.8 4 ) v o t e d . blacks. Y o u w i l l h a v e a b e t t e r i d e a t h e n o f h o w fine a g r o u p i n g y o u w a n t to e n d up with. these suggestions will help meet that rule. if age were If it is true. of discrimination. Anglo-Saxon Protestant. m u c h of the interesting detail of Figure 5-2 w o u l d be lost. is a case in point. a t this p o i n t y o u s h o u l d s a v e all the i n f o r m a t i o n y o u get. 80 WASP. T h e time to ensure that your measures will be as precise as you want is w h e n y o u are gathering data. write d o w n "Piesbyterian. 3 white ethnics. and you would have had to lump t h e m together with either the blacks or the white ethnics." " R o m a n Catholic. E c k s t e i n ' s study. This sort of selection is k n o w n as d r a w i n g a stratified sample.

or w h a t e v e r ) . we k n o w that the t w o differ on the v a r i a b l e and w h i c h one has the h i g i e r s c o r e . Note that s u c h a unit w a s l a c k i n g in the two e x a m p l e s of ordinal m e a s u r e m e n t . p r e c i s i o n in m e a s u r e m e n t . we m a y label people " C a t h o l i c . this is ignored and they are treated simply as interval scales. " and so on. T h e most primitive w a y to measure a variable is s i m p l y to a s s i g n the individuals being studied to categories of that variable. and interval measurement. B u t w e a l s o f i n d v a r y i n g l e v e l s o f information i n the p r o c e s s o f m e a s u r e m e n t itself. e a c h of w h i c h is qualitatively different f r o m the others. if it w e r e . governmental expenditure on a p r o g r a m ( w i t h the unit e x p r e s s e d either in dollars or in percentage points if m e a s u r e d as a percentage of the budget). s u c h as " l o w e r / w o r k i n g / m i d d l e / u p p e r " . if we had an interval measure but c h o s e to ignore our k n o w l e d g e of the distances involved between s c o r e s . we w o u l d still have a n o m i n a l m e a s u r e . If.62 Chapter 5 Problems of Measurement 63 PRECISION IN MEASUREMENT T h e sort o f p r e c i s i o n d i s c u s s e d i n the p r e c e d i n g s e c t i o n . n u m b e r of p l a n e s . p r e c i s i o n i n m e a s u r e s . As a definition of " p r e c i s i o n in measurement. then (1) if the v a r i a b l e is m e a s u r e d n o m i n a l l y . we w o u l d s i m p l y m e a s u r e everything at the interval l e v e l . S o m e variables c o m m o n l y m e a s u r e d i n intervals are i n c o m e ( w i t h the unit e x p r e s s e d in d o l l a r s ) . we have a c h i e v e d ordinal m e a s u r e m e n t . B u t the next sort. we must have s o m e c o m m o n unit of m e a s u r e m e n t for our s c a l e . and (2) party identification. but it does not tell us anything about how the categories relate to e a c h other. are n o m i n a l . if two i n d i v i d u a l s h a v e different s c o r e s on a v a r i a b l e . T h e three major levels of measurement. T h i s is c a l l e d nominal measurement. In s u c h m e a s u r e m e n t . " P r e c i s i o n in m e a s u r e s " referred to k e e p i n g distinctions as fine as w a s p o s s i b l e and p r a c t i c a l . that i s . m e a s u r e d in s o m e w a y . they c o n s i s t of gradations w h i c h . and so on. N o m i n a l measurement places individuals into distinct categories of a variable. we k n o w that the t w o differ on the v a r i a b l e . F o r e x a m p l e . w e w o u l d still have scores o n a n ordinal measure." we c a n say that measurement is relatively precise insofar as it operates at a level that is relatively informative. It is clear that these levels of precision c o m p r i s e a nesting progression. we have s o m e i d e a of a s c a l e that might ideally represent the variable. A n d i f w e had scores on an ordinal measure but c h o s e to ignore our knowledge of the relative Innate Nature of Levels of Measurement O b v i o u s l y . We can assign a score of zero to some point on the scale. percent voting by districts ( w i t h the unit e x p r e s s e d in percentage points). w h i c h one h a s the h i g h e r s c o r e . we c a n rank the categories a c c o r d i n g to " h o w m u c h " of the variable they e m b o d y . figured importantly in data analysis in the social sciences. we k n o w that the two differ on the v a r i a b l e . we c a n say h o w great the differences b e t w e e n the s c o r e s are. w h i c h is in turn more precise than n o m i n a l measurement. in addition to a s s i g n i n g categories. w e m a y label them " B r i t i s h . We are then said to have "ratio" measurement (because we can then take one score as a multiple of another score. m e a s ured as "strong Democrat/weak Democrat/independent/weak Republican/strong r a n k i n g on the variable. and h o w m u c h h i g h e r that s c o r e is than the other p e r s o n ' s . in p r i n c i p l e . to measure religion. 4 Figure 5-3 Levels of Precision To put it a little differently. a set of distinct c l a s s e s to w h i c h individuals w e r e a s s i g n e d . cannot. and all ordinal measurements are also n o m i n a l measurements. in that it contains information not i n c l u d e d in the others. c a n be s u b d i v i d e d infinitely. " " R u s s i a n ." T o measure nationality. which is not possible if there is no zero point). then. 4 . " I f units exist b y w h i c h w e c a n m e a s u r e the intervals b e t w e e n s c o r e s l i k e these. air p o w e r (with the unit e x p r e s s e d in kilotons of b o m b i n g c o v e r a g e . a n d (3) if the v a r i a b l e is m e a s u r e d intervally. w e have a c h i e v e d interval m e a s u r e m e n t . Generally. Republican. " "Protestant. " " G e r m a n . as s h o w n in F i g u r e 5-3. c a n be o v e r e m p h a s i z e d ." "Jew." F u r t h e r p r e c i s i o n in m e a s u r e m e n t is p o s s i b l e if in addition to r a n k i n g the scores a c c o r d i n g to " h o w m u c h " of the variable they represent. w h i c h c a n b e e x p r e s s e d i n thousands o f dollars but A further refinement in precision is possible if in addition to measuring the length of intervals along a scale. " nor c o u l d w e say whether the difference between " w e a k R e p u b l i c a n " and "strong R e p u b l i c a n " w a s greater than that b e t w e e n " i n d e p e n d e n t " a n d " w e a k R e p u b l i c a n . All of the interval scales I noted above could in fact be treated as ratio scales. as yet." or "other. To do this. the l e v e l at w h i c h we m e a s u r e things is not j u s t a c h o i c e we m a k e . ordinal. (2) if the v a r i a b l e is m e a s u r e d o r d i n a l l y . T h e r e are b a s i c a l l y three different w a y s to m e a s u r e . T h a t i s . and the scores we a s s i g n to i n d i v i d u a l s s h o w w h e t h e r they fall h i g h e r or l o w e r than others on s u c h a s c a l e . A l l interval measurements are also ordinal measurements and n o m i n a l measurements. We c o u l d not say whether the difference in status b e t w e e n " w o r k i n g " and " m i d d l e " w a s greater than that between " l o w e r " and " w o r k i n g . body counts (with the unit e x p r e s s e d in n u m b e r of dead p e o p l e ) . however. E x a m p l e s are (1) s o c i a l status. S o m e m e a s u r e s are continuous. A n e x a m p l e i s i n c o m e . I have not included ratio measurement in my discussion here because it has not. Interval measurement is more precise than ordinal measurement.

( B ) age is c o l l a p s e d to an ordinal m e a s u r e and the k n o w l e d g e of unit distance in it is lost. " no unit presents itself by w h i c h categories ( s u c h as " A f r i c a n . and so o n . the l e v e l o f m e a s u r e m e n t w e u s e i s g e n e r a l l y not a p e r s o n a l c h o i c e but i n h e r e s in the thing we are m e a s u r i n g . A m o r e important r e a s o n for f o l l o w i n g the C a r d i n a l R u l e . and so o n . I n F i g u r e 5 . It is for this r e a s o n that p r e c i s i o n i n m e a s u r e m e n t i s more important than p r e c i s i o n i n m e a s u r e s . T h e r e are t w o c a v e a t s to this statement. we c a n do 30 40 50 Age 60 70 80 90 25 •£ 5 0 C D O C D Q. W i t h more p r e c i s e m e a s u r e m e n t we c a n do a greater variety of things w i t h our data a n d thus h a v e more of an opportunity to develop interesting theories. arises if we are interested in u s i n g the m e a s u r e s to study a relationship b e t w e e n two or more variables. writing reports. given the subject y o u are studying. A g e is often grouped into categories s u c h as " y o u n g e s t . do not waste information by imprecise measurement. into dollars. B u t i n a s m u c h as I argued earlier that for practical purposes precision in measures might sometimes be sacrificed. We c o u l d use the n u m b e r of elections that a person h a d experienced as a m e a s u r e of his political exposure.a g e d . the relationship between age a n d percent voting. T h e s e consist naturally of separate points that cannot be further divided. " Low Age The Sin of Wasting Information T i m e and again data c o l l e c t e d at a higher l e v e l of p r e c i s i o n are c o l l a p s e d to a l o w e r level to s i m p l i f y h a n d l i n g the data. " and " o l d e s t " — a n ordinal m e a s u r e ." " m i d d l e . with a few exceptions." Esthetically.l e v e l m e a s u r e (the next s e c t i o n . " T h e S i n o f W a s t i n g I n f o r m a t i o n . we c o u l d not subdivide elections into half-elections. O r i n c o m e i s grouped into "low. w h i c h w o u l d produce an interval m e a s u r e . w h i c h w e l o o k e d at earlier. " "religion.64 Chapter 5 Problems of Measurement 65 w h i c h c o u l d be s u b d i v i d e d into hundreds of d o l l a r s . M e a s u r e s that are not continuous are c a l l e d discrete. it appears that by itself the esthetic consideration is not compelling. T h i s alone should be enough to justify our C a r d i n a l R u l e : " U s e measurements that are as precise as possible. s u c h as "elections. is presented in three f o r m s . that all c o n t i n u o u s m e a s u r e s c a n b e e x p r e s s e d a s i n t e r v a l m e a s u r e s . " E n r i c h i n g the L e v e l o f P r e c i s i o n i n M e a s u r e m e n t . T h e r e are a few discrete m e a s u r e s that are interval. for e x a m p l e . it w o u l d seem better to k n o w more about a subject rather than less. L e t m e provide an e x a m p l e . one that applies only to p r e c i s i o n in m e a s u r e m e n t . " a r g u e s a g a i n s t t h i s ) . W e s e e . however. into tenths of a cent. " " y o u n g e r m i d d l e . s u c h as " r a c e . In this c a s e ." discrete variables are by their nature either ordinal (if they have an ordering to them) or n o m i n a l . into c e n t s . a n d ( 2 ) sometimes w e c a n a d d other i n f o r m a t i o n or theory to a n a t u r a l l y n o m i n a l or o r d i n a l m e a s u r e a n d e n r i c h it to a h i g h e r l e v e l than its natural l e v e l : this is d i s c u s s e d in the s e c t i o n after next.a g e d . F o r most discrete variables. a n d ( C ) age is maintained at an interval l e v e l of m e a s u r e m e n t .4 100 Age and Participation in 2 0 0 2 Election 100 m e a s u r e . c 75 100 (C) Age measured intervally Age High 94 ." "region. T h u s . a c c o r d i n g as ( A ) age is c o l l a p s e d to a n o m i n a l m e a s u r e and all k n o w l e d g e of r a n k i n g and unit distance in it is lost. A l l continuous m e a s u r e s lend t h e m s e l v e s naturally to interval m e a s u r e m e n t b e c a u s e their infinite d i v i s i b i l i t y requires that there be s o m e unit of m e a s u r e m e n t that c a n be d i v i d e d ." T h e measure w o u l d be interval but w o u l d not be continuous. are periodic events that cannot be subdivided. then. with "five e l e c t i o n s " being two units higher than "three elections. a n d treat i t a s a l o w e r ." or " s o c i a l c l a s s . of course. w h e r e a s a l m o s t all d i s c r e t e v a r i a b l e s are n a t u r a l l y either o r d i n a l o r n o m i n a l .4 . or anything like that. A s a r e s u l t .A m e r i c a n " and " w h i t e " ) c a n be c o m p a r e d . E l e c t i o n s ." T h i s esthetic consideration applies both to precision in measures and to precision in measurement. h o w e v e r : (1) W e c o u l d a l w a y s i g n o r e i n f o r m a t i o n w e h a v e about a qualitatively different things w i t h data m e a s u r e d at different levels of p r e c i s i o n . Figure 5 . " and "high. " "older m i d d l e .

Enrichment of the Level of Precision in Measurement So far I have argued that we should try not to drop data carelessly to a lower level of measurement. This suggests an interesting combination of processes: Learning and acquiring the habit of voting during the first years of eligibility causes participation to rise rapidly at first after an initial pause. and that participation peaks around age 70. be interested in whether or not the colonizing country tried to assimilate the native population to its own culture and the effect this had on the stability of the resulting institutions. If age were measured intervally. by raising the level of measurement. for instance. then. One way to do this would be simply to treat the problem as one of a relationship involving nominal measurement. we would expect about the same percentage of individuals in each category to vote. This is indicated by the fact that different age groups participate at different rates. with ordinal measurement. We can now see that the initial increase in participation. that pattern is one of increasing participation with increasing age up to some point. F. Only 16 percent of the individuals in category C vote. we may sometimes be able to do this. For this particular purpose. If you can array the different mother countries along this scale. you want to compare the stability of democratic institutions in former French. and then come B. Often. this is all we can say—that the variables are or are not related. we could say still more about the relationship. though. G. it is followed by a decline. we could see the pattern of the relationship. and D. measured in a common unit. The dashed line tracing the path of the bars approximates the pattern we have already seen in Figure 5-2. for example. we can see the pattern of the relationship. with age measured nominally. years: C E B G A F D 18-20 21-23 24-30 31-50 51-64 65-80 81-91 Examples of Enrichment Example 1. untidy measurement at a higher level is better than neat measurement at a lower level. A." the British colonial experience would rank low and the French quite high. You might. In the second case. we can say a good deal more. you can use them as an ordinal or interval measure of the scale. perhaps due to enfeeblement. . British. with the United States' possibly lower than the Dutch. On the scale "level of assimilation. we can see that there is a relationship between age and participation in elections. depending on which country had done the colonizing. especially a finer breakdown of ages above 40. as compared with 86 percent of those in category F. the information lost through the low precision in measures in part C is slight compared with the information lost by lowering the precision in measurement in parts A and B. Preserving a high level of precision in measurement deserves a strong priority in research. and American colonies. Interval measurement adds information about the magnitude of differences in age expressed by the categories. This is a richer description of the relationship than we could have gotten using the ordinal measurement. In Figure 5-4(c). we can enrich our data in this way if we know something more about the data than is reflected in our measurement—in other words. If age were measured intervally. the bars in the graph have been stretched to take into account the added information that the categories represent the following age groups. But it is likely that you have in mind some underlying scale along which the colonial experience varied. Notice that in part C of Figure 5-4. between the ages of 20 and 30. and we also could see the rate at which one variable changed in response to changes in the other variable. We now know the ranking of the different categories of age: C is youngest. A few examples may be the best way to show how measurement can be "enriched" in this way. greater precision in measures also would have been useful. If you could make a reasonably informed guess at the relative differences (such as Dutch rather close to the French. By a similar argument. E the next to youngest. we probably would have measured at the higher level in the first place. and that you are really interested in the relationship between this scale and the stability of democratic institutions. for example. as this increase levels off. with participation decreasing as age increases. If age and participation were not related. Dutch. You are studying the relationship between the colonial experience of new nations and the stability of democratic institutions in those nations. the colonizing countries comprise at least an ordinal measure of the level of assimilation. That is. large gap between British and American). you might even be able to bring your data up to the level of rough interval measurement. In this case. In addition to seeing whether or not the variables are related. we can see that the peak occurs in the late 70s. If it were. if we have information about the data that would otherwise be wasted. Generally. is slow. With nominal measurement. we could see whether or not there was a relationship. With the greater precision in measures of Figure 5-2. The Dutch and American experiences would fall somewhere between these.66 Chapter 5 Problems of Measurement 67 In the first case. and then a reversal. clean measurement at the higher level. However. Given a certain amount of boldness and ingenuity. This information may not be enough to provide neat. after which there is a decline. that there is a big jump over about the next ten years. we should always try to raise data to a higher level when this is possible.

Alternatively. build involvement a m o n g their m e m b e r s ." see the box on p. If so. abstracting the d e g r e e of " c l o s e n e s s " from religious affiliation suggests o n e ready interval m e a s u r e — a v e r a g e size of c o n g r e g a t i o n s — a l t h o u g h this admittedly w o u l d be only a r o u g h m e a s u r e w h o s e validity w o u l d be questionable. low on assimilation but high on oppression. general university. it w a s found that 39 percent of the m i d s h i p m e n thought the A m e r i c a n military b u d g et was too small. as in the next example.83.83 times as great as the interval between R O T C and n o n . B. old-Reformation Protestant. or whatever. R o m a n Catholic. as a m e a s u r e of different underlying scales. a n d a c a d e m y . Problems of Measurement general university. this information w o u l d have a l l o w e d y o u to reconstruct the interval measure implicit in the ordered variable. extent of oppression. 1971). 3 9 . you might have s o m e additional information on the subject. ROTC. for e x a m p l e . (For a m o r e detailed explanation of "linear relationship. but the Naval A c a d e m y increased budget support by 29 percentage points over R O T C students. F o r instance. T h e s e form an ordinal measure of "militariness" readily enough. near the top. The various types of colonial experience we cited vary on degree of assimilation.0 t o n o n . You can distinguish three types of Notice that the manner in which measurement of nominal variables has been enriched in these examples parallels the "dimensional analysis" that I urged in Chapter 3.R O T C p r o g r a m lowest. y o u can arbitrarily pick the unit in w h i c h t o e x p r e s s y o u r m e a s u r e . Example 4 . p r o b a b l y w o u l d fall a b o v e t h e m i d d l e i n a s s i m i l a t i o n . 3 2 . t h e scores a s s i g n e d m i g h t b e n o n . you well might want to use a given nominal m e a s u r e m e n t twice or educational programs: military academy. You are studying the 5 R O T C . and the Naval A c a d e m y at C. Of course. You might be thinking in terms of the extent to which one's religion promotes an apocalyptic view of the universe. N a v a l A c a d e m y w o u l d t h e n h a v e t o b e 2 7 . a n d w h a t e v e r l i n e a r relationship between the t w o variables exists. B ' .0 to non- tend to encourage pro. J e w s w o u l d fall n e a r t h e b o t t o m o f s u c h a s c a l e and fundamental Protestants. to enrich ordinal data. 0 t o R O T C . you do not have units in advance by which to measure military c o n t e n t . you have in mind an underlying d i m e n s i o n that causes y o u to expect a relationship b e t w e e n the two variables. as did 4 percent of the n o n . Y o u m i g h t a s s i g n 1. 10. speed of economic development. B e c a u s e an interval m e a s u r e d o e s not a s s u m e that a zero point is k n o w n .83. You w a n t to m e a s u r e the extent to w h i c h different educational p r o g r a m s A c a d e m y . T h u s y o u m i g h t be interested in predicting the stability of d e m o c r a t i c institutions from t w o variables simultaneously: (1) the level of assimilation at w h i c h the colonizer a i m e d . a rather p r e c i s e p i e c e of o u t s i d e i n f o r m a t i o n w a s u s e d to tell h o w long t h e intervals s h o u l d b e . T h u s .or antimilitary attitudes. then as in E x a m p l e 1. nominal classifications involve an infinite number of dimensions. Perhaps. Belgium. In a study of A n n a p o l i s m i d s h i p m e n . J e w s w o u l d fall l o w o n t h e " a p o c a l y p t i c " s c a l e b u t h i g h o n t h e "closeness" scale. and C.R O T C students. N o t e that the "right" ordinal or interval arrangement of the n o m i n a l categories d e p e n d s on which underlying scale y o u w a n t t o t a p . you could take the different religions ranked by their apocalyptic content as an ordinal o r i n t e r v a l m e a s u r e o f t h i s s c a l e . In working with ordinal variables (either "enriched" nominal variables or variables which obviously are ordered from the outset). T h e U n i t e d States. however. we m u s t use a g o o d deal of ingenuity.R O T C a n d 2 . C a n y o u m a k e a n interval m e a s u r e out o f this r a n k i n g ? Fortunately. b u t l o w i n o p p r e s s i o n . t h e ratio o f t h e distances is 4. a n e s t i m a t e of the relative differences (intervals) b e t w e e n scores of the measure. without participation in R O T C . Abstracting the appropriate ordered variable(s) from a nominally measured variable is much the same as abstracting the appropriate dimension(s) from a set of multidimensional concepts. 10 percent of the R O T C students agreed. F r a n c e . These scores are A ' . high in assimilation and toward the middle in oppression. and the Naval alienation. 5 R O T C a n d 8. Often. with the military academy highest and the n o n .R O T C . and (2) the extent of oppression and violence during the colonial period.0 t o R O T C . your theory might be that religious groups.) T h i s g i v e s y o u all y o u n e e d t o c o n s t r u c t a n interval m e a s u r e — t h a t is. In E x a m p l e 3. y o u c a n p r o j e c t a c r o s s t h e g r a p h t o s e e w h e r e t h e y s h o u l d fall o n t h e l i n e showing the relationship. a c o m m o n unit m a y appear immediately. This is demonstrated geometrically in Figure 5-5. simply as social organizations. R O T C . N o w look d o w n the graph to see what scores on "military content" must be associated with positions A.68 Chapter 5 Further. a n d n o n .R O T C students (Karsten and others. and thus each w o u l d represent a different m i x of the t w o variables. 69 with participation in m o r e in the s a m e analysis. then (given your assumptions) the interval b e t w e e n military academies and R O T C must have been 4. You might assign 4. multidimensional concepts. If you were willing to a s s u m e that at that time the Naval A c a d e m y was typical of the military academies and that there was a linear relationship between the promilitary orientation of an educational relationship between religion and political p r o g r a m a n d the p e r c e n t a g e o f its s t u d e n t s w h o b e l i e v e d that t h e m i l i t a r y b u d g e t w a s too small. geopolitical position. M o s t s i m p l y . O f t e n s u c h a p r e c i s e g u i d e is s i m p l y . It should be clear that the s a m e p r i n c i p l e w o u l d h o l d for a n y linear relationship. Religion could simply be measured nominally (Jewish. and you can treat the variables readily and directly as interval m e a s ures. In this case you could array the different religions in t e r m s of the size a n d " c l o s e n e s s " of their congregations. the distance from B' to C must be a p p r o x i m a t e l y five t i m e s t h e d i s t a n c e f r o m A ' t o B .R O T C students. 4 . Like vague. B e c a u s e y o u know the scores on budget support for non-ROTC.R O T C . b u t i t i s a p p a r e n t t h a t w h a t e v e r u n i t s y o u c o n c e i v e of. and so on). Example 2. ROTC at B. and C on the line. (To b e exact. Here I have drawn two of the m a n y possible linear relationships between the "military c o n t e n t " of e d u c a t i o n a n d support for the defense budget. h o w d e m o c r a t i c their legal structure is. in w h i c h case Naval A c a d e m y w o u l d have to be 6. Non-ROTC falls at A. or what have you. Example 3 . for instance. 71) If the degree of promilitary orientation in R O T C programs increased support of higher budgets by 6 percentage points over n o n . The colonizers w o u l d be arrayed differently along these t w o scales. a n d so o n . R O T C .

if x is h i g h . I think the a n s w e r is obvious. Y o u m i g h t c h o o s e . but i t c o u l d b e r e f i n e d b y b r i n g i n g t o bear the i m p r e s s i o n s o f m o r e k n o w l e d g e a b l e o b s e r v e r s w i t h r e g a r d t o r e l a tive n a t i o n a l p o w e r . y inc r e a s e s b y o n e . a u n i t c h a n g e in x p r o d u c e s relatively g r e a t c h a n g e in y. M o s t i m p o r t a n t . if x is low. O n e m a r k of a good r e s e a r c h e r is that she boldly seeks out all c h a n c e s — n o t j u s t tl le o b v i o u s or safe o n e s — t o r a i s e the level of m e a s u r e m e n t in her w o r k . a n d R u s s i a . even if we must supplement that information with assumptions and h u n c h e s . H e r e . T h e question remains: A r e y o u better off w i t h s u c h rough interval measures. m e a s u r i n g i n this w a y d o e s not i g n o r e the v a l u a b l e c o l l e c t i o n o f o u t s i d e i n f o r m a t i o n — m o s t o f i t admittedly u n d i g e s t e d — that y o u a l r e a d y p o s s e s s . Great Britain. C o n s i d e r a p r o b l e m i n w h i c h y o u w a n t t o m e a s u r e the n a t i o n a l p o w e r o f Costa R i c a ." and "military power. T h e interval measures y o u derive in E x a m p l e s 3 and 4 m a y not leave y o u fully satisfied. there is plenty of r o o m for errors to occur. It m a k e s sense to use all the information we have about a question. inasmuch as the relationship between the two variables is different for different v a l u e s of x. 10. regardless of the value of the s e c o n d v a r i a b l e . T h e r e l a t i o n s h i p in g r a p h B is n o n l i n e a r . Spain. T h e alternative is to ignore a portion of w h a t we know.70 Chapter 5 Problems of Measurement 71 than others. On the other h a n d .h a l f unit w h e n x c h a n g e s b y o n e unit. T a k i n g the " s a f e r " course a l l o w s us to state our results with more confidence but reduces the importance of what we have to say. S p a i n . t h o u g h quite r o u g h . and y o u k n o w w h i c h i n t e r v a l s are greater 'Although I have dubbed the variable "national power. T h e r e l a t i o n s h i p in g r a p h B m u s t be e x p r e s s e d by a c u r v e w h i c h tilts differently at different v a l u e s of x." "economic power. T h e r e l a t i o n s h i p in g r a p h A is linear. the r e l a t i o n s h i p i n g r a p h A can b e e x p r e s s e d b y a s t r a i g h t line. n o m a t t e r w h a t v a l u e x starts w i t h . and the assumptions required for it. E n r i c h i n g m e a s u r e m e n t in this w a y is a r e s e a r c h technique for w h i c h the r e s e a r c h e r ' s creativity and ingenuity are c r i t i c a l . i n a s m u c h as t h e r e l a t i o n s h i p is the s a m e at all v a l u e s of x. a unit c h a n g e in x p r o d u c e s relatively little c h a n g e in y. or s h o u l d y o u opt to ignore the size of intervals and treat the variables as ordinal m e a s u r e s ? G i v e n the wider range of theoretical c o n c e r n s that one c a n c o v e r using interval data. C o s t a R i c a . T h e m e a s u r e i s r o u g h a n d s u b j e c t i v e ." for most purposes this would be broken down more profitably into component dimensions. B e c a u s e of the indirect c h a i n of reasoning in E x a m p l e 3. such as "diplomatic influence. G r e a t B r i t a i n . but y o u still c o u l d k n o w m o r e than n o t h i n g about the length o f the i n t e r v a l s . interval m e a s u r e o f the n a t i o n a l p o w e r o f the v a r i o u s c o u n t r i e s s i m p l y b y a s s i g n i n g arbitrary interval lengths i n a c c o r d a n c e w i t h y o u r g e n e r a l i d e a o f the m a g n i t u d e s i n v o l v e d . u n a v a i l a b l e . and because of the arbitrary decisions to be made in E x a m p l e 4. and R u s s i a . 6 T h e order i s c l e a r ." . Linear Relationship A r e l a t i o n s h i p b e t w e e n t w o v a r i a b l e s is linear if o n e v a r i a b l e c h a n g e s by a c o n s t a n t a m o u n t w i t h a c h a n g e of o n e unit in the second variable. 1 . Y o u c o u l d create a n a c c e p t a b l e . for e x a m p l e . 7 . a n d a t i m i d r e s e a r c h e r w o u l d be content to treat this as an o r d i n a l v a r i a b l e . y o u k n o w that the intervals are not e q u a l . T h u s . 4 0 .

To return to the quotation from Eckstein with which I introduced this chapter. is by Carter (1971). In this chapter I pointed out that the different levels of measurement were important because it was possible to make a greater variety of statements about relationships between variables that were measured in "advanced" ways. given ratio measurements. One difference between interval and ratio measurement is that whereas it is possible to add and subtract interval measures. It is possible to say that a given score is twice as great as another score. More important. Being a "quantifier" or "nonquantifier" is not an either/or question. but in a different way. Another excellent article. Most of these descriptions had to depend on his subjective judgment rather than on impersonal outside indicators. Personally. Why would the technique in Example 3 not be appropriate if the relationship between "military content" and "support for defense budget" were nonlinear? What might you do in this case? 2. His subject matter. It is still based on subjective descriptions. Fahrenheit temperature is an example of an interval measure that is not a true ratio measure. the less likely he is to press for the interests of his region. the less likely he is to press for the interest of his region. one should be concerned about the possibility that "enriching" a level of measurement may insert either sort of error into the new measure. made in the absence of objective indicators. I have no idea whether this is a reasonable theory. such as "rather strong sense of community. many scholars believe that the costs in carrying out such "enrichment" are greater than the benefits. even though "zero" is arbitrarily assigned to one point on the scale. that the shift continues from there on up the hierarchy at a diminished rate. Working more boldly and imaginatively. the process reverses itself. I think the risk is often justified. a position that would not be accepted by all social scientists. he could use only vague descriptions. Can this sort of work be described in terms of precision in measures and precision in measurement? Do the rules I have prescribed in this chapter hold for this sort of work as they do for more quantitative studies? First. This is certainly a richer statement than the earlier version. especially with numerical data). But the same general conclusions hold for "nonquantifiers" as well. To the extent that one is worried about measurement error.72 Chapter 5 Problems of Measurement 73 Quantifiers and Nonquantifiers Again This discussion of precision in measures and precision in measurement has centered on the work of "quantifiers" (those who work by preference with objective data. and divide ratio measures. Tufte's "Improving Data Analysis in Political Science" (1968-1969). it is clear that he was interested in doing the same things as quantifiers do. The extent to which a researcher uses objective data is generally not a matter of personality but rather of the subject he is studing and the data available for that subject. we may note that measuring variables nonquantitatively requires the same sort of boldness that I urged in enriching data to a higher level of measurement. not of precision in measurement. I have constructed it merely as an example of the wide applicability of pseudo-interval measurement. another student might approximate interval measurement. that a temperature of +20°F is half as "hot" as +40°F. for example. however. this is not possible with simple interval measurement. A fairly technical paper that urges researchers to take seriously the risk of inserting measurement error by treating ordinal variables as if they were interval (an extrapolation similar to enrichment but riskier and less well thought out) is that by Wilson (1971). She might state. in addition to being able to measure the relative interval between different values of the measure. This statement approximates a relationship between two variables measured at the ordinal level ("height in the hierarchy" and "likelihood of lobbying"). He wanted to measure variables and to assess relationships between those variables. that near the top of the hierarchy. but it is no more "quantified" than the other. but there is no particular reason why such research cannot approximate interval measurement. we can see that this is primarily a problem of precision in measures. The only difference is that in the second case the researcher enriches the level of measurement at which she is working. For example. You should note that in this chapter I have presented a fairly radical position in favor of upgrading levels of measurement. it is possible to assign the value zero to some point on the measure (see footnote 3). A nonquantifier can measure things only approximately. It is not true. that this shift occurs very sharply as he moves from some particular level of the hierarchy to another. however. "Ratio" measurement is interval measurement for which. a student of Chinese politics might suggest that the higher a Chinese politician rises in the hierarchy. either random or nonrandom. with the men at the top feeling secure enough to concern themselves more than those immediately below them with the interests of their regions." "highly developed team spirit. For many variables in which he was interested. for example. 8 7 . rarely allowed him to use objective or numerical measures. Two questions you might consider are: 1. that the higher a Chinese politician rises in the hierarchy. and finally. What statements might be made about relationships between ratio-measured variables that could not be made about relationships between interval-measured variables? Let me repeat my earlier admonition." and so on. it is possible to add and subtract. 7 8 FURTHER DISCUSSION A readable presentation along the lines developed in this chapter is Edward R. a bit technical. simply because it was more convenient to introduce it in that way. multiply.

the a s s e r t i o n of a causal relationship differs f r o m the m e r e a s s e r t i o n of a relationship. If. are m o r e apt than brunettes to be R e p u b l i c a n s . A good e x a m p l e of the difficulty of a s c r i b i n g c a u s a l direction is the relationship between central bank independence and inflation. inflation is low. T h i s interpretation is a subjective one. in addition. It is not m e r e l y true that the t w o v a r i a b l e s tend to c o i n c i d e . We cannot a l w a y s m a k e a c a u s a l interpretation w h e n two p h e n o m e n a tend to c o i n c i d e . T w o variables are related if certain v a l u e s of one variable tend to c o i n c i d e w i t h certain v a l u e s of the other v a r i a b l e . we might c o n c l u d e that s m o k i n g c a u s e s heart d i s e a s e . noted a b o v e . As I pointed out earlier. the r e l a t i o n s h i p is a causal relationship. W i n t e r does not c a u s e s p r i n g . T h e example o f c l a s s a n d v o t i n g . that c l a s s d o e s not produce the vote. T h u s . In both c a s e s . and these same groups. C o n s i d e r two further e x a m p l e s : I f w e see that people w h o s m o k e regularly h a v e a greater i n c i d e n c e of heart d i s e a s e than n o n s m o k e r s . that p e r s o n might say. we s a y that s o c i a l c l a s s is a " c a u s e " of party vote. we must add something more. E m p i r i c a l . 74 . r e l i g i o u s . i s a n e x a m p l e o f a c a u s a l r e l a t i o n s h i p . although the one f o l l o w s the other regularly. R a t h e r . although it is probably true in the U n i t e d States that b l o n d s . C e r t a i n l y . T h e notion of c a u s e i n v o l v e s m o r e than that.Chapter 6 Causal Thinking and Design of Research CAUSALITY: AN INTERPRETATION 75 T h e m o s t important thing to note about c a u s a l t h i n k i n g is that it is an interpretation of reality. and then f o l l o w this u p w i t h s o m e ideas o n r e s e a r c h d e s i g n . w h i c h is a rather o b j e c t i v e statement. To qualify as a " c a u s a l " relationship. G e n e r a l l y speaking. T h e question of whether the relationship is a c a u s a l one. a l s o tend to be w o r k i n g c l a s s .o c c u r r e n c e . In this chapter we first d i s c u s s the i d e a of c a u s a t i o n . It might be a r g u e d . and r a c i a l c o n f l i c t s that h a v e s u r f a c e d often i n A m e r i c a n p o l i t i c s . To read causality into s u c h c o . the c o i n c i d e n c e of two p h e n o m e n a must i n c l u d e the idea that one of them produces the other. In a v a l u e s of the other v a r i a b l e . they tend to c o i n c i d e because values of the one tend to produce distinct values of the other. and of w h i c h variable c a u s e s w h i c h . In general. w h e r e central b a n k s are independent. dependent v a r i a b l e s (those we think of as being " p r o d u c e d " ) . all that we k n o w directly from our data is that two variables tend to o c c u r together. S i m i l a r l y . b e c a u s e those w h o vote R e p u b l i c a n tend to be f r o m the m i d d l e and upper c l a s s e s . If we see that those U . senators w h o c o n f o r m to the i n f o r m a l rules of the Senate tend to be the ones w h o s e bills get p a s s e d . 14). e c o n o m i s t s think that if central b a n k s ( s u c h as the F e d e r a l R e s e r v e in the U n i t e d States) are relatively independent of political control. but that both are p r o d u c e d by s o m e t h i n g e l s e — t h e e t h n i c . a theory in its s i m p l e s t f o r m u s u a l l y c o n s i s t s of three things: independent variables (those we think of as d o i n g the " p r o d u c i n g " ) . T h e r e f o r e . In the e x a m p l e of s o c i a l c l a s s and the vote. c o n f o r m i t y and s u c c e s s ) . and h o w we interpret it. they w i l l use monetary p o l i c y more a g g r e s s i v e l y to fight inflation. requires an interpretation. we might c o n c l u d e that c o n f o r m i t y is r e w a r d e d in the Senate. for i n s t a n c e . It is time n o w to l o o k m o r e c l o s e l y at what a " r e l a t i o n s h i p " i s . W e f e e l that there is s o m e t h i n g about a p e r s o n ' s s o c i a l c l a s s that m a k e s that p e r s o n m o r e l i k e l y to vote in a c e r t a i n w a y . the c o i n c i d e n c e o f c l a s s and party votes i s j u s t t h a t — a c o i n c i d e n c e . hair c o l o r does not c a u s e political party preference. T h u s s o m e o n e c o u l d a r g u e — a n d i t w o u l d not be an u n r e a s o n a b l e a r g u m e n t — t h a t c l a s s is not a c a u s e of the vote. w h o are relatively l i k e l y to be w h i t e and Protestant. T h e r e f o r e . it is possible to design the research in w a y s that c a n help us significantly in doing this. S . W e have seen m a n y s u c h e x a m p l e s o f relationships i n earlier chapters. T h e question of whether or not there is a relationship is objectively T h u s far everything we have l o o k e d at in this book has been j u s t i f i e d in terms of h o w u s e f u l it is in establishing relationships b e t w e e n v a r i a b l e s . for i n s t a n c e . s o c i a l c l a s s and party vote are related in the U n i t e d States. s i m p l y by a c c i d e n t . but a c a u s a l interpretation of this r e l a t i o n s h i p is m u c h m o r e s u b j e c t i v e . theory-oriented p o l i t i c a l r e s e a r c h i s almost e x c l u s i v e l y c o n c e r n e d w i t h causal relationships. O u r interpretation of this is that one of the p h e n o m e n a c a u s e s the other. D i s t i n g u i s h i n g b e t w e e n this v e r s i o n of the r e l a t i o n s h i p and the m o r e c o m m o n v e r s i o n is at least partly a matter of j u d g m e n t . and c a u s a l statements l i n k i n g the t w o (refer again to p. R e s e a r c h d e s i g n — t h e w a y i n w h i c h w e structure the gathering of d a t a — s t r o n g l y affects the c o n f i d e n c e w i t h w h i c h we c a n put a c a u s a l interpretation on the results of r e s e a r c h . In this regard. A l m o s t every situation in w h i c h we w i s h to m a k e c a u s a l statements is similar to this example. although as y o u w i l l see. we o b s e r v e that two p h e n o m e n a tend to o c c u r together ( s m o k i n g and heart d i s e a s e . c e r t a i n ethnic groups tend to vote for the D e m o c r a t s . we c o n s i d e r that the values of one v a r i a b l e produce the testable. there is o b j e c t i v e e v i d e n c e for the e x i s t e n c e QJ o) of a r e l a t i o n s h i p .

Accordingly. 94. and now blonds and brunettes no longer differ in their politics. and compared blonds and brunettes. Coincidence without cause gives you no lever. Reviewing the history of the 1920s and 1930s in Latin America. the three countries whose central banks were most independent had averaged only about 3. A slight variation of this often occurs in the social sciences. If voters who like the Republican party also tend to oppose welfare programs. In effect. But there is always one question about causation that remains purely subjective and that cannot be resolved completely by any techniques of data handling. we can settle some of the problems in making causal interpretations. There are three interpretations available to us. Although this convention frequently simplifies things for the researcher. November 20. 1993. In our previous example of hair color. in which variables usually are measured just once and in a single interview. The phenomena coincide because of logical necessity. 3 'The Economist. rather than vice versa. simply because they measure the same thing. if we artificially hold social position constant by looking only at WASPs. we can see whether it has led the original two variables to coincide in such a way as to resemble a causal relationship. It generally is not enough for us to note that two phenomena coincide. the future event caused the prior prophecy to occur. But by setting up a study in certain ways and by manipulating the data appropriately. We generally also want to interpret why they coincide. 2 Summary Let me pull together the argument to this point. there are many instances in which it cannot be used. 2 The technique of holding constant is discussed in greater detail at the conclusion of the chapter. we assume that pulling a trigger causes the shot that follows.76 Chapter 6 Causal Thinking and Design of Research 77 Thus there are techniques by which we can manipulate our data to eliminate some of the alternative causal interpretations of a relationship. is a case in point. and if we are sure that there is causation between the two. Given that two variables are causally related.75 percent inflation.' We see. If we then found that blond WASPs did not tend more often than brunette WASPs to be Republican. We must decide from outside the data at hand whether a coincidence of two phenomena is of this type or whether it involves causation. it is clear which variable must be the cause. such as foreign policy. whose writers believed that some people could prophesy what was to come in the future. we could infer that hair color did not cause party preference. We can then conclude that hair color does not cause political preference. we do not think of either of them as causing the other. which causes which? We might think that voters choose the party that offers the policies they prefer. that there is a relationship between the two. that is. their coincidence is tautologically determined. This is not because their votes on one issue cause them to vote the way they do on the other. but that when high rates of inflation hit in the mid-1930s. the result. By holding constant the third variable. 3 1993 study. even though it is so well established that it seems "natural" to us—is that causation works forward in time. Causation is not involved at all. if we can establish that change in one of the variables precedes change in the other. but it might also be that voters choose the Republican party for other reasons. We would expect them to coincide. ELIMINATION OF ALTERNATIVE CAUSAL INTERPRETATIONS A causal interpretation is something that cannot come solely from our observations. or whether the coincidence of class and party is due to something else that causes both of them? Remember that the ultimate purpose of theories is to give us levers on reality. which causes which? Do independent central banks give us low inflation. A n example of a cultural tradition in which causation does not necessarily work forward in time is that of the Old Testament. . low inflation rates allowed governments to tolerate independent central banks. then. If A and B coincide but A does not cause B. p. the three countries with the most politicized banks had averaged almost 8 percent inflation from 1955 to 1985. spring follows winter. Rather. In general. as the economists believe. That is. both votes are an expression of their general disposition to spend money on social programs. or does low inflation give us independent central banks? If ascribing cause to the coincidence of two things is so tricky. One useful convention in Western culture—but it is only a convention. most of them took away that autonomy. we can infer that the difference in voting was due not to the difference in hair color (which was merely a coincident variable) but to the difference in socioethnic background. which of the variables causes which? Suppose that two phenomena coincide and that there appears to be a causal relationship between them. for instance. by definition. Survey research. we might have looked only at WASPs. For example. Thus. some basis for choosing how to act. It often happens that two slightly different measures of the same concept coincide. that bank independence causes low inflation? Batalla (1993) suggests that the opposite causal interpretation may be true. So. and then are influenced by the party's leaders to adopt its position on welfare programs as their own. we can use our data to test the relationship. but is it true. Only you can decide which is the cause and which. For example. members of Congress who vote to increase aid to education tend also to support increases in welfare spending. while high inflation rates led the governments to pull the banks under their control. Thus. changing A will not change B. and our choice from among them is ultimately subjective. /. yet we do not think of winter as producing spring. where we think that a third variable is causing two other variables to coincide accidentally. as in this hair color example. he notes that many countries had established autonomous central banks by the late 1920s. why do we bother with the notion of causation in our theories? What difference does it make whether class causes voting for a certain party.

78

Chapter 6

Causal Thinking and Design of Research

79

2. The relationship we observe is a result of outside factors that cause the two phenomena at hand, and thus neither of these phenomena causes the
other. The study of hair color and party preference was an example of this. By setting up the study appropriately, we can control for various outside factors in order to concentrate on the relationship in question. In the hair color example, such a control was used. To this extent, we can see from the data whether the coincidence of the phenomena is of this sort. But we are still not exempt from making assumptions, for we must first have assumed the outside factor(s) causally prior to the two coincident phenomena. This is not always an easy decision to make. If it is possible to set up a true experiment (described in the next section), we can eliminate this possibility. But this is often not possible in "field" social sciences such as political science or sociology. 3. One of the phenomena causes the other. Here we have a true causal statement. We are still not finished making assumptions, of course, for we must decide which of the phenomena is the cause and which the effect. That is ultimately a subjective decision, though often we are aided in making it by the convention that causation must run forward in time. As I have said so often in this book, one of the pleasures of research is that nothing in it is automatic. Even the most "quantitative" techniques do not take away our obligation and our right to be creative and imaginative. The fact that causal analysis is ultimately subjective may trouble us—objectivity always seems more comforting than the responsibility imposed by subjective judgment—but in a way it is also a great comfort, inasmuch as it keeps us, and what we do with our minds, at the heart of our research.

decentralize decision making. After the reform, another week's tabulation shows increased output. Conclusion: Decentralized decision making increases output. 2. Reagan's victory over the Soviet Union. During the Reagan administration (1980-1988) the United States steadily increased its military spending. Over the three years from 1989 to 1991 the Soviet Union collapsed, which conservatives hailed as a victory for Reagan's policies. Conclusion: The economic strain of matching Reagan's military buildup had been too much for the Soviet system, and had led to its collapse and the end of the Cold War. 3. Organizing the poor. In anticipation of a major campaign to organize the poor of a city, a survey is taken among them to measure their interest in politics. At the end of the organizing campaign, the same people are asked the same questions a second time. It turns out that those who were contacted by the campaign workers have indeed acquired an increased interest in politics, compared with those who were not. Conclusion: The campaign has increased the political awareness of the poor.

4. Tax-reform mail.

The Congressional Quarterly reports the proportion of each senator's mail which favored tax reform. Comparing these figures with the senators' votes on a tax-reform bill, we see that senators who had received relatively favorable mail tended to vote for the bill, whereas those who had received relatively unfavorable mail tended to vote against it. Conclusion: How favorable a senator's mail was on tax reform affected whether or not she voted for it. 5. Presidential lobbying. In an attempt to measure his influence over Congress, the president randomly selects half the members of the House. He conducts a straw vote to find out how all the members of the House intend to vote on a bill important to him. He then lobbies intensively among the half he has randomly selected. In the final vote in the House, the group that he had lobbied shifted in his favor compared with what he could have expected from the earlier straw vote; the other half voted as predicted from the straw vote. Conclusion: His lobbying helped the bill. Let us look at the design of these studies to see how many alternative causal interpretations each can eliminate.

A FEW BASICS OF RESEARCH DESIGN It should be evident from the discussion so far that the basic problem in causal analysis is that of eliminating alternative causal interpretations. Whenever two variables vary together (are related, coincide), there is a variety of causal sequences that might account for their doing so. A might cause B, B might cause A, both A and B might be caused by something else, or there might be no causation involved. Our task is to eliminate all but one of these, thus leaving an observed relationship, together with a single causal interpretation of it. Some of these alternatives can be eliminated only if we make assumptions from outside the actual study. But we also can design the study in such a way that certain alternatives are impossible. This will leave an interpretation that is dependent on fewer subjective assumptions and can thus lend a greater measure of certainty to the results. Consider these examples: /. Agency study. An organizational analysis of a government agency is made in which workers keep track of their output for a week. The organization is then restructured to

Designs Without a Control Group In the first two examples, the design is of the form:
1. M e a s u r e the d e p e n d e n t v a r i a b l e . 2 . O b s e r v e that t h e i n d e p e n d e n t v a r i a b l e o c c u r s . 3. M e a s u r e the dependent variable again. 4 . I f t h e d e p e n d e n t v a r i a b l e h a s c h a n g e d , a s c r i b e that t o t h e o c c u r r e n c e o f t h e i n d e p e n d e n t variable.

80

Chapter 6
T h u s , in " A g e n c y Study," (1) the w o r k e r s ' o u t p u t is tabulated; (2) the o r g a n i -

Causal Thinking and Design of Research 81
Nations has affected the foreign policy of every country in the world since 1945. H o w c a n a s t u d e n t o f i n t e r n a t i o n a l p o l i t i c s d i s t i n g u i s h its effect o n f o r e i g n p o l i c y from the effects of the Soviet-American rivalry, the development of atomic

zational structure is d e c e n t r a l i z e d ; (3) the w o r k e r s ' o u t p u t is o n c e a g a i n tabulated; a n d ( 4 ) t h e c o n c l u s i o n i s r e a c h e d . T h i s k i n d o f d e s i g n o p e r a t e s without a control group. A s a r e s u l t , t h e r e a r e a n u m b e r o f a l t e r n a t i v e c a u s a l s e q u e n c e s t h a t m i g h t h a v e p r o d u c e d the s a m e result. F o r e x a m p l e , a plausible alternative explanation for the increased productivity m i g h t b e that the initial m e a s u r e m e n t o f p r o d u c t i o n , i n w h i c h e a c h w o r k e r k e p t track of output for a w e e k , focused the w o r k e r s ' attention on productivity in a w a y that had not been done before, leading them to improve their productivity. In other words, i t w a s n o t t h e d e c e n t r a l i z a t i o n o f t h e a g e n c y , b u t t h e s t u d y itself, w h i c h c a u s e d productivity to rise.
4

w e a p o n s , the l i b e r a t i o n o f f o r m e r c o l o n i e s , a n d s o o n , all o f w h i c h h a v e h a p p e n e d at the s a m e time? O n e cannot, of course. It is simply not possible to provide a control g r o u p of c o n t e m p o r a r y countries for w h i c h the U n i t e d N a t i o n s has not existed. T h e s a m e general alternative explanation (that other things h a p p e n i n g at the s a m e time could have been the true cause) also might have applied to "Agency Study." If something else had happened between the two measurements of

productivity—the weather improved, Christmas came, the president urged greater productivity, or whatever—this might have been the true cause of the increased production. Again, using a second agency as a control could eliminate such alternative e x p l a n a t i o n s . S t u d i e s w i t h o u t a c o n t r o l g r o u p p o p u p all t h e t i m e . O n J a n u a r y 4 , 2 0 0 4 , t h e Minneapolis Star Tribune h e a d l i n e d a f r o n t - p a g e s t o r y : " S t a t e S e x E d N o t W o r k i n g , Study Finds." T h e story reported a study by the M i n n e s o t a D e p a r t m e n t of Health, which in 2001 had asked students in grades 7 and 8 of three schools that h a d adopted an abstinence-only sex education curriculum (not teaching contraception or safe sex, but rather providing materials to encourage students to abstain), whether "at any time i n y o u r life h a v e y o u e v e r h a d s e x ( i n t e r c o u r s e ) ? " T h e y t h e n c a m e b a c k i n 2 0 0 2 a n d r e p e a t e d the q u e s t i o n for t h e s a m e g r o u p of students, n o w in g r a d e s 8 a n d 9. F r o m 2001 to 2 0 0 2 the n u m b e r saying that they had ever h a d sexual intercourse rose from 5.8 to 12.4 percent. T h e conclusion of the article w a s that the abstinence-only

H a d the study included a second agency as a control (see the next section), o n e in w h i c h o u t p u t w a s m e a s u r e d a t t h e s a m e t i m e s a s i n t h e first a g e n c y b u t i n w h i c h t h e r e w a s no decentralization, the alternative explanation w o u l d not have been plausible. T h a t is, i f t h e i n c r e a s e d p r o d u c t i v i t y i n " A g e n c y S t u d y " h a d b e e n d u e s i m p l y t o t h e act o f m e a s u r i n g , p r o d u c t i v i t y i n t h e c o n t r o l a g e n c y (in w h i c h t h e s a m e m e a s u r e m e n t s w e r e t a k e n a s i n t h e first a g e n c y ) a l s o s h o u l d h a v e i n c r e a s e d . A c c o r d i n g l y , i f w e f o u n d that productivity increased m o r e in the reorganized agency than in the control agency, w e w o u l d k n o w that this c o u l d not h a v e b e e n b e c a u s e o f t h e act o f m e a s u r i n g , for both agencies would have undergone the same measurements. That particular

alternative interpretation w o u l d h a v e b e e n eliminated by the design of the study. In conducting the study without a control, the alternative interpretation can be eliminated only by a s s u m i n g it away, w h i c h s e e m s very risky. "Reagan's victory over the Soviet U n i o n " provides an example of another sort of alternative explanation that m a y be plausible in studies without a control group. It is quite possible that other things that occurred b e t w e e n the m e a s u r e m e n t of the two events ( 1 9 8 0 - 1 9 8 8 and 1 9 8 9 - 1 9 9 1 ) w e r e the cause of the Soviet Union's collapse. Reform in C o m m u n i s t China, the growth of Muslim minorities in the Soviet Union, feuds among Soviet leaders—all might be proposed as

c u r r i c u l u m h a d failed. An abstinence-only c u r r i c u l u m m i g h t o r m i g h t n o t fail t o r e d u c e sexual

i n t e r c o u r s e , but we c a n n o t tell f r o m this s t u d y w h i c h is the c a s e . S t u d e n t s of that age, as they grow a year older, are in any case probably m o r e likely to have sexual intercourse than when they were younger. A control group of students of the same a g e w h o h a d h a d a different c u r r i c u l u m for sex e d u c a t i o n w o u l d h a v e b e e n easy to add to the study, but without it we have no idea w h e t h e r the students with the abstinence-only curriculum initiated sexual activity at a greater rate than they

alternative c a u s e s of the Soviet U n i o n ' s collapse. If it w e r e possible to find as a control g r o u p a similar system that w e n t through the s a m e other factors but was not engaged in an arms race with the United States, then these alternative no such

explanations system exists.

could be

tested

and perhaps

eliminated.

But of course,

would have done without the curriculum, at a lesser rate, or at the s a m e rate. We s i m p l y c a n n o t tell sexual intercourse. (The problem of interpreting results w a s also exacerbated in this case by the q u e s t i o n , w h i c h a s k e d s t u d e n t s w h e t h e r a t a n y t i m e i n t h e i r life t h e y h a d e v e r h a d s e x u a l i n t e r c o u r s e . T h e 5.8 p e r c e n t w h o a n s w e r e d affirmatively i n 2 0 0 1 m u s t h a v e a n s w e r e d affirmatively again in 2002, so the percent responding " y e s " in 2 0 0 2 whether the abstinence-only curriculum resulted in reduced

T h i s is a g o o d e x a m p l e of h o w difficult it m a y s o m e t i m e s be to i n c l u d e a control group in a design. S o m e circumstances just do not yield parallel cases that can be c o m p a r e d . As another example, consider that the existence of the United

A famous example of this sort is the Hawthorne study, in which an attempt was made to measure how much productivity increased when factories were made brighter and more pleasant. As it turned out, the groups of workers who were placed in better surroundings did show major increases in productivity. But so did control groups whose surroundings had not been improved. The novelty of taking part in an experiment, the attention paid to the workers, and increased social cohesiveness among those groups chosen for the experiment—all these raised productivity irrespective of the experimental changes in physical working conditions that were made for some (but not all) of the groups. See Roethlisberger and Dickson (1939).

4

c o u l d h a v e only risen or s t a y e d the s a m e ; it w a s n o t p o s s i b l e for the p e r c e n t a g e to decline. So, apparent failure of the p r o g r a m w a s baked into the study from the start. A better q u e s t i o n w o r d i n g , w h i c h w o u l d h a v e b e e n m o r e sensitive to the impact of the new program, w o u l d have been: " D u r i n g the past year, have you had sex [intercourse]?")

82

Chapter 6
a c c o m p l i s h e s this i s a

Causal Thinking and Design of Research 83 true experiment; but b e f o r e g o i n g o n to d i s c u s s t h i s , let m e
d i s c u s s a p o o r c o u s i n o f the n a t u r a l e x p e r i m e n t .

U s e of a Control G r o u p

The natural experiment.
encountered. It i s a

" O r g a n i z i n g the P o o r " i s an e x a m p l e o f a d e s i g n i n

w h i c h a control group h a s b e e n added to handle the sorts of p r o b l e m s we j u s t

The natural experiment without premeasurement. T h i s i s a d e s i g n i n w h i c h no
m e a s u r e m e n t s are m a d e before subjects are e x p o s e d to the independent variable. T h i s d e s i g n f o l l o w s the f o r m :

natural experiment,

a d e s i g n i n w h i c h a test group (that i s , a

group e x p o s e d to the independent v a r i a b l e ) a n d a control group (a group not e x p o s e d to the independent v a r i a b l e ) are u s e d , but in w h i c h the investigator has no control over w h o falls into the test group and who falls into the control group. In 1. Measure the dependent variable for subjects, some of whom have been exposed to the independent variable (the test group) and some of whom have not (the control group). 2. If the dependent variable differs between the groups, ascribe this to the effect of the independent variable.

" O r g a n i z i n g the Poor," the matter of w h o w a s contacted by the c a m p a i g n w o r k e r s w a s d e c i d e d b y the w o r k e r s ' o w n c h o i c e o f w h o m t o contact a n d b y the extent to w h i c h different people m a d e t h e m s e l v e s a v a i l a b l e for contact by the c a m p a i g n w o r k e r s . T h e natural e x p e r i m e n t d e s i g n i s o f the form:

T h e " T a x - R e f o r m M a i l " e x a m p l e i s o f this sort. I n this d e s i g n , (1) s e n a t o r s ' votes on a t a x - r e f o r m b i l l w e r e noted, a n d (2) the votes of senators w h o h a d

1. Measure the dependent variable for a specific population before it is exposed to the independent variable. 2. Wait until some among the population have been exposed to the independent variable. 3. Measure the dependent variable again. 4. If between measurings the group that was exposed (called the test group) has changed relative to the control group, ascribe this to the effect of the independent variable on the dependent variable.

r e c e i v e d f a v o r a b l e m a i l w e r e c o m p a r e d w i t h the votes o f those w h o h a d not. T h e s a m e k i n d of alternative e x p l a n a t i o n that has to be dealt w i t h in natural e x p e r i m e n t s has t o b e dealt w i t h i n this d e s i g n a l s o . A s i n " O r g a n i z i n g the Poor," i t m a y b e that h e a v i e r p r o - t a x - r e f o r m m a i l w e n t to senators w h o a l r e a d y w e r e m o v i n g into a taxr e f o r m p o s i t i o n e v e n w i t h o u t the m a i l . S u c h s e n a t o r s , about w h o m there m i g h t h a v e b e e n a great d e a l of s p e c u l a t i o n in the p r e s s , c o u l d h a v e attracted m o r e m a i l than did other senators.

T h u s , in " O r g a n i z i n g the Poor," (1) a n u m b e r of poor people w e r e s u r v e y e d as to their interest in p o l i t i c s ; (2) the c a m p a i g n o c c u r r e d ; (3) the s a m e poor people w e r e s u r v e y e d a g a i n ; and (4) those w h o h a d b e e n contacted by the c a m p a i g n w e r e c o m p a r e d w i t h those w h o h a d not. T h i s design e l i m i n a t e s m a n y o f the alternative explanations that c a n c r o p up in w o r k i n g without a control group. F o r i n s t a n c e , it c o u l d not have b e e n the initial m e a s u r e m e n t that c h a n g e d the group that h a d b e e n contacted, c o m p a r e d w i t h the control group, b e c a u s e both groups h a d b e e n m e a s u r e d in the s a m e w a y . N e v e r t h e l e s s , the n a t u r a l e x p e r i m e n t s t i l l a l l o w s alternative e x p l a n a t i o n s . B e c a u s e the r e s e a r c h e r d o e s not h a v e c o n t r o l o v e r w h o i s e x p o s e d t o the i n d e p e n d e n t v a r i a b l e , it m a y be that the e x p o s e d g r o u p h a s a different p r e d i s p o s i tion to c h a n g e than the c o n t r o l g r o u p . In " O r g a n i z i n g the P o o r , " for i n s t a n c e , the c a m p a i g n w o r k e r s are l i k e l y t o h a v e a p p r o a c h e d t h o s e p o o r w h o m they thought they c o u l d m o s t e a s i l y get i n t e r e s t e d i n p o l i t i c s . A l s o , t h o s e a m o n g the p o o r w h o w e r e m o s t r e s i s t a n t t o c h a n g e m i g h t not h a v e let the c a m p a i g n w o r k e r s i n the d o o r o r m i g h t h a v e b e e n c h r o n i c a l l y a b s e n t w h e n the c a m p a i g n w o r k e r s t r i e d t o c o n t a c t t h e m . A c c o r d i n g l y , a p l a u s i b l e alternative e x p l a n a t i o n i n " O r g a n i z i n g the P o o r " i s that the p o o r w h o w e r e c o n t a c t e d b y the c a m p a i g n i n c r e a s e d their interest i n p o l i t i c s m o r e than t h o s e w h o w e r e not c o n t a c t e d

Moreover,

this

design

permits

many

additional

alternative

explanations

b e y o n d those that a p p l y to a natural e x p e r i m e n t . In the t a x - r e f o r m e x a m p l e , it is p r o b a b l e that p e o p l e w e r e m o r e l i k e l y to w r i t e 1 ;tters f a v o r i n g r e f o r m to senators they thought w o u l d agree w i t h t h e m . I n other w o r d s , i t m i g h t b e that s e n a t o r s ' m a i l d i d not c a u s e their v o t e s , but that their l i k e l y vote c a u s e d t h e m to get c e r t a i n k i n d s of m a i l . H e n c e , the r e l a t i o n s h i p b e t w e e n a s e n a t o r ' s vote on the b i l l a n d the m a i l she r e c e i v e d m i g h t not be a c a u s a t i v e r e l a t i o n s h i p at a l l — m e r e l y a c o i n c i d e n c e b e t w e e n p r o - r e f o r m a n d p r o - m a i l a n d a n t i - r e f o r m a n d a n t i - m a i l . T h i s alternative c o u l d not a p p l y to a natural e x p e r i m e n t . In a natural e x p e r i m e n t it w o u l d h a v e b e e n c l e a r f r o m the i n i t i a l m e a s u r e m e n t w h e t h e r or not those w h o fell into the test group h a d i n i t i a l l y b e e n different f r o m those f a l l i n g into the c o n t r o l group. I n fact, w h a t i s c o m p a r e d i n the natural e x p e r i m e n t i s not the m e a s u r e d v a r i a b l e s t h e m s e l v e s , but h o w the two groups c h a n g e b e t w e e n m e a s u r e m e n t s . T o s u m u p , the n a t u r a l e x p e r i m e n t w i t h o u t p r e m e a s u r e m e n t i s a d e s i g n i n w h i c h the test group a n d the c o n t r o l group are c o m p a r e d w i t h r e s p e c t to a d e p e n d e n t v a r i a b l e o n l y after they h a v e b e e n e x p o s e d to the i n d e p e n d e n t v a r i a b l e . It i n v o l v e s the s a m e sorts o f a l t e r n a t i v e e x p l a n a t i o n s a s the n a t u r a l e x p e r i m e n t d o e s , p l u s s o m e others that r e s u l t f r o m the fact that the i n v e s t i g a t o r d o e s not k n o w w h a t the test g r o u p a n d the c o n t r o l g r o u p l o o k e d l i k e b e f o r e the w h o l e t h i n g started.

simply because they were the ones who showed more potential to become more interested in politics at that time, regardless of the campaign. T h i s alternative m u s t be either a s s u m e d
a w a y o r c o n t r o l l e d b y u s i n g a d e s i g n i n w h i c h the r e s e a r c h e r c a n d e t e r m i n e w h o

True Experiment In neither of the two v e r s i o n s of the natural e x p e r i m e n t j u s t outlined d i d the i n v e s t i gator h a v e any control over w h o fell into the test group a n d w h o fell into the control

f a l l s into the test g r o u p a n d w h o f a l l s into the c o n t r o l g r o u p . A d e s i g n that

and (5) he c o m p a r e d the voting of the t w o groups to see whether his lobbying had m a d e a d i f f e r e n c e . cited at t h e e n d o f this chapter. I n t h e last c o l u m n o f t h e t a b l e . A d m i n i s t e r the i n d e p e n d e n t v a r i a b l e t o t h e test g r o u p . H e r e ( 1 ) the president c h o s e half of the H o u s e r a n d o m l y to be the test group. M e a s u r e the d e p e n d e n t v a r i a b l e for both g r o u p s . This d e s i g n permits o n l y a very f e w alternative e x p l a n a t i o n s . f o r e x a m p l e . or t h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n different from t h o s e in t h e c o n t r o l group even before they e x p e r i e n c e d *. Type Graphic Presentation 3 Example from This Chapter Selected Alternative C a u s a l Interpretations Observation with no control group Test g r o u p : M * M Agency Study. Reagan's Victory T h e first m e a s u r e m e n t itself m a y h a v e c a u s e d the c h a n g e observed in the s e c o n d measurement. Experimental and Quasiexperimental Designs for Research (Chicago: Rand McNally. T h u s the p r o b l e m that t h e i n v e s t i g a t o r s f a c e d i n " O r g a n i z i n g t h e P o o r " i s e l i m i n a t e d . In presenting the designs graphically. N o n e of the alternatives d i s c u s s e d in this c h a p t e r a p p l i e s . Stanley. they c a n set u p theoretically equivalent g r o u p s . w a y to do this is to assign subjects r a n d o m l y t o o n e g r o u p o r the other. ( T h e best. 5. 2 . a s c r i b e t h i s difference to t h e p r e s e n c e of t h e independent variable. an asterisk (*) indicates that a group has been exposed to a stimulus or is distinguished in s o m e other way so as to constitute a "test group". T h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n m o r e likely t o c h a n g e than t h o s e w h o m a d e their w a y into t h e control g r o u p . "Presidential L o b b y i n g " is an e x a m p l e of the true e x p e r i m e n t . or w h a t have y o u — t h e test g r o u p ) vote differently from t h o s e w h o are n o t o f t h i s t y p e ( t h e c o n t r o l g r o u p ) . e d u c a t e d . t h e p r e s i d e n t w o u l d b e pretty c e r t a i n that a disproportionately favorable c h a n g e a m o n g those he l o b b i e d w a s d u e to his efforts. Table 6-1 s u m m a r i z e s the various research d e s i g n s d i s c u s s e d in this chapter and s o m e of the alternative explanations applicable in each case. s h e c a n p e r f o r m a true experiment. T h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n m o r e likely t o c h a n g e than t h o s e w h o m a d e their w a y into t h e control group. I f t h e i n v e s t i g a t o r d o e s h a v e s u c h c o n t r o l . I f t h e g r o u p s are e q u i v a l e n t t o start w i t h . M indicates that the d e p e n d e n t variable has been measured for t h e group.) T h e a d v a n t a g e o f m a k i n g the g r o u p s e q u i v a l e n t i s that r e s e a r c h e r s c a n t h e r e b y e l i m i n a t e a l m o s t all o f t h e a l t e r n a t i v e c a u s a l i n t e r p r e t a t i o n s t h a t h a d t o b e a s s u m e d a w a y i n t h e v a r i o u s natural e x p e r i m e n t d e s i g n s . Each d e s i g n is p r e s e n t e d t h e r e i n a s y m b o l i c s h o r t h a n d that i s e x p l a i n e d i n t h e f o o t n o t e t o t h e t a b l e . A true e x p e r i m e n t i n c l u d e s t h e f o l l o w i n g s t e p s : 1. he w o u l d have been faced with plausible alternative causal interpretations. A f e w e x a m p l e s will indicate the broad u s e of the d e s i g n : ( 1 ) a n y v o t i n g s t u d y that s h o w s that p e r s o n s o f a c e r t a i n t y p e ( w o r k i n g c l a s s . (2) he t o o k a straw vote to m e a s u r e the d e p e n d e n t variable (vote) for both g r o u p s . Assign at random s o m e s u b j e c t s to t h e test g r o u p a n d s o m e to the c o n t r o l g r o u p . 4 . male. and s i m p l e s t .84 Chapter 6 T A B L E 6-1 Causal Selected Research Designs Thinking and Design of Research 85 g r o u p . (4) the bill w a s v o t e d o n . . Campbell and Julian C. o n e o r t w o o f t h e a l t e r n a t i v e i n t e r p r e t a t i o n s left o p e n b y e a c h d e s i g n are h i g h l i g h t e d . W o r k i n g w i t h t h i s d e s i g n . 3 . True e x p e r i m e n t Test group: R M * M Control: RM M Presidential Lobbying Natural e x p e r i ment Test g r o u p : M * M Control: M M Organizing t h e Poor Natural e x p e r i m e n t w i t h o u t premeasurement Test g r o u p : * M Control: M Tax-Reform Mail s o m e t h i n g e l s e that h a p p e n e d at the s a m e t i m e as * m a y h a v e c a u s e d the change. and R (used to describe the "true experiment") indicates that individuals have b e e n assigned randomly to the groups. l e a v i n g the other half as the control. ( 2 ) a n y o f t h e l a r g e n u m b e r o f s t u d i e s estimating h o w m u c h better i n c u m b e n t presidents do in s e e k i n g reelection w h e n the "Notation a d a p t e d from D o n a l d T. a d i f f e r e n c e i n h o w t h e g r o u p s c h a n g e c a n n o t b e d u e t o t h e f a c t that t h e i n d i v i d u a l s i n t h e t e s t g r o u p w e r e m o r e p r o n e to c h a n g e in certain w a y s than w e r e t h o s e in the control group. D E S I G N S FOR POLITICAL RESEARCH T h e natural e x p e r i m e n t w i t h o u t p r e m e a s u r e m e n t i s the s i n g l e m o s t c o m m o n l y u s e d d e s i g n in political research. If t h e test g r o u p h a s c h a n g e d b e t w e e n t h e first a n d s e c o n d m e a s u r e m e n t s in a w a y that is different from t h e c o n t r o l g r o u p . Consult Cook and Campbell (1979). (3) he l o b b i e d the test g r o u p . If he had not b e e n able to control w h o w a s lobbied. 1963). M e a s u r e t h e d e p e n d e n t v a r i a b l e a g a i n for both g r o u p s . or B e c a u s e investigators can control w h o falls into w h i c h group.

w h i c h are difficult t o h a n d l e . b u t t h e s e d e s i g n s c a n b e u s e d o n l y w h e n the r e s e a r c h e r h a s m o r e control over the test variables than m o s t political scientists can usually achieve. O n t h e o t h e r h a n d . direct m a i l a p p e a l s h e l p e d slightly. and so on. T h e s e lend themselves readily to a natural experimental design. electoral systems like those of Germany. s o m e e v e n t s a r e m o r e e a s i l y anticipated: the introduction of a poverty p r o g r a m in a town. the educational backgrounds and past professions of m e m b e r s of C o n g r e s s . high school graduates' entrance into college. w h o t e s t e d w h e t h e r w o m e n c a n d i d a t e s for office w e r e less w e l l financed than m e n b e c a u s e of experiential attributes such as seniority. such as voters from several age groups. this d e s i g n is far f r o m satisfactory. and the Netherlands) should strengthen the interests of producers. It turned out that personal visits were effective in getting p e o p l e to vote. but only if the experimenter can control significant policies and m a n i p u l a t e t h e m for the purposes of investigation. especially the Introduction) argue persuasively that we in political science have overrated the problems and barriers to true experimentation in our field. also known as single-member district. T h e y a l s o m u s t g o t o the e x p e n s e in time and m o n e y of m a k i n g t w o m e a s u r e m e n t s of the subjects. for instance. H o u s e elections and British a n d C a n a d i a n elections) will tend to p r o d u c e e c o n o m i c policies favorable to c o n s u m e r s . W h e n t h e forces t h a t determ i n e t h e e x p e r i m e n t a l a n d c o n t r o l g r o u p s a r e u n k n o w n . the a t t e m p t m a y misfire. points out that a p p a r e n t l y only r a n d o m i z a t i o n p r o v i d e s a c l e a r . in which 30. Sweden.86 Chapter 6 Causal Thinking and Design of Research 87 e c o n o m y h a s i m p r o v e d i n t h e p r e c e d i n g y e a r (test g r o u p ) c o m p a r e d w i t h t h o s e for w h o m the economy has not been improving (control group). a n d t e l e p h o n e calls m a d e no difference at all. It is rare in real political situations that a researcher can exercise this kind of control over output" studies. and Mosteller (1975). Others. the investigator can carry out a true experimental design. Nevertheless. S . for e x a m p l e . 6 . where the dependent variable is compared among groups of subjects distinguished by their values on the independent variable. For instance. T h e latter requires only that the investigator be able to anticipate events. such as assassinations. and probably should use it a good deal m o r e than is now d o n e ." 5 possible. for i n s t a n c e . riots. the k i n d of s y s t e m u s e d in U . E v e n then. But the logic of what is done here is the same as in the strict test/control situation. e v e n t h e c l e v e r e s t statistical a n a l y s i s m e e t s s t r o n g r e s i s t a n c e f r o m o t h e r s c h o l a i s . It is apparent here and in the examples directly preceding this that I am taking some liberty with the notion of "control group. A true experiment requires even greater control over the subjects of the study t h a n d o e s a natural e x p e r i m e n t . the colonial histories of nations. and (4) studies of discrimination. but the true experiment requires that the researcher be able to m a n i p u l a t e t h o s e e v e n t s — s h e m u s t d e c i d e w h o i s t o fall i n t o t h e c o n t r o l g r o u p a n d w h o i s t o fall i n t o t h e e x p e r i m e n t a l g r o u p ." Where rates of participation in the middle class and working class are compared. t h e i m a g i n a t i o n h a s full p l a y t o c r e a t e alternative e x p l a n a t i o n s for t h e d a t a . However. students might c o n d u c t a political c a m p a i g n a m o n g a r a n d o m l y selected portion of the campus community and c o m p a r e that portion over time with a r a n d o m l y selected control. A g o o d e x a m p l e of this sort of e x p e r i m e n t is Gerber and Green (2000). (3) most "policy fit i n t o a n a t u r a l e x p e r i m e n t d e s i g n . T h u s it is no accident that the e x a m p l e I u s e d of a true experiment. 5 A n o t h e r interesting e x a m p l e of a natural situation that p r o d u c e d something a p p r o x i m a t i n g a true e x p e r i m e n t is t h e s t u d y by H o w e l l et al. It is s o m e t i m e s possible to carry out true experiments on a larger scale than this.000 registered voters in New Haven. citing Gilbert. P r o g r a m s w e r e initiated in Dayton. 6). while proportional representation systems (the control group." w a s carried out by the president. In short. it m a y be that t h e test factor (especially if it is o n e that affects only a small part of the p o p u l a t i o n ) will a p p l y to o n l y a few of the p e o p l e i n c l u d e d i n t h e study. In o r d e r to u s e a n a t u r a l e x p e r i m e n t d e s i g n . and so on. regularly scheduled political events such as elections. A c h e n (1986. and (2) infers that the difference on the d e p e n d e n t variable is the result of their difference on whatever it is that distinguishes t h e m as "types. S o m e lie i n t h e past: p e o p l e ' s e x p e r i e n c e s d u r i n g the Depression. I n v e n t i n g h y p o t h e s e s of this sort is e n j o y a b l e . such as Uhlaner and S c h l o z m a n ( 1 9 8 6 ) . u n s t r e n u o u s l a b o r t h a t i s r a r e l y resisted. w h o s e p r o f e s s i o n a l s k e p t i c i s m i s q u i t e n a t u r a l . this r e m a i n s the m o s t widely u s e d d e s i g n in political science. it is not at all clear which class is the "test" group and which is the "control. w e r e e a c h r a n d o m l y assigned o n e of three different a p p r o a c h e s in a nonpartisan get-out-the-vote program. As we s a w in earlier s e c t i o n s . It is convenient and revealing to treat this sort of analysis in terms of the analogy to experiments. a n d c h a n g e s in the b u s i n e s s c y c l e . p. ( 2 0 0 2 ) of t h e effects of a private education on poor children. w h i c h s h o w e d that majoritarian electoral systems (the test group. M a n y i m p o r t a n t v a r i a b l e s i n t h e s o c i a l s c i e n c e s a r e n o t a t all s u s c e p t i b l e t o s t u d y b y n a t u r a l e x p e r i m e n t s . are u n p r e d i c t a b l e . s u c h as a local political c a u c u s or a portion of a c a m p u s . It m a y be that in a s m a l l o r g a n i z a t i o n . t h e i n v e s t i g a t o r m a y b e left w i t h a c o n t r o l g r o u p b u t n o test g r o u p . Ohio: Kinder and Palfrey (1993. T h a t is. or b e c a u s e of direct discrimination. O t h e r designs that have the advantage of control groups are to be preferred because they r e q u i r e less difficult a s s u m p t i o n s . any research is an example of the natural experiment without events.e n o u g h c a u s a l i n t e r p r e t a t i o n t o s e t t l e issues of social-scientific research conclusively: This really describes the bulk of political research. "Presidential Lobbying." The distinction is even muddier when one compares several groups simultaneously. plurality s y s t e m s . It p e r m i t s relatively m a n y alternative causal interpretations. such as one by R o g o w s k i and Kayser (2002). the experiment is so m u c h m o r e powerful than other d e s i g n s — rejects alternative interpretations so decisively—that we should use it whenever 6 p r e m e a s u r e m e n t if it (1) takes t w o or m o r e types of subjects and c o m p a r e s their values on a d e p e n d e n t variable. S u c h variables are difficult to W i t h o u t it. T o d o t h i s i n a f i e l d s i t u a t i o n r e q u i r e s p o w e r over p e o p l e in their n o r m a l lives. Light. r e s e a r c h e r s m u s t be able t o a n t i c i p a t e the o c c u r r e n c e o f the test factor. C o n n e c t i c u t .

on a good day. our city's murder rate declined by a full 18 percent"? What is often missed in such statements is that this effect might also be due to regression to the mean. even if the students had never been involved in team learning. children were chosen by lottery to receive the vouchers. A slightly more complex problem. she will score somewhat higher. Special Design Problem for Policy Analysis: Regression to the Mean I have introduced you above to some simple sources of alternative causal interpretations and some simple designs to control for them. For our purposes here. When it is possible to construct true (or nearly true) experiments in real field situations like this. Not all of them will do this. One remaining alternative explanation is that something akin to the problem we saw in 'Agency Study" was operating. a group of subjects may be placed together and told to reach a decision on some question. so on the average they are likely to go down the next time they are observed. the results can be compelling. We should expect that it is unlikely they would be so lucky twice in a row. by the fact that it was only African Americans who appeared to benefit from the vouchers. however. The result of the study was that African American students who received the scholarships did significantly better on standardized tests two years later than African American students who did not receive the scholarships." This could be true. Thus. It might be that the stronger students' scores declined because when they were measured at the beginning of the experiment with team learning. and low initial values to rise somewhat. Also. the New Haven study by Gerber and Green is an example. If the murder rate has shot up in one year. This selection process approximated a true experiment. But a disproportionate number of the high cases are probably cases for whom things had lined up unusually well the first time. The children who received scholarships knew they had won something special. some of the high scores no doubt are genuinely high and may even rise by the next time they are observed. D. This may have left them more highly motivated than the students who had not won in the lottery. The two alternative causal interpretations can only be sorted out by using an appropriate research design. probably have some element of randomness to their measures. Consider a student in a course. the scores of the weaker students might have improved. If we assume that essentially everything we observe has some element of randomness to it—that is. to see how this influences the result. her measured level of knowledge is generally about right. More often.88 Chapter 6 Causal Thinking and Design of Research 89 The problem is called regression to the mean. she will score somewhat below her usual level. and the scores of the stronger students might have declined. Similarly. is especially important in evaluating the impact of policy initiatives. but it is well within your range of understanding at this point. The investigator then manipulates the way individuals in the group may communicate with each other. the two groups should have been equivalent in all respects other than the experimental treatment. a number of them had done unusually well and then just drifted back to their normal level of performance over time. and New York City that offered children in public schools partial scholarship vouchers to attend private schools. (One is given on p. the important thing about this is that if we observe cases at two points in time. This is a problem most public officials and journalists do not appear to understand at all. they knew that their parents were paying extra money beyond the partial scholarship to further their education. whether or not the city institutes foot patrols in the entertainment district.like: "Team learning strategies are good in that the weakest students improve. while it has many applications in general explanation. When do cities typically institute new policies? When the problem they are concerned about has flared up! But cities' problems. true experimentation is useful for studying general aspects of small-group interactions or individual thought processes relevant to politics. 90) One variant of this problem shows up time and again in assessing the impact of new governmental policies. in addition to its true core value it varies somewhat from time to time—regression to the mean will always be present. For example. (In an ideal experiment. and therefore offers very convincing results on an important policy issue. This study comes very close to a true experiment. in which the investigators tested experimentally the effects of variations in the format of TV news reporting on subjects' political perceptions. people do not know whether they are in the experimental group or in the control. A good example of this sort of experiment is that of Iyengar and Kinder (1987). we should see them rise. But an alternative explanation could be that this is simply regression to the mean.) This worry might be lessened. if the benefit were an artifact of children's knowing they had won in the lottery. the scholarships were assigned randomly. In such studies. one would think that would affect all ethnic groups in the same way. In other words. for example. the random part of their measure is likely to have been positive. on the average. We should therefore be a little suspicious of a statement.C. How often have you seen a statement of the sort: "After foot patrols were introduced in the entertainment district. but if she has had a bad day (isn't feeling good. it is more likely to go down the next year than to go still higher. like students taking tests. we can expect high initial values to drop somewhat from one time to the next. that is. When she is tested. In other words. some of the poorer students might have drifted up to their normal levels. governments are likely to pick a time when a problem is at a high point (which may or may not include some random element) Washington. of course. Similarly. has bad luck in the instructor's choice of questions. inasmuch as the investigator generally can control all the relevant variables. but they are bad in that the strongest students appear to be dragged down by working with the weaker ones.. the lowest scores probably include a disproportionate number of cases that were unusually low because of a negative random factor. and there was a control group of children who applied for scholarships but did not receive them. . and so on). Since there were more applicants than could be funded. No other ethnic group appeared to benefit in the same way. Note that this sort of random variation in a measure is closely related to what we referred to as unreliability in Chapter 4. true experiments are the most appropriate design.

m a k i n g it l i k e l y that the s e c o n d m e a s u r e m e n t w o u l d s h o w a d e c r e a s e whether or not the intervention between the two m e a s u r e s has had any impact. w i t h the intervention indicated by an asterisk and a d a s h e d v e r t i c a l line. a c k n o w l e d g e alternative c a u s a l interpretations. or u s i n g a control group w i t h no p r e m e a s u r e m e n t ) . Figure 6-1 Examples of Interrupted Time Series E v e n if y o u are f o r c e d to rely solely on one of the w e a k e r d e s i g n s (operating A n even better design. in the c a s e of the murder rates.1 . that i s . A d e s i g n that a l l o w s us to sort this out is an AlZ. USE OF VARIED DESIGNS A N D MEASURES 1 2 3 4 5 6 7 Year 8 9 10 11 12 13 14 I s u p p o s e that the easiest c o n c l u s i o n to draw f r o m our d i s c u s s i o n thus far is that any k i n d of r e s e a r c h in political s c i e n c e is difficult and that the results of p o l i t i c a l r e s e a r c h are. on the b a s i s of the information in the statement. 7 (B) T h e interrupted time series i s g r a p h i c a l l y illustrated i n F i g u r e 6 . G r a p h A illustrates an interrupted time series in w h i c h the intervention appears to have c a u s e d a c h a n g e . y o u are better off if y o u set out y o u r d e s i g n f o r m a l l y and m e a s u r e those variables that c a n be m e a s u r e d . T h e d e s i g n in the statement about foot patrols and m u r d e r rates is our old friend: M * M (A) O u r p r o b l e m is that we suspect that the first M m a y have been u n u s u a l l y h i g h . the m e a s urements preceding it should tend to be lower than it i s . we cannot tell whether or not foot patrols r e a l l y c u r b e d the m u r d e r s . D o e s this m e a n that w e cannot a s s e s s the i m p a c t o f c h a n g e s i n p o l i c i e s ? O f c o u r s e not. I did not d i s c u s s these selected r e s e a r c h d e s i g n s to c o n v i n c e y o u not to do p o l i t i c a l r e s e a r c h but to s h o w y o u s o m e of the p r o b l e m s y o u must d e a l w i t h . would add a control to rule out the possibility that it was something else that happened at the same time as the intervention that caused the change: MMMMM * MMMMM MMMMM MMMMM 7 without a control group. in the last a n a l y s i s .90 Chapter 6 Causal Thinking and Design of Research 91 to initiate a p o l i c y to s o l v e the p r o b l e m . T h u s . r e g r e s s i o n to the m e a n w o u l d have l e d us to think the intervention had had an i m p a c t . interrupted time series. B u t it does m e a n that we must find a r e s e a r c h d e s i g n that a l l o w s us to d i s t i n g u i s h b e t w e e n the t w o alternative explanations. if we h a d l o o k e d only at the adjacent "intervention and after" m e a s u r e s . B u t this w o u l d be far too sour. T h e test for whether the average of the several measurements preceding the intervention differs f r o m the average of the measurements intervention has had an effect in this design is whether the following the intervention. Note that in graph B. and try to a s s e s s j u s t h o w . a 1 2 3 4 5 6 7 Year 8 9 10 11 12 13 14 series of m e a s u r e m e n t s over time that is interrupted by the p o l i c y intervention: MMMMM *MMMMM If the measurement that c a m e just before the intervention w a s unusually high. b e c a u s e r e g r e s s i o n to the m e a n provides a p l a u s i b l e alternative explanation. unreliable. where possible. G r a p h B illustrates one in w h i c h it did not.

To this end. whose test group we controlled. his theory was that traditions of civic cooperation. This design still suffered. The choice is between doing this or giving up and relying on your intuition and impressions. Putnam was faced with the question of research design. The various weaknesses of different designs should suggest a solution: Wherever possible. Relying totally on a natural experiment. Anybody. Getting the same result in two different cities would not make it any less likely that the alternative causal interpretation was true. To test the relationship more richly. try to work simultane- ously with a variety of designs. or almost anybody. he first gathered a number of statistics about governmental performance in the 20 regions that he was studying. the end result is more than the sum of its parts. rather than a "true" result. and a personal visit over several weeks. The likelihood that the alternative explanations associated with each design are true decreases because of the confirmation from the other design. each result would bolster the other. But if we could use both of these designs. The strengths of the various designs complement each other. If we can mix research strategies in this way. Consider the "Organizing the Poor" example once again. by showing us that the first result was not an accident or a result of the particular locality chosen. say. there is no reason to think that a similar campaign would not do the same thing in a second city. Once he had decided how to measure governmental performance. all with their own varied sources of invalidity. It will seem less likely to us that the result of the natural experiment is due to unusual people entering the test group. The fact that such linking relationships proved to be true under testing makes us more confident that the overall relationship was not a design fluke. he traced the development of civic involvement in regions over the past century and showed that high civic involvement in the latter part of the nineteenth century led to civic involvement and governmental effectiveness a century later. since willingness to compromise should in turn lead to effective government. But the alternative causal interpretations associated with natural experimental design would in no way be suspended. he also added tests of linking relationships using the same design. But there is a more hopeful side to this chapter. Note that the effect would not be the same if a single research design were repeated twice—if. rather than a "true" result. he approached the problem of measuring "government performance" by using multiple measures. we may wonder whether the result we have obtained might be a result of something about the way the experiment was set up. If the campaign in one city tended to reach mainly those who were already becoming more interested in politics.92 Chapter 6 Causal Thinking and Design of Research 93 likely it is that each alternative is true. by adding analyses over time of the form "observation with no control group. and either . CONCLUSION To pick up once again the refrain of this book. His study sought to explain variation in the quality of governmental institutions. he could be more confident that he had measured governmental effectiveness accurately. for example. in what is really a problem of valid measurement (see Chapter 4) rather than one of research design. He then checked and supplemented the official statistics by having research assistants call government agencies in each region with fictitious requests for help on such things as completing a claim for sickness benefits or applying for admission to vocational school. carry out a fairly obvious test on it using one or two obvious measures. and those with high engagement proved to be those with a high quality of governmental performance. from the possibility that the relationship was a matter of other contemporary things associated with civic engagement. Repeating the same design twice would increase our confidence somewhat.) Finally." In these. First. numerous phone calls. such as the promptness with which the regional governments developed their budget documents. which can at least in part cancel out each other's weaknesses. By combining the complementary strengths of varied measures and varied designs. Putnam was thus able to generate conclusions of which the reader is more confident than if they were based on a single measure or on a single design. creativity and originality lie at the heart of elegant research. a natural experiment were repeated in a different locality. these objective measures of performance were supplemented by several opinion surveys in which people were asked how well they thought the regional government performed. the less political leaders feared compromise. The 20 regions were compared as to their level of civic engagement and the quality of their governments' performance. are what lead to effective government. Robert Putnam (1993) uses the varied-design strategy rather well. For instance. we might wonder whether our results simply reflect a test group comprised of unusual people. in working solely with a randomized experiment in an isolated situation. This makes sense as a component of the relationship between civic engagement and governmental effectiveness. For example. (Governments' response quality varied from responses in under a week after a single letter. Example of Varied Designs and Measures In his Making Democracy Work. such that those regions high in civic engagement were for some other reasons primed for governmental effectiveness. employing either the same or closely related measures. he showed that the higher civic engagement was in communities. to responses that required numerous letters. if we obtain similar results in a true experiment. in part. based on what he calls social capital (networks of social cooperation). We could be more confident of the experimental result if we had gotten a similar result in a field situation. can take a research problem. And we could be more confident of the result of the natural experiment if we had gotten a similar result in an artificial true experiment. Putnam countered this. thus creating the illusion that the campaign had gotten them interested in politics. To the extent that he got the same results across a variety of methods of measurement. His basic design was the natural experiment with no premeasurement. though.

V a r y i n g d e s i g n s i s g e n e r a l l y u s e f u l . T h o s e i n d e m o c r a c i e s c o u l d e x p e c t to live 16 years longer.6 67. A l l o f t h e s e t e c h n i q u e s will h e l p the investigator limit the n u m b e r of alternative interpretations of his findings.001-6. S t a t i s t i c a l t e c h n i q u e s s u c h a s m u l t i p l e r e g r e s s i o n artificially h o l d v a r i a b l e s constant. but it s h o u l d n o t be t h o u g h t of as t h e o n l y a n s w e r . I think. T h e technique demonstrated here of h o l d i n g a variable constant—literally s e p a r a t i n g t h e s u b j e c t s i n t o n e w little g r o u p s a n d d o i n g t h e s a m e a n a l y s i s w i t h i n e a c h o f t h e s e g r o u p s — i s t h e s i m p l e s t t e c h n i q u e o n e c a n u s e t o e l i m i n a t e a third v a r i a b l e . and p o o r countries in general have w o r s e h e a l t h a n d l o w e r l i f e e x p e c t a n c i e s t h a n c o u n t r i e s that are b e t t e r off.001-3. but there are o t h e r w a y s t o e l i m i n a t e alternative interpretations. there still r e m a i n s a n a d v a n t a g e for d e m o c r a c i e s .000 1.4 3. 229. If t h e r e i s .001-4. a n d their m e a n i n g w o u l d thus be uncertain. w i l l cite e v i d e n c e f r o m related studies.2 2. 3 y e a r s .6 16. than those in dictatorships—a big difference. 0 0 0 . 0 0 0 . $ 0-1.2 59. S u c h a student will muster logical arguments. a l t h o u g h i t i s n o t h i n g l i k e 1 6 y e a r s . p.2 56.7 3. D o t h e l o n g e r life e x p e c t a n c i e s in d e m o c r a c i e s indicate anything about w h a t d e m o c r a c y d o e s f o r p e o p l e .001-5.3 73. but t h e d e m o c r a c i e s h a v e a n a d v a n t a g e o f e i g h t . but 2 3 9 d e m o c r a c i e s ( m o s t l y f r o m North A m e r i c a and western Europe). a n o b v i o u s a l t e r n a t i v e e x p l a n a t i o n o f f e r e d itself. 0 0 1 . m a n y o f t h e s e w o u l d b e b a s e d o n rather s m a l l n u m b e r s o f c a s e s .0 68. it w o u l d result in a h o r r e n d o u s n u m b e r of tiny tables. o r i s i t j u s t t h a t d e m o c r a c i e s a r e r i c h e r c o u n t r i e s . l0 9 . s u c h a s b y l o g i c a l a r g u m e n t o r b y indirect e v i d e n c e o f v a r i o u s sorts.000 3.94 Chapter 6 T A B L E 6-2 Per C a p i t a Income. M o r e s e r i o u s l y . interpreting a result a n d h a n d l i n g alternative interpretat i o n s o f t h e result are difficult a n d c h a l l e n g i n g c o n s t i t u e n t s o f research. 0 0 1 . Dictatorships 46. D i c t a t o r s h i p s w e r e m u c h m o r e likely than d e m o c r a c i e s to be poor. and then observe whether within e a c h of these g r o u p s there is a relationship b e t w e e n the first t w o variables.000 6.3 70. A creative r e s e a r c h e r w i l l n o t b e s a t i s f i e d w i t h t h i s b u t w i l l try v e r y h a r d t o a c c o u n t f o r all plausible alternative interpretations. Difference 0. 0 0 0 . on the average. a n d that i s w h y people in democracies live longer? As indicated in Table 6-2.2 71.6 53. A t all l e v e l s o f p r o s p e r i t y .8 4. a u s e f u l o n e .001Total Causal Thinking and Design of Research Life Expectancy U n d e r Dictatorships and D e m o c r a c y Life Expectancy. D o i n g this w i t h the direct t e c h n i q u e w o u l d not be easy. some countries appear in the data set four or more times.6 67. 1 0 T h e a d v a n t a g e o f s u c h s t a t i s t i c a l t e c h n i q u e s i s that t h e y c a n c o n v e n i e n t l y h o l d several variables c o n s t a n t s i m u l t a n e o u s l y . "Based on Przeworski et al.1 5.4 52. r e a c h i n g a s h i g h a s a n i n c r e a s e o f 5 .3 (215) (144) (59) (44) (25) (18) (18) (521) 95 ignore or a s s u m e a w a y the ever-present alternative causal interpretations.0 H O L D I N G A VARIABLE C O N S T A N T e x p e c t a n c y i n b o t h d e m o c r a c i e s a n d d i c t a t o r s h i p s i s l o w .$ 5 . If we w a n t to k n o w w h e t h e r a relationship b e t w e e n t w o variables c a n be a c c o u n t e d for by a third v a r i a b l e that i s r e l a t e d t o b o t h o f t h e m . w e c a n h o l d t h e third v a r i a b l e c o n s t a n t t o s e e w h e t h e r t h e r e l a t i o n s h i p b e t w e e n t h e first t w o c o n t i n u e s t o e x i s t w h e n t h e third variable is not free to vary. if we divide countries into groups with approximately t h e s a m e p e r c a p i t a i n c o m e s .000 2. Democracies 47. 2 y e a r s o f life e x p e c t a n c y for c o u n t r i e s with per capita i n c o m e of $ 4 .3 (11) (39) (34) (30) (19) (23) (239) (395) Life Expectancy. e a c h having a distinct value on the variable to be held constant. F o r c o u n t r i e s w i t h p e r c a p i t a i n c o m e s l e s s than $ 1 . H o w e v e r . and their effect is a p p r o x i m a t e l y the s a m e as the direct t e c h n i q u e I u s e d here. i t c a n n o t b e d u e t o v a r i a t i o n i n t h e third v a r i a b l e . Holding a variable constant is often also called a d v a n t a g e . and an increase of 5. A t t h e o t h e r e n d are o n l y 1 8 dictatorships w i t h per capita i n c o m e o v e r $ 6 . a s u b s t a n t i a l d i f f e r e n c e still e x i s t s b e t w e e n d e m o c r a c i e s and dictatorships e v e n w h e n per capita i n c o m e is taken into account.000 5. f o r w i t h i n e a c h o f t h e s e g r o u p s t h e third v a r i a b l e l i t e r a l l y i s c o n s t a n t .3 63. (2000). and f r o m t h e s e w e c a n s e e w h y the difference w a s s o great b e f o r e w e c o n t r o l l e d for the countries' per capita i n c o m e s .000 4. A g o o d e x a m p l e of h o l d i n g a variable constant is provided by the Przeworski e t al.6 years for countries w i t h p e r c a p i t a i n c o m e g r e a t e r than $ 6 . 9 controlling for the variable. S e e Chapter 9 for an elementary discussion of multiple regression.1 4. ( 2 0 0 0 ) s t u d y o f w h e t h e r p e o p l e l i v e b e t t e r i n d e m o c r a c i e s t h a n i n d i c t a t o r s h i p s . l i f e 8 w i t h o n l y e l e v e n d e m o c r a c i e s ( c o u n t r i e s s u c h a s I n d i a ) . and m a y e v e n vary the d e s i g n s and m e a s u r e s i n h i s o w n study.2 64. T h e poorest group has 2 1 5 dictatorships. T h e reason the number of countries is so high is that the authors counted each country at several different points in its history.2 65. T h e number of countries falling into e a c h type appears in parentheses. T h e y f o u n d that t h e l i f e e x p e c t a n c y o f p e o p l e l i v i n g i n d e m o c r a c i e s w a s 6 9 .t e n t h s o f a year. N o interpretation o f a r e s e a r c h result is cut a n d dried. 3 y e a r s w h i l e that o f p e o p l e living i n dictatorships w a s j u s t 5 3 . The conclusion w e d r a w f r o m h o l d i n g p e r c a p i t a i n c o m e c o n s t a n t i s that although m o s t of the difference in life e x p e c t a n c i e s w a s due to the difference in h o w w e l l o f f d e m o c r a c i e s a n d d i c t a t o r s h i p s are. d e m o c r a c i e s h a v e s o m e I h a v e talked about h o l d i n g a variable constant m o r e than o n c e in this chapter.001-2. compared T h e s i m p l e s t w a y to do this is to divide the subjects into separate g r o u p s .2 69. T h e s u g g e s t i o n m a d e i n t h e p r e c e d i n g s e c t i o n — t o vary m e a s u r e s a n d d e s i g n s — i s . First.

A central theme in this b o o k . T h o m a s D. A further important factor u n d e r l i e s this w h o l e p r o c e s s . S h a d i s h .) It w i l l not surprise y o u . and those h a v i n g telephone s e r v i c e (not a representative group of A m e r i c a n voters in those d a y s ) (see pp. and to s h o w y o u s o m e b a s i c p r i n c i p l e s that w i l l help y o u to select c a s e s in w a y s that w i l l a l l o w y o u a c l e a n e x a m i n a t i o n o f y o u r r e s e a r c h question. y o u must select a set of observations to look at. Achen's The Statistical Analysis of Quasi-experiments (1986) explores these issues with great w i s d o m . it is often the c a s e that the " s e l e c t i o n " o c c u r s by subtle p r o c e s s e s other than y o u r o w n c h o i c e . m e a s u r e the variables i n v o l v e d . and y o u r selection of c a s e s c a n sharply affect or e v e n distort w h a t y o u w i l l find. and my analysis of t h e m has included only a sample of the possible alternative explanations for each design. 2004). C a m p b e l l ' s Quasi- experimentation: Design and Analysis Issues for Field Setting ( 1 9 7 9 ) . T h e literature on the use of true experiments in p o l i t i c a l s c i e n c e is burgeoning. a s p e c i a l i s s u e of the j o u r n a l Political Analysis ( v o l u m e 10. Cook. A u t u m n 2 0 0 2 ) . edited b y D o n a l d G r e e n a n d A l a n G e r b e r . Campbell Experimental and Quasi-Experimental Designs for Generalized Causal Inference ( 2 0 0 2 ) . developed in the face of quite limited research designs. a n d a s p e c i a l i s s u e . or more recent and even m o r e complete (but not quite as readable). w h i c h in 1936 predicted that F r a n k l i n R o o s e v e l t w o u l d l o s e the e l e c t i o n by a l a n d s l i d e on the basis of questionnaires sent to s u b s c r i b e r s to their m a g a z i n e . J a n u a r y So far in this book we h a v e l o o k e d at h o w y o u c a n develop a r e s e a r c h question. Chapters 1 and 2 are easily a c c e s s i b l e to readers with limited technical preparation. T h e c a s e s they h a d c h o s e n to l o o k at did not faithfully mirror the A m e r i c a n electorate. Further. excellent and readable.96 Chapter 6 FURTHER DISCUSSION I have presented here only a f e w examples of the m o r e c o m m o n l y u s e d research designs. S e v e r a l interesting e x a m p l e s of i m p r o v i s e d research designs in field research on animal behavior c a n be found in Tinbergen ( 1 9 6 8 ) . O n e question that y o u might c o n s i d e r i n c o n n e c t i o n w i t h this chapter i s : W h y s h o u l d theory-oriented. C o n s i d e r the f o l l o w i n g e x a m p l e s : • I d e s c r i b e d earlier the strange c a s e o f the Literary Digest.1 . of American Behavioral Scientist ( v o l u m e 4 8 . Tingsten (1937) provides an example of unusually careful and creative interpretations. n o .5 1 ) . no. e m p i r i c a l political r e s e a r c h b e b a s e d a l m o s t e x c l u s i v e l y o n c a u s a l relationships rather than on relationships in g e n e r a l ? 1. for instance. I have relied h e a v i l y on their approach in this chapter. A good treatment of the concept of " c a u s e " in relation to research design is found in B l a l o c k ( 1 9 6 4 . ( T h i s w a s the point of F i g u r e 4 . In this chapter I want to alert y o u to the i m p o r t a n c e of c a s e s e l e c t i o n (whether it is done by y o u or by nature). is provided in T h o m a s D. car o w n e r s . w h i c h y o u s a w e s p e c i a l l y in the two chapters on measurement and w h i c h y o u w i l l see again in later chapters. G o o d e x a m p l e s are K i n d e r and P a l f r e y ( 1 9 9 3 ) . that case selection is j u d g e d by h o w w e l l it produces observed relationships that faithfully mirror the theoretical relationships we are trying to test. then. is that we need to construct our research operations so that the relationships we observe a m o n g the variables we have m e a s u r e d mirror faithfully the theoretical relationships we are interested in. and look for relationships a m o n g them to provide a n s w e r s to the r e s e a r c h question. A more complete treatment of research design. however. W i l l i a m R. 5 0 . To do all these things. and Donald T. 4 . C h r i s t o p h e r H. C o o k and D o n a l d T. 97 . Chapters 1 and 2 ) .

And. would be to set aside those cases in which incumbents have been most successful in generating electoral security. selection of a potentially biased sample did not result from a decision the researcher made. Would students who did badly on the GRE do well in the program? We cannot know.98 Chapter 7 Selection of Observations for Study 99 • College students taking psychology tests are routinely used for experiments to measure psychological processes. he would understate the amount of safety that long-time incumbents accrue. Researchers justify doing this by arguing that the processes they are studying are universal. From time to time. a department will consider dropping the exam because when one looks at the performance of graduate students in the department. they are "ethnic powderkegs" that had been held in check only by Soviet repression. but the one that made the best approximation. We usually cannot study all possible observations to which our theory applies. because the incumbent was so safe that no one wanted to take her on. had to work with another selection problem dealt by nature. the students admitted into the program are an atypical sample from the population. it is argued. only four experienced violent ethnic conflict. As Mihaela Mihaelescu (2004) has pointed out. So he examined the vote margins of incumbents running for reelection. that does not matter. As noted earlier. Therefore. if he ignored the selection problem. for example. and are now prone to explode in ethnic violence. But if instead he treated the unchallenged incumbents as having gotten 100 percent of the vote. this argument relies on data from those admitted to the program to generalize to the population of all students who might apply to the program. SAMPLING FROM A POPULATION OF POTENTIAL OBSERVATIONS Both the Literary Digest example and the discussion of using college students to generalize to all adults are examples of the problem of sampling. that is. • Most graduate departments in political science admit students to their programs based in part on each student's scores on the Graduate Record Examination (GRE). As a result. studying whether incumbent members of Congress had extra advantages in garnering votes. say. they all did well enough on the GRE to get into the program. (p. For instance. In random sampling it is purely a matter of chance which cases from the full population end up in the sample for observation. scholars use computer-generated random numbers to identify which members of the full population should join the sample for observation. We cannot. (In reality. and three of these were various parts of former Yugoslavia. 136) their safety. so even though their test subjects are not at all representative of the human population. however. but inheres in the situation. an experiment using college students might study the effect of drinking coffee on one's ability to memorize long strings of numbers. so their students are as good as any other for the test. each member of the population has an equal chance of being drawn for the sample.) • The general understanding of the new democracies of eastern Europe has been that unless they are culturally homogeneous as Poland is. Clearly. ask every American adult how he or she expects to vote in an election. In fact. as measured by their share of the vote. limitations of time and resources usually bar us from doing so. the "Cadillac" method is to draw a random sample from the full population of possible cases. Nature did it. but at a decreasing rate. to regard victory in an uncontested race as evidence that the MC [member of congress] has captured 100 percent of the vote would probably exaggerate M C s ' actual level of support. a number of seats were uncontested. though it would be (barely) possible to do detailed analysis and fieldwork in each country of the world. (His approximated solution was to estimate from other factors the vote that the unchallenged incumbents would have gotten if they had been challenged. In this case. It is as if we had flipped coins for each possible case and. On the other hand. The relationship they are looking for would be expected to be the same in any kind of group of people. His theory predicted that incumbent members' safety from electoral defeat. in which the full range of possibilities did not get examined. he would be overstating . the guiding rule in choosing observations for study is that the relationships we are looking for in the full universe of possible cases should be mirrored faithfully among the observations we are using. thereby understating the amount of electoral security that develops. at varying levels of seniority. because no such students are in the group we are able to observe. Random Sampling When we are able to draw a fairly large number of observations. However. In this case. Zaller stated his problem: To set [these races] aside. admitted into the sample any case that got heads ten times in a row. of fourteen new states in Eastern Europe with significant ethnic minorities. we usually work with a sample drawn from the universe of all possible observations to which a theory applies. how they did on the GRE has little to do with how well they have done in the graduate program. the test is a nearly irrelevant predictor of success in the graduate program and should be dropped as a tool for assessment. scholars' attraction to the dramatic cases led to a strangely "selected" body of scholarship. and then to insert those simulated estimates as the observations for the unchallenged incumbents—not a perfect solution. • John Zaller (1998). So the available set of congressional races consisted only of those districts in which someone felt it made sense to challenge the incumbent. However. would increase with every term they served. this general impression has come about because almost all scholars who looked at ethnic conflict in new eastern European democracies were drawn to the dramatic outbreaks of violence in the former Yugoslavia. as is sometimes done.) In other words.

000 if they have questioned a sample of two or three thousand people. Some telephones have devices to block unsolicited calls. but it is increasingly difficult to draw a sample in this way that represents the broad population accurately. it would happen only rarely that you could have come up with a group among whom the women were more Republican than the men.100 Chapter 7 Selection of Observations for Study 101 If a sample has been drawn randomly. We saw there that variation in anything we are observing becomes less variable as we use larger and larger samples to examine it. and therefore would be twice as likely to end up in the sample. Some people are at work all day. And statistical adjustments can be made to take into account the clustered structure of the sample. Some families have two phone lines. The Literary Digest poll was drawn before sampling was well understood. exactly how likely it is that the sample result is any given distance from what you would have seen in the population. Even random samples will diverge by accident from the full population. Today almost everyone has a telephone. but it would still happen fairly often. one based in each of the localities chosen. and would have produced an erroneous prediction any time the middle class voted differently than the population as a whole. Others are always gone in the evenings. How much variation there is from one random sample to another is hugely influenced by the size of the sample. In other words. so that it operates almost exactly like a random sample. It always seems surprising that samples do not really need to be all that large to give an accurate rendition of a population. subscribers to the magazine. An alternative system is to draw a truly random sample of telephones by having a computer dial telephones randomly ("random digit dialing. it is rarely used. Now. two groups are randomly chosen. very much skewed toward the middle class. But in practice. which we looked at in Chapter 6. randomization ensures that any two groups chosen are essentially alike. surveys of American citizens are considered to have sufficient cases for accurately mirroring the population of 302. This allows the survey to use approximately 100 interviewers. It is kind of amazing that just a few thousand out of hundreds of millions would be sufficient to estimate the population to within a couple of percentage points. although there is a "gender gap" in American voting. the less variation there will be from one time to another when we draw samples and look at a relationship. But if that is true in experiments. true random sampling is often feasible. across repeated samplings of this sort. might provide a reasonable basis for drawing psychological conclusions about people in general. of course. It is this mathematical system that allows pollsters to say of a national poll. various methods exist to draw "quasi-random" approximations to a random sample. If any group chosen randomly from the population can be expected to be essentially the same as any other. and is then. at least in the sense that. then they must all be essentially the same as the full population. we can see from our consideration of random sampling how they might (or might not) be questionable as samples. at least for large populations.) One nice thing about random samples is that there is a very precisely worked out mathematical system to ascertain. by chance.S. The sample of college students. If you drew samples of a thousand Americans. it would not be unusual for a sample of ten Americans to include a group of women who were more Republican than the men in the sample. so their telephones will ring at any daytime hour but not be answered unless by a machine. the problem with telephone lists was that only the middle class had telephones. The larger the sample. while the true random sample is the Cadillac of sampling. RDD samples are rather problematic as approximations of a random sample of the population. however. Wisconsin. within an error of plus or minus 3 percentage points. Then. another to interview a person in San Diego. that the ten people you drew for a sample would show the reverse of that. Typically. a certain number of individuals are randomly chosen for the sample. it must also be true when trying to generalize to a population from a sample. The National Election Study at the University of Michigan uses what is called a "cluster sample": About 100 localities are first selected. Since in fact men are more likely to be Republicans than women are. so they cannot be expected to differ in any significant way that could interfere with a causal interpretation from the experiment. Looking back now at the first two examples stated. each interviewer can readily reach the 20 or 30 subjects randomly selected from her cluster. the average expression of the gender gap would almost surely mirror faithfully the gender gap in the full population. p. Its sample of telephone owners." or RDD sampling). for example. Quasi-random Sampling When a large population is involved. In randomized experiments. people's use of their phones varies enough so that the group that can actually be reached by phone at a given time is problematic as a sample. and those appearing on state lists of registered automobiles was simply a terrible sample.000. for a sample of any given size. we are assured that any relationship in which we are interested should be mirrored faithfully in the sample. For relatively small populations. For instance. The principle here is the same as that in randomized experiments. and then. even though they are clearly not . such as the population of a city. with men favoring the Republican party more than women do. If you took a large number of such samples. on the average the relationships we would see would be the same as the relationship in the full population. for instance. and averaged them. instead of having interviewers fly or drive hundreds of miles for a single interview. This allows access to a random sample without the difficulty or expense of getting an interviewer to the person's door. on the other hand. (Review the discussion of the Law of Large Numbers. 61. that it shows support of 48 percent for George Bush. so in principle RDD sampling could work well. most samples of ten Americans would show men to be the more Republican. In 1932. the method of choice. The end result approximates a random sample closely. But you can imagine what it would cost to sample the U. at the time of the Literary Digest poll described earlier. within each of these. and so on. population in a truly random way—sending an interviewer to interview a single person in Menominee.

In principle. is fine." will still be huge. rather than purposive. If it faithfully mirrored the population. 106-108) why we do not draw the sample to maximize variation in the dependent variable. 1 Selection of Cases for Case Studies Another kind of purposive sampling comes into play when we are dealing with such small numbers of cases that random sampling would not be very helpful in any case. Another solution. But any given couple of cases—for instance. combining them to construct a sample that is half white and half African American. It might be a study of a particular president's administrative style. Purposive Sampling Sometimes we deliberately want to sample in a nonrandom way.102 Chapter 7 Selection of Observations for Study 103 a random sample of the full population if we are confident that their psychological processes are the same as psychological processes among the full population. In the preceding example. for instance. Zaller." instances in which part of tue range of cases to which a theory applies are cut off and unavailable to the researcher. A purposive sample works only for the particular independent variable or variables on which we have based the sample. too small a group for reliable comparisons in a sample of the usual size. for instance. One solution would be to draw a monstrously large sample. either by the researcher's choice or by circumstances. a study of some aspect of politics in one or a few countries. but because a deliberately constructed. Sometimes there are reasonable ways to fill in the spaces. If we want to study the effect of authoritarian government on economic growth. such as a national cluster sample) would probably not yield enough African American citizens for a good comparison. it should have about one-eighth of the sample African American. In this case. this would not work as a general reflection of the full population. random or quasi-random. but he could observe election outcomes only for those members who were challenged by an opponent. Under these circumstances there is much more to be gained by intelligently choosing the case(s) to pick up the relationship of interest. even if we are looking at only two or three cases. a purely random sample (or good approximation. estimating a relationship or set of relationships based on race. it draws subjects to maximize variation in the independent variable of interest. A purposive sample does not attempt to replicate the full population. the Law of Large Numbers is working so weakly when we study only a few cases that the divergence of any case from the full population. the theory that ancient hatreds would flare up after Soviet repression was lifted from the region obviously was meant to apply to all eastern European democracies with large ethnic minorities. If African Americans were more likely to vote Democratic than whites. the sample would give an unreasonably high estimate of the Democratic vote. estimated on the basis of other variables what vote unchallenged members could have been expected to receive 'We will see (pp. rather than by randomly drawing the case(s). But the one thing for which the sample is going to be used. rather than just taking four randomly chosen states. Like all purposive samples. if the purpose of the study is to look at the politics of race. an intensive study of the attitudes and ideologies of a few people. and can serve as a satisfactory quasi-random sample if processes are indeed the same as for the full population. nonrandom sample serves a purpose for our research. but the researcher can see the performance only of students who were admitted to the program. say. If we wanted to compare the political choices of African Americans and whites. rather than one focused just on the single task we start with. the news-watching habits of the two races should reflect accurately the difference in news-watching habits in the full population. But in practice. any generalization about the importance of test scores is obviously meant to apply to all students who could apply for admission. it is intended to serve multiple purposes. or. a randomly selected sample of even just a couple of cases will mirror the full population. The problem of censored data is always severe. the ones you plan to study—are likely to diverge sharply from the full population. In the case of ethnic violence in eastern Europe. So it is usually safer and more useful to draw up a more general sample. not just for reasons of efficiency or cost. it is much cheaper than trying to draw a random sample of all people. A case study is an intensive study of one or a few cases. even though the sample is not random. 2 CENSORED DATA The last three of the five examples in the introduction to this chapter were examples of "censored data. In the case of GRE scores. it makes sense to seek a couple of authoritarian states and a couple of nonauthoritarian ones for comparison. we might want to look at things in the data other than what we had anticipated when we were first drawing up the sample. we could sample African Americans randomly and sample whites randomly. for instance. so that the relationships we are looking for will be very clear. 2 . an estimate of. is that usually when we draw a sample. for instance. Since both African Americans and whites have been randomly (or quasi-randomly) selected. as it is most frequently seen. random selection of the case or cases would reflect the full population in the long run. This is a situation that seems to fit Keynes' well-known comment that in the long run we shall all be dead. though it will be "only random. Rather. One reason that most sampling is random or quasi-random. And we should probably also anticipate that as we proceed with analysis. is to draw what is called a "purposive" sample. be very expensive. because it requires researchers and their readers to make strong assumptions about what "might have been" in the range of data that are not available. In principle and in the long run. but scholars had chosen to look only at the dramatic cases in which conflict had actually occured. of course. but that would. John Zaller's theory of congressional elections was obviously meant to apply to all members of Congress.

5 4. and the "Asian Tigers": Singapore. As you can see in her graph. we would see that there was little or no advantage for the countries that repressed labor. (GDP per Capital from Penn World Tables. reproduced in Figure 7-1. There may be a slight relationship. the upper left-hand part of the figure is a little less thickly filled with cases. just picking up on what others had not noticed: that most eastern European states with ethnic minorities had not exploded when the Soviets left. and Taiwan.0 0. Figure 7-2 Labor Repression and Growth in the Full Universe of Developing Countries. p. Barbara Geddes (2003) gives a nice example of this. When Scholars Pick the Cases They're Interested In a •Singapore - 6 4 2 0 Mexico* . Scholars of revolution are prone to look at the Soviet Union or Cuba.0 4.5 3. rapid economic growth). South Korea. as in this example of ethnic conflict in eastern Europe. rather than less showy presidents such as Calvin Coolidge. Looking at the full range of labor repression and the full range of economic outcomes. (GDP per Capita from Penn World Tables.0 2. Geddes developed a measure for how much labor organization was repressed by governments.0 3. if we had all of the developing countries of the 1970s before us. investigators doing case studies gravitate to the outcomes in which they are interested.5 4.0 Labor R e p r e s s i o n Figure 7-1 Labor Repression and Growth in the Most Frequently Studied Cases.5 2. indicating little -2 -4 0. p.104 Chapter 7 10 Q O Selection of Observations for Study 105 b had they been challenged. And sometimes the answer is easy: If investigators have ignored some of the range but can study it.5 5. based on case studies of rapidly developing countries. filling in for their short-sightedness.0 0. Mihaelescu was able to do a useful and rewarding study. however.0 4. They then look at those cases to see what caused the thing in which they are interested. which have grown rapidly.5 5. 103 10 •Singapore 8 6 4 Growth in G D P per capita 2 0 -2 -4 0. Most case studies of economic growth at the time focused on Mexico. Brazil. and then placed the five frequently studied countries on a graph of labor repression. that it was important for developing countries' governments to repress organized labor to let industry develop rapidly. 101 relationship between the two variables.0 1. that just makes a neat and interesting study for another investigator to conduct. Taiwan. and inserted those estimates into his study as imputed data. As Geddes points out. 'Taiwan South Korea •Brazil Growth in G D P per capita c d Often. the focus on these several cases of rapid development led to a very wrong-headed conclusion. there is nothing like the kind of relationship that appeared from the censored data in Figure 7-1.) Adapted from Geddes (2003). 1970-1981. But overall. »Taiwan South Korea •Brazil 1. There is very little pattern in the figure.0 Labor R e p r e s s i o n . . As Geddes showed in work adapted here in Figure 7-2. 1970-1981. and the lowest growth (Mexico's) also happens to coincide with a lower repression of labor.0 3. we find about as many low-growth countries with low levels of labor repression as with high labor repression. I will argue later that choosing cases to maximize variation in the dependent variable distorts relationships. Scholars of economic growth are prone to look at countries such as Korea. all five cases fall in the upper -right-hand part of the graph (b: high labor repression. or Brazil. Scholars of the presidency are likely to look at the administrations of dramatically successful presidents such as Franklin Roosevelt or Ronald Reagan. indicating that there were somewhat fewer low-repression countries with high growth. and that one should instead select to maximize variation in the independent variable or variables.5 3.0 2. She points out that a strong argument arose in the 1980s.5 •• Mexico* . related to growth in per capita GDP from 1970 to 1981.5 1 1 1 1.) Source: Geddes (2003).0 1. But another result of choosing cases that exhibit most strongly the thing we are interested in is that it often also results in a censored data set. and pick cases for which the outcome was strong or dramatic.5 2.

000 Support 45% 80% Don't Support Mayor 110. and similarly drawing 200 whites from the pool of whites. In Figure 7-4 I have drawn two samples of 400 from this population. of course. And one cannot decree that all congressional incumbents will be challenged for reelection. and bring it into the analysis. at least if you have enough cases so that the Law of Large Numbers can work for you. And it's a nice challenge. seek out those graduate departments that for whatever reasons do not use the GRE as a criterion.000. A good deal of artfulness is required to analyze data that have been censored by nature. despite the fact that assumptions are required. and the other a purposive sample to maximize variation in support or opposition to the mayor. for drawing "what if?" conclusions. which is. it would require the assumption that departments that did not require the GRE were like other departments in every other way. We might.000 100% 100% 7-3 Race and Support for the Mayor: Hypothetical Polarized City true population figures. the relationship you observe will mirror nicely the true relationship. there are no easy solutions. we find that 80 percent of whites support the mayor.000 310. This is what Mihaelescu or Geddes did. is that this can work well when you select cases to maximize variation on the independent variable. Figure 7-4(A) shows the results of drawing 200 nonwhites randomly from the pool of nonwhites shown in Figure 7-3. as in the example of GRE scores. Zaller had to make a number of assumptions to predict what vote unchallenged members would have gotten had they been challenged. but a creative imagination will be rewarded by substantially improved estimation. One cannot under ordinary circumstances admit to graduate school a group of students who have done badly on the GRE test to see how they perform. by race.000 Figure 1. SELECTION ALONG THE DEPENDENT VARIABLE: DON'T DO IT! When we looked at selection of samples.000 Don't 55% 20% 200. But if the data have been censored by nature. In such cases. however. The missing cases are gone and are not going to reappear. This would not be a perfect solution. And it would ignore the fact that the low-GRE students admitted to those departments would be atypical as well. And it does not matter in this case whether you have a large or a small sample. one a purposive sample to maximize variation in race.000 800. If you maximize variation in the independent variable. and justify them. Similarly. you will distort the true relationship. they are probably unusually high achievers in other ways. or Zaller's work on congressional elections. find the GRE scores of students who attended there. but that you should never select to maximize variation on the dependent variable. This means we must make some assumptions. In Figure 7-3 we see the breakdown of support for the mayor. All one has to do in this case is to seek out a broader range of information.106 Chapter 7 Numbers: Nonwhite Support Mayor White Selection of Observations for Study Percentages: Nonwhite White 107 When Nature Censors Data What can one do about censored data? If the data have been censored because of decisions of researchers. for example.000 200. the problem is easy to solve. for example. While this Figure 7-4 Selection Along Independent. unlikely. since they would have been rejected by other departments but ended up being accepted into the non-GRE departments.000 890. this would be better than ignoring the initial problem of censored data. These are the 90. we are forced to draw conclusions of what might have been had the data not been censored. Let us illustrate this with a hypothetical city with significant racial polarization. and see how they did. but only 45 percent of nonwhites support the mayor. and can in fact offer a nice chance to another researcher to make a significant contribution. Taking percentages. But if you maximize variation in the dependent variable. Nonetheless. A very important rule. Dependent Variables Numbers: Nonwhite White Percentages: Nonwhite White (A) Selection Along Independent Variable Support 86 171 262 Support 43% 85% Don't 114 29 138 Don't 57% 15% 200 200 (B) Selection Along Dependent Variable Numbers: Nonwhite White Percentages: Nonwhite White Support 39 161 200 Support 38% 54% Don't 64 136 200 Don't 62% 46% 103 297 . we noted that you might usefully draw a purposive sample to get sufficient variation in the variables of interest.

It had a proportional representation electoral system. It looks a little artificial—why would anyone do this in the first place? Actually. And operationally. but sampling along the dependent variable does not? It makes sense. ANOTHER STRATEGY. It leaves unaffected the percentages in which we are interested. Case-selection forms the foundation for all of your examination of evidence. offers a good tangible example of about how much variability one gets with random sampling of this size. (Note that sampling on the independent variable distorts the proportions of whites and nonwhites in the population. and try to account for it.) 3 intensive studies of one or a few cases (so-called case studies). in which you do not look at a relationship between variables across cases. with a widely variable estimate of nonwhites' support for the mayor. with regard to This sample. Or. and see whether the things you expected to have been the causes were present. HOWEVER: SINGLE-CASE STUDIES SELECTED FORTHE RELATIONSHIP BETWEEN THE INDEPENDENT AND DEPENDENT VARIABLES So far in this chapter I have dealt with issues having to do with: the selection of cases. compared with 45 percent of nonwhites. There is another kind of case study. Figure 7-4(B) shows the results of a purposive sample of the same size that maximizes variation in the dependent variable. SELECTION OF CASES FOR CASE STUDIES (AGAIN) The preceding example clarifies why one should not sample on the dependent variable. you might choose two cases. For instance. In fact. It is so easy when studying a political outcome to choose a case in which the outcome occurred. the question of sampling from a larger population. but it is wrong. and the logic there is just the same as the logic of the preceding example. Why is it that sampling along the independent variable produces accurate estimates of the relationship. and. and since it is drawn along the independent variable. drawing this sort of sample does not produce a faithful reflection of the relationship in the full population. and see whether the things you expected to have been the causes were present. however. Drawing groups from the two and comparing them fits this very well. with 54 percent of whites supporting the mayor. In the full population. however. support for the mayor. as do 85 percent of whites (compared with 80 percent in the full population). pp. It does come up time and again. changing the relative numbers of supporters and opponents from what one sees in the full population changes the percent of supporters in both racial groups. since that would have produced only a small number of nonwhites. it replicates faithfully the relationship in the full population. 2-3). and it will work well as long as your procedures fulfill the basic principle enunciated so far in this chapter: The relationship to be observed among the cases you have selected must mirror faithfully the true relationship among all potential cases. 3 . But the sample maximizing variation in support appears to indicate a considerably weaker relationship. 43 percent of nonwhites support the mayor (compared with 45 percent in the full population).108 Chapter 7 Selection of Observations for Study 109 sample is on the small side. because the purpose of the analysis is to compare nonwhites' and whites' support for the mayor. the selection of cases affects profoundly what results you see. and so allows you to examine straightforwardly where the chips fall under varying circumstances. As we see. I chose to introduce the idea through a large-scale survey example because the problem is easier to see there. and according to theory. for the same reason we saw in the earlier example—namely. see where the thing you're trying to explain has occurred. Gerhard Loewenberg (1968) was struck by the fact that Germany by the late 1960s looked just the opposite of what we would have expected from theories of electoral systems. For such research. by the way. and the question of whether to select cases on the basis of what you are trying to explain or on the basis of your independent variable. This does not present the same picture of sharp polarization. one in which the outcome had occurred and another in which it had not. It is intuitive. Choosing instead cases that represent varying instances of your explanatory variable allows you to examine the full range over which your explanation is meant to apply. 80 percent of whites support the mayor. it is more accurate than what we would have gotten from a purely random sample of the population. but it may puzzle the reader a bit. but it does not fiddle at all with the likelihood that the outcome occurs. This has all been aimed at research in which we are comparing cases (anywhere from two cases to thousands) to look at the pattern of the relationship between independent and dependent variables. it draws enough nonwhites to make fairly reliable comparisons with whites. this does not come up often with regard to large-scale sampling. In the sample. it increases the number of nonwhites available for analysis. in ways that distort the percentages in both groups. but that this is actually helpful. The numbers diverge by about the amount we would normally expect in drawing a random sample of 400 cases. doing something like this is the most obvious and intuitive thing to do. but the real importance of the argument is with regard to case studies. a relationship of sharp polarization. this should lead to a multiparty system if the country had multiple divisions in society (see the discussion of Duverger's theory. but instead look at a single case to help clarify or illuminate a theory. but the relationship is recognizably the same. In fact. But drawing groups of supporters and opponents of the mayor and then comparing whites' and nonwhites' support does not parallel the research question in the same way. compared with 38 percent of nonwhites. the problem of censored data. if you properly wanted to avoid the problem of censored data. that it distorts the likelihood that the outcome occurs or does not occur.

we s h o u l d then be c l e a r that we are not p r o v i d i n g a test of the theory. K e o h a n e . This argument is developed in Shively (2006). the number of parties represented in the prewar G e r m a n R e i c h s t a g had been 32!) B u t by the late 1960s G e r m a n y had almost a two-party s y s t e m . " H o w the C a s e s Y o u C h o o s e A f f e c t the A n s w e r s Y o u Get. 0 0 0 . C o l l e g e graduates might average $ 5 0 . F i r s t . 0 0 0 a y e a r . O r . T w o v a r i a b l e s are not s i m p l y related or not related. 0 0 0 and the average i n c o m e o f h i g h s c h o o l dropouts m i g h t b e $ 3 0 . for two reasons. though fairly technical. in h o w strongly the independent v a r i a b l e affects the dependent v a r i a b l e . R a t h e r . in order to refine the theory and illuminate how it worked. as noted in footnote 5 in that chapter. but rather to develop the theory by l o o k i n g in detail at a c a s e in w h i c h the theory is w o r k i n g in particular w a y s — we s h o u l d not ignore the dependent variable in s e l e c t i n g our c a s e . ( 2 0 0 0 ) in an appendix. G e d d e s ( 2 0 0 3 ) has a very nice chapter. 0 0 0 a y e a r and s o m e h i g h s c h o o l dropouts might m a k e $ 8 0 . F o r e x a m p l e . it takes on a variety of v a l u e s . w h i l e h i g h s c h o o l dropouts a v e r a g e $ 1 5 . O u r task then is not j u s t to see whether the presence or absence of a test factor affects the value of the dependent variable. e v e n though the average i n c o m e of 111 . the test factor in field r e s e a r c h is frequently not something that is s i m p l y present or absent. although I treated it as a d i c h o t o m y in C h a p t e r 6. 5 4 i n c o m e and h i g h s c h o o l dropouts $ 1 5 . rather. A n d . F o r i n s t a n c e . for i n s t a n c e . Currently there are five parties represented. Chapter 4. S o m e c o l l e g e graduates might m a k e only $ 1 0 . T h i s w a s too s i m p l e . we s h o u l d p i c k the c a s e b e c a u s e of how the theory appears to be w o r k i n g . Rather. with the two largest (the Social Democrats and the Christian democrats) holding about 90 percent of the seats. and our task is to see whether the amount of education a person has affects his or her i n c o m e . Three parties were represented in the parliament in the 1960s. 5 In C h a p t e r 6 we w e r e c o n c e r n e d generally w i t h " r e l a t i o n s h i p s " b e t w e e n independent and dependent v a r i a b l e s . and V e r b a ( 1 9 9 4 ) . " r e l a t i o n s h i p " cannot be d i c h o t o m i z e d . J o h n G e r r i n g (2007) provides a comprehensive and intelligent review of case selection. as indicated by the c o m b i n a t i o n of dependent and independent variables. we do not treat people s i m p l y as " e d u c a t e d " or "not educated. ( A t its highest. Relationships also v a r y in h o w completely the independent v a r i a b l e deter- m i n e s scores on the dependent variable. 0 0 0 Since then the system has moved back somewhat toward multipartism. Rather. we w a n t e d to see whether the p r e s e n c e or absence FURTHER DISCUSSION of a test factor affected the v a l u e of the dependent variable.110 Chapter 7 Chapter 8 G e r m a n y had demonstrated in the 1920s and 1930s that it did indeed have a richly divided society. B u t we s h o u l d also not select the c a s e j u s t in terms of the dependent variable. but instead. to ease the presentation there. 0 0 0 . he p i c k e d it b e c a u s e of the combination of the electoral s y s t e m and historic divisions in society (independent v a r i a b l e s ) and the party system (dependent 4 © Measuring Relationships for Interval D a t a v a r i a b l e ) . 0 0 0 . treatment of c e n s o r e d data is presented in P r z e w o r s k i et a l . C o l l e g e graduates might average $ 5 0 . 0 0 0 . He did not s i m p l y p i c k a c a s e w i t h a dramatic or interesting outcome." A good. R e l a t i o n s h i p s v a r y in t w o w a y s : first. S e c o n d . W h e n a c a s e study is b e i n g u s e d in this w a y — n o t to test a theory by trying it out to see h o w w e l l it w o r k s ." T h e y have v a r y i n g amounts of education. T h e average i n c o m e o f c o l l e g e graduates might b e $ 3 3 . this is a strategy for further refinement of the theory. H o w could this b e ? L o e w e n b e r g studied the G e r m a n case in depth to see what had produced this exception to the theory. e d u c a t i o n might h a v e o n l y a m i n o r effect o n i n c o m e . to see whether and h o w the value of the independent variable affects the value of the dependent variable. Note that L o e w e n b e r g did not p i c k this c a s e j u s t in terms of the dependent variable. " A n o t h e r good treatment of case selection is K i n g . and the "big two" have 73 percent of the seats. yet there might still b e m u c h variation in i n c o m e s that c o u l d not be attributed to variation in p e o p l e ' s education. T h a t i s . it m i g h t h a v e a m a j o r effect. in relating education to i n c o m e . 0 0 0 or $ 9 0 . " S e l e c t i o n M o d e l s .

state. Although for the sake of simplicity these were presented as if we were interested only in whether or not a relationship existed. If we are using a nominal-scale variable." Their relationship may be such that the independent variable has a greater or lesser effect on the dependent variable. as in Figure 8 . dot A represents an observation that . all we know is whether the two values are distinct. This means that we cannot measure how great a difference is induced in the dependent variable by a change in the independent variable. This can be seen in our examples of research design in Chapter 6. given varying scores on the independent variable. I shall call the first way of measuring a relationship effect-descriptive. Thus variables are not simply "related" or "not related. but with how many votes he could swing. and the second correlational. Ordinal and nominal measurement techniques cannot tell us how great a difference in the dependent variable is produced by a given difference in the independent variable. or (2) by how completely the independent variable determines the dependent variable. that is." for instance. I introduce some statistical techniques for measuring the strength of relationships. Our task in evaluating the results of research. because under most circumstances. Recall that we mentioned two ways to measure the "strength" of a relationship between two variables: (1) by how great a difference the independent variable makes in the dependent variable. how complete an explanation of the dependent variable is provided by the independent variable. Each dot in the scattergram represents one observation (a person. In "Presidential Lobbying. "of state affairs. of course. how greatly values of the dependent variable differ. The critical difference between working with interval-measured data and working with data measured at a lower level is that effect-descriptive measurement can apply only to interval-scale data. Generally speaking. we should be able to measure a greater variety of things about it. WORKING WITH INTERVAL DATA Regression Analysis IMPORTANCE OF LEVELS OF MEASUREMENT In Chapter 5 we saw that we have more information about a relationship between variables if we work at a higher level of measurement than if we work at a lower level of measurement. Basically." Statistics includes two main activities: statistical inference and statistical measurement (including the measurement of relationships. The tools we need to accomplish this task are found in the field of statistics. we know whether one value is higher than another. This becomes a particularly important distinction in political research. there are two major types of techniques: those appropriate for data measured at the interval and those suited for lower-level measurements. In this chapter and in Chapter 9. relationships between variables are different depending on the level at which the variables were measured. it was actually political scientists of a sort who first developed the field. effect-descriptive ways of measuring the strength of a relationship are more useful than correlational ways. with which we are concerned in this chapter). Just as higher levels of measurement yield relatively richer information about a variable. This would indicate that although education affected incomes sharply. it is clear that what was of interest to the investigators in each case was finding out how strong a relationship existed. The name statistics derives from the Latin statisticus. If we know more about a relationship. it was relatively incomplete as an explanation of people's incomes. If we are using an ordinal-scale variable. there must be other things affecting income in important ways. Correlational measurement of one sort or another can apply to data measured at any level. for example). and it may be such that the independent variable determines the dependent variable more or less completely. This is what it means to say that education is not a very "complete" explanation of income. I demonstrate this in the next few sections. the president was not simply concerned with whether or not he was able to influence voting for the bill. Because income still varied a good deal within each level of the independent variable. but we do not know how much higher it is. is to measure how strong a relationship exists between the independent variable(s) and the dependent variable. political research is not so much concerned with whether or not two variables are related but with whether or not they have a "strong" relationship (in one or both of the senses used earlier). Statistical inference consists of estimating how likely it is that a particular result could be due to chance. For instance. then. for statistics originally grew out of the need to keep records for the state. or country. so techniques for measuring relationships at high levels of measurement give relatively richer information about those relationships. I discuss inference in Chapter 10.112 Chapter 8 Introduction to Statistics 113 the college graduates was higher than the average income of the dropouts. STATISTICS Although modern political scientists have begun to use statistics extensively only in the past several decades. it tells us how reliable the results of our research are. and we often would guess incorrectly if we tried to predict a person's income solely on the basis of education. It should not be too surprising that methods of measuring A convenient way to summarize data on two interval-scale variables so that we can easily see what is going on in the relationship between them is to plot all the observations on a scattergram. The whole point of nomi lal and ordinal measurement is that in neither do we have available a unit by which to measure the difference between two values of a variable.1 . placed on the graph according to its scores on the two variables. that is. although this is precisely what is required to measure the relationship in an effect-descriptive way.

c o m b i n e s scores of 3. c o m p a r i n g t w o scattergrams gives us o n l y an approximate idea of the differences b e t w e e n t w o relationJ I I I I I I L s h i p s . i t i s u s e f u l t o d e v i s e a p r e c i s e n u m e r i c a l m e a s u r e t o s u m m a r i z e the relevant characteristics of a relationship s h o w n in a scattergram. T h e s i m p l i f y i n g l i n e m a y be t h o u g h t of as a rule for predicting s c o r e s on the d e p e n d e n t v a r i a b l e f r o m s c o r e s o n t h e i n d e p e n d e n t v a r i a b l e s . s u c h a s A . It w o u l d be painful to read such a paper if e a c h relationship w e r e presented in the f o r m of a scattergram. w e c a n s a y t h a t i n t h e first graph the i n d e p e n d e n t variable c a u s e s greater shifts in the d e p e n d e n t variable than in t h e s e c o n d .1 . T h u s w e k n o w that t h e t w o v a r i a b l e s are p o s i t i v e l y related. i n F i g u r e 8 . First. By l o o k i n g at the pattern f o r m e d by the d o t s . W e h a v e o b s e r v e d w h i c h w a y t h e d e p e n d e n t v a r i a b l e m o v e s w i t h c h a n g e s i n t h e i n d e p e n d e n t v a r i a b l e . a straight line s u c h as this c a n be fully described by the equation \ y = a + bx . Its u s e f u l n e s s a s a predictor d e p e n d s on k e e p i n g the average squared v a l u e of "deviant" s c o r e s on the dependent variable as l o w as possible. Graph A s h o w s a nonlinear relationship (the dependent variable increases faster w i t h increases in the i n d e p e n d e n t variable if the i n d e p e n d e n t variable h a s a high value). w e o f t e n m e a s u r e t h e strength o f a r e l a t i o n s h i p b e t w e e n t w o v a r i a b l e s w h i l e h o l d i n g a third v a r i a b l e c o n s t a n t ( s e e t h e d i s c u s s i o n o f this t o p i c o n p p . it can be u n w i e l d y to w o r k with. A l t h o u g h the scattergram tells us a g o o d deal about a relationship.1 . T h i s m e a n s that h i g h s c o r e s o n t h e d e p e n d e n t v a r i a b l e t e n d t o c o i n c i d e w i t h h i g h s c o r e s o n t h e independent variable.5 on the independent variable and 5 on the d e p e n d e n t variable. this relationship appears to be a p p r o x i m a t e l y linear ( s e e the d i s c u s s i o n of "linear relationships" in the b o x on p. It is not u n c o m m o n in a research report to d i s c u s s 30 or 40 F i n a l l y a n d m o s t i m p o r t a n t . Graph B s h o w s a linear relationship in w h i c h the d e p e n d e n t variable i n c r e a s e s m o r e gradually than in the graph in F i g u r e 8 . T o d o t hi s u s i n g s c a t t e r g r a m s m a y b e e x t r e m e l y c u m b e r s o m e . the m a t h e m a t i c a l l y best procedure is to c h o o s e the unique line that m i n i m i z e s t h e s q u a r e d d i f f e r e n c e s b e t w e e n o b s e r v e d v a l u e s o f t h e d e p e n d e n t v a r i a b l e a n d its i d e a l i z e d v a l u e s a s g i v e n b y t h e s i m p l i f y i n g line. W h a t is m o r e . t h e pattern i n t h e d o t s o f a s c a t t e r g r a m i s s u m m e d u p b y t h e s i n g l e l i n e that b e s t a p p r o x i m a t e s it.2 illustrate various other patterns w e m i g h t have observed. b u t w e c a n n o t s a y p r e c i s e l y h o w m u c h g r e a t e r t h e s h i f t s are. T h e l i n e a r r e g r e s s i o n c o e f f i c i e n t i s d e r i v e d i n t h e f o l l o w i n g w a y . C o m p a r i n g t h e g r a p h s i n F i g u r e s 8 . D o t B represents an observation with scores of 12 on the independent variable and 17 on the dependent variable. T h e scattergrams i n F i g u r e 8 . 9 4 . In particular. the j o b would become well-nigh impossible. A single s u m m a r i z i n g line can be described m o r e e a s i l y than a pattern of dots.2 ( B ) . w h e r e a simplifying line has b e e n drawn through a scattergram with observ a t i o n s o n s e v e n h y p o t h e t i c a l c o u n t r i e s t o s u m m a r i z e t h e pattern a c r o s s t h e c o u n t r i e s . F o r all o f t h e s e r e a s o n s . 7 1 ) . or if we w e r e c o m p a r i n g several relationships. Graph D s h o w s a pattern in w h i c h there is no relationship. I f t h e Figure 8-1 Scattergram differences w e r e m o r e subtle. Graph C s h o w s a negative linear relationship in w h i c h the d e p e n d e n t variable d e c r e a s e s as the i n d e p e n d e n t variable increases.1 a n d 8 . T h e s e are b o t h part of an effect-descriptive m e a s u r e m e n t of the relationship.9 5 ) . a n d w e h a v e o b s e r v e d that i t m o v e s a t a s t e a d y rate a t a l l v a l u e s o f t h e i n d e p e n d e n t v a r i a b l e ( a l i n e a r r e l a t i o n ship) rather than at c h a n g i n g rates (a n o n l i n e a r r e l a t i o n s h i p ) . W e h a v e d o n e t w o t h i n g s s o far. T h i s i s illustrated i n Figure 8-3. w e n o t e that t h e r e are f e w d o t s i n t h e l o w e r . a n d t h e p o i n t B a t w h i c h a c o u n t r y h a v i n g A's s c o r e o n t h e i n d e p e n d e n t v a r i a b l e w o u l d b e e x p e c t e d t o fall o n t h e i d e a l i z e d s i m p l i f y i n g l i n e .r i g h t a n d u p p e r . and l o w scores on the dependent variable tend to c o i n c i d e with l o w s c o r e s o n t h e i n d e p e n d e n t v a r i a b l e . Furthermore. F o r i n s t a n c e . It has b e e n drawn to m i n i m i z e the squared differences b e t w e e n e a c h of the observed p o i n t s . For a linear relationship. T h e measure c o m m o n l y u s e d to summarize the effect-descriptive characteristics of a Independent Variable s c a t t e r g r a m is t h e regression coefficient.114 Chapter 8 (A) Introduction to Statistics (D) 115 Independent Variable Figure 8-2 Assorted Scattergrams separate relationships.l e f t c o r n e r s o f t h e g r a p h . we c a n tell a g o o d d e a l about the r e l a t i o n s h i p b e t w e e n t h e t w o v a r i a b l e s .

the slope provides a single. the expected value of y when x equals zero. the formulas will still give us an a and a b. or the slope of the regression equation. Continuing with our example. we do not escape the scholar's obligation to think. this poses no real difficulty.5 with every 1. but the shift of one unit in x in the real world will not be followed by a shift of b units in y. is 6 + (4 x 3). to find the predicted value of the dependent variable for any specified Figure 8 . impersonal numbers. that is. and a and b are numbers relating y to x.4 The Regression Equation The equation of this line is y = 6 + 3x. The slope. how can we tell whether it has a greater impact on diplomatic interaction than does geographic distance.000. the predicted percent voting equals 50. that even though we work with neat. shows by how many units y increases as x increases one unit. often simply called the regression coefficient. or 65.5. you must add a (the predicted value of y when x equals zero) to b times the number of units by which x exceeds zero. If we have guessed the direction of causation between x and y incorrectly. Remember. we might want to know which variable—nations' distance from the United States or the volume of their trade with the United States—has a greater impact on their diplomatic interaction with the United States. y decreases as x increases. The number a. The number b. If income is $30. For example.) In other words.000. or 1 8. is the most valuable part of this equation for most purposes in the social sciences. But often we may be interested in comparing the effects of two or more independent variables on a particular dependent variable. is +0. If the regression coefficient for volume of trade.5.116 Chapter 8 Introduction to Statistics 117 value of the independent variable. If the two independent variables are measured in different sorts of units. is called the intercept of the regression equation. or 60. if diplomatic involvement could be expected to decrease by 10. By telling how great a shift we can expect in the dependent variable if the independent variable shifts by one unit. If we are working with just two variables. that the relationship between income and electoral participation is linear and can be summarized by the regression equation percent voting = 50.0105 with every mile of distance. if there is a regression coefficient of—10. this can be difficult.024.5 for nations' diplomatic involvement with the United States (measured by the number or magnitude of exchanges per year) predicted from their distance from the United States measured in thousands of miles. it is the value of y where the regression line crosses the y axis. measured in millions of dollars. it would be expected to decrease by 0. Independent Variable .000 miles of distance from the United States. plugging in our data and getting numbers out will not make the results valid. V percent more potential voters vote. where x equals zero (see Figure 8-4). The predicted value of y w h e n x is 4.5 + — (income in thousands of dollars) This means that with every additional thousand dollars of income. if income is $20. with its slope of—10. for instance.5? 2 Figure 8-3 T e Regression Line h where y is the predicted value of the dependent variable.0105 for the same dependent variable if distance were measured in miles. Thus the arguments made in Chapter 6 apply even when we work with simple numbers like these. there would be a regression coefficient of —0. If y causes x rather than vice versa. there is a negative relationship between the variables. however. For example. The problem of comparing units. precise summary measure of how great an impact the independent variable has on the dependent variable. x is the value of the independent variable. (If b is negative. the predicted percent voting equals 50. It may be seen from an examination of the concept slope (the number of units by which y changes with a change of one unit in x) that the slope has meaning only with regard to the units in which x and y are measured. Let's assume.5 + (1/2 x 30). That is. for instance.5 + (1/2 x 20).

for there is an infinite variety of nonlinear equations that you might fit to any set of data. two coefficients. are required to express the impact that a change in x will have on y. A nonlinear regression equation may be found to fit the pattern fairly well. If the concepts are not easy to compare. The solution to the problem requires going back to theory development. the regression equation will not summarize the pattern in the data. a nonlinear regression equation y . But these. since that change is not the same at all values of x. but will distort it. Because our theory anticipates a relationship between two variables. One important warning: Remember that the presentation I have made here is only a broad. Competence in using measures like those presented here requires more thorough training than is within the scope of this book. now a bit more complicated. is that the numbers do have meaning. as depicted in Figure 8-7. Any regression line is actually a reflection of the stage that we have reached in developing a theory. this will help you to design an appropriate nonlinear equation. It usually is necessary to play around with alternative nonlinear equations for a while. To the extent that your research is based on a well-thought-out theory. the equation depicted in Figure 8-6 would be inappropriate because it must inevitably reverse direction at some point. Many relationships in political science do turn out to be linear. we measure the relationship between them by . we could compare them in a theory of gravity by comparing their mass. You should always check your data before using linear regression analysis. we would have to develop in our theory dimensions along which they could be meaningfully compared. of course. introductory overview. We can compare them in a nutritional theory. In this case. therefore. The real answer. that x has on y. The regression equation provides a handy summary description of the effect.118 Chapter 8 The units—thousands of miles and millions of dollars—are not comparable. let the computer do it for you) and see whether it looks linear. This is particularly true for linear regression.a + b x — b x has been passed through the scattergram from Figure 8-5. contrary to the old saying about "comparing apples and oranges. It summarizes the pattern in the data more accurately. Otherwise. Apples and oranges actually can be compared. if your theory predicts that the dependent variable will always increase with increases in the independent variable. can be worked out readily enough. For instance. Checking for linearity." but to do so we must first specify along what dimensions our theory calls for them to be compared. Figure 8-5 shows a linear regression equation passed through the scattergram of a nonlinear relationship. Formulas are available to calculate equations for regression lines satisfying the least-squares criteria. But if the relationship you are investigating turns out to be nonlinear. We can see that y increases with x but decreases with the square of x. for instance. that is no reason to give up analyzing it. would be appropriate. including the ones cited at the end of this chapter. The best way to do this is simply to draw a scattergram (or rather. then regression coefficients referring to them will also not be easy to compare. the formulas for a and b are found in every standard statistics text. too. x 2 2 0 Examining the residuals. by measuring how much vitamin C each fruit contains. This regression line fits the data very badly and is not a useful summary of the relationship. but only with relationship to their original concepts. But there are no set "formulas" for nonlinear regression equations. Figure 8-5 Introduction to Statistics 1 1 9 0 Figure 8-6 Nonlinear Regression Equation 2 Linear Regression on a Nonlinear Relationship In Figure 8-6. To compare the effects of diplomatic interaction and geographic distance in the example above. but at a constantly diminishing rate (a "diminishing marginal returns" model). to consider on what grounds your concepts can be related to each other. An equation such as y = a + b log x. b\ and b . Or. the coefficients based on those units are not comparable either. It merely means that the relationship is more complex than you anticipated—and probably more interesting. We must be careful to make certain in using linear regression analysis that the data do fit a more or less linear pattern.

O ..120 Chapter 8 . we m a y notice that cases with similar residuals have s o m e additional characteristic in c o m m o n . for i n s t a n c e .a + b log x Introduction to Statistics 121 that it w o u l d h a v e b e e n difficult to identify the p r e s e n c e of a n o t h e r factor if you h a d n o t first r e g r e s s e d e d u c a t i o n a l s p e n d i n g o n G N P . This difference b e t w e e n the observed value a n d the predicted value is called the residual. a progressive candidate in the 1946 senatorial primary. He found a m o d erately strong relationship b e t w e e n the t w o . It m a y surprise u s — w h e r e we expected a relationship there m a y be none. you m i g h t notice that all o f t h e c o u n t r i e s for w h i c h e d u c a t i o n a l s p e n d i n g w a s h i g h e r t h a n p r e d i c t e d w e r e not d e m o c r a c i e s . K e y w a n t e d t o m e a s u r e t h e i m p a c t of factions in A l a b a m a p r i m a r i e s . c o n v e r s e l y . T h e technique of e x a m i n i n g residuals is illustrated in Figure 8-8. regression analysis is an e x p r e s s i o n of w h a t we already think about the subject. or it m a y be nonlinear." b e y o n d G N P . O. J r . But it is an expression of what we already h a v e b e e n thinking about the subject. ' s Southern Politics ( 1 9 5 0 . beyond the independent variable. T h i s m e a n s t h a t t h e r e i s still u n e x p l a i n e d v a r i a t i o n i n the values of the dependent variable. it indicated the presence of conservative 0 and progressive factions structuring the vote. the regression line can serve a further important function. T h e r i c h e s t d e m o c r a c i e s i n F i g u r e 8-3 s h o w a h i g h e r rate of e d u c a t i o n a l s p e n d i n g t h a n do t h e p o o r e s t n o n d e m o c r a c i e s . that acts as an independent variable. E x a m i n i n g t h e s e r e s i d u a l s p o i n t s o u t t o u s t h o s e c a s e s i n w h i c h t h e " s o m e t h i n g e l s e " h a s t h e effect o f r a i s i n g (or. p . this may then suggest an additional variable that m a y be brought into our theory. Had such Figure 8-7 Model of Diminishing Marginal Returns Figure 8-8 Example of Residual Analysis Source: V. note that the regression line d o e s n o t p r o v i d e perfect p r e d i c t i o n of the v a l u e s on the d e p e n d e n t variable for the c a s e s i n t h e s c a t t e r g r a m . 4 8 ) . p. S o m e of the observations are higher on the dependent variable than variable. we would expect from their value on the independent Something else. c a l c u l a t i n g t h e r e g r e s s i o n e q u a t i o n for t h e line t h a t b e s t s u m m a r i z e s t h e p a t t e r n o f the relationship. July 3 0 . so that it m i g h t n o t h a v e b e e n at all o b v i o u s that " d e m o c r a c y " w a s a variable you should use to explain levels of educational spending. 1 950). In this w a y you might have discovered the identity of the Percent for Sparkman. by 100r pointing out important independent variables that had not yet occurred to us. l o w e r i n g ) the d e p e n d e n t v a r i a b l e . is and s o m e are lower. L o o k i n g b a c k at Figure 8-3. w h i c h w o u l d h a v e b e e n p r e d i c t e d from the r e g r e s s i o n line. Knopf. t h e actual value for c a s e A is h i g h e r than the v a l u e for B . so he related c o u n t i e s ' votes for F o l s o m . also affecting the dependent variable. as Key had expected. I n F i g u r e 8-3. Southern Politics (New York: Alfred A. to their votes for Sparkman. C o n s i d e r Figure 8-3. a progressive candidate in the 1946 gubernatorial primary. and so on. In this sense. K e y . adapted f r o m V . a n d all o f t h e c o u n t r i e s for w h i c h i t w a s u n e x p e c t e d l y l o w were democracies. it can help refine our theory. On e x a m i n i n g the residuals in that figure. However. a c a s e s u c h as A is o n e in w h i c h the effect of t h e " s o m e t h i n g e l s e " is to raise the v a l u e of t h e d e p e n d e n t variable. Notice also . B e c a u s e this m e a n t that counties tended to lean the same way in both elections. Jr. 1 9 4 6 " s o m e t h i n g else. Key. 4 8 . N o w . Special Primary. o n c e the cases have been sorted out in this way.

or as a measure of the basic uncertainty of human affairs. The two extreme models are illustrated in Figure 8-10. Correlation Analysis At the beginning of this chapter I pointed out that there are two ways to measure the strength of a relationship: by measuring how much difference the independent variable makes in the dependent variable and by measuring how completely the independent variable determines the dependent variable. By examining the residuals around the regression line. in B than in A. which were causing additional variations in the vote. But the relationship was not very tight. In this case it turned out that the residuals could be explained in part by the effect of "friends and neighbors. Counties in Sparkman's home part of the state voted less enthusiastically for Folsom than would have been expected from their vote for Sparkman (which presumably was high because he was a local boy). in both graphs. the spread of dots around the regression line is treated as an act of God. compared to "other factors" (the unknown things that cause the residuals to exist). Key got some idea of what those variables might be. Both relationships can be summarized by the same regression line. In graph A of Figure 8-10. A change in the independent Figure 8-9 Two Correlations variable tends to produce the same change in the dependent variable. This indicated the presence of other variables. the residuals tend to be larger in graph B than in graph A. measures how widely such a body of data spreads around a regression line. The correlation coefficient. Similarly.122 Chapter 8 (A) Introduction to Statistics (B) 123 factions not existed. I suspect the reason most of us do not go on to examine this trove is that we have developed a proprietorial sense toward our theories before we ever get to the point of testing them. But this tendency is weaker. A regression line passed through them would leave no residual variation at Figure 8-10 Extreme Models of Correlation (A) (B) . Many counties. such as the one that gave 90 percent of its vote to Folsom but only 45 percent to Sparkman. the data all fall on a straight line through the scattergram. the regression coefficient accomplishes the first of these. In one sense. Consider the graphs in Figure 8-9. r. There is a certain completeness about one's own theory. and assigns to the relationship a score ranging in absolute value from 0 to 1. Usually. and more likely to be disturbed by "other factors. the correlation coefficient accomplishes the second. on the average." in graph B. depending on how closely the data approximate a perfect relationship. The dependent variable is less a result of the independent variable. it is a trove in which new variables lie waiting to be discovered. but the value of the dependent variable in graph B is less closely determined by the independent variable than in graph A. then. the "home" counties of Folsom's opponent went less heavily for Folsom than would have been expected on the basis of their vote for Sparkman. and it does not occur to us to use our theory as a "mere" starting point in the search for explanations. there would have been no particular reason to expect a county to vote in much the same way in the two primaries. in other words. For interval-scale data. one of the major forces retarding the development of stable statewide factions in Alabama politics at that time. This coefficient compares a set of data with ideal models of a perfect relationship and a perfect lack of relationship. Users of regression analysis in political science far too rarely go on to the creative and exploratory labor of examining the residuals to see what additional variables affect the dependent variable. the relationship in graph B is weaker than that in graph A." Counties in Folsom's home part of the state voted for him more enthusiastically than would have been expected on the basis of their vote for Sparkman. voted far differently from what one would have expected simply on the basis of conservative and progressive factions. This pointed out to Key the importance of local solidarity. Much of the looseness in the relationship between the votes for two candidates of the same faction was shown to be a result of people's tendency to vote for a candidate on the basis of where he came from in the state rather than the faction with which he was identified. On the contrary.

F o r i n s t a n c e . that t h e d i f f e r e n c e b e t w e e n r = 0. T h e c o r r e l a t i o n c o e f f i c i e n t f o r this t y p e h a s a n a b s o l u t e v a l u e o f 1 .1 . 6 is t h e s a m e as t h e d i f f e r e n c e b e t w e e n r = 0 . 0 . o r + 1 m e a n .89 T h e f o r m u l a for the variance of any variable x is E(x variance-. the c l o s e r a r e l a t i o n s h i p a p p r o a c h e s t h e s i t u a t i o n i n g r a p h A . . b u t m o r e o f t e n j u s t r ) d o e s h a v e a u s a b l e i n t e r p r e t a t i o n a t all v a l u e s o f r . In a positive relationship. the variance w i l l b e zero."AO + ( / ) + (7 ) 3 3 2 4 2 2 3 121 = — A + "79 + 7 . m a t h e m a t i c a l l y true: T h i s correlation coefficients o f ..0 . b e c a u s e i t i s the a v e r a g e squared d e v i a t i o n of the v a l u e s of y f r o m their o w n m e a n . 2 Variance. v a l u e s o f t h e d e p e n d e n t v a r i a b l e are c o m b i n e d r a n d o m l y w i t h v a l u e s o f t h e i n d e p e n d e n t v a r i a b l e . o n t h e o t h e r h a n d . O n e w a y t o v i e w o u r g o a l i n t h e o r e t i c a l s o c i a l s c i e n c e r e s e a r c h i s t o n o t e that our task is u s u a l l y to a c c o u n t for the variance in a d e p e n d e n t variable. a n d s o o n . t h e c l o s e r i t a p p r o a c h e s t h e m o d e l i n g r a p h A o f F i g u r e 8-10. 0 ( a p e r f e c t p o s i t i v e relationship.xf and then add t h e s e results together. similar to the o n e in graph A but tilted up). B u t i t i s n o t e a s y t o s e e w h a t t h e d i f f e r e n c e b e t w e e n t h e m m e a n s . T h e variance is a measure of h o w w i d e l y the observed values of a variable vary a m o n g t h e m s e l v e s . the square of the correlation coefficient ( s o m e t i m e s c a l l e d the coefficient of determination. Fortunately. in the correlational s e n s e o f " s t r e n g t h o f r e l a t i o n s h i p . It is the variance in s o m e t h i n g that p u z z l e s us and c h a l l e n g e s us to p r o d u c e an explanation. the further e a c h will b e f r o m the m e a n o f t h e m Interpreting the correlation coefficient. T h e e x p r e s s i o n 2 (x (x x xf is equivalent to writing the observation for the - xf + 2 (x 2 - xf + .124 Chapter 8 (-i - Introduction to Statistics 125 all i n t h e d e p e n d e n t v a r i a b l e . x t h e o b s e r v a t i o n for t h e s e c o n d c a s e . T h u s . T h e more the v a l u e s vary a m o n g t h e m s e l v e s . or average: the sum of all the values divided by the number of cases. T h i s i s r e m i n i s c e n t o f t h e d i f f e r e n c e b e t w e e n o r d i n a l a n d i n t e r v a l m e a s u r e m e n t : We k n o w that t h e h i g h e r the a b s o l u t e v a l u e of r. + (x N - xf. i f t h e r e are j u s t three c a s e s . w e c a n s a y w h i c h i s s t r o n g e r . A l t h o u g h it i s c l e a r e n o u g h w h a t all. T h e variance of the dependent variable y can be depicted in a scattergram by d r a w i n g a h o r i z o n t a l l i n e at y = y. T h e least-squares line is a line p a s s e d through the scattergram in any direction s u c h that t h e s q u a r e d d e v i a t i o n s o f v a l u e s o f t h e d e p e n d e n t v a r i a b l e f r o m that l i n e are m i n i m i z e d . T h u s there is no pattern to the relationship. where x x first c a s e . increases in the independent variable produce decreases in the d e p e n d e n t variable. T h e l i n e y = y is a horizontal l i n e p a s s e d t h r o u g h t h e d a t a in t h e s c a t t e r g r a m . T h e s i g n s i m p l y m e a n s that for all t h e o b s e r v a t i o n s . we are to c a l c u l a t e is (x . T h e coefficient ranges from . the m o r e that v a l u e s vary a m o n g t h e m s e l v e s . s o that a n y g i v e n v a l u e o f t h e dependent variable is as likely to coincide with a l o w value on the independent variable as w i t h a h i g h o n e . t h e h i g h e r its c o r r e l a t i o n coefficient will be in absolute value. 4. I m u s t first i n t r o d u c e t h e concept o f variance.3. I f t h e y d o n o t vary a t all but e a c h h a s t h e s a m e v a l u e . = xf N w h e r e x is the m e a n of x and N is the n u m b e r of observations we w i s h to average. the h i g h e r their variance will be. we s h o u l d s e e at l e a s t a s u p e r f i c i a l s i m i l a r i t y b e t w e e n the residuals around the least-squares line and the variance of the dependent variable.8 a n d r = 0 . T h u s the i n d e p e n d e n t v a r i a b l e d e t e r m i n e s t h e d e p e n d e n t v a r i a b l e c o m p l e t e l y .1 + 4 + 5)/3 = 8/3. fall s o m e w h e r e b e t w e e n t h e s e t w o e x t r e m e s . W h y i s i t that s o m e p e o p l e m a k e m o r e m o n e y than other p e o p l e ? T h a t s o m e nations are f r e q u e n t l y i n v o l v e d i n w a r s a n d o t h e r s are n o t ? T h a t s o m e m e m b e r s o f C o n g r e s s v o t e f o r a b i l l a n d o t h e r s o p p o s e it? T h a t s o m e p e o p l e are m o r e p o l i t i c a l l y a l i e n a t e d than others? A l l o f t h e s e q u e s t i o n s s i m p l y ask: T o w h a t c a n the v a r i a n c e i n this v a r i a b l e be a t t r i b u t e d — w h y does this variable vary? C o m p a r i n g F i g u r e s 8-3 a n d 8-11. a n d 5 for a v a r i a b l e . the stronger the relat i o n s h i p . I t i s true t h a t t h e h i g h e r t h e a b s o l u t e v a l u e o f t h e c o e f f i c i e n t . In addition to s h o w i n g h o w c l o s e l y a relationship approaches the situation in graph A. 4 a n d r = 0. h o w e v e r . T h e c o r r e l a t i o n coefficient for this t y p e has a b s o l u t e v a l u e of 0. increases in the dependent variable produce increases in the independent variable. I n g r a p h B . T h i s i s true b e c a u s e e a c h v a l u e w i l l e q u a l the m e a n . 0 (a perfect negative relationship. there i s n o direct w a y o f interpreting w h a t c o e f f i c i e n t s b e t w e e n t h e s e v a l u e s m e a n . Thus the correlation coefficient provides a m e a s u r e by w h i c h the strengths of various relationships can be c o m p a r e d . M o s t r e l a t i o n s h i p s . such as t h e o n e i n g r a p h A o f F i g u r e 8-10) t h r o u g h 0 . f o r i n s t a n c e . T h e a v e r a g e s q u a r e d v a l u e o f t h e r e s i d u a l s . B e f o r e w e c a n c o n s i d e r t h i s .. a n d d r a w i n g in the r e s i d u a l s f r o m this l i n e . T h i s indicates that t h e i n d e p e n d e n t v a r i a b l e h a s n o e f f e c t o n t h e d e p e n d e n t v a r i a b l e . b u t w e d o n o t k n o w how much s t r o n g e r o n e r e l a t i o n s h i p i s t h a n a n o t h e r . and thus the s u m of the squared deviations from the m e a n will equal zero.2. 0 ( g r a p h B ) t o + 1 . A n d it is a l s o n o t true that r = .1 . o f c o u r s e . T h e v a r i a n c e o f a variable i s t h e a v e r a g e s q u a r e d d e v i a t i o n o f v a l u e s o f that v a r i a b l e f r o m their o w n m e a n . in a negative relationship. the greater the s u m o f s q u a r e d d e v i a t i o n s f r o m the m e a n . I t i s not true. 2 y) + 3 2 (4 -y) + 3 2 (5 - 7) 3 2 3 ( . a n d i n s p e c t i o n w i l l s u g g e s t w h a t i s . is the v a r i a n c e of y. 6 is t w i c e as s t r o n g a s r = -0. and c o n s e q u e n t l y . s o that i f w e w i s h t o c o m p a r e t w o different r e l a t i o n s h i p s . their m e a n is (. w i t h s c o r e s o f . i n fact. " I h a v e referred here o n l y to the absolute v a l u e of the correlation coefficient. 3 4 9 = 6. t h e c o r r e l a t i o n c o e f f i c i e n t i n d i c a t e s b y its s i g n w h e t h e r t h e r e l a t i o n s h i p i s p o s i t i v e o r negative. a s i n F i g u r e 8-11. a n d their v a r i a n c e is 1 'The mean of any variable is its arithmetic mean.1 .

w h e r e a s the r e g r e s s i o n coefficient d o e s not. " I s c o r r e l a t i o n o r r e g r e s s i o n a n a l y s i s the better w a y to m e a s u r e the strength of a r e l a t i o n s h i p ? " O b v i o u s l y . I f w e n o w add k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e . the regression line w i l l be h o r i z o n t a l . To the extent that the dependent variable is determined by the independent variable. T h e m a g n i t u d e o f the m i s t a k e s i n e a c h c a s e i s n o w r e p r e s e n t e d b y s q u a r e d d e v i a t i o n s a r o u n d the p r e d i c t i o n s . T h e v a l u e Figure 8 . O p e r a t i n g w i t h o u t any k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e . . A t least this is true if we think of "mistakes" as squared deviations from the true value. p. 4 0 5 . L e t us see w h y this should be so. On the other h a n d . we w o u l d expect the regression coefficient to be about the s a m e in both graphs. T h i s similarity suggests that the squared deviations around the regression line m a y be treated as the variance of the dependent variable around the values predicted for it by the regression equation. T h e r e f o r e . : total variance ^ unexplained variance total v a r i a n c e 3 see Blalock (1979. T h a t i s . y o u s h o u l d mentally square the correlation a n d interpret that statement: " T h e two v a r i a b l e s are negatively related. pp. As already noted. r equals 1 m i n u s this proportion. 5 8 ) .1 2 . " S o m e t i m e s one i s . gives us a useful interpretation of r at all v a l u e s . our 2 ! ! ! y=y i i i i 1 b e s t strategy i n t r y i n g t o p r e d i c t v a l u e s o f the d e p e n d e n t v a r i a b l e for p a r t i c u l a r c a s e s w o u l d b e t o g u e s s that the v a l u e i n a n y g i v e n c a s e i s the m e a n .0 . s o m e t i m e s the other. the u n e x p l a i n e d v a r i a n c e w i l l be zero. the extent of its possible effects on the dependent variable are reduced. T h e e x p l a i n e d v a r i a n c e .1 0 ( B ) . line is the one horizontal line that m i n i m i z e s the squared deviations of values of the dependent variable f r o m itself (see B l a l o c k .126 Chapter 8 Introduction to Statistics 127 that i s . the regression coefficient in B will be approximately the same as that in A. If y o u read that an author h a s f o u n d a c o r r e l a tion o f . 3 0 b e t w e e n two v a r i a b l e s . any other horizontal line p a s s e d through the data w o u l d y i e l d a greater s u m of squared deviations in the dependent variable. T h e s c a t t e r g r a m f r o m g r a p h A h a s b e e n r e p r o d u c e d i n g r a p h B . because the variability of the independent variable has been reduced in graph B. 1979. this u n e x p l a i n e d v a r i a n c e w i l l be s m a l l c o m p a r e d to the total v a r i a n c e . indicating that the s a m e value of the dependent variable is predicted at all values of the i n d e pendent v a r i a b l e . If the dependent variable c a n be predicted perfectly f r o m the independent variable." A n o t h e r h e l p f u l w a y to l o o k at t h i s interpretation is to t h i n k in terms of p r e d i c t i o n . T h u s the u n e x p l a i n e d v a r i a n c e w i l l equal the total v a r i a n c e . the a n s w e r m u s t be.4 0 9 ) . T h e regression line that m i n i m i z e d squared deviations in the full set of data in graph A should continue to m i n i m i z e squared deviations in the partial set of data in graph B. this is the variance in the dependent variable that is still left unaccounted for after the effect of the independent variable has been estimated. including the proof that r = 1 unexplained variance . T h e effect is to r e d u c e the v a r i a b i l i t y of the i n d e p e n d e n t v a r i a b l e w h i l e l e a v i n g the b a s i c r e l a t i o n s h i p b e t w e e n the i n d e p e n d e n t a n d d e p e n d e n t v a r i a b l e s u n c h a n g e d . as in F i g u r e 8 . Under these circumstances. If the dependent variable is unrelated to the independent variable.1 1 The Variance of y r m e a s u r e s the p r o p o r t i o n b y w h i c h w e h a v e r e d u c e d o u r m i s t a k e s i n p r e d i c t i n g 2 the d e p e n d e n t v a r i a b l e b y i n t r o d u c i n g k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e . the proportion of the total v a r i a n c e in the dependent v a r i a b l e that c a n be y T T T 1 1 1 a s c r i b e d to the independent v a r i a b l e . but the correlation coefficient will be sharply lowered in B.1 0 ( A ) . W e w o u l d b e l e s s w r o n g m o r e o f the t i m e than w i t h a n y other g u e s s w e c o u l d m a k e . as in F i g u r e 8 . the regression line w i l l equal the line y = y in this c a s e ." T h e k e y t o d e c i d i n g w h e n to u s e e a c h m e a s u r e lies in the fact that the correlation c o e f f i c i e n t reflects the v a r i a b i l i t y of the independent variable directly. e x c e p t that a l l o b s e r v a t i o n s for w h i c h the i n d e p e n d e n t v a r i a b l e is l e s s than 2 or greater than 4 h a v e b e e n e l i m i n a t e d . therefore. D i v i d i n g the u n e x p l a i n e d variance by the total variance tells us w h a t proportion of the total v a r i a n c e is left after we have a l l o w e d the independent variable to explain as m u c h as it c a n e x p l a i n . C o n s i d e r the t w o s c a t t e r g r a m s i n F i g u r e 8 . T h u s we have two v a r i a n c e s for the dependent variable: its v a r i a n c e around its o w n m e a n (the "total v a r i a n c e " ) and its v a r i a n c e around the r e g r e s s i o n line ("varia n c e left u n e x p l a i n e d by the independent v a r i a b l e " ) . i n a s m u c h as the line y = y is the horizontal line that m i n i m i z e s squared deviations around itself. a n d 9 percent of the v a r i a n c e in one is due to the other. As it happens. o u r best g u e s s b e c o m e s the v a l u e 3 0 p r e d i c t e d f r o m the r e g r e s s i o n e q u a t i o n . or 2 Correlation and Regression Compared O u r d i s c u s s i o n s o far l e a v e s u s w i t h the q u e s t i o n . R e l a t i v e to other c a u s e s of 2 For a good presentation of this interpretation.

r e g a r d l e s s w a n t s o f a n d t o p e o p l e j u d g m e n t w o u l d t h e t r a c t a r e l o c a t e d h o w s t u d y . Stokes (1969) and Achen and Shively (1995). 8 0 t h o r t h e i n s t a n c e .I o f t h e no t a s t h e t h e y w e r e i n g r a p h v a r i a b l e t h a t i s u c A . or states) are used to infer how variables are related among individuals living in those aggregate units. pp. v a r y d i s t r i b u t i o n w h i l e a l m o s t t h e m o s t s u c h t h a t p e r c e n t b e h i g h e r ) r e l a t i o n s h i p b e t w e e n g r a d e s a m o n g g r e a t l y a n d p e r c e n t i n w o u l d m e n ( B l a c k s s p r e a d a r e o r c o n c e n t r a t e d s t u d e n t s c u r r e n t l y g o o d s e t r u l e o f d e p a r t m e n t . t h i s ei n p r o p o r t i o n e c l i n e t o t a l h e i n t h e rw a t t r i b c o r r e l a t i o n b y t h e c o e f f i c i e n t s p a r t i c u l a r i n t e n c ' s t r a c t g e n d e r t h e i n b o t h o f o f t h e t h e s e c a s e s i s t h a t e d . i n F o r w h i c h y o u m a y n o t i n t e n d m i g h t w o u l d t o w a n t b e g e n e r a l i z e . ( T h i s a d v i c e t h e r e i s s u p e r i o r c o r r e l a t i o n t h a t i n h o l d s f a c t . ) r e g r e s s i o n a n a l y s i s a l w a y s h a n d l i n g a r e p a t t e r n s b e s u c h T h e r e c i r c u m s t a n c e s . a n d v o t i n g p o s s i b i l i t y r e l a t i o n s h i p b e t w e e n r a c e r e c o r d f o r C o n g r e s s e s g e n e r a l . m o r e p a r t i c u l a r t h a t t h i s p e r c e n t m a l e s c a r c e l y f i n d i n t h e v a r i e s f r o m n o o n e c e n s u s t r a c t t o a n o t h e r B e c a u s e g u a r a n t e e s m a l e t h e r e p e r c e n t t h a t w o u l d a n d i s r e s e a r c h e r A s w o u l d p r a c t i c a l l y c o r r e l a t i o n s c a t t e r g r a m l i t t l e o f b e t w e e n i n t h e p e r c e n t a n a l y s i s p e r c e n t n e a r v o t i n g v o t i n g . This can occur when data on aggregate units (such as percent black. a p a r t i c u l a r s a y t h e h o w e v e r . ) a b s e n t o t h e r s . i n m o r e a n y s i t u a t i o n c a s e . i n d i c a t e d i n i t . i n that Congress. s o m e o n e I t m a j o r d e t e r m i n a n t T h e r e l i m i t e d u s e f u l n e s s a p a r t i c u l a r t h a t r a c e a n t f a i n a n d c t o f o a C o n g r e s s . o t h e r v a r i a n c e w e r e i n d i f f i c u l t y r e s i d e n t i a l d o e s . median income. a r e c e n s u s p e r c e n t t h e l a r g e c o r r e l a t i n g t h e p e r c e n t o f w i t h i n p e r c e n t w o u l d i n a l l T h e v o t i n g . for census tracts. m o r e o n a t o r e s e a r c h e r w i t h c i t y . w h i l e t h e d a t a a c o e f f i c i e n t a t m o r e a b o u t i n t o o n h a n d . 4 g e n e r a l s h o u l d r e g r e s s i o n c o e f f i c i e n t s .ri sr m u s t c a u s e p a r t i c i p a t i o n p r o b l e m w i t h i s t h a n u s i n g g e n d e r . g e n e r a l i z i n g . P r o b l e m s s u c h a s t h i s a r e a r e s u l t o f c e n s o r e d d a t a ( s e e a b o v e . the ecological fallacy. t h e t o a m e a s u r e o f T h e t h a t w o u l d h a v e s e x u s c o n c l u d e 5 f r o m t h i s t h a t r a c e i s a m o r e i m p o r t a n t i m p o r t a n c e d e c l i n e . r e s e a r c h e r t h e m a k e G R E m a l e a b o u t t r e a t t h e a p p l i c a n t s o f t h e ( a m o n g w h o m v a r i a b i l i t y G R E s c o r e s s c o r e s v a r i a b i l i t y a d o i n d e p e n d e n t m i g h t a b e v a r i a b l e s i n v e s t i g a t o r ' s s e x u s e a n d g e n d e r i n s t a n c e . m e s n e g l i g i b l e h e l p o w e v c o r r e l a t i o n w h a t o u l m e m i m h e t h e i r r s i n w o u l d H e s t a b l i s h e r . i f v e r y F i g u r e 8 . t h e i s d e p a w h i c h a r e j u s t a s f r e e a s a t o o p e r a t e c a u s e o f s . b e c a u s e i n t h e t o t h e i n v e s t i g a t o r c a s e . r e g r e s s i o n d e s p i t e s o m e t h e o r e t i c a l u s u a l l y w h i c h i n I w i l l r e s e a r c h b e a l m o s t t o a l w a y s i n t e r e s t e d a n a l y s i s . t h i s w This example also incorporates a major statistical problem in some correlation and regression analyses.128 Chapter 8 Introduction to Statistics 1 2 9 1 0 0 c o > c o (1) 0 Percent Male 1 0 0 Figure 8-13 P r e d i c t i n g P e r c e n t V o t i n g f r o m P e r c e n t M a l e t h e d e p e n d e n t o f v a r i a b l e . i s i n w h i c h y o u y o u w i s h u s e i n t o e x t r a p o l a t e f r o m a l e s s e v e n l y A s o m e t h e f a c t t h a t t r a c t s t r a c t s . B u t i n t e r e s t e d v o t e d .1 3 . u t a b l et B u t ot h i n d e p e n d e n t s i m p l y e n d e t o n s a y tv a o f v a r i a b l e t h a t r i a b t h e l ed d e p e n d e n t v a r i a n c e o r d s . o f I n t h e r a c e t h e c l e a r l y c e n s u s o r G R E s e x e x t r a p o l a t e t h e p a r t i c u l a r t o h o w a m a k e a c o n t r o l . o p e r a t e d . u n i t s . t o d e s c r i b e b u t h o w t o p e r c e n t s h o w c o u l d a m u c h p e r c e n t v o t e r b l a c k t u r n o u t . i t m i g h t i s t h a t g e n d e r o n l y t o d e s c r i b e s i t u a t i o n . d a t a g e n e r a l t h e t h e s t u d y . a s h y p o t h e t i c a l h e n c e h a n d . 103-106) . a f f e c t e d v a r i a n c e t o i n d e p e n d e n t f r o m t h e s e w a n t s o f v a r i a b l e e d T h i s g e n e r a l l y F o r h a d d a t a w i t h b l a c k h i g h l y a c r o s s b e c o m e s t h e m a t t e r o f c o n s i d e r a b l e i m p o r t a n c e i s i n T h e b e y o n d k n o w i n g f i e l d r e s e a r c h . and so on. b l a c k p e o p l e m a l e w h e t h e r m i g h t v o t i n g i s w h e t h e r p e r s o n r e s e a r c h e r p e r c e n t c i t i e s n o t . o n r e s e a r c h e r o r r a c e s t a t e m e n t i m p a c t c i t y . t h a t 9 2 n d . t h e z e r o c a n m a l e u p a s b e v a r i a n c e d u e t o v a r y p e r c e n t O n t h e a s o f m a l e . See Robinson (1950). n e w b a s i s i n i s d a t a t h e t h a t t o a I n v o t i n g . w a s T h i s a t h e n a p p r o p r i a t e b e t w e e n w e r e d n o i n t d e t h e n y t n o t e b e r s ' p o r t p a r t i c u l a r v o t e s f l u e n o n c i C o n g r e s s . countries. m e n t i o n e d v a r y i n g e a r l i e r . v a r i o u s n g o u t c o t h e r e b i l l s .

A good example of this technique is seen in Bartels (1993). it is possible to correct for it. Refer back to Table 4-1 (p. This should be clear from our earlier discussion of measurement error in Chapter 4. they will appear to us to be unrelated. the true form of the relationship between the variables is more and more lost to us. To the extent that there is random error in two variables. The relationship between the two variables is increasingly attenuated from left to right. and thus reconstruct the true relationship between two variables. As progressively greater amounts of random measurement error are present in two variables. I have purposely refrained from giving formulas for calculating these measures. a possibility that bodes ill for social science. 53). this article and most other discussions of measurement error are difficult for the untrained reader.130 Chapter 8 Introduction to Statistics 131 Problem of Measurement Error Our understanding of relationships between variables is intimately bound up with the extent to which we have been able to measure the variables validly. Figure 8-14 Relationship under Varying Degrees of R n o Measurement E a dm r Moderate Random Measurement Error No Random Measurement Error 5 10 15 Margin 20 25 5 10 15 Margin 20 25 Great Random Measurement Error 40 30 I 20 10 0 5 J 10 L 15 Margin 20 25 . (Unfortunately. The scattergrams for the three sets of data drawn from Table 4-1 are presented in Figure 8-14. One question you might consider is this: What would happen in a regression analysis if the independent variable did not vary at all—if. and Alissa Potter Mee's Statistics for Social Data Analysis (2002) is a good. we are simply unable to state accurately what the relationship between those variables is. say. Bohrnstedt. Not only is it lost to us. George W. but everyone in our study had the same amount of education? This question looks ahead to Chapter 10. Random measurement error also distorts the relationship between variables. but it is systematically distorted. Thus there is a real danger that the only relationships we will be able to perceive are those between variables that are easy to measure accurately. we wanted to relate voting turnout to education. David Knoke. standard text in statistics—one of many—for social scientists. as it might appear under conditions of increasing levels of random measurement error. If we are willing to make some assumptions regarding the nature of the random error. To the extent that there is nonrandom error in our variables.) F R H R DISCUSSION UTE In this chapter I have drawn only the broad outlines of correlation and regression analysis for interval data. so that anyone wishing to put these techniques to use would be forced to seek out more detailed training. In fact.

This is an interesting question. in economic crisis. Or there might be a positive relationship if both questions were tapping an "internationalist" dimension such that those favoring one sort of involvement would favor other sorts as well. Only for the second of these is it possible to develop a standard measure for use with ordinal-scale data. University of Michigan. in which voters' willingness to have the United States use military force to solve problems abroad is related to their willingness to have the United States give financial aid to countries 'This chapter presents advanced material that can be skipped over without any loss in comprehension of the remaining material covered in this book.S. Each number in the table gives the number of people in the 1998 study who exhibited a given combination of attitudes toward the two sorts of international intervention. a positive relationship would mean that relatively few cases should fall in the upper left corner of the table (not many cases willing to intervene militarily but unwilling to send aid) or the lower right corner (not many cases willing to send aid but unwilling to intervene militarily). of course. involvement abroad in one form opposed it in all other ways as well. To measure how much change is produced in the dependent variable. not percentages. National Election Study.Chapter 9 T A B L E 9-1 Introduction to Statistics 133 Relationship Between Support for Military Intervention and Support for E c o n o m i c Aid Willingness to Have United States G i v e Financial Aid to Countries in Crisis Willingness to Have United States U s e Military Force to Solve Problems F u r t h e r Topics on Measurement of Relationships Not Willing 18 47 120 48 Somewhat Willing 66 152 355 119 Very Willing 33 75 112 38 Extremely Willing 29 14 34 5 Extremely w i l l i n g Very w i l l i n g Somewhat willing Not willing Source: 1998 Congressional Election Survey. But this is not available under ordinal measurement. the table shows how frequently the various combinations of the two variables occur. or we can measure how strongly the dependent variable is determined by the independent variable relative to other things that also help determine it. in which voters who favored the use of military force did not want to see more "soft" involvement. imply an even spread across the table. with willingness to use military force increasing somewhat as voters are more willing to give financial aid. Like a scattergram. For instance. and voters who favored helping those in economic crises did not want to see other sorts of involvement. A negative relationship would imply the opposite: relatively few cases in the lower left or upper right. and (3) those that involve several variables simultaneously. only 18 are extremely willing to have 132 . And just as in a scattergram that shows the relationship between two intervally measured variables. A cross-tabulation is to the analysis of ordinal variables what a scattergram is to the analysis of interval variables. In general. And no relationship at all would. while of the 233 who are not willing to give aid. In tins chapter I build on the introduction in Chapter 8 to discuss in brief how to measure three types of relationships: (1) those among ordinal and nominal variables(2) those that involve dichotomous variables. Consider Table 9-1. There might be a negative relationship. These are actual numbers of cases. 1 MEASURES OF RELATIONSHIP FOR ORDINAL DATA As we saw at the beginning of Chapter 8. the most we can hope for from ordinal variables is some form of correlational analysis. just as a scattergram does for interval-scale data. We can measure how much change is produced in the dependent variable by a change in the independent variable. The table reveals a pattern between these variables. 29 are extremely willing to have the United States use military force. we can say that there is a modest positive relationship in the table. Accordingly. of the 82 voters who are extremely willing to give financial aid. there are two main ways in which we can measure the strength of a relationship. it is necessary to have some sort of unit by which to measure that change. and those opposing U.

• ' ••• •. and often t w o rather different patterns will p r o d u c e the s a m e result. Because there are no units by which to measure the difference between two values of a variable. 2 I t t a k e s a s its m o d e l o f p e r f e c t i n d e p e n d e n c e a t a b l e i n w h i c h t h e d a t a are distributed u n i f o r m l y t h r o u g h o u t t h e t a b l e . or vice versa? 3 This measure was described in Goodman and Kruskal (1954). Because there is a limited number of values available for each variable (four for the independent variable and four for the dependent variable). For instance. and vice versa. as w a s d o n e w i t h the correlation coefficient.•* • ••• : « £ S 3 K .»:^v. F r o m a m e a s u r e b a s e d on cell-by-cell c o m p a r i s o n s like this. the similarity to a scattergram becomes more obvious. a n d also develop a measure to indicate h o w close the data in our table c o m e to these two m o d e l s . A m e a s u r e like G o o d m a n .••• *•*»*»•••• • . Is there anything special. T h e n u m b e r s in Table 9-2 are printed in italics to e m p h a s i z e that they are b a s e .134 Chapter 9 T A B L E 9-2 Independent Version of Table 9-1 Introduction to Statistics 135 the United States use military force. a n d t h e r e a r e m o r e i n t h e l o w e r left ( 4 8 v s .) Even though there is not a precise regression line to define residuals. Instead. which notes a w a y to incorporate ordinal measures as independent variables in combination with interval variables.vv. . ••"»••• ' *••*•• • • * • * • • • .•••*••• • •. On the other hand.*%•:•*. however.. 14) t h a n o n e w o u l d e x p e c t i f t h e r e w e r e n o r e l a t i o n s h i p . U n d e r such "perfect i n d e p e n d e n c e . . • • . b u t t h i s i s t h e general principle of h o w they work. as in Figure 9-1. •**••*••>* • *••*****•• « * !!•••„••** • • • • # • • * • * # . . .*• ••••». • • • • •. *•.. (See also 111 footnote 4 on p a g e 138.K r u s k a l ' s w i l l u s e all c e l l s o f t h e t a b l e . There are t w o differences b e t w e e n a table such as this a n d a scattergram. t h e s a m e p e r c e n t o f e a c h category of the dependent variable occurs at each category of the independent variable.. . T h e result is useful. 39) a n d u p p e r r i g h t ( 2 9 v s . the observations are located in one or another of the cells of a table.'S3* * * . "Very willing" might represent only slightly less support than "Extremely willing. B u t n o n e o f t h e m e a s u r e s that h a v e b e e n d e v e l o p e d i s all that satisfactory.*•.:. •. their presence is indicated by counting the number of observations falling in each cell and printing the number there rather than by printing a dot for each observation..r i g h t c o r n e r ( 5 v s . . 9 ) t h a n e x p e c t e d . however: Willingness to Have United States G i v e Financial Aid to Countries in Crisis 1.*.*. that is.:*: distributions in Tables 9-1 a n d 9-2 to see h o w different they are in a positive or negative direction." O n e s u c h m e a s u r e for u s e w i t h ordinal variables i s G o o d m a n a n d K r u s k a l ' s gamma. 27) and l o w e r .l i n e c o u n t e r p a r t s for c o m p a r i s o n w i t h the o b s e r v e d n u m b e r s in Table 9-1.:.. . . How great a change there is in attitudes as we move from "Very willing" to "Extremely willing" tells us nothing specific. If the observations are indicated by dots.•• .: • • • •• • . *lw.. * * • . it is possible to develop measures analogous to the correlation coefficient. •. 2. it is not necessary to use the infinite space of a plane to locate the observations. T h i s indicates a pattern consistent with a mildly positive relationship. • : • • •. • • ••*". . for instance. because for all we know. • '•*•. it might still be useful to look at the "residuals" in Table 9-1—the cases that do not fit the generally positive relationship. " the data in Table 9-1 w o u l d appear as in Table 9-2.:'. the precise pattern in the table is meaningless.. Figure 9-1 Scattergram Version of Table 9-1 Measures like the Goodman-Kruskal gamma are designed to compare the * •*•****• * * • • • • • * * * • * \ * • !•*•*.. we can develop a m e a s u r e o f c o r r e l a t i o n .••*•*••*"•. . •• •••••• .**.-•* • % : •:•.. however. but a bit a m b i g u o u s . T h i s is simply the price we p a y for the fact that ordinal m e a s u r e s are naturally less informative than interval m e a s u r e s . n o t j u s t t h e c o r n e r s . 3 Willingness to Have United States Use Military Force to Solve Problems Extremely willing Very willing Somewhat willing Not willing Not Willing 27 53 114 39 Somewhat Willing 80 157 340 115 Very Willing 30 59 127 42 Extremely Willing 9 19 40 14 T h u s it is not possible to m e a s u r e the strength of relationships in a w a y analogous to regression. about those who buck the general trend by favoring the use of military force but opposing financial aid. w e m u s t d e v e l o p m o d e l s o f t h e perfect relationship a n d of perfect i n d e p e n d e n c e .*. T o d o this. there are fewer cases in the upper-left corner (8 vs. Because so many observations are tied on each value. • «• • ..

they can quite properly be treated as interval scales and can be analyzed using measures appropriate for such scales. just as has been done with ordinal variables. one of which is in A and the other in B.000 . we might regress income at a low-paying factory on gender. 0 30 0 Other 0 26 Preferred X Y Z Prot. the decision of whether A is one unit greater than B. Since a regression line serves as a convenient way of summarizing the central pattern in a set of means." A scattergram that would be consistent with these hypothetical regression results is shown in Figure 9-2. But the problem is really better dealt with by adaptations of regression analysis to nominal variable analysis. referendum vote (yes or no). the position of a cell in the table no longer tells us anything. Dichotomies and Regression Analysis Dichotomies are classifications that involve only two categories: for example. for obvious reasons." As you will see in footnote 4. with gender equal to 0 for men and 1 for women. but also dichotomous nominal scale variables—and mixtures of the two—can be analyzed using regression analysis. the dichotomy "gender: male or female" can also be viewed as a variable "femaleness" (or alternatively. it shows which values of one variable tend to coincide with which values of the other. is one unit. Let us call the categories A and B. The category is either present or not present for each observation.0 for the variable "femaleness" and men have values of 0. (Exactly the same result could be presented. the scattergram for this analysis will look odd.000. Similarly. Regression adaptations for analyzing nominal variables are dealt with in the next two sections. while the average income of women is $12. with a set of data arrayed vertically above "gender = 0" and another set arrayed vertically above "gender = 1.2. in regression and correlation analysis.000 x 1) = $12. There is a well-developed technique described in the section "Multivariate Analysis" T A B L E 9-3 Interchangeable O r d e r in Nominal Data (A) Religion (B) Religion Candidate Candidate Preferred X z Y Cath. it can be used together with other interval measures.000. An alternative way to look at this situation is to think of a dichotomy as measuring the degree to which an observation embodies one of its categories. 20 5 0 Prot.000 gender This would mean that the expected income of men at the factory was $14. The table could not be rearranged in this way if the variables were measured ordinally.) For example. must be arbitrary.000. Such classifications have a useful property. As with ordinally measured data. This greatly expands the value and applicability of regression analysis. We would have no common unit. For instance. Because there usually is no natural ordering to the dichotomy.000 . The trick is that because a dichotomous variable can take on only one of two values. and the distance between two observations.) Where this technique of treating dichotomous variables does become valuable is when we wish to combine a dichotomous variable with other independent variables in an analysis. 20 0 5 Other 0 26 4 4 . Various measures have been developed to do this.000. or one unit less. in contrast to a table relating ordinal variables. in which we are interested in the relationship between an interval-scaled variable and a single dichotomous variable. Since the independent variable can take on only two values. gender (male or female)." If the difference between totally female and totally not female is taken to be one unit. there is yet another trick we can use in this case that will allow us to work with multiple-category variables. This would be the case if we wished to look at the simultaneous effects of sex and education on income. however.136 Chapter 9 Introduction to Statistics 137 M A U E O RELATIONSHIP F R NOMINAL D T ESRS F O AA In relating two nominally measured variables. Consider the example in Table 9-3. for the order of values of a nominal variable means nothing. 0 0 30 Cath. which at first glance appears to be a bit mysterious: Although dichotomies are nominal scales. namely the difference between the two categories. for example. The best-fitting regression line might be income = 14. The regression line will actually pass directly through the mean income of men at gender = 0 and the mean income of women at gender = 1. such as income or age. Because this is a true interval measure.0. or participation (voter or nonvoter).($2. in which versions A and B are equivalent. in which to place Catholics relative to Protestants or Jews in the variable "religion. women have values of +1. the expected income for women would be $14. Each subject is either totally "female" or totally "not female. However. more simply. by noting that the average income of men at the factory is $14. there is no particular advantage to using regression analysis in cases like these. there is by definition a common unit in which differences between values of the variable can be measured. It means that not only intervally measured variables. since the value of "gender" for men is zero. Then the distance between two observations falling into category A is zero units. all we can do in measuring the strength of a relationship between two nominally measured variables is to assess the degree to which one variable is determined by the other. (The same interpretation cannot be placed on nominal-scale variables with more than two categories. "maleness"). cross-tabulation again approximates the function of a scattergram—that is.

however. can be interpreted as the path of expected values of the dependent variable across varying values of the independent variable.000 10. It would not make sense in Figure 9-3. there is a perfect relationship here. but the general explanation holds for both. this works only for dichotomous nominal measures used as independent variables.000 16." They are variants of the logit and probit analysis presented in the next section. The regression line.000 14.000 0 £ 8 c 12. but there is no such technique for nominal measures. as you will recall. and so on. that is the natural logarithm. . P(y= 1). The techniques involve "ordered logit" and "ordered probit. cannot be stated too strongly.7.0 to y = 1 at the point at which x equals 10 and thus cannot register the full extent to which values of y are determined by x. Above levels of x = 10.000 - 6. it cannot follow the jump from y . Since the regression line cannot bend. From the regression line in Figure 8-4. However. but presentation of these techniques goes beyond the scope of this book. But there is still "error" around the regression line. Consider the scattergram in Figure 9-3. would be the probability that the dependent variable equals 1. the logit is the logarithm to base e. about 70 percent of the cases would have y = 1. Such a model would portray. that at x = 5. Such variables generally are known as dummy variables or binary variables. Figure 9-4 provides an example. to predict that the dependent variable would have a value of 0. statisticians have developed an alternative model that is found in two very similar variations. but the logit model allows us to derive estimates of the sort such a model would yield.000 8. It would need to be S-shaped and bounded vertically by 0 and 1. Letting p = P (y = 1). In any reasonable sense. since a probability must be positive and less than or equal to 1. I describe logit analysis here. The ability to include dichotomous nominal variables in such analyses is extraordinarily valuable. for instance. that y would equal 18 when x equaled 4. An analogy to the expected value of a dependent variable. since it can take on values of only 0 or 1. for instance.138 Chapter 9 Introduction to Statistics 139 $18. and when x is less than 10. the probability that y equals 1. It is therefore incapable of serving as the "best summarizing line" to describe the relationship between x and y. The model would predict. we would predict that y would equal 9 when x equaled 1. called logit analysis and probit analysis. The value of this trick. 141-146 that allows us to use interval-level measures simultaneously in this way. That is.000 Gender Figure 9-2 Hypothetical Regression of Income on Gender LOGIT AND PROBIT ANALYSIS To handle the problem of describing the relationship between a continuous independent variable and a dichotomous dependent variable. One indication that this would be problematic is that it would be impossible to construct a meaningful correlation coefficient for such an analysis. of on pp. We do not have any way to work directly with such a model.3 when the independent variable equaled 8. is about 0. this would be nonsensical. y is always 0. across the variation in an independent variable. for example. the varying probability that a dichotomous variable equals 1. We cannot appropriately use them as dependent variables in ordinary regression. y is always 1. 4 Techniques have also been developed for including ordinal variables in regression. whereby we can include nominal scale dichotomies in the powerful technique of regression analysis. This does not make sense for a dichotomous variable.

T h e r e l a t i o n s h i p b e t w e e n t h e o t h e r t w o variables i s t h e n m e a s u r e d for e a c h group.p)) i s a t e a c h o f s e v e r a l v a l u e s o f x . either d i r e c t l y i f t h e y are i n d e p e n d e n t v a r i a b l e s . i t i s s t r a i g h t f o r w a r d t o s o l v e f o r p ( t h e p r o b a b i l i t y t h a t t h e d e p e n d e n t v a r i a b l e i s 1 rather t h a n 0 ) .0 25 35 45 55 65 75 . T h i s p r o b l e m c a n be averted by l u m p i n g subjects into a f e w broad categories of the independent variable. T h i s i s d o n e b y first c a l c u l a t i n g f r o m a a n d b w h a t t h e e x p e c t e d . we m i g h t perform a logit analysis and then present the results: .70 0. the l o g i t m o d e l a l l o w s u s t o use regular regression analysis to analyze a d i c h o t o m o u s dependent variable." a s i n t h e e x a m p l e o n p p . o r e v e n t h e p r o b a b i l i t y o f y . b y separating the subjects into g r o u p s a l o n g t h e control variable ( a g r o u p o f D e m o c r a t s . i t i s n o t y . a g r o u p o f R e p u b l i c a n s . R e m e m b e r . I n C h a p t e r 6 w e s a w that f r e q u e n t l y i t i s i m p o r t a n t t o h o l d o n e v a r i a b l e c o n s t a n t w h i l e m e a s u r i n g the r e l a t i o n s h i p b e t w e e n t w o other variables. a n d t h e w a y i t i s u s u a l ly presented in studies. the a and b in t h e l o g i t e q u a t i o n are n o t i n t u i t i v e l y i n t e r p r e t a b l e . w e m i g h t h a v e a g r o u p o f all t h o s e w i t h a n i n c o m e o f $ 1 8 . T h e b e s t w a y t o i n t e r p r e t l o g i t a n a l y s i s . T h e ability to incorporate d i c h o t o m o u s variables into regression analysis. i t i s t h e n a t u r a l l o g a r i t h m o f p / ( l .p). in the standard form bx h e l d c o n s t a n t ) .68 0. a n d s o o n . t h e w a y t h i s w o r k s i s that t h e d e p e n d e n t v a r i a b l e h a s v a l u e s o f e i t h e r 0 or 1. i f w e w a n t t o control for an intervally m e a s u r e d variable. in this c a s e there probably w o u l d be as m a n y groups as there w e r e subjects in the study. F o r i n s t a n c e . that i s p r e d i c t e d f r o m x i n t h i s e q u a t i o n . T h i s i s a d i f f i c u l t q u a n t i t y t o c o n c e i v e ! B u t i t d o e s e m b o d y t h e p r o b a bility i n w h i c h w e are interested. o r t h r o u g h t h e l o g i t a n d p r o b i t v a r i a n t s o f regression analysis if the d i c h o t o m y is a d e p e n d e n t variable. i f w e w e r e i n t e r e s t e d in the effect of age on voting participation. 9 4 .9 5 . T h e n . T h e e a s i e s t w a y t o d o this i s literally t o " h o l d t h e variable constant.71 141 Age 1. 4 0 2 . T h e c o m p u t e r calculates f r o m the n u m b e r of 0s and 1 s at g i v e n values of the i n d e p e n d e n t variable w h a t the e x p e c t e d logit of the d e p e n d e n t variable is across the range of the independent variable. rather. i n c o n t r o l l i n g f o r i n c o m e . the n u m b e r of "groups" we set up m i g h t be too l a r g e . 2 4 3 . is to calculate by a series of transformations on the logit coefficients what the e x p e c t e d probability of y is at various values of x and to present t h o s e d i s c r e t e l y . T h i s t e c h n i q u e h a s s e r i o u s d i s a d v a n t a g e s . a n o t h e r o f all t h o s e w i t h a n i n c o m e o f $ 4 3 .58 0. F o r i n s t a n c e . Figure 9-4 Path-of-Probabilities M o d e l MULTIVARIATE ANALYSIS p / ( l — p). but d o i n g so throws a w a y a great deal of the precision in the m e a s u r e — s o m e t h i n g o n e m a y not w a n t to do.140 Chapter 9 Introduction to Statistics Estimated Probability of Voting 0. i f registration i s t h e variable t o b e logit of y It turns o u t that the l o g i t c a n appropriately be taken as a linear f u n c t i o n of a c o n tinuous independent variable. First. B e c a u s e the analysis is so indirect. t h o u g h . has m a d e regression analysis a p o w e r f u l statistical tool and the d o m i n a n t tool for data a n a l y s i s in political science. a n d a g r o u p o f i n d e p e n d e n t s .37 0. f r o m t h e n a t u r a l l o g a r i t h m o f p / ( l -p). and m o s t important.2 - A table like this approximates the results o n e m i g h t have gotten f r o m a nonline a r r e g r e s s i o n l i n e . A s e c o n d a n d m o r e s e r i o u s p r o b l e m i n literally h o l d i n g a v a r i a b l e c o n s t a n t i s that the process quite rapidly leaves the researcher w i t h o n l y small numbers of subjects M e c h a n i c a l l y .6 l o g i t ( t h e n a t u r a l l o g a r i t h m o f p / ( l . for e x a m p l e .71 0. T h e r e are a l s o v a r i o u s a n a l o g s t o t h e c o r r e l a t i o n c o e f f i c i e n t f o r logit a n a l y s e s but n o n e that is readily p r e s e n t e d at this l e v e l .

x. and then look at the occupation/vote relationship for e a c h of these groups. respectively. this might leave us with as m a n y as 20 or 30 separate measures to consider. E a c h o b s e r v a t i o n is a dot floating in t h r e e . as noted above. given these particular values of w and x. whites aged 2 0 ." To s k e t c h the t e c h n i q u e in a general w a y . To this sum. T h e t e c h n i q u e of multivariate r e g r e s s i o n a l l o w s us to l o o k at the pattern i n s e v e r a l v a r i a b l e s a m o n g a l l o f our o b s e r v a t i o n s ( w i t h o u t b r e a k i n g these d o w n into separate subgroups) and to estimate w h a t the r e l a t i o n s h i p b e t w e e n the T h i s regression plane w o u l d look l i k e a flat p i e c e of c a r d b o a r d set at s o m e tilt in the m i d d l e of the t h r e e . S u c h s m a l l groups w o u l d give very unreliable estimates of the relationship between occupation and vote. F o r t u n a t e l y . now have the expected value of y. a s seen i n F i g u r e 9 . You 'Refer again to the box "Law of Large Numbers" on p. a n d b tells by h o w m a n y units y 2 c a n be expected to i n c r e a s e if x i n c r e a s e s by one unit and w does not change. 3. the r e l a t i o n s h i p a m o n g three v a r i a b l e s c a n b e plotted i n a t h r e e . q. I t m u s t b e the p l a n e that m i n i m i z e s the s u m of s q u a r e d d e v i a t i o n s of a c t u a l y v a l u e s f r o m the y v a l u e s p r e d i c t e d f r o m the p l a n e . and so on. or the amount by which y could be expected to change if the value of w shifted from zero to the particular value you are using.3 0 . if we wanted to control simultaneously for m o r e than one variable. S i m i l a r l y .v a r i a b l e c a s e . Y o u w i l l r e c a l l that the r e l a t i o n s h i p b e t w e e n two v a r i a b l e s c a n be plotted in a flat s p a c e of two d i m e n s i o n s ( r e p r e s e n t e d by a s c a t t e r g r a m ) . At this point. b tells by h o w m a n y units y c a n be e x p e c t e d to i n c r e a s e x if w i n c r e a s e s by one unit and x does not change.d i m e n s i o n a l s c a t t e r g r a m c o u l d s u m m a r i z e the pattern in a t w o . J u s t as a o n e . a n d e a c h o f the h o r i z o n t a l d i m e n s i o n s is one of the i n d e p e n d e n t v a r i a b l e s . E s p e c i a l l y . might include no more than five or six whites aged 2 0 . add b times x. holding constant race and age. By c o u n t i n g out a certain distance along x and then a certain distance along w.s c a l e v a r i a b l e s ( i n c l u d i n g d i c h o t o m i e s that are taken as i n d e p e n d e n t v a r i a b l e s ) . T h e height of the c a r d b o a r d plane above the grid at this point is the value of y predicted f r o m the g i v e n v a l u e s of w a n d x by the regression p l a n e . H e r e fe. Start with a. Add to this b\ times w. any multicategory n o m i n a l variables included in the a n a l y s i s w i l l show up as a set of several dichotomous variables. I w i l l u s e as an e x a m p l e the c a s e of t w o i n d e p e n d e n t v a r i a b l e s (w and x) and a dependent v a r i a b l e ( y ) .d i m e n s i o n a l s p a c e . 5 A final disadvantage of literal controlling is that it produces a series of measures of the relationship (one measure for e a c h group) that is u n w i e l d y and difficult to absorb. T h e r e g r e s s i o n equation a s s o c i a t e d w i t h the plane is of the f o r m Figure 9-5 Three-Dimensional Scattergram dependent v a r i a b l e a n d any p a r t i c u l a r i n d e p e n d e n t v a r i a b l e w o u l d b e i f n o n e o f the other v a r i a b l e s v a r i e d — t h a t i s . Particularly. you have the expected value of y when x equals zero and w equals the particular value. 6 1 . we c a n locate a particular c o m b i n a t i o n of w and x on this grid. so a t w o . .d i m e n s i o n a l l i n e t h r o u g h the t w o . with about 2.v a r i a b l e r e l a t i o n s h i p . or whatever. B u t the typical national survey. S i m i l a r l y .d i m e n s i o n a l scattergram c a n s u m m a r i z e the pattern in its t h r e e .v a r i a b l e r e l a t i o n s h i p . i f e a c h o f the other v a r i a b l e s w e r e " h e l d constant. y = a + b\w + b x 2 As in the t w o . b l a c k s aged 31—40.3 0 . T h e v e r t i c a l d i m e n s i o n i s the dependent v a r i a b l e . 2. the expected value of y when both x and w equal zero.142 Chapter 9 Introduction to Statistics 143 among w h o m to measure a relationship. a n d y. w h e r e these v a l u e s are p. D o t A h a s h a d c o o r d i n a t e s d r a w n to s h o w h o w it is l o c a t e d in the threed i m e n s i o n a l s p a c e by its v a l u e s on w. Suppose that someone wanted to examine the relationship between occupation and voting. l o c a t e d a c c o r d i n g to its v a l u e s on e a c h of the three v a r i a b l e s .3 0 . as in our example of race and age.6 0 . the amount by which y could be expected to change if x 2 shifted from zero to the particular value you are using and w remained unchanged. and r. there is an e a s i e r w a y to cont rol for a v a r i a b l e — a t least if we are u s i n g i n t e r v a l . T h e plane i s p i c k e d o n the b a s i s o f the s a m e c r i t e r i o n b y w h i c h the r e g r e s s i o n line w a s c h o s e n . blacks aged 5 1 .5 . a is the i n t e r c e p t — t h e predicted v a l u e of y w h e n both x and w equal zero. and b are the r e g r e s s i o n coefficients of the equation.d i m e n s i o n a l s p a c e over a g r i d c o n t a i n i n g all p o s s i b l e c o m b i n a t i o n s of v a l u e s of w and x.d i m e n s i o n a l p l a n e through the t h r e e .d i m e n s i o n a l s p a c e . 2 To c a l c u l a t e f r o m the multivariate r e g r e s s i o n the y v a l u e y o u w o u l d expect f r o m a particular c o m b i n a t i o n of w and x: 1.000 respondents. T h e investigator might divide the population into groups s u c h as blacks aged 2 0 .

8 vary. without requiring us to break our observations into separate s m a l l and a w k w a r d groups. esp. N e w m e m b e r s . 6 / 1 w o u l d indicate that this is true. 136). reaching its apex among those with the highest sophistication score.c u t party d i f f e r e n c e s ) . In this 2 w a y . T h e percentage o f A f r i c a n A m e r i c a n s attending a l l . F o r e x a m p l e . 'Although I have used the case of three variables in this section because that case can be represented by perspective drawings. Because such interaction occurs frequently in the social sciences.1 percent c a n be expected f r o m an i n c r e a s e of 1. we could not have calculated the expected value of y simply by adding a plus b\ times w plus b times x.b l a c k schools. N o w . we had to assume that an increase of one unit in x produced the same change in y.000 d o l l a r s . There is also an additional requirement in multiple linear regression. S . however. In the m u l t i p l e regression equation.2 . N o m i n a l variables m a y be included in T h i s w o u l d indicate that s c h o o l integration i s m o r e w i d e s p r e a d i n r i c h e r counties. As the score on ideological sophistication increased. x. the m e d i a n i n c o m e o f e a c h county. in itself it is an interesting phenomenon. It is exciting to think that a relationship itself may depend on a further variable.h e e l e d counties is s i m p l y a result of the fact that in the richer counties there are so few A f r i c a n A m e r i c a n s that there are not l i k e l y to be any a l l . The same logic would apply to a four-variable case. B e c a u s e there is no satisfactory analog to this 7 This little analysis is wholly fictitious. C o n s i d e r the relationship b e t w e e n party affiliation and h o w m e m b e r s of C o n g r e s s vote on a b i l l . Otherwise. and y. it is important that you be alert to it in your research. As in simple linear regression. interaction is almost always unexpected. O b v i o u s l y . O t h e r w i s e . a relationship appeared. a d e c r e a s e of only 0. It is important that the data best fit a model in w h i c h the relationship between the dependent variable and each independent variable is the same no matter what value the other independent variable(s) takes on. 8 . the T h i s freedom f r o m the need to "hold variables constant" physically is an important advantage of multivariate regression analysis. Among those who scored lowest in ideological sophistication.1 M technique for use with ordinal data.0 . Because "common sense" rarely gets so complicated as to suggest this sort of thing. Philip Converse was able to argue very neatly for the importance of a measure of ideological sophistication by showing that the relationship between social class and voting was strongly affected by it (1964. counties g i v i n g the percentages o f A f r i c a n A m e r i c a n students attending a l l . { 2 } 6 strength of the relationship might be related to a m e m b e r ' s seniority. and discovering it is fun and dramatic. we m i g h t find that the r e g r e s s i o n equation w a s S = 68. there was no relationship between social class and voting.1 4 4 Chapter 9 N o t i c e in this that b d e s c r i b e s the relationship b e t w e e n w and y if x does not { Introduction to Statistics 1 4 5 Interaction in Research and Theory Although interaction among variables is a nuisance from the standpoint of multivariate regression analysis. We w o u l d have had to pick a value for b\ that w a s 2 appropriate for whatever value of x we were using. multivariate r e g r e s s i o n a l l o w s us to e x a m i n e relationships w i t h certain other v a r i a b l e s controlled. might vote obediently along This example is a particularly interesting one because Converse demonstrated yet a second level of interaction (the interaction itself varies with gender) that I have chosen to ignore here. We c o u l d test for this by adding the variable "percent A f r i c a n American" as an additional multiple regression equations by treating them as dichotomies (refer to p.b l a c k s c h o o l s . W i t h the cont rol for "percent A f r i c a n A m e r i c a n " added. On e x a m i n i n g j u s t the relationship b e t w e e n S and M.A m e r i c a n s in the c o u n t y ' s population. T h i s is especially true w h e n working with several variables simultaneously.b l a c k s c h o o l s . it is important in multiple linear regression that the data actually fit a m o d e l in w h i c h all the relationships between the dependent variable and the independent variables are linear. L e t M be the c o u n t y ' s m e d i a n i n c o m e (in thousands of d o l l a r s ) . no matter what the value of w independent variable. all R e p u b l i c a n s the other). 1 M + 0 . we need to control for percent A f r i c a n A m e r i c a n in a s s e s s i n g this relationship. T h e regression equation S = 49. using county data for southern states in the 1950s. and a value of b that w a s appropriate 2 for whatever value of w we were using. T h i s relationship c o u l d be strong (all D e m o c r a t s vote one w a y .000 dollars i n m e d i a n i n c o m e . w h o are rather dependent on party l e a d e r s h i p for f a v o r s .b l a c k s c h o o l s d e c r e a s e s b y 2. W h e n this h a p p e n s — w h e n the relationship b e t w e e n v a r i a b l e s A and B differs. pp. It might be that the apparent integration of the w e l l . the regression plane described by the equation we c o m e up with w i l l be an inaccurate representation of the relationships. see Matthews and Prothro (1963). the relationship b e t w e e n m e d i a n i n c o m e and percentage o f A f r i c a n A m e r i c a n students attending a l l . the general multivariate regression technique applies to any number of variables.2 3 4 ) .2. It is obvious that this w o u l d be a complicated procedure. and that b d e s c r i b e s the relationship b e t w e e n x and y if w does not vary. and the percentage o f A f r i c a n A m e r i c a n s i n e a c h c o u n t y ' s population.5 . L e t A be the percentage of A f r i c a n . w e might find the f o l l o w i n g situation: L e t S be the percentage of A f r i c a n A m e r i c a n students attending a l l . it is often impossible to operate under these conditions. i f w e had data for U .b l a c k s c h o o l s largely disappears. A c o n c r e t e e x a m p l e m a y m a k e m o r e c l e a r what interaction is and h o w it might arise. although no scattergram can be drawn in these four dimensions. depending on the v a l u e of a variable C—there is s a i d to be " i n t e r a c t i o n " a m o n g the variables.1 percent as the m e d i a n i n c o m e of the county i n c r e a s e s by 1. In our three-variable example of w. As one example. or w e a k (less c l e a r . In fact. however. 6 w a s . we c a n add this to our list of arguments for trying to measure variables intervally as often as possible. 2 3 1 . For a multiple regression analysis of black voting registration. with the equation y = a + b w + b x + b z.

52* 0.44 0. it is your responsibility to delve a bit deeper and see just what is being screened out. Appendix A. noting which observations are mavericks in the relationship." How much "vote" changed with a given change in "party" would depend on the value of "seniority.53 0. For instance. but you must let them speak softly. See. Under these circumstances.45 0. It would not be true that "vote" always increased by the same amount with a given change in "party.44 Female 0. This is their great virtue. checking for nonlinearity. Data will talk to you. The relationship between party and vote would change as the third variable.62 0. and examine each of these relationships in detail. Usually. straight party lines. As in logit analysis with a single independent variable. there would be interaction among the variables.50 'Estimated probability of voting Democratic. for various combinations of age and gender. if we were predicting voting for the Democratic Party from age and gender.146 Chapter 9 T A B L E 9-4 Example of Results from a Multivariate Logit Analysis Gender Age 25 35 45 55 65 75 Introduction to Statistics 147 Male 0. Still. 9 of relationships. however. given varying combinations of the independent variables. logit and probit analyses may readily incorporate a number of independent variables. and keep your eye on the question you want to answer. who had built up respect among fellow members. for example. The important thing is that you stay in charge. Those with more seniority. All of these techniques simplify what is going on in a set of data by screening out certain aspects It is possible to add "interaction terms" to the regression equation to take into account certain types of interaction. A computer can chug out hundreds of regression equations in a few seconds. .46 0. you will be better served if you first think carefully about what you want to do. seniority. It is tempting simply to call for a mound of results. and so on." However. "Go out and use these measures." Like ordinary regression analysis.51 0. but the relationships depicted mean quite different things. but that goes beyond the scope of this presentation. especially given the uncertainty involved in working with most social science theories. each of the scattergrams in Figure 9-6 would give approximately the same regression equation. Your measures will then be useful summarizing tools. "Theory Building and the Statistical Concept of Interaction"). and it is a virtue.55 0. 9 CONCLUSION You might expect me to conclude. For instance.48 0. It is critical that you examine as carefully as possible each scattergram on which you base a regression equation. changed. The coefficients for the variables are difficult to interpret. Thus. a regression equation of the form vote = a + b\ party + b seniority 2 Figure 9-6 Four Alternative Scattergrams with the Same Regression Equations would not be appropriate. pick a limited number of relationships of interest to you. I am more interested in cautioning you against using them unwisely.58 0. Use your measures and computer analyses as tools rather than ends in themselves. we might present the results as in Table 9-4. not substitutes for a full look at what is going on. the results are usually made understandable by calculating from transformations of the coefficients what the probability of y is estimated to be. might vote more independently. Blalock (1969. since they are predicting not directly to the variable or its probability but (in the case of the logit) to the natural logarithm of pi (l-p) where p is the probability that y = 1.

It is a very c l o s e a n a l o g y to say that we c a n g a m b l e intelligently on the results of our study by c a l c u l a t i n g the odds that the results are false. and so o n . as in p r e c e d i n g c h a p t e r s . it w o u l d be p o s s i b l e to c o m e up w i t h a group c o n s i s t i n g largely of c o n s e r v a t i v e D e m o c r a t s and liberal R e p u b l i c a n s . 1 LOGIC OF MEASURING SIGNIFICANCE Y o u w e r e able to calculate these probabilities b e c a u s e . T h i s a p p r o a c h m i g h t a l s o s e r v e as a u s e f u l s u p p l e m e n t to a s t a t i s t i c s c o u r s e . I n s t e a d I p r e s e n t the g e n e r a l l o g i c o f s u c h tests t o h e l p y o u u n d e r s t a n d w h a t they m e a n w h e n y o u s e e t h e m . there are t w o m a i n areas o f s t a t i s t i c s : ( 1 ) m e a s u r i n g things. most of the time y o u w o u l d find that the D e m o c r a t s in y o u r group were more liberal than the R e p u b l i c a n s . Now. g i v e n a sufficient set of assumptions. Y o u k n o w that the probability of d r a w i n g a k i n g of hearts both times is 1/52 x 1/52 = 1/2704. T h i s is a c o m m o n problem that must be dealt with w henever anyone tries to describe a w h o l e population of c a s e s by looking only at a sample drawn out of the population. especially measuring relationships between variables. we found that there w e r e qualitatively different w a y s to go about it. H o w e v e r . assuming that you have succeeded in the first round. for instance. it w a s possible 148 . you must accomplish two things. In C h a p t e r s 8 and 9 we l o o k e d at the first of these a s p e c t s . Fortunately. A s I p o i n t e d out i n C h a p t e r 8 . B u t some of the time y o u w o u l d find that the R e p u b l i c a n s in your sample were more liberal than the D e m o c r a t s . G i v e n an honest d e c k of c a r d s . depending on the l e v e l at w h i c h the variables w e r e m e a s u r e d . y o u are instructed to draw a c a r d . let us start w i t h a d e c k of cards. if it is of the s e c o n d (a distorted s a m p l e ) . In other words. one broad procedure h o l d s for all l e v e l s of data. It c o m e s up in conducting social s c i e n c e research. conducting market a n a l y s e s . F r o m that y o u might c o n c l u d e — e r r o n e o u s l y — t h a t there w a s a relationship b e t w e e n party and ideology in the Senate.C h a p t e r 10 Introduction to Statistics 149 subset from that in the Senate as a w h o l e ) . your c o n c l u s i o n w i l l be false. or H o w to G a m b l e on Y o u r Research means of these techniques we c a n c a l c u l a t e h o w m u c h stock to put in our findings. although there are differences in s p e c i f i c techniques. or that the two parties did not differ at all. T h e general procedure by w h i c h we c a l c u l a t e t i e probability that a g i v e n result c o u l d h a v e o c c u r r e d by c h a n c e . w i t h R e p u b l i c a n s the m o r e liberal group. (I m u s t note p a r e n t h e t i c a l l y that. even though the result is not true in general. First. simply because D e m o c r a t i c senators do tend to be more liberal than R e p u b l i c a n senators. senators at r a n d o m . the probability that any particular result w o u l d o c c u r c o u l d be determined. or 0 . W h e n we do this. S . even though the true state of affairs w a s something other than what is i n d i c a t e d by the result. w h i c h s o m e t i m e s t e a c h e s students h o w to u s e the tests w i t h o u t putting the tests in a broader perspective. W i t h s u c h a set of a s s u m p t i o n s . for there are 52 cards and only one of them is the k i n g of hearts. This can be expected to happen only one fifty-second of the time. and in performing numerous other activities. Y o u k n o w that the probability that y o u w i l l draw the k i n g of hearts is 1/52. 0 0 0 3 7 . y o u r c o n c l u s i o n w i l l be true. opinion polling. you can expect to proceed to final success only one fifty-second of one fifty-second of the time. In m e a s u r i n g the statistical s i g n i f i c a n c e of a relationship. if y o u repeatedly drew groups of ten senators. we are s a i d to measure (or "test") the statistical significance of our results. Next. B y Inference.) It is quite p o s s i b l e to get a particular result by c h a n c e . there are w a y s to determine j u s t h o w l i k e l y it is that a particular result c o u l d have o c c u r r e d b y c h a n c e e v e n i f its opposite w e r e gener al l y true. assigning television ratings. selecting the ten at r a n d o m f r o m the w h o l e Senate. replacing the c a r d and reshuffling after the first draw. thoroughly shuffled. in the full two-draw process. As usual w h e n talking about probability. T h e problem in drawing a single group and describing the Senate f r o m it is that y o u cannot k n o w whether your group is of the first type (the relationship between the variables is the same in your subset as in the Senate as a w h o l e ) or of the s e c o n d (the relationship between the variables is different in the 'The logic of this is as follows: In order to draw two kings of hearts in two draws. If y o u c h o s e ten U . In l o o k i n g at h o w to m e a s u r e relationships in C h a p t e r 9. you still can expect success in the second round only one fifty-second of the time. I do not i n t e n d to t e a c h y o u how to use s t a t i s t i c a l tests. y o u are instructed to do this two times. and (2) estimating h o w l i k e l y i t i s that the m e a s u r e y o u h a v e gotten c o u l d h a v e o c c u r r e d b y c h a n c e . If it is of the first type. T h e s e assumptions had to account for all of the factors that c o u l d influence the o u t c o m e of the d r a w i n g . is the s a m e procedure that we u s e in c a l c u l a t i n g the odds of any event h a p p e n i n g . and n o w we e x a m i n e the s e c o n d . you must draw the king of hearts on the first round. administering quality control in manufacturing.

and so on). Statistical inference uses this same logical structure. we decide to treat the set of assumptions as false. The set of assumptions in this case was as follows: 1. If any of these assumptions were not true. How safe would you be to conclude from this limited sample that senatorial Democrats are on the whole more liberal? To find out. none of the cards stick together. using probability theory.' I know that given the amount of evidence I have gathered. There is a probability of 0. using a "chi-square" test (which I introduce later in the chapter). Just from these two assumptions. and we wish to calculate the probability that this event could have occurred given some set of assumptions. that is. you must proceed just as you did in our card-drawing example. In this particular case. This is the way in which we normally use odds and probability. Will you reject that set of assumptions on the basis of your sample result. After all. We cannot be confident of one of these assumptions. the probability of drawing one would have been greater than 1/52. you really are rejecting only one assumption. the probability of drawing one in a single draw would have been 2/52. if there were two kings of hearts (that is. assumption 1 above was wrong). even if there were no difference between Democrats and Republicans. but turns it on its head. and so on. of w h i c h only o n e is the king of hearts. 3. The drawing is blind. the result we have observed could have occurred even if the assumptions were true." 2 HYPOTHESIS TESTING The example we have used is a typical problem in statistical inference: We have a particular result in hand. For instance. An example may be the best way to demonstrate this. for it involves the very thing we are trying to infer. 2. You know whether or not you have drawn the ten senators at random. 2. How you would then treat your research results depends on whether you are a long-shot gambler at heart. given the assumptions. so that each card is equally likely to be drawn ( y o u don't peek. (This usually is referred to as "rejecting the hypothesis" that the assumptions are true. The deck is thoroughly and honestly shuffled before each drawing. Or if the deck had been stacked so as to make drawing the king of hearts likely (that is. but only 33 percent of the Republicans. We have a set of assumptions of which we are confident.264. if in fact Democratic and Republican senators do not differ in the degree to which they are liberal. it is possible for you to calculate how likely it is that you would have gotten any particular result in your sample. are liberal. If there is a sufficiently small probability that the event we have in mind could have occurred. by setting a list of assumptions that would be sufficient to let you calculate the probability of drawing ten senators that look like those you have. 75 percent of the Democrats. TABLE 10-1 Sample Result Democrats Republicans 2 4 Liberal Conservative 3 I There is a great variety of statistical tests available to fit different circumstances of research. Approximately three times out of four you would not have gotten as strong a relationship as you did if the assumptions you wish to test were true. 2 . T h e deck has 52 cards. The chi-square test used here is particularly designed for testing the significance of a relationship between two nominal-scale variables. you can calculate that the probability of having gotten at least as great a difference as you found between the parties. 153-155.264 that I am wrong. not 1/52. is 0. Let us say that you have drawn such a sample and that your results are shown in Table 10-1. See the discussion on pp. What you are saying is: "By rejecting its opposite. what kind of risks ride on your decision. the assumption that there is no relationship between party and ideology in the Senate. Those assumptions determine the probability that any particular event will happen. in rejecting the set of assumptions. your calculation of probabilities would be incorrect. Thus there is a good chance that you could have gotten a relationship as strong as this in a sample of ten even if the assumptions that you set up were true—that is.150 Chapter 10 Introduction to Statistics 151 to calculate exactly how likely any particular result was. From this we know with how much confidence we can predict that the event will occur. given the set of assumptions. There is no relationship b e t w e e n party and i d e o l o g y in the "full" Senate from w h i c h these have been drawn.) And since we are confident of all but one of the assumptions (the thing we want to test). accepting the one chance in four that you are wrong in doing so? If you are willing to reject the assumptions. taking a sample of ten senators to see whether there is a difference in how liberal the two senatorial parties are. assumption 2 above was false). rejecting the set of assumptions really means that we are rejecting that one assumption. These assumptions would be: 1. There is an ideological difference between the senatorial parties. I am running considerable risk in making the statement. Of the ten senators in the sample. Therefore. EXAMPLE OF STATISTICAL INFERENCE Let us return to our previous example. I am making the statement. you purposely have set up the test this way. We run some risk of being wrong in treating the assumptions as false. at what odds we should bet that it will occur. T h e ten senators have been drawn at random. Statistical tests vary in the level of measurement that they require and in the particular set of assumptions that they embody. notice that the truth is questionable with regard to only one of the assumptions in the set.

But because your sample was not that precise a reflection of reality. In using the null hypothesis. if we decide to reject the set. 50. we select a set of assumptions from which we can confidently predict how likely it is that certain things will happen in the future.7309834 percent of voters in the United States voted for George W. It is designed for use with a table relating two nominal-scale variables. It tells us the probability that the result we have observed could have occurred if the assumptions we are rejecting were in fact true—that is. in the Senate example. 100 percent of Democrats liberal. % allows us to estimate the probability that there is no relationship between the variables in the full population from which the sample has been drawn. we really are rejecting only the null hypothesis. Just as in everyday setting of odds. The entries in Table 10-3 are arrived at by calculating how many WASPs. the hypothesis we should like to reject—into the set of assumptions from which we calculate the probability. the probability shows us how likely it is that we are making a mistake. The twisted and convoluted way one must think to understand this process is discouraging to most people when they see it for the first time. In statistical inference. However. It is simply easier to disprove a specific hypothesis than to prove an open-ended hypothesis. Instead. no sample would have produced exactly this figure. on the evidence of this sample. We normally will reject a set of assumptions only when the probability that they could have produced the result we have observed is comfortably low. there also will be such criteria available. one usually has objective outside criteria to "help" in making the decision—an amount of money to be lost if the person is wrong. 51 percent of Republicans liberal. we calculate how likely it is that an observed result could have occurred if some hypothetical and contrary assumption were true. By using a null hypothesis ("there is no difference between the parties"). this is a test that is impossible to set up. which we used in our liberal— conservative example. In Table 10-2. in setting odds. Normally. 1 percent of Republicans liberal. 75 percent of Republicans liberal. Taken to a sufficient number of decimal places. you could never test the hypothesis that your sample result was exactly the same as the true value in the population from which you had drawn the sample. Given the strength of the relationship between the variables in the table. if you wanted to assert that your sample result was sufficiently close to the true relationship for you to accept the sample result as valid. Example: % 2 Null Hypothesis It is clear from our description of statistical inference that we do not test directly the statement we wish to make about a relationship. In much engineering research. It is supposed to apply to occasions as yet unforeseen. based on a sample of 200 respondents to a survey. Bush in 2004. In setting odds. You can see now why I said that statistical inference uses the same logic as setting odds. for instance. To calculate % this table must be compared with a construct designed to show a lack of relationship between the variables. a convention usually is followed of rejecting a set of assumptions only if the probability is less than 0. you would have to choose among an infinite variety of possible true relationships (80 percent of Democrats liberal. there usually are no fixed criteria. but turns it on its head. the probability that we are making a mistake in rejecting the assumptions. "can't we just test our hypothesis directly instead of having to turn it inside out?" But think about how you might go about doing this. however. 3 One popular significance test is % (chi-square)." they ask.152 Chapter 10 Introduction to Statistics 153 The probability we have calculated tells us precisely how likely it is that we are wrong. is a difficult thing to judge. which negates all possible true values that would be consistent with the statement you wish to make. For instance. that there is a relationship between the two variables. By rejecting the null hypothesis we in effect assert its opposite. on the other hand. First of all. but in general. which is the statement we wished to make in the first place. 3 range. you would pick some range of values around the true value and test whether or not the sample result was in that Whether the probability is "comfortably low. Such a construct is shown in Table 10-3. In theory-oriented research. a hypothetical relationship 'between ethnicity and policy priorities has been drawn. it would virtually always not be the same as the true population value. say. Because it is the only questionable member of the set of assumptions. For instance. Accordingly. Therefore. because the result is supposed to hold not just for some particular occasion. and so on). because objective criteria are lacking. because you do not know what the true value is. you can deal with a single statement that negates all possible versions of the statement you want to make. We want to know whether it is safe to assert.05. If the probability is sufficiently low. "Why. would give first priority to employment and welfare if exactly the same proportion of WASPs as of African Americans and "other ethnics" favored the employment/welfare 2 2 2 T A B L E 10-2 Ethnicity and Preferred G o v e r n m e n t Action Priorities Ethnicity Preferred Priority Employment and welfare Foreign affairs Environment Total WASP 30 11 29 70 African American 27 5 40 CO Other 53 14 23 90 Total 110 30 60 200 . you would not want automatically to reject it. a single alternative is chosen. we insert its opposite— which is called the null hypothesis. 100 percent of Democrats liberal. we use the result to reject the assumption." of course.

024 0. The results of this.00 16.0 72.154 Chapter 10 T A B L E 10-3 Table of Nonrelationship Ethnicity Introduction to Statistics 2 155 Preferred Priority Employment and welfare Foreign affairs Environment Total WASP 38. 2 Sampling Distribution priority.5 WASPs who chose employment/welfare. cell C is the upper-right cell. Anglo-Saxon Protestant. inasmuch as each cell's result will be 2 2 Other 49. including the null hypothesis. A sampling distribution embodies all the assumptions you make in a test.25 0. . Similarly. and (3) the squared difference is then divided by the prediction from the hypothetical table.5 21.5 21. it was necessary to maintain the fiction that there could have been one-half of a WASP in favor of the employment/welfare priority. we would expect to find that 110/200 of the 40 African Americans.5 22.0 13. and so on. The more the tables differ. For any particular set of assumptions. chose employment/welfare.0 -4.F ) /F 0 h 2 h 38.167 0. If the tables are exactly the same. It represents the application of probability theory to those assumptions.877 1.0 + 3. the members of each row are equally likely to fall into any given column. % will equal zero.5 13.0 49.5 + 8. a sampling distribution shows what proportion of the time each particular result could be expected to occur if the sampling technique chosen were repeated a very large number of times and the set of assumptions (including the null hypothesis) were true.048 1.444 where for each cell in the table.5 + 0.247 0. (2) this figure is squared.0 -8. because there are 70 WASPs in the sample. and vice versa. we would expect to find that the sample contained 110/200 x 70 = 38.0 90 Total 110 30 60 200 (Fp ~ F f F h h = = F h In Table 10-4. + 0. on the basis of what we found when we looked at the sample.5 6.00 0. cell E falls at the exact center of the table. For instance. are added 0 h C D E LL. and (2) that its entry is the same proportion of its column frequency as its row frequency is of the total sample. cell B is the next cell to the right.25 25. cell by cell.00 16. Table 10-3 embodies the null hypothesis that we want to test—that there is no relationship between the variables. Each cell in the table has the characteristic (1) that its entry is the same proportion of its row frequency as its column frequency is of the total sample. and 110/200 of the sample favor the employment/welfare priority.00 2 1. Thus.0 G H I 4 WASP is the acronym for white. It is calculated from the formula 4 Before I can show how we go from measuring the difference between the observed and null hypothesis tables to calculating the probability that the null hypothesis is true. I must first introduce the idea of a sampling distribution. This is what we should expect if the two variables were unrelated. (1) the prediction from the hypothetical table is subtracted from the actual figure.5 27. until the hypothetical table was filled. to calculate the probability that each possible result would occur.25 1. F is the observed frequency and F is the frequency predicted for the hypothetical table.0 -4. that the full population does not look like the distribution in the hypothetical table? Chi-square is a measure of how different the two tables are. in the card-drawing example on p.5 10.0 70 African American 22.136 0. Cell A is the upper-left cell of Table 10-2 or 10-3 (the cell in which WASPs choosing employment/welfare fall).00 12.0 12.019 3. The question we want to ask is: Are the two tables sufficiently different that we can say. That is. That is. the probability of drawing the T A B L E 10-4 Cell A B Calculations for % 0 2 (F ~ F ) h (F -F ) 0 h 2 (F . the sampling distribution states that the probability of drawing the ace of hearts is 1/52.5 + 5.0.0 6. from all the cells of the table.5 1. Thus for each cell in the table.0 27. cell D is the middle left cell. or 22. it gives the probability of getting a particular result if you applied the techniques you are using to a population for which the null hypothesis is true. and so on. 149. Some simple sampling distributions can be calculated easily.0 40 together (you will recall that this is what the sign "X" means) to give us the %. the greater % will be.5 10.593 X = total = 8.25 64.0 12. % is calculated for the present example. Notice that to construct a table in which the variables are completely unrelated.333 0.

1 Sampling Distribution for Two Flips of a C o i n Figure 1 0 .00 Introduction to Statistics 157 k i n g o f hearts i s 1/52.) F r o m this s a m p l i n g distribution we see that there is a probability of between 0. and (2) random selection. or at least two heads. so the probability of at least one head is 0. suppose that y o u h a d to predict the probability of getting j u s t one head in two flips of a c o i n . 2 x at least as high as the one observed w o u l d have been obtained if the sample were 2 2 drawn by random selection from a parent population in w h i c h the variables were unrelated. F i g u r e 10-2 charts the sampling distribution in this way.156 Chapter 10 1. 2 not related.50 n o a.75 a . we c o u l d only get at least two heads by getting exactly two heads.10 that we w o u l d have f o u n d a % as h i g h as 8. T h e s a m p l i n g distributions for most statistical tests are m o r e c o m p l e x . (Note that this is a cumulative sampling distribution. O f c o u r s e .9.25. a n d s o o n . T h e s e s a m p l i n g distributions are s i m p l e .3 .25 +^ .25) or by getting one head (probability of 0. 2 5 . a n d we must u s e tables developed by statisticians to v i s u a l i z e them. ( T h e sampling distribution is based on assumptions of (1) the null hypothesis y o u w i s h to test. s u c h a s T a b l e s 10-2 and 10-3) i s presented i n F i g u r e 1 0 . if the n u l l h y p o t h e s i s w e r e true).25. the probability of getting a tail f o l l o w e d by a h e a d is 0. in fact. and the probability of getting a tail f o l l o w e d by a tail is 0.2 Flips of a C o i n C h i .50 n o i _ a. the probability of getting j u s t one head in either of the two w a y s is 0.25.50).00 . like the one depicted in F i g u r e 10-2.05 and 0.25.) T h u s . y o u w o u l d k n o w that the probability of getting x as high as this w a s 0.44 f r o m a three-by-three table if there w e r e really no relationship b e t w e e n ethnicity and c h o i c e of priority (that i s . if y o u had found a % of 4. T h i s s a m p l i n g distribution is presented g r a p h i c a l l y in F i g u r e 10-1. w h i c h y o u k n o w y o u have done. and the probability of getting two heads is 0. at least no heads. In another s i m p l e c a s e . the probability of getting a head f o l l o w e d by a head is 0. for Three-by-Three Tables 2 1. T h e probabilities given are the probability that a 1 Number of Heads Cumulative Sampling Distribution for Two 2 .75. this s a m p l i n g d i s t r i b u t i o n d e p e n d s o n the set o f a s s u m p t i o n s y o u m a d e i n that e x a m p l e .3 if the variables y o u were w o r k i n g w i t h w e r e . A s s u m e that (1) the c o i n is honest. (2) the flipper is honest.25 0 . T h e same s a m p l i n g distribution c o u l d be presented in a cumulative graph. t h r o u g h a l l the c a r d s o f the d e c k . We a l w a y s w i l l get at least zero heads ( w e cannot get a negative n u m b e r ) . at least one h e a d . so the probability of at least two heads is 0.3 Sampling Distribution of % . the probability of getting no heads is 0 .0. T h e s a m p l i n g d i s t r i b u t i o n for a different set o f a s s u m p t i o n s w o u l d l o o k different.25. Y o u c a n now calculate that the probability of getting a head f o l l o w e d by a tail is 0.25. B e c a u s e two of the possibilities involve getting j u s t one head in the two throws.75 > ro .50.s q u a r e sampling distributions vary depending on the number of categories in the rows and c o l u m n s of the table used.00 0 1 Number of Heads 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 . T h e c u m u l a t i v e s a m p l i n g distribution of % for a three-by-three table (a table w i t h three categories in the r o w s and three categories in the c o l u m n s . and (3) the c o i n w i l l not stand on its edge. we c o u l d get at least one head either by getting no heads (probability of 0.00 0 Figure 1 0 . . s h o w i n g the probability of getting . T h i s m e a n s that we w o u l d r u n a r i s k of 2 Figure 1 0 . so the probability of that outcome is 1.

depicted in the scattergram in F i g u r e 10-4. in regression a n a l y s i s the amount of c o n f i d e n c e we c a n h a v e in our estimate of w h a t the relationship l o o k s like is a matter not only of how 5 S e e the box "Law of Large Numbers" on p. B u t there m a y be others. In g e n e r a l . To s u m m a r i z e these sections: A sampling distribution is a list of the probabilities that e a c h possible event (out of a set of events) w i l l happen. h o w ever.) If there is relatively little variation in the independent v a r i a b l e . T w o s u c h observations have been added in F i g u r e 10-6. e a c h of w h i c h has an e q u a l s u m of squared deviations about itself. ( I n other w o r d s . 6 N is the most o b v i o u s factor affecting the statistical s i g n i f i c a n c e of findings. 6 . F r o m F i g u r e 10-3 we see 2 that if the null hypothesis of no relationship w e r e true. the number of c a s e s in a s a m p l e . y in the scattergram. If we find f r o m the sampling distribution that it is very unlikely. If we choose our significance test appropriately. T h u s f r o m a table s h o w i n g e x a c t l y the s a m e pattern as T a b l e 10-2 but based on a s a m p l e of 4 0 0 c a s e s instead of the 2 0 0 used in that table. A c o n s i d e r a b l e c h a n g e in the angle at w h i c h the line p a s s e s through the point x. finding a yf of 8. this amounts to rejecting the null hypothesis and asserting its opposite. In m a n y sorts of statistical tests. A l l other things being e q u a l .01. values on x fall so c l o s e together. T h e probabilities in a sampling distribution are based on a set of assumptions. y w i l l not c h a n g e the s i z e of the s q u a r e d deviations f r o m the l i n e by v e r y m u c h . B e c a u s e all the data points fall above a single value of the independent variable. the m o r e certain y o u c a n be that the statement is true.5 . Figure 10-4 Introduction to Statistics 1 5 9 Independent Variable Regression Analysis with Zero V a r i a n c e in the Independent Variable m a n y c a s e s we have used but also of h o w m u c h the independent variable varies. O n e w a y to l o o k at this is to think o f the r e g r e s s i o n line a s w o b b l i n g o n a f u l c r u m . It tells us how likely it is that we c o u l d have gotten our results by c h a n c e a l o n e — t h e probability that in fact the opposite of what we are asserting is true. is a l w a y s a factor in s i g n i f i c a n c e tests. we are s i m p l y unable to c h o o s e any regression line. T h i s is the extreme c a s e . w h i c h is what we had originally wanted to say about the data.44) c o u l d have o c c u r r e d . takes the n u m b e r of c a s e s into account.444. we reject the assumptions on w h i c h the s a m p l i n g distribution is based. for e x a m p l e . a c c o r d i n g l y . w o b b l e in the line does not c h a n g e the e x p e c t e d v a l u e of y v e r y greatly. B e c a u s e we are sure that all but one of those assumptions (the null hypothesis) are true. the set of assumptions on w h i c h its s a m p l i n g distribution is based w i l l consist of (1) a null hypothesis (the assumption that what we really want to say about the data is false). every r e g r e s sion l i n e must p a s s through point x. T h i s is due to the fact that for observations w h o s e v a l u e on x is c l o s e to x. the quantity (F — F ) /F 2 0 h h w o u l d double for each cell. A s i t h a p p e n s . In other w o r d s . as in F i g u r e 1 0 . If e a c h n u m b e r in T a b l e s 10-2 and 10-3 were doubled. the l i n e is free to w o b b l e a g o o d d e a l . given those assumptions. thus the difference b e t w e e n the o b s e r v e d and e x p e c t e d v a l u e s of y also is not greatly changed. the probability of getting a % at least this high w o u l d be less than 0. 5 lines (a few of w h i c h are indicated on the graph) c a n be p a s s e d through the data. The % w o u l d thus be double 8. the m o r e stock we c a n put in our findings.1 5 8 Chapter 10 b e t w e e n 1 c h a n c e in 10 and 1 c h a n c e in 20 of b e i n g w r o n g in asserting on the b a s i s of T a b l e 10-2 that the n u l l h y p o t h e s i s is f a l s e — t h a t there is a relationship between ethnicity and the c h o i c e of priorities. w o u l d h o l d a m u c h firmer grip on the regression line. O b s e r v a t i o n s w i t h w i d e l y v a r y i n g v a l u e s on x. u s i n g this point as a f u l c r u m . it is i m p o s s i b l e to c h o o s e a single " b e s t " l i n e by least-squares criteria. that the event we have observed (in our ethnicity/choice of priorities e x a m p l e above. the greater the n u m b e r of c a s e s on w h i c h a statement is based. or 16.888. an infinite n u m b e r of I m p o r t a n c e of N N. the greater the variation in the i n d e p e n d ent v a r i a b l e . F o r e x a m p l e . T h e following five paragraphs present advanced material that can be skipped over without any loss in comprehension of other material covered in this book. T h e probability that we c o u l d have gotten the observed result if all the assumptions underlying the sampling distribution were true tells us h o w likely it is that we are w r o n g to reject the null hypothesis. this is determined mathem a t i c a l l y by the f o r m u l a s for a a n d b. T h e l i n e i s s i m p l y not h e l d f i r m l y i n p l a c e b y these observations. the e x p e c t e d v a l u e of y w h e n x equals its o w n m e a n must be the m e a n of y. we w o u l d run a risk of less than 1 c h a n c e in 100 of being w r o n g in asserting that there is a relationship between the v a r i a b l e s . In the extreme c a s e of zero variance in the independent variable. and (2) a group of other assumptions that we feel certain are true. T h u s a significance test furnishes a useful c h e c k on our research. No one of the observations is able significantly to affect the regression line in F i g u r e 10-5. it is the only factor we need to w o r r y about. a c c o r d i n g l y . w h o s e Every s i g n i f i c a n c e test. the m o r e firmly fixed the estimate of the r e g r e s s i o n l i n e c a n be a n d . 61.

it might be that some of these units shared a coordinated police force. Thus we might be inappropriately confident of our assertions. see King. without the researcher realizing it. although the 63 townships looked like separate units." The investigator might measure the explicitness of guidelines to patrols in the 11 towns and 63 townships of a county and relate this measure by regression analysis to the roughness with which police in each of those units handle suspects. ch. in calculating how likely it is that our estimate of Figure 1 0 . but also how widely they are spread on the independent variable. If the observations are not independent. Significance tests designed for use with regression analysis take into account both the number of cases and the amount of variation in the independent variable. plus the sheriff's office). Accordingly. part 1). Statistical tests might be applied to the regression results. he or she may overestimate the strength of the evidence.160 Chapter 10 Introduction to Statistics a regression has been due to chance. with one set of guidelines and one set of expectations with regard to roughness. However. The regression line in Figure 10-6 cannot stray very far from points A and B without sharply increasing that sum. president's popularity. 7 Problem of Independent Observations It often happens that a researcher draws a conclusion based on a set of observations that appear to be independent but are really not. and Verba (1994. they would all be separate parts of a single police administration. For a good expository discussion of multicollinearity. Keohane. because statistical tests already assume that the data to be analyzed consist of independent observations. Consider a study that seeks to determine the extent to which "police brutality" is a function of the explicitness of guidelines issued to police patrols. we must take into account not only how many observations have been used in the analysis. 7 Figure 10-5 Regression Analysis with Little Variance in the Independent Variable If the regression line were now to wobble through the same angles as in Figure 10-5. The problem of nonindependent observations does not often occur in a simple form such as this. Remember. The difficulty here is that statistical tests will not be able to alert us to the danger. all statistical tests will underestimate the probability that a relationship might be due to chance. 4. which should raise a specter of doubt about the correctness of an assertion based on these data. with N stated as 74. the problem occurs frequently in a more subtle (and more technical) way under the guise of autocorrelation. Autocorrelation occurs when observations are related to some extent but are not (as in the study of police brutality) identical to each other. if we look at the relationship between trends in the economy and trends in the U. all township police forces might be under the direction of the county sheriff. Thus a few points whose x values are extreme have more impact in determining the slope of the line than do a larger number of points whose x values fall close to the mean. and if it does arise. For example. For example.S. The relationship in question might be due to chance. the size of the deviations of observations A and B (in Figure 10-6) from the line would increase dramatically. we will probably have a T h e general extension of this problem to multivariate regression is the problem of multicollinearity. month by month. An example may help. with close and explicit directions presumed to be associated with low "brutality. Thus there would in fact be only 12 independent observations (each of the 11 town forces.6 The Stabilizing Independent Variable Effect of Variance in the y . it can generally be avoided by simple common sense. the regression line must be the line that minimizes the sum of squared deviations about itself. In this case. Because there are in effect fewer observations in the study than the scholar believes. Unknown to the investigator. however.

however—the way we find out about politics—is by looking at data and seeking out relationships between variables. sample results would fall within plus or minus X percent of the true population value 95 percent of the time. or whatever. 9 USES AND LIMITATIONS OF STATISTICAL TESTS I have tried in this chapter to present a significance test for what it is—a useful check on research results. however. such as the one by Knoke. anything we find out about them is by definition "true generally. There are times when we wish to describe a specific population as it exists at one point in time. A tax administrator who wanted to change the extent to which states relied on sales taxes to generate income would first have to try to find out why states use sales taxes. perhaps plus or minus 1 percent. there is a sense in which all research involves "sampling" of a sort and in which misleading results can occur by chance. a tax administrator may want to know how many states administer sales taxes. for instance. we should seek a more inclusive sort of generality.1 0 3 . but about "state politics. It is a pity that looking at data requires less formal training (but more practice) than does Note. then. the pollster can conclude that according to the sampling distribution of percent approval that one would get using repeated samples of this sort and of this size. Partial solutions are available.000 respondents gave you plus or minus 3 percent." But suppose we think of ourselves as trying to say something not just about the 50 states as they exist at this one point in time. rather than to develop explanatory theories from it. it means that 95 percent of the time it would fall within that range of the true value. and Mee (2002) or Kmenta (1997). Sometimes an author claims that she need not be concerned about the possibility of chance results because her sample is synonymous with the total population of subjects (all the states. this is particularly likely in what I have termed "engineering" research. Thus the pollster claims that the result is accurate to "within plus or minus X percent.S. This is a statement that should arouse suspicion. The real meat of research. by and large. What the administrator is concerned with is a description of the tax situation as it currently exists. or some other level of accuracy. how great a portion of national income these states involve. With infrequent exceptions. or what have you). 8 A ubiquitous activity in our society is polling. asking a sample of the population whom they will vote for. but they are beyond the scope of this book. that changing N changes the width of the "plus or minus" range. In most theory-oriented research. For instance. she must be concerned with the problem of chance results. and what they need is simply to know what the "state of affairs" is. on the other hand. about 2. all the nations in the United Nations. If the author is trying to draw general conclusions from the study. Status quo administrators.000 would give you a lower range. neither can we say that we have as much information here as we would have with two widely separated months. The latter view seems to reflect more accurately what we try to do in developing political theories out of empirical research. discuss the problem. that to the extent that an engineer sees her task as changing and reforming patterns rather than continuing ongoing administrative procedures. In one sense. This operation is a variant of statistical significance testing. Note here." Then we must regard these 50 states as a sample drawn for us by chance and accident from a larger metaphysical population of "states"—all states as they might be in the future or as they might have been had their boundaries been drawn differently or had history proceeded differently.000 respondents are polled to describe a population such as that of the United States. In such research we frequently are interested in measuring some condition of a population so as to react to it or make adjustments in it. with developing theories to explain why things are as they are. . whether they support regulation of handguns. the sorts of things I discussed in Chapters 2 through 9. POLLING AND SIGNIFICANCE TESTS SIGNIFICANCE TEST: ALWAYS NECESSARY? Even when we do not work with a limited sample drawn from a larger group of subjects. this is a problem all of us must take seriously in our research. also. increasing A' to 4. because their chief concern is with plugging established procedures into any given state of affairs. and the results are reported as "accurate to within plus or minus 3 percent"—or 2 percent. there is no question but that this is the "true" result. by the way. we might look at all 50 states and find that the generosity of their welfare budgets is related to the degree of party competition in the states' elections. Most texts in statistics or econometrics." Note that this does not mean that the result absolutely falls within plus or minus X percent of the true value. all the U. I mention the problem at this point only to alert you to it. and with knowledge of how the individuals have been chosen for the sample (usually some variant of random sampling). 8 ''See the discussion of sampling on pp. not. Because we have included all the states in our sample. researchers often place undue emphasis on significance tests. If an N of about 2. propositions that one would expect to be as true of states or nations or senators a decade from now as they are today. and so on. Though one would not go so far as to say that gathering information from these two months is simply equivalent to having information about a single month (as we could say about any two townships in the other example). Typically. explanatory theory will be relatively more important. For instance. Bohrnstedt. 9 9 .162 Chapter 10 Introduction to Statistics 163 problem of autocorrelation. Unfortunately. February 1943 had much in common with May 1943. which of course can be set as desired by the person doing the polling. Based on a given number of cases. can be more content with purely descriptive information. senators.

not positive. Because previous theorists had said that one should expect a stronger relationship between these variables among the middle class than among the working class. however. "Common Misinterpretations of Significance Levels in Sociological Journals" (1968). for instance). along the lines of the example I used. FURTHER DISCUSSION A particularly good discussion of the misapplication of significance tests. in terms of their suitability for a given level of measurement and the particular mix of assumptions (other than the null hypothesis) they require the investigator to guarantee. Evidence? Yes. think about the two following situations.01 that there was no relationship. 2. by contrast. and she uses it as the basis for a chapter and a half of her book. This strikes her as a surprising finding.05 level) between a representative's height and the way he voted on the bill. The significance test is not itself a measure of the strength of a relationship but rather. Among the middle-class respondents. the relationship was more highly significant. A recent article illustrates the problem. she finds a statistically significant relationship (at the 0. with a probability of less than 0. In this study. findings. the relationship was less highly significant. "Proof? No. many researchers give the tests an undue emphasis in their research. For a description of particular tests. In fact.01 and 0. as it happened. The one scholar who did find a statistically significant result publishes it. which is the common thread running through all statistical tests. Their evidence for this was that among the working-class respondents. In this example. the reader should consult a statistics text such as the one cited at the end of Chapter 8. For two of these votes. they were using significance tests for something the tests were not meant to do. Anyone thinking of using these tests should first take a course in statistics. it turned out that there was a Wronger relationship among the middle class. These tests vary among themselves. when the strength of the relationship was measured by an appropriate technique (the Goodman-Kruskal gamma. barring that. A large number of scholars study committee systems to see whether democracy in committee decision making leads the members to be satisfied with their work on the committee. Perhaps because they have spent so much time in courses learning to use significance tests. The second chapter of Siegel (1956) is a particularly useful review of the overall logic of significance tests. reporting a statistically significant relationship between committee democracy and members' satisfaction. See also Winch and Campbell. It aims at much the same presentation as I have attempted in this chapter.164 Chapter 10 Introduction to Statistics calculating significance tests. but these are rejected by journal editors because they are negative. Net result: one published article. What is wrong in each situation ? 1. the middle-class portion of the authors' sample was only about half as large as the working-class portion. but in rather more technical detail. It was because of the smaller number of cases in the middle-class portion that its relationship showed up as less significant than the same relationship for the working-class portion. The status of these tests should be strictly that of a secondary check on the creative work the researcher has done in looking at relationships. with probability between 0. Another very good treatment is that of Simon (1985). This is why there are so many different tests. The Significance of Tests of Significance" (1969). a check on how likely it is that a given measure is due to chance. The authors' conclusion from their own data was wrong. . useful presentations of the techniques are given in the text cited at the end of Chapter 8. A researcher studying Congress examines the vote on 100 or so bills. All but one of these scholars fail to find a statistically significant relationship between these two variables. the authors reported that two variables were more strongly related among the working-class portion of their sample than among the middle-class portion. Most of those who fail to find a relationship leave that question and start to work on other things.05. however. Unfortunately. A few of them write up their results. For further consideration. these authors were understandably excited about their findings. CONCLUSION I have tried in this chapter to give you some idea of the general logic of inference. is Duggan and Dean.

w h e r e a s t h e r e s u l t s o f c o r r e l a t i o n a n a l y s i s c a n s e r v e o n l y a s a description of a particular situation. "observations" in political science are riddled with problems of inaccurate m e a s u r e m e n t . In C h a p t e r 2. then. specifically. p . 2 l b i d . Birkbeck Hill. 166 . m u c h of which is extraneous to what we want to do." and not "smart modern thought. I e q u a t e d the research process with a search for elegant theories. w h e t h e r it is a matter of reliability or validity. Be/swell's Life of Johnson. but t h e i m p o r t a n t p o i n t for m y discussion here is that this a n s w e r begs the question of h o w the original theorist found the theory. I have discussed various aspects of research: concept formation. Suppose that early mechanics had developed by the use of regression equations. I h a v e argued. T h e i r c o m b i n e d effect is to m a k e it very difficult to l o o k at a b a t c h of o b s e r v a t i o n s and pick out the best simplifying pattern. and the nonexperi- ®d4 L® nT9 r L j jo © TD DT 1 m e n t a l c i r c u m s t a n c e s u n d e r w h i c h w e g a t h e r t h e m m a k e i t difficult t o isolate the effects o f s i n g l e v a r i a b l e s . (says he. The wind. W h a t m a k e s a political scientist describe the relationship a m o n g a group of variables in a particular way? Probably it most frequently happens that the A simple example will illustrate some of the difficulties which might arise by this kind of "brick-by-brick" approach to theory. 2 (London: Clarendon Press. T h a t is.) is a natural one. occasions an epidemic cold. is important b e c a u s e without accurate m e a s u r e m e n t we cannot establish the connection b e t w e e n the data we are h a n d l i n g a n d t h e theory w i t h w h i c h w e a r e w o r k i n g . 1887). I h a v e n e v e r dealt w i t h t h e critical q u e s t i o n : J u s t h o w d o e s o n e find a theory on w h i c h to w o r k ? T h e r e is o n e deceptively s i m p l e a n s w e r to this question. K i l d a renders a North-East Wind indispensably necessary before a stranger can land. which is so very obvious as. Christian." but Johnson regarded this as a "prejudice. as well as that of the author. The relation in mechanics is that the velocity attained is equal to the acceleration due to gravity times the time the object has fallen. m e a s u r e m e n t . . w h i c h i s t o c h o o s e t h e o r i e s for s t u d y f r o m t h e a l r e a d y existing b o d y o f w o r k i n political science. J a m e s S. "The cause. b e c a u s e the results of regression analysis apply g e n e r a l l y t o a theory. This problem is not confined to the social sciences. R e g r e s s i o n a n a l y s i s u s u a l l y i s m o r e useful than correlation analysis. T h e situation of St. pp. that the pattern we ideally w o u l d h o p e to pick out is obscured." Eventually the explanation favored by B o s w e l l w a s offered in a letter from a L a d y of Norfolk: v = gt researcher observes a b o d y of data and tries to see a pattern a m o n g them. that an investigation had been carried out relating the length of time a body had fallen through the air and the velocity it attained. of Docking—after ruminating a little. This is the most obvious way to go about devising a theory. that the m o s t appropriate w a y to devise a theory is simply to look at out the most prominent pattern running Unfortunately. b e c a u s e it a l l o w s u s m o r e f l e x i b i l i t y i n s t a t i n g t h e theory w e c h o o s e t o w o r k w i t h .Chapter 11 the j u m b l e of observations and pick through them? Where Do Theories Come From? 167 follow. Reading the book with my ingenious friend. Vol. and so on. In all of this. Suppose. In succeeding chapters. W o u l d it not 'G. There is so m u c h going on in a group of observations. take an existing t h e o r y a n d try it out on s o m e data. for that reason. and of the the residents 1 S c o t t i s h island of St. M y criterion for t h e usefulness o f any o f t h e s e t e c h n i q u e s has been the extent to which that technique aided in the creation of elegant theories. There was at that time already speculation that illnesses were transmitted from one person to another. d a t a analysis. they are often m e a s u r e d imprecisely. C o l e m a n described the w a y the theory of gravity m i g h t not h a v e b e e n d e v i s e d h a d G a l i l e o g o n e a b o u t h i s t a s k i n t h e w a y m o s t d a t a analyzing social scientists do: Now for the explication of this seeming mystery. M e a s u r e m e n t accuracy. A high d e g r e e of "precision in m e a s u r e m e n t " is important. I d i s c u s s e d s o m e o f t h e l i m i t a t i o n s o f this p r o c e d u r e i n C h a p t e r 2 . K i l d a c a u g h t c o l d w h e n e v e r a s h i p arrived t h e r e . it follows quite naturally from the things a theory is s u p p o s e d to achieve. to have escaped the penetration of Dr. This has b e e n a b o o k a b o u t d e v e l o p i n g theories a n d trying t h e m out on reality. 52. James in 1769 about the mystery of w h y Boswell. not the stranger. Johnson and his friend. or In a similar vein. T h e p u r p o s e of a theory is to p r o v i d e a simplified pattern to describe a complicated j u m b l e of observations. 51-52. An amusing example of the difficulty of c h o o s i n g a m o n g potential e x p l a n a t i o n s w i t h o u t the benefit of prior theory some is offered by of their friends a discussion among Samuel Johnson. the late Reverend Mr. T h e s e a r e all p r o b l e m s I d i s c u s s e d e a r l i e r i n t h e b o o k .

e. It is in this spirit that I h a v e tried to stress the "craft" in the "craft of political research. the researcher deduces (from something else. the investigators would have ended with numerous pairs of observations ( V j . A h e a l t h y a w a r e n e s s o f m a j o r t h e o r i e s i n f i e l d s s u c h a s t h e s e .e. Political science. it is h a r d to a r g u e p a t l y that t h e o r y b u i l d i n g must be done deductively. But in every case. T h e art of building a theory r e m a i n s in flux in political science—partly fit. can be a helpful source of theory.u p o n b o d y o f a s s u m p t i o n s f r o m w h i c h m o s t o f its theories can be d e d u c e d (Kuhn. air resistance—which was irrelevant to the fundamentals of mechanics. L a c k i n g this b a s e for d e d u c t i o n . would be air resistance. Now if there had been numerous investigations involving different-sized bodies. which they would locate on a scatter diagram in order to find the line of best t Where Do Theories Come From? 169 of epidemic growth and population dynamics. d e d u c t i v e t h e o r y building is clearly t h e better of t h e t w o . and other things. objects which fell a great distance) and low-density objects (i.e. and there would have been indications that at high velocities the relation of velocity to time was not even linear. m o r a l imagina- tion. it a l l o w s a p l a c e for every sort of imagination to work: literary imagination. such as mass or density of the object. The distinction here is between "inductive" and "deductive" theory building. and especially for high velocities (i. s o m e prior e x p e c t a t i o n s ) w h a t sort of a pattern to e x p e c t a n d t h e n l o o k s for it a m o n g the observations. So may microeconomic theory. T h e r e are s o m e tricks that m a y heighten your awareness of situations in w h i c h deduction is feasible. T h e o r i e s James S. as noted in Chapter 1. A c c o r d i n g to the a r g u m e n t I h a v e p r e s e n t e d so far in this chapter. sociology. s o m e o f w h i c h are n o t all t h a t closely related to our field. w h e r e it is feasible. The reason.. 1964). but it wouldn't have corresponded to the simple velocity-time relation which served as the basis for Galileo's remarkable contribution to the science of mechanics. m a t h e m a t i c a l imagination. which has different effects as a function of the density of the object. O n e w a y of distinguishing an o n g o i n g " s c i e n c e " from a " p r e . different velocities. rational choice theory drawn from m i c r o e c o n o m i c t h e o r y h a s b e e n p a r t i c u l a r l y fruitful for political s c i e n c e . the researcher scans the observations. On the other hand. They might even have served to confound the issue.168 Chapter 11 where g is the acceleration due to gravity. the observed velocity would fall considerably below that which the theoretical equation predicts. Introduction to Mathematical Sociology (New York: The Free Press. and similar social sciences c a n n o t be said to possess such a b o d y of assumptions. scientific imagination. of course." T h e m o r a l o f t h e s e s t o r i e s i s t h a t w e a l m o s t a l w a y s a r e b e t t e r off i f w e h a v e s o m e i d e a o f w h a t k i n d o f pattern w e w a n t t o l o o k for b e f o r e w e start t o l o o k a t data. its velocity. from such fields as epidemiology and ecology. it is important to bear in mind the advantages of deduction. its shape. Copyright 1964 by the Free Press. by bringing in too soon a factor—i. Coleman. The resulting regression equation might have ended up including other variables. If we h a v e in m i n d a p a r t i c u l a r k i n d of pattern. G a l i l e o w a s able to i g n o r e the effects of air resistance b e c a u s e he k n e w that they w e r e not the thing he should use to explain the speed of falling objects. 100-101. it is e a s y to tell..s c i e n c e " is that the f o r m e r i n c l u d e s a g e n e r a l l y a g r e e d . w h a t i s relevant t o that p a t t e r n a n d w h a t i s e x t r a n e o u s . pp. To build theory deductively. O n e of the best s o u r c e s of d e d u c t i v e t h e o r y is a w e l l . T h e r e simply are not m a n y w e l l . a m o n g the j u m b l e of t h i n g s w e find. 3 . To build theory inductively. m a y suggest theories to political scientists or sociologists. feathers).. looking for patterns. The regression equation would of course have been empirically correct. but largely inductive.e s t a b l i s h e d t h e o r y f r o m a n o t h e r field. T h e resulting confusion can be both enjoyable and fruitful. and bodies with differing densities. b e c a u s e m o r e t h a n in m o s t d i s c i p l i n e s .e s t a b l i s h e d premises i n t h e s o c i a l s c i e n c e s f r o m w h i c h t o d e d u c e a n y t h i n g . 1962). 3 deductive. t ). T h e p r o b l e m lies with that " s o m e t h i n g else" from w h i c h the student is supposed to d e d u c e theories.

653-664.: Prentice Hall. 1964. Statistical Methods. "Test-Score "An Institutional Explanation of the Democratic Peace. 2007. 1962. BARTELS. Analysis of Nominal Data. "Some Consequences of Measurement Error in Survey Data. Who Governs! New Haven. SMITH. "Messenger or Message? Military Buildups and the Initiation of Conflict. 1962. GERRING. STEPHEN. N. JAMES D. and W. HUBERT M. New York: Science Editions. 191-217. RICHARD J. ARTHUR A. JAMES D. 1978. New York: Holt." American Journal of Political Science." American Political Science Review. BARBER." Ph. 341-354. PHILLIPS SHIVELY. The Language of Social Research. 2000. 48. 1963. and Direct Mail on Voter Turnout: A Field Experiment.. 2. ASHER. Bennett and Arthur A. BATALLA. 21 (August). BLALOCK. 19. 1987. 18 (May). 1983. Conn.: Prentice Hall. London: George Allen & Unwin.C. PETER. PHILIP E. "Attitudes and the Survey Response. 1964. "Partisanship and Party System Institutionalization. and DONALD P." Journal of Politics. and WILLIAM H. DAVID K. 1979.." American Political Science Review. Rinehart & Winston. N. . ALAN GERBER. ECKSTEIN. Theory Construction: Prom Verbal to Mathematical Formulations. PATRICK J. New York: John Wiley & Sons.: University of California Press.. New York: Free Press. New York: John Wiley & Sons. . THOMAS J. Ann Arbor: University of Michigan Press. ALAN S. GILBERT.. Evaluation and Experiment." Political Methodology. Social Statistics. New York: Free Press. Experimental and Quasi-experimental Designs for Research. SNYDER. BACHRACH. "School Vouchers and Academic Performance: Results from Three Randomized Field Trials" Journal of Policy Analysis and Management. 435-464." Journal of the American Statistical Association. FRED I. DIGESER. pp. "Messages Received: The Political Impact of Media Exposure.: Evidence from Randomized Field Trials. THOMAS D. 947-953. Upper Saddle River.: University of North Carolina Press. BARTON. 1993.2 6 1 . PAUL F. CARTER. 45^16. 747-754. September 2000. and FREDERICK MOSTELLER. 1955. Telephone Calls. DRAPER. "Does Political Stability Hinder Economic Development?" Comparative Politics. 2 0 6 . 56 (December). 977-1007. Cross-Level Inference." Party Politics. 11. The Presidential Character: Predicting Performance in the White House. New York: St. Ideology and Discontent. DEAN. JOHN P. GREEN. N. . . "Common Misinterpretations of Significance Levels ACHEN. 767-777'. Quasi-experimentation: Design and Analysis Issues for Field Setting. "Essays on the Early History of Latin American Central Banking. 139-171. 93 (December)." American Political Science Review. 54 (November). "The Concept of Property-Space in Social Research. CAMPBELL.C. and PAUL E. The American Voter. 1966. . Ohio. EDWARDS. 1971. Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics.: Yale University Press. New York City. "The Two Faces of Power.J. ALLEN L. 1969.: Yale University Press. Chicago: Rand McNally. 1972. 1995." In David Apter. RICHARD J. New Haven. 1966. Applied Regression Analysis. KRUSKAL. CHRISTOPHER H. 96. "Politicization of the Electorate in France and the United States. 1954. 1984. D. CONVERSE. eds. 94 (September). Conn.C. DAHL. ANTHONY. 1999." In Carl A. eds. THOMAS H. 3rd ed. Restructuring of Social and Political Theory. Lazarsfeld and Morris Rosenberg." Paper presented at the annual meetings of the American Political Science Association. DUVERGER. Division and Cohesion in Democracy. LEWIS F. and others. 1-23. 1981. and STEVEN WELDON. Upper Saddle River. 211-244. New York: Free Press. in Sociological Journals. New York: Harper & Row. Chicago: Rand McNally. 12-25. JAMES S. Modern Political Analysis. GREEN. 143—166. and JULIAN C. DAVID. 267-285. "Measures of Association for Cross Classifications. COLEMAN. GOODMAN. 1977. BUENO DE MESQUITA. 170 . BERNSTEIN. 1968. GEDDES.. 39-193. Martin's Press. N. University of California-Los Angeles. ALLAN H. 49 (August). 2004. Philadelphia: University of Pennsylvania Press. MAURICE. 2007. 2002. 1985. Berkeley. New York: Cambridge University Press.. HOWELL. Causal Inferences in Nonexperimental Research. 1973. RUSSELL J. The Presidential Difference: Leadership and Style from FDR to Clinton. An Economic Theory of Democracy. 171 1963. "The Measurement and Meaning of Trust in Government. 10.: Princeton University Press. Chapel Hill. 1218-1231. 1986. STANLEY. LEO A. BARBARA. HERBERT B. and ROY PIERCE. . pp. D. "The Nature of Belief Systems in Mass Publics. DALTON. Beverly Hills. RANDOLPH M. 791-808. FELDMAN. ed.. FRASER. and DONALD T. HARRY. LIGHT. JAMES D. DOWNS." Comparative Political Studies. Effects of School Vouchers in Dayton. "Inadvertent Sociological Theory. Case Study Research: Principles and Practices.. pp. Pressure Group Politics. . 1966. "Measuring Partisanship. PETERSON. 50." Social Forces." In Paul F. and HOWARD ROSENTHAL. LAING..: Prentice Hall. and CHARLES W. BUTLER. 179-196. N. 69 (December). and ALAN GERBER. 3 (February).. BRUCE. 469^185. rev. 1969. Lumsdaine. and JANE M. The Statistical Analysis of Quasi-experiments. New York: Academic Press. BRUNNER. Equal Money: Court-Ordered Redistricting and the Distribution of Public Expenditures in the American States. Chicago: University of Chicago Press." American Sociologist. dissertation in economics. 2003. LARRY M. RONALD D. "The Effects of Personal Canvassing. 1975. New York: Free Press. 2000. MORROW. JOHN. "An 'Intentional' Alternative in Public Opinion Research. 2nd ed.D. 49 (December). 1974. "Equal Votes. 2002. New York: McGraw-Hill. and MORTON BARATZ. 1993. HILDEBRAND. 2nd ed." American Political Science Review. 1961. WILLIAM G. Upper Saddle River. American Behavioral Scientists. "Studying Presidential Performance in Congress. Princeton." Journal of Politics. and JEAN KINGSTON. 1984. 87 (June).." Political Methodology. Calif. CAMPBELL. and DONALD STOKES. GREENSTEIN. DIEHL. CAMPBELL. 26 (Spring). ANSOLABEHERE. Introduction to Mathematical Sociology... PETER. 1957.5 3 . 1992. GERBER. WOLF.Selected Bibliography CAMPBELL. ." American Journal of Political Science. Washington. STANLEY. DONALD. and GEORGES DUPEUX. GOLDSMITH. Political Parties. Political Change in Britain. eds.: Sage. "Assessing Social Innovations: An Empirical Basis for Policy. 1 (January). 21(2). HAMMOND. and H. DAVID E. 9. R. SIVERSON." American Political Science Review. .J. N. 1975. 1960. and ALASTAIR SMITH. 1987. Calif. 1960. 1979. 801-813.J.. 1964. . JR. DONALD T. ROBERT." Public Opinion Quarterly. Political Oppositions in Western Democracies. 4th ed. 13. 1969." American Political Science Review. ANGUS. DUGGAN. COOK." Political Methodology. "The Fourth Face of Power. and Washington. and JAMES M. JAVIER ORTIZ. 4 0 . "Of Time and Partisan Stability.J. 1977.471^180.

New York: McGraw-Hill. Designing Social Inquiry: Scientific Inference in Qualitative Research. TINBERGEN. MIHAELESCU. 1203-1225. K E Y . 33. 344-347. 1969. ROBERT O. New York: Cambridge University Press.. 1968. CHRISTOPHER. eds. . "Cross-Level Inference as a Game Against Nature. 1993. . Mathematical Applications in Political Science. Mich. Parties and Party Systems. Evidence? Yes. Princeton. "Social and Economic Factors and Voter Registration in the South. "Critique of Ordinal Variables. ALVAREZ.Y. and CLYDE KLUCKHOHN. Princeton. 565-570. Conn. 2006. 341-357." American Political Science Review. . 1937." Political Methodology. 415^131. KNOKE. TILLEY. HANNA. Vol. ed. JACOB. KMENTA. MICHAEL E. 2003.. SHIVELY. "Making Social Science Work Across Space and Time: A Critical Reflection on Robert Putnam's Making Democracy Work" American Political Science Review. 3rd ed. L. 1994. 1976. The Concept of Representation. 48 (February). 1974. Charlottesville. . Presidential Power and the Modern Presidents. 46. 2nd ed. Calif. The Double Helix. PALFREY. "The Logic of Political Inquiry: A Synthesis of Opposed Perspectives. A. ed. pp. ROETHLISBERGER. Cambridge. Blalock. New York: Atheneum. T. 1983. Bernd. 1. 1955. 1968. Vol. A. The Language of Social Research. Politicians and Party Politics. 526-539.: The Museum. JAN W. The Structure of Scientific Revolutions. W. pp. and others. MILLER. Curious Naturalists. and FERNANDO LONGI. JR.. New Haven. Vol. REYNOLDS. ZALLER. W. PHILIP E. LEITNER. FELIX. and ALISSA POTTER M E E . "Party Identification in Britain: Does Length of Time in the Electorate Affect Strength of Partisanship?" British Journal of Political Science." In John Geer. N. 1967. TINGSTEN." International Encyclopedia of the Social Sciences. Making Democracy Work. 2002. 2004. Reliability and Validity in Qualitative Research. KIRK. pp. 18. London: Macmillan. Unobtrusive the Family: A Longitudinal Analysis." American Sociologist.J. pp. New York: Harper & Row. SIDNEY. 2002. 135-161. 1968. 1956. Mylai and the Volunteer Army. dissertation in political science. W I N C H . KROEBER. F. 1969. 4. Chicago: Aldine-Atherton. DAVID. Statistics for Social Data Analysis. 283-336. 1949." American Sociological Review." American Political Science Review.: Princeton University Press. Empirical. Va. 1986. 65 (June). 641-654. Representation. New York: Harcourt Brace Jovanovich.J. "Ecological Correlations and the Behavior of Individuals. and KEITH KREHBIEL. JAMES R. "Of Time and partisan Stability Revisited: Australia and New Zealand 1905-90. "The Language of Political Inquiry: Problems of Clarification. 1966. 2002. J. 2 (Spring). SARTORI. 1997. and DONALD T . RICHARD E.. . 1997. CAMPBELL. 30-50. 14. Southern Politics.: Sage. WILSON. V. J. and DONALD T. DONALD T.: Wadsworth. Vol. The Visual Display of Quantitative Information. IYENGAR. 1971.: Harvard University Press. SIDNEY. 140-143. 4th ed. Nonparametric Statistics for the Behavioral Sciences. KEOHANE. 1972. GARY. DICKSON. PROTHRO. 351-357. RICHARD D. Elements of Econometrics. New York: Macmillan. 2000.: Addison-Wesley. MIHAELA. The Chief Executive. JAMES W. 1963. and M A R C L. 1956. Oxford: Oxford University Press. 1984. New York: Random House.: Yale University Press. TARROW.. H. 1997. 1 1 0 4 . CAROLE JEAN. Power: A Radical View. Chicago: University of Chicago Press. 332-344. 5 4 .: University of Michigan Press. 1996. 102 (January).U .. "Case Selection: Insights from Rethinking Social Inquiry'. Baltimore: Johns Hopkins University Press. London: Cambridge University Press. Garden City. 57 (March). JAN. 1968. THOMAS P.. 115-147. and JAMES W. "Interesting but Irrelevant: Social Capital and the Saliency of Politics in Europe. 193-210. The Military Balance. PHILLIPS. Jr. 4 1 . "Improving Data Analysis in Political Science. 1939. 1975. Cambridge. London: Longmans. DONALD 1975. 1965. Cambridge. PETER. Calif. Democracy and Measures. "Institutionalization of Voting Patterns. SCHWARTZ.: The University of Virginia Press. FREDERICK. KINDER. 1982. 2006. Introduction to Models in the Social Sciences. 66 (December). Mass. NEUSTADT. Calif. 69 (March). CAMPBELL. 1977." In Fred Greenstein and Nelson Polsby. 6 2 . and L E E SECHRIST. 2 1 . 24ff. "The Urban-Rural Cleavage in Political Involvement: The Case of France. Beverly Hills. Cambridge. SAMUEL.. OLSON. ROBERT E . KUHN. JULIAN. and KAY LEHMAN SCHLOZMAN.: Addison-Wesley. "What's Driving M e x i c o .: University of California Press. JAMES D. 4 9 .. 1952. DOUGLAS S„ and KRISTIN E. "Candidate Gender and Congressional Campaign Receipts. Experimental and Quasi- Ann Arbor. STEVEN. STOKES. Calif. EDWARD R. Belmont. GIOVANNI." European Journal of Political Science. UHLANER.' Political Analysis. S . Lazarsfeld and Morris Rosenberg. Handbook of Political Science.: Harvard University Press.5 7 . M A R C H . SIEGEL. Mass. Conn. Management and the Worker. "Interethnic Cooperation in Post-Communist Eastern Europe. 939-999. 1950-1990. 1971. HERBERT. "A Multi-dimensional Classification of Atrocity Stories. Mass. News That Matters. 1998. Chicago: University of Chicago Press. ed. Mass. CAMPBELL. 389-398. KOESTLER. DONALD R. MASSEY. Causal Models in the Social Sciences. ROBINSON. DONALD E." Journal of Politics." In Paul F. KARSTEN. 1993. ROBERT D. JOHN. 1971." Journal of Politics. ROGOWSKI. London: Hutchinson. "Party Identification. New York: Alfred A.. Culture: A Critical Review of Concepts and Definitions. "Majoritarian Electoral Systems and Consumer Power: Price-Level Evidence from the OECD Countries. Migration? A Theoretical. RICHARD G. Chicago: Rand McNally." American Political Science Review. 1969. "ROTC. The Act of Creation. University of Minnesota. Berkeley. 1985. pp. expected completion 2005. 1962. ADAM. and others. SHANTO.6 7 . ARTHUR. RONALD. J. MOSTELLER. eds." American Journal of Political Science.8 3 . SIMON. STOUFFER. LAVE. V A N D E T H .. Political Behavior." Foreign Policy. MANCUR.D. and THOMAS R. New York: Atherton. New York: Free Press.: Doubleday. "Errors. JOSE ANTONIO CHEIBUB. M O O N .. 1985." American Journal of Sociology. 4 (May). and JAMES G. and M A R K ANDREAS KAYSER. 131-228." American Journal of Political Science. LUKES. eds. Party Choice. KINDER. or Is Mobilization the Source of Decay?" American Political Science Review." In Joseph L. C O O K . Mass. The American Soldier. PROTHRO. 1975. Reading. 1. 15 (June). MATTHEWS. PITKIN. 1. PRZEWORSKI. 2000. ed." In Fred Greenstein and Nelson Polsby. The Significance of Tests of Significance.. CHARLES. Beverly Hills. rev. . BOHRNSTEDT. DONALD R. O. 1950." World Politics.. The Rise and Decline of Nations. The Logic of Collective Action. EUGENE J. 10. New York: Free Press. 1991. LOUIS W. THOMAS S. KOENIG. N. WATSON. Reading. 1968-1969. 2006. TUFTE. KING. eds.: Sage. and Policy Analysis. W E B B . Experimental Foundations of Political Science. 1969. Development: Political Institutions and Well-Being in the World. "Proof? No. Basic Research Methods in Social Science. JEROME. Mass. "The Nonsense Fight over Scientific Method: A Plea for Peace. PUTNAM. 1975. GEORGE W. Boston: Houghton Mifflin. Knopf. and DONALD R. THOMAS D..1 1 2 7 . 90 (June). N.172 Selected Bibliography Selected Bibliography 173 INTERNATIONAL INSTITUTE FOR STRATEGIC STUDIES. . and SIDNEY VERBA. S. NIEMI. SHADISH. Cheshire. 1987. NIKO." Ph. Analysis of Nominal Data." In Hubert M. "Politicians as Prize Fighters: Electoral Selection and Incumbency Advantage. "The Quality of Surveying Responses About Parents and Experimental Designs for Generalized Causal Inference. ESPINOSA. WILLIAM R. and Voting Stability: The Weimar Case.: Harvard University Press.: Graphics Press. Handbook of Political Science. OPPENHEIM.: Princeton University Press. 1950. Ann Arbor: University of Michigan Press. and W.

5-6 Duggan. Barbara.. 167-168 Continuous measures.. 40 Johnson. Maurice. 30 G o m n Leo A. 54n Baratz. Robert O. 8 Digeser. 87n. 17-20. 38. 62 Iyengar. 7n Abstinence-only sex education. 135n od a . 56 a mn . 40 Alvarez. David. Arthur A. 20 Dean. 96 Gerring. Angus.. 10 Interrupted time series. 40 Batalla... 164 Dependent variable. 2-3. G. 81 Achen.. Charles W...162 Boswell.. 124 Coleman.. 87. 113 Elegant research. Peter. 131. 14 Interaction.Index 175 D Dahl. 103-106 Cheibub.. John.. Jr. 146n Intercept of regression equation. 29-30 Ansolabehere. 119-120 Discrete measures. 127n. Christopher H. John. 23 Extrapolation. 64 Downs. 16 Ethics of research. 47. 161-162 B Gerber. 54-55 Applied research. 77-78 eliminating alternative. 85n. John P. 73 Case studies. W. James. George W... 122-127 compared with regression analysis. 38 Baker v. 54-55. 82 designs without. 96 Greenstein. 57 Ecological fallacy. 88. Harry. Samuel. Jane M. Lewis F.. Stanley. 167 Brunner. Peter. 96 . V. 167n Holding a variable constant. James David. 94-95 Howell.. O.. Robert. true. 29 H H m o d T o a H. 23 English language. 20 174 Campbell. 96. James S. 56 Autocorrelation. 110. 164 Carter. 161n Key. 137-138 Dickson. Jose Antonio. Peter.. 126. 15. Lord. 161-162 Independent variable. 69 Kayser. 87 Goldsmith.. 96 hms Correlational measures of relationship. Russell J. 86 ak Kemeny.. William G„ 87 Hypothesis testing. 5-8 om l Fraser. David E„ 87 Campbell. 4 Engineering research. 29 Bartels. Philip E. 121-122 Kinder. 79-81 Controlling for a variable. Stephen. 80. 94. T o a D„ 85. 131 ar Barton. 16-17 Empirical research. 80n Diehl. 127-130 Correlation coefficient. 31 Bueno de Mesquita. T o a J. 145 Cook. 87. 5. 151-153 I Bachrach. 11-12 Experiment. 40. 47. 167 K Galileo.. 110 Gilbert. 26-27 Butler. statistical. 101 Coefficient of determination.. 56. h m s Hawthorne study. Birkbeck. 52 Feldman. Kristin E. 14 Dichotomies. 94-95 Converse. 110 Analytic political philosophy. 94. 19 Keohane. 34 Epistemic correlation. 164 hms Duverger. 88 J Jacob. Donald T. 76. 32-33. 123 Cross-tabulation. 78-84 Causation. Fred I. Mr Andreas. 40 Dalton. 146n Bohrnstedt. 145-146. Alan S.. 90-91 Interval measures. 56 F r a theory. Donald R. 76 Bernstein. 56 Free rider problem. 83-84 Ex post facto argument. Bruce. Philip E. 113 Correlation analysis. 4 Asher. Donald P. Michael E. 132-134 E Eckstein. 18-19 Campbell. 14. 80n Hill. 38 Diminishing marginal returns model. J. 46n. 40 Blalock. Green. Anthony. 128n Acton. 103 Causal interpretations. Paul E. 7 G Independence of observations. L ry M. G o m n a d Kruskal g m a 135 od a n a m. 56. Carr. 87. Allan H. 74-76 Censored data. 96.. 167-168 Geddes. 38 Barber.. Javier Ortiz.. 128n Effect-descriptive measures of relationship. 45n Espinosa. Herbert B. 104-105 Karsten. 40 Anomalies. Morton. 116 Interesting problems. Hubert M„ 96. 136-137 in regression analysis. Shanto. Ronald D.. Richard J.. 49 F Face validity. 110 Cluster sample... 63-64 Control group.

99-102 purposive. 49-50 Research design. 61. 110 s Sampling. 19 Marx. 58-67 and describing relationships between variables. 168 Q N N (number of cases). 83 Neustadt. Ronald. Mihaela. 31 Krehbiel.5 Null hypothesis.. 3 1 . 149 Significance tests. 83-84 Tufte. 56 u Uhlaner. 82 Test-retest check for reliability. 53-54. Hanna. 139-141 Prothro.176 Index Multidimensional concepts. J. 123-127 r . 139-141 Longi. 20. 87 Pierce. 20. A l i s s a Potter. . 73 Rational choice. Clyde. 25 Natural experiment. 110n. 15. Adam. 163 Power. 141-146 Putnam. 87 Multicollinearity. 131. 4 Nonrandom error. 58. 34-35 Multivariate regression. 94. Arthur. 2. 8. 110. L e e . . Thomas S . Giovanni. 101 National integration. 62 127-130 and measurement error. 26n. 26n L a w of Large Numbers. 38 P M March. 71. Douglas S . 33 Light. Samuel. 45^18 related to validity. Herbert. 60 L Lave. 165 Simplicity. Louis W. 85n Statistics.4 2 Split-half test for reliability. F. 109-110 Logit analysis. 15 R r. K a r l . 104 Mill. 62-67 in measures. 33 Polling. 130-131 checking for. 56 Kluckhohn... 8 Massey. 80n Rogowski. 162-164 Simon. Jan. I28n 111 King. Keith. 29. Ronald. Sidney. 101-102 random. 50-51 Loewenberg. 16 Matthews. 56. 31 Mosteller.. 51. general problem of. 38 Kruskal. 88-90 Relative deprivation.. 92-93 Puzzles. 5 Miller. . 102 quasi-random. 7 3 o Olson. James R. Roy. statistical.. Christopher. 15 True experiment. 65-67 enrichment of. 25-28 Index Selection on the dependent variable 106-108 Shadish. 9 Stratified sample. of a theory. 56 Minneapolis Star Tribune. 2 Slope of regression equation. David. 130-131 Mee. Mancur. 42^15 Measurement error. 6 R D D sampling. 128n Stouffer. 20-21 Siegel. 78-84 Residuals. 106 Leitner. 82-83 without premeasurement. 165 Significance. 135n Kuhn. 96 Peterson. Jean. Carole Jean. Jerome. Julian C . Frederick. 118-119 Niemi. . Felix. 63n. 20. Paul E . 96 Shively. Richard J . S . A. 5-8 Regression analysis. 7 Political party. 51-53 Variables. 161 n Palfrey. 113-114 Schlozman. 117 V Validity. 110 Lukes. . Phillips. James W. 6-7. 144n Przeworski.3 Social status. 99. 96 Tingsten. 99-101 R D D . 52 Tsst group. 4 1 . 23 Tilley. Gary. . 99-101 Ratio measures. 86 Unidimensional concepts. 86 Literary Digest. Steven. 100. 40 Ordinal measures. 128n Roethlisberger. Gerhard. 53-54 Normative philosophy. Donald E . 113-122 compared with correlation. . 49. . 56 Kroeber. 117 Regression to the mean.. 56 Sechrist. 162 Mihaelescu. 96 Topics for research. 79 Recreational research.. Donald. 144n Measurement. 47 Stanley. 47 Theory.. 101 Reagan. 20 Tinbergen. 14-17 Theory-oriented research. L . 37 N A T O . 32 Multidimensional words. 124. Niko. William H. 40 Pluralism. Richard G . 32 Units. William R. J . . Thomas R„ 87n. 4 . 94. comparability in regression analysis. 20 Liberal. James M. Julian. Charles A . 158-161 National Election Study. 29 Quantitative research. . Donald R. . .5 2 Pitkin. Sidney. 24.. 9 Reliability of measures. 26n Markov chain. 30 Oppenheim. 38 Kmenta. 56 Pilot study. 126-127 Random error. 155-158 Sartori. 54-55 Social research. 56 Nominal measures. 101 Sampling distribution. K a y Lehman. 152-153 T Tarrow. 87 Linear relationships. 38-40 Precision. 112 Stokes. 116 Snyder. Marc L . 130-131 Regression coefficient. W. 162 Koenig. 5 1 . . 86 Schwartz. 162 Knoke. Robert D. 29 Koestler. Richard D. 62 Nonempirical research. John Stuart. 40 Scattergram. in regression. 22-23 Trivial theory. 130-131 Random sample. 8 Kirk. 3. 49. Richard E . W. Fernando. 67-71 in measurement. 4. 161n Kingston. 2 . 58-61 Probit analysis. 131. 14 . James G . 48^19. 119-122 Robinson. 81 Moon. Edward R.

William Knowlton. 31 . . Patrick J . 20 Wilson. . Thomas P.. 87 Z a l l e r .. 164 Wolf.178 Index X X\ 153-155 Variance. 56 Weldon. 16In w Watson. Sidney. 124-127 Verba. Robert E. 73 Winch. 98 Zinsser. John. Eugene J . 110. Steven. James D. 1 Webb.

Sign up to vote on this title
UsefulNot useful