Seventh Edition

W. Phillips Shively
University of Minnesota

PEARSON

Upper Saddle River, NJ 07458

Library of Congress Cataloging-in-Publication Data Shively.W. Phillips, 1942The craft of political research / W. Phillips Shively. — 7th ed. p. cm. Includes bibliographical references and index. ISBN-13: 978-0-13-602948-9 (alk. paper) ISBN-10: 0-13-602948-5 (alk. paper) 1. Political science—Methodology. JA71.S45 2009 320.072—dc22 2007047170 Executive Editor: Dickson Musslewhite Editor-in-Chief: Yolanda de Rooy Associate Editor: Rob De George Editorial Assistant: Synamin Ballatt Senior Marketing Manager: Kate Mitchell Marketing Assistant: Jennifer Lang Managing Editor: Joanne Riker Manufacturing Buyer: Wanda Rockwell Creative Director: Christy Mahon Cover Design Director: Jayne Conte Cover Design: Jon Boylan Cover Illustration/Photo: Getty Images, Inc. Composition: Integra Full-Service Project Management: Integra Printer/Binder: Courier-Stoughton Credits and acknowledgments borrowed from other sources and reproduced, with permission, in this textbook appear on appropriate page within text. 2. Political science—Research. I. Title.

To

BARBARA

Copyright © 2009, 2005, 2 0 0 2 , 1 9 9 8 , 1 9 9 0 , 1 9 8 0 by Pearson Education, Inc., Upper Saddle River, New Jersey, 07458. All rights reserved. Printed in the United States of America. This publication is protected by Copyright and permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permission(s), write to: Rights and Permissions Department. Pearson Prentice Hall™ is a trademark of Pearson Education, Inc. Pearson® is a registered trademark of Pearson pic Prentice Hall® is a registered trademark of Pearson Education, Inc. Pearson Pearson Pearson Pearson Education Ltd. Education Singapore, Pte. Ltd. Education, Canada, Ltd. Education-Japan Pearson Pearson Pearson Pearson Education Australia PTY, Ltd. Education North Asia Ltd. Educacion de Mexico, S.A. de C.V. Education Malaysia, Pte. Ltd.

ISBN-13: ISBN-ID:

c

10 9 8 7 6 5 4 3 2 1 l7fl-D-13-bD5T4fl- 1 D-ig-bDSTMfl-S
c

L

Preface

xi 1
2

Doing Research
Social Research

Types of Political Research Research Mix 8

4

Evaluating Different Types of Research
E t h i c s of P o l i t i c a l R e s e a r c h 11

10

Political Theories and Research Topics
C a u s a l i t y and P o l i t i c a l T h e o r y 14 15

What Does Good Theory L o o k L i k e ?

Example of Elegant Research: Philip Converse To Quantify or Not
C h o i c e of a T o p i c 22

20

Engineering Research Theory-Oriented Research

22 23
23

D e v e l o p m e n t of a R e s e a r c h D e s i g n

Observations, Puzzles, and the Construction of Theories 25

Contents Machiavellian Guide to Developing R e s e a r c h Topics 28 31 Contents A F e w B a s i c s of Research Design 78 ix Designs Without a Control Group Use of a Control Group 82 79 Further D i s c u s s i o n Importance of Dimensional Thinking E n g l i s h as a L a n g u a g e for R e s e a r c h 33 32 True Experiment 83 84 D e s i g n s for P o l i t i c a l R e s e a r c h Ordinary Language 34 37 Special Design Problem for Policy Analysis: Regression to the Mean 88 U s e o f V a r i e d D e s i g n s and M e a s u r e s 90 Proper U s e of Multidimensional Words Example of Dimensional 40 Analysis 38 Example of Varied Designs and Measures Conclusion 93 94 92 Further D i s c u s s i o n Problems of Measurement: Accuracy Reliability 45 41 46 Holding a Variable Constant Further D i s c u s s i o n 96 Reliability as a Characteristic of Concepts Testing the Reliability of a Measure Validity 48 47 7 Selection of Observations for Study Random Sampling 99 101 102 103 97 99 S a m p l i n g f r o m a Population of Potential O b s e r v a t i o n s Some Examples Checks for Validity 49 51 53 Quasi-random Sampling Purposive Sampling I m p a c t o f R a n d o m and N o n r a n d o m E r r o r s Importance of A c c u r a c y Further D i s c u s s i o n 56 54 Selection of Cases for Case Studies Censored Data 103 When Scholars Pick the Cases They 're Interested In 104 Problems of Measurement: Precision P r e c i s i o n in M e a s u r e s Precision in Measurement 58 62 57 When Nature Censors Data 106 S e l e c t i o n A l o n g the D e p e n d e n t V a r i a b l e : D o n ' t D o It! 106 108 S e l e c t i o n o f C a s e s for C a s e Studies ( A g a i n ) Innate Nature of Levels of Measurement The Sin of Wasting Information 64 63 A n o t h e r Strategy. H o w e v e r : S i n g l e .C a s e Studies S e l e c t e d for the R e l a t i o n s h i p B e t w e e n the Independent and D e p e n d e n t V a r i a b l e s Further D i s c u s s i o n 110 109 Enrichment of the Level of Precision in Measurement Examples of Enrichment 67 72 Quantifiers and Nonquantifiers Again Further D i s c u s s i o n 73 8 74 Causal Thinking and Design of Research C a u s a l i t y : A n Interpretation 75 Introduction to Statistics: Measuring Relationships for Interval Data 111 Statistics 112 112 E l i m i n a t i o n o f A l t e r n a t i v e C a u s a l Interpretations 76 Importance of L e v e l s of Measurement W o r k i n g w i t h Interval D a t a 113 Summary 77 .

under prodding f r o m r e v i e w e r s . a return to old l o v e ) . I had f o u n d that the students benefited f r o m an introduction that e m p h a s i z e d the internal l o g i c of r e s e a r c h methods and the c o l l e c t i v e . I have m a d e a n u m b e r of additions and deletions over the last c o u p l e of editions to Significance Test: A l w a y s N e c e s s a r y ? P o l l i n g and S i g n i f i c a n c e T e s t s 163 reflect n e w p o s s i b i l i t i e s in technique. perhaps. I c o u l d not find a b o o k that presented things in this w a y at a sufficiently elementary l e v e l to be r e a d i l y a c c e s s i b l e by undergraduates. I h a v e i m p r o v e d on my treatment of the use of n o m i n a l variables in r e g r e s s i o n a n a l y s i s . P u t n a m ' s b o o k a n d the resultant r e s e a r c h p r o g r a m on s o c i a l c a p i t a l are of c o u r s e splendid. It has a l w a y s s e e m e d to me that it fills a needed n i c h e . and has g i v e n me enormous p l e a s u r e . I must a l s o admit to a recent c h a n g e of heart (or. w h e n I w a s an assistant p r o f e s s o r at Y a l e U n i v e r s i t y . A n d so I wrote this book. but I a l w a y s regretted s a c r i f i c i n g C o n v e r s e ' s article. In teaching a n u m b e r of sections of Introduction to R e s e a r c h to undergraduates there. In this seventh edition. and it h a s been a thrill w h e n students have told me that they have benefited f r o m it. Null Hypothesis Example: % 2 152 153 155 158 Independent Observations 161 162 Sampling Distribution Importance of'N Problem of It has f o l l o w e d me through the rest of my career so far. cooperative nature of the r e s e a r c h p r o c e s s . It is still the most xi . I c h a n g e d m y e x a m p l e o f elegant r e s e a r c h f r o m P h i l i p C o n v e r s e ' s "Of Time and Partisan Stability" to R o b e r t P u t n a m ' s Making Democracy Work.Contents Regression Analysis Correlation Analysis 113 122 127 130 Correlation and Regression Compared Problem of Measurement 131 Error Further D i s c u s s i o n Introduction to Statistics: Further Topics on Measurement of Relationships 132 M e a s u r e s o f R e l a t i o n s h i p for O r d i n a l D a t a M e a s u r e s o f R e l a t i o n s h i p for N o m i n a l D a t a 132 136 Dichotomies and Regression Analysis L o g i t and Probit A n a l y s i s Multivariate A n a l y s i s Conclusion 146 141 139 136 Introduction to Statistics: Inference. or How to Gamble on Your Research 148 L o g i c of Measuring Significance E x a m p l e o f Statistical I n f e r e n c e Hypothesis Testing 151 149 150 I wrote this little b o o k in 1970. I have also c h a n g e d significantly the position I take on the question of selecting c a s e s for a n a l y s i s . I am pleased that it still s e e m s to be w o r k i n g for them. W h i l e the general p r i n c i p l e s of good argument and investigation don't c h a n g e . and I have added a n e w d i s c u s s i o n of the p r o b l e m of c o m p a r i n g U s e s and L i m i t a t i o n s of Statistical T e s t s Conclusion 164 164 units in r e g r e s s i o n a n a l y s i s . Further D i s c u s s i o n Where Do Theories Come From? Selected Bibliography Index 174 170 166 R e g u l a r users of this text m a y r e c a l l that a c o u p l e of editions a g o . I have updated a n u m 163 ber of e x a m p l e s .

m a y not see it this way. and s o o n . the s a d fact is that w r i t i n g papers for c o u r s e s is too often something of a drag. A n d so I returned in the sixth e d i t i o n — a n d continue i n this seventh e d i t i o n — t o " O f T i m e and P a r t i s a n Stability. U n i v e r s i t y o f C e n t r a l F l o r i d a . this is a book for w h i c h I have great affection. The Double Helix (New York: Atheneum. the approval of others. and then. Phillips Shively S c h o l a r l y r e s e a r c h is e x c i t i n g and is f u n to do. c o u r s e papers are tied to all sorts of r e w a r d s and p u n i s h m e n t s — y o u r future e a r n i n g s .xii Preface Chapter I beautiful p i e c e of p o l i t i c a l s c i e n c e r e s e a r c h I know. I enjoyed Francis's words. 1 . A l t h o u g h r e s e a r c h c a n be e x c i t i n g in this w a y . S o m e students. indirectly. Constantly he would pop up from his chair. h i s account of their w o r k . The first afternoon following the discovery that A . the period of momentary uncertainty over." and i n c l u d e w i t h it the s a m e e x p l i c a t i o n of M a r k o v c h a i n s that I u s e d in those earlier editions. U n i v e r s i t y o f C a l i f o r n i a a t S a n t a B a r b a r a . but his effort was ineffectual. caught in the grind of daily and term a s s i g n m e n t s . worriedly look at the cardboard models. Francis's preoccupation with D N A quickly became full-time. I hope y o u w i l l e n j o y it as m u c h as I have e n j o y e d it. for professors or for students w h o c a n involve t h e m s e l v e s in a l o n g .T and G . F i r s t of a l l . p. to lodge on the paper. B u t for people w h o c a n carry on r e s e a r c h in a m o r e r e l a x e d w a y . even though they lacked the casual sense of understatement known to be the correct way to behave in Cambridge. ACKNOWLEDGMENTS T h a n k s to the f o l l o w i n g r e v i e w e r s for their helpful suggestions and c o m m e n t s : Terri S u s a n F i n e . W a t s o n d e s c r i b e s his and F r a n c i s C r i c k ' s s e a r c h for the structure of the D N A m o l e c u l e . The Double Helix. A l l o f the anxiety a s s o c i a t e d w i t h these vulnerabilities c o m e s . A f t e r a l l . Garrett G l a s g o w . and P a t r i c k J a m e s . It is m o r e gripping than most mystery novels. r e s e a r c h m a y be a source of fascination and great satisfaction. 198) 1 T h i s i s the w a y J a m e s D . W. 1968).r a n g e project. gives a good picture of the excitement of r e s e a r c h . (Watson. look satisfied and tell me how important our work was. fiddle with other combinations. James D. 1968. As y o u c a n no doubt tell f r o m the tone of this preface. he went back to his thesis measurements. U n i v e r s i t y o f Southern C a l i f o r n i a . Y e t this is probably a l e s s e r c a u s e for frustration in student r e s e a r c h .C base pairs had similar shapes. e a c h of these anxieties m a y a l s o be present for 'Reprinted with permission from Watson.

D u v e r g e r d e v i s e d the theory that (1) if s o c i a l c o n f l i c t s o v e r l a p . as a practice r u n . and of h o w important i n v e n t i v e n e s s . I a l s o hope to m a k e y o u a w a r e of w h a t a c h a l l e n g i n g g a m e it c a n be. In a b o o k s u c h as this. A theory takes a set of s i m i l a r things that h a p p e n — s a y . D u v e r g e r h a d to forget about m a n y other potentially interesting things. the country w i l l develop a multiparty s y s t e m . and the n o n . H i s i d e a w a s that w h e r e there is m o r e than one sort of political c o n f l i c t going on s i m u l t a n e o u s l y in a country. T h e n . Instead of h a v i n g to think about a large n u m b e r of disparate h a p p e n i n g s . for e x a m p l e . there w o u l d be four distinct political positions in the country: the C a t h o l i c w o r k e r position. H a v i n g formulated this theory. w i t h v a r y i n g n u m b e r s and types o f parties present at different times in their histories.2 Chapter 1 Doing Research 3 p r o f e s s i o n a l s c h o l a r s . e c o n o m i c development p r o g r a m s s u c c e e d i n s o m e c o u n t r i e s . for instance. going through the motions of s c h o l a r s h i p . D u v e r g e r no longer h a d to concern h i m s e l f s i m u l - S o c i a l r e s e a r c h is an attempt by s o c i a l scientists to develop and sharpen theories that give us a handle on the universe. W h y w a s this s o ? D u v e r g e r might have cast around to find an explanation for the exception to his theory. I c a n o n l y give my o w n testimony. As y o u c a n see.C a t h o l i c m i d d l e . we n e e d only think of a single pattern w i t h s o m e variations. S u p p o s e . T h i s requirement w o u l d f o r c e s o m e of the distinct groups to c o m p r o m i s e their positions and merge into larger parties that w o u l d h a v e a better c h a n c e of w i n n i n g e l e c t i o n s . not e n o u g h time is a l l o w e d for the student to think l o n g and s e r i o u s l y about the subject. W i t h o u t theories. it a l s o generally requires that we both narrow the range of reality we l o o k at and o v e r s i m p l i f y e v e n the portion of reality that falls w i t h i n that n a r r o w e d range. S u c h a p r o c e s s of c o n s o l i d a t i o n l o g i c a l l y w o u l d c u l m i n a t e in a two-party s y s t e m . F o r e x a m p l e . T h e appropriate n u m b e r o f parties w o u l d then tend to r i s e . as one for w h o m r e s e a r c h is v e r y e x c i t i n g . of w h i c h t i l those party systems were examples. F o r e x a m p l e . others are not. but this attitude toward the r e s e a r c h w o r k cheats them of the p l e a s u r e and excitement that r e s e a r c h c a n b r i n g . H o w e v e r . there u s u a l l y is a feeling on both sides that this is " j u s t a student p a p e r " — t h a t it d o e s n ' t r e a l l y matter h o w g o o d it i s .c l a s s position. To have any hope of controlling w h a t h a p p e n s . w h e n faced with exceptions s u c h as these.C a t h o l i c s w e r e w o r k e r s and s o m e w e r e m i d d l e c l a s s . He needed to think only about a single developmental p r o c e s s . and w h e r e the groups of people i n v o l v e d in these taneously with a great number of idiosyncratic party systems. of votes in a district. e s p e c i a l l y w i t h other papers c o m p e t i n g for attention. To s u m m a r i z e the theory: A country w i l l develop a two-party s y s t e m (1) if there are only two distinct p o l i t i c a l positions in the country. Instead. we a l w a y s have to strike a b a l a n c e b e t w e e n the s i m p l i c i t y of a theory and the n u m b e r of exceptions we are w i l l i n g to tolerate.c l a s s p o s i t i o n . T h e r e is probably no w a y out of this d i l e m m a . we m u s t understand w h y these things h a p p e n . U s u a l l y . O t h e r w i s e . s o m e t i m e s it does not. a c o n f l i c t b e t w e e n w o r k e r s and the m i d d l e c l a s s might o c c u r at the s a m e time as a c o n f l i c t b e t w e e n C a t h o l i c s and n o n . and (2) if the electoral s y s t e m of the country does not p e n a l i z e s m a l l parties. D u v e r g e r thought that this tendency c o u l d be s h o r t . too. the n o n . the electoral l a w forces people of d i v e r s e positions to c o n s o l i d a t e into two large political parties so as SOCIAL RESEARCH to gain an electoral advantage. that D u v e r g e r restricted h i m s e l f in more than j u s t his c h o i c e of a theme: He c h o s e deliberately to play d o w n exceptions to his theory. B y restricting his attention to the n u m b e r of parties c o m p e t i n g in the s y s t e m . A n d e v e n w h e n adequate time is a l l o w e d . rather than a plurality. and he c o u l d have then incorporated that explanation into the original theory to produce a larger theory. he c h o s e to accept them as accidents. s o m e b i l l s are p a s s e d by C o n g r e s s . S o m e t h i n g i s a l w a y s lost w h e n w e s i m p l i f y reality i n this way. if these groups overlapped s o that s o m e o f the C a t h o l i c s w e r e w o r k e r s and s o m e w e r e m i d d l e c l a s s . A more important r e a s o n for the student's l a c k of e n t h u s i a s m is the s i m p l e fact that a paper is g e n e r a l l y regarded. of the f e e l i n g of creating s o m e t h i n g that no one ever s a w before. the development of party s y s t e m s in d e m o c r a c i e s — a n d finds a c o m m o n pattern a m o n g them that a l l o w s us to treat e a c h of these different o c c u r r e n c e s as a repeated e x a m p l e of the s a m e thing. there w i l l be m o r e than two distinct p o l i t i c a l positions in the c o u n try. M a u r i c e D u v e r g e r w a s c o n c e r n e d w i t h the question of w h y s o m e countries develop two-party s y s t e m s and others develop multiparty s y s t e m s ( 1 9 6 3 . As theorists. or (2) if despite the p r e s e n c e of m o r e than two distinct p o l i t i c a l p o s i t i o n s . we are f a c e d w i t h the unreadable c h a o s of reality. B u t I c a n introduce y o u to s o m e selected p r o b l e m s y o u s h o u l d be a w a r e of if y o u want to do good r e s e a r c h y o u r s e l f or to evaluate the w o r k of others. T h e initial reality w a s c h a o t i c . 2 0 6 . in others there are not. although these exceptions might have provided interesting additional information. s u c h as whether any one of the parties w a s revolutionary. the country w i l l develop a two-party s y s t e m . that a student w i l l learn f r o m d o i n g the thing w r o n g . w i t h one party c o r r e s p o n d i n g to e a c h distinct p o s i t i o n . s o m e t i m e s w a r c o m e s . pp. o t h e r w i s e . c o n f l i c t s o v e r l a p . and b o l d n e s s are to g o o d r e s e a r c h . but fail i n others. It w a s necessary for h i m to do this in order to keep the theory simple and to the point.C a t h o l i c s . originality. B e c a u s e the theory s i m p l i f i e s reality for u s . S o c i a l scientists c a r r y out this s i m p l i f i c a t i o n by d e v e l o p i n g theories. We do not r e a l l y h a v e any c h o i c e . w e must s i m p l i f y our perceptions o f reality. Note. or h o w m a n y of the parties h a d any c h a n c e of getting a majority of the votes. I cannot give y o u the d r a m a and excitement of o r i g i n a l r e s e a r c h .c i r c u i t e d if the electoral s y s t e m w e r e set up in s u c h a w a y as to p e n a l i z e s m a l l p a r t i e s — b y requiring that a candidate have a majority. in s o m e elections there are m a j o r shifts. it might have grown as c o m p l e x as the reality that it sought to simplify. the C a t h o l i c m i d d l e .2 8 0 ) .C a t h o l i c w o r k e r p o s i t i o n . that a country for w h i c h his theory had predicted a two-party s y s t e m developed a multiparty s y s t e m instead. w h i l e s o m e o f the n o n . s c o r e s o f countries w e r e i n v o l v e d . for instance. by both teacher and student. A n d t o have any h o p e o f understanding w h y they h a p p e n . i n his book o n political parties. Students must have the c h a n c e to learn f r o m their o w n m i s t a k e s . R e a l i t y unrefined by theory is too c h a o t i c for us to absorb. S o m e people vote and others do not. there are costs in setting up a theory. .

a n d derive theories f r o m them. a n d (2) d e m o c r a t i c government. we must look largely outside a c a d e m i c life to such sources as the R A N D C o r p o r a t i o n a n d the Weekly Standard. A y n R a n d . a n d (3) p o l i t i c a l events are not perfectly predictable. S o c i a l theory. It t y p i c a l l y takes certain p o l i t i c a l facts as given and c o m b i n e s t h e m w i t h m o r a l arguments to p r e s c r i b e p o l i t i c a l action. it is "recreation. R e s e a r c h m a y be c l a s s i f i e d a c c o r d i n g to these two criteria. It is a l s o n o n e m p i r i c a l in that it does not c o n s i s t p r i m a r i l y of investigating matters of fact. may be thought of as applied versus basic research. Like normative philosophy. then. c o n sists o f a l l theories that have been d e v e l o p e d to e x p l a i n political behavior. a w o r k by w h i c h m a n y w o u l d date the e m e r g e n c e of f o r m a l theory as a distinct field in p o l i t i c a l s c i e n c e — i s A n t h o n y Downs' An Economic Theory of Democracy (1957). to i m p r o v e p o l i t i c a l theory. S o m e e x a m p l e s w o u l d be m e a s u r i n g the effects of various reapportionment m e t h o d s . K a r l M a r x . without c o m m e n t ." Formal theory. T h i s m e a n s that it is not intended so m u c h to develop p o l i t i c a l theory as to u s e what political theory tells us about society and p o l i t i c s as a b a s i s for m a k i n g p o l i t i c a l d e c i s i o n s . I m a g i n e a p h y s i c i s t — o r a fruit p i c k e r for that m a t t e r — operating in the a b s e n c e of theory. L i k e normative p h i l o s o p h e r s . or interpreting. w i l l do this (factual a s s u m p t i o n ) . A c t u a l l y . and (3) democratic governments tend to redistribute income. P r o b a b l y the oldest f o r m of p o l i t i c a l r e s e a r c h . T h e two m a i n w a y s b y w h i c h t o d i s t i n g u i s h o n e p i e c e o f r e s e a r c h f r o m another are: political s c i e n c e . At the other e n d of the c o n t i n u u m f r o m applied r e s e a r c h is recreational 1. A g l a n c e at T a b l e 1-1 s h o w s us the four types of p o l i t i c a l r e s e a r c h b a s e d on different combinations of these two dimensions. T h e s e two forms of applied r e s e a r c h exist in s o m e estrangement f r o m a c a d e m i c Types of Political R e s e a r c h T h e w a y a particular p o l i t i c a l scientist c o n d u c t s r e s e a r c h w i l l depend both o n the u s e s that she v i s u a l i z e s for the project a n d on the w a y she m a r s h a l s e v i d e n c e . Without theories. its goal is p r o b l e m s o l v i n g . engineering research is geared to s o l v i n g prob- l e m s . a n d d e s i g n i n g methods o f riot c o n t r o l . (2) p o l i t i c a l c o n f l i c t o c c u r s on o n l y one i s s u e at a t i m e ." Normative philosophy is taught extensively. or it may be carried on largely for its own sake. based on the uses for which research is designed. It is a p p l i e d r e s e a r c h . l a r g e l y a p o s t . Research may be directed toward providing the answer to a particular problem. they posit facts as e m p i r i c a l conditions rather than as the foundation for m o r a l arguments. it is c o n c e r n e d w i t h a s c e r t a i n i n g the facts needed to s o l v e p o l i t i c a l p r o b l e m s . f o r m a l theorists posit certain facts about p o l i t i c s . Political engineering is a thriving industry a n d m a n y courses relevant to it are taught in political s c i e n c e departments. or a s s u m p t i o n s . political research can be characterized r e s e a r c h . is the m o s t r e c e n t l y by the extent to which it seeks to provide new factual information (empirical versus nonempirical). a subset of s o c i a l theory. Research may also be intended primarily to discover new facts. A good e x a m p l e T A B L E 1-1 Types of Political R e s e a r c h Applied Recreational Formal theory Theory-oriented research theory f r o m a set of a s s u m p t i o n s s u c h as: (1) voters a n d parties b e h a v e rationally. in w h i c h he urges the adoption of d e m o c r a t i c representative g o v e r n m e n t b e c a u s e (1) the c h i e f end of g o v e r n m e n t s h o u l d be to facilitate the d e v e l o p m e n t in e a c h citizen of h i s full potential ( m o r a l a r g u m e n t ) . students of society are trapped. A l l she c o u l d do if she s a w an apple falling f r o m a tree w o u l d be to d u c k . This distinction. P a u l K r u g m a n . (2) it m a y be rational for the voter to r e m a i n uninformed. by giving the people responsibility. . but in contrast to normative p h i l o s o p h e r s . S o c i a l scientists m e r e l y interpret reality in a m o r e s y s t e m a t i c a n d explicit way.W o r l d W a r II p h e n o m e n o n . It is u s u a l l y c a l l e d " p u r e " or " b a s i c " r e s e a r c h . a n d she w o u l d not even k n o w w h i c h w a y to m o v e . Normative philosophy c o n s i s t s of introduced f o r m o f political r e s e a r c h . to add to our general understanding of politics. i t i n c l u d e s a m o n g its practitioners P l a t o . w h a t s o c i a l scientists do in developing theories is not different f r o m w h a t we n o r m a l l y do every day in perceiving. they w i l l not. for this is r e s e a r c h c a r r i e d on for its o w n s a k e . its stance is e m p i r i c a l .4 Chapter 1 is John Stuart Mill's argument in Considerations Doing Research on Representative 5 Government. T h e y are r e d u c e d to m e r e l y observing events. A n d they distinctively operate by d e r i v i n g further i m p l i c a t i o n s of the posited conditions by p r e c i s e l o g i c a l and m a t h e m a t i c a l operations. not the active formulation of normative arguments. our environment. that i s . F o r both forms of applied r e s e a r c h . P o l i t i c a l theory. G e o r g e W i l l . a n d others. trying to d e s i g n a d i p l o m a t i c strategy to effect d i s a r m a m e n t p r o c e d u r e s . and r e s e a r c h is carried on under that n a m e . parties w i l l tend to agree very c l o s e l y on i s s u e s . but generally this m e a n s the history of normative philosophy a n d its development. In a h i g h s e n s e of the w o r d . is the s u m total of a l l those theories d e v e l o p e d by s o c i a l scientists to e x p l a i n h u m a n behavior. Thus. but research in it is often relegated to a separate institute or " s c h o o l of public policy. but this c a r r i e s the u n p l e a s ant implication that applied r e s e a r c h is either i m p u r e or of l i m i t e d v a l u e . P o l i t i c a l scientists pursue this type of r e s e a r c h for the t w i n p l e a s u r e s of e x e r c i s i n g their m i n d s a n d i n c r e a s i n g their understanding of things. T h e i r e n d goal is to develop reasonably b r o a d a n d general theories b a s e d on a s m a l l n u m b e r of agreed-upon assumptions. S o m e of the predictions generated f r o m h i s theory are (1) in a two-party s y s t e m . T h i s type of research is r e a l l y not as flippant as the c h o i c e of the term recreational might m a k e it s e e m . T h e i r c o n c e r n is to take the posited facts. or it may be intended to provide new ways of looking at old facts. H o w e v e r . A g o o d e x a m p l e of f o r m a l t h e o r y — i n d e e d . w h e r e a s Nonempirical Empirical Normative philosophy Engineering research in a multiparty s y s t e m . Downs builds a wide-ranging arguments about what should be in p o l i t i c s . 2.

the large store cannot get its v a l u e d g o o d u n l e s s it takes the lead in setting up the a s s o c i a t i o n . 9 9 9 . T h e m a i n u s e of f o r m a l theory. he c o n c l u d e d . the difference if I do or do not contribute is a budget of $ 2 9 . We k n o w that m a n y people do contribute to political organizations even though. R e m e m b e r that the rational c h o i c e a s s u m p t i o n states that i n d i v i d u a l s choose their actions in order to m a x i m i z e a v a l u e d object. if e a c h of us contributes $ 100 to the c a u s e . If y o u j o i n K X X X .) A good e x a m p l e of f o r m a l theory that illustrates the rational c h o i c e a s s u m p tion is M a n c u r O l s o n ' s The Logic of Collective Action ( 1 9 6 5 ) . and m i n i m i z e the cost e x p e n d e d in a c h i e v i n g it. B a s e d on the rational c h o i c e a s s u m p t i o n . w h i l e m i n i m i z i n g the c o s t expended in a c h i e v i n g it. T h u s . in a w i d e array of settings O l s o n ' s theory predicts behavior rather w e l l . k n o w s that a D o w n t o w n M e r c h a n t s ' A s s o c i a t i o n cannot function i f i t does not j o i n and contribute. T h e virtue of O l s o n ' s theory in this c a s e is that instead of v i e w i n g s u c h contributions as " n a t u r a l " and therefore i gnor i ng t h e m .e n c o m p a s s i n g . s i n c e every potential m e m b e r of s u c h an organization is in this s a m e situation.T V today we w i l l send y o u this beautiful coffee m u g ! " ) bears testimony to the power of O l s o n ' s logic. participation in the group is u s u a l l y nonrational. m o s t of s o c i e t y ' s interests w o u l d be actively o r g a n i z e d . if the a s s u m p t i o n s p r o d u c e d an untenable result. the 2 pluralist school. or p o l i t i c a l p o w e r itself. O l s o n laid out several conditions under w i i i c h organizations might nonetheless arise. O n e s u c h condition is that one potential m e m b e r might h a v e s u c h large resources that she k n o w s no organization is p o s s i b l e without her participation. and let all those other people m a k e the contributions. W h a t is rational. a great deal of f o r m a l theory in political s c i e n c e has b a s e d itself on the e c o n o m i s t s ' c o r e a s s u m p t i o n of interests existed i n s o c i e t y — r a c i a l m i n o r i t i e s . 0 0 0 . However. he w o u l d go b a c k and r e e x a m i n e them. B u t b e c a u s e f o r m a l theory consists of taking a set of a s s u m p t i o n s and w o r k i n g out w h e r e they l e a d — t h a t i s . then ( c ) c a n we d e d u c e what sort of taxes in the context of the a s s u m p t i o n s of (b) w o u l d best a c h i e v e ( a ) ? L i k e normative p h i l o s o p h y . one m u s t r e c o g n i z e that excerpts s u c h as these do e v e n m o r e than the u s u a l v i o l e n c e to a r i c h net of theories. b e c a u s e rationally. instead. To the organization this a m o u n t w o u l d be trivially s m a l l . and in fact one function of theory m a y be that it highlights exceptions for c l o s e r e x a m i n a t i o n . L e t us say there are 3 0 0 . we c a n count on an organization b e i n g set up. I w i l l h a v e e x p e n d e d a significant cost without getting any m o r e of my v a l u e d good. and (b) if we a s s u m e that governmental investment is inefficient and that i n d i v i d u a l taxpayers act so as to m a x i m i z e their i n c o m e . I k n o w that u n l e s s I have v e r y u n u s u a l r e s o u r c e s . we are f o r c e d to treat the contributions as a p u z z l e requiring further investigation. A l l o f the c o n c l u s i o n s derive l o g i c a l l y f r o m a limited set of e x p l i c i t a s s u m p t i o n s . and a l l o w e d h i m to stand r e c e i v e d w i s d o m o n its h e a d . O l s o n wrote o n the v e r y b a s i c question o f 2 political organization in society. p r o f e s s i o n s . as in the e x a m p l e above. we c a n u s e f o r m a l theory to construct a n a l y s e s of the f o l l o w i n g f o r m : If we want to a c h i e v e X. the m a r v e l s h o u l d be that any interest or gani zations exist at a l l . c a n we d e v i s e a set of r e a s o n a b l y true a s s u m p t i o n s and an action w h i c h . H o w e v e r . O l s o n r e a s o n e d that there w a s nothing natural about organization at a l l .t a x p r o p o s a l s are a g o o d e x a m p l e : T h e y originated in argument of the f o l l o w i n g f o r m : (a) I f w e want t o m a x i m i z e investment and e c o n o m i c growth. D o w n s ' p u r p o s e in this is s i m p l y to see w h e r e the a s s u m p t i o n s he started w i t h w i l l lead h i m . it is irrational for t h e m to do s o . A w h o l e s c h o o l of political s c i e n c e . but to me $ 1 0 0 m a k e s a real difference. T h e excruciating efforts of public television stations to get their viewers to j o i n rather than be free riders ("Please! O n l y one in ten of our viewers is a member. s c h o l a r s had a s s u m e d that w h e n For example. a f o r m a l theory is u s e d to construct a set of conditions f r o m w h i c h the thing we w i s h to e x p l a i n w o u l d have l o g i c a l l y f l o w e d . 2 .) It is important to e m p h a s i z e that this sort o f w o r k i s a l m o s t s o l e l y a n e x e r c i s e i n deduction. 0 0 0 . rational choice: the a s s u m p t i o n that i n d i v i d u a l s c h o o s e their actions in order to m a x i m i z e s o m e v a l u e d object. No theory c a n ever be a l l . is to be a free rider. is explanation. G o o d work in formal theory w i l l take a set of seemingly reasonable assumptions and w i l l show by logical deduction that those assumptions lead inescapably to c o n clusions that surprise the reader. F o l l o w i n g f r o m D o w n s . in the context of those a s s u m p t i o n s . P r e s u m a b l y .6 Chapter 1 Doing Research 7 ( O f c o u r s e . S o m e t i m e s the object m a y e v e n be left u n s p e c i f i e d in the theory. S u c h explanatory f o r m a l theories are then often tested e m p i r i c a l l y through theory-oriented r e s e a r c h . as O l s o n has p r o v e d . T h a t i s . If I am a p e r s o n c o n c e r n e d w i t h h i s t o r i c preservation. though. We s h o u l d thus expect to see a w i d e range of parties and interest groups engaged in p o l i t i c s . T h e largest department store in town. Duverger (1963) assumed this in the theory I described on pp. B e f o r e his b o o k . F l a t . ( I n e c o n o m i c s the v a l u e d object is gene r a l l y taken to be m o n e y . groups w i t h special c o n c e r n s s u c h as historic p r e s e r v a t i o n — p o l i t i c a l organizations c o u l d be expected to emerge naturally to represent those i n t e r e s t s . to argue for v a r i o u s w a y s to set up e l e c t i o n s . In the next chapter y o u w i l l see that it m a y also help to explain w h y s m a l l nations typically do not pull their "fair" weight in international alliances. 9 0 0 v e r s u s $ 3 0 . T h e rational c h o i c e a s s u m p t i o n pointed O l s o n to a question no one h a d a s k e d before. If I contribute. w h i c h is not rational. T h e B i j o u x T e e . F r o m the standpoint of any i n d i v i d u a l in a group w i t h a shared object. T h e reader must then either accept the surprising conclusion or reexamine the assumptions that had seemed plausible. O l s o n pointed out. w a s organized ar ound the expectation that most of the time. f o r m a l theory interacts w i t h e m p i r i c a l r e s e a r c h . in political s c i e n c e it m a y be m o n e y — a s in theories of w h y and h o w c o m m u n i t i e s s e e k pork-barrel s p e n d i n g — b u t theories m a y also posit that the v a l u e d object is a nonmonetary p o l i c y s u c h as abortion. is not in that situation. what they l o g i c a l l y i m p l y — i t is a l s o u s e f u l for d e v e l o p i n g and a n a l y z i n g strategies for p o l i t i c a l action. 0 0 0 people around the country w h o share my interest. F o r m a l theorists u s u a l l y try to start w i t h a s s u m p t i o n s that are in a c c o r d w i t h existing k n o w l e d g e about p o l i t i c s . formal theory provides insights by logical argument. my i n d i v i d u a l contribution to an interest group p u r s u i n g preservation w i l l not m a k e a m e a s u r a b l e difference. or for v a r i o u s w a y s to arrange taxes so as to get the o u t c o m e s we want. and at the end they m a y c o m p a r e their final m o d e l s w i t h this body of k n o w l e d g e . . for e x a m p l e . h o w e v e r . b u s i n e s s e s .3 . U n d e r these c i r c u m s t a n c e s . for i n s t a n c e . B u t they are not t h e m s e l v e s c o n c e r n e d w i t h turning up n e w factual i n f o r m a t i o n . w i l l l o g i c a l l y lead to XI F o r m a l theory is u s e d in this w a y . not by a direct examination of political facts.S h i r t S h o p on the corner.

T h e normative p h i l o s o p h e r m u s t be free to i m a g i n e realities that have never existed before. T h o s e w h o w e r e not p r o m o t e d w e r e d i s a p pointed. T h e fact that s o m a n y air c o r p s m e n w e r e N C O s apparently m a d e the c o r p s m e n feel that p r o m o t i o n w a s the n o r m a l thing. facts are sought out if they are n e e d e d to s o l v e a p r o b l e m . T h e y e x a m i n e d c h a n g e s i n military e x p e n diture by m a j o r p o w e r s f r o m 1816 to 1976 to see w h e t h e r i n c r e a s e s tended to be f o l l o w e d b y i n v o l v e m e n t i n war. it is c o n c e r n e d w i t h d i s c o v e r i n g facts about p o l i t i c s . t o c o n d u c t a . T h e y d i d this by d e v e l o p i n g the theory of relative deprivation to a c c o u n t for their s e e m i n g l y contradictory findings. M o r e important. any s p e c i f i c p i e c e of w o r k is a m i x of m o r e than one of the types. A c c o r d i n g l y . a m i x is so m u c h the u s u a l situation that w h e n I tried to m a k e a r o u g h h e a d count of the f r e q u e n c y of the P r a c t i c a l l y no r e s e a r c h is a pure e x a m p l e of any of the types I h a v e presented here. let us l o o k a bit m o r e c l o s e l y at normative p h i l o s o p h y . F i r s t . and those w h o w e r e p r o m o t e d d i d not feel particularly h o n o r e d . This type of research is concerned with p e r s o n a l i n v e s t i g a t i o n o f this f a c t u a l b a s i s . however. on the other h a n d . A researcher m a y be relatively more w i l l i n g t o a s s u m e these facts f r o m g e n e r a l e x p e r i e n c e a n d / o r f r o m the r e s e a r c h o f others. one of w h i c h is that a r m s expenditure m u s t therefore be determined m o r e by d o m e s t i c p o l i t i c a l c o n s i d e r a t i o n s than by the international situation. A s a s e c o n d e x a m p l e o f the w a y i n w h i c h types o f r e s e a r c h are m i x e d i n any one w o r k . but of whether her c o n d i t i o n c o m p a r e s favorably or unfavorably w i t h a standard that she p e r c e i v e s as n o r m a l . h e m a y w i s h . satisfaction w i t h o n e ' s c o n d i t i o n is not a function of h o w w e l l . T h i s sort of paradox occurred a number of times in their study. there w i l l a l m o s t a l w a y s b e s o m e interaction b e t w e e n the different types i n any g i v e n w o r k . M i l i t a r y engagements w e r e no m o r e or l e s s l i k e l y to o c c u r f o l l o w i n g military buildups than under other c i r c u m s t a n c e s . o f c o u r s e . u s i n g K a r l M a r x ' s w o r k as an e x a m p l e . a s M i l l w a s . T h o s e w h o w e r e not p r o m o t e d w e r e not disappointed. M P s were objectively less likely to be promoted than were members o f the A r m y A i r C o r p s . here they are sought out if they w i l l be u s e f u l in d e v e l o p i n g theories. and b e c a u s e the state and the e c o n o m y function in b a d effects. D u v e r g e r ' s study of p o l i t i c a l parties is an e x a m p l e of theory-oriented r e s e a r c h . cannot b e "tested. A m o n g the M P s ." B u t M a r x w a s less w i l l i n g than M i l l t o s i m p l y assume the factual portions of his argument. It s h o u l d be e v i d e n t that a n y o n e d e v e l o p i n g n o r m a t i v e theories about p o l i t i c s must begin with some factual assumptions. let us l o o k at a c a s e in w h i c h r e s e a r c h e r s w o r k i n g on a p r i m a r i l y e n g i neering project f o u n d they h a d to develop a theory to m a k e s e n s e out of their w o r k . all U t o p i a n dreams w o u l d h a v e to be thrown out. and these. most r e s e a r c h and teaching in p o l i t i c a l s c i e n c e is still of the fourth type suggested in Table 1-1. 1949). L i k e p o l i t i c a l e n g i n e e r i n g . w h o w e r e l e s s l i k e l y to be p r o m o t e d . A c c o r d i n g to this theory. o n the other h a n d . A l t h o u g h one m e t h o d w i l l u s u a l l y predominate. the fact that normative p h i l o s o p h e r s are not required to p r o v i d e e v i d e n c e for all their a s s u m p t i o n s l e a v e s t h e m free to devote m o r e energy to other parts of the r e s e a r c h task. N o matter h o w they adjusted t h i n g s — l o o k i n g for d e l a y e d r e a c t i o n s . l o o k i n g only a t " a r m s r a c e s " i n w h i c h two rivals s i m u l t a n e o u s l y i n c r e a s e d their military f o r c e s . O f Stouffer's sample o f M P s . A n o t h e r g o o d e x a m p l e is a test by D i e h l and K i n g s t o n ( 1 9 8 7 ) of the theory that arms buildups l e a d t o m i l i t a r y confrontations. theory-oriented research. F o r one thing. felt that c h a n c e s for promotion i n the A r m y w e r e g o o d ! I h a v e m e n t i o n e d these two e x a m p l e s to illustrate my point that m o s t r e s e a r c h w o r k i n v o l v e s s o m e m i x of the four types of r e s e a r c h .o f f a p e r s o n is objectively. he spent y e a r s of r e s e a r c h trying to w o r k out the p r e c i s e e c o n o m i c effects of c a p i t a l i s m . that the r e s e a r c h e r n e e d not f e e l required to p r o d u c e the f u l l factual b a s i s for h i s argument. l i k e M a r x . 2 4 percent were n o n c o m m i s sioned officers ( N C O s ) . i n v o l v e h i m t o s o m e degree i n e n g i n e e r i n g r e s e a r c h . we s h o u l d strive to c h a n g e the state and the e c o n o m y in w h i c h w i l l eliminate the b a d effects. w h o w e r e m o r e l i k e l y to be p r o m o t e d . c o m p a r e d with 47 percent of the air c o r p s m e n . T w o e x a m p l e s m a y help illustrate this point." I f normative p h i l o s o p h e r s w e r e h e l d to the s a m e standards of factual e v i d e n c e as e m p i r i c a l r e s e a r c h e r s . the air c o r p s m e n . it is e m p i r i c a l .8 Chapter 1 Doing Research 9 A l t h o u g h f o r m a l theory is the fastest g r o w i n g type of political r e s e a r c h . T h e y c o n c l u d e d by e x p l o r i n g the i m p l i c a t i o n s of this finding. G e n e r a l l y . and the researchers felt they had to m a k e some sense of it if their efforts were to help the A r m y improve morale. In e n g i n e e r i n g . and so o n — t h e y found no relationship. H i s argument takes the s a m e general f o r m as that in M i l l ' s e s s a y on representative government: " B e c a u s e aspects of the h u m a n condition today are w a y s to p r o d u c e these ways. I n this r e s p e c t n o r m a t i v e p h i l o s o p h y differs f r o m the e m p i r i c a l types o f p o l i t i c a l r e s e a r c h . A group of s o c i o l o g i s t s l e d by S a m u e l Stouffer w a s e m p l o y e d by the A r m y to study the m o r a l e o f A m e r i c a n soldiers during W o r l d W a r I I (Stouffer and others. types o f e m p h a s i s found i n particular p i e c e s o f r e s e a r c h . b a d . w h i c h deals w i t h facts o n l y for their u s e f u l n e s s in s p e c i f i c p o l i t i c a l p r o b l e m s . the m o s t important activity in this r e s e a r c h is the development of theories l i n k i n g o b s e r v e d facts about politics. B u t u n l i k e e n g i n e e r i n g . the M P s w e r e m u c h more likely than the air c o r p s m e n to think that soldiers e x p a n d i n g our k n o w l e d g e of w h a t happens in p o l i t i c s and of w h y it happens as it does. and those w h o w e r e promoted felt h o n o r e d . T h e s e are abstract distinctions. this r e s e a r c h deals w i t h them to develop n e w p o l i t i c a l theories or to c h a n g e or c o n f i r m old ones. T h e distinction is an important one. felt that c h a n c e s for p r o m o t i o n in the A r m y w e r e poor. S u c h a c t i v i t y w i l l . M a r x ' s theory of the d i a l e c t i c is p r i m a r i l y a w o r k in normative philosophy. Stouffer and his c o w o r k e r s w e r e p u z z l e d by the fact that often a s o l d i e r ' s m o r a l e h a d little to do w i t h h i s objective situation. and the M P s . p a r a d o x i c a l l y . F o r instance. I n d e e d . promotion w a s sufficiently infrequent that not being promoted w a s seen as the n o r m . they often n e e d to a s s u m e facts that cannot p o s s i b l y be tested against reality. T h u s . Research Mix with ability h a d a good chance to advance in the A r m y . h o w e v e r . Paradoxically. I t i s c h a r a c t e r i s t i c o f n o r m a t i v e p h i l o s o p h y . I n s t e a d . o f c o u r s e .

T h e s e goals have a lot to do w i t h the w a y a study s h o u l d be set up and the criteria a c c o r d i n g to w h i c h it should be judged. I w i l l d i s c u s s this in s o m e detail in Chapter 2. F o r e x a m p l e . R e c r e a t i o n a l r e s e a r c h . by p r o v i d i n g e v i d e n c e for the factual b a s i s on w h i c h its a s s u m p t i o n s rest. m a k e s a m o r e interesting p r o b l e m in n o r m a tive theory than a p r o b l e m dealing w i t h an argument about d y n a s t i c s u c c e s s i o n . We operate w i t h i n rules of e v i d e n c e in interpreting reality. T o d a y . although a normative argument m a y b e m a d e m o r e c o n v i n c i n g . F o r either sort of a p p l i e d r e s e a r c h . In other w o r d s . however. we s h o u l d let the c h i p s fall w h e r e they may. H o w to c h o o s e an interesting p r o b l e m is one of the most difficult and c h a l l e n g i n g parts of e m p i r i c a l r e s e a r c h . i n b i o l o g y . this b o o k i s c o n c e r n e d w i t h e m p i r i c a l r e s e a r c h . we must take personal r e s p o n s i b i l i t y for the results of our r e s e a r c h . that our results are to s o m e extent a subjective interpretation of reality. for i n s t a n c e . In either f o r m of n o n e m p i r i c a l r e s e a r c h . We s h o u l d s i m p l y s e e k truth and not w o r r y about its effects. It is j u s t not often the c a s e that a r e s e a r c h e r c a n e a s i l y be labeled a normative philosopher. r e s e a r c h o n the difference b e t w e e n m e n ' s and w o m e n ' s voting i n I c e l a n d in the 1920s and 1930s w o u l d s o u n d absurd f r o m the standpoint of an engineer." A l s o . ETHICS OF POLITICAL RESEARCH C o n d u c t i n g r e s e a r c h is an act by y o u . i t i s v e r y much a matter of personal beliefs and cultural context. A n o t h e r c o l l e a g u e . in that w h a t s e e m s " d e h u m a n i z i n g " to a p e r s o n d e p e n d s o n w h a t the p e r s o n t h i n k s " h u m a n " m e a n s — t h a t i s . Y o u m u s t therefore be c o n c e r n e d about the ethics of y o u r r e s e a r c h . R e c e n t results in p s y c h o l o g y suggesting that a w i d e range of b e h a v i o r s are genetically c o n trolled go against our prevailing d i s p o s i t i o n to think of h u m a n s as free agents in w h a t they do. r e s e a r c h noted a b o v e m a y s e e m w h o l l y a p p r o p r i a t e . so we are c o n s t r a i n e d in w h a t we c a n assert and cannot s i m p l y p u l l findings out of a hat. the s o c i a l scientist rarely deals i n u n q u e s t i o n e d truths. the r e s e a r c h e r s h o u l d be of political p e r s u a s i o n . W i t h i n e m p i r i c a l r e s e a r c h . requires p r o b l e m s that w i l l have a substantial i m p a c t on existing bodies of theory. on the other h a n d . S i m i l a r l y . one type of r e s e a r c h is dominant in any given p i e c e of w o r k . a further p r o b l e m a r i s e s . I n g e n e r a l . T h e s e types interact in the w o r k of every political scientist. w a s upset w h e n h e l e a r n e d that h i s r e s e a r c h o n f r o g s ' e y e s t u r n e d out t o h a v e a p p l i c a t i o n s i n the d e s i g n o f g u i d a n c e s y s t e m s for m i s s i l e s ! I n the c a s e o f d e m e a n i n g o r d e h u m a n i z i n g r e s e a r c h . w e m u s t c o n c e r n o u r s e l v e s w i t h the effects on society of what we discover.10 Chapter 1 Doing Research 11 different types of r e s e a r c h in p o l i t i c a l s c i e n c e j o u r n a l s . W e w o r k under sufficient difficulties. B u t I w i l l r i s k suggesting t w o standards for r e s e a r c h that w i l l s e r v e as e x a m p l e s of the w a y in w h i c h the type (or types) of r e s e a r c h we are doing dictates the w a y we s h o u l d c o n d u c t that r e s e a r c h . B u t these voting patterns. T h a t most r e s e a r c h i n v o l v e s a m i x of the types does not p r e c l u d e the i m p o r tance of the d i s t i n c t i o n s . j u s t as y o u are w i t h all of y o u r a c t i o n s . In the first p l a c e . O n e r e s p o n s e to s u c h difficulties might be to take a "pure s c i e n c e " a p p r o a c h . as we m a y fear that innocent r e s e a r c h results might reinforce preexisting stereotypes. T h e r e are two broad c l a s s e s o f e t h i c a l questions regarding our r e s e a r c h . I w a s u n a b l e to do s o . a c c o r d i n g to i n f o r m a l and rather flexible criteria. p r o b l e m s s h o u l d be c h o s e n w h i c h are of r e a l i m p o r t a n c e for c o n t e m p o r a r y p o l i c y . R e s e a r c h on r a c i a l or ethnic groups is particularly s e n s i t i v e . A s a n o t h e r e x a m p l e . E t h i c a l q u e s t i o n s o f this sort are e s p e c i a l l y d i f f i c u l t b e c a u s e the r e s u l t s o f our r e s e a r c h are s o h a r d t o p r e d i c t . arguing that b e c a u s e it is so hard to j u d g e the results of k n o w l e d g e a n y w a y . d e p e n d i n g o n o n e ' s v i e w o f h u m a n n e s s . an engineer. a f e w h u n d r e d y e a r s ago the reverse w o u l d probably h a v e been the c a s e . I w a s s i m p l y u n w i l l i n g to a s s i g n most articles to one or another of the categories. G e n e r a l l y . good r e s e a r c h of any sort s h o u l d be directed to an interesting problem. We are not s i m p l y neutral agents of truth. I devote s o m e w h a t m o r e attention to theory-oriented r e s e a r c h than to . for e x a m p l e . F o r i n s t a n c e . r e s e a r c h s h o u l d be evaluated subjectively. engineering. M a n y topics that are of c o n s i d e r a b l e importance to an engineer show little promise for theory-oriented research. only to learn later that it w a s u s e d by a military j u n t a in a L a t i n A m e r i c a n country to figure out h o w to p r o d u c e a controllable "democracy. o c c u r r i n g j u s t after the extension of the vote to w o m e n . the results of a r e s e a r c h c a n be d e m e a n i n g or d e h u m a n i z i n g . it is p o s s i b l e that w h a t y o u learn c o u l d be u s e d by a political charlatan to do b a d things. m a n y p r o m i s i n g topics for recreational r e s e a r c h are not directly relevant. an argument about c i v i l d i s o b e d i e n c e . might be important for theories of h o w voting patterns b e c o m e established a m o n g new voters. As we w i l l see throughout this book. this is not n e c e s s a r y . L i k e any creative w o r k . in the c o m m o n usage of the w o r d . a f o r m a l theorist. depending on the goals of the researcher. or a theory-oriented e m p i r i c a l researcher. if y o u study t e c h n i q u e s Evaluating Different Types of Research It is dangerous to set d o w n s i m p l e standards for good r e s e a r c h . h o w e v e r . T h e r e are two r e a s o n s for this: (1) It is the m o r e c o m m o n k i n d of research in p o l i t i c a l s c i e n c e . but still our results i n v o l v e i n d i v i d u a l c h o i c e s and j u d g m e n t by u s . B u t w h a t sort of p r o b l e m is "interesting" depends largely on the motivation of the study. the t h e o r y o f e v o l u t i o n a p p e a r s d e h u m a n i z i n g to m a n y f u n d a m e n t a l i s t C h r i s t i a n s but d o e s not a p p e a r so to m a n y other p e o p l e . and (2) it poses rather m o r e difficult instructional problems than does e n g i n e e r i n g . o r a n e x e r c i s e i n f o r m a l theory m a y b e m a d e m o r e interesting. e s p e c i a l l y the fact that we u s u a l l y cannot operate by experimentation (see C h a p t e r 6 ) . F i r s t . The psychological held responsible for demonstrating the factual basis of his conclusions. A c o l l e a g u e o n c e p u b l i s h e d a study of the effects of electoral s y s t e m s on representation. In the s e c o n d p l a c e . applied r e s e a r c h s h o u l d be relevant. in either f o r m of e m p i r i c a l r e s e a r c h . difficult though these e t h i c a l questions m a y be.

as m a y be true of political r e s e a r c h that d e l v e s into racketeering or c o r r u p t i o n ? T h e most horrible historic e x a m p l e of s c i e n c e gone bad is that of the N a z i doctors w h o k i l l e d prisoners b y i m m e r s i n g t h e m i n i c e water t o see h o w long people c o u l d s u r v i v e in f r e e z i n g water. in evaluating the effects of a program to get people off the welfare rolls and into jobs. p a i n of w h i c h they are i n f o r m e d in a d v a n c e ) and the social benefits great. I n s u m . from the South. w h i c h w a s p u r c h a s e d at great h u m a n p a i n . Particular problems arise in the f o l l o w i n g w a y s : 1. 5. B u t no matter w h i c h type of r e s e a r c h one is currently e n g a g e d i n . or submitting them to embarrassing situations. as I pointed out in C h a p t e r 1. studies m e a s u r i n g the malapportionment of state legislatures. for e x a m p l e . If a r e s e a r c h that w i l l benefit society c a n be c o n d u c t e d only by mistreating s u b j e c t s . we w o u l d g e n e r a l l y s a y y e s . O t h e r s i n c l u d e the G a l l u p P o l l . Public officials may get a hundred questionnaires a year. If the costs to s u b j e c t s are slight ( i n c o n v e n i e n c e . to withhold the program from some deserving people while administering it to others. but w h i c h m a y potentially help i n s a v i n g lives a n d — w e h o p e — w i l l never b e available again f r o m any s o u r c e . D o e s u s i n g the results o f the r e s e a r c h j u s t i f y it? I f s o . Harming the subjects of your study. Don't demand more of them than is reasonable. y o u s h o u l d not a s s u m e that it is important only for w h a t I h a v e c a l l e d theory-oriented r e s e a r c h . but it is also an annoying time if you have 15 minutes' worth of questions to ask. Harm to subjects. and c o m parisons of the relative military strength of v a r i o u s countries. M u c h a p p l i e d r e s e a r c h on the probl e m of e n r i c h i n g the education of underprivileged c h i l d r e n . the subjects of your study will wish to have their privacy protected. To effect a c h a n g e in s o m e given p h e n o m e n o n . perhaps w e s h o u l d destroy the results. You should avoid shaming people into participating in your study. B u t w h a t if it puts subjects in danger of death. s h o u l d it be d o n e ? T h e r e is no c l e a r a n s w e r . Generally. As an overall rule. T h e Stouffer study. they s i m p l y i n v o l v e m e a s u r i n g things that need to be m e a s u r e d . In that c a s e . Dinnertime is a good time to reach people by phone. On the other h a n d . for me at least. Confidentiality. either by doing harmful things to them or by withholding good things from them. In this chapter we look more c l o s e l y at the nature of political theories and at the f a c tors that i n f l u e n c e the d e c i s i o n to do r e s e a r c h on a particular theory. B u t might that not lead to greater h u m a n pain for v i c t i m s o f f r e e z i n g and exposure w h o m w e might have h e l p e d ? T h e one f i r m rule. the senior member of her committee). T h e o r y is a tool in one type of r e s e a r c h . As you will see in Chapter 6. y o u m a y need to develop a theory that a c c o u n t s for several factors and a l l o w s y o u to T h e p r o b l e m s I have noted here p o s e difficult ethical questions of the " e n d s and m e a n s " sort. everyone has the right not to be fooled and not to be used without his or her consent. e n g i n e e r i n g r e s e a r c h m a y o r m a y not i n v o l v e the development o f political theories. it s h o u l d be done. A p a i n f u l ethical question today is w h e t h e r e v e n to u s e the results of that r e s e a r c h . manipulate them to produce the d e s i r e d c h a n g e . should generally be avoided. 4. the results of your study might well be more valid if the people you study are unaware that they are being studied. A l t h o u g h this chapter deals w i t h p o l i t i c a l theories. However. S . You should take care to truly mask the subjects who have helped you. T a k i n g the U . c e n s u s i s one e x a m p l e o f s u c h engineering r e s e a r c h . Is it ethically right. Stouffer and his collaborators h a d t o e x p l a i n w h y M P s h a d higher m o r a l e than air c o r p s m e n . Embarrassment or psychological stress. for example. female. A l o n g the w a y I w i l l d i s c u s s s o m e standards to u s e in d e c i d i n g w h e t h e r a theory is w e a k or strong. theory-oriented r e s e a r c h a l w a y s does. Fooling or misleading the subjects. is that people s h o u l d never be c o e r c e d or tricked into participation and s h o u l d a l w a y s be fully i n f o r m e d before they agree to participate. 3. c i t e d in C h a p t e r 1. more specific than the ones described earlier but not necessarily easier to answer.12 Chapter 1 A s e c o n d c l a s s of questions. it is w o r t h taking a c l o s e r l o o k at the nature of theory. Relevant details you include in your report might make it easy to identify the subject (a member of Congress. m a n y e n g i n e e r i n g studies do not require that a theory be developed. the k e y to s o l v i n g m a n y engineering p r o b l e m s m a y be a political theory of s o m e sort. has h a d to c o n c e r n itself w i t h d e v e l o p i n g theories to e x p l a i n w h y one c h i l d learns things m o r e q u i c k l y than another. it is an e n d in itself in the other. deals with our treatment of the people we are studying. Imposition. We are responsible to treat the subjects of our study fairly and decently. is another e x a m p l e of an engineering study in w h i c h it w a s n e c e s s a r y to develop a theory. in order to see how effective it is? 2. T h i s w a s n e c e s s a r y if they w e r e to devise w a y s to raise the m o r a l e of A r m y p e r s o n n e l i n general. you should make certain that your subjects know exactly what they will be doing and what use you will make of them. You are asking your subjects to help you. It is not enough just to withhold publishing their names. I n d e e d . 13 . keep yours short.

2. for e x a m p l e . F o r i n s t a n c e . In theory 2. the authors tested a theory that if a country i n c r e a s e s its m i l i t a r y expenditure. then it might be e x p e c t e d to go to war. It all depends on the theory: In engineering r e s e a r c h . ories. A l l of these factors are c a l l e d " v a r i a b l e s " s i m p l y b e c a u s e it is the variation of e a c h that m a k e s it of interest to u s . It does not help to have a simple. F o r i n s t a n c e . T h e s a m e v a r i a b l e m a y b e a n i n d e p e n d e n t v a r i a b l e i n o n e theory a n d a d e p e n d e n t v a r i a b l e i n another. if e a c h country h a d had e x a c t l y the s a m e n u m b e r o f p a r t i e s — t h e r e w o u l d h a v e b e e n nothing for D u v e r g e r to e x p l a i n . A c a u s a l theory a l s o i n c l u d e s one or m o r e factors that are thought to affect the dependent variable. o n e theory m i g h t u s e the s o c i a l status of a p e r s o n ' s father (the i n d e p e n d e n t v a r i a b l e ) to e x p l a i n the p e r s o n ' s s o c i a l status (the dependent v a r i a b l e ) . a n d it is not s o m e t h i n g about the variable itself. In the D u v e r g e r e x a m p l e . v a r i a b l e w a s the n u m b e r of parties. to explain why people vote the way they do. but it is an important part of the e n g i n e e r i n g r e s e a r c h project. A n o t h e r theory might u s e the p e r s o n ' s s o c i a l status as an independent v a r i a b l e to e x p l a i n s o m e t h i n g e l s e . T h i s is the dependent variable. the theory s h o u l d give a h a n d l e on as big a portion of the u n i v e r s e as p o s s i b l e . a theory o f electoral change might take on importance partly f r o m the p h e n o m e n a it e x p l a i n e d d i r e c t l y — c h a n g e s in p e o p l e ' s votes. If party s y s t e m s h a d not v a r i e d — t h a t i s . p e r h a p s the w a y the p e r s o n v o t e s . for e x a m p l e . Y o u r r e s e a r c h report s h o u l d i n c l u d e s o m e d i s c u s s i o n of the i m p o r tance of the p r o b l e m a n d of p o s s i b l e applications for y o u r findings. T h u s . T h e shape it takes " d e p e n d s " on the configuration of the other factors. Predictive accuracy. y o u s h o u l d try to j u s t i f y y o u r c h o i c e of topic. the independent v a r i a b l e s are thus d e s i g n a t e d b e c a u s e in terms of the particular theory. However." T h e e x a m p l e s we l o o k e d at in C h a p t e r 1 w e r e all of this f o r m . It should use no more than a few independent variables. might predict quite a c c u r a t e l y for that s p e c i f i c situation. Importance. W h e n we say that a theory s h o u l d apply " b r o a d l y " a n d "generally. A theory should give us as simple a handle on the universe as possible. democracy functions as an independent variable. It would not be very useful to develop a theory that used 30 variables. effective theories: 1. e a c h o f w h i c h i n v o l v e s its o w n portion o f reality. it w o u l d not help us very m u c h to r e d u c e the c h a o s of N e w Y o r k C i t y p o l i t i c s . then Y w i l l f o l l o w as a result. but an i m a g i n a t i v e r e s e a r c h e r w h o sits d o w n a n d thinks about it for a w h i l e m a y be able to point up additional. A c a u s a l theory a l w a y s i n c l u d e s s o m e p h e n o m e n o n that is to be e x p l a i n e d or a c c o u n t e d for. so we shall consider them separately. if a certain configuration of political c o n f l i c t s e x i s t s . A theory of the organization of borough p r e s i d e n c i e s in N e w Y o r k C i t y . a n d if the country adopts a certain electoral law. general theories of attitude c h a n g e . but also to h o w great a variety of preexisting theories are affected by the n e w theory. no variable is innately either independent or dependent. i n a s m u c h a s there w e r e n o differences. but it w o u l d a l s o shed light on the w i d e r u n i v e r s e dealt w i t h by the other theories. then the n u m b e r of political parties in the c o u n t r y c a n be e x p e c t e d to grow or s h r i n k to a certain number. the tendency to w a g e w a r depends on whether or not a country is a democracy. a theory s h o u l d address a p r o b l e m that is currently p r e s s i n g . if all countries h a d h a d the s a m e electoral s y s t e m . In theory 1. It is e a s y to develop a trivial theory. Countries with high per capita incomes are more likely to be democracies than poor countries are. let alone the c h a o s of p o l i t i c s in general. S i m i l a r l y ." we are referring not o n l y to h o w large a selection of items f r o m reality the theory deals w i t h . A theory should make accurate predictions. If one or the other of h i s independent v a r i a b l e s h a d not v a r i e d . s i m p l y to turn in a c o m p u t e r printout as a paper. T h u s . it s h o u l d apply broadly a n d generally. or whatever. democracy functions as a dependent variable. In D u v e r g e r ' s theory. in intricate combinations. 3. but it w o u l d be a m o r e v a l u a b l e tool if it c o u l d be s h o w n to h a v e significant i m p l i c a t i o n s for other areas of s o c i a l t h e o r y — d e m o c r a t i c theory. A theory c a n attain great generality rather e c o n o m i c a l l y if it h e l p s to recast older theTheory 1: Theory 2: Democracies do not tend to initiate wars. In the D i e h l a n d K i n g s t o n study. T h e dependent v a r i a b l e is so n a m e d b e c a u s e in terms of the particular theory used it is thought to be the result of other factors (the independent v a r i a b l e s ) . Such a theory would be about as chaotic and as difficult to absorb as the reality it sought to simplify. In effect. T h e s e are c a l l e d the independent variables. D u v e r g e r u s e d t w o independent v a r i a b l e s in h i s theory: the nature of s o c i a l c o n f l i c t s in a country a n d the c o u n t r y ' s electoral s y s t e m . " T r u e . that factor w o u l d have b e e n u s e l e s s i n e x p l a i n i n g the dependent variable. the obvious a p p l i c a t i o n s are o b v i o u s . the dependent T h r e e things are important if we are to develop g o o d .14 Chapter 2 POLITICAL THEORY Political Theories and Research Topics W H A T D O E S G O O D T H E O R Y L O O K LIKE? 15 CAUSALITY AND In the s o c i a l s c i e n c e s . Simplicity. In theory-oriented r e s e a r c h . T h i s is a subjective j u d g m e n t . whether or not a country is l i k e l y to be a d e m o c r a c y depends on its per capita i n c o m e . broad theory which gives predictions that are not much better than one could get by guessing. b e c a u s e "the applications are o b v i o u s . but before y o u begin y o u r r e s e a r c h . one that is often c a r r i e d out s l o p p i l y a n d in an i n c o m p l e t e way. m o r e v a r i e d w a y s in w h i c h the results c a n be u s e d . the variations in party s y s t e m s that p u z z l e d h i m c o u l d not h a v e b e e n due to differences in the c o u n t r i e s ' electoral s y s t e m s . what makes a theory important is different in engineering research than in theory-oriented research. . they are not taken as determined by any other factor u s e d in this particular theory. not o n l y to y o u r s e l f but a l s o to y o u r a u d i e n c e . of c o u r s e . A theory should be important. theories g e n e r a l l y are stated in a c a u s a l m o d e : " I f X h a p p e n s . it w o u l d p e r f o r m two s i m p l i f y i n g f u n c t i o n s : It w o u l d not o n l y give us a handle on the rather l i m i t e d portion of our environment that it sought to e x p l a i n directly. Students have been k n o w n . that i s . I n d e p e n d e n c e and d e p e n d e n c e are the two roles a variable m a y p l a y in a c a u s a l theory. B u t i n a s m u c h as the borough presidents h a v e little power. It m a y s e e m u n n e c e s s a r y to point this out.

that the study w o u l d borr o w i m p o r t a n c e f r o m this u n d e r l y i n g p h e n o m e n o n . I m m i g r a t i o n and N a t u r a l i z a t i o n S e r v i c e added extra guards and i m p o s e d p u n i s h m e n t s on e m p l o y e r s found to be h i r i n g i l l e g a l i m m i g r a n t s . s i m p l i c i t y . In the late 1980s and early 1 9 9 0 s . If a t h e o r y ' s p r e d i c t i o n s are r e a s o n a b l y a c c u r a t e . w h a t the h e c k . there is also an element of "beautiful s u r p r i s e " to elegant r e s e a r c h . we s a w that he might h a v e i m p r o v e d the a c c u r a c y of h i s t h e o r y ' s predictions by bringing in additional explanatory v a r i a b l e s . making it very c o m p l e x . " T h i s s h o u l d n ' t be that h a r d . E s p i n o s a ( 1 9 9 7 ) found that s i n c e the border c r o s s i n g h a d been m a d e tougher. but this w o u l d have r e d u c e d the s i m p l i c i t y of the theory. P o l i t i c a l scientists often j o k i n g l y refer to this element as the "interocular subjectivity test" of r e s e a r c h — D o e s it hit us b e t w e e n the e y e s ? A g o o d e x a m p l e of r e s e a r c h w i t h beautiful surprise is a study of the i m p a c t of "get-tough" p o l i c i e s against i l l e g a l i m m i g r a t i o n a c r o s s the U n i t e d S t a t e s . it's not s o c i a l s c i e n c e . T h e r e a l test of a theory's v a l u e is whether its subject matter is important and h o w c l o s e it has c o m e to elegance. whereas about 75 percent of A m e r i c a n s w h o were polled identified with some political party. i f the b o r o u g h p r e s i d e n c i e s w e r e taken a s e x a m p l e s o f s o m e b r o a d e r c o n c e p t i n u r b a n p o l i t i c s . . A s i d e f r o m its utility and simplicity. C o n v e r s e wanted to k n o w w h y the level of party identification varied as it did from country to country. 2 A b o u t 80 p e r c e n t of those w h o did k n o w their father's Another reason for the difficulty of attaining elegance in social research is simply that most social science terms are imprecise and ambiguous. he a n d D u p e u x h a d f o u n d that the difference in percentage of party identifiers b e t w e e n F r a n c e a n d the U n i t e d States s e e m e d to b e e x p l a i n e d a l m o s t w h o l l y b y the fact that m o r e A m e r i c a n s than F r e n c h h a d s o m e i d e a o f w h a t party their fathers h a d identified w i t h . S . " Example of Elegant R e s e a r c h : Philip C o n v e r s e I n his article " O f T i m e and P a r t i s a n S t a b i l i t y " ( 1 9 6 9 ) . A s w e c a n see i n T a b l e 2 . and l o w e r levels of party identification in G e r m a n y and Italy. given that subject matter. and predictive a c c u r a c y — i t w i l l be u s e f u l as a tool for s i m p l i f y i n g reality. B e c a u s e the overall extent to w h i c h citizens of a particular country felt bound to the existing parties s e e m e d likely to have something to do w i t h h o w stable politics in that country w o u l d be. not d e c r e a s e d . a s i m i lar poll conducted in F r a n c e s h o w e d that less than 45 percent of the respondents did so ( C o n v e r s e & D u p e u x .M e x i c a n border.1 . P h i l i p C o n v e r s e c a m e about as c l o s e to d e v e l o p i n g an "elegant" theory as one c a n c o m m o n l y do in the s o c i a l s c i e n c e s . Remember that this was an early study. c o m p a r e d w i t h other s c i e n t i f i c a r e a s . At the time of their e a r l i e r study. it 1 T h e fact that most social s c i e n c e theory is not very elegant does not m e a n that it is not good. s e r v e as a u s e f u l m i c r o c o s m for s t u d y i n g the w o r k i n g s of patronage. If a theory c a n s u c c e e d r e a s o n a b l y w e l l at meeting these three c r i t e r i a — i m p o r t a n c e . T h e e n d result w a s that the n u m b e r of illegal i m m i g r a n t s present at any g i v e n time w a s i n c r e a s e d . w h a t the h e c k . M a s s e y and K r i s t i n E . In the e x a m p l e of D u v e r g e r ' s theory. done in 1962. O n the other h a n d . 1 am a l w a y s a m u s e d w h e n people s a y of a question that is b e i n g m a d e to look more difficult than it r e a l l y i s . If it is important to understand h u m a n k i n d ' s behavior. by the stepped-up enforcement. P e r h a p s one day the old s a w w i l l b e c o m e . S i m i l a r l y . s o m e t h i n k i t m a y p e r h a p s b e l a r g e l y b e y o n d e x p l a n a t i o n . it is rare for theory in the s o c i a l s c i e n c e s to a c h i e v e e l e g a n c e . O n e thousand extra border patrol officers w e r e then added e a c h y e a r for several y e a r s afterward. within each row there was practically no difference between the French and A m e r i c a n l e v e l s of party identification. for e x a m p l e . but that we h a v e not y e t c o m e up w i t h a s o c i a l theory that c o u l d s h o w the true potential of our f i e l d . B u t i t m i g h t b e p o s s i b l e . Other studies h a d s h o w n high levels of party identification in B r i t a i n and N o r w a y . In an earlier study.r o u n d b e c a u s e they k n e w it w o u l d be h a r d to get b a c k into the U n i t e d States if they w e n t h o m e to M e x i c o . This problem is addressed in Chapter 3. but plotting the trajectory of an object in a v a c u u m is far s i m p l e r than understanding the motivation of a h u m a n b e i n g . he and G e o r g e s D u p e u x h a d found that. O n e difficulty in creating an elegant theory is that trying to meet any one of the three b a s i c criteria tends to m a k e it harder to meet the other two. In both countries about 50 percent of those who did not know 3 their father's party expressed identification with s o m e party t h e m s e l v e s . illegal i m m i grants w h o o r i g i n a l l y w o u l d h a v e c o m e to the U n i t e d States for only a f e w months of s e a s o n a l labor n o w stayed y e a r . it's not rocket s c i e n c e " — i m p l y i n g that rocket s c i e n c e is the e s s e n c e of difficulty and complexity. S u c h a theory is s o m e t i m e s d e s c r i b e d as elegant. a theory to e x p l a i n the o r g a n i z a t i o n of b o r o u g h Political Theories and Research Topics 17 i s u s u a l l y b e c a u s e the s c o p e o f the theory i s r e s t r i c t e d o r b e c a u s e m a n y o f the e x c e p t i o n s to the theory h a v e b e e n a b s o r b e d into it as a d d i t i o n a l v a r i a b l e s . H i s study is worth l o o k i n g at in s o m e detail. C o n v e r s e took as his dependent variable the strength of the "party identification" of i n d i v i d u a l s — t h e i r sense that they are supporters of one or another of the political parties. N o t to take a w a y f r o m the difficulty of rocket s c i e n c e . D o u g l a s S . an attempt to m a k e a theory m o r e general often w i l l cost us s o m e t h i n g in either the s i m p l i c i t y of the theory or the a c c u r a c y of its predictions. that m a k e s us rethink our w o r l d . 1962). H u m a n b e h a v i o r i s m o r e c o m p l e x than the b e h a v i o r o f p h y s i c a l o b j e c t s — i n fact. it is important to try to develop theories about it. A p i e c e of r e s e a r c h that goes against our expectations. the U . gives us a s p e c i a l k i n d of pleasure.16 Chapter 2 In the e x a m p l e j u s t c i t e d . 2 p r e s i d e n c i e s i n N e w Y o r k . It was not long before further work showed similar effects for mothers! 3 'The choice of this word typifies the aesthetic pleasure—and the vanity—with which researchers approach their work. I t a p p e a r s t o b e p a r t i c u l a r l y h a r d t o a c h i e v e elegant r e s e a r c h i n the s o c i a l s c i e n c e s . " T h i s s h o u l d n ' t be that hard. i t m a y b e that h u m a n b e h a v i o r c a n be u n d e r s t o o d . At any rate. the theory a c c r u e s s o little i m p o r t a n c e d i r e c t l y a s t o l o o k a b s u r d . T h e b o r o u g h p r e s i d e n c i e s m i g h t . even if things do not fall as neatly into p l a c e as we w o u l d like.

7 the older he w a s the m o r e l i k e l y he w a s to identify strongly w i t h a party. F r a n c e m u s t be at an e a r l i e r stage of the p r o c e s s than America.8 X 77%) + (0. ( F r e n c h w o m e n . o n e c o u l d e x p e c t the next g e n e r a t i o n i n F r a n c e t o b e m u c h m o r e l i k e the A m e r i c a n s . the r e l a t i v e l y l o w l e v e l of party i d e n tification i n F r a n c e m u s t h a v e r e s u l t e d b e c a u s e the vote w a s extended later a n d l e s s c o m p l e t e l y there than i n A m e r i c a . simple mathematics that you can play with for yourself. then if 30 percent of the population of country A presently identify with a party. Snell. C a m p b e l l e t a l .g e n e r a t i o n voters w o u l d e x p r e s s i d e n t i f i c a tion. ( S i n c e o f those w h o s e fathers h a d not identified w i t h any party. 19) C o n v e r s e further r e a s o n e d that the 80 percent a n d 50 percent figures might be u n i v e r s a l l y true. called a "Markov chain. W o r k i n g f r o m these two a n g l e s . M o r e o v e r . In the first e l e c t i o n . Inc. " on p. and (0.8 X 9 0 % ) + (0. the difference b e t w e e n the two c o u n t r i e s w a s a result of the fact that the A m e r i c a n s k n e w their father's party so m u c h m o r e frequently than the F r e n c h d i d . J. Kemeny. Thus. 1957). Thompson. B u t i n " O f T i m e and P a r t i s a n Stability. B u t for the p u r p o s e s o f a r g u m e n t . If the rates of transferring identifications are in fact the same in two countries. we would expect them to converge rapidly. " M a r k o v c h a i n s . w e r e f i r s t given the vote in 1945. of c o u r s e . 1 6 1 .5 X 70%) = 59% of country A having an identification.8 X 59%) + (0. and that.7% of country A having an identification. A c c o r d i n g to this s c h e m e . In the next generation after that. It c o u l d h a v e b e e n . s c a r c e l y any voters in a g i v e n c o u n t r y w o u l d identify w i t h a party. Introduction to Finite Mathematics (Englewood Cliffs. Voting studies c o m m o n l y h a d s h o w n that w i t h i n an i n d i v i d u a l ' s life s p a n . C o n v e r s e a n d D u p e u x a c c e p t e d this a s a n interesting finding and d i d not elaborate o n it. Given these figures. would have moved to similar levels of party identification. and (0. H e then s h o w e d that i f h i s a s s u m p t i o n about the p r e v i o u s g e n e r a t i o n ' s l o w l e v e l o f party i d e n t i f i c a t i o n w e r e true. h e r e a s o n e d that the d i f f e r e n c e b e t w e e n F r a n c e a n d the U n i t e d States c o u l d b e e x p l a i n e d e a s i l y i f the p r e v i o u s g e n e r a t i o n i n F r a n c e h a d i n d e e d i n c l u d e d v e r y f e w voters w h o i d e n t i f i e d w i t h a party.4 47. for one thing. A l s o . F i r s t ." C o n v e r s e u s e d the earlier f i n d i n g to suggest a general theory of the p r o c e s s by w h i c h countries developed stable patterns of party preference. T h e first of these derived f r o m the earlier r e s e a r c h on i n d i v i d u a l development. A t the time. of those whose fathers had not identified with a party. the Markov Chains Converse's reasoning is based on some cute. 1 9 6 0 . ( T h i s d e v e l o p m e n t c a n b e seen i n the b o x . C o n v e r s e d e v e l o p e d a s i m p l e theory that predicts the strength of a voter's party identification f r o m j u s t two things: (1) the number of y e a r s the p e r s o n has b e e n eligible to vote ( w h i c h is a d u a l f u n c t i o n of age K n o w father's party Do not know father's party 79. about 50 percent developed an identification of their own. pp.7 party w e r e p o l i t i c a l party adherents. rather than of h i s age itself ( s e e . The process involved here. Converse and Dupeux estimated for France and the United States that about 80 percent of those whose fathers had identified with a party developed an identification of their own. . F r a n c e m u s t b e m o v i n g t o w a r d the l e v e l o f party i d e n t i f i c a t i o n f o u n d i n the U n i t e d S t a t e s .) If this w e r e s o . . pp.5 X 41%) = 67. and h o w l o n g e l e c t i o n s have b e e n h e l d in h i s / h e r c o u n t r y ) . then both F r a n c e a n d the U n i t e d States m i g h t s i m p l y b e e x a m p l e s o f a g e n e r a l p r o c e s s that a l l c o u n t r i e s undergo w h e n their c i t i z e n s are first g i v e n the vote. ( H e knew o n l y that they h e l d for F r a n c e a n d the U n i t e d States. and G. 171-178.. which had started out being quite different. ) T h u s .5 X 10%) = 77% of country B having an identification.1 6 4 ) . but 50 percent of s e c o n d .5 X 23%) = 73. C o n v e r s e c h o s e t o a s s u m e that this w a s not the c a s e . in two generations the two countries. then even though the countries differ greatly in the level of identification at present. and 90 percent of the population of country B presently identify with a party. we would expect to see (0. N.18 Chapter 2 Political Theories and Research Topics 19 T A B L E 2-1 Percent Having Same Sort of Party Identification France USA 81.) T h u s . for e x a m p l e . T h u s . i f the a s s u m p t i o n w e r e true. g r a d u a l l y party a d h e r e n c e w o u l d r e a c h a stable l e v e l . 5 0 percent d e v e l o p e d a n identification o f their o w n . I n d o i n g s o h e b r o u g h t t w o strands o f theory together.: Prentice-Hall.6 50.1% of country B having an identification. a n d N o r w a y ." is described in J. For example. T h e s e c o n d strand o f theory c a m e into play w h e n C o n v e r s e tied h i s theory o f national d e v e l o p m e n t to s o m e older findings on i n d i v i d u a l voters in the A m e r i c a n electorate.J. and assuming that party identifiers have the same number of children as nonidentifiers.8 X 30%) + (0. in the next generation we would expect to see (0. this h a d b e e n s h o w n to be a result of h o w long he h a d b e e n e x p o s e d to the party by being able to vote for it. B r i t a i n . a n d (2) the l i k e l i h o o d that the i n d i v i d u a l ' s father h a d identified w i t h a party ( w h i c h in turn depends on w h a t portion of the father's adult life e l e c t i o n s w e r e h e l d in w h i c h he w a s e l i g i b l e to vote). that the d i f f e r e n c e w a s due to the fact that the F r e n c h d i d not talk to their c h i l d r e n about p o l i t i c s as m u c h as the A m e r i c a n s d i d .

O v e r the y e a r s after it appeared. A f e w e x a m p l e s of predictions f r o m his theory are (1) at the time e l e c t i o n s are first h e l d in a country. or the U n i t e d States. but the s e n s e of the two statements is the s a m e . all s h o u l d identify at the s a m e l o w l e v e l s . party identification is predicted f r o m the i n d i v i d u a l ' s age and the length of time that the country h a s been h o l d i n g e l e c t i o n s . T i l l e y ." but it has not c o o l e d sufficiently in the intervening y e a r s . A n d it included the startling suggestion of a convergence of "mature" electorates to a c o m m o n level of party identification approximately equal to that of B r i t a i n . and b e c o m e s e n m e s h e d i n . S t u d i e s e m p l o y i n g m o r e quantitative data. To Quantify or Not A side issue in the question of h o w to develop elegant theory is the o l d chestnut: S h o u l d political s c i e n c e be "quantitative" or not? T h e r e has been m u c h rhetoric spilled over this. and they are certainly m o r e l i k e l y to give us a c l e a r i d e a of h o w accurate a theory's predictions are. T h u s . 8 6 y e a r s p e r d e c a d e after 1 9 2 2 . B u t it m a y be that the time a n d attention spent in gathering p r e c i s e data m a k e it difficult for the r e s e a r c h e r to appreciate the larger a s p e c t s of a theory. the important thing is to see w h a t relationship degrees of quantification bear to e l e g a n c e in r e s e a r c h . As long ago as 1956. (3) (/the transition rates for a l l countries w e r e r o u g h l y the s a m e as for F r a n c e a n d the U n i t e d States. 2 0 0 3 . R e c o r d s f r o m e a r l i e r e l e c t i o n s are u s u a l l y kept i n f a i r l y g o o d order. although C o n v e r s e ' s theory w a s quite simple. the p a r t i c u l a r s u b j e c t w e are s t u d y i n g affects the extent t o w h i c h i t i s p o s s i b l e for u s t o quantify. I n e l e c t i o n s t u d i e s . H o u s e i n c r e a s e d f r o m 1880 t o 2 0 0 5 o n the a v e r a g e b y 0. greater openness to totally n e w theories. S . b e c a u s e s o m e k i n d s of data by their very nature are m o r e a m e n a b l e to p r e c i s e f o r m u l a t i o n . " s a y s a p p r o x i m a t e l y the s a m e thing a s " F r o m 1 8 8 0 t o 2 0 0 5 . (2) if e l e c t i o n s w e r e interrupted in a country (as in G e r m a n y f r o m 1933 to 1 9 4 5 ) . i t s e e m e d to predict fairly accurately. It is s t r i k i n g . T h u s . T h e theory w a s s i m p l e . R e s e a r c h that is less c o n c e r n e d w i t h m e a s u r i n g things n u m e r i c a l l y . E a c h a p p r o a c h i n v o l v e s c o s t s a n d benefits for r e s e a r c h . but rapidly decreasing. It is a bit hard to pin d o w n w h a t the term r e s e a r c h that p a y s a g o o d d e a l of attention to tends to m a k e best that studies w i t h varying degrees of quantification be c a r r i e d on in any g i v e n field of p o l i t i c a l r e s e a r c h . for the different levels of quantification c o m p l e m e n t e a c h other.68 y e a r s p e r d e c a d e before 1 9 2 2 . essentially. T y p i c a l l y . e a c h a p p r o a c h has its o w n costs a n d benefits. and M e x i c o to test the theory. however. the C o n v e r s e article stimulated a great deal of further r e s e a r c h . . potential for electoral instability in a new electorate. is c o n s i d e r e d quantitative. T h i s oversight m a y w e l l be due to the difficulty of getting " h a r d " data on w h a t goes on in that office. the quantitative r e s e a r c h e r is a b l e to do a great d e a l . On the other h a n d . the pattern we t y p i c a l l y o b s e r v e in E u r o p e a n d A m e r i c a (the y o u n g b e i n g w e a k l y identified. p. is c o n s i d e r e d relatively less quantitative. however. r e p r e s e n tatives s e r v e d a s t e a d i l y i n c r e a s i n g length of time in the H o u s e . I n v i r t u a l l y every f i e l d o f political r e s e a r c h . the w h o l e b o d y o f theoretical exploration. " T h e first f o r m of the statement g i v e s m o r e p r e c i s e i n f o r m a t i o n . I deal w i t h the i s s u e of proper l e v e l s of p r e c i s i o n in C h a p t e r 5. 1997. J a m e s Prothro c a l l e d the dispute "the nonsense fight over scientific method. It is in this w a y that a good p i e c e of theoretical w o r k f e e d s . D a l t o n & W e l d o n . and a greater a w a r e n e s s of the c o m p l e x i t y of s o c i a l p h e n o m ena. the r e s u l t s o f m a n y attitude s u r v e y s are a l s o a v a i l a b l e . the c h a n g e p r o c e e d e d a bit m o r e r a p i d l y in the latter h a l f of that p e r i o d . there is the danger that o v e r c o n c e r n w i t h p r e c i s i o n m a y restrict our c h o i c e of r e s e a r c h to those v a r i a b l e s we c a n m o r e e a s i l y quantify. a n d m o s t voters d o not regard their a c t i o n s a s s o m e t h i n g about w h i c h they n e e d t o m a i n t a i n s e c r e c y . p r e s i d e n c y i s studied b y p o l i t i c a l s c i e n t i s t s . 1 9 7 5 ) . N o r w a y . T h u s . G e r m a n y . s o u r c e s o f quantitative data are quite r e s t r i c t e d . 1972. a n d it is w e l l to r e m e m b e r that no particular degree of quantification h a s a corner on e l e g a n c e . F i r s t o f a l l . A l s o . for i n s t a n c e . S . but generally. a n d i t w a s b r o a d l y a p p l i c a b l e . are m o r e l i k e l y to p r o d u c e s i m p l e . numerical m e a s u r e s of things. " U s i n g data f r o m B r i t a i n . i n C h i n e s e s t u d i e s . there i s c o n s i d e r a b l e s c o p e for q u a n t i f i c a t i o n . a n d tends to make verbal statements about t h e m . u s a b l e theories. p r e s i d e n c y . T h e y a l s o s t i m ulated r e s e a r c h e r s to investigate w h e t h e r in fact the transition probabilities on w h i c h the M a r k o v c h a i n is b a s e d are the s a m e in all i n d u s t r i a l i z e d countries ( B u t l e r & Stokes. o r i n s t u d i e s d e a l i n g w i t h the U . less quantitative r e s e a r c h provides greater breadth. a n d m o s t r e s e a r c h m u s t b e r e l a t i v e l y n o n quantitative. and to test w h e t h e r n e w electorates actually b e h a v e as C o n v e r s e ' s theory predicts they w o u l d ( S h i v e l y . a l l other things b e i n g e q u a l . the rate o f i n c r e a s e w a s 0. 5 3 ) . It simultaneously explained individual behavior and characteristics of political s y s t e m s . thus f u l f i l l i n g the third criterion for " e l e g a n c e . the U n i t e d States. w o r k c a n b e c o n d u c t e d in either a p r i m a r i l y quantitative or a p r i m a r i l y nonquantitative m o d e . F o r our p u r p o s es here. It i m p l i e d a more or less universal f o r m of political development at the m a s s l e v e l — w i t h a prediction of initial. 1 9 6 9 . L e i t n e r . Italy. w h i c h i s w h a t one w o u l d expect o f elegant w o r k . H i s f i n d i n g s s e r v e d a s a s s u m p t i o n s for f o r m a l theoretic w o r k ( P r z e w o r s k i . T h u s . quantitative m e a n s . then party identification levels in a l l electoral d e m o c r a c i e s s h o u l d converge over the next f e w generations toward a s i n g l e v a l u e of about 72 percent. a n d 0 . h o w little the U . it w a s applicable to a w i d e variety of questions. W h a t w a s m o r e . S . l e v e l s of party identification s h o u l d d e c l i n e at a predictable rate. 2 0 0 7 ) . T o g i v e a c r u d e e x a m p l e : " T h e length o f s e r v i c e o f m e m b e r s o f the U .78 y e a r s e v e r y d e c a d e . M o s t people w o u l d agree that p r e c i s e i n f o r m a t i o n is m o r e u s e f u l than i m p r e c i s e i n f o r m a t i o n . C o n v e r s e f o u n d that the theory predicted quite w e l l for a l l five c o u n t r i e s .20 Chapter 2 Political Theories and Research Topics 21 A n y t h i n g i n p o l i t i c a l s c i e n c e c a n b e stated w i t h v a r y i n g degrees o f q u a n t i f i c a t i o n . It is p r o b a b l y s e c o n d f r o m h i s a n d D u p e u x ' s c o m p a r a t i v e study o f F r a n c e and the U n i t e d States. a n d mathematical statements about them. the o l d strongly) s h o u l d not h o l d .

of c o u r s e . what the r e s e a r c h e r in this area must do is to decide w h i c h r e s e a r c h topic is going to p r o d u c e the greatest c h a n g e in the status of e x i s t i n g theories. stating it in the f o r m of a set of h y p o t h e s e s to be tested. B u t o n c e y o u h a v e c h o s e n a general area to w o r k i n . and so o n . a n d y o u s h o u l d r e a d t o f i n d out h o w l i k e l y i t i s that y o u r t o p i c w i l l y i e l d u s e f u l r e s u l t s . y o u s h o u l d r e a d t o see w h a t other w o r k h a s b e e n d o n e o n the p r o b l e m . T h i s is something for w h i c h no rules c a n be laid d o w n . theories about elite political behavior. F i n a l l y . but that she also have s o m e i d e a of w h e r e an existing body of r e s e a r c h is weakest and most needs to be supported or c h a n g e d . s o that y o u w i l l s e e w h e r e y o u are m o s t l i k e l y t o p r o d u c e r e s u l t s that are new. y o u w i l l w a n t to state y o u r thesis so that y o u r results w i l l not duplicate an earlier study. it is p r i m a r i l y a question of u s i n g y o u r time and talents efficiently. Ex post facto argument results when an investigator forms a theory on the basis of certain evidence. o r o n s i m i l a r p r o b l e m s . By c h o o s i n g to study political s c i e n c e y o u h a v e already begun to n a r r o w the field. for e x a m p l e . O n e difficulty in c h o o s i n g the topic is that y o u probably w i l l have to c o m p r o m i s e a m o n g y o u r g o a l s . to attain the three criteria for e l e g a n c e : s i m p l i c i t y . C h o o s i n g a topic that w i l l p r o d u c e important results for theory is undoubtedly m o r e difficult than formulating a question that m a y y i e l d important practical applications. for a single p i e c e of r e s e a r c h m a y s i m u l t a n e o u s l y affect m a n y different theories. T h i s m e a n s that in f r a m i n g any topic for r e s e a r c h . T h i s task requires not o n l y that she be f a m i l i a r w i t h as broad a range of existing theories as p o s s i b l e . as m u c h as p o s s i b l e . M a n y topics relating to defense or to the executive are of this sort. s o m e s o c i a l scientists h a v e developed a very restrictive p r o c e d u r e to serve as a standard in c a r r y i n g on r e s e a r c h . In selecting a topic. but in theory-oriented r e s e a r c h it is harder to d e c i d e h o w important the results of a study are l i k e l y to be. D e c i d i n g w h e r e y o u are l i k e l y to produce theoretical results that are s i m p l e Engineering Research C h o o s i n g a topic is s o m e w h a t s i m p l e r in e n g i n e e r i n g r e s e a r c h than it is in theoryoriented r e s e a r c h . it s h o u l d have a broad and general effect on theory. the topic s h o u l d be one that deals w i t h a p r e s s i n g p r o b l e m and one on w h i c h y o u think y o u are l i k e l y to c o m e up w i t h findings that are both accurate and s i m p l e enough to be u s e f u l . y o u s i m p l y f o l l o w y o u r interests. and then carried out a study of the House Appropriations Committee to test the theory. for example." the r e s e a r c h results must either p r o d u c e totally n e w theories or lead to s o m e c h a n g e in the status of older theories. the r e s e a r c h e r should first frame a theory. and these are likely to be included in any theory based on it. or indirectly through the variety of other theories it affects. but b e c a u s e y o u h a v e seen s o m e r e s e a r c h on it that y o u think w o u l d be rather e a s y to correct. B u t it is certainly something for w h i c h rules have been l a i d d o w n . It is an art. T h u s . y o u are i n v o l v e d at o n c e in the full body of political s c i e n c e theory. T h e s e h y p o t h e s e s p r e s u m a b l y are b a s e d on w o r k others h a v e done in the past. R e s e a r c h on h o w a c o n g r e s s i o n a l c o m m i t t e e reaches its d e c i s i o n s . and y o u certainly w i l l narrow things m o r e before y o u are ready to begin. it will look as if the unique aspects are indeed general. T h e r e is no difficulty in this. or at least point up w h e r e that w o r k p r o d u c e d m i s t a k e n results. and (2) y o u want y o u r results. It is a l s o the most difficult aspect of r e s e a r c h to teach a n y o n e . It is at this s t e p — s e e i n g that a p r o b l e m exists and that there is a good c h a n c e y o u c a n provide n e w insight into i t — that originality and talent are most c r i t i c a l .22 Chapter 2 Theory-Oriented Research Political Theories and Research Topics 23 CHOICE OF ATOPIC T h e c h o i c e of a r e s e a r c h topic is intimately b o u n d up w i t h the elegance of what c o m e s out of the r e s e a r c h effort. T h e m a i n thing t o d o i n l o o k i n g for a t o p i c i s t o r e a d . as I say. T h e r e is no s e n s e in w a s t i n g y o u r time r u n n i n g over ground that has already been w o r k e d u n l e s s y o u think y o u are l i k e l y to d i s c o v e r d i s c r e p a n c i e s . S i m i l a r l y . those parts of the theory would have been found wanting. 4 A c c o r d i n g to this procedure. Y o u m a y decide that for the p r o b l e m nearest y o u r heart. T h i s effect c a n be a c h i e v e d either directly through the p h e n o m e n a it e x p l a i n s . to be "new. whereas if a different test situation had been used. p i c k i n g a particular topic to r e s e a r c h is difficult. there s i m p l y is not e n o u g h material a v a i l a b l e to let y o u study it satisfactorily. c a n affect theories about p o w e r in C o n g r e s s . general theories about c o m m i t t e e s and organizations. Y o u s h o u l d r e a d s o that y o u are c e r t a i n y o u are p i c k i n g a n i m p o r t a n t p r o b l e m . Or it m a y be that a topic interests y o u not b e c a u s e it deals w i t h the most p r e s s i n g p r o b l e m y o u c a n think of. Y o u w i l l r e c a l l that if theory-oriented r e s e a r c h is to be important. DEVELOPMENT OF A RESEARCH DESIGN It m a y be true. then uses that evidence to affirm the theory. 4 and predict a c c u r a t e l y requires the s a m e sort of g u e s s i n g as in engineering r e s e a r c h . this would be ex post facto argument. The danger in this is that any given situation has certain unique aspects. If a political scientist formed a theory of congressional committees on the basis of intimate experience with the House Appropriations Committee. predictive a c c u r a c y . H e r e . T h i s is the c r i t i c a l d e c i s i o n in doing r e s e a r c h . Y o u must j u g g l e all of these d e c i s i o n s around so as to get the best m i x — a topic that w i l l produce results that are as new and as elegant as p o s s i b l e . the first step is to c h o o s e a general area that is interesting and significant for y o u . that c h o o s i n g a topic is not s o m e t h i n g for w h i c h rules c a n be laid d o w n . At the s a m e time. T h e r e s e a r c h e r s h o u l d then gather fresh data . B e c a u s e of an exaggerated fear of ex post facto argument. If the same situation is then used to test the theory. theories about e x e c u t i v e c o n g r e s s i o n a l relations. and importance. T h e important thing in c h o o s i n g a topic is to p i c k one that s h o w s p r o m i s e of g i v i n g y o u n e w and elegant results. To y i e l d elegant results. T h i s i m p l i e s two things: (1) Y o u want t o f o r m u late y o u r topic question so that y o u r results w i l l be l i k e l y to alter existing o p i n i o n on a subject.

In short. general category of causal relationships (that is. b a s e d on our daily e x p e r i e n c e s . it effectively reduces the research task to a selection of obvious hypotheses. T h e i r procedure is m u c h l e s s f o r m a l than the one they p r e s c r i b e for students. I hasten to add that it s h o u l d not be rejected completely. learning more about the subject. the r e s e a r c h e r s h o u l d either reject it or e n s h r i n e it solely on the basis of those new data. the theory qualifies as a plausible explanation of the thing we are trying to explain. A n d they m o s t c e r t a i n l y do not i g n o r e older e v i d e n c e .0 1. C o m p a r e d with the Seven Smallest M e m b e r s of N A T O G r o s s National Product. it is m o r e m e t h o d i c a l and easier to apply for the b e g i n n i n g researcher. A l s o . h a v i n g tested the theory. T h i s is h o w we train p e o p l e to do r e s e a r c h . The Military Balance. It is true that this procedure erects f o r m i d a b l e barriers to protect us f r o m ex post facto argument.6 1. then gradually l o o s e n i n g up as e x p e r i e n c e is gained. a theory). P u z z l e s . but m o s t of us h a v e better sense than to f o l l o w our o w n precepts. B e c a u s e students u s u a l l y operate under stricter deadlines than other r e s e a r c h e r s . w h i c h historically have armed heavily because of their c o n f l i c t s w i t h e a c h other. P e r h a p s a good w a y to learn is by starting w i t h the m o r e c l e a r and obvious p r o c e d u r e s . but it has a n u m b e r of serious d r a w b a c k s . contribute a n y t h i n g n e a r as h i g h a p e r c e n t a g e of their g r o s s n a t i o n a l product t o the c o m m o n a l l i a n c e a s the U n i t e d States d o e s . T h e epitome of this type of thinking is the research design—a c o m m o n student exercise in w h i c h students are instructed to frame s o m e hypotheses (presumably based on their reading) and s h o w h o w they might gather data to test those hypotheses. i m m e r s e t h e m s e l v e s in w h a t other people h a v e written. and think. If they are. and the C o n s t r u c t i o n of Theories O n e w a y to l o o k at c h o o s i n g topics and developing theories is w i t h the realization that they are at heart v e r y c o m m o n s e n s i c a l p r o c e s s e s . . although they u s u a l l y h a v e s o m e operational h u n c h e s about w h a t they e x p e c t t o f i n d . T h e other s m a l l a l l i e s all appear to v a r y i n g degrees to ride on the coattails of the T A B L E 2-2 U . 2. In doing s o . p i c k a f e w propositions about voting behavior. To account for it. S . they grope for interesting theories.500 363 297 295 258 226 184 37 Source: International Institute for Strategic Studies. T h e y are m e r e l y supposed to react to old theories and concepts rather than to think up e n tirely new problems for explanation. 3. 2006. S e c o n d and m o r e important. theories that are elegant and give a n e w slant to things. it m a y m a k e sense for t h e m to w o r k w i t h s p e c i f i c goals in m i n d so that they c a n estimate a c c u r a t e l y at the b e g i n n i n g of a project w h e n it w i l l be c o m p l e t e d . a s s e e n in Table 2-2. N o r do they u s u a l l y " f r a m e h y p o t h e s e s " in any f o r m a l s e n s e before they start to w o r k . that evidence p r e s u m a b l y is to be ignored if the new test gives contradictory results. in w h i c h s c h o l a r s frequently deal w i t h abstractions (and w i t h e a c h other's rival abstractions). B e c a u s e it requires that hypotheses be fixed firmly at the beginning of the research p r o c e s s . He needed to find a topic for his dissertation and he thought that a good w a y to do this w o u l d be to look through a b o o k l a y i n g out theories of elections. In the first p l a c e .7 United States Turkey Belgium Norway Denmark Greece Portugal Luxembourg 12. T h i s is often lost in the forest of s c h o l a r s h i p .4 3. we put it in a broader. A l t h o u g h I h a v e c o n d e m n e d the f o r m a l p r o c e d u r e for d e s i g n i n g r e s e a r c h . R e s e a r c h e r s are not supposed to r e m o l d theory as they go along. argue w i t h c o l l e a g u e s . A s a n e x a m p l e . Defense Spending. F u r t h e r m o r e .0 3. and test them w i t h s o m e data. 2 0 0 5 4. the u s u a l procedure deters researchers f r o m casting about creatively for research topics and theories. it is h a r d to teach s o m e o n e to grope for interesting topics and theories. O n e of its advantages is safeguarding against ex post facto argument. F i n a l l y . E v e n in c a s e s w h e r e a variety of previously existing evidence favors a particular theory. e v e n the e v i d e n c e that suggested a theory to t h e m in the first p l a c e . To test whether the broader theory is valid as an explanation. 2 0 0 5 (billions of $) Defense Spending as Percentage of G r o s s National Product. A s e a r c h of articles in p o l i t i c a l s c i e n c e j o u r n a l s w i l l turn up o n l y a f e w that report r e s e a r c h that f o l l o w s the r u l e s . B u t at heart all theory-oriented research in the s o c i a l s c i e n c e s is of the f o l l o w i n g sort: 1. Something in our lives puzzles us.2 1. we draw other specific predictions from the theory and test these to see whether the theory's predictions are generally true. O n l y G r e e c e and Turkey. O n e of the betterkept secrets in p o l i t i c a l s c i e n c e is that g o o d p o l i t i c a l scientists g e n e r a l l y do not d r a w up r e s e a r c h d e s i g n s before they start to w o r k on a topic. it lends an exaggerated significance to the results of the new study. this approach encourages the researcher to function as a clerk. T h e y p l a y w i t h data. c o n s i d e r the p u z z l e that the U n i t e d States h a s a l w a y s c o n tributed p r o p o r t i o n a l l y m o r e than a l m o s t all other m e m b e r s o f the N A T O m i l i t a r y a l l i a n c e . e s p e c i a l l y a s c o m p a r e d w i t h the s m a l l e r m e m b e r s o f the a l l i a n c e .6 1. O b s e r v a t i o n s .24 Chapter 2 Political Theories and Research Topics 25 w i t h w h i c h to test the theory. A doctoral candidate w h o m I o n c e talked w i t h s e e m e d to me the perfect e x a m p l e of repeated exposure to e x e r c i s e s s u c h as these. It offers researchers no encouragement to think about their theories once research has begun. e v e n though the r e s e a r c h d e s i g n undoubtedly stifles initiative and creativity. and we try to think of an explanation to account for it.6 0.

as the diffuse scientific "conversation" has proceeded. (1999). The structure of argument discussed in this section—see a puzzle. the stakes are higher for democratic leaders. But beyond these two broad positions. This observation. the chance that two democracies will enter a war. leaders typically need satisfy only a small number—the army. many scholars argue that democracies share a number of values in common. In a democracy. It is extremely rare in the social sciences for any relationship to be as invariant as this one. March in Introduction to Models in the Social Sciences (1975). make democracies unlikely to wage war on those with whom they agree in so many areas. 3. and once involved in wars. let me review for you a puzzle currently in real play. One way to explain this would be to treat it as a specific instance of a more general relationship—that in any voluntary cooperative group the member with the greatest resources always tends to make disproportionate contributions. "war" is not always a black-and-white concept. was that a case of democracies attacking another democracy? Similarly. in an autocracy. Accordingly. In the late nineteenth century. They also share an international norm of compromise and bounded resolutions to conflicts. a number of scholars have questioned whether the observation is as infallible as it is claimed to be. they will be more careful about entering wars than will autocrats. for instance. to show the uncertainty and the progression of steps by which scientific discussion of a puzzle usually proceeds in real life. Since the stakes are higher for democratic leaders. it is said. . If the theory holds up well across a variety of such tests. it is unlikely that either of two democrats involved in a conflict will think that he or she has a preponderant chance of winning a war.) It is hard to imagine a democratic leader losing a war and not being voted out of office. but little Denmark hardly makes a difference one way or another. (1999) point out that both of the broad explanations are hard to reconcile with the fairly strong willingness of democracies to initiate wars of conquest or empire. because the democratic leader has to reward more people. especially in the first three chapters. is low. Therefore. 4. (See [1. was that war? 5 6 Granting the gray areas of definition. a relatively insignificant member will see that the group would do about equally well whether or not he contributes. 1. and each will know that the other (because it is a democracy) will fight very determinedly if there is a war. and will tend to sit back and be a free rider. and the influence of "the neople" in these democracies. the democratic United States enthusiastically engaged Spain in a war intended to lift some parts of the Spanish Empire such as Puerto Rico and the Philippines. These characteristics. it will be a plausible potential explanation for the "Denmark problem. That is. A democratic leader needs a higher ratio of potential benefits to costs to justify going to war than an autocrat does. First of all. Or we might look at trade union-supported political parties such as the Labour Party of Great Britain to see whether the largest unions carry a disproportionate share of the burden.] above. a member who sees that the group would fail without her contribution will come through strongly. which was discussed in pp. a leader must satisfy a fairly large number of people in order to stay in power. democratic leaders will The theory used in this example derives from the broader theoretical structure of Mancur Olson's The Logic of Collective Action (1965). As one example. did not prevent these countries from entering into warfare rather happily. democratic France and Great Britain frequently sent armed forces to help acquire additions to their empires. Bueno de Mesquita and his coauthors propose instead a slightly complicated explanation based on rational choice theory. so it is not surprising that this has attracted the attention of some very good minds. People generally do not want to fight wars. In the NATO example. An autocrat might lose a war yet stay in power (Saddam Hussein of Iraq was an example. Bueno de Mesquita et al. he was ousted only later by direct action of American troops in 2003. Since no country is a pure democracy.) Therefore. most political scientists consider the democratic peace to be an enduring relationship of quite unusual regularity. Both "democracy" and "war" are subject to a certain amount of interpretation. how much democracy is enough to have the country count as democratic? When the NATO alliance attacked the Yugoslavian government of Slobodan Milosevic in 1998 (a government which had been elected. T h e discussion that follows draws heavily on the useful review of the democratic peace literature in Bueno de Mesquita et al. but in an election without much real competition). if in both of them the people's concerns count for a good deal. testable predictions from the theory to see whether it is generally valid. poses the puzzle: Why is it that democracies do not attack one another? There have been two primary explanations offered. Given reasonably good information about each other. however. the countries can be expected to try to negotiate their disputes rather than go to war over them." Lest working out a theory or a puzzle should seem too easy or too pat from these examples. devise other unrelated predictions from the general principle in order to test it—is presented skillfully by Charles A. 2. 6-7. Lave and James G. the alliance languishes. and in democracies the wishes of the broad populace count for more than in nondemocracies. These were not wars waged on democracies.S. for instance—or a single party. but it was certainly the case that democratic values. and the relationship is also of great importance in the assessment of democracy. Alternatively. A war involves costs. 6 fight them more determinedly. but also benefits if won. then. Central Intelligence Agency gave covert support to a military coup against democratically elected President Allende of Chile. continuing in office after he lost the first Gulf War in 1991. such as respect for individual liberties and a high value on competition. frame an explanation based on a more general principle. And in the early twentieth century. In this example we might examine chambers of commerce to see whether the biggest merchants in town usually carry most of the freight. A lot of the political scientist's creativity will then come into play in devising other. First. if the United States does not contribute vigorously. A great deal of attention has been directed over the past decade to the so-called democratic peace—the observation that no democracy has ever been observed to initiate war with another democracy. a number of scholars point to the institutional structure of democracies as the explanation. many subtly nuanced theories have been developed.26 Chapter 2 Political Theories and Research Topics 27 United States. When the U.

Perhaps if we view the development task from the perspective of political research in general. you will be doing your best job when you maximize your effect on theory with a given investment of time and money. A broader range of theory is aimed at in studies of the U. though. 8 'Needless to say. for any phenomenon can be examined at different levels of generality.) Note that throughout this discussion of puzzles and explanations. As an example from political science research. I have said that what is produced is a plausible potential explanation. presidency as an example of sovereigns in general and seek to explain the sources and limitations of sovereign power. As we have seen. which may analyze the nature of the office. Of course. especially during his term in office. a question on which previous research has been contradictory. Another way to follow this strategy is to pick an anomaly—that is.28 Chapter 2 Political Theories and Research Topics 29 This is a good explanation. The narrowest range of theory is found in biographies of particular presidents. 2000). so as to resolve the apparent contradiction. fitting these theories to the real world to see how accurately they predict things. 8 MACHIAVELLIAN GUIDE TO DEVELOPING RESEARCH TOPICS There are really no guidelines that I can give you for developing a research topic other than to remind you once again that you are working toward results that are both new and elegant. derived in just the way that I have outlined here. presidency. that "new. Pick a weak theory to work on. 1968). The physical activity of the "study" is the same in both cases. that this rule is not something absolute. . A good example of research stimulated by an anomaly is the study by Sidney Tarrow (1971) of the political participation of French peasants. however. for instance. Richard Neustadt's Presidential Power and the Modern Presidents (1991). and feeding the results of such research back into the body of theory. Perhaps the best way to use the strategy of picking a weak theory is to state a new.S. This is basically a restatement of the first criterion for elegant research. From this description of the process of research. /. we will gain some clue as to its place in the entire scheme. or an analysis of presidents' leadership (Greenstein. they also regularly turned out to vote in greater numbers than did the urban French. Scholars involved in developing theory form a kind of priesthood—admittedly sometimes run less on faith and more according to the laws of laissez-faire and caveat emptor—focused on the common goal of perfecting elegant theories. a theory that is as complicated as the reality it is meant to explain has not gotten you very far). it is not quite as neat as this. 1972). It then becomes the task of the scholar to decide among them on the basis of how broadly they apply. original theory yourself. how accurately they predict. a study of the nature of the presidential office (such as Koenig. (A good exercise for you would be to frame rebuttals of the rational choice explanation from the standpoint of the first two explanations—the "shared values" explanation and the "institutional role of the people" explanation. This is mixed. or Emile Durkheim. or Robert Putnam. To do this.S. This anomaly led Tarrow to probe more deeply into what political involvement means to the French. It typically is the case that more than one plausible potential explanation can be produced for anything needing explanation. another may have the same experience and form a theory of universal gravitation. They also shed new light on F o r example. a given person usually does not handle all these aspects of a particular problem. Most researchers can expect to carry on any or all activities at any time. however. I mentioned earlier the pleasure that researchers feel in creating something that no one has seen before. the sources of executive power. you must: Maximize the generality of the theory you intend to examine. you have a greater probability of refuting a weak theory. them. His conclusions led to the rejection of the traditional "interest in politics" measure. and a third may relate the new evidence to the body of older theory. The researcher in such a biography generally is concerned only with explaining what happened during a particular president's life. and so on. Remember. the greater your contribution will be. though. the celebrants carry on this process by developing new theories and adapting old ones. how simple they are (remember. One person may be hit on the head by an apple and form a theory of falling apples. another may do a descriptive study of a partcular case. Tarrow was struck by the fact that although French peasants regularly responded to surveys by stating that they were not very interested in politics. But also. but it also draws on Karl Marx. If empirical research is motivated by a desire to affect the state of theories. and so on. but it certainly has not represented the last word in political scientists' collective effort to work out an explanation of the puzzle. In this case. original theories" that are also elegant are hard to come up with. One's work is something brand new. either by confirming them or by working changes in 7 2. Note. the difference lies solely in the level at which the researcher works. For one thing. A still broader range of theory is seen in studies that use the U. Implicit in this chapter is the view that scholarly research represents a loose cooperative effort among many people. But the basic building blocks of political explanation are plausible potential explanations. One person may work simply at clarifying theories. a study of presidents' political personalities (Barber. the way in which presidents' personalities can influence their behavior in office. and modifies the meaning of their work. often operates at this level of generality. with a sense of pride in being part of an ongoing tradition. The weaker the previous confirmations of a theory have been. if your research does confirm the theory. consider the variety of studies done on the presidency. your hypotheses are necessarily in need of proof. we can derive a set of rules to guide the individual researcher. and any evidence you can buttress them with will be important. your work will again be more significant than if the theory already had a good deal of confirming evidence.

though he hastened to add that his measure of "instability" w a s questionable and might have been a source of the disparity. On this hinged the apparent contradiction. FURTHER DISCUSSION A n intriguing b o o k relevant to this chapter is A r t h u r K o e s t l e r ' s The Act of Creation ( 1 9 6 9 ) . so it m a k e s sense to m a k e y o u r reader's life e a s i e r and y o u r m e s s a g e more c o m p e l l i n g . If y o u c a n find an anomaly having to do with a significant area of political theory. because earlier investigators generally have noticed them already and have tried to resolve them. Presumably. T u f t e ' s The Visual Display of Quantitative Information ( C h e s h i r e . T h o u g h they were in fact interested in politics. Arthur A. An excellent introduction to good graphic presentation HarperCollins. M u c h of the rest of this b o o k f o c u s e s on s u c h p r o b l e m s . Although this is the usual procedure. however. T h e T h i r d W o r l d w o u l d provide a laboratory of less-developed countries with a wider range of cultures and e c o n o m i c circumstances. y o u might replicate a study in a different context from the original one. People often think how y o u say s o m e t h i n g is separable f r o m what y o u say. elegant theories is found in the first three chapters of L a v e and M a r c h ( 1 9 7 5 ) . It i n v o l v e s s u c h things as the a c c u r a c y of y o u r deductions f r o m the theory to the s p e c i f i c situation. I stated in this chapter that most social science theories are causal. E = mc . G o l d s m i t h ' s negative results m a y thus point up the importance of organized interest groups in O l s o n ' s theory. A n o t h e r m a y note an a n o m a l y in a theory and organize an experiment to resolve the problem. B u t e a c h o f the r u l e s . T h i s m e a n s . T h u s . What would a noncausal theory look like? Under what circumstances would it be likely to be used? {Hint: Consider Einstein's famous theory. y o u might choose a problem y o u believe has just not been sufficiently researched. that prolonged political stability produces rigid social and e c o n o m i c structures that retard e c o n o m i c growth. it is neither the only nor necessarily the best approach. 1983). W h a t sort of theory might y o u d e v i s e to e x p l a i n t h i s ? B e s i d e s anomalies. A n e x c e l l e n t introduction t o b u i l d i n g 3. is that the context is different. T h i s really j u s t boils d o w n to m a k i n g sure y o u say w h a t y o u think y o u are s a y i n g . or simpler interpretation on them. Present your theory as clearly and vividly as possible. G o l d s m i t h found that examination of these c o u n tries did not support O l s o n ' s theory. is E d w a r d R. rather c o n t r o v e r s i a l ideas about appropriate w a y s to build and test theory is R o n a l d B r u n n e r ' s " A n ' I n t e n t i o n a l ' Alternative i n P u b l i c O p i n i o n R e s e a r c h " ( 1 9 7 7 ) . S u c h groups are strong and active in the countries O l s o n studied. write w e l l and present any g r a p h i c information w e l l . and so o n . w a s based mostly on the experiences of the West and Japan.) . y o u c a n be certain that any plausible efforts at resolution w i l l be interesting. What drawbacks might it involve? In what alternative ways might one develop a theory? 3. for e x a m p l e . but not in the T h i r d World context that G o l d s m i t h studied. A l l are following the rule of m a x i m i z i n g their impact on theory. they denied this because of their antipathy to the parties. also has a b e n e f i c i a l effect on the field as a w h o l e .k n o w n theory. Make the connection between the general theory and your specific operations as clear as possible. at least to my taste. broader. A n o t h e r e x a m p l e of a n i c e a n o m a l y j u s t waiting to be a n a l y z e d is the fact that in the 2 0 0 0 e l e c t i o n . N e e d l e s s to say. perhaps one in w h i c h all variables have not been covered. In fact. What changes would this require in the definition of elegance? 2. H o w to write w e l l and d e s i g n g r a p h i c d i s p l a y s w e l l are b e y o n d the scope of this book. I c a n suggest two truly good b o o k s that w i l l help y o u . F o r w r i t i n g . the entire field w i l l benefit f r o m an e x a m i n a t i o n of those theories m o s t in need of investigation. the a c c u r a c y with w h i c h y o u have m e a s u r e d things. If i n d i v i d uals c h o o s e those p r o b l e m s of theory that have so far h a d the w e a k e s t v e r i f i c a t i o n . B u t if y o u look at the " r e d s t a t e s " — t h e states he c a r r i e d — they are on the w h o l e poor states w i t h low average i n c o m e s . since F r e n c h peasants were fiercely independent and understood "interest in politics" to i m p l y some sort of approval of the existing parties. C o n n . Donald Moon (1975). just as empirical research is. I r e c o m m e n d W i l l i a m K n o w l t o n Zinsser's On Writing Well: The Classic Guide to Writing Nonfiction ( N e w Y o r k : 1998). I f the purpose o f theory is to c h a n g e p e o p l e ' s understanding of the w o r l d . work in normative philosophy or in formal theory could be evaluated in terms of elegance. the h i g h e r voters' i n c o m e s w e r e . these guidelines s h o u l d remain flexible enough to allow different m i x e s of research strategy. Y o u m a y have noticed that these three rules r e s e m b l e the criteria for elegance fairly c l o s e l y . A Machiavellian researcher wants to i n f l u e n c e as m a n y people as p o s s i b l e . This chapter has implied that the usual way to come up with a theory is to focus on a body of observations and look for regular patterns in them. An article that presents s o m e interesting. G o l d s m i t h (1987) noted that M a n c u r O l s o n ' s (1982) w e l l . 4. T h e most likely reason. A third m a y look over previous research findings and place a new. T h e r e is no one "scientific method" involved here. O l s o n ' s theory relies on the negative e c o n o m i c effects of organized interest groups. S o m e questions y o u might c o n s i d e r are: 1. the m o r e l i k e l y they w e r e to vote for G e o r g e W. Fortunately. then the w a y the theory is c o m m u nicated to them is an integral part of the development of the theory. A n o m a l i e s s u c h as this are hard to c o m e by. the nature of F r e n c h voters' attachment to political parties. B u s h . : G r a p h i c s P r e s s .30 Chapter 2 Political Theories and Research Topics 31 O n e person m a y find a tool that measures a variable better than had been done before and then s i m p l y apply it to sharpen previously e x a m i n e d relationships. Y o u also m a y have noticed that the b a s i c p h i l o s o p h y b e h i n d t h e m — " D o r e s e a r c h that m a k e s as big a s p l a s h as p o s s i b l e " — r e a d s like a guide for ruthless and h u n g r y assistant p r o f e s s o r s . A n o t h e r perspective i s offered persuasively by J. d e r i v e d f r o m m y u n d e r l y i n g M a c h i a v e l l i a n outlook. but that is s i m p l y not true. e a c h r e a l l y requires a b o o k in its o w n right.

the m e a n i n g of that theory w o u l d be left to the reader's judgment. f r o m hot to c o l d . 2. those people w h o are registered with the party. is a m u l t i d i m e n s i o n a l concept that involves a m o n g other things temperature. In C h a p t e r 2. it often connotes "free-spending. for e x a m p l e : 3 ." might m e a n : 1. however. T h e a d m i n i s t r a t i o n p r i v a t e l y p a s s e d t h e w o r d t o its s u p p o r t e r s t o u n d e r c u t the bill. those people w h o do voluntary w o r k for the party at s o m e g i v e n time. 5. Just to look at one of its dimensions. it is preferable to frame t h e o r i e s i n terms of u n i d i m e n s i o n a l rather than multidimensional concepts." "go all-out. it is natural to use this flexible w o r d as a club in describing candidates. W h e n r e s e a r c h e r s write about "the political party. Debates in political science sometimes resolve themselves into the fact that participants are using the same term in different w a y s . the " g o o d n e s s " of weather varies. if a theory states that a temperature b e l o w x degrees c a u s e s the spotted ibex to stop breeding. In the election of 1996. I shall c a l l such simple concepts unidimensional. In this chapter we shall look at the E n g l i s h language in general and at the varied theories that m a y be hidden in ordinary language. " T h e w e l f a r e b i l l failed in the H o u s e because the administration did not fight for it. but did not c a m p a i g n a s h a r d a s i t u s u a l l y d o e s for bills it w a n t s . f r e e d o m — a r e i m p o r t a n t in d o i n g g o o d r e s e a r c h . some of w h i c h are probably even inconsistent." "want the bill. m mkM Unfortunately. the officers of the party. the E n g l i s h language is b a d l y d e s i g n e d to function as a m e d i u m for stating r e s e a r c h results c l e a r l y and without ambiguity." they m a y be referring to any or all of these v a r i o u s d i m e n s i o n s of the term. the late Senator Paul Wellstone of Minnesota 32 . T h e a d m i n i s t r a t i o n c a m p a i g n e d as h a r d as it u s u a l l y d o e s for bills it w a n t s . but failed to get it p a s s e d — a n d t h e w r i t e r is in t h e o p p o s i n g party. M o s t E n g l i s h w o r d s c a n take on a variety of m e a n i n g s . very simple concepts w h o s e meaning is unequivocal. humidity. we shall be concerned with framing theories in terms of concepts that cannot easily be subdivided into further distinct meanings. c o n s i d e r political party. 6. and love. but n o t t h e o n e the w r i t e r w a n t e d . A w e l f a r e bill w a s p a s s e d . u n a m b i g u o u s w a y — a f o r m o f s e l f ." on the other h a n d . If the theory stated that " b a d weather" c a u s e d the spotted ibex to stop breeding. T h u s . depending on the context and on the m o o d of the reader. they m a y m e r e l y s u c c e e d in creating c o n f u s i o n . Note that in these statements there are many additional phrases that are ambiguous: "pass the word. T h e r e f o r e . multidimensional concepts. in contrast to concepts w h o s e meaning m a y vary f r o m reader to reader and even from time to time. but not as hard as the writer would have done. No w e l f a r e bill p a s s e d at all. In this c h a p t e r I s h a l l stress the n e e d to state t h e o r i e s i n a c l e a r . economic development. T h i s c o n c e p t c a n be b r o k e n d o w n into at least six c o m p o n e n t d i m e n s i o n s : those people w h o vote for the party in a given election. and degree of c l o u d cover. w i n d velocity. " G o o d weather. W h i c h e v e r of these is the c a s e . for instance. Is it rain that discourages the i b e x ' s ardor? Or do high w i n d s m a k e h i m think of other things? Is it the heat? T h e h u m i d i t y ? As we c a n see f r o m this e x a m p l e . C o n s i d e r the f o l l o w i n g e x a m p l e : As a description of c l i m a t e s . u n l e s s p o l i t i c a l scientists s p e c i f y w h i c h d i m e n s i o n s apply. 4 . T h e w o r d liberal. T h e a d m i n i s t r a t i o n w a n t e d t h e bill. either denotative or connotative—in other words. and those w h o are r u n n i n g or elected as candidates under the party l abel . w h i c h is still fairly current in E u r o p e .Chapter 3 Importance of Dimensional Thinking ENGLISH AS A L A N G U A G E FOR RESEARCH 33 I have a s s u m e d so far that if p o l i t i c a l r e s e a r c h is to be u s e f u l . the statement. a m i n i m a l requirement is that its results s h o u l d m e a n the s a m e thing to any two different readers." "campaign for the bill." S i n c e liberal has generally unpopular connotations in the United States and is multidimensional in complex w a y s . If there is a variation along any one of these separate d i m e n s i o n s ." although in its original usage. conservative. that is. "temperature" is a u n i d i m e n s i o n a l concept. T h e a d m i n i s t r a t i o n w e n t all-out for t h e bill.d i s c i p l i n e that m u s t a c c o m pany such freedom. amount of precipitation." As another e x a m p l e of a m u l t i d i m e n s i o n a l concept. the failure c o u l d have resulted f r o m any n u m b e r of other events. that statement w i l l mean the s a m e thing to every reader. those people w h o identify t h e m s e l v e s in their m i n d s w i t h the party. enjoyment. Temperature c a n vary along only one d i m e n s i o n ." "undercut the bill. A f e w further e x a m p l e s of m u l t i d i m e n s i o n a l terms are: power. it means "favoring a limited role for government. I a r g u e d that f l e x i b i l i t y a n d o r i g i n a l i t y — i n a w o r d . has many dimensions. intelligent. T h e E n g l i s h language does not a l w a y s m a k e this easy. To use an e v e r y d a y e x a m p l e . for instance. In particular.

followed by A and B and by B and C. p a c i f i s m . Suppose that this political scientist discovered the following things about countries A. For example. T h e polite substitution of the t e r m developing nations for underdeveloped countries in the 1 9 5 0 s a n d 1 9 6 0 s i s another e x a m p l e . for e x a m p l e . and C: Between Countries A and B B and C A and C Volume Trade/Year $10 billion $5 billion $20 billion Items of Mail/Year 20 million 30 million 10 million p r a c t i c a l refutation m u c h l o n g e r than i f they w e r e m a d e e x p l i c i t . and alliance—there would have been no problem in comparing the countries. So. they confront an impossible task unless they have defined the variable in such a way that it consists of a single dimension. can help us to keep clear the meanings of the words we use and to avoid this kind of confusion. By associating several distinct things with a single term. T h i s p r o c e s s of theory d e v e l o p m e n t is u n c o n t r o l l e d . He responded that his vote w a s actually "deeply conservative" because it reflected the moral values of Minnesotans. it is a p r o c e s s that g i v e s us little c o n f i d e n c e in the n e w t h e o r i e s that are p r o d u c e d . the word wise has long connoted (1) broad practical knowledge. "interaction" involves a number of dimensions: alliances. T h e r e is no p r o v i s i o n m a d e for greater i n f l u e n c e o n the p r o c e s s b y those w h o k n o w m o r e about the s u b j e c t . D i c t i o n a r i e s . if I use the word wise rather than intelligent. T h e j o b of the social analyst. Ordinary Language A s w e have j u s t seen. But unless its component dimensions are made explicit. change as c o m m o n u s a g e r e f l e c t s c h a n g e d m o o d s a n d n e w e x p e r i e n c e s . that A and C share the greatest trade. and (4) a background of long practical experience. it has implied the theory that the older (and thus the more experienced) one got. In a s e n s e it is v e r y d e m o c r a t i c — p r o b a b l y m o r e d e m o c r a t i c than w e w o u l d l i k e . B and C exchange a great deal of mail but do not trade together as much as A and B." I t i s i n j u s t s u c h a w a y that w o r d u s a g e g r a d u a l l y c h a n g e s . w h i c h are so important to art. another fault in multidimensional words is that each such word is itself a theory of sorts.34 Chapter 3 3. A n o t h e r p r o b l e m w i t h the p r o c e s s is that these c h a n g e s often p r o c e e d at a s n a i l ' s p a c e . a multidimensional word implies the theory that these things go together. . we pay a high price for this benefit. my reader picks up this little theory subliminally embedded in my statement. Difficulty with measurement. B u t in the social s c i e n c e s . B e c a u s e s u c h attitudes as a d e s i r e for e c o n o m i c a c t i v i s m o n the part o f g o v e r n m e n t . of course. w h i c h h a s l e d to d i m i n i s h e d u s e o f the w o r d i n e g a l i t a r i a n A m e r i c a . Among other things. Ambiguous associations. the better one understood things. From the standpoint of the social scientist. and South Korea. B u t the p r o c e s s b y w h i c h they c h a n g e is not a v e r y s a t i s f a c t o r y o n e . m a n y E n g l i s h words involve more than one d i m e n s i o n . (2) a highly developed understanding of relationships. r e f l e c t i n g different v i e w s o f the w o r l d . it is a poorly articulated and poorly controlled theory. for instance. Poor communication. w h i c h o n l y Between which two countries is there greater interaction? A and C trade together a great deal but are not allies. but their m e a n i n g s w e r e c o n s i d e r a b l y c h a n g e d . exchanges of mail. ( T h e w o r d s w e r e b o r r o w e d f r o m a E u r o p e a n c o n t e x t . a d d e d t o the s u b s e q u e n t c o n f u s i o n . "amount of interaction" is not a unidimensional variable. A r t and rhetoric c o u l d not be restricted to unidimensional vocabularies. T h e w o r d s Alliance Yes Yes No liberal a n d conservative are a c a s e in point. T h e connotations accruing to a w o r d that means many different things simultaneously add richness to our language. a n d its d e g r a d a t i o n i n s u c h s l a n g a s " D o n ' t get w i s e w i t h m e ! " a n d " w i s e guy. Importance of Dimensional ininmng ->3 w a s described by his opponent as "embarrassingly liberal" for his vote against a bill reducing welfare payments. and so on. (3) a contemplative bent. as is indicated by their popularity. T h u s . It m a y be an u n a r t i c u l a t e d d i s l i k e for the theory i m p l i e d by the w o r d wise. reality has caught up with rhetoric in the explosive economic growth of countries such as China. W a s Wellstone's vote "liberal" or "conservative"? T h e argument proceeded almost irrespective of its merits. s e r v e t o s l o w d o w n the p r o c e s s of c h a n g e . For example. T h i s c h a n g e i n u s a g e r e f l e c t e d the c h a n g e d status o f the T h i r d W o r l d a n d a t least s o m e h o p e f u l t h i n k i n g about w h a t w a s happening there. Had the investigator chosen three separate measures—trade. 2. Because a reader can never be certain what meaning an author Whenever social scientists want to measure a variable. it is the poet's j o b to create a rich confusion of varied nuances. M u l t i d i m e n s i o n a l concepts are valuable and useful in ordinary language. and i n t e r n a t i o n a l i s m often s e e m e d t o g o together w h e n o n e l o o k e d at the p o l i t i c a l elite in the U n i t e d S t a t e s . India. volume of trade. A poet is pleased if she chooses a w o r d w h o s e meanings give the reader pause for thought. embedded in multidimensional words. tourism. multidimensional words in effect hamper communication. the t e r m liberal c a m e to denote their p r e s e n c e a n d conservative their a b s e n c e . B. and these component d i m e n s i o n s are rarely specified in ordinary language. b y c o d i f y i n g w o r d u s a g e . t h e o r i e s that are e m b e d d e d i n m u l t i d i m e n s i o n a l w o r d s s u r v i v e intended. 1 1. We return to such problems of measurement in Chapter 4. framing theories in terms of unidimensional concepts. c o n c e r n for the legal rights o f i n d i v i d u a l s . M u l t i d i m e n s i o n a l w o r d s h a m p e r s o c i a l scientists in at least three w a y s : T h e o r i e s such as this. T h e strategy I am introducing here. get in the w a y of analytic thinking. however. R i c h connotations. E v e r y o n e w h o u s e s a p a r t i c u l a r w o r d takes part in the p r o c e s s . We can confidently state. is to bring order out of social chaos by means of simple theories. ) T h e i m p l i e d t h e o r y — t h a t these attitudes tended to c o i n c i d e — w a s n e v e r v e r y a c c u r a t e as a d e s c r i p t i o n e v e n of the elite a n d 'Today. and a p e r s o n ' s i n f l u e n c e is m o r e or l e s s p r o p o r t i o n a l to the n u m b e r of p e o p l e w h o h e a r her u s e the w o r d . consider a political scientist who wanted to measure the "amount of interaction" between various nations. mail flow. Now. Clearly. but they do not affect the quality of the c h a n g e .

political scientists o v e r the last f e w d e c a d e s have explicitly developed the term national integration. they must invent the w o r d s t h e m s e l v e s . T h e r e are many d i m e n s i o n s i m p l i e d in the term: political c o n s e n s u s w i t h i n the nation. the word has the same root as "prove. H o w e v e r . A l t h o u g h it is a l w a y s n e c e s s a r y to think and w o r k w i t h u n i d i m e n s i o n a l c o n c e p t s in the s o c i a l s c i e n c e s . " "velocity. too. T h e s e are to be v a l u e d in their o w n right. Natural scientists must create their o w n vocabularies. It does not bother us w h e n a p h y s i c i s t tries to reduce the complex motion of a particle to unidimensional concepts. If ordinary language does not provide u n i d i m e n sional w o r d s for the things s o c i a l scientists want to say. even though the changed meaning of the word has rendered the phrase meaningless. w h i c h l e a d s me to 1. T h e m e a n i n g of a theory is not u n a m b i g u o u s l y clear if it is c o u c h e d in m u l t i d i m e n s i o n a l words. y o u s h o u l d not use them c a s u a l l y . as if they h a d g e n e r a l v a l i d i t y . and so on. w i d e spread c o m m u n i c a t i o n and personal interchanges w i t h i n the nation. yet s o c i a l scientists often must create their o w n vocabulary. T h e y m a y add clarity. writers in any analytic field suffer the same handicap. however. A l t h o u g h e x p l i c i t m u l t i d i m e n s i o n a l w o r d s are more u s e f u l than ordinary language. . s o that even today there are still many people who use 2 "hurtles through space. that they all b e l o n g together. B u t there is another side to the q u e s t i o n . that y o u have built y o u r o w n j u x t a p o s i t i o n o f d i m e n s i o n s into the w o r d . we must a c k n o w l e d g e the fact that the E n g l i s h language contains many words that hold a rich variety of meanings and w o u l d be p o s s i b l e to do without the term national integration. temper that s t a n c e s o m e w h a t ." The phrase about exceptions has continued. W h a t most people m e a n b y " s o c i a l s c i e n c e j a r g o n " i s the u n i d i m e n sional v o c a b u l a r y invented by s o c i a l scientists for their use in analytic r e s e a r c h ." and was used in phrases such as "submit to a proof by fire. B u t somehow the loss of richness seems more painful in the social sciences than in other fields. " N a t i o n a l integration" m a y provide an e x a m p l e of s u c h a concept. the thing we care most passionately about. w h e n f a c e d with a h i g h score on a variable that c o m b i n e s d i m e n s i o n s A. but it need not prevent them f r o m w r i t i n g c l e a r and graceful prose. an amusing one. At the s a m e time. T h i s is because the social scientist deals with people. as c o m p a r e d w i t h ordinary l a n g u a g e . 3. is offered by the old saying "the exception that proves the rule. or whatever. w h i l e c h a n g e s in usage do p r o d u c e c h a n g e s in the theories i m p l i e d by m u l t i d i m e n s i o n a l w o r d s . A l l t h i s w e n t u n n o t i c e d u n t i l a t l e a s t the late 1 9 4 0 s . for e x a m p l e . for e x a m p l e . W r i t i n g f r o m w h i c h the r i c h n e s s o f v a r i e d connotations h a s been r e m o v e d i s flat. legal integration. I n c l u s i o n of m u l t i d i m e n s i o n a l w o r d s in a t h e o r y c o n f u s e s that t h e o r y w i t h a d d i t i o n a l t h e o r i e s that are i m p l i e d b y t h e e x i s t e n c e o f the m u l t i d i m e n s i o n a l w o r d s t h e m s e l v e s . this is p r e s u m a b l y not the w a y we want to d e v e l o p s o c i a l theory. but should cause us to reconsider the rule. T h e result h a s not been fully satisfactory. if y o u are w o r k i n g w i t h so m a n y d i m e n s i o n s s i m u l t a n e o u s l y that readers w o u l d have difficulty k e e p i n g track o f t h e m ." and "feedback loops. T h e r e are a great n u m b e r of d i m e n s i o n s i n v o l v e d . we are disturbed.36 Chapter 3 Importance of Dimensional Thinking 37 i t h a s p r o v e d t o b e quite i n a c c u r a t e i n d e s c r i b i n g p e o p l e i n g e n e r a l . Actually. S o c i a l scientists s h o u l d be uncomfortable if hidden a w a y in their statement of a theory are additional little theories that are implicit a n d uncontrolled. Of c o u r s e . L e t me s u m up the argument thus far: S o c i a l scientists s h o u l d use u n i d i m e n sional language for three r e a s o n s : PROPER USE OF MULTIDIMENSIONAL W O R D S I have a r g u e d that we s h o u l d try to use o n l y i n d i v i s i b l e d i m e n s i o n s as c o n c e p t s in p o l i t i c a l s c i e n c e . It D e s p i t e our prescription. B. or of protests. a m o n g the nation's people. the words liberal a n d conservative in this w a y . T h i s is one r e a s o n that s o c i a l s c i e n c e writing so often strikes readers as flat and c o l d . T h e use of an explicit c o m b i n a t i o n of separate d i m e n s i o n s s u c h as this has s o m e of the advantages of both " o r d i n a r y l a n g u a g e " and u n i d i m e n s i o n a l l a n g u a g e . B u t w h e n the political scientist describes politics as consisting of " s y s t e m inputs. and C. S u c h s u m m a r y constructs m a y add grace and interest to the presentation of y o u r results and ideas. A c c o r d i n g l y . to refer to the general p r o c e s s ." and "acceleration" rather than saying that it d i m e n s i o n s . the p r o c e s s by w h i c h these c h a n g e s are made is c a p r i c i o u s . and w o r k directly with these d i m e n s i o n s . there is a w i d e s p r e a d feeling a m o n g political scientists that these things tend to go together to constitute a general p r o c e s s . T h i s is s i m p l y one of the costs we must pay to write a n a l y t i c a l l y . in fact. A n d they help t o keep y o u r theory p a r s i m o n i o u s . or that B w a s h i g h and A and C w e r e low. I t t u r n s out. it m a y be c o n v e n i e n t to put separate d i m e n s i o n s together in explicit m u l t i d i m e n s i o n a l c o m b i n a t i o n s ." and the m i n d does not rebel. and s o m e of the disadvantages of e a c h . and they m a y c o m b i n e in odd w a y s . the family. these contrived concepts retain m a n y of the disadvantages "Another example of meanings of words being slow to change. that p e o p l e w h o are l i b e r a l o n i n d i v i d u a l s ' rights are often c o n s e r v a tive o n e c o n o m i c i s s u e s . of ordinary l a n g u a g e . A reader cannot tell. T h i s p r o c e s s might not be at all e a s y to d i s c e r n if we were f o r c e d to w o r k s i m u l t a n e o u s l y w i t h the large n u m b e r of u n i d i m e n s i o n a l c o n cepts i n v o l v e d in it. w h e t h e r this means that A and B w e r e h i g h and C low. An exception does not prove a rule. this is not meant to e x c u s e poor writing in the s o c i a l s c i e n c e s or a n y w h e r e e l s e . Variables c a n n o t be m e a s u r e d u n a m b i g u o u s l y if they h a v e b e e n defined in a m u l t i d i m e n s i o n a l way. but w h e n social scientists try to simplify the c o m p l e x reality of politics." the mind does rebel. with " p r o o f included in it. U n i d i m e n s i o n a l language is a m i n o r h a n d i c a p under w h i c h s o c i a l scientists operate. but it w o u l d be a w k w a r d . but there does appear to be a need for s u c h a s u m m a r y term. a f e e l i n g . S u c h explicit c o m b i n a t i o n s of d i m e n s i o n s have the advantage. 2. T h u s . too. composed of these various connotations. The secret is that centuries ago to "prove" meant to submit to a test. T h e physicist m a y describe a body's motion in terms of " m a s s ." Taken at face value—as it usually is—this phrase is idiotic." " s y s t e m outputs.

B a c h r a c h and Baratz. F i g u r e 3-1 l a y s these out as four d i m e n s i o n s of power. and in determining the agenda of i s s u e s f a c i n g the city but m a y not C o n c e p t u a l a n a l y s i s of this sort is terribly important to r e s e a r c h . Rather. i t m i g h t b e that a s m a l l g r o u p o f p o w e r f u l f i g u r e s h a d d e t e r m i n e d w h a t i s s u e s w o u l d b e o p e n for d i s c u s s i o n . e x p l i c i t d i m e n s i o n a l a n a l y s i s o f a body o f s c h o l a r l y w o r k . B u t this c o n c e p t is m u l t i d i m e n s i o n a l and notoriously difficult to pin d o w n . C o n n e c t i c u t ( D a h l . D e s p i t e the fact that a w i d e r a n g e of p e o p l e h a d p a r t i c i p a t e d i n N e w H a v e n d i s c u s s i o n s . as I have i n d i c a t e d . whether one has influenced people's political identities. T r u e p o w e r lay not in the m e c h a n i c s of d e c i s i o n m a k i n g . If one c o u l d c o n v i n c e white w o r k e r s in N e w H a v e n that the m o s t important thing to them w a s education for their c h i l d r e n rather than defending w h i t e s ' prerogatives against b l a c k s — o r i f y o u d i d the reverse o f t h i s — y o u w o u l d h a v e already pretty w e l l determined the p o l i t i c a l o u t c o m e s . e a c h set up for one particular r e s e a r c h u s e . nor at that t i m e h a d there b e e n a n y interest in the r e l a t i v e s i t u a t i o n of women vis-a-vis men. E a c h of these conceptualizations actually emphasizes a different d i m e n s i o n of power: D a h l e m p h a s i z e s whether one has participated in a decision. for instance. Major Dimension Four D i m e n s i o n s of Power A Political Decision debates o f the last f e w d e c a d e s c a n b e c l a r i f i e d b y d i m e n s i o n a l a n a l y s i s . and Digeser. they m a y p r o d u c e c o n f u s i o n as great as that r e s u l t i n g f r o m ordinary language. a n e w s p a p e r editor m a y have b e e n i n v o l v e d in the setting of identities. for i n s t a n c e . H o w e v e r . A l t h o u g h the scholars involved in these arguments have generally v i e w e d their o w n version of power as ruling out one or more of the others. either in the d e c i s i o n itself or in setting the agenda. whether one has helped set the agenda for public decisions. L u k e s . On a question of park policy. It m a y h a p p e n . Peter D i g e s e r ( 1 9 9 2 ) drew o n the w o r k o f M i c h e l F o u c a u l t t o suggest that real p o w e r c o n s i s t s not of determining w h a t people w i l l see as in their Identity influenced did did did did did did did did did did did people's not not not not not not definition of their identities did did did did not not Interests influenced people's understanding of interests Participation participated did not did not did did not did did not did did not . H e noted that o n different i s s u e s . K r o e b e r a n d K l u c k h o h n ( 1 9 5 2 ) counted several h u n d r e d different w a y s i n w h i c h the w o r d culture w a s u s e d by anthropologists. S t e v e n L u k e s a d d e d a further c o m p l i c a t i o n . for i n s t a n c e . true p o w e r l a y in the ability to define for people w h a t their interests w e r e . in i n f l u e n c i n g p e o p l e ' s understanding of their interests. W h e n these related w o r d s are taken as a group. T h e p o w e r any person holds in a particular political d e c i s i o n m a y be a n a l y z e d in terms of her position on the four d i m e n s i o n s . R o b e r t D a h l l e d off the m o d e r n debate o v e r the nature o f p o w e r w i t h a study o f p o w e r i n l o c a l p o l i t i c s i n N e w H a v e n . In g e n e r a l . A s a f i n a l h o o k .38 Chapter 3 Importance of Dimensional Thinking 39 I h a v e u s e d as my " b a d " e x a m p l e s w o r d s f r o m ordinary language rather than w o r d s s p e c i f i c a l l y d e s i g n e d for u s e by s c h o l a r s . e v e n s c h o l a r l y v o c a b u l a r y is often not as tight as it c o u l d be. that determines a l l that f o l l o w s . s o h e c o n c l u d e d that l o c a l p o l i t i c a l p o w e r w a s not c o n c e n t r a t e d i n a n elite but w a s w i d e l y d i s t r i b u t e d . c a n help us to c l a r i f y the m e a n i n g of the w h o l e body of work and may point u p new directions for theoretical development. T h e f o l l o w i n g a n a l y s i s of different w a y s of u s i n g the c o n c e p t power is an extended e x a m p l e o f h o w s u c h d i m e n s i o n a l a n a l y s i s might p r o c e e d . w h e n s c h o l a r s d e s i g n t e r m i n o l o g y it is a good deal tighter than ordinary language. whether one has influenced people's understanding of their interests. In 1974. B a c h r a c h a n d B a r a t z ( 1 9 6 2 ) r e s p o n d e d that the true e s s e n c e o f p o w e r l a y not i n p a r t i c i p a t i o n i n d e c i s i o n s but i n b e i n g a b l e t o l a y out the a g e n d a for w h a t q u e s t i o n s w e r e to be d i s c u s s e d . 1 9 6 1 ) . Figure 3-1 Example of Dimensional Analysis T h e b a s i c b u i l d i n g b l o c k o f all p o l i t i c a l s c i e n c e a n a l y s i s i s the c o n c e p t o f power. P e r h a p s the most frequent f a i l i n g in r e s e a r c h is that this part of the w o r k is done c a s u a l l y a n d sloppily. A c c o r d i n g l y . e v e n i f it is already at a quite abstract l e v e l . If the N e w H a v e n w o r k e r s c a n be l e d to identify t h e m s e l v e s as w h i t e s rather than as w o r k e r s . T h e q u e s t i o n o f the c i t y g o v e r n m e n t t a k i n g o v e r the p u b l i c utilities h a d not b e e n o n the t a b l e . dimensional analysis c a n show h o w they all w o r k together. In one study. interests but rather in determining w h a t leads t h e m to define their v e r y identity. different sorts o f p e o p l e w e r e i n v o l v e d . that several s c h o l a r s w o r k i n g independently of e a c h other d e v i s e related u n i d i m e n s i o n a l w o r d s .

40

Chapter

3

Chapter 4

have been i n v o l v e d in the actual d e c i s i o n s . A m e m b e r of the city park board m a y have been i n v o l v e d in the d e c i s i o n but m a y not have e x e r c i s e d p o w e r in any of its other d i m e n s i o n s . A n elementary s c h o o l teacher m a y have been i n v o l v e d i n the p r o c e s s o f f o r m i n g identities but m a y not have e x e r c i s e d p o w e r in its other d i m e n s i o n s ; another s c h o o l teacher, interested and active on park questions, m a y have been i n v o l v e d in the p r o c e s s of f o r m i n g identities, and m a y have participated in the d e c i s i o n s , but not have e x e r c i s e d p o w e r in the other two d i m e n s i o n s . D i m e n s i o n a l a n a l y s i s of this sort a l l o w s us to see the interrelations of v a r i o u s a p p r o a c h e s to a question and c a n a l s o give us a r i c h f r a m e w o r k w i t h w h i c h to apply a m u l t i d i m e n s i o n a l c o n c e p t . It a l s o a l l o w s us to c o m p a r e the i m p o r t a n c e of the d i m e n s i o n s . I m p l i c i t in e a c h later s c h o l a r in the d i s c u s s i o n s k e t c h e d above w a s the i d e a that h i s notion of p o w e r w a s deeper and more b a s i c than those that went before. A d i m e n s i o n a l a n a l y s i s g i v e s us a structure w i t h i n w h i c h we c a n address this q u e s t i o n .

WClLTTTOClLTl] Accuracy

FURTHER DISCUSSION

F o r m a l treatment o f d i m e n s i o n a l a n a l y s i s w a s i n t r o d u c e d b y A l l a n H . B a r t o n , " T h e Concept of Property-Space in Social R e s e a r c h " (1955). Philip E. Jacob's "A Multid i m e n s i o n a l C l a s s i f i c a t i o n o f A t r o c i t y S t o r i e s " ( 1 9 5 5 ) f u r n i s h e s a good e x a m p l e o f dimensional analysis in practice. S o m e e x a m p l e s f r o m p o l i t i c a l s c i e n c e are C h a p t e r Political Oppositions in evant dimensions Western Democracies ( 1 9 6 6 ) , 11 of R o b e r t D a h l ' s

In this chapter I e x p l o r e p r o b l e m s of accurate m e a s u r e m e n t . T h e s e are p r o b l e m s that arise w h e n we try to relate the a c t u a l operations in a p i e c e of r e s e a r c h — t h a t i s , m e a s u r e s of t h i n g s — t o the c o n c e p t s that f o r m the b a s i s of our u n d e r l y i n g theory. C o n c e p t s , o f c o u r s e , exist o n l y i n the m i n d . O n e n e c e s s a r y a s s u m p t i o n , i f w e are t o c l a i m that a p i e c e of r e s e a r c h h a s tested a given theory, is that the things m e a s u r e d in the r e s e a r c h c o r r e s p o n d to the things in the theorist's m i n d . T h i s is often a difficult a s s u m p t i o n to m a k e . In the p r e c e d i n g chapter, y o u s a w one k i n d o f p r o b l e m that c a n stand i n the w a y o f it. T h e p o l i t i c a l scientist w h o w a n t e d to m e a s u r e the a m o u n t of interaction b e t w e e n nations f o u n d that there w a s no s i n g l e satisfactory indicator of "interaction." A n u m b e r of t h i n g s — t r a d e , m a i l e x c h a n g e s , a l l i a n c e , and so o n — p a r t o o k of "interaction," but no one of t h e m alone w a s s y n o n y m o u s w i t h the mental construct. In the social s c i e n c e s , only rarely are we able to measure our concepts directly. C o n s i d e r , for example, the concepts " s o c i a l c l a s s , " "respect for the presidency," and "power in the community." A n y variables we w o u l d c h o o s e to measure these concepts correspond only indirectly or in part to our mental constructs. T h i s is the basic problem of measurement in the social s c i e n c e s . C o n s i d e r the concept " s o c i a l status." A m o n g s o c i a l scientists there are two popular v e r s i o n s of this concept: " s u b j e c t i v e s o c i a l status," the c l a s s that i n d i v i d u a l s c o n s i d e r t h e m s e l v e s as b e l o n g i n g to; and "objective s o c i a l status," an i n d i v i d u a l ' s rank w i t h regard to prestige along s o c i a l h i e r a r c h i e s s u c h as education, i n c o m e , and o c c u p a t i o n . N e i t h e r v e r s i o n of the c o n c e p t c a n be m e a s u r e d directly. I n the c a s e o f " s u b j e c t i v e s o c i a l status," w e c a n n o t m e a s u r e d i r e c t l y w h a t i n d i v i d u a l s f e e l about their status. W e k n o w w h a t they report, but their r e p l i e s t o our i n q u i r i e s m a y not b e w h a t w e are l o o k i n g for. T h e y m a y not k n o w w h a t they " r e a l l y " f e e l , for i n s t a n c e ; o r they m a y m i s u n d e r s t a n d the q u e s t i o n a n d g i v e a

a first-rate a n a l y s i s of the r e l Harry Eckstein's Pressure Group

for c l a s s i f y i n g

"opposition";

Politics ( 1 9 6 0 ) , e s p e c i a l l y pp. 1 5 - 4 0 , in w h i c h he c l a s s i f i e s pressure group activities; and the third chapter of h i s Division and Cohesion in Democracy ( 1 9 6 6 ) , an e x c e l lent dimensional analysis of "political division"; Hanna Pitkin's Representation

( 1 9 6 9 ) , a c o l l e c t i o n of v a r i o u s w r i t i n g s on the c o n c e p t of representation, a m o n g w h i c h P i t k i n ' s o w n e s s a y is p a r t i c u l a r l y insightful; Representation ( 1 9 6 7 ) ; also P i t k i n ' s The Concept of

and G i o v a n n i S a r t o r i ' s Parties and Party Systems ( 1 9 7 6 ) .

O n e b r a n c h o f p o l i t i c a l p h i l o s o p h y i s the " a n a l y t i c p o l i t i c a l p h i l o s o p h y " a p p r o a c h , w h i c h s e e k s t o s t u d y p o l i t i c a l i d e a s b y a c l o s e e x a m i n a t i o n o f the meaning of concepts used to describe politics. T h i s approach is reviewed in

Richard

B e r n s t e i n ' s Restructuring
(1975).

of Social and Political

Theory

(1978)

and

in

Oppenheim

F i n a l l y , a s a n e x e r c i s e , y o u m i g h t c o n s i d e r the c o n c e p t u a l p r o b l e m s i n v o l v e d i n the w e l l - w o r n a p h o r i s m o f L o r d A c t o n : " P o w e r tends t o corrupt a n d absolute p o w e r c o r r u p t s a b s o l u t e l y . " H o w w o u l d y o u a n a l y z e this c o n c e p t u a l l y ? Can i t b e analyzed?

41

42

Chapter 4

Problems of Measurement

43

m i s l e a d i n g a n s w e r . T h e n a g a i n , a p e r s o n m a y f e e l d i f f e r e n t l y f r o m one d a y t o the next, i n w h i c h c a s e the m e a s u r e o f h i s status w i l l d e p e n d o n our rather arbitrary c h o i c e o f day. In the c a s e of "objective s o c i a l status," again we cannot m e a s u r e the variable directly. " O b j e c t i v e status" has something to do w i t h i n c o m e , something to do w i t h e d u c a t i o n , s o m e t h i n g to do w i t h o c c u p a t i o n , and s o m e t h i n g to do w i t h various other h i e r a r c h i e s , s o m e of w h i c h we m a y not k n o w about. N o n e of these provides a sufficient m e a s u r e in itself. F o r e x a m p l e , if we tried to u s e o c c u p a t i o n alone as a m e a s u r e of s o c i a l status, we w o u l d be f a c e d w i t h the difficult question of whether a p l u m b e r w h o m a d e $ 4 0 , 0 0 0 a y e a r w a s r e a l l y of l o w e r s o c i a l status than a bank teller w h o made $ 2 5 , 0 0 0 a year. S i m i l a r l y , if we tried to use i n c o m e alone as a m e a s u r e , we w o u l d be f a c e d w i t h the p r o b l e m of what to do w i t h a retired teacher, w h o s e i n c o m e might be b e l o w the poverty l e v e l . " S o c i a l status" in this c a s e is a c o n c e p t that is related to a n u m b e r of m e a s u r a b l e things but is related only i m p e r f e c t l y to e a c h of them. T h e best we c a n do in m e a s u r i n g it is to c o m b i n e the v a r i o u s m e a s u r a b l e indicators into a p o o l e d m e a s u r e that is r o u g h l y related to the c o n c e p t "objective s o c i a l status." We encounter s i m i l a r p r o b l e m s in m e a s u r i n g the other concepts I have cited as e x a m p l e s . L i k e m a n y other v a r i a b l e s i n p o l i t i c a l s c i e n c e , these concepts are o f c o n siderable interest and use in theories but are by their nature i m p o s s i b l e to measure directly. T h e general p r o b l e m p o s e d b y s u c h v a r i a b l e s i s presented s c h e m a t i c a l l y i n Figure 4-1. A s y o u s a w i n C h a p t e r 2 , i n p o l i t i c a l r e s e a r c h w e are c o m m o n l y interested i n relating c o n c e p t s through a theory. T h i s is a l w a y s true in theory-oriented r e s e a r c h , and it is often true in e n g i n e e r i n g r e s e a r c h as w e l l . If we cannot measure directly the concepts w e w i s h t o u s e , w e find o u r s e l v e s i n the position depicted i n F i g u r e 4 - 1 . We w a n t to say, " C o n c e p t A bears a relationship of t h u s - a n d - s o f o r m to c o n c e p t B . " B u t all that we c a n o b s e r v e is the relationship between m e a s u r e A and m e a s u r e B.

W h e t h e r w h a t we say about the m e a s u r e s is an accurate statement of the relationship b e t w e e n the c o n c e p t s depends on the u n o b s e r v e d relationships b e t w e e n c o n c e p t A and m e a s u r e A and between c o n c e p t B and m e a s u r e B. We c a n only a s s u m e what these relationships are. L i k e the theory itself, the relationships cannot be o b s e r v e d . A s a n e x a m p l e , suppose y o u want t o a s s e s s the theory tested b y D i e h l and K i n g s t o n , that countries that are i n c r e a s i n g their a r m a m e n t s tend to engage in aggressive international p o l i c i e s (see above, p. 8). Y o u might be f a c e d w i t h the f o l l o w i n g relationships:

1. Relationship between the concept "increasing armaments " and the measure of "increasing
armaments." You clearly cannot measure increases in a nation's armaments directly; only a national intelligence apparatus has the facilities to do that, and even then, the result is imperfect. Therefore, you might take as your measure the country's reported expenditures on armaments. Now, a country that is not preparing to launch an aggressive military venture would have less reason to lie about an arms buildup than would a country (such as Germany in 1933) that is consciously preparing for aggression. Therefore, the relationship between concept and measure in this case might be: When a country is not building up its armaments, or when it is building them up in order to launch an aggressive action, its reported expenditures on arms will not increase.

2. Relationship between the concept "increasing armaments" and the concept "aggressive
international policies." Let us assume, for this example, that countries that are increasing their armaments do tend to engage in aggressive international policies.

3. Relationship between the concept "aggressive international policies" and the measure
of "aggressive international policies." Let us assume, for this example, that we are able to develop a measure that corresponds almost perfectly to the concept "aggressive international policies." (In practice, of course, this would be a difficult variable to measure, and it certainly would be necessary first to analyze the varied dimensions involved in "aggression" and "policies" in order to state more clearly just what was meant by the concept.)

W e n o w find o u r s e l v e s i n the position depicted i n F i g u r e 4 - 2 . H e r e , b e c a u s e o f Figure 4-1 The Problem of Measurement peculiarities in the relationships b e t w e e n the concepts a n d the m e a s u r e s of these c o n c e p t s , the relationship y o u c a n o b s e r v e between the m e a s u r e s turns out to be the opposite of the true relationship b e t w e e n the c o n c e p t s . W o r s e yet, i n a s m u c h as the Theory Concept A -« >Concept B two m e a s u r e s and the c o n n e c t i o n between them are all that y o u c a n o b s e r v e , y o u w o u l d h a v e no w a y of k n o w i n g that this w a s h a p p e n i n g . T h i s is w h y I h a v e c a l l e d indirect m e a s u r e m e n t of concepts the p r o b l e m of m e a s u r e m e n t . O n e solution to the p r o b l e m might be to m e a s u r e v a r i a b l e s only directly. S o m e 9 I I 9 concepts are directly m e a s u r a b l e . A few e x a m p l e s are p e o p l e ' s votes if an election is nonsecret and y o u tabulate the result y o u r s e l f ; statements y o u hear m a d e on the Senate floor; a b o m b y o u see dropped or t h r o w n .
1

Relationship Measure A •* *Measure B

"it is reasonable, also, to include here reliable observers' accounts of such events. Even though the measurement of these things is technically indirect, if you accept another observer's account of them, you should be able to achieve a very tight fit between the concept "event happens" and the measure "reliable observer says that event happens."

44

Chapter

4

Problems of Measurement

45

Concept Increases in Armaments Increases in armaments tend to be followed by aggression

Concept Aggressive International Policies

elections are held with a secret ballot). Therefore, most of the time we must work with variables that are indirect measures of the concepts in which we are interested. This means that there are interposed, between our (concrete) operations and the (abstract) theory we want to work on, the relationships between our concrete concepts and their abstract measures. This is the situation illustrated graphically in Figures 4-1 and 4-2.
2

As concept increases, measure increases except when policies are aggressive

The chief problem of measurement is to ensure, as much as possible, that the relationships between concepts and measures are such that the relationship between the measures mirrors the relationship between the concepts.
As concept increases, measure increases

Problems we may encounter in trying to achieve this correspondence between measures and concepts fall under two headings: problems of measure reliability and problems of measure validity.

Measure Increases in Reported Arms Expenditures Increased reported expenditures are never followed by aggression

Measure Aggressive International Policies

RELIABILITY A measure is reliable to the extent that it gives the same result again and again if the measurement is repeated. For example, if people are asked several days in a row whether they are married, and their answers vary from one day to the next, the measure of their marital state is unreliable. If their answers are stable from one time to the next, the measure is reliable. The analogy of measuring with a yardstick may help make the meaning of reliability clear. If an object is measured a number of times with an ordinary wooden yardstick, it will give approximately the same length each time. If the yardstick were made of an elastic material, its results would not be so reliable. It might say that a chair was 20 inches high one day, 16 the next. Similarly, if it were made of a material that expanded or contracted greatly with changes in temperature, its results would not be reliable. On hot days it would say that the chair was shorter than on cold days. In fact, the choice of wood as a material for yardsticks is in part a response to the problem of reliability in measurement, a problem certainly not confined to the social sciences. Wood is cheap, rigid, and relatively unresponsive to changes in temperature. There are many sources of unreliability in social science data. The sources vary, depending on what kinds of data are used. Official statistics, for example, may be unreliable because of an unusual number of clerical errors or because of variability in how categories are defined from one time to the next. ("Votes cast in an election," for instance, may mean "all votes, including spoiled ballots" at one time, "all valid votes" at another.) Attitude measures may be unreliable because a question is hard for respondents to understand, and they interpret it one way at one time, another way the next. Or the people entering their responses into the computer may make mistakes.

Figure 4 - 2

Example of the Problem of Measurement

The difficulty with such a solution is that concepts that can be measured directly are usually trivial in and of themselves. They are too idiosyncratic to use in general, interesting theories. I would hate to say that particular statements by U.S. senators lead nowhere, but as far as political theory is concerned, it is true. Any given statement can apply only to itself. It takes on a general meaning only if it is placed in a category, so that it can be compared with other statements. For instance, Senator 's statement, "The president's policies are bankrupting the people of my state," is not intrinsically of theoretical interest. It can be placed in various categories, however: "statements of opposition to the president," "statements of concern for constituents' needs," "bombastic statements." Placing it in one of these categories allows us to compare it with other senatorial statements and to develop theories about the causes and effects of such statements. Note, though, that by placing it into a category, we have used the statement as an indirect measure of the concept which the category represents. No given statement is a perfect case of the "bombastic statement" or of the "statement of opposition to the president." Rather, a number of statements approximate each category, and we choose to use these statements as indirect measures of the abstract concept we cannot measure. To sum up the argument thus far: For a concept to be useful in building theories, it usually must be an abstraction, which cannot be measured directly. Further, of those interesting concepts that are in principle directly measurable (how individuals vote in an election, for example), many cannot be measured directly for practical reasons (the

t e c h n i c a l l y , these relationships are called epistemic correlations. "Correlation" means relationship, and "epistemic" has the same root as "epistemology"—the study of how we know.

inasmuch as straightforward informational items like this one can usually be measured with reasonable reliability." which is made by combining such items as an individual's income. Converse (1964) noted that on many standard questions of political policy.5 percent of the time from one interview to the next. This source of unreliability would be easily distinguishable from others." comprising such disparate items as welfare payments. or a measure of "welfare policy expenditures. s o that r e s p o n d e n t s s o m e t i m e s interpreted i t t o m e a n . Converse had to assume that he had effectively eliminated other sources of unreliability. is that there is no way to distinguish instability that stems from "real" unreliability in the concept being measured from instability due to problems in the measurement process. A number of tests have been developed to help researchers check the reliability of a measure. such as attitudinal questions. 6. a measure of "social status. the researcher randomly divides these assorted items into two groups and then composes a summary "measure" out of each of the groups. should answer. On the average. An additional source of unreliability in the measure is variability in the "true value" of the concept. 2. D i s h o n e s t i n t e r v i e w e r s m i g h t b e p l a y i n g g a m e s w i t h t h e r e s e a r c h e r b y filling o u t t h e f o r m s t h e m s e l v e s i n s t e a d o f g o i n g t h r o u g h t h e t r o u b l e o f getting the r e s p o n d e n t s t o answer them. We often wish to know how successfully we have reduced unreliability. If the second measure strongly resembles the first—that is. For example. such as gender and race. Even on attributes that should be relatively easy to measure reliably. and he went on to draw interesting comparisons between elite and mass opinion. because it would show up as a recognizable pattern of stable answers up to a certain point. the reported gender of respondents changed 0. avoids this problem. and neighborhood into a single summary measure. Note that to reach this conclusion. But the same sorts of conditions affect the reliability of less straightforward survey questions. T h e r e m i g h t be simple clerical errors in copying d o w n the answers. unemployment relief. the As an illustration. followed by changed. I shall describe two of them briefly. perhaps some people got married. Characteristics on which it is somewhat easier to be vague or mistaken showed substantial unreliability. house size. it is usually a barrier we want to elm inate. some errors appeared when the same respondents were interviewed at two-year intervals by a well-administered survey. but not d i v o r c e d . questions that permit a considerable degree of interpretation. or divorced. Christopher Achen (1975) later challenged Converse's conclusions on just these grounds. and school lunch programs. R e s p o n d e n t s ' a n s w e r s m i g h t d e p e n d on the c o n t e x t of t h e interview. or " n o " w h e n t h e y h a d h a d a b a d day. some of these possibilities are farfetched. R e s p o n d e n t s ' a n s w e r s m i g h t d e p e n d o n their m o o d . P e r h a p s t h e y w o u l d a n s w e r " y e s " w h e n t h e y h a d h a d a g o o d day. such as interviewer error and confusion about the meaning of questions. Careful work is the best way to achieve reasonable reliability—double-checki rig all clerical work. 3 Reliability as a Characteristic of Concepts Thus far I have treated the unreliability of a measure as if it were a result of unpredictability in the relationship between the concept and its measure. 4 . It is particularly useful whenever a measure is multidimensional—for instance. Presumably. would show even more unreliability. 5. trying out the questionnaire on a small pilot study in order to catch and correct unclear questions. Admittedly. occupation. A p e r s o n m i g h t s a y " n o " t o a n attractive i n t e r v i e w e r a n d " y e s " t o e v e r y o n e e l s e . however. " H a v e y o u ever b e e n m a r r i e d ? " I t m i g h t not b e c l e a r h o w p e o p l e w h o w e r e s e p a r a t e d from their s p o u s e s . people's attitudes appeared to vary randomly across time. 3 These are drawn from Asher (1974). either by the interviewer on the spot or by the person w h o transcribes the interviewers' copy into a computer. the split-half check for reliability. he could then treat the unreliability in his measure as a reflection of unreliability in the concept. . from one time to the next. education. at least in American survey research. whereas race did not change at all. Because all of the items are taken to be measures of the same thing. the report of respondents' educational background showed lower education two years later (which is logically impossible) an average of 13 percent of the time. Testing the Reliability of a Measure Although unreliability may sometimes spur on further theoretical research. such as "What do you like about candidate X?" "What social class do you consider yourself to be a member of?" and "Do you feel people generally can be trusted?" A few examples will help give you a sense of the magnitude of this problem. a n s w e r i n g questions randomly. He concluded that on certain issues. The example itself is a bit strained. This situation sometimes provides the basis for interesting theories. answers. R e s p o n d e n t s m i g h t be playing g a m e s with the interviewer. of course. allowing for a suitable interval of time between the two measurements. as it did in this case. let us list the various sources of unreliability that might be involved in our example of tabulating responses to a simple question about marital status: 1 .46 Chapter 4 Problems of Measurement 47 In our previous example. and so on. T h e q u e s t i o n m i g h t b e p h r a s e d badly. One problem with this test. 3 . the mass public simply did not form stable opinions. In one study. ' A r e y o u n o w m a r r i e d ? " a n d s o m e t i m e s . In the split-half test. based on that conclusion. The test-retest check for reliability simply consists of repeating the measurement a second time. Another test. hospital subsidies. but once again stable. Having first eliminated these sources of unpredictability in the relationship between concept and measure. if the measure is stable over the elapsed time—it is considered relatively reliable. A more interesting case is presented when the true value itself varies randomly.

T h u s . T h u s . T h e relationship between the concept and the m e a s u r e is s u c h that w h e n "increases in a r m a m e n t s " are h i g h . we c a n a s s u m e that any error i n the m e a s u r e i s r a n d o m . 4 6 ) . b e c a u s e the people w h o write s u c h letters are not an accurate c r o s s s e c t i o n of the p u b l i c as a w h o l e . T h e r e are m a n y w a y s that a m e a s u r e c a n be i n v a l i d . In one y e a r there m i g h t be no natural d i s a s t e r s . w h i c h w i l l also be important w h e n we l o o k at m e a s u r i n g the strength of relationships in C h a p t e r 8. on the average. c o u l d result if they lack reliability is a n e c e s s a r y but not a sufficient c o n d i t i o n of validity. depending on the reason for the arms buildup. B u t it c a n be r e l i a b l e a n d yet not be v a l i d . if we a s k e d people w h e t h e r they h a d a p r i s o n r e c o r d . U s i n g letters to the editor as a n indicator o f p u b l i c o p i n i o n w o u l d b e u n w i s e . on the average. any one of them c o u l d be rather far f r o m the true v a l u e of the concept. it is l i k e l y that there w o u l d be a g o o d deal of n o n r a n d o m error in the m e a s u r e . i f there is in p r i n c i p l e a relationship of equivalence b e t w e e n a m e a s u r e and Some Examples A f e w e x a m p l e s are in order. for its erratic variation w o u l d m a k e the s c o r e b a s e d o n the group i n w h i c h i t w a s i n c l u d e d l e s s l i k e l y to e q u a l the s c o r e b a s e d on the group that d i d not i n c l u d e it. s u c h as the effect of the length of time s i n c e p a y d a y or of c h a n g e s in the weather. or both. T h e m e a s u r e might be reliable (a country that reported l o w increases in one year. A m e a s u r e is v a l i d i f it a c t u a l l y m e a s u r e s w h a t it purports to m e a s u r e . that if the c o n c e p t w e r e m e a s u r e d a large n u m b e r of t i m e s . T h e s p l i t . shots at the c e n t e r of a target. for i n s t a n c e — v a r i e s greatly f r o m state to state a n d f r o m one y e a r to the next in any o n e state. w e encountered a good deal of r a n d o m error. A m o r e s e r i o u s f a i l i n g of our m e a s u r e m e n t s . a m e a s u r e is v a l i d to the extent that it is free of both sorts of error. for instance. T h e s h o o t i n g m a y s u f f e r f r o m either r a n d o m error (the a r c h e r i s e r r a t i c ) o r n o n r a n d o m error (the a r c h e r h a s s o m e s y s t e m a t i c p r o b l e m — p e r h a p s there i s a w i n d f r o m the left. people tended to report their l e v e l of e d u cation differently f r o m one time to the next. T h e i r o p i n i o n s w o u l d not be a v a l i d m e a s u r e of " p u b l i c o p i n i o n . c o n s i d e r a state-by-state m e a s u r e o f " w e l f a r e p o l i c y e x p e n d i tures. random error a n d nonrandom error. a n d D but a c h i e v e s validity. We have already d i s c u s s e d s e v e r a l i n s t a n c e s o f r a n d o m error. " i n c r e a s e s in reported arms expenditures" m a y be . i n a s m u c h as a l l of the subitems p r e s u m a b l y are m e a s u r e d at the s a m e time. h o w e v e r . T h e t w o sorts o f error are p r e s e n t e d v i s u a l l y i n F i g u r e 4 . If in the l o n g r u n . a c c i d e n t a l reporting errors w o u l d c a n c e l e a c h other out and the average of m a n y s u c h reports w o u l d give a true m e a s u r e of the c o n c e p t for a particular group of people. the m e a s u r e s of a c o n c e p t tend to be true. T h i n k o f a n a r c h e r w h o is t r y i n g to p r o d u c e " v a l i d " s h o t s . We a s s u m e d . E v e n in the l o n g r u n . T h e relationship between validity and reliability c a n b e c l a r i f i e d b y introducing the notion of two s u m m a r y m e a s u r e s s h o u l d tend to be the s a m e .h a l f test to s h o w a r e l a t i v e l y l o w reliability. A c c o r d i n g l y . T h a t p a r t i c u l a r i t e m w o u l d be a s o u r c e of u n r e l i a b i l i t y in the overall m e a s u r e . it is " r e l i a b l y i n v a l i d . s o w e w i l l c o n f i n e ourselves here to e x a m p l e s of n o n r a n d o m error. I n m e a s u r i n g education.48 Chapter 4 Problems of Measurement 49 either high or low. VALIDITY R e l i a b i l i t y has to do w i t h h o w d e p e n d a b l y a m e a s u r e m i r r o r s its c o n c e p t . the m e a s u r e is r e l i a b l e . i t c a n m i s s s o m e s o u r c e s o f instability i n the m e a s u r e m e n t p r o c e s s . B u t this is actually an important benefit: If we are able to s c r e e n out true c h a n g e over time in the c o n c e p t . so that it does not tend to m i r r o r it faithfully. " T h e relationship between the measure " i n c r e a s e s in reported arms expenditures" and the concept " i n c r e a s e s in a r m a m e n t s " in F i g u r e 4 . as people s y s t e m a t i c a l l y tried to suppress their p r i s o n r e c o r d s . T h e s e t w o sorts o f error r e s u l t i n the f o u r p o s s i b l e c o m b i n a t i o n s s h o w n i n F i g u r e 4 . b e c a u s e the m e a s u r e does not mirror the concept accurately. n o n r a n d o m error is s y s t e m a t i c error that tends in the l o n g r u n . A s a n e x a m p l e . T h e s e two c h e c k s for reliability c o m p l e m e n t e a c h other. but it w o u l d still be invalid. T h e test-retest c h e c k is appropriate for any sort of m e a s u r e that c a n be replicated. in other w o r d s . we a s s u m e d i m p l i c i t l y that the m e a s u r e tended to m i r r o r the c o n c e p t faithfully a n d that the p r o b l e m of reliability w a s s i m p l y that this tendency m a y be a rather loose one. for e x a m p l e (see p . A m e a s u r e c a n n o t be v a l i d a n d yet not be r e l i a b l e . It a l s o s h o u l d c a u s e the s p l i t . in another there m i g h t be floods or a h u r r i c a n e . A m e a s u r e of h o w c l o s e they are to e a c h other p r o v i d e s a c h e c k on h o w r e l i a b l e the total s u m m a r y m e a s u r e i s . T h e r e w a s no r e a s o n to expect that people w e r e deliberately m i s r e p r e s e n t i n g their educational l e v e l .3 ." It m i g h t be that one p a r t i c u l a r i t e m — d i s a s t e r relief. " its c o n c e p t .2 is an example of invalid measurement. but this often i n c l u d e s c h a n g e s in the true v a l u e of the c o n c e p t rather than o n l y the instability that is due to the m e a s u r e m e n t p r o c e s s .h a l f c h e c k is appropriate for m e a s u r e s c o m p r i s i n g a group of subitems. on the average. It c h e c k s o n l y for those s o u r c e s of unreliability that do not operate over time. It c h e c k s for all s o u r c e s of unreliability. on respondents' a n s w e r s to an i n t e r v i e w question. In thinking about reliability.3 . By contrast. a n d the a r c h e r h a s not y e t l e a r n e d to c o r r e c t for i t ) . A m e a s u r e is reliable to the extent that it is free of r a n d o m error alone. T h e a r c h e r a c h i e v e s " r e l i a b i l i t y " on b o t h targets B " v a l i d i t y " o n l y o n target D . R a n d o m error is the sort of error we have a d d r e s s e d in d i s c u s s i n g reliability. this m e a s u r e w o u l d not give an accurate estimate of the true v a l u e . s h o u l d be l i k e l y to report low increases the next y e a r a l s o ) . the average of those m e a s u r e s w o u l d reflect our " i d e a l " c o n c e p t . we w i l l have a m u c h better i d e a of any instability due to the m e a s u r e m e n t p r o c e s s . however. T h a t i s . for i n s t a n c e . s o w e w o u l d expect that in the long r u n . to distort a g i v e n m e a s u r e of a c o n c e p t . Q u i t e s i m p l y . In effect. but it c o u l d distort the c o n c e p t in the s a m e w a y e a c h of these t i m e s . O n e c o m m o n s o u r c e of i n v a l i d m e a s u r e s is extrapolation f r o m a s a m p l e to a population that is not r e a l l y represented by that s a m p l e . the m e a s u r e is v a l i d . T h e p r o b l e m in reliability is that s i n c e the m e a s u r e s vary. If it g i v e s the s a m e result repeatedly. that i s .

" At the same time. you should ask a few people to answer your questions and then to relate their understanding of the questions themselves. however. with many of them vehemently rejecting political parties. depicted in Figure 4-1. We say that a measure is valid if it is a true measure of its concept. then. it was striking that voting participation was higher among farmers than among most groups in the French population. Roosevelt initiated the New Deal. We cannot A comic case of sampling problems from the early days of opinion polls is the Literary Digest poll. Also. which broadened his support among the poor and drove the middle class to the Republicans. Thus." Checks for Validity Taking precautions. It had always been thought. middle-class sample the Digest drew upon did not vote much differently from the rest of the country from 1924 through 1932. the sympathies of the middle class were no longer a valid measure of the way the country would vote. Part of the "craft" in the craft of political research is cleverness and care in developing measures that appear likely to be valid. A questionnaire item may also result in an invalid measure when respondents attach a significantly different meaning to the question than was intended by the researcher. this seems to have been a rather select sample. the interested.S. Accordingly. For example. The most important thing is to think through the measurement process carefully and to be on guard against any way in which the relationship between concept and measure might be distorted. of course. How. Some techniques are available to help in developing valid measures. Starting in 1924. know what the relationship between a concept and its measure is. which was sent out to a mailing list obtained from telephone directories and state automobile registration lists. the Digest ran an ambi- tious poll in presidential election years. 1928. Sidney Tarrow discovered that the innocent question about political interest had been spreading confusion. If they were not interested in politics.50 Chapter 4 Problems of Measurement 51 Random Error High Low Figure 4-3 R n o and Nonrandom Er r a dm ro Apparently. When the Literary Digest went out of business the next year. But the general problem of measurement is precisely the fact that usually all we can observe is the measures. that French farmers were essentially apolitical. This may alert you to questions that mean something different to your respondents than they mean to you. its sympathies were a valid measure of the way the country was going to vote in those elections. After 1936. These deceptively simple strategies consist of taking various precautions against invalidity during the construction of a measure. . in determining the final form of a questionnaire that you hope to use in a study. When asked in surveys "How interested are you in politics?" they had generally responded. Between 1932 and 1936. many farmers who were interested in politics but considered themselves independents responded "Not at all" to this invalid measure of "interest in politics. First. it was attacked in the Congressional Record and featured in New Yorker cartoons. Thus it was a shock when the poll's prediction of a landslide victory for Landon in 1936 was gainsaid by FDR's decisive victory. coming within a few percentage points of the actual outcome in each of those elections. 29). the Digest poll was very successful. The Literary Digest was a giant magazine in the United States in the early part of the twentieth century. By 1932 the poll had become an institution. we now know that drawing a sample in certain ways (drawing a random sample. and again in 1932. the Digest had over 2 million responses each time. Also. In 1924. for example. French farmers apparently interpreted "interest in politics" to mean commitment to some particular party. why did they vote? In a study designed to explore this paradox (see p. is that there is no pat way to do so. Our problem in checking the validity of a measure is similar to the general problem of measurement. population in two ways. but even at that. can we assess the validity of the measure? The answer. The Digest sample distorted the U. "Not at all. Virtually everyone who owned a car or a telephone was reached by the poll. for instance) guards against a fiasco like that which destroyed the Literary Digest poll. it was thought that the shock and loss of reputation from having called the election so badly was a factor in the magazine's demise. it sampled only those who were interested enough and energetic enough to return the sample ballot. Only about 20 percent of the sample ballots mailed out were returned. inasmuch as those who did not have a car or a telephone—at a time when cars and telephones were far less universally owned than today—did not get onto the mailing list. it essentially sampled only the upper and middle classes. Because only 20 percent of those who received the ballot returned it.

it is worth c o n s i d e r i n g what happens under those c i r c u m s t a n c e s . "political interest" corresponds to A. w h e n it c a n be u s e d . Test of validity. True Values Seniority 32 24 23 20 14 11 10 6 5 Margin of Victory (%) 18 12 11 11 8 6 6 4 3 1 Seniority 32. It is apparent from the figures in the first set of c o l u m n s (True V a l u e s ) that there is a relationship between the two. c o n s i d e r a measure of nations' hostility to e a c h o t h e r — b a s e d on content a n a l y s i s of the nations' n e w s p a p e r s ( a ) . L e t us say that we want to decide whether m e a s u r e a is a v a l i d m e a s u r e of c o n c e p t A. i t has face validity. r a n d o m error.6 6.9 In this example.6 8. T h e relationship is also quite strong.0 3. To the extent that m e a s u r e s are unreliable.1 .4 21.0 3. It w i l l p a r a l l e l the true relationship but w i l l appear w e a k e r than is actually the c a s e . w i l l greatly strengthen y o u r findings. or even usually. T h u s far our strategies have not a c t u a l l y p r o v i d e d a test of the It s h o u l d be o b v i o u s that the o n l y g o o d m e a s u r e is one that is v a l i d . B e c a u s e s o c i a l scientists often operate w i t h m e a s u r e s that suffer f r o m one or the other sort of error. If the m e a s u r e s are sufficiently u n r e l i a b l e . the strategy I have suggested here requires o n l y that the investigator c o n s i d e r c a r e fully h o w p l a u s i b l e it is that the m e a s u r e mirrors the concept.2 13. If we found that two of the nations went to w a r against e a c h other (/3) yet the measure did not show an a c c o m p a n y i n g increase in feelings of hostility between the two. we c a n c h e c k to see whether m e a s u r e ji is related to m e a s u r e a. B e c a u s e h e c o u l d not c o n c e i v e o f high electoral participation o c c u r r i n g in the a b s e n c e of h i g h political interest.8 6. If y o u think it does (and people w h o read y o u r w o r k agree w i t h y o u ) . A s i t h a p p e n s . T h e effect of unreliability ( i f the m e a s u r e s are o t h e r w i s e valid) is m o r e subtle.3 0.8 21. we w o u l d be s u s p i c i o u s of the validity of the m e a s u r e s .0 12. an even greater degree of TABLE 4-1 Safe Districts Related to Seniority. the b a s i c relationship c a n be so w e a k e n e d that it w i l l appear. a n d our a s s u m p t i o n of a relationship between /3 and A is true. as if there is no relationship at a l l . If a m e a s u r e is s y s t e m a t i c a l l y i n v a l i d .0 11. T h e study b y T a r r o w cited earlier provides a n e x a m p l e o f this l o g i c . w h i c h had s h o w e d l o w political interest c o i n c i d i n g w i t h h i g h participation. however. 4 As another e x a m p l e of this k i n d of test. T h u s . T a r r o w ' s initial c o n c l u s i o n that the u s u a l question " H o w interested are y o u in p o l i t i c s ? " w a s p r o v i d i n g a n i n v a l i d m e a s u r e o f "political interest" a m o n g F r e n c h farmers c a m e f r o m his observation that farmers h a d in fact one of the highest levels of electoral participation a m o n g F r e n c h c i t i z e n s .4 5.2 22. and it is a l w a y s rather subjective. as w e l l as their representatives' seniority. that i s . this test is not a l w a y s .52 Chapter 4 Problems of Measurement IMPACT O F R A N D O M A N D N O N R A N D O M ERRORS 53 S u c h p r e l i m i n a r y testing of o n e ' s m e a s u r e s is c a l l e d a "pilot study. seniority increases without exception as the representatives' margins of victory increase. how broadly they are intended to apply.5 22. then a c o u l d not be a v a l i d m e a s u r e of A.6 4. S u c h a test c a n be m a d e . T h e m o s t general test of validity is w h a t is c a l l e d face validity. At the m o s t general l e v e l . If there is s o m e m e a s u r e /3 that we are certain is strongly related to c o n c e p t A. B u t a s s e s s i n g the validity of m e a s u r e s is so important that an indirect test. p o s s i b l e .6 0. In the third and fourth c o l u m n s . 4 2 .6 12. In the fifth and sixth c o l u m n s . s u c h as might o c c u r f r o m clerical errors or other sources of unreliability.0 30. S u c h an indirect test of validity is p o s s i b l e only w h e n a r e s e a r c h e r is quite certain that /3 must go along w i t h A.4 Margin of Victory (%) 21. y o u s h o u l d go thoroughly into the b a c k g r o u n d of the things reported t h e r e — h o w they w e r e d e v e l o p e d ." S i m i l a r l y . the relationship at the m e a s u r e l e v e l w i l l tend to be looser and w e a k e r than the true relationship.1 2.2 Margin of Victory (%) 9.6 0.8 4.6 12. although it is n e c e s s a r i l y subjective and o p e n to v a r y i n g interpretations. farmers' responses to the question on political interest correspond to a. T h a t k i n d of certainty is u n c o m m o n and m a y not be shared e q u a l l y by every observer. Is it v a l i d "on its f a c e " ? A f t e r a l l .5 2. and farmers' electoral participation corresponds to B. T h e effect of n o n r a n d o m error is s i m p l e and severe.5 12.5 13. f r o m w h a t we c a n o b s e r v e .0 12. must not have b e e n m e a s u r i n g political interest v a l i d l y . T h e s e techniques s i m p l y require that the investigator think ahead to p r o b l e m s that c o u l d o c c u r in the relationship b e t w e e n c o n c e p t and m e a s u r e and act either to prevent these or to c h e c k to see whether they are present. E a c h set of two c o l u m n s tabulates the c l o s e n e s s of elections in ten congressional districts. i n a s m u c h as representatives f r o m safe districts tend to have greater seniority than those f r o m marginal districts. one that has in it neither r a n d o m nor n o n r a n d o m error.8 12.1 2. T h i s is illustrated in Table 4 . a n d so o n .1 4.2 11. in u s i n g o f f i c i a l d o c u m e n t s . If it is not.8 1.5 Seniority 12.0 3. he c o n c l u d e d that the conventional m e a s u r e s .0 19.0 14. y o u have c o n s i d e r a b l e e x p e r i e n c e w i t h politics and m u s t j u d g e for y o u r s e l f whether the m e a s u r e does w h a t y o u want it to do. the t w o sorts o f error have different effects on the development of theory. Using Simulated D a t a Less Reliable Measures Very Unreliable Measures validity of the m e a s u r e . what the terms m e a n . T h i s is j u s t a f a n c y term for whether a m e a s u r e l o o k s right to y o u . there is no r e a s o n for us to expect any c o r r e s p o n d e n c e b e t w e e n the relationship w e actually o b s e r v e f r o m our m e a s u r e s and the i d e a l i z e d relationship we w i s h to investigate.0 10. m a k i n g them less reliable.0 9. has been added to the original m e a s u r e s .

we w o u l d probably c o n c l u d e that there w a s no relationship. If we w e r e to test the relationship between safe districts and seniority and had only the data f r o m the last two c o l u m n s in hand. 3. they w i l l be m o r e l i b e r a l . s p e n d i n g s h o u l d h a v e r i s e n . the extent to w h i c h u n i m a g i n a t i v e s c h o l a r s w i l l a l l o w moderate errors of m e a s u r e m e n t to a c c u m u l a t e in a statement until the statement loses m u c h of its m e a n i n g . w e r e not a l l that c l e a r l y either D e m o c r a t i c o r R e p u b l i c a n . 2. population 935. T h i s simple fact has tended to be forgotten in a general ethos of " m a k i n g d o " with poor measures. what we do with them is irrelevant. T h r e e d e c a d e s o f r e s e a r c h and c o m m e n t a r y h a d m i s s e d the point. "Understanding of politics" indicated by years of education. had had one seat in the state senate and one seat in the state house of representatives. they f o u n d d r a m a t i c p o l i c y effects f r o m r eappor t i onm ent . " It m i g h t w e l l be that a p o l i t i c a l s c i e n t i s t w o u l d o p e r a t i o n a l i z e the three v a r i a b l e s of this theory in a w a y s u c h as the f o l l o w i n g : /. Carr 369 US 186 (1962). .047. the e m p i r i c a l a n a l o g of the theory b e c o m e s : " T h e more educated a person i s . In the w a k e of the Baker v. but its importance must be constantly underscored. This would seem reasonable. and J a m e s M . Jr. but only slightly. C o n s i d e r a test for the s i m p l e theory: " T o the extent that they understand p o l i t i c s . scholars c o n c l u d e d that c h a n g i n g the electoral r u l e s h a d h a d n o i m p a c t o n p u b l i c p o l i c y . Jefferson County. r e a c h e d e x a c t l y the w r o n g c o n c l u s i o n . Test your measures for possible inadequacies.54 Chapter 4 Problems of Measurement 55 random error has been added. "Need for public services" indicated by the size of the person's family. W h e n A n s o l a b e h e r e a n d h i s c o a u t h o r s tested the effect i n this w a y . before the Court's decision. in Florida. A l a n G e r b e r . Carr d e c i s i o n . In other w o r d s . T h e i m p l i c i t a s s u m p t i o n w a s that rural areas w o u l d h a v e preferred t o k e e p s p e n d i n g d o w n . S n y d e r . understanding should be fairly closely related to education. so backward rural areas that had not experienced much growth in their populations were hugely advantaged relative to rapidly growing cities and suburbs. it would seem that the more dependents a person had. T h e r e a s o n w a s pretty c l e a r : Baker v. the more that person would depend on a variety of public services. the question of how precisely a measure should be calibrated. 5 problems turn out to be of two basic types. s c h o l a r s had l o o k e d to see w h e t h e r taking a w a y the unfair overrepresentation of rural c o u n t i e s h a d led to an e x p a n s i o n of state s p e n d i n g . S o r e d r e s s i n g the i m b a l a n c e s w a s a p p r o x i m a t e l y a w a s h in terms of party strength. the stronger the relationship between the s i z e of that p e r s o n ' s f a m i l y and the probability that the person w i l l vote D e m o c r a t i c .e n o u g h understood by political scientists. using " n e w s p a p e r s read per 1. In the next chapter we look at another aspect of measurement. For instance. I have exaggerated here slightly. you wish to examine. In this decision the Supreme Court ruled that it was unconstitutional to have state legislative districts with widely varying populations. ( 2 0 0 2 ) d e m o n strated this p r o b l e m w e l l in a paper s h o w i n g that three d e c a d e s of r e s e a r c h on whether electoral reapportionment affected public policy had been subtly m i s d i r e c t e d and h a d . Prior to the decision many states had done no legislative redistricting for half a century. L e t me r e e m p h a s i z e the important pitfalls to be c a r e f u l of in m e a s u r e m e n t : w h e t h e r the l e v e l o f s p e n d i n g r o s e — i t w a s pretty o b v i o u s w h y i t d i d n o t — b u t w h e t h e r it w a s distributed m o r e e q u a l l y as a result of m o r e e q u a l r e p r e s e n t a t i o n . Note that the relationship b e c o m e s w e a k e r in the s e c o n d set of data (there are more exceptions to the general tendency) and virtually disappears in the last. B e c a u s e the variables we w o r k w i t h are difficult to m e a s u r e . we have in m a n y c a s e s c o m e to accept measures that we k n o w full w e l l are inadequate. Miami's Dade County. In this chapter I have d i s c u s s e d problems in the a c c u r a c y of measurement. the true test of w h e t h e r e q u a l i t y of representation affected p o l i c y w a s not I n a c c u r a c y in measurement is a critical p r o b l e m w h o s e potential for m i s c h i e f does not yet s e e m w e l l . w h i c h had benefited. " T h i s statement is r i d i c u l o u s . or on the o v e r a l l s p e n d i n g l e v e l of the state. with a population of 9. depending on whether they stem from flux in the measure (the problem of reliability) or from a basic lack of correspondence between measure and concept (the problem of nonrandom error). To the extent that our measures are not valid. that the true test of w h e t h e r e l e c t o r a l institutions affected p o l i c y w a s not the effect on the t w o parties. recurrent n a g g i n g i n a d e q u a c i e s in the c h o s e n m e a s u r e s m a y debilitate a theory so that it b e c o m e s a l m o s t a trivial e x e r c i s e . w h i l e r u r a l a r e a s . t h o u g h . T h e c u r e for this p r o b l e m is s i m p l y to u s e c a r e and imagination i n d e v e l o p i n g m e a s u r e s .000 population" as a measure of the political awareness of various electorates) is another e x a m p l e of this problem. Again. S t e p h e n A n s o l a b e h e r e . T h e s e W h e n they f o u n d that s p e n d i n g l e v e l s after reapportionment w e r e no higher than they h a d b e e n before reapportionment. U n d e r the b a d . 2. Even when a measurement p r o b l e m is not so central as to n u l l i f y the results of a study. 5 N o w . at least. as a result. T h e loose acceptance of cross-nationa l indicators (for instance. i f p e o p l e ' s n e e d for p u b l i c s e r v i c e s i s r e l a t i v e l y great. but w h e t h e r c o u n t i e s that g a i n e d fair IMPORTANCE OF ACCURACY representation thereafter got a m o r e equal share of state e x p e n d i t u r e s .543. "Liberalism" indicated by voting Democratic. Fortunately. Be sure that the measures you choose fit the relationship among concepts that 1. although this is a rough measure. an interesting question is lost through m i s t a k e s in setting up e m p i r i c a l operations to parallel the theoretical question. O f t e n . old s y s t e m both R e p u b l i c a n suburbs and D e m o c r a t i c c i t i e s h a d been d i s e n f r a n c h i s e d . political scientists are now b e c o m i n g more aware of the problem of measurement. In Chapter 8 we w i l l return to tackle the problem of measuring relationships w h e n random or nonrandom error is present. s o i f they lost i n f l u e n c e o n p o l i c y . had had one seat in the state senate and three seats in the state house of representatives. M u c h of survey r e s e a r c h tends to fall into this category. A n s o l a b e h e r e a n d h i s c o a u t h o r s pointed out.

is w e l l worth reading. I conceive of all social behavior as a vast "data bank." only some of which is quantitatively aggregated in yearbooks and the like. or h o w finely calibrated. D o n a l d T. In this chapter we deal w i t h the " q u a l i t y " of the m e a s u r e i t s e l f — h o w p r e c i s e it s h o u l d be. Webb. 7 9 . [M]any of the indicators used in the text may not be readily recognized as such by contemporary social scientists. " in the International Encyclopedia of the Social Sciences (1968). u n e m p l o y m e n t . C a m p b e l l . In a study of N o r w e g i a n p o l i t i c s . 57 . T h e ideas presented in the book are both creative and sound. S e e also K i r k and Miller. and L e e Sechrist (1966). S e v e r a l good examples of specific measurement problems are N i e m i and K r e h b i e l ( 1 9 8 4 ) . In overemphasizing quantities we sometimes miss the most telling data—in any case." A u s e f u l e x e r c i s e w o u l d be to list as m a n y factors as y o u c a n that w o u l d l e a d t o r a n d o m o r n o n r a n d o m error for e a c h o f the f o l l o w i n g m e a s u r e s : i n t e n d e d vote (in s u r v e y s ) . strength o f a r m e d f o r c e s . 1966. if used circumspectly. (Eckstein. and " s e n s e o f c o m m u n i t y " ) rather than p r e c i s e n u m e r i c a l quantities. for quantitative indicators are not always as "indicative" of what one wants to know as other observations.56 Chapter 4 Chapter 5 D FURTHER DISCUSSION A delightful book with a unique approach to handling certain problems of validity is Unobtrusive Measures. with a rather technical d i s c u s s i o n of w a y s to handle them. and C o n v e r s e and Pierce ( 1 9 8 5 ) . S c h w a r t z . Mosteller details a variety of possible sources of invalidity and unreliability. footnote pp. however. data that may be reliable in their own right or used as checks on the inferences drawn from quantitative data. " S t u d y i n g Presidential Performance in C o n g r e s s " : F e l d m a n ( 1 9 8 3 ) . H a m m o n d and F r a s e r (1984). I do use such quantities in what follows. t r i b a l i s m . if it is to be u s e f u l . and personal income. Herbert A s h e r (1974) presents some good examples of reliability problems. nor always obtainable. More often. " T h e M e a s u r e m e n t and M e a n i n g of Trust in G o v e r n m e n t " . This strikes me as both defensible and desirable. readily observable "qualities" are used as indicators of not-so-readily observable qualities. h i e r a r c h i c a l c o n t r o l i n a n a g e n c y . and the text itself is filled w i t h interesting and highly unusual examples. The preceding chapter dealt Precision with problems of the reliability and validity of m e a s u r e s . Harry. Division and Cohesion in Democracy: A Study of Norway (Copyright 1966 by Princeton University Press). Reprinted by permission of Princeton University Press. By an indicator we usually mean nowadays a precisely ascertainable quantity that stands for some imprecise quantity (as G N P may indicate level of economic development. R i c h a r d D. T h o s e p r o b l e m s c o n c e r n e d the relationship b e t w e e n a m e a s u r e and the c o n c e p t that m e a s u r e is intended to mirror. " T h e Quality of S u r v e y i n g R e s p o n s e s A b o u t Parents and the F a m i l y " ." for i n s t a n c e . H a r r y E c k s t e i n felt that it w a s n e c e s s a r y to a p o l o g i z e for the fact that s o m e m e a s u r e s he w o u l d u s e w e r e subjective intangibles ( " w a r m t h i n s o c i a l relations. Reliability and Validity in Qualitative Research (1985). or as the number of casualties in revolutionary violence may indicate its intensity). " M e a s u r i n g Partisanship.8 0 ) 1 'From Eckstein. Frederick Mosteller's article " E r r o r s . a g r e e m e n t w i t h the p r e s i d e n t (in c o n g r e s s i o n a l v o t i n g ) . by E u g e n e J. and much of the rest of which may speak volumes to our purposes.

too m u c h p r e c i s i o n i n m e a s u r e s c a n be a n u i s a n c e . nor i n m a n y c a s e s p o s s i b l e . and it may cause such techniques to give systematically inaccurate results. S i m i l a r l y . w e are limited to a population of 4 5 . i n the e n s u i n g . T h e figure is so chaotic that it is h a r d to d e c i p h e r a relationship. we are interested in l o o k i n g at the entire range of variation and have no particular cutoff point in m i n d . T h i s m e a n s that the p e r c e n t voting fluctuates a g o o d d e a l f r o m one age group to the next. it is difficult to p i c k out a s y s t e m a t i c pattern in the r e l a t i o n s h i p b e t w e e n age a n d p a r t i c i p a t i o n i n e l e c t i o n s . G e n e r a l l y . though. even i n theory-oriented r e s e a r c h . I n s t e a d . C o n s i d e r F i g u r e 5 . we are not able to limit the n e c e s s a r y l e v e l of p r e c i s i o n in this way. If it is true that s o m e t i m e s we m a y be better off w i t h l e s s p r e c i s i o n in our m e a s u r e s . w o u l d be to e x p a n d the study by c o n s i d e r i n g m o r e i n d i v i d u a l s i n e a c h category. A n o t h e r w a y to handle the problem is to decrease the precision of the measure.1 . Data provided by the Interuniversity Consortium for Political and Social Research. PRECISION IN MEASURES P r e c i s i o n in m e a s u r e s c o r r e s p o n d s r o u g h l y to our c o l l o q u i a l u s e of the w o r d precision—that i s . G i v e n the l i m i t e d n u m b e r o f i n d i v i d u a l s (about 1. ( S e e the b o x " L a w o f L a r g e N u m b e r s ." A l t h o u g h as a general rule. National Election Study. k e e p i n g the units of m e a s u r e m e n t relatively fine. a l t h o u g h w e c a n see that participation g e n e r a l l y r i s e s w i t h age. precision in measurement. w h i c h s h o w s the relationship b e t w e e n age and participation in the 2 0 0 2 c o n g r e s s i o n a l election. precise measurement has become enough of a fetish in political science that it is no doubt a l w a y s possible to find someone w h o w i l l d i s m i s s a piece of w o r k because it is not "quantitative. I must first distinguish between two kinds of precision with w h i c h we shall be concerned.58 Chapter 5 Problems of Measurement 59 It is wrong that E c k s t e i n should have felt the need to apologize. for i n s t a n c e . o n h o w p r e c i s e its m e a s u r e s are. T h i s should have been an unnecessary justification. reporting a p e r s o n ' s i n c o m e in dollars is m o r e p r e c i s e than rounding it off to the nearest thousand dollars. and the second. O n e w a y to handle this p r o b l e m . for e x a m p l e . creating a smaller number of categories with a larger number of cases in e a c h . F i r s t .300) c o n s u l t e d i n the p o l l ." In this chapter I sort out j u s t what kinds of precise quantification are helpful in political research. I c a l l the first of these precision in measures. p r e c i s i o n in m e a s u r e s is o b v i o u s l y a g o o d thing. there i s o n l y a s m a l l n u m b e r o f r e s p o n d e n t s for e a c h p a r t i c u l a r age. S . " " G r e e k O r t h o d o x . Data analysis techniques such as regression analysis handle random noise in their own way. presidents. " p ." To discuss it further. given the subject you are studying. do not waste information by imprecise measurement. 2 . If we w e r e registering voters. I f w e study U . the degree of p r e c i s i o n we n e e d is determined by what we w i s h to do w i t h the data. l a r g e l y r a n d o m fluctuation of p e r c e n t v o t i n g . " and so on is m o r e p r e c i s e than reporting it as "Protestant. o r e v e n largely. Reducing precision in measures is thus u n n e c e s s a r y when one is analyzing data. with age measured to the nearest half-decade rather than to the nearest year. and r o u n d i n g i n c o m e off to the nearest ten thousand dollars is still l e s s p r e c i s e . and a drop-off among the very aged. " and "other. O n e theme of this chapter w i l l be that this rule is as susceptible to violation by "quantifiers" as by "nonquantifiers. then it appears l i k e l y that this sort of p r e c i s i o n is not so important that r e s e a r c h s h o u l d b e j u d g e d s o l e l y . in theory-oriented r e s e a r c h . T h e C a r d i n a l R u l e of P r e c i s i o n might read: Use measurements that are as precise as possible." " C a t h o l i c . 6 1 ) T h u s . For example. B u t this u s u a l l y i s neither p r a c t i c a l Reducing precision in this way to eliminate random noise in the data is appropriate only for chartmaking and visual presentation. reporting a p e r s o n ' s r e l i g i o n as " P r e s b y t e r i a n . with the most rapid increases c o m i n g in the first couple of decades. T h i s has been done in F i g u r e 5-2. the measures of percent voting fluctuate less and the f o r m of the relationship between age and participation becomes clearer. S o m e t i m e s . (See Chapter 7 for a description of regression analysis. its importance c a n be overrated.) 2 18 30 40 50 Age 60 70 80 85 Figure 5-1 Age and Participation in 2 0 0 2 Election: Age Measured by Years Source: 2002 Congressional-Election Survey. of c o u r s e . " " R e f o r m e d J e w i s h . any p r e c i s i o n in m e a s u r i n g age that w e n t b e y o n d l a b e l i n g people "under 18" and " o v e r 18" w o u l d be u n n e c e s s a r y and p o s s i b l y a n u i s a n c e as w e l l . W i t h the larger number of cases in e a c h age c l a s s . but unfortunately.

is a case in point.D e c a d e s 60 70 80 8 t h e d a t a later i f y o u w a n t t o . Anglo-Saxon Protestant. so that you will not have to r e d u c e the precision m o r e than you w o u l d like. for e x a m p l e . if age were If it is true. A l t h o u g h w e w o u l d still s e e a strong relationship in these figures. you might draw up a sample consisting of equal numbers of 'WASP is the acronym for white.5 0 ) v o t e d . w e w o u l d find that 5 4 p e r c e n t o f t h o s e i n t h e first unit ( 1 8 . and with groups of 1 (the smallest possible "group") all groups would be either 0 or 100 percent male. (It would not be unlikely for there to be 60 males in 100 people. Precision in measures is important. one should not conclude that precision is not helpful or important. m u c h of the interesting detail of Figure 5-2 w o u l d be lost. given the subject w e are studying. 80 WASP. w e c o u l d a g a i n l o s e s o m e o f t h e pattern that a p p e a r e d there." or "Jewish ' In short. m a k e sure that you include as large a n u m b e r of cases as is practical.) With groups of 10. there would be wild fluctuation. (This law forms the basis for a great deal of statistics.g a t h e r i n g s t a g e is c a s u a l l y t h r o w i n g a w a y precision that m i g h t later p r o v e useful. or stop studying political corruption because many facts are hidden or grouped together. Y o u w i l l h a v e a b e t t e r i d e a t h e n o f h o w fine a g r o u p i n g y o u w a n t to e n d up with. With groups of 100. or stop studying past historical periods b e c a u s e data are limited and m a n y kinds are " p r e g r o u p e d " in inconvenient ways. these suggestions will help meet that rule. you probably would have drawn too few Puerto Ricans and Chicanos. as the Cardinal R u l e states. of discrimination. M a n y questions in political science simply do not admit of great precision of this type. W h e n asking people their age. not " u n d e r 30. It w o u l d be foolish to give up such studies on this account.60 100 Chapter 5 Problems of Measurement 61 L a w of Large Numbers The fact that the smaller the group of individuals sampled. Had you simply drawn a random sample of the population. record the n u m b e r of years old. for example. If age w e r e m e a s u r e d in units of 33 years. w e m u s t be careful not to overemphasize the importance of precision in measures. and you would have had to lump t h e m together with either the blacks or the white ethnics. For e x a m p l e . it m a k e s sense to get an adequate n u m b e r of the group into your study. If a research team selected groups of 1.) The fact is intuitively obvious. they would expect to get very nearly the national figure in all of them. T h e time to ensure that your measures will be as precise as you want is w h e n y o u are gathering data. If y o u ask people their religion. the more any measure based on that group will deviate from its norm is one part of the L a w of Large Numbers. That c o n c e p t d o e s not lend itself to finely calibrated m e a s u r e s ." " R o m a n Catholic." "Catholic. if y o u w a n t e d to study the effects of different k i n d s presidency because the n u m b e r of cases is limited. a t this p o i n t y o u s h o u l d s a v e all the i n f o r m a t i o n y o u get." " 3 0 . No one should stop studying the m e a s u r e d still less p r e c i s e l y t h a n i n F i g u r e 5-2. and Chicanos.4 0 . but not indispensable. a n d that 7 7 p e r c e n t o f t h o s e i n t h e s e c o n d u n i t ( 5 1 . You can always g r o u p 30 40 50 Age Figure 5-2 Age and Participation in 2 0 0 2 Election: Age Measured by H a l f .000 people randomly and calculated the percentage male. even if the resultant data are not "typical" of the entire population. " and the like. A n u m b e r of p r e c a u t i o n s are in order. Thus. if y o u r study c o n c e r n s a specific group within the population. do not group the informat i o n y o u h a v e u n t i l y o u h a v e f i n i s h e d g a t h e r i n g it. write d o w n "Piesbyterian. E c k s t e i n ' s study. . To continue with the example of age and participation. A n o t h e r pitfall to a v o i d at t h e d a t a . He wanted to study the degree of " c o m m u n i t y " in Norway." and so o n — n o t "Protestant. O n t h e o t h e r h a n d . 3 white ethnics. blacks. First. This sort of selection is k n o w n as d r a w i n g a stratified sample. a l t h o u g h t h e r e a r e s i t u a t i o n s i n w h i c h w e m a y w a n t t o b a c k off from precise measures.8 4 ) v o t e d . w h i c h he d e f e n d e d in the quotation given at the start of this chapter. they would get increasingly erratic measurement. B e c a u s e o f t h e l i m i t a t i o n s i m p o s e d b y a s p e c i f i c r e s e a r c h t o p i c . that it is important to be as precise as possible and not to waste information. Puerto Ricans. B u t t h e r u l e a l s o s t a t e s t h a t w e s h o u l d b e a s p r e c i s e a s w e c a n .

as yet. Republican. T h e most primitive w a y to measure a variable is s i m p l y to a s s i g n the individuals being studied to categories of that variable. w e m a y label them " B r i t i s h . we c a n rank the categories a c c o r d i n g to " h o w m u c h " of the variable they e m b o d y . T h i s is c a l l e d nominal measurement. to measure religion. We can assign a score of zero to some point on the scale. in p r i n c i p l e . We c o u l d not say whether the difference in status b e t w e e n " w o r k i n g " and " m i d d l e " w a s greater than that between " l o w e r " and " w o r k i n g . It is clear that these levels of precision c o m p r i s e a nesting progression. B u t w e a l s o f i n d v a r y i n g l e v e l s o f information i n the p r o c e s s o f m e a s u r e m e n t itself. percent voting by districts ( w i t h the unit e x p r e s s e d in percentage points). this is ignored and they are treated simply as interval scales. We are then said to have "ratio" measurement (because we can then take one score as a multiple of another score. m e a s ured as "strong Democrat/weak Democrat/independent/weak Republican/strong r a n k i n g on the variable. In s u c h m e a s u r e m e n t . that i s . or w h a t e v e r ) . however. Note that s u c h a unit w a s l a c k i n g in the two e x a m p l e s of ordinal m e a s u r e m e n t . a set of distinct c l a s s e s to w h i c h individuals w e r e a s s i g n e d ." "Jew. and the scores we a s s i g n to i n d i v i d u a l s s h o w w h e t h e r they fall h i g h e r or l o w e r than others on s u c h a s c a l e . s u c h as " l o w e r / w o r k i n g / m i d d l e / u p p e r " . and all ordinal measurements are also n o m i n a l measurements. they c o n s i s t of gradations w h i c h . B u t the next sort. Interval measurement is more precise than ordinal measurement. a n d (3) if the v a r i a b l e is m e a s u r e d intervally. c a n be s u b d i v i d e d infinitely. I have not included ratio measurement in my discussion here because it has not." we c a n say that measurement is relatively precise insofar as it operates at a level that is relatively informative. w h i c h c a n b e e x p r e s s e d i n thousands o f dollars but A further refinement in precision is possible if in addition to measuring the length of intervals along a scale. we w o u l d still have a n o m i n a l m e a s u r e . w h i c h one h a s the h i g h e r s c o r e . " "Protestant. A l l interval measurements are also ordinal measurements and n o m i n a l measurements. N o m i n a l measurement places individuals into distinct categories of a variable. governmental expenditure on a p r o g r a m ( w i t h the unit e x p r e s s e d either in dollars or in percentage points if m e a s u r e d as a percentage of the budget). we have a c h i e v e d ordinal m e a s u r e m e n t . which is not possible if there is no zero point). S o m e variables c o m m o n l y m e a s u r e d i n intervals are i n c o m e ( w i t h the unit e x p r e s s e d in d o l l a r s ) . " P r e c i s i o n in m e a s u r e s " referred to k e e p i n g distinctions as fine as w a s p o s s i b l e and p r a c t i c a l . and (2) party identification. but it does not tell us anything about how the categories relate to e a c h other. ordinal. " " G e r m a n . To do this. and h o w m u c h h i g h e r that s c o r e is than the other p e r s o n ' s . p r e c i s i o n i n m e a s u r e s ." F u r t h e r p r e c i s i o n in m e a s u r e m e n t is p o s s i b l e if in addition to r a n k i n g the scores a c c o r d i n g to " h o w m u c h " of the variable they represent. are n o m i n a l . A n e x a m p l e i s i n c o m e . n u m b e r of p l a n e s . S o m e m e a s u r e s are continuous. if it w e r e . we c a n say h o w great the differences b e t w e e n the s c o r e s are. (2) if the v a r i a b l e is m e a s u r e d o r d i n a l l y . F o r e x a m p l e . w h i c h is in turn more precise than n o m i n a l measurement. " and so on. we w o u l d s i m p l y m e a s u r e everything at the interval l e v e l . we m a y label people " C a t h o l i c . c a n be o v e r e m p h a s i z e d ." T o measure nationality. in addition to a s s i g n i n g categories. T h e r e are b a s i c a l l y three different w a y s to m e a s u r e . 4 Figure 5-3 Levels of Precision To put it a little differently. p r e c i s i o n in m e a s u r e m e n t . cannot. and interval measurement. if we had an interval measure but c h o s e to ignore our k n o w l e d g e of the distances involved between s c o r e s . we k n o w that the t w o differ on the v a r i a b l e . we have s o m e i d e a of a s c a l e that might ideally represent the variable. we k n o w that the two differ on the v a r i a b l e . Generally. as s h o w n in F i g u r e 5-3. m e a s u r e d in s o m e w a y . we must have s o m e c o m m o n unit of m e a s u r e m e n t for our s c a l e . As a definition of " p r e c i s i o n in measurement. If. 4 . we k n o w that the t w o differ on the v a r i a b l e and w h i c h one has the h i g i e r s c o r e . w e have a c h i e v e d interval m e a s u r e m e n t ." or "other. and so on.62 Chapter 5 Problems of Measurement 63 PRECISION IN MEASUREMENT T h e sort o f p r e c i s i o n d i s c u s s e d i n the p r e c e d i n g s e c t i o n . T h a t i s . All of the interval scales I noted above could in fact be treated as ratio scales. in that it contains information not i n c l u d e d in the others. if two i n d i v i d u a l s h a v e different s c o r e s on a v a r i a b l e . air p o w e r (with the unit e x p r e s s e d in kilotons of b o m b i n g c o v e r a g e . w e w o u l d still have scores o n a n ordinal measure. e a c h of w h i c h is qualitatively different f r o m the others. E x a m p l e s are (1) s o c i a l status. then. body counts (with the unit e x p r e s s e d in n u m b e r of dead p e o p l e ) . A n d i f w e had scores on an ordinal measure but c h o s e to ignore our knowledge of the relative Innate Nature of Levels of Measurement O b v i o u s l y . then (1) if the v a r i a b l e is m e a s u r e d n o m i n a l l y . T h e three major levels of measurement. figured importantly in data analysis in the social sciences. " I f units exist b y w h i c h w e c a n m e a s u r e the intervals b e t w e e n s c o r e s l i k e these. " " R u s s i a n . " nor c o u l d w e say whether the difference between " w e a k R e p u b l i c a n " and "strong R e p u b l i c a n " w a s greater than that b e t w e e n " i n d e p e n d e n t " a n d " w e a k R e p u b l i c a n . the l e v e l at w h i c h we m e a s u r e things is not j u s t a c h o i c e we m a k e .

Figure 5 ." or " s o c i a l c l a s s . a n d ( 2 ) sometimes w e c a n a d d other i n f o r m a t i o n or theory to a n a t u r a l l y n o m i n a l or o r d i n a l m e a s u r e a n d e n r i c h it to a h i g h e r l e v e l than its natural l e v e l : this is d i s c u s s e d in the s e c t i o n after next. I n F i g u r e 5 . A g e is often grouped into categories s u c h as " y o u n g e s t . " T h e S i n o f W a s t i n g I n f o r m a t i o n . we c o u l d not subdivide elections into half-elections." "region. " no unit presents itself by w h i c h categories ( s u c h as " A f r i c a n ." discrete variables are by their nature either ordinal (if they have an ordering to them) or n o m i n a l .4 100 Age and Participation in 2 0 0 2 Election 100 m e a s u r e . for e x a m p l e . or anything like that. " and " o l d e s t " — a n ordinal m e a s u r e . w h i c h w o u l d produce an interval m e a s u r e . " "older m i d d l e . T h e s e consist naturally of separate points that cannot be further divided.a g e d . one that applies only to p r e c i s i o n in m e a s u r e m e n t . w h i c h w e l o o k e d at earlier. however. arises if we are interested in u s i n g the m e a s u r e s to study a relationship b e t w e e n two or more variables.64 Chapter 5 Problems of Measurement 65 w h i c h c o u l d be s u b d i v i d e d into hundreds of d o l l a r s . a n d treat i t a s a l o w e r . O r i n c o m e i s grouped into "low." Esthetically. given the subject y o u are studying. W e s e e . T h u s . of course. into dollars. s u c h as " r a c e . A l l continuous m e a s u r e s lend t h e m s e l v e s naturally to interval m e a s u r e m e n t b e c a u s e their infinite d i v i s i b i l i t y requires that there be s o m e unit of m e a s u r e m e n t that c a n be d i v i d e d . " "religion. with "five e l e c t i o n s " being two units higher than "three elections. T h e r e are a few discrete m e a s u r e s that are interval.A m e r i c a n " and " w h i t e " ) c a n be c o m p a r e d . W i t h more p r e c i s e m e a s u r e m e n t we c a n do a greater variety of things w i t h our data a n d thus h a v e more of an opportunity to develop interesting theories. do not waste information by imprecise measurement. L e t m e provide an e x a m p l e . T h i s alone should be enough to justify our C a r d i n a l R u l e : " U s e measurements that are as precise as possible. In this c a s e . with a few exceptions. s u c h as "elections.a g e d . M e a s u r e s that are not continuous are c a l l e d discrete." T h i s esthetic consideration applies both to precision in measures and to precision in measurement. A m o r e important r e a s o n for f o l l o w i n g the C a r d i n a l R u l e . that all c o n t i n u o u s m e a s u r e s c a n b e e x p r e s s e d a s i n t e r v a l m e a s u r e s . the relationship between age a n d percent voting. F o r most discrete variables. are periodic events that cannot be subdivided. T h e r e are t w o c a v e a t s to this statement.4 . It is for this r e a s o n that p r e c i s i o n i n m e a s u r e m e n t i s more important than p r e c i s i o n i n m e a s u r e s . " and "high. " E n r i c h i n g the L e v e l o f P r e c i s i o n i n M e a s u r e m e n t . it w o u l d seem better to k n o w more about a subject rather than less. h o w e v e r : (1) W e c o u l d a l w a y s i g n o r e i n f o r m a t i o n w e h a v e about a qualitatively different things w i t h data m e a s u r e d at different levels of p r e c i s i o n . the l e v e l o f m e a s u r e m e n t w e u s e i s g e n e r a l l y not a p e r s o n a l c h o i c e but i n h e r e s in the thing we are m e a s u r i n g ." T h e measure w o u l d be interval but w o u l d not be continuous. c 75 100 (C) Age measured intervally Age High 94 . a c c o r d i n g as ( A ) age is c o l l a p s e d to a n o m i n a l m e a s u r e and all k n o w l e d g e of r a n k i n g and unit distance in it is lost. into c e n t s . and so o n . and so o n . We c o u l d use the n u m b e r of elections that a person h a d experienced as a m e a s u r e of his political exposure. is presented in three f o r m s . " a r g u e s a g a i n s t t h i s ) . w h e r e a s a l m o s t all d i s c r e t e v a r i a b l e s are n a t u r a l l y either o r d i n a l o r n o m i n a l . " " y o u n g e r m i d d l e . into tenths of a cent. E l e c t i o n s . B u t i n a s m u c h as I argued earlier that for practical purposes precision in measures might sometimes be sacrificed. " Low Age The Sin of Wasting Information T i m e and again data c o l l e c t e d at a higher l e v e l of p r e c i s i o n are c o l l a p s e d to a l o w e r level to s i m p l i f y h a n d l i n g the data. ( B ) age is c o l l a p s e d to an ordinal m e a s u r e and the k n o w l e d g e of unit distance in it is lost. writing reports." " m i d d l e .l e v e l m e a s u r e (the next s e c t i o n . a n d ( C ) age is maintained at an interval l e v e l of m e a s u r e m e n t . A s a r e s u l t . then. it appears that by itself the esthetic consideration is not compelling. we c a n do 30 40 50 Age 60 70 80 90 25 •£ 5 0 C D O C D Q.

it is followed by a decline. If age and participation were not related. F. greater precision in measures also would have been useful. for instance. Generally. as compared with 86 percent of those in category F. after which there is a decline. especially a finer breakdown of ages above 40. The dashed line tracing the path of the bars approximates the pattern we have already seen in Figure 5-2. large gap between British and American). In Figure 5-4(c). British. One way to do this would be simply to treat the problem as one of a relationship involving nominal measurement. But it is likely that you have in mind some underlying scale along which the colonial experience varied. we could say still more about the relationship. In this case. and D. perhaps due to enfeeblement. we can see that the peak occurs in the late 70s. This suggests an interesting combination of processes: Learning and acquiring the habit of voting during the first years of eligibility causes participation to rise rapidly at first after an initial pause. this is all we can say—that the variables are or are not related. with participation decreasing as age increases. years: C E B G A F D 18-20 21-23 24-30 31-50 51-64 65-80 81-91 Examples of Enrichment Example 1. G. we can see that there is a relationship between age and participation in elections. if we have information about the data that would otherwise be wasted. though. we can enrich our data in this way if we know something more about the data than is reflected in our measurement—in other words. we could see whether or not there was a relationship. then. we may sometimes be able to do this. Enrichment of the Level of Precision in Measurement So far I have argued that we should try not to drop data carelessly to a lower level of measurement. with the United States' possibly lower than the Dutch. A. and that participation peaks around age 70. you might even be able to bring your data up to the level of rough interval measurement. we should always try to raise data to a higher level when this is possible. If age were measured intervally. with ordinal measurement. measured in a common unit. for example." the British colonial experience would rank low and the French quite high. We now know the ranking of the different categories of age: C is youngest. We can now see that the initial increase in participation. as this increase levels off. With nominal measurement. For this particular purpose. However. Notice that in part C of Figure 5-4. Only 16 percent of the individuals in category C vote. If you can array the different mother countries along this scale. untidy measurement at a higher level is better than neat measurement at a lower level. You might. E the next to youngest. Given a certain amount of boldness and ingenuity. This is indicated by the fact that different age groups participate at different rates. You are studying the relationship between the colonial experience of new nations and the stability of democratic institutions in those nations. we would expect about the same percentage of individuals in each category to vote. with age measured nominally. In addition to seeing whether or not the variables are related. between the ages of 20 and 30. that there is a big jump over about the next ten years. by raising the level of measurement. With the greater precision in measures of Figure 5-2. If you could make a reasonably informed guess at the relative differences (such as Dutch rather close to the French. for example. we can say a good deal more. you can use them as an ordinal or interval measure of the scale. Interval measurement adds information about the magnitude of differences in age expressed by the categories. is slow. The Dutch and American experiences would fall somewhere between these. we probably would have measured at the higher level in the first place. That is. . By a similar argument. and that you are really interested in the relationship between this scale and the stability of democratic institutions. that pattern is one of increasing participation with increasing age up to some point. A few examples may be the best way to show how measurement can be "enriched" in this way. If it were. and then a reversal. On the scale "level of assimilation. the bars in the graph have been stretched to take into account the added information that the categories represent the following age groups. Preserving a high level of precision in measurement deserves a strong priority in research. you want to compare the stability of democratic institutions in former French. we could see the pattern of the relationship. and we also could see the rate at which one variable changed in response to changes in the other variable. the colonizing countries comprise at least an ordinal measure of the level of assimilation. Often. Dutch. This information may not be enough to provide neat. depending on which country had done the colonizing. and then come B. we can see the pattern of the relationship. be interested in whether or not the colonizing country tried to assimilate the native population to its own culture and the effect this had on the stability of the resulting institutions. clean measurement at the higher level. In the second case. This is a richer description of the relationship than we could have gotten using the ordinal measurement. and American colonies. the information lost through the low precision in measures in part C is slight compared with the information lost by lowering the precision in measurement in parts A and B. If age were measured intervally.66 Chapter 5 Problems of Measurement 67 In the first case.

R O T C . speed of economic development. The various types of colonial experience we cited vary on degree of assimilation. but the Naval A c a d e m y increased budget support by 29 percentage points over R O T C students. Example 2. t h e scores a s s i g n e d m i g h t b e n o n .0 to non- tend to encourage pro. N o w look d o w n the graph to see what scores on "military content" must be associated with positions A. simply as social organizations. In a study of A n n a p o l i s m i d s h i p m e n . J e w s w o u l d fall n e a r t h e b o t t o m o f s u c h a s c a l e and fundamental Protestants. it w a s found that 39 percent of the m i d s h i p m e n thought the A m e r i c a n military b u d g et was too small. Perhaps. (To b e exact.or antimilitary attitudes. and C on the line. h o w d e m o c r a t i c their legal structure is. If you were willing to a s s u m e that at that time the Naval A c a d e m y was typical of the military academies and that there was a linear relationship between the promilitary orientation of an educational relationship between religion and political p r o g r a m a n d the p e r c e n t a g e o f its s t u d e n t s w h o b e l i e v e d that t h e m i l i t a r y b u d g e t w a s too small. You are studying the 5 R O T C . with the military academy highest and the n o n . old-Reformation Protestant.83. (For a m o r e detailed explanation of "linear relationship. or what have you. T h e U n i t e d States. 1971). low on assimilation but high on oppression. Example 3 . These scores are A ' .83 times as great as the interval between R O T C and n o n .68 Chapter 5 Further. 10. B e c a u s e y o u know the scores on budget support for non-ROTC.R O T C p r o g r a m lowest. Problems of Measurement general university. and you can treat the variables readily and directly as interval m e a s ures. Abstracting the appropriate ordered variable(s) from a nominally measured variable is much the same as abstracting the appropriate dimension(s) from a set of multidimensional concepts. J e w s w o u l d fall l o w o n t h e " a p o c a l y p t i c " s c a l e b u t h i g h o n t h e "closeness" scale. p r o b a b l y w o u l d fall a b o v e t h e m i d d l e i n a s s i m i l a t i o n . geopolitical position. B.0 t o n o n . 4 . high in assimilation and toward the middle in oppression. general university. a n d a c a d e m y . you do not have units in advance by which to measure military c o n t e n t . and C. T h e s e form an ordinal measure of "militariness" readily enough. N a v a l A c a d e m y w o u l d t h e n h a v e t o b e 2 7 . and so on). Example 4 . T h u s y o u m i g h t be interested in predicting the stability of d e m o c r a t i c institutions from t w o variables simultaneously: (1) the level of assimilation at w h i c h the colonizer a i m e d . F o r instance. for e x a m p l e . In this case you could array the different religions in t e r m s of the size a n d " c l o s e n e s s " of their congregations. y o u c a n p r o j e c t a c r o s s t h e g r a p h t o s e e w h e r e t h e y s h o u l d fall o n t h e l i n e showing the relationship. ROTC. Belgium. You can distinguish three types of Notice that the manner in which measurement of nominal variables has been enriched in these examples parallels the "dimensional analysis" that I urged in Chapter 3." see the box on p. T h u s . Often. O f t e n s u c h a p r e c i s e g u i d e is s i m p l y . a n d n o n . extent of oppression.R O T C . You might assign 4. If so. to enrich ordinal data.) T h i s g i v e s y o u all y o u n e e d t o c o n s t r u c t a n interval m e a s u r e — t h a t is. Of course. 3 9 . b u t l o w i n o p p r e s s i o n . 71) If the degree of promilitary orientation in R O T C programs increased support of higher budgets by 6 percentage points over n o n . a rather p r e c i s e p i e c e of o u t s i d e i n f o r m a t i o n w a s u s e d to tell h o w long t h e intervals s h o u l d b e . a c o m m o n unit m a y appear immediately.0 t o R O T C . Non-ROTC falls at A. B ' . or whatever. You w a n t to m e a s u r e the extent to w h i c h different educational p r o g r a m s A c a d e m y . then as in E x a m p l e 1. It should be clear that the s a m e p r i n c i p l e w o u l d h o l d for a n y linear relationship. you have in mind an underlying d i m e n s i o n that causes y o u to expect a relationship b e t w e e n the two variables. In E x a m p l e 3.R O T C a n d 2 . a n d so o n . nominal classifications involve an infinite number of dimensions. Y o u m i g h t a s s i g n 1. M o s t s i m p l y . Alternatively. without participation in R O T C . 5 R O T C a n d 8. near the top. in w h i c h case Naval A c a d e m y w o u l d have to be 6.R O T C . then (given your assumptions) the interval b e t w e e n military academies and R O T C must have been 4. and (2) the extent of oppression and violence during the colonial period. and the Naval alienation. a n e s t i m a t e of the relative differences (intervals) b e t w e e n scores of the measure. R o m a n Catholic. this information w o u l d have a l l o w e d y o u to reconstruct the interval measure implicit in the ordered variable. Here I have drawn two of the m a n y possible linear relationships between the "military c o n t e n t " of e d u c a t i o n a n d support for the defense budget. as in the next example. the distance from B' to C must be a p p r o x i m a t e l y five t i m e s t h e d i s t a n c e f r o m A ' t o B . your theory might be that religious groups.R O T C students. N o t e that the "right" ordinal or interval arrangement of the n o m i n a l categories d e p e n d s on which underlying scale y o u w a n t t o t a p . t h e ratio o f t h e distances is 4. we m u s t use a g o o d deal of ingenuity. as a m e a s u r e of different underlying scales. b u t i t i s a p p a r e n t t h a t w h a t e v e r u n i t s y o u c o n c e i v e of. and thus each w o u l d represent a different m i x of the t w o variables. you might have s o m e additional information on the subject. however. you well might want to use a given nominal m e a s u r e m e n t twice or educational programs: military academy. F r a n c e . you could take the different religions ranked by their apocalyptic content as an ordinal o r i n t e r v a l m e a s u r e o f t h i s s c a l e . B e c a u s e an interval m e a s u r e d o e s not a s s u m e that a zero point is k n o w n . Religion could simply be measured nominally (Jewish. C a n y o u m a k e a n interval m e a s u r e out o f this r a n k i n g ? Fortunately. for instance. Like vague. abstracting the d e g r e e of " c l o s e n e s s " from religious affiliation suggests o n e ready interval m e a s u r e — a v e r a g e size of c o n g r e g a t i o n s — a l t h o u g h this admittedly w o u l d be only a r o u g h m e a s u r e w h o s e validity w o u l d be questionable. multidimensional concepts.83. 10 percent of the R O T C students agreed. build involvement a m o n g their m e m b e r s . The colonizers w o u l d be arrayed differently along these t w o scales.R O T C students. This is demonstrated geometrically in Figure 5-5. y o u can arbitrarily pick the unit in w h i c h t o e x p r e s s y o u r m e a s u r e . as did 4 percent of the n o n . and the Naval A c a d e m y at C. 0 t o R O T C . a n d w h a t e v e r l i n e a r relationship between the t w o variables exists. In working with ordinal variables (either "enriched" nominal variables or variables which obviously are ordered from the outset). 69 with participation in m o r e in the s a m e analysis.R O T C students (Karsten and others. 3 2 . You might be thinking in terms of the extent to which one's religion promotes an apocalyptic view of the universe. ROTC at B. R O T C .

y inc r e a s e s b y o n e . and R u s s i a . T h e alternative is to ignore a portion of w h a t we know." and "military power. a n d R u s s i a . n o m a t t e r w h a t v a l u e x starts w i t h . and because of the arbitrary decisions to be made in E x a m p l e 4. t h o u g h quite r o u g h . E n r i c h i n g m e a s u r e m e n t in this w a y is a r e s e a r c h technique for w h i c h the r e s e a r c h e r ' s creativity and ingenuity are c r i t i c a l . T h e m e a s u r e i s r o u g h a n d s u b j e c t i v e . T a k i n g the " s a f e r " course a l l o w s us to state our results with more confidence but reduces the importance of what we have to say. a n d a t i m i d r e s e a r c h e r w o u l d be content to treat this as an o r d i n a l v a r i a b l e . T h u s . 10. and the assumptions required for it. but y o u still c o u l d k n o w m o r e than n o t h i n g about the length o f the i n t e r v a l s . C o s t a R i c a . for e x a m p l e ." "economic power. 4 0 .h a l f unit w h e n x c h a n g e s b y o n e unit. T h e question remains: A r e y o u better off w i t h s u c h rough interval measures. T h e r e l a t i o n s h i p in g r a p h B is n o n l i n e a r . O n e m a r k of a good r e s e a r c h e r is that she boldly seeks out all c h a n c e s — n o t j u s t tl le o b v i o u s or safe o n e s — t o r a i s e the level of m e a s u r e m e n t in her w o r k . Y o u m i g h t c h o o s e . I think the a n s w e r is obvious." . and y o u k n o w w h i c h i n t e r v a l s are greater 'Although I have dubbed the variable "national power. if x is low. S p a i n . m e a s u r i n g i n this w a y d o e s not i g n o r e the v a l u a b l e c o l l e c t i o n o f o u t s i d e i n f o r m a t i o n — m o s t o f i t admittedly u n d i g e s t e d — that y o u a l r e a d y p o s s e s s . C o n s i d e r a p r o b l e m i n w h i c h y o u w a n t t o m e a s u r e the n a t i o n a l p o w e r o f Costa R i c a . but i t c o u l d b e r e f i n e d b y b r i n g i n g t o bear the i m p r e s s i o n s o f m o r e k n o w l e d g e a b l e o b s e r v e r s w i t h r e g a r d t o r e l a tive n a t i o n a l p o w e r . On the other h a n d . 6 T h e order i s c l e a r . T h e r e l a t i o n s h i p in g r a p h A is linear. y o u k n o w that the intervals are not e q u a l . Great Britain. if x is h i g h . a u n i t c h a n g e in x p r o d u c e s relatively g r e a t c h a n g e in y. u n a v a i l a b l e . there is plenty of r o o m for errors to occur. 1 . T h e interval measures y o u derive in E x a m p l e s 3 and 4 m a y not leave y o u fully satisfied. G r e a t B r i t a i n . or s h o u l d y o u opt to ignore the size of intervals and treat the variables as ordinal m e a s u r e s ? G i v e n the wider range of theoretical c o n c e r n s that one c a n c o v e r using interval data. Y o u c o u l d create a n a c c e p t a b l e ." for most purposes this would be broken down more profitably into component dimensions. B e c a u s e of the indirect c h a i n of reasoning in E x a m p l e 3. a unit c h a n g e in x p r o d u c e s relatively little c h a n g e in y. It m a k e s sense to use all the information we have about a question. such as "diplomatic influence. regardless of the value of the s e c o n d v a r i a b l e . inasmuch as the relationship between the two variables is different for different v a l u e s of x. Spain. T h e r e l a t i o n s h i p in g r a p h B m u s t be e x p r e s s e d by a c u r v e w h i c h tilts differently at different v a l u e s of x. Linear Relationship A r e l a t i o n s h i p b e t w e e n t w o v a r i a b l e s is linear if o n e v a r i a b l e c h a n g e s by a c o n s t a n t a m o u n t w i t h a c h a n g e of o n e unit in the second variable. interval m e a s u r e o f the n a t i o n a l p o w e r o f the v a r i o u s c o u n t r i e s s i m p l y b y a s s i g n i n g arbitrary interval lengths i n a c c o r d a n c e w i t h y o u r g e n e r a l i d e a o f the m a g n i t u d e s i n v o l v e d . H e r e . 7 . the r e l a t i o n s h i p i n g r a p h A can b e e x p r e s s e d b y a s t r a i g h t line. i n a s m u c h as t h e r e l a t i o n s h i p is the s a m e at all v a l u e s of x. M o s t i m p o r t a n t .70 Chapter 5 Problems of Measurement 71 than others. even if we must supplement that information with assumptions and h u n c h e s .

Being a "quantifier" or "nonquantifier" is not an either/or question." "highly developed team spirit. A nonquantifier can measure things only approximately. In this chapter I pointed out that the different levels of measurement were important because it was possible to make a greater variety of statements about relationships between variables that were measured in "advanced" ways. I have constructed it merely as an example of the wide applicability of pseudo-interval measurement. One difference between interval and ratio measurement is that whereas it is possible to add and subtract interval measures. we may note that measuring variables nonquantitatively requires the same sort of boldness that I urged in enriching data to a higher level of measurement. 7 8 FURTHER DISCUSSION A readable presentation along the lines developed in this chapter is Edward R. It is still based on subjective descriptions. multiply. for example. either random or nonrandom. but it is no more "quantified" than the other. To return to the quotation from Eckstein with which I introduced this chapter. More important. for example. such as "rather strong sense of community. one should be concerned about the possibility that "enriching" a level of measurement may insert either sort of error into the new measure. It is not true. I think the risk is often justified. You should note that in this chapter I have presented a fairly radical position in favor of upgrading levels of measurement. He wanted to measure variables and to assess relationships between those variables. and finally. that the shift continues from there on up the hierarchy at a diminished rate. another student might approximate interval measurement. we can see that this is primarily a problem of precision in measures." and so on. the less likely he is to press for the interests of his region. it is possible to add and subtract. the less likely he is to press for the interest of his region. this is not possible with simple interval measurement. I have no idea whether this is a reasonable theory. in addition to being able to measure the relative interval between different values of the measure. Can this sort of work be described in terms of precision in measures and precision in measurement? Do the rules I have prescribed in this chapter hold for this sort of work as they do for more quantitative studies? First. the process reverses itself. given ratio measurements. She might state. that this shift occurs very sharply as he moves from some particular level of the hierarchy to another. a bit technical. rarely allowed him to use objective or numerical measures. Tufte's "Improving Data Analysis in Political Science" (1968-1969). Another excellent article. he could use only vague descriptions. with the men at the top feeling secure enough to concern themselves more than those immediately below them with the interests of their regions. For many variables in which he was interested. To the extent that one is worried about measurement error. This is certainly a richer statement than the earlier version. many scholars believe that the costs in carrying out such "enrichment" are greater than the benefits. but there is no particular reason why such research cannot approximate interval measurement. The only difference is that in the second case the researcher enriches the level of measurement at which she is working. A fairly technical paper that urges researchers to take seriously the risk of inserting measurement error by treating ordinal variables as if they were interval (an extrapolation similar to enrichment but riskier and less well thought out) is that by Wilson (1971). it is clear that he was interested in doing the same things as quantifiers do. a position that would not be accepted by all social scientists. Most of these descriptions had to depend on his subjective judgment rather than on impersonal outside indicators. made in the absence of objective indicators. What statements might be made about relationships between ratio-measured variables that could not be made about relationships between interval-measured variables? Let me repeat my earlier admonition. It is possible to say that a given score is twice as great as another score. especially with numerical data). not of precision in measurement. But the same general conclusions hold for "nonquantifiers" as well. 8 7 . however. This statement approximates a relationship between two variables measured at the ordinal level ("height in the hierarchy" and "likelihood of lobbying"). For example.72 Chapter 5 Problems of Measurement 73 Quantifiers and Nonquantifiers Again This discussion of precision in measures and precision in measurement has centered on the work of "quantifiers" (those who work by preference with objective data. The extent to which a researcher uses objective data is generally not a matter of personality but rather of the subject he is studing and the data available for that subject. however. simply because it was more convenient to introduce it in that way. that the higher a Chinese politician rises in the hierarchy. Why would the technique in Example 3 not be appropriate if the relationship between "military content" and "support for defense budget" were nonlinear? What might you do in this case? 2. is by Carter (1971). that a temperature of +20°F is half as "hot" as +40°F. a student of Chinese politics might suggest that the higher a Chinese politician rises in the hierarchy. Working more boldly and imaginatively. even though "zero" is arbitrarily assigned to one point on the scale. but in a different way. "Ratio" measurement is interval measurement for which. Personally. it is possible to assign the value zero to some point on the measure (see footnote 3). Fahrenheit temperature is an example of an interval measure that is not a true ratio measure. that near the top of the hierarchy. His subject matter. and divide ratio measures. Two questions you might consider are: 1.

and these same groups. are m o r e apt than brunettes to be R e p u b l i c a n s . although it is probably true in the U n i t e d States that b l o n d s . If. but that both are p r o d u c e d by s o m e t h i n g e l s e — t h e e t h n i c . It might be a r g u e d . We cannot a l w a y s m a k e a c a u s a l interpretation w h e n two p h e n o m e n a tend to c o i n c i d e . T h i s interpretation is a subjective one. To read causality into s u c h c o . In general. requires an interpretation. although the one f o l l o w s the other regularly. T h e r e f o r e . S i m i l a r l y . s o c i a l c l a s s and party vote are related in the U n i t e d States. T h u s s o m e o n e c o u l d a r g u e — a n d i t w o u l d not be an u n r e a s o n a b l e a r g u m e n t — t h a t c l a s s is not a c a u s e of the vote. and then f o l l o w this u p w i t h s o m e ideas o n r e s e a r c h d e s i g n . but a c a u s a l interpretation of this r e l a t i o n s h i p is m u c h m o r e s u b j e c t i v e . inflation is low. theory-oriented p o l i t i c a l r e s e a r c h i s almost e x c l u s i v e l y c o n c e r n e d w i t h causal relationships. 74 . If we see that those U . there is o b j e c t i v e e v i d e n c e for the e x i s t e n c e QJ o) of a r e l a t i o n s h i p . the c o i n c i d e n c e of two p h e n o m e n a must i n c l u d e the idea that one of them produces the other. In the e x a m p l e of s o c i a l c l a s s and the vote. W i n t e r does not c a u s e s p r i n g . To qualify as a " c a u s a l " relationship. w h i c h is a rather o b j e c t i v e statement. for i n s t a n c e . A l m o s t every situation in w h i c h we w i s h to m a k e c a u s a l statements is similar to this example. we o b s e r v e that two p h e n o m e n a tend to o c c u r together ( s m o k i n g and heart d i s e a s e . that c l a s s d o e s not produce the vote. T h e example o f c l a s s a n d v o t i n g . c o n f o r m i t y and s u c c e s s ) . As I pointed out earlier. In this regard. C o n s i d e r two further e x a m p l e s : I f w e see that people w h o s m o k e regularly h a v e a greater i n c i d e n c e of heart d i s e a s e than n o n s m o k e r s . and r a c i a l c o n f l i c t s that h a v e s u r f a c e d often i n A m e r i c a n p o l i t i c s . T h e question of whether the relationship is a c a u s a l one. senators w h o c o n f o r m to the i n f o r m a l rules of the Senate tend to be the ones w h o s e bills get p a s s e d . s i m p l y by a c c i d e n t . for i n s t a n c e . w h o are relatively l i k e l y to be w h i t e and Protestant. b e c a u s e those w h o vote R e p u b l i c a n tend to be f r o m the m i d d l e and upper c l a s s e s . r e l i g i o u s . A good e x a m p l e of the difficulty of a s c r i b i n g c a u s a l direction is the relationship between central bank independence and inflation. E m p i r i c a l . O u r interpretation of this is that one of the p h e n o m e n a c a u s e s the other. G e n e r a l l y speaking. we c o n s i d e r that the values of one v a r i a b l e produce the testable. T w o variables are related if certain v a l u e s of one variable tend to c o i n c i d e w i t h certain v a l u e s of the other v a r i a b l e . a theory in its s i m p l e s t f o r m u s u a l l y c o n s i s t s of three things: independent variables (those we think of as d o i n g the " p r o d u c i n g " ) . it is possible to design the research in w a y s that c a n help us significantly in doing this.o c c u r r e n c e . w h e r e central b a n k s are independent. dependent v a r i a b l e s (those we think of as being " p r o d u c e d " ) . they w i l l use monetary p o l i c y more a g g r e s s i v e l y to fight inflation. and of w h i c h variable c a u s e s w h i c h . the a s s e r t i o n of a causal relationship differs f r o m the m e r e a s s e r t i o n of a relationship. although as y o u w i l l see. in addition. and h o w we interpret it. T h e notion of c a u s e i n v o l v e s m o r e than that. we s a y that s o c i a l c l a s s is a " c a u s e " of party vote. 14). In both c a s e s . R e s e a r c h d e s i g n — t h e w a y i n w h i c h w e structure the gathering of d a t a — s t r o n g l y affects the c o n f i d e n c e w i t h w h i c h we c a n put a c a u s a l interpretation on the results of r e s e a r c h . we might c o n c l u d e that c o n f o r m i t y is r e w a r d e d in the Senate. that p e r s o n might say. S . In this chapter we first d i s c u s s the i d e a of c a u s a t i o n . D i s t i n g u i s h i n g b e t w e e n this v e r s i o n of the r e l a t i o n s h i p and the m o r e c o m m o n v e r s i o n is at least partly a matter of j u d g m e n t . i s a n e x a m p l e o f a c a u s a l r e l a t i o n s h i p . In a v a l u e s of the other v a r i a b l e . T h u s . T h e r e f o r e . they tend to c o i n c i d e because values of the one tend to produce distinct values of the other. W e have seen m a n y s u c h e x a m p l e s o f relationships i n earlier chapters. all that we k n o w directly from our data is that two variables tend to o c c u r together. e c o n o m i s t s think that if central b a n k s ( s u c h as the F e d e r a l R e s e r v e in the U n i t e d States) are relatively independent of political control. T h e question of whether or not there is a relationship is objectively T h u s far everything we have l o o k e d at in this book has been j u s t i f i e d in terms of h o w u s e f u l it is in establishing relationships b e t w e e n v a r i a b l e s . we might c o n c l u d e that s m o k i n g c a u s e s heart d i s e a s e . and c a u s a l statements l i n k i n g the t w o (refer again to p.Chapter 6 Causal Thinking and Design of Research CAUSALITY: AN INTERPRETATION 75 T h e m o s t important thing to note about c a u s a l t h i n k i n g is that it is an interpretation of reality. C e r t a i n l y . It is not m e r e l y true that the t w o v a r i a b l e s tend to c o i n c i d e . hair c o l o r does not c a u s e political party preference. the r e l a t i o n s h i p is a causal relationship. a l s o tend to be w o r k i n g c l a s s . It is time n o w to l o o k m o r e c l o s e l y at what a " r e l a t i o n s h i p " i s . c e r t a i n ethnic groups tend to vote for the D e m o c r a t s . we must add something more. R a t h e r . noted a b o v e . the c o i n c i d e n c e o f c l a s s and party votes i s j u s t t h a t — a c o i n c i d e n c e . W e f e e l that there is s o m e t h i n g about a p e r s o n ' s s o c i a l c l a s s that m a k e s that p e r s o n m o r e l i k e l y to vote in a c e r t a i n w a y .

we could infer that hair color did not cause party preference. members of Congress who vote to increase aid to education tend also to support increases in welfare spending. Accordingly. Survey research. That is. why do we bother with the notion of causation in our theories? What difference does it make whether class causes voting for a certain party. The phenomena coincide because of logical necessity. and then are influenced by the party's leaders to adopt its position on welfare programs as their own. In our previous example of hair color. is a case in point. which causes which? We might think that voters choose the party that offers the policies they prefer. we might have looked only at WASPs. their coincidence is tautologically determined. we can use our data to test the relationship. that bank independence causes low inflation? Batalla (1993) suggests that the opposite causal interpretation may be true. Thus. Reviewing the history of the 1920s and 1930s in Latin America. whose writers believed that some people could prophesy what was to come in the future. Given that two variables are causally related. If A and B coincide but A does not cause B. most of them took away that autonomy. but is it true. there are many instances in which it cannot be used. Rather. There are three interpretations available to us. One useful convention in Western culture—but it is only a convention. the result. we can see whether it has led the original two variables to coincide in such a way as to resemble a causal relationship. but it might also be that voters choose the Republican party for other reasons. for instance. the three countries whose central banks were most independent had averaged only about 3. p. In general. November 20. we do not think of either of them as causing the other. So. 94. . low inflation rates allowed governments to tolerate independent central banks. Coincidence without cause gives you no lever. yet we do not think of winter as producing spring. simply because they measure the same thing. he notes that many countries had established autonomous central banks by the late 1920s. but that when high rates of inflation hit in the mid-1930s. But by setting up a study in certain ways and by manipulating the data appropriately. the three countries with the most politicized banks had averaged almost 8 percent inflation from 1955 to 1985. such as foreign policy. Only you can decide which is the cause and which. Causation is not involved at all. if we can establish that change in one of the variables precedes change in the other. 3 'The Economist. 2 Summary Let me pull together the argument to this point. or does low inflation give us independent central banks? If ascribing cause to the coincidence of two things is so tricky. We must decide from outside the data at hand whether a coincidence of two phenomena is of this type or whether it involves causation. which causes which? Do independent central banks give us low inflation. by definition. it is clear which variable must be the cause. If voters who like the Republican party also tend to oppose welfare programs. then. Thus. and if we are sure that there is causation between the two. some basis for choosing how to act. while high inflation rates led the governments to pull the banks under their control. If we then found that blond WASPs did not tend more often than brunette WASPs to be Republican. that is. as in this hair color example. and our choice from among them is ultimately subjective. It often happens that two slightly different measures of the same concept coincide. 3 1993 study.76 Chapter 6 Causal Thinking and Design of Research 77 Thus there are techniques by which we can manipulate our data to eliminate some of the alternative causal interpretations of a relationship. the future event caused the prior prophecy to occur. we can settle some of the problems in making causal interpretations. and now blonds and brunettes no longer differ in their politics. But there is always one question about causation that remains purely subjective and that cannot be resolved completely by any techniques of data handling. For example. It generally is not enough for us to note that two phenomena coincide. we can infer that the difference in voting was due not to the difference in hair color (which was merely a coincident variable) but to the difference in socioethnic background. A n example of a cultural tradition in which causation does not necessarily work forward in time is that of the Old Testament. For example. We generally also want to interpret why they coincide. as the economists believe. if we artificially hold social position constant by looking only at WASPs. which of the variables causes which? Suppose that two phenomena coincide and that there appears to be a causal relationship between them. /. where we think that a third variable is causing two other variables to coincide accidentally. 2 The technique of holding constant is discussed in greater detail at the conclusion of the chapter. In effect. spring follows winter. 1993. in which variables usually are measured just once and in a single interview. This is not because their votes on one issue cause them to vote the way they do on the other. that there is a relationship between the two. or whether the coincidence of class and party is due to something else that causes both of them? Remember that the ultimate purpose of theories is to give us levers on reality. we assume that pulling a trigger causes the shot that follows. and compared blonds and brunettes. We can then conclude that hair color does not cause political preference. rather than vice versa. By holding constant the third variable. ELIMINATION OF ALTERNATIVE CAUSAL INTERPRETATIONS A causal interpretation is something that cannot come solely from our observations. even though it is so well established that it seems "natural" to us—is that causation works forward in time.75 percent inflation. Although this convention frequently simplifies things for the researcher. both votes are an expression of their general disposition to spend money on social programs. A slight variation of this often occurs in the social sciences. We would expect them to coincide. changing A will not change B.' We see.

78

Chapter 6

Causal Thinking and Design of Research

79

2. The relationship we observe is a result of outside factors that cause the two phenomena at hand, and thus neither of these phenomena causes the
other. The study of hair color and party preference was an example of this. By setting up the study appropriately, we can control for various outside factors in order to concentrate on the relationship in question. In the hair color example, such a control was used. To this extent, we can see from the data whether the coincidence of the phenomena is of this sort. But we are still not exempt from making assumptions, for we must first have assumed the outside factor(s) causally prior to the two coincident phenomena. This is not always an easy decision to make. If it is possible to set up a true experiment (described in the next section), we can eliminate this possibility. But this is often not possible in "field" social sciences such as political science or sociology. 3. One of the phenomena causes the other. Here we have a true causal statement. We are still not finished making assumptions, of course, for we must decide which of the phenomena is the cause and which the effect. That is ultimately a subjective decision, though often we are aided in making it by the convention that causation must run forward in time. As I have said so often in this book, one of the pleasures of research is that nothing in it is automatic. Even the most "quantitative" techniques do not take away our obligation and our right to be creative and imaginative. The fact that causal analysis is ultimately subjective may trouble us—objectivity always seems more comforting than the responsibility imposed by subjective judgment—but in a way it is also a great comfort, inasmuch as it keeps us, and what we do with our minds, at the heart of our research.

decentralize decision making. After the reform, another week's tabulation shows increased output. Conclusion: Decentralized decision making increases output. 2. Reagan's victory over the Soviet Union. During the Reagan administration (1980-1988) the United States steadily increased its military spending. Over the three years from 1989 to 1991 the Soviet Union collapsed, which conservatives hailed as a victory for Reagan's policies. Conclusion: The economic strain of matching Reagan's military buildup had been too much for the Soviet system, and had led to its collapse and the end of the Cold War. 3. Organizing the poor. In anticipation of a major campaign to organize the poor of a city, a survey is taken among them to measure their interest in politics. At the end of the organizing campaign, the same people are asked the same questions a second time. It turns out that those who were contacted by the campaign workers have indeed acquired an increased interest in politics, compared with those who were not. Conclusion: The campaign has increased the political awareness of the poor.

4. Tax-reform mail.

The Congressional Quarterly reports the proportion of each senator's mail which favored tax reform. Comparing these figures with the senators' votes on a tax-reform bill, we see that senators who had received relatively favorable mail tended to vote for the bill, whereas those who had received relatively unfavorable mail tended to vote against it. Conclusion: How favorable a senator's mail was on tax reform affected whether or not she voted for it. 5. Presidential lobbying. In an attempt to measure his influence over Congress, the president randomly selects half the members of the House. He conducts a straw vote to find out how all the members of the House intend to vote on a bill important to him. He then lobbies intensively among the half he has randomly selected. In the final vote in the House, the group that he had lobbied shifted in his favor compared with what he could have expected from the earlier straw vote; the other half voted as predicted from the straw vote. Conclusion: His lobbying helped the bill. Let us look at the design of these studies to see how many alternative causal interpretations each can eliminate.

A FEW BASICS OF RESEARCH DESIGN It should be evident from the discussion so far that the basic problem in causal analysis is that of eliminating alternative causal interpretations. Whenever two variables vary together (are related, coincide), there is a variety of causal sequences that might account for their doing so. A might cause B, B might cause A, both A and B might be caused by something else, or there might be no causation involved. Our task is to eliminate all but one of these, thus leaving an observed relationship, together with a single causal interpretation of it. Some of these alternatives can be eliminated only if we make assumptions from outside the actual study. But we also can design the study in such a way that certain alternatives are impossible. This will leave an interpretation that is dependent on fewer subjective assumptions and can thus lend a greater measure of certainty to the results. Consider these examples: /. Agency study. An organizational analysis of a government agency is made in which workers keep track of their output for a week. The organization is then restructured to

Designs Without a Control Group In the first two examples, the design is of the form:
1. M e a s u r e the d e p e n d e n t v a r i a b l e . 2 . O b s e r v e that t h e i n d e p e n d e n t v a r i a b l e o c c u r s . 3. M e a s u r e the dependent variable again. 4 . I f t h e d e p e n d e n t v a r i a b l e h a s c h a n g e d , a s c r i b e that t o t h e o c c u r r e n c e o f t h e i n d e p e n d e n t variable.

80

Chapter 6
T h u s , in " A g e n c y Study," (1) the w o r k e r s ' o u t p u t is tabulated; (2) the o r g a n i -

Causal Thinking and Design of Research 81
Nations has affected the foreign policy of every country in the world since 1945. H o w c a n a s t u d e n t o f i n t e r n a t i o n a l p o l i t i c s d i s t i n g u i s h its effect o n f o r e i g n p o l i c y from the effects of the Soviet-American rivalry, the development of atomic

zational structure is d e c e n t r a l i z e d ; (3) the w o r k e r s ' o u t p u t is o n c e a g a i n tabulated; a n d ( 4 ) t h e c o n c l u s i o n i s r e a c h e d . T h i s k i n d o f d e s i g n o p e r a t e s without a control group. A s a r e s u l t , t h e r e a r e a n u m b e r o f a l t e r n a t i v e c a u s a l s e q u e n c e s t h a t m i g h t h a v e p r o d u c e d the s a m e result. F o r e x a m p l e , a plausible alternative explanation for the increased productivity m i g h t b e that the initial m e a s u r e m e n t o f p r o d u c t i o n , i n w h i c h e a c h w o r k e r k e p t track of output for a w e e k , focused the w o r k e r s ' attention on productivity in a w a y that had not been done before, leading them to improve their productivity. In other words, i t w a s n o t t h e d e c e n t r a l i z a t i o n o f t h e a g e n c y , b u t t h e s t u d y itself, w h i c h c a u s e d productivity to rise.
4

w e a p o n s , the l i b e r a t i o n o f f o r m e r c o l o n i e s , a n d s o o n , all o f w h i c h h a v e h a p p e n e d at the s a m e time? O n e cannot, of course. It is simply not possible to provide a control g r o u p of c o n t e m p o r a r y countries for w h i c h the U n i t e d N a t i o n s has not existed. T h e s a m e general alternative explanation (that other things h a p p e n i n g at the s a m e time could have been the true cause) also might have applied to "Agency Study." If something else had happened between the two measurements of

productivity—the weather improved, Christmas came, the president urged greater productivity, or whatever—this might have been the true cause of the increased production. Again, using a second agency as a control could eliminate such alternative e x p l a n a t i o n s . S t u d i e s w i t h o u t a c o n t r o l g r o u p p o p u p all t h e t i m e . O n J a n u a r y 4 , 2 0 0 4 , t h e Minneapolis Star Tribune h e a d l i n e d a f r o n t - p a g e s t o r y : " S t a t e S e x E d N o t W o r k i n g , Study Finds." T h e story reported a study by the M i n n e s o t a D e p a r t m e n t of Health, which in 2001 had asked students in grades 7 and 8 of three schools that h a d adopted an abstinence-only sex education curriculum (not teaching contraception or safe sex, but rather providing materials to encourage students to abstain), whether "at any time i n y o u r life h a v e y o u e v e r h a d s e x ( i n t e r c o u r s e ) ? " T h e y t h e n c a m e b a c k i n 2 0 0 2 a n d r e p e a t e d the q u e s t i o n for t h e s a m e g r o u p of students, n o w in g r a d e s 8 a n d 9. F r o m 2001 to 2 0 0 2 the n u m b e r saying that they had ever h a d sexual intercourse rose from 5.8 to 12.4 percent. T h e conclusion of the article w a s that the abstinence-only

H a d the study included a second agency as a control (see the next section), o n e in w h i c h o u t p u t w a s m e a s u r e d a t t h e s a m e t i m e s a s i n t h e first a g e n c y b u t i n w h i c h t h e r e w a s no decentralization, the alternative explanation w o u l d not have been plausible. T h a t is, i f t h e i n c r e a s e d p r o d u c t i v i t y i n " A g e n c y S t u d y " h a d b e e n d u e s i m p l y t o t h e act o f m e a s u r i n g , p r o d u c t i v i t y i n t h e c o n t r o l a g e n c y (in w h i c h t h e s a m e m e a s u r e m e n t s w e r e t a k e n a s i n t h e first a g e n c y ) a l s o s h o u l d h a v e i n c r e a s e d . A c c o r d i n g l y , i f w e f o u n d that productivity increased m o r e in the reorganized agency than in the control agency, w e w o u l d k n o w that this c o u l d not h a v e b e e n b e c a u s e o f t h e act o f m e a s u r i n g , for both agencies would have undergone the same measurements. That particular

alternative interpretation w o u l d h a v e b e e n eliminated by the design of the study. In conducting the study without a control, the alternative interpretation can be eliminated only by a s s u m i n g it away, w h i c h s e e m s very risky. "Reagan's victory over the Soviet U n i o n " provides an example of another sort of alternative explanation that m a y be plausible in studies without a control group. It is quite possible that other things that occurred b e t w e e n the m e a s u r e m e n t of the two events ( 1 9 8 0 - 1 9 8 8 and 1 9 8 9 - 1 9 9 1 ) w e r e the cause of the Soviet Union's collapse. Reform in C o m m u n i s t China, the growth of Muslim minorities in the Soviet Union, feuds among Soviet leaders—all might be proposed as

c u r r i c u l u m h a d failed. An abstinence-only c u r r i c u l u m m i g h t o r m i g h t n o t fail t o r e d u c e sexual

i n t e r c o u r s e , but we c a n n o t tell f r o m this s t u d y w h i c h is the c a s e . S t u d e n t s of that age, as they grow a year older, are in any case probably m o r e likely to have sexual intercourse than when they were younger. A control group of students of the same a g e w h o h a d h a d a different c u r r i c u l u m for sex e d u c a t i o n w o u l d h a v e b e e n easy to add to the study, but without it we have no idea w h e t h e r the students with the abstinence-only curriculum initiated sexual activity at a greater rate than they

alternative c a u s e s of the Soviet U n i o n ' s collapse. If it w e r e possible to find as a control g r o u p a similar system that w e n t through the s a m e other factors but was not engaged in an arms race with the United States, then these alternative no such

explanations system exists.

could be

tested

and perhaps

eliminated.

But of course,

would have done without the curriculum, at a lesser rate, or at the s a m e rate. We s i m p l y c a n n o t tell sexual intercourse. (The problem of interpreting results w a s also exacerbated in this case by the q u e s t i o n , w h i c h a s k e d s t u d e n t s w h e t h e r a t a n y t i m e i n t h e i r life t h e y h a d e v e r h a d s e x u a l i n t e r c o u r s e . T h e 5.8 p e r c e n t w h o a n s w e r e d affirmatively i n 2 0 0 1 m u s t h a v e a n s w e r e d affirmatively again in 2002, so the percent responding " y e s " in 2 0 0 2 whether the abstinence-only curriculum resulted in reduced

T h i s is a g o o d e x a m p l e of h o w difficult it m a y s o m e t i m e s be to i n c l u d e a control group in a design. S o m e circumstances just do not yield parallel cases that can be c o m p a r e d . As another example, consider that the existence of the United

A famous example of this sort is the Hawthorne study, in which an attempt was made to measure how much productivity increased when factories were made brighter and more pleasant. As it turned out, the groups of workers who were placed in better surroundings did show major increases in productivity. But so did control groups whose surroundings had not been improved. The novelty of taking part in an experiment, the attention paid to the workers, and increased social cohesiveness among those groups chosen for the experiment—all these raised productivity irrespective of the experimental changes in physical working conditions that were made for some (but not all) of the groups. See Roethlisberger and Dickson (1939).

4

c o u l d h a v e only risen or s t a y e d the s a m e ; it w a s n o t p o s s i b l e for the p e r c e n t a g e to decline. So, apparent failure of the p r o g r a m w a s baked into the study from the start. A better q u e s t i o n w o r d i n g , w h i c h w o u l d h a v e b e e n m o r e sensitive to the impact of the new program, w o u l d have been: " D u r i n g the past year, have you had sex [intercourse]?")

82

Chapter 6
a c c o m p l i s h e s this i s a

Causal Thinking and Design of Research 83 true experiment; but b e f o r e g o i n g o n to d i s c u s s t h i s , let m e
d i s c u s s a p o o r c o u s i n o f the n a t u r a l e x p e r i m e n t .

U s e of a Control G r o u p

The natural experiment.
encountered. It i s a

" O r g a n i z i n g the P o o r " i s an e x a m p l e o f a d e s i g n i n

w h i c h a control group h a s b e e n added to handle the sorts of p r o b l e m s we j u s t

The natural experiment without premeasurement. T h i s i s a d e s i g n i n w h i c h no
m e a s u r e m e n t s are m a d e before subjects are e x p o s e d to the independent variable. T h i s d e s i g n f o l l o w s the f o r m :

natural experiment,

a d e s i g n i n w h i c h a test group (that i s , a

group e x p o s e d to the independent v a r i a b l e ) a n d a control group (a group not e x p o s e d to the independent v a r i a b l e ) are u s e d , but in w h i c h the investigator has no control over w h o falls into the test group and who falls into the control group. In 1. Measure the dependent variable for subjects, some of whom have been exposed to the independent variable (the test group) and some of whom have not (the control group). 2. If the dependent variable differs between the groups, ascribe this to the effect of the independent variable.

" O r g a n i z i n g the Poor," the matter of w h o w a s contacted by the c a m p a i g n w o r k e r s w a s d e c i d e d b y the w o r k e r s ' o w n c h o i c e o f w h o m t o contact a n d b y the extent to w h i c h different people m a d e t h e m s e l v e s a v a i l a b l e for contact by the c a m p a i g n w o r k e r s . T h e natural e x p e r i m e n t d e s i g n i s o f the form:

T h e " T a x - R e f o r m M a i l " e x a m p l e i s o f this sort. I n this d e s i g n , (1) s e n a t o r s ' votes on a t a x - r e f o r m b i l l w e r e noted, a n d (2) the votes of senators w h o h a d

1. Measure the dependent variable for a specific population before it is exposed to the independent variable. 2. Wait until some among the population have been exposed to the independent variable. 3. Measure the dependent variable again. 4. If between measurings the group that was exposed (called the test group) has changed relative to the control group, ascribe this to the effect of the independent variable on the dependent variable.

r e c e i v e d f a v o r a b l e m a i l w e r e c o m p a r e d w i t h the votes o f those w h o h a d not. T h e s a m e k i n d of alternative e x p l a n a t i o n that has to be dealt w i t h in natural e x p e r i m e n t s has t o b e dealt w i t h i n this d e s i g n a l s o . A s i n " O r g a n i z i n g the Poor," i t m a y b e that h e a v i e r p r o - t a x - r e f o r m m a i l w e n t to senators w h o a l r e a d y w e r e m o v i n g into a taxr e f o r m p o s i t i o n e v e n w i t h o u t the m a i l . S u c h s e n a t o r s , about w h o m there m i g h t h a v e b e e n a great d e a l of s p e c u l a t i o n in the p r e s s , c o u l d h a v e attracted m o r e m a i l than did other senators.

T h u s , in " O r g a n i z i n g the Poor," (1) a n u m b e r of poor people w e r e s u r v e y e d as to their interest in p o l i t i c s ; (2) the c a m p a i g n o c c u r r e d ; (3) the s a m e poor people w e r e s u r v e y e d a g a i n ; and (4) those w h o h a d b e e n contacted by the c a m p a i g n w e r e c o m p a r e d w i t h those w h o h a d not. T h i s design e l i m i n a t e s m a n y o f the alternative explanations that c a n c r o p up in w o r k i n g without a control group. F o r i n s t a n c e , it c o u l d not have b e e n the initial m e a s u r e m e n t that c h a n g e d the group that h a d b e e n contacted, c o m p a r e d w i t h the control group, b e c a u s e both groups h a d b e e n m e a s u r e d in the s a m e w a y . N e v e r t h e l e s s , the n a t u r a l e x p e r i m e n t s t i l l a l l o w s alternative e x p l a n a t i o n s . B e c a u s e the r e s e a r c h e r d o e s not h a v e c o n t r o l o v e r w h o i s e x p o s e d t o the i n d e p e n d e n t v a r i a b l e , it m a y be that the e x p o s e d g r o u p h a s a different p r e d i s p o s i tion to c h a n g e than the c o n t r o l g r o u p . In " O r g a n i z i n g the P o o r , " for i n s t a n c e , the c a m p a i g n w o r k e r s are l i k e l y t o h a v e a p p r o a c h e d t h o s e p o o r w h o m they thought they c o u l d m o s t e a s i l y get i n t e r e s t e d i n p o l i t i c s . A l s o , t h o s e a m o n g the p o o r w h o w e r e m o s t r e s i s t a n t t o c h a n g e m i g h t not h a v e let the c a m p a i g n w o r k e r s i n the d o o r o r m i g h t h a v e b e e n c h r o n i c a l l y a b s e n t w h e n the c a m p a i g n w o r k e r s t r i e d t o c o n t a c t t h e m . A c c o r d i n g l y , a p l a u s i b l e alternative e x p l a n a t i o n i n " O r g a n i z i n g the P o o r " i s that the p o o r w h o w e r e c o n t a c t e d b y the c a m p a i g n i n c r e a s e d their interest i n p o l i t i c s m o r e than t h o s e w h o w e r e not c o n t a c t e d

Moreover,

this

design

permits

many

additional

alternative

explanations

b e y o n d those that a p p l y to a natural e x p e r i m e n t . In the t a x - r e f o r m e x a m p l e , it is p r o b a b l e that p e o p l e w e r e m o r e l i k e l y to w r i t e 1 ;tters f a v o r i n g r e f o r m to senators they thought w o u l d agree w i t h t h e m . I n other w o r d s , i t m i g h t b e that s e n a t o r s ' m a i l d i d not c a u s e their v o t e s , but that their l i k e l y vote c a u s e d t h e m to get c e r t a i n k i n d s of m a i l . H e n c e , the r e l a t i o n s h i p b e t w e e n a s e n a t o r ' s vote on the b i l l a n d the m a i l she r e c e i v e d m i g h t not be a c a u s a t i v e r e l a t i o n s h i p at a l l — m e r e l y a c o i n c i d e n c e b e t w e e n p r o - r e f o r m a n d p r o - m a i l a n d a n t i - r e f o r m a n d a n t i - m a i l . T h i s alternative c o u l d not a p p l y to a natural e x p e r i m e n t . In a natural e x p e r i m e n t it w o u l d h a v e b e e n c l e a r f r o m the i n i t i a l m e a s u r e m e n t w h e t h e r or not those w h o fell into the test group h a d i n i t i a l l y b e e n different f r o m those f a l l i n g into the c o n t r o l group. I n fact, w h a t i s c o m p a r e d i n the natural e x p e r i m e n t i s not the m e a s u r e d v a r i a b l e s t h e m s e l v e s , but h o w the two groups c h a n g e b e t w e e n m e a s u r e m e n t s . T o s u m u p , the n a t u r a l e x p e r i m e n t w i t h o u t p r e m e a s u r e m e n t i s a d e s i g n i n w h i c h the test group a n d the c o n t r o l group are c o m p a r e d w i t h r e s p e c t to a d e p e n d e n t v a r i a b l e o n l y after they h a v e b e e n e x p o s e d to the i n d e p e n d e n t v a r i a b l e . It i n v o l v e s the s a m e sorts o f a l t e r n a t i v e e x p l a n a t i o n s a s the n a t u r a l e x p e r i m e n t d o e s , p l u s s o m e others that r e s u l t f r o m the fact that the i n v e s t i g a t o r d o e s not k n o w w h a t the test g r o u p a n d the c o n t r o l g r o u p l o o k e d l i k e b e f o r e the w h o l e t h i n g started.

simply because they were the ones who showed more potential to become more interested in politics at that time, regardless of the campaign. T h i s alternative m u s t be either a s s u m e d
a w a y o r c o n t r o l l e d b y u s i n g a d e s i g n i n w h i c h the r e s e a r c h e r c a n d e t e r m i n e w h o

True Experiment In neither of the two v e r s i o n s of the natural e x p e r i m e n t j u s t outlined d i d the i n v e s t i gator h a v e any control over w h o fell into the test group a n d w h o fell into the control

f a l l s into the test g r o u p a n d w h o f a l l s into the c o n t r o l g r o u p . A d e s i g n that

l e a v i n g the other half as the control. M indicates that the d e p e n d e n t variable has been measured for t h e group. A d m i n i s t e r the i n d e p e n d e n t v a r i a b l e t o t h e test g r o u p . o n e o r t w o o f t h e a l t e r n a t i v e i n t e r p r e t a t i o n s left o p e n b y e a c h d e s i g n are h i g h l i g h t e d . This d e s i g n permits o n l y a very f e w alternative e x p l a n a t i o n s . "Presidential L o b b y i n g " is an e x a m p l e of the true e x p e r i m e n t . True e x p e r i m e n t Test group: R M * M Control: RM M Presidential Lobbying Natural e x p e r i ment Test g r o u p : M * M Control: M M Organizing t h e Poor Natural e x p e r i m e n t w i t h o u t premeasurement Test g r o u p : * M Control: M Tax-Reform Mail s o m e t h i n g e l s e that h a p p e n e d at the s a m e t i m e as * m a y h a v e c a u s e d the change. H e r e ( 1 ) the president c h o s e half of the H o u s e r a n d o m l y to be the test group. A f e w e x a m p l e s will indicate the broad u s e of the d e s i g n : ( 1 ) a n y v o t i n g s t u d y that s h o w s that p e r s o n s o f a c e r t a i n t y p e ( w o r k i n g c l a s s . Assign at random s o m e s u b j e c t s to t h e test g r o u p a n d s o m e to the c o n t r o l g r o u p . . Consult Cook and Campbell (1979). t h e p r e s i d e n t w o u l d b e pretty c e r t a i n that a disproportionately favorable c h a n g e a m o n g those he l o b b i e d w a s d u e to his efforts. f o r e x a m p l e . Type Graphic Presentation 3 Example from This Chapter Selected Alternative C a u s a l Interpretations Observation with no control group Test g r o u p : M * M Agency Study. male. (2) he t o o k a straw vote to m e a s u r e the d e p e n d e n t variable (vote) for both g r o u p s . they c a n set u p theoretically equivalent g r o u p s . cited at t h e e n d o f this chapter. w a y to do this is to assign subjects r a n d o m l y t o o n e g r o u p o r the other. I f t h e i n v e s t i g a t o r d o e s h a v e s u c h c o n t r o l .84 Chapter 6 T A B L E 6-1 Causal Selected Research Designs Thinking and Design of Research 85 g r o u p . or w h a t have y o u — t h e test g r o u p ) vote differently from t h o s e w h o are n o t o f t h i s t y p e ( t h e c o n t r o l g r o u p ) . Experimental and Quasiexperimental Designs for Research (Chicago: Rand McNally. T h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n m o r e likely t o c h a n g e than t h o s e w h o m a d e their w a y into t h e control g r o u p . a d i f f e r e n c e i n h o w t h e g r o u p s c h a n g e c a n n o t b e d u e t o t h e f a c t that t h e i n d i v i d u a l s i n t h e t e s t g r o u p w e r e m o r e p r o n e to c h a n g e in certain w a y s than w e r e t h o s e in the control group. or t h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n different from t h o s e in t h e c o n t r o l group even before they e x p e r i e n c e d *. (3) he l o b b i e d the test g r o u p . W o r k i n g w i t h t h i s d e s i g n . an asterisk (*) indicates that a group has been exposed to a stimulus or is distinguished in s o m e other way so as to constitute a "test group". 4 . ( T h e best. Table 6-1 s u m m a r i z e s the various research d e s i g n s d i s c u s s e d in this chapter and s o m e of the alternative explanations applicable in each case. N o n e of the alternatives d i s c u s s e d in this c h a p t e r a p p l i e s . s h e c a n p e r f o r m a true experiment. and s i m p l e s t . e d u c a t e d . M e a s u r e the d e p e n d e n t v a r i a b l e for both g r o u p s . Stanley. and R (used to describe the "true experiment") indicates that individuals have b e e n assigned randomly to the groups. 2 . M e a s u r e t h e d e p e n d e n t v a r i a b l e a g a i n for both g r o u p s . In presenting the designs graphically. T h u s the p r o b l e m that t h e i n v e s t i g a t o r s f a c e d i n " O r g a n i z i n g t h e P o o r " i s e l i m i n a t e d . 5. 1963). I n t h e last c o l u m n o f t h e t a b l e . Reagan's Victory T h e first m e a s u r e m e n t itself m a y h a v e c a u s e d the c h a n g e observed in the s e c o n d measurement. Campbell and Julian C. he w o u l d have been faced with plausible alternative causal interpretations. and (5) he c o m p a r e d the voting of the t w o groups to see whether his lobbying had m a d e a d i f f e r e n c e . 3 . a s c r i b e t h i s difference to t h e p r e s e n c e of t h e independent variable. A true e x p e r i m e n t i n c l u d e s t h e f o l l o w i n g s t e p s : 1. (4) the bill w a s v o t e d o n .) T h e a d v a n t a g e o f m a k i n g the g r o u p s e q u i v a l e n t i s that r e s e a r c h e r s c a n t h e r e b y e l i m i n a t e a l m o s t all o f t h e a l t e r n a t i v e c a u s a l i n t e r p r e t a t i o n s t h a t h a d t o b e a s s u m e d a w a y i n t h e v a r i o u s natural e x p e r i m e n t d e s i g n s . T h o s e w h o m a d e their w a y into t h e test g r o u p m a y h a v e b e e n m o r e likely t o c h a n g e than t h o s e w h o m a d e their w a y into t h e control group. If t h e test g r o u p h a s c h a n g e d b e t w e e n t h e first a n d s e c o n d m e a s u r e m e n t s in a w a y that is different from t h e c o n t r o l g r o u p . Each d e s i g n is p r e s e n t e d t h e r e i n a s y m b o l i c s h o r t h a n d that i s e x p l a i n e d i n t h e f o o t n o t e t o t h e t a b l e . ( 2 ) a n y o f t h e l a r g e n u m b e r o f s t u d i e s estimating h o w m u c h better i n c u m b e n t presidents do in s e e k i n g reelection w h e n the "Notation a d a p t e d from D o n a l d T. or B e c a u s e investigators can control w h o falls into w h i c h group. I f t h e g r o u p s are e q u i v a l e n t t o start w i t h . D E S I G N S FOR POLITICAL RESEARCH T h e natural e x p e r i m e n t w i t h o u t p r e m e a s u r e m e n t i s the s i n g l e m o s t c o m m o n l y u s e d d e s i g n in political research. If he had not b e e n able to control w h o w a s lobbied.

the k i n d of s y s t e m u s e d in U . w h i c h are difficult t o h a n d l e . but only if the experimenter can control significant policies and m a n i p u l a t e t h e m for the purposes of investigation." The distinction is even muddier when one compares several groups simultaneously. In o r d e r to u s e a n a t u r a l e x p e r i m e n t d e s i g n . ( 2 0 0 2 ) of t h e effects of a private education on poor children. T h e s e lend themselves readily to a natural experimental design. E v e n then. H o u s e elections and British a n d C a n a d i a n elections) will tend to p r o d u c e e c o n o m i c policies favorable to c o n s u m e r s . T h e latter requires only that the investigator be able to anticipate events. t h e i n v e s t i g a t o r m a y b e left w i t h a c o n t r o l g r o u p b u t n o test g r o u p . direct m a i l a p p e a l s h e l p e d slightly. such as one by R o g o w s k i and Kayser (2002). I n v e n t i n g h y p o t h e s e s of this sort is e n j o y a b l e . M a n y i m p o r t a n t v a r i a b l e s i n t h e s o c i a l s c i e n c e s a r e n o t a t all s u s c e p t i b l e t o s t u d y b y n a t u r a l e x p e r i m e n t s . It turned out that personal visits were effective in getting p e o p l e to vote. and so on. O t h e r designs that have the advantage of control groups are to be preferred because they r e q u i r e less difficult a s s u m p t i o n s . It is convenient and revealing to treat this sort of analysis in terms of the analogy to experiments. the experiment is so m u c h m o r e powerful than other d e s i g n s — rejects alternative interpretations so decisively—that we should use it whenever 6 p r e m e a s u r e m e n t if it (1) takes t w o or m o r e types of subjects and c o m p a r e s their values on a d e p e n d e n t variable. C o n n e c t i c u t . it m a y be that t h e test factor (especially if it is o n e that affects only a small part of the p o p u l a t i o n ) will a p p l y to o n l y a few of the p e o p l e i n c l u d e d i n t h e study. b u t t h e s e d e s i g n s c a n b e u s e d o n l y w h e n the r e s e a r c h e r h a s m o r e control over the test variables than m o s t political scientists can usually achieve. and (4) studies of discrimination. w h o s e p r o f e s s i o n a l s k e p t i c i s m i s q u i t e n a t u r a l . It m a y be that in a s m a l l o r g a n i z a t i o n . plurality s y s t e m s . w h o t e s t e d w h e t h e r w o m e n c a n d i d a t e s for office w e r e less w e l l financed than m e n b e c a u s e of experiential attributes such as seniority. S . A g o o d e x a m p l e of this sort of e x p e r i m e n t is Gerber and Green (2000). u n s t r e n u o u s l a b o r t h a t i s r a r e l y resisted. s u c h as a local political c a u c u s or a portion of a c a m p u s . also known as single-member district. w h i c h s h o w e d that majoritarian electoral systems (the test group. for instance. O n t h e o t h e r h a n d . where the dependent variable is compared among groups of subjects distinguished by their values on the independent variable. P r o g r a m s w e r e initiated in Dayton. T h e y a l s o m u s t g o t o the e x p e n s e in time and m o n e y of m a k i n g t w o m e a s u r e m e n t s of the subjects. any research is an example of the natural experiment without events." 5 possible. a n d t e l e p h o n e calls m a d e no difference at all. A c h e n (1986. and (2) infers that the difference on the d e p e n d e n t variable is the result of their difference on whatever it is that distinguishes t h e m as "types. and probably should use it a good deal m o r e than is now d o n e .86 Chapter 6 Causal Thinking and Design of Research 87 e c o n o m y h a s i m p r o v e d i n t h e p r e c e d i n g y e a r (test g r o u p ) c o m p a r e d w i t h t h o s e for w h o m the economy has not been improving (control group). especially the Introduction) argue persuasively that we in political science have overrated the problems and barriers to true experimentation in our field. for i n s t a n c e . or b e c a u s e of direct discrimination. However. It p e r m i t s relatively m a n y alternative causal interpretations. and so on. It is rare in real political situations that a researcher can exercise this kind of control over output" studies. "Presidential Lobbying. S o m e lie i n t h e past: p e o p l e ' s e x p e r i e n c e s d u r i n g the Depression. and the Netherlands) should strengthen the interests of producers. for e x a m p l e . W h e n t h e forces t h a t determ i n e t h e e x p e r i m e n t a l a n d c o n t r o l g r o u p s a r e u n k n o w n . points out that a p p a r e n t l y only r a n d o m i z a t i o n p r o v i d e s a c l e a r . but the true experiment requires that the researcher be able to m a n i p u l a t e t h o s e e v e n t s — s h e m u s t d e c i d e w h o i s t o fall i n t o t h e c o n t r o l g r o u p a n d w h o i s t o fall i n t o t h e e x p e r i m e n t a l g r o u p . It is apparent here and in the examples directly preceding this that I am taking some liberty with the notion of "control group. it is not at all clear which class is the "test" group and which is the "control. r e s e a r c h e r s m u s t be able t o a n t i c i p a t e the o c c u r r e n c e o f the test factor. T h a t is. p. Light. s o m e e v e n t s a r e m o r e e a s i l y anticipated: the introduction of a poverty p r o g r a m in a town. A true experiment requires even greater control over the subjects of the study t h a n d o e s a natural e x p e r i m e n t . t h e i m a g i n a t i o n h a s full p l a y t o c r e a t e alternative e x p l a n a t i o n s for t h e d a t a . But the logic of what is done here is the same as in the strict test/control situation. high school graduates' entrance into college. students might c o n d u c t a political c a m p a i g n a m o n g a r a n d o m l y selected portion of the campus community and c o m p a r e that portion over time with a r a n d o m l y selected control. are u n p r e d i c t a b l e ." Where rates of participation in the middle class and working class are compared. in which 30. 5 A n o t h e r interesting e x a m p l e of a natural situation that p r o d u c e d something a p p r o x i m a t i n g a true e x p e r i m e n t is t h e s t u d y by H o w e l l et al. Ohio: Kinder and Palfrey (1993. S u c h variables are difficult to W i t h o u t it. As we s a w in earlier s e c t i o n s .e n o u g h c a u s a l i n t e r p r e t a t i o n t o s e t t l e issues of social-scientific research conclusively: This really describes the bulk of political research. 6 . such as Uhlaner and S c h l o z m a n ( 1 9 8 6 ) . the colonial histories of nations. this d e s i g n is far f r o m satisfactory. T h u s it is no accident that the e x a m p l e I u s e d of a true experiment.000 registered voters in New Haven. e v e n t h e c l e v e r e s t statistical a n a l y s i s m e e t s s t r o n g r e s i s t a n c e f r o m o t h e r s c h o l a i s . this r e m a i n s the m o s t widely u s e d d e s i g n in political science. the educational backgrounds and past professions of m e m b e r s of C o n g r e s s . such as assassinations. and Mosteller (1975). In short. riots. regularly scheduled political events such as elections. a n d c h a n g e s in the b u s i n e s s c y c l e . Sweden. electoral systems like those of Germany. (3) most "policy fit i n t o a n a t u r a l e x p e r i m e n t d e s i g n ." w a s carried out by the president. while proportional representation systems (the control group. Others. For instance. Nevertheless. the investigator can carry out a true experimental design. the a t t e m p t m a y misfire. 6). citing Gilbert. w e r e e a c h r a n d o m l y assigned o n e of three different a p p r o a c h e s in a nonpartisan get-out-the-vote program. T o d o t h i s i n a f i e l d s i t u a t i o n r e q u i r e s p o w e r over p e o p l e in their n o r m a l lives. It is s o m e t i m e s possible to carry out true experiments on a larger scale than this. such as voters from several age groups.

she will score somewhat below her usual level. our city's murder rate declined by a full 18 percent"? What is often missed in such statements is that this effect might also be due to regression to the mean. When she is tested. people do not know whether they are in the experimental group or in the control. however. (In an ideal experiment.. But a disproportionate number of the high cases are probably cases for whom things had lined up unusually well the first time. the important thing about this is that if we observe cases at two points in time. For our purposes here. the random part of their measure is likely to have been positive. it is more likely to go down the next year than to go still higher. while it has many applications in general explanation." This could be true. even if the students had never been involved in team learning. like students taking tests. (One is given on p. in which the investigators tested experimentally the effects of variations in the format of TV news reporting on subjects' political perceptions. some of the high scores no doubt are genuinely high and may even rise by the next time they are observed. The two alternative causal interpretations can only be sorted out by using an appropriate research design. probably have some element of randomness to their measures. Note that this sort of random variation in a measure is closely related to what we referred to as unreliability in Chapter 4. Thus. We should therefore be a little suspicious of a statement.C. When do cities typically institute new policies? When the problem they are concerned about has flared up! But cities' problems. we can expect high initial values to drop somewhat from one time to the next. the results can be compelling. and there was a control group of children who applied for scholarships but did not receive them. inasmuch as the investigator generally can control all the relevant variables. her measured level of knowledge is generally about right. Consider a student in a course. the scores of the weaker students might have improved. We should expect that it is unlikely they would be so lucky twice in a row. The children who received scholarships knew they had won something special. This is a problem most public officials and journalists do not appear to understand at all. if the benefit were an artifact of children's knowing they had won in the lottery. How often have you seen a statement of the sort: "After foot patrols were introduced in the entertainment district. In such studies. One remaining alternative explanation is that something akin to the problem we saw in 'Agency Study" was operating. but it is well within your range of understanding at this point. on the average. If we assume that essentially everything we observe has some element of randomness to it—that is. and the scores of the stronger students might have declined. they knew that their parents were paying extra money beyond the partial scholarship to further their education. In other words. by the fact that it was only African Americans who appeared to benefit from the vouchers. has bad luck in the instructor's choice of questions. A good example of this sort of experiment is that of Iyengar and Kinder (1987). one would think that would affect all ethnic groups in the same way. but they are bad in that the strongest students appear to be dragged down by working with the weaker ones. But an alternative explanation could be that this is simply regression to the mean. of course. If the murder rate has shot up in one year. and New York City that offered children in public schools partial scholarship vouchers to attend private schools. Not all of them will do this. that is. for example. the two groups should have been equivalent in all respects other than the experimental treatment. A slightly more complex problem. 90) One variant of this problem shows up time and again in assessing the impact of new governmental policies. and therefore offers very convincing results on an important policy issue. true experimentation is useful for studying general aspects of small-group interactions or individual thought processes relevant to politics. true experiments are the most appropriate design. the New Haven study by Gerber and Green is an example. the scholarships were assigned randomly. No other ethnic group appeared to benefit in the same way. The investigator then manipulates the way individuals in the group may communicate with each other. This may have left them more highly motivated than the students who had not won in the lottery. The result of the study was that African American students who received the scholarships did significantly better on standardized tests two years later than African American students who did not receive the scholarships. some of the poorer students might have drifted up to their normal levels. Since there were more applicants than could be funded. children were chosen by lottery to receive the vouchers. to see how this influences the result. a group of subjects may be placed together and told to reach a decision on some question. a number of them had done unusually well and then just drifted back to their normal level of performance over time. In other words. whether or not the city institutes foot patrols in the entertainment district. governments are likely to pick a time when a problem is at a high point (which may or may not include some random element) Washington. When it is possible to construct true (or nearly true) experiments in real field situations like this. This selection process approximated a true experiment. in addition to its true core value it varies somewhat from time to time—regression to the mean will always be present. . Special Design Problem for Policy Analysis: Regression to the Mean I have introduced you above to some simple sources of alternative causal interpretations and some simple designs to control for them. Similarly. on a good day. and low initial values to rise somewhat.like: "Team learning strategies are good in that the weakest students improve. so on the average they are likely to go down the next time they are observed. and so on). This study comes very close to a true experiment. More often. we should see them rise.) This worry might be lessened. For example.88 Chapter 6 Causal Thinking and Design of Research 89 The problem is called regression to the mean. is especially important in evaluating the impact of policy initiatives. Also. Similarly. It might be that the stronger students' scores declined because when they were measured at the beginning of the experiment with team learning. but if she has had a bad day (isn't feeling good. D. the lowest scores probably include a disproportionate number of cases that were unusually low because of a negative random factor. she will score somewhat higher.

B u t it does m e a n that we must find a r e s e a r c h d e s i g n that a l l o w s us to d i s t i n g u i s h b e t w e e n the t w o alternative explanations. T h u s . 7 (B) T h e interrupted time series i s g r a p h i c a l l y illustrated i n F i g u r e 6 . unreliable. T h e d e s i g n in the statement about foot patrols and m u r d e r rates is our old friend: M * M (A) O u r p r o b l e m is that we suspect that the first M m a y have been u n u s u a l l y h i g h . w i t h the intervention indicated by an asterisk and a d a s h e d v e r t i c a l line. r e g r e s s i o n to the m e a n w o u l d have l e d us to think the intervention had had an i m p a c t . interrupted time series. in the last a n a l y s i s . USE OF VARIED DESIGNS A N D MEASURES 1 2 3 4 5 6 7 Year 8 9 10 11 12 13 14 I s u p p o s e that the easiest c o n c l u s i o n to draw f r o m our d i s c u s s i o n thus far is that any k i n d of r e s e a r c h in political s c i e n c e is difficult and that the results of p o l i t i c a l r e s e a r c h are. we cannot tell whether or not foot patrols r e a l l y c u r b e d the m u r d e r s . and try to a s s e s s j u s t h o w . B u t this w o u l d be far too sour. m a k i n g it l i k e l y that the s e c o n d m e a s u r e m e n t w o u l d s h o w a d e c r e a s e whether or not the intervention between the two m e a s u r e s has had any impact. where possible. Note that in graph B. A d e s i g n that a l l o w s us to sort this out is an AlZ. on the b a s i s of the information in the statement. I did not d i s c u s s these selected r e s e a r c h d e s i g n s to c o n v i n c e y o u not to do p o l i t i c a l r e s e a r c h but to s h o w y o u s o m e of the p r o b l e m s y o u must d e a l w i t h . the m e a s urements preceding it should tend to be lower than it i s . G r a p h B illustrates one in w h i c h it did not. y o u are better off if y o u set out y o u r d e s i g n f o r m a l l y and m e a s u r e those variables that c a n be m e a s u r e d . a 1 2 3 4 5 6 7 Year 8 9 10 11 12 13 14 series of m e a s u r e m e n t s over time that is interrupted by the p o l i c y intervention: MMMMM *MMMMM If the measurement that c a m e just before the intervention w a s unusually high. if we h a d l o o k e d only at the adjacent "intervention and after" m e a s u r e s .1 . Figure 6-1 Examples of Interrupted Time Series E v e n if y o u are f o r c e d to rely solely on one of the w e a k e r d e s i g n s (operating A n even better design. T h e test for whether the average of the several measurements preceding the intervention differs f r o m the average of the measurements intervention has had an effect in this design is whether the following the intervention. or u s i n g a control group w i t h no p r e m e a s u r e m e n t ) . b e c a u s e r e g r e s s i o n to the m e a n provides a p l a u s i b l e alternative explanation. G r a p h A illustrates an interrupted time series in w h i c h the intervention appears to have c a u s e d a c h a n g e . a c k n o w l e d g e alternative c a u s a l interpretations.90 Chapter 6 Causal Thinking and Design of Research 91 to initiate a p o l i c y to s o l v e the p r o b l e m . that i s . D o e s this m e a n that w e cannot a s s e s s the i m p a c t o f c h a n g e s i n p o l i c i e s ? O f c o u r s e not. would add a control to rule out the possibility that it was something else that happened at the same time as the intervention that caused the change: MMMMM * MMMMM MMMMM MMMMM 7 without a control group. in the c a s e of the murder rates.

whose test group we controlled. since willingness to compromise should in turn lead to effective government. a natural experiment were repeated in a different locality." In these. By combining the complementary strengths of varied measures and varied designs. The strengths of the various designs complement each other. in working solely with a randomized experiment in an isolated situation. he could be more confident that he had measured governmental effectiveness accurately. he first gathered a number of statistics about governmental performance in the 20 regions that he was studying. numerous phone calls. we might wonder whether our results simply reflect a test group comprised of unusual people. If we can mix research strategies in this way. (Governments' response quality varied from responses in under a week after a single letter. he traced the development of civic involvement in regions over the past century and showed that high civic involvement in the latter part of the nineteenth century led to civic involvement and governmental effectiveness a century later. or almost anybody. the end result is more than the sum of its parts. in part. The fact that such linking relationships proved to be true under testing makes us more confident that the overall relationship was not a design fluke. To the extent that he got the same results across a variety of methods of measurement. And we could be more confident of the result of the natural experiment if we had gotten a similar result in an artificial true experiment. The 20 regions were compared as to their level of civic engagement and the quality of their governments' performance. First. It will seem less likely to us that the result of the natural experiment is due to unusual people entering the test group. and either . the less political leaders feared compromise. If the campaign in one city tended to reach mainly those who were already becoming more interested in politics. To test the relationship more richly. The various weaknesses of different designs should suggest a solution: Wherever possible. Example of Varied Designs and Measures In his Making Democracy Work. To this end. rather than a "true" result. The likelihood that the alternative explanations associated with each design are true decreases because of the confirmation from the other design. in what is really a problem of valid measurement (see Chapter 4) rather than one of research design. This design still suffered.92 Chapter 6 Causal Thinking and Design of Research 93 likely it is that each alternative is true. CONCLUSION To pick up once again the refrain of this book. For example. This makes sense as a component of the relationship between civic engagement and governmental effectiveness. his theory was that traditions of civic cooperation. can take a research problem. and those with high engagement proved to be those with a high quality of governmental performance. there is no reason to think that a similar campaign would not do the same thing in a second city. by adding analyses over time of the form "observation with no control group. Relying totally on a natural experiment. all with their own varied sources of invalidity. But there is a more hopeful side to this chapter. he showed that the higher civic engagement was in communities. each result would bolster the other. Repeating the same design twice would increase our confidence somewhat. We could be more confident of the experimental result if we had gotten a similar result in a field situation. Getting the same result in two different cities would not make it any less likely that the alternative causal interpretation was true. His basic design was the natural experiment with no premeasurement. though. employing either the same or closely related measures. try to work simultane- ously with a variety of designs. Putnam countered this. But the alternative causal interpretations associated with natural experimental design would in no way be suspended. thus creating the illusion that the campaign had gotten them interested in politics. and a personal visit over several weeks. But if we could use both of these designs.) Finally. rather than a "true" result. The choice is between doing this or giving up and relying on your intuition and impressions. such as the promptness with which the regional governments developed their budget documents. Consider the "Organizing the Poor" example once again. say. if we obtain similar results in a true experiment. these objective measures of performance were supplemented by several opinion surveys in which people were asked how well they thought the regional government performed. Note that the effect would not be the same if a single research design were repeated twice—if. to responses that required numerous letters. Anybody. for example. He then checked and supplemented the official statistics by having research assistants call government agencies in each region with fictitious requests for help on such things as completing a claim for sickness benefits or applying for admission to vocational school. based on what he calls social capital (networks of social cooperation). are what lead to effective government. from the possibility that the relationship was a matter of other contemporary things associated with civic engagement. he approached the problem of measuring "government performance" by using multiple measures. carry out a fairly obvious test on it using one or two obvious measures. For instance. Putnam was faced with the question of research design. Putnam was thus able to generate conclusions of which the reader is more confident than if they were based on a single measure or on a single design. we may wonder whether the result we have obtained might be a result of something about the way the experiment was set up. Once he had decided how to measure governmental performance. such that those regions high in civic engagement were for some other reasons primed for governmental effectiveness. His study sought to explain variation in the quality of governmental institutions. which can at least in part cancel out each other's weaknesses. he also added tests of linking relationships using the same design. by showing us that the first result was not an accident or a result of the particular locality chosen. creativity and originality lie at the heart of elegant research. Robert Putnam (1993) uses the varied-design strategy rather well.

and p o o r countries in general have w o r s e h e a l t h a n d l o w e r l i f e e x p e c t a n c i e s t h a n c o u n t r i e s that are b e t t e r off.4 52.$ 5 .1 5. f o r w i t h i n e a c h o f t h e s e g r o u p s t h e third v a r i a b l e l i t e r a l l y i s c o n s t a n t . T h e number of countries falling into e a c h type appears in parentheses.3 73.3 (11) (39) (34) (30) (19) (23) (239) (395) Life Expectancy. M o r e s e r i o u s l y . D i c t a t o r s h i p s w e r e m u c h m o r e likely than d e m o c r a c i e s to be poor. l0 9 . and m a y e v e n vary the d e s i g n s and m e a s u r e s i n h i s o w n study. r e a c h i n g a s h i g h a s a n i n c r e a s e o f 5 . S t a t i s t i c a l t e c h n i q u e s s u c h a s m u l t i p l e r e g r e s s i o n artificially h o l d v a r i a b l e s constant. and their effect is a p p r o x i m a t e l y the s a m e as the direct t e c h n i q u e I u s e d here. on the average. T h e technique demonstrated here of h o l d i n g a variable constant—literally s e p a r a t i n g t h e s u b j e c t s i n t o n e w little g r o u p s a n d d o i n g t h e s a m e a n a l y s i s w i t h i n e a c h o f t h e s e g r o u p s — i s t h e s i m p l e s t t e c h n i q u e o n e c a n u s e t o e l i m i n a t e a third v a r i a b l e .001-4.3 70. it w o u l d result in a h o r r e n d o u s n u m b e r of tiny tables. H o w e v e r . 229. T h e reason the number of countries is so high is that the authors counted each country at several different points in its history. If t h e r e i s . a u s e f u l o n e . s u c h a s b y l o g i c a l a r g u m e n t o r b y indirect e v i d e n c e o f v a r i o u s sorts.7 3. but there are o t h e r w a y s t o e l i m i n a t e alternative interpretations. a n d that i s w h y people in democracies live longer? As indicated in Table 6-2. d e m o c r a c i e s h a v e s o m e I h a v e talked about h o l d i n g a variable constant m o r e than o n c e in this chapter.2 71. m a n y o f t h e s e w o u l d b e b a s e d o n rather s m a l l n u m b e r s o f c a s e s . A t t h e o t h e r e n d are o n l y 1 8 dictatorships w i t h per capita i n c o m e o v e r $ 6 .2 59. and an increase of 5. S u c h a student will muster logical arguments.0 H O L D I N G A VARIABLE C O N S T A N T e x p e c t a n c y i n b o t h d e m o c r a c i e s a n d d i c t a t o r s h i p s i s l o w . A g o o d e x a m p l e of h o l d i n g a variable constant is provided by the Przeworski e t al.001-5. but t h e d e m o c r a c i e s h a v e a n a d v a n t a g e o f e i g h t . (2000). some countries appear in the data set four or more times.001-6. 3 y e a r s . "Based on Przeworski et al. but it s h o u l d n o t be t h o u g h t of as t h e o n l y a n s w e r .000 3. First. A creative r e s e a r c h e r w i l l n o t b e s a t i s f i e d w i t h t h i s b u t w i l l try v e r y h a r d t o a c c o u n t f o r all plausible alternative interpretations. S e e Chapter 9 for an elementary discussion of multiple regression. The conclusion w e d r a w f r o m h o l d i n g p e r c a p i t a i n c o m e c o n s t a n t i s that although m o s t of the difference in life e x p e c t a n c i e s w a s due to the difference in h o w w e l l o f f d e m o c r a c i e s a n d d i c t a t o r s h i p s are.8 4. o r i s i t j u s t t h a t d e m o c r a c i e s a r e r i c h e r c o u n t r i e s . a n o b v i o u s a l t e r n a t i v e e x p l a n a t i o n o f f e r e d itself. w i l l cite e v i d e n c e f r o m related studies. 9 controlling for the variable.6 67. V a r y i n g d e s i g n s i s g e n e r a l l y u s e f u l .0 68. i t c a n n o t b e d u e t o v a r i a t i o n i n t h e third v a r i a b l e . A t all l e v e l s o f p r o s p e r i t y .000 2.94 Chapter 6 T A B L E 6-2 Per C a p i t a Income.2 56. than those in dictatorships—a big difference. D o i n g this w i t h the direct t e c h n i q u e w o u l d not be easy.t e n t h s o f a year.000 5.2 64. Difference 0.000 1. 2 y e a r s o f life e x p e c t a n c y for c o u n t r i e s with per capita i n c o m e of $ 4 . e a c h having a distinct value on the variable to be held constant.2 69. ( 2 0 0 0 ) s t u d y o f w h e t h e r p e o p l e l i v e b e t t e r i n d e m o c r a c i e s t h a n i n d i c t a t o r s h i p s . A l l o f t h e s e t e c h n i q u e s will h e l p the investigator limit the n u m b e r of alternative interpretations of his findings. interpreting a result a n d h a n d l i n g alternative interpretat i o n s o f t h e result are difficult a n d c h a l l e n g i n g c o n s t i t u e n t s o f research.3 63.2 2. and f r o m t h e s e w e c a n s e e w h y the difference w a s s o great b e f o r e w e c o n t r o l l e d for the countries' per capita i n c o m e s . there still r e m a i n s a n a d v a n t a g e for d e m o c r a c i e s . a s u b s t a n t i a l d i f f e r e n c e still e x i s t s b e t w e e n d e m o c r a c i e s and dictatorships e v e n w h e n per capita i n c o m e is taken into account.001-2. 1 0 T h e a d v a n t a g e o f s u c h s t a t i s t i c a l t e c h n i q u e s i s that t h e y c a n c o n v e n i e n t l y h o l d several variables c o n s t a n t s i m u l t a n e o u s l y . 0 0 0 . Democracies 47.000 4. 0 0 0 . compared T h e s i m p l e s t w a y to do this is to divide the subjects into separate g r o u p s . 0 0 0 . $ 0-1.4 3. T h e y f o u n d that t h e l i f e e x p e c t a n c y o f p e o p l e l i v i n g i n d e m o c r a c i e s w a s 6 9 .3 (215) (144) (59) (44) (25) (18) (18) (521) 95 ignore or a s s u m e a w a y the ever-present alternative causal interpretations. F o r c o u n t r i e s w i t h p e r c a p i t a i n c o m e s l e s s than $ 1 .6 16. Dictatorships 46.6 years for countries w i t h p e r c a p i t a i n c o m e g r e a t e r than $ 6 .000 6.1 4. T h e poorest group has 2 1 5 dictatorships. but 2 3 9 d e m o c r a c i e s ( m o s t l y f r o m North A m e r i c a and western Europe). D o t h e l o n g e r life e x p e c t a n c i e s in d e m o c r a c i e s indicate anything about w h a t d e m o c r a c y d o e s f o r p e o p l e . 0 0 1 .001-3.2 65.6 67. T h o s e i n d e m o c r a c i e s c o u l d e x p e c t to live 16 years longer. w e c a n h o l d t h e third v a r i a b l e c o n s t a n t t o s e e w h e t h e r t h e r e l a t i o n s h i p b e t w e e n t h e first t w o c o n t i n u e s t o e x i s t w h e n t h e third variable is not free to vary. 0 0 1 . p.6 53. Holding a variable constant is often also called a d v a n t a g e . if we divide countries into groups with approximately t h e s a m e p e r c a p i t a i n c o m e s . I think. a l t h o u g h i t i s n o t h i n g l i k e 1 6 y e a r s .001Total Causal Thinking and Design of Research Life Expectancy U n d e r Dictatorships and D e m o c r a c y Life Expectancy. l i f e 8 w i t h o n l y e l e v e n d e m o c r a c i e s ( c o u n t r i e s s u c h a s I n d i a ) . 3 y e a r s w h i l e that o f p e o p l e living i n dictatorships w a s j u s t 5 3 . T h e s u g g e s t i o n m a d e i n t h e p r e c e d i n g s e c t i o n — t o vary m e a s u r e s a n d d e s i g n s — i s . N o interpretation o f a r e s e a r c h result is cut a n d dried. If we w a n t to k n o w w h e t h e r a relationship b e t w e e n t w o variables c a n be a c c o u n t e d for by a third v a r i a b l e that i s r e l a t e d t o b o t h o f t h e m . a n d their m e a n i n g w o u l d thus be uncertain. and then observe whether within e a c h of these g r o u p s there is a relationship b e t w e e n the first t w o variables.

J a n u a r y So far in this book we h a v e l o o k e d at h o w y o u c a n develop a r e s e a r c h question. and to s h o w y o u s o m e b a s i c p r i n c i p l e s that w i l l help y o u to select c a s e s in w a y s that w i l l a l l o w y o u a c l e a n e x a m i n a t i o n o f y o u r r e s e a r c h question. A u t u m n 2 0 0 2 ) . developed in the face of quite limited research designs. Achen's The Statistical Analysis of Quasi-experiments (1986) explores these issues with great w i s d o m . is that we need to construct our research operations so that the relationships we observe a m o n g the variables we have m e a s u r e d mirror faithfully the theoretical relationships we are interested in. 4 . Tingsten (1937) provides an example of unusually careful and creative interpretations. y o u must select a set of observations to look at. Chapters 1 and 2 ) . and Donald T. for instance. G o o d e x a m p l e s are K i n d e r and P a l f r e y ( 1 9 9 3 ) . A more complete treatment of research design. Campbell Experimental and Quasi-Experimental Designs for Generalized Causal Inference ( 2 0 0 2 ) . To do all these things. is provided in T h o m a s D. a s p e c i a l i s s u e of the j o u r n a l Political Analysis ( v o l u m e 10. Cook. C h r i s t o p h e r H. n o . it is often the c a s e that the " s e l e c t i o n " o c c u r s by subtle p r o c e s s e s other than y o u r o w n c h o i c e . of American Behavioral Scientist ( v o l u m e 4 8 .) It w i l l not surprise y o u . I have relied h e a v i l y on their approach in this chapter. no. S h a d i s h . T h e literature on the use of true experiments in p o l i t i c a l s c i e n c e is burgeoning. In this chapter I want to alert y o u to the i m p o r t a n c e of c a s e s e l e c t i o n (whether it is done by y o u or by nature). a n d a s p e c i a l i s s u e . e m p i r i c a l political r e s e a r c h b e b a s e d a l m o s t e x c l u s i v e l y o n c a u s a l relationships rather than on relationships in g e n e r a l ? 1. then. A good treatment of the concept of " c a u s e " in relation to research design is found in B l a l o c k ( 1 9 6 4 . A central theme in this b o o k . C a m p b e l l ' s Quasi- experimentation: Design and Analysis Issues for Field Setting ( 1 9 7 9 ) . ( T h i s w a s the point of F i g u r e 4 . S e v e r a l interesting e x a m p l e s of i m p r o v i s e d research designs in field research on animal behavior c a n be found in Tinbergen ( 1 9 6 8 ) . and look for relationships a m o n g them to provide a n s w e r s to the r e s e a r c h question. C o o k and D o n a l d T. that case selection is j u d g e d by h o w w e l l it produces observed relationships that faithfully mirror the theoretical relationships we are trying to test. 5 0 . edited b y D o n a l d G r e e n a n d A l a n G e r b e r . 97 . Further. w h i c h y o u s a w e s p e c i a l l y in the two chapters on measurement and w h i c h y o u w i l l see again in later chapters.96 Chapter 6 FURTHER DISCUSSION I have presented here only a f e w examples of the m o r e c o m m o n l y u s e d research designs. and my analysis of t h e m has included only a sample of the possible alternative explanations for each design. O n e question that y o u might c o n s i d e r i n c o n n e c t i o n w i t h this chapter i s : W h y s h o u l d theory-oriented. m e a s u r e the variables i n v o l v e d . W i l l i a m R. w h i c h in 1936 predicted that F r a n k l i n R o o s e v e l t w o u l d l o s e the e l e c t i o n by a l a n d s l i d e on the basis of questionnaires sent to s u b s c r i b e r s to their m a g a z i n e . A further important factor u n d e r l i e s this w h o l e p r o c e s s . C o n s i d e r the f o l l o w i n g e x a m p l e s : • I d e s c r i b e d earlier the strange c a s e o f the Literary Digest. 2004). T h o m a s D. T h e c a s e s they h a d c h o s e n to l o o k at did not faithfully mirror the A m e r i c a n electorate. excellent and readable. or more recent and even m o r e complete (but not quite as readable). and those h a v i n g telephone s e r v i c e (not a representative group of A m e r i c a n voters in those d a y s ) (see pp. and y o u r selection of c a s e s c a n sharply affect or e v e n distort w h a t y o u w i l l find. Chapters 1 and 2 are easily a c c e s s i b l e to readers with limited technical preparation.5 1 ) .1 . however. car o w n e r s .

so their students are as good as any other for the test. We usually cannot study all possible observations to which our theory applies. His theory predicted that incumbent members' safety from electoral defeat. a department will consider dropping the exam because when one looks at the performance of graduate students in the department. Clearly. As noted earlier. • Most graduate departments in political science admit students to their programs based in part on each student's scores on the Graduate Record Examination (GRE). would be to set aside those cases in which incumbents have been most successful in generating electoral security. selection of a potentially biased sample did not result from a decision the researcher made. SAMPLING FROM A POPULATION OF POTENTIAL OBSERVATIONS Both the Literary Digest example and the discussion of using college students to generalize to all adults are examples of the problem of sampling. ask every American adult how he or she expects to vote in an election. 136) their safety. but at a decreasing rate. And. Therefore. So the available set of congressional races consisted only of those districts in which someone felt it made sense to challenge the incumbent. scholars use computer-generated random numbers to identify which members of the full population should join the sample for observation. however. In this case. they are "ethnic powderkegs" that had been held in check only by Soviet repression. In this case. at varying levels of seniority. because no such students are in the group we are able to observe. As a result. Researchers justify doing this by arguing that the processes they are studying are universal. as is sometimes done. Zaller stated his problem: To set [these races] aside. if he ignored the selection problem. and three of these were various parts of former Yugoslavia. • John Zaller (1998). (His approximated solution was to estimate from other factors the vote that the unchallenged incumbents would have gotten if they had been challenged.) In other words. Nature did it. that does not matter. the test is a nearly irrelevant predictor of success in the graduate program and should be dropped as a tool for assessment.98 Chapter 7 Selection of Observations for Study 99 • College students taking psychology tests are routinely used for experiments to measure psychological processes.) • The general understanding of the new democracies of eastern Europe has been that unless they are culturally homogeneous as Poland is. thereby understating the amount of electoral security that develops. From time to time. scholars' attraction to the dramatic cases led to a strangely "selected" body of scholarship. On the other hand. because the incumbent was so safe that no one wanted to take her on. and then to insert those simulated estimates as the observations for the unchallenged incumbents—not a perfect solution. of fourteen new states in Eastern Europe with significant ethnic minorities. admitted into the sample any case that got heads ten times in a row. but inheres in the situation. a number of seats were uncontested. this general impression has come about because almost all scholars who looked at ethnic conflict in new eastern European democracies were drawn to the dramatic outbreaks of violence in the former Yugoslavia. would increase with every term they served. he would understate the amount of safety that long-time incumbents accrue. (p. and are now prone to explode in ethnic violence. the "Cadillac" method is to draw a random sample from the full population of possible cases. For instance. However. in which the full range of possibilities did not get examined. However. he would be overstating . to regard victory in an uncontested race as evidence that the MC [member of congress] has captured 100 percent of the vote would probably exaggerate M C s ' actual level of support. In fact. In random sampling it is purely a matter of chance which cases from the full population end up in the sample for observation. the students admitted into the program are an atypical sample from the population. an experiment using college students might study the effect of drinking coffee on one's ability to memorize long strings of numbers. how they did on the GRE has little to do with how well they have done in the graduate program. as measured by their share of the vote. only four experienced violent ethnic conflict. but the one that made the best approximation. that is. We cannot. limitations of time and resources usually bar us from doing so. though it would be (barely) possible to do detailed analysis and fieldwork in each country of the world. this argument relies on data from those admitted to the program to generalize to the population of all students who might apply to the program. so even though their test subjects are not at all representative of the human population. The relationship they are looking for would be expected to be the same in any kind of group of people. for example. each member of the population has an equal chance of being drawn for the sample. It is as if we had flipped coins for each possible case and. As Mihaela Mihaelescu (2004) has pointed out. it is argued. the guiding rule in choosing observations for study is that the relationships we are looking for in the full universe of possible cases should be mirrored faithfully among the observations we are using. But if instead he treated the unchallenged incumbents as having gotten 100 percent of the vote. studying whether incumbent members of Congress had extra advantages in garnering votes. (In reality. they all did well enough on the GRE to get into the program. we usually work with a sample drawn from the universe of all possible observations to which a theory applies. had to work with another selection problem dealt by nature. Random Sampling When we are able to draw a fairly large number of observations. So he examined the vote margins of incumbents running for reelection. Would students who did badly on the GRE do well in the program? We cannot know. say.

For instance. and therefore would be twice as likely to end up in the sample. Typically. it would happen only rarely that you could have come up with a group among whom the women were more Republican than the men. then they must all be essentially the same as the full population. but it is increasingly difficult to draw a sample in this way that represents the broad population accurately. If you drew samples of a thousand Americans. The sample of college students. for example. one based in each of the localities chosen. people's use of their phones varies enough so that the group that can actually be reached by phone at a given time is problematic as a sample. which we looked at in Chapter 6. An alternative system is to draw a truly random sample of telephones by having a computer dial telephones randomly ("random digit dialing. Its sample of telephone owners. p. the average expression of the gender gap would almost surely mirror faithfully the gender gap in the full population. on the average the relationships we would see would be the same as the relationship in the full population. But in practice. Today almost everyone has a telephone. such as the population of a city. Some people are at work all day. population in a truly random way—sending an interviewer to interview a single person in Menominee. and then. In other words. of course. 61. For relatively small populations. surveys of American citizens are considered to have sufficient cases for accurately mirroring the population of 302. In randomized experiments. The larger the sample. so they cannot be expected to differ in any significant way that could interfere with a causal interpretation from the experiment. we can see from our consideration of random sampling how they might (or might not) be questionable as samples. Then. the method of choice. It always seems surprising that samples do not really need to be all that large to give an accurate rendition of a population. very much skewed toward the middle class. But you can imagine what it would cost to sample the U. Some families have two phone lines. Now. The principle here is the same as that in randomized experiments. How much variation there is from one random sample to another is hugely influenced by the size of the sample. the problem with telephone lists was that only the middle class had telephones. two groups are randomly chosen. In 1932. within each of these. another to interview a person in San Diego. If you took a large number of such samples. and averaged them. so their telephones will ring at any daytime hour but not be answered unless by a machine. while the true random sample is the Cadillac of sampling. we are assured that any relationship in which we are interested should be mirrored faithfully in the sample. so in principle RDD sampling could work well. randomization ensures that any two groups chosen are essentially alike. so that it operates almost exactly like a random sample. true random sampling is often feasible. Wisconsin.000 if they have questioned a sample of two or three thousand people. It is this mathematical system that allows pollsters to say of a national poll. the less variation there will be from one time to another when we draw samples and look at a relationship. And statistical adjustments can be made to take into account the clustered structure of the sample. that it shows support of 48 percent for George Bush. it must also be true when trying to generalize to a population from a sample.000. within an error of plus or minus 3 percentage points. however." or RDD sampling). for instance. each interviewer can readily reach the 20 or 30 subjects randomly selected from her cluster.100 Chapter 7 Selection of Observations for Study 101 If a sample has been drawn randomly. might provide a reasonable basis for drawing psychological conclusions about people in general.) One nice thing about random samples is that there is a very precisely worked out mathematical system to ascertain. This allows access to a random sample without the difficulty or expense of getting an interviewer to the person's door. The National Election Study at the University of Michigan uses what is called a "cluster sample": About 100 localities are first selected. But if that is true in experiments. instead of having interviewers fly or drive hundreds of miles for a single interview. by chance. although there is a "gender gap" in American voting. on the other hand. Quasi-random Sampling When a large population is involved. with men favoring the Republican party more than women do. and is then. and so on. and would have produced an erroneous prediction any time the middle class voted differently than the population as a whole. The Literary Digest poll was drawn before sampling was well understood. It is kind of amazing that just a few thousand out of hundreds of millions would be sufficient to estimate the population to within a couple of percentage points. and those appearing on state lists of registered automobiles was simply a terrible sample. Looking back now at the first two examples stated. This allows the survey to use approximately 100 interviewers. Others are always gone in the evenings. exactly how likely it is that the sample result is any given distance from what you would have seen in the population. If any group chosen randomly from the population can be expected to be essentially the same as any other. at least in the sense that. Since in fact men are more likely to be Republicans than women are. subscribers to the magazine. even though they are clearly not . that the ten people you drew for a sample would show the reverse of that. it would not be unusual for a sample of ten Americans to include a group of women who were more Republican than the men in the sample.S. most samples of ten Americans would show men to be the more Republican. (Review the discussion of the Law of Large Numbers. at least for large populations. We saw there that variation in anything we are observing becomes less variable as we use larger and larger samples to examine it. but it would still happen fairly often. The end result approximates a random sample closely. a certain number of individuals are randomly chosen for the sample. for a sample of any given size. various methods exist to draw "quasi-random" approximations to a random sample. at the time of the Literary Digest poll described earlier. Some telephones have devices to block unsolicited calls. Even random samples will diverge by accident from the full population. it is rarely used. RDD samples are rather problematic as approximations of a random sample of the population. across repeated samplings of this sort.

but that would. is fine. it makes sense to seek a couple of authoritarian states and a couple of nonauthoritarian ones for comparison. In principle and in the long run. rather than by randomly drawing the case(s). Another solution. 2 CENSORED DATA The last three of the five examples in the introduction to this chapter were examples of "censored data. 2 . a randomly selected sample of even just a couple of cases will mirror the full population. In the case of GRE scores. In this case. One solution would be to draw a monstrously large sample. too small a group for reliable comparisons in a sample of the usual size. an intensive study of the attitudes and ideologies of a few people. we might want to look at things in the data other than what we had anticipated when we were first drawing up the sample. for instance. A purposive sample works only for the particular independent variable or variables on which we have based the sample. an estimate of. rather than just taking four randomly chosen states. the ones you plan to study—are likely to diverge sharply from the full population. but because a deliberately constructed. rather than purposive. the Law of Large Numbers is working so weakly when we study only a few cases that the divergence of any case from the full population. random or quasi-random. In the preceding example. such as a national cluster sample) would probably not yield enough African American citizens for a good comparison. even if we are looking at only two or three cases. as it is most frequently seen. though it will be "only random. The problem of censored data is always severe. Since both African Americans and whites have been randomly (or quasi-randomly) selected. a study of some aspect of politics in one or a few countries. If we wanted to compare the political choices of African Americans and whites. estimated on the basis of other variables what vote unchallenged members could have been expected to receive 'We will see (pp. it is much cheaper than trying to draw a random sample of all people. any generalization about the importance of test scores is obviously meant to apply to all students who could apply for admission. Under these circumstances there is much more to be gained by intelligently choosing the case(s) to pick up the relationship of interest. is that usually when we draw a sample. because it requires researchers and their readers to make strong assumptions about what "might have been" in the range of data that are not available. Purposive Sampling Sometimes we deliberately want to sample in a nonrandom way. If African Americans were more likely to vote Democratic than whites. If it faithfully mirrored the population. But the one thing for which the sample is going to be used. not just for reasons of efficiency or cost. combining them to construct a sample that is half white and half African American. This is a situation that seems to fit Keynes' well-known comment that in the long run we shall all be dead. In principle. random selection of the case or cases would reflect the full population in the long run. the theory that ancient hatreds would flare up after Soviet repression was lifted from the region obviously was meant to apply to all eastern European democracies with large ethnic minorities. a purely random sample (or good approximation. 106-108) why we do not draw the sample to maximize variation in the dependent variable. It might be a study of a particular president's administrative style. And we should probably also anticipate that as we proceed with analysis. But in practice. Zaller. if the purpose of the study is to look at the politics of race. or. In the case of ethnic violence in eastern Europe. nonrandom sample serves a purpose for our research. 1 Selection of Cases for Case Studies Another kind of purposive sampling comes into play when we are dealing with such small numbers of cases that random sampling would not be very helpful in any case. the sample would give an unreasonably high estimate of the Democratic vote. So it is usually safer and more useful to draw up a more general sample. If we want to study the effect of authoritarian government on economic growth. estimating a relationship or set of relationships based on race. this would not work as a general reflection of the full population. Sometimes there are reasonable ways to fill in the spaces. Rather. it draws subjects to maximize variation in the independent variable of interest. either by the researcher's choice or by circumstances. and can serve as a satisfactory quasi-random sample if processes are indeed the same as for the full population. One reason that most sampling is random or quasi-random.102 Chapter 7 Selection of Observations for Study 103 a random sample of the full population if we are confident that their psychological processes are the same as psychological processes among the full population. for instance. we could sample African Americans randomly and sample whites randomly." will still be huge. A purposive sample does not attempt to replicate the full population. John Zaller's theory of congressional elections was obviously meant to apply to all members of Congress. of course. say. but he could observe election outcomes only for those members who were challenged by an opponent. But any given couple of cases—for instance. but the researcher can see the performance only of students who were admitted to the program. A case study is an intensive study of one or a few cases. be very expensive. is to draw what is called a "purposive" sample. for instance. Like all purposive samples. so that the relationships we are looking for will be very clear. the news-watching habits of the two races should reflect accurately the difference in news-watching habits in the full population. rather than one focused just on the single task we start with. even though the sample is not random. for instance. it should have about one-eighth of the sample African American. but scholars had chosen to look only at the dramatic cases in which conflict had actually occured. it is intended to serve multiple purposes." instances in which part of tue range of cases to which a theory applies are cut off and unavailable to the researcher.

Scholars of the presidency are likely to look at the administrations of dramatically successful presidents such as Franklin Roosevelt or Ronald Reagan. Mihaelescu was able to do a useful and rewarding study.5 3.5 3. Looking at the full range of labor repression and the full range of economic outcomes. As Geddes points out. . based on case studies of rapidly developing countries. »Taiwan South Korea •Brazil 1. if we had all of the developing countries of the 1970s before us.5 5. And sometimes the answer is easy: If investigators have ignored some of the range but can study it.0 0. which have grown rapidly.5 4. indicating little -2 -4 0.) Source: Geddes (2003). 'Taiwan South Korea •Brazil Growth in G D P per capita c d Often. and the "Asian Tigers": Singapore. Figure 7-2 Labor Repression and Growth in the Full Universe of Developing Countries. there is nothing like the kind of relationship that appeared from the censored data in Figure 7-1. 103 10 •Singapore 8 6 4 Growth in G D P per capita 2 0 -2 -4 0. however.0 1. p. There may be a slight relationship.0 0.0 Labor R e p r e s s i o n . She points out that a strong argument arose in the 1980s. and that one should instead select to maximize variation in the independent variable or variables. Scholars of economic growth are prone to look at countries such as Korea. the upper left-hand part of the figure is a little less thickly filled with cases. Most case studies of economic growth at the time focused on Mexico. that just makes a neat and interesting study for another investigator to conduct.0 2. filling in for their short-sightedness. or Brazil.5 •• Mexico* . that it was important for developing countries' governments to repress organized labor to let industry develop rapidly.0 4. But overall.0 Labor R e p r e s s i o n Figure 7-1 Labor Repression and Growth in the Most Frequently Studied Cases.5 4. and then placed the five frequently studied countries on a graph of labor repression. South Korea. related to growth in per capita GDP from 1970 to 1981. and the lowest growth (Mexico's) also happens to coincide with a lower repression of labor. As Geddes showed in work adapted here in Figure 7-2.0 3. There is very little pattern in the figure. 1970-1981. the focus on these several cases of rapid development led to a very wrong-headed conclusion. Scholars of revolution are prone to look at the Soviet Union or Cuba. They then look at those cases to see what caused the thing in which they are interested. Taiwan. indicating that there were somewhat fewer low-repression countries with high growth.5 2. just picking up on what others had not noticed: that most eastern European states with ethnic minorities had not exploded when the Soviets left. Brazil. Geddes developed a measure for how much labor organization was repressed by governments.0 2. Barbara Geddes (2003) gives a nice example of this. reproduced in Figure 7-1. as in this example of ethnic conflict in eastern Europe. 101 relationship between the two variables.5 2. When Scholars Pick the Cases They're Interested In a •Singapore - 6 4 2 0 Mexico* . we find about as many low-growth countries with low levels of labor repression as with high labor repression. As you can see in her graph. (GDP per Capital from Penn World Tables. p. all five cases fall in the upper -right-hand part of the graph (b: high labor repression.5 1 1 1 1. and inserted those estimates into his study as imputed data. rather than less showy presidents such as Calvin Coolidge. we would see that there was little or no advantage for the countries that repressed labor. 1970-1981.0 3. I will argue later that choosing cases to maximize variation in the dependent variable distorts relationships. But another result of choosing cases that exhibit most strongly the thing we are interested in is that it often also results in a censored data set.) Adapted from Geddes (2003). (GDP per Capita from Penn World Tables. investigators doing case studies gravitate to the outcomes in which they are interested. rapid economic growth).0 4.5 5.104 Chapter 7 10 Q O Selection of Observations for Study 105 b had they been challenged. and Taiwan.0 1. and pick cases for which the outcome was strong or dramatic.

and the other a purposive sample to maximize variation in support or opposition to the mayor. the problem is easy to solve. there are no easy solutions. or Zaller's work on congressional elections. And it's a nice challenge. the relationship you observe will mirror nicely the true relationship. All one has to do in this case is to seek out a broader range of information. find the GRE scores of students who attended there. they are probably unusually high achievers in other ways. one a purposive sample to maximize variation in race. Zaller had to make a number of assumptions to predict what vote unchallenged members would have gotten had they been challenged. for example. One cannot under ordinary circumstances admit to graduate school a group of students who have done badly on the GRE test to see how they perform. we are forced to draw conclusions of what might have been had the data not been censored.000 890. and justify them. as in the example of GRE scores. Dependent Variables Numbers: Nonwhite White Percentages: Nonwhite White (A) Selection Along Independent Variable Support 86 171 262 Support 43% 85% Don't 114 29 138 Don't 57% 15% 200 200 (B) Selection Along Dependent Variable Numbers: Nonwhite White Percentages: Nonwhite White Support 39 161 200 Support 38% 54% Don't 64 136 200 Don't 62% 46% 103 297 . Figure 7-4(A) shows the results of drawing 200 nonwhites randomly from the pool of nonwhites shown in Figure 7-3. This means we must make some assumptions.000 310. In Figure 7-4 I have drawn two samples of 400 from this population. it would require the assumption that departments that did not require the GRE were like other departments in every other way. This would not be a perfect solution. While this Figure 7-4 Selection Along Independent. which is. since they would have been rejected by other departments but ended up being accepted into the non-GRE departments. Let us illustrate this with a hypothetical city with significant racial polarization. you will distort the true relationship.000 800. seek out those graduate departments that for whatever reasons do not use the GRE as a criterion. And it does not matter in this case whether you have a large or a small sample. This is what Mihaelescu or Geddes did. Similarly. unlikely. But if you maximize variation in the dependent variable. for drawing "what if?" conclusions. The missing cases are gone and are not going to reappear. for example. Taking percentages. If you maximize variation in the independent variable. and see how they did. however.000 Support 45% 80% Don't Support Mayor 110.000. despite the fact that assumptions are required. and similarly drawing 200 whites from the pool of whites. A very important rule. but that you should never select to maximize variation on the dependent variable. but only 45 percent of nonwhites support the mayor.106 Chapter 7 Numbers: Nonwhite Support Mayor White Selection of Observations for Study Percentages: Nonwhite White 107 When Nature Censors Data What can one do about censored data? If the data have been censored because of decisions of researchers. We might. we noted that you might usefully draw a purposive sample to get sufficient variation in the variables of interest. And it would ignore the fact that the low-GRE students admitted to those departments would be atypical as well. and bring it into the analysis. this would be better than ignoring the initial problem of censored data. In Figure 7-3 we see the breakdown of support for the mayor. SELECTION ALONG THE DEPENDENT VARIABLE: DON'T DO IT! When we looked at selection of samples.000 100% 100% 7-3 Race and Support for the Mayor: Hypothetical Polarized City true population figures.000 Don't 55% 20% 200. These are the 90.000 200. at least if you have enough cases so that the Law of Large Numbers can work for you. In such cases. But if the data have been censored by nature. of course. we find that 80 percent of whites support the mayor. is that this can work well when you select cases to maximize variation on the independent variable.000 Figure 1. And one cannot decree that all congressional incumbents will be challenged for reelection. Nonetheless. and can in fact offer a nice chance to another researcher to make a significant contribution. by race. A good deal of artfulness is required to analyze data that have been censored by nature. but a creative imagination will be rewarded by substantially improved estimation.

Or. Case-selection forms the foundation for all of your examination of evidence. Gerhard Loewenberg (1968) was struck by the fact that Germany by the late 1960s looked just the opposite of what we would have expected from theories of electoral systems. because the purpose of the analysis is to compare nonwhites' and whites' support for the mayor. It looks a little artificial—why would anyone do this in the first place? Actually. with 54 percent of whites supporting the mayor. compared with 38 percent of nonwhites. offers a good tangible example of about how much variability one gets with random sampling of this size. you might choose two cases. with a widely variable estimate of nonwhites' support for the mayor. but it is wrong. and try to account for it. it replicates faithfully the relationship in the full population. I chose to introduce the idea through a large-scale survey example because the problem is easier to see there. 80 percent of whites support the mayor. doing something like this is the most obvious and intuitive thing to do. For instance. In fact. pp. as do 85 percent of whites (compared with 80 percent in the full population). In fact. In the full population. and see whether the things you expected to have been the causes were present. The numbers diverge by about the amount we would normally expect in drawing a random sample of 400 cases. changing the relative numbers of supporters and opponents from what one sees in the full population changes the percent of supporters in both racial groups. It had a proportional representation electoral system. It is intuitive. this should lead to a multiparty system if the country had multiple divisions in society (see the discussion of Duverger's theory. As we see. and since it is drawn along the independent variable. in ways that distort the percentages in both groups.) 3 intensive studies of one or a few cases (so-called case studies). the question of sampling from a larger population. (Note that sampling on the independent variable distorts the proportions of whites and nonwhites in the population. for the same reason we saw in the earlier example—namely. For such research. however. but the real importance of the argument is with regard to case studies. it is more accurate than what we would have gotten from a purely random sample of the population. and the question of whether to select cases on the basis of what you are trying to explain or on the basis of your independent variable. but it does not fiddle at all with the likelihood that the outcome occurs. if you properly wanted to avoid the problem of censored data. but sampling along the dependent variable does not? It makes sense. This does not present the same picture of sharp polarization. 2-3). drawing this sort of sample does not produce a faithful reflection of the relationship in the full population. and it will work well as long as your procedures fulfill the basic principle enunciated so far in this chapter: The relationship to be observed among the cases you have selected must mirror faithfully the true relationship among all potential cases. It does come up time and again. There is another kind of case study. this does not come up often with regard to large-scale sampling. in which you do not look at a relationship between variables across cases. It leaves unaffected the percentages in which we are interested. that it distorts the likelihood that the outcome occurs or does not occur. compared with 45 percent of nonwhites. and. it increases the number of nonwhites available for analysis. This has all been aimed at research in which we are comparing cases (anywhere from two cases to thousands) to look at the pattern of the relationship between independent and dependent variables. but the relationship is recognizably the same. It is so easy when studying a political outcome to choose a case in which the outcome occurred. but it may puzzle the reader a bit. the selection of cases affects profoundly what results you see. In the sample. Figure 7-4(B) shows the results of a purposive sample of the same size that maximizes variation in the dependent variable. and according to theory. with regard to This sample. it draws enough nonwhites to make fairly reliable comparisons with whites. SELECTION OF CASES FOR CASE STUDIES (AGAIN) The preceding example clarifies why one should not sample on the dependent variable. and so allows you to examine straightforwardly where the chips fall under varying circumstances. since that would have produced only a small number of nonwhites. Choosing instead cases that represent varying instances of your explanatory variable allows you to examine the full range over which your explanation is meant to apply. however. but instead look at a single case to help clarify or illuminate a theory. ANOTHER STRATEGY. and the logic there is just the same as the logic of the preceding example. But the sample maximizing variation in support appears to indicate a considerably weaker relationship. support for the mayor. But drawing groups of supporters and opponents of the mayor and then comparing whites' and nonwhites' support does not parallel the research question in the same way. see where the thing you're trying to explain has occurred. Drawing groups from the two and comparing them fits this very well. the problem of censored data. HOWEVER: SINGLE-CASE STUDIES SELECTED FORTHE RELATIONSHIP BETWEEN THE INDEPENDENT AND DEPENDENT VARIABLES So far in this chapter I have dealt with issues having to do with: the selection of cases. a relationship of sharp polarization. but that this is actually helpful. by the way. 43 percent of nonwhites support the mayor (compared with 45 percent in the full population). 3 . And operationally.108 Chapter 7 Selection of Observations for Study 109 sample is on the small side. one in which the outcome had occurred and another in which it had not. and see whether the things you expected to have been the causes were present. Why is it that sampling along the independent variable produces accurate estimates of the relationship.

Chapter 4. we s h o u l d p i c k the c a s e b e c a u s e of how the theory appears to be w o r k i n g . and V e r b a ( 1 9 9 4 ) . O u r task then is not j u s t to see whether the presence or absence of a test factor affects the value of the dependent variable. This argument is developed in Shively (2006). with the two largest (the Social Democrats and the Christian democrats) holding about 90 percent of the seats. 5 4 i n c o m e and h i g h s c h o o l dropouts $ 1 5 . " A n o t h e r good treatment of case selection is K i n g . he p i c k e d it b e c a u s e of the combination of the electoral s y s t e m and historic divisions in society (independent v a r i a b l e s ) and the party system (dependent 4 © Measuring Relationships for Interval D a t a v a r i a b l e ) . in relating education to i n c o m e . R e l a t i o n s h i p s v a r y in t w o w a y s : first. as indicated by the c o m b i n a t i o n of dependent and independent variables. 0 0 0 and the average i n c o m e o f h i g h s c h o o l dropouts m i g h t b e $ 3 0 . but instead. 0 0 0 . w h i l e h i g h s c h o o l dropouts a v e r a g e $ 1 5 . to ease the presentation there. 0 0 0 or $ 9 0 . in order to refine the theory and illuminate how it worked. T h a t i s . Rather. the test factor in field r e s e a r c h is frequently not something that is s i m p l y present or absent." T h e y have v a r y i n g amounts of education.110 Chapter 7 Chapter 8 G e r m a n y had demonstrated in the 1920s and 1930s that it did indeed have a richly divided society. the number of parties represented in the prewar G e r m a n R e i c h s t a g had been 32!) B u t by the late 1960s G e r m a n y had almost a two-party s y s t e m . S e c o n d . we w a n t e d to see whether the p r e s e n c e or absence FURTHER DISCUSSION of a test factor affected the v a l u e of the dependent variable. O r . 5 In C h a p t e r 6 we w e r e c o n c e r n e d generally w i t h " r e l a t i o n s h i p s " b e t w e e n independent and dependent v a r i a b l e s . W h e n a c a s e study is b e i n g u s e d in this w a y — n o t to test a theory by trying it out to see h o w w e l l it w o r k s . 0 0 0 . " S e l e c t i o n M o d e l s . Currently there are five parties represented. A n d . Rather. rather. for i n s t a n c e . ( 2 0 0 0 ) in an appendix. T h e average i n c o m e o f c o l l e g e graduates might b e $ 3 3 . e d u c a t i o n might h a v e o n l y a m i n o r effect o n i n c o m e . G e d d e s ( 2 0 0 3 ) has a very nice chapter. and the "big two" have 73 percent of the seats. 0 0 0 Since then the system has moved back somewhat toward multipartism. " r e l a t i o n s h i p " cannot be d i c h o t o m i z e d . J o h n G e r r i n g (2007) provides a comprehensive and intelligent review of case selection. He did not s i m p l y p i c k a c a s e w i t h a dramatic or interesting outcome. 0 0 0 . F i r s t . 0 0 0 . in h o w strongly the independent v a r i a b l e affects the dependent v a r i a b l e . this is a strategy for further refinement of the theory. for two reasons. but rather to develop the theory by l o o k i n g in detail at a c a s e in w h i c h the theory is w o r k i n g in particular w a y s — we s h o u l d not ignore the dependent variable in s e l e c t i n g our c a s e . it m i g h t h a v e a m a j o r effect. ( A t its highest. Three parties were represented in the parliament in the 1960s. yet there might still b e m u c h variation in i n c o m e s that c o u l d not be attributed to variation in p e o p l e ' s education. T w o v a r i a b l e s are not s i m p l y related or not related. C o l l e g e graduates might average $ 5 0 . treatment of c e n s o r e d data is presented in P r z e w o r s k i et a l . e v e n though the average i n c o m e of 111 . S o m e c o l l e g e graduates might m a k e only $ 1 0 . F o r i n s t a n c e . R a t h e r . B u t we s h o u l d also not select the c a s e j u s t in terms of the dependent variable. F o r e x a m p l e . we do not treat people s i m p l y as " e d u c a t e d " or "not educated. as noted in footnote 5 in that chapter. K e o h a n e . T h i s w a s too s i m p l e . though fairly technical. we s h o u l d then be c l e a r that we are not p r o v i d i n g a test of the theory. 0 0 0 a y e a r . Note that L o e w e n b e r g did not p i c k this c a s e j u s t in terms of the dependent variable. and our task is to see whether the amount of education a person has affects his or her i n c o m e ." A good. " H o w the C a s e s Y o u C h o o s e A f f e c t the A n s w e r s Y o u Get. H o w could this b e ? L o e w e n b e r g studied the G e r m a n case in depth to see what had produced this exception to the theory. C o l l e g e graduates might average $ 5 0 . to see whether and h o w the value of the independent variable affects the value of the dependent variable. although I treated it as a d i c h o t o m y in C h a p t e r 6. Relationships also v a r y in h o w completely the independent v a r i a b l e deter- m i n e s scores on the dependent variable. it takes on a variety of v a l u e s . 0 0 0 a y e a r and s o m e h i g h s c h o o l dropouts might m a k e $ 8 0 .

The tools we need to accomplish this task are found in the field of statistics. is to measure how strong a relationship exists between the independent variable(s) and the dependent variable. If we are using a nominal-scale variable.1 . as in Figure 8 . I demonstrate this in the next few sections. political research is not so much concerned with whether or not two variables are related but with whether or not they have a "strong" relationship (in one or both of the senses used earlier). and we often would guess incorrectly if we tried to predict a person's income solely on the basis of education." Their relationship may be such that the independent variable has a greater or lesser effect on the dependent variable. then. that is. Thus variables are not simply "related" or "not related. for example). Ordinal and nominal measurement techniques cannot tell us how great a difference in the dependent variable is produced by a given difference in the independent variable. WORKING WITH INTERVAL DATA Regression Analysis IMPORTANCE OF LEVELS OF MEASUREMENT In Chapter 5 we saw that we have more information about a relationship between variables if we work at a higher level of measurement than if we work at a lower level of measurement. with which we are concerned in this chapter). the president was not simply concerned with whether or not he was able to influence voting for the bill. The whole point of nomi lal and ordinal measurement is that in neither do we have available a unit by which to measure the difference between two values of a variable. Correlational measurement of one sort or another can apply to data measured at any level. Generally speaking. Each dot in the scattergram represents one observation (a person. Just as higher levels of measurement yield relatively richer information about a variable. placed on the graph according to its scores on the two variables. Recall that we mentioned two ways to measure the "strength" of a relationship between two variables: (1) by how great a difference the independent variable makes in the dependent variable. state. For instance. It should not be too surprising that methods of measuring A convenient way to summarize data on two interval-scale variables so that we can easily see what is going on in the relationship between them is to plot all the observations on a scattergram. This is what it means to say that education is not a very "complete" explanation of income. The name statistics derives from the Latin statisticus. If we are using an ordinal-scale variable. and it may be such that the independent variable determines the dependent variable more or less completely. In "Presidential Lobbying. This would indicate that although education affected incomes sharply. given varying scores on the independent variable." Statistics includes two main activities: statistical inference and statistical measurement (including the measurement of relationships. If we know more about a relationship. This means that we cannot measure how great a difference is induced in the dependent variable by a change in the independent variable. how complete an explanation of the dependent variable is provided by the independent variable. This can be seen in our examples of research design in Chapter 6. Statistical inference consists of estimating how likely it is that a particular result could be due to chance. The critical difference between working with interval-measured data and working with data measured at a lower level is that effect-descriptive measurement can apply only to interval-scale data. although this is precisely what is required to measure the relationship in an effect-descriptive way. how greatly values of the dependent variable differ. it is clear that what was of interest to the investigators in each case was finding out how strong a relationship existed. all we know is whether the two values are distinct. we should be able to measure a greater variety of things about it. but with how many votes he could swing. there are two major types of techniques: those appropriate for data measured at the interval and those suited for lower-level measurements. relationships between variables are different depending on the level at which the variables were measured. Although for the sake of simplicity these were presented as if we were interested only in whether or not a relationship existed. or (2) by how completely the independent variable determines the dependent variable." for instance. I introduce some statistical techniques for measuring the strength of relationships. it was relatively incomplete as an explanation of people's incomes. Our task in evaluating the results of research. Basically. we know whether one value is higher than another. that is. This becomes a particularly important distinction in political research. so techniques for measuring relationships at high levels of measurement give relatively richer information about those relationships. there must be other things affecting income in important ways.112 Chapter 8 Introduction to Statistics 113 the college graduates was higher than the average income of the dropouts. I discuss inference in Chapter 10. "of state affairs. STATISTICS Although modern political scientists have begun to use statistics extensively only in the past several decades. it tells us how reliable the results of our research are. or country. for statistics originally grew out of the need to keep records for the state. Because income still varied a good deal within each level of the independent variable. because under most circumstances. of course. effect-descriptive ways of measuring the strength of a relationship are more useful than correlational ways. dot A represents an observation that . it was actually political scientists of a sort who first developed the field. but we do not know how much higher it is. I shall call the first way of measuring a relationship effect-descriptive. and the second correlational. In this chapter and in Chapter 9.

the m a t h e m a t i c a l l y best procedure is to c h o o s e the unique line that m i n i m i z e s t h e s q u a r e d d i f f e r e n c e s b e t w e e n o b s e r v e d v a l u e s o f t h e d e p e n d e n t v a r i a b l e a n d its i d e a l i z e d v a l u e s a s g i v e n b y t h e s i m p l i f y i n g line.r i g h t a n d u p p e r . First. Graph C s h o w s a negative linear relationship in w h i c h the d e p e n d e n t variable d e c r e a s e s as the i n d e p e n d e n t variable increases. F o r all o f t h e s e r e a s o n s . w e c a n s a y t h a t i n t h e first graph the i n d e p e n d e n t variable c a u s e s greater shifts in the d e p e n d e n t variable than in t h e s e c o n d . t h e pattern i n t h e d o t s o f a s c a t t e r g r a m i s s u m m e d u p b y t h e s i n g l e l i n e that b e s t a p p r o x i m a t e s it. T h i s i s illustrated i n Figure 8-3. T h e scattergrams i n F i g u r e 8 . It is not u n c o m m o n in a research report to d i s c u s s 30 or 40 F i n a l l y a n d m o s t i m p o r t a n t .114 Chapter 8 (A) Introduction to Statistics (D) 115 Independent Variable Figure 8-2 Assorted Scattergrams separate relationships. c o m p a r i n g t w o scattergrams gives us o n l y an approximate idea of the differences b e t w e e n t w o relationJ I I I I I I L s h i p s . Graph B s h o w s a linear relationship in w h i c h the d e p e n d e n t variable i n c r e a s e s m o r e gradually than in the graph in F i g u r e 8 . F o r i n s t a n c e .1 a n d 8 . D o t B represents an observation with scores of 12 on the independent variable and 17 on the dependent variable.2 illustrate various other patterns w e m i g h t have observed.1 .9 5 ) . i n F i g u r e 8 . W e h a v e o b s e r v e d w h i c h w a y t h e d e p e n d e n t v a r i a b l e m o v e s w i t h c h a n g e s i n t h e i n d e p e n d e n t v a r i a b l e . By l o o k i n g at the pattern f o r m e d by the d o t s . It has b e e n drawn to m i n i m i z e the squared differences b e t w e e n e a c h of the observed p o i n t s . Furthermore. b u t w e c a n n o t s a y p r e c i s e l y h o w m u c h g r e a t e r t h e s h i f t s are. a n d t h e p o i n t B a t w h i c h a c o u n t r y h a v i n g A's s c o r e o n t h e i n d e p e n d e n t v a r i a b l e w o u l d b e e x p e c t e d t o fall o n t h e i d e a l i z e d s i m p l i f y i n g l i n e . c o m b i n e s scores of 3. 7 1 ) . and l o w scores on the dependent variable tend to c o i n c i d e with l o w s c o r e s o n t h e i n d e p e n d e n t v a r i a b l e . A single s u m m a r i z i n g line can be described m o r e e a s i l y than a pattern of dots. In particular. it can be u n w i e l d y to w o r k with. 9 4 . A l t h o u g h the scattergram tells us a g o o d deal about a relationship. W e h a v e d o n e t w o t h i n g s s o far. For a linear relationship. T h e s e are b o t h part of an effect-descriptive m e a s u r e m e n t of the relationship. W h a t is m o r e . I f t h e Figure 8-1 Scattergram differences w e r e m o r e subtle. w e n o t e that t h e r e are f e w d o t s i n t h e l o w e r . T h e measure c o m m o n l y u s e d to summarize the effect-descriptive characteristics of a Independent Variable s c a t t e r g r a m is t h e regression coefficient. C o m p a r i n g t h e g r a p h s i n F i g u r e s 8 . T o d o t hi s u s i n g s c a t t e r g r a m s m a y b e e x t r e m e l y c u m b e r s o m e . It w o u l d be painful to read such a paper if e a c h relationship w e r e presented in the f o r m of a scattergram. or if we w e r e c o m p a r i n g several relationships. a n d w e h a v e o b s e r v e d that i t m o v e s a t a s t e a d y rate a t a l l v a l u e s o f t h e i n d e p e n d e n t v a r i a b l e ( a l i n e a r r e l a t i o n ship) rather than at c h a n g i n g rates (a n o n l i n e a r r e l a t i o n s h i p ) . T h i s m e a n s that h i g h s c o r e s o n t h e d e p e n d e n t v a r i a b l e t e n d t o c o i n c i d e w i t h h i g h s c o r e s o n t h e independent variable. T h e s i m p l i f y i n g l i n e m a y be t h o u g h t of as a rule for predicting s c o r e s on the d e p e n d e n t v a r i a b l e f r o m s c o r e s o n t h e i n d e p e n d e n t v a r i a b l e s . w e o f t e n m e a s u r e t h e strength o f a r e l a t i o n s h i p b e t w e e n t w o v a r i a b l e s w h i l e h o l d i n g a third v a r i a b l e c o n s t a n t ( s e e t h e d i s c u s s i o n o f this t o p i c o n p p . Graph A s h o w s a nonlinear relationship (the dependent variable increases faster w i t h increases in the i n d e p e n d e n t variable if the i n d e p e n d e n t variable h a s a high value). i t i s u s e f u l t o d e v i s e a p r e c i s e n u m e r i c a l m e a s u r e t o s u m m a r i z e the relevant characteristics of a relationship s h o w n in a scattergram. w h e r e a simplifying line has b e e n drawn through a scattergram with observ a t i o n s o n s e v e n h y p o t h e t i c a l c o u n t r i e s t o s u m m a r i z e t h e pattern a c r o s s t h e c o u n t r i e s .1 . T h e l i n e a r r e g r e s s i o n c o e f f i c i e n t i s d e r i v e d i n t h e f o l l o w i n g w a y . the j o b would become well-nigh impossible. a straight line s u c h as this c a n be fully described by the equation \ y = a + bx . T h u s w e k n o w that t h e t w o v a r i a b l e s are p o s i t i v e l y related. this relationship appears to be a p p r o x i m a t e l y linear ( s e e the d i s c u s s i o n of "linear relationships" in the b o x on p.2 ( B ) . Its u s e f u l n e s s a s a predictor d e p e n d s on k e e p i n g the average squared v a l u e of "deviant" s c o r e s on the dependent variable as l o w as possible. Graph D s h o w s a pattern in w h i c h there is no relationship.l e f t c o r n e r s o f t h e g r a p h .5 on the independent variable and 5 on the d e p e n d e n t variable. we c a n tell a g o o d d e a l about the r e l a t i o n s h i p b e t w e e n t h e t w o v a r i a b l e s . s u c h a s A .

x is the value of the independent variable. but the shift of one unit in x in the real world will not be followed by a shift of b units in y. y decreases as x increases. you must add a (the predicted value of y when x equals zero) to b times the number of units by which x exceeds zero. But often we may be interested in comparing the effects of two or more independent variables on a particular dependent variable. where x equals zero (see Figure 8-4). if there is a regression coefficient of—10. For example. shows by how many units y increases as x increases one unit.5 + — (income in thousands of dollars) This means that with every additional thousand dollars of income. is called the intercept of the regression equation. The predicted value of y w h e n x is 4.5. V percent more potential voters vote.0105 with every mile of distance. to find the predicted value of the dependent variable for any specified Figure 8 .5? 2 Figure 8-3 T e Regression Line h where y is the predicted value of the dependent variable. that even though we work with neat. there is a negative relationship between the variables. If the regression coefficient for volume of trade.000. this poses no real difficulty. measured in millions of dollars. we might want to know which variable—nations' distance from the United States or the volume of their trade with the United States—has a greater impact on their diplomatic interaction with the United States.) In other words. the predicted percent voting equals 50. how can we tell whether it has a greater impact on diplomatic interaction than does geographic distance. the expected value of y when x equals zero.116 Chapter 8 Introduction to Statistics 117 value of the independent variable. is 6 + (4 x 3). the formulas will still give us an a and a b. or 65. or the slope of the regression equation. often simply called the regression coefficient. For example. that the relationship between income and electoral participation is linear and can be summarized by the regression equation percent voting = 50. the predicted percent voting equals 50.0105 for the same dependent variable if distance were measured in miles.5 for nations' diplomatic involvement with the United States (measured by the number or magnitude of exchanges per year) predicted from their distance from the United States measured in thousands of miles. If y causes x rather than vice versa. or 60. and a and b are numbers relating y to x. If we are working with just two variables. precise summary measure of how great an impact the independent variable has on the dependent variable.000 miles of distance from the United States.5.5 + (1/2 x 20). is the most valuable part of this equation for most purposes in the social sciences.000. impersonal numbers. there would be a regression coefficient of —0.4 The Regression Equation The equation of this line is y = 6 + 3x. plugging in our data and getting numbers out will not make the results valid. Independent Variable . however. with its slope of—10. that is. is +0. we do not escape the scholar's obligation to think. Thus the arguments made in Chapter 6 apply even when we work with simple numbers like these. Remember. That is.024. By telling how great a shift we can expect in the dependent variable if the independent variable shifts by one unit. it would be expected to decrease by 0. If income is $30. this can be difficult.5 + (1/2 x 30). (If b is negative. the slope provides a single. The problem of comparing units. if income is $20. if diplomatic involvement could be expected to decrease by 10. If we have guessed the direction of causation between x and y incorrectly.5 with every 1. Continuing with our example. or 1 8. If the two independent variables are measured in different sorts of units. Let's assume. The number a. The number b. for instance. it is the value of y where the regression line crosses the y axis. It may be seen from an examination of the concept slope (the number of units by which y changes with a change of one unit in x) that the slope has meaning only with regard to the units in which x and y are measured. for instance. The slope.

but will distort it. Checking for linearity. For instance. Because our theory anticipates a relationship between two variables. we measure the relationship between them by . It merely means that the relationship is more complex than you anticipated—and probably more interesting. to consider on what grounds your concepts can be related to each other. We must be careful to make certain in using linear regression analysis that the data do fit a more or less linear pattern. The best way to do this is simply to draw a scattergram (or rather. too. It usually is necessary to play around with alternative nonlinear equations for a while. is that the numbers do have meaning. The regression equation provides a handy summary description of the effect. To the extent that your research is based on a well-thought-out theory. The real answer. the regression equation will not summarize the pattern in the data. One important warning: Remember that the presentation I have made here is only a broad. the formulas for a and b are found in every standard statistics text. would be appropriate. the equation depicted in Figure 8-6 would be inappropriate because it must inevitably reverse direction at some point. You should always check your data before using linear regression analysis. that x has on y. if your theory predicts that the dependent variable will always increase with increases in the independent variable. of course. contrary to the old saying about "comparing apples and oranges. for there is an infinite variety of nonlinear equations that you might fit to any set of data. can be worked out readily enough. by measuring how much vitamin C each fruit contains. therefore. x 2 2 0 Examining the residuals. since that change is not the same at all values of x. but at a constantly diminishing rate (a "diminishing marginal returns" model). Many relationships in political science do turn out to be linear. now a bit more complicated. We can compare them in a nutritional theory. Or. we would have to develop in our theory dimensions along which they could be meaningfully compared. let the computer do it for you) and see whether it looks linear. Apples and oranges actually can be compared. It summarizes the pattern in the data more accurately. a nonlinear regression equation y . for instance. are required to express the impact that a change in x will have on y. Figure 8-5 shows a linear regression equation passed through the scattergram of a nonlinear relationship. We can see that y increases with x but decreases with the square of x. This regression line fits the data very badly and is not a useful summary of the relationship. The solution to the problem requires going back to theory development. Formulas are available to calculate equations for regression lines satisfying the least-squares criteria. But there are no set "formulas" for nonlinear regression equations. In this case. Figure 8-5 Introduction to Statistics 1 1 9 0 Figure 8-6 Nonlinear Regression Equation 2 Linear Regression on a Nonlinear Relationship In Figure 8-6. b\ and b . An equation such as y = a + b log x. Otherwise. then regression coefficients referring to them will also not be easy to compare. but only with relationship to their original concepts. two coefficients. This is particularly true for linear regression. Competence in using measures like those presented here requires more thorough training than is within the scope of this book. that is no reason to give up analyzing it. we could compare them in a theory of gravity by comparing their mass. To compare the effects of diplomatic interaction and geographic distance in the example above. as depicted in Figure 8-7. Any regression line is actually a reflection of the stage that we have reached in developing a theory. But these. the coefficients based on those units are not comparable either." but to do so we must first specify along what dimensions our theory calls for them to be compared. introductory overview. If the concepts are not easy to compare. A nonlinear regression equation may be found to fit the pattern fairly well. But if the relationship you are investigating turns out to be nonlinear.a + b x — b x has been passed through the scattergram from Figure 8-5. including the ones cited at the end of this chapter. this will help you to design an appropriate nonlinear equation.118 Chapter 8 The units—thousands of miles and millions of dollars—are not comparable.

that acts as an independent variable. Knopf. the regression line can serve a further important function. adapted f r o m V . K e y w a n t e d t o m e a s u r e t h e i m p a c t of factions in A l a b a m a p r i m a r i e s . L o o k i n g b a c k at Figure 8-3. so that it m i g h t n o t h a v e b e e n at all o b v i o u s that " d e m o c r a c y " w a s a variable you should use to explain levels of educational spending. a c a s e s u c h as A is o n e in w h i c h the effect of t h e " s o m e t h i n g e l s e " is to raise the v a l u e of t h e d e p e n d e n t variable. I n F i g u r e 8-3. ' s Southern Politics ( 1 9 5 0 . this may then suggest an additional variable that m a y be brought into our theory. C o n s i d e r Figure 8-3. But it is an expression of what we already h a v e b e e n thinking about the subject. we would expect from their value on the independent Something else. T h i s m e a n s t h a t t h e r e i s still u n e x p l a i n e d v a r i a t i o n i n the values of the dependent variable. S o m e of the observations are higher on the dependent variable than variable. so he related c o u n t i e s ' votes for F o l s o m . p. and so on. also affecting the dependent variable. by 100r pointing out important independent variables that had not yet occurred to us. c a l c u l a t i n g t h e r e g r e s s i o n e q u a t i o n for t h e line t h a t b e s t s u m m a r i z e s t h e p a t t e r n o f the relationship.120 Chapter 8 . beyond the independent variable. O . Special Primary. is and s o m e are lower. Notice also . 1 9 4 6 " s o m e t h i n g else. a progressive candidate in the 1946 gubernatorial primary. t h e actual value for c a s e A is h i g h e r than the v a l u e for B . 4 8 ) ." b e y o n d G N P . p . July 3 0 . B e c a u s e this m e a n t that counties tended to lean the same way in both elections. However. l o w e r i n g ) the d e p e n d e n t v a r i a b l e . T h e technique of e x a m i n i n g residuals is illustrated in Figure 8-8.a + b log x Introduction to Statistics 121 that it w o u l d h a v e b e e n difficult to identify the p r e s e n c e of a n o t h e r factor if you h a d n o t first r e g r e s s e d e d u c a t i o n a l s p e n d i n g o n G N P . Jr. In this sense. a progressive candidate in the 1946 senatorial primary. T h e r i c h e s t d e m o c r a c i e s i n F i g u r e 8-3 s h o w a h i g h e r rate of e d u c a t i o n a l s p e n d i n g t h a n do t h e p o o r e s t n o n d e m o c r a c i e s . N o w . In this w a y you might have discovered the identity of the Percent for Sparkman. He found a m o d erately strong relationship b e t w e e n the t w o . On e x a m i n i n g the residuals in that figure. we m a y notice that cases with similar residuals have s o m e additional characteristic in c o m m o n . or it m a y be nonlinear. o n c e the cases have been sorted out in this way. J r . you m i g h t notice that all o f t h e c o u n t r i e s for w h i c h e d u c a t i o n a l s p e n d i n g w a s h i g h e r t h a n p r e d i c t e d w e r e not d e m o c r a c i e s . a n d all o f t h e c o u n t r i e s for w h i c h i t w a s u n e x p e c t e d l y l o w were democracies. it indicated the presence of conservative 0 and progressive factions structuring the vote. regression analysis is an e x p r e s s i o n of w h a t we already think about the subject. for i n s t a n c e . Key. E x a m i n i n g t h e s e r e s i d u a l s p o i n t s o u t t o u s t h o s e c a s e s i n w h i c h t h e " s o m e t h i n g e l s e " h a s t h e effect o f r a i s i n g (or. It m a y surprise u s — w h e r e we expected a relationship there m a y be none. it can help refine our theory. O.. Southern Politics (New York: Alfred A. to their votes for Sparkman. This difference b e t w e e n the observed value a n d the predicted value is called the residual. Had such Figure 8-7 Model of Diminishing Marginal Returns Figure 8-8 Example of Residual Analysis Source: V. 1 950). c o n v e r s e l y . K e y . w h i c h w o u l d h a v e b e e n p r e d i c t e d from the r e g r e s s i o n line. as Key had expected. note that the regression line d o e s n o t p r o v i d e perfect p r e d i c t i o n of the v a l u e s on the d e p e n d e n t variable for the c a s e s i n t h e s c a t t e r g r a m . 4 8 .

This coefficient compares a set of data with ideal models of a perfect relationship and a perfect lack of relationship. and more likely to be disturbed by "other factors. But this tendency is weaker. By examining the residuals around the regression line. I suspect the reason most of us do not go on to examine this trove is that we have developed a proprietorial sense toward our theories before we ever get to the point of testing them. the regression coefficient accomplishes the first of these. in other words. A change in the independent Figure 8-9 Two Correlations variable tends to produce the same change in the dependent variable. the spread of dots around the regression line is treated as an act of God. In graph A of Figure 8-10. Key got some idea of what those variables might be. it is a trove in which new variables lie waiting to be discovered. This indicated the presence of other variables." Counties in Folsom's home part of the state voted for him more enthusiastically than would have been expected on the basis of their vote for Sparkman. depending on how closely the data approximate a perfect relationship. but the value of the dependent variable in graph B is less closely determined by the independent variable than in graph A. Users of regression analysis in political science far too rarely go on to the creative and exploratory labor of examining the residuals to see what additional variables affect the dependent variable. Many counties. Similarly. and assigns to the relationship a score ranging in absolute value from 0 to 1. the residuals tend to be larger in graph B than in graph A. one of the major forces retarding the development of stable statewide factions in Alabama politics at that time. But the relationship was not very tight. The two extreme models are illustrated in Figure 8-10.122 Chapter 8 (A) Introduction to Statistics (B) 123 factions not existed. which were causing additional variations in the vote. the correlation coefficient accomplishes the second. Both relationships can be summarized by the same regression line. the relationship in graph B is weaker than that in graph A. Counties in Sparkman's home part of the state voted less enthusiastically for Folsom than would have been expected from their vote for Sparkman (which presumably was high because he was a local boy). A regression line passed through them would leave no residual variation at Figure 8-10 Extreme Models of Correlation (A) (B) . in B than in A. there would have been no particular reason to expect a county to vote in much the same way in the two primaries. compared to "other factors" (the unknown things that cause the residuals to exist). the data all fall on a straight line through the scattergram. such as the one that gave 90 percent of its vote to Folsom but only 45 percent to Sparkman. The dependent variable is less a result of the independent variable. Correlation Analysis At the beginning of this chapter I pointed out that there are two ways to measure the strength of a relationship: by measuring how much difference the independent variable makes in the dependent variable and by measuring how completely the independent variable determines the dependent variable. In one sense." in graph B. In this case it turned out that the residuals could be explained in part by the effect of "friends and neighbors. The correlation coefficient. on the average. the "home" counties of Folsom's opponent went less heavily for Folsom than would have been expected on the basis of their vote for Sparkman. There is a certain completeness about one's own theory. On the contrary. Consider the graphs in Figure 8-9. Usually. in both graphs. or as a measure of the basic uncertainty of human affairs. Much of the looseness in the relationship between the votes for two candidates of the same faction was shown to be a result of people's tendency to vote for a candidate on the basis of where he came from in the state rather than the faction with which he was identified. measures how widely such a body of data spreads around a regression line. r. For interval-scale data. then. This pointed out to Key the importance of local solidarity. voted far differently from what one would have expected simply on the basis of conservative and progressive factions. and it does not occur to us to use our theory as a "mere" starting point in the search for explanations.

T h e c o r r e l a t i o n c o e f f i c i e n t f o r this t y p e h a s a n a b s o l u t e v a l u e o f 1 . a n d i n s p e c t i o n w i l l s u g g e s t w h a t i s . T h e l i n e y = y is a horizontal l i n e p a s s e d t h r o u g h t h e d a t a in t h e s c a t t e r g r a m .89 T h e f o r m u l a for the variance of any variable x is E(x variance-. It is the variance in s o m e t h i n g that p u z z l e s us and c h a l l e n g e s us to p r o d u c e an explanation. m a t h e m a t i c a l l y true: T h i s correlation coefficients o f . 3 4 9 = 6. T h e v a r i a n c e o f a variable i s t h e a v e r a g e s q u a r e d d e v i a t i o n o f v a l u e s o f that v a r i a b l e f r o m their o w n m e a n . b e c a u s e i t i s the a v e r a g e squared d e v i a t i o n of the v a l u e s of y f r o m their o w n m e a n . Fortunately. 2 Variance. and c o n s e q u e n t l y . = xf N w h e r e x is the m e a n of x and N is the n u m b e r of observations we w i s h to average. T h e variance of the dependent variable y can be depicted in a scattergram by d r a w i n g a h o r i z o n t a l l i n e at y = y. h o w e v e r . t h e h i g h e r its c o r r e l a t i o n coefficient will be in absolute value. i n fact. the stronger the relat i o n s h i p . W h y i s i t that s o m e p e o p l e m a k e m o r e m o n e y than other p e o p l e ? T h a t s o m e nations are f r e q u e n t l y i n v o l v e d i n w a r s a n d o t h e r s are n o t ? T h a t s o m e m e m b e r s o f C o n g r e s s v o t e f o r a b i l l a n d o t h e r s o p p o s e it? T h a t s o m e p e o p l e are m o r e p o l i t i c a l l y a l i e n a t e d than others? A l l o f t h e s e q u e s t i o n s s i m p l y ask: T o w h a t c a n the v a r i a n c e i n this v a r i a b l e be a t t r i b u t e d — w h y does this variable vary? C o m p a r i n g F i g u r e s 8-3 a n d 8-11. B u t i t i s n o t e a s y t o s e e w h a t t h e d i f f e r e n c e b e t w e e n t h e m m e a n s . we are to c a l c u l a t e is (x . T h u s the i n d e p e n d e n t v a r i a b l e d e t e r m i n e s t h e d e p e n d e n t v a r i a b l e c o m p l e t e l y . and thus the s u m of the squared deviations from the m e a n will equal zero. T h e more the v a l u e s vary a m o n g t h e m s e l v e s ..0 . T h e coefficient ranges from . A l t h o u g h it i s c l e a r e n o u g h w h a t all. o r + 1 m e a n .8 a n d r = 0 . I t i s not true. the c l o s e r a r e l a t i o n s h i p a p p r o a c h e s t h e s i t u a t i o n i n g r a p h A . 0 (a perfect negative relationship. a s i n F i g u r e 8-11. v a l u e s o f t h e d e p e n d e n t v a r i a b l e are c o m b i n e d r a n d o m l y w i t h v a l u e s o f t h e i n d e p e n d e n t v a r i a b l e . 0 ( a p e r f e c t p o s i t i v e relationship. a n d their v a r i a n c e is 1 'The mean of any variable is its arithmetic mean. or average: the sum of all the values divided by the number of cases.1 + 4 + 5)/3 = 8/3. there i s n o direct w a y o f interpreting w h a t c o e f f i c i e n t s b e t w e e n t h e s e v a l u e s m e a n . T h u s . b u t w e d o n o t k n o w how much s t r o n g e r o n e r e l a t i o n s h i p i s t h a n a n o t h e r . 2 y) + 3 2 (4 -y) + 3 2 (5 - 7) 3 2 3 ( . T h e a v e r a g e s q u a r e d v a l u e o f t h e r e s i d u a l s . 4. 4 a n d r = 0.1 . T h e c o r r e l a t i o n coefficient for this t y p e has a b s o l u t e v a l u e of 0. a n d 5 for a v a r i a b l e . I m u s t first i n t r o d u c e t h e concept o f variance. increases in the dependent variable produce increases in the independent variable. T h e s i g n s i m p l y m e a n s that for all t h e o b s e r v a t i o n s . o n t h e o t h e r h a n d . in the correlational s e n s e o f " s t r e n g t h o f r e l a t i o n s h i p . their m e a n is (. i f t h e r e are j u s t three c a s e s . B e f o r e w e c a n c o n s i d e r t h i s .xf and then add t h e s e results together. F o r i n s t a n c e . the further e a c h will b e f r o m the m e a n o f t h e m Interpreting the correlation coefficient. I t i s true t h a t t h e h i g h e r t h e a b s o l u t e v a l u e o f t h e c o e f f i c i e n t . In a positive relationship. . the greater the s u m o f s q u a r e d d e v i a t i o n s f r o m the m e a n . A n d it is a l s o n o t true that r = . T h e variance is a measure of h o w w i d e l y the observed values of a variable vary a m o n g t h e m s e l v e s . T h e least-squares line is a line p a s s e d through the scattergram in any direction s u c h that t h e s q u a r e d d e v i a t i o n s o f v a l u e s o f t h e d e p e n d e n t v a r i a b l e f r o m that l i n e are m i n i m i z e d . the variance w i l l b e zero. fall s o m e w h e r e b e t w e e n t h e s e t w o e x t r e m e s . T h u s there is no pattern to the relationship. x t h e o b s e r v a t i o n for t h e s e c o n d c a s e . T h i s indicates that t h e i n d e p e n d e n t v a r i a b l e h a s n o e f f e c t o n t h e d e p e n d e n t v a r i a b l e . s o that i f w e w i s h t o c o m p a r e t w o different r e l a t i o n s h i p s . M o s t r e l a t i o n s h i p s . b u t m o r e o f t e n j u s t r ) d o e s h a v e a u s a b l e i n t e r p r e t a t i o n a t all v a l u e s o f r . that t h e d i f f e r e n c e b e t w e e n r = 0. the m o r e that v a l u e s vary a m o n g t h e m s e l v e s . " I h a v e referred here o n l y to the absolute v a l u e of the correlation coefficient. such as t h e o n e i n g r a p h A o f F i g u r e 8-10) t h r o u g h 0 .. I f t h e y d o n o t vary a t all but e a c h h a s t h e s a m e v a l u e . is the v a r i a n c e of y."AO + ( / ) + (7 ) 3 3 2 4 2 2 3 121 = — A + "79 + 7 . o f c o u r s e . t h e c o r r e l a t i o n c o e f f i c i e n t i n d i c a t e s b y its s i g n w h e t h e r t h e r e l a t i o n s h i p i s p o s i t i v e o r negative. increases in the independent variable produce decreases in the d e p e n d e n t variable.2. f o r i n s t a n c e .1 . In addition to s h o w i n g h o w c l o s e l y a relationship approaches the situation in graph A. we s h o u l d s e e at l e a s t a s u p e r f i c i a l s i m i l a r i t y b e t w e e n the residuals around the least-squares line and the variance of the dependent variable. where x x first c a s e . w e c a n s a y w h i c h i s s t r o n g e r . in a negative relationship. T h i s i s true b e c a u s e e a c h v a l u e w i l l e q u a l the m e a n . T h e e x p r e s s i o n 2 (x (x x xf is equivalent to writing the observation for the - xf + 2 (x 2 - xf + . O n e w a y t o v i e w o u r g o a l i n t h e o r e t i c a l s o c i a l s c i e n c e r e s e a r c h i s t o n o t e that our task is u s u a l l y to a c c o u n t for the variance in a d e p e n d e n t variable. 6 is t h e s a m e as t h e d i f f e r e n c e b e t w e e n r = 0 . I n g r a p h B . a n d s o o n . the square of the correlation coefficient ( s o m e t i m e s c a l l e d the coefficient of determination. the h i g h e r their variance will be. w i t h s c o r e s o f . 0 ( g r a p h B ) t o + 1 . similar to the o n e in graph A but tilted up). + (x N - xf. 6 is t w i c e as s t r o n g a s r = -0. t h e c l o s e r i t a p p r o a c h e s t h e m o d e l i n g r a p h A o f F i g u r e 8-10. 0 . s o that a n y g i v e n v a l u e o f t h e dependent variable is as likely to coincide with a l o w value on the independent variable as w i t h a h i g h o n e . Thus the correlation coefficient provides a m e a s u r e by w h i c h the strengths of various relationships can be c o m p a r e d .1 . a n d d r a w i n g in the r e s i d u a l s f r o m this l i n e .3. T h i s i s r e m i n i s c e n t o f t h e d i f f e r e n c e b e t w e e n o r d i n a l a n d i n t e r v a l m e a s u r e m e n t : We k n o w that t h e h i g h e r the a b s o l u t e v a l u e of r.124 Chapter 8 (-i - Introduction to Statistics 125 all i n t h e d e p e n d e n t v a r i a b l e .

our 2 ! ! ! y=y i i i i 1 b e s t strategy i n t r y i n g t o p r e d i c t v a l u e s o f the d e p e n d e n t v a r i a b l e for p a r t i c u l a r c a s e s w o u l d b e t o g u e s s that the v a l u e i n a n y g i v e n c a s e i s the m e a n . therefore. line is the one horizontal line that m i n i m i z e s the squared deviations of values of the dependent variable f r o m itself (see B l a l o c k . A t least this is true if we think of "mistakes" as squared deviations from the true value. the u n e x p l a i n e d v a r i a n c e w i l l be zero.1 0 ( B ) . including the proof that r = 1 unexplained variance . D i v i d i n g the u n e x p l a i n e d variance by the total variance tells us w h a t proportion of the total v a r i a n c e is left after we have a l l o w e d the independent variable to explain as m u c h as it c a n e x p l a i n . as in F i g u r e 8 . but the correlation coefficient will be sharply lowered in B. " I s c o r r e l a t i o n o r r e g r e s s i o n a n a l y s i s the better w a y to m e a s u r e the strength of a r e l a t i o n s h i p ? " O b v i o u s l y . pp. To the extent that the dependent variable is determined by the independent variable. Under these circumstances. the proportion of the total v a r i a n c e in the dependent v a r i a b l e that c a n be y T T T 1 1 1 a s c r i b e d to the independent v a r i a b l e .126 Chapter 8 Introduction to Statistics 127 that i s . s o m e t i m e s the other. On the other h a n d . or 2 Correlation and Regression Compared O u r d i s c u s s i o n s o far l e a v e s u s w i t h the q u e s t i o n . p. this u n e x p l a i n e d v a r i a n c e w i l l be s m a l l c o m p a r e d to the total v a r i a n c e . If y o u read that an author h a s f o u n d a c o r r e l a tion o f .1 2 . the extent of its possible effects on the dependent variable are reduced. T h e effect is to r e d u c e the v a r i a b i l i t y of the i n d e p e n d e n t v a r i a b l e w h i l e l e a v i n g the b a s i c r e l a t i o n s h i p b e t w e e n the i n d e p e n d e n t a n d d e p e n d e n t v a r i a b l e s u n c h a n g e d . a n d 9 percent of the v a r i a n c e in one is due to the other.1 1 The Variance of y r m e a s u r e s the p r o p o r t i o n b y w h i c h w e h a v e r e d u c e d o u r m i s t a k e s i n p r e d i c t i n g 2 the d e p e n d e n t v a r i a b l e b y i n t r o d u c i n g k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e . gives us a useful interpretation of r at all v a l u e s . w h e r e a s the r e g r e s s i o n coefficient d o e s not. O p e r a t i n g w i t h o u t any k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e . If the dependent variable is unrelated to the independent variable. T h e e x p l a i n e d v a r i a n c e . If the dependent variable c a n be predicted perfectly f r o m the independent variable. 1979. As already noted. As it happens. indicating that the s a m e value of the dependent variable is predicted at all values of the i n d e pendent v a r i a b l e . r equals 1 m i n u s this proportion. T h a t i s .0 . y o u s h o u l d mentally square the correlation a n d interpret that statement: " T h e two v a r i a b l e s are negatively related. C o n s i d e r the t w o s c a t t e r g r a m s i n F i g u r e 8 . T h e r e f o r e . L e t us see w h y this should be so.1 0 ( A ) . . T h u s we have two v a r i a n c e s for the dependent variable: its v a r i a n c e around its o w n m e a n (the "total v a r i a n c e " ) and its v a r i a n c e around the r e g r e s s i o n line ("varia n c e left u n e x p l a i n e d by the independent v a r i a b l e " ) . T h u s the u n e x p l a i n e d v a r i a n c e w i l l equal the total v a r i a n c e . 5 8 ) . I f w e n o w add k n o w l e d g e o f the i n d e p e n d e n t v a r i a b l e .4 0 9 ) . the regression coefficient in B will be approximately the same as that in A. 4 0 5 . because the variability of the independent variable has been reduced in graph B. T h e m a g n i t u d e o f the m i s t a k e s i n e a c h c a s e i s n o w r e p r e s e n t e d b y s q u a r e d d e v i a t i o n s a r o u n d the p r e d i c t i o n s . o u r best g u e s s b e c o m e s the v a l u e 3 0 p r e d i c t e d f r o m the r e g r e s s i o n e q u a t i o n . 3 0 b e t w e e n two v a r i a b l e s . T h e v a l u e Figure 8 . T h e s c a t t e r g r a m f r o m g r a p h A h a s b e e n r e p r o d u c e d i n g r a p h B . we w o u l d expect the regression coefficient to be about the s a m e in both graphs. as in F i g u r e 8 . T h e regression line that m i n i m i z e d squared deviations in the full set of data in graph A should continue to m i n i m i z e squared deviations in the partial set of data in graph B. R e l a t i v e to other c a u s e s of 2 For a good presentation of this interpretation. any other horizontal line p a s s e d through the data w o u l d y i e l d a greater s u m of squared deviations in the dependent variable. " S o m e t i m e s one i s . e x c e p t that a l l o b s e r v a t i o n s for w h i c h the i n d e p e n d e n t v a r i a b l e is l e s s than 2 or greater than 4 h a v e b e e n e l i m i n a t e d . the a n s w e r m u s t be. the regression line w i l l be h o r i z o n t a l . T h i s similarity suggests that the squared deviations around the regression line m a y be treated as the variance of the dependent variable around the values predicted for it by the regression equation. the regression line w i l l equal the line y = y in this c a s e ." A n o t h e r h e l p f u l w a y to l o o k at t h i s interpretation is to t h i n k in terms of p r e d i c t i o n . W e w o u l d b e l e s s w r o n g m o r e o f the t i m e than w i t h a n y other g u e s s w e c o u l d m a k e . i n a s m u c h as the line y = y is the horizontal line that m i n i m i z e s squared deviations around itself. this is the variance in the dependent variable that is still left unaccounted for after the effect of the independent variable has been estimated." T h e k e y t o d e c i d i n g w h e n to u s e e a c h m e a s u r e lies in the fact that the correlation c o e f f i c i e n t reflects the v a r i a b i l i t y of the independent variable directly. : total variance ^ unexplained variance total v a r i a n c e 3 see Blalock (1979.

or states) are used to infer how variables are related among individuals living in those aggregate units. 8 0 t h o r t h e i n s t a n c e . t h i s ei n p r o p o r t i o n e c l i n e t o t a l h e i n t h e rw a t t r i b c o r r e l a t i o n b y t h e c o e f f i c i e n t s p a r t i c u l a r i n t e n c ' s t r a c t g e n d e r t h e i n b o t h o f o f t h e t h e s e c a s e s i s t h a t e d . o n r e s e a r c h e r o r r a c e s t a t e m e n t i m p a c t c i t y . t h e t o a m e a s u r e o f T h e t h a t w o u l d h a v e s e x u s c o n c l u d e 5 f r o m t h i s t h a t r a c e i s a m o r e i m p o r t a n t i m p o r t a n c e d e c l i n e . u t a b l et B u t ot h i n d e p e n d e n t s i m p l y e n d e t o n s a y tv a o f v a r i a b l e t h a t r i a b t h e l ed d e p e n d e n t v a r i a n c e o r d s . See Robinson (1950). a f f e c t e d v a r i a n c e t o i n d e p e n d e n t f r o m t h e s e w a n t s o f v a r i a b l e e d T h i s g e n e r a l l y F o r h a d d a t a w i t h b l a c k h i g h l y a c r o s s b e c o m e s t h e m a t t e r o f c o n s i d e r a b l e i m p o r t a n c e i s i n T h e b e y o n d k n o w i n g f i e l d r e s e a r c h . r e g a r d l e s s w a n t s o f a n d t o p e o p l e j u d g m e n t w o u l d t h e t r a c t a r e l o c a t e d h o w s t u d y . pp. ) r e g r e s s i o n a n a l y s i s a l w a y s h a n d l i n g a r e p a t t e r n s b e s u c h T h e r e c i r c u m s t a n c e s . a r e c e n s u s p e r c e n t t h e l a r g e c o r r e l a t i n g t h e p e r c e n t o f w i t h i n p e r c e n t w o u l d i n a l l T h e v o t i n g . P r o b l e m s s u c h a s t h i s a r e a r e s u l t o f c e n s o r e d d a t a ( s e e a b o v e . B u t i n t e r e s t e d v o t e d . o f I n t h e r a c e t h e c l e a r l y c e n s u s o r G R E s e x e x t r a p o l a t e t h e p a r t i c u l a r t o h o w a m a k e a c o n t r o l . n e w b a s i s i n i s d a t a t h e t h a t t o a I n v o t i n g . a n d v o t i n g p o s s i b i l i t y r e l a t i o n s h i p b e t w e e n r a c e r e c o r d f o r C o n g r e s s e s g e n e r a l . w h i l e t h e d a t a a c o e f f i c i e n t a t m o r e a b o u t i n t o o n h a n d . This can occur when data on aggregate units (such as percent black. t h e z e r o c a n m a l e u p a s b e v a r i a n c e d u e t o v a r y p e r c e n t O n t h e a s o f m a l e . a s h y p o t h e t i c a l h e n c e h a n d . and so on. 103-106) . t h a t 9 2 n d . v a r i o u s n g o u t c o t h e r e b i l l s . t o d e s c r i b e b u t h o w t o p e r c e n t s h o w c o u l d a m u c h p e r c e n t v o t e r b l a c k t u r n o u t . r e g r e s s i o n d e s p i t e s o m e t h e o r e t i c a l u s u a l l y w h i c h i n I w i l l r e s e a r c h b e a l m o s t t o a l w a y s i n t e r e s t e d a n a l y s i s . i n F o r w h i c h y o u m a y n o t i n t e n d m i g h t w o u l d t o w a n t b e g e n e r a l i z e . s o m e o n e I t m a j o r d e t e r m i n a n t T h e r e l i m i t e d u s e f u l n e s s a p a r t i c u l a r t h a t r a c e a n t f a i n a n d c t o f o a C o n g r e s s . i f v e r y F i g u r e 8 . o p e r a t e d . countries. t h e i s d e p a w h i c h a r e j u s t a s f r e e a s a t o o p e r a t e c a u s e o f s . g e n e r a l i z i n g . Stokes (1969) and Achen and Shively (1995). o t h e r v a r i a n c e w e r e i n d i f f i c u l t y r e s i d e n t i a l d o e s . i s i n w h i c h y o u y o u w i s h u s e i n t o e x t r a p o l a t e f r o m a l e s s e v e n l y A s o m e t h e f a c t t h a t t r a c t s t r a c t s . m o r e o n a t o r e s e a r c h e r w i t h c i t y . u n i t s . ( T h i s a d v i c e t h e r e i s s u p e r i o r c o r r e l a t i o n t h a t i n h o l d s f a c t .128 Chapter 8 Introduction to Statistics 1 2 9 1 0 0 c o > c o (1) 0 Percent Male 1 0 0 Figure 8-13 P r e d i c t i n g P e r c e n t V o t i n g f r o m P e r c e n t M a l e t h e d e p e n d e n t o f v a r i a b l e . m e s n e g l i g i b l e h e l p o w e v c o r r e l a t i o n w h a t o u l m e m i m h e t h e i r r s i n w o u l d H e s t a b l i s h e r . t h i s w This example also incorporates a major statistical problem in some correlation and regression analyses. the ecological fallacy. r e s e a r c h e r t h e m a k e G R E m a l e a b o u t t r e a t t h e a p p l i c a n t s o f t h e ( a m o n g w h o m v a r i a b i l i t y G R E s c o r e s s c o r e s v a r i a b i l i t y a d o i n d e p e n d e n t m i g h t a b e v a r i a b l e s i n v e s t i g a t o r ' s s e x u s e a n d g e n d e r i n s t a n c e . i n m o r e a n y s i t u a t i o n c a s e . i n that Congress. ) a b s e n t o t h e r s . m e n t i o n e d v a r y i n g e a r l i e r . a p a r t i c u l a r s a y t h e h o w e v e r . for census tracts.1 3 . v a r y d i s t r i b u t i o n w h i l e a l m o s t t h e m o s t s u c h t h a t p e r c e n t b e h i g h e r ) r e l a t i o n s h i p b e t w e e n g r a d e s a m o n g g r e a t l y a n d p e r c e n t i n w o u l d m e n ( B l a c k s s p r e a d a r e o r c o n c e n t r a t e d s t u d e n t s c u r r e n t l y g o o d s e t r u l e o f d e p a r t m e n t . b e c a u s e i n t h e t o t h e i n v e s t i g a t o r c a s e . 4 g e n e r a l s h o u l d r e g r e s s i o n c o e f f i c i e n t s .ri sr m u s t c a u s e p a r t i c i p a t i o n p r o b l e m w i t h i s t h a n u s i n g g e n d e r .I o f t h e no t a s t h e t h e y w e r e i n g r a p h v a r i a b l e t h a t i s u c A . w a s T h i s a t h e n a p p r o p r i a t e b e t w e e n w e r e d n o i n t d e t h e n y t n o t e b e r s ' p o r t p a r t i c u l a r v o t e s f l u e n o n c i C o n g r e s s . median income. i t m i g h t i s t h a t g e n d e r o n l y t o d e s c r i b e s i t u a t i o n . b l a c k p e o p l e m a l e w h e t h e r m i g h t v o t i n g i s w h e t h e r p e r s o n r e s e a r c h e r p e r c e n t c i t i e s n o t . m o r e p a r t i c u l a r t h a t t h i s p e r c e n t m a l e s c a r c e l y f i n d i n t h e v a r i e s f r o m n o o n e c e n s u s t r a c t t o a n o t h e r B e c a u s e g u a r a n t e e s m a l e t h e r e p e r c e n t t h a t w o u l d a n d i s r e s e a r c h e r A s w o u l d p r a c t i c a l l y c o r r e l a t i o n s c a t t e r g r a m l i t t l e o f b e t w e e n i n t h e p e r c e n t a n a l y s i s p e r c e n t n e a r v o t i n g v o t i n g . d a t a g e n e r a l t h e t h e s t u d y . i n d i c a t e d i n i t .

(Unfortunately. Bohrnstedt. standard text in statistics—one of many—for social scientists. Not only is it lost to us. George W. David Knoke. this article and most other discussions of measurement error are difficult for the untrained reader. 53). we wanted to relate voting turnout to education. A good example of this technique is seen in Bartels (1993). As progressively greater amounts of random measurement error are present in two variables. One question you might consider is this: What would happen in a regression analysis if the independent variable did not vary at all—if. but everyone in our study had the same amount of education? This question looks ahead to Chapter 10. Random measurement error also distorts the relationship between variables. but it is systematically distorted. we are simply unable to state accurately what the relationship between those variables is. To the extent that there is random error in two variables. it is possible to correct for it. The scattergrams for the three sets of data drawn from Table 4-1 are presented in Figure 8-14. Thus there is a real danger that the only relationships we will be able to perceive are those between variables that are easy to measure accurately. as it might appear under conditions of increasing levels of random measurement error. Figure 8-14 Relationship under Varying Degrees of R n o Measurement E a dm r Moderate Random Measurement Error No Random Measurement Error 5 10 15 Margin 20 25 5 10 15 Margin 20 25 Great Random Measurement Error 40 30 I 20 10 0 5 J 10 L 15 Margin 20 25 . a possibility that bodes ill for social science. they will appear to us to be unrelated. so that anyone wishing to put these techniques to use would be forced to seek out more detailed training.) F R H R DISCUSSION UTE In this chapter I have drawn only the broad outlines of correlation and regression analysis for interval data. The relationship between the two variables is increasingly attenuated from left to right. If we are willing to make some assumptions regarding the nature of the random error. and thus reconstruct the true relationship between two variables. I have purposely refrained from giving formulas for calculating these measures.130 Chapter 8 Introduction to Statistics 131 Problem of Measurement Error Our understanding of relationships between variables is intimately bound up with the extent to which we have been able to measure the variables validly. say. the true form of the relationship between the variables is more and more lost to us. To the extent that there is nonrandom error in our variables. Refer back to Table 4-1 (p. and Alissa Potter Mee's Statistics for Social Data Analysis (2002) is a good. In fact. This should be clear from our earlier discussion of measurement error in Chapter 4.

just as a scattergram does for interval-scale data. But this is not available under ordinal measurement. the table shows how frequently the various combinations of the two variables occur. National Election Study. imply an even spread across the table. and (3) those that involve several variables simultaneously. the most we can hope for from ordinal variables is some form of correlational analysis. This is an interesting question. in which voters' willingness to have the United States use military force to solve problems abroad is related to their willingness to have the United States give financial aid to countries 'This chapter presents advanced material that can be skipped over without any loss in comprehension of the remaining material covered in this book. And no relationship at all would. In general. The table reveals a pattern between these variables. Only for the second of these is it possible to develop a standard measure for use with ordinal-scale data. a positive relationship would mean that relatively few cases should fall in the upper left corner of the table (not many cases willing to intervene militarily but unwilling to send aid) or the lower right corner (not many cases willing to send aid but unwilling to intervene militarily). we can say that there is a modest positive relationship in the table. in which voters who favored the use of military force did not want to see more "soft" involvement. in economic crisis. A cross-tabulation is to the analysis of ordinal variables what a scattergram is to the analysis of interval variables. of the 82 voters who are extremely willing to give financial aid. it is necessary to have some sort of unit by which to measure that change.Chapter 9 T A B L E 9-1 Introduction to Statistics 133 Relationship Between Support for Military Intervention and Support for E c o n o m i c Aid Willingness to Have United States G i v e Financial Aid to Countries in Crisis Willingness to Have United States U s e Military Force to Solve Problems F u r t h e r Topics on Measurement of Relationships Not Willing 18 47 120 48 Somewhat Willing 66 152 355 119 Very Willing 33 75 112 38 Extremely Willing 29 14 34 5 Extremely w i l l i n g Very w i l l i n g Somewhat willing Not willing Source: 1998 Congressional Election Survey. not percentages. 1 MEASURES OF RELATIONSHIP FOR ORDINAL DATA As we saw at the beginning of Chapter 8. Or there might be a positive relationship if both questions were tapping an "internationalist" dimension such that those favoring one sort of involvement would favor other sorts as well. there are two main ways in which we can measure the strength of a relationship. Consider Table 9-1. A negative relationship would imply the opposite: relatively few cases in the lower left or upper right. or we can measure how strongly the dependent variable is determined by the independent variable relative to other things that also help determine it. Accordingly. and voters who favored helping those in economic crises did not want to see other sorts of involvement. of course. There might be a negative relationship. In tins chapter I build on the introduction in Chapter 8 to discuss in brief how to measure three types of relationships: (1) those among ordinal and nominal variables(2) those that involve dichotomous variables. And just as in a scattergram that shows the relationship between two intervally measured variables. involvement abroad in one form opposed it in all other ways as well. To measure how much change is produced in the dependent variable. Like a scattergram.S. while of the 233 who are not willing to give aid. Each number in the table gives the number of people in the 1998 study who exhibited a given combination of attitudes toward the two sorts of international intervention. and those opposing U. These are actual numbers of cases. 29 are extremely willing to have the United States use military force. University of Michigan. For instance. with willingness to use military force increasing somewhat as voters are more willing to give financial aid. We can measure how much change is produced in the dependent variable by a change in the independent variable. only 18 are extremely willing to have 132 .

n o t j u s t t h e c o r n e r s . w e m u s t d e v e l o p m o d e l s o f t h e perfect relationship a n d of perfect i n d e p e n d e n c e .*%•:•*. a n d also develop a measure to indicate h o w close the data in our table c o m e to these two m o d e l s . however. T h i s indicates a pattern consistent with a mildly positive relationship.'S3* * * ..*. T h i s is simply the price we p a y for the fact that ordinal m e a s u r e s are naturally less informative than interval m e a s u r e s . • • . the observations are located in one or another of the cells of a table..:. B u t n o n e o f t h e m e a s u r e s that h a v e b e e n d e v e l o p e d i s all that satisfactory. • «• • . or vice versa? 3 This measure was described in Goodman and Kruskal (1954). 9 ) t h a n e x p e c t e d . a n d t h e r e a r e m o r e i n t h e l o w e r left ( 4 8 v s . however: Willingness to Have United States G i v e Financial Aid to Countries in Crisis 1. " the data in Table 9-1 w o u l d appear as in Table 9-2. •• •••••• . the similarity to a scattergram becomes more obvious. .-•* • % : •:•.. ..•* • ••• : « £ S 3 K . 27) and l o w e r . as w a s d o n e w i t h the correlation coefficient. "Very willing" might represent only slightly less support than "Extremely willing." O n e s u c h m e a s u r e for u s e w i t h ordinal variables i s G o o d m a n a n d K r u s k a l ' s gamma. 39) a n d u p p e r r i g h t ( 2 9 v s . and often t w o rather different patterns will p r o d u c e the s a m e result. and vice versa.) Even though there is not a precise regression line to define residuals.134 Chapter 9 T A B L E 9-2 Independent Version of Table 9-1 Introduction to Statistics 135 the United States use military force. because for all we know. 14) t h a n o n e w o u l d e x p e c t i f t h e r e w e r e n o r e l a t i o n s h i p . the precise pattern in the table is meaningless. There are t w o differences b e t w e e n a table such as this a n d a scattergram. .*.»:^v. • • • • •.••*•*••*"•. • '•*•. • : • • •. however.:'. 2 I t t a k e s a s its m o d e l o f p e r f e c t i n d e p e n d e n c e a t a b l e i n w h i c h t h e d a t a are distributed u n i f o r m l y t h r o u g h o u t t h e t a b l e . which notes a w a y to incorporate ordinal measures as independent variables in combination with interval variables. • ' ••• •. How great a change there is in attitudes as we move from "Very willing" to "Extremely willing" tells us nothing specific. Instead.: • • • •• • .. as in Figure 9-1. . we can develop a m e a s u r e o f c o r r e l a t i o n . their presence is indicated by counting the number of observations falling in each cell and printing the number there rather than by printing a dot for each observation. T o d o this.•••*••• • •. ••"»••• ' *••*•• • • * • * • • • . Because there is a limited number of values available for each variable (four for the independent variable and four for the dependent variable).*. *lw. Because there are no units by which to measure the difference between two values of a variable.. *•. •.•• . Because so many observations are tied on each value. that is. . . •**••*••>* • *••*****•• « * !!•••„••** • • • • # • • * • * # . . T h e result is useful. Is there anything special.. about those who buck the general trend by favoring the use of military force but opposing financial aid. For instance. there are fewer cases in the upper-left corner (8 vs. it might still be useful to look at the "residuals" in Table 9-1—the cases that do not fit the generally positive relationship. but a bit a m b i g u o u s . If the observations are indicated by dots.••• *•*»*»•••• • . it is not necessary to use the infinite space of a plane to locate the observations. . it is possible to develop measures analogous to the correlation coefficient.:*: distributions in Tables 9-1 a n d 9-2 to see h o w different they are in a positive or negative direction. 3 Willingness to Have United States Use Military Force to Solve Problems Extremely willing Very willing Somewhat willing Not willing Not Willing 27 53 114 39 Somewhat Willing 80 157 340 115 Very Willing 30 59 127 42 Extremely Willing 9 19 40 14 T h u s it is not possible to m e a s u r e the strength of relationships in a w a y analogous to regression. 2. * * • . . b u t t h i s i s t h e general principle of h o w they work. Figure 9-1 Scattergram Version of Table 9-1 Measures like the Goodman-Kruskal gamma are designed to compare the * •*•****• * * • • • • • * * * • * \ * • !•*•*. (See also 111 footnote 4 on p a g e 138. for instance. •.r i g h t c o r n e r ( 5 v s . On the other hand. F r o m a m e a s u r e b a s e d on cell-by-cell c o m p a r i s o n s like this. A m e a s u r e like G o o d m a n . T h e n u m b e r s in Table 9-2 are printed in italics to e m p h a s i z e that they are b a s e .*•.l i n e c o u n t e r p a r t s for c o m p a r i s o n w i t h the o b s e r v e d n u m b e r s in Table 9-1. U n d e r such "perfect i n d e p e n d e n c e .*• ••••». ..K r u s k a l ' s w i l l u s e all c e l l s o f t h e t a b l e .. • • ••*".**. t h e s a m e p e r c e n t o f e a c h category of the dependent variable occurs at each category of the independent variable.:..vv.

which at first glance appears to be a bit mysterious: Although dichotomies are nominal scales. just as has been done with ordinal variables. As with ordinally measured data. cross-tabulation again approximates the function of a scattergram—that is.000 gender This would mean that the expected income of men at the factory was $14. Each subject is either totally "female" or totally "not female. Regression adaptations for analyzing nominal variables are dealt with in the next two sections. the position of a cell in the table no longer tells us anything.0 for the variable "femaleness" and men have values of 0. in regression and correlation analysis. more simply.($2. This greatly expands the value and applicability of regression analysis. Various measures have been developed to do this. Because there usually is no natural ordering to the dichotomy. However. or one unit less. 20 5 0 Prot. must be arbitrary. Dichotomies and Regression Analysis Dichotomies are classifications that involve only two categories: for example. in which to place Catholics relative to Protestants or Jews in the variable "religion. There is a well-developed technique described in the section "Multivariate Analysis" T A B L E 9-3 Interchangeable O r d e r in Nominal Data (A) Religion (B) Religion Candidate Candidate Preferred X z Y Cath.000. (The same interpretation cannot be placed on nominal-scale variables with more than two categories. and the distance between two observations. Since a regression line serves as a convenient way of summarizing the central pattern in a set of means. by noting that the average income of men at the factory is $14. with a set of data arrayed vertically above "gender = 0" and another set arrayed vertically above "gender = 1. This would be the case if we wished to look at the simultaneous effects of sex and education on income. 20 0 5 Other 0 26 4 4 . in contrast to a table relating ordinal variables. all we can do in measuring the strength of a relationship between two nominally measured variables is to assess the degree to which one variable is determined by the other. The category is either present or not present for each observation. such as income or age." A scattergram that would be consistent with these hypothetical regression results is shown in Figure 9-2. namely the difference between the two categories. It means that not only intervally measured variables.000. they can quite properly be treated as interval scales and can be analyzed using measures appropriate for such scales. while the average income of women is $12. The regression line will actually pass directly through the mean income of men at gender = 0 and the mean income of women at gender = 1. one of which is in A and the other in B.) For example. An alternative way to look at this situation is to think of a dichotomy as measuring the degree to which an observation embodies one of its categories. gender (male or female).2. We would have no common unit. for the order of values of a nominal variable means nothing.000 x 1) = $12. with gender equal to 0 for men and 1 for women.) Where this technique of treating dichotomous variables does become valuable is when we wish to combine a dichotomous variable with other independent variables in an analysis. is one unit." If the difference between totally female and totally not female is taken to be one unit. since the value of "gender" for men is zero. "maleness"). it shows which values of one variable tend to coincide with which values of the other. 0 30 0 Other 0 26 Preferred X Y Z Prot. there is by definition a common unit in which differences between values of the variable can be measured. (Exactly the same result could be presented." As you will see in footnote 4. we might regress income at a low-paying factory on gender. but also dichotomous nominal scale variables—and mixtures of the two—can be analyzed using regression analysis. For instance. the decision of whether A is one unit greater than B. Then the distance between two observations falling into category A is zero units. Consider the example in Table 9-3. Such classifications have a useful property. But the problem is really better dealt with by adaptations of regression analysis to nominal variable analysis. however.136 Chapter 9 Introduction to Statistics 137 M A U E O RELATIONSHIP F R NOMINAL D T ESRS F O AA In relating two nominally measured variables.000. it can be used together with other interval measures. in which we are interested in the relationship between an interval-scaled variable and a single dichotomous variable. in which versions A and B are equivalent. Because this is a true interval measure. The table could not be rearranged in this way if the variables were measured ordinally.000. for example. referendum vote (yes or no). The trick is that because a dichotomous variable can take on only one of two values. Let us call the categories A and B. the expected income for women would be $14. the dichotomy "gender: male or female" can also be viewed as a variable "femaleness" (or alternatively. The best-fitting regression line might be income = 14.000 . there is yet another trick we can use in this case that will allow us to work with multiple-category variables.000 . 0 0 30 Cath. women have values of +1. or participation (voter or nonvoter). Since the independent variable can take on only two values. for obvious reasons.0. there is no particular advantage to using regression analysis in cases like these. the scattergram for this analysis will look odd. Similarly.

000 14. We cannot appropriately use them as dependent variables in ordinary regression. The value of this trick. there is a perfect relationship here. this would be nonsensical. It would need to be S-shaped and bounded vertically by 0 and 1. 141-146 that allows us to use interval-level measures simultaneously in this way. . but the logit model allows us to derive estimates of the sort such a model would yield.0 to y = 1 at the point at which x equals 10 and thus cannot register the full extent to which values of y are determined by x.3 when the independent variable equaled 8. One indication that this would be problematic is that it would be impossible to construct a meaningful correlation coefficient for such an analysis. That is. Since the regression line cannot bend. and when x is less than 10. cannot be stated too strongly. that y would equal 18 when x equaled 4. can be interpreted as the path of expected values of the dependent variable across varying values of the independent variable. y is always 0. called logit analysis and probit analysis. The techniques involve "ordered logit" and "ordered probit. y is always 1. about 70 percent of the cases would have y = 1. This does not make sense for a dichotomous variable. since it can take on values of only 0 or 1. statisticians have developed an alternative model that is found in two very similar variations. but presentation of these techniques goes beyond the scope of this book.000 0 £ 8 c 12. of on pp. whereby we can include nominal scale dichotomies in the powerful technique of regression analysis. The model would predict.138 Chapter 9 Introduction to Statistics 139 $18. But there is still "error" around the regression line. Consider the scattergram in Figure 9-3. we would predict that y would equal 9 when x equaled 1.000 Gender Figure 9-2 Hypothetical Regression of Income on Gender LOGIT AND PROBIT ANALYSIS To handle the problem of describing the relationship between a continuous independent variable and a dichotomous dependent variable. would be the probability that the dependent variable equals 1. it cannot follow the jump from y .7. for instance. Such a model would portray. since a probability must be positive and less than or equal to 1. as you will recall. An analogy to the expected value of a dependent variable. 4 Techniques have also been developed for including ordinal variables in regression. The ability to include dichotomous nominal variables in such analyses is extraordinarily valuable. The regression line. We do not have any way to work directly with such a model. but there is no such technique for nominal measures. this works only for dichotomous nominal measures used as independent variables. however.000 16. Such variables generally are known as dummy variables or binary variables. the probability that y equals 1." They are variants of the logit and probit analysis presented in the next section. Figure 9-4 provides an example.000 8. It would not make sense in Figure 9-3. is about 0. From the regression line in Figure 8-4. that at x = 5. It is therefore incapable of serving as the "best summarizing line" to describe the relationship between x and y.000 10. I describe logit analysis here. Letting p = P (y = 1). the varying probability that a dichotomous variable equals 1. In any reasonable sense. across the variation in an independent variable. to predict that the dependent variable would have a value of 0. for instance. the logit is the logarithm to base e. that is the natural logarithm. but the general explanation holds for both. However. and so on.000 - 6. for example. Above levels of x = 10. P(y= 1).

o r e v e n t h e p r o b a b i l i t y o f y . 4 0 2 . has m a d e regression analysis a p o w e r f u l statistical tool and the d o m i n a n t tool for data a n a l y s i s in political science. T h i s t e c h n i q u e h a s s e r i o u s d i s a d v a n t a g e s . I n C h a p t e r 6 w e s a w that f r e q u e n t l y i t i s i m p o r t a n t t o h o l d o n e v a r i a b l e c o n s t a n t w h i l e m e a s u r i n g the r e l a t i o n s h i p b e t w e e n t w o other variables. i n c o n t r o l l i n g f o r i n c o m e . f r o m t h e n a t u r a l l o g a r i t h m o f p / ( l -p). T h e e a s i e s t w a y t o d o this i s literally t o " h o l d t h e variable constant. T h e b e s t w a y t o i n t e r p r e t l o g i t a n a l y s i s . a n d t h e w a y i t i s u s u a l ly presented in studies.71 141 Age 1. a n o t h e r o f all t h o s e w i t h a n i n c o m e o f $ 4 3 .6 l o g i t ( t h e n a t u r a l l o g a r i t h m o f p / ( l . i f w e w e r e i n t e r e s t e d in the effect of age on voting participation. b y separating the subjects into g r o u p s a l o n g t h e control variable ( a g r o u p o f D e m o c r a t s . we m i g h t perform a logit analysis and then present the results: . and m o s t important.58 0. T h i s i s d o n e b y first c a l c u l a t i n g f r o m a a n d b w h a t t h e e x p e c t e d . the n u m b e r of "groups" we set up m i g h t be too l a r g e . t h o u g h . i f w e w a n t t o control for an intervally m e a s u r e d variable. R e m e m b e r . w e m i g h t h a v e a g r o u p o f all t h o s e w i t h a n i n c o m e o f $ 1 8 . T h e ability to incorporate d i c h o t o m o u s variables into regression analysis. Figure 9-4 Path-of-Probabilities M o d e l MULTIVARIATE ANALYSIS p / ( l — p). i t i s t h e n a t u r a l l o g a r i t h m o f p / ( l .0 25 35 45 55 65 75 . T h e n . 2 4 3 . but d o i n g so throws a w a y a great deal of the precision in the m e a s u r e — s o m e t h i n g o n e m a y not w a n t to do. rather. is to calculate by a series of transformations on the logit coefficients what the e x p e c t e d probability of y is at various values of x and to present t h o s e d i s c r e t e l y .p)) i s a t e a c h o f s e v e r a l v a l u e s o f x . a n d a g r o u p o f i n d e p e n d e n t s . i t i s n o t y . o r t h r o u g h t h e l o g i t a n d p r o b i t v a r i a n t s o f regression analysis if the d i c h o t o m y is a d e p e n d e n t variable. T h e c o m p u t e r calculates f r o m the n u m b e r of 0s and 1 s at g i v e n values of the i n d e p e n d e n t variable w h a t the e x p e c t e d logit of the d e p e n d e n t variable is across the range of the independent variable. t h e w a y t h i s w o r k s i s that t h e d e p e n d e n t v a r i a b l e h a s v a l u e s o f e i t h e r 0 or 1. T h i s i s a d i f f i c u l t q u a n t i t y t o c o n c e i v e ! B u t i t d o e s e m b o d y t h e p r o b a bility i n w h i c h w e are interested. that i s p r e d i c t e d f r o m x i n t h i s e q u a t i o n ." a s i n t h e e x a m p l e o n p p . F o r i n s t a n c e . a g r o u p o f R e p u b l i c a n s . 9 4 .68 0. T h e r e are a l s o v a r i o u s a n a l o g s t o t h e c o r r e l a t i o n c o e f f i c i e n t f o r logit a n a l y s e s but n o n e that is readily p r e s e n t e d at this l e v e l . i f registration i s t h e variable t o b e logit of y It turns o u t that the l o g i t c a n appropriately be taken as a linear f u n c t i o n of a c o n tinuous independent variable.70 0.71 0. F o r i n s t a n c e . in the standard form bx h e l d c o n s t a n t ) . i t i s s t r a i g h t f o r w a r d t o s o l v e f o r p ( t h e p r o b a b i l i t y t h a t t h e d e p e n d e n t v a r i a b l e i s 1 rather t h a n 0 ) .37 0.p).140 Chapter 9 Introduction to Statistics Estimated Probability of Voting 0. for e x a m p l e . the l o g i t m o d e l a l l o w s u s t o use regular regression analysis to analyze a d i c h o t o m o u s dependent variable. T h e r e l a t i o n s h i p b e t w e e n t h e o t h e r t w o variables i s t h e n m e a s u r e d for e a c h group. either d i r e c t l y i f t h e y are i n d e p e n d e n t v a r i a b l e s . a n d s o o n . B e c a u s e the analysis is so indirect. T h i s p r o b l e m c a n be averted by l u m p i n g subjects into a f e w broad categories of the independent variable. A s e c o n d a n d m o r e s e r i o u s p r o b l e m i n literally h o l d i n g a v a r i a b l e c o n s t a n t i s that the process quite rapidly leaves the researcher w i t h o n l y small numbers of subjects M e c h a n i c a l l y .9 5 . First. the a and b in t h e l o g i t e q u a t i o n are n o t i n t u i t i v e l y i n t e r p r e t a b l e .2 - A table like this approximates the results o n e m i g h t have gotten f r o m a nonline a r r e g r e s s i o n l i n e . in this c a s e there probably w o u l d be as m a n y groups as there w e r e subjects in the study.

a n d e a c h o f the h o r i z o n t a l d i m e n s i o n s is one of the i n d e p e n d e n t v a r i a b l e s . if we wanted to control simultaneously for m o r e than one variable. Y o u w i l l r e c a l l that the r e l a t i o n s h i p b e t w e e n two v a r i a b l e s c a n be plotted in a flat s p a c e of two d i m e n s i o n s ( r e p r e s e n t e d by a s c a t t e r g r a m ) . . q. or the amount by which y could be expected to change if the value of w shifted from zero to the particular value you are using. E a c h o b s e r v a t i o n is a dot floating in t h r e e . l o c a t e d a c c o r d i n g to its v a l u e s on e a c h of the three v a r i a b l e s . holding constant race and age. i f e a c h o f the other v a r i a b l e s w e r e " h e l d constant. 6 1 .000 respondents. T h e r e g r e s s i o n equation a s s o c i a t e d w i t h the plane is of the f o r m Figure 9-5 Three-Dimensional Scattergram dependent v a r i a b l e a n d any p a r t i c u l a r i n d e p e n d e n t v a r i a b l e w o u l d b e i f n o n e o f the other v a r i a b l e s v a r i e d — t h a t i s . b l a c k s aged 31—40. and then look at the occupation/vote relationship for e a c h of these groups. with about 2.6 0 . I t m u s t b e the p l a n e that m i n i m i z e s the s u m of s q u a r e d d e v i a t i o n s of a c t u a l y v a l u e s f r o m the y v a l u e s p r e d i c t e d f r o m the p l a n e . whites aged 2 0 . as in our example of race and age. a is the i n t e r c e p t — t h e predicted v a l u e of y w h e n both x and w equal zero. T h e investigator might divide the population into groups s u c h as blacks aged 2 0 . Suppose that someone wanted to examine the relationship between occupation and voting. might include no more than five or six whites aged 2 0 . B u t the typical national survey. S i m i l a r l y . a n d y. any multicategory n o m i n a l variables included in the a n a l y s i s w i l l show up as a set of several dichotomous variables. 2. 5 A final disadvantage of literal controlling is that it produces a series of measures of the relationship (one measure for e a c h group) that is u n w i e l d y and difficult to absorb. you have the expected value of y when x equals zero and w equals the particular value. Add to this b\ times w. At this point. and r.s c a l e v a r i a b l e s ( i n c l u d i n g d i c h o t o m i e s that are taken as i n d e p e n d e n t v a r i a b l e s ) . 3.v a r i a b l e r e l a t i o n s h i p . H e r e fe. Start with a.3 0 . To this sum.d i m e n s i o n a l s p a c e .v a r i a b l e c a s e . J u s t as a o n e . there is an e a s i e r w a y to cont rol for a v a r i a b l e — a t least if we are u s i n g i n t e r v a l . S i m i l a r l y . the r e l a t i o n s h i p a m o n g three v a r i a b l e s c a n b e plotted i n a t h r e e . E s p e c i a l l y . T h e height of the c a r d b o a r d plane above the grid at this point is the value of y predicted f r o m the g i v e n v a l u e s of w a n d x by the regression p l a n e . S u c h s m a l l groups w o u l d give very unreliable estimates of the relationship between occupation and vote. T h e t e c h n i q u e of multivariate r e g r e s s i o n a l l o w s us to l o o k at the pattern i n s e v e r a l v a r i a b l e s a m o n g a l l o f our o b s e r v a t i o n s ( w i t h o u t b r e a k i n g these d o w n into separate subgroups) and to estimate w h a t the r e l a t i o n s h i p b e t w e e n the T h i s regression plane w o u l d look l i k e a flat p i e c e of c a r d b o a r d set at s o m e tilt in the m i d d l e of the t h r e e .d i m e n s i o n a l s p a c e over a g r i d c o n t a i n i n g all p o s s i b l e c o m b i n a t i o n s of v a l u e s of w and x.d i m e n s i o n a l scattergram c a n s u m m a r i z e the pattern in its t h r e e .v a r i a b l e r e l a t i o n s h i p . as noted above." To s k e t c h the t e c h n i q u e in a general w a y .d i m e n s i o n a l l i n e t h r o u g h the t w o . a s seen i n F i g u r e 9 . 2 To c a l c u l a t e f r o m the multivariate r e g r e s s i o n the y v a l u e y o u w o u l d expect f r o m a particular c o m b i n a t i o n of w and x: 1. and so on. Particularly.d i m e n s i o n a l p l a n e through the t h r e e . given these particular values of w and x.5 . the expected value of y when both x and w equal zero. D o t A h a s h a d c o o r d i n a t e s d r a w n to s h o w h o w it is l o c a t e d in the threed i m e n s i o n a l s p a c e by its v a l u e s on w. now have the expected value of y. we c a n locate a particular c o m b i n a t i o n of w and x on this grid. this might leave us with as m a n y as 20 or 30 separate measures to consider. b tells by h o w m a n y units y c a n be e x p e c t e d to i n c r e a s e x if w i n c r e a s e s by one unit and x does not change. T h e v e r t i c a l d i m e n s i o n i s the dependent v a r i a b l e . x.3 0 . the amount by which y could be expected to change if x 2 shifted from zero to the particular value you are using and w remained unchanged. By c o u n t i n g out a certain distance along x and then a certain distance along w. respectively. F o r t u n a t e l y .142 Chapter 9 Introduction to Statistics 143 among w h o m to measure a relationship.d i m e n s i o n a l s c a t t e r g r a m c o u l d s u m m a r i z e the pattern in a t w o . a n d b tells by h o w m a n y units y 2 c a n be expected to i n c r e a s e if x i n c r e a s e s by one unit and w does not change. w h e r e these v a l u e s are p. blacks aged 5 1 . add b times x. so a t w o . You 'Refer again to the box "Law of Large Numbers" on p.d i m e n s i o n a l s p a c e .3 0 . T h e plane i s p i c k e d o n the b a s i s o f the s a m e c r i t e r i o n b y w h i c h the r e g r e s s i o n line w a s c h o s e n . or whatever. I w i l l u s e as an e x a m p l e the c a s e of t w o i n d e p e n d e n t v a r i a b l e s (w and x) and a dependent v a r i a b l e ( y ) . y = a + b\w + b x 2 As in the t w o . and b are the r e g r e s s i o n coefficients of the equation.

6 w a s . however.1 4 4 Chapter 9 N o t i c e in this that b d e s c r i b e s the relationship b e t w e e n w and y if x does not { Introduction to Statistics 1 4 5 Interaction in Research and Theory Although interaction among variables is a nuisance from the standpoint of multivariate regression analysis. the T h i s freedom f r o m the need to "hold variables constant" physically is an important advantage of multivariate regression analysis. W h e n this h a p p e n s — w h e n the relationship b e t w e e n v a r i a b l e s A and B differs.b l a c k schools. no matter what the value of w independent variable. a relationship appeared. see Matthews and Prothro (1963). It might be that the apparent integration of the w e l l . the relationship b e t w e e n m e d i a n i n c o m e and percentage o f A f r i c a n A m e r i c a n students attending a l l .b l a c k s c h o o l s . The same logic would apply to a four-variable case. reaching its apex among those with the highest sophistication score. counties g i v i n g the percentages o f A f r i c a n A m e r i c a n students attending a l l . depending on the v a l u e of a variable C—there is s a i d to be " i n t e r a c t i o n " a m o n g the variables.000 dollars i n m e d i a n i n c o m e . pp. On e x a m i n i n g j u s t the relationship b e t w e e n S and M. and the percentage o f A f r i c a n A m e r i c a n s i n e a c h c o u n t y ' s population. 8 . esp. Philip Converse was able to argue very neatly for the importance of a measure of ideological sophistication by showing that the relationship between social class and voting was strongly affected by it (1964. w h o are rather dependent on party l e a d e r s h i p for f a v o r s . T h i s is especially true w h e n working with several variables simultaneously.A m e r i c a n s in the c o u n t y ' s population. in itself it is an interesting phenomenon. we need to control for percent A f r i c a n A m e r i c a n in a s s e s s i n g this relationship. O t h e r w i s e . In fact. w e might find the f o l l o w i n g situation: L e t S be the percentage of A f r i c a n A m e r i c a n students attending a l l .1 M technique for use with ordinal data. As one example. the regression plane described by the equation we c o m e up with w i l l be an inaccurate representation of the relationships. For a multiple regression analysis of black voting registration. 6 / 1 w o u l d indicate that this is true. L e t M be the c o u n t y ' s m e d i a n i n c o m e (in thousands of d o l l a r s ) . the general multivariate regression technique applies to any number of variables. We c o u l d test for this by adding the variable "percent A f r i c a n American" as an additional multiple regression equations by treating them as dichotomies (refer to p. although no scattergram can be drawn in these four dimensions.b l a c k s c h o o l s . it is important in multiple linear regression that the data actually fit a m o d e l in w h i c h all the relationships between the dependent variable and the independent variables are linear.b l a c k s c h o o l s largely disappears. It is exciting to think that a relationship itself may depend on a further variable. without requiring us to break our observations into separate s m a l l and a w k w a r d groups. 2 3 1 . x. N e w m e m b e r s . In the m u l t i p l e regression equation. with the equation y = a + b w + b x + b z. might vote obediently along This example is a particularly interesting one because Converse demonstrated yet a second level of interaction (the interaction itself varies with gender) that I have chosen to ignore here. T h e regression equation S = 49. 136). i f w e had data for U . and that b d e s c r i b e s the relationship b e t w e e n x and y if w does not vary. and y. 8 vary. we could not have calculated the expected value of y simply by adding a plus b\ times w plus b times x. W i t h the cont rol for "percent A f r i c a n A m e r i c a n " added.c u t party d i f f e r e n c e s ) . A c o n c r e t e e x a m p l e m a y m a k e m o r e c l e a r what interaction is and h o w it might arise. We w o u l d have had to pick a value for b\ that w a s 2 appropriate for whatever value of x we were using. it is often impossible to operate under these conditions. or w e a k (less c l e a r .2 3 4 ) . a d e c r e a s e of only 0. Among those who scored lowest in ideological sophistication.1 percent as the m e d i a n i n c o m e of the county i n c r e a s e s by 1. all R e p u b l i c a n s the other). we m i g h t find that the r e g r e s s i o n equation w a s S = 68. In our three-variable example of w. and a value of b that w a s appropriate 2 for whatever value of w we were using. F o r e x a m p l e .0 . 'Although I have used the case of three variables in this section because that case can be represented by perspective drawings. it is important that you be alert to it in your research. using county data for southern states in the 1950s. T h e percentage o f A f r i c a n A m e r i c a n s attending a l l . As the score on ideological sophistication increased. O b v i o u s l y . T h i s relationship c o u l d be strong (all D e m o c r a t s vote one w a y . however.1 percent c a n be expected f r o m an i n c r e a s e of 1. S . and discovering it is fun and dramatic. It is obvious that this w o u l d be a complicated procedure. Because "common sense" rarely gets so complicated as to suggest this sort of thing. multivariate r e g r e s s i o n a l l o w s us to e x a m i n e relationships w i t h certain other v a r i a b l e s controlled. interaction is almost always unexpected. N o w .000 d o l l a r s . In this 2 w a y .2.2 . we had to assume that an increase of one unit in x produced the same change in y.b l a c k s c h o o l s d e c r e a s e s b y 2. It is important that the data best fit a model in w h i c h the relationship between the dependent variable and each independent variable is the same no matter what value the other independent variable(s) takes on. there was no relationship between social class and voting. C o n s i d e r the relationship b e t w e e n party affiliation and h o w m e m b e r s of C o n g r e s s vote on a b i l l . As in simple linear regression.h e e l e d counties is s i m p l y a result of the fact that in the richer counties there are so few A f r i c a n A m e r i c a n s that there are not l i k e l y to be any a l l . we c a n add this to our list of arguments for trying to measure variables intervally as often as possible. N o m i n a l variables m a y be included in T h i s w o u l d indicate that s c h o o l integration i s m o r e w i d e s p r e a d i n r i c h e r counties. There is also an additional requirement in multiple linear regression. Otherwise. Because such interaction occurs frequently in the social sciences. { 2 } 6 strength of the relationship might be related to a m e m b e r ' s seniority. the m e d i a n i n c o m e o f e a c h county. 1 M + 0 . B e c a u s e there is no satisfactory analog to this 7 This little analysis is wholly fictitious. L e t A be the percentage of A f r i c a n .5 .

straight party lines. See. The important thing is that you stay in charge.44 0. pick a limited number of relationships of interest to you. For instance. and keep your eye on the question you want to answer. Appendix A.52* 0. we might present the results as in Table 9-4.50 'Estimated probability of voting Democratic. All of these techniques simplify what is going on in a set of data by screening out certain aspects It is possible to add "interaction terms" to the regression equation to take into account certain types of interaction. the results are usually made understandable by calculating from transformations of the coefficients what the probability of y is estimated to be. and so on. however. Your measures will then be useful summarizing tools. As in logit analysis with a single independent variable. checking for nonlinearity. This is their great virtue. The relationship between party and vote would change as the third variable. not substitutes for a full look at what is going on.53 0.46 0. The coefficients for the variables are difficult to interpret. 9 CONCLUSION You might expect me to conclude. it is your responsibility to delve a bit deeper and see just what is being screened out. It would not be true that "vote" always increased by the same amount with a given change in "party. For instance. who had built up respect among fellow members. A computer can chug out hundreds of regression equations in a few seconds. "Theory Building and the Statistical Concept of Interaction"). might vote more independently.45 0.62 0. since they are predicting not directly to the variable or its probability but (in the case of the logit) to the natural logarithm of pi (l-p) where p is the probability that y = 1. noting which observations are mavericks in the relationship. Those with more seniority.44 Female 0. . if we were predicting voting for the Democratic Party from age and gender. Data will talk to you. especially given the uncertainty involved in working with most social science theories. I am more interested in cautioning you against using them unwisely. Still. "Go out and use these measures. changed. logit and probit analyses may readily incorporate a number of independent variables. but that goes beyond the scope of this presentation. seniority." How much "vote" changed with a given change in "party" would depend on the value of "seniority." However. Use your measures and computer analyses as tools rather than ends in themselves. It is tempting simply to call for a mound of results. 9 of relationships. Usually. for example. It is critical that you examine as carefully as possible each scattergram on which you base a regression equation. and examine each of these relationships in detail. and it is a virtue. each of the scattergrams in Figure 9-6 would give approximately the same regression equation. there would be interaction among the variables. but you must let them speak softly. a regression equation of the form vote = a + b\ party + b seniority 2 Figure 9-6 Four Alternative Scattergrams with the Same Regression Equations would not be appropriate.58 0. given varying combinations of the independent variables. Under these circumstances.48 0.51 0. Thus. Blalock (1969.146 Chapter 9 T A B L E 9-4 Example of Results from a Multivariate Logit Analysis Gender Age 25 35 45 55 65 75 Introduction to Statistics 147 Male 0. you will be better served if you first think carefully about what you want to do.55 0." Like ordinary regression analysis. but the relationships depicted mean quite different things. for various combinations of age and gender.

It c o m e s up in conducting social s c i e n c e research. even though the true state of affairs w a s something other than what is i n d i c a t e d by the result. and in performing numerous other activities. T h e general procedure by w h i c h we c a l c u l a t e t i e probability that a g i v e n result c o u l d h a v e o c c u r r e d by c h a n c e . B u t some of the time y o u w o u l d find that the R e p u b l i c a n s in your sample were more liberal than the D e m o c r a t s . or 0 . is the s a m e procedure that we u s e in c a l c u l a t i n g the odds of any event h a p p e n i n g . T h e s e assumptions had to account for all of the factors that c o u l d influence the o u t c o m e of the d r a w i n g . the probability that any particular result w o u l d o c c u r c o u l d be determined. G i v e n an honest d e c k of c a r d s . y o u r c o n c l u s i o n w i l l be true. you must accomplish two things. y o u are instructed to do this two times. assuming that you have succeeded in the first round. First. 1 LOGIC OF MEASURING SIGNIFICANCE Y o u w e r e able to calculate these probabilities b e c a u s e . administering quality control in manufacturing. In other words. w h i c h s o m e t i m e s t e a c h e s students h o w to u s e the tests w i t h o u t putting the tests in a broader perspective. If it is of the first type. your c o n c l u s i o n w i l l be false. opinion polling. even though the result is not true in general. assigning television ratings. A s I p o i n t e d out i n C h a p t e r 8 . (I m u s t note p a r e n t h e t i c a l l y that. it w a s possible 148 . As usual w h e n talking about probability. Now. let us start w i t h a d e c k of cards. for there are 52 cards and only one of them is the k i n g of hearts. one broad procedure h o l d s for all l e v e l s of data. senators at r a n d o m . we found that there w e r e qualitatively different w a y s to go about it. or that the two parties did not differ at all. there are w a y s to determine j u s t h o w l i k e l y it is that a particular result c o u l d have o c c u r r e d b y c h a n c e e v e n i f its opposite w e r e gener al l y true. g i v e n a sufficient set of assumptions. S . Y o u k n o w that the probability that y o u w i l l draw the k i n g of hearts is 1/52. w i t h R e p u b l i c a n s the m o r e liberal group. you can expect to proceed to final success only one fifty-second of one fifty-second of the time. in the full two-draw process. or H o w to G a m b l e on Y o u r Research means of these techniques we c a n c a l c u l a t e h o w m u c h stock to put in our findings. W h e n we do this. 0 0 0 3 7 . for instance. it w o u l d be p o s s i b l e to c o m e up w i t h a group c o n s i s t i n g largely of c o n s e r v a t i v e D e m o c r a t s and liberal R e p u b l i c a n s . and so o n . and (2) estimating h o w l i k e l y i t i s that the m e a s u r e y o u h a v e gotten c o u l d h a v e o c c u r r e d b y c h a n c e . if it is of the s e c o n d (a distorted s a m p l e ) . although there are differences in s p e c i f i c techniques. T h i s is a c o m m o n problem that must be dealt with w henever anyone tries to describe a w h o l e population of c a s e s by looking only at a sample drawn out of the population. F r o m that y o u might c o n c l u d e — e r r o n e o u s l y — t h a t there w a s a relationship b e t w e e n party and ideology in the Senate. Y o u k n o w that the probability of d r a w i n g a k i n g of hearts both times is 1/52 x 1/52 = 1/2704. especially measuring relationships between variables. H o w e v e r . simply because D e m o c r a t i c senators do tend to be more liberal than R e p u b l i c a n senators. conducting market a n a l y s e s . T h i s a p p r o a c h m i g h t a l s o s e r v e as a u s e f u l s u p p l e m e n t to a s t a t i s t i c s c o u r s e . T h e problem in drawing a single group and describing the Senate f r o m it is that y o u cannot k n o w whether your group is of the first type (the relationship between the variables is the same in your subset as in the Senate as a w h o l e ) or of the s e c o n d (the relationship between the variables is different in the 'The logic of this is as follows: In order to draw two kings of hearts in two draws. It is a very c l o s e a n a l o g y to say that we c a n g a m b l e intelligently on the results of our study by c a l c u l a t i n g the odds that the results are false. thoroughly shuffled. Next. In l o o k i n g at h o w to m e a s u r e relationships in C h a p t e r 9. you still can expect success in the second round only one fifty-second of the time. y o u are instructed to draw a c a r d . depending on the l e v e l at w h i c h the variables w e r e m e a s u r e d . there are t w o m a i n areas o f s t a t i s t i c s : ( 1 ) m e a s u r i n g things. Fortunately.C h a p t e r 10 Introduction to Statistics 149 subset from that in the Senate as a w h o l e ) . as in p r e c e d i n g c h a p t e r s .) It is quite p o s s i b l e to get a particular result by c h a n c e . selecting the ten at r a n d o m f r o m the w h o l e Senate. In C h a p t e r s 8 and 9 we l o o k e d at the first of these a s p e c t s . If y o u c h o s e ten U . W i t h s u c h a set of a s s u m p t i o n s . we are s a i d to measure (or "test") the statistical significance of our results. you must draw the king of hearts on the first round. most of the time y o u w o u l d find that the D e m o c r a t s in y o u r group were more liberal than the R e p u b l i c a n s . and n o w we e x a m i n e the s e c o n d . This can be expected to happen only one fifty-second of the time. if y o u repeatedly drew groups of ten senators. I do not i n t e n d to t e a c h y o u how to use s t a t i s t i c a l tests. I n s t e a d I p r e s e n t the g e n e r a l l o g i c o f s u c h tests t o h e l p y o u u n d e r s t a n d w h a t they m e a n w h e n y o u s e e t h e m . B y Inference. In m e a s u r i n g the statistical s i g n i f i c a n c e of a relationship. replacing the c a r d and reshuffling after the first draw.

none of the cards stick together. Statistical tests vary in the level of measurement that they require and in the particular set of assumptions that they embody. but turns it on its head. Let us say that you have drawn such a sample and that your results are shown in Table 10-1. These assumptions would be: 1. What you are saying is: "By rejecting its opposite. (This usually is referred to as "rejecting the hypothesis" that the assumptions are true. I am running considerable risk in making the statement. Statistical inference uses this same logical structure. How safe would you be to conclude from this limited sample that senatorial Democrats are on the whole more liberal? To find out. Will you reject that set of assumptions on the basis of your sample result. For instance. From this we know with how much confidence we can predict that the event will occur. An example may be the best way to demonstrate this. accepting the one chance in four that you are wrong in doing so? If you are willing to reject the assumptions. given the set of assumptions. and so on)." 2 HYPOTHESIS TESTING The example we have used is a typical problem in statistical inference: We have a particular result in hand. The chi-square test used here is particularly designed for testing the significance of a relationship between two nominal-scale variables. assumption 2 above was false). is 0. your calculation of probabilities would be incorrect. We have a set of assumptions of which we are confident. The drawing is blind. Thus there is a good chance that you could have gotten a relationship as strong as this in a sample of ten even if the assumptions that you set up were true—that is. You know whether or not you have drawn the ten senators at random. This is the way in which we normally use odds and probability. After all. that is. it is possible for you to calculate how likely it is that you would have gotten any particular result in your sample.' I know that given the amount of evidence I have gathered.150 Chapter 10 Introduction to Statistics 151 to calculate exactly how likely any particular result was. T h e ten senators have been drawn at random. we decide to treat the set of assumptions as false. T h e deck has 52 cards. 2 . the result we have observed could have occurred even if the assumptions were true. the probability of drawing one in a single draw would have been 2/52. TABLE 10-1 Sample Result Democrats Republicans 2 4 Liberal Conservative 3 I There is a great variety of statistical tests available to fit different circumstances of research. so that each card is equally likely to be drawn ( y o u don't peek. what kind of risks ride on your decision. even if there were no difference between Democrats and Republicans. See the discussion on pp. 3. using a "chi-square" test (which I introduce later in the chapter). in rejecting the set of assumptions. Those assumptions determine the probability that any particular event will happen. The deck is thoroughly and honestly shuffled before each drawing. There is an ideological difference between the senatorial parties. you purposely have set up the test this way. In this particular case. but only 33 percent of the Republicans. There is no relationship b e t w e e n party and i d e o l o g y in the "full" Senate from w h i c h these have been drawn. you really are rejecting only one assumption. using probability theory. are liberal. There is a probability of 0. The set of assumptions in this case was as follows: 1. rejecting the set of assumptions really means that we are rejecting that one assumption. the probability of drawing one would have been greater than 1/52. 75 percent of the Democrats. not 1/52. 2. 2. We run some risk of being wrong in treating the assumptions as false. Or if the deck had been stacked so as to make drawing the king of hearts likely (that is. If any of these assumptions were not true. and we wish to calculate the probability that this event could have occurred given some set of assumptions.) And since we are confident of all but one of the assumptions (the thing we want to test). if in fact Democratic and Republican senators do not differ in the degree to which they are liberal. If there is a sufficiently small probability that the event we have in mind could have occurred. Therefore. We cannot be confident of one of these assumptions. you can calculate that the probability of having gotten at least as great a difference as you found between the parties. given the assumptions.264. of w h i c h only o n e is the king of hearts.264 that I am wrong. if there were two kings of hearts (that is. Just from these two assumptions. How you would then treat your research results depends on whether you are a long-shot gambler at heart. by setting a list of assumptions that would be sufficient to let you calculate the probability of drawing ten senators that look like those you have. 153-155. you must proceed just as you did in our card-drawing example. assumption 1 above was wrong). EXAMPLE OF STATISTICAL INFERENCE Let us return to our previous example. at what odds we should bet that it will occur. for it involves the very thing we are trying to infer. taking a sample of ten senators to see whether there is a difference in how liberal the two senatorial parties are. Of the ten senators in the sample. and so on. Approximately three times out of four you would not have gotten as strong a relationship as you did if the assumptions you wish to test were true. I am making the statement. notice that the truth is questionable with regard to only one of the assumptions in the set. the assumption that there is no relationship between party and ideology in the Senate.

you can deal with a single statement that negates all possible versions of the statement you want to make. the probability that we are making a mistake in rejecting the assumptions. It tells us the probability that the result we have observed could have occurred if the assumptions we are rejecting were in fact true—that is. You can see now why I said that statistical inference uses the same logic as setting odds. It is supposed to apply to occasions as yet unforeseen. By using a null hypothesis ("there is no difference between the parties"). the probability shows us how likely it is that we are making a mistake. Given the strength of the relationship between the variables in the table." of course. we really are rejecting only the null hypothesis. Taken to a sufficient number of decimal places. Example: % 2 Null Hypothesis It is clear from our description of statistical inference that we do not test directly the statement we wish to make about a relationship. a hypothetical relationship 'between ethnicity and policy priorities has been drawn. because objective criteria are lacking. we calculate how likely it is that an observed result could have occurred if some hypothetical and contrary assumption were true. "can't we just test our hypothesis directly instead of having to turn it inside out?" But think about how you might go about doing this. Therefore. but in general. would give first priority to employment and welfare if exactly the same proportion of WASPs as of African Americans and "other ethnics" favored the employment/welfare 2 2 2 T A B L E 10-2 Ethnicity and Preferred G o v e r n m e n t Action Priorities Ethnicity Preferred Priority Employment and welfare Foreign affairs Environment Total WASP 30 11 29 70 African American 27 5 40 CO Other 53 14 23 90 Total 110 30 60 200 . a convention usually is followed of rejecting a set of assumptions only if the probability is less than 0. no sample would have produced exactly this figure. 3 One popular significance test is % (chi-square). if we decide to reject the set. you would not want automatically to reject it. based on a sample of 200 respondents to a survey. we insert its opposite— which is called the null hypothesis. you would have to choose among an infinite variety of possible true relationships (80 percent of Democrats liberal. % allows us to estimate the probability that there is no relationship between the variables in the full population from which the sample has been drawn. In setting odds. because the result is supposed to hold not just for some particular occasion. on the evidence of this sample. The twisted and convoluted way one must think to understand this process is discouraging to most people when they see it for the first time. We normally will reject a set of assumptions only when the probability that they could have produced the result we have observed is comfortably low. 75 percent of Republicans liberal. 51 percent of Republicans liberal. in setting odds. 3 range. If the probability is sufficiently low.152 Chapter 10 Introduction to Statistics 153 The probability we have calculated tells us precisely how likely it is that we are wrong. Such a construct is shown in Table 10-3. Accordingly. Just as in everyday setting of odds. there also will be such criteria available. But because your sample was not that precise a reflection of reality. Bush in 2004. However. we select a set of assumptions from which we can confidently predict how likely it is that certain things will happen in the future. Because it is the only questionable member of the set of assumptions. It is designed for use with a table relating two nominal-scale variables. in the Senate example. is a difficult thing to judge. 50. we use the result to reject the assumption. We want to know whether it is safe to assert. it would virtually always not be the same as the true population value. for instance. however. if you wanted to assert that your sample result was sufficiently close to the true relationship for you to accept the sample result as valid. For instance. on the other hand. In theory-oriented research. 100 percent of Democrats liberal." they ask. In much engineering research. this is a test that is impossible to set up. For instance. Instead. In Table 10-2. the hypothesis we should like to reject—into the set of assumptions from which we calculate the probability.05. 100 percent of Democrats liberal. By rejecting the null hypothesis we in effect assert its opposite. Normally. a single alternative is chosen. because you do not know what the true value is. you could never test the hypothesis that your sample result was exactly the same as the true value in the population from which you had drawn the sample. but turns it on its head. there usually are no fixed criteria. that there is a relationship between the two variables. It is simply easier to disprove a specific hypothesis than to prove an open-ended hypothesis. say. In statistical inference. To calculate % this table must be compared with a construct designed to show a lack of relationship between the variables. which is the statement we wished to make in the first place.7309834 percent of voters in the United States voted for George W. First of all. The entries in Table 10-3 are arrived at by calculating how many WASPs. one usually has objective outside criteria to "help" in making the decision—an amount of money to be lost if the person is wrong. 1 percent of Republicans liberal. you would pick some range of values around the true value and test whether or not the sample result was in that Whether the probability is "comfortably low. and so on). In using the null hypothesis. "Why. which negates all possible true values that would be consistent with the statement you wish to make. which we used in our liberal— conservative example.

Cell A is the upper-left cell of Table 10-2 or 10-3 (the cell in which WASPs choosing employment/welfare fall).0 72.25 0. inasmuch as each cell's result will be 2 2 Other 49.5 + 5.0 -8. . that the full population does not look like the distribution in the hypothetical table? Chi-square is a measure of how different the two tables are. cell B is the next cell to the right. and vice versa.25 1.0 70 African American 22.0 90 Total 110 30 60 200 (Fp ~ F f F h h = = F h In Table 10-4. + 0.154 Chapter 10 T A B L E 10-3 Table of Nonrelationship Ethnicity Introduction to Statistics 2 155 Preferred Priority Employment and welfare Foreign affairs Environment Total WASP 38. (2) this figure is squared. in the card-drawing example on p.593 X = total = 8. That is.024 0. Anglo-Saxon Protestant. on the basis of what we found when we looked at the sample. and (2) that its entry is the same proportion of its column frequency as its row frequency is of the total sample. the sampling distribution states that the probability of drawing the ace of hearts is 1/52. the greater % will be.0 27. chose employment/welfare.167 0.333 0. For any particular set of assumptions. That is.048 1. we would expect to find that 110/200 of the 40 African Americans. (1) the prediction from the hypothetical table is subtracted from the actual figure. Table 10-3 embodies the null hypothesis that we want to test—that there is no relationship between the variables.5 6. If the tables are exactly the same.5 + 8. F is the observed frequency and F is the frequency predicted for the hypothetical table. It represents the application of probability theory to those assumptions.0 G H I 4 WASP is the acronym for white. the members of each row are equally likely to fall into any given column.444 where for each cell in the table.247 0. Thus for each cell in the table. to calculate the probability that each possible result would occur.0 12. It is calculated from the formula 4 Before I can show how we go from measuring the difference between the observed and null hypothesis tables to calculating the probability that the null hypothesis is true.5 21. 149.0 + 3.5 27.0 12. % is calculated for the present example.5 1. until the hypothetical table was filled. I must first introduce the idea of a sampling distribution.0 13.0 40 together (you will recall that this is what the sign "X" means) to give us the %. a sampling distribution shows what proportion of the time each particular result could be expected to occur if the sampling technique chosen were repeated a very large number of times and the set of assumptions (including the null hypothesis) were true.5 + 0.0 49. it gives the probability of getting a particular result if you applied the techniques you are using to a population for which the null hypothesis is true. it was necessary to maintain the fiction that there could have been one-half of a WASP in favor of the employment/welfare priority. from all the cells of the table.00 12.25 25. Each cell in the table has the characteristic (1) that its entry is the same proportion of its row frequency as its column frequency is of the total sample. cell D is the middle left cell.5 13.00 16. Some simple sampling distributions can be calculated easily. The question we want to ask is: Are the two tables sufficiently different that we can say.F ) /F 0 h 2 h 38.5 22.0 -4. cell E falls at the exact center of the table.5 10. This is what we should expect if the two variables were unrelated. are added 0 h C D E LL.00 2 1.25 64.00 0. cell by cell.0 -4. and so on. the probability of drawing the T A B L E 10-4 Cell A B Calculations for % 0 2 (F ~ F ) h (F -F ) 0 h 2 (F . including the null hypothesis. A sampling distribution embodies all the assumptions you make in a test.5 WASPs who chose employment/welfare.136 0. 2 Sampling Distribution priority. because there are 70 WASPs in the sample.0.019 3. cell C is the upper-right cell. Thus. For instance. or 22.5 21. and so on.00 16. and 110/200 of the sample favor the employment/welfare priority. Notice that to construct a table in which the variables are completely unrelated. and (3) the squared difference is then divided by the prediction from the hypothetical table. we would expect to find that the sample contained 110/200 x 70 = 38. % will equal zero.5 10. The more the tables differ. Similarly.0 6. The results of this.877 1.

and the probability of getting a tail f o l l o w e d by a tail is 0. we c o u l d only get at least two heads by getting exactly two heads. T h e same s a m p l i n g distribution c o u l d be presented in a cumulative graph.9. for Three-by-Three Tables 2 1. at least one h e a d . 2 5 . (2) the flipper is honest.) T h u s .) F r o m this s a m p l i n g distribution we see that there is a probability of between 0. T h i s m e a n s that we w o u l d r u n a r i s k of 2 Figure 1 0 .75.1 Sampling Distribution for Two Flips of a C o i n Figure 1 0 .25. T h e s e s a m p l i n g distributions are s i m p l e . 2 x at least as high as the one observed w o u l d have been obtained if the sample were 2 2 drawn by random selection from a parent population in w h i c h the variables were unrelated. T h e s a m p l i n g d i s t r i b u t i o n for a different set o f a s s u m p t i o n s w o u l d l o o k different.00 . in fact.0. so the probability of at least one head is 0.25 +^ .3 . so the probability of that outcome is 1. we c o u l d get at least one head either by getting no heads (probability of 0.25. (Note that this is a cumulative sampling distribution. the probability of getting a tail f o l l o w e d by a h e a d is 0.05 and 0. T h e c u m u l a t i v e s a m p l i n g distribution of % for a three-by-three table (a table w i t h three categories in the r o w s and three categories in the c o l u m n s .50 n o a. the probability of getting a head f o l l o w e d by a head is 0.50). and (3) the c o i n w i l l not stand on its edge.75 a .25.2 Flips of a C o i n C h i . O f c o u r s e .25) or by getting one head (probability of 0. s u c h a s T a b l e s 10-2 and 10-3) i s presented i n F i g u r e 1 0 . In another s i m p l e c a s e . and (2) random selection. ( T h e sampling distribution is based on assumptions of (1) the null hypothesis y o u w i s h to test. B e c a u s e two of the possibilities involve getting j u s t one head in the two throws.00 0 1 Number of Heads 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 . t h r o u g h a l l the c a r d s o f the d e c k . Y o u c a n now calculate that the probability of getting a head f o l l o w e d by a tail is 0. We a l w a y s w i l l get at least zero heads ( w e cannot get a negative n u m b e r ) .00 Introduction to Statistics 157 k i n g o f hearts i s 1/52.50 n o i _ a. T h e s a m p l i n g distributions for most statistical tests are m o r e c o m p l e x .10 that we w o u l d have f o u n d a % as h i g h as 8.3 Sampling Distribution of % .25. F i g u r e 10-2 charts the sampling distribution in this way. and the probability of getting two heads is 0. a n d we must u s e tables developed by statisticians to v i s u a l i z e them.25. at least no heads.3 if the variables y o u were w o r k i n g w i t h w e r e . w h i c h y o u k n o w y o u have done.156 Chapter 10 1.s q u a r e sampling distributions vary depending on the number of categories in the rows and c o l u m n s of the table used. so the probability of at least two heads is 0. or at least two heads. if the n u l l h y p o t h e s i s w e r e true). if y o u had found a % of 4. suppose that y o u h a d to predict the probability of getting j u s t one head in two flips of a c o i n . like the one depicted in F i g u r e 10-2.25.50. T h e probabilities given are the probability that a 1 Number of Heads Cumulative Sampling Distribution for Two 2 . a n d s o o n . the probability of getting no heads is 0 .75 > ro . T h i s s a m p l i n g distribution is presented g r a p h i c a l l y in F i g u r e 10-1. y o u w o u l d k n o w that the probability of getting x as high as this w a s 0. A s s u m e that (1) the c o i n is honest. 2 not related. the probability of getting j u s t one head in either of the two w a y s is 0. this s a m p l i n g d i s t r i b u t i o n d e p e n d s o n the set o f a s s u m p t i o n s y o u m a d e i n that e x a m p l e . s h o w i n g the probability of getting . .44 f r o m a three-by-three table if there w e r e really no relationship b e t w e e n ethnicity and c h o i c e of priority (that i s .25 0 .00 0 Figure 1 0 .

it is i m p o s s i b l e to c h o o s e a single " b e s t " l i n e by least-squares criteria. finding a yf of 8. T h u s a significance test furnishes a useful c h e c k on our research. O b s e r v a t i o n s w i t h w i d e l y v a r y i n g v a l u e s on x. Figure 10-4 Introduction to Statistics 1 5 9 Independent Variable Regression Analysis with Zero V a r i a n c e in the Independent Variable m a n y c a s e s we have used but also of h o w m u c h the independent variable varies. w o b b l e in the line does not c h a n g e the e x p e c t e d v a l u e of y v e r y greatly. this amounts to rejecting the null hypothesis and asserting its opposite. No one of the observations is able significantly to affect the regression line in F i g u r e 10-5.444. ( I n other w o r d s . this is determined mathem a t i c a l l y by the f o r m u l a s for a a n d b. as in F i g u r e 1 0 . y in the scattergram. and (2) a group of other assumptions that we feel certain are true. takes the n u m b e r of c a s e s into account. A s i t h a p p e n s . y w i l l not c h a n g e the s i z e of the s q u a r e d deviations f r o m the l i n e by v e r y m u c h . the l i n e is free to w o b b l e a g o o d d e a l . A l l other things being e q u a l . we w o u l d run a risk of less than 1 c h a n c e in 100 of being w r o n g in asserting that there is a relationship between the v a r i a b l e s . e a c h of w h i c h has an e q u a l s u m of squared deviations about itself. 6 N is the most o b v i o u s factor affecting the statistical s i g n i f i c a n c e of findings. the m o r e firmly fixed the estimate of the r e g r e s s i o n l i n e c a n be a n d . w h o s e Every s i g n i f i c a n c e test. T h e l i n e i s s i m p l y not h e l d f i r m l y i n p l a c e b y these observations. that the event we have observed (in our ethnicity/choice of priorities e x a m p l e above. A c o n s i d e r a b l e c h a n g e in the angle at w h i c h the line p a s s e s through the point x. T h i s is due to the fact that for observations w h o s e v a l u e on x is c l o s e to x. T w o s u c h observations have been added in F i g u r e 10-6. If we choose our significance test appropriately. the number of c a s e s in a s a m p l e . thus the difference b e t w e e n the o b s e r v e d and e x p e c t e d v a l u e s of y also is not greatly changed. T h u s f r o m a table s h o w i n g e x a c t l y the s a m e pattern as T a b l e 10-2 but based on a s a m p l e of 4 0 0 c a s e s instead of the 2 0 0 used in that table. values on x fall so c l o s e together. a c c o r d i n g l y . w o u l d h o l d a m u c h firmer grip on the regression line. we reject the assumptions on w h i c h the s a m p l i n g distribution is based. T h e probability that we c o u l d have gotten the observed result if all the assumptions underlying the sampling distribution were true tells us h o w likely it is that we are w r o n g to reject the null hypothesis. 5 lines (a few of w h i c h are indicated on the graph) c a n be p a s s e d through the data. the set of assumptions on w h i c h its s a m p l i n g distribution is based w i l l consist of (1) a null hypothesis (the assumption that what we really want to say about the data is false). we are s i m p l y unable to c h o o s e any regression line. In m a n y sorts of statistical tests. T h i s is the extreme c a s e . the probability of getting a % at least this high w o u l d be less than 0. every r e g r e s sion l i n e must p a s s through point x. If we find f r o m the sampling distribution that it is very unlikely. If e a c h n u m b e r in T a b l e s 10-2 and 10-3 were doubled. w h i c h is what we had originally wanted to say about the data. It tells us how likely it is that we c o u l d have gotten our results by c h a n c e a l o n e — t h e probability that in fact the opposite of what we are asserting is true.44) c o u l d have o c c u r r e d . 6 . T h e probabilities in a sampling distribution are based on a set of assumptions. F o r e x a m p l e . B u t there m a y be others. 61. h o w ever. B e c a u s e we are sure that all but one of those assumptions (the null hypothesis) are true. it is the only factor we need to w o r r y about. In the extreme c a s e of zero variance in the independent variable. the greater the n u m b e r of c a s e s on w h i c h a statement is based. given those assumptions. or 16. the quantity (F — F ) /F 2 0 h h w o u l d double for each cell.) If there is relatively little variation in the independent v a r i a b l e . is a l w a y s a factor in s i g n i f i c a n c e tests. the greater the variation in the i n d e p e n d ent v a r i a b l e . O n e w a y to l o o k at this is to think o f the r e g r e s s i o n line a s w o b b l i n g o n a f u l c r u m . the m o r e certain y o u c a n be that the statement is true.01.1 5 8 Chapter 10 b e t w e e n 1 c h a n c e in 10 and 1 c h a n c e in 20 of b e i n g w r o n g in asserting on the b a s i s of T a b l e 10-2 that the n u l l h y p o t h e s i s is f a l s e — t h a t there is a relationship between ethnicity and the c h o i c e of priorities. F r o m F i g u r e 10-3 we see 2 that if the null hypothesis of no relationship w e r e true. In other w o r d s . for e x a m p l e . To s u m m a r i z e these sections: A sampling distribution is a list of the probabilities that e a c h possible event (out of a set of events) w i l l happen. the m o r e stock we c a n put in our findings. depicted in the scattergram in F i g u r e 10-4.888. u s i n g this point as a f u l c r u m . T h e following five paragraphs present advanced material that can be skipped over without any loss in comprehension of other material covered in this book. In g e n e r a l . an infinite n u m b e r of I m p o r t a n c e of N N. the e x p e c t e d v a l u e of y w h e n x equals its o w n m e a n must be the m e a n of y. in regression a n a l y s i s the amount of c o n f i d e n c e we c a n h a v e in our estimate of w h a t the relationship l o o k s like is a matter not only of how 5 S e e the box "Law of Large Numbers" on p.5 . The % w o u l d thus be double 8. a c c o r d i n g l y . B e c a u s e all the data points fall above a single value of the independent variable.

7 Problem of Independent Observations It often happens that a researcher draws a conclusion based on a set of observations that appear to be independent but are really not. the regression line must be the line that minimizes the sum of squared deviations about itself. see King. Autocorrelation occurs when observations are related to some extent but are not (as in the study of police brutality) identical to each other. However. with one set of guidelines and one set of expectations with regard to roughness. month by month. we must take into account not only how many observations have been used in the analysis. Statistical tests might be applied to the regression results. Thus there would in fact be only 12 independent observations (each of the 11 town forces. Consider a study that seeks to determine the extent to which "police brutality" is a function of the explicitness of guidelines issued to police patrols. The regression line in Figure 10-6 cannot stray very far from points A and B without sharply increasing that sum. 4. however. For example. because statistical tests already assume that the data to be analyzed consist of independent observations. president's popularity. Thus a few points whose x values are extreme have more impact in determining the slope of the line than do a larger number of points whose x values fall close to the mean. although the 63 townships looked like separate units. Accordingly. in calculating how likely it is that our estimate of Figure 1 0 . An example may help. and if it does arise. but also how widely they are spread on the independent variable. For example. the size of the deviations of observations A and B (in Figure 10-6) from the line would increase dramatically. plus the sheriff's office). the problem occurs frequently in a more subtle (and more technical) way under the guise of autocorrelation. Because there are in effect fewer observations in the study than the scholar believes. it can generally be avoided by simple common sense. if we look at the relationship between trends in the economy and trends in the U. he or she may overestimate the strength of the evidence.6 The Stabilizing Independent Variable Effect of Variance in the y . If the observations are not independent." The investigator might measure the explicitness of guidelines to patrols in the 11 towns and 63 townships of a county and relate this measure by regression analysis to the roughness with which police in each of those units handle suspects. 7 Figure 10-5 Regression Analysis with Little Variance in the Independent Variable If the regression line were now to wobble through the same angles as in Figure 10-5. and Verba (1994. The problem of nonindependent observations does not often occur in a simple form such as this. they would all be separate parts of a single police administration. Unknown to the investigator.160 Chapter 10 Introduction to Statistics a regression has been due to chance. which should raise a specter of doubt about the correctness of an assertion based on these data. In this case.S. Keohane. Remember. part 1). all township police forces might be under the direction of the county sheriff. Significance tests designed for use with regression analysis take into account both the number of cases and the amount of variation in the independent variable. it might be that some of these units shared a coordinated police force. The relationship in question might be due to chance. ch. For a good expository discussion of multicollinearity. with N stated as 74. we will probably have a T h e general extension of this problem to multivariate regression is the problem of multicollinearity. The difficulty here is that statistical tests will not be able to alert us to the danger. all statistical tests will underestimate the probability that a relationship might be due to chance. Thus we might be inappropriately confident of our assertions. with close and explicit directions presumed to be associated with low "brutality. without the researcher realizing it.

this is a problem all of us must take seriously in our research. about 2. we should seek a more inclusive sort of generality. there is no question but that this is the "true" result. The latter view seems to reflect more accurately what we try to do in developing political theories out of empirical research." Then we must regard these 50 states as a sample drawn for us by chance and accident from a larger metaphysical population of "states"—all states as they might be in the future or as they might have been had their boundaries been drawn differently or had history proceeded differently. This operation is a variant of statistical significance testing. which of course can be set as desired by the person doing the polling. I mention the problem at this point only to alert you to it. This is a statement that should arouse suspicion. Bohrnstedt. and so on. the pollster can conclude that according to the sampling distribution of percent approval that one would get using repeated samples of this sort and of this size. by the way. for instance. Note here. because their chief concern is with plugging established procedures into any given state of affairs. February 1943 had much in common with May 1943. asking a sample of the population whom they will vote for. perhaps plus or minus 1 percent. 8 A ubiquitous activity in our society is polling.000 respondents are polled to describe a population such as that of the United States. but about "state politics. or whatever. however—the way we find out about politics—is by looking at data and seeking out relationships between variables. Status quo administrators. we might look at all 50 states and find that the generosity of their welfare budgets is related to the degree of party competition in the states' elections. and with knowledge of how the individuals have been chosen for the sample (usually some variant of random sampling). with developing theories to explain why things are as they are. then. not. With infrequent exceptions. If the author is trying to draw general conclusions from the study. that to the extent that an engineer sees her task as changing and reforming patterns rather than continuing ongoing administrative procedures.000 respondents gave you plus or minus 3 percent. all the nations in the United Nations.S. If an N of about 2. Based on a given number of cases. For instance. Sometimes an author claims that she need not be concerned about the possibility of chance results because her sample is synonymous with the total population of subjects (all the states. explanatory theory will be relatively more important. or what have you). she must be concerned with the problem of chance results. There are times when we wish to describe a specific population as it exists at one point in time. discuss the problem. and what they need is simply to know what the "state of affairs" is. such as the one by Knoke. Because we have included all the states in our sample. Most texts in statistics or econometrics. In one sense. it means that 95 percent of the time it would fall within that range of the true value. POLLING AND SIGNIFICANCE TESTS SIGNIFICANCE TEST: ALWAYS NECESSARY? Even when we do not work with a limited sample drawn from a larger group of subjects. however. whether they support regulation of handguns. senators. Typically. how great a portion of national income these states involve. this is particularly likely in what I have termed "engineering" research. Though one would not go so far as to say that gathering information from these two months is simply equivalent to having information about a single month (as we could say about any two townships in the other example)." Note that this does not mean that the result absolutely falls within plus or minus X percent of the true value. Thus the pollster claims that the result is accurate to "within plus or minus X percent. For instance. rather than to develop explanatory theories from it. In such research we frequently are interested in measuring some condition of a population so as to react to it or make adjustments in it. 8 ''See the discussion of sampling on pp.000 would give you a lower range. a tax administrator may want to know how many states administer sales taxes." But suppose we think of ourselves as trying to say something not just about the 50 states as they exist at this one point in time. on the other hand. 9 9 . A tax administrator who wanted to change the extent to which states relied on sales taxes to generate income would first have to try to find out why states use sales taxes. In most theory-oriented research. 9 USES AND LIMITATIONS OF STATISTICAL TESTS I have tried in this chapter to present a significance test for what it is—a useful check on research results. What the administrator is concerned with is a description of the tax situation as it currently exists. anything we find out about them is by definition "true generally. increasing A' to 4. Partial solutions are available. researchers often place undue emphasis on significance tests. that changing N changes the width of the "plus or minus" range. propositions that one would expect to be as true of states or nations or senators a decade from now as they are today. and the results are reported as "accurate to within plus or minus 3 percent"—or 2 percent. .162 Chapter 10 Introduction to Statistics 163 problem of autocorrelation. neither can we say that we have as much information here as we would have with two widely separated months. can be more content with purely descriptive information. also. by and large. Unfortunately. but they are beyond the scope of this book. sample results would fall within plus or minus X percent of the true population value 95 percent of the time. or some other level of accuracy.1 0 3 . all the U. the sorts of things I discussed in Chapters 2 through 9. It is a pity that looking at data requires less formal training (but more practice) than does Note. there is a sense in which all research involves "sampling" of a sort and in which misleading results can occur by chance. and Mee (2002) or Kmenta (1997). The real meat of research.

a check on how likely it is that a given measure is due to chance. Evidence? Yes. the authors reported that two variables were more strongly related among the working-class portion of their sample than among the middle-class portion. For two of these votes. Anyone thinking of using these tests should first take a course in statistics. is Duggan and Dean. with probability between 0. The significance test is not itself a measure of the strength of a relationship but rather. it turned out that there was a Wronger relationship among the middle class. CONCLUSION I have tried in this chapter to give you some idea of the general logic of inference. the middle-class portion of the authors' sample was only about half as large as the working-class portion. A recent article illustrates the problem. but these are rejected by journal editors because they are negative. The status of these tests should be strictly that of a secondary check on the creative work the researcher has done in looking at relationships. A few of them write up their results. A researcher studying Congress examines the vote on 100 or so bills.05. This strikes her as a surprising finding. 2. "Common Misinterpretations of Significance Levels in Sociological Journals" (1968). The Significance of Tests of Significance" (1969). reporting a statistically significant relationship between committee democracy and members' satisfaction. For a description of particular tests. The one scholar who did find a statistically significant result publishes it. Among the middle-class respondents. useful presentations of the techniques are given in the text cited at the end of Chapter 8. the relationship was less highly significant.05 level) between a representative's height and the way he voted on the bill. It was because of the smaller number of cases in the middle-class portion that its relationship showed up as less significant than the same relationship for the working-class portion.01 and 0. findings. which is the common thread running through all statistical tests. however. This is why there are so many different tests. What is wrong in each situation ? 1. The second chapter of Siegel (1956) is a particularly useful review of the overall logic of significance tests. the relationship was more highly significant. but in rather more technical detail. when the strength of the relationship was measured by an appropriate technique (the Goodman-Kruskal gamma. In fact. . For further consideration. she finds a statistically significant relationship (at the 0. See also Winch and Campbell. barring that. The authors' conclusion from their own data was wrong. Unfortunately. Another very good treatment is that of Simon (1985). Their evidence for this was that among the working-class respondents. many researchers give the tests an undue emphasis in their research. A large number of scholars study committee systems to see whether democracy in committee decision making leads the members to be satisfied with their work on the committee. with a probability of less than 0. In this example. however. In this study. along the lines of the example I used. FURTHER DISCUSSION A particularly good discussion of the misapplication of significance tests. as it happened. they were using significance tests for something the tests were not meant to do. "Proof? No. the reader should consult a statistics text such as the one cited at the end of Chapter 8. for instance). Perhaps because they have spent so much time in courses learning to use significance tests. by contrast. These tests vary among themselves.164 Chapter 10 Introduction to Statistics calculating significance tests. Most of those who fail to find a relationship leave that question and start to work on other things. All but one of these scholars fail to find a statistically significant relationship between these two variables.01 that there was no relationship. Because previous theorists had said that one should expect a stronger relationship between these variables among the middle class than among the working class. Net result: one published article. and she uses it as the basis for a chapter and a half of her book. think about the two following situations. in terms of their suitability for a given level of measurement and the particular mix of assumptions (other than the null hypothesis) they require the investigator to guarantee. these authors were understandably excited about their findings. It aims at much the same presentation as I have attempted in this chapter. not positive.

b e c a u s e the results of regression analysis apply g e n e r a l l y t o a theory. w h i c h i s t o c h o o s e t h e o r i e s for s t u d y f r o m t h e a l r e a d y existing b o d y o f w o r k i n political science. b e c a u s e it a l l o w s u s m o r e f l e x i b i l i t y i n s t a t i n g t h e theory w e c h o o s e t o w o r k w i t h .) is a natural one. A high d e g r e e of "precision in m e a s u r e m e n t " is important. 2 (London: Clarendon Press. M e a s u r e m e n t accuracy. which is so very obvious as. This has b e e n a b o o k a b o u t d e v e l o p i n g theories a n d trying t h e m out on reality. K i l d a renders a North-East Wind indispensably necessary before a stranger can land. that the pattern we ideally w o u l d h o p e to pick out is obscured. occasions an epidemic cold. T h e situation of St. T h e s e a r e all p r o b l e m s I d i s c u s s e d e a r l i e r i n t h e b o o k . M y criterion for t h e usefulness o f any o f t h e s e t e c h n i q u e s has been the extent to which that technique aided in the creation of elegant theories. I h a v e n e v e r dealt w i t h t h e critical q u e s t i o n : J u s t h o w d o e s o n e find a theory on w h i c h to w o r k ? T h e r e is o n e deceptively s i m p l e a n s w e r to this question. that an investigation had been carried out relating the length of time a body had fallen through the air and the velocity it attained. In all of this. In C h a p t e r 2. K i l d a c a u g h t c o l d w h e n e v e r a s h i p arrived t h e r e . take an existing t h e o r y a n d try it out on s o m e data. This is the most obvious way to go about devising a theory. This problem is not confined to the social sciences. I d i s c u s s e d s o m e o f t h e l i m i t a t i o n s o f this p r o c e d u r e i n C h a p t e r 2 . Christian. The wind. d a t a analysis. to have escaped the penetration of Dr. I have discussed various aspects of research: concept formation. of Docking—after ruminating a little. T h a t is. then. Suppose that early mechanics had developed by the use of regression equations. m e a s u r e m e n t . Vol. There is so m u c h going on in a group of observations. not the stranger. that the m o s t appropriate w a y to devise a theory is simply to look at out the most prominent pattern running Unfortunately. I h a v e argued. 51-52. Johnson and his friend. w h e t h e r it is a matter of reliability or validity. Suppose. . specifically. There was at that time already speculation that illnesses were transmitted from one person to another. Be/swell's Life of Johnson.Chapter 11 the j u m b l e of observations and pick through them? Where Do Theories Come From? 167 follow. T h e i r c o m b i n e d effect is to m a k e it very difficult to l o o k at a b a t c h of o b s e r v a t i o n s and pick out the best simplifying pattern. I e q u a t e d the research process with a search for elegant theories. James in 1769 about the mystery of w h y Boswell. C o l e m a n described the w a y the theory of gravity m i g h t not h a v e b e e n d e v i s e d h a d G a l i l e o g o n e a b o u t h i s t a s k i n t h e w a y m o s t d a t a analyzing social scientists do: Now for the explication of this seeming mystery. as well as that of the author." and not "smart modern thought. the late Reverend Mr. T h e p u r p o s e of a theory is to p r o v i d e a simplified pattern to describe a complicated j u m b l e of observations. but t h e i m p o r t a n t p o i n t for m y discussion here is that this a n s w e r begs the question of h o w the original theorist found the theory. (says he. An amusing example of the difficulty of c h o o s i n g a m o n g potential e x p l a n a t i o n s w i t h o u t the benefit of prior theory some is offered by of their friends a discussion among Samuel Johnson. w h e r e a s t h e r e s u l t s o f c o r r e l a t i o n a n a l y s i s c a n s e r v e o n l y a s a description of a particular situation. 166 . Reading the book with my ingenious friend. and so on. or In a similar vein. Birkbeck Hill. In succeeding chapters. they are often m e a s u r e d imprecisely. W h a t m a k e s a political scientist describe the relationship a m o n g a group of variables in a particular way? Probably it most frequently happens that the A simple example will illustrate some of the difficulties which might arise by this kind of "brick-by-brick" approach to theory. 2 l b i d . W o u l d it not 'G. "The cause. and of the the residents 1 S c o t t i s h island of St. p . and the nonexperi- ®d4 L® nT9 r L j jo © TD DT 1 m e n t a l c i r c u m s t a n c e s u n d e r w h i c h w e g a t h e r t h e m m a k e i t difficult t o isolate the effects o f s i n g l e v a r i a b l e s . it follows quite naturally from the things a theory is s u p p o s e d to achieve. is important b e c a u s e without accurate m e a s u r e m e n t we cannot establish the connection b e t w e e n the data we are h a n d l i n g a n d t h e theory w i t h w h i c h w e a r e w o r k i n g . pp. m u c h of which is extraneous to what we want to do. for that reason. "observations" in political science are riddled with problems of inaccurate m e a s u r e m e n t ." Eventually the explanation favored by B o s w e l l w a s offered in a letter from a L a d y of Norfolk: v = gt researcher observes a b o d y of data and tries to see a pattern a m o n g them. R e g r e s s i o n a n a l y s i s u s u a l l y i s m o r e useful than correlation analysis." but Johnson regarded this as a "prejudice. J a m e s S. The relation in mechanics is that the velocity attained is equal to the acceleration due to gravity times the time the object has fallen. 1887). 52.

and similar social sciences c a n n o t be said to possess such a b o d y of assumptions. 1964). would be air resistance. scientific imagination. it is e a s y to tell. and there would have been indications that at high velocities the relation of velocity to time was not even linear. Copyright 1964 by the Free Press. different velocities.s c i e n c e " is that the f o r m e r i n c l u d e s a g e n e r a l l y a g r e e d . T h e resulting confusion can be both enjoyable and fruitful. The resulting regression equation might have ended up including other variables. the researcher scans the observations. 3 deductive.. The distinction here is between "inductive" and "deductive" theory building. m o r a l imagina- tion. w h a t i s relevant t o that p a t t e r n a n d w h a t i s e x t r a n e o u s . its shape. and other things. To build theory inductively. feathers). air resistance—which was irrelevant to the fundamentals of mechanics. On the other hand. O n e w a y of distinguishing an o n g o i n g " s c i e n c e " from a " p r e . T h e art of building a theory r e m a i n s in flux in political science—partly fit. can be a helpful source of theory. sociology. objects which fell a great distance) and low-density objects (i. but it wouldn't have corresponded to the simple velocity-time relation which served as the basis for Galileo's remarkable contribution to the science of mechanics.e. of course. 1962).e. by bringing in too soon a factor—i. but largely inductive. t ). looking for patterns. from such fields as epidemiology and ecology.e s t a b l i s h e d premises i n t h e s o c i a l s c i e n c e s f r o m w h i c h t o d e d u c e a n y t h i n g . which they would locate on a scatter diagram in order to find the line of best t Where Do Theories Come From? 169 of epidemic growth and population dynamics. pp. a m o n g the j u m b l e of t h i n g s w e find. w h e r e it is feasible. such as mass or density of the object." T h e m o r a l o f t h e s e s t o r i e s i s t h a t w e a l m o s t a l w a y s a r e b e t t e r off i f w e h a v e s o m e i d e a o f w h a t k i n d o f pattern w e w a n t t o l o o k for b e f o r e w e start t o l o o k a t data. T h e o r i e s James S. They might even have served to confound the issue. d e d u c t i v e t h e o r y building is clearly t h e better of t h e t w o . O n e of the best s o u r c e s of d e d u c t i v e t h e o r y is a w e l l . its velocity. which has different effects as a function of the density of the object. T h e r e simply are not m a n y w e l l .168 Chapter 11 where g is the acceleration due to gravity.u p o n b o d y o f a s s u m p t i o n s f r o m w h i c h m o s t o f its theories can be d e d u c e d (Kuhn. and especially for high velocities (i. the observed velocity would fall considerably below that which the theoretical equation predicts. it is h a r d to a r g u e p a t l y that t h e o r y b u i l d i n g must be done deductively. 3 . as noted in Chapter 1. So may microeconomic theory. and bodies with differing densities. s o m e o f w h i c h are n o t all t h a t closely related to our field. Political science... The regression equation would of course have been empirically correct. rational choice theory drawn from m i c r o e c o n o m i c t h e o r y h a s b e e n p a r t i c u l a r l y fruitful for political s c i e n c e . Now if there had been numerous investigations involving different-sized bodies. A c c o r d i n g to the a r g u m e n t I h a v e p r e s e n t e d so far in this chapter. the investigators would have ended with numerous pairs of observations ( V j . L a c k i n g this b a s e for d e d u c t i o n . Coleman. m a y suggest theories to political scientists or sociologists. 100-101. it a l l o w s a p l a c e for every sort of imagination to work: literary imagination. m a t h e m a t i c a l imagination. T h e r e are s o m e tricks that m a y heighten your awareness of situations in w h i c h deduction is feasible. But in every case. the researcher deduces (from something else. The reason. Introduction to Mathematical Sociology (New York: The Free Press. To build theory deductively. s o m e prior e x p e c t a t i o n s ) w h a t sort of a pattern to e x p e c t a n d t h e n l o o k s for it a m o n g the observations. T h e p r o b l e m lies with that " s o m e t h i n g else" from w h i c h the student is supposed to d e d u c e theories. it is important to bear in mind the advantages of deduction. It is in this spirit that I h a v e tried to stress the "craft" in the "craft of political research. A h e a l t h y a w a r e n e s s o f m a j o r t h e o r i e s i n f i e l d s s u c h a s t h e s e . If we h a v e in m i n d a p a r t i c u l a r k i n d of pattern. b e c a u s e m o r e t h a n in m o s t d i s c i p l i n e s .e.e s t a b l i s h e d t h e o r y f r o m a n o t h e r field. G a l i l e o w a s able to i g n o r e the effects of air resistance b e c a u s e he k n e w that they w e r e not the thing he should use to explain the speed of falling objects.

and FREDERICK MOSTELLER. CAMPBELL. COOK. SIVERSON. New York: St. New York: Academic Press. GOODMAN." American Journal of Political Science. DAVID.." American Political Science Review.J. 1987." Journal of Politics. HUBERT M. 1977. N. 2007. ANTHONY. "The Fourth Face of Power. New Haven. DAHL. New York: Harper & Row. "The Concept of Property-Space in Social Research. 1985. Case Study Research: Principles and Practices. "The Effects of Personal Canvassing. HOWELL. Ideology and Discontent. Quasi-experimentation: Design and Analysis Issues for Field Setting. 26 (Spring). Washington. 9. 1954. 1984. .. JOHN P. 2000. September 2000. New York City. 1955." American Political Science Review." American Political Science Review. CAMPBELL. ed. BATALLA. . 2002. New York: Free Press.J. "Messenger or Message? Military Buildups and the Initiation of Conflict. CHRISTOPHER H. 12-25. "The Measurement and Meaning of Trust in Government. Beverly Hills. New York: John Wiley & Sons. Ann Arbor: University of Michigan Press. "Some Consequences of Measurement Error in Survey Data.. 1975. PETER. COLEMAN. ALAN S. LARRY M. GREEN. Upper Saddle River. pp. BARTON. Conn. DOWNS. 2004.: University of North Carolina Press." American Political Science Review.. BUTLER. 1969. eds. THOMAS J.. "Does Political Stability Hinder Economic Development?" Comparative Politics. 791-808. N. STEPHEN. BARBARA. HAMMOND. New York: Free Press. 435-464. 1969. HERBERT B. Conn. PAUL F. "An 'Intentional' Alternative in Public Opinion Research. PETERSON. New York: Cambridge University Press. "Politicization of the Electorate in France and the United States. STANLEY. Cross-Level Inference. 49 (December).J. "Common Misinterpretations of Significance Levels ACHEN. and JEAN KINGSTON. HILDEBRAND." American Political Science Review. and ROY PIERCE. 19. DAVID E. 49 (August). The American Voter. Theory Construction: Prom Verbal to Mathematical Formulations. MAURICE. 10. Chicago: University of Chicago Press. rev. Ohio. BRUNNER.C. 1-23. The Presidential Character: Predicting Performance in the White House. 1 (January).: Prentice Hall. and HOWARD ROSENTHAL.: Evidence from Randomized Field Trials.: Prentice Hall. 801-813. PETER. R. RANDOLPH M. 13. "Assessing Social Innovations: An Empirical Basis for Policy. 1962. Political Oppositions in Western Democracies. and W. 1966. eds. and GEORGES DUPEUX. An Economic Theory of Democracy. 1983.. THOMAS H. 947-953. DUVERGER. Social Statistics. pp. ALLEN L. Statistical Methods. JAMES D. 1974. Division and Cohesion in Democracy. "Of Time and Partisan Stability. Philadelphia: University of Pennsylvania Press. New York: Free Press. HARRY. Chicago: Rand McNally. N. 45^16. 1979. "Messages Received: The Political Impact of Media Exposure. PHILIP E.J. SNYDER. THOMAS D. WILLIAM G. 4th ed. 1981.2 6 1 . 2 0 6 . "Measuring Partisanship. EDWARDS." Paper presented at the annual meetings of the American Political Science Association. 267-285. . and JULIAN C. The Statistical Analysis of Quasi-experiments.. ECKSTEIN. . Pressure Group Politics. ALAN GERBER. New York: John Wiley & Sons. Introduction to Mathematical Sociology." Party Politics. SMITH. CAMPBELL.: Sage. The Presidential Difference: Leadership and Style from FDR to Clinton.C. Political Change in Britain. 2003. 21(2). 87 (June). 1987. Lumsdaine. 1957. BLALOCK. 96." In David Apter." Political Methodology. and others. 1993. JAVIER ORTIZ. 143—166. Calif. .. 1961. New York: Free Press. BACHRACH. RICHARD J. ASHER. BRUCE.: Princeton University Press. 48. New York: Holt. "The Nature of Belief Systems in Mass Publics. 653-664. 2nd ed. and CHARLES W. 2." Social Forces. Berkeley. 56 (December). Evaluation and Experiment. "School Vouchers and Academic Performance: Results from Three Randomized Field Trials" Journal of Policy Analysis and Management. 1964. 2nd ed. Restructuring of Social and Political Theory. DONALD. "Equal Votes. and DONALD T. LIGHT. 1973. 54 (November). New York: Science Editions. 1963. and Direct Mail on Voter Turnout: A Field Experiment. 1972. GILBERT. Lazarsfeld and Morris Rosenberg. 179-196. JAMES S." In Carl A. 1964. 18 (May). Modern Political Analysis. Bennett and Arthur A. 94 (September). 1966.: Yale University Press. 139-171. 21 (August). "Attitudes and the Survey Response. Princeton. D. BUENO DE MESQUITA. 1986. and DONALD STOKES. PATRICK J." Political Methodology. ANSOLABEHERE. STANLEY. D. DONALD T. 2000. Effects of School Vouchers in Dayton. FELDMAN." American Journal of Political Science. BARTELS. 1975.: Prentice Hall. 1995.C. 1992." Political Methodology. London: George Allen & Unwin. Calif. 171 1963. DIGESER. ..5 3 . Equal Money: Court-Ordered Redistricting and the Distribution of Public Expenditures in the American States.D. KRUSKAL." In Paul F. Causal Inferences in Nonexperimental Research. GOLDSMITH. 767-777'. JAMES D. 11. University of California-Los Angeles. Chicago: Rand McNally. 2002. Chapel Hill.. 469^185. 1969. 50. BARBER. eds. 2007. Experimental and Quasi-experimental Designs for Research. Analysis of Nominal Data. New York: McGraw-Hill. Upper Saddle River. "Inadvertent Sociological Theory. 1979. JOHN. and MORTON BARATZ. ALLAN H. ANGUS. 3rd ed. 1964. 4 0 . in Sociological Journals. 191-217. 1999. CARTER. "Test-Score "An Institutional Explanation of the Democratic Peace. LEWIS F. and STEVEN WELDON. DEAN. GREENSTEIN. 1968. 1966. and ALASTAIR SMITH." Ph.." American Political Science Review. DUGGAN. Martin's Press.. The Language of Social Research.. 1978. GREEN. "Essays on the Early History of Latin American Central Banking. ." Comparative Political Studies. 977-1007. CONVERSE. LAING. . 747-754. 1960. "Measures of Association for Cross Classifications. ARTHUR A. BERNSTEIN. American Behavioral Scientists.: University of California Press. 1984. 1977. 1962. DALTON. DAVID K. JAMES D. "The Two Faces of Power. DRAPER. DIEHL. Political Parties. GERBER. PHILLIPS SHIVELY. RONALD D. and H. 1993." Public Opinion Quarterly. GEDDES. .. N. pp. Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics. 1971." Journal of Politics. and WILLIAM H. Rinehart & Winston. N. "Studying Presidential Performance in Congress.Selected Bibliography CAMPBELL. LEO A. 93 (December). and PAUL E. dissertation in economics. Who Governs! New Haven. RUSSELL J. and JAMES M." Journal of the American Statistical Association. and ALAN GERBER. RICHARD J. 1218-1231. JR. Applied Regression Analysis. 211-244. and JANE M. 170 . MORROW. Telephone Calls. ROBERT. and DONALD P.: Yale University Press. "Partisanship and Party System Institutionalization. 341-354." American Sociologist. 69 (December). 39-193. and Washington. N. FRASER. Upper Saddle River. FRED I. . GERRING. 3 (February). WOLF. 1960.471^180.

Chicago: Rand McNally. "Institutionalization of Voting Patterns.. Migration? A Theoretical. OLSON. Baltimore: Johns Hopkins University Press. LUKES. "The Urban-Rural Cleavage in Political Involvement: The Case of France. 2006. Mathematical Applications in Political Science." American Political Science Review. 1968. "The Quality of Surveying Responses About Parents and Experimental Designs for Generalized Causal Inference.. 140-143. KOENIG.: Sage. S. IYENGAR. 1956. 1975. Bernd. eds. Cambridge. University of Minnesota. SARTORI." American Political Science Review. 1969. 1997. The Act of Creation." In John Geer. and THOMAS R. L.J. 2002. 2006. MOSTELLER. Chicago: University of Chicago Press. JAMES W. MILLER. Blalock. Management and the Worker. 193-210. W I N C H . . CAROLE JEAN. WATSON. Mass.U . 15 (June). 1974. pp. ed. V. STEVEN. 1998. THOMAS D. 344-347.. 1986. New York: Atheneum. MICHAEL E. Beverly Hills. rev. REYNOLDS. JOSE ANTONIO CHEIBUB.: Princeton University Press. TINBERGEN. BOHRNSTEDT.. The Military Balance. "Interesting but Irrelevant: Social Capital and the Saliency of Politics in Europe. Development: Political Institutions and Well-Being in the World. Vol. W. 641-654. ADAM. Mass. New York: McGraw-Hill. . DONALD R. Vol.: Princeton University Press. Mass. Vol. 1952. 2004. 1994. The Visual Display of Quantitative Information. 332-344. Elements of Econometrics. Handbook of Political Science. New Haven. and JAMES W. F. Experimental and Quasi- Ann Arbor. 1997." Journal of Politics. 1965. and others. 1968. MATTHEWS. 10. Mass. and M A R K ANDREAS KAYSER. London: Macmillan. KNOKE. "Interethnic Cooperation in Post-Communist Eastern Europe. 33. THOMAS S. 14. PHILIP E. 1. DAVID. ZALLER. N.8 3 . SIEGEL. SHANTO. 1971. ESPINOSA. J. Presidential Power and the Modern Presidents. C O O K . LEITNER. Unobtrusive the Family: A Longitudinal Analysis.. The American Soldier. NIKO. 1203-1225. PROTHRO. MASSEY. 2003. Mass. V A N D E T H . Calif. 18. The Structure of Scientific Revolutions. M O O N . 1997.: Addison-Wesley. "Proof? No. 1968-1969. 1985. TINGSTEN. KOESTLER.. 1977. 1993. Princeton.. KROEBER.. S . ROBERT O.. 1950-1990. 1969. Calif. 1996. CHRISTOPHER. PITKIN. RICHARD E. Making Democracy Work. JAMES R. 1939. Designing Social Inquiry: Scientific Inference in Qualitative Research. and ALISSA POTTER M E E . Cheshire. and M A R C L. PALFREY. Political Behavior. THOMAS P. WILLIAM R. and SIDNEY VERBA." In Joseph L. 1969. KUHN. DONALD T. 1971. 1972. GIOVANNI." American Journal of Sociology. pp. ed. DICKSON. New York: Free Press. Princeton. SHADISH.: Harvard University Press. Handbook of Political Science. "Case Selection: Insights from Rethinking Social Inquiry'. Introduction to Models in the Social Sciences. and DONALD R. Analysis of Nominal Data. and Policy Analysis. 4th ed.: The University of Virginia Press." Ph. 1949.. Berkeley. PROTHRO." In Hubert M. "Critique of Ordinal Variables. and FERNANDO LONGI. New York: Cambridge University Press. Evidence? Yes. Curious Naturalists." Foreign Policy. JAN. EUGENE J. A. 1975. Politicians and Party Politics.D. "Majoritarian Electoral Systems and Consumer Power: Price-Level Evidence from the OECD Countries. "ROTC. J. ALVAREZ. JULIAN. 4. 283-336. Ann Arbor: University of Michigan Press. ROBERT D. LOUIS W. Representation. Power: A Radical View. W.. 1956. Nonparametric Statistics for the Behavioral Sciences. RICHARD D. "Party Identification in Britain: Does Length of Time in the Electorate Affect Strength of Partisanship?" British Journal of Political Science." American Political Science Review. 4 1 . The Concept of Representation. . JAMES D. Southern Politics.1 1 2 7 . 1. The Rise and Decline of Nations. CAMPBELL. HANNA. 66 (December). PUTNAM. and JAMES G. OPPENHEIM. 135-161. EDWARD R. Charlottesville. ." American Journal of Political Science." Journal of Politics. FREDERICK. The Chief Executive. 389-398. H." In Fred Greenstein and Nelson Polsby. 1968. JOHN. Statistics for Social Data Analysis. 341-357. "A Multi-dimensional Classification of Atrocity Stories. 1991.. JEROME. ed. 526-539. 4 9 . CAMPBELL. 1967.: Graphics Press." In Paul F. RICHARD G. LAVE. O. "Party Identification. 1984. W E B B . eds. SCHWARTZ. New York: Free Press. 565-570. Calif. K E Y . Knopf. pp. "The Language of Political Inquiry: Problems of Clarification. 1985. ed. 5 4 . 1976. and DONALD T. or Is Mobilization the Source of Decay?" American Political Science Review. 4 (May). Boston: Houghton Mifflin. London: Hutchinson. and DONALD T . and CLYDE KLUCKHOHN. 1987. SIMON. "Ecological Correlations and the Behavior of Individuals. NEUSTADT. 2002. Va. ROETHLISBERGER. 57 (March). Empirical. 1 1 0 4 . 1. TARROW. 351-357. The Language of Social Research. PRZEWORSKI.: Doubleday. Democracy and Measures. UHLANER. "Candidate Gender and Congressional Campaign Receipts. 69 (March).' Political Analysis. Mylai and the Volunteer Army.: Harvard University Press.172 Selected Bibliography Selected Bibliography 173 INTERNATIONAL INSTITUTE FOR STRATEGIC STUDIES.: Addison-Wesley. 2006. KMENTA. Chicago: University of Chicago Press.. and W. M A R C H . New York: Alfred A. FELIX. GEORGE W. Beverly Hills. Party Choice. The Double Helix. 415^131. JR. Belmont. DONALD 1975. J. KINDER.5 7 .: Sage.. 102 (January).: University of Michigan Press. GARY. 48 (February). 2 (Spring). dissertation in political science. and KAY LEHMAN SCHLOZMAN. TUFTE. Mich. CHARLES. Basic Research Methods in Social Science. 1963. DOUGLAS S„ and KRISTIN E. Garden City. PHILLIPS. Vol. "What's Driving M e x i c o . 1962. Mass. 46. London: Cambridge University Press. Cambridge. ROBERT E . HERBERT. NIEMI. eds. 2002. JAN W. CAMPBELL. "Cross-Level Inference as a Game Against Nature. KING. Calif. Conn. 1983. PETER. New York: Harper & Row. "Politicians as Prize Fighters: Electoral Selection and Incumbency Advantage. 1975. Reliability and Validity in Qualitative Research. "Of Time and partisan Stability Revisited: Australia and New Zealand 1905-90. SIDNEY. ROGOWSKI. 1969. 1937. 1971. MANCUR.. Lazarsfeld and Morris Rosenberg. pp. "Improving Data Analysis in Political Science. 2 1 . 3rd ed. TILLEY. Cambridge." International Encyclopedia of the Social Sciences. WILSON. 90 (June). 65 (June). and L E E SECHRIST." World Politics. A. SIDNEY." Political Methodology." American Sociological Review. New York: Harcourt Brace Jovanovich. SAMUEL. 1966. and Voting Stability: The Weimar Case." In Fred Greenstein and Nelson Polsby. "The Nonsense Fight over Scientific Method: A Plea for Peace. 24ff. STOUFFER. News That Matters. 1993.Y.: Harvard University Press. KINDER. KARSTEN. 1950. N.6 7 . Reading.: Wadsworth. KEOHANE." American Journal of Political Science.: Yale University Press. N. 6 2 . 2000. Experimental Foundations of Political Science. MIHAELA. 2nd ed. DONALD R. KIRK. ROBINSON. DONALD E. "Making Social Science Work Across Space and Time: A Critical Reflection on Robert Putnam's Making Democracy Work" American Political Science Review. Culture: A Critical Review of Concepts and Definitions. Parties and Party Systems. expected completion 2005. "The Logic of Political Inquiry: A Synthesis of Opposed Perspectives. The Logic of Collective Action.: The Museum. "Errors. RONALD. 1982. and others. ARTHUR. pp. . 939-999. "Social and Economic Factors and Voter Registration in the South. New York: Atherton. eds. Cambridge. Chicago: Aldine-Atherton. . JACOB. SHIVELY. Conn. Causal Models in the Social Sciences." European Journal of Political Science. 30-50. New York: Random House. MIHAELESCU.J. 115-147. 1968. and KEITH KREHBIEL. London: Longmans. The Significance of Tests of Significance. Jr. STOKES. Oxford: Oxford University Press. 131-228. ." American Sociologist.: University of California Press. New York: Macmillan. Reading. T. 1955. 1950. 2000.

Russell J. 146n Bohrnstedt. 38 Diminishing marginal returns model. W.. Christopher H. 76 Bernstein. William G„ 87 Hypothesis testing. 96 . 161-162 Independent variable. 11-12 Experiment. 30 G o m n Leo A. 7 G Independence of observations. Hubert M„ 96. 83-84 Ex post facto argument. 5. Bruce. 101 Coefficient of determination. Paul E. John. Donald T. 46n.. Green. 7n Abstinence-only sex education. Donald P. 73 Case studies. 161-162 B Gerber. Mr Andreas. 20 Dean.. 132-134 E Eckstein. 40 Batalla. 31 Bueno de Mesquita. 136-137 in regression analysis. Michael E. Philip E. 145-146. 90-91 Interval measures. 137-138 Dickson. 54n Baratz. 76. James David. J. 56 Autocorrelation. 29-30 Ansolabehere. 145 Cook. 29 Bartels. V. 4 Engineering research. 80n Diehl. 45n Espinosa. Carr. 96 hms Correlational measures of relationship.. 122-127 compared with regression analysis. James. Harry. 119-120 Discrete measures. 80. 40 Alvarez. 52 Feldman. 164 hms Duverger. David E„ 87 Campbell. 69 Kayser. Birkbeck. 64 Downs. Herbert B. 5-8 om l Fraser. Kristin E. Peter. 94-95 Converse.. Allan H. 161n Key. 19 Keohane. 40 Anomalies. 151-153 I Bachrach. 63-64 Control group. Stanley... 56 F r a theory. Morton. Samuel. 104-105 Karsten. 121-122 Kinder. 94. 5-6 Duggan. 56.. 96. 164 Carter. 14. 14 Interaction. 8 Digeser.162 Boswell. 57 Ecological fallacy.. 16 Ethics of research. 16-17 Empirical research. Ronald D. Arthur A. 34 Epistemic correlation.. 164 Dependent variable. 15. 94-95 Howell. 40 Dalton. Angus.. 124 Coleman. 80n Hill. 110 Analytic political philosophy.Index 175 D Dahl. Charles W. 87. 17-20. true. John P. 79-81 Controlling for a variable.. 128n Effect-descriptive measures of relationship. 56 Free rider problem. Robert. 23 English language. T o a D„ 85.. 87n. 103-106 Cheibub.. Anthony. 85n. 62 Iyengar. 2-3. 131. Richard J. Peter. 40 Johnson. 47. 40.. 167n Holding a variable constant. L ry M. 78-84 Causation. 74-76 Censored data. 14 Dichotomies. James S. O.. 88. 29 H H m o d T o a H. 116 Interesting problems. Philip E. 94. 32-33. Lewis F.. G o m n a d Kruskal g m a 135 od a n a m. 49 F Face validity. 77-78 eliminating alternative.. T o a J. 23 Extrapolation. John. 88 J Jacob. 81 Achen... 87. 56 a mn . 54-55 Applied research. David. 131 ar Barton. 10 Interrupted time series. Fred I. George W. 38. Stephen. Donald R. statistical. Maurice. 54-55.. Lord. Jose Antonio. Shanto. h m s Hawthorne study. 110 Gilbert. Barbara. 127n. 96 Gerring.. 56. 26-27 Butler. 167-168 Geddes. Jr. Robert O. 20 174 Campbell. G. 47.. 40 Blalock. Alan S. 167 Brunner. 38 Barber. 103 Causal interpretations.. 113 Elegant research. 167 K Galileo. 128n Acton. 110.. 167-168 Continuous measures. Peter. 86 ak Kemeny. 135n od a . 87.. 126. 96 Greenstein.. 82 designs without. Javier Ortiz. 127-130 Correlation coefficient.. 38 Baker v. 96. 18-19 Campbell. 146n Intercept of regression equation. 4 Asher. 123 Cross-tabulation. 110 Cluster sample.. 87 Goldsmith. Jane M.. 113 Correlation analysis.

47 Theory. 56 Minneapolis Star Tribune. J. Richard D. 56 Kluckhohn. 16 Matthews. 6 R D D sampling. . James W. 61. 86 Unidimensional concepts. 130-131 Random sample. 83-84 Tufte. general problem of. . 99. 4 Nonrandom error. 19 Marx. Herbert. . 109-110 Logit analysis. Keith. 20. F. A. 119-122 Robinson. 54-55 Social research. 14-17 Theory-oriented research. Felix. Adam. 20. W. 2 Slope of regression equation. 101 National integration. 130-131 checking for. Niko. Frederick.. 155-158 Sartori. 130-131 Regression coefficient. 42^15 Measurement error.3 Social status. 94. Jerome. 62-67 in measures.4 2 Split-half test for reliability. 124. John Stuart. 126-127 Random error... 25-28 Index Selection on the dependent variable 106-108 Shadish. 149 Significance tests. 96 Shively. 96 Tingsten. Hanna. 139-141 Prothro. 81 Moon. 29. Christopher. 2. comparability in regression analysis. J . 7 3 o Olson. 56 Sechrist... 52 Tsst group. Donald R. 7 Political party. 31 Mosteller. 53-54. 47 Stanley. 152-153 T Tarrow. 22-23 Trivial theory. 60 L Lave. 110 s Sampling. 51-53 Variables. 20. 24. of a theory. 116 Snyder. 34-35 Multivariate regression. 67-71 in measurement. . Charles A . Arthur. 135n Kuhn.. 49. Gerhard. Jan. 15. 62 Nonempirical research. 38 P M March. Carole Jean. 40 Ordinal measures. Donald E . 87 Pierce.176 Index Multidimensional concepts. Sidney. 15 R r. Clyde. Ronald. 38-40 Precision. 8. 101-102 random. Paul E . Donald. 168 Q N N (number of cases). 65-67 enrichment of. 162 Mihaelescu. David.. Richard G . 49. 45^18 related to validity. 92-93 Puzzles. 29 Quantitative research. L . 165 Significance. 113-114 Schlozman. 96 Peterson. James R. 49-50 Research design. Sidney. 20 Liberal. 48^19. 99-102 purposive. 118-119 Niemi. 20 Tinbergen. . Mihaela. . 3 1 . 130-131 Mee. Gary. W. James M. . 5 Miller. 33 Light. . Ronald. Mancur. . 8 Massey. 162 Koenig. 58. 30 Oppenheim. 9 Reliability of measures. . 31 Krehbiel. . 101 Reagan. 85n Statistics. Samuel. Thomas S . L e e . . 29 Koestler. Julian. 62 127-130 and measurement error. 128n Roethlisberger. 14 . Fernando. 104 Mill. 56. 165 Simplicity. 5-8 Regression analysis. 78-84 Residuals. 56 Nominal measures. 82 Test-retest check for reliability. 86 Literary Digest. 163 Power. I28n 111 King. statistical. A l i s s a Potter. 71. 162 Knoke.. William H. 110. Giovanni. 8 Kirk. 80n Rogowski. K a r l . 123-127 r .5 Null hypothesis. . 15 True experiment. 87 Linear relationships. 117 Regression to the mean. 53-54 Normative philosophy. Thomas R„ 87n. 2 . 96 Topics for research. 6-7. 58-67 and describing relationships between variables. 4 . 87 Multicollinearity. 38 Kmenta. 158-161 National Election Study. 3. James G . 20-21 Siegel. 58-61 Probit analysis. 25 Natural experiment. 32 Units. 110n. K a y Lehman. Robert D. 26n L a w of Large Numbers. 110 Lukes. 23 Tilley. 32 Multidimensional words. 131. 73 Rational choice. 50-51 Loewenberg. 139-141 Longi. 51. 40 Pluralism.. 101 Sampling distribution. 128n Stouffer.5 2 Pitkin. S . 99-101 Ratio measures. 113-122 compared with correlation. 94. 33 Polling. 106 Leitner. 161 n Palfrey. 4. 82-83 without premeasurement. 9 Stratified sample. 26n Markov chain. Marc L . 141-146 Putnam. 56 Pilot study. 5 1 .. 131. 56 u Uhlaner. 40 Scattergram. 63n. 88-90 Relative deprivation. 4 1 . Jean. 37 N A T O . Phillips. 144n Przeworski. 99-101 R D D . 86 Schwartz. William R. Richard E . 56 Kroeber. 144n Measurement. Richard J . 162-164 Simon. 79 Recreational research. 26n. Julian C . Steven. 112 Stokes. Roy. 83 Neustadt. 117 V Validity. 161n Kingston. 102 quasi-random. 100. in regression. Edward R. Louis W. 38 Kruskal. Douglas S .

20 Wilson.. Steven. Thomas P. .. William Knowlton. 87 Z a l l e r . 98 Zinsser. 124-127 Verba. John. James D. 1 Webb. Eugene J .178 Index X X\ 153-155 Variance. 56 Weldon. . 73 Winch. 164 Wolf. 31 . Robert E. 110. 16In w Watson. Sidney. Patrick J .

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.