You are on page 1of 7

07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999

SIMPSON'S PARADOX AND CORNFIELD’S CONDITIONS
Milo Schield, Augsburg College
Dept. of Business & MIS. 2211 Riverside Drive. Minneapolis, MN 55454

Abstract
In another example, it was found that whites were more
Simpson's Paradox occurs when an observed associa-
likely to be sentenced to death for murder than blacks.
tion is spurious – reversed after taking into account a
But after taking into account the confounding factor of
confounding factor. At best, Simpson's Paradox is used
the race of the victim, it was found that blacks were
to argue that association is not causation. At worst,
more likely to be sentenced to death than whites. A
Simpson's Paradox is used to argue that induction is
death sentence was more likely if the victim was white.
impossible in observational studies (that all arguments
Since blacks were more likely to kill blacks, they were
from association to causation are equally suspect) since
less likely to be sentenced to death. But whether the
any association could possibly be reversed by some yet
victim was white or black, a death sentence was more
unknown confounding factor. This paper reviews Corn-
likely for blacks than for whites. (Agresti 1984)
field's conditions – the necessary conditions for Simp-
son's Paradox – and argues that a simple-difference
form of these conditions can be used to establish a 3. IS SIMPSON'S PARADOX IMPORTANT?
Simpson's Paradox is vitally important for several rea-
minimum effect size for any potential confounder.
sons. (1) It clearly demonstrates that correlation is not
Cornfield's minimum effect size is asserted to be a key
element in statistical literacy. In order to teach this always causation. If the direction of an association can
be reversed, any assertion about direct causation is
important concept, a graphical technique was developed
clearly disputable. (2) It demonstrates that associations
to illustrate percentage-point difference comparisons.
Some preliminary results of teaching these ideas in an are sometimes conditional. Students often think of nu-
merical associations as immutable —as unconditional.
introductory statistics course are presented.
By studying Simpson's Paradox students overcome this
Keywords: Statistical Literacy, Teaching; Epistemo l- mistaken perception. (3) It introduces the minimum
effect size necessary for a confounder to explain a spu-
ogy; Philosophy of science; Observational studies.
rious association. The measurement of the minimum
effect size is the point of this paper and is developed in
1. STATISTICAL LITERACY
a later section.
Statistical literacy studies the use of statistics and sta-
tistical associations as evidence in arguments (Schield,
4. UNDERSTANDING SIMPSON'S PARADOX
1998). Many arguments involving statistical associa-
tions are based on observational studies and are directed It is not easy to understand the reasons for—much less
the cause of—a reversal of an association, i.e., Simp-
at supporting claims on causation. Interpreting such
son's Paradox. Consider three types of explanations:
associations is a major problem due to the possibility of
confounding. Simpson's Paradox is a striking example Mathematical explanation: “Consider 8 variables: A, B,
of this problem. C, D, a, b, c and d. If it is true that A/B > a/b and C/D
> c/d, is it also true that (A+C)/(B+D) > (a+c)/(b+d)?”
2. WHAT IS SIMPSON'S PARADOX? The reply: “Not in general. For example, 1/1 > 3/4 and
Simpson's Paradox is the reversal of an association be- 1/4 > 0/1. Now (1+1)/(4+1) = 2/5 and (3+0)/(4+1) =
tween two variables after a third variable (a confound- 3/5. Now is 2/5 > 3/5? No!” (sci.stat.edu, 12/96). In
ing factor) is taken into account. For an overview of this explanation, the reversal is just a consequence of
association reversals, see Samuels (1993). A confound- the particular numbers involved.
ing factor is a factor — a lurking variable — which is
Group inhomogeneity explanation. Suppose A, B, ... d
found or mixed with another.
are as above. Let A/B = P1 and C/D = P3. Let a/b = P2
Simp son's paradox has been observed in several real- and c/d = P4. Let the size of the groups being com-
life situations. One well-known example occurred in pared be illustrated by the number of “x” symbols.
the Graduate Division of the University of California at
Berkeley. Women were rejected more often than men xx A/B=P1 > P2=a/b xxxxxxxxxx
xxxxxx C/D=P3 > P4=c/d xx
at the overall college level, but men were rejected more
often than women at the individual departmental level. The ⊕ symbol indicates the merging of the groups:
The confounding factor was the choice of department. P1 ⊕ P3 = (A+C)/(B+D)
Women were more likely to choose departments with P2 ⊕ P4 = (a+c)/(b+d)
higher rejection rates than were men (Freedman, Pisani, xxxxxxxx P1 ⊕ P3 < P2 ⊕ P4 xxxxxxxxxxxx
Purves, and Adhikari 1991, p. 16).

99ASA.doc Page 1 Milo Schield

Consider an example: considerable influence on smoking. relative to the prevalence among an observational study. relative to those After studying Simpson’s Paradox. so students need to learn how to deal cigarette smokers must be at least 9 times greater with associations based on observational studies. association assuming the association is totally spuri- stood. for those exposed to A. into account strongly influences our conclusions (Kelly “There can be little doubt that the genotype exercises 1994). They wrote (Cornfield et al. C lung cancer. The passengers immedi. When the father did nothing to control the cancer was not caused by smoking per se but was children some of the passengers became irritated. then the proportion of hormone-X-producers among ways possible. and this is ignore observational studies and deal only with ran. caused by that part of the genotype that also caused One of them asked the father to control the children people to smoke. not so exposed. A. Appendix A). The father lifted his head and explained that he whether they smoke or not. Textbooks seldom indicate a way to estimate disease. Students understand this principle: a more important factor can easily change one’s standard for evaluation. experiments are not al. However. He concluded. and on the particu- A father and his young children were riding on the lar habit of smoking adopted…” New York subway. one student con. They derived a necessary condition for a ately reversed their evaluation once they took account confounding factor to explain away an observed asso- of the influence of the death on the family. B. it is not easily anticipated. one on whether to smoke or on what (cigarette. with no causal effect upon the risk of a 1999). P de- notes a probability. shows an ap- parent risk. among cluded one should never trust any association based on those exposed to A. he presented actual data on smoking than Bantam (190 = P4). batting average than Bantam (330 = P2). his first half of the season. If an agent. A denotes the apparent cause. FROM CORRELATION TO CAUSATION ninefold. 7. if cigarette smokers have 9 times the risk of cur. lated the percentage of twins in which there were dis- ting average than Slugger if Bantam bats much more tinct differences in smoking (smoker versus non- than Slugger in the first half and much less than Slugger smoker or cigarette smoker versus pipe smoker). than that of non-smokers. His in the second half. Thus people who are disposed to (implying the father was derelict in his responsibili. smoke would contract lung cancer at the same rate ties). or sented using baseball batting averages. planation. determining whether an association is spurious (Pearl. because of a positive correla- the likelihood of a Simpson's Paradox reversal. One solution is to nonsmokers for developing lung cancer. If the relative prevalence of hormone-X-producers is considerably less than 6. this choices among fraternal and identical twins. In the second He did not just allude to the possibility of some con- half Slugger (200 = P3) had a higher batting average founding factor. Suppose in the cigar) to smoke. Bantam can have a higher bat. Cornfield et al (1959) countered Fisher’s alternate ex- their mother. but P1 ⊕ P3 < so it was possible that an unknown confounding factor P2 ⊕ P4: the Simpson’s Paradox reversal.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 We see above that P1 > P2 and P3 > P4. The children were out of control. and the children had left the hospital where his wife. then hormone X cannot account for the A serious concern about the possibility of Simpson’s magnitude of the apparent effect. CORNFIELD’S CONDITION Cornfield et al deduced the minimum effect size neces- 5. had just died. The differing size of the groups explains the reversal in Fisher (1958) noted that genetic factors might dispose the association. This explanation is sometimes pre. 1959. He calcu- association can reverse. There is no test for ous. nevertheless. might significantly change the associations. not because cigarette smoke is a causal agent." Paradox arose in the late 1950s when several research projects found an association between smoking and Cornfield's condition can be stated algebraically. then the prevalence of B. But these associations were observational denotes the common cause and E denotes an observable 99ASA. Thus. ticipate when a Simpson's Paradox reversal could oc. only because cigarette smokers produce hormone X. The father was slumped over with his head in his Fisher used this association to suggest that perhaps lung hands. ANTICIPATING SIMPSON'S PARADOX sary for a potential confounder to explain an observed But even if Simpson’s Paradox were readily under. data showed that there were distinct differences in smoking choice among 51% of the fraternal twins as Confounding factor explanation: What we fail to take opposed to 24% of the identical twins. must be greater than r. ing is minimized. Although Fisher was a smoker. Yet for the entire season. ciation—assuming the association was totally spurious. And if there is no way to an. but domized experiments where the problem of confound. pipe.doc Page 2 Milo Schield . r. those not so exposed. this student is absolutely right. tion with some other causal agent. Slugger (370 = P1) had a higher article demonstrated his allegiance to the power of data.

only those exceeding certain nec- essary conditions could be relevant.2 Regression Coefficients and the more the burden of proof was shifted onto those The influence of a confounding factor can be expressed arguing against causation. served associations. (Smoking) (lung cancer) servational studies as is the use of randomized assign. Thus P(C|A) is the vance. Cornfield et al derived the risk-difference condition in [Actually. need a hidden bias of a particular magnitude. to explain it is large. tween treatment and outcome does not imply proved that the prevalence of the actual cause (C) must causation. 1 Start with P(E|A) = P(E|C∩A) P(C|A) + P(E|C'∩ A) P(C'|A). Instead of saying that an association be- an observable effect E (lung cancer). Fisher had noted a 2 to 1 relative prevalence (51% vs. the real cause than among those without. to the best of their knowledge. But Cornfield's condition Since [P(C|A) . And prevalence of the real cause must be at least P(E|C'∩A) = P(E|C'∩A') = P(E|C'). It allowed statisticians P percentage points greater among smokers to conclude that.P(A). Cornfield et al. Their statement is an important conceptual ad- The vertical bar (|) denotes “given”. Rosenbaum (1995) said of Cornfield's condition: ment of the condition (A' = non-A) so P(A') = 1 . than among non-smokers. No longer could one re- (Bad Gene) C fute an ostensive causal association by simply asserting that some new factor (such as a genetic factor) might be If smoking has no effect on lung cancer the true cause. then the equality: P(E|C∩A) = P(E|C∩A') = P(E|C). While there might be many as a bias in the expected value of a regression coeffi- confounding factors. Necessary Relationship among Absolute Dif- ferences to Explain a Totally Spurious Association.doc Page 3 Milo Schield . P(E|A) = P(E|C) P(C|A) + P(E|C') P(C'|A) 1a This necessary prevalence—Cornfield’s condition— P(E|A') = P(E|C) P(C|A') + P(E|C') P(C'|A') 1b blunted Fisher's argument.P(C|A')] ≤ 1. This 1 gives: N times as great among smokers as non-smokers. The advance consists in replacing a gen- probability of C given A. servational studies by a quantitative statement that is specific to what is observed in a particular If factor A (smoking) had no effect on the likelihood of study. one would Prevalences to Explain a Totally Spurious Association. P(C|A)/P(C|A') – the relative prevalence of bad genes among smokers versus non-smokers. the hidden bias needed as great among smokers as among non-smokers. 8. Now one had to argue that the relative then the risk of lung cancer must be at least prevalence of this potentially confounding factor was P percentage points greater among those with greater than the relative risk for the ostensive cause. 99ASA. Figure 2.1 Cross-Rate Equality (Bad Gene) A sufficient condition for “no effect” is cross-A rate If smoking has no effect on lung cancer. Apparent Observed Spurious Cause Effect Association Cornfield's minimum effect size is as important to ob. P(E|A') = P(E|C∩A') P(C|A') + P(E|C'∩ A') P(C'|A'). If Suppose the prevalence of lung cancer is N times the association is strong. Fisher's comparison was of the form (1c) but dismis sed it saying it “leads to no useful con- P(A|C)/P(A|C') – the relative prevalence of smokers clusion. The necessary condition of Cornfield et al is the posi. The higher the relative risk in the observed association.P(E|A')] 1d non-smokers. Necessary Relationship among Relative association seen in a particular study. METHOD OF DIFFERENCES Association The minimum effect size can also be a simple difference (Smoking) (lung cancer) of two percentages. The A "Real Cause" E ment to experimental studies. they say that to explain the Figure 1. smo k. ing caused cancer – based on observational studies. Apparent Observed Cause Spurious Effect 8. 24%) in smoking behavior P(E|A)-P(E|A') = [P(E|C)-P(E|C')][P(C|A)-P(C|A')] 1c for the two types of twins. P(C|A') is the probability of C eral qualitative statement that applies in all ob- given the absence of A. required that Fisher show the prevalence of his genetic factor was nine times as great among smokers as among [P(E|C) . Fisher never replied. the stronger the argument in favor of direct causation. A single quote following a letter is the comple.” This paper argues that this risk -difference among bad genes versus good genes -. Suppose the risk of lung cancer is tive side of Simpson's Paradox. that hidden biases can explain ob- satisfy: P(C|A)/P(C|A') > P(E|A)/P(E|A'). Consider three approaches: A The E "Real Cause" C 8.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 effect.P(E|C')] ≥ [P(E|A) .instead of condition is extremely useful (see Section 9).

(Friedman 1994. Let b3 be the coefficient regressing C on A. 420). then the association between A and E. we obtain (1c). we get: then one should be concerned about the possibility of a b 2 = P(E|C∩A) . Although the partial correlation coefficient field’s nullification.P(E|C'∩A) = P(E|C) . the change in ex.P(C|A') [P(E|C) . let the uncontrolled coefficient regressing When the variables involved are binary.[rAC rCE ]}/ √[(1-r2 AC) (1-r2 CE)] 3a is not sufficient to imply a Simpson’s Paradox rever- If the apparent association between A and E (rAE ) is sal—it is only a necessary condition.C) x φ (C. the Pearson E on A be b0 . Equation (1d) gives a very simple method for determin- ent influence due to a failure to take account of the con. they have absolute values no greater than 1.P(E|C') Simpson's Paradox reversal. The “indi. The death rate is 3% of cases at the city hos- rAE = rAC rCE 3b pital versus 2% at the rural. If all the variables are binary. b0 = b2 × b3. In the It follows that r| AC| and |rCE | must each be at least as case of three variables: A. This estimated change in E based as summarized by (1c) and (1d). is zero (rAE. p. C and E. b 3 = P(C|A) . Consider two hospitals: a city hospital and a rural hospital.P(E|C')]√{[P(C)P(C')]/[P(E)P(E')]}} b 0 = b 1 + (b 2 × b 3 ). the “whole effect”. Since these slopes are differences in probabilities. This relationship is well known. as shown in (1d). of E for a one unit change in A will be b0 . p. approach is more theoretical.P(E|A'). For the past three years students in introductory statis- 8. the expected change large as |rAE |. 8. ing whether a third variable (C) has the strength – the founding factor.differences The influence of a confounding factor can be expressed in percentage points -. Thus we can 10.P(C|A')]√{[P(A)P(A')]/[P(C)P(C')]}} 3d If we fail to include C. powers of two binary variables. a common cause (C). then the regression slope Students need only compare two simple differences coefficients are the difference in the associated percent. “For a in the response variable E given a change in A can be confounding variable to explain an association of a biased whenever one ignores the influence of a con. let b2 be the coefficient involving C. Students have entirely spurious and is due entirely to associations with used these ideas as follows. 415).in comparing the explanatory using partial correlation. rect effect” is the product of b2 and b3 . 2b The difference between the uncontrolled effect (b 0 ) and 9. it can be shown to meas- tained earlier in (1d). 1.A) = φ (E. 2a {[P(C|A) .doc Page 4 Milo Schield . the change in the expected value which reduces to (1c). the ure the strength of association without knowing the direct association is completely spurious and prevalence of C in A. the whole effect. both involving E. TEACHING SIMPSON'S PARADOX deduce that b2 ≥ b 0 . the indirect effect.P(E|A')] 1d Assuming A has “no effect” on E. To illustrate. establishes a minimum effect size for any confounding factor to nullify or reverse an observed association. The regression approach is b 3 . EXPLANATORY POWER the direct effect (b1 ) can be viewed as bias—an appar. we can obtain the same result ob. Thus. measured in percentage points. most powerful since it can be generalized to multiple confounding factors (Wonnacott and Wonnacott 1979. If.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 cient (Wonnacott and Wonnacott 1990.3 Partial Correlation Coefficients tics were taught to use simple differences -. This bias is the product of two slope with both the possible causal factor and the disease” coefficients. The cross-rate equal- on the whole effect will be biased by the amount of b 2 × ity approach is simplest. Students were cau- tioned that the truth of the percentage-point difference rAE. given strength. φ (E.C = 0).4 Comparison of Approaches pected value of E for a one-unit change in A should be All three “difference” approaches give the same result b 1 . 210 and 214). If C is a confounding factor.A). Let b1 be the coefficient involving A (the φ (E.C)=[P(E|C)-P(E|C')]√[P(C)P(C')]/[P(E)P(E')] 3c “direct effect”). the direct effect. The combined death rate is 99ASA.P(E|A')] √{[P(A)P(A')]/[P(E)P(E')]} Wonnacott show that the whole effect (b 0 ) is the sum of the direct effect (b 1 ) and the indirect effect (b 2 x b3 ): = {[P(E|C) . effect size – necessary to nullify or reverse an observed association between two other variables (A and E). This simple requirement If b 0 = b 2 × b 3 . When regressing E on correlation coefficient reduces to phi ( φ ): A and controlling for C. In relating this regression coefficient approach to Corn.C = {rAE . ages: b 0 = P(E|A) .P(E|C')] ≥ [P(E|A) . Thus. Under (3b). p. conditioned on C. With no direct effect (b 1 = 0). there are two coefficients. it must have a much stronger association founding factor C. Wonnacott and [P(E|A) .

However.3% Here the simple difference in death rates by patient 67. 3. With six kinds of subscriptions. p.of a confounding factor.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 2.2% better captures the spirit of being true in one sense. But if we eliminate all types of subscriptions ex- Death Rates cept the two largest groups. Death sentence rates metic as conditional. Students need extensive reinforcement to see that associations can be conditional. Renewal Rates 78. See Figure 4. the cause is difficult to Figure 3.6 Pct. whereas the concept of ‘spurious’ 10. 11. Students found this graphical device (Figures 3.6 percentage points) is greater than the Feb 67. 93).2%. Cryer and Miller discuss renewal rates of magazine way of evaluating an association.9% in January City Poor 3. the type of subscription.0% Race of as “spurious” rather than as “biased”.0% Overall and 67. when the sentences are classified by differences. The difference in renewal rates 2. See Figure 3.5 simple difference in death rates by hospital (1 percent.4% of the base line. Subscription 47.4%. Pts 1 Pct.the ex- take into account the race of the victim when studying planatory power -. They don't see arith- Figure 4.4% authenticity. Furthermore.3% while that for subscrip- tion agents was 20.6%.0% by type of subscription (67. 4 sentences by race of murderer (1.0% of the cases with a white victim and 5.Pt. Based on this data. In a group of convicted murderers. a graphical technique was developed. The two-month renewal rate 2.9% of white murderers and 10.9% Overall cerned about a possible Simpson's Paradox reversal of Agent the association between hospital and death rate. The the race of the victim.2% By Hospital By Patient Condition month (19. P(E). ing students about the Cornfield conditions: alty was given for 11. The overall rate was 47.9% ciation reversal better when describing the association 1. it seems that the rural hospital is safer than newal rate was increased between January and Febru- the city hospital. Knowing an association can be unbiased (true) but still be spurious gives them a more powerful 3. Thus to understand the month-to-month difference.4 percentage points). was used as the 14. Students seem to understand the problem of asso- 11.1% Regular condition (2. ary. so why should statistics be differ- Death Sentence ent.8% Figure 5. the death pen. To better illustrate these whites. Type of Jan 53.6 percentage ally evident (See Figures 3.6 Pct. To Month 20. Students found the algebraic form of Cornfield's one might argue that the legal system is biased against condition to be unintuitive. but Race of Black not in another.7 PctPt 19. ‘Spurious’ indicates 'real' but lacking in Murderer 5. It seems to be a simple To guard against a Simpson's Paradox reversal we must and sensible way to measure the importance -. points) is greater than the difference in rate of death 2. 4 and 5). we find the overall renewal 3.8% rate was 53.4% age point). RESULTS OF TEACHING Following are some observations of the results of teach- 2. Renewal rates by month and subscription while that among patients in good condition is 1. we must take into account Now consider a plausible confounding factor: the con. Thus we have strong reason to be con.7 percentage points) is Good Rural much greater than the difference in renewal rates by 1. and 5) to be visually intuitive.1% in February.5% of black murderers (Agresti 1984).5 percentage points). In one year the overall re- 99ASA.7% for regular renewal was 78. The four probabilities being compared were cases with a black victim. the association between the death penalty and the race of the mu rderer. Yet the renewal rate in every category went down. 2.7%. Rates 14. know- subscriptions (1991. Thus. death rate among patients in poor condition is 3. 1.doc Page 5 Milo Schield .Pts 11.6% guard against such a reversal we can take into account (control for) patient condition when comparing the death rates for these two hospitals.7 PctPt White 8.0% White 4. We find that overall the a time difference is susceptible to Simpson's Paradox. The difference in the rate of grouped so that percentage point differences were vis u- death sentences by race of victim (8. This example shows that even dition of the patient’s health. The concept of Black Overall Victim ‘bias’ implies error. Death rates by hospital and patient condition see. the death penalty was given in overall probability of the effect.

there are multiple confounding factors. tralia at the University of Newcastle and at the Statisti- cal Society of Australia. suggested the use of conditional probability to of the necessary condition. NSW. able to appreciate the distinction between total and 173-203.html. pends on the goal. Simpson's Paradox is unimportant. University of Western Ontario in Canada. Thomas H. teachers often skip it. Introductory Statistics. Jon and Robert Miller (1991). second hand smoke). In observational studies.. the use of regression to analyze bias due to an omitted sible). sary condition (minimum effect size) for a reversal of a provided David Lewis's graphical explanation of Simp- spurious association is the key to proper understanding son's Paradox while Bob Gibberd. Institut für Informatik I in Data. 2. Neil Thomason. Without proper understanding castle. 99ASA. independently spotted an unjustified claim. Thus Cornfield’s conditions are a most impor.edu/ppages/~schield. and Ansgar Grüne. In Australia. There are too many other statistical concepts that Phenomena. Norton & Co. Reply: Fundamentality de.cs. David (1994). 1993. 88. p.W. Tom Wonna- doorway to subjectivis m (i. When present. W. Myra (1993). Statistical Literacy and Eviden- tal if the goal is to reason deductively about statistical inference. E. This paper is based on talks given in Aus- tant statistical contribution to human thought. and Ronald J. ciation. 5 th ed. Letter to the Editor. these Kelley. it is Friedman.. M. Hammond.augsburg. Nether- Agresti. but the observa- tional studies most susceptible to Simpson's Paradox Rosenbaum. There is no known statistical test for For Confounding. Wonnacott. John Wiley. Wonnacott. 1984. McGraw Hill if in the text. Milo (1998). March. 3. Robert Pisani. W.f. Nature. POSSIBLE OBJECTIONS Fisher.e. Cornfield's Inequality. Simpson's Paradox is Pearl. Jan Hajek.. 1. Cryer. but is most fundamental if the goal is to rea. Bonn. partial correlations in multivariate regression. A. Simpson's Paradox can be a obtain a simple difference comparison. Journal of the National Cancer Institute. Observational Studies. real statistical studies. Wonnacott (1990). Reply: True. partial correlation and multivariate regression. This paper is at www. University of Melbourne. and Wynder. Norton & Co. 12. They are more concerned about what is taken into account. Primer Of Epidemiology 4th sometimes just a problem or an optional section.. And Why confounding. derstand Simpson's Paradox. J. cott.. ASA Proceedings of the Section on Sta- tistical Education. Students who are trained in this way seem better questions.. Paul R. Shimkin. Understanding the neces. and Ronald J. PWS-Kent. Why Many Think There Is. Reply: True but with Cornfields' min i- They Are Almost Right. E.ucla. Encyclo- are often the studies used as evidence for important pedia of Biostatistics. Donald Macnaughton REFERENCES made suggestions on earlier drafts. used as an introduction to multiple regression. to Cornfield's essay in his talk at the 1998 ASA JSM.. Lilienfeld. tial Statistics. 81. p. Reply: Yes. P. they must un. Statistics 2nd ed.. A. This single-factor emphasis should be (1979).W. Simpson's Paradox is not fundamen- Schield. if an argument is not de. (1959). Smoking and lung cancer: Recent evidence and a discussion of some 5. John Wiley.edu/jp_home. but they are not arguments. founders and thus strengthen an inductive argument.. noted ductively valid. 22. The author can be reached at schield@augsburg. confounder. W. Roger Purves and Ani Adhikari. CONCLUSION If students are to understand proper inductive reasoning Acknowledgments: Paul Rosenbaum called attention about causality in observational studies. Haenszel.edu. Judea (1999). It is omitted from many introductory texts. Following are some arguments against featuring Simp- son's Paradox in teaching introductory statistics: Freedman. Why There Is No Statistical Test always possible. University of Ne w- of Simpson's Paradox. Ronald (1958). Analysis of Ordinal Categorical lands. Samuels. Typically. Gary (1994). nothing is certain and anything is pos. 137. John Wiley 13. Simpson's Paradox and Related 4. Statistics for Business: Data Analysis and Modelling. are signs of unimportance. son inductively from association to direct causation. (1995). Technical Report R-256. Rosenbaum. mum effect-size conditions we can eliminate many con- http://singapore. David. Simpson's Paradox is seldom encountered in doing Springer-Verlag.doc Page 6 Milo Schield . changes in public policy (c. The Art of Reasoning. Wonnacott 5. Reply: True. Journal of the American Statistical Asso- are more fundamental. 2nd ed. 2nd ed.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 ing an association can be spurious sets the stage for Cornfield. Paul R. Even ed. Thomas H. Econometrics..

the second factor is less than pendix B). r21. let the proportion exposed to B be p1 . Thus. then = R2 r22 [1+{[p/(1-p)](r21 / r22 )}] the apparent relative risk of developing disease I on exposure to A will be less than that for disease II (Ap. by either agent is small (in a sense to be defined). but the less restrictive specification. The first justification for use of the relative meas. If. is the same whether A is present or absent. while the absolute denominator. r. R1 . Let the disease rate for those exposed to diseases by the magnitude of the effect will depend on the causal agent B. From (1) we obtain ure can be stated more precisely. because of a positive correlation with Since p 1 > p2 and R1 /R2 > 1. R1 r12 [1+{[p/(1-p)](r11 / r12 )}] ease I. but not disease II. also has agent B present. but only be.p 2 . parent relative risk for A. be r1 and for those not exposed.r2 )(p 1 . The absolute measure would be important in appraising the public health significance of an effect (1) R1 /R2 = {p 1 r1 +(1-p 1 )r2 } / {p 2 r1 +(1-p 2 )r2 } known to be causal. the relationship. Then same effect. r12 . and ing the possible noncausal nature of an agent having an of those not exposed to A. then Since p 1 /p 2 is the ratio of the prevalence of B among the prevalence of B. and if dis. those exposed to A. nevertheless. the ranking of the justification. for right is positive and p 1 /p 2 > R1 /R2 . A. the greater than r. R1 = rate for those exposed to A = pr11 +(1-p)r12 The presence of other real causes thus reduces the ap.” If a causal agent A increases the risk for disease I and has no effect on the risk for disease II. Of those ex- purpose. then the relative risk of [Editorial comment: In Appendix A. and this. with no causal effect upon the risk of a dis- ease. as follows: p 1 /p 2 = R1 /R2 + [r2 /(p 2 r1 )] [(1-p 2 )(R1 /R2 )-(1-p 1 )] If an agent. and canceling r1 . if the absolute difference.R2 . and 3) properly reflecting the effects of R1 = rate for those exposed to A = p 1 r1 +(1-p 1 )r2 disease misclassification or further refinement of classi- R2 = rate for those not exposed to A = p 2 r1 +(1-p 2 )r2 fication.p2 ). among those exposed to A. Appendix A We feel obliged to give proof of the rather obvious Measures of Difference: statement on the magnitudes of relative risk because it “…we now discuss the use of relative and absolute has been suggested that the use of a relative measure- measures of differences in risk. is less than the risk for A in the Then absence of B. 99ASA. statement is proved. r.07/ 23/ 03 Updated Simpson's Paradox and Cornfield's Conditions ASA-JSM 1999 Appendix I: Quotes from Cornfield et al (1959). proportion of hormone-X-producers among cigarette Appendix B smokers must be at least 9 times greater than that of non-smokers. A. If the relative prevalence of hormone-X. used. and R1 /R2 is the apparent relative risk. absence of A and presence of B.doc Page 7 Milo Schield . Since p 1 > p 2 and r1 > r2 . and this is not be. r. (R1 . Because of the assumed positive correlation agent with respect to other possible agents inducing the between A and B. alone. the risk in the The second reason for using a relative measure absence of both A and B. unity and (R1 /R2 ) < (r12/r22 ) which proves the proposi- The third reason for using a relative measure is: tion. the relative risk of and R1 /R2 = apparent relative risk developing either disease I or disease II on exposure to A is the same in the absence of other causes. When an agent has an ment is merely “instinctive” and lacking in rational apparent effect on several diseases.R2 ) = (r1 . The relative measure is helpful in 1) apprais- posed to A. is greater than the relative risk ing (1) is obtained by multiplying (1) by the right-hand of developing disease I and II combined. then the to no useful conclusion about p1 . let the proportion exposed to apparent effect. is cause cigarette smoke is a causal agent. r2 . R2 = rate for those not exposed to A = pr21 +(1-p)r22 parent relative risk. Let r1 > r2 . for example. if cigarette smokers have 9 times the risk of non- smokers for developing lung cancer. then hor. hypothesis. the present of A and absence of B. the result follow- developing disease I. whether an absolute or a relative measure is used. On the other hand. 2) appraising the importance of an B be p 2 . and if the risk of the disease in the absence of population exposed to B is denoted by p. A and B. The proof again is simple. … each rate being unaffected by exposure or nonexposure Both the absolute and the relative measures serve a to the noncausal agent. It is reasonable to assume r22 may be phrased as follows: = 0. and r22 .] measure is unaffected. Since r22 /r21 < r12/r11 . relative to those not so exposed. Let r11 denote the risk of the producers is considerably less than ninefold. disease in the presence of both A and B. p 1 > p 2 . each increase the risk r11 is sufficient for what follows. r22 < r12 r21 / If two uncorrelated agents. B. shows an apparent risk. must be exposed. the risk in the ent effect (Appendix A). then the ap. the risk in mone X cannot account for the magnitude of the appar. dividing by p 2 r1 . The proportion of the of a disease. leads cause cigarette smokers produce hormone X. relative to those exposed to A relative to that among those not so the prevalence among those not so exposed. it follows that R1 /R2 > 1. the second term on the some other causal agent.