11.1 Introduction “se Uses of the chi-square test Function of analysis of variance {inferences about population variances 568° Ghapter.11 7.779. The data are not well described by a normal distribution with p. and g = 15. Jeff is wrong. 114 Analysis of Variance : Earlier in this chapter, we used the chi-square test to examine the differences among more than two sample proportions and to make inferences about whether suth samples are drawn from populations each having the same proportion. In this section, we will learn a technique known as analysis of variance (often abbreviated ANOVA) that will enable us to test for the significance of the differences among more than two sample means. Using analysis of variance, we will be able Yo make inferences about whether our samples are drawn from. populations having the same mean, ‘Analysis of variance is useful in such situations as comparing the mileage achieved by five different brands of gasoline, testing which of four different training methods produces the fastest learning record, or comparing the first-year earnings of the graduates of half a dozen different business schools. In each of these cases, we would compare the means of more than two samples. Statement of the Problem In the training director's problem that opened this chapter, she wanted to evaluate three dif- ferent training methods to determine whether there weré any differences in effectiveness. ‘After completion of the training period, the company’s statistical staff chose 16 new em- ployees assigned at random to the three training methods.* Counting the production output by these 16 trainces, the staff has summarized the data and calculated the mean production of the trainees (see Table 11-12). Now if we wish to determine the grand mean, or ¥ (the mean for the entire group of 16 trainees), we can use one of two methods: is 15 +18 +19 +22 +11 + 22427 +18 +21 + 17+ 18424419 + 16422415 L t= 16 Sut 16 = 19 + Grand mean using al te + akhough neal practice, 16 trainees would not constiue an adequate statistical sample, we bave limited the num berhere wo beable to demonstrate the basic echniques of analysis of varance and to avoid tedious calculations, Ss “analysis of Varance 591 Scanned with CamScanner‘Stating the problem symbolically Anterpretng the results Tar | Method 2 Method 3 perecrcry Dally Production Cauley Enon) 2 F = G/16)(17) + /16)(21) + (6/16)(19) 304 16 = 19 © Gandmeanas a veptd aerage ofthe sample means, wg ta ate angles te ier Statement of the Hypotheses In this case, our reason for using analysis of variance is to decide whether thes tree sam. ples (a sample is the small group of employees trained by any one method) were dum from populations (a population isthe total number of employees who could be tained y that method) having the same means. Because we are testing the effectiveness ofthe thee training methods, we must determine whether the three samples, represented by the samle means 3, = 17, % = 21, and % = 19, could have been drawn from populations baving te same mean, j1, A formal statement of the null and alternative hypotheses we wish totst | would be 4 Ho: Ba = pa = pig Nl hypothesis Hy! Hy Ha and pis are riot all equal © Abematve pats | Tf we can conclude from our test that the sample means do not differ significantly, we can infer that the choice of training method does not influence the productivity of the em- ployee. On the other hand, if we find differences among the sample means that are too ge to attribute to chance sampling error, we can infer that the method used in training doesia- fluence the productivity of the employee. In that case, we would adjust our trainiig gram accordingly. Analysis of Variance: Basic Concepts Inorder to se analysis of variance, we must assume that each ofthe samples is dawn fo anormal population and that each of these populations has the same variance, 0”, Hower if te sample sizes are large enough, we do not need the assumption of normaly. In our training-methods problems, our null hypothesis states that the three Pop4 abe 4 have the same mean. If this hypothesis is true, classifying the data into three columns ia 11-12 is unnecessary and the entire set of 16 measurements of productivity can be ‘a5 a sample from one population. This overall population also has a variance of 0° (Chi-Square and Analysis of Varlance Scanned with CamScannerAnalysis of va lance is based on a comparison of two different estimates of the vari- ance, *", af our overall population. In this ease, we can calculate one of these estimates by examining the variance among the three sample means, which are 17, 21, and 19. The other estimate of the population variance is determined by the variation within the three samples themselves, that is, (15, 18, 19, 22, 11), (22, 27, 18, 21, 17), and (18, 24, 19, 16, 22, 15). Then we compare these two estimates of the population variance. Because both are estimates of 0”, they should be approximately equal in value when the null hypothesis is true. If the null hypothesis is nor true, these two estimates will differ considerably. The three steps in analysis of variance, then, are poy ovorince 1. Determine one estimate of the population variance from the variance among oe the sample means, 2, Determine a second estimate of the population variance from the variance within the samples, 3, Compare these two estimates. If they are approximately equal in value, accept the null hypothesis. In the remainder of this section, we shall learn how to calculate these two estimates of the population variance, how to compare these two estimates, and how to perform a hy- pothesis test and iriterpret the results. As we learn how to do these computations, however, keep in mind that all are based on the above three steps. Calculating the Variance among the Sample Means feéatefctesinte ofthe Step 1 in analysis of variance indicates that we must obtain one estimate of the population variance from the variance among the three sample means. In statistical language, this es- timate is called the between-column variance. Tn Chapter 3, we used Equation 3-17 to calculate the sample variance: 2@-xP at wa Sample vaiance —> 5? = ‘in be vaiane ameng: Now, because we are working with three sample means and a grand mean, let’s substitute ¥ for, X for, and k (the number of samples) for n to geta formula for the variance among the sample means: . Sie pan tone Next, we can retum for a moment to Chapter 6, where we defined the standard error /*tine anongsanple of the mean as the standard deviation of all possible samples of a given size. The for- tuula to derive the standard errot of the tein 1s Equation 6-1: : 11,40 Analysis of Varianes” 5932 Scanned with CamScanner“Which sample sie to use 594: “chapter 14 Poulton sand Standard eof he mean ttn isan etn a posible songl met on Sven soles) Srl sang sey We can simplify this equation by cross-multiplying the terms and then squari onder to change the population standard deviation, o, into the population vasa P 203 xn 7 ‘Standard error squared lity} Stic rane gh caret aap ne i For our training-method problem, we do not have all the information We need tous. ‘equation to find o*. Specifically, we donot knowo3. We could, however, calclae te ance among the three sample means, s3, using Equation 11-4. So Why not subsite de o2 in Equation 11-5 and calculate an estimate of the population variance? This wl ge There is a sli . the sample size, but which sample size should we use when the different samples haved ferent sizes? We solve this problem with Equation 11-6, where each ( ~)* ism by its own appropriate nj.‘ Fstestimae of the ppubtionariance where © 62 = our first estimate of ‘the population variance based on the among the sample means (the between-column varianie) ny = size of the jth sample 3 = sample mean of the jth samp! %=grandmean k = number of samples ‘Now we can use Equation 11-6 and the data from Table 11-12 to calculate the PANS column variance, Table 11-13 shows how to do these calculations. ‘Chi-Square and Analysis of Variance, Scanned with CamScanner| LSSOnRE) Se ty 7-19=-2 (yas peeerecinny Mose a ed Nas 19-19= 0 or =0 Calculating the Variance within the Samples sing te secondestinate ofthe Step 2 in ANOVA requires a second estimate of the population variance based on the vari- opuition vriance Ua a te iteration ox disposal ance within the samples. In statistical terms, this can be called the within-column variance. (Our employee-training problem has three samples of five or six items each. We can calcu- late the variance within each of these three samples using Equation 3-17: Ba - B17] n-1 Sarge weince ——> $? = ‘Because we have assumed that the variances of our three populations are the same, we could use any one of the three sample variances (s7 or s3 or 53) as the second estimate of the population variance. Statistically, we can get a better estimate of the population vari- ance by using a weighted average of all three samiplé Vatiances: The general formula for this second estimate of o* is ‘Second estinat of ‘ the polation variance ——> 63, = >| © 62 = our second estimate of the population variance based on the variances within the samples (the within-column dariance) size of the jth sample on, a ©. s? = sample variance of the jth sample © & =number of samples © nz = %n,= total sample size ‘This formula uses all the information we have at our disposal, not. just a portion of it, Had there been seven samples instead of three, we would have taken a weighted average of all seven, The weights used in Equation 11-7 will be explained shortly. Table 11-14 illustrates how to caleutate tis second estimate ofthe population variance using the variances within. all three of our samples. Wie 114 Analysis ofVariance 595 Scanned with CamScannerTraining Method 1 Training Method 2 Sample Mean: = 17 Sample Mean: = 21 Scio ps Ae sea 1 wre 6 (or 36 2 area 18-21 = ~3 ("9 19-19= 0 5 Gea N-ws 0 r= 0 =~ 198-3 T= 6-9 = 36 W-2= 4 (A119 3 Lu-H 6215 ~ 19-4 Be-H = 70 Bia ant a 5-4 62 Sample variance -9 53 = 155 Sample variance — 3}= 129 sf = (4/1397) + /12KI5S) + 6/1342.) ina = 132 second estimate ofthe population 13 variance based on the vaiance within ‘= 14.769 © the samples the within-clumn variance) The F Hypothesis Test: Computing and Interpreting the F Statistic Step 3 in ANOVA compares these two estimates of the popillation variance by computing their ratio, called F, as follows: Finding the F ratio first estimate of the population variance * p= based on the variance among the sample means second estimate of the population variance based on the variances within the samples ural If we subsite the statistical shorthand Yor the numerator and denominator of ts Equation 11-8 becomes _, between-column variance _ 0% Twithin-column variance 8! Fw between-column variance’ coh: within-column:- variance: 62, Scanned with CamScannergue Fratio il FRE 11-8 ‘he davtutions (ist value fiteses equals number ‘eso reedom inthe ea the rai second Having found this F ratio of 1,354, how can vie interpret it? First, examine the dé nominator, which is hased on the variance within the samples, The denominator is a 2004 estimator of « (the population variance) whether the null hypothesis is true or not. What about the numerator? IF the null hypothesis that the three methods of training have equal effects is true, then the numerator, or the variation among the sample means of the three methods, is also a good estimate of a? (the population variance). As a result, the de~ Hominator and numerator should be about equal If the null hypothesis fs trae, The nearer the F ratio comes to 1, then the more we are inclined to accept the null hypothe sis, Conversely, as the F ratio becomes larger, we will be more inclined to reject the null hypothesis and accept the alternative (that a difference does exist in the effects of the three training methods), Shortly, we shail learn a more formal way of deciding when to accept or reject the null hypothesis. But even now, you should understand the basic logic behind the F statistic. ‘When populations are not the same, the between-column variance (which was derived from the variance among the sample means) tends to be larger than the within-col- umn variance (which was derived from the variances within the samples), and the value of F tends to be large. This leads us to reject the null hypothesis. The F Bistribution Like other statistics we have studied, ifthe null hypothesis is tue, then the F statistic has a particular sampling distribution. Like the t and chi-square distributions, this F distribution is actually a whole family of distributions, three of which are shown in Figure 11-8. Notice that each is identified by a pair of degrees of freedom, unlike the ¢ and chi-square distribu- tions, which have only one value for the number of degrees of freedom. The first number is the number of degrees of freedom in the numerator of the F ratio; the second is the degrees of freedom in the denominator. ‘As we can seé in Figure 11-8, the F distribution has a single mode. The specific shape of an F distribution depends on the number of degrees of freedom in both the numerator and the denominator of the F ratio, But, in general, the F distribution is skewed to the right and tends to become more symmetrical as the numbers of degrees of freedom in the nu- merator and denominator increase. es a (65) degrees of reedom (2.1 degrees of teedom 11.4 Analysis of Variance’. 597 Scanned with CamScannerCalaoting Sores of treo Finding the rumeratr degrees, ot eedom Fed th denoricate ates ot treedom 598° chapter11 Using the F Distribution: Degrees of een . P as a pair of degrees of freedom, one forthe ny, Av av i oe enna ow can we alesse First, think about the numerator, the between-column variance, pantie 11-13, we three valves of ~ ¥, one foreach sample, to calculate 2i(% ~ *)” Once we knew ty of these x ~ x values, the third was automatically determined and could not be freely spec. fom is lost when we calculate the between-column vari ified, Thus, one degree of freed er sostecanb vt degrees of freedom for the numerator of the F ratio is always one feye, than the number of samples. The rule, then, is Ti Mra Dearest of Freedom Number of degrees of freedom (number of samples ~ 1 inthe numerator of the F ratio ~ ° ) (11-19) Now, what of the denominator? Look at Table 11-14 for a moment. There we calculated the variances within the samples, and we used all three samples. For the jth sample, we used ny values of (x ~ ¥) to calculate the X(x ~ ¥)° for that sample. Once we knew all but one of these (x ~ ¥) values, the last was automatically determined and could not be freely spee- ified.’Thus, we lost 1 degree of freedom in the calculations for each sample, leaving us with 4,4, and 5 degrees of freedom in the samples. Because we had three samples, we were left with 4 +4 +5 = 13 degrees of freedom (which could also be calculated as 5 + 5 +6— 3 = 13). We can state the rule like this: ae en eter aria Number of degrees of freedom in the denominator of the F ratio. ~~ ) = my — k (tt) Freedom aes where © ny = size of the jth sample © k =number of samples © ny = Xn, = total sample size Now we can see thatthe weight assigned tos? in Equat aa tion 11-7 on p, $95 was justi fra tion of the total number of degrees of freedom in the denominator af the F ratio. Using the F Table. To do F hypothesis tests, we shall use an F table in whic the columns represent the nz ber of ege of freedom for the numerator andthe rows sopresat the egres of freedom sg nominator. Separate abe exist for each level of significance. : ut one et ting a hyporbess atthe 0.01 level of significance, using the Fit b depres freedom ate 8 for the numerator and 11 forthe denominator. In this instance, tum to Appendix Table 6(b). In the body of that table, the approprisl® CChi’Square and Analysis of Varlance Scanned with CamScanneree fe 8 and 11 degrees of freedom is 4.74, If our calculated sample value ot Fibcests table value of 4.74, we Would reject the null hypothesis, If not, we would accept it. Testing the Hypothesis oe TarysGtt our hypothed thatthe thre different training methods prodace identical » Using the material \ repens , "a ry 2 how we calculated the Fea ‘Wave developed to this point. Let's begin by reviewin: first estitate of the population variance Fs base onthe variance among the sample means fs} ‘second estimate of the population variance ¥ based on the variances within the samples 20 14.769 Crass (C1354 rasa Next, calculate the number of degrees of freedom in the numerator of the F ratio, using ‘Equation 11-10 as follows: 7 Number of degrees of freedom in the numerator of the F ratio' = (number of samples — 1) {11-40) =3-1 = 2& pages fedomin the amex And we can calculate the number of degrees of freedom in the denominator of the F ratio by use of Equation 11-11: Number of degrees of freedom : in the denominator of the F ratio = £(nj-1) = np—k et =6-D+6-D+6-1 =44445 = 13} vaguest eto ate eon ‘Suppose the director of training wants to test at the 0.05 level the hypothesis that there are no differences among the three training methods. We can look in Appendix Table 6(a) for 2 de- ‘grees of freedom in the numerator and 13 in the denominator. The value we find there is 3.81. Figure 11-9 shows this hypothesis test graphically. The colored region represents the level of significance, The table value of 3.81 sets the upper limit of the acceptance region. Becaiuse the ‘calculated sample value for F of 1.354 lies within the acceptance region, we would accept the null hypothesis and conclude that, according to the sample information we have, there are no significant differences in the effect ofthe three training methods on employee productivity. ~ Precautions about Using the F Test ‘As we stated earlier, our sample sizes in this problem are too small for us to be able to draw valid inferences about the effectiveness of the various training methods. We chose small sam- sles so that we could explain the logic of analysis of variance without tedious calculations. In tal practice, our methodology would be'the sume, but our samples would be larger. 114) Analysis of Variance; 599 Scanned with CamScannerFIGURE 11-9 is test at the 0.05 level ‘of significance, using the F disti= ‘bution and showing the accep- tance region and the sample Frvalue 381 -eample, we have assumed the absence of many factors that might hae oe 3 ce We accepted as given, for example, the fact that all the new employs’ sampled had the same demonstrated aptitude for learning, which may or may sore We assumed that all the instructors ofthe three training methods had the same teach and to manage, which may not be true. And we assumed that the company’, cal staf collected the data on productivity during work periods that were similar of time of day, day of the week, time of the year, and so on. To be able to makes decisions based on analysis of variance, we need to be certain that all these factors ax, fectively controlled. ‘tes for one factor only Finally, notice that we have discussed only one-way, or one-factor, analysis of ra Our problem examined the effect ofthe type of training method on employee pr nothing else. Had we wished to measure the effect of two factors, such as the tra gram and the age of the employee, we would need the ability to use two-way: variance, a statistical method best saved for more advanced textbooks. i Using the Computer for Analysis of Variance ‘ ‘Once again, let us repeat that we used small sample sizes in our ANOVA examples could explain the logic of the method without getting bogged down in tedious cal ‘Using SAS for ANOVA For a realistic problem, it would be very convenient to use the ANOVA routines that [-————- Acceptance region ———* } ‘Aces thr hypoesis te ample vaeis P8009 found in all the commonly used statistical packages. So that you can compare one 0 ‘with the analysis that we did by hand, Figure 11-10 gives the output when SASi analyze the data in our training-method problem. i Let’s look first at the column of SAS’s ANOVA table headed “MEAN SQUARE"! row labeled “MODEL,” this column contains the value 20.000, which we recogniz: between-column variance we calculated in Table 11-13. In the row labeled “ERROR find the value 14.769, which is the within-column variance we calculated in Table UE Notice also the column headed “DF” (meaning degrees of freedom). It tells us tht MODEL MEAN SQUARE (the between-column variance) has 2 degrees of fedom the ERROR MEAN SQUARE (the within-column variance) has 13 degrees of fe The last line of the output gives the value of the F statistic, F = 1.35, and the PO 0.2923, which s the probability of geting an F statistic as large or ager than 13514 true. Because the prob value is larger than our significance level of a = 0.05, WO fat wo cannet ert On the basis of the sample evidence, these three training ™=" » do.not appear to have significantly different effects on employee productivity. + Figure 11-11 illustrates ihe use Minti topertrm esac tes. Maia OS _ sentially the same ANOVA table, along with n, %, and s for the three samples. ‘Chi Square and Analysis of Variance Scanned with CamScannerFIGURE 11-10 a from SAS forthe oneeaning problem FIGURE 11-11 Wiiab output forthe enplee training problem ‘Grn about student complaints por ciUSTRATING THE USE OF SAS FOR ANOVA BS PRODUCTIVITY DEPEND ON TRAINING MBTHOD? ANALYSTS OF VARIANCE PROCEDURE DEPENDENT VARIABLE: UNET@INTTS PRODUCED BY TRAINEE SOURCE DF SUM OF SQUARES MEAN SQUARE ee 2” 49.00000000 —20.00000000 BRROR 13 192.00000000 14,76923077 CORRECTED TOTAL 15 232.00000000 MODEL F = 1.35 PR > F = 0.2923 One-Way Analysis of Variance: Analysis of Variance Source DF ss Ms F P Factor 2 40.0 20.0 1.35 0.292 Error 23 192.0 14.800 = Total 15 232.0 Level N Mean stDev METHOD 1 5 17.000 4.183 METHOD 2 5 21.000 3.937 METHOD 3 6 ~—-19..000 3.464 Pooled StDev = 3.843 Now that we've seen how to interpret the SAS and Minitab output, let’s analyze a much more realistic example. Despite the conclusion we reached on pages 489-491 that students in the faculty-taught sections of our statistics course didn't do significantly bet- ter on the final exam than students who were taught by teaching assistants, we still re- ceived complaints. “I'm in Mr. Jackson's class, and my friends in Professor Rubin's class are learning. much more than I am” was typical of what we were hearing. We began to wonder whether perhaps there were significant differences among the individual sections of the course, even if the teaching assistants as a group were not significantly different from the faculty as a group. ‘We used the Minitab ANOVA procedure to check this out. The formal statement of our hypotheses was Hy: Alll six ps are the same (no difference among sections) Hi: The six ps are not the same (sections differ significantly) ‘The output from this analysis is shown in Figure 11-12, The calculated value of the F statis- ticiis 1.75, and the probability of observing such a large value of F if Ho is true (the prob value for this test) is 0:126. With such a large prob value, we must accept H and conclude that there were no significant differences inthe six sections’ performances on the final exam. S50) ght 4 ot Analysis of Variance 601 Scanned with CamScannerFIGURE 11-12 ‘Minitab output for ANOVA, cof final exam scores Using ANOVA on the earings data Antetpreting the resus FIGURE 11-13 {Using Minitab for ANOVA ‘of eamings changes 602;; chapter 4 sd One-Way Analysis of Variance Analysis of Variance on FINAL Source ss i P SECTION 5 859.4 “75 0,428 Error 193 °7218996.8 Total 198 19656.2 vel N Mean stDev a 27 45.742 10.679 2 46. 44.761 11.900 3 37 49,081 7.365 4 26 0 44,923 8.064 5 36 © 44.333 -10.373 6 22. 921 Pooled StDev = Let's use Minitab to do an analysis of Variance on the earnings data in Appendix Tae 11. Recall that on pages 491-492, we used Minitab to test whether the year-to-yearchange, in ast quarter earings for New York Stock Exchange stocks had a different mean ha ge ‘year-to-year changes in last-quarter earnings for American Stock Exchange and overte. ‘counter stocks, We grouped the latter two together because at that point we didn’t yet ka how to compare more than two means. Now let’s look at all three groups separately a analysis of variance to see whether the three means differ significantly. He: Hore = Mase = Huvse (no differences by exchange) H,: the p’s are not equal (exchanges differ significantly) ‘We used the ONEWAY command in Minitab to perform this analysis of variance. Recall that the MARKET variable is 1 for OTC, 2 for ASE, and 3 for NYSE. The results are given in Figure 11-13. The calculated value of the F statistic is 0.88 and the prob value fortes One-Way Analysis of Variance Analysis of Variance on CHANGE ‘Source DF ss Ms F P MARKET 2 ‘2.421 0.712 0.88 -0.415 Error 221, 177.906 0.805 Total 223° 179.327 , Level, N Mean StDev A. 1121 -0.1120 0.5195 2 38 0.0876 0.9121 3°) 75: 0.0156 "1.2598 : Pooled StDev'= “0.8972. ‘Chi-Square and Analysts of Varlance Scanned with CamScannerels of significance (w = 0.10, 0,05, 0.01, and so on), we cannot reject Ho; we conclude that the mean values of year-to-year changes in last-quarter earnings on the three exchanges are | ing our hypotheses is 0.415. Because this prob value is larger than all of our customary lev- not significantly different from cach other. —_—_—_—————————— 15 The focus of analysis of variance is to test whether three or more samples have WN RSSUMPTIONS been drawn from populations having the same mean. Analysis of variance is im- portant in research such as the evaluation of new drugs, where we need to exam- ine the effects of dose, frequency of medication, effects of other drugs, and patient differences in a single study. Analysis of variance compares two estimates of the | population variance. One estimate comes from the variance among the sample means, the other from the variance within the samples themselves. If they are ap- | proximately equal, the chances are high that the samples came from the same pop- | ulation. Warning: It’s vital not to abandon common sense when interpreting re- sults, While it may be true that a study can identify differences in brand preferences for instant coffee that apply to coffee purchases made on weekday mornings, it’s hard to say what a coffee company should do with this information. c Exercises 11.4 SC 11-5 A study compared the effects of four 1-month point-of-purchase pro- . motions on sales. The unit sales for five stores using all four promotions in different months follow. ‘ ’ Free sample 8 7 a 8s ‘One-pack gift “84 1 7 90 88 4 Cents off 3 B 69 B 6 Refund by mail 9 3 3 2 a (a) Compute the mean unit sales for each promotion and then deter- mine the grand mean. (b) Estimate the population variance using the between-column vari- ance (Equation 11-6). (©). Estimate the population variance using the within-column vari- ance computed from the variance within the samples. (d) Calculate the F ratio, At the 0.01 level of significance, do the pro- motions produce different effects on sales? C1146 Aresearch company has designed three different systems to clean up oi spills, The following table contains the results, measured by how much surface area (in square meters) is cleared in 1 hour. The data were found by testing each method in several trials. Are the three systems equally effective? Use the 0.08 level of signifiéance SystemA 58. 6 7 8 5689) 85, sytemB S78 Bg Systeme SR 37 11.4 Analysis of Variance = 603 Scanned with CamScanner