Psychological Bulletin

1080. Vol. 87, No. 1, 166-184

Analysis of Variance and the "Second Discipline"

of Scientific Psychology: A Historical Account
Anthony J. Rucci and Ryan D. Tweney
Bowling Green State University

A historical analysis of the period from 1925 to 1950 was conducted to inves-
tigate the incorporation of analysis of variance (ANOVA) techniques in psycho-
logical research. In addition to attempts to identify the earliest uses of ANOVA
in psychology, the gradual incorporation of the technique was examined by
counting its appearance in 6,457 articles appearing in major American psycho-
logical journals from 1935 through 1952. Expository articles and texts directed
at psychologists were also identified, and a questionnaire survey of graduate
psychology programs was conducted to establish how psychologists were intro-
duced to ANOVA. Finally, the phylogeny of major contributors to American psy-
chological statistics was established for the period. The data suggest a segmen-
tation into three historical periods: (a) an initial, expository phase lasting until
the onset of World War II, (b) a wartime interregnum during which use of
ANOVA declined, and (c) a postwar resurgence, characterized by the institu-
tionalization of ANOVA training. Although ANOVA certainly had a major impact
on experimental psychology, the data do not permit the conclusion that the
incorporation of ANOVA in psychology in itself constituted a revolutionary
"paradigm shift."

In 1957 Cronbach, in a presidential ad-
dress to the American Psychological Associa-
tion, described two disciplines of scientific
psychology: the correlational and the experi-
mental. He characterized the experimental
school by its reliance on analysis of variance
(ANOVA) techniques for research design, in
contrast to the correlational techniques fa-
vored by the other school. Referring to the
English statistician who developed ANOVA,
The authors would like to acknowledge contribu-
tions in the form of personal communications from
Harold Bechtoldt, Isadore Blumen, Alphonse Cha-
panis, C. J. Cranny, Lee Cronbach, Allen L. Ed-
wards, Nathaniel Gage, Bert Green, J. P. Guilford,
Harold Gulliksen, Lloyd Humphreys, E. F. Lind-
quist, Quinn McNemar, Don Ragusa, T. A. Ryan,
Patricia C. Smith, and Ray Tucker.
A brief version of this article was presented at
the annual convention of the American Psychologi-
cal Association in Toronto, Ontario, Canada, August
1978.
Requests for reprints should be sent to Anthony
J. Rucci, who is now at the American Hospital
Supply Corporation, 1 American Plaza, Evanston,
Illinois 60201.
Cronbach asserted that "Fisher made the
experimentalist an expert puppeteer, able to
keep untangled the strands to half-a-dozen
independent variables (p. 675)."
Although it may be superfluous to reassert
that the ANOVA has been the workhorse sta-
tistic of experimental psychology, it is not
superfluous to consider how this technique
carne to dominate experimental design in psy-
chology. Not until the late 1930s did ANOVA
begin to appear in psychological journals, and
yet by 1952 it was fully established as the
most frequently used technique in experi-
mental research. It took less than 15 years
for psychology to incorporate ANOVA.
The purpose of the present study was to
develop a historical account of psychology's
incorporation of ANOVA from Fisher's initial
presentation of the technique (1925b)
through the early 1950s. It became clear in
the course of this research that experimental
design in psychology changed radically in the
period in question: Single-variable designs
were being used less frequently from 1940
on, whereas factorial designs were used more

Copyright 1980 by the American Psychological Association, Inc. 0033-2909/80/8701-0166S00.75

166
 ANALYSIS OF VARIANCE
frequently. The temptation, therefore, is to
conclude that variance techniques caused a
profound change in the way psychologists
approached research. It is, of course, impossi-
ble to assess causal relations in this type of
historical investigation. Although it might
appear that variance techniques had sub-
stantial impact on psychology, it is just as
plausible that the discipline had developed in
such a way that the time was ripe for the
adoption of the technique. Did ANOVA affect
psychological design, or did psychological
design affect the adoption of ANOVA? The
question is unanswerable, given our data. It
is clear, however, that changes in psychologi-
cal design were concomitant with the incorpo-
ration of variance techniques. A characteri-
zation of this incorporation, rather than an
assessment of causality, was the goal of this
investigation.
Five approaches were taken to trace psy-
chology's incorporation of variance tech-
niques: (a) identification of early published
uses and expositions of the technique, (b) a
frequency count to trace the growth in use
of the technique from 193S on, (c) a review
of early statistical research on variance tech-
niques done by psychologists, (d) identifica-
tion of the earliest behavioral science texts
to include the technique, and (e) a question-
naire survey of graduate curricula. The results
are presented following a brief summary of
the statistical development of ANOVA, a topic
that has been thoroughly discussed by others
(e.g., Hotelling, 1951; Johnson, 1943; Ma-
halanobis, 1938; Stanley, 1966; Yates, 1951).
Statistical Development of ANOVA
Fisher pointed out in 1935 that
statistical procedure and experimental design are
only two different aspects of the same whole, and
that whole is the logical requirements of the com-
plete process of adding to natural knowledge by
experimentation, (p. 3)
Although Fisher's (1925b, 1935) two texts
are the seminal works on ANOVA, statistical
developments leading to the technique go as
far back as Gauss who, as early as 1826, de-
scribed the concepts of degrees of freedom
and least squares (cited in Hotelling, 1943,
167
1951; and in Walker, 1940). In 1876, an
astronomer named Helmert had mathemati-
cally derived the exact distribution of the
sum of squared deviations of a normal variate
about the population mean (cited in Wilks,
1932). Prior to 1950 developments in design
were overwhelmingly centered in agricultural
research. Boussingault began field experimen-
tation as early as 1834, and in 1841 Lawes
established in England what would become
the nucleus of research on experimental de-
sign—the Rothamsted Experimental Station.
Longitudinal research at Rothamsted from
1843 to the present day has had a profound
influence on modern experimental design and
analysis. It was there that Fisher developed
the ANOVA technique in the 1920s.
Gossett, writing under the pseudonym of
Student, was the first to suggest the form of
the distribution of the sum of squared devia-
tions about a sample mean (Gossett, 1908).
Student's work was of immediate relevance to
the development of ANOVA. Fisher (1915)
rigorously developed the sampling distribu-
tion for the t statistic and showed its appli-
cation to experimental data (Fisher, 1925a).
Fisher's work led to the development of the
r-to-2 statistic, which is equal to one half of
the difference between the natural logarithms
of two estimated variances: z = -}|log (1 —
/•) — log (1 + > ' ) ] • When r in this equation is
equal to the intraclass correlation, z provides
a test statistic for the difference between two
independent variance estimates. In Fisher's
1925(b) presentation of ANOVA technique,
this variance difference logarithm was tabled
for use in determining the significance of
difference between two variances. It was not
until 1934 that Snedecor tabled mean square
ratios in the now common F tables. Snedecor
chose to call his tabled values F ratios in
commemoration of Fisher. It is interesting to
note that through all later editions of the
1925(b) text (the last edition was 1970)
Fisher never included F tables but rather re-
tained only the tabled r-lo-z values.
One must be cautious in citing the first
presentation of any concept in research litera-
ture. Although most writers cite Fisher's
1925(b) text as the original reference to
ANOVA, two articles referred to the technique
in 1923. Fisher and MacKenzie (1923) sub-
168 ANTHONY J. RUCCI AND RYAN D. TWENEY
divided the total variance of the results of an
experiment to test whether different varieties
of potatoes responded differently to potash
fertilizers. The subcomponents of variance for
differences between manures and differences
between varieties were tested for significance
against a variance due to variations between
plots. These data were reanalyzed in the
1925(b) text because, according to Yates
(1964), Fisher realized the need to partition
the error term due to the arrangement of
plots in the original design.
The second 1923 reference to ANOVA was an
article by Student (Gossett, 1923), on testing
varieties of cereals. Student quoted a letter
that he received from Fisher regarding the
breakdown of variances for trials and varie-
ties. Fisher presented a table in this letter
that is nearly identical to the form in which
ANOVA results are reported today.
The 1925(b) text, Statistical Methods for
Research Workers, was the first didactic pre-
sentation of ANOVA, It was not a compre-
hensive presentation, however, and in fact
only two chapters out of eight were directly
related to ANOVA. The remainder of the text
was devoted to a discussion of the scope and
characteristics of statistics, the Poisson dis-
tribution, chi-square, and the t statistic. Be-
sides the presentation of ANOVA, the fourth
edition of this text (Fisher, 1932) included
Fisher's first presentation of the analysis of
covariance (ANCOVA).
Soon after the initial presentation of ANOVA,
Fisher became acutely aware of the relation-
ship between design and analysis (Fisher,
1926). The 1935 text, The Design oj Experi-
ments, was the culmination of the interfacing
of design and analysis. This was the first
comprehensive presentation of factorial design
and analysis. The first 113 pages of this text
are an elegant presentation of the concepts
of null hypothesis testing, randomization, and
inductive inference.
Fisher's contributions to statistics were
numerous, and although some colleagues
claimed that his work was not new in a revo-
lutionary sense, he clearly exerted a profound
influence on contemporary experimental de-
sign and analysis:
It might be stated that experiment design as known
today, had its beginning with Sir Ronald A. Fisher
and began in the second quarter of the twentieth
century. (Federer & Balaam, 1973, p. 40)
Early Uses of ANOVA in Psychology
Our first approach was to trace the devel-
opment of variance techniques in psychology
by identifying early published articles in be-
havioral research that used ANOVA or proposed
it as a design and analysis alternative. An
extensive review was made of all located
articles that used ANOVA prior to 1940. In ad-
dition, a review was made of all located arti-
cles through 1950 that recommended the use
of variance techniques in psychology or dem-
onstrated their application to psychological
research problems.
Table 1 lists 17 published studies in psy-
chological research that used ANOVA prior to
1940. These studies were identified using
personal communications from Cronbach
(Note 1) and a systematic survey of approxi-
mately 50 pre-1940 journals. Although some
pre-1940 studies may not have been discov-
ered, Table 1 should contain the majority of
published psychological research articles to
use ANOVA prior to 1940.
A number of similar articles appear in
Table 1. Three used covariance analysis
(Dressel, 1939; Gaskill & Cox, 1937; Snede-
cor, 1935), 2 investigated memorization of
piano music (Rubin-Rabson, 1937, 1939), 2
studied differences in alpha rhythms between
normal and abnormal individuals (Kreezer,
1939; Rubin, 1938), 3 applied ANOVA to
evaluating validity of test items and test
reliability (Baker, 1939; Jackson, 1939; Lev,
1938), and 4 were concerned with the quanti-
tative study of trance personalities (Caring-
ton, 1934, 1936, 1937; Thouless, 1937). Of
the 17 articles listed, 16 involved research
primarily relating to or using human subjects.
Only 1 study (Crutchfield, 1939) used ANOVA
in research with animals. Twelve of the arti-
cles were in psychological publications, 4
appeared in educational journals, and 1 ap-
peared in a statistical journal.
The earliest behavioral research to use
ANOVA was reported by Reitz (1934) in the
Journal of Experimental Education. Reitz
presented the formulas for computing a z
value, noted the assumption of homogeneity
 ANALYSIS OF VARIANCE 169

Table 1
Psychological Articles Using Analysis of Variance Prior to 1940

Author Source Description Design

Reitz (1934) Journal of Experimental Tested grade differences One-way ANOVA

Education between college classes
Carington Proceedings of the Society for Psychical research on One-way ANOVA
(1934) Psychical Research trances
Snedecor Journal of the American Covariance applied to One-way ANOVA
(1935) Statistical A ssociation college math grades
Carington Proceedings of the Society for Psychical research on One-way ANOVA
(1936) Psychical Research trances
Carington Proceedings of the Society for Psychical research on One-way ANOVA
(1937) Psychical Research trances
Gaskill & Cox Journal of General Covariance applied to One-way ANOVA
(1937) Psychology respiration rates
Rubin- Rabson Archives of Psychology Investigated memorization Factorial
(1937) of piano music (4X4X4X4)
Thouless Proceedings of the Society for Psychical research on Factorial
(1937) Psychical Research trances
Rubin (1938) Journal of Psychology Studied alpha rhythms, Factorial
normal vs. schizophrenic
Lev (1938) Journal of Educational Evaluated test-item One-way ANOVA
Psychology validity
Gottsdanker Journal of Experimental Studied student test Factorial (5 X 5)
(1939) Psychology response patterns
Baker (1939) Psychological Record Evaluated test-item One-way ANOVA
validity
Rubin-Rabson Journal of Educational Investigated memorization Factorial
(1939) Psychology of piano music (4X4X4X4)
Kreezer American Journal of Studied alpha rhythms, One-way ANOVA
(1939) Psychology normals vs. Mongoloid
Dressel Journal of Educational Covariance applied to One-way ANOVA
(1939) Psychology high school grades
Crutchfield Journal of Psychology Studied string pulling in Factorial
(1939) rats (3X3X3X3X3)
Jackson British Journal of Psychology Evaluated test reliability One-way ANOVA
(1939)

Note. ANOVA = analysis of variance.

of variance, and checked his computed z
value by outlining the relationship between
the correlation ratio (rj-) and z. Given that
only Fisher's editions of the 1925(b) text
were available to Reitz, this was a remarkably
competent use of the technique.
Four of the earliest articles to use ANOVA
were in research on trance personalities
(Carington, 1934, 1936, 1937; Thouless,
1937) through the London-based Society for
Psychical Research. After these four very
early articles, however, use of variance tech-
niques completely disappeared from the jour-
nal.
Although Snedecor's (1935) covariance
analysis of grades appeared in a statistical
journal, we have included it here since it
was a direct application of covariance analy-
sis to a behavioral research problem. This
article represents one of the earliest uses of
applied variance techniques. Smith, the head
of the mathematics department at Iowa State,
wanted to determine if different grading prac-
tices by teachers were responsible for dif-
ferences between mathematics classes or if the
different grades reflected different levels of
achievement. Snedecor applied covariance
analysis to the problem and tested for class
differences on a common exam.
A wide range of statistical sophistication
was demonstrated by these early articles. The
Snedecor (193S) and Gaskill and Cox (1937)
articles were complex statistical applications,
particularly in view of the newness of the
170 ANTHONY J. RUCCI AND RYAN D. TWENEY
techniques. Even more striking, however, was
the extent to which the authors of these
early articles acknowledged the new technique
being used. Reitz (1934), Gaskill and Cox
( 1 9 3 7 ) , and Thouless (1937), for example,
made a point of telling the reader that the
statistical technique being used was both new
and particularly appropriate for research
problems in psychology. On the other hand,
Rubin-Rabson (1939) did not acknowledge
the newness of variance analysis, and Rubin
(1938) cited no statistical text or article rele-
vant to the statistical treatment used!
Although the pre-1940 articles were pio-
neering applications of ANOVA, some of these
authors displayed uncertainty regarding the
technique. Kreezer (1939) provides the best
example of this uneasiness. After pointing out
that F was equal to t~, Kreezer analyzed his
data using both t and F, saying, "Both meth-
ods have, nevertheless, been used in this study
so that they may provide a check on each
other'' (p. 5 2 2 ) .
A systematic survey of psychological jour-
nals through 19SO turned up 66 articles that
were identified as expository, written with
the stated intention of presenting a new sta-
tistical technique to be applied to psychologi-
cal research. These included articles that en-
dorsed ANOVA for psychological research and
also those that demonstrated its application
to psychological research problems.
Three articles, Crutchfield (1938), Baxter
(1941), and Garrett and Zubin (1943), were
major efforts toward "psychologizing" the
ANOVA technique. In 1938, Crutchfield il-
lustrated the application of factorial design to
a study of the perseverative performance of
rats in a string-pulling apparatus. Only the
layout of the research design, in accordance
with Fisher's technique, was reported, not the
results of the study. Crutchfield was un-
equivocal in his endorsement of factorial
designs:
Whenever, in experimental, comparative, or social
psychology, a systematic investigation of the pri-
mary effects and the interacting effects of a number
of experimentally controllable factors is being con-
ducted, the principles of efficient factorial design
can be invoked with inestimable benefit, (p. 341)
Contrast this quote by Crutchfield with that
of Peters and VanVoorhis (1940) in a chap-
ter entitled "The Technique of Controlled
Experimentation":
ll seems scarcely necessary for us to say here that
care must be exercised to keep all other conditions
constant in the two situations except the one experi-
mental factor; this is the law of the single variable,
so fundamental to all scientific experimentation.
(p. 44V)
The pervasiveness of the single-variable law
in behavioral research was felt well into the
late 1930s and early 1940s.
Baxter (1941) noted that Fisher's methods
had only recently begun to appear in psycho-
logical journals, but these methods frequently
appeared in journals of agriculture and biol-
ogy. Baxter's article outlined the analogy
between the agricultural terminology associ-
ated with Fisher's method and psychological
terminology. The fundamental analogy pro-
posed by Baxter was
Treatment: Soil: :
Experimental variable: Organism.
Further, he stated that "treatment corre-
sponds to the traditional term independent
variable" (p. 270). An article by Garrett and
Zubin (1943) gave extensive treatment to po-
tential applications of ANOVA in psychological
research. Grant (1944) published a rejoinder
to this article, pointing out errors of omission
in Garrett and Zubin's discussion of some
early psychological articles using ANOVA.
Many other articles that need not be re-
viewed here endorsed ANOVA for psychology
(Bloomers & Lindquist, 1942; Burt, 1938;
Crutchfield & Tolman, 1940; Dunlap, 1938,
1940, 1941; Garrett, 1943; Grant, 1944,
19SO; Hotelling, 1935; Royce, 19SO; Thou-
less, 1939).
A major portion of the 66 expository arti-
cles were applications of variance analysis.
The titles of a number of these articles began
with the words "An application of" ANOVA or
factorial design to a particular research issue.
Jackson's (1940) research bulletin was fre-
quently cited by later researchers. Other arti-
cles on applications ranged from studies of
educational development (Johnson & Tsao,
1945) to galvanic skin response data (Hag-
gard, 1949), from liberalism scores (Schrader,
 ANALYSIS OF VARIANCE
1940) to reaction times (Baxter, 1942), and
from lightness and saturation (Helson, 1942)
to practice effects (Owens, 1942).
The Latin square design received much
early attention from psychologists. No fewer
than seven articles were found that were
devoted wholly or partially to the Latin
square in research design (Baxter, 1941;
Bugelski, 1949; Edwards, 19SOb; Garrett &
Zubin, 1943; Grant, 1944; Thomson, 1941).
An issue that began to receive attention in
the late 1940s was that of post hoc tests. The
common practice was to compute t tests be-
tween individual comparisons following a sig-
nificant overall F value. Webb and Lemmon
(19SO) and Walker (1947), however, com-
mented on this practice. Webb and Lemmon
maintained that it was too rigidly applied,
and Walker saw post hoc techniques as a yet
unresolved problem in research.
An examination of publication dates showed
that 45 out of the 66 expository articles
(687c) appeared between 1938 and 194S.
This was a critical period with respect to the
development of ANOVA techniques in psychol-
ogy. Nearly all of these articles were explicit
introductions to variance techniques. This
suggests that at the time of the publication
of these articles, the ANOVA was still not
widely used by psychological researchers. In
1935 36 38 40
YEAR OF JOURNALS
Figure 1. Proportion of articles using selected statistical tests as a function of year.
171
fact, as late as 1944, Grant used the future
tense to refer to variance techniques in psy-
chology: "Analysis of variance and kindred
techniques developed by Fisher and his co-
workers will very likely become valuable
statistical tools in psychological research" (p.
158).
ANOVA Growth Curve
We carried out a frequency count of articles
appearing in major psychological journals
from 1935 through 1952 to determine the
size of the impact of ANOVA. Six major Amer-
ican psychological journals were selected for
analysis: American Journal oj Psychology,
Journal oj Applied Psychology, Journal oj
Educational Psychology, Journal of Experi-
mental Psychology, Journal oj General Psy-
chology, and Journal oj Psychology.
Every article from 1935 to 1952 in each
journal was reviewed to determine what type
of statistical analysis was employed. A total
of 6,457 articles was included in the count.
Editorial articles, notes and discussions, and
articles reporting new apparatus were not
included.
Figure 1 shows the proportion of articles
that used ANOVA, t tests, critical ratio tests,
and correlational analysis for each year
CRITICAL RATIO • • • • •
CORRELATIONAL
42 44 46 48 50 52
172 AXTHOXY J. RUCCI AND RYAX D. TWENEY
from 1935 through 1952. These categories are
the most reflective of the transition from the
conventional method (critical ratio) to the
Fisher techniques. Use of both t and ANOVA
increased gradually prior to World War II,
declined during the war, and increased im-
mediately thereafter. Use of the critical ratio
technique declined in exact proportion to the
increase in ANOVA and t, which confirms the
idea that Fisherian procedures were supplant-
ing the conventional technique for examining
group differences. Note the marked similarity
in the profiles for ANOVA and t. Every in-
crease or decrease in the use of ANOVA is par-
alleled by a concomitant increase or decrease
in /-test usage. By f949 use of the / test sur-
passed that of the critical ratio, and in 1950
ANOVA usage surpassed the critical ratio. Sur-
prisingly, use of the t test did not precede
that of ANOVA, although it had been developed
much earlier and was very similar to the
already established critical ratio technique.
Figure 1 shows a decline in the use of
Fisher's techniques during the war years,
1942-194S. Note also the general increase in
the use of the critical ratio during the same
period. Those research psychologists young
enough to be inducted into military service
were also those most likely to have had
graduate training in variance techniques
(Chapanis, Note 2). Therefore, those re-
searchers most likely to use ANOVA were taken
out of the publishing ranks during the war
years, which could account for the decline. In
addition, those who continued to publish dur-
ing the war may have been those too old to
be inducted and who lacked variance training.
Their published research, then, would rely
more heavily on the critical ratio technique,
and this could account for the increase in its
use from 1943 to 1945. This explanation of
the war year effect is consistent with Kuhn's
( 1 9 7 0 ) account of paradigm shifts. Kuhn has
suggested that new ideas are adopted differ-
entially by young and old scientists.
The use of correlational analysis (also
shown in Figure 1) from 1935 to 1952 sug-
gests a striking conclusion: The Fisher meth-
ods did not supplant the use of correlational
analysis. The curve of the use of correlational
analysis runs parallel to the abscissa. The
proportion of articles that used correlation in
1952 is nearly equal to that in 1935. This
indicates that the use of correlational meth-
ods remained relatively constant, even after
the Fisher methods were incorporated. With
the aid of hindsight, it becomes clear that
variance techniques enabled researchers to
fill the void that existed in experimental
methodology. To use Cronbach's terminology,
the correlational school probably would have
been sustained whether Fisher's methods ap-
peared or not.
Some aspects of the frequency distribution
for individual journals deserve brief com-
ment: The American Journal of Psychology
and the Journal oj Applied Psychology
showed increases in the use of ANOVA and t
up to 1942. From 1942 to 1945, however,
there was a decrease in the use of these
statistics. From 1945 on, use of ANOVA and t
increased, with a concomitant decrease in the
use of the critical ratio. The percentage of
articles that used both ANOVA and the t test
surpassed that of articles that used the criti-
cal ratio only after 1947. The Journal oj Ex-
perimental Psychology and the Journal oj
Psychology showed the same pattern, though
growth of ANOVA was faster in the former and
slower in the latter. The Journal oj General
Psychology was late to include ANOVA and did
not use it at all in 1948, although it increased
rapidly thereafter.
The results for the Journal oj Educational
Psychology were anomalous compared with
the results of the other five. It evidenced the
largest initial increase in ANOVA through 1942.
This may be attributed to a text by Lind-
quist (1940) on ANOVA for educational re-
searchers, which was widely cited. Although
a decrease in the use of variance techniques
from 1942 to 1946 was also evident in this
journal, the depression lasted much longer
than in the other journals. It was not until
1950 that the use of ANOVA surpassed its
prewar use. Thus the paradox—Although ed-
ucational researchers initially adopted the
technique more quick!)' than others, the im-
mediate postwar upsurge never materialized.
It was not until 1952 that usage of the criti-
cal ratio was surpassed by either ANOVA or /.
 ANALYSIS OF VARIANCE
Statistical Research on ANOVA by
Psychologists
Our third approach traced statistical re-
search done by psychologists relevant to
ANOVA. Table 2 lists the relevant located
articles.
Kogan (19S3) reported that psychologists
had done much to adapt ANOVA techniques to
repeated measures designs. Gilliland and D.
W. Humphreys (1943) acknowledged L.
Humphreys for providing the computational
formulas for the repeated measures analysis
used in their study. Edwards (19SOa) and L.
G. Humphreys (Note 3) cite Gilliland and
D. W. Humphreys and L. G. Humphreys
(1943) as the first to use repeated measures
ANOVA in psychology. Alexander (1946) and
Lindquist (1947) provided variance tests for
trends of repeated trials data. Alexander
(1946) also gave tests for comparing trends
between two groups and tests for estimating
reliability across repeated trials (Alexander,
1947).
One problem frequently faced by psycholo-
gists was that of unequal or disproportionate
subclass sizes, Tsao (1942, 1946), Tsao and
Johnson (1946) and Ansbacher and Mather
(194S) all gave formulas for ANOVA or ANCOVA
with disproportionate subclasses or unequal
subclass sizes.
The 1947 issue of Biometrics covered three
extremely important issues related to ANOVA.
Eisenhart (1947) outlined the difference be-
tween fixed and random models and cited the
need to devise ways of interpreting and test-
ing mixed models. The entire issue of fixed
versus random models received little early
attention from psychologists and is, in fact,
still a subject of debate (Clark, 1973; Clark,
Cohen, Smith, & Keppel, 1976; Wike &
Church, 1976). In the same issue of Bio-
metrics, Cochran (1947) examined the effects
of departures from the assumptions of ANOVA
and Bartlett outlined transformations that
could be applied to eliminate nonhomo-
geneity of variance. Other articles related to
the same issue were published by Edwards
(19SOb), Mueller (1949), Baxter (1941),
and Godard and Lindquist (1940).
A number of articles dealt with the in-
terpretation of interaction effects and the
173
relationship between interaction in ANOVA and
intercorrelation (Coombs, 1948; Peters, 1944;
Taylor, 1942), Edwards (19SOc) discussed
the use of the interaction mean square as the
error term in random model factorial designs.
Other articles listed in Table 2 covered a
variety of topics. Festinger (1943a, 1943b)
used an F test for determining significance of
data with exponential or skewed distributions.
The relationship between F and t- was shown
by at least three researchers: Engelhart
(1941), Rulon (1943), and Brozek and Alex-
ander (1950). A frequently cited article by
Walker (1940) gave a geometrical and con-
ceptual presentation of the concept of degrees
of freedom.
The majority of articles on statistical issues
published by psychologists were highly utili-
tarian. Most were concerned with facilitating
the application of ANOVA to psychological
research. The unique research problems en-
countered in psychology (e.g., repeated mea-
sures designs) necessitated extensions and
revisions of variance analyses as developed by
statisticians. Dunlap (1938) foresaw the need
for psychologists to enlarge their mathemati-
cal and statistical armamentarium:
I can only extend my sympathy to the psychologist
of the future, for it seems as if he must first be
a mathematician, then a statistician, . . . and, if he
is not dead of old age bv then, a psychologist, (p.
571)
The New Textbooks
The mark of establishment is incorporation
in textbooks. Accordingly, we reviewed text-
books that appeared through 19SO that con-
tained coverage of ANOVA. So few were pub-
lished by psychological researchers prior to
1950, however, that it was necessary to in-
clude books by statistical and mathematical
researchers that were cited in early articles
using variance analysis.
Table 3 lists the authors, years of publica-
tion, and affiliations of the located textbooks
through 1950. There are two noteworthy
facts in Table 3. First of all, many of the
names in this table have not appeared in
prior sections of this article, since only a
handful of the authors were psychologists.
Secondly, the affiliations are dominated by
174 ANTHONY J. RUCCI AND RYAN D. TWENEY

Table 2
.1 rticles Containing Statistical Treatments Related to Analysis of Variance.

Author Source Description

Godard & Psycliometrika Tested effect of violating homogeneity of

Lindquist (1940) variance assumption
Shen (1940) Journal of Experimental Education Outlined differences between critical ratio,
/ test, and AN'OVA and ANCOVA
\Valkcr (1940) Journal of Educational Psychology Conceptual and geometrical discussion of
degrees of freedom
Engelhart (1941) Psycliometrika Demonstrated that I- = F
Hoyt (1941) Psycliometrika Used AXOVA to estimate test reliability
Sattcrthwaitc Psycliometrika Extended z test to situations in which a
(1941) chi-square estimate would be inexact
Taylor (1942) British Journal of Psychology Relationship shown between interaction
effects and intercorrelation
Tsao (1942) Psyclwmelrika Formulas for AXOVA with unequal
subclass sixes
Humphreys (1943) Journal of Comparative Psychology Formulas for repeated measures AXOVA
Festinger (1943a) Psychonielrika Test of significance for means from an
exponential distribution
Festinger (1943b) Psycliometrika Test of significance for means from a
skewed distribution
Rulon (1943) Journal of Experimental Education Demonstrated that t'2 = F
Johnson & Tsao Psycliomelriku Factorial design and formulas for d e t e r m i n -
(1944) ing psychophysical thresholds
Kcllcy (1944) Psycliometrika A variance-ratio lest for principal-axis
components
Peters (1944) Psychological Bulletin Interaction in ANOVA related to
intercorrelation
Schoenfeld (1944) American Journal of Psychology Computational formulas for factorial
analysis
Ansbacher & Psyclioinelrika Formulas for AXOVA with unequal
Mather (1945) subclass sizes
Johnson & Tsao Psychonielrika Factorial design and formulas for applying
(1945) covariance analysis
Alexander (1946) Psychological Bulletin Test of trends and differences between
trends for repeated measures of d a t a
Tsao (1946) Psycliometrika. Formulas for AXOVA and ANCOVA with
unequal subclass sixes
Tsao & Johnson Journal of Experimental Education Formulas for ANOVA with unequal
(1946) subclass sizes
Alexander (1947) Psyclwmelrika Estimated reliability across repeated trials
Bartlett (1947) B-iometrics Transformations to eliminate heterogeneity
of variance
Brozck & American. Journal of Psychology Formulas for ANOVA with repeated
Alexander (1947) measures
Cochran (1947) Biometrics Evaluated the effect of departures from
AXOVA assumptions
Eisenhart (1947) Biometrics Discussion of fixed, mixed, and random
models
Lindquist (1947) Psycliometrika Test of trends and differences between
trends for repeated measures data
Coombs (1948) Psycliometrika Interaction in AXOVA related to inter-
correlation
Grant (1948) Psychological IIullelin Formulas for computing sums of squares
in a Latin square design
Kogan (1948) Psychological Bulletin Formulas for repeated measures AXOVA
Mueller (1949) Psychological Bulletin Transformations to use to meet assump-
tions of AXOVA
Rulon (1949) Psyclwmelrika Matrix models of ANOVA and ANCOVA
 ANALYSIS OF VARIANCE 175

Table 2 (continued)

Author Source Description

Brozek & American Journal of Psychology Demonstrated that t- = F

Alexander (1950)
Edwards (1950c) Educational and Psychological Use of interaction sums of squares as
Measurement error terms
Edwards (1950b) Psychological Bulletin Applied homogeneity of variance tests to a
Latin square
Edwards & Psychomelrika Formulas for computing the highest
Horst (1950) interaction sum of squares in a factorial
design

Note. ANOVA = analysis of variance; ANCOVA = analysis of covariance.

researchers from Columbia University, Iowa
State College, the University of Iowa, and the
University of Minnesota. This is discussed in
more detail later in this article.
A small subset of the textbooks listed in
Table 3 was heavily cited by psychological
articles using ANOVA. The two Snedecor
(1934, 1937) texts, Fisher's (1935) text,
Goulden (1939), Rider (1939), Peters and
VanVoorhis (1940), and Walker (1943) ac-
count for most of the citations through the
early 1940s. In general, however, the texts by

Table 3
Statistical Textbooks Containing Analysis of Variance Coverage

Author Affiliation

Fisher (1925b) Rothamsted Experimental Station

Tippett (1931) British Cotton Industry
Snedecor (1934) Iowa State College
Fisher (1935) University of London
Davenport & Ekas (1936) Carnegie Institution and Northwest High School, Philadelphia
Treloar (1936) University of Minnesota
Yule & Kendall (1937) St. John's College
Snedecor (1937) Iowa State College
Fisher & Yates (1938) University of Cambridge and Rothamsted Experimental Station
Croxten & Crowden (1939) Columbia University
Goulden (1939) University of Manitoba
Rider (1939) Washington University
Peters & VanVoorhis (1940) Perm State College
Lindquist (1940) University of Iowa
Walker (1943) Columbia University
Smith & Duncan (1944) Princeton University
Edwards (1946) University of Washington
Wilks (1946) Princeton University
Eisenhart, Hastay, &
Wallis (1947) Columbia University Research Group
Garrett (1947) Columbia University
Kelley (1947) Harvard University
Lewis (1948) University of Iowa
Chapanis, Garner, & Morgan
(1949) Johns Hopkins University
Mann (1949) Ohio State University
McNemar (1949) Stanford University
Johnson (1949) University of Minnesota
Cochran & Cox (1950) Iowa State College
Edwards (1950) University of Washington
Guilford (1950) University of Southern California
Mood (1950) Iowa State College
176 ANTHONY J. RUCCI AND RYAN D. TWENEY
Snedecor (1937) and Lindquist (1940) had
the largest impact on psychological research-
ers well into the late 1940s. The Lindquist
book represents the first comprehensive text
on ANOVA that was intended for behavioral
researchers. Snedecor was a statistician at the
Iowa State College agricultural station, and
his text was oriented to agricultural research.
However, a number of the early psychology
programs offering variance training used this
text.
The texts by Kelley (1947), McNemar
(1949), Johnson (1949), Cochran and Cox
(19SO), Edwards (T9SOa), and Guilford
( 1 9 S O ) proved to be important to psycho-
logical researchers through the 1950s.
Once again the effect of the war years on
the use of ANOVA is seen in Table 3. Only
three texts appeared between 1940 and 1946.
From immediately after the war, however,
through 1950, the production of texts on
ANOVA averaged nearly three per year, not
including later editions of earlier texts.
ANOVA Training in Psychology
Our fifth and final approach was to deter-
mine when ANOVA was incorporated into the
training of psychologists. It was expected
that graduate training in ANOVA would have
lagged behind incorporation into the research
literature.
A questionnaire survey was conducted of
88 psychology departments that offered grad-
uate degrees in 1940. The questionnaire was
intended to determine when ANOVA training
was introduced into the graduate program at
each department. Of the 88 surveys mailed,
41 ( 4 7 / J - ) were returned. Fourteen of the
returned surveys were unusable due to incom-
plete information. Therefore, Table 4 lists
the results of the 27 usable questionnaires.
The universities are listed in the order in
which the}' began to offer the doctorate in
psychology degree. In all three categories,
those departments offering the doctorate in
psychology prior to 1940 incorporated ANOVA
training earlier than other departments. The
27 departments were rank ordered by the
year in which they began to offer the doc-
torate and by the year of the three categories.
Rank-order correlations were computed for
the ordering by doctorate with each of the
three category orderings. The rank-order cor-
relation (Spearman's rho corrected for ties)
between doctoral year and year of first vari-
ance training was .53; between doctoral year
and year of first psychology course in ANOVA,
p = .63; between doctoral year and first year
of program requirement, p = AS (all ps <
.01). The older the doctoral program, (a) the
earlier variance training was started, (b) the
earlier the department offered a variance
course, and (c) the earlier variance training
became a graduate degree requirement.
In the 5 years after World War II, there
was a dramatic increase in the number of
psychology departments in which graduate
students were receiving training in ANOVA
techniques. This result coincides with a similar
increase in journal and textbook use of and
attention to ANOVA immediately following the
war. Again, the return of young psychologists
to academic positions following the war may
have brought an increased emphasis on vari-
ance techniques.
Phylogeny of Variance Training
Based on the results of the preceding
analyses, the line of training was traced for
those researchers identified as prominent con-
tributors to the development of variance tech-
niques in psychology. (See Ben-David & Col-
lins, 1966, and Boring & Boring, 1948, for
examples of the method used.) Figure 2 was
constructed using 1940 editions of the Ameri-
can Psychological Association membership
directory, textbook prefaces and acknowledge-
ments, and personal communications (Cron-
bach, Note 1; Chapanis, Note 2; Humph-
reys, Note 3; Bechtoldt, Note 4; Cronbach,
Note 5; Edwards, Note 6; Gage, Note 7;
Green, Note 8; Gulliksen, Note 9; Lind-
quist, Note 10; McNemar, Note 11; Ryan,
Note 12; Smith, Note 13) from a number of
individuals.
The phylogeny of the development of Fish-
er's techniques in psychology and in Ameri-
can statistics as well was centered around
three statistical researchers: G. W. Snedecor
at Iowa State College, Harold Hotelling at
Columbia University, and Palmer Johnson at
the University of Minnesota. These three
 ANALYSIS OF VARIANCE 177

Table 4
Summary of Questionnaire Results for Incorporation of Variance Training

Year degree first

offered
ANOVA
PhD MA
in psy- in psy- Require-
University chology chology Training" Course1' ment 0

Johns Hopkins 1886 1949 1952 1952

Cornell 1892 1950 1950 1954d
Stanford 1915 1935 1935 1935
of Kansas 1916 1916 1949 1949 1951
Indiana 1919d 1900d 1947 1951 1947
Northwestern 1924 1940 1940 1946
of Texas, Austin 1925d 1910d I947d 1949d 1950
Brown 1927« 1892 1946 1946 1946
of Michigan 1928' 1947d 1952 1948
of North Carolina, Chapel Hill 1920s 1954" 1954'1 1954d
Duke 1930 1930 1951 1951
1931 1923
—
of North Dakota 1958 1958 1958
Case Western Reserve 1932 1960d — —
Temple 1938 1938 1966 1966 1970
Purdue 1941 1935 1941 1941 1941
Loyola 1947 1930 1966 1966 1966
of Utah 1949 1931 1952 1956 1957
Vanderbilt 1950 1949* 1950 1950 1950
Tufts 1952 1940 1950 1950'1 1950
of Missouri, Columbia 1953 1920s 1960d 1968 1960d
of Georgia 1956 1930s 1956d 1956d 1971
of Arkansas 1959 1929 1956 1956 1959
of South Dakota 1963 1903 1964 1964 1964
Iowa State 1965 1920 d '>' 1934 — 1948
of Wyoming 1966 1914 1940s — 1970d
Howard 1969 1928 1950 1952 1952
of Nebraska, Omaha 1973 1937 1966 1966 1968

Note. ANOVA = analysis of variance; dash = not yet incorporated.

a
Year in which graduate students in psychology began receiving variance training, possibly through other
departments (i.e., biology, agriculture, or mathematics). For departments that offered a master's degree
prior to 1940, median year is 1950; for departments that offered a doctorate prior to 1940, median year is
1949.
b
Year in which psychology department first offered a course in ANOVA. For departments that offered a
master's degree prior to 1940, median year is 1952; for departments that offered a doctorate prior to 1940,
median year is 1951.
0
Year in which ANOVA training became a graduate program requirement. For departments that offered a
master's degree prior to 1940, median year is 1954; for departments that offered a doctorate prior to 1940,
median year is 1951.
d
Estimate provided by individual completing the questionnaire.
0
Previously offered through philosophy department.
' Offered through philosophy department 1906-1928.
e
Previously offered through philosophy and psychology departments.
h
Previously offered through history, philosophy, and psychology departments.

stand at the beginning of the entire line of
training in Fisherian statistics in the United
States. All three gained exposure to variance
techniques directly from Fisher himself.
Hotelling visited Fisher's laboratory at the
Rothamsted Experiment Station in 1929. In
1931 he took a position as professor of Eco-
nomics at Columbia University. Under Ho-
telling's influence Columbia University be-
came one of the centers of statistical research
in the United States. Hotelling (1931) him-
self is responsible for T-, the multivariate
178 ANTHONY J. RUCCI AND RYAN D. TWENEY
1925 •1935-
^,'Cox
0**
**''^, ..--Cochran •
/ I Iowa St.)
S''\
-Hotel lings:-!-
I Columbia )"%>„
v\ *%
S
\v
^Johnson-
Tsao
I Minnesota I^VV*«.S """ 1
""."^^ ~~" Broiek
\^> v
N "Baxter
Figure 2. Phylogeny of major contributors to psychological statistics.
generalization of Student's t. His associate,
S. S. Wilks (1932), developed the multivari-
ate generalization of the ANOVA. Of those hav-
ing a direct effect on psychology's adoption
of variance techniques, H. E. Garrett, J. Zu-
bin, and H. Walker were associated with Co-
lumbia University. Garrett and Zubin's
(1943) article can legitimately be cited as
the initial comprehensive presentation of vari-
ance techniques to psychology.
Johnson visited the Gallon Laboratories in
London and received exposure to variance
techniques from Fisher. After returning to
Minnesota, Johnson had a profound influence
on the development of variance techniques in
behavioral research. He published numerous
articles dealing with ANOVA, and Johnson and
Neyman (1936) developed a covariance tech-
nique for testing linear hypotheses that came
to be referred to as the Johnson-Neyman
technique. The effects of Johnson's research
and training were felt through other research-
ers from Minnesota. Baxter's (1941) article
on variance techniques was an excellent trans-
lation of agricultural terminology into psy-
•1940 •1945 •1950
-Lindquist
[U. Iowa)
• Gaskill
,, Bartlett
Walker
[Wisconsin |
MeNemar :*''
„' Edwards
I Stanford I "**»
N
3errett
s
" * Humphreys ''.
I Northwestern I
IU. Washington)
""•Chapanis
Zubin (Johns Hopkins!
x
Alexander
chological terminology. Brozek, Tsao, and
Alexander were also at Minnesota, and each
of these researchers published several articles
related to ANOVA.
Snedecor received exposure to ANOVA at
Iowa State College in the mid-1930s while
Fisher was a visiting professor there. Snede-
cor's work at Iowa State College was impor-
tant to the development of variance tech-
niques in the United States. One or the other
of Snedecor's textbooks (1934, 1937) was
nearly always cited by early psychological
research using variance analysis. His influence
was also felt through others at Iowa State
College. Gaskill and Cox (1937) published
one of the earliest articles that dealt with
covariance analysis. Cochran and Cox (1950),
who were also from Iowa State College, pub-
lished a text on experimental design that was
widely used by psychologists in the 1950s.
At the University of Iowa, but through
contacts with Snedecor and Cochran, E. F.
Lindquist made a major contribution to the
incorporation of variance techniques into psy-
chology. His (1940) textbook on ANOVA for
 ANALYSIS OF VARIANCE
educational researchers was the first bona fide
statistics text devoted to the application of
variance techniques in behavioral research.
This text was widely cited by researchers in
psychology who used ANOVA prior to 19SO.
Of psychologists directly involved in the
statistical development of ANOVA, the line of
training clearly traces back to Quinn McNe-
mar at Stanford University. McNemar, how-
ever, received his exposure to statistical tech-
niques from Hotelling, who was an associate
professor at Stanford University until 1931.
Through McNemar, at Stanford University,
L. G. Humphreys was trained before going to
Northwestern University, where he was sta-
tistical adviser for the research of Allen Ed-
wards. Edwards became a prominent figure in
the development of variance techniques in
psychology, publishing numerous articles, an
elementary textbook (Edwards, 1946), and in
1950 a comprehensive textbook of ANOVA.
David Grant graduated from Stanford Uni-
versity in 1941 before going to the University
of Wisconsin. Grant was perhaps the most
active of all psychologists both in introduc-
ing ANOVA to psychology and in applying it
in his own research. From Stanford Univer-
sity, therefore, came directly and indirectly
four of the most prominent names in the
development of variance techniques in psy-
chology: McNemar, L. G. Humphreys, Ed-
wards, and Grant.
L. G. Humphreys may even have played an
important role in introducing variance train-
ing at Yale University. While on a postdoc-
toral appointment at Yale in 1938, Humph-
reys reported on Fisher's techniques in a
seminar offered by Donald Marquis. In 1942,
at Yale, Chapanis received variance training
from Carl Hovland. Chapanis went on to
introduce the ANOVA into applied experimental
research (Chapanis, 1951; Chapanis, Garner,
& Morgan, 1949).
Discussion
Our results show that ANOVA was incorpo-
rated into psychology in logical and orderly
steps. First came the expository articles from
1938 to 1945. The use of variance techniques
in journals began to increase in the early
1940s, was deterred during World War II, and
179
began an uninterrupted increase immediately
following the war. Psychologists became ac-
tively involved in statistical issues related to
the technique in the middle-to-late 1940s.
Textbooks in statistics written explicitly for
psychological researchers did not appear until
the late 1940s. Finally, graduate training in
ANOVA was the last in chronological order to
develop, the median year being 1951.
Thus, introductory articles were followed
by a gradual increase in the application of
the technique in the research literature. When
it became clear that the technique was es-
tablishing itself as a primary method of de-
sign and analysis, psychologists began refin-
ing the technique to meet the unique
requirements of behavioral research. Text-
books written by psychologists for psycholo-
gists then appeared. When it was clear that
competent psychologists must receive ex-
posure to variance techniques, graduate pro-
grams began offering and requiring courses in
ANOVA.
The history of psychology's incorporation
of variance techniques can be logically di-
vided into three periods: (a) the expository
period from 1925 to 1942, (b) the war period
from 1942 to 1945, and (c) the postwar
institutionalization period from 1946 to 1957.
The expository period is characterized by
early attempts to use ANOVA, but even more
so by articles exhorting the use of variance
analysis by psychologists. The majority of
these articles appeared between 1936 and
1945, with some of the most important (e.g.,
Baxter, 1941; Crutchfield, 1938; Crutchfield
& Tolman, 1940; Garrett & Zubin, 1943)
appearing between 1938 and early 1943.
The war years effect was pronounced. The
gradual upsurge in the use of ANOVA between
1934 and 1942 was wiped out during World
War II. It was 1947 before use of the tech-
nique regained the high point it had reached
just before the war. The war, therefore, was
instrumental in how quickly the technique
was adopted and may have delayed its in-
corporation by some 5 years. This conclusion
must be qualified, however. To establish such
an effect conclusively would require extensive
review of the research conducted by psycholo-
gists for the military during the war. Many
psychologists may have received their initial
180 ANTHONY J. RUCCI AND RYAN D. TWENEY
exposure to variance analysis and experience
with the technique while in military service
(Chapanis, Note 2). If so, then the war may
actually have accelerated use of the technique
in the late 1940s.
The postwar institutionalization period was
dramatic. The use of ANOVA in the journals
began an uninterrupted increase immediately
following the war. By 19S2 nearly all psy-
chology graduate programs were offering
course work in ANOVA, clearly signaling its
institutionalization as an accepted analytic
technique familiar to virtually all research
psychologists.
The proposed trichotomy of the history of
psychology's adoption of variance techniques
qualifies the usual claim (Hearnshaw, 1964;
Stanley, 1966) that the incorporation of vari-
ance techniques into psychology was a post-
World War II phenomenon. By 19S2 psychol-
ogy had completed a shift with respect to
experimental design and analysis, a process
that began well before the war, in the 1930s.
Can this change be considered an instance
of a paradigm shift in Kuhn's (1970) sense?
The criteria that would have to be met to
substantiate such a claim are (a) that a major
change was produced in psychologists' way of
perceiving and/or approaching basic concep-
tual issues, (b) that the new techniques were
differentially adopted by younger psycholo-
gists against the resistance of older workers,
(c) that anomalies that could not be ac-
counted for with older techniques were in-
corporated in the newer.
The first criterion can be supported by
reference to Cronbach's (1957) characteri-
zation of the experimental discipline of psy-
chology. The approach to basic experimental
issues was radically different following the
adoption of Fisherian techniques. It is unde-
niable that factorial design and analysis fa-
cilitated the demise of the single-variable law
in psychology. By 1957 it was standard ex-
perimental procedure to manipulate multiple
levels of multiple independent variables.
ANOVA was, therefore, the ideal analytic sta-
tistic for such designs.
That the second condition held is suggested
by the effect of the war on incorporation of
ANOVA. By and large, it was younger, less
established researchers who entered wartime
service as psychologists. This period shows a
clear drop in the publication of articles using
ANOVA from 1942 to 1946, whereas, at the
same time, ANOVA techniques were increas-
ingly used by military research psychologists
(e.g., Chapanis & Schachter, Note 14;
Schachter & Chapanis, Note 15). Following
the war, these psychologists entered the aca-
demic world and began to publish in regular
journals, using ANOVA. Further, there was
clearly resistance from older researchers.
Peters (1943, 1944; Peters & VanVoorhis,
1940) was the most vociferous critic of Fish-
er's techniques. He pointed out that Fisher
presented only extensions of traditional sta-
tistics, nothing fundamentally new. He re-
ferred to the ANOVA as "magic" on two occa-
sions in print and warned,
if educationalists and psychologists, out of some
sort of inferiority complex, grab indiscriminately at
them [variance techniques] and employ them where
they are unsuitable, education and psychology will
suffer another slump in prestige such as they have
often hitherto suffered in consequence of the pursuit
of fads. (Peters, 1943, p. 549)
The last criterion for a paradigm—success-
ful incorporation of anomalies—does not seem
to fit this case. Although examples can be
found of pre-ANOVA research that could have
benefitted from use of ANOVA, it is not the
case that felt anomalies were present that
needed to be attended to. ANOVA is, of course,
a technique, not a theory, and one could ar-
gue that no methodological technique could
ever meet this criterion, by definition. If,
however, one considers the concepts of
crossed classifications and interaction as being
anomalies prior to ANOVA, then, in that sense,
ANOVA did incorporate anomalies. Even so, we
feel that it is more reasonable to regard
ANOVA not as something fundamentally new,
in a revolutionary sense, but as an extension
and explication of a set of design strategies.
Although ANOVA gave the experimenter a
powerful set of analytic tools, the nature of
psychological experimentation changed during
the period in question as psychologists moved
from single-variable studies to multiple-varia-
ble studies. Thus, the history of ANOVA is
likely to be most interpretable as a component
of a broader, as yet unattempted, history of
 ANALYSIS OF VARIANCE
the use of experimental research methods in
psychology.
Thus, we feel that our study represents
only the beginning of a necessary but much
larger effort. We cannot legitimately assess
the place of ANOVA in the history of psychol-
ogy without the context provided by knowl-
edge of the explicit ways in which experimen-
tation changed during the period in question.
Furthermore, neither statistical techniques nor
design techniques can be assessed indepen-
dently of the concomitant changes in the
empirical issues considered by psychologists
or independently of the changes in the theo-
retical structures underlying psychological
research. Our study is thus very much a
beginning effort only. By characterizing the
growth and development of a single impor-
tant dimension of psychological method, we
hope it will contribute to the emergence of a
broader, more comprehensive history of scien-
tific method in psychology.
Reference Notes
1. Cronbach, L. J. Personal communication, July
16, 1977.
2. Chapanis, A. Personal communication, July 19,
1977.
3. Humphreys, L. G. Personal communication, July
18, 1977.
4. Bechtoldt, H. P. Personal communication, Au-
gust 10, 1977.
5. Cronbach, L. J. Personal communication, July
25, 1977.
6. Edwards, A. L. Personal communication, Decem-
ber 6, 1977.
7. Gage, N. L. Personal communication, July 21,
1977.
8. Green, B. F. Personal communication, August 10,
1977.
9. Gulliksen, H. Personal communication, July 15,
1977.
10. Lindquist, E. F. Personal communication, De-
cember 4, 1977.
11. McNcmar, Q. Personal communication, Decem-
ber 20, 1977.
12. Ryan, T. A. Personal communication, July 27,
1977.
13. Smith, P. C. Personal communication, May 10,
1977.
14. Chapanis, A., & Schachter, S. Depth perception
through a P-80 canopy and through distorted
glass (Memorandum Rep. TSEAL-69S-48N).
Wright Field, Dayton, Ohio: Army Air Forces
181
Air Technical Service Command, Engineering
Division, Aero Medical Laboratory, October
1945.
15. Schachter, S., & Chapanis, A. Distortion in
glass and its effect on depth perception (Memo-
randum Rep. TSEAL-695-48B). Wright Field,
Dayton, Ohio: Army Air Forces Air Technical
Service Command, Engineering Division, Aero
Medical Laboratory, April 1945.
References
Alexander, H. W. A general test for trend. Psycho-
logical Bulletin, 1946, 43, 533-557.
Alexander, H. W. The estimation of reliability when
several trials are available. Psychometrika, 1947,
12, 79-99.
Ansbacher, H. L., & Mather, K. Group differences in
size estimation. Psychometrika, 1945, 10, 37-56.
Baker, K. H. Item validity by the analysis of vari-
ance. Psychological Record, 1939, 3, 242-248.
Bartlett, M. S. The use of transformations. Bio-
metrics, 1947, 3, 39-52.
Baxter, B. Problems in the planning of psychologi-
cal experiments. American Journal oj Psychology,
1941, 54, 270-280.
Baxter, B. A study of reaction time using factorial
design. Journal oj Experimental Psychology, 1942,
31, 430-437.
Ben-David, J., & Collins, R. Social factors in the
origin of a new science: The case of psychology.
American Sociological Review, 1966, 31, 451-465.
Bloomers, P., & Lindquist, E. F. Experimental and
statistical studies: Application of newer statistical
techniques. Review of Educational Research, 1942,
12, 501-520.
Boring, M. D., & Boring, E. G. Masters and pupils
among the American psychologists. American
Journal oj Psychology, 1948, 61, 527-534.
Brozek, J., & Alexander, H. A note on estimation of
the components of variation in a two-way table.
American Journal oj Psychology, 1947, 60, 629-
637.
Brozek, J., & Alexander, H. The formula t~ = F.
American Journal of Psychology, 1950, 63, 262-
269.
Bugelski, B. R. A note on Grant's discussion of the
Latin square principle in the design of experi-
ments. Psychological Bulletin, 1949, 46, 49-50.
Hurt, C. Recent developments of statistical methods
in psychology: 1. Occupational Psychology Land-
marks, 1938, 12, 169-177.
Carington, W. The quantitative study of trance
personalities: 1. Proceedings oj the Society for
Psychical Research, 1934, 42, 173-240.
Carington, W. The quantitative study of trance
personalities: 2. Proceedings oj the Society for
Psychical Research, 1936, 43, 319-361.
Carington, W. The quantitative study of trance
personalities: 3. Proceedings of the Society for
Psychical Research, 1937, 44, 189-222.
Chapanis, A. Theory and methods for analyzing
errors in man-machine systems. Annals of the
182 ANTHONY J. RUCCI AND RYAN D. TWENEY
.\eui York Academy of Sciences, 1951, 51, 1179-
1203.
Chapanis, A., Garner, VV. R., & Morgan, C. T.
Applied experimental psychology. New York:
Wilej', 1949.
Clark, H. H. The language-as-fixed-effect fallacy:
A critique of language statistics in psychological
research. Journal of Verbal Learning and Verbal
Behavior, 1973, 12, 335-359.
Clark, H. H., Cohen, J., Smith, J. E. K., & Keppel,
G. Discussion of Wike and Church's comments.
Journal oj Verbal Learning and Verbal Behavior,
1976, IS, 257-266.
Cochran, W. G. Some consequences when the as-
sumptions for the analysis of variance are not
satisfied. Biometrics, 1947, 3, 22-28.
Cochran, VV. G., & Cox, G. M. Experimental designs,
New York: Wiley, 1950.
Coombs, C. H. The role of correlation in analysis of
variance. Psychometrika, 1948, 13, 233-243.
Cronbach, L. The two disciplines of scientific psy-
chology. American Psychologist, 1957, 12, 671-
684.
Croxten, F. E., & Crowden, D. J. Applied general
statistics. New York: Prentice-Hall, 1939.
Crutchfield, R. S. Efficient factorial design and
analysis of variance illustrated in psychological
experimentation. Journal oj Psychology, 1938, 5,
339-346.
Crutchfield, R. S. The determiners of energy expendi-
ture in string pulling by the rat. Journal oj Psy-
chology, 1939, 7, 163-178.
Crutchfield, R. S., & Tolman, E. C. Multiple-varia-
ble design for experiments involving interaction of
behavior. Psychological Review, 1940, 47, 38-42.
Davenport, C. B., & Ekas, M. P. Statistical methods
in biology, medicine, and psychology. New York:
Wiley, 1936.
Dressel, P. The effect of high school grades on col-
lege grades. Journal oj Educational Psychology,
1939, JO, 611-616.
Dunlap, J. W. Recent advances in statistical theory
and applications. American Journal of Psychology,
1938, 51, 558-S71.
Dunlap, J. W. Applications of analysis of variance
to educational problems. Journal oj Educational
Research, 1940, 33, 434-442.
Dunlap, J. W. Recent advances in statistical theory
and applications. American Journal oj Psychology,
1941, 54, 583-601.
Edwards, A. L. Statistical analysis for students in
psychology and education. New York: Rinehart,
1946.
Edwards, A. L. Experimental design in psychologi-
cal research. New York: Rinehart, 1950. (a)
Edwards, A. L. Homogeneity of variance and the
Latin square design. Psychological Bulletin, 1950,
47, 118-129. (b)
Edwards, A. L. On the use of interactions as "error
terms" in the analysis of variance. Educational
and Psychological Measurement, 1950, 10, 214-
223. (c)
Edwards, A. L., & Horst, P. The calculation of sums
of squares for interactions in the analysis of vari-
ance. Psychometrika, 1950, 14, 17-24.
Eisenhart, C. The assumptions underlying the analy-
sis of variance. Biometrics, 1947, 3, 1-21.
Eisenhart, C., Hastay, M. W., & Wallis, W. A.
Selected techniques of statistical analysis. New
York: McGraw-Hill, 1947.
Engelhart, M. D. The analysis of variance and
covariance techniques in relation to the conven-
tional formulas for the standard error of dif-
ference. Psychometrika, 1941, 6, 221-234.
Federer, W. T., & Balaam, L. N. Bibliography on
experiment and treatment design—pre 1968. New
York: Hafner, 1973.
Festinger, L. An exact test of significance for means
of samples drawn from populations with expo-
nential frequency distribution. Psychometrika,
1943, S, 153-160. (a)
Festinger, L. A statistical test for means of samples
from skew populations. Psychometrika, 1943, 8,
205-210. (b)
Fisher, R. A. On the distribution of the standard
deviation of small samples: Appendix 1. to papers
by "Student" and R. A. Fisher. Biometrika, 1915,
W, 522-529.
Fisher, R. A. Applications of "Student's" distribu-
tion. Metron, 1925, 5, 90-104. (a)
Fisher, R. A. Statistical methods for research work-
ers. London: Oliver & Boyd, 1925. (b)
Fisher, R. A. The arrangement of field experiments.
Journal of the Ministry of Agriculture, 1926, 33,
503-513.
Fisher, R. A. Statistical methods for research work-
ers (4th ed.). London: Oliver & Boyd, 1932.
Fisher, R. A. The design of experiments. London:
Oliver & Boyd, 1935.
Fisher, R. A. Statistical methods for research work-
ers (14th ed.). London: Oliver & Boyd, 1970.
Fisher, R. A., & MacKenzie, W. A. Studies in crop
variation: 2. The manurial response of different
potato varieties. Journal of Agricultural Science,
1923, 13, 311-320.
Fisher, R. A., & Yates, F. Statistical tables for bio-
logical, agricultural, and medical research. New
York: Hafner, 1938.
Garrett, H. E. AN OVA in psychological research.
Journal of Educational Research, 1943, 36, 631-
632.
Garrett, H. E. Statistics in psychology and educa-
tion. New York: Longmans, Green, 1947.
Garrett, H. E., & Zubin, J. The analysis of vari-
ance in psychological research. Psychological Bul-
letin, 1943, 40, 233-267.
Gaskill, H. V., & Cox, G. M. 1. Respiration: Use
of analysis of variance and covariance in psycho-
logical data. Journal of General Psychology, 1937,
16, 21-38,
Gilliland, A. R., & Humphreys, D. W. Age, sex,
method and interval as variables in time estima-
tion. Journal oj Genetic Psychology, 1943, 63,
123-130.
Godard, R. H., & Lindquist, E. F. An empirical
study of the effect of heterogeneous within-groups
 ANALYSIS OF VARIANCE
variance upon certain /''-tests of significance in
analysis of variance. Psychometrika, 1940, S, 263-
274.
Gossett, W. S. (pseudonym, Student). The probable
error of a mean. Blomelriha, 1908, 6, 1-25.
Gossett, W. S. (pseudonym, Student). On testing
varieties of cereals. Biometrika, 1923, 25, 271-293.
Gottsdanker, R. M. An experimental study of fixa-
tion of response by college students in a multiple
choice situation. Journal oj Experimental Psy-
chology, 1939, 25, 431-444.
Goulden, C. H. Methods of statistical analysis.
Minneapolis, Minn.: Burgess, 1939.
Grant, D. A. On "The analysis of variance in psy-
chological research." Psychological Bulletin, 1944,
41, 158-166.
Grant, D. A. The Latin square principle in the
design and analysis of psychological experiments.
Psychological Bulletin, 1948, 45, 427-442.
Grant, D. A. Statistical theory and research design.
In C. P. Stone (Ed.), Annual review of psychol-
ogy (Vol. 1). Stanford, Calif.: Annual Reviews,
19SO.
Guilford, J. P. Fundamental statistics in psychology
and education. New York: McGraw-Hill, 1950.
Haggard, E. A. On the application of analysis of
variance to GSR data: 1. The selection of an
appropriate measure. Journal of Experimental
Psychology, 1949, 39, 378-392.
Hearnshavv, L. S. A short history of British psy-
chology. New York: Barnes & Noble, 1964.
Helson, H. Multiple-variable analysis of factors
affecting lightness and saturation. American Jour-
nal of Psychology, 1942, 55, 46-57.
Hotelling, H. The generalization of "Student's"
ratio. Annals oj Mathematical Statistics, 1931, 2,
360-378.
Hotelling, H. Review of Snedecor, G. W.: Calcula-
tion and interpretation of the analysis of variance
and covariance. Journal oj the American Statisti-
cal Association, 1935, 30, 118.
Hotelling, H. Dr. Peters' criticism of Fisher's statis-
tics. Journal oj Educational Research, 1943, 36,
707-711.
Hotelling, H. The impact of R. A. Fisher on statis-
tics. Journal of the American Statistical Associa-
tion, 1951, 46, 35-46.
Hoyt, C. Test reliability by analysis of variance.
Psycttometrika, 1941, 6, 153-160.
Humphreys, L. G. The strength of a Thorndikian
response as a function of the number of practice
trials. Journal oj Comparative Psychology, 1943,
35, 101-110.
Jackson, R. W. B. Reliability of mental tests. Brit-
ish Journal oj Psychology, 1939, 29, 267-287.
Jackson, R. W. B. Application of the analysis of
variance and covariance methods to educational
problems. Department of Educational Research,
University of Toronto Bulletin, 1940, 11, 67-74.
Johnson, P. 0. Use of Fisherian statistics. Journal
oj Educational Research, 1943, 36, 627-630.
Johnson, P. 0. Statistical methods in research. New
York: Prentice-Hall, 1949.
183
Johnson, P. 0., & Neyman, J. Tests of certain linear
hypotheses and their application to some educa-
tional problems. Statistical Research Memoirs,
1936, ;, 57-93.
Johnson, P. O,, & Tsao, F. Factorial design in the
determination of differential limen values. Psycho-
metrika, 1944, 9, 107-146.
Johnson, P. 0., & Tsao, F. Factorial design and
covariance in the study of individual educational
development. Psychometrika, 1945, 10, 133-162.
Kelley, T. L. A variance-ratio test of the uniqueness
of principal-axis components as they exist at any
stage of the Kelley iterative process for their
distribution. Psychometrika, 1944, 9, 199-200.
Kelley, T. L. Fundamentals of statistics. Cambridge,
Mass.: Harvard University Press, 1947.
Kogan, L. S. Analysis of variance—Repeated mea-
surements. Psychological Bulletin, 1948, 45, 131-
143.
Kogan, L. S. Variance designs in psychological re-
search. Psychological Bulletin, 1953, 50, 1-40.
Krcezcr, G. Intelligence level and occipital alpha
rhythm in the Mongolian type of mental defi-
ciency. American Journal oj Psychology, 1939, 52,
503-532.
Kuhn, T. S. The structure oj scientific revolutions.
Chicago: University of Chicago Press, 1970.
Lev, J. Evaluation of test items by the method of
analysis of variance. Journal of Educational Psy-
chology, 1938, 29, 623-630.
Lewis, D. Quantitative methods in psychology. Iowa
City, Iowa: Bookshop, 1948.
Lindquist, E. F. Statistical analysis in educational
research. Boston: Houghton-Mifflin, 1940.
Lindquist, E. F. Goodness of fit and trend curves
and significance of trend differences. Psycho-
metrika, 1947, 12, 65-78.
Mahalanobis, P. C. Professor Ronald Aylmer Fisher.
Sankhya, 1938, 4, 265-272. •
Mann, H. B. Analysis and design oj experiments.
New York: Dover, 1949.
McNcmar, Q. Psychological statistics. New York;
Wiley, 1949.
Mood, A. Introduction to the theory oj statistics.
New York: McGraw-Hill, 1950.
Mueller, C. G. Numerical transformations in the
analysis of experimental data. Psychological Bul-
letin, 1949, 46, 198-223.
Owens, VV. A. A new technique in studying the
effects of practice upon individual differences.
Journal oj Experimental Psychology, 1942, 30,
180-183.
Peters, C. C. Misuses of the Fisher statistics. Jour-
nal of Educational Research, 1943, 36, 546-549.
Peters, C. C. Interaction in analysis of variance in-
terpreted as intercorrelation. Psychological Bulle-
tin, 1944, 41, 287-299.
Peters, C. C., & VanVoorhis, W. R. Statistical pro-
cedures and their mathematical bases. New York:
McGraw-Hill, 1940.
Reitz, W. Statistical techniques for the study of
institutional differences. Journal oj Experimental
Education, 1934, 3, 11-24.
184 ANTHONY J. RUCCI AND RYAN D. TWENEY
Rider, P. R. An introduction to modern statistical
analysis. New York: Wiley, 1939.
Royce, J. R. A synthesis of experimental designs in
program research. Journal of General Psychology,
1950, 43, 295-303.
Rubin, M. A. A variability study of the normal and
schizophrenic alpha rhythm: 1. Journal of Psy-
chology, 1938, 6, 325-334.
Rubin.Rabson, G. The influence of analytical pre-
study in memorizing piano music. Archives of
Psychology, 1937, 31 (No. 220).
Rubin-Rabson, G. Studies in the psychology of
memorizing piano music. Journal of Educational
Psychology, 1939, 30, 321-34S.
Rulon, P. J. Fisher's t-test as a special case of his
s-test. Journal oj Experimental Education, 1943,
11, 245-249.
Rulon, P. J. Matrix representation of models for the
analysis of variance and covariance. Psychomet-
rika, 1949, 14, 259-278.
Satterthwaite, F. E. The synthesis of variance.
Psyclwmetrika, 1941, 6, 309-316.
Schocnfeld, W. N. The treatment of multiple entries
in analysis of variance with three criteria of
classification. American Journal of Psychology,
1944, 57, 500-508.
Schrader, W. A. B. Analysis of variance applied to
liberalism scores. Journal of Experimental Educa-
tion, 1940, 8, 267-270.
Shen, E. Experimental design and statistical treat-
ment in educational research. Journal of Experi-
mental Education, 1940, S, 346-353.
Smith, J. G., & Duncan, A. J. Elementary statistics
and applications. New York: McGraw-Hill, 1944.
Snedecor, G. W. Calculation and interpretation of
the analysis of variance and covariance. Ames,
Iowa: Collegiate Press, 1934.
Snedecor, G. W. Analysis of variance and covariance
of statistically controlled grades. Journal of the
American Statistical Association, 1935, 30, 263-268.
Snedecor, G. W. Statistical methods. Ames, Iowa:
Collegiate Press, 1937.
Stanley, J. C, The influence of Fisher's "The design
of experiments" on educational research thirty
years later. American Educational Research Jour-
nal, 1966, 3, 223-229.
Taylor, W. S. The use of interactions in analyzing
psychological data. British Journal of Psychology,
1942, 32, 248-258.
Thomson, G. H. The use of the Latin square in
designing educational experiments. British Journal
of Educational Psychology, 1941, 11, 135-137.
Thouless, R. H. Review of Mr. Whatley Carington's
work on trance personalities. Proceedings of the
Society for Psychical Research, 1937, 44, 223-275.
Thouless, R. H. Scientific method and the use of
statistics. In F. C. Bartlett, F. Ginsberg, E. J.
Lindgren, & R. L. Thouless (Eds.), The study of
society. London: Kegan Paul, 1939.
Tippett, L. H. C. The methods of statistics. London:
Williams and Norgate, 1931.
Treloar, A. E. An outline oj biometric analysis.
Minneapolis, Minn.: Burgess, 1936.
Tsao, F. Tests of statistical hypotheses in the case
of unequal or disproportionate numbers of obser-
vations in the subclasses. Psychometrika, 1942, 7,
195-212.
Tsao, F. General solution of the analysis of vari-
ance and covariance in the case of unequal or
disproportionate numbers of observations in the
subclasses. Psychometrika, 1946, 11, 107-128.
Tsao, F., & Johnson, P. 0. A note on solutions of
analysis of variance for the problem of unequal
or disproportionate subclass numbers. Journal oj
Experimental Education, 1946, 14, 253-255.
Walker, H. M. Degrees of freedom. Journal of Edu-
cational Psychology, 1940, 31, 253-269.
Walker, H. M. Elementary statistical methods. New
York: Holt, 1943.
Walker, H. M. Certain unsolved statistical problems
of importance to psychological research. Harvard
Educational Review, 1947, 17, 297-304.
Webb, W. B., & Lemmon, V. W. A qualification in
the use of analysis of variance. Psychological
Bulletin, 1950, 47, 130-136.
Wike, E. L., & Church, J. D. Comments on Clark's
"The language-as-fixed-effect fallacy." Journal of
Verbal Learning and Verbal Behavior, 1976, IS,
249-256.
Wilks, S. S. Certain generalizations in ANOVA. Bio-
metrika, 1932, 24, 471-474.
Wilks, S. S. Mathematical statistics. Princeton, N.J.:
Princeton University Press, 1946.
Yates, F. The influence of "Statistical methods for
research workers" on the development of the
science of statistics. Journal oj the American Sta-
tistical Association, 1951, 46, 19-34.
Yates, F. Sir Ronald Fisher and the design of ex-
periments. Biometrics, 1964, 20, 307-321.
Yule, G. U., & Kendall, M. G. An introduction to
the theory of statistics. London: Griffin, 1937.
Received October 23, 1978 •