STATISTICS

Contents
An Overview of Statistics in Education
Analysis and Interpretation of Multivariate Data
Analysis of Covariance
Analysis of Extreme Values in Education
Analysis of Variance
Bayesian Statistical Analysis
Bootstrap Method
Canonical Correlation
Categorical Data Analysis
Causal Inference
Cluster Analysis: Overview
Cognitive Psychology and Educational Statistics
Computational Statistics
Continuous Probability Distributions
Correspondence Analysis
Data Mining
Decision Theory
Design of Experiments
Discrete Probability Distributions
Discrimination and Classification
Empirical Bayes Methods
Evaluation Research
Exploratory Data Analysis
Factor Analysis: An Overview and Some Contemporary Advances
Generalized Linear Mixed Models
Generalized Linear Models
Generating Random Numbers
Goodness-of-Fit Testing
Graphical Models
Growth Modeling
Hierarchical Linear Models
Hypothesis Testing and Confidence Intervals
Instrumental Variables
Jackknife Methods
Large-sample Statistical Methods
Latent Class Models
Markov Chain Monte Carlo
Matrix Algebra
Measure of Association
Measures of Central Tendency
Measures of Dispersion, Skewness and Kurtosis
Meta Analysis
Missing Data
Model Selection
Monte Carlo Methods
Multidimensional Scaling
Multiple Comparisons
Multivariate Analysis of Variance
Multivariate Linear Regression

1
2 Statistics

Multivariate Normal Distribution

Nonlinear Regression Analysis
Nonparametric Statistical Methods
Observational Studies
Order Statistics
Point Estimation Methods with Applications to Item Response Theory Models
Principal Components Analysis
Probability Theory
Recursive Partitioning
Sampling
Sequential Probability Ratio Test
Sequential Testing
Signal Detection Theory
Small Area Estimation
Statistical Analysis of Functional Data
Statistical Inequalities
Statistical Paradoxes
Statistical Power Analysis
Statistical Significance Versus Effect Size
Stochastic Processes
Structural Equation Models
Survival Data Analysis
The Normal Distribution and its Applications
Time Series Analysis
Univariate Linear Regression
Value-Added Models

An Overview of Statistics in Education

S Sinharay, ETS, Princeton, NJ, USA
ã 2010 Elsevier Ltd. All rights reserved.

Introduction
This article intends to provide an overview of the appli-
cation of statistics to the field of education. Statistics is a
vast field, with new topics such as proteomics, ensemble
sampling, and statistics in opthalmology being intro-
duced every now and then. The Statistics section of the
encyclopedia did not attempt to provide a summary of all
possible topics under the subject. Instead, this section
focuses on the topics in statistics that have found appli-
cations in the field of education. Most applications of
statistics to education are found in the area of educa-
tional measurement for the simple reason that statistics,
the science that deals with quantitative analysis of data,
inherently is related to measurement. (Several such appli-
cations – such as automated scoring, differential item
functioning, generalizability theory, and item response
theory (IRT) – are covered in the educational measure-
ment section of the encyclopedia and will not be
repeated here.) Hence, one could argue that some of
the topics included in the statistics section, or, that sev-
eral examples included in the articles in this section, deal
mostly with educational measurement and should have
belonged in the Educational measurement section of
this encyclopedia. In addition, some of the topics may
overlap to a small extent with other topics in this section
(e.g., generalized linear models are tools in categorical
data analysis, but this section has two separate articles on
the two topics). In any case, the number of applications
of statistics to education is on the rise. This is because of
(1) increases in computing power that have led people to
ask questions that could not have been answered 20 years
ago in a timely manner, and, (2) increases in the number
of educational tests, partially due to the No Child Left
Behind (NCLB) Act of 2001 in the USA that requires
annual testing in the schools and produces a lot of data.
Practitioners in education should find this section, together
with the Educational measurement section, helpful as these
two provide a comprehensive overview of the statistical
methods used in education.
 An Overview of Statistics in Education 3

Next, several topics in statistics that are relevant to the

field of education are described in brief along with, for
some topics, examples of actual applications to the field of
education. All of these topics and many more topics are
covered in one or more articles in the statistics section.

Exploratory Data Analysis

Examinees
After data are collected, often the first step in analyzing
the data is exploratory data analysis (EDA), which consists
of looking at data to see what they seem to say (Tukey,
1977) while relying on simple arithmetic and easy-to-
draw pictures or plots. The techniques used in EDA
include the following:

Plotting the data in bar charts, pie charts, histograms,
Youden plots, etc.,

Plotting simple statistics in plots such as mean plots,
standard deviation (SD) plots, box plots, etc., and

Positioning such plots so as to extract the maximum
information possible from them. Items
Consider Figure 1, which shows responses of 325 exam- Figure 1 A plot of the responses of 325 examinees to 15 mixed
inees to 15 items regarding mixed-number subtraction number subtraction items.
(Tatsuoka, 1984). An example item is 457  147. The items,
sorted according to decreasing proportion correct (i.e., 9 and 13 were higher in 2004 than in 1971 and the average
increasing difficulty), are shown along the x-axis in the score for 17-year-olds in 2004 was similar to that in 1971.
figure; the examinees, sorted according to increasing raw Measures of central tendency and measures of dispersion,
scores, are shown along the y-axis. A short black horizontal skewness, and kurtosis are discussed in the statistics sec-
line for an examinee and an item indicates a correct tion of the encyclopedia.
response. Some patterns are immediately visible from the
figure. For example, several examinees (24 out of 325) at
the top answer all items correctly (clear from the top of Measures of Association
the plot being completely black). The examinees with the
In education, it is often of interest to examine the amount
lowest scores could answer only two items correctly. Inter-
of association between a group of variables. For example,
estingly, these items ð34  34 and 378  2Þ are not the two
test administrators administering several tests simulta-
easiest items and can be solved without any knowledge of
neously to students will like that scores on different tests
mixed-number subtraction. Further, the lower half of the
(e.g., in reading, writing, mathematics, and science) do
examinees rarely answered any difficult items correctly.
not correlate highly with each other. High correlations
Wainer (2000, 2005) provided several applications of
between such scores may raise questions about the need
EDA to education.
of so many tests. Choice of the appropriate measures of
association depends on whether the variables of interest
are continuous, discrete ordinal, or discrete nominal. For
Simple Summary Measures
example, if the scores on the abovementioned tests are
It is often important to examine simple summary measures given on a scale of 1 to 100 in 1-point increments, a
such as the mean and standard deviation (SD) of numerical correlation coefficient may be the appropriate measure
information. For example, the Digest of Education Statistics, of association. On the other hand, if scores are given as
an annual publication of the National Center for Educa- 0 or 1 (where a score of 1 means that a student is good on a
tion Statistics (NCES) in the United States of America, subject and 0 otherwise), an odds ratio or the Kendall’s
includes the number of schools and colleges, teachers, tau (see, e.g., Agresti, 2002) may be the appropriate mea-
enrolments, and graduates, in addition to other informa- sure. It is often of interest to examine the association
tion on education. Further, the 2007 Digest reports that the between two groups of variables, X ¼ ðX1 ; X2 ; . . . Xk Þ
average salary for teachers in 2005–06 was US$49 109, and Y ¼ ðY1 ; Y2 ; . . . Y1 Þ. Canonical correlation analysis
about 1% higher than in 1995–96, after adjustment for attempts to find linear combinations of the two groups
inflation, and that the average reading scores at ages with high correlations.
4 Statistics
Probability Theory
Providing a mathematical description of our beliefs about
the systematic properties of a random phenomenon is the
first step in several statistical analyses. Usually this is
accomplished with the help of probability theory, the
branch of mathematics that describes the pattern of chance
outcomes. A key idea in probability theory is that of a
random variable, which is a variable whose value is a
numerical outcome of a random phenomenon, and its
distribution. There are several types of random variables,
and the articles in the statistics section, on discrete and
continuous probability distributions, provide detailed
descriptions of them. A knowledge of random variables
that follow a normal distribution is essential in several
applications of statistics to education (e.g., the distribution
plays a key role in topics such as factor analysis, analysis of
variance, linear regression, and multilevel models) – hence
two articles in the statistics section are devoted to these
variables. Stochastic processes are sequences of random
variables and are often of interest in probability theory
(e.g., the path traced by a molecule as it travels in a liquid
or a gas can be modeled using a stochastic process). Order
statistics is another area of some importance in probability
theory. For example, the ordered scores on a test of a
sample of students represent the order statistics for the
test score variable for the sample.
Statistical Inference
Statistical inference consists in the use of statistics to draw
conclusions about some unknown aspect of a population
based on a random sample from that population. Some
preliminary conclusions may be drawn by the use of EDA
or by the computation of summary statistics as well, but
formal statistical inference uses calculations based on
probability theory to substantiate those conclusions. Sta-
tistical inference can be divided into two areas: estimation
and hypothesis testing. In estimation, the goal is to
describe an unknown aspect of a population, for example,
the average scholastic aptitude test (SAT) writing score
of all examinees in the State of California in the USA.
Estimation can be of two types, point estimation and
interval estimation, depending on the goal of the applica-
tion. The goal of hypothesis testing is to decide which of
two complementary statements about a population is true.
Two such complementary statements may be: (1) the stu-
dents of California score higher on an average on SAT
writing than the students of Texas, and (2) the students of
California score lower on an average on SAT writing than
the students of Texas. Point estimation is discussed in the
statistics section of the encyclopedia. Details on interval
estimation and hypothesis testing, and power analysis,
which play a key role in hypothesis testing are also dis-
cussed in the statistics section of the encyclopedia. Often,
an investigator has to perform several hypothesis tests
simultaneously. For example, one may want to compare
the SAT critical reading scores of several pairs of schools
belonging to a geographical region. The article on multi-
ple comparison in the statistics section of the encyclope-
dia, discusses how to handle such a situation in an
appropriate manner.
Principal-Components Analysis
Principal-component analysis (PCA) is often used to
reduce multidimensional data sets to a lower number of
dimensions for analysis. PCA retains those characteristics
of the data set that contribute most to its variance, by
keeping lower-order principal components (the ones that
explain a large part of the variance present in the data)
and ignoring higher-order ones (that do not explain much
of the variance present in the data). Such low-order com-
ponents often contain the most important aspects of
the data. For example, in the National Assessment of
Educational Progress (NAEP), a large-scale educational
survey conducted in the USA (see, e.g., von Davier et al.,
2007), values of several hundred background variables are
collected for each examinee. As it is important to use the
information contained in these background variables in
the statistical model used in NAEP, several principal
components are computed from the original background
variables and are used, instead of the original variable
values, in the final NAEP statistical model (see, e.g.,
Jenkins et al., 2001: 377–378).
Factor Analysis
The beginning of factor analysis lies in the early attempts
of Karl Pearson, Charles Spearman, and others to define
and measure intelligence – hence factor analysis has been
a popular tool in education. When several tests are admi-
nistered to a group of examinees, one aspect of validation
may involve determining whether there are a few under-
lying abilities or skill variables that govern the examinees’
performances on the tests. Factor analysis, which is often
used in such situations, attempts to describe the covari-
ance among several variables in terms of a few underlying,
but unobservable, random variables called factors. For
example, correlations from the group of test scores in
classics, French, English, mathematics, and music col-
lected by Spearman suggested an underlying intelligence
factor (Johnson and Wichern, 1998). Factor analysis can be
considered to be an extension of PCA – both of these
techniques approximate the covariance matrix among the
variables. A factor analysis is exploratory if the investiga-
tor does not have a hypothesis about the number of factors
measured by the tests, and confirmatory if the investigator
has such hypotheses and conducts statistical tests of them.

Structural-Equation Modeling
Structural-equation modeling is an extension of factor
analysis and is a methodology designed primarily to test
substantive theory from empirical data. For example, a
theory may suggest that certain mental traits do not affect
other traits and that certain variables do not load on
certain factors, and that structural equation modeling
can be used to test the theory. (A mental trait is a habitual
pattern of behavior, thought and emotion.) A structural-
equation model (SEM) is a system of linear equations
among several unobservable variables (constructs) and
observed variables. An SEM is composed of two parts: a
structural part, linking the constructs to each other (usu-
ally, this part expresses the endogenous or dependant
constructs as linear functions of the exogenous or inde-
pendent constructs), and a measurement part, linking the
constructs to observed measurements. The second part
resembles a confirmatory factor analysis model. The
SEMs can be displayed in visual form – these displays
are called path diagrams. The full model is then estimated
from a data set and inferences drawn.
Classification and Discriminant Analysis
The goal of classification and discriminant analysis are
to describe, either graphically or algebraically, the differ-
ent features of observations from several known groups,
and to sort new objects (whose group membership is
unknown) into the groups. For example, consider a medi-
cal school applications office that has the test scores and
other college records of several students who became
MDs and several others who did not. When a new appli-
cation comes, the office may want to classify the applicant
into likely to become MD and unlikely based on the test
scores and college records.
Cluster Analysis
Cluster analysis is a technique to group similar observa-
tions into a number of clusters based on the observed
values of several variables for each individual. Cluster
analysis is similar in concept to discriminant analysis.
The group membership of a sample of observations is
known upfront in the latter while it is not known for any
observation in the former. As an application of cluster
analysis to education, Everitt (1990) describes a data set
that has achievement test scores on reading and arithme-
tic for children in the fourth and sixth grades of 25
schools and the interest is in identifying different levels
of performance and assessing similarities and differences
in the patterns of change from fourth to sixth grade –
cluster analysis is the most appropriate technique for
the example.
An Overview of Statistics in Education 5
Multidimensional Scaling
Multidimensional scaling is related to cluster analysis and
assigns a location of each sample observation in a low-
dimensional space so that their distances are close to their
actual distances in multiple dimensions. This idea can be
illustrated by reference to map construction. Suppose that
one is given a table with the distances between several
American cities (and not given the exact locations of them
on a map). One could attempt to place the cities on a map
so that the distances between the cities on the map are as
close as possible to the distances on the table. Once this
step is done, the map is used to draw conclusions on the
sample; often, a cluster analysis is performed on these
low-dimensional representations of the observations.
Categorical Data Analysis
A categorical variable has a measurement scale that con-
sists of a set of categories. For example, the response of a
student to a test question is often measured as correct or
incorrect.
Categorical variables have two types of scales – nominal
(when the categories do not have a natural ordering; e.g.,
gender and religion) and ordinal (when the categories have
a natural ordering; e.g., patient condition being measured as
good, fair, and serious). Several statistical methods and
concepts have been devised for categorical data. For exam-
ple, distributions such as the binomial and Poisson distribu-
tions, models such as generalized linear models, generalized
linear mixed models, logistic regression model, and log-
linear models, and methods such as analysis of contingency
tables were devised to handle categorical data.
Analysis of Variance, Analysis of Covariance,
and Multivariate Analysis of Variance
Analysis of variance (ANOVA) is the statistical procedure
of comparing the means of a variable across several groups
of individuals. For example, ANOVA may be used to
compare the average SAT critical reading scores of several
schools. The name of the technique arises from the fact
that the first step in an ANOVA is to partition the variance
present in the observations into several components. The
ANOVA method was the second most frequently used
data-analysis procedure in a survey of articles published
between 1971 and 1998 in three reputed educational-
research journals (Hsu, 2005). Generalizability theory
(Cronbach et al., 1963), which is a competitor to the
classical theory of reliability of tests, usually applies
ANOVA procedures to test scores.
Analysis of covariance (ANCOVA) is used when, like
in ANOVA, the interest is in comparing several means, but
the investigator also has the values of an additional variable
that influences the variable of interest. For example,
6 Statistics
ANCOVA may be used to compare the average SAT
critical reading scores of several schools where the pre-
liminary scholastic aptitude test/national merit scholar-
ship qualifying test (PSAT/NMSQT) critical reading
score of each examinee is available in addition to the
SAT critical reading score. (The PSAT/NMSQT is sup-
posed to provide firsthand practice for the SAT.)
Multivariate analysis of variance (MANOVA) is used
to compare means of several variables simultaneously
across several groups of individuals. For example, one
could apply MANOVA to simultaneously compare the
average scores on several subjects across several schools.
Longford (1990) provides such an example.
Design of Experiments
An experiment is a test in which purposeful changes are
made to the input variables of a process or system so that
one may observe and identify the reasons for changes
that may be observed in the output response. Design of
experiments is the science of planning and conducting
experiments and analyzing the resulting data so that
valid and objective conclusions can be drawn. For exam-
ple, the education ministry of a country may be interested
in conducting an experiment to find out if a particular
style of teaching mathematics helps children of fourth
grade to learn the subject better than the existing style.
In designing an experiment, the ministry has to make sure
that any difference that they might observe in the out-
come for students who were taught using the new style
and those who were taught using the existing style cannot
be attributed to a factor other than the teaching style (e.g.,
if they assign all students from rural areas to the new style
and all students from urban areas to the existing style,
then a difference can be attributed to the rural vs. urban
difference). There are three basic principles in design of
experiments – randomization (which means that the
assignment of the experimental material and the order in
which the individuals receive the experimental material are
randomly determined), replication (which refers to repeats
of each experimental condition), and blocking (which is the
grouping of individuals to create several homogeneous
groups before assigning the experimental material).
Observational Studies
In some situations, it is not possible (for reasons such as
budget constraints and ethical issues) to design an exper-
iment to answer a question or to test a hypothesis.
For example, consider that the interest is in finding
whether watching television is affecting the class grades
of students. As watching television may have adverse
effects, it will be unethical for one to design an experiment
and randomly assign students to watch television for dif-
ferent number of hours per day. In such situations, often
the only way is to conduct an observational study, that is, to
collect data on a sample of individuals, and apply an
appropriate method to draw conclusions. In the example
above, one could record the number of hours of television
watched by the students and examine its association with
their grades. Only limited number of conclusions can be
drawn from an observational study. Any observed differ-
ence or association has several reasonable alternative
explanations. For example, an observed lower score of
students watching television longer can be caused by
such students having less appropriate home and school
inputs, such as fewer books at home or parents who read
less to them. Huang and Lee (2009) investigated whether
television watching at ages 6–7 and 8–9 affects cognitive
development measured by math and reading scores at age
8–9 using a rich childhood longitudinal sample.
Causal Inference and Instrumental Variables
Suppose that an investigator is interested in testing a
hypothesis, for example, about the comparison of a new
educational program versus the existing program. Whether
the investigator performs a randomized experiment or an
observational study, he is faced with the question of how to
draw inferences about the causal effects of the new program.
In other words, if there is a performance difference between
the students who were administered the new educational
program and those who were administered the existing
program, the investigator would like an answer to the ques-
tion ‘‘Is the difference in performance of the two groups
caused by the difference in the educational program?’’ An
article in the statistics section discusses how causal infer-
ences can be made. Instrumental variables (IVs) are used to
estimate causal relationships when controlled experiments
are not feasible. An overview of instrumental variables and
of their possible applications to education is discussed in the
statistics section of the encyclopedia.
Sampling
There is a growing importance of survey information on
individuals, households, institutions, businesses, and envi-
ronmental resources. Typically, one wants to gather infor-
mation on a large group of individuals. However, time and
cost usually does not allow obtaining information from
each individual in the group. In such cases, one usually
gathers information on only a sample, which is a small
part of the large group. Sampling plays an essential role in
drawing conclusions about the large group (which is
called the population) from the information contained
in the sample. An example of an application of sam-
pling is the National Assessment of Educational Progress
(NAEP), an educational sampling survey (Allen et al., 2001).
NAEP is the only ongoing measure of what students in the
USA know and can do in a variety of subject areas and it
reports scores for different demographic groups based on

gender, ethnicity, school type, school location, etc. NAEP
draws a sample of students that is representative of the
whole student population, applies several statistical tech-
niques, and draws conclusions (an example of a conclusion
is that between 1992 and 2000, the percentage of fourth-
graders at or above the proficient-achievement level in
reading increased by a small, but statistically significant
amount) on the whole population based on information
contained in the sample.
Bayesian and Empirical Bayes Methods
In a traditional or frequentist statistical analysis, the
parameter of a probability model is considered an
unknown but nonrandom quantity and only the informa-
tion contained in the observed data is relevant for any
inference. On the contrary, a Bayesian analysis (see, for
e.g., Gelman et al., 2003) assumes that the parameter is a
random variable with a certain probability distribution,
referred to as the prior distribution. The prior distribution
quantifies the experimenter’s beliefs about the parameter
before observing the data. The next step in a Bayesian
approach is to update the prior distribution on the basis of
the likelihood function of the observed data through
Bayes’ theorem (Bayes, 1763). The resulting distribution
is referred to as the posterior distribution of the parameter
and summarizes the information in both the prior distri-
bution and in the data. The influence of the prior dis-
tribution on the posterior distribution becomes weaker as
the size of the observed data sample increases. The varia-
tion of the Bayesian methods in which the parameters of
the prior distribution are estimated from the observed
data is called empirical Bayes methods. Sinharay (2006)
provided a review of the applications of Bayesian methods
to educational measurement. Novick and Jackson (1974)
included several applications of Bayesian methods to
educational measurement. Other examples of applications
of Bayesian methods to education are Rubin (1983), who
applied Bayesian methods to three problems in educa-
tional measurement, Zwick et al. (1999), who applied an
empirical Bayes method to differential item function-
ing, and Sinharay (2005), who applied Bayesian model-
checking methods to assess the goodness of fit of IRT
models. Further details on Bayesian methods and empir-
ical Bayes methods (see, for example, Carlin and Louis,
1996) are discussed in the statistics section of the encyclo-
pedia. Decision theory, which is a Bayesian approach, is
concerned with identifying the values, uncertainties, and
other issues relevant in a given decision and the resulting
optimal decision. An article in the statistics section pro-
vides more details on decision theory.
Resampling Methods
Resampling methods (see, e.g., Efron, 1982) draw samples
from the observed data to draw certain conclusions about
An Overview of Statistics in Education 7
the population of interest. Two of the most popular resam-
pling methods are the jackknife and bootstrap. Both of these
are examples of nonparametric statistical methods.
Jackknife is used in statistical inference to estimate the
bias and standard error of a test statistic. The basic idea
behind jackknife lies in systematically recomputing the
statistic a large number of times, leaving out one observa-
tion or a group of observations at a time from the sample.
Estimates of the bias and variance of the statistic can be
calculated from this set of jackknife replications of the
statistic. The jackknife finds several applications in com-
plex sampling schemes, such as multistage sampling with
varying sampling weights – an example of such applica-
tion is NAEP, where the jackknife method is employed to
compute standard errors of estimates.
Bootstrap is a statistical method for estimating the
sampling distribution of an estimator by sampling with
replacement from the original sample, most often with the
purpose of deriving robust estimates of standard errors
and confidence intervals of a population parameter like a
mean, median, and correlation coefficient. It is often used
as a robust alternative to procedures based on parametric
assumptions, especially when those assumptions are in
doubt, or where parametric inference is impossible or
requires very complicated formulas for the calculation of
standard errors. See, for example, Hanson et al. (1993),
who applied the bootstrap method to compute the stan-
dard error of an equating method.
Nonparametric Inference
Nonparametric methods, or distribution-free methods,
are statistical methods that do not rely on assumptions
that the data are drawn from a given probability distribu-
tion. Nonparametric methods are often applied when less
is known about the data (so that a probability distribution
cannot be assumed). Due to the reliance on fewer assump-
tions, nonparametric methods are more robust (i.e., less
vulnerable to violations of assumptions). They are also
often applied because of their simplicity. Examples of
nonparametric methods are Pearson’s w2 test for assessing
independence in a contingency table, jackknife and boot-
strap methods for estimating the bias and variance of an
estimator, the Wilcoxon Mann–Whitney rank-sum test,
the permutation test, the Kolmogorov–Smirnov test for
assessing whether two distributions are the same, and
spline regression for estimating regression curves of a
dependent variable on several independent variables.
Multiple Linear Regression Models
Multiple linear regression models have been extensively
used in education (see, e.g., Hsu, 2005). Interestingly, the
name regression, borrowed from the title of the first
article on this subject (Galton, 1885), does not reflect
8 Statistics
either the importance or breadth of application of this
method. Multiple regression is the statistical procedure to
predict the values of a response (dependent) variable from
a collection of predictor (independent) variable values.
For example, if scores on multiple predictors and one
criterion are available, multiple regression may be used
to develop a single equation to predict criterion perfor-
mance from the set of predictors. Several applications of
multiple regression models can be found in the prediction
of first-year grade-point average in college from the SAT
scores and high school grade-point average (see, e.g.,
Kobrin et al., 2008). Multiple regression and multivariate
multiple regression, the case when there are more than
one dependent variables of interest and the interest is in
predicting them simultaneously from a set of predictor
variables, are discussed in the statistics section of the
encyclopedia.
Hierarchical Linear Models and Growth Models
In an application of linear regression, the observations are
assumed to be independent. When the assumption of
independence is likely to be violated, for example, in an
application in which one has data on several students who
belong to a few schools (so that the responses of the
students within each school are dependent), a popular
option is to employ hierarchical linear models (HLMs).
These models are also referred to as multilevel models
and random-effects regression models. The students con-
stitute the lower level while the schools constitute the
higher level in the example. Note that HLMs can also
be applied to repeated measures design or longitudinal
studies, where individuals are followed and their re-
sponses recorded several times over a certain period
of time; the repeated measures constitute the lower level
and the individuals constitute the higher level; exam-
ples of such models are growth models, where, for ex-
ample, the investigator measures the cognitive growth
of students by giving them several tests over a certain
period of time. Growth models are increasingly popular
in the US due to the NCLB Act of 2001 that puts special
emphasis on the cognitive growth of students. For exam-
ple, in December 2007, the US Secretary of Education
Margaret Spellings invited all eligible US states to submit
a growth model proposal for the 2007–08 school year.
Growth models and HLMs are discussed in the statistics
section of the encyclopedia.
Value-Added Models
The NCLB Act of 2001 in the US requires states to ensure
that there are quality teachers in every classroom, with
quality defined in terms of traditional criteria such as
academic training and fully meeting the state’s licensure
requirements (Braun, 2005). Interestingly, in this respect,
some states have taken the lead by seeking a quantitative
evaluation of teachers based on an analysis of the test-
score gains of their students. Such evaluations employ a
class of models called value-added models (VAMs). These
models require data that track individual students’ aca-
demic growth over several years in different subjects in
order to estimate the contributions that teachers make to
that growth. Thus, VAMs can be viewed as a special case of
growth models and, hence, of HLMs. Given their current
state of development, VAMs can be used to identify a group
of teachers who may reasonably be assumed to require
targeted professional development. These are the teachers
with the lowest estimates of relative effectiveness. Despite
the enthusiasm these models have generated among many
policymakers, several technical reviews of VAMs have
revealed a number of serious concerns and it is important
that such concerns be properly addressed before VAMs are
used to make important decisions.
Generalized Linear Models and Generalized
Linear Mixed Models
Linear regression models apply when the response vari-
able can be assumed to be a continuous variable or to be
normally distributed. However, in several applications in
education, the response does not belong to either of those
types. Suppose the interest is in finding out how the
socioeconomic status and average parents’ education for
a class of students affects their performance on a test. If we
have the scores on the test for each student, we can
employ a linear regression model regressing the test
scores on the socioeconomic status and average parent
education. However, if we do not have the scores, but only
know who passed the test and who did not (which is a
binary response), we cannot employ linear regression.
Generalized linear models (GLMs) can be used in situa-
tions like this. GLMs are extensions of the linear regres-
sion model to a wider class of response type such as binary
or count data. A GLM requires the specification of two
defining characteristics – the distribution of the response
and the link function that describes how the mean of the
response is linked to a linear combination of the predic-
tors. Generalized linear mixed models (GLMM) are
extensions of GLMs to the case when the individuals are
clustered (e.g., students belonging to different schools).
The statistics section of the encyclopedia includes two
articles, one each on GLM and GLMM.
Nonlinear Regression Methods
One is often interested is in studying how a set of inde-
pendent variables affect a dependent variable, but the
relationship between them cannot be assumed linear. So

the abovementioned models, all of which assume a linear
relationship, cannot be applied. Nonlinear regression
methods, which may be applicable in such situations to
predict the dependent variable from the independent
variables and recursive partitioning, or, classification and
regression trees method, which is another method that
may be applicable in such situations, are discussed in the
statistics section of the encyclopedia.
IRT Models
These models, with numerous applications in education,
are discussed in an article in the educational measure-
ment section and are not covered here.
Latent Class Models
A latent class model (LCM) relates a set of observed
discrete multivariate variables to a set of latent variables
(latent variables are not directly observed but are rather
inferred, mostly through a mathematical model, from
other variables that are observed; e.g., quality of life or
intelligence of a person is a latent variable). It is called an
LCM because the latent variable is discrete and divides
the population into several classes. A class is characterized
by a pattern of conditional probabilities that indicate the
chance that variables take certain values. For example,
Dayton and Macready (2006) discuss an application in
which the observed variables are the responses to ten
questions on matrix algebra on a test, the latent variable
refers to the knowledge of matrix algebra of students, and
the latent classes refer to masters and nonmasters on
matrix algebra. Given class membership, the conditional
probabilities specify the chance certain answers are cho-
sen. Within each latent class, the observed variables are
statistically independent (this is often called local inde-
pendence). This is an important aspect of LCMs. Usually,
the observed variables are statistically dependant. By
introducing the latent variable, independence is restored
in the sense that variables are independent within classes.
The association between the observed variables is thus
explained by the latent classes.
Time-Series Analysis
A time series is a sequence of data points, measured
typically at successive time points. Time series analysis
comprises methods that attempt to understand such time
series, often either to understand the underlying context
of the data points, or to make forecasts (predictions).
Forecasting using a time-series analysis consists of the
use of a model to forecast future events based on known
past events. An example in education is the prediction of
An Overview of Statistics in Education 9
the number of students who will take a test (e.g., SAT) at
an administration based on the numbers from the previ-
ous administrations of the test. A time-series model gen-
erally reflects the fact that observations close together
in time will be more closely related than observations
further apart. Three broad classes of time-series models
of practical importance are the autoregressive (AR) mod-
els, the integrated (I) models, and the moving average
(MA) models. There are models, such as the autoregres-
sive moving average (ARMA) and autoregressive inte-
grated moving average (ARIMA) that are combinations
of the above three.
Model Fit and Model Selection
Model fit analysis refers to an examination of whether
the statistical model employed in an application ade-
quately explains the important features of the data set
at hand. Model selection refers to the choice of the
statistical model that describes the data best among
several competing models. Model fit and model selec-
tion analysis for the linear models employed in educa-
tion do not pose any problems and proceed in a similar
manner as in any other statistics field, for example, by
using residual analysis, Akaike information criterion
(AIC) and Bayesian information criterion (BIC) (see,
e.g., Draper and Smith, 1998). However, model fit and
model selection analysis for the nonlinear models, espe-
cially for the IRT models, are not trivial, primarily
because the computations are not straightforward with
these models, the response variable is discrete so that
normality of the response cannot be assumed, and the
number of possible responses is huge so that there is
sparseness in the data. Fortunately, with the advent of
faster computers, there has been substantial work in
these areas. Swaminathan et al. (2006) provided a detailed
review of the literature on model fit of IRT models.
Several model fit statistics have been suggested for test-
ing different aspects of an IRT model: statistics for test-
ing the unidimensionality assumption of the IRT model,
item fit statistics, person fit statistics, and overall model
fit statistics. In applications of IRT models, it is impor-
tant to employ the appropriate model fit statistics
depending on the intended use of the model, and to
evaluate not only statistical significance, but also practi-
cal significance. For example, it may happen that the
value of a fit statistic is statistically significant so that
the model does not predict an aspect of the data, but the
misfit has negligible consequences operationally.
Kang and Cohen (2007) provided a detailed review of
model selection methods for IRT models. More techni-
ques such as the penalty criteria and the theoretical back-
ground behind several techniques are discussed in the
statistics section of the encyclopedia.
10 Statistics
Other Topics in Statistics
Large-sample methods are essential in the application of
several statistical methods as it is often important to know
the distribution of a test statistic for a large sample. An
overview of how to handle multivariate data, which is
common in the field of education, is discussed in the
statistics section of the encyclopedia. There are several
statistics topics that are of increasing interest to the field
of education and will most likely become more popular in
the coming years. Examples of such topics are analysis of
extreme values, association mining, the relationship
between cognitive psychology and educational statistics,
correspondence analysis, data mining, functional data
analysis, graphical models, instrumental variables, latent
class analysis, missing data analysis, Monte Carlo meth-
ods, Markov chain Monte Carlo methods, sequential
probability ratio test, small-area estimation, survival anal-
ysis, and signal detection theory. Hence, articles on these
topics are included in this section. Computational meth-
ods play a key role in statistics in optimizing the time
required to run computationally intensive statistical ana-
lyses on data. A knowledge of matrix algebra is important
in several topics in statistics (such as linear regression,
GLM, and ANOVA). Meta analysis, a technique for com-
bining information from several studies, statistical
inequalities and statistical paradoxes, which are three
topics that any user of statistical methods should be
aware of, are discussed in the statistics section of the
encyclopedia. Random number generation is often an
important issue in applications of statistics to education,
especially those involving Monte Carlo or Markov chain
Monte Carlo methods. Evaluation research, which is a
form of disciplined and systematic inquiry that is carried
out to arrive at an assessment or appraisal of an object,
program, practice, activity, or system, is discussed in the
statistics section of the encyclopedia.
Conclusions
The list of topics in statistics that is provided above, along
with the examples of applications of these topics to edu-
cation, demonstrate how statistics can be useful in educa-
tion. As more students are tested, access to data becomes
easier, computers become faster, and statistical software
packages become more accessible, statistics is sure to find
more applications in education. However, this situation
also suggests the use of caution on several grounds.
Unless the investigator is careful, the resulting inference
from a statistical analysis may be inappropriate. See, for
example, Kramer and Gigerenzer (2005), of how statistics
can be confusing and has been misused, Haller and Kraus
(2002) to see how the concept of statistical significance is
misunderstood by university students and their instruc-
tors alike, and Stigler (2005) to see how correlation is often
misinterpreted as causation. These works show that there is
a need to apply appropriate statistical methods to educa-
tional applications. The articles in the statistics and educa-
tional measurement sections of this encyclopedia (along
with the references and the further readings therein) will
help practitioners in the field of education to learn more
about statistics and to do a better job in choosing appropri-
ate statistical methods in their applications.
There is often a tendency to fit complicated statistical
models to real data. However, before using such a model, a
researcher should always make sure that the model per-
forms considerably better than a simpler alternative, both
substantively and statistically, and that the model is well
identified. It is important to remember the following
comment in Rubin (1983)
William Cochran once told me that he was relatively
unimpressed with statistical work that produced methods
for solving non-existent problems or produced compli-
cated methods which were at best only imperceptibly
superior to simple methods already available. Cochran
went on to say that he wanted to see statistical methods
developed to help solve existing problems which were
without currently acceptable solutions.
See also: Analysis and Interpretation of Multivariate Data;
Analysis of Covariance; Bayesian Statistical Analysis;
Bootstrap Method; Canonical Correlation; Categorical Data
Analysis; Cluster Analysis: Overview; Computational Statis-
tics; Continuous Probability Distributions; Decision Theory;
Design of Experiments; Discrete Probability Distributions;
Discrimination and Classification; Empirical Bayes Methods;
Evaluation Research; Exploratory Data Analysis; Factor
Analysis: An Overview and Some Contemporary Advances;
Generalizability Theory; Generalized Linear Mixed Models;
Generalized Linear Models; Generating Random Numbers;
Goodness-of-Fit Testing; Growth Modeling; Hierarchical
Linear Models; Hypothesis Testing and Confidence Inter-
vals; Instrumental Variables; Jackknife Methods; Large-
sample Statistical Methods; Latent Class Models; Matrix
Algebra; Measure of Association; Measures of Central
Tendency; Measures of Dispersion, Skewness and Kurtosis;
Meta Analysis; Model Selection; Multidimensional Scaling;
Multiple Comparisons; Multivariate Analysis of Variance;
Multivariate Linear Regression; Nonlinear Regression Analy-
sis; Nonparametric Statistical Methods; Observational Stud-
ies; Point Estimation Methods with Applications to Item
Response Theory Models; Probability Theory; Recursive
Partitioning; Sampling; Statistical Inequalities; Statistical
Paradoxes; Statistical Power Analysis; Statistical Signifi-
cance Versus Effect Size; Structural Equation Models; Time
Series Analysis; Univariate Linear Regression; Value-Added
Models.

Bibliography
Agresti, A. (2002). Categorical Data Analysis, 2nd edn. New York: Wiley.
Allen, N. L., Donoghue, J. R., and Schoeps, T. L. (2001). The NAEP
1998 Technical Report (NCES 2001-509). Washington, DC:
National Center for Education Statistics, U.S. Department of
Education.
Bayes, T. (1763). An essay towards solving a problem in the doctrine of
chances. Philosophical Transactions of the Royal Society of London
61.53, 370–418. [Reprinted with biographical note by Barnard,
G. A. (1958) Biometrika 45, 293–315.]
Braun, H. (2005). Using Student Progress to Evaluate Teaching:
A Primer on Value-Added Models. Princeton, NJ: Policy Information
Center, Educational Testing Service.
Carlin, B. P. and Louis, T. A. (1996). Bayes and Empirical Bayes
Methods for Data Analysis. London: Chapman and Hall.
Cronbach, L. J., Nageswari, R., and Gleser, G. C. (1963). Theory of
generalizability: A liberation of reliability theory. British Journal of
Statistical Psychology 16, 137–163.
Dayton, C. M. and Macready, G. B. (2006). Latent class analysis in
psychometrics. In Rao, C. R. and Sinharay, S. (eds.) Handbook of
Statistics, vol. 26, pp 421–446. Amsterdam: North-Holland/Elsevier.
Draper, N. R. and Smith, H. (1998). Applied Regression Analysis, 3rd
edn. New York: Wiley.
Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling
Plans. Philadelphia, PA: Society for Industrial and Applied
Mathematics.
Everitt, B. S. (1990). Cluster analysis. In Husen, T. and
Postlethwaite, N. (eds.) International Encyclopedia of Education,
2nd edn., pp 825–831. Oxford: Pergamon.
Galton, F. (1885). Regression towards mediocrity in heredity stature.
Journal of the Anthropological Institute 15, 246–263.
Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2003). Bayesian
Data Analysis. New York: Chapman and Hall.
Haller, H. and Kraus, S. (2002). Misinterpretations of significance:
A problem students share with their teachers? Methods of
Psychological Research 7, 1–20.
Hanson, B. A., Zeng, L., and Kolen, M. J. (1993). Standard
errors of Levine linear equating. Applied Psychological Measurement
17, 225–237.
Hsu, T. (2005). Research methods and data analysis procedures used
by educational researchers. International Journal of Research and
Method in Education 28(2), 109–133.
Huang, F. and Lee, M. (2009). Dynamic treatment effect analysis of TV
effects on child cognitive development. No 0906, Discussion Paper
Series, Institute of Economic Research, Korea University. http://
econpapers.repec.org/RePEc:iek:wpaper:0906.
Jenkins, F., Kaplan, B., and Lim, Y. (2001). Data analysis for the
national writing samples. In Allen, N. (ed.) The NAEP 1998
Technology Report, 359–370. Washington, DC: National Center
for Education Statistics.
Johnson, R. A. and Wichern, D. W. (1998). Applied
Multivariate Statistical Analysis, 4th edn. Upper Saddle River, NJ:
Prentice-Hall.
Kang, T. and Cohen, A. S. (2007). IRT model selection methods
for dichotomous items. Applied Psychological Measurement 31(4),
331–358.
Kobrin, J. L., Patterson, B. F., Shaw, E. J., Mattern, K. D., and Barbuti,
S. M. (2008). Validity of the SAT for predicting first-year college grade
An Overview of Statistics in Education 11
point average. College Board Research Report No. 2008-5.
New York: College Board.
Kramer, W. and Gigerenzer, G. (2005). How to confuse with statistics or:
The use and misuse of conditional probabilities. Statistical Science
20, 223–230.
Longford, N. T. (1990). Multivariate variance component analysis: An
application in test development. Journal of Educational Statistics
15(2), 91–112.
Novick, M. R. and Jackson, P. H. (1974). Statistical Methods
for Educational and Psychological Research. New York:
McGraw-Hill.
Rubin, D. B. (1983). Some applications of Bayesian statistics to
educational data. Statistician 32, 55–68.
Sinharay, S. (2005). Assessing fit of unidimensional item response
theory models using a Bayesian approach. Journal of Educational
Measurement 42, 375–394.
Sinharay, S. (2006). Bayesian methods in educational measurement. In
Upadhyay, S. K., Singh, U., and Dey, D. K. (eds.) Bayesian Statistics
and Its Applications, pp 422–437. New Delhi: Anamaya.
Stigler, S. M. (2005). Correlation and causation: A comment.
Perspective in Biology and Medicine 48(1), 88–94.
Swaminathan, H., Hambleton, R. K., and Rogers, H. J. (2006).
Assessing the fit of item response theory models. In Rao, C. R. and
Sinharay, S. (eds.) Handbook of Statistics, vol. 26, pp 683–718.
Amsterdam: North-Holland/Elsevier.
Tatsuoka, K. K. (1984). Caution indices based on item response theory.
Psychometrika 49, 95–110.
Tukey, J. W. (1977). Exploratory Data Analysis. Reading, MA:
Addison-Wesley.
von Davier, M., Sinharay, S., Oranje, A., and Beaton, A. (2007). The
statistical procedures used in national assessment of educational
progress: Recent developments and future directions.
In Rao, C. R. and Sinharay, S. (eds.) Handbook of Statistics, vol. 26,
pp 1039–1055. Amsterdam: Elsevier.
Wainer, H. (2000). Visual Revelations: Graphical Tales of Fate and
Deception from Napoleon Bonaparte to Ross Perot, 2nd edn.
Hillsdale, NJ: Erlbaum.
Wainer, H. (2005). Graphic Discovery: A Trout in the Milk and
Other Visual Adventures. Princeton, NJ: Princeton
University Press.
Zwick, R., Thayer, D. T., and Lewis, C. (1999). An empirical Bayes
approach to Mantel–Haenszel DIF analysis. Journal of Educational
Measurement 36, 1–28.
Further Reading
Crocker, L. and Algina, J. (1986). Introduction to Classical and Modern
Test Theory. New York: Harcourt Brace Jovanovich College
Publishers.
Relevant Website
http://www.ed.gov – US Department of Education.