t Tests see Catalogue of Parametric
one may resort to hypotheses about the nature of the
Tests
Tanimoto Coefficient see
Proximity Measures
Tau-Equivalence see Conditional
Standard Errors of Measurement
Tau-Equivalent and
Congeneric Measurements
It is a well-known fact that the reliability of a test,
defined as the ratio of true score to observed score
variance, cannot generally be determined from a
single test administration, but requires the use of a
parallel test. More often than not, parallel tests are not
available. In such cases, two approaches are popular
to obtain indirect information on the reliability of the
test: either lower bounds to reliability can be used, or
test parts.
Evaluating lower bounds to the reliability, such
as Guttman’s λ3 [6], better known as coefficient
alpha [4] has gained wide popularity. A lower bound
that is nearly always better than alpha is Guttman’s
λ4 . It is the highest alpha that can be obtained by
splitting up the items in two parts (not necessar-
ily of equal numbers) and treating those two parts
as novel ‘items’. Jackson and Agunwamba [8] pro-
posed the greatest lower bound (glb) to reliability.
It exceeds all conceivable lower bounds by using the
available information implied by the observed covari-
ance matrix exhaustively. A computational method
for the glb has been proposed by Bentler & Wood-
ward [3], also see [19]. Computation of the glb has
been implemented in EQS 6.
When lower bounds are high enough, the reliabil-
ity has been shown adequate by implication. How-
ever, when lower bounds are low, they are of limited
value. Also, some lower bounds to reliability involve
a considerable degree of sampling bias. To avoid
these problems, it is tempting to look to alternative
approaches, by introducing hypotheses on the nature
of the test parts, from which the reliability can be
determined at once. Two of such hypotheses are well-
known in classical test theory.
Tau Equivalence
The first hypothesis is that of (essentially) tau-
equivalent tests. Test parts X1 , . . . , Xk are essentially
1992 Tau-Equivalent and Congeneric Measurements
tau-equivalent when for i, j = 1, . . . , k,
Tj = Ti + aij . (1)
This implies that the true scores of the test parts
are equal up to an additive constant. When the
additive constants are zero, the test parts are said
to be tau-equivalent. Novick and Lewis [14] have
shown that coefficient alpha is the reliability (instead
of merely a lower bound to it) if and only if the test
parts are essentially tau-equivalent.
Unfortunately, the condition for essential tau-
equivalence to hold in practice is prohibitive: All
covariances between test parts must be equal. This
will only be observed when k = 2 or with con-
trived data. Moreover, the condition of equal covari-
ances is necessary, but not sufficient for essential
tau-equivalence. For instance, let Y1 , Y2 and Y3 be
three uncorrelated variables with zero means and unit
variances, and let the test parts be X1 = Y2 + Y3 ,
X2 = Y1 + Y3 , and X3 = Y1 + Y2 . Then, the covari-
ance between any two test parts is 1, but the test
parts are far from being essentially tau-equivalent,
which shows that having equal covariances is neces-
sary but not sufficient for essential tau-equivalence.
Because the necessary condition will never be satis-
fied in practice, coefficient alpha is best thought of
as a lower bound (underestimate) to the reliability of
a test.
Congeneric Tests
A weaker, and far more popular hypothesis, is that of
a congeneric test, consisting of k test parts satisfying
Tj = cij Ti + aij , (2)
which means that the test parts have perfectly cor-
related true scores. Equivalently, the test parts are
assumed to fit a one-factor model. Essential tau equiv-
alence is more restricted because it requires that the
weights cij in (2) are unity. For the case k = 3,
Kristof has derived a closed-form expression for the
reliability of the test, based on this hypothesis. It
is always at least as high as alpha, and typically
better [10]. Specifically, there are cases where it coin-
cides with or even exceeds the greatest lower bound
to reliability [20].
For k > 3, generalized versions of Kristof’s coef-
ficients have been proposed. For instance, Gilmer and
Feldt [5] offered coefficients that could be evaluated
without having access to powerful computers. They
were fully aware that these coefficients would be
supplanted by common factor analysis (see History
of Factor Analysis: A Psychological Perspective)
based coefficients by the time large computers would
be generally available. Nowadays, even the smallest
of personal computers can evaluate the reliability of
a test in the framework of common factor analysis,
assuming that the one-factor hypothesis is true. For
instance, McDonald [13], also see Jöreskog [9] for a
similar method, proposed estimating the loadings on
a single factor and evaluating the reliability as the
ratio of the squared sum of loadings to the test vari-
ance. When k = 3, this yields Kristof’s coefficient.
Coefficients like Kristof’s and McDonald’s have been
considered useful alternatives to lower bounds like
glb, because they aim to estimate, rather than under-
estimate, reliability, and they lack any reputation of
sampling bias. However, much like the hypothesis of
essential tau-equivalence, the one-factor hypothesis is
problematic.
The Hypothesis of Congeneric Tests is
Untenable for k > 3, and Undecided
Otherwise
The hypothesis of congeneric tests relies on the
existence of communalities to be placed in the
diagonal cells of the item covariance matrix, in
order to reduce the rank of that matrix to one.
The conditions under which this is possible have
been known for a long time. Spearman [17] already
noted that unidimensionality is impossible (except
in contrived cases) when k > 3. Accordingly, when
k > 3, factor analysis with only one common factor
will never give perfect fit. More generally, Wilson
and Worcester [23], Guttman [7], and Bekker and
De Leeuw [1] have argued that rank reduction of
a covariance matrix by communalities does not
carry a long way. Shapiro [15] has proven that the
minimal reduced rank that can possibly be achieved
will be at or above the Ledermann bound [11]
almost surely. It means that the minimal reduced
rank is almost surely at or above 1 when k = 3,
at or above 2 when k = 4, at or above 3 when
k = 5 or 6, and so on. The notion of ‘almost
surely’ reflects the fact that, although covariance
matrices that do have lower reduced rank are easily
 Tau-Equivalent and Congeneric Measurements
constructed, they will never be observed in practice.
It follows that the hypothesis of congeneric tests
is nearly as unrealistic as that of essential tau-
equivalence. It may be true only when there are three
or fewer items.
Even when reduction to rank 1 is possible, this is
not sufficient for the hypothesis to be true: We merely
have a necessary condition that is satisfied. The
example of X1 , X2 , and X3 with three uncorrelated
underlying factors Y1 , Y2 , and Y3 , given above in the
context of tau-equivalence, may again be used to
demonstrate this: There are three underlying factors,
yet communalities that do reduce the rank to 1 do
exist (being 1, 1, and 1). The bottom line is that the
hypothesis of congeneric tests cannot be rejected, but
still may be false when k = 3 or less, and it has to be
rejected when k > 3. ‘Model-based coefficients are
not useful if the models are not consistent with the
empirical data’ [2]. Reliability coefficients based on
the single-factor hypothesis are indeed a case where
this applies.
Sampling Bias
Lower bounds to reliability do not rest on any
assumption other than that error scores of the test
parts correlate only with themselves and with the
observed scores they belong with. On the other hand,
lower bounds do have a reputation for sampling bias.
Whereas coefficient alpha tends to slightly underesti-
mate the population alpha [21, 24], Guttman’s λ4 , and
the greatest lower bound in particular, may grossly
overestimate the population value when computed
in small samples. For instance, when k = 10 and
the population glb is 0.68, its average sample esti-
mate may be as high as 0.77 in samples of size
100 [16].
It may seem that one-factor-based coefficients
have a strong advantage here. But this is not true.
When k = 3, Kristof’s coefficient often coincides
with glb, and for k > 3, numerical values of McDon-
ald’s coefficient are typically very close to the glb. In
fact, McDonald’s coefficient demonstrates the same
sampling bias as glb in Monte Carlo studies [20].
Because McDonald’s and other factor analysis–based
coefficients behave very similarly to the glb and have
the same bias problem, and, in addition, rely on a
single-factor hypothesis, which is either undecided
or false, the glb is to be preferred.
1993
Bias Correction of the glb
Although the glb seems superior to single-factor-
based coefficients of reliability, when the test is
hypothesized to be unidimensional, this does not
mean that the glb must be evaluated routinely for
the single administration of an arbitrary test. The
glb has gained little popularity, mainly because of
the sampling bias problem. Bias correction methods
are under construction [12, 22], but still have not
reached the level of accuracy required for practical
applications. The problem is especially bad when the
number of items is large relative to sample size. Until
these bias problems are over, alpha will prevail as the
lower bound to reliability.
Reliability versus Unidimensionality
Reliability is often confused with unidimensional-
ity. A test can be congeneric, a property of the true
score parts of the items, yet have large error vari-
ances, a property of the error parts of the items, and
the reverse is also possible. Assessing the degree
of unidimensionality is a matter of assessing how
closely the single factor fits in common factor anal-
ysis. Ten Berge and Sočan [20] have proposed a
method of expressing unidimensionality as the per-
centage of common variance explained by a single
factor in factor analysis, using the so-called Minimum
Rank Factor Method of Ten Berge and Kiers [18].
However, this is a matter of taste and others pre-
fer goodness-of-fit measures derived from maximum-
likelihood factor analysis.
References
[1] Bekker, P.A. & De Leeuw, J. (1987). The rank
of reduced dispersion matrices, Psychometrika 52,
125–135.
[2] Bentler, P.M. (2003). Should coefficient alpha be
replaced by model-based reliability coefficients? Paper
Presented at the 2003 Annual Meeting of the Psychome-
tric Society, Sardinia.
[3] Bentler, P.M. & Woodward, J.A. (1980). Inequalities
among lower bounds to reliability: with applications to
test construction and factor analysis, Psychometrika 45,
249–267.
[4] Cronbach, L.J. (1951). Coefficient alpha and the internal
structure of tests, Psychometrika 16, 297–334.
1994 Teaching Statistics to Psychologists
[5] Gilmer, J.S. & Feldt, L.S. (1983). Reliability estimation
for a test with parts of unknown lengths, Psychometrika
48, 99–111.
[6] Guttman, L. (1945). A basis for analyzing test-retest
reliability, Psychometrika 10, 255–282.
[7] Guttman, L. (1958). To what extent can communalities
reduce rank, Psychometrika 23, 297–308.
[8] Jackson, P.H. & Agunwamba, C.C. (1977). Lower
bounds for the reliability of the total score on a
test composed of non-homogeneous items: I. Algebraic
lower bounds, Psychometrika 42, 567–578.
[9] Jöreskog, K.G. (1971). Statistical analysis of sets of
congeneric tests, Psychometrika 36, 109–133.
[10] Kristof, W. (1974). Estimation of reliability and true
score variance from a split of the test into three arbitrary
[11]
parts, Psychometrika 39, 245–249.
Ledermann, W. (1937). On the rank of reduced correla-
tion matrices in multiple factor analysis, Psychometrika
2, 85–93.
[12] Li, L. & Bentler, P.M. (2004). The greatest lower
bound to reliability: Corrected and resampling estima-
tors. Paper presented at the 82nd symposium of the
Behaviormetric Society, Tokyo.
[13] McDonald, R.P. (1970). The theoretical foundations of
principal factor analysis, canonical factor analysis, and
alpha factor analysis, British Journal of Mathematical
and Statistical Psychology 23, 1–21.
[14] Novick, M.R. & Lewis, C. (1967). Coefficient alpha and
the reliability of composite measurements, Psychome-
trika 32, 1–13.
[15] Shapiro, A. (1982). Rank reducibility of a symmetric
matrix and sampling theory of minimum trace factor
analysis, Psychometrika 47, 187–199.
[16] Shapiro, A. & Ten Berge, J.M.F. (2000). The asymptotic
bias of minimum trace factor analysis, with applications
to the greatest lower bound to reliability, Psychometrika
65, 413–425.
[17] Spearman, C.E. (1927). The Abilities of Man, McMillan,
London.
[18] Ten Berge, J.M.F. & Kiers, H.A.L. (1991). A numer-
ical approach to the exact and the approximate mini-
mum rank of a covariance matrix, Psychometrika 56,
309–315.
[19] Ten Berge, J.M.F., Snijders, T.A.B. & Zegers, F.E.
(1981). Computational aspects of the greatest lower
bound to reliability and constrained minimum trace
factor analysis, Psychometrika 46, 357–366.
[20] Ten Berge, J.M.F. & Sočan, G. (2004). The great-
est lower bound to the reliability of a test and the
hypothesis of unidimensionality, Psychometrika 69,
611–623.
[21] Van Zijl, J.M., Neudecker, H. & Nel, D.G. (2000). On
the distribution of the maximum likelihood estimator of
Cronbach’s alpha, Psychometrika 65, 271–280.
[22] Verhelst, N.D. (1998). Estimating the reliability of
a test from a single test administration, (Measure-
ment and Research Department Report No. 98-2),
Arnhem.
[23] Wilson, E.B. & Worcester, J. (1939). The resolution of
six tests into three general factors, Proceedings of the
National Academy of Sciences 25, 73–79.
[24] Yuan, K.-H. & Bentler, P.M. (2002). On robustness
of the normal-theory based asymptotic distributions of
three reliability coefficient estimates, Psychometrika 67,
251–259.
JOS M.F. TEN BERGE
Teaching Statistics to
Psychologists
In the introduction to his 1962 classic statistical text,
Winer [7] describes the role of the statistician in a
research project as similar to that of an architect; that
is, determining whether the efficacy of a new drug
is superior to that of competing products is similar
to designing a building with a particular purpose
in mind and in each case, there is more than one
possible solution. However, some solutions are more
elegant than others and the particulars of the situation,
whether they are actual patient data or the size and
placement of the building site, place boundaries on
what can and cannot be accomplished.
What is the best way to teach the science and art
of designing and conducting data analysis to today’s
graduate students in psychology? Perhaps history
can be our guide, and so we begin by first asking
what graduate education in statistics was like when
today’s senior faculty were students, then ask what
is current common practice, and finally ask what the
future might hold for tomorrow’s graduate students.
I propose the following:
• Graduate training in statistics has greatly changed
over the last few decades in ways that are both
helpful and harmful to students attempting to
master statistical methodology.
• The major factor in this change is the
development of computerized statistical packages
(see Software for Statistical Analyses) which,
when used in graduate education, cause students
trained in experimental design to be more
broadly but less thoroughly trained.