Chapter 36

An Introduction to Item
Response Theory Models
and Their Application in the
Assessment of Noncognitive
Traits
Steven P. Reise and Tyler M. Moore
Copyright American Psychological Association. Not for further distribution.

Item response theory (IRT; Embretson & Reise,
2000; Hambleton, Swaminathan, & Rogers, 1991;
de Ayala, 2009) refers to a class of mathematical
models relating individual differences on one or
more latent variables to the probability of respond-
ing to a scale item in a specific way. A response of
“3” on a 5-point personality item, a correct answer
on a college entrance exam item, and a clinician’s
rating of an adolescent’s anxiety are all item
responses that can potentially be related (probabilis-
tically) to a latent psychological variable. IRT mod-
els, which focus on characterizing how individual
differences on a latent variable interact with item
properties to produce a response, contrast sharply
with classical test theory (Lord & Novick, 1968)
procedures, which focus on understanding the
statistical properties of a composite scale score
(e.g., estimating reliability).
Development of IRT models and associated
methods was originally motivated by applied prob-
lems in large-scale multiple-choice aptitude testing
(e.g., how to efficiently administer different test
items to individuals but still compare them on the
same scale, how to link different sets of items mea-
suring the same construct onto the same scale).
Lately, however, applications of IRT models to per-
sonality, psychopathology, and patient-reported
outcomes measurement have been increasing (see
Reise & Waller, 2009, for review). Given the
authors’ interests and expertise, this chapter
describes IRT models mostly through the lens of
these latter, noncognitive measurement contexts.
Regardless of context, researchers from a variety
of fields have been keenly interested in the potential
of IRT modeling as an alternative to traditional psy-
chometric approaches to scale construction, item
analysis, scale administration, and scoring individ-
ual differences. As reviewed in Reise, Ainsworth,
and Haviland (2005), IRT models potentially offer
many attractive features. For example, through
inspection of item and scale information functions, a
researcher can gain a better understanding of how
well an item, or scale, functions (e.g., measurement
precision) across different ranges of a latent variable.
This inspection is often done graphically. Moreover,
because of the IRT item and person parameter
invariance properties, IRT models can be used to
either place items from different instruments onto a
common scale, or place individuals who responded
to different items onto a common scale. In turn, this
facilitates the analysis of differential item function-
ing across demographic groups (i.e., do the items
measure the same latent variable in the same way
across different groups?) as well as the creation of

This work was supported by the Consortium for Neuropsychiatric Phenomics and National Institutes of Health (NIH) Roadmap for Medical Research
Grants UL1-DE019580 (Robert Bilder, Principal Investigator [PI]) and RL1DA024853 (Edythe London, PI). Additional research support was obtained
through NIH Roadmap for Medical Research Grant AR052177 (David Cella, PI), National Cancer Institute (NCI) Grant 4R44CA137841-03 (Patrick
Mair, PI) for item response theory software development for health outcomes and behavioral cancer research, and through Institute of Educational
Sciences Grant 00362556 (Noreen Webb, Program Director). The content is solely the responsibility of the authors and does not necessarily represent
the official views of the NCI or the NIH.

DOI: 10.1037/13619-037
APA Handbook of Research Methods in Psychology: Vol. 1. Foundations, Planning, Measures, and Psychometrics, H. Cooper (Editor-in-Chief)
699
Copyright © 2012 by the American Psychological Association. All rights reserved.
 Reise and Moore
item banks that can be administered efficiently via
computerized adaptive testing (Wainer, 2000).
Nevertheless, this chapter neither focuses on the
virtues of IRT nor compares IRT with traditional
psychometric procedures. Such articles are plentiful
(see, e.g., Embretson, 1996; Reise & Henson, 2003).
Rather, this chapter is divided into two sections. In
the first, we describe commonly applied unidimen-
sional IRT models that are appropriate for dichoto-
mous or polytomous item response data; space
considerations prevent us from extending these
models to the multidimensional case. Our primary
goal in this section is to inform readers of the
Copyright American Psychological Association. Not for further distribution.
most popular IRT models, their origin, and the
interpretation of parameters. In short, this first sec-
tion is oriented toward researchers who are rela-
tively unfamiliar with modern measurement theory
and who desire a basic understanding of IRT model-
ing options.
In the second section, we discuss conceptual and
technical issues that arise in the application of IRT
models to psychological data. Among the topics we
consider are (a) the applicability of IRT models
across various types of constructs and domains,
(b) sample size and model selections issues, and
(c) some lessons learned thus far from the research
literature on application of IRT to noncognitive
measures. This second section is oriented toward
both novice researchers who are considering apply-
ing IRT to their data and to more experienced inves-
tigators who may not have considered some of the
issues raised herein.
Many key topics are not addressed. For example,
technical topics (such as item and person parameter
estimation) and applications of IRT models are not
covered, including using IRT (a) to score individu-
als on a latent variable, (b) to evaluate measures for
differential item functioning, or (c) as a basis for
computerized adaptive testing. Interested readers
are referred to Embretson and Reise (2000); Ham-
bleton, Swaminathan, and Rogers (1991); and de
Ayala (2009) for treatment of these topics. Finally,
given space constraints, we cannot do justice to the
seemingly endless variety of alternative IRT models,
such as nonparametric (Sijtsma & Molenaar, 2002),
unfolding (Roberts, Donoghue, & Laughlin, 2000),
explanatory (De Boeck & Wilson, 2004), hierarchical
700
(Fox & Glas, 2001), or sequential-steps models
(Tutz, 1990).
Unidimensional Dichotomous IRT
Models
IRT modeling begins with a Person (i = 1 . . . I) ×
Item (j = 1 . . . J) matrix of item responses (X).
When items are dichotomously scored, such as cor-
rect versus incorrect or endorsed versus not
endorsed, the item response matrix consists entirely
of zeros and ones. Given this matrix, the chief objec-
tive of IRT modeling is to fit a mathematical func-
tion that characterizes the relation between
individual differences on an assumed latent variable
(labeled θ) and the probability of endorsing an item.
Herein, for dichotomous items, this function will be
termed an item response curve (IRC). The goal of
fitting IRT models is to find a model such that the
estimated IRC best represents or fits the observed
item response data. In this section, we provide
detailed review of the most commonly observed uni-
dimensional IRT models for describing dichotomous
item response data. We proceed slowly in develop-
ing these models for the sake of audiences with little
prior exposure to IRT.
As noted, the basic goal of IRT modeling is to find
a function that relates an individual’s standing on a
latent variable with the probability of endorsing an
item. One such IRC must be found for each scale item.
In considering an appropriate model, note that as indi-
vidual trait levels increase, the probability of item
endorsement should increase monotonically. Stated
differently, groups of individuals who are higher on
the latent variable measured by an item should have
higher item endorsement rates relative to groups of
individuals who are lower on the latent variable.
At first blush, the above observation may suggest
that a straight-line function could be used to
describe the relation between the latent variable and
item endorsement probability. However, a straight-
line function will not suffice because probabilities
are bounded between zero and one, and any line
will eventually predict values above one as the latent
variable increases and predict values below zero
as the latent variable decreases. Alternatively, a
function that increases monotonically and is
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
bounded between zero and one is the cumulative
function of the normal distribution:
∞
1 lative normal distribution to model the proportions
∫
2
P( x = 1 | ) = () = e −[1/ 2]Z dz. (1)
− Z=−( − )/  2 
Equation 1 states that to find the probability of
endorsement, conditional on a latent variable, we
need to integrate (find the area under the curve)
between −Z and infinity. As −Z becomes more nega-
tive, proportions increase; as −Z becomes more posi-
tive, proportions decrease. To clarify, in Equation 1,
θ represents individual differences on a latent vari-
Copyright American Psychological Association. Not for further distribution.
able. (For now, think of θ as a standardized raw
scale score.) The μ and σ parameters in Equation 1
are the population mean and standard deviation of a
normal distribution. The term (θ − μ)/σ thus indi-
cates how many standard deviation units an individ-
ual’s level on the latent variable is relative to the
mean. When different values of μ or σ are plugged
into Equation 1, the resulting cumulative distribu-
tion takes different forms. By decreasing the stan-
dard deviation, we can produce a steeper curve, or
by increasing the standard deviation, we can
decrease the acceleration of the ogive. In turn, the
mean always corresponds to the point on the latent
continuum at which the proportion is .50.
To illustrate, we examined a large set of item
response data drawn from a 24-item measure of ado-
lescent Social Discomfort (Williams, Butcher, Ben-
Porath, & Graham, 1992) administered to a large
sample. Parts of this data set were previously ana-
lyzed in Reise and Waller (2003). For each item, we
began our analysis by simply summing raw item
responses (not including the item under study) and
then standardizing those raw scale scores. We then
plotted item endorsement proportions (i.e., means)
for groups of individuals with similar Z-scores.
These empirical IRCs are shown for items 16, 22,
and 18 in the three panels of Figure 36.1, top, middle,
and bottom, respectively. We then estimated best-
fitting two-parameter cumulative normal ogives
(2PNO), and these estimated IRCs are shown as
solid lines in the figures. Clearly, these lines appear
to describe the data quite nicely. Specifically, the
three curves are cumulative ogives from three different
normal distributions with means and standard devi-
ations of .17, .66 (top panel); .58, 2.17 (middle
panel); and 2.19, 1.78 (bottom panel).
This example illustrates simple use of the cumu-
of individuals endorsing an item as a function of the
latent variable. By using different normal distribu-
tions and their associated cumulative ogives, we can
accommodate items regardless of difficulty or
strength of relationship to the latent variable. Yet,
Equation 1 is not the actual normal-ogive model
referred to in the IRT literature. For that, we need to
rearrange and redefine some terms. We start by rela-
beling the mean (μ) as an item location parameter
(b) and defining a slope parameter (a; sometimes
called a discrimination parameter) as the reciprocal
of the standard deviation: a = 1/σ. Replacing and
rearranging terms leaves us with Equation 2:
∞
1
∫
2
P( x ij = 1 | ) = () = e −[1/ 2]Z dz. (2)
− Z=−[a ( −b )] 2 
Equation 2 makes clear that the probability of
endorsing an item is now a function of an individu-
al’s standing on a latent variable, and two item prop-
erties, location and slope (to be defined more sharply
in the following paragraphs). Individuals with trait
level values above the item location will have high
probabilities of endorsing the item, and conversely,
individuals with trait levels below the item location
will have low probabilities of item endorsement.
Finally, note that Equation 2 is in slope and loca-
tion form, meaning, to find the probability of item
endorsement given θ, we must know this particular
item’s slope (a) and location (b). We can rewrite
Equation 2 to be in slope and intercept form by defin-
ing an intercept d as −ab:
∞
1
∫
2
P( x ij = 1 | ) = () = e −[1/ 2]Z dz. (3)
2
− Z=−( a+d ) 
Equations 2 and 3 express exactly the same thing;
both are two-parameter normal-ogive IRT (2PNO)
models. Equation 2 is the parameterization most fre-
quently reported and discussed, but Equation 3 is crit-
ically important to understand because (a) it is the
model that is most easily transformed in a factor ana-
lytic model (i.e., slopes transformed into factor load-
ings and intercepts transformed into factor thresholds;
701
 Reise and Moore
Copyright American Psychological Association. Not for further distribution.

Figure 36.1.  Empirical (dots) and estimated (solid lines) item response curves for three social discomfort items.

McLeod, Swygert, & Thissen, 2001, p. 199), and thus of the model that is estimated in most popular IRT
(b) it is the model that generalizes most easily to mul- software programs (Baker & Kim, 2004).
tidimensional models (i.e., models with more than Thus far, we have been thinking of θ as a stan-
one latent variable), and (c) it is the parameterization dardized scale score. Of course, in IRT applications,

702
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits

θ is a latent variable with an arbitrary scale, and an
individual’s position on that latent variable needs to
be estimated. In most IRT applications, the scale of
the latent variable is identified by specifying that the
mean in the population is zero and the standard
deviation is one. As a consequence, a person’s
standing on the latent variable is like a z score (if we
assume normality of the latent variable), and impor-
tantly, the item parameters (location and slope) are
estimated in reference to this latent trait scale. If we
identified the latent variable in a different way, say
using a mean of 500 and standard deviation of 100,
the scale for the item parameters would change
Copyright American Psychological Association. Not for further distribution.
software program (Zimowski, Muraki, Mislevy, &
Bock, 2003). The first two columns of Table 36.1
report the item–test correlations and the item means
(proportions endorsed). Clearly, items do not vary
greatly in their response rates—there are no highly
endorsed items—but they do vary in item–test cor-
relation. We note that the raw score distribution was
highly skewed positively; most people score low on
the measure.
The third column of Table 36.1 reports the esti-
mated item intercepts (−ab). Generally speaking,
intercepts tend to range between −3 and 3 with high
values representing easy (high proportion endorsed)

accordingly. items and negative values reflecting difficult items

To illustrate and better define the 2PNO, Table 36.1
shows item parameter estimates for the 24 Adoles-
cent Social Discomfort items noted earlier. Item
parameters were estimated using marginal maxi-
mum likelihood as implemented by the BILOG-MG
(low proportion endorsed). Technically, the inter-
cept corresponds to the z score on a normal distri-
bution that cuts off the predicted proportion of
endorsements for individuals at θ = 0. If the latent
variable is perfectly normally distributed, the

Table 36.1

Descriptive Statistics and Two-Parameter IRT Parameter Estimates of the 24-Item Social Discomfort Scale

Item–test Proportion Normal ogive (IRT) Logistic (IRT)

Item correlation endorsed Int. Location Slope Int. Location Slope
1 .44 .31 −.53 .72 .74 −.90 .72 1.26
2 .61 .35 −.51 .36 1.41 −.87 .36 2.39
3 .36 .51 .08 −.13 .59 .13 −.13 1.00
4 .35 .56 .22 −.40 .54 .37 −.40 .92
5 .56 .41 −.23 .20 1.13 −.39 .20 1.92
6 .45 .41 −.21 .27 .77 −.36 .27 1.32
7 .42 .29 −.62 .88 .70 −1.05 .88 1.19
8 .52 .37 −.38 .38 .98 −.64 .38 1.67
9 .39 .54 .18 −.27 .65 .30 −.27 1.10
10 .53 .26 −.84 .79 1.06 −1.43 .79 1.80
11 .43 .11 −1.67 1.64 1.02 −2.84 1.64 1.74
12 .29 .20 −.90 1.86 .48 −1.53 1.86 .82
13 .60 .29 −.82 .59 1.40 −1.40 .59 2.39
14 .43 .38 −.31 .43 .71 −.52 .43 1.21
15 .57 .33 −.55 .46 1.21 −.94 .46 2.06
16 .62 .41 −.26 .17 1.51 −.44 .17 2.57
17 .35 .50 .04 −.07 .56 .07 −.07 .95
18 .29 .14 −1.22 2.19 .56 −2.08 2.19 .95
19 .54 .33 −.56 .52 1.09 −.96 .52 1.86
20 .50 .39 −.29 .31 .93 −.49 .31 1.58
21 .52 .23 −1.00 .94 1.07 −1.70 .94 1.81
22 .31 .39 −.27 .58 .46 −.45 .58 .79
23 .36 .44 −.12 .22 .56 −.21 .22 .96
24 .45 .40 −.24 .31 .76 −.41 .31 1.30

Note. Int. = intercept; IRT = item response theory.

703
 Reise and Moore
intercept is nothing more than the z-score cutpoint
corresponding to the proportion who endorsed the
item in the calibration sample. Consider, for exam-
ple, that the intercepts in Table 36.1 are very close
to the z-score cutpoints that correspond to the pro-
portions endorsed. For example, for Item 1, the pro-
portion endorsed is 31%. The z score that cuts off
31% from 69% is −0.49, very close to the estimated
intercept of −.53. For Item 4, the endorsement rate
is 56%. The z score that cuts off 56% from 44% is
.16, which is again close to the estimated intercept
of .22.
The fourth column of Table 36.1 shows the esti-
Copyright American Psychological Association. Not for further distribution.
mated item location parameters. These parameters
are expressed on the same scale as the latent variable
and typically range between −3 and 3. In this two-
parameter model, the location parameter reflects the
point on the latent variable continuum at which the
endorsement rate is .50. Finally, the fifth column
shows the estimated slope or discrimination parame-
ters. Slope parameters indicate the steepness of the
IRC in the area around the item location; higher
slopes indicate more discriminating items. More
technically, Equation 4 reveals that the slope in the
normal-ogive metric, which typically ranges
between .5 and 1.2, determines how fast −Z changes
as the latent variable changes. Finally, observe that
the slope is nearly perfectly correlated with the item-
test correlation.
Before moving on, a few comments are in order
regarding the intercept and location parameters.
First, as noted, the intercept is essentially a simple
transformation of the observed proportion endorsed
into a z-score metric. In turn, because b = −(d/a),
items can have equal intercepts but different loca-
tions if the items have different slopes. This observa-
tion calls into question the use of the common label
difficulty for the location parameter, b; it appears
more reasonable to consider the intercepts as the dif-
ficulty of an item. Moreover, even if item parameters
from multiple groups are linked on a common met-
ric, a researcher cannot simply compare estimated
location parameters across two or more groups with-
out first establishing that the slopes are invariant.
This is much like multiple-group confirmatory fac-
tor analysis, which requires first establishing loading
invariance before investigating threshold invariance.
704
The normal-ogive model has the advantages of
being familiar and easily related to the factor analy-
sis of ordinal items, and it can be extended to multi-
ple dimensions. However, there are some
well-known mathematical difficulties in estimating
model parameters. For this reason, most IRT appli-
cations estimate the IRC using a logistic-ogive
model in place of the normal ogive. In fact, some
statistical estimation software (MULTILOG; This-
sen, 2003) only includes logistic models. Equations
4 and 5 show the two-parameter logistic model
(2PLM) in both slope-and-location and slope-and-
intercept forms, respectively.
exp[1.7a( − b )]
P( x = 1 | ) = () = . (4)
1 + exp[1.7a( − b )]
exp(1.7a + d )
P( x = 1 | ) = () = . (5)
1 + exp(1.7a + d )
As written in Equations 4 and 5, because of the
inclusion of the constant 1.7 in the model, these
are actually two-parameter logistic approxima-
tions of the normal-ogive model. The purpose of
including this scaling factor is so that the slope
parameter estimated in the logistic model is the
same as its value estimated in the normal-ogive
model. If the 1.7 scaling factor were not included
in the model, the slope parameter in the pure
logistic model would be 1.7 times higher than its
value in the normal ogive. To demonstrate the
equivalence of the logistic and normal-ogive mod-
els, in Table 36.1, we also display the estimated
parameters from the logistic model using
BILOG-MG (Zimowski et al., 2003).
Interpretation of the item parameters under the
2PLM remains essentially the same as under the
2PNO model. As usual, the scale for the latent vari-
able is arbitrary and must be identified by setting the
mean and standard deviation. Values of zero and
one are typically chosen. The location parameter
remains the point on the latent continuum at which
the proportion endorsed is .50; these values are the
same in the 2PNO and 2PLM. The slope indicates
how rapidly response rates change as a function of
the latent variable in the area of the item location.
The slopes in a pure logistic model (no D = 1.7) are
1.7 times higher than in the normal ogive.
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits

Reduced and Expanded Dichotomous
Models
The 2PLM and 2PNO allow IRCs to vary in two
ways: location and slope. Other IRT models for
dichotomous responses can be viewed as either
expansions of these models or as nested models
derived by placing restrictions on the item slopes.
For example, if we retain the identification con-
straint that the mean of the latent variable is zero
with standard deviation of one in the population, we
can impose the constraint that all the items have the
same slope parameter. In Table 36.2, we show the
Social Discomfort items under a logistic model with
Copyright American Psychological Association. Not for further distribution.
the difference between an individual’s trait standing
and the item’s location, weighted by a constant (the
item slope).
The 1PLM, although similar, is not to be con-
fused with the Rasch model (Bond & Fox, 2007). In
a Rasch model, it is customary to impose the con-
straint that all the slopes are equal to one, and thus
the slope parameter disappears from the model. In
turn, to accommodate this constraint, one needs to
free up the variance of the latent variable by estimat-
ing it rather than fixing its value (a variety of other
identification constraints are possible). Over the
past 30 years, many researchers have been champi-

the equal slope constraint. This is called the one-pa-
rameter logistic model (1PLM). In this model, the
IRCs will not intersect because they all constrained
to have the same slope. Moreover, in the 1PLM, the
probability of endorsement is solely a function of
oning the potential virtues of Rasch models (e.g.,
sufficient statistics for estimating item parameters,
nonintersecting IRCs, specific objectivity) and an
equal number have been questioning their utility in
real-world psychological data. Such debates are

Table 36.2

1PLM and 3PLM Parameter Estimates for the Social Discomfort Scale

1PLM 3PLM
Item Intercept Slope Location Intercept Slope Location c-P
1 −0.92 1.34 0.69 −0.63 0.85 0.74 0.00
2 −0.69 1.34 0.51 −0.99 1.96 0.51 0.03
3 0.18 1.34 −0.14 −1.63 2.47 0.66 0.34
4 0.46 1.34 −0.34 −0.34 0.90 0.38 0.27
5 −0.35 1.34 0.26 −0.40 1.24 0.32 0.00
6 −0.35 1.34 0.26 −0.30 0.86 0.35 0.00
7 −1.09 1.34 0.82 −0.71 0.80 0.88 0.00
8 −0.59 1.34 0.44 −0.52 1.10 0.47 0.00
9 0.35 1.34 −0.26 0.10 0.66 −0.16 0.01
10 −1.28 1.34 0.96 −1.04 1.28 0.81 0.00
11 −2.58 1.34 1.93 −1.98 1.44 1.37 0.00
12 −1.75 1.34 1.31 −0.96 0.58 1.66 0.00
13 −1.09 1.34 0.82 −1.95 2.98 0.65 0.04
14 −0.53 1.34 0.40 −0.48 0.86 0.55 0.03
15 −0.80 1.34 0.60 −1.22 2.05 0.59 0.05
16 −0.35 1.34 0.27 −0.77 2.08 0.37 0.04
17 0.11 1.34 −0.08 −1.07 1.60 0.67 0.30
18 −2.31 1.34 1.73 −1.31 0.70 1.87 0.00
19 −0.85 1.34 0.64 −1.82 2.74 0.67 0.09
20 −0.47 1.34 0.35 −1.39 2.29 0.61 0.15
21 −1.52 1.34 1.14 −1.28 1.40 0.92 0.01
22 −0.48 1.34 0.36 −1.58 1.61 0.98 0.25
23 −0.20 1.34 0.15 −0.36 0.74 0.48 0.09
24 −0.40 1.34 0.30 −0.53 1.04 0.51 0.07

Note. 1PLM = one-parameter logistic model; 3PLM = three-parameter logistic model; c-P = c parameter.

705
 Reise and Moore
beyond the present treatment (see Borsboom, 2005,
for extended discussion), but two features of Rasch
modeling are worth noting.
First, Rasch modeling, by its nature, emphasizes
the meaningfulness and interpretability of the latent
variable metric and items arrayed along that dimen-
sion. This is a laudable goal and contrasts sharply
with most psychometric work in its consideration of
the unit of measurement. Second, as a general phi-
losophy, following the factor analytic tradition, IRT
modeling typically focuses on finding a model that
best fits the data. On the other hand, a Rasch model
is usually considered the only real measurement
Copyright American Psychological Association. Not for further distribution.
model, and the goal of research is to find a set of
items that provide responses fitting that model. In
short, the philosophies underlying application of
Rasch and non-Rasch models can be very different
(Wilson, 2005).
Moving beyond restricted models, we now con-
sider models that add parameters to the 2PLM (or
2PNO). Consider that on multiple-choice tests, it is
arguable that the 2PLM is inadequate to describe the
item response process because individuals may pro-
duce a correct response by chance, regardless of
their levels on the latent variable. Notice that in a
two-parameter model, the IRC has a lower asymp-
tote of zero (very low-scoring individuals have zero
chance of answering the item correctly) and an
upper asymptote of one (very high scoring individu-
als always get the item right). Thus, the model may
be unrealistic in describing certain types of multiple-
choice tests. Moreover, some personality researchers
have argued that a two-parameter model is inade-
quate because individuals at low levels of the latent
variable may have a nonzero probability of endors-
ing the item (Rouse, Finger, & Butcher, 1999). To
accommodate this fact in either cognitive or non-
cognitive measurement, the canonical 2PLM can be
expanded to include a lower-asymptote or pseudo-
guessing parameter, as in Equation 6.
exp[a( − b )]
P( x ij = 1 | ) = () = c + (1 − c)
1 + exp[a( − b )] 
In this three-parameter logistic model (3PLM),
the lower asymptote parameter (c) places a lower
boundary on the IRF. For example, if c is estimated
to be .20, then regardless of how low an individual’s
706
standing is on the latent variable, the IRC will go no
lower than .20. The location (b) in Equation 6 is no
longer the point on the latent trait continuum at
which the probability of endorsing is .50. Rather,
the probability of endorsing at θ = b is (100 + c)/2
(Hambleton & Swaminathan, 1985).
To illustrate, the last set of columns in Table 36.2
shows the 3PLM parameters estimated for the Adoles-
cent Social Discomfort data (logistic model). For
most items, there is no c parameter, indicating that as
levels on the latent variable decrease, response proba-
bilities go to zero. However, there are several items
for which there is evidence for a c parameter (i.e.,
Items 3, 4, 17—three items with relatively high pro-
portion endorsed—and 22). For example, Item 22 is
interesting in that it has somewhat extreme content,
abbreviated as “spend most of spare time alone” (Ben-
Porath, Tellegen, & Kaemmer, 2005), which is keyed
positively for Social Discomfort. Thus, the 3PLM tells
us that even individuals low on Social Discomfort will
say “yes” to this item approximately 25% of the time.
Moving beyond the 3PLM, Reise and Waller
(2003) and Waller and Reise (2010) considered a
4PLM model that includes an upper asymptote
parameter (γ) as well as a lower asymptote for
psychopathology items.
exp[a( − b )]
P( x = 1 | ) = () = c + ( − c) . (7)
1 + exp[a( − b )] 
This model was motivated by inspection of
empirical IRCs and by theoretical considerations.
For example, even in clinical populations (such as
depressed populations), it would be unrealistic to
expect that 100% of patients display any one symp-
tom (e.g., suicide ideation, hopelessness). Hence,
the upper asymptote for an item may not be 100%,
as in the 2PLM or 3PLM, but rather some smaller
value. Relative to the 2PLM, the interpretation of the
item parameters in this 4PLM model changes
slightly. Specifically, the location (b) is now the
point on the latent scale at which the response pro-
. (6) portion is (100 + c – γ)/2. We will not attempt to
estimate and display parameters for the 4PLM
herein. Instead, we refer readers to the original arti-
cles for discussion of the possible causes and conse-
quences of upper and lower asymptote parameters
in personality and psychopathology data.
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
Unidimensional Polytomous Item
Response Models
Although dichotomously scored multiple-choice
tests continue to dominate cognitive assessment, the
fields of personality, psychopathology, and patient-
reported outcomes assessment rely heavily on mea-
sures with ordered multicategory response options.
Over the past 10 years, the application of polyto-
mous IRT models has increased, especially in the
field of patient-reported outcomes assessment (see
Cella et al., 2007, 2010; Reise & Waller, 2009). The
goals of this section are to introduce the logic of
polytomous IRT models and to highlight important
Copyright American Psychological Association. Not for further distribution.
differences. There are many important polytomous
models, but here, we focus discussion on only a
small subset. In addition, we limit the description to
only the logistic versions of each model. Readers
interested in fuller treatment of the diverse family of
polytomous IRT models should consult Nering and
Ostini (2010) and Ostini and Nering (2006).
Just as the basic goal of IRT modeling for dichot-
omous items is to estimate an IRC that best repre-
sents the data, the chief objective of polytomous IRT
models is to estimate a set of best-fitting category
response curve (CRC). These CRCs model the rela-
tion between level on a latent variable and the prob-
ability of responding in a particular response
category for an item. A central distinction between
polytomous IRT models is the distinction between
difference models and divide-by-total models
(Thissen & Steinberg, 1986). As shown in the next
section, difference models require a two-step com-
putation procedure to derive CRCs, whereas in
divide-by-total models, only a single equation is
needed to derive the CRCs.
Graded-Response Model
We begin by describing the iconic graded-response
model (GRM; Samejima, 1969), which appears to be
the most frequently applied polytomous IRT model
in the noncognitive assessment domain (Reise &
Waller, 2009). One way to view the GRM is to think
of it as an extension of the 2PLM to the polytomous
response case. Consider that, with a dichotomous
item, there are only two response options, and thus
there is only a single boundary or threshold between
a response of zero and one. As a consequence, only a
single IRC is needed to describe how increases on
the latent variable increase the chances of an indi-
vidual endorsing the item (i.e., responding one
instead of zero).
Now consider an item with four ordered response
options (0, 1, 2, 3). This item can be thought of as
containing three (number of categories minus one)
dichotomies: 0 versus 1, 2, 3; 0, 1 versus 2, 3; and 0,
1, 2 versus 3. When represented in this way, it is easy
to understand that the first step in estimating a GRM
is to estimate the number of categories minus one
threshold response curve (TRC), one for each of the
possible dichotomizations. These TRCs, shown in
Equation 8, are simply 2PLM IRCs with equal slopes
within an item (but not necessarily between items):
Px* () =
( )
exp   − b j 
, (8)
(
1 + exp   − b j  )
where, j = 1 . . . number of response categories
minus one.
Given that they are 2PLM functions, the TRCs
indicate how the probability of responding in or
above a given category changes as a function of the
latent variable. In other words, for a four-category
item, in the GRM, each item is described by one
item slope parameter (a) and the number of catego-
ries minus one location parameter (bj)—one for
each threshold between the response categories. The
TRCs are important, but they do not directly yield
the desired CRCs. Rather, once the parameters of
the TRCs are estimated, computing the conditional
category response probabilities for x = 0 . . . 3
requires a second step that is done by subtraction:
Px () = P(*x ) () − P(*x+1) () . (9)
By definition, the probability of responding in or
above the lowest response category is P(*x =0 ) () = 1.0,
and the probability of responding above the highest
response category is P(*x =4 ) () = 0.0. The CRCs
derived from Equation 9 represent the probability of
an individual responding in a particular category
conditional on the latent variable.
The item parameters in the GRM dictate the
shape and location of the TRCs (and thus the
CRCs). The higher the slope parameters (α), the
steeper the TRCs, and the more narrow and peaked
707
 Reise and Moore
the CRCs, indicating that the response categories
differentiate among individuals at different levels of
the latent variable well. The location parameters (bj)
determine the location of the TRCs along the latent
variable continuum and the point at which each of
the CRCs for the middle response options peak. Spe-
cifically, the CRCs peak in the middle of two adja-
cent location parameters. To illustrate the model,
Table 36.3 displays the estimated item parameters
for a set of Depression items scored with four
response options. These items were drawn from the
National Institutes of Health (NIH) PROMIS project
(Cella et al., 2007) and item parameters are based on
Copyright American Psychological Association. Not for further distribution.
a small subsample drawn from a much larger project
(Pilkonis et al., 2011). The darker lines in Figure 36.2
show the TRCs and the CRCs for two items, namely,
Figure 36.2.  Threshold response curves (top) for the most and least discriminating depression items under the
GRM, and CRCs (bottom) for these same items under the GRM (darker lines) and GPCM (lighter lines).
708
the two items with the highest estimated (Item 21,
“I felt hopeless”) and lowest (Item 27, “I felt guilty”)
slopes.
Nominal Response Model
We now turn our attention to a set of nested models
that belong to a distinct class of divide-by-total poly-
tomous IRT models. We begin by introducing the
most general direct model, namely, Bock’s (1972)
nominal response model (NRM). It is called a “nom-
inal” response because the model does not assume
that category responses are ordered within an item.
Rather, the model treats category ordering as a prop-
erty to be discovered, as we will see. In contrast, the
GRM assumed that response categories are strictly
ordered.
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits

Table 36.3

Graded-Response Model and Generalized Partial Credit Model Parameter Estimates for an Example
Depression Scale

Graded-response model Generalized partial credit model

Item α b1 b2 b3 α δ1 δ2 δ3
1 3.13 0.81 1.55 2.35 2.99 0.97 1.45 2.12
2 2.64 0.65 1.35 2.28 2.49 0.89 1.19 2.11
3 2.80 0.70 1.47 2.45 3.08 0.90 1.29 2.28
4 1.98 0.29 1.19 2.24 1.58 0.67 1.02 1.93
5 2.73 0.67 1.48 2.58 2.42 0.84 1.31 2.42
6 2.11 0.43 1.30 2.40 1.93 0.77 1.14 2.13
7 2.50 −0.37 0.72 1.94 2.11 −0.27 0.73 1.79
Copyright American Psychological Association. Not for further distribution.

8 2.67 0.97 1.67 2.60 2.46 1.22 1.43 2.39

9 2.16 0.21 1.20 2.42 2.05 0.43 1.09 2.24
10 2.91 0.49 1.17 2.35 2.74 0.70 1.01 2.23
11 2.07 0.08 0.95 2.07 1.96 0.38 0.79 1.86
12 2.40 −0.21 0.74 1.94 2.20 −0.03 0.70 1.80
13 2.34 0.52 1.35 2.32 1.93 0.78 1.21 2.09
14 2.03 0.12 1.00 2.11 1.43 0.45 0.86 1.87
15 2.89 0.09 1.02 1.95 2.80 0.21 1.00 1.76
16 1.88 0.03 1.24 2.49 1.89 0.24 1.16 2.25
17 2.27 −0.10 0.72 1.99 1.98 0.16 0.57 1.88
18 2.30 0.13 1.01 2.20 2.02 0.38 0.87 2.07
19 3.08 −0.27 0.77 1.88 3.23 −0.20 0.76 1.75
20 2.37 1.46 2.20 2.89 2.13 1.77 1.99 2.49
21 3.43 0.73 1.52 2.31 3.16 0.83 1.42 2.12
22 1.86 0.15 1.20 2.44 1.69 0.46 1.08 2.19
23 2.02 0.56 1.48 2.75 2.12 0.88 1.26 2.57
24 2.08 0.32 1.41 2.81 2.18 0.51 1.26 2.65
25 2.04 −0.14 0.86 2.06 1.61 0.10 0.77 1.87
26 2.47 0.60 1.20 2.33 3.11 0.98 0.88 2.20
27 1.79 0.33 1.38 2.58 1.52 0.70 1.19 2.31
28 2.33 −0.01 0.90 1.94 2.03 0.21 0.83 1.74

Note. α denotes an item discrimination parameter, b denotes a threshold, and δ denotes an intersection.

In the NRM, the probability of an individual
responding in category x (x = 0 . . . number of cate-
gories minus one) on an item, conditional on the
latent variable (θ) is
exp (a  + c x )
Px () = NCAT −1 x . (10)
∑ ( x x)
exp a  + c
x =0 
To identify the model, some constraint must be
set: Σax = Σcx = 0. This constraint forces one
response option (the one with the highest a) to have
a monotonically increasing CRC and one response
option (the one with the lowest a) to have a mono-
tonically decreasing CRC. In Equation 10, cx is an
intercept parameter for category x, and the αx is the
slope of the linear regression of the latent variable
on the log-odds of the probability of responding in a
particular category. The cx parameters reflect the
popularity of a particular response category, where
larger values reflect more popular options (Thissen,
Steinberg, & Fitzpatrick, 1989).
Simply stated, Equation 10 is called a divide-by-
total model because the numerator contains a single
conditional probability and the denominator con-
tains a scaling factor, which is the sum of condi-
tional probabilities for each response distinction. In
short, the denominator ensures that the sum of the
probabilities of each category response conditional

709
 Reise and Moore
on the latent trait equals one. In other divide-by-to-
tal models to follow, the denominator will become
more complicated but will always serve the exact
same purpose.
This description of the NRM does not readily
reveal some of its most intriguing properties. How-
ever, Thissen et al. (1989) and Thissen, Cai, and
Bock (2010) have shown how the NRM can be
thought of in terms of the choice between two adja-
cent response categories. Specifically, if one thinks
of an item response as a choice between two options
x and x − 1 (3 vs. 2), then the NRM for this choice
can be written in the form of the 2PLM:
Copyright American Psychological Association. Not for further distribution.
1 cepts in the NRM can be transformed into useful
Px | x = x or x − 1 = , (11)
1 + exp[−a* ( − )] 
where a* = ax − ax–1 and δ = cx–1 – cx/(ax − ax–1).
That is, the difference between the category slope
parameters for adjacent categories functions is like a
slope parameter in a two-parameter model (Equa-
tion 8), indicating how discriminating the choice
between two adjacent categories is. Clearly, if ax is
larger than ax–1, then a* is positive, and the response
categories are ordered along the latent variable;
higher levels of the latent variable mean one is more
likely to select x than x − 1.
Thus, inspection of the a* parameters in the
NRM allows an empirical test of the ordering of
response categories. Preston, Reise, Cai, and Hays
(2011) called the a* parameters category boundary
discriminations (CBDs). They argued that inspection
of these values is important not only to test for the
ordering of categories but also to determine whether
the item contains too many response categories (e.g.,
when an a* is near zero). It can also be used to eval-
uate whether the response categories are equally dif-
ferentiating. For example, they cited examples of
multipoint items for which the CBD between the
first and second category is very high but the
remaining CBDs are very low, suggesting that only
the first response distinction is meaningful.
Finally, they argued that a lack of consistency
between the CBDs within an item may call into
question the application of models that specify only
a single item slope parameter (e.g., the generalized
partial credit model to be described shortly, or
the GRM). For example, Table 36.4 displays the
710
category slopes (a) and the CBDs (a*) for the exam-
ple Depression items estimated using MULTILOG
(Thissen, 2003). In the present data, it appears that
all categories (1 = never, 2 = rarely, 3 = sometimes,
4 = often) provide some useful discrimination. How-
ever, for some items, such as 3, 11, and 16, there
appears to be a trend for the first CBD to be signifi-
cantly higher than the second or third. If statistical
tests (e.g., a Wald test) indicated that the a* within
an item varied significantly, one would have to ques-
tion whether any model that proposed a single item
slope is appropriate.
Finally, it can be shown that the category inter-
and interpretable information by taking c* = (cx−1 −
cx)/aj* to obtain category intersection parameters.
These intersection parameters, of which there are
the number of categories minus one, indicate the
point on the latent variable scale at which two CRCs
intersect. These parameters contrast sharply with
the location parameters in the GRM. Specifically,
locations in the GRM indicate where the probability
of responding in and above a category is .50,
whereas intersections indicate where the selection of
one response category becomes more likely than the
previous category. Moreover, category intersections
are not necessarily ordered, but locations in the
GRM must be. Unordered category intersections do
not indicate that the response options are unordered
but rather merely that one response category is
never the most likely response for any value of the
latent trait.
Generalized Partial Credit Model
It may seem odd to describe Muraki’s (1992) gener-
alization of the partial credit model (GPCM) before
describing the partial credit model (PCM; Masters,
1982).Our decision to include this model here is
based on the fact that the NRM highlighted in the
previous section is the most general direct polyto-
mous IRT model. In turn, our goal is to emphasize
that Muraki’s model is simply a restricted version of
the NRM, where item responses are assumed
ordered, and CBD parameters (a*) are set equal
within an item. Thus, although the NRM allows
within-item response category distinctions to vary
in slope, the GPCM assumes that CBDs are equally
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits

Table 36.4

Nominal Response Model (NRM) and Generalized Partial Credit Model (GPCM) Slope/Discrimination
Parameter Estimates for Depression Items

NRM slope (a) and CBD (a*) estimates GPCM

Item a1 a2 a3 a4 a*1 a*2 a*3 slopes
1 −4.54 −1.62 1.53 4.64 2.92 3.15 3.11 2.99
2 −3.54 −0.77 0.69 3.62 2.77 1.46 2.93 2.49
3 −3.82 −0.63 1.26 3.19 3.19 1.89 1.93 3.08
4 −2.48 −0.58 0.93 2.13 1.90 1.51 1.20 1.58
5 −3.86 −0.86 1.24 3.49 3.00 2.10 2.25 2.42
6 −2.73 −0.91 0.92 2.72 1.82 1.83 1.80 1.93
7 −3.65 −1.66 1.14 4.16 1.99 2.80 3.02 2.11
Copyright American Psychological Association. Not for further distribution.

8 −3.43 −0.74 0.95 3.21 2.69 1.69 2.26 2.46

9 −2.94 −0.70 0.76 2.88 2.24 1.46 2.12 2.05
10 −4.15 −1.24 1.15 4.24 2.91 2.39 3.09 2.74
11 −2.75 −0.50 0.83 2.42 2.25 1.33 1.59 1.96
12 −3.17 −1.22 0.94 3.45 1.95 2.16 2.51 2.20
13 −3.18 −0.76 0.93 3.00 2.42 1.69 2.07 1.93
14 −2.55 −0.68 0.85 2.37 1.87 1.53 1.52 1.43
15 −4.24 −1.33 1.65 3.92 2.91 2.98 2.27 2.80
16 −2.66 −0.47 0.82 2.31 2.19 1.29 1.49 1.89
17 −2.97 −0.80 0.74 3.03 2.17 1.54 2.29 1.98
18 −3.13 −0.65 0.94 2.85 2.48 1.59 1.91 2.02
19 −4.89 −1.79 1.69 4.99 3.10 3.48 3.30 3.23
20 −2.83 −0.28 0.41 2.70 2.55 0.69 2.29 2.13
21 −5.19 −1.09 1.68 4.59 4.10 2.77 2.91 3.16
22 −2.37 −0.76 0.79 2.34 1.61 1.55 1.55 1.69
23 −2.60 −0.61 0.93 2.28 1.99 1.54 1.35 2.12
24 −2.92 −0.62 0.72 2.82 2.30 1.34 2.10 2.18
25 −2.74 −0.74 0.80 2.67 2.00 1.54 1.87 1.61
26 −3.32 −0.75 1.05 3.02 2.57 1.80 1.97 3.11
27 −2.21 −0.41 0.78 1.84 1.80 1.19 1.06 1.52
28 −3.19 −0.79 1.14 2.84 2.40 1.93 1.70 2.03

Note. a denotes category slopes and a* denotes category boundary discriminations. CBD = category boundary
discrimination.

differentiating within an item but that items can for each item. The category intersection parameters
vary in their overall discrimination. The GPCM can (δ) in this model are interpreted in the same way as
be written as x the c* parameters in the NRM—as the intersection
EXP∑   −  j ( ) point of two adjacent CRCs. They are the points on
Px () = j=0
, (12)
the latent variable scale at which one category
NCAT−1  x  response becomes relatively more likely than the
∑ 
(
EXP∑   −  j )

  preceding response. The slope (αi) parameters have
x =0 j=0
0 the usual interpretation as a kind of discrimination
( )
where ∑   −  j ≡ 0. parameter, that is, they “indicate the degree to
j=0 which categorical responses vary among items as θ
For the GPCM (see Table 36.3), one unique level changes” (Muraki, 1992, p. 162).
slope parameter (α) and one minus the number of The GPCM and the GRM are very similar models
categories intersection parameters (δ) are estimated but are derived from two different families (and thus

711
 Reise and Moore
are not nested models or comparable using a likeli-
hood ratio test). Models like the GPCM are built by
considering the choice between two adjacent
response categories x and x − 1—ignoring all other
categories—and then deriving how the latent vari-
able interacts with item slope and intersection
parameters to determine the response choice. This is
certainly different than the GRM for which all
response options are considered in deriving each
within-item TRC. For comparison sake, the esti-
mated parameters from the GPCM are shown in
Table 36.3 alongside the GRM. Although the values
look different, the two bottom graphs in Figure 36.2
Copyright American Psychological Association. Not for further distribution.
show examples in which the CRCs estimated under
the GRM (dark lines) and GPCM (lighter lines) are
almost visually indistinguishable.
A Polytomous Rasch Model
The PCM (Masters, 1982) is the prototypical Rasch
model suitable for analyzing ordered polytomous
items. Recall that the GRM is built around dichoto-
mizing items at the boundaries between the
response categories (e.g., 1, 2 vs. 3, 4) and then fit-
ting 2PLMs to each of the number of categories
minus one of the boundaries between categories. The
PCM is also built around dichotomizing items at the
boundaries between categories, but this process dif-
fers from the GRM in important ways. Specifically,
the PCM is based on fitting Rasch’s 1PLM to the
number of categories minus one ordered dichoto-
mies but considering only two response categories at
a time. Thus, with a four-category item, the PCM fits
a Rasch model considering the dichotomy 0 versus 1;
1 versus 2; and 2 versus 3, as opposed to a GRM
where the dichotomies are 0 versus 1, 2, 3; 0, 1
versus 2, 3; and 0, 1, 2, versus 3.
Interestingly, it is easy to show that the PCM is
simply Equation 12 without the slope parameter.
Thus, in the PCM, the probability of response is
determined solely by the difference between an indi-
vidual’s level on the latent variable and the category
intersections. An interesting feature of the PCM is
that the category intersections are not necessarily
ordered. Recall that in the GRM, locations (b) must
be ordered. When the intersection parameters are
not ordered, this phenomena is known as a “reversal”
(Dodd & Koch, 1987). As a rule, if the intersection
712
parameters are ordered within an item, then there is
at least one location on the latent variable where
every response option is most likely (Andrich, 1988).
Thus, the divide-by-total or direct polytomous
IRT models are a nested family going from least
restricted NRM, to allowing items to vary in slope
(GPCM), to allowing items to vary only in category
intersection parameters (PCM). Of course, only the
PCM has features associated with Rasch models
(specific objectivity). We note that the model hierar-
chy considered here did not include the well-known
rating scale version of the PCM (Andrich, 1978).
We believe this model to be more of theoretical
interest and thus was not detailed here.
Practical Applications and
Considerations
The first section was dedicated to introducing key
IRT models, their properties, and their interpreta-
tion. Understanding the mechanics of these models
is crucial because all applications of IRT (e.g., com-
puterized adaptive testing, analysis of item and scale
information functions, linking distinct measures
onto a common scale, analysis of differential item
functioning) rest squarely on the validity of these
representations. By validity of the model, we mean
that term in the broadest sense to include the viabil-
ity of IRT model assumptions, the accuracy of item
parameter estimates, and the proper representation
of the construct (i.e., is there really a latent variable
that accounts for the relations among the items?).
With this in mind, the following section is geared
toward applied research. Specifically, we consider
(a) the applicability of IRT models across various
types of constructs and domains, (b) sample size
and model selection issues, and (c) some lessons
learned thus far from the research literature on the
application of IRT to noncognitive measures.
The Applicability of IRT Models
As noted by many authors, IRT models developed in
the context of large-scale cognitive testing; applica-
tions outside this domain emerged slowly and are
only now starting to fully blossom. In considering
the application of IRT models across a broad range
of constructs, we consider the following three
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
issues: (a) latent versus emergent variables, (b) cog-
nitive versus noncognitive applications, and (c) the
measurement of narrow versus broadband con-
structs. First, we point out that IRT models can be
viewed as latent factor models (not components)
that are applied to item-level data. As such, IRT
models are clearly latent variable models for which
the items are effect indicators in the Bollen and
Lennox (1991) sense. In short, it makes little sense
to apply IRT models unless one has reason to believe
that a single common latent variable is causing vari-
ation in item responses and thus causing items to be
intercorrelated.
Copyright American Psychological Association. Not for further distribution.
Second, although it is challenging to cite any spe-
cific author, we have observed that some researchers
are hesitant to embrace the application of IRT mod-
els to measures of noncognitive constructs (i.e.,
achievement, ability, competence). In this regard,
we agree with the following: (a) noncognitive con-
structs, such as personality traits, psychopathology,
attitudes, values, health outcomes, and so on, are
conceptually different than such constructs as nurs-
ing competency, eighth-grade spelling knowledge,
or verbal ability; and in turn (b) the psychometric
properties of noncognitive measures are often very
different than what is typically seen in the measure-
ment of cognitive constructs (see concluding section
for examples); and, finally, (c) the measurement
context (i.e., the goals of measurement) between
noncognitive and cognitive assessment is often dra-
matically different (see Reise, 2009, for further
commentary).
Yet, despite these differences, we see little if any
reason to question the application of IRT models to
measures of well-examined personality, psychopa-
thology, or health outcomes constructs. Latent vari-
ables models are justified when one can propose
that the empirical relations among a set of content-
diverse items is explained by individual differences
on a common psychological process (a latent trait).
In this regard, we see no empirical evidence that
data derived from noncognitive measures are more
multidimensional than cognitive measures or that a
dominance response process (higher trait levels lead
to higher item scores) does not apply. In fact, the
history of factor analysis, both exploratory and con-
firmatory, leads us to believe that many noncognitive
measures should be entirely consistent with the
assumptions of IRT (i.e., unidimensionality, mono-
tonicity of response).
Finally, we note that researchers in both cogni-
tive and noncognitive domains have long recognized
the hierarchical nature of psychological constructs.
Accordingly, it has long been recognized that con-
structs and their associated measures vary in con-
ceptual bandwidth, ranging from narrow (physical
attractiveness self-esteem, tooth brushing self-
efficacy, test anxiety) to broad (general self-esteem,
neuroticism). Commonly applied IRT models are
unidimensional—they assume the existence of a sin-
gle common variable affecting item responses. As
pointed out by multiple authors over the years (see
Humphreys, 1970; Gustafsson & Aberg-Bengtsson,
2010), attempting to create pure “unidimensional”
scales can create quite a quandary and actually lead
to poor measurement.
The way to achieve unidimensionality, and thus
be consistent with IRT modeling, is to focus on a
narrow construct and write items that are essentially
replicates (this can increase coefficient alpha as
well). Yet, it is questionable whether we really need
the power of IRT to measure such conceptually nar-
row constructs in cases in which the correlation
between the items can be explained merely by the
semantic similarity of the items, rather than by the
need to postulate an underlying psychological trait.
On the other hand, the measurement of broadband
constructs demands the inclusion of content diverse
indicators. In turn, this almost guarantees violations
of the IRT unidimensionality assumption. Yet, it
appears to us that for these types of substantively
complex constructs, this is exactly where the power
of IRT and other latent variable methods are most
needed and interestingly applied.
We raise this issue because it is important for
future directions of IRT modeling. Specifically, we
see the future as rejecting the unidimensional model
as an appropriate foundation for IRT modeling and
for defining good measurement in general. Never-
theless, we do not necessarily see the future as mov-
ing toward multidimensional correlated traits IRT
models either (see Reckase, 2010) but rather as rec-
ognizing a broader definition of what the common
latent variable is in IRT modeling.
713
 Reise and Moore
For example, the bifactor model described by
Gibbons and Hedeker (1992) has seen increased use
as a tool for exploring the applicability of IRT mod-
els (e.g., Reise, Moore, & Haviland, 2010; Reise,
Morizot, & Hays, 2007). This model allows each
item to reflect a single common latent variable and
one additional latent variable caused by clusters of
items that share item content (i.e., sometimes
referred to as content parcels). Thus, when a mea-
sure consists of multiple content parcels necessary
to validly measure complex constructs, the bifactor
model can easily accommodate such a structure. In
other words, a bifactor framework affords the ability
Copyright American Psychological Association. Not for further distribution.
to use multidimensional item response data to
achieve unidimensional measurement. Thus, we
view the bifactor model not only as a tool for evalu-
ating the distorting effects of forcing multidimen-
sional data into a unidimensional model (Reise
et al., 2007) but also for expanding the range of
constructs that can be accommodated by IRT mod-
els (see also Chapter 35 of this volume).
Sample Size and Model Choice
A common question in seminars and workshops is
“how many subjects do I need to estimate IRT
model parameters?” This is a fundamental question,
but unfortunately there is no simple rule-of-thumb
answer. The needed sample size for reasonably
accurate item parameter estimates depends on the
interplay among a variety of intertwined factors,
including (a) the model selected (constrained mod-
els are easier to estimate); (b) how large the item
slope parameters are expected to be (higher is bet-
ter); (c) how well the data fit the model (if the data
do not fit the model, then parameter accuracy is
meaningless); (d) the underlying latent trait distri-
bution (normal vs. skewed); (e) sample heterogene-
ity (more is better); and, importantly, (f) what the
ultimate research goal is (high-stakes testing vs.
research-only scale).
There may be applications where accuracy of
item parameter estimation is not critically important
(e.g., in exploring basic item functioning). More-
over, given reasonable scale length (e.g., 20 items),
minor biases in item parameter estimates or even
estimating the wrong model may not greatly affect
model outcomes, such as the scale response curve or
714
the relative order of trait level estimates (e.g.,
Wainer & Thissen, 1987). On the other hand, in
some contexts, such as the evaluation of different
CRCs or IRCs across different groups of individuals
(e.g., men versus woman), high-stakes testing in
health care settings, or large-scale computerized adap-
tive testing, item parameter accuracy is paramount.
To clarify the number of subjects issue, some
researchers have conducted Monte Carlo simulation
studies. For example, Reise and Yu (1990) showed
that the GRM can be estimated with MULTILOG
(Thissen, 2003) with as few as 250 examinees, but
they recommend around 500 (the number of items
is not critical with marginal maximum likelihood
estimation). Yet simulation studies are useful only if
the real data match the simulated data, and they
should not be relied on to judge the viability of a
particular application. Rather than suggesting some
citable magic N or subjects-to-parameters ratio, we
recommend the following.
First, thoroughly explore the data using nonpara-
metric methods (Sijtsma & Molenaar, 2002). If the
data considered at the nonparametric level do not
reasonably conform to a parametric model, a large
sample size is pointless—there is no sense precisely
estimating parameters that do not exist. Second,
evaluate model assumptions (dimensionality, mono-
tonicity) carefully, even if sample size is small.
Again, item parameter estimation accuracy is moot if
the data are not appropriate for the model in the first
place. Third, for models that include item slopes,
consider their magnitude as being the key influence
on parameter estimation. Research (MacCallum,
Widaman, Zhang, & Hong, 1999) has shown that
the size of a factor loading is a key factor in how
many subjects are needed to conduct a factor analy-
sis (i.e., high loadings can be estimated with fewer
subjects). The IRT slope parameter is analogous to a
factor loading, and the same principle should apply.
Finally, it is important to recognize the location
parameters in the GRM are, generally speaking, eas-
ier to estimate than intersection parameters in
divide-by-total models. Assuming that the items
have a reasonable degree of slope, and that there are
sufficient responses in each category, location
parameters are relatively easy to estimate because all
the data are used. An intersection parameter, on the
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
other hand, is relatively harder to estimate than a
location because the intersection considers only
responses in x and x − 1. Thus, it is particularly
important to have sufficient responses in every cate-
gory for models like the NRM, GPCM, and PCM.
Finally, having decided to pursue IRT modeling,
a critical question is which model should be esti-
mated. Although the choice of model is often dic-
tated by what software an individual has access to,
for the sake of argument, we assume that any model
described herein can be estimated. With this in
mind, we note that from a statistical perspective, the
4, 3, 2, and 1 parameter logistic models for dichoto-
Copyright American Psychological Association. Not for further distribution.
mous item responses are nested, and thus a likeli-
hood ratio test may be useful in model comparisons.
Moreover, the same likelihood-ratio test also applies
to divide-by-total models where the NRM, GPCM,
and PCM also form a nested series. Unfortunately,
comparison of the GRM with the GPCM cannot be
made on the basis of a likelihood-ratio test, but
rather it must be made on the basis of other statisti-
cal criteria (e.g., the Akaike information criterion)
as well as substantive considerations.
Conversely, researchers seldom are concerned
with overall model fit but rather are concerned with
a comparison between the observed and estimated
IRC or CRCs on an item-by-item basis. In consider-
ing the various modeling options in IRT, we feel that
researchers spend too much effort worrying about
relative model fit and item parameter accuracy
rather than model outcomes. In any particular data
set, it may be the case that a model can be shown to
provide a better relative fit than another, but in
terms of outcomes, model choice is inconsequential.
Ostini and Nering (2006) pointed out that although
the polytomous IRT models may differ in a variety
of ways, few researchers have attempted to compare
the models in terms of outcomes as opposed to fit. It
is arguable that in terms of ultimate outcomes, such
as scale-response curves or individuals trait-level
estimates, the impact of model choice (or even poor
parameter estimation) is minimal. Similar argu-
ments were made by Embretson and Reise (2000),
who fitted a personality scale to a number of differ-
ent polytomous models and found that trait-level
estimates were above .90 regardless of model. This
demonstration should not be taken to mean that the
model never makes a difference. In fact, recent work
has argued that model accuracy is important in scal-
ing individuals correctly in certain trait ranges
(Waller & Reise, 2010).
Lessons Learned From the Application
of IRT to Noncognitive Measures
IRT modeling and its by-products (e.g., computer-
ized adaptive testing) can be considered standard
practice in education measurement, large-scale abili-
ties testing, and licensure testing. Although this
trend is by no means true in noncognitive assess-
ment, even in those domains, IRT modeling has
moved well beyond the stage of didactic articles and
proof-of-concept demonstrations. In fact, there is
now quite a substantial body of research that has
considered IRT modeling outside the cognitive
domain (see Cella et al., 2007, 2010; Reise &
Waller, 2009). In this section, we consider some
observations we have made regarding this research.
In turn these observations may suggest future
research issues and directions.
We have observed that several interesting phe-
nomena occur relatively frequently when polytomous
IRT models are applied to personality, health out-
comes, and psychopathology measures, namely the
following: (a) extreme item locations, (b) extremely
high item slopes, (c) bunched item locations, and
(d) an unusually large range of item slopes. In this
section, we discuss each of these phenomena in turn
and propose reasons why they occur. For simplicity
and continuity, we discuss these issues in the context
of the slope and location parameters resulting from
the application of the GRM. These same issues arise
in application of any model.
We begin with the phenomena of extremely high
item slope parameters. To be conservative, we define
high slopes as any value larger than 3 (logistic met-
ric), but note that even a slope of two is relatively
large compared with what is typically found in
dichotomous IRT models. It is easy to demonstrate
that as an item slope parameter moves beyond 3, the
response categories are providing a high degree of
discrimination among individuals. In other words,
with a slope of three in the GRM, if a researcher
knew an individual’s trait level, he or she could pre-
dict which the individual’s likely response category
715
 Reise and Moore
with high accuracy. Another way of thinking about
this issue is as follows. As item slopes increase
beyond three, the TRCs begin to look more like
Guttman step functions than monotonically increas-
ing ogives. Do such high slopes mean good mea-
surement, or is there something wrong?
We argue that the answer to this question
depends on understanding the causes of the high
slope parameters. To begin, it is well known that
one possible source of a high item slope is a problem
known as local dependency. Local dependency means
that two (or more) items share common variance
above and beyond that because of the latent trait
Copyright American Psychological Association. Not for further distribution.
being measured. Local dependence can be caused by
item content redundancy (i.e., asking essentially the
same question twice) and results in inflated item
slope parameters (Steinberg & Thissen, 1996). To
the extent that slopes are inflated by local dependen-
cies, the slope parameter estimates provide a mis-
leading gauge of measurement precision. However,
local dependence is not an explanation for high
slopes when all or most of the items on a scale dis-
play exceptionally high slopes. In this situation, we
need to turn to other explanations.
A second possible explanation for high slopes is
that the measured construct has a narrow conceptual
bandwidth (e.g., “knee joint pain”). Sparing the reader
the technical details, in IRT, the magnitude of an
item’s average correlation with other items determines
the item’s slope (or factor loading in factor analysis).
In measures of narrowband constructs, homogeneous
item content is expected. In addition, because of a
lack of diverse trait manifestations, in measures of
narrowband constructs the conceptual distance
between the item and the construct is often very small
(e.g., “I experience pain when bending my knee”).
These factors result in narrow measures containing
items with very high intercorrelations, which in turn
result in high IRT slope parameters. This is especially
true if a 5-, 7-, or 9-point rating scale is used for each
item because the more response options, the more
room is left for individual differences in response style
to operate, further inflating correlations.
When high item slopes are caused by the narrow-
band nature of the measured construct, their values
are perfectly valid indicators of measurement preci-
sion. However, there is one big caveat. Namely, that
716
measurement precision should not be viewed as
resulting from quality measurement, but rather as
resulting from the narrow construct that is being
measured. It is unsurprising to observe high slopes
on narrowband measures because individual differ-
ences are much easier to discriminate in that
context. Consider this example: The items 3 + 4 = ?,
2 + 6 = ?, and 1 + 3 = ? would form a highly discrimi-
nating set of indicators of the construct of “adding
two single digits.” In short, the fewer and more
homogeneous the trait (or ability) manifestations,
the easier it is to discriminate between those that are
high or low on the trait with high precision.
A narrow construct and the resulting item con-
tent homogeneity can explain some cases of
extremely high slopes, but it is irrelevant when high
slopes are observed on measures of complex and
multifaceted broadband constructs that contain
diverse item content. In this situation, we propose
two additional sources as possible explanations,
namely, (a) a mixture of extreme groups (e.g., clini-
cal and nonclinical) included in the calibration sam-
ple, and (b) a skewed or quasi-trait. These latter two
concepts are not synonyms, but discussion of them
is joined because in practice we argue that it is very
difficult to differentiate between a highly skewed
latent distribution and a quasi-dimensional trait.
In the measurement of psychopathology or health
outcomes, psychometricians have long warned
against the use of combined clinical and nonclinical
samples in judging the psychometric properties of
instruments. The obvious danger is that if a nonclini-
cal group with a floor effect is combined with a clini-
cal group with a ceiling effect, an instrument can
look deceptively good in terms of item intercorrela-
tions, item-test correlations, factor loadings, and
coefficient alpha. In short, in mixed clinical and non-
clinical samples, strong psychometrics reflect merely
the fact that the item differentiates between extreme
groups rather than indicating a good measure of a
dimensional construct across the entire range of the
construct. Because IRT slope parameters are complex
transformations of item intercorrelations, a mixed
clinical and nonclinical sample can easily result in
very high slope parameter estimates.
Finally, we propose that a second possible cause
of unusually high slopes on broadband measures is a
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
highly skewed or a quasi-trait. Although these con-
cepts are difficult to differentiate in practice, we
consider a skewed trait to be a fully continuous trait,
conceptually definable at both ends, but with a
strong floor or ceiling effect (e.g., an optimism–
pessimism scale applied in a culture in which almost
everyone is an optimist). A quasi-trait is a dimen-
sional construct that is defined only on one end of
the scale (i.e., meaningful individual differences
exist only on one end of the scale). Many constructs
in psychology, although assumed dimensional (e.g.,
aggression, self-esteem, spirituality), are possibly
only quasi-traits.
Copyright American Psychological Association. Not for further distribution.
For example, in our work on psychopathology
(Reise & Waller, 2003; Waller & Reise, 2010), we
noted that the low end of our measures, say depres-
sion, is not happiness or well-being but rather is
lack of depression (i.e., no symptoms). In working
with either highly skewed traits or quasi-traits, cau-
tion must be used in interpreting high slope parame-
ters. For the same reasons as articulated for mixed
samples, item slopes may be artificially inflated
because of either the non-normality or quasi-trait
status of the construct. In such cases, a high slope
may indicate that the item differentiates between
not-traited (not depressed) and traited (depressed)
individuals, but it may not necessarily provide a pre-
cise differentiation among people along a meaning-
ful continuous latent dimension.
The discussion of skewed traits or quasi-traits
leads directly to the next topic, namely, the issue of
bunched item location parameters. By bunched item
location parameters, we mean location parameters
(e.g., in the GRM) that are clumped closely together
along the latent variable (usually at the trait
extremes) rather than spread out over the trait con-
tinuum. When location parameters are bunched
together, this implies that the instrument affords
measurement precision in only a narrow trait range.
This is an interesting occurrence given that the only
legitimate purpose of having a multipoint response
format is to allow people to validly make discrimina-
tions across the continuum. If a researcher uses a
multipoint response scale in a diverse sample, and
location parameters still clump together, this is evi-
dence that either respondents cannot differentiate
among response options, or more likely, that what is
being measured is a highly skewed trait or quasi-
trait that is not a fully dimensional construct.
It appears that many researchers are operating
under the assumption that all constructs are fully
continuous, defined at both ends of the construct,
and that items can be found that measure (have
location parameters) across an entire trait range.
For example, in a different context, Andrich (1995)
conveyed the following sentiments:
Although measuring instruments have
operating ranges, a measurement is not
taken to be a function of the operating
range of any instrument—instead, if a
measurement is contaminated by the
operating range of the instrument (e.g.,
floor or ceiling effects), another instru-
ment with a range more compatible with
the location of the entity or object is
sought. (p. 101)
Such a statement assumes that, in theory, a
researcher could find an alternative instrument that
would not produce a floor or ceiling effect. For
another example, Fraley, Waller and Brennan
(2000), on observing that in attachment measures
item locations are highly bunched together on one
end of the trait, suggested that new items be written
to better spread out the measurement precision over
the complete range of the latent variable. But what if
such a search for new items or measures is fruitless?
With a highly skewed construct or a quasi-trait,
it may not be possible to find items with location
parameters spread out across the range. Consider a
research study by Gray-Little, Williams, and Han-
cock (1997), who applied the GRM to a 5-point ver-
sion of the Rosenberg Self-Esteem Scale (Rosenberg,
1989). This study is fascinating because although
the items contain five response options, the four
location parameters per item are closely bunched at
the low end of the latent trait (i.e., low self-esteem).
For example, even the third location (x = 1, 2, 3, vs.
4, 5) was in the negative range for 8 out of the 10
items. To make this more concrete, the third loca-
tion parameter for Item 1 (“On the whole, I am sat-
isfied with myself”) was b = −1.48, implying that
even individuals who are a standard deviation and a
half below the mean on the trait are most likely to
717
 Reise and Moore
respond in the highest fourth or fifth category. In
other words, even people far below the mean on
self-esteem rate themselves highly self-satisfied.
Although reasonable minds can disagree, one expla-
nation for this effect is that it is not due to poor
items or poor choice of response options. Rather, it
is due to the nature of the self-esteem construct;
items only differentiate people with low self-esteem
because that is the only end of the construct that is
meaningful.
Related to the topic of bunched item location
parameters is the topic of extreme item locations.
By extreme we mean that the location parameters
Copyright American Psychological Association. Not for further distribution.
that do not fall within a reasonable range of values,
say −2 to 2 (assuming that the latent variable has a
mean of zero and standard deviation of one). As
mentioned, one reason why an extreme location
may occur is if an item has a relatively low slope
parameter (recall that the location is a function of
the intercept divided by the slope). A second obvi-
ous reason is that either the lowest or highest
response category is too extreme given the content
of the item. A third, more interesting reason is that
the latent variable is either highly skewed or is a
quasi-trait.
We again use the results of the Gray-Little et al.
(1997) study to illustrate the phenomena. In their
calibration of the Rosenberg self-esteem scale, the
category locations not only were bunched in the low
end of the latent variable but also were very
extreme. For example, for all 10 items the first cate-
gory location (x = 1 vs. 2, 3, 4, 5) was an absurdly
low value. For example, for Item 1 (“On the whole, I
am satisfied with myself”), the estimated location
was b = −3.45, suggesting that even individuals who
are nearly three and a half standard deviations below
the mean will be likely to respond in at least cate-
gory 2. Moreover, for many items, even the fourth
location (x = 1, 2, 3, 4 vs. 5) is barely above the trait
mean. For example, for Item 3 (“I feel that I have a
number of good qualities”) the fourth location is
only b = 0.22. Thus, even people barely above the
mean on self-esteem are most likely to respond in
the highest category on this item. The only item
that displayed a more reasonable fourth location
was Item 9 (“All in all I am inclined to feel that I
am a failure”—reversed) that had a fourth threshold
718
of 2.13. This item has very low slope (a = 1.16),
however, relative to the high slope items that have
slopes ranging from 2.0 to 2.5. It is thus plausible
that this item is not measuring the same common
trait as the high slope items.
Our final practical issue in applying IRT models
to noncognitive data is the observation that on some
measures we have noticed a very wide spread of item
slopes. For example, Van Der Ark (2001) reported
slopes of 0.8, 3.6, 0.4, 6.4, and 0.2 for a measure of
coping strategy to industrial odor annoyance. We
call this a steep descent pattern, where one or two
items have a relatively high slope, and then the
value of the slope parameter decreases rapidly for
the remainder of the items. Although it is reasonable
that different trait indicators can be related to the
latent trait to different degrees, a wide variance in
slopes may also be a sign of problems. For example,
in Hall, Reise, and Haviland (2007), a nine-item
spiritual instability measure had GRM slopes rang-
ing from 2.63 (“When I sin, I am afraid of what God
will do to me”) to 0.74 (“When I sin, I tend to with-
draw from God”). The remaining items had slopes
of 1.9, 1.9, 1.5, 1.5, 1.4, 1.0, and 0.9, respectively.
Such a variable pattern of slopes is troubling in a
number of respects. First, one could argue that the
construct is so narrow that one item essentially
defines the latent variable and the other items are
only tangentially related. If that were true, one could
argue that the remaining items are unnecessary.
More technically, a second potential problem is that
if IRT methods were used to estimate standing on
the construct, then in this example the best item has
3.7 times the influence compared with the worst
item because the sufficient statistic for the latent
trait estimate is the raw item response times the item
slope. Moreover, because an item contributes to
measurement precision by the square of the slope
parameter, such results argue that the best item con-
tributes 12.7 times more error reduction than the
worst item; that is, it takes almost 13 items like the
worst item to equal one item like the best.
This wide-ranging slope phenomenon is not
unique to spirituality constructs and their associated
measures. In fact, this phenomena is rather easy to
observe in the IRT literature and non-IRT literature
(in the form of variable item–test correlations or
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
factor loadings). Ignoring multidimensionality as a
possible cause, we propose that this phenomena is
due to a combination of narrowband constructs
(where one or two good items essentially define the
construct), combined with limited item pools. Many
constructs in personality, health, and psychopathol-
ogy have an extremely limited indicator pool (e.g.,
how many ways are there to react to industrial
odor?). Even a relatively complex and multifaceted
construct such as depression has a finite set of indi-
cators (e.g., sad moods, social isolation, suicide cog-
nitions, feelings of hopelessness, and somatic
disturbances). Importantly, when only a few items
Copyright American Psychological Association. Not for further distribution.
have high slopes and the remainder has much lower
slopes, a researcher must be cautious in interpreting
the latent variable. It could well be that the latent
variable does not reflect variance on a common
latent variable shared by all the items, but rather it
merely reflects individual differences on the item
with the highest slope.
In sum, this section is not meant to disparage or
discourage IRT applications to typical performance
constructs. Our goal is merely to draw attention to
some interesting challenges that researchers may
face in applying polytomous models to noncognitive
constructs. As we stated, IRT emerged in the context
of large-scale cognitive ability testing. In that con-
text, it is relatively easy to conceive of normally dis-
tributed continuous traits and unlimited item pools.
For example, it is easy to envision an endless num-
ber of spelling, algebra, analogy, constitutional law,
or nursing skills questions. In turn, it is relatively
easy to envision these items as varying in their loca-
tion across the trait range. Moreover, large-scale
cognitive ability researchers are typically working
with broadband constructs (verbal ability, algebra,
knowledge of the law, nursing expertise) for which
domains of item content (learning or skill domains)
are well articulated.
In contrast, as the IRT technology is transported
from the cognitive abilities realm into the broader
world of performance assessment, special issues and
problems are bound to emerge (Reise, 2009). This is
especially true given that noncognitive researchers
have to work with constructs of varying conceptual
breath, for which a set of limited indicators exist,
and for which the underlying distribution cannot
possibly be standard normal. To the degree that IRT
applications in noncognitive settings raise issues
that have not caught the attention of previous
researchers (e.g., the Gray-Little et al., 1997, study),
or call into question the quality of legacy measures
developed under traditional coefficient alpha–
centric scale construction practices, this is a positive
development for the field of psychometrics. Indeed,
part of the excitement of current IRT research lies in
identifying new problems and working toward their
solutions.
References
Andrich, D. (1978). A rating formulation for ordered
response categories. Psychometrika, 43, 561–573.
doi:10.1007/BF02293814
Andrich, D. (1988). A general form of Rasch’s extended
logistic model for partial credit scoring. Applied
Measurement in Education, 1, 363–378. doi:10.1207/
s15324818ame0104_7
Andrich, D. (1995). Distinctive and incompatible proper-
ties of two common classes of IRT models for graded
responses. Applied Psychological Measurement, 19,
101–119. doi:10.1177/014662169501900111
Baker, F. B., & Kim, S.-H. (2004). Item response theory:
Parameter estimation techniques (2nd ed.). New York,
NY: Marcel Dekker.
Ben-Porath, Y. S., Tellegen, A. T., & Kaemmer, B.
(2005). MMPI–A: Minnesota Multiphasic Personality
Inventory—Adolescent™: Booklet of abbreviated items.
Minneapolis: University of Minnesota Press.
Bock, R. D. (1972). Estimating item parameters and
latent ability when responses are scored in two or
more latent categories. Psychometrika, 37, 29–51.
doi:10.1007/BF02291411
Bollen, K., & Lennox, R. (1991). Conventional wis-
dom on measurement: A structural equation per-
spective. Psychological Bulletin, 110, 305–314.
doi:10.1037/0033-2909.110.2.305
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch
model: Fundamental measurement in the human sci-
ences (2nd ed.). Mahwah, NJ: Erlbaum.
Borsboom, D. (2005). Measuring the mind: Conceptual
issues in contemporary psychometrics. Cambridge,
England: Cambridge University Press. doi:10.1017/
CBO9780511490026
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve,
B., Yount, S., . . . Hays, R. D. (2010). Initial item
banks and first wave testing of the Patient-Reported
Outcomes Measurement Information System
(PROMIS) network: 2005–2008. Journal of Clinical
Epidemiology, 63, 1179–1194.
719
 Reise and Moore
Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook,
K., Reeve, B., & Rose, M. (2007). The Patient-
Reported Outcomes Measurement Information
System (PROMIS): Progress of an NIH Roadmap
Cooperative Group during its first two years. Medical
Care, 45(5, Suppl. 1), S3–S11. doi:10.1097/01.
mlr.0000258615.42478.55
de Ayala, R. J. (2009). The theory and practice of item
response theory. New York, NY: Guilford Press.
De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory
item response models: A generalized linear and nonlin-
ear approach. New York, NY: Springer.
Dodd, B. G., & Koch, W. R. (1987). Effects of variations
in item step values on item and test information
in the partial credit model. Applied Psychological
Copyright American Psychological Association. Not for further distribution.
Measurement, 11, 371–384. doi:10.1177/01466216
8701100403
Embretson, S. E. (1996). The new rules of measure-
ment. Psychological Assessment, 8, 341–349.
doi:10.1037/1040-3590.8.4.341
Embretson, S. E., & Reise, S. P. (2000). Psychometric
methods: Item response theory for psychologists.
Mahwah, NJ: Erlbaum.
Fox, J-P., & Glas, C. A. W. (2001). Bayesian estima-
tion of a multilevel IRT model using Gibbs sam-
pling. Psychometrika, 66, 271–288. doi:10.1007/
BF02294839
Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000).
An item response theory analysis of self- report mea-
sures of adult attachment. Journal of Personality and
Social Psychology, 78, 350–365. doi:10.1037/0022-
3514.78.2.350
Gibbons, R. D., & Hedeker, D. (1992). Full-information
item bi-factor analysis. Psychometrika, 57, 423–436.
doi:10.1007/BF02295430
Gray-Little, B., Williams, V. S. L., & Hancock, T. D.
(1997). An item response theory analysis of the
Rosenberg Self-Esteem Scale. Personality and Social
Psychology Bulletin, 23, 443–451. doi:10.1177/
0146167297235001
Gustafsson, J. E., & Aberg-Bengtsson, L. (2010).
Unidimensionality and the interpretability of psy-
chological instruments. In S. E. Embretson (Ed.),
Measuring psychological constructs (pp. 97–121).
Washington, DC: American Psychological Association.
doi:10.1037/12074-005
Hall, T. W., Reise, S. P., & Haviland, M. G. (2007). An
item response theory analysis of the spirituality
assessment inventory. International Journal for the
Psychology of Religion, 17, 157–178.
Hambleton, R. K., & Swaminathan, H. (1985). Item
response theory. Boston, MA: Kluwer-Nijhoff.
Hambleton, R. K., Swaminathan, H., & Rogers, H.
J. (1991). Fundamentals of item response theory.
Newbury Park, CA: Sage.
720
Humphreys, L. G. (1970). A skeptical look at the factor
pure test. In C. E. Lunneborg (Ed.) Current problems
and techniques in multivariate psychology: Proceedings of
a conference honoring Professor Paul Horst (pp. 23–32).
Seattle: University of Washington.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of
mental test scores. Reading, PA: Addison-Wesley.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S.
(1999). Sample size in factor analysis. Psychological
Methods, 4, 84–99. doi:10.1037/1082-989X.4.1.84
Masters, G. N. (1982). A Rasch model for partial credit
scoring. Psychometrika, 47, 149–174. doi:10.1007/
BF02296272
McLeod, L. D., Swygert, K. A., & Thissen, D. (2001).
Factor analysis for items scored in two categories.
In D. Thissen & H. Wainer (Eds.), Test scoring
(pp. 189–216). Mahwah, NJ: Erlbaum.
Muraki, E. (1992). A generalized partial credit
model: Application of an EM algorithm. Applied
Psychological Measurement, 16, 159–176.
doi:10.1177/014662169201600206
Nering, M. L., & Ostini, R. (Eds.). (2010). Handbook of
polytomous item response theory models. New York,
NY: Taylor & Francis.
Ostini, R., & Nering, M. L. (2006). Polytomous item
response theory models. Thousand Oaks, CA: Sage.
Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M.,
Riley, W. T., & Cella, D. (2011). Item banks for
measuring emotional distress from the Patient-
Reported Outcomes Measurement Information
System (PROMIS): Depression, anxiety, and anger.
Assessment, 18, 263–283.
Preston, K. S. J., Reise, S. P., Cai, L., & Hays, R. D.
(2011). Using the nominal response model to evalu-
ate response category discrimination in the PROMIS
emotional distress item pools. Educational and
Psychological Measurement, 71, 523–550.
Reckase, M. D. (2010). Multidimensional item response
theory. New York, NY: Springer.
Reise, S. P. (2009). The emergence of item response
theory (IRT) models and the Patient-Reported
Outcomes Measurement Information System
(PROMIS). Austrian Journal of Statistics, 38, 211–220.
Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005).
Item response theory: Fundamentals, applications,
and promise in psychological research. Current
Directions in Psychological Science, 14, 95–101.
doi:10.1111/j.0963-7214.2005.00342.x
Reise, S. P., & Henson, J. M. (2003). A discussion
of modern versus traditional psychometrics as
applied to personality assessment scales. Journal of
Personality Assessment, 81, 93–103. doi:10.1207/
S15327752JPA8102_01
 An Introduction to Item Response Theory Models and Their Application in the Assessment of Noncognitive Traits
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010).
Bifactor models and rotations: Exploring the extent
to which multidimensional data yield univocal
scale scores. Journal of Personality Assessment, 92,
544–559.
Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role
of the bifactor model in resolving dimensionality
issues in health outcomes measures. Quality of Life
Research, 16, 19–31.
Reise, S. P., & Waller, N. G. (2003). How many IRT
parameters does it take to model psychopathol-
ogy items. Psychological Methods, 8, 164–184.
doi:10.1037/1082-989X.8.2.164
Reise, S. P., & Waller, N. G. (2009). Item response
theory and clinical measurement. Annual Review of
Copyright American Psychological Association. Not for further distribution.
Clinical Psychology, 5, 27–48. doi:10.1146/annurev.
clinpsy.032408.153553
Reise, S. P., & Yu, J. (1990). Parameter recovery in the
graded response model using MULTILOG. Journal of
Educational Measurement, 27, 133–144. doi:10.1111/
j.1745-3984.1990.tb00738.x
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E.
(2000). A general item response theory model for
unfolding unidimensional polytomous responses.
Applied Psychological Measurement, 24, 3–32.
doi:10.1177/01466216000241001
Rosenberg, M. (1989). Society and the adolescent self-
image (rev. ed.). Middletown, CT: Wesleyan
University Press.
Rouse, S. V., Finger, M. S., & Butcher, J. N. (1999).
Advances in clinical personality measurement: An
item response theory analysis of the MMPI–2 PSY-5
scales. Journal of Personality Assessment, 72, 282–307.
doi:10.1207/S15327752JP720212
Samejima, F. (1969). Estimation of latent ability using
a response pattern of graded scores. Psychometrika,
Monograph Supplement 17.
Sijtsma, K., & Molenaar, I. W. (2002). Introduction to
nonparametric item response theory. Thousand Oaks,
CA: Sage.
Steinberg, L., & Thissen, D. (1996). Uses of item
response theory and the testlet concept in the mea-
surement of psychopathology. Psychological Methods,
1, 81–97. doi:10.1037/1082-989X.1.1.81
Thissen, D. (2003). MULTILOG 7: Multiple categorical
item analysis and test scoring using item response
theory [Computer software]. Chicago, IL: SSI.
Thissen, D., Cai, L., & Bock, R. D. (2010). The nominal
categories item response model. In M. Nering & R.
Ostini (Eds.), Handbook of item response theory mod-
els (pp. 43–75). Philadelphia, PA: Taylor & Francis.
Thissen, D., & Steinberg, L. (1986). A taxonomy of
item response models. Psychometrika, 51, 567–577.
doi:10.1007/BF02295596
Thissen, D., Steinberg, L., & Fitzpatrick, A. R. (1989).
Multiple-choice models: The distractors are also part
of the item. Journal of Educational Measurement, 26,
161–176. doi:10.1111/j.1745-3984.1989.tb00326.x
Tutz, G. (1990). Sequential item response models with an
ordered response. British Journal of Mathematical and
Statistical Psychology, 43, 39–55.
Van Der Ark, L. A. (2001). Relationships and proper-
ties of polytomous item response theory models.
Applied Psychological Measurement, 25, 273–282.
doi:10.1177/01466210122032073
Wainer, H. (2000). Computerized adaptive testing: A
primer (2nd ed.). Mahwah, NJ: Erlbaum.
Wainer, H., & Thissen, D. (1987). Estimating ability with
the wrong model. Journal of Educational Statistics, 12,
339–368. doi:10.2307/1165054
Waller, N. G., & Reise, S. P. (2010). Measuring psychopa-
thology with nonstandard item response theory mod-
els: Fitting the four-parameter model to the Minnesota
Multiphasic Personality Inventory. In S. Embretson
(Ed.), Measuring psychological constructs: Advances in
model-based approaches (pp. 147–173). Washington,
DC: American Psychological Association. doi:10.1037/
12074-007
Williams, C. L., Butcher, J. N., Ben-Porath, Y. S.,
& Graham, J. R. (1992). MMPI–A content
scales: Assessing psychopathology in adolescents.
Minneapolis: University of Minnesota Press.
Wilson, M. (2005). Constructing measures: An item
response modeling approach. Mahwah, NJ: Erlbaum.
Zimowski, M., Muraki, E., Mislevy, R., & Bock, R. D.
(2003). BILOG-MG (Version 3) [Computer
software]. Lincolnwood, IL: SSI.
721