Measurement 46 (2013) 3766–3774

Contents lists available at SciVerse ScienceDirect

Measurement
journal homepage: www.elsevier.com/locate/measurement

Using the concept of a measurement system to characterize

measurement models used in psychometrics
Mark Wilson ⇑
University of California, Berkeley, CA 94720, USA

a r t i c l e i n f o a b s t r a c t

Article history: The philosophy of measurement in the social and behavioral sciences is seen (from with-
Available online 18 April 2013 out) as typically following the representational viewpoint. However, in practice, this is
not the case for the great majority of measures that are developed in this area. The paper
Keywords: surveys several approaches to measurement in the social sciences (i.e., Classical Test The-
Measurement system ory, Guttman Scaling, Item Response Theory, Rasch Scaling, and Construct Modeling), as
Classical Test Theory examples of measurement approaches in the area of psychometrics, and uses the founda-
Guttman Scaling
tional concept of a measuring system, as developed by Mari [1], to explicate the logic on
Item Response Theory
Rasch Scaling
which these approaches are based and thus enable a comparison with measurement
Construct Modeling approaches used by other fields such as engineering and physics. The paper uses the under-
lying concept of the standard reference set (one of the essential features of Mari’s formaliza-
tion) to show how the five approaches differ, and also how they are related. The
importance of these differences, and the consequences for measurement using those
approaches are also explicated and discussed.
Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction sciences measurement than what is seen currently as being

In the social sciences in general, and in the behavioral
sciences in particular, the study of measurement is most
commonly sustained within the discipline called ‘‘psycho-
metrics’’. There are for each discipline within the social sci-
ences, particular subdomains that pertain to measurement
within those disciplines in particular—for example, in soci-
ology, there is the sub-discipline of ‘‘sociometrics’’, etc.,
but, in the main, the major debates and developments in
these specific sub-disciplines reflect the situation that per-
tains in psychometrics. Thus, in the paper, I will use psy-
chometrics as a locale for my discussing approaches to
measurement in the social sciences. My particular take
on this is to sketch an alternative conceptualization for
the philosophical foundations of measurement in the social
sciences that aligns better with the actual practice of social
⇑ Tel.: +1 510 708 0322; fax: +1 510 642 4803.
E-mail address: MarkW@berkeley.edu
the dominant philosophical viewpoint on measurement.
In psychology and the behavioral sciences in general,
and in psychometrics in particular, the representational
viewpoint [2] is widely seen as dominating the philosoph-
ical foundations of measurement. This is quite understand-
able, as the combined philosophical works of the authors of
Ref. [2] stand head and shoulders above the works of any
others in this area. Not that there are not significant works
by others, but that the scope and comprehensiveness of [2]
is generally seen as being without equal. However, this
dominance in formal modeling has little or no correspon-
dence with the reality of most actual measures that are
constructed in these domains. The sad state of philosophiz-
ing in the area of psychometrics is that the philosophical
grounding provided by these giants of the field is ‘‘More
honor’d in the breach than the observance’’ [3]. In fact, it
is somewhat rare to find examples of applications of the
representational approach beyond the works of the
authors of [2]. One reaction to this has been for some
authors to amend the tenets of the representational ap-

0263-2241/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.measurement.2013.04.005
 M. Wilson / Measurement 46 (2013) 3766–3774
proach to incorporate a probabilistic element (e.g., [4,5]).
Another reaction has been to seek alternative philosophical
bases for measurement, such as ‘‘scientific realism’’ (e.g.,
[6,7]). The debate about this is still in its early stages, with
several presentations given and planned at psychometric
conferences, and only little of it yet having reached publi-
cation (though see [8] for some background to this debate).
In fact, my view is that most practicing psychometricians
do not actually subscribe to a representational measure-
ment philosophy. In fact, it seems to me that there is very
little representationalism present in mainstream psycho-
metrics, except insofar as Stevens’ definition [9] of mea-
surement can be seen as being consistent with
representationalism.
As a contribution to this debate, this paper utilizes the
functional concept of a measuring system, as developed by
Mari [1], to explicate the logic of several measurement ap-
proaches used in psychometrics (for a summary see [10]),
and thus establish grounds for the comparison of these
with measurement approaches used by other fields such
as engineering and physics.
2. Brief background on the concept of a measuring
system
A very comprehensive account of the concept of a mea-
surement system can be found in Mari [1]—in this section,
only a brief account is given. The operation of a measuring
system (MS), as described by Mari [1], is summarized by
Fig. 1.
In this figure, the evaluation of the ‘‘thing’’ is accom-
plished by the following 3-step procedure.
(1) Establish a calibration: create a ‘‘standard reference
set’’ (of things) A, by associating them with a set of
symbols H. Generally these symbols will be the ele-
Fig. 1. Summary of the operation of a measuring system.
Fig. 2. Establishing a calibration.
Fig. 3. Data acquisition.
3767
Fig. 4. Data Presentation.
ments of a mathematical structure such as a set of
integers or the real numbers along with their usual
arithmetic relations: mI(a) = h (see Fig. 2).
(2) Carry out data acquisition: through empirical inter-
action, select an element of A that corresponds to
the thing up (for person p) using the ‘‘thing selection
function’’ v(), i.e., v(up) = a (see Fig. 3).
(3) Make the data presentation (i.e., re-use calibration):
select the symbol to measure by compounding 1
and 2: m(up) = mI(v(up)) = h (see Fig. 4).
What makes this a measuring system is that this map-
ping m is a homomorphism between the empirical rela-
tional structure of things (U, RU), and the symbolic
relational structure of symbols (H, RH): that is rU(up)
whenever m(rU)(m(up)). In Mari’s approach, evaluations
are measurements if and only if this latter is the case. In
the following we will see how this looks for some measure-
ment approaches that have been used in psychometrics.
3. Guttman Scaling
For all the examples of measurement approaches in this
paper, the basic situation under consideration is that we
have an instrument composed of a set of items, I = {I1, . . . , II}
for which prior substantive theory indicates that a person’s
response to those items (indicated by the response vector
up for person p) is an indicator of the construct to be mea-
sured, H (i.e., the measurand). Without loss of generality, I
will assume that the responses to the items are dichoto-
mous, that is 0 or 1.
The Likert style of item has been a very common form of
item in the social sciences. The most generic form of this is
the provision of a stimulus statement (sometimes called a
‘‘stem’’), and a set of standard options that the respondent
must choose among. Possibly the most common set of op-
tions is the set {Strongly Agree, Agree, Disagree and
Strongly Disagree}, with sometimes a middle or neutral
option. The particular set of options used may be adapted
from those given above to match the context. For example,
an example of s Likert-style item would be:
I believe in fairies. (Choose one option)
Strongly Agree, Agree, Disagree, Strongly Disagree.
It is relatively easy to come up with many items when
all that is needed is a new stem for each one. Although this
makes it a very popular approach, there is certain dissatis-
faction with the way that the response options relate to the
construct. The problem is that there is very little to guide a
respondent in judging what is the difference between the
options, say, ‘‘Strongly Disagree’’ and ‘‘Agree’’. Respondents
may well have radically different ideas about these distinc-
tions. This problem is greatly aggravated when the option
offered are not even words, but a set of numerals or letters,
3768 M. Wilson / Measurement 46 (2013) 3766–3774
such as {1, 2, . . . , 8, 9}—in this sort of array, the respondent
does not even get a hint as to what it is that s/he is sup-
posed to be making distinctions between!
One alternative that has been developed is to build into
each option set meaningful statements that give the
respondent some context in which to make the desired dis-
tinctions. The aim here is to try and make the relationship
between each item and the overall scale interpretable. This
approach was formalized by Guttman [11] who created his
scalogram approach (also known as Guttman Scaling):
If a person endorses a more extreme statement, he
should endorse all less extreme statements if the state-
ments are to be considered a [Guttman] scale. . .We
shall call a set of items of common content a scale if a
person with a higher rank than another person is just
as high or higher on every item than the other person
[11, p. 62].
Thus, for example, suppose we hypothesize four dichot-
omous attitude (Likert-style) items that do indeed form a
Guttman scale. If the order of items, 1, 2, 3, 4 is in this case
also the scale order, and the responses are ‘‘Agree’’ and
‘‘Disagree’’, then only the following responses are possible
under the Guttman scale requirement:
(Agree, Agree, Agree, Agree),
(Agree, Agree, Agree, Disagree),
(Agree, Agree, Disagree, Disagree),
(Agree, Disagree, Disagree, Disagree),
(Disagree, Disagree, Disagree, Disagree).
If all the responses are of this type, then, when they are
scored (say, ‘‘Agree’’ = 1, and ‘‘Disagree’’ = 0), then there is a
one-to-one relationship between the sumscores and the
set of item responses. A person with a sumscore of 1 must
have agreed with Item 1 and not the rest, and thus can be
interpreted as being somewhere between Item 1 and Item
2 in his/her views. Similarly, a person who scored 3 must
have agreed with the first three items and disagreed with
the last, so can be interpreted as being somewhere be-
tween Item 3 and Item 4 in his/her views. Other responses
Fig. 5. Examples of Guttman’s items [11].
to the item set, such as ‘‘Disagree, Disagree, Agree, Dis-
agree’’, would indicate that the items did not form a perfect
Guttman scale.
Four items developed by Guttman using this approach
are shown in Fig. 5. These items were used in a study of
American soldiers returning from the Second World
War—they have more than two categories, which makes
them somewhat more complicated to interpret as Guttman
items, but nevertheless, they can still be thought of in the
same way.
Thus, in Guttman Scaling the essential idea is that, on
the basis of substantive theory and practical knowledge
about items, one can order, in terms of expected responses,
a set of items from ‘‘easiest’’ to ‘‘hardest’’ (or ‘‘least posi-
tive’’ to ‘‘most positive’’, etc., depending on the context).
Then the ordered set of possible response vectors is as-
sumed to constitute a standard reference set, directly indi-
cating the value of H in terms of the rank of the ‘‘highest’’
item for which the response is 1:
mðup Þ ¼ guttmanscoreðup Þ;
which is the rank of v(up) minus 1 if it is defined, and
undefined otherwise. As there is a finite number of items,
I, the usual symbol set is the integers 1 to I + 1. The stan-
dard reference set can be written as A = {a0, a1, . . . , aI}
where
a0 ¼ fð0; 0; . . . ; 0Þg
a1 ¼ fð1; 0; . . . ; 0Þg
a2 ¼ fð1; 1; 0; . . . ; 0Þg
..
.
aI ¼ fð1; 1; . . . ; 1; 1Þg:
Note these particular response vectors in A are called
‘‘Guttman true scale-types’’. There is a frustrating incom-
pleteness in this approach, as the there are many possible
response vectors that are not scale-types, response such as
(1, 0, 1, 0, . . . , 0), etc. Hence, one aspect of the creation of
the instrument I is the selection of items that are suitable
for use in a Guttman scale—intuitively, one would seek
 M. Wilson / Measurement 46 (2013) 3766–3774
items that are increasing in ‘‘difficulty’’ as one went from
the first to the last item, and where the increments in dif-
ficulty were as large as possible (although this will become
more difficult to achieve if the number of items, I, is large).
Guttman recommended the use of a quantitative indicator,
the coefficient of reproducibility [11], which is the ratio of
the observed number of true scale-type response vectors
to the total number of responses, to gauge the suitability
of the set of items for use in a Guttman scale, both in a rel-
ative sense (i.e., sets with larger coefficients are better),
and in an absolute sense (e.g., accept item sets with coeffi-
cient values greater than .85).
The problem of what to do with persons with non-
scale-types, which can constitute a high proportion of the
sample, has led to the low usage of this approach in most
areas of application. According to Kofsky [12, pp. 202–203],
. . . the scalogram model may not be the most accurate
picture of development, since it is based on the assump-
tion that an individual can be placed on a continuum at
a point that discriminates the exact [emphasis added]
skills he has mastered from those he has never been
able to perform. ... A better way of describing individual
growth sequences might employ probability statements
about the likelihood of mastering one task once another
has been or is in the process of being mastered.
The next approach can be seen as a step towards deal-
ing with this problem, although historically it much pre-
dates Guttman Scaling.
4. Classical Test Theory
In Classical Test Theory (CTT) [13–16], the problem of
what to do about non-scale-types is finessed by simply
ignoring it. Examples of item-types that are used in mea-
sures that use a CTT approach include the Likert-style
and Guttman type items discussed in the previous section,
as well as the common multiple choice type items used in
educational achievement test, as well as open-ended essay
type items that require ratings by judges. Response vectors
are given a symbol (usually called a ‘‘score’’) that is equal
to what the Guttman score would have been if the 1s
and 0s had been ordered according to a scale-type, or
equivalently, the sum score of the response vector:
m(up) = sumscore (up) = R upi, where upi is the ith response
in the vector up. The symbol set H will then be the integers
from 0 to the maximum score I. The standard reference set
can be written as A = {a0, a1, . . . , aI} where
a0 ¼ fð0; 0; . . . ; 0Þg
a1 ¼ fð1; 0; . . . ; 0Þ; ð0; 1; . . . ; 0Þ; . . . ; ð0; 0; . . . ; 0; 1Þg
a2 ¼ fð1; 1; 0; . . . ; 0Þ; ð1; 0; 1; 0; . . . ; 0Þ; . . . ; ð0; 0; . . . ; 0; 1; 1Þg
..
.
aI ¼ fð1; 1; . . . ; 1; 1Þg:
Note that this standard set now accounts for every possible
response vector under the assumptions. Just as for Gutt-
man Scaling, the item set I is typically a smaller set that re-
sults from some item selection based on empirical data.
However, unlike Guttman Scaling the criteria are not based
3769
on substantive theory about the interpretation of the
items. Instead the standard criteria are based on statistical
considerations having to do with various aspects of uncer-
tainty. These are grouped together under the term ‘‘item
analysis’’ [17]—typical criteria are:
(a) Reliability of the item set, reliability(I).
(b) Discriminations of the items, discrimination(Ii).
(c) Etc.
Uncertainty in the measure is estimated in terms of the
standard error of measurement (sem) for the scores, where
pffiffiffiffiffiffiffiffiffiffiffi
sem ¼ S 1  r;
where S is the standard deviation of the scores, and r is the
reliability [17].
Using this concept, each measure should be more accu-
rately expressed as a binary: (h, sem). Effectively, this
moves the symbol set H beyond the set of integers in the
interval [0, I], as it was above, to encompass the segment
of the real number line, [0  d, I + d], where the value of
d is dependent on the level of uncertainty one wishes to
express. The most common representation of CTT is Xp = -
Tp + Ep where Xp, is the observed score for person p (or,
sumscore(up) above), Ep is the error (or sem above) and Tp
is the ‘‘true score’’ (i.e., theoretical average of person p’s ob-
served score over a large (infinite) number of observa-
tions). Note that, in historical terms, CTT was developed
long before Guttman Scaling (as is indicated by the publi-
cation dates on the references above). However, the order
has been reversed in this account, in order to emphasize
the potential importance of the meaning of the standard
set.
One alternative to the sum score symbols that is often
used is the percentile. This is simply the value of the cumu-
lative distribution function for the sum score in a chosen
reference sample of persons, expressed as a percentage.
This conceals the differences between instruments that
have different numbers of items, and can be used as a basis
for the ‘‘equating’’ of such instruments. This approach is of-
ten referred to as ‘‘norm-referenced testing’’. For this ap-
proach, the symbol set (ignoring the sem issue) is the real
numbers between 0 and 100. All of the remaining points
above hold, however.
Sometimes, when a decision is to be made on the basis
of the measures whether a person is ‘‘above’’ some point
(or, equivalently, ‘‘below’’). This requires the setting of a
cut-score. If that is the sole purpose of the instrument, this
can be seen as equivalent to establishing a new, coarser,
standard reference set A0 ¼ fa00 ; a01 g, where (assuming the
cut-off is k)1
a00 ¼ fa0 ; a1 ; . . . ; ak g
a01 ¼ fakþ1 ; akþ2 ; . . . ; aI g:
where the ais are as in the definition above. The cut-score
will usually be set using a procedure that invokes substan-
tive knowledge from among professionals in areas related
1
Note that in this expression, {a0, a1, . . . , ak} is taken to mean the set of all
the response vectors in the sets a0, a1, . . . , ak rather than the set of the k sets
a0, a1, . . . , ak. This is true for similar expressions below.
3770 M. Wilson / Measurement 46 (2013) 3766–3774
to the construct and the typical applications. Clearly, this
same procedure can be repeated to have multiple cut-
scores.
5. Item Response Theory
In Item Response Theory [18] the CTT approach is
amended and extended to (a) formalize the relationship
between the person and the item (i.e., rather than the
instrument as a whole) using a mathematical model, (b)
adopt a metric (most commonly the log of the odds) that
frees the scale from a dependence on the (largely) inciden-
tal aspect of the number of items. There are several differ-
ent mathematical models in common use—I will illustrate
only the simplest, the Rasch model: for this model, which
assumes dichotomous data, represented as 0 and 1, the
mathematical relationship is given by:
Prðupi ¼ 1jhp ; di Þ ¼ ppi1 ¼ expðhp  di Þ=Wpi ;
where hp is the person symbol (often called the ‘‘location’’,
or the ‘‘ability’’ depending on the context), di is an item
parameter (often termed the ‘‘difficulty’’), and Wpi is a nor-
ming value equal to [1 + exp(hp  di)]. The connection to
the log-odds is evident as:
logðppi1 =ppi0 Þ ¼ hp  di
(where ppi0 has an obvious definition). The probability of a
response vector is given by applying the local indepen-
dence assumption:
Y
I mid-way between CTT and Guttman Scaling, the symbols
Prðup jhp ; dÞ ¼ Prðupi jhp ; di Þ;
i¼1
where d is the vector of item parameters. This special case
is commonly termed Rasch Scaling.
In psychometric modeling, many other functions
(termed ‘‘item response models’’) besides the simple logistic
function are used. Other models that are commonly used are
the two-parameter (2PL) and the three-parameter (3PL) lo-
gistic models [18]: In particular, the 2PL adds a so-called dis-
crimination parameter for each item, ai, and the 3PL adds a
pseudo-guessing parameter, ci, for each item. There are also
versions of these models that are based on the cumulative
distribution of the Gausian probability distribution.
In the special case of the Rasch model, just as for CTT, a
symbol is found for each possible response vector, and, due
to the fact that the sumscores are sufficient statistics for
the person location [19,20], the initial standard reference
set is the same as for CTT (see above). However, unlike
the case for CTT, the symbols are not automatically as-
signed via a simple explicit function based on the sum-
scores, but must be statistically estimated [18], and may
take values anywhere on the real number line (1,
1)2—we denote the estimated value of hp, for person p with
response vector up, as h (up).
2
The minimum and maximum cases, (0, 0 . . . , 0) and (1, 1 . . . , 1) receive
the symbols 1 and 1, respectively. (In practical terms, these symbols are
not very useful, and practitioners typically either refrain from giving
persons with those response vectors symbols, or they assign finite values to
them, following certain procedures.)
Just as for Guttman Scaling and CTT, the item set I is
typically a smaller set that results from some item selec-
tion based on empirical data. As for CTT the criteria are
based on statistical considerations having to do with vari-
ous aspects of uncertainty, such as uncertainty that the
items are a satisfactory source of data for the measurand,
and uncertainty whether the logistic function specified
above is a good choice.. These are grouped together under
the term ‘‘item fit analysis’’ [18–20], and generally capital-
ize on the probability of response vectors to accumulate
the likelihood of observing the responses to a given item,
given its difficulty, and the person parameters for the ob-
served persons (more traditional CTT item analysis proce-
dures are also commonly used). Other considerations are
also involved, such as the match between the observed
and expected distribution of persons on the logit scale,
and the match of item locations to that distribution3.
Uncertainty in the measure is expressed in terms of the
standard error for the person estimates, which is found as
a by-product of the estimation procedures, and differs from
the standard error for CTT in that it is conditional on the
location itself: s(h). As for CTT, each person’s measure should
be more accurately expressed as a binary: (h, s(h)).
Note that one way to see the standard reference set for
Guttman Scaling is that it is the standard set for CTT where
we have accepted some response vectors as being possible
(i.e., the Guttman true-types), and others as not (i.e., the
non-Guttman types). Put another way, the Guttman true-
types could be seen as having a non-zero probability, and
the rest as having a zero probability. In a sense that is
in the Rasch Scaling standard reference set are not ac-
corded equal standing—each response vector has a proba-
bility of being observed (conditional on the estimated
parameters d. Thus, some response vectors will have smal-
ler probabilities than others—this would be the case, for
example for the response vector (0, 0, . . . , 0, 1) in the case
where the items are ordered in terms of item difficulty.
This fact has been used as the basis for developing several
‘‘person fit statistics’’ [18,21], which are then used to de-
cide that some persons with low-probability response vec-
tors should then not be assigned symbols (estimates). This
results in a reduced standard set B, although the number of
symbols (estimates) remains the same. Thus the standard
reference set can be written as B = {b0, b1, . . . , bI} where
b0  a0 ¼ fð0;0; . .. ; 0Þg
b1  a1 ¼ fð1; 0; .. . ; 0Þ;ð0; 1; . .. ; 0Þ; . .. ; ð0; 0; .. . ; 0; 1Þg
b2  a2 ¼ fð1; 1; 0; .. . ; 0Þ; ð1;0; 1;0; . .. ; 0Þ; .. . ; ð0; 0; .. . ; 0; 1; 1Þg
..
.
bI  aI ¼ fð1; 1; . .. ; 1; 1Þg;
(i.e., each bi is a subset of the respective ai in CTT), and the
rules for reducing the sets a0 through aI are determined by
the specific person fit procedures chosen.
In the 2PL and 3PL, the standard reference set will (in
non-trivial cases) be much larger than that shown above,
3
Note that the graphical device showing both persons and items on the
same scale is sometimes referred to as a ‘‘Wright map.’’
 M. Wilson / Measurement 46 (2013) 3766–3774
as, in general, every different response vector u, will have a
different estimate h (up) (i.e., this can be seen as a conse-
quence of the property of the 2PL that the sufficient statis-
tic for h (up) is given by a weighted sumscore, R ai upi, and
this implies that the 3PL also has the same larger symbol
set (although with different estimates than for 2PL). A par-
ticular consequence of this is that the set of estimates for
response vectors that have the same sumscore will not
necessarily be contiguous on the h scale.
Some special characteristics of Rasch Scaling: Due to (a)
its simplicity, and (b) its unique properties, such as ‘‘spe-
cific objectivity’’ [19] which confers particular strengths
on item sets that are found to be amenable to Rasch mod-
eling (i.e., item sets that ‘‘fit’’ the model), the Rasch model
can be used to bring the item and the person parameters
onto the same metric, allowing a wide range of possibilities
for the development of analogical and figurative aids to
interpretation (e.g., see [20]). With these advantages, Rasch
Scaling represents an interesting, and potentially powerful,
compromise between the strictness of the Guttman Scale
adherence to the pre-eminence of the substantive theory
(via the item ordering implied by the substantive interpre-
tation), and CTT’s flexibility in accepting all response vec-
tors. Thus, Rasch Scaling has been seen as a way to
reconcile the perspectives of CTT and Guttman Scaling
[22].
6. Construct Modeling
Construct Modeling [23] builds upon the reconciliation
of CTT and Guttman Scaling, as represented by Rasch Scal-
ing. It takes as its technical side the ground-work of Rasch
Scaling, and moves one step further along the path towards
adhering to the substantive theory. In this case, it is as-
sumed that the substantive theory takes a particularly sim-
ple form: the construct consists of a simple linear
succession of discrete segments of a continuum, from a
lowest level to a highest level, and when these are laid
out in a figure, it is termed a ‘‘construct map’’—see Fig. 6.
A concrete example of this is shown in Fig. 7, which was
developed in the context of a test of students’ knowledge
about the practice of designing and critiquing displays of
data for students in the middle school, with the lowest le-
vel at the bottom of the figure. In this case, the Rasch Scal-
ing described above serves as a starting place for
establishing the standard set. All of the steps described
above for Rasch Scaling are followed. Simultaneously with
that, a second set of steps is followed that takes into ac-
count the construct map, and additional substantive infor-
mation concerning each item, to wit, a substantive link
from each item to a (single) level of the construct map.
When deciding on the item set, an additional criterion
that is used is this link from each item in the set to the con-
struct map—for each item it needs to be judged whether
the estimated item location is well-matched to the hypoth-
esized order in the construct map. An example of an item
used to assess the Data Display construct is shown in
Fig. 8 and 9 shows sample student responses that have
been matched to certain levels of the DaD construct map.
Of course, accumulated empirical information that it does
3771
not match could lead to a revision of the construct map
and/or the hypothesized link, as well as modification/dele-
tion of the item.
Once an item set is established, ‘‘banding’’ or ‘‘standard
setting’’ takes place—this is the equivalent of Mari’s cali-
bration: the placement of the values into segments of the
logit scale (the ‘‘Wright Map’’). The upper and lower limits
of these bands are determined by the locations of the lower
and upper limits of the locations of the items linked to each
level of the construct map. Most often, these item locations
do not result immediately in a ‘‘clean’’ segmentation of the
logit scale—hence a judgmental process [24] is required to
determine reasonable locations for the band edges4. This
process may result in further decisions regarding the suit-
ability of certain items, and may also involve information
about the persons, when that is available. Denote these lim-
its by H = (H1, H2, . . . , HK), the K boundary values between
the K + 1 segments. Then we see that:
Band 1 is ð1; H1 ;
Band 2 is ðH1 ; H2 ;
:
Band k is ðHk1 ; Hk ;
:
Band K þ 1 is ðHK ; þ1Þ:
Or, equivalently, the standard reference set is C, composed
of the set of response vectors ck, k = 1, . . . , K, and
ck ¼ fbi : Hk1 < hðuÞ < Hk Þ; uebi Þ;
where the bi are the same as those constructed for Rasch
Scaling. These bands can be used as a basis for criterion-
referenced interpretations of the measurements in that
each band may be associated with the items whose diffi-
culties fall within its range5. This provides ways to enhance
and deepen the interpretations available to those who apply
the measurements. At the same time, the existence of the
underlying Rasch scale means that (a) technical aspects of
the measures are available, such as standard errors, and
(b) technical advantages of item response scales are still
available, such as flexibility in item choice, ability to link
forms through items, and the possibility of computerized
adaptive item administration. This is one reason to choose
the Construct Modeling approach compared to the alterna-
tive of latent class modeling [26], which might be seen as
potentially appropriate, given what is shown in Fig. 5.
7. Discussion and conclusion
Mari [1, p. 80] characterizes measurement as needing to
attain both objectivity and intersubjectivity. In his words:
4
The label for this is different depending on when the set B is developed:
if it is developed before the scaling (although it may be adapted after), then
the process is termed ‘‘construct modeling’’ [23]; if it is developed after the
scaling, then the resulting scale is termed a ‘‘described variable’’ [25].
5
Depending on the interpretive context, the specification of the
difficulties of the items may vary. For example, whereas the standard is
to set it at the point where the probability of endorsement is .5, for some
contexts, it may be more readily interpretable if the probability is set at,
say, .8, for ‘‘mastery’’ in achievement testing.
3772 M. Wilson / Measurement 46 (2013) 3766–3774

Fig. 6. A graphical representation of a construct map with K levels.

Fig. 7. The levels of the Data Display construct map.

Fig. 8. An example item related to the DaD construct: ‘‘Shimmering Candle’’.

 M. Wilson / Measurement 46 (2013) 3766–3774
Fig. 9. Examples of responses to the DaD construct levels for the Shimmering Candles item.
objectivity implies that the MS is able to discriminate
the measurand from the various influence quantities
so that the acquisition component of the MS is sensitive
only to the measurand;
intersubjectivity implies that the MS is able to refer the
measurand to the primary standard, so that all the mea-
surements expressed in terms of that standard are com-
parable with each other.
From the point of view adopted here, we need to specify
how these properties would appear in the context of Con-
struct Modeling.
First, consider objectivity. If we think of ‘‘the various
influence quantities’’ as being embodied by the possibility
of using different sets of items as the instrument, then this
amounts to independence from the specific set of items
used. This is indeed what Rasch’s ‘‘specific objectivity’’ is
concerned with (i.e., if the set of items fits the Rasch model,
then it does not matter which items are used to measure
the person). Hence, when using a Rasch scale as the basis
for Construct Mapping, (and where the items do indeed
fit a Rasch model) it would remain to check whether the
banding was relatively robust to choices of the items as
representatives.
Second, consider intersubjectivity. Unlike the case for
CTT, Construct Modeling inherits the Rasch Scaling (and
Guttman Scaling) characteristic of disallowing some re-
sponse vectors. However, these are not represented in
the standard reference set, hence this is not a formal prob-
3773
lem. In practical terms, this amounts to a situation where
the measurement system does not give some people mea-
sures. The best procedure in this case is to seek a re-admin-
istration of the measurement process for that individual,
with perhaps the possibility of gathering extra information
to check on the conditions under which these misfit re-
sponse vectors tend to be found.
In the text above, it was noted that there are many item
response models available beyond the Rasch model. Where
these are being used, some, but not all, of the development
above in the Construct Modeling section can be developed.
In particular, banding is not readily possible, as the concept
of the ‘‘location’’ of an item on the logit scale does not have
a straightforward interpretation. Also, the specific objectiv-
ity possible under the Rasch model is not attainable for
other models [18]. Thus, for other item response model ap-
proaches, the development here seems difficult.
The generic type of construct (measurand) that is used
to motivate the development of Construct Modeling (i.e.,
a simple linear succession of discrete segments of a contin-
uum) may seem quite restrictive on first glance. However,
most published measures in the social sciences are in fact
of just this type, or simpler (i.e., they have no segments,
just a continuum). That said, where there are more com-
plex constructs under consideration, many of them repre-
sent quite simple extensions of the generic construct
discussed above. For example, where there are multiple
linear continua (i.e., a ‘‘multi-dimensional’’ construct), then
the scaling can be accomplished using multi-dimensional
3774 M. Wilson / Measurement 46 (2013) 3766–3774
versions of the Rasch model [27,28], and each dimension
can be treated then as a separate case for banding. Where
there are polytomous items and/or multiple substantive
categories within a particular polytomous score [29], the
banding procedure can be generalized to deal with the sit-
uation [24]. Where the latent class is posited to be an or-
dered latent class rather than a latent continuum, the
methods described above can be applied, with the proviso
that one should check for the most appropriate model
using fit procedures [30]. Where a more complex construct
is under consideration, such as a ‘‘learning progression’’
[31] (which posits level-based links among different
dimensions), there are also methods analogous to those
described above, although these are still under develop-
ment [32]. Of course, there are more complex constructs
yet, but the list above contains a very large proportion of
the extant types.
This paper has built on an earlier conference paper [33]
to explore how to use the functional concept of a Measur-
ing System6 to explicate the logic of several measurement
approaches used in psychometrics, and thus enable a com-
parison with measurement approaches used by other fields
such as engineering and physics. It surveyed Guttman Scal-
ing, Classical Test Theory, Rasch Scaling and Construct Mod-
eling, as examples of measurement approaches in the area of
psychometrics, and explicated the underlying standard ref-
erence set that is one of the essential features of Mari’s for-
malization [1], and showed how these differ among the four
approaches. The importance of these differences, and the
consequences for measurement using those approaches,
hinge on the capacity to identify theoretically tractable sub-
stantive properties capable of supporting both objectivity
and intersubjectivity. Connecting psychometric approaches
to measurement with Mari’s formalization of the functional
concept of a measuring system opens up new opportunities
for productive dialogue between the natural and social
sciences.
Acknowledgements
I would like to thank Luca Mari and Roman Morowski
for organizing the symposium in which some of these ideas
were originally presented, and for their encouragement of
this work.
References
[1] L. Mari, Beyond the representational viewpoint: a new formalization
of measurement, Measurement 27 (2000) 71–84.
[2] D.H. Krantz, R.D. Luce, P. Suppes, A. Tversky, Foundations of
Measurement. Volume 1: additive and Polynomial Representations,
Academic Press, New York, 1971.
[3] A. Thompson, N. Taylor, Hamlet, Arden, London, 2006.
[4] R. Perline, B. Wright, H. Wainer, The Rasch model as additive
conjoint measurement, Applied Psychological Measurement 3
(1979) 237–256.
[5] G. Karabatsos, The Rasch model, additive conjoint measurement, and
new models of probabilistic measurement theory, Journal of Applied
Measurement 2 (4) (2001) 389–423.
6
This paper has not sought to expand the viewpoint adopted here to
include the new insights from Mari’s later work [34]. This remains as an
important step to be carried out, but is beyond the scope of the current
paper.
[6] J. Michell, An introduction to the logic of psychological
measurement, Lawrence Erlbaum Associates, Mahwah, New Jersey,
1990.
[7] D. Borsboom, G. Mellenbergh, H. Van, The theoretical status of latent
variables, Psychological Review 110 (2) (2003) 203–219.
[8] R. Lissitz, The concept of validity: revisions, new directions, and
applications, Information Age Publishing, Charlotte, North Carolina,
2009.
[9] S.S. Stevens, On the theory of scales of measurement, Science 103
(1946) 677–680.
[10] K. Sijtsma, Introduction to the measurement of psychological
attributes, Measurement 44 (2011) 1209–1219.
[11] L.A. Guttman, The basis for scalogram analysis, in: S.A. Stouffer, L.A.
Guttman, F.A. Suchman, P.F. Lazarsfeld, S.A. Star, J.A. Clausen (Eds.),
Studies in Social Psychology in World War Two. Volume. 4:
Measurement and Prediction, Princeton University Press,
Princeton, 1950, pp. 60–90.
[12] E. Kofsky, A scalogram study of classificatory development, Child
Development 37 (1966) 191–204.
[13] F. Edgeworth, The statistics of examinations, Journal of the Royal
Statistical Society 51 (1888) 599–635.
[14] F. Edgeworth, Correlated Averages, Philosophical Magazine, 5th
series, 34 (1892) 190–204.
[15] C. Spearman, The proof and measurement of association between
two things, American Journal of Psychology 15 (1904) 72–101.
[16] C. Spearman, Demonstration of formulae for true measurement of
correlation, American Journal of Psychology 18 (1907) 161–169.
[17] J. Nunnally, I. Bernstein, Psychometric Theory, 3rd ed., McGraw-Hill,
Columbus, Ohio, 1994.
[18] W. van der Linden, R. Hambleton, Handbook of Item Response
Theory, Springer, New York, 1997.
[19] G. Rasch, 1960/1980, Probabilistic Models for Some Intelligence and
Attainment Tests, University of Chicago Press, Chicago, 1980.
[20] B. Wright, M. Stone, Best Test Design, MESA Press, Chicago, 1979.
[21] R. Meijer, K. Sijtsma, Methodology review: evaluating person fit,
Applied Psychological Measurement 25 (2001) 107–135.
[22] B. Wright, G. Masters, Rating Scale Analysis, MESA Press, Chicago,
1981.
[23] M. Wilson, Constructing Measures: An Item Response Modeling
Approach, Erlbaum, Mahwah, New Jersey, 2005.
[24] M. Wilson, K. Draney, A Technique for Setting Standards and
Maintaining Them over Time, in: S. Nishisato, Y. Baba, H.
Bozdogan, K. Kanefugi (Eds.), Measurement and Multivariate
Analysis, Springer-Verlag, Tokyo, 2002, pp. 325–332.
[25] Organisation for Economic Co-operation and Development, PISA
2000 Technical Report, OECD, Paris, 2002.
[26] M. Croon, Latent class analysis with ordered latent classes, British
Journal of Mathematical and Statistical Psychology 43 (2) (1990)
171–192.
[27] R. Adams, M. Wilson, W. Wang, The multidimensional random
coefficients multinomial logit model, Applied Psychological
Measurement 21 (1) (1997) 1–23.
[28] M. Wu, R. Adams, M. Wilson, S. Haldane. ACERConQuest 2.0
[computer program], ACER, Hawthorn, Australia, 2008.
[29] M. Wilson, R. Adams, Marginal maximum likelihood estimation for
the ordered partition model, Journal of Educational Statistics 18 (1)
(1993) 69–90.
[30] D. Torres Irribarra, R. Diakow, Model selection for tenable
assessment: selecting a latent variable model by testing the
assumed latent structure, in: Paper Presented at the 17th
International Meeting of the Psychometric Society, Hong Kong SAR,
2011.
[31] A. Alonzo, A. Gotwals (Eds.), Learning Progressions in Science, Sense
Publishers, Rotterdam, The Netherlands, 2012.
[32] M. Wilson, Responding to a challenge that learning progressions
pose to measurement practice: hypothesized links between
dimensions of the outcome progression, in: A.C. Alonzo, A.W.
Gotwals (Eds.), Learning Progressions in Science, Sense Publishers,
Rotterdam, The Netherlands, 2012.
[33] M. Wilson. The role of mathematical models in measurement: a
perspective from psychometrics. Paper Presented at the Joint
International IMEKO TC1+TC7+TC13 Symposium ‘‘Intelligent
Quality Measurement – Theory, Education and Training’’ August
31–September 02, Jena, Germany, 2011.
[34] A. Frigerio, A. Giordani, L. Mari, Outline of a general model of
measurement, Synthese 175 (2010) 123–149.