1 CONSEQUENCES OF TEST

INTERPRETATION AND USE: THE

FUSION OF VALIDITY AND VALUES IN
PSYCHOLOGICAL ASSESSMENT 1

Samuel Messick

This paper addresses the role of social values in educational and psychological
measurement, with special attention to the consequences of testing as validity
evidence, which is an inherently value-dependent enterprise. The prinuuy
measurement standards that must be met to legitimize a proposed test use are those
of reliability, validity, and fairness, which are also value-laden concepts. Evidence
of reliability signifies that something is being measured~ the major concern is score
consistency or stability. Evidence of validity circumscribes the nature of that
something~ the rruyor concern is score meaning. Evidence of fairness indicates that
score meaning does not differ consequentially across individuals, groups, or
settings~ the major concern is comparability.

The appropriate level of reliability depends on the meaning of the construct

being measured because some constructs are conceived theoretically to be more
consistent or stable than others. Hence, evidence of reliability consistent with the
construct's meaning is simultaneously also evidence of construct validity. Within
these limits, the measurement intent is to achieve sufficient score consistency or
stability to warrant the use of the scores in decision making. Another way of
putting it is that the uncertainty involved in determining score levels should be
inconsequential for the proposed test use.

Validity is mainly concerned with the meaning and consequences of

measurement. Accordingly, validation studies aim to accrue convergent evidence
supportive of score meaning and its action implications as well as discriminant
evidence discounting plausible rival interpretations. Validity is a unitary concept,

R. D. Goffin et al. (eds.), Problems and Solutions in Human Assessment

© Springer Science+Business Media New York 2000
316 Problems and Solutions in Human Assessment

& Chartrand, 1999; Gollwitzer, 1999; Locke, 1968; Locke & Latham, 1990;
Locke, Latham, & Erez, 1988). Goal commitment is related to performance in two
ways.First, empirical studies, including a meta-analysis, have shown that goal
commitment moderates the relationship between goal difficulty and performance
(Erez & Zidon, 1984; Hollenbeck, Williams & Klein, 1989a; Klein, Wesson,
Hollenbeck, & Alge, 1999a; Tubbs, 1993). The correlation between goal difficulty
and performance is higher among individuals with high goal commitment than
among those with low goal commitment.

Second, the level of goal commitment can have a main effect on performance
(Hollenbeck & Klein, 1987; Klein et aI., 1999a; Locke & Latham, 1990).
Individuals who are highly committed to a difficult goal perform at significantly
higher levels than individuals who are not committed to the same goal
(Hollenbeck, Klein, O'Leary-Kelly, & Wright, 1989b; Klein et al., 1999a; Locke &
Shaw, 1984; Seijts & Latham, 1999a). This is because highly committed
individuals exert more effort, and are more persistent toward goal attainment, than
individuals who are less committed to the goal. However, when an assigned goal
is easy or moderately difficult, high commitment may lead to poorer performance
than low commitment to the same goal (Locke & Latham, 1990). Individuals with
low commitment to the assigned goal may set a higher personal goal that, in tum,
may lead to higher performance.

Given the critical role of goal commitment in goal setting theory, one would
expect that the assessment of this construct has played a prominent role in goal
setting research. However, in a review of the goal setting literature through the
mid 1980s, Hollenbeck and Klein (1987) found that in the majority (61 percent) of
goal setting studies they reviewed (66 of 109), no mention was made of goal
commitment. In 12 percent of the studies, goal commitment was mentioned but
not empirically assessed. In none of the studies did the authors attempt to establish
the construct validity of their measure of goal commitment. In addition,
Hollenbeck and Klein (1987) found that many goal setting researchers used a
single-item measure (e.g., "How committed are you to attaining the goal set?").
Hence the reliability of the measure of goal commitment was not known.

Hollenbeck and Klein's (1987) findings were troubling for at least two reasons.
First, they found that the critical role of goal commitment to goal setting theory
had been largely ignored. Second, they reported that unmeasured goal
commitment effects were often used as post hoc explanations for non-significant
goal setting effects. That is, even when goal commitment was not assessed, the
varying effect-sizes for goal setting, as well as inconsistent results with moderating
variables (e.g., participation, feedback, and rewards) were often attributed to
unmeasured goal commitment effects. Hollenbeck and Klein concluded that, given
the central role of goal commitment in goal setting theory, subsequent research
must assess goal commitment.