Measures and Interpretations of Vigilance Performance:

Evidence against the Detection Criterion

J. D. Balakrishnan, Purdue University, West Lafayette, Indiana

Operators’ performance in a vigilance task is often assumed to depend on their

choice of a detection criterion. When the signal rate is low this criterion is set
high, causing the hit and false alarm rates to be low. With increasing time on
task the criterion presumably tends to increase even further, thereby further
decreasing the hit and false alarm rates. Virtually all of the empirical evidence
for this simple interpretation is based on estimates of the bias measure β from
signal detection theory. In this article, I describe a new approach to studying
decision making that does not require the technical assumptions of signal detec-
tion theory. The results of this new analysis suggest that the detection criterion
is never biased toward either response, even when the signal rate is low and
the time on task is long. Two modifications of the signal detection theory
framework are considered to account for this seemingly paradoxical result. The
first assumes that the signal rate affects the relative sizes of the variances of the
information distributions; the second assumes that the signal rate affects the
logic of the operator’s stopping rule. Actual or potential applications of this
research include the improved training and performance assessment of operators
in areas such as product quality control, air traffic control, and medical and clin-
ical diagnosis.

Editor’s note: The following article was the INTRODUCTION

subject of considerable – yet constructive and
informative – debate throughout the review In many operational and applied settings,
process. The reason is that it challenges cer- fast and accurate decisions about the status of
tain widely accepted assumptions associated a system (e.g., the safety conditions of an air-
with estimates derived from the theory of sig- craft, a surgery patient’s vital signs) must be
nal detectability (TSD) and proposes an alter- made by human operators on the basis of data
native approach. Given that TSD is one of the presented on an information display of some
more robust human performance models we kind (e.g., a video terminal). The conditions
have, and has been widely used in many task requiring action are often relatively rare and
contexts including vigilance, it is not surpris- unpredictable, making it necessary for the
ing that such a challenge would generate con- operator to maintain a high level of alertness
troversy. Therefore, I felt that the article over an extended period. Any breakdown in
should be published and widely read so that this vigilance or sustained attention to the dis-
you, the readers, could form your own opin- play could cause the operator to overlook or
ion. We must always remember that science is respond too slowly to a mission-critical event.
an endless search for better explanations, not A classic example of the failure of sustained
final answers. We must continually challenge attention in an applied setting was the tenden-
convention if we are to move forward. cy of airborne British radar observers during
Requests for reprints should be sent to J. D. Balakrishnan, Department of Psychological Sciences, Purdue University, West
Lafayette, IN 47907; jdb@psych.purdue.edu. HUMAN FACTORS, Vol. 40, No. 4, December 1998, pp. 601–623.
Copyright © 1998, Human Factors and Ergonomics Society. All rights reserved.

Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016

602
World War II to begin to miss the radar signals
indicating the presence of enemy submarines in
the area after about 30 min on watch (N. H.
Mackworth, 1948, 1961). This so-called vigi-
lance decrement has since been documented in
a variety of real-world and laboratory settings
(e.g., Davies & Parasuraman, 1982; See, Howe,
Warm, & Dember, 1995; Warm & Jerison,
1984).
Typically, when a vigilance decrement is
observed, it is accompanied by a decline in the
number of incorrect responses to nonsignal
events, or false alarms. Both effects are consis-
tent with the notion that the operator’s atten-
tion level is somehow impaired by the passage
of time, but both results could also be attrib-
uted to a reduction in the operator’s willingness
to report a signal event; this could occur for a
number of reasons. One of the main objectives
of empirical vigilance research, therefore, has
been to establish a general set of guidelines, or
a “vigilance taxonomy,” that can be used to pre-
dict when and how the vigilance decrement will
be manifested (e.g., Parasuraman, 1979;
Parasuraman & Davies, 1977; Parasuraman &
Mouloua, 1987; Parasuraman, Warm, &
Dember, 1987; See et al., 1995; Warm, 1984).
For many vigilance researchers, the preferred
method of separating the effects of attentional
capacity on performance from the effects of an
operator’s attitudes or performance strategy is
to compute the sensitivity (d′) and response
bias (β) statistics of signal detection theory
(e.g., Davies & Parasuraman, 1982; Green &
Swets, 1974; Proctor & Van Zandt, 1994).
According to this general framework for dis-
crimination behavior, the series of mental
events that takes place when an operator mon-
itors an information display over an extended
period can be broken up into pairs of tempo-
rally distinct operations: encoding and deci-
sion making.
An overt response to a signal occurs when
the encoded information, combined in some
manner with the operator’s objectives and
knowledge of the task, exceeds a detection cri-
terion. This criterion is under the control of
the operator, and its value ultimately defines
the efficiency (or optimality) and the bias of
the operator’s decision-making strategy. Some
of the most widely accepted views of the gen-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
eral properties and limitations of human vigi-
lance, including the effect of the signal rate on
the operator’s detection rate and the change in
detection rate over time, depend heavily on
the notion of a detection criterion.
In its most general form (i.e., as a nonpara-
metric description of the two stages involved
in discrimination), the signal detection theory
model is nonfalsifiable, but it is also unin-
formative about decision-making aspects of
performance. In order to measure the contri-
bution of the decision-making process, some
additional, technical assumptions about the
two processes (e.g., normality of the encoding
distributions and a single detection criterion)
are required. These assumptions can be – and
have been – challenged for a number of rea-
sons (e.g., Green & Swets, 1974; Lockhart &
Murdock, 1970), and a number of alternatives
to d′ and β have been proposed and studied
(e.g., Macmillan & Creelman, 1990; See,
Warm, Dember, & Howe, 1997; Swets, 1986).
Few theorists, however, have found any seri-
ous reason to question the general concept of
the detection criterion.
In this article, I review and apply a new set
of “distribution-free” performance measures
that I have recently developed to study decision-
making processes in vigilance and other kinds
of two-choice classification tasks (Balakrish-
nan, 1998; Balakrishnan & Ratcliff, 1996).
These measures are based on the same funda-
mental principles of signal detection theory
(i.e., statistical decision making) and, hence,
the same concept of response bias (i.e., a
description of the mapping between informa-
tion states and responses). However, instead of
making assumptions about the shapes of the
encoding distributions, the new measures take
advantage of the extra information about the
decision-making process provided by subjec-
tive confidence judgments.
The results of these new analyses suggest
that operators do not change the value of a
detection criterion in response to changes in
the signal rate in a vigilance setting. The fact
that the signal rate does have a substantial
impact on the operator’s detection rate must
therefore be attributed to an entirely different
type of response bias. On this basis, I ar-
gue that the significance of some of the most
DECISION PROCESSES IN VIGILANCE
well-established findings in the vigilance litera-
ture – including the supposed conservatism of
the criterion placement and the supposed cri-
terion increment with time on task – need to
be reexamined.
In the first part of the paper, I review the
basic concepts and measures of signal detec-
tion theory and their application to vigilance. I
then describe the new distribution-free mea-
sures and use these to show that the detection
criterion account of vigilance makes a testable
prediction about subjective confidence judg-
ments that is consistently contradicted by the
data. This result reopens the question of why
the operator’s detection rate is directly tied to
the signal rate in vigilance. Accordingly, in the
second part of the paper, I describe two potential
modifications of the classical signal detection
theory framework that would be consistent
with both the effect of signal rate on detection
rate and the observed properties of subjective
confidence. In the final section, I consider some
of the implications of rejecting the detection
criterion construct for researchers whose main
purpose is to measure and compare the per-
formance levels of human operators under dif-
ferent experimental conditions.
BACKGROUND
In order to mimic some of the more com-
mon features of real-world vigilance settings,
experimental studies of vigilance typically
require the participant to monitor a series of
distinct stimulus events presented at either
fixed or irregular intervals. Most of these
events are no-signal events, which require no
overt response by the participant. Occasion-
ally, however, a signal stimulus occurs, and
the participant is asked to report this event.
Under most conditions, participants occasion-
ally respond incorrectly to a no-signal event
(false alarms) or fail to respond to a signal
event (misses). The rates of these two kinds of
errors usually provide the primary basis for
measuring and interpreting performance.
One of the more important dimensions of
the original taxonomy developed by Parasur-
aman and Davies (1977) to organize the vigi-
lance literature is the type of discrimination
task carried out over the course of the vigi-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
603
lance period. In the successive form, one of
two stimulus objects is presented, and the par-
ticipant must refer to his or her memory of the
objects in order to identify whether this object
represents a signal or a no-signal event. In the
simultaneous form, each event is defined by
the presentation of two stimulus objects. These
two objects are identical in the no-signal con-
dition and different in some way (e.g., in size)
in the signal condition. Given that the two ob-
jects are visible simultaneously, the task does
not require any long-term memory for the
physical properties of the two stimuli.
The second important dimension identified
by Parasuraman and Davies (1977) is the aver-
age time between stimulus presentations (i.e.,
the event rate). By combining the two dimen-
sions (type of task and event rate) Parasur-
aman and Davies found reasonably consistent
evidence for decreases in perceptual sensitivity
(i.e., a sensitivity decrement) in successive dis-
crimination tasks when the event rate was
high and for criterion effects under other con-
ditions. More recent work (e.g., Deaton &
Parasuraman, 1993; Parasuraman et al., 1987;
See et al., 1995; Warm, Dember, Murphy, &
Dittmar, 1992) has identified other variables
(e.g., signal discriminability and cognitive vs.
sensory tasks) that can influence the type of
decrement observed. A detailed review and
meta-analysis of previous findings is given by
See et al. (1995).
Statistical Decision Making and Signal
Detection Theory
When the signal detection theory measures
are used to analyze and interpret vigilance
data, the main assumption involved is that vig-
ilance can be formally represented as a series
of simple, two-step hypothesis tests. In Step 1,
the operator collects some information from
the display, and in Step 2, a strategy or rule is
used to assign the resulting internal informa-
tion state to a judgment (e.g., a detection
response). Figure 1 illustrates the additional
assumptions of the model associated with the
d′ and β statistics.
In the two panels of the figure, the stimulus
conditions – signal and no signal – represent
identical perceptual sensitivity (equal spacing
of the distributions). In the upper panel the
604 December 1998 – Human Factors

Figure 1. The effects of signal rate in the normal, equal variance model of signal detection theory. In the
lower panel, the decision rule is biased toward the no-signal response but not enough to maximize the per-
centage of correct responses.

response criterion (XC) is unbiased, and in the
lower panel it is biased toward the no-signal
response. The bias measure β is defined as the
ratio of the equal variance normal distribu-
tions (signal to no signal) at the point XC (e.g.,
Warm & Jerison, 1984). The interval labeled v
in the lower panel of the figure represents a
set of “biased mental states” – that is, in this
region the states are mapped to a no-signal
response, and the height of the no-signal dis-
tribution, fN(t), is less than that of the signal
distribution, fS(t).
In order to appreciate the significance of
the results that are described later, several
implicit properties of this two-stage model
must be clearly understood. First, note that bias
and optimality are completely independent
constructs in the model: Depending on the
prior probabilities (or rates) of the no-signal
and signal events, the decision rules represent-
ed in Figure 1 may or may not be optimal with
respect to accuracy (i.e., they may or may not
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
maximize the overall proportion of correct
discrimination responses). Assuming that the
signal occurs on one of four trials on average,
the region labeled w in the lower panel of
Figure 1 is a set of mental states for which the
decision rule is suboptimal.
It is also important to recognize that the dis-
tributions in Figure 1 are assumed to represent
the true or objective likelihoods (probability
densities) of the states induced by the stimuli.
Thus a decision rule is or is not biased depend-
ing on the mapping between mental states and
their true likelihoods under the two stimulus
conditions. These likelihood values presumably
depend on both external, physical properties of
the stimuli and internal properties of the percep-
tual system (i.e., on the noise level). Whether or
not the decision maker consciously attempts to
use a biased or suboptimal decision rule is an
entirely different issue (i.e., an issue of the cor-
rect psychological interpretation of the observed
decision rule; Balakrishnan, 1997).
DECISION PROCESSES IN VIGILANCE
Finally, given that the effect of a change in
decision-making variables (e.g., the signal
rate) is represented in the model as a shift in
the location of the detection criterion, it is im-
portant to notice that the mental states nearest
to the criterion are the first to be remapped to
a different response when these decision vari-
ables are manipulated. That is, a small change
in the signal rate or payoff structure does not
lead to an arbitrary remapping of states to
responses but, rather, to a remapping of the
states near the criterion. An important implicit
assumption of this model, therefore, is that
the decision process considers the states near
the criterion to be associated with weak evi-
dence or high uncertainty. More generally, the
decision maker’s perception of the strength of
the evidence (i.e., subjective confidence) is a
monotonic, increasing function of the distance
of the perceptual state from the detection cri-
terion (Balakrishnan & Ratcliff, 1996). If this
were not so, then the parameters of the model
could still be estimated, but the theory could
not explain why the degree of the criterion
shift is a direct function of the degree of the
bias manipulation.
The Ratings Paradigm
In a popular variation of the classical yes-no
detection paradigm, the all-or-none detection
response is replaced by a graded response (i.e.,
a confidence rating on a single, bipolar scale).
High confidence in a no-signal response is indi-
cated by lower values on the scale, and high
confidence in a signal response is indicated by
higher values on the scale. Given that the par-
ticipant’s confidence state should be a monoto-
nic function of the encoding effect of the
stimulus, asking participants to rate their con-
fidence is equivalent to asking them to set
additional criteria on the continuum of encod-
ing effects (e.g., Dorfman & Alf, 1969).
The ratings paradigm is often preferred to
the yes-no detection task because it allows the
researcher to test hypotheses about the sensory
distributions using data from a single condition
(i.e., without varying the stimulus presentation
rates or the values of a payoff matrix). In order
to take advantage of the additional information
about the decision process provided by confi-
dence ratings, the standard vigilance task must
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
605
be modified so that the participants choose
both a discrimination response (signal or no-
signal) and a confidence rating after each stim-
ulus event. The most efficient way to obtain
both types of data is to add an explicit cutoff
somewhere (e.g., in the middle) of the bipolar
rating scale. That is, somewhere on the scale
are two adjacent rating responses that repre-
sent the least confident no-signal and signal
responses. The highest and lowest values on
the scale represent the highest confidence sig-
nal and no-signal responses, respectively.
Alternatively, the participant may be asked
to first give a signal or no-signal response and
then a confidence rating. The same representa-
tion as the bipolar rating scale is easily obtained
from these data by multiplying the no-signal
response confidence ratings by –1. This devia-
tion from the more common vigilance design
obviously could have some effect on a partici-
pant’s performance. However, because the
detection problem remains the same, there is
no basis within signal detection theory for pre-
dicting that soliciting confidence judgments
should fundamentally alter the nature of the
detection process. In support of this, the data
reported later replicate the two major results
previously attributed to the placement of a
detection criterion: (a) The hit rate is substan-
tially lower than the correct rejection rate when
the signal rate is low, and (b) the hit and false
alarm rates both decrease with time on task.
DISTRIBUTION-FREE TESTS
OF OPTIMALITY AND BIAS
Suboptimal Decision Rules
If the participant’s objective is to maximize
the percentage of correct responses, then the
optimal decision rule is to choose the response
that is most likely to be correct on each trial.
In other words, the participant should choose
the response associated with the maximum of
the two objective posterior probabilities, P[no
signal | Ψ] and P[signal | Ψ], where Ψ is a
random variable representing the information
state. (Note that P[signal | Ψ] = 1 – P[no
signal | Ψ]).
A simple and direct consequence of this fact
leads immediately to a test for optimality that
does not depend on any assumptions about the
606
distributions of the stimulus information (Ψ)
under the two event conditions. Specifically, no
matter what the transformation is from stimulus
information (Ψ) to a confidence state E, the
probability that the response associated with this
state is correct must be greater than or equal to
the probability that the response is incorrect (or,
equivalently, greater than 0.5). If this were not
true, then the decision rule would not be opti-
mal, because the decision maker could increase
the percentage of correct responses by reversing
the discrimination response associated with
these suboptimal confidence states.
To apply this assumption-free test to empir-
ical data, the experimenter calculates the pro-
portion of correct responses, P[no signal | R =
kN], for each rating response R = kN associated
with a no-signal response, and the proportion
of correct responses, P[signal | R = kS], for
each rating response associated with a signal
response. If any of these values is less than 0.5
(allowing for estimation error), then it follows
immediately that participants could improve
their performance merely by changing their
decision rule (i.e., by making the alternative
discrimination response for each suboptimal
rating response).
According to classical signal detection theo-
ry analyses of human discrimination perfor-
mance, participants tend to be conservative in
their response to objective bias manipula-
tions. For example, when the signal rate is de-
creased, the estimated value of the detection
criterion, XC, increases, but not enough to
maximize the percentage of correct responses
(Creelman & Donaldson, 1968; Green &
Swets, 1974; Macmillan & Creelman, 1991;
Maloney & Thomas, 1991). In other words,
empirical estimates of the parameters of this
model indicate that the participant’s decision
rule is in fact biased when the signal rate is
low, but it is not biased enough to be optimal
(e.g., as in the lower panel of Figure 1). This
means that there should exist a set of mental
states for which the optimality test described
earlier fails. Data confirming this prediction
are presented later.
A Distribution-Free Index of Response Bias
In addition to assuming a specific distribu-
tion model, the bias statistic β of signal detec-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
tion theory also assumes that there is only
one, if any, contiguous region of biased mental
states. In principle, several different biased
regions could exist, and in fact the same de-
cision rule could be biased simultaneously
toward both responses (i.e., there could be one
region of bias toward the no-signal response
and another toward the signal response). A
simple way to avoid both the distributional
assumption and the assumption about the
number of biased regions in signal detection
theory is to redefine the bias index as the
probability (relative frequency) of a biased
response (or, in other words, the probability
that the perceptual effect of the stimulus will
fall into a biased response region),
pbias = pN ∫v fN(s)ds + pS ∫v fS(s)ds,
where pS and pN are the signal and no-signal
rates, respectively, and v denotes all regions of
confidence values for which the decision rule
is biased. This distribution-free index is per-
fectly compatible with β of signal detection
theory, given that for each pair of values β and
d′ there exists a unique value of pbias (see
Figure 1). Using the symbol βp to refer to this
particular, distribution-dependent value, the
distribution-free construct pbias and the signal
detection theory statistic βp will be equal
when the distributional assumptions of signal
detection theory are in fact satisfied.
To estimate pbias without relying on a distri-
butional assumption, the method proposed by
Balakrishnan (1998) is to examine the empiri-
cally observable function,
UR(k) = FR,N(k) – FR,S(k), (1)
where FR,N(k) and FR,S(k) are the cumulative
relative frequency histograms of confidence
rating k on no-signal (N) and signal (S) trials,
respectively. When this function is decreasing
for any value of k associated with a no-signal
response or increasing for any value of k asso-
ciated with a signal response, then the decision
rule is biased. Further, the total proportion of
these “biased rating responses” in the data
(henceforth, Ωp) provides a distribution-free
estimate of the probability of a biased re-
sponse, pbias.
DECISION PROCESSES IN VIGILANCE 607

The basic idea behind this new, distribution-
free test is very simple. As is well-known (e.g.,
Green & Swets, 1974), the slope of the receiv-
er operating characteristic (ROC) curve at any
information state t is equal to the likelihood
ratio of the two underlying distributions,
fS(t)/f N(t), where f denotes the density func-
tion of the encoding effect of the stimulus.
Note that this result refers to a plot of the hit
rate against the false alarm rate in probability
coordinates. A plot of these values using
normal-normal coordinates, or z-score trans-
forms, produces a linear function when the
underlying distributions are normal. It is easy
to show (see Balakrishnan, 1998) that the
function UR(k) is increasing when the slope of
the ROC curve is less than one and decreasing
when this value is greater than one.
Because of the relationship in signal detec-
tion theory between the slope of the ROC
curve and the likelihood ratio at the detection
criterion (i.e., β), this model makes a testable
prediction about the location of the peak of
the function UR(k). Specifically, if the detec-
tion criterion is shifted in favor of the no-signal
response, then the peak of the UR(k) function
should be shifted toward the no-signal side of
the bipolar rating scale. If the bias is toward
the signal response, then the peak should
occur on the signal response side of this scale.
This fundamental prediction of signal
detection theory is illustrated in Figure 2,

Figure 2. An illustration of the relationship between properties of the confidence ratings distributions and
the location of the detection criterion in signal detection theory. Changes in the signal rate change the loca-
tion of the detection criterion and hence the objective likelihood ratios corresponding to the lowest confi-
dence states. When the decision rule is biased, the log likelihood function crosses the value 0 (i.e., the
likelihood ratio is greater than 1) between Rating Responses 3 and 4, and the log likelihood ratio at the
detection criterion is therefore greater than 0.
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
608 December 1998 – Human Factors

which plots the UR(k) function corresponding

to biased and unbiased decision rules in an
experiment with four confidence levels per
response (i.e., the bipolar rating scale is com-
posed of the integers 1–8, and the yes-no cut-
off is between 4 and 5). In this example, the
decision rule is unbiased when the criterion
defining the upper bound of Response 4 (i.e.,
the detection criterion) is set at the intersec-
tion of the two distributions, as in the left
panel Figure 2. Note that for each rating
response k, the log likelihood ratio,

L(k) = log ( P(R = k | S )

P(R = k | N) )
increases with k and crosses the value 0 (i.e.,
the likelihood ratio crosses the value 1)
between the lowest confidence no-signal and
signal responses (i.e., Responses 4 and 5; see
lower left panel of Figure 2).
When the detection criterion is shifted to
the right, as in the upper right panel of Figure
2 (e.g., as is presumably the case when the no-
signal rate is increased), the log likelihood
ratios associated with the different rating
responses cross the value 0 prior to the cutoff
between no-signal and signal responses. In the
unbiased case (left panels), the function UR(k)
reaches its peak at the yes-no cutoff (Rating
Response 4), whereas in the biased case (right
panels), the function reaches its peak earlier
(in this example, at Rating Response 3).
Underestimation of Bias Using the
Distribution-Free Measure Ωp
If the test function UR(k) is found to be
unimodal with its maximum at the yes-no cut-
off (e.g., Rating Response 4 on an eight-point
bipolar scale), then Ωp will be zero, suggesting
that the decision rule is completely unbiased.
However, in such a case it is possible that a
biased response region near the detection cri-
terion did exist but was too small to have any
effect on the discrete function UR(k); a more
detailed explanation of this potential limita-
tion of Ωp is given in Balakrishnan (1998).
Because of this, when Ωp is zero, the researcher
should also calculate the proportion of least con-
fident no-signal responses and the proportion
of least confident signal responses. The maxi-
mum of these two values is an upper bound
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
Figure 3. Illustration of the potential for the func-
tion UR(k) to miss the bias in the decision rule if
the spacing between criteria is too large. Given that
the interval defining the response bias (i.e., the
interval between the upper bound on “4” responses
and the point at which the two density functions
intersect) is contained within the interval between
the upper bounds on “4” and “3” responses, the
proportion of biased responses is less than the pro-
portion of “4” responses.
on the true proportion of biased responses.
This property of the measure Ωp is illustrated
in Figure 3.
Although the decision rule is biased, the
spacing between the detection criterion and
the immediately adjacent criterion on the left
is too large for this bias to be detected from
the function UR(k). To see why the proportion
of “4” responses is an upper bound on the pro-
portion of biased responses in this example,
note that the actual proportion of biased re-
sponses is determined by the areas under the
two distributions (see top panel of Figure 3)
between the detection criterion (the upper
bound for Rating Response 4) and the point
DECISION PROCESSES IN VIGILANCE
of intersection between the two distributions.
This interval is contained within the interval
between the detection criterion and the upper
bound for Response 3, and hence the propor-
tion of “4” responses is larger than the pro-
portion of biased responses, pbias .
Given that the upper bound on the total
amount of response bias when Ωp equals zero
is determined by the proportion of lowest confi-
dence no-signal and signal responses (and,
hence, by the spacing between the detection
criterion and the two immediately adjacent cri-
teria), there are several possible ways a
researcher can keep this upper bound to a mini-
mum. These include (a) adding rating respons-
es to the confidence scale (adding more criteria
to decrease the spacing between them); (b)
instructing the participants to be conservative
about making the least confident responses on
the rating scale (in effect, asking the participant
to use small spacing near the detection criteri-
on); and (c) using a sliding scale response pro-
cedure (essentially, a continuous measure of
confidence; e.g., Watson, Rilling, & Bourbon,
1964). Each of these methods can be employed
without increasing the required number of
experimental conditions or the sample size. The
same arguments given earlier about the effects
of adding a confidence rating judgment to the
vigilance task would also apply to the effects of
these manipulations on the detection process.
New Methods and Their Relationship to
Signal Detection Theory
The new analyses described previously differ
from traditional signal detection theory analy-
ses of ratings data in three ways. First, a cutoff
between no-signal and signal responses must be
added to the rating scale. Traditionally, the rat-
ing scale was used exclusively to study sensitivi-
ty and the shape of the ROC curve, and
therefore an explicit cutoff between the two
discrimination responses was unnecessary.
Second, the signal rate or some other factor
presumably tied to the decision process should
be set in such a way as to induce a bias. In the
traditional approach, the purpose of the rating
task was to avoid the need to vary the decision
maker’s response bias, and therefore there was
no obvious reason to combine the rating task
with the standard bias manipulations. Con-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
609
sequently, the rating task was always per-
formed with equal likely stimuli.
Finally, instructions or a continuous valued
rating scale must be introduced so that the pro-
portion of the lowest confidence responses is
small. In signal detection theory’s representa-
tion of rating data, the relative frequencies of
the different rating responses are determined
by the spacing of the criteria: A small response
proportion indicates a small response bin.
Small spacing gives the experimenter an esti-
mate of the UR(k) function (or, equivalently, the
slopes of the ROC curve) at positions close to
the detection criterion, which is needed in order
to place a sharp upper bound on the degree
of response bias. Obviously, any one – or all
three – of these manipulations may have some
effect on the operator’s detection performance.
However, because these manipulations are all
perfectly consistent with the fundamental prin-
ciples of signal detection theory (i.e., that there
is a continuum of information states that the
participant is obliged to divide into response
bins using response criteria), there is no basis
within this theoretical framework for claiming
that the fundamental nature of the operator’s
performance (i.e., the issue of whether or not
an adjustable criterion is set on an informa-
tion continuum) should change because of
these changes in the details of the design.
EMPIRICAL APPLICATIONS
Although several aspects of vigilance, in-
cluding the exact form of the vigilance decre-
ment, appear to depend on the specific aspects
of the vigilance task, two empirical results are
consistent throughout this literature. First,
when the signal rate is low, the correct rejec-
tion rate is substantially larger than the detec-
tion rate (the signal rate effect). As the signal
rate increases, the difference between these
two proportions decreases (e.g., Davies & Para-
suraman, 1982). The most common interpre-
tation of this well-established effect is the one
associated with signal detection theory (i.e.,
the detection criterion is biased toward the
more frequent stimulus event in the manner
represented in the lower panel of Figure 1).
Second, as the time on task increases, both the
hit and the false alarm rates decrease, suggesting
610
that the initial bias against the detection re-
sponse increases with time or, in other words,
that the detection criterion increases (the cri-
terion increment).
The main purpose of the experiments reported
here was to apply the methods described pre-
viously to test the detection criterion account
of the effect of signal rate and time-on-task
on vigilance performance. In the first experi-
ment, participants performed successive or
simultaneous discrimination tasks for a con-
tinuous 50-min period. The participants were
asked to respond to each stimulus presenta-
tion using a bipolar rating scale with a yes-no
cutoff in the middle rather than to respond
only to a perceived signal event, so that the
necessary confidence rating data could be
obtained. In the simultaneous discrimination
condition, the signal rate was fixed at .1. The
reason for performing both the simultaneous
and successive discrimination tasks was not to
compare performances between the two tasks
but, instead, to add some generality to the
conclusions about the relationship of decision-
making processes to the signal rate and the
time on task.
In addition to these two standard vigilance
conditions, a third, mixed successive discrimi-
nation condition was run in which the signal
rate varied between blocks of 100 events. On
half of these blocks the probability of a signal
event was .1, and on the other half it was .5.
These data provide a basis for comparing the
test function, U R(k), under conditions in
which the optimal decision rule was either
biased or unbiased for the same group of par-
ticipants. Under no objective bias manipula-
tion, both measures Ωp and βp would be
expected to be zero or close to zero, with little
implication for the study of the nature of
response bias. Given that one demonstration
of this easily predicted result seemed adequate
by itself, the analogous simultaneous discrimi-
nation condition with equal no-signal and sig-
nal event rates was not included in the design.
EXPERIMENT 1
Method
Participants. Sixty undergraduate students
from the psychology department participant pool
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
at Purdue University participated in a single 50-
min session in partial fulfillment of an introduc-
tory course requirement. All were prescreened
for normal or corrected-to-normal vision, and
they performed one of three experimental condi-
tions (20 participants per condition): successive
discrimination, simultaneous discrimination,
or mixed successive discrimination.
Stimuli. In the two successive discrimina-
tion conditions, the signal and no-signal stim-
uli were defined by the length and height of an
L-shaped figure presented in the approximate
center of a computer monitor (14-inch, or
35.6-cm diagonal) for 0.25 s. The vertical and
horizontal lengths of this figure were 4.9 cm
and 2.1 cm, respectively, for the no-signal
stimulus. For the signal stimulus, the vertical
length was increased to 5.4 cm and the hori-
zontal length was increased to 2.4 cm.
In the simultaneous discrimination condi-
tion, the events were defined by the simultane-
ous presentation of two L-shaped figures, one
on the left (rightmost edge 2.2 cm to the left
of the screen center), and the other on the
right (leftmost edge 2.2 cm to the right of the
horizontal screen center).The two figures were
either identical (the no-signal stimulus) on
both dimensions (horizontal and vertical
length) or different on both dimensions (the
signal stimulus). The horizontal and vertical
lengths of the segments were 2.1 cm and 4.9
cm, respectively, for the figure presented on
the left in both the no-signal and the signal
stimulus. The lengths of these segments for
the figure presented on the right were
increased to 2.3 cm (horizontal) and 5.2 cm
(vertical) for the signal stimulus. In all condi-
tions, the two segments in each stimulus were
1 mm wide and viewing distance was approxi-
mately 40 cm. The physical sizes of the stimuli
were chosen on the basis of pilot data so that
the tasks would be moderately difficult. Given
that the comparisons of interest were within
conditions, no attempt was made to equate
the difficulty levels of different conditions for
the average participant.
Procedure. At the beginning of the session,
the two stimuli (no-signal and signal) were
presented simultaneously on the display
screen with their correct identification labels.
The experimenter explained the nature of the
DECISION PROCESSES IN VIGILANCE
task and asked the participants to press one
key to begin the experimental trials or another
key to repeat the demonstration (i.e., there was
no practice or warm-up period). Each stimulus
event began with a random foreperiod during
which the screen was blank. This delay value
was obtained by sampling from a truncated
exponential distribution with a mean of 2.0 s.
If this sampled value was less than 0.3 s or
greater than 30 s, a new sample was taken
(i.e., the minimum foreperiod was 0.3 s and
the maximum foreperiod was 30 s). After the
stimulus presentation (0.25 s), the participants
entered a response using a 14-point bipolar
rating scale.
There were 14 adjacent keys on the upper
row of the computer keyboard labeled from
left to right with decreasing integers from 7 to
1 (no-signal responses) and then increasing
integers from 1 to 7 (signal responses). The
two sides of the scale were labeled with the
discrimination response that they represented,
and the participants were asked to be conserv-
ative in their use of the extreme (low and high
confidence) rating responses.
The participants were given a 2-s deadline
to respond to the presentation of the stimulus.
To indicate the amount of time remaining
before the deadline was reached, an expand-
ing horizontal line was presented in the upper
portion of the display (9.4 cm below the top
edge of the viewing screen) after the stimulus
was removed. The length of this line increased
at a constant rate (9.54 cm/s for 1.75 s) until
its right-hand endpoint reached a target point
indicating the completion of the time allowed
for a response. To encourage participants to
TABLE 1: Performance Statistics for the Four Conditions of Experiment 1
Condition d′ A′
Successive 1.908 .815
Simultaneous 1.482 .727
Mixed successive 1.542 .747
(Ps = .1)
Mixed successive 1.456 .800
(Ps = .5)
Note: CR and HR are the correct rejection and hit rates, respectively, and Max(pbias) is the estimated
upper bound on the proportion of biased responses.
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
611
respond prior to the deadline as often as pos-
sible, an artificial reward system was used
(participants were not paid). If a participant
failed to respond prior to the end of the dead-
line, the screen flashed and a penalty of 50
points was assessed. This occurred on less
than 3% of the trials across conditions. Parti-
cipants were awarded 10 points for each cor-
rect response and were penalized 10 points
for each incorrect response.
The correct response and the cumulative
point scores as well as the points allotted after
each event were presented on the screen for 1 s.
The foreperiod for the subsequent event began
immediately after the feedback for the previ-
ous response was removed from the display
screen. Including the time to respond and the
feedback, the total time for each event was
between 1.3 and 33 s, and the average event
rate was therefore less than 20 events/min.
In the mixed successive discrimination con-
dition, the signal rate was chosen randomly
from the two values .1 and .5 and was held
constant at this value within a block of 100
events. At the beginning of each block, a mes-
sage indicating the signal rate for the next
block of trials was presented on the screen for
5 s. In the other two conditions, participants
were told at the beginning of the session that
the no-signal event would occur nine times
more often than the signal event.
Results and Discussion
Two estimates of sensitivity, d′ and A′ (area
under the confidence rating ROC curve; e.g.,
Green & Swets, 1974), and the two bias mea-
sures βp and Ωp are listed in Table 1. These
CR HR βp Ωp Max(pbias )
.945 .619 .125 .000 .031
.923 .521 .162 .000 .002
.932 .518 .163 .004 .039
.798 .733 .032 .000 .011
612 December 1998 – Human Factors

Figure 4. Estimates of the UR(k) functions for the four conditions of Experiment 1. The dashed vertical line
indicates the predicted location of the maximum of the function implied by the estimated value of βp given
in Table 1.

and subsequent estimates were obtained by com-
bining the individual participants’ data sets to-
gether to form a single data set. Estimates of
the UR(k) functions for each condition are
shown in Figure 4. As expected, the correct
rejection rate was considerably larger than the
hit rate in the three low signal rate conditions,
suggesting a substantial bias toward the no-
signal response in each of these conditions.
Given that the signal detection theory measure
βp (and all of the other signal detection theory
measures reviewed recently by See et al.,
1997) is directly related to the size of this dif-
ference, its value was large in each of these
conditions.
Also as expected, the difference between the
two correct response proportions was consid-
erably smaller when the signal and no-signal
rates were equal. In contrast, the distribution-
free measure Ωp was zero or very close to zero
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
in all four conditions, suggesting that the deci-
sion rule was uniformly unbiased and therefore
unrelated to the signal rate.
Given that there is only one region of
biased mental states in signal detection theo-
ry’s representation of the decision process, this
model predicts that UR(k) will be unimodal,
with its modes shifted to the left of the yes-no
cutoff when βp is large (see Figure 2 and
Balakrishnan, 1998). Although the estimated
functions are in fact unimodal, there is no left-
ward shift in the position of their maxima in
any of the conditions. In each case, the peak
of the function occurs at or very near the yes-
no cutoff. Because the sample sizes were large
enough for the estimated functions to be
smooth and unimodal (i.e., the result cannot
be attributed to estimation error), an upper
bound on the proportion of biased responses
is the estimate of Ωp (zero in all but one case)
DECISION PROCESSES IN VIGILANCE 613

Figure 5. Proportion correct conditioned on the discrimination response and confidence level for the four
conditions of Experiment 1. Any percentage correct value less than 0.5 indicates that overall proportion cor-
rect could be improved by changing the decision rule.

plus the proportion of rating responses at the
maximum of the function (Rating Response 7
in all but one case). These values – also very
small – are given in the right-hand column of
Table 1.
Finally, to illustrate in a more concrete way
the discrepancy between the signal detection
theory representation of bias and the true like-
lihood ratios associated with the confidence
ratings, Figure 4 also shows the point at which
the UR(k) functions should have reached their
maxima under the distributional assumptions
of this model. These values are indicated by
the dashed vertical lines connecting the esti-
mated UR(k) function to the abscissa. Note
that each of these predicted maxima occur to
the left of the observed maxima.
The Optimality Test
When the signal and no-signal event rates
are equal, the unbiased decision rule is also
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
the optimal decision rule (maximizing per-
centage correct). However, when the signal
rate is low, an unbiased decision rule is neces-
sarily suboptimal no matter what the shapes
of the underlying distributions may be (i.e.,
equal variance normal or otherwise). Thus, if
the decision rule in each of the four conditions
of the experiment was in fact unbiased, it
must also be suboptimal in three of the four
conditions (i.e., when the signal rate was low)
and optimal in the other (i.e., when the signal
and no-signal rates were equal). Results of the
optimality test defined earlier are shown in
Figure 5. Consistent with the interpretation
that the decision rule was in fact unbiased, the
test clearly failed when the signal rate was low
(the proportion of correct responses associat-
ed with several rating responses was less than
0.5), but it was clearly satisfied for all 14 rat-
ing responses when the signal and no-signal
rates were equal.
614
EXPERIMENT 2
Method
The main purpose of the second experiment
was to replicate the results of Experiment 1
using a slightly different design. In the first
experiment, the feedback was given immedi-
ately after the participant’s response to the
stimulus event, and the foreperiod for the sub-
sequent event began immediately thereafter
(i.e., after the 1-s feedback display). This
meant that by responding quickly to an event,
the participant could decrease the total time
between stimulus presentations, raising the
possibility that he or she might introduce a
subjective value on the speed of responses
(i.e., in order to increase the pace of the task).
Because response time incentives could poten-
tially play an important role in alternative
accounts of the decision-making process, the
time between the stimulus presentation and
the feedback display in Experiment 2 was
fixed at 2 s, regardless of when the participant
entered his or her response. Thus, responding
more quickly would not decrease the total
length of time between events.
There were 22 participants in the successive
and simultaneous discrimination conditions
(12 and 10 per condition, respectively), and
15 participants performed the mixed succes-
sive discrimination condition. The methods
were otherwise identical to those of Exper-
iment 1.
Results
Performance statistics for the four condi-
tions are listed in Table 2. Overall, the results
TABLE 2: Performance Statistics for the Four Conditions of Experiment 2
Condition d′ A′
Successive 2.365 .887
Simultaneous 2.109 .860
Mixed successive 1.778 .812
(Ps = .1)
Mixed successive 2.045 .873
(Ps =.5)
Note: CR and HR are the correct rejection and hit rates, respectively, and Max(pbias) is the estimated
upper bound on the proportion of biased responses.
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
were very similar to those in Experiment 1.
Most importantly, when the signal rate was
low, the signal detection theory bias measure
βp was large, whereas the distribution-free
measure Ω p was zero or close to zero.
Estimates of the UR(k) functions are shown in
Figure 6. Once again, there was no indication
of any leftward shift in the maximum of the
function with respect to the yes-no cutoff. In
addition, the point at which the UR(k) func-
tion should have reached a maximum under
the assumptions of signal detection theory was
consistently to the left of the observed loca-
tion, indicating that βp overestimates the
amount of bias. Results of the assumption-free
test for optimality, shown in Figure 7, were
also similar to those of Experiment 1.
GENERAL DISCUSSION
Implications for the Fixed-Sample
Models of Vigilance
Although the statistical analyses described
previously are new and somewhat complex,
the conclusion to be drawn from them is fairly
simple. Specifically, the signal rate had a signif-
icant impact on the detection rate but no effect
on the logic of the decision rule (i.e., no effect
on response bias as this notion is defined in
signal detection theory). This seemingly para-
doxical result has at least two important impli-
cations. First, given that the optimal decision
rule is biased toward the more frequent stimu-
lus event, the participants’ unbiased decision
rule is necessarily suboptimal. Second, because
the participants’ detection rate did depend on
the signal rate, the relative frequency of the
CR HR βp Ωp Max(pbias)
.947 .772 .070 .007 .037
.942 .702 .095 .000 .034
.929 .621 .123 .024 .098
.886 .799 .043 .000 .014
DECISION PROCESSES IN VIGILANCE 615

Figure 6. Estimates of the UR(k) functions for the four conditions of Experiment 2. The dashed vertical line
indicates the predicted location of the maximum of the function implied by the estimated value of βp given
in Table 2.

two stimulus events must have substantial
effects on the encoding distributions.
For obvious reasons, few if any theorists
have explicitly predicted that the signal rate
should have absolutely no effect on the partici-
pant’s decision-making process. Several theo-
rists, however, have argued that this variable
should have at least some effect on the encod-
ing distributions. J. F. Mackworth (1970) and
others (see Craig, 1979), for example, have
pointed out that because the signal stimulus
occurs infrequently when the signal rate is low,
it is not unreasonable to expect more across-
trial variability in the participant’s internal rep-
resentation of this stimulus. In order to account
for the results described earlier, the signal rate
must be assumed to affect only the variances of
the two encoding distributions, as illustrated in
Figure 8. Because the detection criterion is
always set at the intersection of the two distrib-
utions, an increase in the variance of the signal
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
distribution causes the hit rate to decrease and
the correct rejection rate to increase.
One way to test the hypothesis that the dis-
tribution variances are inversely related to pre-
sentation rates is to plot the ROC curves for
different signal rate conditions. Under some
weak assumptions (e.g., Green & Swets, 1974),
increasing the differences in the variances of
the two distributions would increase the asym-
metry of their corresponding ROC curves.
Empirical ROC curves from the two mixed suc-
cessive conditions of Experiment 1 are shown
in Figure 9. With respect to the minor diagonal
(i.e., the diagonal line connecting the upper left
and lower right corners of the graph), the
asymmetry in these curves is clearly more pro-
nounced when the signal rate was low, consis-
tent with J. F. Mackworth’s (1970) hypothesis.
(For more detailed discussions of the analysis
of vigilance ROC curves and their implications,
see Davies & Parasuraman, 1982.)
616
Figure 7. Percentage correct conditioned on the discrimination response and confidence level for the four
conditions of Experiment 2.
However, even granting that less frequent
stimuli give rise to distributions with more
variance, several questions about the nature of
the discrimination process still remain to be
answered. In particular, it is not clear why the
decision rule is completely unbiased when the
signal rate is low (i.e, why the detection criteri-
on is always set exactly at the intersection of
the distributions). It is also unclear why the
same effect on the sizes of the distribution vari-
ances occurred in the simultaneous discrimina-
tion condition, in which the participant can
directly compare the physical properties of the
stimuli during each presentation.
Sequential Sampling Models of Vigilance
From the experimenter’s point of view, the
fixed sample assumption is appropriate for
vigilance tasks because no specific incentives
are given to the participants to respond quick-
ly. Under these conditions, the participants
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
should continue to process the stimulus (or a
memory trace) until all of the stimulus infor-
mation has been exhausted. In such a case, the
total time taken to respond might vary from
trial to trial, but the participant’s behavior
would still be formally equivalent to the fixed
sample model. For a variety of reasons, how-
ever, it is not unreasonable to suppose that the
participant might place some value on the
speed of responses. For example, continued
processing of the stimulus requires continued
effort, which may lead to increased fatigue
over the course of the vigilance period.
Further, participants do not always wait
until (or respond immediately after) the stimu-
lus has been removed from the screen.
Presumably, they respond quickly when they
believe they have sufficient information to
make a correct response. However, they some-
times make errors when their response times
are short, suggesting that they might be able
DECISION PROCESSES IN VIGILANCE
Figure 8. A distribution model that predicts the
effect of signal rate on vigilance performance.
Increasing the difference in the presentation rates
of the two stimulus events changes the relative
sizes of the variances of the distributions, but it has
no effect on the degree of response bias.
to improve their performance by taking more
time to respond.
In the variable or sequential sampling mod-
els of human performance, the participant con-
tinues to collect information until a condition
for terminating the sampling process is reached.
Once this stopping rule condition is satisfied, a
decision rule determines the response to be
emitted and, if necessary, the confidence level.
(For a more detailed discussion of these kinds
of models, see Green & Swets, 1974; Link &
Heath, 1975; Luce, 1986; Thomas, 1971; Town-
send & Ashby, 1983.) Given that the stopping
rule in a sequential sampling model is inherent-
ly a decision-making process, these models
have two potential sources of bias: the stop-
ping rule and the decision rule. In order to
develop statistical tests of these types of mod-
els, therefore, a different kind of bias measure
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
617
is needed. Once again, such a measure can be
derived by analyzing the behavior of an opti-
mal decision maker.
Whatever the objective of the stopping rule
might be, the optimal decision maker must
still use whatever stimulus information was
collected to compute the two objective proba-
bilities, P[no signal | Ψi ] and P[signal | Ψi ],
where Ψi is now a random vector containing
the samples collected up to time i (the point at
which sampling was terminated). The larger
of these two probabilities would determine the
response selected when the sampling process
is terminated as well as the decision maker’s
confidence in the accuracy of this response.
Given that the reported confidence level of an
optimal decision maker should be essentially a
readout of the objective probability of a cor-
rect response at the point of the decision, a
general test of bias for sequential sampling
models is to determine whether the same
nominal degree of reported confidence is asso-
ciated with the same objective degree of accu-
racy; that is,
P[no signal | R = kN] = P[signal | R = kS], (2)
where kN and kS are equivalent confidence
levels associated with no-signal and signal
responses, respectively (e.g., on an eight-point
bipolar scale with a cutoff between Responses
4 and 5, Responses 2 and 7 both represent
Confidence Level 3).
Notice that these same conditional proba-
bilities also determine whether the decision
rule is optimal in the sense defined earlier for
the fixed sample model. Thus, for an optimal
decision maker in a dynamic decision making
context, each of the terms involved in the new
test should be greater than or equal to .5.
To account for the transformation from
continuous confidence values to discrete rat-
ing responses, each confidence rating in the
optimal decision maker should be defined by
an upper and a lower bound on the probabili-
ty of a correct response. The probability of a
correct response conditioned on the rating
response will fall somewhere between these
two bounds, and a more general test is there-
fore to verify that a higher confidence level is
always associated with higher accuracy, even
618
Figure 9. ROC curves for the mixed successive con-
ditions of Experiment 1. The skew of the curve
with respect to the minor diagonal (i.e., the line
connecting the upper left and lower right corners of
the figure) can be taken as an index of the differ-
ences in variances of the signal and no-signal distri-
butions.
when comparisons are drawn between differ-
ent discrimination responses. For example, the
proportion of correct responses corresponding
to signal responses given at Confidence Level
3 should be larger than the proportion of cor-
rect responses corresponding to no-signal
responses given at Confidence Level 2.
Although there are other possible forms of
response bias in a dynamic system, this new
test identifies a bias that is clearly a signature
for the effects of signal rate on participants’
performance. Referring once more to the esti-
mated functions in Figures 6 and 8, the type
of discrimination response (i.e., signal or no
signal) has no systematic effect on the propor-
tion of correct responses associated with the
different confidence levels when the signal
and no-signal rates are equal, and it has a sub-
stantial effect on these values when the signal
rate is low. Thus, response biases of the form
defined by Equation 2 do exist empirically and
are directly tied to the signal rate.
Psychological Interpretations
In addition to recovering the original con-
cept of response bias as a factor directly relat-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
ed to the signal rate and affecting only the
decision-making process (i.e., the stopping
rule), the dynamic modeling framework for
vigilance also provides a basis for explaining
both the lack of bias in the decision rule and
the apparently strong bias in the stopping rule.
To predict the results of the Equation 2 bias
test, it is enough to suppose that the partici-
pants accurately compute the heights of the
encoding distributions associated with their
stimulus samples but overestimate the signal
rate when this value is low. Such a mecha-
nism, already incorporated in classical, fixed-
sample theories of discrimination, would
cause a decision maker in a dynamic decision-
making context to overestimate the probability
P[signal | Ψi ], leading to premature termina-
tion of the sampling process.
A similar approach would be to assume
that the participant sets a higher value on cor-
rect signal responses than on correct no-signal
responses. In this case, the participant inten-
tionally adjusts his or her stopping rule to
maximize the expected reward (which pre-
sumably would also depend on the speed of
the response). Because they are mathematically
equivalent, empirical tests to distinguish these
two interpretations are difficult to formulate
(Balakrishnan, 1997).
To illustrate the effect of this miscalculation
on the objective stimulus probabilities associat-
ed with the different subjective confidence
states, suppose that the sampling process is ter-
minated when the perceived value of the proba-
bility of a correct response (i.e., the perceived
value of the maximum of P[signal | Ψi ] and
P[no signal | Ψi] is equal to .9. In this case,
when the participant makes a signal response,
the perceived value of P[signal | Ψi] at the point
of the decision was .9, but the actual value of
this probability was lower (because the signal
rate was overestimated). Hence, the percentage
correct associated with the signal responses will
be less than .9. Similarly, when the participant
makes a no-signal response, the perceived value
of P[no signal | Ψi] at the point of the decision
was .9, but the actual value of this probability
was higher (because the no-signal rate was
underestimated), and so the percentage of cor-
rect responses associated with this discrimina-
tion response will be greater than .9.
DECISION PROCESSES IN VIGILANCE
TABLE 3: Performance Statistics for the First and Second Half of the Sessions from the Successive and
Simultaneous Discrimination Conditions of Experiment 1
Condition d′ A′
Successive
1st half 1.799 .820
2nd half 2.096 .811
Simultaneous
1st half 1.438 .747
2nd half 1.594 .706
To illustrate the relationship between the
percentage of correct responses and the objec-
tive probability of a correct response at the
point of the decision in a sequential sampling
model, consider the following example. Sup-
pose that a gambler will make a bet only when
the odds are (exactly) four to one in his or her
favor. Examining the (infinite) history of this
gambler’s bets, exactly four out of five will
turn out to be winners.
Vigilance Decrements and Criterion
Increments
If the difference between the hit and cor-
rect rejection rates in vigilance experiments
cannot be attributed to a biased decision rule,
then it follows that the increase in this differ-
ence with time on task cannot be attributed to
an increase in this type of bias. In this section,
I use the new bias measures described previ-
ously to see how dynamic models of vigilance
would interpret the effect of time on the
detection rate. To do this, the data from the
successive and simultaneous conditions of
Experiment 1 were split into two parts (rough-
ly 25 min each) and reanalyzed. Sensitivity
and bias statistics for the two halves of the
session are given in Table 3. Given that the
event rate in both conditions was low, little or
no decrease in sensitivity would be predicted
by the taxonomy developed by Parasuraman
and Davies (1977).
Although d′ actually increased slightly with
time on task, the distribution-free index A′
should be considered a more appropriate mea-
sure given the violations of the equal variance
assumption indicated by the ROC curves
shown in Figure 9. This value decreased
slightly for the two conditions of this experi-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
619
CR HR βp Ωp
.922 .645 .113 .026
.968 .594 .130 .000
.891 .581 .132 .000
.955 .458 .183 .000
ment. More important, the experiment also
replicated the classical effect of time on task:
In each condition, the hit and false alarm rates
decreased in the second half of the session,
resulting in an increase in the bias measure βp.
In contrast, the distribution-free measure Ωp
was zero or close to zero, suggesting that the
detection criterion account of this phenome-
non is once again invalid.
Estimates of the new bias measure repre-
sented by Equation 2 are shown in Figure 10.
For the no-signal responses, there was little
effect of time on task on the probability of a
correct response under the different confidence
levels. There was a substantial effect of time,
however, on the level of this function for the
signal response trials: According to this new
measure, the participants’ response bias de-
creased with time on task, reducing (but not
eliminating) the suboptimal character of their
decision rule.
Interpretations of the Effects of Time on
Task in the Fixed and Variable Sample
Models
In previous accounts of the vigilance decre-
ment, the decrease in detection rate was attrib-
uted to an increment in the detection criterion
corresponding to a decrease in the operator’s
willingness to make a detection response.
Assuming that the initial location of the criteri-
on was conservative with respect to its optimal
location, the increment would increase the
overall percentage of correct decisions. One
possible explanation for the decrease in the
detection rate, therefore, was that participants
learn to adjust their detection criterion as they
gain experience with the task (e.g., Williges,
1969; see, however, Vickers, Leary, & Barnes,
620
Figure 10. Effects of time on task on the accuracy
predictions of subjective confidence (Equation 2
bias test) in the successive and simultaneous dis-
crimination conditions of Experiment 1.
1977, for a demonstration that estimates of the
parameter β do not always approach the puta-
tive optimal value with time on task).
Although this description of the decision
rule is inconsistent with the finding that the
decision rule is always unbiased, a similar
mechanism could account for the results
shown in Figure 10 if it is added to the
sequential sampling framework for discrimi-
nation. Specifically, if the participants initially
overestimate the signal rate but the size of this
estimation error decreases with experience
(i.e., their representation of the signal rate
becomes more accurate), then their tendency
to terminate the sampling process prematurely
would also be reduced. This would cause the
objective probabilities associated with the
confidence levels for the two discrimination
responses to converge in the manner shown in
Figure 9. Under the fixed sample model, the
results shown in Figure 10 could be taken as
evidence that the disparity in the variances of
the signal and no-signal distributions decreases
with time on task. Presumably, whatever mech-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
anism is assumed to cause the difference in
variances could also be assumed to weaken
with time on task.
SUMMARY AND FINAL COMMENTS
A fundamental assumption of signal detec-
tion theory is that the objective likelihood ratio
at the point XC (i.e., the location of the detec-
tion criterion) changes as a function of the par-
ticipant’s decision-making attitude. This
attitude is presumably rational at least to some
degree. That is, if the signal stimulus becomes
more likely, the participant shifts the detection
criterion to some degree to favor a signal re-
sponse. For several decades, this relatively sim-
ple, one-parameter representation of human
decision processes has seemed to work ex-
tremely well, providing a highly plausible and
consistent picture of human discrimination per-
formance in a wide variety of experimental set-
tings, including vigilance settings. For example,
the model predicts that as the signal rate
decreases, the hit and false alarm rates should
both decrease, whereas the sensitivity measure,
d′, should remain roughly the same. Empirical
estimates of d′ do in fact remain roughly con-
stant under the standard bias manipulations
(e.g., Green & Swets, 1974), providing what
appears to be a strong confirmation of the basic
tenets of this formal theory.
Although many researchers would probably
expect this model’s underlying assumptions to
be violated to some degree (e.g., the distribu-
tions might not be exactly normal), few would
expect them to be violated in an insidious way
(i.e., in a way that would cause the model to
fundamentally misrepresent the qualitative
properties of human decision-making behav-
ior). This is true despite the fact that for many
years it has also been known that another
class of models (sequential sampling models)
can also account for these classical data,
although they interpret them in very different
ways (e.g., the bias is defined by the relative
distances from the starting point of the two
response thresholds).
The results described in this article present
what appears to be a major challenge to the
static interpretation of signal detection theory.
Specifically, for the same reasons that this
DECISION PROCESSES IN VIGILANCE
model predicts a decrease in the hit and false
alarm rates as the signal rate decreases (i.e., a
change in the likelihood ratio at the detection
criterion), it also predicts that the peak of the
empirical function, UR(k), should shift toward
the no-signal response side of a bipolar confi-
dence rating scale. This prediction was consis-
tently violated by empirical data in the two
vigilance experiments reported here.
Although the signal rate did have a sub-
stantial impact on the detection rate, there
was no shift in the peak of the estimated func-
tion UR(k). Similarly, if the detection criterion
shifts further in favor of the no-signal re-
sponse with increasing time on task (the crite-
rion increment), then the peak of the UR(k)
function should shift further in the direction
of the no-signal response side of the rating
scale. In both the first and second halves of
the vigilance periods, the peak of the function
was consistently fixed at the center of the rat-
ing scale, despite the fact that the hit and false
alarm rates decreased with time on task. Thus,
with respect to both the signal rate and time
on task, the standard findings of previous vigi-
lance studies were replicated, but an analysis
of the distributions of subjective confidence
revealed some apparently serious problems
with signal detection theory’s representation
of the decision-making process.
Alternative Interpretations of the
Empirical Results
In order to estimate the value of the likeli-
hood ratio at the criterion value, XC, the stan-
dard experimental methods associated with
vigilance experiments needed to be modified.
For example, the participants were asked to
indicate not only their discrimination judg-
ment but also their degree of confidence in
this response. Further, they were also instruct-
ed to use the rating scale in a particular way
(i.e., reserving the extremely underconfident
responses of the scale for extremely uncertain
states). Obviously, it is possible to simply dis-
miss confidence rating data as a measure of
human performance.
In terms of the signal detection theory
model, however, the rating scale is a logical
extension of the yes-no detection task. In the
latter case, the participant sets a single detec-
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
621
tion criterion on some psychological dimen-
sion, and in the former case, the participant
sets this detection criterion and several others
on the same psychological dimension. The dif-
ference is that the detection criterion deter-
mines the accuracy of the participant (the hit
and false alarm rates) and, hence, is not arbi-
trary. The other criteria determine the relative
frequencies of the different rating responses
and the percentage correct values associated
with these responses, and they are arbitrary in
the sense that they do not affect the partici-
pant’s hit or correct rejection rates.
It is possible that subjective confidence as
measured by the rating scale is contaminated
in some way, causing the location of the detec-
tion criterion to be misrepresented. However,
no mechanism exists within the framework of
signal detection theory that would produce
such an effect, and no obvious modification of
this theory would account for the confidence
ratings data. At the very least, therefore, the
data presented here need to be accounted for
before the interpretations associated with sig-
nal detection theory are taken for granted.
In addition to the confidence rating re-
sponse, the methods employed in the two
experiments described here also differed from
the more typical vigilance tasks in two other
ways. First, the participants were informed
about the relative frequencies of the signal
and no-signal conditions prior to performing
the task (or prior to each block of trials in the
mixed successive discrimination condition).
The participants were also given feedback after
each response. Both manipulations may be
expected to have some effect on performance,
but neither should be expected to cause the
fundamental assumptions of signal detection
theory to be violated. In fact, if anything, this
information should have encouraged the par-
ticipants to bias their decision rule in the appro-
priate way.
Finally, in other experiments (e.g., Balakrish-
nan, 1998), asymmetric payoffs were used to
induce a response bias, and the results were
the same (i.e., there was no evidence for bias-
es of the type defined within the signal detec-
tion theory model). Thus, although it is
possible that an alternative explanation could
be formulated for the results described here
622
that preserves the notion of response bias
from signal detection theory, it is not obvious
what this would be, and certainly no alterna-
tive account would be possible without sub-
stantial revisions, additions, or both to the
classical signal detection theory model.
Other Measures of Response Bias
As noted earlier, most experimental analy-
ses of performance indices for vigilance data
have involved quantitative comparisons of the
degree of change in the measures or their sta-
tistical properties (e.g., their variance or corre-
lations to sensitivity indices) under different
experimental conditions. For example, in a
recent review of the several bias measures
derived from signal detection theory by Mac-
millan and Creelman (1991), See et al. (1997)
concluded that the measure β was less “ade-
quate” than the alternatives proposed by
Macmillan and Creelman because it was less
sensitive to supposed bias manipulations.
Several points about the relevance and impli-
cations of this earlier work are worth clarify-
ing. First, all of the measures compared in See
et al. were based exclusively on the hit and
false alarm rates. In the present study, the dif-
ference between these two values was, not
surprisingly, substantial when the signal rate
was low (see Tables 1 and 2), and therefore all
of these bias measures would indicate a sub-
stantial bias toward the no-signal response.
Second, all of these measures depend,
implicitly or explicitly, on a specific distribu-
tion model, and when they indicate that a bias
exists, they also indicate that there exists a set
of “biased mental states,” as illustrated in
Figure 1. In other words, the differences in
these measures are quantitative rather than
qualitative. The results described in this article
raise questions about the validity of all bias
indices that represent response bias in this
general way, not just the measure β.
Finally, and most important, the fact that
biases of this particular kind do not appear to
exist under standard bias manipulations (e.g.,
changes in the signal rate) should not be con-
fused with the empirical fact that these manip-
ulations have qualitatively different effects on
the hit and false alarm rates than do sensitivity
manipulations (e.g., changes in the physical
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
December 1998 – Human Factors
similarity of the stimuli or the noise level).
This difference clearly must have important
implications for theories of vigilance – the
main point of the present study is not that
response biases do not exist but, merely, that
response biases of the type defined in signal
detection theory are not produced by standard
bias manipulations.
Implications for Performance Measures
Given that all formal models of vigilance
must define an information state at the point at
which a decision is made about the current dis-
play condition, all models (fixed and variable
sample) must define two distributions, one for
each of the two stimulus events. Sensitivity
indices such as d′ and A′ are measures of the
relationship between these two distributions
under the assumption that they are univariate
(e.g., as is subjective confidence defined on a
bipolar scale). Thus, although they were origi-
nally developed with the fixed-sample model in
mind, these measures can also be used to ana-
lyze performance under the variable sample
assumption, under the proviso that any differ-
ences in sensitivity level could be attributable
entirely or in part to differences in the opera-
tor’s stopping rule. For example, when a sensi-
tivity index decreases with time on task (i.e.,
the sensitivity decrement), it is not necessarily
true that the operator’s attention level has
decreased. At least in principle, the same result
could also be attributed to a change in the
stringency of the stopping rule.
Practical Applications
Understanding the nature of the decision
processes involved in vigilance is obviously
important with respect to theories of vigi-
lance, but it may also have important ramifica-
tions for applied problems associated with
information display. For example, if the sub-
optimal performance of an operator is related
to the amount of time spent processing the
display before an action is taken, then opera-
tors’ performance should improve if they are
encouraged to respond more slowly to appar-
ent signal occurrences. This kind of interven-
tion might be particularly effective if the
vigilance sensitivity decrement turns out to be
a stopping rule effect.
DECISION PROCESSES IN VIGILANCE
Another approach to manipulating the
operator’s decision-making strategy would be
to solicit confidence rating responses and then
inform the operator about the objective proba-
bility of a correct response corresponding to
their different confidence levels (i.e., in the
manner of the Equation 2 bias test). In this
way, a trainer or personnel manager could pro-
vide a concrete goal for operators that would
allow them to optimize their decision-making
behavior with respect to whatever objective is
most appropriate for their task. This type of
bias manipulation is not possible using tradi-
tional methods (e.g., defining an explicit pay-
off system or setting a target for the hit or
false alarm rates), because the information
needed to reveal the logic of the operator’s
decision-making strategy is not available in the
hit and false alarm rates alone.
ACKNOWLEDGMENTS
Parts of this research were supported by
National Science Foundation Grant SBR-
9709789 and NASA Dryden FRC Grant
NCC2-374.
REFERENCES
Balakrishnan, J. D. (1997). Form and objective of the decision
rule in absolute identification. Perception & Psychophysics,
59, 1049–1058.
Balakrishnan, J. D. (1998). Some more sensitive measures of sen-
sitivity and response bias. Psychological Methods, 3, 68–90.
Balakrishnan, J. D., & Ratcliff, R. (1996). Testing models of deci-
sion making using confidence ratings in classification. Journal
of Experimental Psychology: Human Perception and Perfor-
mance, 22, 615–633.
Craig, A. (1979). Nonparametric measures of sensory efficiency
for sustained monitoring tasks. Human Factors, 21, 69–78.
Creelman, C. D., & Donaldson, W. (1968). ROC curves for dis-
crimination of linear extent. Journal of Experimental Psychology,
77, 514–516.
Davies, D. R., & Parasuraman, R. (1982). The psychology of vigi-
lance. London: Academic.
Deaton, J. E., & Parasuraman, R. (1993). Sensory and cognitive
vigilance: Effects of age on performance and subjective work-
load. Human Factors, 6, 71–97.
Dorfman, D. D., & Alf, E., Jr. (1969). Maximum-likelihood esti-
mations of parameters of signal-detection theory and determi-
nation of confidence intervals – Rating-method data. Journal
of Mathematical Psychology, 6, 487–496.
Green, D. M., & Swets, J. A. (1974). Signal detection theory and
psychophysics. Huntington, NY: Krieger.
Link, S. W., & Heath, R. A. (1975). A sequential theory of psy-
chological discrimination. Psychometrika, 40, 77–105.
Lockhart, R. S., & Murdock, B. B., Jr. (1970). Memory and the
theory of signal detection. Psychological Bulletin, 74, 100–109.
Luce, R. D. (1986). Response times: Their role in inferring elemen-
tary mental organization. New York: Oxford University Press.
Mackworth, J. F. (1970). Vigilance and attention. Harmondsworth,
England: Penguin.
Downloaded from hfs.sagepub.com at NORTHWESTERN UNIV LIBRARY on June 4, 2016
623
Mackworth, N. H. (1948). The breakdown of vigilance during
prolonged visual search. Quarterly Journal of Experimental
Psychology, 1, 6–21.
Mackworth, N. H. (1961). Researches on the measurement of
human performance. In H. W. Sinaiko (Ed.), Selected papers
on human factors in the design and use of control systems
(pp. 174–331). New York: Dover.
Macmillan, N. A., & Creelman, C. D. (1990). Response bias:
Characteristics of detection theory, threshold theory, and “non-
parametric” indexes. Psychological Bulletin, 107, 401–413.
Macmillan, N. A., & Creelman, C. D. (1991). Detection theory: A
user’s guide. Cambridge, England: Cambridge University Press.
Maloney, L. T., & Thomas, E. A. C. (1991). Distributional assump-
tions and observed conservatism in the theory of signal
detectability. Journal of Mathematical Psychology, 35, 443–470.
Parasuraman, R. (1979). Memory load and event rate control sensi-
tivity decrements in sustained attention. Science, 205, 924–927.
Parasuraman, R., & Davies, D. R. (1977). A taxonomic analysis of
vigilance performance. In R. R. Mackie (Ed.), Vigilance:
Theory, operational performance, and physiological correlates
(pp. 559–574). New York: Plenum.
Parasuraman, R., & Mouloua, M. (1987). Interaction of signal dis-
criminability and task type in vigilance decrement. Perception
& Psychophysics, 41, 17–22.
Parasuraman, R., Warm, J. S., & Dember, W. N. (1987).
Vigilance: Taxonomy and utility. In L. S. Mark, J. S. Warm, &
R. L. Huston (Eds.), Ergonomics and human factors: Recent
research (pp. 11–32). New York: Springer-Verlag.
Proctor, R. W., & Van Zandt, T. (1994). Human factors in simple
and complex systems. Boston: Allyn and Bacon.
See, J. E., Howe, S. R., Warm, J. S., & Dember, W. N. (1995).
Meta-analysis of the sensitivity decrement in vigilance.
Psychological Bulletin, 117, 230–249.
See, J. E., Warm, J. S., Dember, W. N., & Howe, S. R. (1997).
Vigilance and signal detection theory: An empirical evaluation
of five measures of response bias. Human Factors, 39, 14–29.
Swets, J. A. (1986). Indices of discrimination or diagnostic accura-
cy: Their ROCs and implied models. Psychological Bulletin,
99, 110–117.
Thomas, E. A. C. (1971). Sufficient conditions for monotone haz-
ard rate and application to latency-probability curves. Journal
of Mathematical Psychology, 8, 303–332.
Townsend, J. T., & Ashby, F. G. (1983). Stochastic modeling of
elementary psychological processes. Cambridge, England:
Cambridge University Press.
Vickers, D., Leary, J., & Barnes, P. (1977). Adaptation to decreas-
ing signal probability. In R. R. Mackie (Ed.), Vigilance:
Theory, operational performance and physiological correlates
(pp. 679–703). New York: Plenum.
Warm, J. S. (1984). An introduction to vigilance. In J. S. Warm
(Ed.), Sustained attention in human performance (pp. 1–14).
Chichester, England: Wiley.
Warm, J. S., Dember, W. N., Murphy, A. Z., & Dittmar, M. L.
(1992). Sensing and decision-making components of the signal-
regularity effect in vigilance performance. Bulletin of the
Psychonomic Society, 30, 297–300.
Warm, J. S., & Jerison, H. J. (1984). The psychophysics of vigi-
lance. In J. S. Warm (Ed.), Sustained attention in human per-
formance (pp. 15–60). Chichester, England: Wiley.
Watson, C. S., Rilling, M. E., & Bourbon, W. T. (1964). Receiver
operating characteristics determined by a mechanical analog
to the rating scale. Journal of the Acoustical Society of
America, 36, 283–288.
Williges, R. C. (1969). Within-session criterion changes compared
to an ideal observer criterion in a visual monitoring task.
Journal of Experimental Psychology, 81, 61–66.
J. D. Balakrishnan is an assistant professor in the
Department of Psychological Sciences at Purdue
University. He received his Ph.D. in cognitive math-
ematical psychology from the University of
California, Santa Barbara, in 1991.
Date received: January 9, 1997
Date accepted: April 28, 1998