Professional Documents
Culture Documents
decision error shows naturalistic decisions to be ment involves defining the problem as well as
non-optimal when compared with a normative assessing the levels of risk associated with it and
mathematical model. Often naturalistic situations the amount of time available for solving it. Once
are not amenable to optimizing strategies. It may the problem is defined, a course of action must
not be appropriate to apply a model such as be chosen. The course of action is selected from
Multiattribute Utility Theory (MAUT) because the options available in the situation. Building
normative approaches require time and large on Rasmussen (1986), Orasanu and Fischer
computational capacity to thoroughly evaluate all proposed three types of option structures: rule-
options, neither of which may be available to the based, choice and creative. In rule-based
decision maker. Manipulating large amounts of decisions, there is a single prescribed action to
data is not consistent with human information take in response to a particular condition. Once
processing capability, especially under stress the problem situation is recognized, the solution
(Stokes et al, 1992). People work within a should be evident to someone with domain
'bounded rationality' (Simon, 1957); satisficing expertise, e.g. icing conditions. The second
is a function of this. type, choice decisions, involve multiple options
with legitimate trade-offs between them, e.g.
From the perspective of bounded rationality, selecting alternative airports. Choices depend on
apparently irrational, biased or heuristic prevailing goals and constraints. The third type,
approaches to decision making are often an creative, pertains to situations in which no
individual's response to the computational suitable options are readily available and the
complexity of a decision task. In fact, this decision maker must invent at least one to meet
approach is not necessarily defective, given the the demands of the situation, perhaps using
constraints of the situation (Cohen, 1993; analogies to similar situations.
Payne, Bettman & Johnson, 1988). Many
assumptions are built into normative theory, Thus, there are two major ways in which error
including that people will always seek to may arise. People may (a) develop a wrong
optimize outcomes, that their preferences are interpretation of the problem, which leads to a
stable, and that information and options are wrong decision because they are solving the
known. These assumptions are not always valid wrong problem -- an SA error, or (b) establish
in real world decision making (Brehmer, 1987) an accurate picture of the situation, but choose
and typically ignore the contributions of the the wrong course of action -- a CoA error.
decision maker's expertise and flexibility.
Situation assessment errors can be of several
Decision processes and errors types: situation cues may be misinterpreted,
misdiagnosed, or ignored, resulting in a wrong
Considering the nature of decision making in picture; risk (threat or danger) levels may be
naturalistic contexts, the question remains misassessed (Orasanu, Dismukes & Fischer,
whether it is reasonable to say the decision 1993); or the amount of available time may be
maker made an error or simply did not select the misjudged (Orasanu & Strauch, 1993).
best solution, e.g. choosing to attempt a landing
in bad weather conditions rather than going Errors in choosing a course of action may also
around. Since the outcome of a decision cannot be of several types. In rule-based decisions, the
necessarily be used to identify error, it is more appropriate response may not be retrieved from
appropriate to look at the process by which memory and applied, either because it was not
decisions are made (Woods, et al., 1996). known or because some contextual factor
mitigated against it. In choice decisions, options
Orasanu and Fischer (1996) described a decision also may not be retrieved from memory, or only
process model for aviation that involves two one may be retrieved when in fact multiple
components: situation assessment (SA) and options exist. Constraints or factors that
choosing a course of action (CoA). We will use determine the adequacy of various options may
this rough process model as a frame for not be retrieved or used in evaluating the
considering decision errors. Situation assess- options. Finally, the consequences of various
HESSD ’98 102
options may not be considered. The decision the decision maker does not have the right
maker may fail to mentally simulate the possible knowledge to construct an accurate represent-
outcomes of each considered option. Creative ation. Experience can also influence availability
decisions may be the most difficult because they of response options. Information errors are
involve the least support from the environment. those in which the decision maker does not have
The absence of available options means enough information to make a decision--
candidate solutions must be invented to fit the technically not cognitive errors. The third
goals and existing conditions. source of error is inadequate mental simulation
which may lead to 'wrong choice' errors.
Using this framework, we see that the Failure to simulate outcomes associated with
confidence with which errors can be identified various courses of action may lead to poor
may depend on the type of decision structure and choices. This is a process error. Thus, Klein's
the component in which the error lies. Mistakes content analysis supports our suggestion that
in situation assessment may be more clearly there are two basic classes of error in naturalistic
classified as errors than mistakes in choosing a situations.
course of action, since in principle it is possible
to know the "correct" definition of a problem. II. What Factors Contribute to Decision
Risk or threat levels in principle are knowable, Errors?
as is the time available to make a decision.
Confidence in identifying course of action errors To explore factors that contribute to decision
varies depending on the determinacy of the errors, we examined cases in the NTSB's
options. In rule-based decisions, it may be (1994) set of 37 "crew-caused" accidents that
possible to say that an error occurred because the involved "tactical decision errors". A common
correct response was not applied (assuming the pattern was the crew's decision to continue with
situation was correctly assessed) e.g. being over their original plan when conditions suggested
take-off weight. However, in choice situations, that other courses of action might be more
defining the best option may be more prudent3. In other words, they decided to "go"
problematic, e.g. one alternative airport may in a "no go" situation, usually in the face of
have the maintenance facilities you require whilst ambiguous or dynamically changing conditions,
another would provide better connections for e.g. continuing with a landing when it might
your passengers. These are cases in which have been more appropriate to go-around. Four
satisficing may rule. Here, it may only be factors are hypothesized as possible contributors
possible to say that the process was faulty if the to these decision errors:
decision maker did not consider readily available • The situations were not recognized as ones
options, failed to take into account relevant that should trigger a change of course of
constraints, or did not project consequences of action, due to the ambiguity of the cues;
the various courses of action. These would
constitute process errors. In creative decisions, • Risk was underestimated;
it is also difficult to say that an error occurred • Goals conflicted (safety vs. productivity,
since no options exist. Any solution that met a mission completion or social factors);
goal and major constraints would have to be • Consequences were not anticipated or
considered a success, even if other better evaluated.
solutions might be generated in hindsight. If no
response at all were made, then we might call it A. Ambiguity: Cues that signal a problem
an error. are not always clear-cut. Conditions can
deteriorate gradually, and the decision maker's
Our analysis is supported by Klein's (1993) situation assessment may not keep pace. If
analysis of decision errors across a variety of events occur infrequently, the decision maker
naturalistic domains. He found three sources of may not have amassed the experience to
error: lack of experience, lack of information, recognize the signals associated with a different
and inadequate simulation. Lack of experience CoA. Flight crews are typically in a "go" frame
may contribute to situation assessment errors if of mind. A substantial weight of evidence is
HESSD ’98 103
needed to change the plan being executed. C. Goal conflicts: Organizational factors
Weather and certain system malfunctions can emphasize productivity, e.g. on time arrivals and
change dynamically and pose a challenge for departures, or saving fuel, which may conflict
situation assessment. For decisions that have with safety4. Pilots may be willing to take a risk
consequences, such as rejecting a takeoff or with safety (a possible loss) to arrive on time (a
diverting, the decision maker needs to justify a sure benefit). Social factors are also influential:
course of action that may entail a cost. If the among pilots peer pressure may encourage risky
situation is ambiguous, the decision is harder to behavior. Meeting organizational and social
justify than if the situation is clear-cut, which goals often appears to outweigh safety goals,
may work against a decision to change the CoA. especially in ambiguous conditions.
support would be most helpful? Technologies to from, they will be able to select a model which
assist decision makers have not always had their more closely fits the problem. The importance
focus on the decision maker and the areas in of having a choice of exemplars in memory is
which they have difficulty. Decision aids may illustrated by Klein (1997), amongst others,
benefit from an interpretation in this light on who notes a distinguishing feature of chess
areas of likely errors. masters is the large number of board positions
they have committed to memory.
Wiener (1993) separates technologies for
reducing human error into mechanistic and SA will also be assisted by addressing two
behaviorally based interventions. Mechanistic components of naturalistic situations: the risk
interventions include hardware interlocks, such and time available. It is difficult to embed
as gust locks on aircraft controls, or software information about risk and time in displays,
traps; behavioral ones include training, check- since these often depend on the context. Aircraft
lists and procedures. While mechanistic system information, perhaps, could have risk
approaches may be more reliable, behavioral estimates attached. Weather and traffic displays
ones are more adaptable. If we think in terms of already have some risk information (color
the two major decision components, situation coding of weather severity; TCAS warnings for
assessment and choosing a course of action, we traffic). Aids providing judgments of time
can begin to suggest the kinds of aids that might available for making a decision or taking action
be helpful under these two categories. would require predictive models. They could
indicate how long it will take for a condition to
Assisting SA degrade to a critical state, e.g. fuel consumption
or reserve battery life span, how soon weather
Given that accident analysis suggests that a will improve or a storm hit, or when traffic will
major impact could be made on decision making dissipate in target region. Again, contextual
by improving situation assessment so pilots factors are likely to make such predictions
recognize situations that require revision of their difficult.
CoA, what are the essential components? To
improve pilots’ understanding of their problem Assisting the pilot’s SA addresses two of the
requires better diagnostic information and more error sources identified by Klein (1993), those
accessible, comprehensible, integrated displays of lack of information and experience.
that show trends (see Flach et al. 1994, Information can be provided at the point of
Norman, 1986). These improvements apply decision in enhanced displays which are user-
most clearly to system diagnostics (in which centered. Experience can be built through
there has been great improvement already), directed training which presents patterns of
providing more up to date information on developing process errors to pilots, enabling
unpredictable, dynamic attributes of the situ- them to recognize the trend of a sequence of
ation. It is more difficult to present weather events and hence act to prevent an incident
information in the integrated format required to occurring.
show trends, although this is not impossible as
the new terminal area weather displays CoA Assistance
demonstrate (e.g. ITWIS, MIT/LL). However,
weather information should highlight inform- The second decision making component to aid is
ation critical to pilots depending on their phase choosing a course of action, and with this the
of flight. Traffic displays are also being third source of error - inadequate simulation. To
improved, especially in anticipation of "free do this, aids may present options, constraints
flight" and will be integrated with weather and the likelihood of outcomes. At the moment,
information (e.g. NASA’s AATT project). flight manuals typically list options for dealing
with system malfunctions. For example, when
A second avenue is to improve decision makers' certain kinds of engine anomalies are encount-
experience by giving them better training: if they ered the conditions under which it would be
have a large number of exemplars to choose most prudent to shut down the engine, reduce
HESSD ’98 105
power to idle, or leave it running are described. ures do not guarantee that every incident
Aiding may consist of prompting crews to situation will be caught. Changing the system in
consider options prior to jumping to action, to an effort to catch errors may also alter it in a way
consider the disadvantages of the selected to change pilot behavior within it, the source of
option, and the likelihoods of various outcomes. the error may therefore change. Thus, pilot
Klein (1997) included such recommendations in decision error may be a moving target which will
a training model for military decision making. have to be periodically redefined as the field
Tools for doing "what if" reasoning and advances.
managing multiple hypotheses may also be
helpful, encouraging forward thinking. Means While better displays are already in evidence and
et al (1993) stress the importance of making automation already controls many aircraft
decision makers aware of the worst case functions, we still are left with human judgment
scenario and training them to manage this. in cases that cannot be automated. The challenge
of the future is to provide support for the weak
It will be more difficult to present risk ratings for links in human decision processes, while
goal conflict situations. We know that people exploiting the strengths and adaptability of the
are notoriously poor at integrating numerical human agent through specifically designed tools
probabilities, faring better when information is and training.
presented as a graphical representation.
Context-sensitive estimates of event likelihoods
would be needed to enable trade-offs to be References
made.
Brehmer, B. (1987). The psychology of
Conybeare (1980), and others, have suggested risk. In W.T. Singleton & J. Hovden (Eds.)
that people recalculate the risks involved under a Risk and Decisions (pp. 25-39). New York:
new system and compensate their behavior to John Wiley & Sons.
bring the level of risk back to its previous level. Cohen, M.S. (1993). The naturalistic
This suggests decision aids may not have a basis of decision biases. In G. Klein, J .
lasting effect. However, McKenna (1985) Orasanu, R. Calderwood & C. Zsambok (Eds.),
refutes this homeostasis theory on a number of Decision making in action: Models and methods
counts, among these are that people are unable to (pp. 51-99). Norwood, NJ: Ablex.
calculate risk probabilities in this way and that Conybeare, J. (1980). Evaluation of
non-obvious safety measures could not be automobile safety regulation: the case of
accounted for by the individual. Whilst agreeing compulsory seat belt legislation in Australia.
with McKenna, there is anecdotal evidence that Policy Sciences, 12, 27-39.
pilots (like motorists) will ‘push the envelope’ Flach, J., Hancock, P., Caird, J. &
when new measures are introduced, altering Vicente, K. (Eds.) (1994). Global Perspectives
their behavior to exploit the increased on the Ecology of Human-Machine Systems
functionality of the systems, e.g. driving faster (pp.1-13). Hillsdale, NJ: Lawrence Erlbaum
when safety belts became mandatory. An Associates.
example of this is possibly the Airbus 320 crash Hollenbeck, J., Ilgen, D., Phillips, J. &
as the Habsheim airshow. New systems may Hedlund, J. (1994). Decision risk in dynamic
lead the pilot to change the way they evaluate two-stage contexts: Beyond the status-quo.
options, or the weights they assign to attributes. Journal of Applied Psychology, 79(4), 592-598.
This may not necessarily lead to more risky Jensen, R. (1998). Presentation at
behavior but is likely to lead to a change in Training for safety in general aviation and air
behavior and the process of pilot decision taxi operations: Research needs and oppor-
making. tunities. NASA-Ames Research Center, Moffett
Field, CA. January 1998.
Assisting pilots with decision making elements Kahneman, D. & Tversky, A. (1984).
that are difficult for humans, may help them to Choices, values, and frames. American
avert potential incident. However, these meas- Psychologist, 39(4) 341-350.
HESSD ’98 106
Klein, G. (1993). Sources of error in Santa Monica, CA: Human Factors and
naturalistic decision making. In Proceedings of Ergonomics Society.
the Human Factors and Ergonomics Society Orasanu, J. & Fischer, U. (1997).
37th Annual Meeting, 1, 368-371. Santa Finding decisions in natural environments: The
Monica, CA: Human Factors and Ergonomics view from the cockpit. In C. Zsambok & G .
Society. Klein (Eds.) Naturalistic Decision Making. (pp.
Klein, G. (1997). The current status of 343-357). Hillsdale, NJ: Lawrence Erlbaum
the naturalistic decision making framework. In Associates.
R. Flin, E. Salas, M. Strub & L. Martin (Eds.) Payne, J.W., Bettman, J.R. & Johnson,
Decision Making Under Stress: Emerging E.J. (1988). Adaptive strategy selection in
Themes and Applications. Aldershot, UK: decision making. Journal of Experimental
Ashgate. Psychology: Learning, Memory and Cognition.
Klein, G., Orasanu, J., Calderwood, R. & 14 (3), 534-552.
Zsambok, C. (Eds.). Decision making in action: Pidgeon, N. (1988). Risk assessment and
Models and methods. Norwood, NJ: Ablex. accident analysis. In B. Rohrmann, L. Beach,
McKenna, F. (1985). Do safety measures C. Vlek & S. Watson (Eds.) Advances in
really work? An examination of risk homeostasis Decision Research (pp. 355-368). New York:
theory. Ergonomics, 28(2), 489-498. Elsevier Science Publishers.
Means, B., Salas, E., Crandall, B. & Rasmussen, J. (1985). The role of
Jacobs, T.O. (1993). Training decision makers hierarchical knowledge representation in
for the real world. In G. Klein, J. Orasanu, R. decision making and system management. IEEE
Calderwood, & C. Zsambok (Eds.), Decision Transactions on Systems, Man and Cybernetics,
making in action: Models and methods (pp. 51- 2, (SMC - 15) 234-243.
99). Norwood, NJ: Ablex. Reason, J. (1990). Human Error.
National Transportation Safety Board. Cambridge, UK: CUP.
(1994). A Review of Flightcrew-Involved, Schuch, H.P. (1992). The influence of
Major Accidents of U.S. Air Carriers, 1978 flight experience on midair collision risk
through 1990. (PB94-917001, NTSB/SS-94/ perception. Accident Analysis & Prevention,
01). Washington, DC: Author. 24(4), 655-660.
Norman, D.A. (1981). Categorization of Simon, H. (1957). Models of Man. New
action slips. Psychological Review, 88, 1-14. York: Wiley.
Norman, D.A. (1986). Cognitive Sitkin, S. (1992). Learning through
Engineering. In D.A. Norman & S.W. Draper failure: The strategy of small losses. Research
(Eds.), User Centered System Design (pp. 87- in Organizational Behavior, 14, 231-266.
124). Hillsdale, NJ: Lawrence Erlbaum Stokes, A.F., Kemper, K.L. & Marsh, R.
Associates. (1992). Time-stressed flight decision making:
Orasanu, J. & Connolly, T. (1993). The A study of expert and novice aviators (Tech.
reinvention of decision making. In G. Klein, J . Rep. No. 4). Arlington, VA: Office of Naval
Orasanu, R. Calderwood & C. Zsambok (Eds). Research.
Decision making in action: Models and methods Wiener, E.L. (1993). Intervention
(pp. 3-20). Norwood, NJ: Ablex. Strategies for the Management of Human Error
Orasanu, J., Dismukes, K. & Fischer, U . (NASA Contractor Report No. 4547). Moffett
(1993). Decision errors in the cockpit. In Field, CA: NASA Ames Research Center.
Proceedings of the Human Factors and Woods, D.D., Johannesen, L.J., Cook,
Ergonomics Society 37th Annual Meeting, 1, R.I. & Sarter, N.B. (1994). Behind Human
363-367. Santa Monica, CA: Human Factors Error: Cognitive Systems, Computers, And
and Ergonomics Society. Hindsight. Wright-Patterson Air Force Base,
Orasanu, J. & Strauch, B. (1994). Ohio: Crew Systems Ergonomics Information
Temporal factors in aviation decision making. Analysis Center.
In Proceedings of the Human Factors and Zsambok, C. & Klein, G. (1997).
Ergonomics Society Meeting (pp. 935-939). Naturalistic Decision Making. Hillsdale, NJ:
Lawrence Erlbaum Associates.
HESSD ’98 107
Notes
jo/lm, hew2, !/ /#