aI | Expert | Competence (Human Resources)

Competence in Experts: The Role of Task Characteristics

James Shanteau

Kansas State University Department of Psychology Bluemont Hall Manhattan, Kansas 66506

Running Head: Competence in Experts

From: Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53, 252266.

the discrepancy in the conclusions drawn from the two literatures appears to be a function of the different domains studied. (2) the psychological traits associated with experts. the theory has implications both for the analysis of experts and for the design and use of expert systems. the Theory of Expert Competence. whereas cognitive investigators have concentrated on experts in static domains (e.. to the biasing effects of judgmental heuristics. . physics). Judgment and decision research has shown that experts make flawed decisions due. the proposed theory contains several new components. An alternative view developed here. and (5) a task with suitable characteristics. Decision researchers have looked primarily at experts in behavioral domains (e. the theory assumes competence depends on five components: (1) a sufficient knowledge of the domain. Although similar to approaches such as Cognitive Continuum Theory.Abstract The previous literature on experts presents two contrasting views. In particular. Insufficient attention has been paid to task and domain characteristics in prior research.g. clinical psychology).g.. views experts as competent and different from novices in nearly every aspect of cognitive functioning. Cognitive science research. (4) the ability to use appropriate decision strategies. in contrast. but incomplete. (3) the cognitive skills necessary to make tough decisions. suggests that both analyses are correct. In addition. Thus. The latter is the focus of this paper. in part.

such as . 19 B.Competence in Experts: The Role of Task Characteristics “Experto credito” (Virgil. The final section considers implications of these observations for analyses of experts and for design of expert systems. Psychometric analyses revealed that judgments by experts are lacking in validity (Oskamp. 1965) and reliability (Trumbo. 1966). for more a detailed account.C. The following provides a brief overview of the literature. Adams. Second. the topic of experts has been of interest to writers over the centuries. 1877). with an emphasis on task characteristics. BACKGROUND Decision-Making Research With some exceptions. Deficiencies have been reported in nearly every type of decision analysis. Similar findings have been reported for a variety of experts. “No lesson seems to be so deeply inculcated by the experience of life as that you never should trust experts” (Lord Salisbury. The fourth section compares the proposed approach to Cognitive Continuum Theory (Hammond. These quotes illustrate two facts: First. The purpose of this proposal is to explore expertise from an alternative perspective and to use that perspective to provide new insights on the competence of experts. the judgment and decision making literature paints a dismal picture of the ability of experts. 1962).). there is considerable disagreement about the competence of experts. & Schipper. Milner. see Shanteau (1989).” The next develops a new Theory of Expert Competence. The paper is organized into five sections. The second presents proposed definitions of “expert” and “competence. The first section summarizes the prevailing views of experts in the judgment/decision making and cognitive science literatures.

1970). 1981) and coherence (Chan. 1982). 1974). 1975). Fryback. & Coles. and clinical psychologists (Goldberg. 1970). medical doctors (Einhorn. meteorologists (Williams. Moreover.” Thus. which in turn lead to systematic biases (Kahneman. 1982). 1951). & Thornbury. Fischhoff. 1974). experts should use all relevant information. 1976). Comparable deficiencies have been reported for various experts. Moreover. 1954). expert judgments lack the complexity expected from superior decision makers.Shanteau agricultural judges (Foss. 1977). Wright. and nurses (Grier. including physicians (DeSmet. clinical psychologists (Oskamp. however. Various studies. presumably. & Phillips. 1982). 1954). have reported that models of experts reflect surprisingly low information use (Goldberg. parole board members (Carroll & Payne. clinical psychologists (Lichtenstein & Fischhoff. & Tversky. the evidence suggests their judgments can be described by simple linear models. 1976). A common explanation for this low level of performance is that experts often rely on heuristics. experts apparently are unaware of their shortcomings (Meehl. forecasters (Lichtenstein. 1975). experts apparently are . Under use of available information has been reported for criminal court judges (Ebbesen & Konecni. 1979). medical pathologists (Einhorn. Slovic. 1962). 1975). Dawes and Corrigan (1974) concluded “the whole trick is to decide what variables to look at and then to know how to add. Because of heuristic decision strategies. After reviewing studies from a variety of domains. Not only do experts make use of little information. and court judges (Ebbesen & Konecni. the experience of experts is not related to their judging ability (Meehl. Studies of probabilistic judgments have reported deficiencies in calibra-tion (Christensen-Szalanski & Bushyhead. Another approach to analyzing expert judgment is to examine the amount of 2 information used in making decisions.

1965) and to have better memory for chess positions (Chase & Simon. Bushyhead. 1981). 1971). 1984). 1983). Phelps and Shanteau (1978) found that livestock judges were capable of using large amounts of information. Studies within this tradition have shown expert superiority over novices in nearly every aspect of cognitive functioning. "Numerous studies show that intelligent people have great difficulty judging probabilities. These examples of strong performance. Experts in physics. 1983). Diehr. have been largely ignored in the literature (Christensen-Szalanski & Beach. Research with auditors has produced many examples of competent performance (Ashton. and Wood (1982) to make good decisions. making predictions. previous decision making research presents a discouraging view of the abilities of experts. Fischhoff. attempting to cope with uncertainty. however. mathematics.Shanteau limited in the same way as naive subjects (Tversky & Kahneman. 3 There have been some exceptions to the general trend of expert incompetence. Frequently these difficulties can be traced to the use of judgmental heuristics" (Slovic. . 1973). and otherwise. from memory and learning to problem solving and reasoning (Anderson. for instance. Weather forecasters’ predictions were reported by Murphy and Winkler (1977) to be well calibrated. Altogether. 1985).” Cognitive-Science Research A quite different view of experts has emerged from research in cognitive psychology. & Lichtenstein. Medical doctors were observed by Christensen-Szalanski. and computer programming reveal similar superior skills (Mayer. Kahneman (1991) stated a widely held conclusion: “There is much evidence that experts are not immune to the cognitive illusions that affect other people. Chess masters. have been found to perceive patterns of play more effectively (deGroot.

For instance. Although other methods have been proposed (Hoffman. expertise is acquired through stages of development. 4 Because of their special abilities. in contrast. & Simon. expertise is domain specific. 1977). Third. Expert physicists. First. reason forward using stored "functional units" from the givens to the goal (Larkin. 1990). rely on controlled processes. 1990). expert processes are reflected by and can be studied through verbal protocols. expertise produces more efficient approaches to thinking about problem solving and decision making (Anderson. Fourth. qualitatively rich accounts of an expert's reasoning processes become accessible (Ericsson & Simon. 1980). 1987). experts use different thinking strategies. The next is the "associative stage. The special skills of an expert are diminished outside his/her area of expertise: "Chess experts do not appear to be better thinkers for all their genius in chess" (Anderson. 1980). 1987). McDermott. the thinking of experts is "domain adapted" (Slatter. Apparently. Therefore. Novices. . novices have been found to reason backwards from the unknowns to the givens in solving physics problems." where specific facts are memorized to perform the task.Shanteau Several themes have emerged from this body of research. Simon. According to Fitts and Posner (1967). The last is the "autonomous stage. more like deductive reasoning (Larkin. which are linear and sequential. Second." where the skills become practiced and rapid. somewhat akin to the mental development of children. These automated processes generally operate in parallel and function somewhat like visual perception or pattern recognition. the thinking of experts is more automated (Shiffrin & Schneider. the first is the "cognitive stage. in contrast." where connections between successful elements are strengthened. 1979). By asking experts to think aloud.

Novices are intermediate in skill and knowledge. In my research. There are almost as many definitions of “expert” as there are researchers who study them (Hoffman. 1988). competent. there are some who are considered by their peers to be best at what they do. Chi. Unfortunately. the cognitive science view is that experts within their domains are skilled. Burton. Dreyfus & Dreyfus. 1987). This skill can be tapped to provide a sufficient basis for building an expert system (Coombs. a naive decision maker has little or no skill in making decisions in a specific area.Shanteau protocol analyses are commonly used to provide the raw data for building expert systems (Slatter. SOME DEFINITIONS Before trying to resolve these different views of expertise. My suggestion (Shanteau. Glaser. Moreover. they frequently have studied for years and may even work at subexpert levels. For instance. undergraduate students generally are naive about the kinds of decisions made by experts. In some domains this is reflected by official recognition or job titles. novices lack one or more of the abilities needed to . numerous hierarchies have been proposed to describe lower and intermediate levels of expertise (e. Benner. & Akerstrom. However. 1981. 1984. efforts to provide objective definitions have proved elusive..g. In every field. experts are operationally defined as those who have been recognized within their profession as having the necessary skills and abilities to perform at the highest level. 1986). and think in qualitatively different ways than novices 5 (Anderson. The definition of expert is an obvious precondition to any analysis of expertise. 1984). it is necessary to define several key terms. Shadbolt. 1987a) is to let those in a domain define the experts. 1991). In contrast. In others. it comes from consensual acclamation. Shanteau. & Farr. In total.

this approach has been used successfully in many studies of experts. the primary research question has been: What factors lead experts to do well and what factors lead them to do poorly? My view is that experts are neither as deficient as suggested in the decision making literature nor as unique as implied by the cognitive science perspective. And these opinions can shift over time. 6 Further. because circumstances change or experts adopt new standards. The problem is that standards are defined from subjective opinions of experts (not the other way around). many “expert-novice” studies are better described as “novice-naive” studies. psychological traits. THEORY OF EXPERT COMPETENCE In my research. according to the Theory of Expert Competence. Typically. external standards are seldom available for expert domains -that’s why experts were needed in the first place. From this view. Thus. advanced (graduate) students are novices in making skilled decisions. the skills and abilities that emerge (or don't emerge) in experts depend on five factors: domain knowledge.Shanteau function as experts. it appears that at least some functioning expert systems mistakenly have been based on novices instead of experts (Levitt. Unfortunately. . much research on “experts” in fact has used novices. This would be easy if standards existed for determining what is a good decision and what is not. they can be elusive and subject to debate. 1991). Although there is some danger of circularity. My resolution is to rely on the views of acknowledged experts about what is a competent decision and what is not. These opinions are backed up by professional guidelines and commonly accepted standards in the field. Even where objective criteria exist. Defining the quality or competence of a decision is also difficult. Instead.

The traits are part of a decision style found in many experts. 1988). but also insights gained from experience in working on real problems. 1984). however. I now believe they should be considered separately. In previous papers (e. these skills were combined with the psychological traits under the label “psychological characteristics”. Shanteau. maybe even as much as experts. excellent communication skills. 1988). In other respects. an overview of the key ideas are presented here. their knowledge is generally accessed through stories about past cases (Shanteau. I argued that experts often display a common set of psychological traits. the ability to adapt to new situations. and task characteristics. 7 Having an adequate grasp of domain knowledge is obviously a prerequisite for being an expert. Many novices know a great deal.. In short. As such.g. (Previously.) Some of these skills exhibited by experts .. Shanteau. These traits include strong self-confidence. Although knowledge of the domain is necessary. Based on conversations with experts. This represents not only textbook knowledge. These reflect what Goffman (1959) describes as “self presentation” -. decision strategies. I described various cognitive skills commonly possessed by experts. These anecdotal accounts appear to provide both a mnemonic to remember and a convenient way to organize vast amounts of information.g. The specifics of this theory are described elsewhere (Shanteau.Shanteau cognitive skills. and a clear sense of responsibility. it is necessary to act like one. they lack what it takes to behave as an expert. it is not sufficient for expertise.the creation and maintenance of a public image. they are consistent with efforts to build expert systems through “case-based reasoning” (Kolodner. in press). In prior papers (e. 1984). to be accepted as an expert.

8 My observation of experts also has revealed the use of a variety of formal and informal decision strategies (Shanteau.g. Although many strategies are unique to given domains. and strategies. there are several that are widely used.1 the literature in each field is clear about the level of competence. experts seem incapable of performing much above the level of novices. Except for nurses. the ability to identify exceptions to rules. a sense of what is relevant.. They include making use of dynamic feedback. but often overlooked.Shanteau are highly developed attention abilities. 1989). the competence observed in an expert depends on the task. and prethinking solutions to tough situations. relying on decision aids. traits.. These strategies help systematize decision making and have the effect of helping experts overcome cognitive limitations. There is a final factor which is crucial. and the capability to work effectively under stress. physicians. The question is: What do the tasks on each side have in common? . The left side of the table lists judgment domains in which competent performance has been reported in the literature. clinical psychologists (Oskamp. In other tasks. e. Task Characteristics The relationship between domains and decision performance is illustrated in Table 1. There are some tasks that experts do well. even in the face of considerable difficulty. decomposing complex decision problems. The remainder of this section will be devoted to discussion of task variables. 1977).g. Even with the appropriate knowledge. The task characteristics determine whether it is possible for experts to behave competently or not. weather forecasters (Murphy & Winkler. 1962). and auditors (listed on both sides). e. The right side lists domains in which deficient performance has been reported. skills.

consequently.similar conditions arise from time to time. are more changeable -. Another difference is that left-side tasks tend to be repetitive -. With right-side domains. there appear to be fewer chances to learn. they do less well. the stimuli are dynamic and generally involve human behavior. experts are held to higher standards of performance. With domains on the left side. That is. Other insights have been offered about this table. in contrast. an expert can better his/her decisions. there are more chances to learn from past decisions. therefore. An insightful observation was offered by Gigerenzer (1989). the experts are being asked to evaluate and make decisions about stimuli that are 9 relatively constant.Shanteau My original answer (see Table 2) was that domains with competent performance involve static objects or things (Shanteau. however. weather forecasters moved from relying on feelings and guesswork to using detailed climatic information. As understanding of meteorology developed. Right-side tasks. Based on previous successes and failures. 1987b). Clinical psychologists. many unstructured right-side tasks performed .conditions differ frequently. Dawes also noted that the competence expected by clients (or the public) varies for the two sides. Dawes (1987) observed that predictability is different for the two sides: human behavior is inherently less predictable than physical stimuli. for example. With the advancement of science. are expected to be always correct. Where poor performance is observed. Because experts are being asked to evaluate and decide about what is in effect a moving target. judges are faced with a stationary target. who noted that historically most left-side tasks began as right-side tasks. This has implications for the opportunity to receive and respond to feedback. for example. whereas weather forecasters are allowed to make occasional mistakes. Paradoxically in the less predictable behavioral domains.

1975). any conclusions must take task into account. This distinction is supported by the widespread presence of decision aids for left-side tasks (Shanteau. That means experts cannot be described generically. the lesser the competence seen in experts. The more a task contains left-side characteristics. Theoretical Hypotheses Although the ideas behind the theory are primarily qualitative. Sometimes these aids are formal. the more right-side characteristics a task contains. As Edwards and von Winterfeldt (1986) point out. The first hypothesis relates to task characteristics which lead to compe-tent performance. . 1977) to help in learning and to prevent hindsight biases (Fischhoff. their competence depends on the task characteristics. Instead. These concern conditions under which experts will or will not make competent decisions. the aids are informal. the “unaided expert may be an oxymoron since competent experts will adopt whatever aids are needed to assist their decision making. my view is that expert performance is neither uniformly good nor bad. Three hypotheses are described here. 1989). it is possible to propose several testable hypotheses. Rather. At other times. 1984). That means the performance by experts in a given situation should be a function of the number of task characteristics associated with each side. such as the written records kept by livestock judges (Phelps. Conversely.” In summary. The same expert may behave competently in some settings and not in others. such as the “soil triangle” used by agronomy judges to define soil classification boundaries (Gaeth & Shanteau. the greater the competence that should be seen in experts.Shanteau 10 by less-competent experts eventually become structured left-side tasks performed by competent experts.

nonsequential. COGNITIVE CONTINUUM THEORY The left-side.Shanteau The second hypothesis involves the decision strategies associated with different tasks. That is. right-side strategies. On the other hand. According to the theory. even when there are no objectively verifiable correct answers. We will refer to this composite as QUASI-RATIONAL thought. use of left-side strategies. That means it should be possible to prescriptively improve expert competence. should be more frequent in right-side tasks. 340). should be greater for leftside tasks. sequential.” They go on to define a middle category: “Most instances of thinking will have both intuitive and analytic components. Performance in right-side tasks should improve if they can be made more like left-side tasks. The final hypothesis involves prescriptive procedures for producing more (or less) competent decisions. right-side distinction has parallels to Cognitive Continuum Theory (CCT) developed by Hammond (1966) from the ideas of Brunswik (1956). the thinker can report no more than the outcome of his thinking (p.. Hammond and Brehmer (1973) define the continuum as follows: “The ANALYTICAL end of the continuum is characterized by a form of thinking that is explicit.. is implicit.. there should be a direct relation between the strategies used and the type of task. Thus. INTUITIVE thinking. (which) can be reported by the thinker. such as making unaided decisions without use of decomposition or feedback.. and recoverable. such as 11 problem decomposition with decision aids and feedback. This might be done either by changing task characteristics or by encouraging the use of left-side strategies. and nonrecoverable. Usually.. it consists of a series of steps that transform information according to certain rules.” . on the other hand.

and low conscious awareness. task decomposition. the . First. with no classifiable middle ground. Grassia. The former includes reliably measured cues. et al (1987) distinguish analysis-inducing and intuition-inducing task characteristics. in contrast. Hamm. whether expert or naive. presence of organizing principles. In contrast. lack of organizing principles. The origin of the continuum comes from Brunswik’s (1956) analysis of general perceptual processes. slow processing. is based on a conceptualization of competence that goes from low to high as a function of task characteristics. et al (1987) argue that accuracy of judgment will be greatest when there is a correspondence between task properties and cognitive properties: “At some point on the cognitive continuum. In comparison. the present approach differs from CCT in at least three respects. The latter includes unreliably measured cues.Shanteau 12 In applying CCT to expertise. and simultaneous display of cues. the theory proposed here has competence and incompetence of expert performance at the two extremes. In contrast. the continuum in CCT represents cognitive style. Hammond. These task characteristics are consistent with the differences outlined in Table 2.” The present theory. with analytic and intuitive thinking at the extremes and quasirationality between. and sequential display of cues. analysis has high cognitive control. nondecomposable task. Second. performance will be best and accuracy will fall off as the expert becomes either more analytic or more intuitive. It is unclear how to make the connection between these two continua. However. Intuition is seen as having little cognitive control. CCT is based on a general approach to human judgment applicable to all people. Hammond. and Pearson (1987) identify several cognitive properties that distinguish between intuition and analysis. and high awareness. rapid processing.2 Third. Hammond.

such as MYCIN. other aspects of expertise have been overlooked by builders of expert systems. There are several potentially valuable systems. getting experts to interact with expert systems has often proved difficult (Michie. having the facts and relevant experience are essential for any expert (Naylor. 1982). It is unclear how to make extensions to naive subjects. that are either unused or misused by the very people the systems were designed to help (Ham. 1987). in part because of the lack of cooperativeness of experts (Rose. 1987). 1988). expert systems are increasingly being proposed to aid or even replace skilled decision makers. The analyses of experts here may contribute to a greater understanding of when and where expert systems are likely to be useful.g.. there has been a debate whether computer systems can mimic experts (e. 1984). Slatter. Domain knowledge and experience are clearly necessary for expertise. However. since they are incapable of doing or even understanding most expert tasks." Some argue that eventually expert systems will provide "replacements for humans" (Cebrzynski. 1986. According to Kolodner (1984). 1988). Many investigators see great potential for expert systems (Barrett & Beerel. Nonetheless.Shanteau 13 present analysis is concerned only with experts and the tasks they perform. In other cases. extended efforts to develop expert systems had to be abandoned. 1988. although others question whether that potential can ever be realized (Dreyfus & Dreyfus. the goal is to build systems which "contain all or most of the compiled knowledge an expert has. 1987). Haugeland. IMPLICATIONS FOR EXPERT SYSTEMS3 Using techniques from artificial intelligence. Graubard. By concentrating on knowledge and production rules. . 1985). At the same time. knowledge is not sufficient for expertise.

are rigid. and use of expert systems. and strategies such as dynamic feedback should be incorporated into expert systems. greater attention should be paid to alternative methods for analyzing experts. However. The traditional expert systems may be adequate for the former. not the other way around. Can an expert system be built that incorporates the present approach? Not enough is known yet to answer this question. First. unfortunately. Third. Systems should be as flexible as the experts they are designed to simulate. at least. not as something to be defined within the constraints of available hardware and software. different types of expert systems may be needed to reflect left-side and right-side expertise. at best verbal protocols are inefficient and at worst they may be misleading for representing expertise (Hoffman. Lastly. . & Uppuluri. The difficult cases are what distinguish experts from those less skilled and the present analyses suggest how competent experts deal with these cases. Instead. seem necessary to build such a system. such as those used in judgment and decision making research. Renn. 1987). Traits such as communication ability.Shanteau I believe the psychological insights described may improve the design. these systems generally work best on wellstructured problems (Mumpower. Second. Verbal protocols typically are used to capture the information needed to build an expert system. Phillips. Expert systems should reflect the physical and psychological worlds of experts. skills. experts cannot be expected to explain everything about what they do. However. 1989) and most systems. 14 construction. expertise must be looked at from the perspective of experts. and strategies of human experts when building computer systems. But the following would. 1987). skills such as identifying exceptions. as suggested by Table 1. Rigidity is a characteristic of novices (Shanteau. more emphasis should be placed on the traits.

et al.. Payne. psychological traits. The present research has supplied some tentative answers that hopefully will stimulate further research. Howell & Kerkar. Beach. 1990. In contrast. 1982. Keller. subjects often invent shortterm decision strategies as they go along in an experimental task. decision strategies. von Winterfeldt & Edwards. The importance of understanding task characteristics has been stressed repeatedly in judgment and decision research (Hammond. 1982. CONCLUDING COMMENTS Five factors associated with the competence of experts have been identified: domain knowledge. The competence seen in experts depends on having stable strategies developed in response to their environment. There are many questions remaining to be answered about the effect of task variables on experts.Shanteau linear judgment models may be better suited for the latter. and task characteristics. 1986) have focused primarily on contingencies in ongoing decision behavior.g. 1985. Thus. . the ability of linear approximations to provide good fits to noisy data is well established (Dawes & Corrigan. As Payne (1982) argues. The 15 previous analyses (e. cognitive skills. 1974). the present analysis highlights the connection between task characteristics and asymptotic expert behavior. 1987). The stress here was on the last factor. the concern here is with the long-term instead of short-term influence of task characteristics.

G. W. Chichester. 16 Ashton. evidence and implications. J. In J. NJ: Erlbaum. Anderson. P. (1982). K. G. & Simon. (1983).). R. Perception in chess. Berkeley: University of California Press. Beach.. From novice to expert: Excellence and power in clinical nursing practice. Expert systems are seen as replacements for humans. (1984). Payne (Eds. Chase. L. (1990). E. B. H. Carroll & J. W. A. W. Marketing News. H. 63. (1988). 1. C. Brunswik. Cognitive Psychology. 6). Chan. The psychology of parole decision processes: A joint application of attribution theory and information-processing psychology. Vancouver: Canadian Certified General Accountants. (1976). S. February 27). NJ: Erlbaum. Cognitive Skills and their acquisition. G. M. 428-444. J. Perception and the representative design of psychological experiments (2nd ed. Cognition and social psychology (pp. (1990). L.). MA: Addison-Wesley.. p. J. (1981). (1973). S. Hillsdale. Benner. (1987. R. Cognitive psychology and its implications (3rd ed. Hillsdale. A. 4. New York: W. (1956). Cebrzynski. Chichester. Expert judgments made under uncertainty: Some evidence and suggestions.). Freeman and Company. (Monograph No. 55-81. Carroll. Research in audit decision making: Rationale. Barrett. S. J. R.: Wiley. 13-32). Social Science Quarterly. & Beerel.. Reading. . H.: Halsted. Expert systems in business: A practical approach. & Payne.Shanteau References Anderson. R. Image theory: Decision making in personal and organizational contexts. U.

R. American Psychologist. (1975). (1984). H.. (1974). Diehr. T. D. The citation bias: Fad and fashion in the judgment and decision making literature.. Medical Decision Making. Physician's use of probabilistic information in a real clinical setting. Bushyhead. Dreyfus. Fryback. Dawes. A. 132. D. Hillsdale. V. London: Academic Press. Mind over machine. Linear models in decision making. Personal communication. B. Glaser. & Konecni. J. J.. B. Two studies of good clinical judgment.. (1986)... deGroot. 95-106. (1978)... & Beach. (1965).. Psychological Bulletin. J. R. J. DeSmet. Christensen-Szalanski.Shanteau 17 Chi. E. M.. Christensen-Szalanski. R. A. 2. M. 805-821. W. 7. 39.. E. (1988). G.. & Bushyhead. 59(2). & Farr. The nature of expertise. 81. (1984). 275-284. R. J. J. Journal of Experimental Psychology: Human Perception and Performance. A second look at the utility of radiographic skull examination for trauma. N. W. (1981). American Journal of Radiology. L. J. & Wood. R. 928-935. R. M. P.. New York: The Free Press. Developments in expert systems. & Corrigan. J. & Dreyfus. J. (1987. 75-78. Southern California Law Review. J. Christensen-Szalanski. (1982). S.: Erlbaum. 32. L. . M. J. Journal of Personality and Social Psychology. 95-99. D. Coombs. B. Thought and choice in chess. November). & Thornbury. J. M. J. H. (1986). Dawes. & von Winterfeldt. Decision making and information integration in the courts: The setting of bail. H. The Hague: Mouton. Ebbesen. A. Edwards. On cognitive illusions and their implications. J. J. 401-451.

Goffman. & Shanteau.. plus some evidence. (1967). Ham. Decision making about patient care. Journal of Applied Psychology. Nursing Research(25). Journal of Experimental Psychology: Human Perception and Performance. Organizational Behavior and Human Performance. Fischhoff. Human performance. & Polson. G. The presentation of self in everyday life. 800-802. R. J. Hindsight • foresight: The effect of outcome knowledge on. Man vs. (1984). W. R. S. Psychological Review. (1974). Playing by the rules. M. (1984.Shanteau 18 Einhorn. 562-571. 105-110. Testing the accuracy of field textures. (1975). 59. (1975). May). model of man: A rationale. Wright. H. PC World. M. B. H. . Foss. 73(422432). 33. & Coles. I. (1988). Personal communication. January). K. real foundations. R. & Simon. Expert judgment: Some necessary conditions and an example. Cambridge. J. E.. Garden City: Doubleday-Anchor. 215-251. Gigerenzer. J. Grier. E. L. 34-41. A. Psychological Bulletin. P. CA: Brooks Cole.. (1976). R. M. H. Gaeth. 39. Belmont. Verbal reports as data. MA: MIT Press. 1. Fitts. The artificial intelligence debate: False starts. Soil Science Society of America Proceedings. (1980). M. (1970). p. (1989. Graubard. Reducing the influence of irrelevant information on experienced decision makers. G. 263-282. 288-299. 87. Ericsson. (1959). R.. for a method of improving clinical inferences.. Goldberg.

& Winston. Hammond (Eds. (1991). Kahneman. & Akerstrom. (1985). Man. and Winston. R.. theory. Unpublished manuscript. Psychological Science. D. (1991). (1987).. Cambridge. MA: MIT Press. Kahneman. R.. A. Slovic. Hamm. R. (1973). J. Probabilistic functionalism: Egon Brunswik's integration of the history. & Tversky.. Summer). Grassia. Judgment under uncertainty: Heuristics and biases. SMC17(5). R. Rinehart. 2(3). Shadbolt. Hoffman. & Pearson. Shanteau. Human judgment and social interactions New York: Holt. Haugeland. R. In L. Direct comparison of the efficacy of intuitive and analytic cognition in expert judgment.. Rappoport & D. and method of psychology. K. Quasi-rationality and distrust: Implications for international conflict. N. R. Judgment and decision making: A personal view. Rineholt. The cognitive psychology of expertise: A review of theories and findings. New York: Cambridge University Press. The problem of extracting the knowledge of experts from the perspective of experimental psychology. & Kerkar. & Brehmer. Hoffman. J. p. (1966). 30. T. Howell. (1982). A.). In K. Organizational Behavior and Human Performance. . M. and Cybernetics. K.. Hammond. M. W.). K.. B. Hammond. A test of task influence in uncertainty measurement. Summers (Eds. R. R. (1982). Adelphi University..Shanteau 19 Hammond.. J. Artificial intelligence: The very idea. (Ed.). The psychology of Egon Brunswik New York: Holt. A. IEEE Transactions on Systems. R. AI Magazine. 5367. Burton. R. S. R. (1987. 142-145. P. D. 753-770... 365390.

Slovic. & Uppuluri.). Towards an understanding of the role of experience in the evolution from novice to expert. . London: Academic Press. B. Do those who know more also know more about how much they know? Organizational Behavioral and Human Performance. Developments in expert systems (pp. 159-183. Simon. Clinical versus statistical prediction: A theoretical analysis and a review of the evidence.). Fischhoff. J... 20 Kolodner. In M. (1982). (1990. (1987). (1980). Mumpower. D. Michie. (Ed. E. S. Lochhead & J. cognition. L. 738-751. L.Shanteau Keller. Cognitive process instruction. February). Expert judgment and expert systems. J.). V. In J. (1984). D. Levitt. L. & Phillips. (1982). J. S. B. problem solving. Information processing and science instruction. O. Renn. P.. (1983).. Calibration of probabilities: The state of the art to 1980. (1979). J. & Fischhoff. The effects of problem representation on the sure-thing and substitution principles. R. Clement (Eds. 1335-1342. Science. Meehl. (1977). Personal communication. (1954). Expert and novice performance in solving physics problems. New York: Gordon and Breach. R. Lichtenstein.. H. McDermott. Mayer. H. 95-116). Management Science. A. New York: Freeman.. Philadelphia: Franklin Institute Press. Thinking. & Simon. P. D. UK: Cambridge University Press. Tversky (Eds. Judgment under uncertainty: Heuristics and biases Cambridge. J. T. Larkin. Kahneman. Berlin: Springer-Verlag. (1985).).. P. Introductory readings in expert systems. Coombs (Eds. L. 31. Minneapolis: University of Minnesota Press. L. J. R. & A. Phillips.. Larkin. 208. D.. H. In D. R. 20. Lichtenstein.

& Winkler. Can weather forecasters formulate reliable forecasts of precipitation and temperature? National Weather Digest. J. L. F. Phelps. Expert judgment and expert systems (pp. 2-9. Berlin: Springer-Verlag.. 92. J. H. August 12). H. (1978). . Kansas State University. 14. 382402. Psychological Monographs. S. Seattle. (1982). Rose. (1977). In M. J. (1987b). C. Payne.. L. Shanteau. J. Uppuluri (Eds. R. 261-265. Oskamp. & V. Psychological Bulletin. El-Hawary (Ed. 29. New York: Halsted Press. Some unasked questions about the psychology of expert decision makers. 68. Acta Psychologica.). (1987). Build your own systems (2nd ed). J. Psychological characteristics of expert decision makers. R. 2. . Contingent decision behavior. Naylor. Oskamp. A. (1984). Journal of Consulting Psychology. (1988. D. Shanteau. Thinking machine: An 'electronic clone' of a skilled engineer is very hard to create. and Cybernetics. Mumpower.). 21. (1977) Expert livestock judgment: A descriptive analysis of the development of expertise. 289-304). R. What about experts? Paper presented at Judgment/Decision Making Society meeting. & Shanteau. Phelps.Shanteau 21 Murphy. L. In J. 209219. p. H. Overconfidence in case-study judgments. R. Man. Shanteau. The Wall Street Journal. Phillips. New York: IEEE. (1988). Shanteau. Renn. (1962). E. R. Doctoral Dissertation. 203-215. Livestock judges: How much information can an expert use? Organizational Behavior and Human Performance. (1987a). (1965). J. The relationship of clinical experience and training methods to several criteria of clinical prediction. S. O. Psychological characteristics and strategies of expert decision makers. 76. IEEE Conference on Systems.

: Ellis Horwood. New York: Cambridge University Press. & Schipper. (In press). C. Trumbo. 76. Tversky. (1971).'. P.).. von Winterfeldt. England: Cambridge University Press. The use of confidence factors in forecasting. and a general theory..). A. J. L. Shanteau. automatic attending.. M. Beach. Rohrmann. Social science and regulatory policy Berkeley: University of California Press. Cereal Science Today. (1986). (1951). G. & Schneider. & S. & Kahneman. Adams. Building expert systems: Cognitive emulation. 279-281. Reliability and accuracy in the inspection of hard red winter wheat. . Milner. The belief in the 'law of small numbers. Shiffrin. Williams. L. Psychological Bulletin. R. M. 84. Vlek. Slovic. (Ed. & Edwards. P. Advances in decision research (pp. E.. & Lichtenstein. Slatter. P. Noll (Eds.. C. B. Decision analysis and behavioral research. 203-215). D.Shanteau 22 Shanteau.. 105-110. 32. Controlled and automatic human information processing: II.). Expert decision making: Psychological explorations of competence. D.. B. Chichester. (1989). Fischhoff. D. Bulletin of the American Meteorological Society. Cambridge. 7. In B. Amsterdam: North Holland. (1985). R. (1987). (1962). R. Psychological characteristics and strategies of expert decision makers. S. 127-190. In R. Watson (Eds. Psychological Review. W.. J. W. Regulation of risk: A psychological perspective. Perceptual learning. (1977).

Table 1. _____________________________________________________________________ Domains with Good Performance Poor Performance _____________________________________________________________________ Weather Forecasters Livestock Judges Astronomers Test Pilots Soil Judges Chess Masters Physicists Mathematicians Accountants Grain Inspectors Photo Interpreters Insurance Analysts Nurses Physicians Auditors Clinical Psychologists Psychiatrists Astrologers Student Admissions Court Judges Behavioral Researchers Counselors Personnel Selectors Parole Officers Polygraph (Lie Detector) Judges Intelligence Analysts Stock Brokers Nurses Physicians Auditors .Domains in which good (left side) and poor (right side) expert performance have been observed. -.

_____________________________________________________________________ Characteristics Associated with Good Performance Poor Performance _____________________________________________________________________ Static Stimuli Dynamic (Changeable) Stimuli Decisions About Things Decisions About Behavior Experts Agree on Stimuli Experts Disagree on Stimuli More Predictable Problems Less Predictable Problems Some Errors Expected Few Errors Expected Repetitive Tasks Unique Tasks Feedback Available Feedback Unavailable Objective Analysis Available Subjective Analysis Only Problem Decomposable Problem Not Decomposable Decision Aids Common Decision Aids Rare . -.Task characteristics associated with good (left side) and poor (right side) performance in experts.Table 2.

Department of Psychology. and Gary Gaeth for their insightful contributions during the initial stages of this project. Richard Ettenson.Author Notes The research described in this paper was supported in part by grants from the Army Research Institute. and the Bureau of General Research at Kansas State University. Bluemont Hall. Risk. I wish to thank Geri Dino. The first draft this paper was prepared while the author was serving as Program Director of the Decision. the Department of Health and Human Services. Manhattan. Kansas State University. . and Management Science program of the National Science Foundation. Requests for reprints or for copies of the cited research can be obtained from: James Shanteau. KS 66506-5302.

3 Much of the material in this section is drawn from the discussion in Shanteau (1989). For instance. the different tasks performed by these experts lead to wide variation in performance level. 2 As argued in Shanteau (1989). livestock judges vary widely in their competence. Nonetheless.Footnotes 1 The domains placed on both sides reflect literatures where there is evidence of both competent and less-competent performance. there are no external criteria available to evaluate the quality of decisions made by livestock judges. . experts often work on tasks that do not have correct answers. A similar situation arises for test pilots and grain inspectors. Apparently.

Sign up to vote on this title
UsefulNot useful