4/12/22

CONFIDENCE, CONSISTENCY,
CALIBRATION, & THE CONFIRMATION
BIAS
Decision Making
Robert Mauro, PhD
University of Oregon

Self-Test of Overconfidence
90% Confidence
Range
Low High
Martin Luther King’s age at death
Length of the Nile River
Number of countries that are members of OPEC
Number of books in the Old Testament
Diameter of the moon in miles
Weight of an empty Boeing 747
Year in which Wolfgang Amadeus Mozart was born
Gestation period (in days) of an Asian elephant
Air distance from London to Tokyo
Deepest (known) point in the ocean (in feet)

Self-Test of Overconfidence
Actual
Answer

Martin Luther King’s age at death (in years) 39

Length of the Nile River (in miles) 4187
Number of countries that are members of OPEC 13
Number of books in the Old Testament 39
Diameter of the moon in miles 2160
Weight of an empty Boeing 747 (in pounds) 390,000
Year in which Wolfgang Amadeus Mozart was born 1756
Gestation period (in days) of an Asian elephant 645
Air distance from London to Tokyo 5959
Deepest (known) point in the ocean (in feet) 36,198

1
84 VOLUME 12, NUMBER 3, JUNE 2003

class, we asked 141 students to tell

us how well they had done on an
exam, just before they walked out
of the classroom. We asked the re-
spondents to estimate their perfor-
mance and mastery of the course ma-
4/12/22
terial relative to the other students
taking the exam. We also asked
them to estimate their raw score on
the test.
Figure 1 presents the data from
the comparison questions. In the
figure, we have separated respon-
dents into four groups based on
their actual performance on the test,
from the bottom 25% of performers
to the top 25%. As the figure shows,
students in the bottom quartile
greatly overestimated their perfor-
mance on the test. Whereas their
performance actually put them in
the 12th percentile, they estimated Fig. 1. Perceived percentile rankings for mastery of course material and test perfor-
mance as a function of actual performance rank.
their mastery of the course mate-
rial to fall in the 60th percentile
and their test performance to fall
in the 57th. Figure 2 reveals a sim- solving ability in the lab (Haun,
ilar pattern in estimates of raw Zeringue, Leach, & Foley, 2000). THE DOUBLE CURSE
scores, with bottom performers This pattern even appears, un-
overestimating their performance checked, after participants are prom- People fail to recognize their

by roughly 30%.
This example is not an isolated
case. Participants taking tests in
their ability to think logically, to
write grammatically, and to spot
funny jokes tend to overestimate
their percentile ranking relative to
their peers by some 40 to 50 points,
thinking they are outperforming a
majority of their peers when, in
fact, they are the ones being outper-
formed (Kruger & Dunning, 1999).
This pattern also emerges in more
real-world settings: among debate
teams taking part in a college tour-
nament and hunters quizzed about
their knowledge of firearms just be-
fore the start of hunting season
(Ehrlinger, Johnson, Banner, Dun-
ning, & Kruger, 2003); among med-
ical residents evaluating their pa-
tient-interviewing skills (Hodges,
Regehr, & Martin, 2001); and
among medical lab technicians as-
sessing their knowledge of medical
terminology and everyday problem-
Overconfidence
ised up to $100 for accurate as- own incompetence because that in-
sessments of their performance competence carries with it a double
(Ehrlinger et al., 2003). curse. In many intellectual and so-
■ People generally have positive
self-images which leads them
to be overly confident in their
abilities.
■ Novices often don’t know
what they don’t know
■ Overestimate performance
■ Experts often don’t realize
that everyone doesn’t know
what they know
■ Overestimate performance of
Fig. 2. Perceived versus actual test score as a function of actual test performance.
others
Dunning, Johnson, Ehrlinger, & Kruger (2003)
Published by Blackwell Publishing Inc.

Overconfidence
Perceived Safety Perceived Skill
40 30

35
25

30

20
25
P e rc e n t

P e rc e n t

20 15

US US

15 Sw eden Sw eden
10

10

5
5

0 0
0

0
10

10
-20

-30

-40

-50

-60

-70

-80

-90

-20

-30

-40

-50

-60

-70

-80

-90
-10

-10
0-

0-
11

21

32

41

51

61

71

81

11

21

32

41

51

61

71

81
91

91

E s tim a te d P o s itio n (P e rc e n tile ) E s tim a te d P e rc e n tile

Everyone thinks that they are better than average drivers (Svenson, 1981)

Overconfidence
■ Bay of Pigs
■ 17 April 1961
■ 1,400 paramilitaries, divided into five
infantry battalions and one
paratrooper battalion,
■ Launched from Guatemala and
Nicaragua by boat.
■ Surrendered on 20 April 1961
■ Cuban exiles who opposed Fidel
Castro's Cuban Revolution.
■ Covertly financed and directed by the
U.S. government.

2
 4/12/22

Consistency & Commitment

■ A panoply of effects that result from attempting to main consistency with previous plans, behaviors,
intentions, attitudes and beliefs.

– Sunk cost phenomena

■ Continuing to invest time, effort, wealth in a previously chosen option when an explicitly better option is
available.
– Plan Continuation Error
■ Continuing with a previously accepted plan despite evidence that the plan will not succeed when other
better options are available
– Consistency effects
■ Change attitudes, beliefs, behaviors to maintain consistency with actions
– Overconfidence
■ Consistent with a positive self image, people are frequently overconfident in their knowledge, abilities,
good fortune
– Confirmation Bias
■ Seeking information that supports previously held attitudes and beliefs rather than seeking all relevant
information.
– Motivated Cognition
■ Interpreting and weighting evidence to support previously held attitudes and beliefs

Plan Continuation Error

■ July 6, 2013,
■ Asiana Airlines flight 214
■ Boeing 777-200ER,
■ on approach to San Francisco
International Airport (SFO) runway 28L
when it struck a seawall
■ Three of the 291 passengers were fatally
injured; 40 passengers, 8 of the 12
flight attendants, and 1 of the 4 flight
crewmembers received serious injuries.
■ The other 248 passengers, 4 flight
attendants, and 3 flight crewmembers
received minor injuries or were not
injured. The airplane was destroyed.

Consistency: When Prophecy Fails

■ When Prophecy Fails: A Social and
Psychological Study of a Modern Group
That Predicted the Destruction of the
World is a classic work of social
psychology by Leon Festinger, Henry
Riecken, and Stanley Schachter
published in 1956, which studied a small
UFO religion in Chicago called the
Seekers that believed in an imminent
apocalypse and its coping mechanisms
after the event did not occur. Festinger's
theory of cognitive dissonance can
account for the psychological
consequences of disconfirmed
expectations.

3
 4/12/22

Consistency: Cognitive Dissonance

Festinger & Carlsmith, (1959)

10

Cognitive Dissonance Phenomena (some)

■ Severe Initiation
– Compared to mild
initiation, observe
increased liking for
group following severe
initiation
■ Counter-attitudinal advocacy
– After being subtly
manipulated into
advocating for disliked
position, individuals
demonstrate increased
liking for that position
– Effect increased if
advocacy is public;
decreased if there is
obviously forced
compliance

11

Manufactured Attitudes
■ Self Perception Theory
– No attitude
– Asked to decide
– Examine behavior
– Infer attitude from behavior
– Act accordingly
■ Symbolic Attitudes
– No attitude
– Asked to decide
– Examine perceived attitudes of
reference group
– Infer attitude from reference
group
– Act accordingly

12

4
 4/12/22

General Consistency Diagram

(Balance Theory)
Out of Balance In Balance

Object Object

- + + +
Person Person Person Person
1 2 1 2

+ +

13

General Consistency Diagram Extensions

■ Number of nodes
– People Object
– Objects
■ People, attitudes
?
■ Types of relations
– Possession
– Symmetry
■ Importance of relations Person Person
■ Degree of consistency 1 2

14

Confirmation bias

“If a card has a vowel on one side, then it has an even number on the other
side."

Which of the cards do you need to turn over to see if the statement is true or
false?

15

5
 4/12/22

Confirmation Bias: N-Rays

■ Reported by well-respected French
physicist Prosper-René Blondlot in
1903
■ Confirmed by approximately 120 other
scientists in 300 published articles.
■ N-rays detected emanating from most
substances, including the human body
(but not green wood and some treated
metals).
■ But many scientists were unable to
reproduce the effect.

16

Confirmation Bias: N-Rays

■ Robert W. Wood went to Blondlot’s
laboratory
■ Surreptitiously altered laboratory
equipment while in the dark (required
to observe phenomenon) to interfere
with the ”N-Rays.”
■ Blondlot continued to report seeing
unchanged results, demonstrating
that the phenomenon was illusory, a
result of perceptual effects; published
in Nature.
■ Morals: Scientists are human; science
is a methodology (not a set of facts)
that is self-correcting.

Robert W. Wood Prosper-René Blondlot

17

© 1980 SCIENTIFIC AMERICAN, INC

This content downloaded from

184.171.112.49 on Tue, 12 Apr 2022 17:39:17 UTC
All use subject to https://about.jstor.org/terms

Summary: Problems
■ Attention
– Seek confirmation; don’t attend to
• Attention
disconfirmatory input that is available.
■ Perception • Perception
– Misperceive available information;
■ Memory • Memory
– Fail to recall inconsistent information;
doesn’t fit mental schemas
■ Motivation • Motivation
– Motivated cognition; dissonance is
unpleasant
• Information Processing
■ Information Processing
– Information that is consistent with prior
beliefs is overweighted • Decision Making
– Sunk costs – commitment effects

18

6
 4/12/22

Summary: Solutions
■ Obtain information
– Seek feedback
– Realize limits of available information
– Attempt to disconfirm (science)
– Validate & verify
– Admit personal limitations
■ Seek alternative perspectives
– Agree on (true) goals, not necessarily methods
■ Knowledge is power
– Implement prophylactic and recovery procedures
■ Confess error, correct course

19

Don’t Isolate
■ For mental health
– Delusions
■ For making good individual decisions
– Disconfirming evidence
– Additional options
■ For making good group decisions
– Disconfirming evidence
– Alternative viewpoints
– Additional options
Kennedy and the Cuban Missile Crisis

20

Don’t Isolate
■ For mental health
– Delusions
■ For making good individual decisions
– Disconfirming evidence
– Additional options
■ For making good group decisions
– Disconfirming evidence
– Alternative viewpoints
– Additional options

21

7
 4/12/22

End

22