SPE DISTINGUISHED LECTURER SERIES

is funded principally
through a grant of the

SPE FOUNDATION
The Society gratefully acknowledges
those companies that support the program
by allowing their professionals
to participate as Lecturers.
And special thanks to The American Institute of Mining, Metallurgical,
and Petroleum Engineers (AIME) for their contribution to the program.

Would You Know a Good
Decision if You Saw One?
Reidar B Bratvold
University of Adelaide
Australian School of Petroleum

Psychological and Judgmental
Aspects in Decision-Making

Acknowledgements
„

Prof. Steve Begg
– Australian School of Petroleum, University of
Adelaide

„

Dr. John Campbell
– DecisionsDecisions, Norman, OK

„

Dr. Michael Lee
– Department of Psychology, University of
Adelaide
3

Today’s Menu
„

Introduction

„

Motivation - A cult of mediocrity

„

How do people make decisions?

„

What is a good decision?

„

Heuristics and biases

„

Summary

4

‘Growth’ in Oil & Gas Index v DJIA
1400%

Dow Jones Industrial Average
1200%

Oil & Gas Index

1000%
800%
600%
400%
200%
0%
1985

1987

1989

1991

1993

1995

1997

1999

2001
5

Taking on a Cult of Mediocrity
„

“The last 10 years might be called ‘a
decade of unprofitable growth’ for many
upstream companies.”
» Ed Morrow, Independent Project Analysis (IPA)

– Based on the analysis of more than 1000 E&P
projects:
• 2/3 offshore, average CapEx = $670 million

– One in eight of all major offshore
developments in the last decade falls into the
‘disaster’ category.
– Record even worse for mega-projects
• CapEx of $1 billion or more

6
Source: UPSTREAM, 23 May 2003

Asset cost
growth

Operability index

-10%

-10%
-5%

Schedule slip

Best practice

95%

-5%
0%

0%
5%
5%
10%

10%

Industry average

Best practice
Best practice

75%

Industry average

Industry average

15%
20%

15%

BETTER

BETTER

Taking on a Cult of Mediocrity

55%

25%
20%
25%
30%

35%

Disasters

40%

Disasters

Disasters
35%

WORSE

WORSE

30%

7
Ref: Merrow, Ed: Taking on a Cult of Mediocrity, Upstream, 23 May 2003

What’s Amiss - 1?
Over-estimating returns and/or
under-estimating risk of loss

Uncertainty
8

What’s Amiss - 1?
Over-estimating returns and/or
under-estimating risk of loss
„ Uncertainty

around current “states-of-nature” (e.g.
OOIP) and uncertainty around predictions of future
“states-of-the-world” (e.g. oil price, recovery
efficiency).

„ People

tend to grossly underestimate uncertainties,
particularly
- the number of uncertain factors;
- the magnitude of the uncertainties;
- the complexity of the relationships among the
uncertainties, and
- their consequences

9

What’s Amiss - 2?
Flawed Decision-Making

Cognitive Illusions

10

What’s Amiss - 2?
Flawed Decision-Making
„ When

forecasting the outcomes of risky projects,
decision-makers all too easily fall victim to what
psychologists call the planning fallacy.

„ They

overestimate benefits and underestimate
costs.

„ They

spin scenarios of success while overlooking
the potential for mistakes and miscalculations.

„ As

a result, decision-makers pursue initiatives that
are unlikely to come in on budget or on time – or to
deliver the expected returns.
11

Visual Illusions
Which Edge, A or B, is Longer?
B
A

12

Visual Illusions - Which Figure is Tallest?

13

Linda Question
„

Linda is 31 years old, single, outspoken,
and very bright. She majored in
philosophy. As a student, she was deeply
concerned with issues of discrimination
and social justice, and also participated
in anti-nuclear demonstrations.

„

Please choose the most likely alternative:
a) Linda is a bank teller
b) Linda is a bank teller and is active in the
feminist movement
14

Discussion of Linda Question
„

Kahneman and Tversky (1982) asked this
question of 86 participants, with nearly 90%
choosing the second alternative (bank teller and
active in the feminist movement), even though this
is logically incorrect

bank tellers

bank tellers
who are not
feminists

feminists

feminist bank tellers

feminists
who are not
bank tellers

15

Discussion of Linda Question
Kahneman and Tversky (1982) asked this
question of 86 participants, with nearly 90%
choosing the second alternative (bank teller and
active in the feminist movement), even though this
Cognitive
biases
or illusions are similar to
is logically
incorrect
„

optical illusions in that the error remains
compelling
even when one is fully
aware of
bank tellers
feminists
its nature.
bank tellers
who are not
feminists

feminist bank tellers

feminists
who are not
bank tellers

16

How do People Make Decisions?
„

People have well defined preferences and they
take action to maximize some objective function:
– Maximize profit;
– Minimize cost;
– Maximize shareholder value.

„

Problems with this formulation.
– People have limited information processing
capabilities;
– Personal insights about uncertainty and preference
can be both limiting and misleading.
17

Perspectives in the Field of
Decision-Making

Normative
– how should people make
decisions
− principle of maximizing
expected value (utility)
− traditional decision theory
− rational decision-making

Descriptive
− how do people make
decisions
− limited cognitive capacity;
bounded rationality;
heuristics or rule-based;
satisficing; myopic
18

Good Decisions vs. Good Outcome
„

„

„

The weather report on the
evening news predicts a
warm, dry sunny day
tomorrow
When you get up and look
out the window in the
morning there’s not a cloud in
sight
You decide to leave your
umbrella at home and get
soaked in an unexpected
afternoon thundershower

Did you make a bad decision?

19

Decision Outcome is Influenced by Three
Factors
Doing
• implementation
and other factors
under your control

Deciding

Outcome

• the thinking
and decision
process

Chance
• uncontrollable
factors, luck

20

What Makes a Good Decision?
Russian Roulette
„ $10 million offered
„ One loaded chamber, 5 empty
„ Six possible outcomes of equal
probabilities
„

„

Is the player who survives a
good decision-maker, …

„

or is he a Lucky-Fool?

If outcome is used to judge the quality of a
decision, this person should be used as a role
21
model for decision-makers!

A Good Outcome is NOT Synonymous with a
Good Process!
Outcome

Good

Bad

Good

Justice

Bad Break

Bad

Dumb Luck

Justice

Promoted

Fired

Process Used to
Make the Decisions

Possible outcome for
the decision-maker

22

A Good Outcome is NOT Synonymous with a
Good Process!
Outcome

Good

Bad

We do not appoint commissions of
Justice
Bad Break
Good
inquiry
to
study
the
causes
of
good
Process Used to
Make the Decisions outcomes!
Bad
Possible outcome for
the decision-maker

Dumb Luck

Justice

Promoted

Fired

23

It is Important to Distinguish Between
Decision and Outcome
„

A good outcome is a future state of the world
that we prize relative to other possibilities.

„

A good decision is an action we take that is
logically consistent with the alternatives we
perceive, the information we have, and the
preferences we feel.

„

The quality of a decision should be judged by
the knowledge and information available at the
time the decision was made.

24

Too Much Focus on Outcome is
Driving Poor Decision-Making
„ Performance

incentives for a more balanced
focus are required


process vs outcome
long-term vs short-term
organizational drivers

„A

large number of decisions are required to
expose the lucky-fool or to recognize the
astute decision-maker

“in the long-run”
25

Performance Incentives
„

Bringing Down the House – Ben Mezrich

„

“Player pay is based on expected return
per hand. Not actual return; it doesn’t
matter if luck swings one way or another,
if you hit a losing streak or a winning
streak. You earn what our charts say
you’re supposed to earn based on perfect
application of our system.”
26

Common Biases and Traps in
Managerial Decision-Making
„

Hindsight Foresight
and the “curse of knowledge”

„

Overconfidence

„

Intuition and Repetition

„

Availability and Vividness

„

Anchoring

„

Knowing when enough
is enough

„

Illusion of control

„

The confirmation trap

27

Heuristics and Biases
„

Heuristics
– simple rules of thumb and mental
shortcuts

„

Biases
– systematic errors that are a result of the
use of heuristics

28

“I Knew It Would Happen” –
Remembered Probabilities of OnceFuture Things
„

Prior to President Nixon’s trip to China
and Russia in 1972, students were asked
to consider 15 possible outcomes:
• “the US will establish a permanent diplomatic
mission in Peking, but not grant diplomatic
recognition,”
• “Nixon will meet Mao Tse-tung at least once,”
• “Nixon will meet Soviet demonstrators,”
• …

„

The students assigned probabilities to
each outcome.
29

Ref: Fischhoff and Beyth (1975)

The “We Knew it All Along” Phenomenon
„

After the trip the students were asked in
hindsight to assess the likelihood of these
various outcomes and also asked to remember
or reconstruct their original probabilities.

„

Two weeks between pre- and post-trip
– 67% thought their original estimates were closer to
the truth then they really were.

„

Four to eight months between pre- and post
– 84% thought they had predicted the outcome.
30

Hindsight Foresight
„ In

hindsight, people consistently exaggerate
what could have been anticipated in foresight.

„ People

believe that others should have been
able to anticipate events much better than was
actually the case.

“they even misremember their own
predictions so as to exaggerate in
hindsight what they knew in foresight”
Fischhoff and Beyth, 1975

31

Hindsight Foresight
„ People

not only tend to view what happened as
having been inevitable but also to view it as
having appeared relatively inevitable before it
happened.

„“Dear

Diary, The Hundred Years’
War started today.”
32

Extreme Overconfidence
„ Heavier-than-air

flying
machines are impossible

Lord Kelvin, British
mathematician, physicist, and
president of the British Royal
Society, spoken in 1895

„A

severe depression like that
of 1920-21 is outside the
range of probability

Harvard Economic Society,
Weekly Letter, November 16,
1929
33

Extreme Overconfidence
„I

think there is a world market
for about five computers

Thomas J. Watson, chairman
of IBM, 1943

„ They

couldn't hit an elephant
at this dist…

General John B. Sedgwick,
Union Army Civil War officer's
last words, uttered during the
Battle of Spotsylvania, 1864
34

Lake Wobegon effect:
“all the women are strong, all the men
are good looking, and all the children are
above average” – Garrison Keillor
„
„
„
„
„

„

82% of people say they are in the top 30% of safe drivers;
86% of MBA students say they are better looking than their
classmates;
68% of lawyers in civil cases believe that their side will prevail;
Doctors consistently overestimate their ability to detect certain
diseases;
81% of new business owners think their business has at least
70% chance of success, but only 39% think that any business
like theirs would be likely to succeed;
Mutual fund managers, analysts, and business executives at a
conference were asked to write down how much money they
would have at retirement and how much the average person in
the room would have. The average figures were $5 million and
$2.6 million, respectively.

35

In the oil & gas industry pressures
towards overconfidence are rife:
„
„
„

„
„

“Explorationists need to be optimists.”
“Surprises” are more common than would be
hoped.
Experts may place overconfident bounds on
uncertain quantities to demonstrate their
expertise relative to peers or competitors.
It is common to have to “sell” ones projects.
Managers may create a climate that
discourages a true expression of uncertainty:
– “you are paid to know, not to not know …”
– “you are just covering your …”
36

Illusion of Control:
“Victory has a thousand fathers; defeat
is an orphan.”
-the Duke of Wellington
„

People often (knowingly and unknowingly)
take credit for positive outcomes and
attribute negative outcomes to external
factors, no matter what their true cause.

„

Study of letters to shareholders:
– Executives tend to attribute favourable outcomes
to factors under their control, and
– Unfavourable outcomes were more likely to be
attributed to uncontrollable external events such
as weather or inflation.

37

Illusion of Control
Doing

Skill

• implementation
and other factors
under your control

Presumed Cause
Success

Deciding

Outcome

• the thinking
and decision
process

Failure
Chance
• uncontrollable
factors, luck

Presumed Cause

38

Illusion of Control
Doing
• implementation
and other factors
Presumed
under your
control
control
frequently
cause Cause
people

Illusion
Skillof
to repeat actions that in the past were Success
followed by success.
Deciding
is true even if there’s no reason to
•„theThis
thinking
and
decision the actions did anythingOutcome
believe
to cause
process
the success.
„ Only by realistically assessing the role of
Failure
Cause of
chance in successes can you Presumed
learn which
Chance
your actions you
should repeat and which
• uncontrollable
factors, luck
could be improved.
39
„

Optimism and Organizational Pressure
„
„

„

Every company has limited investment
funds and time to devote to new projects.
Competition for this time and money is
intense, as individuals and units jockey to
present their own proposals as being the
most attractive for investment.
=> Big incentives to accentuate the
positive in project forecasts.
40

Optimism and Organizational Pressure
„

This has two ill effects:
– It ensures that the forecasts used for planning
are overoptimistic.
– It raises the odds that the projects chosen for
investment will be those with the most
overoptimistic forecasts – and hence the
highest probability of disappointment.
41

The Confirmation Trap
„

Consider the following series of numbers:
2–4–6

„

This series conforms to some rule. Your job is to
discover the rule. To do that you will be allowed to
pose any new sequences of numbers and you will be
told (truthfully) whether or not your sequence
conforms to the rule.

„

What is the rule?

„

How will you know when you have enough evidence
to guess the rule?

42

The Confirmation Trap
„

Common guesses:
– Numbers that go up by two
– The difference between the first two
numbers equals the difference
between the last two numbers

„

The actual rule in this experiment
(Wason, 1960)

Any three ascending numbers
43

The Confirmation Trap
This solution requires participants to accumulate
disconfirming, rather than confirming, evidence.
„ Wason’s conclusions:
– “Obtaining the correct solution necessitates
a willingness to attempt to falsify hypothesis,
and thus test the intuitive ideas that so often
carry the feeling of certitude.”
„

„

Our tendency is to search for information that
support our decision before making the final
commitment.
44

Financial Analysts
Presented with 14 hypothetical events that could
affect stock price.
„ They were asked how likely it was that each
event would occur.
„ They were also asked to determine whether
each event would be good for their investments,
or not.
„ Result:
„

– The more positive they thought an event was for their
investments, the more likely they thought it was to
occur.
45

Something out of Nothing
Misperceptions of
random events
„ The latter stages of
WWII
„

– The Germans
bombarded London with
their “vengeance
weapons” – the V-1
buzz bomb and the V-2
rocket

Source: Johnson (1981)

46

Something out of Nothing

Source: Johnson (1981)

18

4

11

31

47

Something out of Nothing
11

21

16

23

Source: Johnson (1981)

48

“Man thinks he believes only what
he sees, but he is much better at
believing than at seeing.”
George Santayana

49

Good Decision-Making Requires Not
Only Knowing the Facts, But
Understanding the Limits of Our
Knowledge
„ To

know that we know what
we know and that we do not
know what we do not know,
that is true knowledge
–Confucius

“It’s not what we don’t know that gets
us into trouble, it’s what we know that
ain’t so”
Will Rogers

50

Geometry Problem
Imagine two one-mile long pieces of railroad track, put end
to end, and attached to the ground at the extremes. When
it gets hot, the track expands by one inch, forcing it to rise
above the ground. How high is the track off the ground at
peak?
1 mile + 1 inch

x?
1 mile

Give a high and low estimate such that you are 90% sure
the correct answer lies between them.
51

Geometry Problem
„

Give a high and low estimate such that you are 90%
sure the correct answer lies between them.
a) Between 1/2 and 1 inch?
b) Between 1 and 3 inches?
c) More than 3 inches?

52

Geometry Problem
„

Typical Answers:

„

Give a high and low estimate such that you are 90% sure
the correct answer lies between them.
– Median Low Guess:

½ Inch

– Median High Guess:

2 Inches

– Ranges containing True Value:

13 of 82 (15.9%)

53

Normative Rule
„ Pythagorean

Theorem

z2 = x 2 + y2 ⇒ x = z2 − y2
x = (1 mile + 1 inch) 2 − (1 mile) 2
= (5, 280*12 + 1) 2 − (5, 280*12) 2

= 126, 721 = 355.98 inches = 29.6 feet!
„ Descriptive

reality:
– Most people underestimate x
– Why: We anchor on 1 inch and adjust insufficiently54

Anchoring and Adjustment
„

Students took the last three digits of their student ID
numbers, added 400 and wrote the number down.

„

They were then asked the following:
„

The Huns under Attila invaded Europe and penetrated
deep into what is now France where they were
defeated and forced to return eastward.

„

Did these events occur before or after AD _____
[date]?
„

„

Before____

After____

In what year did Attila's defeat occur? ____

55

Anchoring and Adjustment
Data from 38002 students:
If [date] was between… then average response was…
400-700

676AD

701-1000

738AD

1001-1200

848AD

1201-1400

759AD
[correct answer: 451 AD]
56

Ref.: Thaler, Richard

Anchoring is a Common Shortcut in
Estimation
„

The tendency to excessively anchor opinions
in a single statistic or fact that from then on
dominates the thinking process

„

Random anchors can have just as large
effects as credible anchors

„

Subtle changes in wording of a question can
have significant impact on how people
respond
57

Anchoring - Subtle changes in wording
of a question can have significant
impact on how people respond
Question

Outcome

Do you get headaches frequently,
and if so, how often?

2.2 / week

Do you get headaches occasionally,
and if so, how often?

0.7 / week

58

Anchoring - Subtle changes in wording
of a question can have significant
impact on how people respond
Question

Outcome

How long was the movie?

130 min

How short was the movie?

100 min

59

Anchoring - Subtle changes in wording
of a question can have significant
impact on how people respond
Question

Outcome

How tall was the basketball player?

79 inches

How short was the basketball
player?

69 inches

60

Knowing When Enough is Enough
40%

ƒ In our desire to reduce
uncertainty, we often
ask for too much
information

% correct
predictions

30%

Confidence does
increase

20%

Accuracy does not
increase

10%

5

10

20

40

Items of information available

ƒ We believe – mistakenly
– that more information
will reduce uncertainty

ƒ More information
helps only to the
extent we can use it
intelligently
61

Heuristics are Useful, Sometimes
„

The logic of heuristics is that, on average, any loss in
decision quality will be outweighed by time saved.

„

As demonstrated, there are many instances in which the
loss of decision quality far outweighs the time saved by
heuristics.
The logic above suggests that we have voluntarily accepted
the quality tradeoffs associated with heuristics.
In reality we have not as most of us are unaware of their
existence and their ongoing impact on our decision-making.

„
„

„

=> we fail to distinguish between situations in which they are
beneficial and situations in which they are potentially
harmful.
62

Heuristics are Useful, Sometimes
„

The logic of heuristics is that, on average, any loss in
decision quality will be outweighed by time saved.

As demonstrated, there are many instances in which the
loss of decision quality far outweighs the time saved by
The
key to improved judgement lies in
heuristics.
„ The logic
suggests thatbetween
we have voluntarily
accepted
learning
toabove
distinguish
appropriate
the quality tradeoffs associated with heuristics.
inappropriate
use
„ and
In reality
we have not as most
of usof
are heuristics.
unaware of their
existence and their ongoing impact on our decision-making.
„

„

=> we fail to distinguish between situations in which they are
beneficial and situations in which they are potentially
harmful.
63

What Can We Do?
Experts: US Weather Forecasters
38

1.0

9

0.9
0.8

Actual Fraction

159

82

0.7
0.6

147
203

0.5
0.4

172

0.3

257

0.2
0.1

589

146

161

282

0.0
0.0

575

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Forecast Probability
Source: Russo and Schoemaker

64

Experts: Physicians
Data: Physicians, after completing history and physical
examination, estimated the probability that patients had
pneumonia (Source: Christensen-Szalanski & Bushyhead, 1981)
1.0

„

Why are physicians
lousy and weather
forecasters great?
We cannot learn to
calibrate unless
we get systematic
feedback.

0.9

% Radiographically assigned
pneumonia

„

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Subjective Probability of Pneumonia

65

What Can We Do?
„

Cognitive illusions are a ubiquitous part of life, and
if unchecked, can lead to serious consequences
Awareness
– Understand the limits of your knowledge
– Actively challenge yourself – Engage in contrary analysis
– Abandon false comfort of single-point predictions
– Use ranges instead of single-point estimates

– Use multiple anchors

Calibration
– Feedback and accountability
– Meta-knowledge, to learn to recognize how much we
know and don’t know and not to come close to actual
answers
– Do we need to change our incentive systems and/or
organizational structures to drive good decision-making 66
behavior?

In Praise of Uncertainty
„ Decision-makers

must reduce
uncertainty as far as they can justify,
then they must manage it.

„ Managing

uncertainty doesn’t mean
accepting vague projections, making
wishy-washy recommendations, or
abandoning planning.

67

In Praise of Uncertainty
„ Requires

redefining rigor:

– Rigor is NOT found in precise single point
predictions.
– Rigor is NOT obtained by selecting the one
right vision for the future.
– Rigor requires accurately defined
uncertainty estimates.
– Rigor is defined through a rigorous process
that will enable you to anticipate and
prepare for multiple futures.
68

In Praise of Uncertainty

„

“Accept the fact of uncertainty and learn
to live with it. Since no magic will provide
certainty, our plans must work without it.”
» Historian Roberta Wohlstetter (1962)

69

The Pioneers
„

Kahneman and Tversky

„

2002 Nobel Price in Economics

Russo and Schoemaker
Decision Traps
– Winning Decisions

„

Thaler

„

Bazerman

„

The Winner’s Curse
Judgment in Managerial Decision Making

Plous

The Psychology of Judgment and Decision Making
70