Decision Making Under Uncertainty

Decision Making Everyday

• Should I bring an umbrella with me?

• Which bus to take?
• Where to have dinner?
• …
 Objective

• To introduce a formal framework for analyzing decision

problems that involve uncertainty
• The framework is to use a decision analysis tool called
Decision Trees
 The Truel
• Mr. Black, Mr. Gray, and Mr. White are fighting in a truel. They each get a gun and take
turns shooting at each other until only one person is left. Mr. Black, who hits his shot 1/3
of the time, gets to shoot first. Mr. Gray, who hits his shot 2/3 of the time, gets to shoot
next, assuming he is still alive. Mr. White, who hits his shot all the time, shoots next,
assuming he is also alive. The cycle repeats. If you are Mr. Black, where should you shoot
first for the highest chance of survival?
 Elements of Decision Analysis
• Although decision making under uncertainty occurs in a
wide variety of contexts, all decision problems have
three common elements:

1. A set of decisions (or strategies) available,

2. A set of possible outcomes, with associated probabilities, and
3. A value model that prescribes monetary or utility values for the
various decision-outcome combinations.

• Once these elements are known, the decision maker can

systematically find an optimal decision, depending on
the optimality criterion chosen.
 Case Example – Contract Bidding
• SciTools Inc., a company that specializes in scientific
instruments, has been invited to bid for a government
contract.
• The contract calls for a specific number of these
instruments to be delivered during the coming year.
• The bids must be sealed, so that no company knows what
others are bidding, and the lowest bid wins the contract.
• Cost estimates:
– $5,000 to prepare a bid, and
– $95,000 to supply the instruments if it wins the contract.
• SciTools believes that there is a 30% chance that there will
be no competing bids.
SciTools Case Example Continued
• In case there is competition, based on past contracts of
this type, possible lowest bids from competitors and
associated probabilities are given below:

Lowest Bid from Competitors Probability

Less than $115,000 0.2
Between $115,000 and $120,000 0.4
Between $120,000 and $125,000 0.3
Greater than $125,000 0.1
 Four Factors In A Decision Tree

• Decisions i.e. options/strategies

available to decision maker

• Outcomes i.e. externalities

outside control of decision maker

• Probabilities i.e. risk/uncertainty

level of the outcomes

• Payoffs i.e. rewards accruing to

different decisions & outcomes
 Decisions (In SciTools Case)

• Options available for SciTools:

– Don’t bid
– Bid
• $115,000
• $120,000
• $125,000

(Q: Why not bid any amount less than $115,000? Or more than
$125,000? Or any other amount in between?)
 Outcomes (In SciTools Case)

• What may happen?

– No bid from competitors
– Lowest bid from competitors < $115,000
– Lowest bid from competitors is between $115,000 and
$120,000
– Lowest bid from competitors is between $120,000 and
$125,000
– Lowest bid from competitors > $125,000
Probabilities Of Outcomes
 Payoffs (In SciTools Case)
Actual Outcomes

Competitors’ lowest bid (in $’000)

Payoff Table
No bid <115 115~120 120~12 >125
5
No bid 0 0 0 0 0
Decisions

115 15 -5 15 15 15
SciTools’ Bid
(in $’000) 120 20 -5 -5 20 20

125 25 -5 -5 -5 25

Probability 0.3 0.14 0.28 0.21 0.07

 Simplified Payoffs
Summarized Outcomes

Monetary Value ($’000) Probability

that
SciTools SciTools SciTools
Wins Loses Wins
SciTools’ No Bid 0 0 0.00
Bid
Decisions

(in $’000) 115 15 -5 0.86

120 20 -5

125 25 -5 0.37
 Decision Tree Modeling

• For the same problem, different decision trees may be used

to model it, depending on:
– How events (and thus outcomes) are defined?
– How decisions are defined and sequenced?
 Decision Tree For SciTools (V1.1)
Outcome 0.3 15k
Decision Probability
0

0.14
-5k

0.28
15k

Decision 0.21
Node 15k
Payoff
Probability
Node 0.07
15k
 Complete Decision Tree
0.3 15k
0
0.14 -5k

Low bid 115k~120k 0.28 15k

0.21 15k 20k

0.3
0.07 0.14 -5k
15k

Low bid 115k~120k 0.28 -5k

0.21 20k
0.3 25k
0.07
20k
0.14 -5k

Low bid 115k~120k 0.28 -5k

0.21 -5k
0.07
25k
 Evaluating A Decision Tree

• Decision tree is drawn from left to right (following

the sequence of time).
• Evaluation is done from right to left.
• Each node on the decision tree will have an EMV
associated with it after evaluation.
 EMV

• The expected monetary value, or EMV, for a

decision is a weighted average of all possible
payoffs for this decision, weighted by
probabilities of the outcomes.
• “Playing the averages”: Choosing the decision
with the largest EMV
– EMV for each probability node is calculated (see
following example)
– EMV for decision node = largest EMV amongst all
decisions
 EMV of “Bid $115k” Decision
0.3 15k

0.14
12.2k -5k

0.28
15k

0.21
15k

0.07

15k
 Maximax Criterion
in Evaluating a Decision Node
Don’t bid 0

12.2k Maximum among all EMVs

12.2k Bid 115k of the probability nodes
Bid 120k 9.5k

6.1k
Bid 125k
 Complete Decision Tree
0.3 15k
0
0.14 -5k

12.2k Low bid 115k~120k 0.28 15k

0.21 15k 20k

0.3
0.07 0.14 -5k
15k
12.2k
9.5k
Low bid 115k~120k 0.28 -5k

0.21 20k
0.3 25k
0.07
20k
0.14 -5k

6.1k Low bid 115k~120k 0.28 -5k

0.21 -5k
0.07
25k
 Optimal Decision (For SciTools)

• The best strategy for SciTools is to bid $115,000 for the

contract.
• The expected return (or profit) is $12,200.
 Decision Tree V1.2
0
0.86 15k
Don’t bid Win
12.2k

12.2k Lose 0.14 -5k

Bid 115k
0.58 20k
9.5k Win

Bid 120k Lose 0.42

-5k

Bid 125k Win 0.37 25k

6.1k

Lose 0.63
-5k

Based on summarized payoffs

 Decision Tree Modeling

• For the same problem, different decision trees may be used

to model it, depending on:
– How events (and thus outcomes) are defined?
– How decisions are defined and sequenced?
 Decision Tree V1.3
0.3 0.2 -5k
15k
0 No Competition Low bid
<115k
12.2k
Don’t bid Low bid
0.7 0.4
115k~120k
With 15k
12.2k Bid 115k
Competition Low bid
120k~125k
12.2k Bid 9.5k
120k 0.3
Bid 15k
Low bid
Bid 6.1k
>125k
125k
0.1

15k
 Decision Tree V1.4
0.3 0.2 -5k
15k
0 No Competition
Lose
12.2k
Don’t bid
0.7 Win 0.8
Has 15k
12.2k Bid 115k
Competition

12.2k Bid 9.5k

120k
Bid

Bid 6.1k
125k
Multi-Stage Decision Problems
The value of information
 Caroline Jane’s Decision Problem
Case: Development of New Consumer Product
– Caroline Jane to decide whether to produce “Suds-Away”
– Market for product may be strong or weak

If market is strong : company will make $18m

If market is weak : company will lose $8m

Caroline Jane estimates:

30% chance market will be strong
70% chance market will be weak
 Caroline Jane’s Decision Problem
Case: Development of New Consumer Product
– Caroline Jane to decide whether to produce “Suds-Away”
– Market for product may be strong or weak

$0
$18 m

Produce

 $8 m
 Caroline Jane’s Decision Problem
Case: Development of New Consumer Product
– Caroline Jane to decide whether to produce “Suds-Away”
– Market for product may be strong or weak

$0
$18 m For this problem,
the maximum
$0.0M $0.2M EMV ($0) is
obtained if
Produce Caroline decides
not to produce
 $8 m Suds-Away.
Caroline Jane’s Decision Problem (with Survey)

Suppose Caroline can conduct a market survey first Cost : $2.4 m

– Survey may predict positive or negative market conditions

Caroline Jane’s Decision Problem (with Survey)

Suppose Caroline can conduct a market survey first Cost : $2.4 m

– Survey may predict positive or negative market conditions

Conduct Survey
 Caroline Jane’s Problem

Q: Is it worthwhile to conduct the survey?

 Caroline Jane’s Problem

For a start , let us assume that : the survey is perfectly accurate.

That is : Positive  Strong market

Negative  Weak market
 Caroline Jane’s Problem (with Perfect Survey)
Q: Assuming the
market survey is
perfectly accurate,
what is the max that
you would pay for
the survey?

Conduct Survey
 Caroline Jane’s Problem (with Perfect Survey)
Q: Assuming the market
survey is
perfectly accurate,
what is the max that
you would pay for $0
the survey?

$18 m
Produce

$0
Conduct Survey

 $8 m
Produce
 Caroline Jane’s Problem (with Perfect Survey)
Q: Assuming the market
survey is
perfectly accurate,
what is the max that
you would pay for $0
the survey?
$18 m
$18 m
Produce
$5.4 m $5.4 m

$0
Conduct Survey
$0
 $8 m
EMV = $5.4m (minus the cost of survey) Produce
 Expected Value of Perfect Information
Q : Assuming the market survey is perfectly accurate, what is the
maximum amount that you would pay for the survey?

A : Should not pay more than $5.4m for the survey

The maximum amount that a decision maker would pay for perfect
information is called the Expected Value of Perfect Information (EVPI).

EVPI = (Maximum EMV with perfect information)

 (Maximum EMV without additional information)
Caroline Jane’s Decision Problem (with Survey)

Suppose Caroline can conduct a market survey first Cost : $2.4 m

– Survey may predict positive or negative market conditions

Conduct Survey

Caroline now has to decide :

(i) whether to conduct the survey, and
(ii) whether to produce Suds Away
 Caroline Jane’s Decision Problem (with Survey)
Q: Is it worthwhile
Suppose Caroline can conduct a market survey first Cost : $2.4 m
to conduct the
–survey?
Survey may predict positive or negative market conditions

Depends on the accuracy of the survey,

and whether the potential gains
outweigh the cost.

Conduct Survey
 Caroline Jane’s Decision Problem (with Survey)
Q: Is it worthwhile
to conduct the
survey? Survey cannot predict market with certainty.
Past results indicate :
Depends on the accuracy of the survey,
•If market is weak, there is a 10% chance that
test and whether
will be the potential gains
positive
outweigh the cost.
•If market is strong, there is a 20% chance
that the test will be negative.
Need more info regarding
How do we incorporate this
accuracy of survey.
knowledge into our analysis?
Caroline Jane’s Decision Problem (with Survey)

•If market is weak, there is a 10% chance that test will be positive
•If market is strong, there is a 20% chance that the test will be negative.

Event Denote by Probability

Market is Strong S 0.3
Market is Weak W 0.7
Survey Result is Positive Q P(Q)
Survey Result is Negative N P(N)

Accuracy P(Q|W) = 0.1

of Survey P ( N | S ) = 0.2
 Caroline Jane’s Decision Problem (with Survey)
Event Denote by Probability
Market is Strong S 0.3
P(Q |W) = 0.1
Market is Weak W 0.7
P ( N | S ) = 0.2
Survey Result is Positive Q P(Q )
Survey Result is Negative N P(N)

In order to solve the problem

we need to work out the P(S|Q)

following probabilities: P(Q)

P(W|Q)

Conduct Survey
P(S|N)
P(N)

P(W|N)
Recall Bayes’ Theorem
Consider an event B and n mutually exclusive and collectively
exhaustive events A1, A2 , . . , An .
Given that B has occurred, we can use Bayes’ theorem to find
the posterior probabilities that the event Ai has occurred.

𝑃(𝑨𝒊 ∩ 𝑩) 𝑃 𝑩 𝑨𝒊 𝑃(𝑨𝒊 )
𝑃 𝑨𝒊 𝑩 =
𝑃(𝑩)

𝑃 𝑩 𝑨𝟏 𝑃 𝑨𝟏 + 𝑃 𝑩 𝑨𝟐 𝑃 𝑨𝟐 + . . . +𝑃 𝑩 𝑨𝒏 𝑃 𝑨𝒏
Caroline Jane’s Decision Problem (with Survey)
To summarize, P(S) = 0.3 P ( W ) = 0.7
we know :
P(Q |W) = 0.1 P(N |W) = 0.9
P ( N | S ) = 0.2 P ( Q | S ) = 0.8

Need to find : P(Q) = 0.31 P(N) = 1  P(Q) = 0.69

P(S|Q) P(W|Q) = 1  P(S|Q)
P(S|N) P(W|N) = 1  P(S|N)

𝑃 𝑄 = 𝑃 𝑄∩𝑆 + 𝑃 𝑄∩𝑊

= 𝑃 𝑄|𝑆 𝑃(𝑆) + 𝑃 𝑄|𝑊 𝑃(𝑊)

= 0.8 × 0.3 + 0.1 × 0.7 = 0.31
Caroline Jane’s Decision Problem (with Survey)
To summarize, P(S) = 0.3 P ( W ) = 0.7
we know :
P(Q |W) = 0.1 P(N |W) = 0.9
P ( N | S ) = 0.2 P ( Q | S ) = 0.8

Need to find : P(Q) = 0.31 P(N) = 1  P(Q) = 0.69

P(S|Q)  0.7742 P(W|Q) = 1  P(S|Q)  0.2258
P(S|N) P(W|N) = 1  P(S|N)
𝑃(𝑆 ∩ 𝑄) 𝑃 𝑄 𝑆 𝑃(𝑆)
𝑃 𝑆𝑄 = =
𝑃(𝑄) 𝑃 𝑄

0.8 × 0.3 . 24
= = ≅ 0.7742
0.31 . 31
 Caroline Jane’s Decision Problem (with Survey)
To summarize, P(S) = 0.3 P ( W ) = 0.7
we know :
P(Q |W) = 0.1 P(N |W) = 0.9
P ( N | S ) = 0.2 P ( Q | S ) = 0.8

Need to find : P(Q) = 0.31 P(N) = 1  P(Q) = 0.69

P(S|Q)  0.7742 P(W|Q) = 1  P(S|Q)  0.2258
P(S|N)  0.0870 P(W|N) = 1  P(S|N)  0.9130

Similarly :
𝑃(𝑆 ∩ 𝑁) 𝑃 𝑁 𝑆 𝑃(𝑆)
𝑃 𝑆𝑁 = = ≅ 0.0870
𝑃(𝑁) 𝑃 𝑁
 Caroline Jane’s Decision Problem (with Survey)
To summarize, P( S ) = 0. 3 P ( W ) = 0.7
we know :
P( Q |W ) = 0. 1 P (N |W ) = 0 . 9
P ( N | S ) = 0. 2 P ( Q | S ) = 0.8

Need to find : P(Q ) = 0.31 P (N) = 1  P(Q) = 0. 69

P(S |Q )  0.7742 P (W |Q ) = 1  P ( S |Q )  0.2258
P(S |N)  0.0870 P(W |N) = 1  P( S |N)  0.9130

P(S|Q)

P(Q)

P(W|Q)

Conduct Survey
P(S|N)
P(N)

P(W|N)
Caroline Jane’s Decision Problem (with Survey)
Solving the decision tree :
0.7742

0.31

0.2258

Conduct Survey
0.31 0.0870
0.69

0.9130
Caroline Jane’s Decision Problem (with Survey)
Solving the decision tree :
0.7742
$9.729m $9.729 m
0.31

$1.36 m
0.2258

Conduct Survey
0.31 0.0870
 $2.4m  $8.138 m
0.69

0.9130
 Expected Value of Sample Information
Q : Given that the market survey is imperfect, what is the
maximum amount that Caroline would pay for the survey?

A : She should not pay more than $3.76m for the survey

The maximum amount that a decision maker would pay for (sample)
information is called the Expected Value of Sample Information (EVSI).

EVSI = (Maximum EMV with sample information)

 (Maximum EMV without additional information)
 Caroline Jane’s Decision Problem - Conclusion

Caroline’s optimal strategy:

Do the survey.
Produce if survey result is positive,
otherwise do not produce.
EMV will be $1.36m.

$1.36m
$1.36m

EVSI = $3.76m

End