MSBA7003 Quantitative Analysis Methods

Tutorial 02
Haobo Yu
2021 – 2022

1
 2

Agenda

• Solution of Assignment 01

• Dynamic Programming
• Determining Reject Allowances
• Winning in Las Vegas

• Python Code of Carpark Revenue Management

 3

Question 01

• The joint probability distribution among two random variables X and Y is

given below.
Joint Probabilities Y = 1 Y=2 Y=3

X=0 0.1 0.2 0.3

X=1 0.2 0.1 0.1

• (A) X=0 and Y=1 are unconditionally independent.

• (B) The event of X=0 is more likely to occur given Y=2 than given Y=1.
• (C) E[Y|X=0]=1.4.
• (D) E[X*Y]=0.84.
• (E) None of the above.
 4

Question 02

• Question 02: according to the Venn diagram below, which of the

following statement(s) is (are) true?

• (A) Event A and event B are dependent

without knowing event C.
• (B) If event C did not happen, event A
and event B may be independent.
• (C) Given that C occurred, A and B are
independent.
• (D) P(BC)=P(BC|A).
• (E) None of the above.
 5

Question 03

• Question 03:
• The useful information is summarized below.

Initial Profit Favorable Unfavorable

Product investment Report Report
Weak
(Cost) Strong Market
Market
Strong Market 65% 35%
X 15 100 20
Weak Market 45% 55%
Y 5 -80 -9

Do
0 0 0
nothing

• The company’s prior belief of a strong market is 60%.

 6

Question 03
• When the consulting company is not hired
Payoff
Strong Market (0.6)
85
EMV_X = 13
Weak Market (0.4)
-95

EMV_Y = 3.4 Strong Market (0.6) 15

Product Y
Weak Market (0.4)
-14

EMV_0 = 0
0
Without survey: EMV= 13
 7

Question 03

• Decision Tree with Perfect Report

Second Decision Payoffs
Point
85 Strong Market (1)
85
With a perfect report, 2 Weak Market (0)
–95
EMV = 85*0.6+0*0.4=51 Product
15 Strong Market (1)
15

85
Y
3 Weak Market (0)
-14
Do nothing 0
1 -95 Strong Market (0) 85
4 Weak Market (1) –95
-14 Strong Market (0)
Product 15
5 Weak Market (1)
0

Y –14
Do nothing
0

• The value of the consulting firm’s report is at most $38 million.

 8

Question 03

• When a consulting company is hired:

Strong Market Weak Market Marginal

Favorable report 0.65*0.6 0.45*0.4 0.57

Unfavorable report 0.35*0.6 0.55*0.4 0.43

Updated belief
0.6842 0.3158
under Fav
Updated belief
0.4884 0.5116
under Unfav
Prior belief 0.6 0.4
 9

Question 03

• Decision Tree with Report

Second Decision Payoffs
Point
25.156 Strong Market (0.6842)
If a consulting survey is 85
2 Weak Market (0.3158)
–95
provided: 5.8418 Strong Market (0.6842)

28.156
Product 15
EMV = Y
3 Weak Market (0.3158)
-14
28.156*0.57+0.1636*0.43 Do nothing 0
=16.12
1 -7.088 Strong Market (0.4884) 85
4 Weak Market (0.5116) –95
0.1636 Strong Market (0.4884)
0.1636

Product 15
Y
5 Weak Market (0.5116)
–14
Do nothing
0

• The expect value of the consulting firm’s report is about $3.12 million.
 10

Question 04

• Question 04
• There are three possibilities for the smart phone screen 𝑄 : high quality 𝐻 ,
medium quality 𝑀 , and low quality 𝐿 .
• Three tests: The first one did not break; the second one broke; the third one
broke.
• After the first test
The first one did High Medium Low Marginal
not break

Joint Probability 0.7/3 0.5/3 0.3/3 0.5

Prior 1/3 1/3 1/3

Updated 7/15 1/3 1/5

 11

Question 04
• After the second test
The second one High Medium Low Marginal
broke
Joint Probability 0.3 ∗ 7/15 0.5/3 0.7/5 67/150

Prior 7/15 1/3 1/5

Updated 21/67 25/67 21/67

• After the third test

The third one High Medium Low Marginal
broke
Joint Probability 0.3 ∗ 21/67 0.5 ∗ 25/67 0.7 ∗ 21/67 335/647

Prior 21/67 25/67 21/67

Update 63/335 25/67 147/335

• The screens are most likely to be of low quality.

 12

Question 04

• The expected probability of breaking the fourth screen is

63 25 147
0.3 ∗ + 0.5 ∗ + 0.7 ∗ = 0.5501
335 67 335

• If we drop the first screen again, the probability of breaking is the

same as the one above.

• Dropping two screens at the same time can be regarded as

dropping two screens sequentially.
 13

Question 04
• After the fourth test
The fourth one High Medium Low Marginal
broke
Joint Probability 0.3 ∗ 63/335 0.5 ∗ 25/67 0.7 ∗ 147/335 1843/3350

Prior 63/335 25/67 147/335

Updated 189/1843 625/1843 1029/1843

• If the fourth screen breaks, the expected probability of breaking the

fifth screens equals
189 625 1029
0.3 ∗ + 0.5 ∗ + 0.7 ∗ = 0.5912
1843 1843 1843
• Combing with the probability of the fourth breaking, the target
probability is 0.5912 ∗ 0.5501 = 0.3252.
 14

Determining Reject Allowances

• The HIT-AND-MISS MANUFACTURING COMPANY has received an

order to supply one item of a particular type. However, the customer
has specified such stringent quality requirements that the
manufacturer may have to produce more than one item to obtain an
item that is acceptable. The number of extra items produced in a
production run is called the reject allowance. Including a reject
allowance is common practice when producing for a custom order,
and it seems advisable in this case.
• The manufacturer estimates that each item of this type that is
produced will be acceptable with probability 0.5 and defective with
probability 0.5. Thus, the probability of producing no acceptable
items in a lot of size 𝐿 is 0.5𝐿 .
 15

Determining Reject Allowances

• Marginal production costs for this product are estimated to be $100

per item (even if defective), and excess items are worthless. In
addition, a setup cost of $300 must be incurred whenever the
production process is set up for this product, and a completely new
setup at this same cost is required for each subsequent production
run if a lengthy inspection procedure reveals that a completed lot has
not yielded an acceptable item. The manufacturer has time to make
no more than three production runs. If an acceptable item has not
been obtained by the end of the third production run, the cost to the
manufacturer in lost sales income and penalty costs will be $1,600.
• The objective is to determine the policy regarding the lot size (1 +
reject allowance) for the required production run(s) that minimizes
total expected cost for the manufacturer.
 16

Determining Reject Allowances

• Objective: to minimize the total expected cost

• State: (𝑛, 𝑠𝑛 )
• 𝑛: 𝑛th production run (𝑛 = 1,2,3) or 𝑛𝑡ℎ stage
• 𝑠𝑛 : number of acceptable items still needed (1 or 0) at beginning of stage 𝑛
• Action: 𝑥𝑛
• 𝑥𝑛 : lot size for stage 𝑛
• State-action value: 𝑄(𝑛, 𝑠𝑛 , 𝑥𝑛 )
• 𝑄(𝑛, 𝑠𝑛 , 𝑥𝑛 ): total expected cost (from stage 𝑛 to the last stage) if system
starts in state (𝑛, 𝑠𝑛 ) and immediate decision is 𝑥𝑛 , and optimal decisions are
made thereafter
• State value: 𝐽 𝑛, 𝑠𝑛 = min 𝑄 𝑛, 𝑠𝑛 , 𝑥𝑛
𝑥𝑛 =0,1,…
• 𝐽 𝑛, 𝑠𝑛 : total expected cost (from stage 𝑛 to the last stage), given that
system is in state (𝑛, 𝑠𝑛 ) and optimal policy is adopted
• 𝐽 𝑛, 0 = 0
 17

Determining Reject Allowances

• Using $100 as the unit of money, the contribution to cost from stage
𝑛 is [𝐾 𝑥𝑛 + 𝑥𝑛 ] regardless of the next state, where 𝐾 𝑥𝑛 is a
function of 𝑥𝑛 such that
0, if 𝑥𝑛 = 0
• 𝐾 𝑥𝑛 = ቊ
3, if 𝑥𝑛 > 0.
 18

Determining Reject Allowances

• The basic structure for determining reject allowances
Subsequent cost
𝐽 𝑛, 1 = min 𝑄 𝑛, 1, 𝑥𝑛 𝑥
𝑥𝑛 =0,1,… Acceptable 1 − 0.5 𝑛
𝐽 𝑛 + 1,0 = 0

No Acceptable 0.5 𝑥𝑛
𝐽(𝑛 + 1,1)
.
.
. Acceptable 1 − 0.5 𝑥𝑛 𝐽 𝑛 + 1,0 = 0
𝑥𝑛 = 2
𝒏, 𝟏
𝐾 𝑥𝑛 + 𝑥𝑛 = 5 No Acceptable 0.5 𝑥𝑛
𝐽(𝑛 + 1,1)

𝐽(𝑛 + 1,1)
𝑄 𝑛, 1, 𝑥𝑛 = 𝐾 𝑥𝑛 + 𝑥𝑛 + 0.5 𝑥𝑛 𝐽 𝑛 + 1,1
 19

Determining Reject Allowances

• Stage 3:

1
𝑄 3,1, 𝑥3 = 𝐾 𝑥3 + 𝑥3 + ( )𝑥3 16
2
𝑠3 𝐽 3, 𝑠3 𝑥3∗
𝑥3 = 0 𝑥3 = 1 𝑥3 = 2 𝑥3 = 3 𝑥3 = 4 𝑥3 = 5

0 0 0 0

1 16 12 9 8 8 8.5 8 3 or 4
 20

Determining Reject Allowances

• Stage 2:
𝑥2
1
𝑄 2,1, 𝑥2 = 𝐾 𝑥2 + 𝑥2 + 𝐽(3,1)
2
𝑠2 𝐽 2, 𝑠2 𝑥2∗
𝑥2 = 0 𝑥2 = 1 𝑥2 = 2 𝑥2 = 3 𝑥2 = 4

0 0 0 0

1 8 8 7 7 7.5 7 2 or 3
 21

Determining Reject Allowances

• Stage 1:
𝑥1
1
𝑄 1,1, 𝑥1 = 𝐾 𝑥1 + 𝑥1 + 𝐽(2,1)
2
𝑠1 𝐽 1, 𝑠1 𝑥1∗
𝑥1 = 0 𝑥1 = 1 𝑥1 = 2 𝑥1 = 3 𝑥1 = 4

1 3 7 7 3
1 7 7 6 6 7 6 2
2 4 8 16 4

• Thus, the optimal policy is to produce two items on the first

production run; if none is acceptable, then produce either two or
three items on the second production run; if none is acceptable, then
produce either three or four items on the third production run. The
total expected cost for this policy is $675.
 22

Winning in Las Vegas

• A young statistician believes that she has developed a system for

winning a popular Las Vegas game. Her colleagues do not believe that
her system works, so they have made a large bet with her that if she
starts with three chips, she will not have at least five chips after three
plays of the game. Each play of game involves betting any desired
number of available chips and then either winning or losing this
number of chips. The statistician believes that her system will give
her a probability of 2Τ3 of winning a given play of the game.
• Assuming the statistician is correct, what is her optimal policy
regarding how many chips to bet (if any) at each of the three plays of
game.
 23

Winning in Las Vegas

• Objective: to maximize the probability that the statistician will win

her bet
• State: (𝑛, 𝑠𝑛 )
• 𝑛: 𝑛th play of game (𝑛 = 1,2,3) or 𝑛th stage
• 𝑠𝑛 : number of chips in hand to begin stage 𝑛
• Action: 𝑥𝑛
• 𝑥𝑛 : number of chips to bet at stage 𝑛 and 0 ≤ 𝑥𝑛 ≤ 𝑠𝑛 .
• State-action value: 𝑄(𝑛, 𝑠𝑛 , 𝑥𝑛 )
• 𝑄(𝑛, 𝑠𝑛 , 𝑥𝑛 ): probability of ending with at least five chips, given that the
statistician starts stage 𝑛 in state (𝑛, 𝑠𝑛 ), makes immediate decision 𝑥𝑛 , and
makes optimal decisions thereafter
• State value: 𝐽 𝑛, 𝑠𝑛 = max 𝑄(𝑛, 𝑠𝑛 , 𝑥𝑛 )
𝑥𝑛 =0,1,…,𝑠𝑛
• 𝐽 𝑛, 𝑠𝑛 : probability of ending with at least five chips, given that the
statistician starts stage n in state (𝑛, 𝑠𝑛 ) and optimal policy is adopted
 24

Winning in Las Vegas

• The basic structure for the Las Vegas problem
Probability
𝐽 𝑛, 𝑠𝑛 = max 𝑄 𝑛, 𝑠𝑛 , 𝑥𝑛 to win finally
𝑥𝑛 =0,1,…,𝑠𝑛 Win (2/3)
𝐽(𝑛 + 1, 𝑠𝑛 + 1)

Loss (1/3)
𝐽(𝑛 + 1, 𝑠𝑛 − 1)

Win (2/3) 𝐽(𝑛 + 1, 𝑠𝑛 + 2)

𝑥𝑛 = 2
𝒏, 𝒔𝒏 (=2)
Loss (1/3)
𝐽(𝑛 + 1, 𝑠𝑛 − 2)

𝐽(𝑛 + 1, 𝑠𝑛 )
1 2
𝑄 𝑛, 𝑠𝑛 , 𝑥𝑛 = 𝐽 𝑛 + 1, 𝑠𝑛 − 𝑥𝑛 + 𝐽 𝑛 + 1, 𝑠𝑛 + 𝑥𝑛
3 3
 25

Winning in Las Vegas

• Stage 3:
𝑠3 𝐽(3, 𝑠3 ) 𝑥3∗

0 0 -

1 0 -

2 0 -

3 2/3 2 (or more)

4 2/3 1 (or more)

≥5 1 0 (or ≤ 𝑠3 − 5)
 26

Wining in Las Vegas

• Stage 2:
1 2
𝑄 2, 𝑠2 , 𝑥2 = 𝐽 3, 𝑠2 − 𝑥2 + 𝐽 3, 𝑠2 + 𝑥2
𝑠2 3 3 𝐽 2, 𝑠2 𝑥2∗
𝑥2 = 0 𝑥2 = 1 𝑥2 = 2 𝑥2 = 3 𝑥2 = 4
0 0 0 -

1 0 0 0 -

2 0 4/9 4/9 4/9 1 or 2

3 2/3 4/9 2/3 2/3 2/3 0, 2 or 3

4 2/3 8/9 2/3 2/3 2/3 8/9 1

≥5 1 0 (or ≤ 𝑠2 − 5)
 27

Winning in Las Vegas

• Stage 1:
1 2
𝑄 1, 𝑠1, 𝑥1 = 𝐽 2, 𝑠1 − 𝑥1 + 𝐽 2, 𝑠1 + 𝑥1
𝑠1 3 3 𝐽 1, 𝑠1 𝑥1∗
𝑥1 = 0 𝑥1 = 1 𝑥1 = 2 𝑥1 = 3
3 2/3 20/27 2/3 2/3 20/27 1

• Therefore, the optimal policy is

• This policy has a probability of 20/27 to win.

 28

Python Code of
Carpark Revenue Management
• Please refer to CarparkMCTS.py.