Solutions Markov Chains 1

1) Given the following one-step transition matrices of a Markov chain,

determine the classes of the Markov chain and whether they are
recurrent.

a.
1 2
0 0
3 3
P 1 0 0 0 0 1 2 3
0 1 0 0
0 1 0 0

All states communicate, all states recurrent

b. 1 0 0 0
1 1
0 0
2 2
P 1 1 0 1 2 3
0 0
2 2
1 1
0 0
2 2

Communicate {0}, {1, 2}, {3}

states 0, 1, 2 recurrent
state 3 transient

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 3
2) The leading brewery on the West Coast (A) has hired a TM specialist to
analyze its market position. It is particularly concerned about its
major competitor (B). The analyst believes that brand switching can
be modeled as a Markov chain using 3 states, with states A and B
representing customers drinking beer produced from the
aforementioned breweries and state C representing all other brands.
Data are taken monthly, and the analyst has constructed the following
one-step transition probability matrix.
A B C
0.70 0.20 010
.
P  0.20 0.75 0.05
010
. 010
. 0.80

What are the steady-state market shares for the two major breweries?

Soln:
 A 0.7 A  0.20 B  01. C (1)
 B 0.2 A  0.75 B  01. C (2)
 C 01
.  A  0.05 B  0.8 C (3)

Let,  A 1 Then, from (1),

0.3 0.2 B  01
. C

 C 3  2 B (4)
This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 4
2) (cont.)

 A 0.7 A  0.20 B  01. C (1)

 B 0.2 A  0.75 B  01. C (2)

 C 01
.  A  0.05 B  0.8 C (3)
 C 3  2 B (4)

from (2),
 B .2(.1)  .75 B  .1(3  2 B )
.25 B .2  .3  .2 B
.45 B .5
 B 1111
.

from (4)

 C 3  2 (1111
. )
0.777

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 5
2) (cont.)

 A 1000
.
 B 1111
.
 C 0.777
 2.888
But,  A   B   C 1

 A  1000
.
2.888 0.346
 B  1111
.
2.888 0.385
 C  0.777 2.888 0.269

Note: This could also be checked by Pn for some large n.

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 6
3) Louise Ciccone, a dealer in luxury cars, faces the following weekly
demand distribution.
Demand 0 1 2
Prob. .1 .5 .4

She adopts the policy of placing an order for 3 cars whenever the
inventory level drops to 2 or fewer cars at the end of a week. Assume
that the order is placed just after taking inventory. If a customer arrives
and there is no car available, the sale is lost. Show the transition matrix
for the Markov chain that describes the inventory level at the end of each
week if the order takes one week to arrive. Compute steady state
probabilities and the expected number of lost sales per week.

Soln
S= inventory level at week’s end = 1, 2, 3, 4, 5
note: we can have at most 5 cars in the lot, 2 cars at week’s end
followed by no sales followed by the arrival of 3 cars at the end of the
week.
note: we could start with 0 cars, but once an arrival of 3 cars
comes we will never get back to 0 cars since we never sell more than 2
and we get below 2 at time t, 3 arrive at t+1

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 7
Soln cont.

State Event End State Probability

5 sell 0 5 .1
5 sell 1 4 .5
5 sell 2 3 .4

4 sell 0 4 .1
4 sell 1 3 .5
4 sell 2 2 .4

3 sell 0 3 .1
3 sell 1 2 .5
3 sell 2 1 .4

2 3 arrive at week’s end

sell 0 5 .1
sell 1 4 .5
sell 2 3 .4

1 3 arrive at week’s end

sell 0 4 .1
sell 1 3 .5+.4 since demand of 1 or 2 deletes
inventory during the week

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00
 Solutions Markov Chains 8
Soln cont.

0 0 .9 .1 0  1 0 1  0 2  .4 3  0 4  0 5 (1)

0 0 .4 .5 .1  2 0 1  0 2  .5 3  .4 4  0 5 ( 2)
P .4 .5 .1 0 0  3 .9 1  .4 2  .1 3  .5 4  .4 5 (3)
0 .4 .5 .1 0  4 .1 1  .5 2  0 3  .1 4  .5 5 ( 4)
0 0 .4 .5 .1  5 0 1  .1 2  0 3  0 4  .1 5 (5)
 1   2   3   4   5 1

Let,  1 1, then from (1), 3 1 / .4 2.5

from (5), .9 5 .1 2

from ( 2),  2 .5 3  .4 4

.5( 2.5)  .4 4

Continue substitutions and renormalize or

.15 .26 .37 .19 .03 Lost sales if we1car on lot and demand is 2,
.15 .26 .37 .19 .03
P (10 ) .15 .26 .37 .19 .03
E[lost sales ]  1P( Dt 2) .15(.4) .06
.15 .26 .37 .19 .03
.15 .26 .37 .19 .03

This study source was downloaded by 100000835991620 from CourseHero.com on 04-08-2023 08:13:38 GMT -05:00

Powered by TCPDF (www.tcpdf.org)