|
 |

 

|
 

‡ The Characteristics of Markov Analysis

‡ The Transition Matrix
‡ Steady-State Probabilities
‡ Additional Examples of Markov Analysis
‡ Special Types of Transition Matrices
‡ Excel Solution of the Debt Example

|
 |
  
 |
 
 

- |
  
is a 
 


 technique.
- It provides information about a decision situation.
- It is a  


, not an optimizing technique.
- Specifically applicable to systems that exhibit probabilistic
movements from one  (or condition) to another.

|
 |
  
    |
 
- The brand-switching problem used for discussion.
- The brand-switching problem analyses the probability of customers¶ changing brands of a
product over time.

á
Probabilities of Customer Movement per Month

- The state of a system is where the system is at a point in time.

- A transition probability is the probability of moving from one state to another
|
 |
  
during one period of time.
    |
  

@. The transition probabilities for a given beginning state of the

system sum to one.
2. The probabilities apply to all participants in the system.
3. The transition probabilities are constant over time.
4. The states are independent over time.

|
 |
  
 |
   ! " #

- Information available from Markov analysis is the probability of being in a state at some future time.
- In example, the manager wants to know the probability that a customer would trade with them in
month 3 given that the customer trades with them in month @.
- Probability of a customer¶s trading with Petroco in month 3 given that the customer initially traded
with Petroco in month @ is the sum of two branch probabilities associated with Petroco: .36 + .08 = .44.
- For probability of a customer¶s purchsasing gasoline from National in month 3, add probabilities
associated with National: .24 + .32 = .56.



Probabilities of future states
given that a customer trades
with Petroco this month

|
 |
  
 |
   ! " #

- For each starting state, the probabilities of ending up in either state in

month 3 sum to one.

robabilit of rade in
onth 3
tarting tate etroco ational um
etroco . . 1.
ational . .7 1.

|
 |
  $
 |
   ! " #
- Given that a customer initially purchased gasoline from National, the probability of
purchasing gasoline from National in month 3 is .08 + .64 = . 2.
- The probability of the customer trading with Petroco in month 3 is .@2 + .@6 = .28.



Probabilities of future states
given that a customer trades
with National this month

|
 |
  %
    | '

- The probabilities of being in a particular state in the future can be determined by using
°atrix algebra
- A transition °atrix includes the transition probabilities for each state of nature.

First month Next month

Petroco National
T = Petroco .60 .40

.20 .80
National 

|
 |
  &
 )!*  +
| ', 
- Symbology
Probability of trading with Petroco in period i, given customer started with Petroco: Pp (i);
Probability of trading with National in period i, given customer started with Petroco: Np (i).
- Probability of a customer¶s trading at National in month 2, given that customer initially
traded at Petroco is
Np(2)
- Probability of a customer¶s trading with Petroco and National in a future period i, given
that customer traded initially with national is
Pn(i) and Nn (i)
- If a customer is presently trading with Petroco, the following probabilities exist
Pp (@) = @.0 and Np (@) = 0
- These probabilities in matrix form:
[Pp (@) Np (@)] = [@.0 0.0]

|
 |
  (
 | '|
- Computing probabilities of a customer trading at either station in future months using
matrix multiplication. In future time periods, the 
 


   
Month 2: [Pp(2) Np (2)] = [@.0 0.0] .60 .40

.20 .80

= [.60 .40]
Month 3: [Pp (3) Np (3)] = [.60 .40] .60 .40

.20 .80

= [.44 .56]
Month 4: [Pp (4) Np (4)] = [..44 .56] .60 .40

.20 .80 

= [.38 .62]
Month 5: [Pp (5) Np (5)] = [.35 .65]
Month 6: [Pp (6) Np (6)] = [.34 .66]
Month : [Pp ( ) Np ( )] = [.34 .66]
Month 8: [Pp (8) Np (8)] = [.33 .6 ]
Month 9: [Pp (9) Np (9)] = [.33 .6 ]

|
 |
  -
  ) **
- State probabilities in future time periods:
[Pp(i) Np (i)] = [.33 .6 ]



The probability
Pp(i) for future
values of i

|
 |
  
  ! + ) **
- The probability of ending up in a state in the future is independent of the starting state.
- Computing future state probabilities when initial starting state is National:
Month @: [Pn(@) Nn (@)] = [0.0 @.0]
.60 .40

Month 2: [Pn (2) Nn (2)] = [0.0 @.0] 
.20 .80
= [.20 .80]
Month 3: [Pn (3) Nn (3)] = [.20 .80] .60 .40

 
= [.28 . 2] .20 .80
Month 4: [Pn (4) Nn (4)] = [.3@ .69]
Month 5: [Pn (5) Nn (5)] = [.32 .68]
Month 6: [Pn (6) Nn (6)] = [.33 .6 ]
Month : [Pn ( ) Nn ( )] = [.33 .6 ]
Month 8: [Pn (8) Nn (8)] = [.33 .6 ]
Month 9: [Pn (9) Nn (9)] = [.33 .6 ]

|
 |
  
 )
) **

-   
 


 are average, constant probabilities that the system will
be in a state in the future.
- For service station example:
.33 = probability of a customer¶s trading at Petroco after a number of months
regrdless of where customer traded in month one.
.6 = probability of a customer¶s trading at National after a number of months
regardless of where customer traded in month one.

|
 |
  
 )
) **
"  
#
- Combining operations into one matrix:
 p(2) p(2)
 @ 0
 .60 .40

onth 2 : 

 
 n(2) n(2) 
0 @ .20 .80
.60 .40

 


.20 .80
 p(3) p(3)
 .60 .40
.60 .40

onth 3 : 

   
 n(3) n(3) 
.20 .80 .20 .80
 p(4) p(4)
 .44 .56
.60 .40

onth 4 : 

   
 n(4) n(4) 
.28 . 2 .20 .80
.38 .62

 


.3@ .69

 p(9) p(9)
 .33 .6

onth 9 : 

  
 n(9) n(9) 
.33 .6 

|
 |
  
 . +* . !   )
)
 **
- At some point in the future the state probabilities remain constant from period to period.
- After steady state is reached it is not necessary to designate the time period.
-Steady state probabilities can be computed by developing a set of equations using matrix
operations and solving them simultaneously:
.60 .40

[Pp Np] = [Pp Np] .20 .80

Pp = .6Pp + .2Np
Np = .4Pp + .8Np
Pp + Np = @.0
Np = @.0 - Pp
Pp = .6Pp + .2(@.0 - Pp) = .6Pp+ .2 - .2Pp = .2 + .4Pp

..6Pp = .2
Pp = .2/.6 = .33
Np = @.0 - Pp = @.0 -.33 = .6
[Pp Np] = [.33 .6 ]

|
 |
  
   )
) **
" #

- Steady-state probabilities can be multiplied by the total system participants to determine

the expected number in each state in the future.
- Example: percentage of customers who will trade at a service station during any given
month in the long run given that there are 3,000 customers in the community who purchase
gasoline:
Petroco: Pp(3,000) = .33(3,000)
= 990 customers
National: Np(3,000) = .6 (3,000)
= 2,0@0 customers

|
 |
  $
   )
) **" #

- Re-evalution with modified transition probabilities resulting from Petroco improving its
service.
- Re-evalution indicates Petroco will get @,200 customers in any given month in the long
run, increasing its customer base by 2@0 customers (@,200 - 990).
- Management must evaluate trade-off of cost of improved service and increase of 2@0.
customers.
- Analysis with new transition probabilities:

. 0 .30

 
[Pp Np] = [Pp Np]  
.20 .80
solving, Pp = .2/.5 = .4
and thus, Np = @- Pp = @ - .4 = .6

|
 |
  %
 

 /'! |
 
| 0 

 

- Machine with daily transition matrix:

Day @ Day 2
Operate Breakdown
.90 .@0

T = Operate  
 
. 0 .30
Breakdown

.88 = steady-state probability of the machine¶s operating

.@2 = steady-state probability of the machine¶s breaking down
- Management must decide if it should decrease breakdown probability and if cost of doing
so is covered by profit achieved by resulting increased production.

|
 |
  &
 

 /'! |
 
 
1  !
- Rents in three states; trucks are rented on a daily basis and can be rented and returned in
any of the three states.
- Transition matrix: V M NC
Virginia .60 .20 .20



Maryland .30 .50 .20
 
North Carolina .40 .@0 .50


- Steady-state probabilities: Virginia Maryland North Carolina

[.4 @ .244 .285]
- Given total fleet of 200 trucks, long run expectation of number of trucks in each state:
[ 94 49 5 ]

|
 |
  (
 )    | 
- Once the system leaves a transient state, it will never return (state 3 in example):
@ 2 3
@ .40 .60 0

 

T = 2 .30 . 0 0
 
3 @.0 0 0

- A transition matrix is cyclic when the system moves back and forth between states:
@ 2
T = @  0 @.0

 
 
@.0 0
2
- Once the system moves into an  

 , it is trapped and cannot leave (state 3 in
example): @ 2 3
@ .30 .60 .@0



T = 2 .40 .40 .20
 
 0 0 @.0
3 
|
 |
  -
 .*/'!
" $#

- Example of absorbing state matrix.

- States are the months during which customer carries a debt.
- States: p = debt paid, @ = debt held @ month, 2 = debt held 2 months, b = ³bad´ debt.
- p and b are absorbing states.
- Transition matrix for A to Z Office Supply Company accounts receivable:
p @ 2 b
p  @ 0 0 0

 

T = @ . 0 0 .30 0 
 
2 .50 0 0 .50
 
 0 0 0 @

b

|
 |
  
 .*/'!
" $#

- Rearrange the transition matrix:

p b @ 2
p  @ 0 0 0

 

T = b  0 @ 0 0 
 
@ . 0 0 0 .30
 
.50 .50 0 0
2 

- Label submatrices:

 I 0

T =  

 Q

|
 |
  
 .*/'!
" $#
@ 0

 
- Form identity matrix, I =  
0 @

0 0

- Matrix of zeros, 0= 

0 0

- Transition probabilities of the debt¶s being absorbed in next period,

. 0 0

R=  
 
.50 .50

- Transition probabilities for movement between both nonabsorbing states,

0 .30


=  
0 0
|
 |
  
 .*/'!
" $#
- The ˜   

, F, indicates the expected number of times the system will be in
any of the nonabsorbing states before absorbtion occurs.
- Determine fundamental matrix, F:
F = (I - )-@
- Determine F by taking inverse of the difference between identity matrix, I, and :

@ .30

F= 
 
0 @

|
 |
  
 .*/'!
" $#

- The F ÔR matrix reflects the probability that the debt will eventually be absorbed given
any starting state.
- Calculating the the F Ô R matrix:

@ .30
. 0 0

FÔR=   Ô  
   
0 @ .50 .50

.85 .@5


= 
.50 .50

|
 |
  
 .*/'!
"$ $#

- Determine what portions of these funds will be collected and what will result in bad debts
given that company has accounts receivable of $4,000 in month @ and $6,000 in month 2.

.85 .@5

Determination of accounts receivable = [4,000 6,000] Ô 

.50 .50

= [6,400 3,600]

- Of $@0,000 owed, bank can expect to receive $6,400 and $3,600 will become bad debt.

|
 |
  $
 + ! 
2  *!$33&333%3
(

|
 |
  %