Chapter 17 Markov Chains 17-4Set 17.1a States: Models M1, M2, and M3 MIM? M3 mi{ 065 02 «Ol M2 06 Ors | ox) M3 05 on os sl: car on patrol s3: car at call scene Si 82 s2: car responding to a call 4: apprehension made. 85 transport to police station 3 s4 3 States: QO, QI, Q2, Q3, Q4, paid, bad debt p{bad,bad}=P {paid.paid}=1 P{Q0,paid}=2000/10000, P{Q0, P{Q0.Q2}=3000/10000, P{Q0,Q3 P£QI,paid}=4000/25000, P{Q1,Q2}=12000/25000, P{Q1,Q3}=6000/25000, P{QI.Q4) 13=3000/10000, Input Markov chain: Q0_a1 a2 03 a4 PAID BAD aof.00 30 30 20 00 «62 on at}.00 00 48 24 12 16 00 @2|.00 00 00 30 55 15 09 @3|.00 00 00 09 16 = 09 a4|.00 00 00 09 00 650 50 PAID} .00 .09 .00 .00 00 1.00 00 BaD | .00 .00 00 00 .00 001.00 States: dialysis, cadaver transplant, living donor transplant, >lyear survivors, death 17-2Set 17.2a M1 M2 M3 Mt 0.65 02 0.15 Initial probabilities: M2 08 015 025 Qo 1 G2 3 a4 PAID BAD M3 03 ot os 02 01 03 02 0 0 02 ee eee Input Markov chain: ee Qo at a2 a3 as PAD BAD Mi 0.6175, 0.175 0.2075 Ot | 00) ane 29 2 oe os M2 0.605 0.1675 0.2275 G1}.00 00 48 24 12 016 00 io | ose ee a @2].00 00 100 30 55 015 09 PEMIMI}=.6175 : Se ee Pana tee a4].00 00 00 .00 00 050 50 PIM3IM3}= 26 PAID|.00 .00 00 00 .00 1.00 09 Bad | 00 00 00 00 00 0.00 1. 2 Output (2-step) transition matrix Qo a1 a2 a3 O4 PAID BAD Initial probabilities: Q0 [00.00 14.16 23 0.46 0.00 sl S23 St Ss Qt] .00 00 00 14 30 0.49 0.06 er ra) 2] .00 00 00 .00 05 068 028 Topi Mavky ehala? 3.00 00 .00 .00 .00 092 0.08 a Sie 4] .00 .00 .00 .00 .00 050 0.50 al oie oocgeuaee crass PAID |.00 .00 09 .00 .00 1.00 0.00 S82 On 03 06 0 0 BAD [.00_.00_.09 00.0 9.00 _1.00 33 o1 0 05 04 0 st nn ee 85 1 0 0 o 0 Output (2-step or 2 patrols) transition matrix P ‘Absonate si 2 ss S485 State $500, SI 0.22 042 0.36 oO 0 Q0 oO oO 82 013 015 048 024 0 at o 0 83 0.25 0,06 0.25 02 0.24 Q2 0.0288 14400 S4 0.76 0.24 oO 0 0 Q3 0.0468 23400 S83 O4 0.6 o o o Q4 0.09124 45620 Absolute 2-step probabilities =(0 0 1 0 0)P* PAID 0.52866 264330 as BAD 0.3045 _ 152250 a '$500,000 31 0.25 s2 0.06 83 025 st 02 85 024 P{apprehension, $4, in 2 patrols}=.2 17-3Set 17.2a 4 ® orig ° Initial probabilities: Sr Day Chansons _>ty$ Dash ul o o o oO 2 7 . o 4 2 oe Tp di Dialy _CTrans__LTrans_>lyr8_Death Dialy Trans _LTrans_>lyi8. a Do gama] | ed | so CTrans 03 eo a a he ‘LTrans | 0.15 oO 0 07 OL time | 0380s aa i ee ele ee oe welts 8 8 elf om Lo up emanate Dat Cham Listes Dah Bah _Isic_tav_atu8 Death | 45 Foray oom oe 0x O37 Dialy ue oe hed CTrans | 0.1128 0.0425 0.014. 0465 0.37 Grins | 0178 000m 048025 Lite | Gow? oeatt eott oc 026 itm [ors 00 dois oem ois] | Mame] G07 OCT gon oem 026 Sins |"om 00 to ost t ge EG bet | oy 1 Absolute Cm sue Gay Day ost bead 0.355 CTrans 0.02423 Cras os ‘L Trans. 0.05 se eee ae 022s = = Deh o3tm9 Ptransplant)=.154.05—2 P{surviving 4 more years} =.68197 extund 17-4Set 17.3a 1 (@) Using excelMarkovChains.xls, all the states of the chain are periodic with period 3. 01 0 007 P=|0 0 1p =|1 0 of, 100 910, f00 010 P=/0 1 o|,pt=|0 0 1 OO] 100 (b) States 1, 2, and 3 are transient, State 4 is absorbing, (© State 1 is transient. States 2 and 3 form a closed set. State 4 is absorbing, States 5 and 6 form a closed set. (@ All the states communicate and the chain is ergodic. 17-5inner sub rural total= 400 appellate tases) Chicago and Atlanta will have pea space availability problem SS ‘ean un time Se _(wks) _ Phx 32.17 Den 4.09 Chi 242 Atl @ Tally of i followed by j @) o 1 2 3 sum Initial probabilities: 0 [oes 8 Equal initial probabitties a1 2 3 7 Phx Den Chit! 2,2 3 4 7 [0.25 0.25 0.25 0.25 | Sa 7 | Input Markov chai = chain: Phx Den Chi All ano Te 2 3 Pix] 0.7 0.08 0.18 0.06 0.25 025 0.425 037 Den} 0 07 0.418 0.12 0.28571 0.142857 0.285714 0.28571) chi] 0 015 07 0.45 0.28571 0.428571 0.28571 Atl | 0.03 0.03 0.24 07 © Output Results ‘Steady Mean rim sate stte___time 0 0.275862 3.624995 1 0218779 46343799 2 0.270638 3,6949792 3_0.237722 42065916 = 0.275862 continued. continued. 17-8Set 17.4a & 14 Results (@) State=(ij.)-(¢ in yr -2,¢ in yr, in curyt) a a ij, k= or) = Example: (1-0-0) this yr links to (0-0-1) if a contract is ‘State state. time, secured next yr. 0 144578 6.9166613 Of OG GO fF tO 1.180723 5.5333285 OO tO fH OF 1 2 240964 4.1499977 p99 __1_0_4_4_4 3 361448 2.7666647 Como 8 8 8 6 og 4 072289 13.833323 100] 2 0 0 8 0 0 0 0 010 | 0 2 0 0 0 8 0 Ol Ay dstops bet 13.83 00-4 o 0 2 0 0 0 8 oO mand a 140] 0 3 0 0 0 7 0 0 101} 0 0 3 0 0 0 7 0 ort | 0 0 0 0 3 0 0 71] (Fines paid=$400 +1 Loo o o 5 0 0 5 State _state 0-0-0 014859 1-0-0 066865 0-1-0 066865 0-0-4 066865 11-0 178308 1-0-1 178306 O41 1783068 = 249629 | Expected # contracts in 3 yrs = | 1(.066865+.066865+.066865)+ 2(.178306+.178306+.178306)+ 3(.249629)= 2.01932 Expected # contracts/yr=2.01932/3=67311 (@) States:0, 1, 2,3, 4 15 Input Markov chain o 1 2 3 4 of 5 5 0 0 o 1] 0 6 4 0 Oo 2} 0 0 7 3 0 3} 0 0 0 8 2 41 0 0 0 0 continued. 17-12Set 17.5a a 4 HAA +) oa (a) Initial probabilities: 3 1 875 oe : 2 3 a 5 4 7 3 51.3333. 1 o o 0 o Mu Terr : 7 33333333 0 2) 3.8333 3333 0 3333 Ce 3) 3.8333 33333333 0 0 3333 4 | 3.3333 Ros Bal 2 = = ce Te aie ot HS oe cee eet SOOT ARE 4 oO 0 0 1 0 Com SU se (c) Hs = 214286 Matrix P: © jo Aa el ee ees ceteris 5 oO 0 0 oO 1 5 EN i 2 3 A ane 2 3 4 5 1 1 +333 333 +3333 4 S 0 oO 0 5 inv-N) bad hesinesereeaccn: af Fa tas aa] saa .. 3 1 (8 18667 4.4666} 4.4666 es 1 2 3 4 4 7 6 1.0667 4.2666 4.2666 2] -333 1 -333 0 | | (as opposed to 4.6666 in Part () of Problem 1) a 17-13Set 17.5a @ Matrix I: 1 @ [4 Initial probabilities: (im-Joe=(-j) S| a 32.23 4 4d 0550 Blo io 0 0 0 0 4] 0 Tnput Markov chains s| 0 3223 14 4 Os 50 s|_o sefo os 0 5 0 oO Matrix 23/5 0 $ 0 0 o 32.2 wu} o 5 0 0 5 0 0 41/5 0 0 0 0 8 5 os} 30 0 0 0 7 0 o sol3 0 0 0 0 7 5 as © 3 Output (3-step) transition matrix 3223 It 41 05 50 | 05 EN 32 [075 375 «025 as a7s a) Bais Gs 0 is ss 23/-5 1 «3 0 0 14] 405 328 0 27 uw} o a 1 0 0 41/335 075 2s 075 07 er 05 | 297 105.075 108343075 sol-.3 0 0 0 3 $0 [297 105 075 105 _0_a18 | inv(N) nfo f 23 5 P{Joe wins in 3 tosses}=P{3-2-+0-5}=.125 wl 4 4 2 2 33 ‘P{Jim wins in 3 tosses}=P{3-2—»5-0}=.175 14 2 2 2 i 41] 6 4 2 4 67 © sol 6 4 2 3 83 Output Results Ma ‘Abeohute Steady Mean return 0-5 Site_Gatep) state time 32 ‘expected number of| 320752571433, 8888801 23] 153] tosses till Joe wins 23 378171429 5.8333335 14] 87 14 0 085714 1.666665 41) 22.7 41281285717. 7777801 50 L233 0S 2S .142887 7.000019 50178214286 4.666668 13-0 EN 32[ 1 a 0-30 P{game ends in Jim’s favor}= 1, . = a {game ends in Joe’s favor}= 1,5 =.143 a So 4 Oo 08 oo 0 3 continued. continue. 17-14Set 17.5a inv(l-N) 2 [aes 2) 23] 4 4 2 233 ws] 4 33 267 2 44 41] 2013 67 2 1 50|_4 27 1332 56 Mu 05 3-2 [BZ] —expocted number of 2:3] 13.3 | tosses tll Jim wins 14] 16.4 4a] 71 50 [15.6 @) Input Markov chain: pink __red__orange white pink 2 oO oO 4 red 5 4 1 0° orange 5 0 25 25 white Sine, Oe ) Initial probabilities: jink __red__orange white aes) ‘Absolute Steady State __(6-step) state pink 0.55555 0.555556 red 0.00256 o orange 0.00179 0 white 0.4401 0.444445 After 5 years, 56% pink, 44% white. Red and orange will vanish. Approximately same result in the long run. © My JeM{white) pink red _ orange pik (igeeaccemo ° red] 58 ot orange|__-5 075 continued, inven) Mu pink red orange white pink [2.5 0 oO] 25 red | 2.36111 1.66667 .22222| 4.25 orange | 1.60867 o 133333] 3 5S years from red to whit A Bc [ts 1 16 A B 2 05 c 125 75 A 394737 B 307018 | rn A: 39.5%, B: 30.7%, C: 29.8% © LN i=2@) A Al 25 c L125 inv) A c A (5.71429 3.42857] A cL2ss7i4_ 5.71420 B 14286 | 8.57143 i=3() AB af -a BL.2 25 1 2 A 588235 2.35204] A B|_4.70588 5.88235 | B c | 8.23529 | 1.5882 AB: 9.14 years AOC: 8.23 years 17-15Set 17.6a 1.07 | a_np =| 27 16 12 08 04 (-N)'A= 1.02 98 1.07 1.03 0 1.07 0 0.07 84 88 92 96, 0.93 0.98 1.02 1.07 Labor cost={S20*{1.07(30/60)+.98(20/60)] + $18[1.02(10/60)+.93(10/600]}/(.84) average. a) States: Iwk, 2wk, 3wk, Library Matrix P: + 2 3 1 [003 0 o7 2 | 0 0 o1 08 3 |o0 0 0 14 ® oe eed © inven) Mu 12 3 tb 1) 1°03 03] 1) 4.33 2}o 4 of 2! 14 } slo o af 3[_4 Ikeep the book 1.33 wks on the @ Matrix P: 3 See 20 1 6 2 0 0 3} 0 0 4 oo 5 40 6 + 0 ° o 4 inv(-N) 4 2 3 4 ss 1 [75865 09774 057i 0.3008 0.1203 2| 1.4062 2.4436 1.4286 0.7519 0.3008 3| 12857 2.1420 27143 1.4286 05714 4| 10150 16917 21429 24496 0.9774 5 [06090 _1.0150_1.2857 1.4662 1.5865, Mu Piito) Absorption 6 0 1 [ s5s6301 | 1 [o.oa 052 2| 6300077 |2| 012 o8e 3 |istazasr'| 3 fo229 077 4] 8270677 | 4| 0.301 0.609 5 | 5.962406 | 5 | 0.635 0.365 (b)Average # of bets to termination =8.14286 (©) Phowin double}=.229, P {lose all}=.771© Pate pve (Peat at 57}=44 2 M (4) P{ofT payrol}=3.25 years 1 [0628 Dears 2 | 0625 0375 3 | 0625 037s 4 | or ® 6 P{Master}=.375 (d) Q4_PAID_BAD Expected pay= : 0 2770 $15,000(5%.625+3%.375)=$63,750 at a 5 az 1500 as} BO : as 5 5 (@) States:55, 56, ons 7; G Matrix P BAD o 4 55_56_ 57 i 5 Qi_a2 a3 a4 | 0 0 49 0 3 46 at 35] 249 2 at 1 48 38 45) 2.31 oO 2 o 1 3 8| 19 e as 0 0 64 46} 1.16 z a4 OOO Ot : Expected # qrirs till absorption= 2.49 o ) PGitoD 5758 59 60 __61_| PAID BAD 8 7 61 83 448 89 78 68 59 498 1 88 7 68 56 0 1 87 75 636 rr ee ee 0 0 0 4 (8s d 00 0 0 4 | P{Q0-+Paid}=.83 Mu Piitop °o e2a__62 Q Ls 499] 448 —~562 444| 498 502 ao [249 3.38 58 Mm at} 231 3.25 636 364 oe) 259 731 269 as} 1.16 185 85 15 asl _4 1 1 ° Pirie at OO= Nbr. of qrtrs tll settled = 1.9 contoued. 17-17Set 17.6a (a) State (i-j)=(Sets won by Andre-Sets won by John) Matrix. P: 30 03 13 31 32 23 20 24 12 o2 0.0 0-4 22 +9. 13 at 32 (b) invi-n) 22 Cs +9. ot 48 8 ° 16 o2 9 14 12 20 24 22 qgeeegee zy sequeagsg g # 10 23 oo ot 02 +0 nT 12 20 24 407 327 1.98 2.093 2.48 16 1.56 14 Mu 22 00 on o2 +0 4 12 20 24 22 Average # of sat ead of match= 407 Probability Andre will win -3 0s favor, oF in TWO set it sum of (PsoHP31#Ps) given 0-0 start= 69 can be 2-3 (P's favor) or 3.2 (A’s favor). The average numberof sets to termination i thus more than 1 and les then 2 (= 1.6). (@) The average numberof sets ill termination is 1.6. In ONE se the termination score can be | (c)P{AAndre wins eurent score 1 17-18Set 17.6a 10 @ @ ba Matrix P: Matrix P: ies F o 1 2 3D 1 2 8 oO oO oO 5 5 0 0 0 2) oo a ae ° 1] 4 0 68 0 0 3 0 oO 225 oO 2 3 o o 7 oO 4 Le Zz 3) 2 0 0 0 8 FL_o oo 1 plo o 0 o 4 (b) States: 0, 1, 2,3, Delete inv(I-N) Mu 1 2 3 4 F 4/125 1.282 1.333 1420] 1[5.29 inv(-N) Mu 2} 0 1.282 1.333 1.420| 2] 4.04 o 1 2 3 oD 3} 0 0 1333 1420] 3] 2.76 ° of 2] 4[_o0 0 0 1420] 4] 143 1 4] 9.96 a: 2| 6.96 (©) Tobe able to fake Cal I, the student must finish in | 3 3| 3.39 16 wesks (4 rastons) ories. Average mmber of | @y transitions needed = 5.29. Hence, an average student ‘will not be able to finish Cal Ton time. ‘A new customer stays 12 years on the list (Ne (c) 6.96 years ® 215 14 States: 0,1, 2,3, 4, $, promotion — states: 108, 109, 110, 111, 112, 107,113 Matrix P: 108109 110 411 112 107 113 o i 2 3 4 5 P 108 “o] 33 0 eee ee een 109 o| o o 1 Oo Se 08) 0 110 o} o oO 2) 8 6 2 2 4 0 of | in 333] 0 0 3 oO O. 0 2.4 Al o 112, 333. 0 33 400 OO By 4 107 o 1 0 50 oo oo) 1 113 oo 1 p[ oo 0 0 0 o 14 o) () inven) Mu Oso) Oe of 125 1008 4113 411 11 9]0[ 057 1] 0 425 1004 111 4.4 89/1] 546 gan ana 2] 0 0 425 109 11 29/2] 435 re 3] 9 0 0 425 44 o/3}aas|/ $98) 28 2 18 1S Hee fe 8 Bee 0] 15 3 450 3815 sLo_o 0 o o sjs|_aj) 4 [5 ie 412 145 2 25 It takes 6.57 on the averages to be promoted. contrued 17-19Set 17.6a mu Pitoi) absorb 107113 tos[ 7.5] 108[ 83 17 tos] 12) 109| 67 33, 110) 135) 110) 5 5 111 12] 111] 33 67 a2{_75| 12| 17 83 ‘The last two columns (low=107, high=L13) provide the answer as a function of the current voltage. For example, if current voltage is 109, P{low}=.67, P{high}=.33 © Setting current voltage at 110 guarantees an average time to failure of 13.5(15)= 202.5 minutes. 12 Dialysis _1st¥rc_1st¥r._>tyrS_Death Dialysis fi tst¥rc O20 25] 2 1st S51 >tyrS 0 o 8} 05 Death oo (a) # years on dialysis3.54 years. (b) Longevity = 12.92 years. (©) Life expectaney = 16.46 years (@) 14 years. (©) >lyrSurvivor has the highest longevity (= 16.46 years) and the least number of years on dialysis (= 1.7699 years). 17-20