Classification of Markov chains. Definition (Homogeneous Markov chain) LON LE RCO ed eden cutee ce Rc Ru clad an ra COs ene nee , Definition (Regular Markov chain) A Markov chain is called a regular markov chain if its tpm is regular. eetu 0.75 0.25 0 0.56 0.31 0.13 Consider P= | 0 05 0.5], then P? = ]0.30 0.45 0.25] Therefore P is regular. naan PCa ae eeu ene Pane) (ee) Consider P=] 0 1 0|,thenP?=|0 1 0],P=| 0 1 0 001 0 0 1 | Since P, P?, P® have zeros at identical places, therefore P is not regular. Oat ecgDefinition (Irreducible (ergodic) and reducible Markov chains) A Markov chain is said to be irreducible or ergodic Markov chain when pi? Se ee Rae} eee cue see aL Note- (i) In an irreducible Markov chain, every state can be reached from every other state in finite De ee ese et ae ea ace cra (ii) If a Markov chain is irreducible, then all the states are recurrent. (iii) A Markov chain may be ergodic but not regular. Set ecgetna a) Enya) le ae 0 0.5 0.5], then P? = 30 0.45 0.25] Therefore P is irreducible. en 045 0.35 0.20 Sere 0 015 085 0 jer p_|032 9 0 068 Consider P = eae) eee) aur) 0 064 036 0 (a a eZ) 0 050 050. 0 (OZ pe [Ow oP OD ro ys ay cn vee a CU ss ho) 0 051 049 0 |’ ci nwa) 0 0.50 050 0 030 0 0 070 CeCe a ee a Therefore we see that all even powers of P have zeros at identical places and all odd powers of P have zeros at identical places. Therefore P is not regular. But P is ergodic as every state can be reached ee een ae Oe ae cgera Three boys A, B and C are throwing a ball to each other. A always throws the ball to B and B always throws ball to C, but C is just as likely to throw the ball to B as to A. Find the tpm and show that (i) the Markov chain is irreducible and regular. and (ii) all the states are aperiodic Le o 1 0 can Ce eae ea) id a we ean i PnP a SUP ees neat Lae aes Oe ee ee oe EC cae ee (i) Since pQ) icc ello Oat ecgry a (ee eres emer meres Pl=|1/4 1/4 1/2],P°=|1/4 1/2 1/4 SU, eee Ue BURR Cee esse (ii) Since pS > 0, pQ) > 0, py) > 0, SS Bee ee Re ace Pe eRe eek Oe Ee Poe Oat ecgenna 0 2/3 1/3 Dees CRUG Rane Cae me ed Ue ea UC o1/2 1/2 0 seal Reo aL a 0 2/3 1/3 Soln. Let P= ]1/2 0 1/2 eee eee ROCCO UR RC ee ee ey a Ciel PSererererrey ie peed ei errs Cree tind cei oie nod eed cy Since p\") > 0 for n = 2,3,4,...Therefore the period d; = ged {2,3,4,...} = 1 ie. all the states are peers Set ecg