Understanding Limiting Distribution of Markov

Chains
Cai Yufei (zhmgczh@foxmail.com)
May 12, 2022

Suppose there is an irreducible, aperiodic and positive recurrent Markov

chain {Xn } with k states. Assume

the stationary distribution of the Markov
chain is π = π1 π2 ... πk .
Then we have known that limn→∞ Pijn = πj for i, j = 1, 2, ..., k.
The n-step transition matrix is given by
 n n n

P11 P12 . . . P1k
P21 n n n
P22 . . . P2k
Pn =  .
 
. .. .
 .. .. . .. 
n n n
Pk1 Pk2 ... Pkk
.
Then    
π1 π2 ... πk π
π1 π2 ... πk  π 
 
P∞ = . ..  =  .. 

.. ..
 .. . . .  .
π1 π2 ... πk π
.

Now assume the initial state X0 has the distribution a = a1 a2 ... ak
Pk
where ai ≥ 0, i=1 ai = 1 and ai = P (X0 = i) for i = 1, 2, ..., k.
Then aP is the distribution of X1 with P (X1 = i) = (aP )i , i = 1, 2, ..., k,
aP 2 is the distribution of X2 with P (X2 = i) = (aP 2 )i , i = 1, 2, ..., k, . . .
∀n ∈ {1, 2, ...}, aP n is the distribution of Xn with P (Xn = i) = (aP n )i ,
i = 1, 2, ..., k.
As a result, aP ∞ is the distribution of Xn when n is sufficiently large.
(Actually we can use ϵ − N language to describe it, but we do not need.)
Now, we want to figure out what aP ∞ is exactly and whether aP ∞ depends
on a or not.

1
 Actually,
 
π
π 
aP ∞ = a .  (1)
 
 .. 
π
 
π
π 

 
= a1 a2 ... ak  .  (2)
 .. 
π
k
X
= ai π (3)
i=1
Xk
=( ai )π (4)
i=1
=π (5)

.
So we can conclude that no matter what initial distribution from which a
Markov chain starts, the limiting distribution of the r.v. Xn will never change
and will always be the stationary distribution π when the Markov chain is
irreducible, aperiodic and positive recurrent.
In other words, when n is sufficiently large, P (Xn = i) ≈ (aP ∞ )i = πi (or
limn→∞ P (Xn = i) = (aP ∞ )i = πi ) for i = 1, 2, ..., k.
This conclusion depends on the fact that limn→∞ Pijn = πj .