UPPSALA UNIVERSITET Markov Processes, 1MS012

Matematiska Institutionen Assignment 1, 2022

Örjan Stenflo

Assignment 1
Due: February 3, 2022

1. A Markov chain, (Xn )∞ n=0 , with state-space S = (0, 1, 2) has transition probabilities
p00 = p01 = p11 = p12 = p20 = p22 = 1/2.
Suppose P (X0 = 0) = P (X0 = 1) = P (X0 = 2) = 1/3. Determine P (X2 = 0, X4 = 0).

2. A Markov chain, (Xn )∞

n=0 has state space S = (1, 2, 3, 4, 5, 6), and transition probability
matrix  
0.5 0.5 0 0 0 0
 0.5 0.5 0 0 0 0 
 
 0 0 0.25 0.75 0 0 
P=  
 0 0 0.75 0.25 0 0  
 0.5 0 0 0.3 0.1 0.1 
0 0.2 0.2 0.2 0.2 0.2

(a) Express S as a disjoint union of sets of transient states and closed irreducible sets
of recurrent states.
(b) Calculate
P (Xn = j for some n ≥ 1 | X0 = i),
for any i, j ∈ S.

3. Consider a standard deck of cards labelled 1, .., 52. Every second we shuffle the cards by
drawing a card at random and placing it at the top of the deck. Calculate the expected
amount of time (in years) until the deck returns to the original order. (Hint: consider
a Markov chain on the set of permutations of {1, ..., 52}.)