MAT442 Assignment 6 Winter 2023

Due on April 7.
Assignment Problem 1. Every time that a team wins a game, it wins its next game with
probability 0.8; every time it loses a game, it wins its next game with probability 0.3. If
the team wins a game, then it has dinner together with probability 0.7, whereas if the team
loses then it has dinner together with probability 0.2. What proportion of games result in
a team dinner?
Assignment Problem 2. Consider a Markov chain with states 0,1,2,3,4. Suppose P0,4 = 1;
and suppose that when the chain is in state i, where i > 0, the next state is equally likely
to be any of the states 0, 1, . . . , i − 1. Find the limiting probabilities of this Markov chain.

Assignment Problem 3. Show that the stationary probabilities for the Markov chain
having transition probabilities Pi,j are also the stationary probabilities for the Markov chain
whose transition probabilities Qi,j are given by
k
Qi,j = Pi,j

for some fixed positive integer k.

Assignment Problem 4. For the Markov chain with states 1, 2, 3, 4 whose transition
probability matrix is  
0.4 0.2 0.1 0.3
 
 
0.1 0.5 0.2 0.2
 
P=



0.3 0.4 0.2 0.1
 
 
 
0 0 0 1

find fi3 and si3 for i = 1, 2, 3.

Assignment Problem 5. For a branching process, calculate π0 when

(a) P0 = 1/4, P2 = 3/4.

(b) P0 = 1/4, P1 = 1/2, P2 = 1/4.

(c) P0 = 1/6, P1 = 1/2, P2 = 1/3.
Assignment Problem 6. For a time reversible Markov chain, argue that the rate at which
transitions from i to j to k occur must equal the rate at which transitions from k to j to i
occur.