Math 180B, Winter 2010

Homework 4 Solutions
Ex. III.1.4.
P[X1 = 1, X2 = 1|X0 = 0] = P0,1 P1,1 = .1 .2 = .02,
and P[X2 = 1, X3 = 1|X1 = 0] has the same value because the transition probabilities do
not depend on the absolute time.
Ex. III.1.5.
P[X0 = 1, X1 = 1, X2 = 0] = .5 .1 .5 = .025.
P[X1 = 1, X2 = 1, X3 = 0] = P[X1 = 1] .1 .5 = .05 P[X1 = 1].
But
P[X1 = 1] = .5 P[X1 = 1|X0 = 0] + .5 P[X1 = 1|X0 = 1] = .5(.2 + .1) = .15,
so
P[X1 = 1, X2 = 1, X3 = 0] = .05 .15 = .075.
Pr. III.1.4. If Xn = k then a transition of the form k k + 1 occurs if and only if
n+1 = k + 1, and a transition of the form k k occurs if and only if n+1 k. No other
transitions have positive probability. The transition matrix is therefore
0.1
0
P=
0
0

Ex. III.2.2. We have

0.3 0.2 0.4

0.4 0.2 0.4
0 0.6 0.4
0
0
1

2
1
2
P =
1
4
1

2
1
P3 = 3
8
3
and

6
1
4
P =
5
16
5
1

3
2
3

1
1,
2

3
3,
2

5
6
5

5
5.
6

1
2
1

Therefore,
P[X0 = 0|X0 = 0] = 1,

P[X1 = 0|X0 = 0] = 0,

P[X2 = 0|X0 = 0] = 1/2,

and
P[X3 = 0|X0 = 0] = 1/4,
Pr. III.2.2. If

P[X4 = 0|X0 = 0] = 3/8.

1
P=

,
1

then by straightforward (if tedious) computation (or by Mathematica) the element in row
0, column 0 of the matrix P5 is 102 (1 )3 + 54 (1 ) + (1 )5 . This is the required
probability.
Pr. III.2.5. Bearing in mind that P[X0 = 0] = 1,
P[X3 = 0|T > 3] =

P[X1 6= 2, X2 6= 2, X3 = 0]
P[X1 6= 2, X2 6= 2, X3 6= 2]

The numerator in this expression is equal to

P0,1 P1,1 P1,0 + P0,1 P1,0 P0,0 + P0,0 P0,0 P0,0 + P0,0 P0,1 P1,0
= (.2)(.7)(.3) + (.2)(.3)(.7) + (.7)3 + (.7)(.2)(.3)
= 3(.042) + .343 = .469.
The denominator is equal to the numerator plus
P0,1 P1,1 P1,1 + P0,1 P1,0 P0,1 + P0,0 P0,0 P0,1 + P0,0 P0,1 P1,1
= (.2)(.7)(.5) + (.2)(.3)(.3) + (.7)2 (.2) + (.7)(.2)(.5)
= 2(.07) + .018 + .098 = .256.
Thus,
P[X3 = 0|T > 3] =

.469
= .647.
.469 + .256

Ex. III.3.2. For k = 0, 1, 2, . . . , N 1, a k k + 1 transition occurs if the chosen ball

comes from urn B and then urn A is chosen to receive it; this happens with probability
(1 k/N )p. Likewise, for k = 1, 2, . . . , N , a k k 1 transition occurs if the chosen ball
comes from urn A and then urn B is chosen to receive it; this happens with probability
(k/N )q. Finally, for k = 0, 1, 2, . . . , N , a transition of type k k occurs is the chosen
2

urn is the same as the urn from which the chosen ball came; this happens with probability
(k/N )p + (1 k/N )q. These are the only transitions with non-zero probability.
Pr. III.3.1.

1
P = 15
0
0

14
15
4
15

0
0
11
15
6
15

0
0
.
0
9
15

Pr. III.3.4. Notice that the probability that the service of customer currently being
served is finished at the end of the current period is ; this is the memoryless property
of the geometric distribution. A k k transition can happen in two ways: (i) a new
customer arrives and the customer being served finishes, or (ii) no new customer arrives
and the customer in service does not finish. The total probability of a k k transition is
therefore + (1 )(1 ) = 1 . (If k = 0 this formula must be emended; the
probability of a 0 0 transition is simply 1 .) A k k + 1 transition occurs if a new
customer arrives but the customer in service does not finish; this has probability (1 )
( if k = 0). Finally, for k 1, the probability of a k k 1 transition is (1 ).
Pr. III.3.7.

0 a1
1 0

0 1

0 0
P=

0 0

0 0
.. ..
. .

a2
0
0
1
0
0
..
.

a3
0
0
0
1
0
..
.

a4
0
0
0
0
1
..
.

..
.