Solutions of assignment #2

1. Sol. (a) The main idea for this question is to condition on the rst outcome of and use the renewal property of the renewal event. rn = P ( occurs on trail n) (Summation over the rst occurrence time of )
n

=
k=1 n

P ( rst occurs on trail k and occurs on trail n) P ( rst occurs on trail k )P ( occurs on trail n| rst occurs on trail k )
k=1 n

= =
k=1

fk rnk .

In the above step, due to the renewal property, we have that P ( occurs on trail n| rst occurs on trail k ) = P ( occurs on trail n k ) = rnk . Since f0 = 0, we further have rn =
k=0 n

fk rnk .

(From our class, we note that it is somehow hard to nd fn . The above formula tells us we can nd it recursively.) (b) The main tool for this part is the convolution of two power series. First,

R (s) =
n=0

rn sn

= 1+ = 1+

rn sn
n=1 n

fk rnk sn
n=1 k=0 n

= 1+
n=0 k=0

fk rnk sn ( Note that f0 = 0)

= 1 + R (s)F (s). (
n=0 n n k=0 fk rnk s

is the convolution of R (s) and F (s).

Please come to check the denition of the convolution of the power series.) From the above formula, we can easily get that F (s) = 1 2. Sol. 1 1 . R (s)

(a) Note

E (V235 ) =
n=1

rn =

1 = . 103 n=3

So 235 is recurrent. (b)

R235 (s) =
n=0

rn sn = 1 +
n=3

sn = 1 +

(1 s) + s3 /103 s3 /103 = . 1s 1s

F235 (s) = 1

by Renewal Relation R235 (s) 1s = 1 (1 s) + s3 /103 (1 s) + s3 /103 (1 s) = (1 s) + s3 /103 s3 /103 = . (1 s) + s3 /103

So F235 (1) = (c)

1/103 , 1/103

then f235 = 1 and 235 is recurrent.

E (T235 ) = F235 (1) = 103 = 1000 < (P ositive recurrent). Note that f7 = P (X5 = 2, X6 = 3, X7 = 5, the rst 4 outcomes can not contain 235) = P (X5 = 2, X6 = 3, X7 = 5)P (the rst 4 outcomes can not contain 235) 1 = ( )3 (1 P (rst 3 outcomes=235) P (X2 = 2, X3 = 3, X4 = 5)) 10 1 1 = ( )3 1 2( )3 . 10 10 3. (a) The overlap for two consecutive TTHHTTTH is TTH. So E (TT T HHT T T H ) = E (TT T H ) + E (TT T HHT T T H ) = 1 1 + . 2 (1 p) p (1 p)5 p3

(b) The overlap for two consecutive TTTHHTTT is TTT. So E (TT T T HHT T T ) = E (TT T T ) + E (TT T T HHT T T ) = E (TT T T ) + Note that E (TT T T ) = E (TT T )+E (TT T T ) = E (TT )+E (TT T )+E (TT T T ) = So E (TT T T HHT T T ) = 1 1 1 + + . 2 1 p (1 p) (1 p)3 1 . (1 p)6 p2

1 1 1 1 + + + . 2 3 1 p (1 p) (1 p) (1 p)6 p2 2

(c) The overlap for two consecutive 262333262 is 262. So E (T262333262 ) = E (T262 ) + E (T262333262 ) = E (T2 ) + E (T262 ) + E (T262333262 ) 1 1 1 = + + = 6 + 63 + 6 9 . 3 1/6 (1/6) (1/6)9 4. Sol. (a) From the denition of delayed renewal sequence {dn } n0 , we have d0 = 0, d1 = d2 = d3 = d4 = 0, and dn = (1/6)5 , for n 5. From the denition of the associated renewal sequence {r n } n0 , we have r 0 = 1, r 1 = 0, r 2 = (1/6)2 , r 3 = 0, r 4 = (1/6)4 , and r n = (1/6)5 for n 5. So the generating function D (s) is

D (s) =
n=0

dn s =

(1/6)5 sn =
n=5

(1/6)5 s5 . 1s

The generating function R (s) is

R (s) =
n=0

r n sn = 1 + (1/6)2 s2 + (1/6)4 s4 +

(1/6)5 s5 . 1s

From the formula in the handouts, we have F (s) = D (s) (1/6)5 s5 . = R (1/6)5 s5 + (1 s){1 + (1/6)2 s2 + (1/6)4 s4 } (s)

So F (1) = 1 and the probability that occurs at least once in the sequence is 1. We can use E (T ) = F (1) = 6 + 63 + 65 or by renewal theorem, E (T ) = E (T121 )+E (T12121 ) = E (T1 )+E (T121 )+E (T12121 ) = (b) From the renewal relationship, we have F (s) = 1 1s 1 =1 . 5 s5 + (1 s){1 + (1/6)2 s2 + (1/6)4 s4 } R ( s ) (1 / 6) 1 1 1 + + = 6+63 +65 . 3 (1/6) (1/6) (1/6)5

So F (1) = 1, which means 12121 will reoccur with probability one if 12121 occurs once. By renewal theorem, we have
5 E (T ) = 6 .

(c) P ( will occur innitely often in the sequence) = P ( will occur at least once in the sequence) P ( will occur innitely often in the sequence | occurs once) = 1 1 = 1. The expected number of tosses required to see this event 5 times is 6 + 63 + 65 + 4 65 . 5. First, we need to note that T00 = 1 + T20 , where T20 is the rst waiting time for 20 . (a,b) Note that f00 = P (T00 < ) = P (T20 < ) = and E (T00 ) = 1 +
2 q p

1 q 1/2 (q/p)2 q < 1/2

2 1 + q p

q > 1/2 = q 1/2

q > 1/2 . q 1/2

So, when q > 1/2, 00 is positive recurrent; when q = 1/2, 00 is null recurrent; and when q < 1/2, 00 is transient. (c) Note that E (V00 ) = f00 = 1 f00
(q/p)2 1(q/p)2

q 1/2 . q 1/2
15 q p

6. (a) E (the number of games required f or her to go bankrupt) = E (T0,15 ) = (b) P(Marys worth reaches 3000 at some point)

= 75.

= P (M ary s worth reaches 3000 bef ore she goes bankrupt) = P15 by the gambler s problem, here i = 15, k = 30 1 (q/p)15 = 1 (q/p)30 = 0.002278. (c) Let = M ary s worth reaches 3000 at some point. Let us call it one cycle whenever Mary goes bankrupt or reaches $3000 and A = M ary reaches 0 bef ore $3000 in the f irst circle. Then P () = P (|A) P (A) + P (|Ac ) P (Ac ) = (1 0.002278) P () + 0.002278 1 = 1.

(d) P (T0,15 < ) = [P (T0,1 < )]15 = ( 0.4 15 ) = 0.002284. 0.6

Dierence: For (b), 0 acts as an absorbing barrier and the game ends as soon as Marys net worth hits 0. For (c), each time Mary goes bankrupt before reaching $3000, Mary receives an instant infusion of cash from her aunt and her worth becomes $1500 again. The game then starts over from the beginning. So we can look at the whole procedure as a geometric distribution with probability p = 0.002278 of reaching $3000 (30) on each try. Therefore, Marys worth will reach $3000 (30) at some try (waiting for Success in a geometric distribution is a recurrent event). For (d), if Mary goes bankrupt before hitting $3000 (30), then Marys worth will become 0. Moreover, she can continue to move to the left in the walk (arbitrarily far into debt) if she continues to lose. By contrast, in (c), hitting 0 is a kind of bouncing barrier that kicks her back up into state 15. All in all, Part (b) is the Gamblers ruin model or the random walk with two absorbing boundaries 0 and 30; Part (c) is the random walk with one reecting boundary (reecting to 15 after reach 0) and one absorbing boundary; Part (d) is the random walk with one absorbing boundary at 30.