Stat 150 Stochastic Processes

Spring 2009

Lecture 11: Return times for random walk

Lecturer: Jim Pitman

Recurrence/Transience
From last time: look at the simple random walk on Z with p q . Sn := a + X1 + + Xn . Compute: u2n = P0 (S2n = 0) f2n = P0 (T0 = 2n) T0 = rst return time to 0 = inf {n : Sn = 0} (= if no such n exists) We already know that 1 q P0 (T0 < ) = p p + q 1 = 2q p q q + p 1 = 2p To see this, P1 (T0 < ) =
q p

if p = q = if p > q if p < q

1 2

if p q if p < q

let P1 (x < ) = x, then x = q 1 + p x2 = x = 1 or q/p.

x 2 x

Lecture 11: Return times for random walk

Recurrence/Transience We know u2n = What is f2n ? Let N0 =

n=0

2n (pq )n n

1(Sn = 0) = # of visits of sums to 0. From last class,

E0 N0 =
n=0

2n (1/2)2n (4pq )n n (4pq )n = 1 <1 if p = q if p = q

2n c (1/2)2n n n then E0 N0 = 1 0

<

if p = q if p = q

In fact, P0 (N0 = ) =

for p = q . for p = q

Obviously, E0 N0 < = P0 (N0 < ) = 1, but E0 N0 = does not so obviously imply P0 (N0 = ) = 1. Idea: Look at the rst return probability.

f :=
n=1

f2n =
n=1

P0 (T0 = 2n) = P0 (T0 < ) = probability of ever returning to 0

4 Notice that P0 (N0 = 5) = f (1 f ). Its easy to see that k 1 P0 (N0 = k ) = f (1 f ) Geometric(1 f )

E0 N0 =

1 1 f

< =

i f < 1 i f = 1

Lecture 11: Return times for random walk

2 1 3

4 5

Summary: If P0 (T0 < ) < 1, then E0 N0 < . If P0 (T0 < ) = 1, then E0 N0 = . How to nd a formula for f2n = P0 (T0 = 2n)? Idea: Develop a relation between sequences f2n (unknown) and u2n (known). For every n = 1, 2, 3, . . . , u2n = P0 (S2n = 0)
n

=
m=1 n

P0 (T0 = 2m, S2n = 0) f2m u2n2m

m=1

Idea: Handle a sequence by its generating function. Try the generating function derived from u2n :

U (z ) =
n=0

u2n z 2n .

Also F (z ) =

f2n z 2n
n=1

See text for examples of probability generating functions.

Lecture 11: Return times for random walk

Look at

F (z )U (z ) = ( = =
n=1

f2m z (

2m

)(
k=0

u2k z 2k )

m=1 n

f2m u2n2m )z 2n

n=1 m=1

u2n z 2n

= U (z ) 1 Progress: We have turned the nasty looking convolution formula into a nicer relation of generating functions F (z )U (z ) = U (z ) 1 or F (z ) = 1 U (z ) 1 =1 . U (z ) U (z )

U (1) =
n=0

u2n = E0 N0 1 1 P0 (T0 < ) 1 = 1 F (1) =

Algebra: It is a key observation that for p = q = 1/2, u2k = (1)k

1 2 . See k e.g. Fellers book, An introduction to probability theory and its applications Vol I, Page 96. Recall that
n

(1 + x) =
k=0

n k x k

for any positive integer n

generalizes to (1 + x) =
a

k=0

a k x k

for |x| < 1

for all real numbers a, where a k := a(a 1) (a k + 1) . k!

Lecture 11: Return times for random walk

Claim: Proof:

2n n

= 2 (1)

2n

1 2 . n

2n n

1 2n(2n 1) (n + 1)(n)(n 1) n! n(n 1) 1 1 = 2(2n 1)2(2n 3)2 (5)2(3)2(1) n! 1 = 2n 1 3 (2n 1) n! 1 1 1 1 1 = 22n ( )( + 1)( + 2) ( + n 1) n! 2 2 2 2 1 2n 1 1 1 = 2 (1)n ( )( 1) ( n + 1) n! 2 2 2 1 2 = 22n (1)n n =

Now apply the binomial expansion for the power 1/2

U (z ) :=
n=0

u2n z

2n

=
n=0

2n 2n 2n 2 z n
1 2 (z 2 )n n
1

=
n=0

= (1 z 2 ) 2

F (z ) =
n=1

f2n z 2n ,

f2n = P0 (T0 = 2n)

F (z ) = 1 U (z )1 = 1 (1 z 2 ) 2

=1
n=0

1 2

(z 2 )n

=
n=1

1 2

(1)n1 z 2n

Finally, compare coecients to deduce that f2n = (1)n1

1 2

u2n 2n 1

Lecture 11: Return times for random walk

Check: For n = 1, f2 /u2 =

2/4 2/4

1 1

For n = 2,

f4 2/8 1 = = u4 6/8 3

n=2

n=1

0
0

For a biased coin, u2n (p) = 2n (pq )n = u2n (4pq )n n

where u2n = u2n (1/2) as before. So

Up (z ) =
n=0

1 2 (4pq )n (z 2 )n n
1

= (1 4pqz 2 ) 2

Fp (z ) = 1 (1 4pqz 2 ) 2 In particular, P0 (T0 < ) = Fp (1) = 1 (1 4pq ) 2 = 1 |1 2p| Check: For p q , P0 (T0 < ) = pP1 (T0 < ) + q P1 (T0 < ) = p q/p + q 1 = 2q = 2 2p. And similarly for p < q .
(p) (p) (p) (p)
1