QF204 Additional Examples

Week 10: Random Walk

1 Questions
1. A disoriented person is standing on the number line {0, 1, 2, 3, , 1000}, where 0 and 1000
are death and home, respectively. On each step, the person moves one unit to the left or right
with equal probability of 12 . Starting at 80, determine

(a) the probability of the person reaching 0 (i.e. death).

(b) the expected number of steps the person takes before reaching 0 or 1000.

2. In exactly the same setting as the previous question, suppose now that the person moves one
unit to the left with probability 0.501, and one unit to the right with probability 0.499. Starting
at 80, determine the probability of the person reaching 0.

3. Suppose a person is standing on 0 along the number line {0, 1, 2}, and the person moves
according to the following rules:

Location Movement
0 take a step to the right with probability 1
1 take a step to the left or right with probability 21 each
2 absorption state the random walk terminates

(a) What is the probability of hitting 2?

(b) What is the expected number of steps required to reach 2?

4. Consider a game where you follow the bold play strategy and bet x [0, 1] : min{x, 1 x}
on each game, with 1 being your target wealth. You win each game with probability p = 0.7.
Let f (x) denote the probability of you attaining your target wealth. Work out f (0.1), f (0.2),
f (0.3), f (0.4), f (0.5), f (0.6), f (0.7), f (0.8), f (0.9).

5. Consider the game played by Akio Kashiwagi with an initial capital of $12,000,000, he
needed to bet $200,000 each time until he either doubled his initial capital or went broke. The
probability of him winning each game is p = 0.49375. Determine

(a) the probability of him attaining the target wealth of $24,000,000.

(b) the expected number of games to be played for him to either attain the goal or to go
broke.

1
2 Suggested Solutions
1. (a) This is a symmetric random walk with p = q = 2.
1
Let f (k) denote the probability of
reaching 1000 starting at k, we have
80
f (80) = = 0.08.
1000
Hence, the probability of reaching 0 is 1 0.08 = 0.92.
(b) Let g(k) denote the expected number of steps taken before absorption (reaching 0 or
1000) starting at k, we have

g(80) = 80 (1000 80) = 73, 600.

2. Let p denote the probability of a right move, and q denote the probability of a left move. We
have p = 0.499 and q = 0.501. The probability of reaching 1000 is
 80
q
p 1
f (80) =  1000 = 0.007036
q
p 1

Hence, the probability of reaching 0 becomes 1 0.007036 0.993.

3. We only need to consider 3 locations: 0, 1, 2. At location 0, we move to the right with

probability 1, since theres no other place to move. At location 1, we move to 0 with probability
2 , and to 2 with probability 2 . So the probability of hitting 2 is
1 1

 2  3
1 1 1
+ + + = 1.
2 2 2

The expected number of steps taken to reach 2 is

 
1 1 1 1 1 1
2 + 4 2 + 6 3 + ... = 2 + 4 + 6 2 + ...
2 2 2 2 2 2
" #
1
1 2 2 2
= +
2 = 4.
2 1 12 1 1 2

In the last step, we have used the innite series formula for arithmetico-geometric series

a rd
S = a + (a + d)r + (a + 2d)r2 + . . . = + .
1 r (1 r)2

A more elegant solution is to make use of the recursive nature of the problem. Let X denote
the number of steps required to reach 2. We have
1 1
X =1+ (1 + X) + (1) X = 4.
2 2

2
4. The recurring relationship is given as follow:

f (0.8) = p + qf (0.6)
= p + pq + q 2 f (0.2)
= p + pq + pq 2 f (0.4)
= p + pq + p2 q 2 f (0.8)
p + pq
f (0.8) =
1 p2 q 2
f (0.9) = p + qf (0.8)
f (0.7) = p + qf (0.4) = p + pqf (0.8)
f (0.6) = p + qf (0.2) = p + p2 qf (0.8)
f (0.5) = p
f (0.4) = pf (0.8)
f (0.3) = pf (0.6)
f (0.2) = pf (0.4)
f (0.1) = pf (0.2)

Hence, we can work out:

f (0.9) = 0.9856
f (0.8) = 0.952
f (0.7) = 0.8999
f (0.6) = 0.8399
f (0.5) = 0.7
f (0.4) = 0.6664
f (0.3) = 0.588
f (0.2) = 0.4665
f (0.1) = 0.3265

5. Since 12,000,000 24,000,000

200,000 = 60 and 200,000 = 120, this question can be seen as a random walk on the
number line {0, 1, 2, 3, , 120}, starting at the location 60.

(a) With p = 0.49375 and q = 1 p, we have

 60
q
p 1
f (60) =  120 0.1824
q
p 1

(b) The expected number of games played is

60 120
g(60) = f (60) 3049
qp qp