Assignment #3 Solution

1. (a) {Xn } n=0 a Markov chain (Why?) and its state space is transition matrix is 0 1 0 0 0 1/4 0 3 / 4 0 0 P = 0 2/4 0 2/4 0 0 0 3/4 0 1/4 0 0 0 1 0 {0, 1, 2, 3, 4}. The one-step

(b) First, we can easily nd P 2 . From that the second row is the distribution of X2 given that X0 = 1. So P (X2 = 1|X0 = 1) = 5/8, P (X2 = 3|X0 = 1) = 3/8 and P (X2 = 0, 2, 4|X0 = 1) = 0. (c) For this part, we need to nd the equilibrium distribution of the Markov chain. Solve P = , we have 1 = 40 2 = 60 3 = 40 4 = 0 . (Due to the symmetry, we can imagine that 1 = 3 and 0 = 4 ). Using 4 i=0 i = 1, we have 0 = 1/16 and 4 = 1/16. So the proportion of times when at least one urn is empty, is 0 + 4 = 1/8. (d) Let pk be the the probability that number of balls in urn A reaches 0 before the number of balls in urn B reaches 0, given that X0 = k for k = 0, 1, 2, 3 and 4. Immediately, we have p0 = 1 and p4 = 0. Note that p2 is also the probability that the number of balls in urn A reaches 0 before 4. Note that 2 is just the middle of 0 and 4, so by symmetry, we should have p2 = 1/2. Conditioning on the value of X1 , we have p1 = 1/4p0 + 3/4p2 = 5/8 p3 = 3/4p2 + 1/4p4 = 3/8.

2. (a) 1 3 C1 = {1, 3} and it is closed; 2 4 C2 = {2, 4} and it is closed; 0 5 and 5 0 1 C1 can tell us C3 = {0, 5} and it is open.

The simple form is State State = State State State State 1 3 2 4 0 5

P simp

(b) For all three classes, there exists a state such that the one-step transition probability is greater than 0 and therefore the period for all states is 1. (c) For C2 , the transition matrix for C2 is doubly stochastic, so the equilibrium distribution 1 is C = ( 1 2 , 2 ). 2 For C1 , solve P = subject to j C1 j = 1 1/31 + 3/54 = 1 1 + 4 = 1 Solving the equations, we can get 1 = (d) Conditional on the rst movement,
5 9 19 10 19 . 9 10 So C = ( 19 , 19 ).
1

State 1 State 3 State 2 State 4 State 0 State 5 1 2 0 0 0 0 3 3 2 3 0 0 0 0 5 5 1 1 0 0 0 0 2 2 1 1 0 0 0 0 2 2 1 1 1 1 1 0 5 5 5 5 5 1 1 1 0 0 0 4 2 4

and 4 =

A0,C1

=
k =0

P0,k Ak,C1

= 1/5 A0,C1 + 1/5 A1,C1 + 1/5 A2,C1 + 1/5 A4,C1 + 1/5 A5,C1 = 1/5 A0,C1 + 1/5 A5,C1 + 1/5 and
5

A5,C1

=
k =0

P5,k Ak,C1

= 1/4 A0,C1 + 1/4 A1,C1 + 1/2 A4,C1 = 1/4 A0,C1 + 1/4. Solve the above two equations, we get A0,C1 = 1/3, A5,C1 = 1/3 and therefore A0,C2 = 2/3, A5,C2 = 2/3. (e) Then when X0 = 0, the equilibrium distribution is = A0,C1 (0, 9/19, 0, 10/19, 0, 0) + A0,C2 (0, 0, 1/2, 0, 1/2, 0). (f) Let T5 be the waiting time for going to the closed class, starting from state 5. Conditional on the rst movement, E (T0 ) = 1/5(1 + E (T0 )) + 1/5 + 1/5 + 1/5 + 1/5(1 + E (T5 )) = 1/5 E (T0 ) + 1/5 E (T5 ) + 1 2

and E (T5 ) = 1/4(1 + E (T0 )) + 1/4 + 1/2 = 1/4 E (T0 ) + 1. Solve the above two equations, we get E (T0 ) = 8/5, E (T5 ) = 7/5. 3. The transition matrix for modied symmetric random walk on {0, 1, . . . , N } is

P =

0 1 0 0 1/2 0 1/2 0 0 1/2 0 1/2 0 0 1/2 0 . . . . . . . . . . . . . . . 0 0 0 0 1/2 0 0 0 0 0

0 0 0 0 . . .

0 1/2

0 0 0 0

One might guess at the equilibrium distribution. Intermediate states are all treated equally by this walk (just as all states are treated equally by the balanced ordinary random walk). Each intermediate state can be approached from the right or from the left with equal probability 1/2. However, the two boundary states (the two reecting barriers 0 and N ) can only be approached from one side and we might expect them to be visited half as often as the intermediate states. This suggests the equilibrium distribution = (1/(2N ), 1/N, 1/N, . . . , 1/N, 1/N, 1/(2N )). One could check this hypothesis by verifying P = . If this solution does not occur to you,
N

Solve P = subject to
j =0

j = 1.

Solve recursively in terms of 0 .

(0 , 1 , . . . , N )

0 1/2 0 0 . . . 0 0 1 0 1/2 0 . . . 0 0

(0 , 1 , . . . , N )

= 0 = 1 1 , so 1 = 20 . 2 = 1 = 0 + 1 2 , so 2 = 2(1 0 ) = 20 . 2

Carrying on this manner, we obtain j = 20 for j = 1, . . . , N 1.

While (0 , 1 , . . . , N ) 1/2

(0 , 1 , . . . , N )

0 1/2 0 1/2 . . . 0 0

= 2 = 1/21 + 1 3 , so 3 = 22 1 = 20 . 2

0 0 0 0 . . . 0

= 1/2N 1 = N . So N = 1 N 1 = 0 . 2
N

Thus we observe N = 0 and j = 20 for j = 1, . . . , N 1. Using

1 = 2N ,
j =0

j = 1, we conclude

1 1 1 1 , ,..., , N N N N

N 1 intermediate steps Expected return time:

1 2N .

E (T00 ) =

1 1 = = 2N. 0 1/(2N ) 1 1 = = N. E (T11 ) = 1 1/(N )

4. Let N (t) be the number of telephone calls arriving at a switchboard at time t (minutes), N1 (t) be the number of long-distance calls at time t (minutes) and N2 (t) be the number of local-distance calls at time t (minutes) . Then, we learn from the question that (i) N (t) is a Poisson process with rate 2 per minute; (ii) N1 (t)|N (t) = n follows Binomial(n, 0.1) distribution; (iii) N (t) = N1 (t) + N2 (t). With these two things in mind, we can have the following three properties regarding N1 (t), N2 (t) and N (t): (i) N (t) P oisson(2t); (ii) N1 (t) P oisson(0.2t); (iii) N2 (t) P oisson(1.8t) and it is independent of N1 (t). With those properties, we come to work out the following problems. (a) The probability that no call arrives between 9:00-9:05am is P (N (5) = 0) = e10 . 4

(b) The probability that at least 2 calls arrive between 10:00-10:02am is P (N (2) 2) = 1 4e4 e4 = 1 5e4 . (c) The probability of at least one long distance call in a ten minute period is P (N1 (10) 1) = 1 P (N1 (10) = 0) = 1 e2 . (d) We need to nd E (N (60)|N1 (60) = 8). Then E (N (60)|N1 (60) = 8) = E [(N1 (60)+N2 (60))|N1 (60) = 8] = 8+E (N2 (60)) = 8+1.860 = 116. (e) Given that there were 90 calls in an hour, the probability that 10 were long distance is P (N1 (60) = 10|N (60) = 90) = 90 0.110 0.980 . 10

5. Let Ti be the service time for ith customer and Si be the waiting time for ith arrival. Let T is the waiting time for the rst departure. (a) Under the assumption that there is no new arrival before the departure, then the waiting time for the rst departure is T1 and the expectation is 1/. (b) Under the assumption that there is no new arrival before the departure, then the waiting time for the rst departure is {T1 , T2 } and the expectation is 1/(2). (c,d) Since there are at least 3 customers at the bank, so the waiting time for the rst departure is T = min{T1 , T2 , T3 } and the expectation of the waiting time is E (T ) = 31 . (e) The expected length of time until an arrival is
1 .

(f) If all three tellers are busy, the probability that the customer at Teller #1 departs rst is 1/3.