METU EE531 Prof.

Elif Uysal-Biyikoglu Homework 6 Not to be graded

1. (Markov Processes) Exercise 7.1 from the Textbook. 2. (Markov Processes) Exercise 7.14 from the Textbook. 3. (Stopping rules, Walds Inequality, Martingales) (this is an exam problem from last year.)
Consider a Poisson process with inter-arrival times Xi, i=1,2, with rate a (arrivals per second). Every arrival brings a reward equal to the square of the preceding inter-arrival duration, such that Ri = Xi 2, i=1,2, The total reward collected by the nth arrival is: NOTE: Most parts of this problem can be answered independently of other parts! (a) Compute the expectation of the total reward collected by the nth arrival. (b) Compute the expectation of the total reward collected in a duration of t seconds (c) Write an expression for the probability that among the first n arrivals, at least n/2 of them bring a reward greater than (Hint: a reward greater than corresponds to an interarrival greater than greater than . ) (d) Compute the probability that, in a duration of t seconds, there is at most 1 arrival. (e) We shall stop observing the process when the total reward collected becomes greater or equal to some value r>0. Using Wald's equality, compute a bound for the expectation of N, the total number of arrivals we observe. } is a Martingale. (Hint: note (f) Define . Show that the process { that you do not need the specific distribution of to show this.) (g) Given that =3/a, compute the expected value of .

(h) Is the process {Zn} stationary? Explain your answer.