THE CHINESE UNIVERSITY OF HONG KONG

Department of Statistics

STAT3007: Introduction to Stochastic Processes

Problem Sheet 3

The deadline for this Problem Sheet is 5.30pm on Friday 16th November.
Please hand in your answers to Yan Tingjin in LSB 143 or Xi Yiru in G24 or
alternatively to the box outside the computer lab on the first floor of LSB. No
late submissions will be accepted. A late submission will receive a
mark of zero. Students may discuss set problems with others, but their final
submissions must be their own work.
Please answer the following problems. (All are based on questions that can
be found in An Introduction to Stochastic Modeling, 4th Ed. by Mark Pinsky
and Samuel Karlin.)

1. (Problem 4.4.1 in Pinsky and Karlin) Consider the Markov chain on {0, 1}
whose transition probability matrix is
 
1−α α
, 0 < α, β < 1.
β 1−β

(a) Verify that (π0 , π1 ) = (β/(α + β), α/(α + β)) is a stationary distri-
bution.
(1)
(b) Show that the first return distribution to state 0 is given by f00 =
(n)
(1 − α) and f00 = αβ(1 − β)n−2 for n = 2, 3, . . . .
P∞ (n)
(c) Calculate the mean return time m0 = n=1 nf00 and verify that
π0 = 1/m0 .
2. (Problem 5.1.1 in Pinsky and Karlin) Let ξ1 , ξ2 , . . . be independent ran-
dom variables, each having an exponential distribution with parameter λ.
Define a new random variable X as follows: If ξ1 > 1, then X = 0; if
ξ1 ≤ 1 but ξ1 + ξ2 > 1, then set X = 1; in general, set X = k if

ξ1 + · · · + ξk ≤ 1 < ξ1 + · · · + ξk + ξk+1 ,

Show that X has a Poisson distribution with parameter λ. (Thus the

method outlined can be used to simulate a Poisson distribution.) Hint: the
sum of k independent exponential random variables each with parameter
λ has a gamma density with parameters λ and k.
3. (Problem 5.1.2 in Pinsky and Karlin) Suppose that minor defects are
distributed over the length of a cable as a Poisson process with rate α, and
that, independently, major defects are distributed over the cable according
to a Poisson process of rate β. Let X(t) be the number of defects, either
major or minor, the cable up to length t. Argue that X(t) must be a
Poisson process of rate α + β.

1
4. (Problem 5.1.5 in Pinsky and Karlin) For each value of h > 0, let X(h)
have a Poisson distribution with parameter λh. Let pk (h) = P (X(h) = k)
for k = 0, 1, . . . . Verify that

1 − p0 (h)
lim = λ, or p0 (h) = 1 − λh + o(h); (1)
h→0 h
p1 (h)
lim = λ, or p1 (h) = λh + o(h); (2)
h→0 h
p2 (h)
lim = 0, or p2 (h) = o(h). (3)
h→0 h
Here o(h) stands for any remainder term of order less than h as h → 0.
Hint: you may find L’Hôpital’s rule useful for the first statement - look it
up.
5. (Problem 5.2.4 in Pinsky and Karlin) Suppose that N points are uniformly
distributed over the interval [0, N ). Determine the probability distribution
for the number of points in the interval [0, 1) as N → ∞.

6. (Problem 5.3.1 in Pinsky and Karlin) Let X(t) be a Poisson process of

rate λ. Validate the identity

{W1 > w1 , W2 > w2 }

if and only if

{X(w1 ) = 0, X(w2 ) − X(w1 ) = 0 or 1}.

Use this to determine the joint upper tail probability

P r{W1 > w1 , W2 > w2 } = P r{X(w1 ) = 0, X(w2 ) − X(w1 ) = 0 or 1}

= e−λw1 [1 + λ(w2 − w1 )]e−λ(w2 −w1 )

Finally, differentiate twice to obtain the joint density function

f (w1 , w2 ) = λ2 e−λw2 for 0 < w1 < w2 .

7. (Problem 5.3.6 in Pinsky and Karlin) Customers arrive at a holding facility

at random according to a Poisson process having rate λ. The facility
processes in batches of size Q. That is, the first Q − 1 customers wait
until the arrival of the Qth customer. Then all are passed simultaneously,
and the process repeats. Service times are instantaneous. Let N (t) be
the number of customers in the holding facility at time t. Assume that
N (0) = 0 and let T = min{t ≥ 0 : N (t) = Q} be the first dispatch time.
RT
Show that E[T ] = Q/λ and E[ 0 N (t)dt] = [1 + 2 + · · · + (Q − 1)]/λ =
Q(Q − 1)/2λ.

2
8. (Problem 5.4.7 in Pinsky and Karlin) Let W1 , W2 , . . . be the event times
in a Poisson process {X(t); t ≥ 0} of rate λ, and let f (w) be an arbitrary
function. Verify that
X(t) Z t
X
E[ f (Wi )] = λ f (w)dw.
i=1 0

9. (Problem 5.4.9 in Pinsky and Karlin) Customers arrive at a service facility

according to a Poisson process of rate k customers per hour. Let N (t) be
the number of customers that have arrived up to time t, and let W1 , W2 , . . .
be the successive arrival times of the customers. Determine the expected
value of the product of the waiting times up to time t. (Assume that
W1 W2 · · · WN (t) = 1 when N (t) = 0.)