DEPARTMENT OF STATISTICS

STOCHASTIC PROCESSES 1: HSTS431

ASIGNMENT 2

1. Suppose that the rate of the process, instead of being constant varies with time 𝑡 and is in fact
equal to 𝑡.

(a) Show that, for this process 𝑃0 (𝑡) = 𝑒 −𝜆𝑡 and find the expected time to the first incident.
∞ √𝜋
[HINT: Use the result ∫0 √𝑢 𝑒 −𝑢 = 2
]

(b) If the first incident is known to have occurred at time s, show that the probability that no
1 2
incidents occur in (𝑠, 𝑠 + 𝑡] is given by 𝑃0 (𝑡|𝑠) = 𝑒𝑥𝑝 (−𝑠𝑡 − 𝑡 ). Deduce that the
2
independence property of the intervals between incidents, which is known to hold for the
constant rate process, fails for this variable rate process.

2. (a) Define the term a “continuous time Markov chain”.

(b) Consider a machine that works for an exponential amount of time having mean 1⁄𝜆 before
breaking down and suppose that it takes an exponential amount of time having mean 1⁄ µ to
repair the machine. If the machine is in working condition at time 0, then what is the
probability that it will be working at time 𝑡 = 10?

HINT: 𝑃′0𝑗 = 𝜆0 [𝑃1𝑗 (𝑡) − 𝑃0𝑗 (𝑡)], 𝑃𝑖𝑗′ (𝑡) = 𝜆𝑖 𝑃𝑖+1,𝑗 (𝑡) + µ𝑖 𝑃𝑖𝑗 (𝑡) − (𝜆𝑖 − µ𝑖 )𝑃𝑖𝑗 (𝑡), 𝑖 > 0.

3. Consider an office at the Registrar General’s office were people apply for national identity cards.
A customer upon arrival goes to point A where a form is filled and finger prints taken. A
customer then proceeds to point B where s/he photographed and given an identity card. Service
times are exponential distributed with respective times µ1 and µ2 . Potential customers arrive at a
rate of 𝜆 and will enter the system only when both points are empty.

(a) State the possible states of the resulting Markov chain.

(b) Explain why this is not a birth and death process.
(c) Use limiting probabilities to determine the proportion of time spent in at each point.
4.