&DVV8QGHUJUDGXDWH6FKRRO

%6F+RQV    'HJUHHLQ$FWXDULDO6FLHQFH
%6F+RQV    'HJUHHLQ0DWKHPDWLFDO6FLHQFH





$66WRFKDVWLF0RGHOV>&7D@
3DUWRI6XEMHFW&7RIWKH,QVWLWXWHDQG)DFXOW\RI$FWXDULHV
([DPLQDWLRQV
6WDJH([DPLQDWLRQ 

-DQXDU\ KRXUVPLQV

,QVWUXFWLRQVWRVWXGHQWV

&DQGLGDWHVVKRXOGDQVZHU$//)285TXHVWLRQVIURP6HFWLRQ$SOXVDQ\
7:2RIWKH)285TXHVWLRQVIURP6HFWLRQ%&DQGLGDWHVVKRXOGEHJLQHDFK
TXHVWLRQRQDQHZVKHHWRISDSHU
 
7KHQXPEHURIPDUNVDOORFDWHGLVVKRZQDWWKHHQGRIHDFKTXHVWLRQ:KHUH
PDUNVKDYHEHHQTXRWHGIRUSDUWVRITXHVWLRQVWKHVHDUHLQWHQGHGWREHD
KHOSIXOTXLGHWRWKHFDQGLGDWHV

7KLVH[DPLQDWLRQSDSHUFRQVLVWVRISULQWHGSDJHVH[FOXGLQJWKHWLWOHSDJH

0DWHULDOV

1XPEHURIDQVZHUERRNVWREHSURYLGHG
 2QO\WKH&DVLRFDOFXODWRUV);06(6*7RU*7;RU);06
(6*7RU*7;DUHSHUPLWWHGIRUXVHLQWKLVH[DP
'LFWLRQDULHVDUHQRWSHUPLWWHG
7KLVH[DPLQDWLRQSDSHUPD\EHUHPRYHGIURPWKHH[DPLQDWLRQURRP


External Examiner'U$OHW5RX[
Internal Examiner0LFKDLO&KURQRSRXORV
 SECTION A: Compulsory Questions

QUESTION 1
A Markov chain X on state space S = {1, 2, 3, 4} has transition matrix:
⎛ ⎞
0 0 0.5 0.5
⎜ ⎟
⎜1 0 0 0 ⎟
P =⎜⎜
⎟
⎟
⎝ 0 1 0 0 ⎠
0 1 0 0
(i) Give the definition of an irreducible Markov chain. [2 marks]

(ii) Show that X is irreducible but periodic, and find the period. [4 marks]

(iii) Explain why there is exactly one probability vector π such that π T = π T P .
[1 mark]

(iv) Is π the equilibrium distribution for X? Give a reason for your answer. [3 marks]

(v) Calculate the stationary distribution π. [3 marks]

[Total: 13 marks]

QUESTION 2
The Ruritanian political system has 3 parties: the Traditional Party, the Radical Party
and the Sensible Party. A general election might result in a majority Traditional Party
government (event A), a majority Radical Party government (event B), a majority Sensible
Party government (event C), a coalition between the Traditional and Sensible Parties (event
D) or a coalition between the Radical and Sensible Parties (event E). Since the country
gained independence, the results of the 20 elections have been:

BBEADAABDDCAABAEBBDA

(i) Fit a Markov chain model to the data by estimating the transition matrix.
[4 marks]

(ii) Use your fitted model to estimate the probability that the 22nd general election will
result in a government, which includes the Sensible Party.
[4 marks]

(iii) Let αn denote the probability that the nth election results in a majority Traditional
Party government. Explain why it must be the case that αn converges to some limit as n
→ ∞. [4 marks]

[Total: 12 marks]

Page 1 of 6
QUESTION 3

(i) A Linear Congruential Generator (LCG) is to be used to produce uniformly distributed

pseudo-random numbers in the range [0, 1]. The base is m = 1000, the multiplier is
a = 7 and the additive constant is c = 3.

(a) Demonstrate how the LCG works by generating three pseudo-random numbers
u1 , u2 and u3 starting from the seed x0 = 169. [6 marks]
(b) Comment on whether m = 1000 is a good choice of base for the LCG method.
[3 marks]

(ii) A random observation is required from a continuous distribution with density function
f (x) and distribution function F (x). Show that X = F −1 (U ) has the correct distribu-
tion, where U is a uniformly distributed random variable on [0, 1]. [4 marks]

[Total: 13 marks]

QUESTION 4
A business school appoints a new Dean, who immediately starts to receive invitations to
visit partner universities. On average, one invitation is received every 2 days (except on
Saturdays and Sundays, when no invitations are received). You may assume that a Poisson
process model is appropriate.

(i) Let Di denote the number of invitations received by the Dean in the ith week. Is
{D1 , D2 , . . . } a stationary process? Give a reason for your answer. [3 marks]

(ii) In the first week, the Dean receives six invitations. Given this information, what is
the conditional probability that she receives fewer than half that number during the
second week? [3 marks]

(iii) Evaluate the probability that, starting from midday on Thursday, the next time the
Dean receives an invitation is the following Monday. [2 marks]

(iv) Each invitation, independently, has a probability of 30% of being from a partner uni-
versity in the Far East. Calculate the probability that, during a single week, the Dean
receives no invitations from Far Eastern partners. [Note: you should condition on the
total number of invitations received during the week.] [4 marks]

[Total: 12 marks]

Page 2 of 6
 SECTION B: Optional Questions

QUESTION 5
In the Championship in 2013–14 Watford Football Club won 16 games, drew 14 games and
lost 16 games. The sequence of results was as follows:

Aug – Dec WWDDLDWWWLWLDLDLLDDLDWD

Jan – May LDLWDWWDLWLWLDWDWWWLLLL

The number of observed transitions in each half of the season was

WW WD WL DW DD DL LW LD LL
Aug–Dec 3 2 2 2 2 4 1 5 1
Jan–May 3 3 3 3 0 2 3 2 3

Let Rn denote the result of the nth game, with state space S = {W, D, L}, and let Xn denote
the number of points Watford have scored after n games, where X0 = 0 and teams score 3
points for a win, 1 point for a draw and 0 points for a loss.

(i) Model I states that {Xn : n ≥ 0} is a random walk.

(a) Explain what it means that {Xn : n ≥ 0} possesses stationary, independent

increments. [2 marks]
(b) Prove that

E[Xn |X0 = x0 ] = x0 + nμ and Var[Xn |X0 = x0 ] = nσ 2

for some constants μ and σ. [5 marks]

(c) Use the data from the first half of the season only to estimate the distribution of
the increments of the random walk. [2 marks]
(d) Calculate the mean and variance of the fitted distribution and use these to provide
a 95% confidence interval for the number of points Watford have achieved by the
end of the season. [7 marks]

(ii) Model II states that the results Rn follow a Markov chain model with transition matrix
P . Model III states that the results in the first half of the season follow a Markov chain
model with transition matrix P1 , while the results in the second half of the season follow
a Markov chain model with transition matrix P2 .

(a) Estimate the matrices P1 and P2 . [4 marks]

(b) Explain how these estimates could be used to test whether Model III fits the data
better than Model II. You should describe how the test is to be carried out, but
you are not expected to perform the test. [5 marks]

[Total: 25 marks]

Page 3 of 6
QUESTION 6

(i) Give a brief description of the way a discrete-time model is fitted to a collection of data.
Your description should include discussion of model identification, model validation and
the purpose of simulation. [5 marks]

(ii) Explain what is meant by a stationary process and give an example of an observable
process, which could reasonably be modelled by a stationary process. [4 marks]

(iii) A Markov chain {Xn : n ≥ 0} on state space S = {1, 2, 3, 4, 5, 6} has transition diagram
as shown in Figure 1. Each arrow represents a non-zero transition probability, and all
transitions with non-zero probability are shown.

(a) Divide the state space into communicating classes, stating whether each is tran-
sient or recurrent, periodic or aperiodic. [5 marks]
(b) Explain why it is possible to deduce that the limit

lim P[Xn = j|X0 = i]

n→∞

exists for each i and j and is independent of i. [5 marks]

(c) Let πj denote the limit defined in (b). If the initial value of the Markov chain,
X0 , is chosen randomly with P[Xn = j|X0 = i], prove that {Xn : n ≥ 0} is a
stationary process. [6 marks]

Figure 1
[Total: 25 marks]

Page 4 of 6
QUESTION 7

(i) Let {X(t) : t ≥ 0} be a Markov jump process on state space S and denote by P (s) the
time-s transition matrix, whose entries are pij (s) = P[X(t + s) = j|X(t) = i].

(a) Explain what it means that {X(t) : t ≥ 0} has the Markov property.
[2 marks]
(b) Explain the difference between time-homogeneous and time-inhomogeneous Markov
chain. [2 marks]
(c) Prove that P (t + s) = P (s)P (t). Ensure that you state explicitly the point in
your explanation where you use the Markov property. [5 marks]
(d) Derive the Kolmogorov Forward Equation P  (s) = P (s)Q. [4 marks]

(ii) A film director chooses his technical crew after a set of interviews. With probability
75% they are already on the film studio’s payment system, but with probability 25%
they are not, in which case they have to wait for an average of 10 days for their name
to be added to the system. Once the name is on the system, it takes an average of half
a day for the contract to be set up and , when that has been done, it takes an average
of 2 days for payment to be authorised.
This arrangement is to be modelled by a Markov jump process, with states A: name
not yet added to the system; B: name added but contract not set up; C: contract set
up but payment not authorised; D: payment authorised.

(a) Write down the generator matrix, Q. [3 marks]

(b) List two assumptions which must hold if the Markov jump process model is to
be a good representation of the process. [2 marks]
(c) State the values of pAA (t) and pBB (t). [3 marks]
(d) The integrated form of the Kolmogorov Backward Equation for pAB (t) can be
written in the form
 t
pAB (t) = f (u) × rAB × pBB (t − u)du.
0

Identify the function f (u) and state the value of the constant rAB . [4 marks]

[Total: 25 marks]

Page 5 of 6
QUESTION 8
Let {X(t) : t ≥ 0} be a time-homogeneous Markov jump process on a state space S =
{1, 2, 3, 4} with generator matrix
⎛ ⎞
−3λ 2λ λ 0
⎜ ⎟
⎜ 0 −2λ λ λ⎟
⎜
Q=⎜ ⎟
⎝ 0 0 −λ λ⎟⎠
0 0 0 0

and let pij (t) denote P[X(t) = j|X(0) = i].

(i) What do you deduce from the fact that the bottom row of the generator matrix consists
entirely of zeros? [1 marks]

(ii) Draw a transition diagram for the process. [2 marks]

(iii) Write down the Kolmogorov Forward Equation for p11 (t) and show that the solution
is p11 (t) = e−3λt . [5 marks]

(iv) (a) Write down the Kolmogorov Forward Equation satisfied by p22 (t). [3 marks]
(b) Derive the value of p21 (t). [1 marks]
(c) Derive the solution of the Kolmogorov Forward Equation for p22 (t).
[3 marks]

(v) (a) Explain why it is the case that

 t
p12 (t) = 2λe−3λu p22 (t − u)du.
0

[4 marks]
(b) Simplify the above integral to derive an expression for p12 (t). [4 marks]
(c) Write down the value of p13 (t). [2 marks]

[Total: 25 marks]

Page 6 of 6