EGERTON UNIVERSITY

DEPARTMENT OF MATHEMATICS
BACS 346/STAT 433 STOCHASTIC PROCESSES CAT II
1. Consider an arrival process and let X (t ) be the number of arrivals in the time interval (0, t ) .
Let the probability of n arrivals in the time interval (0, t ] be pn (t )  pr[ X (t )  n] .
a) Derive the differential equations for pure birth process
b) Solve the differential equations to determine p n (t ) for the Yule-Furry process
n  n assuming that X(0)  1
c) Find E(X(t )) and var(X(t ))
2. Consider a simple Birth-Death process and let X (t ) be the number of arrivals in the time
interval (0, t ) . Suppose the probability of n arrivals in the time interval (0, t ]
is pn (t )  pr[ X (t )  n] . Find the
i) Differential equations for pn (t )  pr[ X (t )  n] , n  0,1, 2,...
ii) E(Xt ) for all t assuming that X (0)  n0
3. Suppose that a production process changes state according to a Markov process whose
transition matrix is given by
 0.2 0.2 0.4 0.2 
 
 0.5 0.2 0.2 0.1 
T 
0.2 0.3 0.4 0.1 
 
 0.1 0.2 0.4 0.3 
 
a) Calculate { pr{ X 3  4 / X 1  1}
b) Given p( X 0  1)  p( X 0  2)  p( X 0  3)  0.2 and p( X 0  4)  0.4 ,find
p( X1  3 X 2  2 X 3  4)
c) Explain what is meant by the statement that a Markov chain is an irreducible recurrent
chain, and show, stating any general results that you assume, that this statement is true
for the present chain.
d) Determine the limiting distribution for the process.

4. A six state {E1 , E2 ,..., E6 } Markov-chain has transition matrix

2 3 0 0 1 3 0 0
 0 12 0 0 1 2 0 

 0 0 1 3 1 3 1 3 0
T  
1 2 0 0 1 2 0 0
 0 12 0 0 1 2 0
 
1 2 0 0 1 2 0 0
a) Show that state E4 is persistent and non-null.
b) Confirm that state E 3 is transient.
c) Identify all closed chains. Which ones are irreducible
5. A motor insurer operates a no claims discount system with the following levels of discount
{0%, 25%, 50%, 60%}. The rules governing a policyholder’s discount level, based upon the
number of claims made in the previous year, are as follows:
 Following a year with no claims, the policyholder moves up one discount level, or
remains at the 60% level.
 Following a year with one claim, the policyholder moves down one discount level, or
remains at 0% level.
 Following a year with two or more claims, the policyholder moves down two
discount levels (subject to a limit of the 0% discount level).
The number of claims made by a policyholder in a year is assumed to follow a Poisson
distribution with mean 0.30.
a) Determine the transition matrix for the no claims discount system.
b) Calculate the stationary distribution of the system, π.
6. The number of claims lodged with an insurance company is according to a Poisson process
of rate 3 a month. Calculate the probability that
i) Exactly 4 claims in the first month and exactly 8 claims in the first three months.
ii) At least 2 claims in the first month and at least 4 claims in the first two months.
iii) At most three claims in the first four months given that at least two claims in the first
one month.
iv) Average number of months that must elapse in order to get 50 claims.
7. A gambler has 5 pounds and has the opportunity of playing a game in which the probability
is 0.3 that she wins an amount equal to her stake, and probability 0.7 that she loses her stake.
If her capital is increased to 8 pounds, she will stop playing and if her capital becomes 0, she
has to stop playing of course. She is allowed to decide how much to stake at each game and
she decides each time to bet, when this is possible, just sufficient in order to increase her
capital to 8 pounds, or if she does not have enough for this, to bet at all that she has. Let
Z 0  5 and Z n , n  1, 2,... be her capital after nth game.
i) Find the transition matrix.
ii) Compute the probability of ultimately increasing her capital to 8 pounds.
8. At a professional school, students need to take and pass an English writing/speech class in
order to get their professional degree. Students must take the class during the first quarter that
they enroll. If they do not pass the class they take it again in the second semester. If they fail
twice, they are not permitted to retake it again, and so would be unable to earn their degree.
Students can be in one of 4 states: passed the class (P), enrolled in the class for the first time
(C), retaking the class (R) or failed twice and cannot retake (F). Experience shows 70% of
students taking the class for the first time pass and 80% of students taking the class for the
second time pass.
Write the transition matrix and identify the absorbing states. Find the probability of being
absorbed eventually in each of the absorbing states.
9. A gambler has $3,000, and she decides to gamble $1,000 at a time at a Black Jack table in a
casino in Las Vegas. She has told herself that she will continue playing until she goes broke
or has $5,000. Her probability of winning at Black Jack is .40. Write the transition matrix,
identify the absorbing states, find the solution matrix, and determine the probability that the
gambler will be financially ruined at a stage when she has $2,000.