King Saud University Final Exam, S1 1441

College of Sciences M 380 - Stochastic Processes

Department of Mathematics Time: 3 hours
‫ــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــ‬
Answer the following questions:

Q1: [4+4]

(a)Consider the Markov chain whose transition probability matrix is given by

0 1 2 3
0 1 0 0 0
1 0.1 0.4 0.1 0.4
P=
2 0.2 0.1 0.6 0.1
3 0 0 0 1

(i) Starting in state 2, determine the probability that the Markov chain ends in state 0.

(ii) Determine the mean time to absorption.

b) Let X n denote the quality of the nth item that produced in a certain factory with Xn  0
meaning “good” and X n  1 meaning “defective”. Suppose that Xn  be a Markov chain whose
transition matrix is

0 1
0 0.99 0.01
P
1 0.12 0.88

i) What is the probability that the fourth item is defective given that the first item is defective?

ii) In the long run, what is the probability that an item produced by this system is good?

Q2: [4+4]

(a) The following experiment is performed: An observation is made of a Poisson random variable
N with parameter . Then N independent Bernoulli trials are performed, each with probability
p of success. Let Z be the total number of successes observed in the N trials.

i) Formulate Z as a random sum and thereby determine its mean and variance.

ii) What is the distribution of Z ?

1
(b) Consider a sequence of items from a production process, with each item being graded as good
or defective. Suppose that a good item is followed by another good item with probability  and
is followed by a defective item with probability 1  . Similarly, a defective item is followed by
another defective item with probability  and is followed by a good item with probability 1   .
Answer each of the following:

i) If the first item is good, what is the probability that the first defective item to appear is the
fifth item?

ii) If the first item is bad, what is the probability that the first good item to appear is the fifth
item?

Q3: [5+4]

(a) Suppose that the weather on any day depends on the weather conditions for the previous 2
days. Suppose also that if it was sunny today but cloudy yesterday, then it will be sunny
tomorrow with probability 0.4; if it was cloudy today but sunny yesterday, then it will be sunny
tomorrow with probability 0.6; if it was sunny today and yesterday, then it will be sunny
tomorrow with probability 0.2; if it was cloudy for the last 2 days, then it will be sunny
tomorrow with probability 0.9. Transform this model into a Markov chain, and then find the
transition probability matrix. Find also the long run fraction of days in which it is sunny.

(b) Suppose that customers arrive at a facility according to a Poisson process having rate   2 .
Let X (t ) be the number of customers that have arrived up to time t . Determine the following:

i) pr X (1)  2 and X (3)  6

ii) pr X (3)  6 X (1)  2

Q4: [5+4]

(a) If X (t ) represents a size of a population where X (0)  1, using the following differential
equations
dp 0 (t )
 0 p 0 (t ) (1)
dt
dp n (t )
 n 1p n 1 (t )  n p n (t ), n =1,2,3, ... (2)
dt
Prove that: X (t )  geom (p ), p  e  t when 0  0 and n  n  , and then find the mean and
variance of this process.

2
(b) Let X (t ) be a Yule process that is observed at a random time U , where U is uniformly
distributed over [0,1). Show that pr X (U )  k   pk / (  k ) for k  1, 2,... , with p  1  e   .

Q5: [6]

A pure birth process starting from X (0)  0 has birth parameters 0  1, 1  3, 2  2 and 3  5.
Determine Pn (t ) for n  0, 1, 2.

‫ـــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــ‬

3
 Model Answer
Q1: [4+4]

(a)

0 1 2 3
0 1 0 0 0
1 0.1 0.4 0.1 0.4
P=
2 0.2 0.1 0.6 0.1
3 0 0 0 1

ui  pr XT  0 X 0  i for i=1,2,

and vi =E[T X 0  i ] for i=1,2.

(i)

u1  p10  p11u1  p12u 2

u 2  p20  p21u1  p22u 2

u1  0.1  0.4u1  0.1u 2

u 2  0.2  0.1u1  0.6u 2

6u1  u 2  1 (1)
u1  4u 2  2 (2)

Solving (1) and (2), we get

u1  6
23 and u 2  13
23

Starting in state 2, the probability that the Markov chain ends in state 0 is

u 2  u 20  13
23

(ii) Also, the mean time to absorption can be found as follows

v1  1  p11v1  p12v 2
v 2  1  p21v1  p22v 2

4


v1  1  0.4v1  0.1v 2
v 2  1  0.1v1  0.6v 2

6v1  v 2  10 (1)
v1  4v 2  10 (2)

Solving (1) and (2), we get v 2  v 20  70

23

(b)

i)

0.9737 0.0263
P3   
0.3152 0.6848
pr X 3  1 X 0  1  p113  0.6848

ii)

In the long run, the probability that an item produced by this system is good is given by:

0.12
b / (a  b ) 
0.01  0.12
12
  92.13 % ,
13

 a bb a
a b 
where lim P   bn
a 
n 
 a b a b 

5
Q2: [4+4]

(a)

i) Z  1   2  ... +N , N  0

E (k )    p , Var(k )= 2  p (1  p )

E (N )  v   , Var(N )= 2  
E (Z )  v
 E (Z )   p
Var(Z)=v 2   2 2
 Var(Z)= p (1  p )  p 2 
= p

ii) Z Poisson( p )

(b)

i)

Pr X 2  G , X 3  G , X 4  G , X 5  D X1  G 
 Pr X 5  D , X 4  G , X 3  G , X 2  G X1  G 
 Pr X 5  D X 4  G   Pr X 4  G X 3  G   Pr X 3  G X 2  G   Pr X 2  G X1  G 
 pGD pGG
3

 (1   ) 3
  3 (1   )

Also, you can solve it as follows.

p1p12 p23p34 p45 , p1  Pr(X1  G )  1

 pG pGG
3
pGD
  3 (1   )

ii)

Similarly,

6
Pr X 2  D , X 3  D , X 4  D , X 5  G X1  D 
 Pr X 5  G , X 4  D , X 3  D , X 2  D X1  D 
 Pr X 5  G X 4  D   Pr X 4  D X 3  D   Pr X 3  D X 2  D   Pr X 2  D X1  D 
 pDG p 3
DD

 (1   )  3
  3 (1   )

Also, you can solve it as follows.

p1p12 p 23p34 p 45 , p1  Pr(X1  D )  1

 pD pDD
3
pDG
  3 (1   )

Q3: [5+4]

(a)

(S , S ) (S ,C ) (C , S ) (C ,C )
(S , S ) 0.2 0.8 0 0
(S ,C ) 0 0 0.6 0.4
(C , S ) 0.4 0.6 0 0
(C ,C ) 0 0 0.9 0.1

In the long run, the limiting distribution is   ( 0 ,  1 ,  2 ,  3 )

0.2 0  0.4 2   0   2  2 0 (1)

0.8 0  0.6 1   1   1  2 0 (2)
8
0.4 1  0.1 3   3   3   0 (3)
9

And  0  1   2   3  1 (4)

9
 0   0.1698
53
 9 18 18 8 
  , , , 
 53 53 53 53 

The long run fraction of days in which it is sunny is

7
 9 18
 0  1  
53 53
27
=  0.5094
53

(b)

i)

pr X (1)  X (0)  2, X (3)  X (1)  4

indpendent r.vs
22e 2 44e 4
 .
2! 4!
64
 e 6  0.05288
3

ii)

pr X (3)  6 X (1)  2
 pr X (3)  X (1)  4 X (1)  X (0)  2
indpendent r.vs
 pr X (3)  X (1)  4
44e 4

4!
64
 e 4  0.1953668
6

Q4: [5+4]

(a) dp 0 (t )
 0 p 0 (t ) (1)
dt
dp n (t )
 n 1p n 1 (t )  n p n (t ), n =1,2,3, ... (2)
dt

The initial condition is X (0)  1  p1 (0)  1

1 , n=1
 pn (0)  
0 , otherwise

8
 dp 0 (t )
0  0 (1)  0
dt
 p 0 (t )  0 (3)

dp n (t )
(2)   n 1p n 1 (t )  n p n (t )
dt
dp n (t )
  n p n (t )  n 1p n 1 (t ), n  1, 2, ...
dt

n  n  , n 1  (n  1)
dp n (t )
  n  p n (t )  (n  1) p n 1 (t ), n=1,2, ...
dt

Multiply both sides by e nt

dp (t ) 
e n t  n  n  p n (t )   (n  1) p n 1 (t )e n t
 dt 
d
  p n (t )e n t   (n  1) p n 1 (t )e n t
dt
t t
  d  p n (x )e n x   (n  1)  p n 1 (x )e n xdx
0 0
t t
  p n (x )e n x   (n  1)  p n 1 (x )e n xdx
0 0

 p n (t )  e n t  p n (0)  (n  1)  p n 1 (x )e nxdx  , n  1, 2, ... (4)

t

 0 

which is a recurrence relation.

at n  1

p 1 (t )  e  t  p 1 (0)  0  e  t (5)

at n  2

p 2 (t )  e 2t  p 2 (0)    p 1 (x )e 2xdx 

t

 0 

9
(5)  p 1 (x )  e  x

 p 2 (t )  e 2 t    e  xe 2 xdx 
t

 0 
t
 p 2 (t )  e 2 t  e xdx
0
 t
=e (1  e  t )1 (6)
Similarly as (5) and (6), we deduce that
p n (t )  e  t (1  e  t )n 1
 p (1- p )n 1 , p  e  t , n  1, 2, ...
 X (t )  geom ( p ), p  e  t
Mean X (t )   1/ p  e t ,
1  p 1  e  t
Variance X (t )    2 t
p2 e

(b) For Yule process,

p n (t )  e   t (1  e   t )n 1 , n  1

1
pr X (U )  k    e   u (1  e   u )k 1du
0
1
1

  u k 1
= (1  e ) . e   udu
0
1
1  (1  e   u )k 
=  
 k 0
1
= (1  e   )k 
k 
pk
 pr X (U )  k  = , k  1, 2, ... where p  1  e  
k

Q5: [6]

For pure birth process,

10
p0 (t )  e  0t , (1)
 1 1 
p1 (t )  0  e  0t  e  1t  , (2)
 1  0 0  1 
and pn (t )  pr X (t )  n X (0)  0
=0 1...n 1 B0,ne  0t  ...  Bk ,ne  k t  ...  Bn ,ne  nt  , n  1, (3)

where

n
 1 
Bk ,n     i  k , 0  k  n,
i  0  i  k 

n
 1 
B0,n    
i 1  i  0 

and
n 1
 1 
Bn ,n    
i  0  i  n 

at n  0 (1)  p0 (t )  e  0t , 0  1
 p0 (t )  e t

at n  1 (2)  p1 (t )  12 e t  e 3t 

at n  2 (3)  p2 (t )  01 B0,2e 0t  B1,2e 1t  B2,2e 2t  ,

1
where, B0,2 
(1  0 )(2  0 )
= 12 ,
1
B1,2 
(0  1 )(2  1 )
= 12
and
1
B2,2 
(0  2 )(1  2 )
 -1

 p2 (t )  3  12 e t  12 e 3t  e 2t 

‫ــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــــ‬

11