Branching Process

( ) ( ) 2
1
1
2 1
, ( ) , ( )
, 1
( ) , ( )
1
, 1
1
n
X
n n
n i i i
i
n n
n
n n
X E Var
n
E X Var X

+
=

= = =
=

= =

Extinction Probability
1
0
( ) ( 0)
k
n n k n
k
u P N n P X p u

=
= = = =

Markov Inequality/Maximal Inequality
0
0
0, 0, ( ) ( ) /
(max ) ( ) / , 0
n
n m
X P X E X
E X E X m



>


Expectation
0
1
Discrete: ( ) ( ) Continuous: ( ) [1 ( )]
X
x
E X P X x E X F x dx

=
= =

Markov Chain
Chapman-Kolmogorov Equation
( ) ( 1) ( ) ( 1)
0
0
, ( )
n n n n n T n
ij ik kj n
k
P P P

=
= = = =

P PP P u P u

Tower Expectation
0 1 0 0
1
0
( ( ) | ) ( ( ( ) | , ) | )
( ( ) | )
ik
k
E g X i E E g X X i X i
E g X k P

=
= = = =
= =

X X
X

First Step Analysis
1
0
0
1
0
0
1
[ ] 1 1,
1
1 1
1 1
0
0 0
1
( | )
( | ) 1
( ) , ( ,..., )
( ) , ( ,..., )
( ) |
( )
r
ik T ik ij jk
j
r
i ij j
j
T
k k k k r k
T
r
T r
ij n ij ik kj
n k
u P X k X i P P u
v E T X i P v
I Q R u u
I Q v v
W E I X j X i P W
I Q

= =

= = = = +
= = = +
= =
= =
(
= = = = +
(

=

u u
v 1 v
W

Long Run Behavior of Markov Chain
( )
0
regular: s.t. 0
if , , s.t. 0 regular
limiting distribution: , lim
k
ii
T n
k
i j i j i P
>
>
= =
P
P P u

Periodicity
( )
( ( ))
( ) ( ( ))
( ) gcd( : 0), ( ) ( )
0 for large enough
if 0, 0 for large enough
aperiodic and irreducible regular
n
ii
nd i
ii
m m nd i
ii ii
d i n P i j d i d j
P n
P P n
+
= > =
>
> >

Recurrent and Transient states
( )
( ) ( ) ( ) (0)
0
( )
0
0
0
Prob of return to state at the transition
, 0
Prob of return to state
no. of times returning to , ( | ) ( )
[ | ]
1
th n
ii
n
n k n k
ii ii ii ii
k
n
ii ii
n
k
ii
ii
ii
i n f
P f P f
i f f
M i P M k X i f
f
E M X i
f
r

=
=
= =
= =
= = =
= =

( )
0
1
( )
0
1
1 [ | ]
1 [ | ]
n
ii ii
n
n
ii ii
n
ecurrent f P E M X i
transient f P E M X i

=
= = = =
< < = <

Basic Limit Theorem of Markovian

( )
0
0
1 return time min( 1: )
( | )
irreducible, recurren, aperiodic Markovian
1
( | )
st
j n
n
j jj
j j
j
R n X j
P R n X j f
E R X j m

= = =
= = =
= = =

Poisson Process
Poisson Distribution
1 1 2 2 1 2 1 2
~ ( ) if ( ) , ( ) ( )
!
~ ( ), ~ ( ), ~ ( )
| ~ ( , ), ~ ( ) ~ ( )
x
e
X Poi P X x E X Var X
x
X Poi X Poi indep X X Poi
M N Bin N p N Poi M Poi p

= = = =
+ +

Homogenous Poisson Process

2 1 4 3 1 2 3 4
{ ( )} is a Homogenous Poisson process with rate if
( ) ( ), ( ) ( ) are indep,
( ) ( ) ~ ( ), (0) 0
*** ( ) ~ ( )
X t
X t X t X t X t t t t t
X s t X s Poi t X
X t Poi t

< <
+ =

Non-Homogenous Poisson Process
2 1 4 3 1 2 3 4
0
1
{ ( )} is a Non-Homogenous Poisson process with
rate function ( ) if
( ) ( ), ( ) ( ) are indep,
( ) ( ) ~ ( ( ) ), (0) 0
*** ( ) ~ ( ( ) )
if ( ) ~ ( ) ( ) | (
s t
s
t
X t
t
X t X t X t X t t t t t
X s t X s Poi d X
X t Poi d
X t Poi t X t X

+
< <
+ =

1
2 1 2
2
1 2 1
1 2
2
1 2 1
2
1 2 1 1 1
2 2 2
) ~ ( , ) for
( ( ) , ( ) ( ) )
( ( ) | ( ) )
( ( ) )
( ( ) ) ( ( ) ( ) )
( ( ) )
( ) !
1
!( )!
k N k
k N k
N
t
t N Bin N t t
t
P X t k X t X t N k
P X t k X t N
P X t N
P X t k P X t X t N k
P X t N
N
t t t t t N
k k N k t t t

= <
= =
= = =
=
= =
=
=
| | | | | |
= =
| | |

\ \ \
Poisson Related Distribution
1
2
0
1
2
1
~ ( , ), ( ) , ( ) , ( )
( )
waiting time for the event, 0
1
( ) ~ ( ), ~ ( , ), ( )
( 1)!
( )
1 1
~ ( ), ( ) , ( ) , ( )
soj
n
x
W
th
n
n n
t
n W
k
x
S
n n n
x e
W f x E W Var W
W n W
t
X t Poi t W n f t e
n
N t k W t
S Exp f x e E S Var S
S W W

+
= = =

= =
=

= = =
= =
1
. . .
1
0
1
,..., | ( ) 1
ourn time between the and ( 1) event
( ) ~ ( ), ~ ( ), (1, ) (1/ )
1
~ ( , ), ( | ) ( )
,..., | ( ) ~ ordered uniform
! , 0 ...
n
th th
i i d
n
n
n k
k
n
n
W W X t n n
n n
X t Poi t S Exp Exp
W S n P S x y S x P S y
W W X t n
f n t W W t

=
+
=
= > + > = >
=
= < < <

1
. . .
1 1
if ( ,..., ) is symmetric, then
( ( ,..., ) | ( ) ) ( ( ,..., )), ~ (0, )
n
i i d
n n i
g w w
E g W W X t n E g U U U U t = =
Spatial Poisson Process
1 2 1 2
1
1 1
1
{ ( )} is a homogenous Poisson point process with
intensity if
( ), ( ) are indep,
( ) ~ ( | |), ( ) 0
( 1,..., ) and and
| | !
( ( ) ,..., ( ) )
!... ! |
m
i i j k
k
k
m m
m
X A
X A X A A A
X A Poi A X
A A i m A A A A
A n
P X A x X A x
x x

=
=
=
= = =
= = =

1
1 2 1 2
|
{ ( )} is a non-homogenous Poisson point process with
intensity ( ) if
( ), ( ) are indep,
( ) ~ ( ( ) ), ( ) 0
k
x
m
k
A
A
X A
A
X A X A A A
X A Poi d X

=
| |
|
\
=
=

Compound Poisson Process

( )
1
2 2 2
( )
0
( )
( ) ~ ( ), i.i.d., ( ) is Compound Poisson Process
if ( ) for ( ) 0, ( ) 0 otherwise
( ), ( ) ( ( )) , ( ( )) ( )
( )
( ) ( ) ( )
!
( )
i
X t
i
i
i i
n t
n
Z
n
n
X t Poi t Y Z t
Z t Y X t Z t
E Y Var Y E Z t t Var Z t t
t e
F z P Z z G z
n
G z

=

=
= > =
= = = = +
= =
=

( 1)
1
(0)
( )
0
( ) ( ) '( )
1 for 0
( )
0 for 0
1
min( : ( ) ), ( ) ( )
n
n
i
i
n
n
P Y z G z G d
z
G z
z
T t Z t E T G

=
=

=

<

= > =

Marked Poisson Process

( )
1
( )
1 1
1
waiting time for the event, i.i.d. r.v. happened at
( , ) is a marked Poisson process
( ) ( ) ( ) indep, ( ) ~ ( ( ) )
( ) ( ) ( ) ~ ( ( ) )
If
th
i i i
i i
X t
i k i i k
k
X t
k k
k
i
W i Y W
W Y
X t I Y i X t X t Poi P Y i t
X t I Y A X t Poi P Y A t
Y

=
=

= = =
=

is continuous i.i.d. r.v. with p.d.f. ( ), ( , )form a 2D

non-homogeneous Poisson point process in the plane
(W, )( ) ~ ( ( ) )
Y i i
i i Y
A
f y W Y
t y
Y A Poi f y dydt

Small o-function
0
( )
lim 0
h
o h
h

=

Pure Birth Process
1,
2,
0 0 0
( ) is a pure birth process with birth rate if
( ( ) ( ) 1| ( ) ) ( )
( ( ) ( ) 0 | ( ) ) 1 ( )
( ( ) ( ) 0 | ( ) ) 0
(0) 0
( ( ) ) ( )
'( ) ( )
'( )
k
k k
k k
n
n n
X t
P X t h X t X t k h o h
P X t h X t X t k h o h
P X t h X t X t k
X
P X t n P t
P t P t
P t P

+ = = = +
+ = = = +
+ < = =
=
= =
=
=
0
0
1 1
0
0 1 0, ,
,
0 1
( ) ( ), 1
Sojourn time time between and ( 1) event ~ ( )
( ) , indep, ( )
if , , ( ) ... ... , 1
1
( )...( )
i
i
n
n n n
th th
i i
s t
S i i
t t
i j n n n n n
k n
k k k
t P t n
i i S Exp
f s e S P t e
i j P t B e B e n
B

+
= + =
= =
( = + +

=

0 1
0 0 0
1
( )...( )
1
( ) 1
n
i k k n k i k
i k
n n
n n n n
P t S

= +

= = =
=

= = =

Yule Process/Linear Birth Process
| |
1
1
1
1
, 1
'( ) ( ) ( 1) ( )
(0) 1, (0) 0, 2
( ) (1 )
( 1)
( 1)!( )!
k
n n n
n
t t n
n
k
k n n
k
P t nP t n P t
P P n
P t e e
B
k n k

=
=
= =
=

=


Homogeneous
Poisson Process
Yule Process/Linear
Birth Process
k
k
( )
n
P t
( )
!
n
t
t
e
n

1
(1 )
t t n
e e


( ) ~ X t ( ) Poi t ( )
t
Geo e


, k n
B
N/A
1
1
( 1)
( 1)!( )!
k
n
k n k

Pure Death Process
1,
2,
( ) is a pure death process with death rate if
( ( ) ( ) 1| ( ) ) ( )
( ( ) ( ) 0 | ( ) ) 1 ( )
( ( ) ( ) 0 | ( ) ) 0
0,1,...,
( ( ) | (0) ) ( )
(0) 1, (0) 0
k
k k
k k
n
N n
X t
P X t h X t X t k h o h
P X t h X t X t k h o h
P X t h X t X t k
k N
P X t n X N P t
P P

+ = = = +
+ = = = +
+ > = =
=
= = =
= =
1 , ,
,
0 1 1
( )
if , , ( ) ... ... ,
1 1
( )...( )( )...( )
Sojourn time minimum lifetime for the survivors after the death
N
n N
t
N
t t
i j n N n n n N n
N n
k n
i N k k k k k n k N i k
i k
P t e
i j P t A e A e n N
B
k

= +

=
( = + + <

= =

=

of ( 1) survivor ~ ( ), ( ) , indep
i
k
s th
k k S k k
k S Exp f s e S

+ = =
Linear Death Process
, 1
1 1
0
!
( ) (1 ) ,
!( )!
1
( 1) ( )!( )!
... min( 0 : ( ) 0)
( ) ( ) ( ( ) 0) ( ) (1 )
Sojourn time ~ ( )
k
n t t N n
n
k n N n k n
N N N
t N
T
k
k
N
P t e e n N
n N n
A
N k k n
W S S S t X t T
F t P T t P X t P t e
S Exp k

=
=

=

= + + + = = =
= = = = =
=

Birth and Death Process
,
, 1 1,
, 1 1,
,
( ) ( ( ) | ( ) )
( ) is a birth and death process with birth rate and death rate if
( ) ( ( ) ( ) 1| ( ) ) ( )
( ) ( ( ) ( ) 1| ( ) ) ( )
( )
i j
k k
k k k k
k k k k
k k
P t P X t s j X s i
X t
P h P X t h X t X t k h o h
P h P X t h X t X t k h o h
P h

= + = =
= + = = = +
= + = = = +

2,
,
0 0
, , ,
0
0, 0 0,
( ( ) ( ) 0 | ( ) ) 1 ( ) ( )
1 ,
(0) ( ( ) | ( ) )
0 ,
0, , , 0, 1, 2,...
( ) ( ) ( ) (backward equation: (0, ), ( , ))
'( ) ( )
k k k
k l
k k
i j i k k j
k
j j
P X t h X t X t k h o h
l k
P P X t l X t k
l k
k
P t h P h P t h h t h
P t P t

=
= + = = = + +
=
= = = =

= > =
+ = +
= +

0 ,
, 1, , 1,
, , ,
0
,0 ,0 ,1
, 1 , 1 , 1
( )
'( ) ( ) ( ) ( ) ( ), 1
( ) ( ) ( ) (foward equation: (0, ), ( , ))
'( ) ( ) ( )
'( ) ( ) ( ) ( )
i j
i j i i j i i i j i i j
i j i k k j
k
i i i i i
i j j i j j j i j j
P t
P t P t P t P t i
P t h P t P h t t t h
P t P t P t
P t P t P t P

=
+
= + +
+ = +
= +
= + +

, 1
( )
0 0
1 1 1 1
2 2 2 2
( ), 1
Sojourn time the time in the state
~ ( ), ( ) ( ) , indep
infinitesimal generator of ( )
0 0
( ) 0 sum of a row
0 ( )
i i
i
i j
th
s
i i i S i i i
t j
i
S Exp f s e S
X t A





+
+

= =
+ = +
= =
| |
|
+
|
| +
|
\

0 =
Limiting Behavior of Birth and Death Process
, ,
0 1
0
0 1 1
0
1 2
0
0
'( ) ( ) , [ ( )] ( )
lim ( ) [ , ,..., ,...], ( ( ) )
0 , 1
...
1, for 1
...
if ,
if , 0 for all
Growth with
i j i j
i j j
t
j
j
j
j
j
j
j j j
k
k
j j
t A t t P t
t P X j
A
j
j

=
= =
= = = =
= =
= =
< = =
= =

P P P
P

immigration : ,
( ) [ ( )], '( ) ( ) ( )
n n
n a n
M t E X t M t a M t


= + =
= = +

Renewal Process
1 2
0
1
, ,..., ,... 0 are i.i.d.(lifetime of the renewal item)
the no. of event happened before ( ) ( ) is
a renewal process, ...
( )
if ~ ( ) ( ) ~ ( )
( ( ) ) (
th
i
i
i
i i
k
i
X X X i
t N t I W t
W X X
N t k W t
X Exp N t Poi t
P N t k P

=
>
= =
= + +


=

) ( )
k k
W t F t =
Renewal Function
1 1 1
( ) 1
1 1
1 1 1 1
1 1 1 1
( )
( ) ( ( ))
( ) ( ) ( ) ( ( ( ) ) )
( ) ( ( ) 1)
Proof: { ( ) 1} { ... }
[ { ... }] [ ] [ { ... }]
[ ] { ... } ( )
[
k k
k k k
N t
j
j j j j
j j
N t
M t E N t
M t P W t F t P N t k
E W M t
I N t j I X X t
E X I X X t E X E I X X t
E X P X X t F t
E W

= = =
+

=
= = =
= +
= + +
+ + = + +
= + + =

1 1 ( ) 1
( ) 1
1
2
2
1 1
2
1
2
] [ ... ]
[ ]
{ ( ) 1 }
{ ... }
( ) ( ( ) 1)
N t
N t
j
j
j
j
j j
j
j
j
E X X
E X E X
E X I N t j
E X I X X t
F t M t

+ +
+
=

=
= + +
(
= +
(

(
= + +
(

(
= + + +
(

= + = +

Elementary Renewal Theorem

1
( ) 1 1
lim
( )
t
M t
t E X

= =

Residual lifetime/excess R.V. of the N(t)
th
item=
( ) 1 t N t
W t
+
=
Current lifetime/excess R.V. of the N(t)
th
item=
( ) t N t
t W =
Total lifetime of the N(t)
th
item=
( ) 1 ( ) t t t N t N t
W W
+
= + =
For Poisson Process:
( ) [ ( )]
{ } { ( ) ( ) 0} { ( ) 0}
{ }
no renewal in ( , ] { ( ) 0}, 0
0 ,
1 , 0
{ }
1 ,
x
t
t
t
x
t
M t E N t t
P x P N t x N t P N x e
P x
t x t P N x x t
t t x
e x t
P x
t x

= =
> = + = = = =
> =
= <

<

<
=


0
0
1
[ ] [ ] [ ] { }
1 1 1
(1 )
t
t t t t
t
x t
E E E P x dx
e dx e

= + = + >
= + = +

Examples
(1) (2)
11 11
2 2
( )
11
( )
11 11
1
1/ 5 1/ 5 1/ 5 1/ 5 1/ 5
0 1/ 3 0 2 / 3 0
Class:{0, 2, 4},{1, 3} 0 0 1/ 4 0 3/ 4
0 2 / 3 0 1/ 3 0
1/ 3 0 1/ 3 0 1/ 3
1 2 2 4
,
3 3 3 9
2 1 2 4 1
for 3,
3 3 3 9 3
1
n n
n
i
i
f f
n f
f f

=
| |
|
|
|
|
|
|
\
| || |
= = =
| |
\ \
| || | | | | |
= =
| | | |
\ \ \ \
= =

(1) (2)
22 22
( )
2 0
(1) (1)
02 42
( ) ( 1) ( 1)
02 02 42
( ) ( 1) ( 1)
42 02 42
(
02
state 1 and 3 are recurrent
1 3 1 1
,
4 4 3 4
Let { 2, 2, 1, 2,..., 1| }, 0, 4
1 1
,
5 3
1 1
5 5
1 1
3 3
n
i n r
n n n
n n n
f f
f P X X r n X i i
f f
f f f
f f f
f


| || |
= = =
| |
\ \
= = = = =
= =

= +

= +

) ( ) ( 1) ( 1)
42 02 42
1
(1) (1)
02 42
1 1
( )
5 3
1 1 1 1
( ) , 3
5 3 5 3
n n n n
n n
f f f
f f n

| |
+ = + +
|
\
| | | |
= + + = +
| |
\ \

( ) ( 2) ( 2)
22 02 42
( )
22 22
1
1 2 1 2 1 2
3 1
( )
4 3
11
1 {0, 2, 4} is transient
14
Limiting distribution of {1, 3}
1 3
, 1 1/ 3 2 / 3
3 2 / 3 1/ 3
1
( ) 0 , 1
2
Mean time spending on r
n n n
i
i
T
T
f f f
f f
I

=
= +
= = <
= =
| |
|
\
= = + = = =

P P
P
1
1
ecurrent class
( ) (7.5, 13.667, 11.833)
1
Mean time returning to state 1 2
T
I Q

= =
= =
v 1

( ) Poisson Process with rate , given ( ) , find ( )
( | ( ) ) ( ( ) | ( ) ) ~ ( , )
1 , ( ) 1
r
r
W
r
k n k k n k
n n
W
k r k r
X t X t n f w
w
P W w X t n P X w r X t n Bin n
t
n n
w w d w w
f w
k k t t dw t t
n
d w
k dw t

= =
=
= = =
| | | |
| | | | | | | |
= =
| | | | | |
\ \ \ \
\ \
| |
|
=
|
\
\

1
1 1
1
1
1 1
1
1 1
1 ( ) 1
1
1
1
k n k n
n
k r
k n k k n k
n
k r
n
k n k
k
w d w
t dw t
n
w w w w
k n k
k t t t t t t
w
n
t t
n
w w w
k n
k t t t t

( (
| | | | |
+
( (
| | |
\ \
( (

(
| |
| | | | | | | |
=
(
| | | | |
\ \ \ \
\ (

| |
+
|
\
| |
| | | | (
=
| | |
(
\ \
\

1
1
1 1 1
1
0
1 1
1
1
1 1
1
where: 1
!
1
( 1)!( )!
n
n
r
k r n k r n
n r
k
k r n k r
k r
w
n
t t
n
w w w w
k r n n
k r t t t t t t
n
w w w
k r n
k r t t t
n w w
k r n k r t t

=
+

=
+
+
| |
+
|
\
| |
| | | | ( | |
= + +
| | | |
(
+
\ \ \
\
| |
| | | | (
+
| | |
(
+
\ \
\
| | |
=
|
+
\ \

1
1
1
1
!
1
( )!( )!
!
1
( 1)!( )!
!
1 ( 1) ( )
( )!( 1)!
!
( ) 1
( )!( 1)!
n k r
k r n k r
k r n k r
k r n k r
k r
n w w
n
k r n k r t t
n w w
k r n k r t t
n w w
f k f k
k r n k r t t
n w w
f k
k r n k r t t

+
+
+
+
|
|

| | | |

| |
+
\ \
| | | |
=
| |
+
\ \
| | | |
=
| |
+
\ \
| |
=
|
+
\
1
1
1
1 1
( ) [ ( 1) ( 1)]
! 1
1
( 1)!( )!
r
n k r
n
W
r n r
w
f w f f n r n
t t t
n w w
r n r t t t

| |
|
\
| |
= +
|
\
| | | |
=
| |

\ \

(4)
1 1 1
4
1
4 (4)
1 1
4
2000 2000 | (4) ( (4) )
(1 )
| (4) ( ) , ~ (0, 4)
4
(1 )
2000 2000 ( (4) )
4
(1 )
500 ( (4)
k k
k
k
X n
W W
k n k
n
W U
k
X
W
k n
E e E e X n P X n
n e
E e X n nE e U Uni
n e
E e P X n
e
nP X

= = =

= =

( | |
= = =
( |
\
| |
= = =
|
\
(
= =
(

= =

4
1
4
4
5
(1 )
) 500 ( (4))
(1 ) 10000
500 (4 ) | (1 )
n
e
n E X
e
e

= =

2
2
2
2
2 0
0 0
Consider 2 Poisson process of particles with intesity
distance of a particle with its nearest neighbour
( ) ( ) 1 ( ) 1 ( ( ) 0)
( )
1 1
0!
( ) (1 ( ))
D
x
x
x
D
D
D
F x P D x P D x P X x
e x
e
E D F x dx e dx

=
= = > = =
= =
= =

2
0
1
2
1 1 1
2 2
y
e dy

2

=
= =

1
1
( ) ( )
0 1
1 1 1
Shock Model: ~ ( ), ( ) (1 ) , 1, 2,...
~ ( , )
min{ : ( ) 0}
1 1
( ) ( ) (1 ( ))
1
1 1
(1 ( )) (1 (1 ) )
1
1
1
(1
1
k
n
i
i
n n
n n
n
n k n
i
n i n k n
Y Geo p P Y k p p k
n Y NegBin n p
T t Z t
E T G G
k
P Y p p
n
k
n

=

= =

= = = =
= = =

= >
= = +
| |
= + = +
|

\
|
= +

1 1
1
1 0
(1 ) )
1
1 1
(1 (1 ) ) (1 )
k
n k n
k n
k
n k n
k n
p p
k
p p p p
n

= =

= =
|

|

| |
= + = +
|
\

. . .
1 2 ( )
1 2 ( )
( ) Poisson Process of intensity
~ ( ) , (0,1), ( ) min{ , ,... }
( ( ) )
( ( ) | ( ) 0)
( ( ) 0)
( ( ) ) (min{ , ,... } )
( ( ) 0) ( ( ) 0)
i i d
i N t
N t
N t
Y G y y y Z t Y Y Y
P Z t z
P Z t z N t
P N t
P Z t z P Y Y Y z
P X P X

= =
>
> > =
>
> = >
= = >

Y
N(t)



W
N(t)

0 0
0
Parameter of ( ) '( )
Parameter of ( ) '( ) (1 )
t z
t
z
X G y dyd tz
X G y dyd t z

= =
= =



1
0 (1 ) 0
1
1
P( ( ) 0) ( ( ) 0)
( ) ( (1 ))
1
0! 0!
( ( ) )
( ( ) | ( ) 0)
( ( ) 0) 1
lim ( ( ) ) lim ( ( ) ) lim( )
tz t z
tz t
t
t
t
t
t t t
X P X
e tz e t z
P Z t z e e
P Z t z N t
P N t e
P t Z t P Z t e e
t

| |
|

|
|

\

= >
| || |

=
| |
| |
\ \
>
> > = =
>
> = > =
e


=
1
1 1
1
1
0
( ... )
0
1
0
( ) Yule Process, , (0) 1, ~ ( )
( ( ) 1) ( ) ( ) ( )
( ) [ ] [ ] [ ]
[ ]
/
( ( ) 1)
n
n n i
n
i i i
n
n W
n
W S S S w
W
i
S s i s
i
X t n X T Exp
P X T n P T W P T w f w dw
e f w dw E e E e E e
i i
E e e i e ds
i i
i
P X T n
i

+
+
+

+ +
=

= =
+ = =
= = = =
= = =
+ +
+ =

1
/
n
i

=
+

Repairman Model
total machines, can operate at most machines
repairmen
min( , )
min( , )
3 machines, 2 repairmen, at most 2 operating
time the machine works ~ exp with mean 5
time for a repairman to fix a m
n
n
N M
R
M n
R N n


=
=
( )
( )
0 1 2 3
1
achine ~ exp with mean 4
( ) 0 1 2 3
Birth 2 / 4 2 / 4 2 / 4 0
Death 0 1/ 5 2 / 5 2 / 5
2 / 4 2 / 4 0 0
1/ 5 1/ 5 2 / 4 2 / 4 0
0 2 / 5 2 / 5 1/ 4 1/ 4
0 0 2 / 5 2 / 5
64 160 200 125
0 , 1 , , ,
549 549 549 549
Average Machines Op
i
i
N
i
i
X t
A
A

=
| |
|
+
|
=
|
+
|

\
= = = = = =

1 2 1
1 2
1
1 2
erating
2 ... ( ... )
Average Machines Operating
Long Run Utilization
Capacity
2 ...
( ... )
Average Idle per Capacity 1 2 ...
M M N
M
M N
N R N R N
M M
M
M
R




+
+
+ +
= + + + + + +
=
+ + +
= + + +
= + + +


A system has two components and , both switching
between two states: 0 OFF and 1 OPERATING. In
state 0, component remains there for an exponentially
distributed random time with parameter .
A
A B
A

= =
In state 0,
component remains there for an exponentially distributed
random time with parameter . Component behaves
similarly with parameters and . The system is
operational as long as one
A
B B
A
B

of the components is operational.
i) In the long run, what fraction of time is the system
inoperational?
ii) Once the system enters the failed state, what is the mean
duration there prior to returning to operation?
iii) Define a cycle as the time between the instant that the
system first enters the failed state and the next such instant.
Using renewal theory, find the mean duration of a cycle.
iv)
0 1
What is the mean system operating duration between
successive system failures?
i) Consider component only
0 1
0
1
0 , ( , )
,
A A A
A A
A A
A A A A A
A A A A
B B
B
B B B B
A
G
G

= | |
|

\
| |
= =
|
+ +
\
| |
=
|
+ +
\

0 1
0 0
( , )
(Inoperational)
( )( )
ii) Let be the time prior to returning to operation for
Let be the time prior to returning to operation for
(min( , )) (min( , )
B B
A B
A B
A A B B
OA
OB
OA OB OA OB
P
S A
S B
E S S P S S




= =
+ +
=

0
0 0
( )
0
B
( )
1
)
( ) ( )
1
iii) Define as the time of cycle
( operating time failure time)
as the failure time
*** (Inoperational)
( )
lim lim
A B
A B
x x
OA OB
x
A
i
i
N t
i
i
t
x dx
P S x P S x dx e e dx
e dx
X
Y
P
E Y
t

+
=

>
= > > =
= =
+
= +
= =

( ) ( ( ))
( ( )) 1
( ) lim ( )
( )
( ( )) 1
elementary renewal theorem: lim
( )
i
t
i i
t
i
t
i
E Y E N t
t
E N t
E Y E Y
t E X
E N t
t E X

= =
=

1
i: (Inoperational) , ii: ( )
( )( )
1 1
( )( ) ( )
( )
( )
( )( )
( ) 1
iv) ( ) ( ) ( )
( )( )
In a Markov Chain there
A B
i
A A B B A B
A B
A A B B A B i
A B A B
i
A A B B
A B A B
i i i i
A A B B A B
P E Y
E X
E X
E X Y E X E Y








= =
+ + +
=
+ + +
+
=
+ +
+
= =
+ + +
( )
( )
are two classes A and B, both are recurrent.
For any and and any , 0
Proof: Suppose , i.e. s.t. 0
is recurrent 1
s.t. probability from state to state to state
n
ij
n
ij
ii
i A j B n P
i j n P
i f
m i j
=
>
=

( ) ( )
( )
( )
( )
after steps 0
i.e. 0
0
s.t. 0
A and B are the same class contradiction
A, B, , 0
n m n
ij ji
m n
ji
m n
ji
n
ij
i m
P P
P
m n P j i i j
i j n P

>
>
>
>

=

{ }
2 0 1
0
1 2 1 1
1 2
1 2 1
, 2, 3,...( , given), i.i.d. can be
viewed as a Markov process
{ } Markov process ( | , ,..., ) ( | )
0 1
Let
1 0 0
n n n n
n
n n n n n
n n n n n
n
n n n
y y n y y
X E X X X X E X X
y y y
Z
y y y

= + =
=
+ | | | | | | | |
= = + =
| | | |
\ \ \ \
1 2 1 1 1 1
1
1 1 1 1
0 1
( | , ,..., ) | ,...,
1 0
0 1
( )
1 0
0 1 0 1
( | ) | ( )
1 0 1 0
( ) is a Poisson Process with rate 2
i) ( (1) 2)
n n n n n
n n
n n n n n n n
E Z Z Z Z E Z Z Z
Z E
E Z Z E Z Z Z E
X t
P X

| |
|
\
| | | |
= +
| |
\ \
| |
= +
|
\
| | | | | |
= + = +
| | |
\ \ \
=

0 1 2
2
ii) ( (1) 1, (3) 6)
iii) ( (2) 3| (1) 1) iv) ( (2) 6 | (4) 8)
i) ( (1) 2) ( (1) 0) ( (1) 1) ( (1) 2)
0.6767
0! 1! 2!
ii) ( (1) 1, (3) 6) ( (3) (1) 5, (1) 1)
( (3) (1)
P X X
P X X P X X
P X P X P X P X
e e e
P X X P X X X
P X X

=
= =
= =
= = + = + =
= + + =
= = = = =
= =
2 5 1
2
5| (1) 1) ( (1) 1)
(2 )
0.0423
5! 1!
iii) ( (2) 3| (1) 3) 1
(2) (1) ( (2) (1))
{ (1) 1} { (1) 1} { (1) 2} { (1) 3}
( (2) 3, (1) 1)
( (2) 3| (1) 1)
( (1) 1)
( (2) 3, (1) 1) ( (2) 3, (1)
X P X
e e
P X X
X X X X
X X X X
P X X
P X X
P X
P X X P X X

=
= =
= =
=
= +
= = =

=

= +
=

(4 2) 2 2 6
4 8
2) ( (2) 3, (1) 2)
( (1) 1)
0.8305
( (4) 8| (2) 6) ( (2) 6)
iv) ( (2) 6 | (4) 8)
( (4) 8)
( (4) (2) 2) ( (2) 6) ( (2 ) / 2!) ( (2 ) / 6!)
( (4) 8) (4 ) / 8!
0.109375
P X X
P X
P X X P X
P X X
P X
P X X P X e e
P X e

= +

=
= = =
= = =
=
= =
= =
=
=
1 1 2 2
1 2 1 2
11 1 12 2
11 12 11 12 12 12
( ) Poisson process with rate , Poisson process with rate
, indep, ( ( ) 1 before ( ) 1)
waiting time to get ( ) 1, waiting time to get ( ) 1
( ) ( | )
W
X t X
X X P X t X t
W X t W X t
P W W P W w W w f

= =
= =
< = < =
12
12
1 11 2 12
1 12 2 12
12 12
0
1 11 2 12
0 0
2 12
0
1
2
1 2 1 2
3 1
2 2
3 1
2 2
( )
(1 )
1
1
1
Given (1) 3, find joint distribution of and
1
1
( , | (1) 3) ( , | (1
1
w
w w
w w
w dw
e dw e dw
e e dw
W W
X U V
W W
W W
P U u V v X P u v X
W W

=
=

= + =
+ +

= = =

= =

2
2 2
1 2 3 2
1 1 1 1
3
1 3 2 2 3 2
0 1 0 0 1
1 1
2
2 2 3 2 2 2 3 2
0 0
2
2
,
) 3)
( , 1 | (1) 3)
3!1 6
1
6 ( ) 6 6
6
( , ) ( , | (1) 3) 1 for ( , ) (0,1)
uw
v vw v vw
U V
P W uW W v vW X
dwdw dw uw dw dw
uw v vw dw dw uv w w dw dw uv uv
f u v P U u V v X u v
u v

+ +
=
= + =
= =
= = = =

= = =




Let be the excess life and the age in a renewal process having
inter-occurrence distribution function ( ). Determine the conditional
probability ( | ) and the conditional mean ( | )
(
t t
t t t t
F x
P y x E x
P

> = =
( )
( ) ( ) ( )
( ) ( )
( )
0 0
| ) ( | )
( | )
( , )
1 ( )
( ) 1 ( )
1
( | ) ( | ) 1 ( )
1 ( )
t t N t t t t
N t t t N t N t t t
N t N t
N t
t t t t
y x P S x y x
P S x y S x S x
P S x y S x
F x y
P S x F x
E x P y x dy F x y dy
F x




> = = = + > + =
= = + > + = +
> +
+
= =

= = > = = +

1 2
Certain particles appear in time according to a Poisson process
of intensity . Each particle exists for a random duration and is
then annihilated. Suppose that the successive lifetimes , ,...
of d
Y Y

istinctive particles are independent random variables having

the common distribution ( )=Pr{ }. Let ( ) count the number
of particles existing at time . Then ( ) follows a Poisson distribution
k
G y Y y M t
t M t

0
1
1
with parameter , where is given by [1 ( )] .
Let (t) be the number of particles created up to time t, We observe
that ( ) ( ); Conditional on ( ) and let ,..., be the
times o
t
n
pt p p G z dz
t
X
M t X t X t n W W t
=
=

1
1
f the particle creation. Then particle existed at time if and
only if . Therefore
Pr{ ( ) | ( ) } Pr{ [ ] | ( ) }
!
Pr{ [ ] } (1 ) ( ( , ))
!( )!
Final
k k
n
k k
k
n
m n m
k k
k
k t
W Y t
M t m X t n I W Y t m X t n
n
I U Y t m p p Bin n p
m n m
=

=
+
= = = + = =
= + = =

(1 )
ly Pr{ ( ) } Pr{ ( ) | ( ) }Pr{ ( ) }
! ( )
(1 )
!( )! !
( ) (1 ) ( )
! ( )!
( ) ( )
.
! !
n m
n t
m n m
n m
m n m n m
t
n m
m m
t t p pt
M t m M t m X t n X t n
n t e
p p
m n m n
pt p t
e
m n m
pt pt
e e e
m m

=

= = = = =
=

= =

Shock Model
( )
0
( )
0 0
0
( ) 1 ( )
0
0 0
. . .
1 2
min{ : ( ) } , { } { ( ) }
( )
( ) ( ( ) ) ( )
!
( )
( ) ( ) ( )
!
( )
( ) ( )
!
if ~ ( ), ... ~
n t
n
n
n t
n
n
n t
n n
n n
i i d
i n
T t Z t T t Z t
t e
P T t P Z t G
n
t e
E T P T t dt G dt
n
t e
G dt G
n
Y Exp Y Y Y Gam

=


=

= =
= > > <
> = < =
= > =
= =
+ + +

1
( )
1
0
1
( )
0 0 0 0
0
( ) ( )
( ... ) ( ) 1
! !
( ) Poisson process with rate , ( ( ) ) ( ... )
( ) ( )
( )
! !
( )
(1 )
!
k z k z n
n
n
k k n
n
k a k a k
n
n n k n k n
k a
k
ma
z e z e
P Y Y z G z
k k
X t P X z k P Y Y z
a e a e
G a
k k
a e
k
k

= =

= = = = =

=
+ + = = =
= + +
= =
= + =

1
1 [ ]
a
a E T

+
+ =

( )
( ) ( )
( )
0 0
1 1
1 1
2 1
2
3 2 1
1
1
1
1
Gambler's ruin problem
= P{ reaches 0 before | } 1, 0
First Step Analysis
Put 0
/
/ /
/
Moreover, we have
1 ( )
i n N
k k k
k k k k k
N
N
k
k j j j
j
u X N X k u u
u pu qu
d u u pd qd
d q p d
d q p d q p d
d q p d
u u u d
+
+

=
= = =
= +
= =
=
= =
=
= =

( )
1
1
1 0
1
1
0
1
1 1 1
0
0
1
1
2
0
1
0
0
/
1 ( / )
1
0 1 ( / )
( / )
1 / , if ( / )
1
1 ( / )
1 , if
( / )
1 ( / )
[ | ]
(
k k
j
j j
k
j
k
j
N
j
N N
j j
j
k
j
j
k
k N
j
N
j
i
i
q p d
u d q p
u d q p d
q p
k N p q q p
u
q p
p q
q p
q p
v E T X i
k N
v

= =

=
=

=
=
= +
= = + =
= =

= =

= =

1
2
1
1 2 1
1 2
1 2
1
1
), if
1 1
, 1
(1 ) 1
...
, 1,..., 1 if 1, i.e. 0
1 ...
...
where:
...
...
(
1 ...
i
N
k N
k k
N
k
k
k
k
k
N
k p q
q
N i
p p
u k N p q r
q q q
p p p
v

= =

( | |
=
( |

\
+ +
= = + >
+ + + +
=
+ +
=
+ + +
1 1
1
1 1
1 2 1 1
)(1 ... )
( ... ) , 1,..., 1
1 1 1
( ... ) , 1,..., 1
k
k
k k
k k
k N
k N
q q q

+ + +
+ + =
= + + + =