Stochastic Models

Lecture 4
(Birth-Death process, Equations of Motions,
Equilibrium)

by
Dr. Olivia Morad
2018
1
 COURSE GROUP
https://drive.google.com/drive/folders/0B15 •
cWEHDjCvpcjRCVzF6aTlEWjA

2
A-Birth-Death Processes

• Birth– death is the special case of a Markov process in which

the transitions from state Ek are permitted only to neighboring
state Ek+1, Ek, Ek-1.

Sk-1 Sk Sk+1

3
• Our Main interest will focus on continuous – time birth death
process with discrete state space in which transitions only to
neighboring state Sk+1 or Sk-1 from state Sk are permitted.

Sk-1 Sk Sk+1

• Birth –death process → modeling changes in the size of a

population.

• When the process in state Sk → the population at that time of

siz k.

• A transition from Sk → Sk+1 → birth within a population,

whereas a transition from Sk → Sk-1 → death in the population.
4
• We consider changes in size of a population where transitions
from state Sk take place to nearest neighbors only.

• Birth rate λk : rate at which births occur when the population is

of size k.

• Death rate µk : rate at which deaths occur when the population

of size k.

λi are independent
µi are independent

5
 λ0 λ1 λk-1 λk

….. …..
0 1 2 k-1 k k+1

µ1 µ2 µk µk+1

State-transition-rate diagram for the birth-death process

6
B- Equations of Motion

Sk+1 Death

No change
Sk Sk

Birth
Sk-1

Time
t t + ∆t

"Possible transitions into Sk"

7
1- That we had k in the population at time t and no state
changes occurred.

2- That we had k-1 in the population at time t and we had a

birth during the interval (t, t+ ∆t).

3- That we had k+1 members in the population at time t and

we had one death during the interval (t, t+ ∆t).

8
 B1: P [Exactly 1 birth in (t, t + ∆t) k in population]
= λk ∆ t + O (∆ t)

D1: P [Exactly 1 death in (t, t + ∆) k in population]

= µk ∆ t + O(∆ t)

B2: P [Exactly 0 births in (t, t + ∆) k in population]

= 1- λk ∆ t + O (∆ t)

D2: P [Exactly 0 death in (t, t + ∆) k in population]

= 1- µk ∆ t + O(∆ t)

9
Pk (t + ∆ t ) = Pk (t ) Pk , k (∆ t )
+ Pk −1 (t ) Pk −1, k (∆ t )
+ Pk +1 (t ) Pk +1, k (∆ t )
+ O (∆ t ) k >= 1 .......... ......( 1)

P0 (t + ∆ t ) = P0 (t ) P0 , 0 (∆ t )
+ P1 (t ) P1, 0 (∆ t )
+ O (∆ t ) k = 0 .......... ........( 2 )
10
 • To solve the system represented by Eqs (1) & (2), we must
make use our assumptions B1, D1, B2, and D2, in order to
evaluate the coefficients in these equations.

11
Pk (t + ∆t ) = Pk (t )[1 − λk ∆t + O (∆t )][1 − µk ∆t + O (∆t )]
+ Pk −1 (t )[λk −1 ∆t + O (∆t )]
+ Pk +1 (t )[µk −1 ∆t + O (∆t )]
+ O (∆t ) k >= 1 .................(1)

P0 (t + ∆t ) = P0 (t )[1 − λk ∆t + O (∆t )]
+ P1 (t )[µk −1 ∆t + O (∆t )]
+ O (∆t ) k = 0...................( 2)

12
Taking the limit as ∆t approaches 0: Equations of
Motion
dPk (t )
= −(λk + µk ) Pk (t ) + λk −1Pk −1 (t ) + µk +1Pk +1 (t )
dt
k >= 1....................(1)

dP0 (t )
= − λ0 P0 (t ) + µ P1 (t )
dt
k = 0......................(2)
13
• The set of equations given by (1) & (2) is clearly a set of
differential-difference equations and represents the dynamics of our
probability system, their solution will give the behavior of Pk(t)
(equations of motions for the probabilities P k(t)).

14
• We observe that the rate at which probability “ flows” into
this state at time t is given by:

Flow rate into Sk : λ k − 1 Pk − 1 (t ) + µ k + 1 Pk + 1 (t )

Flow rate out of Sk: (λ k + µ k ) P k (t )

15
• Clearly the difference between these two is the effective
probability flow rate into this state, that is,

dPk (t )
= λk −1Pk −1 (t ) + µk +1Pk +1 (t ) − (λk + µk ) Pk (t )
dt

16
C- Birth -Death in Equilibrium

0 = − (λ k + µ k ) Pk + λ k −1 Pk −1 + µ k +1 Pk +1 k≥ 1

0 = − λ 0 P0 + µ1 P1 k=0

Flow rate into Sk λk −1 Pk −1 + µk +1 Pk +1

Flow rate out Sk : (λk + µk ) Pk
* In equilibrium these two most be the same:

λ k −1 Pk −1 + µ k +1 Pk +1 = (λ k + µ k ) Pk
17
Steady – State probability: Equilibrium
Probability (General Solution):

k −1
λi
Pk = P0 ∏ k = 0, 1, 2,… (1)
i = 0 µ i +1

P0 = 1
∞ k −1 λ
1 + ∑∏ i (2)
k =1 i = 0 µ i +1

18
 Served Particle Queuing
System

C S
Particles Queue S
C Service
C S Facility
C C C C CCC S
C

Served Particle

“ An elementary queuing system (each particle is

indicated by C and each server by S)”

19
Queuing System Characteristics:
• Interarrival time distribution
• Service time distribution
• # of servers (Waiting line/line configuration)
• Population Size
• Queue Size (Storage Capacity)
• Queuing Discipline

20
 a.Single server queue b- Multiple severs single queue

. . . .

.
.
.
.
.
.

21
 c-Multiple severs with multiple queues d-Tandem queue

. ..... .
. . .
. . .
.
.

22
 Thank You

23