You are on page 1of 49

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Birth-death processes

Jorge Julvez´ University of Zaragoza

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Example: Consider cells which reproduce according to the following rules:

A cell present at time t has probability λh + o(h) of splitting in two in the interval (t, t + h)

This probability is independent of age

Events betweeen different cells are independent

> Time

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Example: Consider cells which reproduce according to the following rules:

A cell present at time t has probability λh + o(h) of splitting in two in the interval (t, t + h)

This probability is independent of age

Events betweeen different cells are independent

> Time

What is the time evolution of the system?

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Non-Probabilistic Analysis

Let n(t) =number of cells at time t

Let λ be the birth rate per single cell

Thus λn(t)∆(t) births occur in (t, t + ∆t) Then:

n(t + ∆t) = n(t) + n(t)λt

n(t + ∆t) n(t)

t

= n(t)λ dn dt = n (t) = n(t)λ

The solution of this differential equation is: n(t ) = Ke λt

If n(0) = n 0 then

n(t) = n 0 e λt

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Probabilistic Analysis Notation:

N(t) = number of cells at time t

P{N(t) = n} = P n (t)

Assumptions:

A cell present at time t has probability λh + o(h) of splitting in two in the interval (t, t + h)

The probability of more than one birth occurring in time interval (t, t + h) is o(h)

All states are transient

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Assumptions:

Probability of splitting in (t, t + h): λh + o(h)

Probability of more than one split in (t, t + h): o(h)

The probability of birth in (t, t + h) if N(t) = n is nλh + o(h). Then,

P n (t + h) = P n (t)(1 nλh o(h)) + P n1 (t)((n 1)λh + o(h))

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Assumptions:

Probability of splitting in (t, t + h): λh + o(h)

Probability of more than one split in (t, t + h): o(h)

The probability of birth in (t, t + h) if N(t) = n is nλh + o(h). Then,

P n (t + h) = P n (t)(1 nλh o(h)) + P n1 (t)((n 1)λh + o(h))

P n (t + h) P n (t) = nλhP n (t) + P n1 (t)(n 1)λh + f (h), with f (h) o(h)

P n (t + h) P n (t)

h

Let h 0,

= nλP n (t) + P n1 (t)(n 1)λ + f(h)

h

P n (t) = nλP n (t) + (n 1)λP n1 (t)

Initial condition P n 0 (0) = P{N(0) = n 0 } = 1

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Probabilities are given by a set of ordinary differential equations.

n (t) = nλP n (t) + (n 1)λP n1 (t)

P

P n 0 (0) = P{N(0) = n 0 } = 1

Solution

P n (t) =

n 0 e λn 0 t (1 e λt ) nn 0

n

1

n

n = n 0 , n 0 + 1,

where

n =

k

n! k!(n k )! .

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Solution

P n (t) =

n 1

n

n 0 e λn 0 t (1 e λt ) nn 0

n = n 0 , n 0 + 1,

Observation: The solution can be seen as a negative binomial distribution, i.e., probability of obtaining n 0 successes in n trials. Suppose p =prob. of success and q = 1 p =prob. of failure. Then, the probability that the ﬁrst (n 1) trials result in (n 0 1) successes and (n n 0 ) failures followed by success on the n th trial is:

n 1

n

n 0 p n 0 1 q nn 0 p =

n 1

n

n 0 p n 0 q nn 0 ;

n = n 0 , n 0 + 1,

If p = e λt and q = 1 e λt , both equations are the same.

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process (Yule-Furry Process)

Yule studied this process in connection with the theory of evolution, i.e., population consists of the species within a genus and creation of a new element is due to mutations.

This approach neglects the probability of species dying out and size of species.

Furry used the same model for radioactive transmutations.

Birth Processes
Birth-Death Processes
Relationship to Markov Chains
Linear Birth-Death Processes
Examples
Pure Birth Processes. Generalization
• In a Yule-Furry process, for N(t) = n the probability of a
change during (t, t + h) depends on n.
• In a Poisson process, the probability of a change during
(t, t + h) is independent of N(t).
0
1
2 i-1
i

Generalization

Assume that for N(t) = n the probability of a new change to n + 1 in (t, t + h) is λ n h + o(h).

The probability of more than one change is o(h).

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Processes. Generalization

Generalization

Assume that for N(t) = n the probability of a new change to n + 1 in (t, t + h) is λ n h + o(h).

The probability of more than one change is o(h).

Then,

P n (t + h) = P n (t)(1 λ n h) + P n1 (t)λ n1 h + o(h), n

P 0 (t + h) = P 0 (t)(1 λ 0 h) + o(h)

= 0

P n (t) =

λ n P n (t) + λ n1 P n1 (t)

P 0 (t) = λ 0 P n (t)

Equations can be solved recursively with P 0 (t) = P 0 (0)e λ 0 t

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Birth Process. Generalization

Let the initial condition be P n 0 (0) = 1.

The resulting equations are:

P n (t) = λ n P n (t) + λ n1 P n1 (t), n > n 0 P n 0 (t) = λ n 0 P n 0 (t)

Yule-Furry processes assumed λ n = nλ

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Birth-Death Processes

Linear Birth-Death Processes

Examples

Notation

Pure Birth process: If n transitions take place during (0, t), we may refer to the process as being in state E n .

Changes in the pure birth process:

E n E n+1 E n+2

Birth-Death Processes consider transitions E n E n1 as well as E n E n+1 if n 1. If n = 0, only E 0 E 1 is allowed.

0
1
2
0
1 2
1
2
3
i-1
i
i
i+1
i i+1
i
i+1 i+1
i+2

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Birth-Death Processes

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Birth-Death Processes

Assumptions If the process at time t is in E n , then during (t, t + h):

Transition E n E n+1 has probability λ n h + o(h)

Transition E n E n1 has probability µ n h + o(h)

Probability that more than 1 change occurs = o(h).

P n (t + h) = P n (t)(1 λ n h µ n h)

+ P n1 (t)(λ n1 h) + P n+1 (t)(µ n+1 h) + o(h)

Time evolution of the probabilities

P n (t) = (λ n + µ n )P n (t) + λ n1 P n1 (t) + µ n+1 P n+1 (t)

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Birth-Death Processes

For n = 0

Linear Birth-Death Processes

Examples

P 0 (t + h) = P 0 (t)(1 λ 0 h) + P 1 (t)µ 1 h + o(h)

P 0 (t) = λ 0 P 0 (t) + µ 1 P 1 (t)

If λ 0 = 0, then E 0 E 1 is impossible and E 0 is an absorbing state.

If λ 0 = 0, then P 0 (t) = µ 1 P 1 (t) 0 and hence P 0 (t) increases monotonically.

Note:

lim t P 0 (t) = P 0 () = Probability of being absorbed.

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

P 0 (t) = λ 0 P 0 (t) + µ 1 P 1 (t) P n (t) = (λ n + µ n )P n (t) + λ n1 P n1 (t) + µ n+1 P n+1 (t)

As t → ∞, P n (t) P n (limit). Hence, P 0 (t) 0 and P n (t) 0. Therefore,

0 = λ 0 P 0 + µ 1 P 1

P 1 = λ 0

µ 1

P

0

0 = (λ 1 + µ 1 )P 1 + λ 0 P 0 + µ 2 P 2

P 2 = λ 0

λ 1 P 0

µ 1 µ 2

P 3 = µ λ 1 0 µ λ 1 2 λ µ 2 2 P 0

etc

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

P 1 = λ 0 P 0 ; P 2 = λ 0 µ 2 P 0 ; P 3 = λ 0 λ 1 λ 2 2 P 0 ; P 4 =

µ

1

λ

1

µ

1

µ 1 µ 2 µ

Examples

The dependence on the initial conditions has disappeared.

After normalizing, i.e.,

n=1

P n = 1:

P 0 =

1

1 +

n=1

n1

i=0

λ

i

µ

i+1

;

P n =

n1

i=0

λ

i

µ

i+1

1 +

n=1

n1

i=0

λ

i

µ

i+1

, n 1

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

n−1
λ i
1
µ i+1
i=0
P 0 =
; P n =
, n ≥ 1
∞ n−1
∞ n−1
λ i
λ i
1 +
1 +
µ i+1
µ i+1
n=1
i=0
n=1
i=0
Ergodicity condition
P n > 0, for all n ≥ 0, i.e.,:
∞ n−1
λ i
< ∞
µ i+1
n=1
i=0

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Example. A single server system

1

0

0
1

Exp( ) Exp( )

constant arrival rate λ (Poisson arrivals)

stopping rate of service µ

(exponential distribution)

states of the system: 0 (server free), 1 (server busy)

P 0 (t) = λP 0 (t) + µP 1 (t)

P

1 (t) = λP 0 (t) µP 1 (t)

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Example. A single server system

0
1
 P 0 (t) = −λP 0 (t) + µP 1 (t) 1 0 Exp( ) Exp( ) P 1 (t) = λP 0 (t) − µP 1 (t) Given that: P 0 (t) + P 1 (t) = 1, P 0 (t) + (λ + µ)P 0 (t) = µ. P 0 (t) = λ + µ + P 0 (0) − µ P 1 (t) = λ + µ + P 1 (0) − λ + µ e −(λ+µ)t λ µ + µ e −(λ+µ)t λ λ Solution = Equilibrum distribution + Deviation from the equilibrium with exponential decay.

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Poisson Process. Probabilities

Poisson Process

Birth probability per time unit is constant λ

The population size is initially 0

0
1 2
i-1
i

All states are transient

Equations

P (t) = λP i (t) + λP i1 (t), P 0 (t) = λP 0 (t)

i

i > 0

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Poisson Process. Probabilities

Equations

P (t) = λP i (t) + λP i1 (t),

i

P 0 (t) = λP 0 (t)

i > 0

P 0 (t) = e λt

dt [e λt P i (t)] = λP i1 (t)e λt P i (t) = e λt λ t P i1 (t )e λt dt

d

0

P 1 (t) = e λt λ t e λt e λt dt = e λt (λt)

0

−λt
Recursively: P i (t) = (λt) i
e
i!
Number of births in interval (0, t) ∼ Poisson(λt).

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Death Process. Probabilities

Pure Death Process

All the individuals have the same mortality rate µ

The population size is initially n

0
1
2
n

2

3

n-1
(n-1)

n

State 0 is an absorbing state. The rest are transient.

Equations

P n (t) = nµP n (t) P (t) = (i + 1)µP i+1 (t) iµP i (t),

 i = 0, , n − 1 i

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Pure Death Process. Probabilities

Equations

P n (t) = nµP n (t)

 i = 0, , n − 1 i

P (t) = (i + 1)µP i+1 (t) iµP i (t),

P n (t) = e nµt

dt [e iµt P i (t)] = (i+1)µP i+1 (t)e iµt P i (t) = (i+1)e iµt µ t

d

0

P i+1 (t )e iµt dt

P n1 (t) = ne (n1)µt µ t

0

e nµt e (n1)µt dt = ne (n1)µt (1 e µt )

Recursively: P i (t) = n

Binomial distribution: The survival probability at time t is e µt

independent of others.

i

(e µt ) i (1 e µt ) ni

27 / 47

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relation to CTMC

Relationship to Markov Chains

Linear Birth-Death Processes

Inﬁnitesimal generator matrix:

Q =

λ 0

µ 1

0

.

.

.

.

.

.

λ

0

(λ 1 + µ 1 )

µ

2

0

.

0

λ 1

(λ 2 + µ 2 )

µ

3

.

0

λ 2

.

.

(λ 3 + µ 3 )

. .

0
1
2
0
1
2
1
2
3
i-1
i
i
i+1
i i+1
i
i+1 i+1
i+2

.

.

0

λ 3

.

.

.

.

.

.

.

.

.

.

.

0

.

.

.

.

Examples

.

.

.

.

.

.

.

.

.

.

.

.

.

 

 

Birth Processes

Birth-Death Processes

Relation to DTMC

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Embedded Markov chain of the process. For t → ∞, deﬁne:

P(E n+1 |E n ) = Prob. of transition E n E n+1

= Prob. of going to E n+1 conditional on being inE n

Deﬁne P(E n1 |E n ) similarly. Then

P(E n+1 |E n ) λ n , P(E n1 |E n ) µ n

P(E n+1 |E n ) =

λ n λ + n µ n , P(E n1 |E n ) =

µ n

λ n + µ n

The same conditional probabilities hold if it is given that a transition will take place in (t, t + h) conditional on being in E n .

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Linear Birth-Death Processes

Examples

Linear Birth-Death Process

λ n = nλ

µ n = nµ

P 0 (t) = µP 1 (t)

P n (t) = (λ + µ)nP n (t) + λ(n 1)P n1 (t) + µ(n + 1)P n+1 (t)

Steady state behavior is characterized by:

t→∞ P

lim

0 (t) = 0

Similarly as t → ∞ P n () = 0

P 1 () = 0

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Linear Birth-Death Processes

Examples

Steady state behavior is characterized by:

t→∞ P

lim

0 (t) = 0

Similarly as t → ∞ P n () = 0

P 1 () = 0

Two cases can happen:

If P 0 () = 1 the probability of ultimate extinction is 1.

If P 0 () = P 0 < 1, the relations P 1 = P 2 = P 3

= 0

imply with probability 1 P 0 that the population can

increase without bounds.

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Mean of a Linear Birth-Death Process

P n (t) = (λ + µ)nP n (t) + λ(n 1)P n1 (t) + µ(n + 1)P n+1 (t)

Deﬁne Mean by M (t ) = n=1 nP n (t)

and consider M (t) =

n=1

nP n (t), then:

M (t) = (λ + µ)

n=1

n 2 P n (t) + λ

n=1

(n

1)nP n1 (t)

+ µ

n=1

(n

+ 1)nP n+1 (t)

Write (n 1)n = (n 1) 2 + (n 1), (n + 1)n = (n + 1) 2 (n + 1)

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Mean of a Linear Birth-Death Process

M (t) = (λ + µ)

n=1

n 2 P n (t)

+ 1) 2 P n1 (t) + µ

λ

n=1

(n

n=1

(n + 1) 2 P n+1 (t) + P 1 (t)

+

λ

n=1

1)P n1 (t) µ

nP n (t) µ

n=1

(n

M (t) = λ

n=1

n=1

(n + 1)P n+1 (t) + P 1 (t)

nP n (t) = (λ µ)M(t)

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Mean of a Linear Birth-Death Process

M(t) = n 0 e (λµ)t

If λ > µ then M(t) → ∞

If λ < µ then M(t) 0

Similarly if M 2 (t) = n=1 n 2 P n (t) one can show that:

M 2 (t) = 2(λ µ)M 2 (t) + (λ + µ)M(t)

and when λ > µ, the variance is:

n 0 e 2(λµ)t 1 e (µλ)t

λ + µ

λ µ

Birth Processes

Outline

Birth-Death Processes

Relationship to Markov Chains

 1 Birth Processes 2 Birth-Death Processes 3 Relationship to Markov Chains 4 Linear Birth-Death Processes 5 Examples

Linear Birth-Death Processes

Examples

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Linear Birth-Death Process. Example

Let X (t) be the number of bacteria in a colony at instant t. Evolution of the population is described by:

the time that each of the individuals takes for division in two (binary ﬁssion), independently of the other bacteria

the life time of each bacterium (also independent)

Assume that:

Time for division is exponentially dist. (rate λ)

Life time is also exponentially dist. (rate µ)

M(t) = n 0 e (λµ)t

If λ > µ then the population tends to inﬁnity

If λ < µ then the population tends to 0

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

A queueing system

Linear Birth-Death Processes

s servers
µ
K places
µ
λ
µ
µ

s servers

K waiting places

λ arrival rate (Poisson)

µ Exp(µ) holding time

(expectation 1)

Examples

Is it a birth-death process?

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

A queueing system

s servers
µ
K places
µ
λ
µ
µ

s servers

K waiting places

λ arrival rate (Poisson)

µ Exp(µ) holding time

(expectation 1)

Examples

Let “N =number of customers in the system” be the state variable.

N determines uniquely the number of customers in service and waiting room.

After each arrival and departure the remaining service times of the customers in service are Exp(µ) distributed (memoryless).

0
1
2
3
4
5
6
7
8
9
2
3
4

40 / 47

Birth Processes

Birth-Death Processes

Relationship to Markov Chains

Linear Birth-Death Processes

Examples

Call blocking in an ATM network

An ATM network offers calls of two different types.

R 1 = 1Mbps λ 1 = arrival rate µ 1 = mean holding time

R 2 = 2Mbps λ 2 = arrival rate µ 2 = mean holding time