BIRTH-AND-DEATH PROCESSES
and serve as reasonable approximations to
GUVENC DEGIRMENCI
Department of Industrial
Engineering, University
of Pittsburgh,
Pittsburgh,
Pennsylvania
INTRODUCTION
A birth-and-death (BD) process is a
continuous-time Markov chain (CTMC),
{X(t) : t ≥ 0}, defined on the countable state
space S = {0, 1, 2, . . .} and whose transitions
are restricted to only its nearest neighbors.
In other words, if at any time the process is
in state i ∈ S \ {0}, then in an infinitesimal
time interval, it will next transition to
either state i + 1 or to state i − 1. If the
current state is 0, then it can only transition
to state 1. BD processes have been used
to model population dynamics, queueing
systems, inventory systems, computer and
communications networks, and biological
systems, to name only a few. For example,
a BD process can be used to describe the
temporal and stochastic evolution of the
population of a geographical region when
the birth and death rates of the region can
be estimated from historical data. In such
models, the random variable X(t) represents
the population of the region at time t. In a
queueing context, the BD process can be used
to describe the total number of customers in
a stochastic service system (i.e., the number
of customers currently in service and those
waiting to receive service) when customers
arrive according to a Poisson process and
bring an exponentially distributed service
requirement.
The BD process can be analyzed in a tran-
sient sense (for any finite t such that t ≥ 0),
or in an asymptotic sense (as t → ∞). The
asymptotic behavior is often used to highlight
the salient features of the system’s dynam-
ics. Moreover, asymptotic results are often
Wiley Encyclopedia of Operations Research and Management Science, edited by James J. Cochran
Copyright © 2010 John Wiley & Sons, Inc.
1
mathematically tractable, easier to obtain,
the transient behavior when t is large. The
ergodic theorem for CTMCs gives the condi-
tions under which the limiting distribution
of a BD process coincides with the long-run
fraction of time that the process occupies
any state in S. For instance, in a high-speed
communications network, it is important to
know the fraction of time each node in the
network is in a congested state in order to
characterize important delay and congestion
measures. When the joint process of buffer
content at all the nodes can be modeled as
a multivariate BD process, the distribution
can be obtained in a simple, closed form. The
limiting behavior can also be used to design
new systems that satisfy quality-of-service
guarantees.
In the remainder of this article, we
formally define the standard BD process,
review some basic facts about its transient
and asymptotic behavior, and discuss some
important special cases of the BD process.
In addition, we highlight two applications in
queueing theory, discuss some extensions,
and provide guidance for further reading.
FORMAL MODEL DESCRIPTION
A BD process, {X(t) : t ≥ 0}, is a CTMC on the
countable state space S = {0, 1, 2, . . .} whose
distinguishing characteristic is that it tran-
sitions only to its nearest neighbors. Specif-
ically, from any state i ∈ S\{0}, the process
may transition to state i − 1, or to state i + 1;
however, it may not transition to any state
i ± k where k ≥ 2. If i = 0, it may only tran-
sition to i + 1. Intuitively, one may think of
a BD process as representing the evolution
of a population process, and the state, X(t),
is the population size at time t. An increase
in the population size (by unit increment)
represents a ‘‘birth’’ and a decrease (by unit
decrement) represents a ‘‘death.’’ If the birth
and/or death rates depend on the current pop-
ulation, then the BD process is called state
dependent. In other words, whenever X(t) = i,
2 BIRTH-AND-DEATH PROCESSES

the birth rate is a positive number λi , and the The infinitesimal generator matrix of the
death rate is μi , i ∈ S. BD process, denoted by Q, is given by
For any s, t ≥ 0 and i, j ∈ S, the transi-
tion probability functions of a BD process are Q=
⎡ ⎤
given by −λ0 λ0 0 0 0 ···
⎢μ1 −(λ1 + μ1 ) λ1 0 0 · · ·⎥
⎢ ⎥
⎢0 μ2 −(λ2 + μ2 ) λ2 0 · · ·⎥
⎢ ⎥
⎢ .. ⎥ , (1)
pij (s, t) ≡ P(X(t + s) = j|X(s) = i), ⎢ ⎥
⎢0 0 μ3 −(λ3 + μ3 ) λ3 .⎥
⎣ ⎦
.. .. .. .. .. ..
. . . . . .
and the BD process is called time-
homogeneous if pij (s, t) = pij (0, t) =: pij (t) which is seen to possess a tridiagonal
for any s ≥ 0. For i > 0, and a small interval structure. This structure provides some
of length t, the transition functions of computational advantages, particularly
the time-homogeneous version satisfy the when evaluating the asymptotic behavior
following equations: of the process (as t → ∞). The transition
functions, pij (t), can be shown to satisfy the
Kolmogorov backward equations, a set of
⎧
⎪
⎪λi t + o(t), j = i + 1, ordinary differential equations given by
⎪
⎨μ t + o(t),
i j = i − 1, ⎧
pij (t) = ⎪λ0 p1j (t) − pij (t) , i = 0,
⎪
⎪
⎪1 − (λi + μi ) t + o(t), j = i, dpij (t) ⎨
⎩ = λi pi+1,j (t) + μi pi−1,j (t)
o(t), otherwise, dt ⎪
⎩
− (λi + μi ) pij (t), i > 0.

where o(t)/t → 0 as t → 0. These probabil- Suppose we let P(t) = [pij (t)], i, j ∈ S,

ities are interpreted as follows. Given that
the current state is i (i > 0), in an infinites-
imal time interval, the next event is either
a birth with probability λi t + o(t) or a death
with probability μi t + o(t), and the popula-
tion size remains the same with probability
1 − (λi + μi ) t + o(t). The probability that a
‘‘jump’’ magnitude exceeds unity is of order
o(t). Practically speaking, in a small interval
of time, multiple births, multiple deaths, and
simultaneous births and deaths do not occur.
Whenever i = 0, the process next transitions
to state 1 after an exponentially distributed
amount of time with mean 1/λ0 . It is not hard
to see that the infinitesimal transition rates
of the process are given by the birth rates {λi }
and the death rate {μi }. The transition rate
diagram of this CTMC is depicted in Fig. 1.
denote the matrix of transition functions for
any t ≥ 0. Obtaining P(t) for a generic CTMC
with finite state space S is not too difficult.
The Kolmogorov backward equations can
be written in a convenient matrix form as
follows:
dP(t)
= Q P(t).
dt
This (matrix) differential equation has the
obvious solution
 
P(t) = P(0) exp Q t , t ≥ 0,
where exp(A) denotes matrix exponentiation
of the matrix A, and P(0) = I, the iden-
tity matrix. Hence, P(t) can be obtained by
employing any number of numerical rou-
tines for performing matrix exponentiation.

l0 l1 l2 l3

0 1 2 3
Figure 1. Transition rate diagram of a
standard birth-and-death process. m1 m2 m3 m4

However, because the BD process is defined
on S = {0, 1, 2, . . .}, solving for the transition
functions is generally challenging except in a
few special cases. For instance, if {X(t) : t ≥ 0}
is a homogeneous Poisson process with λi = λ
and μi = 0 for all i ∈ S, then the Kolmogorov
backward equations can be solved exactly
using standard Laplace transform techniques
(cf. Kulkarni [1]). For a general BD process
(with state-dependent births and deaths), the
method of continued fractions has been suc-
cessfully applied to analyze the transient
behavior. Three representative examples of
the use of continued fractions in analyzing
BD processes are given in Parthasarathy
[2–4]. Uniformization methods can also be
used, particularly if the state space can be
truncated at a finite level [1,5].
Some Special Cases
It is easy to see that some well-known
stochastic processes are special cases of the
standard BD process. Specifically, if μi = 0
for all i ∈ S, then {X(t) : t ≥ 0} is called a pure
birth process. In case μi = 0 and λi = λ for all
i ∈ S, the BD process is a homogeneous Pois-
son process (see Poisson Process and its
Generalizations). A prototypical example
of the pure birth process is cell reproduction
where X(t) is the number of cells living in a
colony at time t. Each cell in the colony acts
independent of the others and splits into
two cells after an exponentially distributed
amount of time with parameter λ. Therefore,
the birth rates are λi = iλ, i ∈ S, since each
of the i members gives birth with the same
exponential rate λ. In this case, {X(t) : t ≥ 0}
is called a Yule process.
A pure death process is a BD process
with λi = 0 for all i ∈ S (i.e., only deaths
occur until the process is absorbed at state
0). For example, consider a machine that
operates initially with i (i > 0) components
in parallel such that each component has
an exponentially distributed lifetime with
mean μ−1 < ∞, and suppose X(t) denotes the
number of functioning components at time t.
The component lifetimes are assumed to be
mutually independent, and the system can
continue to function as long as at least one
component is functioning. At the time of a
BIRTH-AND-DEATH PROCESSES 3
component failure, the system state is decre-
mented by unity, and the failed component
cannot be repaired. Here, the death rates are
given by μi = iμ, i ∈ S, since each component
fails at the same exponential rate μ.
LIMITING BEHAVIOR
In this section, we discuss the limiting behav-
ior of the transition probabilities pij (0, t) as
t → ∞. Initially, let us assume the existence
of a set of values, {pj : j ∈ S}, such that
lim pij (0, t) = lim P(X(t) = j|X(0) = i)
t→∞ t→∞
= pj , j ∈ S, (2)
and note that pj is independent of the initial
state i. If the limit in Equation (2) exists, then
the row vector p ≡ (p0 , p1 , p2 , . . .) is called
the limiting distribution of {X(t) : t ≥ 0}. The
existence of the limit is ensured when {X(t) :
t ≥ 0} is irreducible (i.e., all states in S com-
municate) and positive recurrent (i.e., start-
ing from any state, the expected time to
return to that state is finite). The following
result from the asymptotic analysis of posi-
tive recurrent CTMCs shows how to obtain
the limiting distribution p.
Theorem 1. If {X(t) : t ≥ 0} is an irre-
ducible and positive recurrent CTMC, then
its limiting distribution p exists and is the
unique positive solution to the system of
equations

pQ = 0; pj = 1, (3)
j∈S
where 0 denotes the zero vector.
Whenever these probabilities exist, the BD
process is said to be ergodic, and the limit
in Equation (2) has two important interpre-
tations. First, by definition, pj is the limiting
probability that the process {X(t) : t ≥ 0} is
in state j. Second, pj can be interpreted as
the long-run proportion of time that the pro-
cess spends in state j. For the BD process,
the matrix Q takes the form of Equation (1).
4 BIRTH-AND-DEATH PROCESSES

Solving the system of equations (3) for the service or in the waiting area) and note that
BD process, it is not difficult to see that {X(t) : t ≥ 0} is a BD process with constant
k−1  birth and death rates λj = λ for j ≥ 0 and
 λi μj = μ for j ≥ 1, respectively. The ratio of
pk = p0 , k ≥ 1. (4) the arrival rate to the service rate is the
μi+1
i=0 traffic intensity given by ρ = λ/μ. Then using
Equations (4) and (5), we obtain
Applying the normalization condition,
 ∞  k
−1
∞
  λ
pk = 1, p0 = 1 + .
μ
k=0 k=1

the probability p0 is obtained by The infinite series converges if and only if

ρ < 1; therefore, the system is stable if and
 ∞ k−1
−1
  λi only if λ < μ. In such a case, the limiting
p0 = 1 + . (5) probabilities are given by
μi+1
k=1 i=0
 k
λ
Equation (4) shows that each of the prob- pk = p0 = (1 − ρ) ρ k , k ≥ 1.
abilities depends explicitly on the limiting μ
probability p0 that the process is in state 0.
Hence, the limiting distribution exists if and The M/M/s Queue
only if p0 > 0, that is, if The M/M/s is similar to the M/M/1 queue
∞ except that it has s (s ≥ 1) independent and
 λ0 λ1 · · · λk−1
< ∞. identical servers (see The M/M/s Queue).
μ1 μ2 · · · μk Since there are s servers, the death rates of
k=1
this BD process are given by
Hence, the ergodicity of the process depends

explicitly on the convergence of the above jμ, if j < s,
infinite series. Should the series diverge to ( + μj =
sμ, if j ≥ s.
∞), then p0 = 0 and a limiting distribution
does not exist.
The traffic intensity for this system is ρ =
λ/sμ. Using Equations (4) and (5), the limit-
APPLICATIONS IN QUEUEING THEORY ing probabilities are given by
⎧ k
BD processes are commonly used to model ⎪
⎨p0 (sρ) , 0 ≤ k ≤ s,
Markovian queueing systems. Here, we dis- k!
pk =
cuss the application of BD processes in two of ⎪
⎩ ρ k ss
p0 , k ≥ s.
the most basic queueing systems. s!

The M/M/1 Queue and

In the M/M/1 queueing system, customers  s−1 −1

 (sρ)k (sρ)s 1
arrive according to a Poisson process with p0 = + .
rate λ, and each customer brings a service k! s! 1 − ρ
k=0
requirement that is exponentially distributed
with parameter μ. The system has only a Therefore, the limiting distribution of the
single server for processing customers, and it number of customers in the system exists
has an infinite waiting room for customers to if ρ < 1, that is, if λ < sμ. This means that
wait for service (see The M/M/1 Queue for the maximum service rate must exceed the
further details). Let X(t) denote the number arrival rate of customers for the system to
of customers in the system at time t (i.e., in remain stable.
 BIRTH-AND-DEATH PROCESSES 5

EXTENSIONS AND FURTHER READING process are given by

An important extension of the BD pro- P(X(s + t) = v + uek − ωel |X(s) = v)

cess is the quasi-birth-and-death (QBD) ⎧
⎪
⎪λk (v) t + o(t), u = 1, ω = 0,
process [6–8]. The QBD is a bivari- ⎪
⎪
⎪
⎪μ (v) t + o(t), u = 0, ω = 1,
ate Markov process on the state space ⎪
⎪
l
⎨γ (v) t + o(t), u = ω = 1,
S = {(i, j) : i ≥ 0, 1 ≤ j ≤ Mi }, (0 < Mi < ∞), =
lk
 
where i is the level of the process and j is ⎪
⎪1− λ (v)+ μ (v)
⎪  
⎪ k k l l
referred to as the phase. Its block tridiagonal ⎪
⎪+ k l γkl (v) t + o(t), u = ω = 0,
⎪
⎪
infinitesimal generator matrix is given by ⎩
o(t), otherwise,

⎡ ⎤ where o(t)/t → 0 as t → 0 and ek is the kth

A(0)
1 A(0)
0 0 0 0 ...
⎢ ⎥ unit vector. The transition from state v to
⎢ A(1)
2 A(1)
1 A(1)
0 0 0 ... ⎥ v + ek may be described as an arrival to the
⎢ ⎥
Q=⎢ 0 A(2) A(2) A(2) 0 ... ⎥. kth colony, the transition from v to v − ek as
⎢ 2 1 0 ⎥
⎢ 0 0 A(3) A(3) A(3) ... ⎥ a departure from k, and the transition from
⎣ 2 1 0 ⎦
.. .. .. .. .. .. v to v + ek − el as a transfer from l to k,
. . . . . .
l = k (assuming m ≥ 2). For certain classes
of Markov population processes, Kingman
The positive integer Mi is the number of [10] obtained the equilibrium distributions
phases at the ith level. The QBD process as t → ∞. The reader is referred to McNeil
is called level independent if, for i ≥ 0, and Schach [11], Barbour [12], and Klebaner
Mi = M, A(i) (i) [13] for further details on this subject.
0 = A0 and for i ≥ 1, A1 = A1 and
(i)
A2 = A2 (see Level-Independent Quasi- One of the earliest descriptions of the BD
Birth-and-Death Processes). Otherwise, process, its practical implications, and the
the QBD process is called level dependent limiting distribution were provided by Feller
(see Level-Dependent Quasi-Birth-and- [14]. Classical textbooks that provide further
Death Processes). The analysis of QBDs details include Ross [15], Kulkarni [1], and
is facilitated by the matrix-geometric Asmussen [16]. Properties of the BD pro-
cess and its transient analysis are described
method, developed primarily by Neuts [6].
in Karlin and McGregor [17–19] and more
By employing matrix-geometric methods,
recently by Jouini and Dallery [20]. The BD
many researchers have addressed the
process has been extended to include more
transient and asymptotic analysis of (level-
complex dynamics. For instance, Kendall [21]
independent) QBD processes, as well as their
developed the non-Markovian birth process.
many and varied applications. A method for
Age-dependent BD processes are analyzed by
computing the stationary distribution in the
Waugh [22] and Weiner [23]. Griffiths [24,25]
level-dependent case is given in Bright and
analyzed bivariate and multivariate BD pro-
Taylor [9].
cesses and discussed transient properties.
The multivariate generalization of a BD The BD process with immigration and emi-
process is called a Markov population process gration is studied by Zheng et al. [26], Swift
[10]. Let Sm be the set of all m-dimensional [27], and Aksland [28]. Karlin and Tavar [29],
vectors v = (v1 , v2 , . . . , vm ) whose components and van Doorn and Zeifman [30] analyzed the
vk ∈ S, 1 ≤ k ≤ m are nonnegative integers. BD process with killing.
Intuitively, one may think of v as the pop-
ulation vector of a system consisting of m
colonies with vk as the population of the kth REFERENCES
colony. Let u and ω be indicator variables 1. Kulkarni VG. Modeling and analysis of
such that u = 1 (ω = 1), if there is an arrival stochastic systems. London: Chapman and
to (departure from) the kth colony. For any Hall; 1995.
s, t ≥ 0 and vk ∈ S, 1 ≤ k ≤ m, the transition 2. Parthasarathy PR. Some unusual birth-and-
probability functions of a Markov population death processes. Math Sci 2003;28:79–90.
6 BIRTH-AND-DEATH PROCESSES
3. Parthasarathy PR, Lenin RB. Birth-
and-death process (BDP) models with
applications–queueing, communication sys-
tems, chemical models, biological models: the
state-of-the-art with a time-dependent per-
spective. New York: American Scienc Press;
2004.
4. Parthasarathy PR. Exact transient solution of
a state-dependent birth-death process. J Appl
Math Stoch Anal 2006;2006:1–16.
5. Heyman DP, Sobel MJ. Volume I, Stochas-
tic models in operations research. New York:
McGraw Hill; 1982.
6. Neuts MF. Matrix-geometric solutions in
stochastic models. Baltimore (MD): Johns
Hopkins University Press; 1981.
7. Latouche G, Ramaswami V. Introduction to
matrix analytic methods in stochastic model-
ing. ASA-SIAM series on statistics and applied
probability. Philadelphia (PA): SIAM; 1999.
8. Zhang J, Coyle EJ. Transient analysis of
quasi-birth-death processes. Stoch Models
1989;5:459–496.
9. Bright LW, Taylor PG. Calculating the
equilibrium distribution in level depen-
dent quasi-birth-and-death processes. Com-
mun Stat–Stoch Models 1995;11:497–525.
10. Kingman JFC. Markov population processes.
J Appl Probab 1969;6:1–18.
11. McNeil DR, Schach S. Central limit analogues
for Markov population processes. J R Stat Soc
B 1973;35:1–23.
12. Barbour AD. Equilibrium distributions
Markov population processes. Adv Appl
Probab 1980;12:591–614.
13. Klebaner FC. Asymptotic behaviour of Markov
population processes with asymptotically
linear rate of change. J Appl Probab
1994;31:614–625.
14. Feller W. An introduction to probability theory
and its applications. New York: John Wiley &
Sons, Inc.; 1950.
15. Ross S. Stochastic processes. New York: John
Wiley & Sons, Inc.; 1996.
16. Asmussen S. Applied probability and queues.
New York: Springer; 2003.
17. Karlin S, McGregor J. The classification of
birth and death processes. Trans Am Math
Soc 1957;86:366–3400.
18. Karlin S, McGregor J. The differential
equations of birth-and-death processes and
the Stieltjes moment problem. Trans Am Math
Soc 1957;85:489–546.
19. Karlin S, McGregor J. Linear growth,
birth and death processes. J Math Mech
1958;7:643–662.
20. Jouini O, Dallery Y. Moments of first passage
times in general birth–death processes. Math
Methods Oper Res 2008;68:49–76.
21. Kendall DG. On the generalized birth-and-
death process. Ann Math Stat 1948;19:1–15.
22. O’N Waugh WA. An age-dependent birth and
death process. Biometrica 1955;42:291–306.
23. Weiner HJ. Applications of the age distribu-
tion in age dependent branching processes.
J Appl Probab 1966;3:179–201.
24. Griffiths DA. A bivariate birth-death pro-
cess which approximates to the spread of
a disease involving a vector. J Appl Probab
1972;9:65–75.
25. Griffiths DA. Multivariate birth-and-death
processes as approximations to epidemic pro-
cesses. J Appl Probab 1973;10:15–26.
26. Zheng Y, Chao X, Ji X. Transient analysis
of linear birth-death processes with immigra-
tion and emigration. Probab Eng Inf Sci Arch
2004;18:141–159.
27. Swift RJ. Transient probabilities for a simple
birth-death-immigration process under the
influence of total catastrophes. Int J Math
Math Sci 2001;25:689–692.
28. Aksland M. A birth, death and migration
process with immigration. Adv Appl Probab
1975;7:44–60.
29. Karlin S, Tavaré S. Linear birth and
death processes with killing. J Appl Probab
1982;19:477–487.
30. van Doorn EA, Zeifman AI. Extinction prob-
ability in a birth-death process with killing.
J Appl Probab 2005;42:185–198.