Applied Probability Trust (18 August 2005

)
EOQ ANALYSIS UNDER STOCHASTIC
PRODUCTION AND DEMAND RATES
VIDYADHAR KULKARNI,

UNC-Chapel Hill
KEQI YAN,
∗∗
UNC-Chapel Hill
Abstract
In this paper we study a type of production-inventory system in which the
production and demand rates are modulated by a background state process
modeled as a finite state Continuous Time Markov Chain (CTMC). When the
production rate exceeds the demand rate, the inventory level increases, and
when the demand rate exceeds the production rate, it decreases. When the
inventory level reaches zero, an order is placed from an external supplier, and
it arrives instantaneously. We model this system as a bivariate Markovian
stochastic process and derive the limiting distribution of the inventory level.
Assuming linear holding costs and fixed ordering costs, we show that the
classical deterministic Economic Order Quantity (EOQ) policy minimizes the
long-run average cost if one replaces the deterministic demand rate by the
expected demand - production rate in steady state. Finally, we extend the
model to allow backlogging.
Keywords: inventory theory, CTMC, uniform distribution, optimal ordering
policy, EOQ
AMS 2000 Subject Classification: Primary 90B05
Secondary 60J25
1. Introduction
In this paper we study a production-inventory model operating in a stochastic
environment that is modulated by a finite state CTMC. A production rate and a
demand rate are associated with each state of the CTMC. The inventory on hand thus
fluctuates according to the state of the CTMC. Once the inventory level drops to 0 a

Postal address: Department of Statistics and Operations Research, University of North Carolina,
Chapel Hill, NC, 27599, USA
1
2 V. Kulkarni and K. Yan
replenishment order of size q is placed. We assume that the lead time is zero, i.e., the
replenishment order is delivered instantaneously, thus the inventory level jumps from
0 to q instantaneously. Figure 1 illustrates a sample path of the inventory level.
Figure 1: A sample path of the inventory level.
This model reflects situations in which the production and demand rates undergo
recurring changes in a stochastic fashion. For instance, the demand rates can change
seasonally; or, in a machine shop the production rate can change according to the
number of working machines, etc. We assume the order size is q regardless of the state
of the CTMC in which the inventory level hits zero. This is an appropriate model
when we can base our inventory replenishment decisions only on the inventory level
and not on the state of the CTMC. This may be because knowledge of the state of the
background CTMC is unavailable, or to simplify the ordering policies. We shall study
the ordering policies based on the state of the CTMC in a subsequent paper.
The objective is to find the replenishment order size q that minimizes the long-run
average cost. The total cost includes costs to hold products in inventory, to purchase
and to produce. There is also a fixed set-up cost every time an order is placed with
an external supplier. To begin with, we assume backlogging is not allowed. (We treat
the backlogging case in Section 6.) Since we assume zero lead time, it is optimal to
place an order only when the inventory reaches zero. In this paper, we establish the
stochastic EOQ theorem that shows the standard deterministic EOQ formula remains
optimal if we replace deterministic demand rate by the expected net demand rate in
steady state.
In the literature, a large variety of inventory models is studied, although many of
them are deterministic [13]. For stochastic models, fluid models are widely used as
one type of approximations [9]. There are several papers concerning fluid models when
EOQ Analysis under Stochastic Production and Demand Rates 3
the production and demand rates depend on the inventory level [1]. As for the cases
when the production and demand rates are determined by the system environmental
state, Berman, Stadje, and Perry recently studied an EOQ model with a two-state
random environment [2]. They consider order sizes that depend on the state of the
environmental state and derived the EOQ policy to optimize the system revenue/cost.
In the general n-state systems, Browne and Zipkin studied a model with continuous
demand driven by a Markov process [3], which can be regarded as a special case of
the model in this paper. Similarly, the clearing processes [11] can be regarded as the
reverse of the inventory level process in a special case when the production rate is
always less than or equal to the demand rate. In that case we show that the inventory
level in steady state is uniformly distributed. However, the stationary distribution of
the content in a clearing system has been proved to be uniform in [4] only under certain
conditions. These conditions are too restrictive and are not satisfied by our model. In
[11] and [12] the authors show that the limiting distribution of the content in a clearing
system is almost never uniform. In this context our result about asymptotically uniform
distribution is even more unexpected.
The rest of this paper is organized as follows. In section 2, we describe the model
mathematically. In Section 3 we derive a system of differential equations for the joint
distribution function of the inventory level and environmental state in steady state and
then solve the equations. We give explicit expressions for two special examples: one
is when the production rate is always less than or equal to the demand rate for every
state. The other is a two-state model and we consider two cases: when the production
rate is less than or equal to the demand rate, or not. In section 4 we compute the
long-run average cost. In section 5, we present the optimal order size q

that minimizes
this cost. We show the optimality of a stochastic version of the classical EOQ policy.
Furthermore, in section 6 we extend the results to a more general case that allows
backorders. We show that all the results for the limiting behaviors and expected costs
hold in this new model, and the optimal policy is equivalent to the classical backlogging
EOQ in deterministic models under certain conditions.
4 V. Kulkarni and K. Yan
2. The model
Consider a production-inventory system that is modulated by a stochastic process
{Z(t), t ≥ 0} on state space S = {1, 2, ..., n}. We assume that {Z(t), t ≥ 0} is an
irreducible CTMC on {1, 2, ..., n} with rate matrix Q = [q
ij
]. When Z(t) is in state i,
the production occurs continuously at rate r
i
, and there is a demand at rate d
i
. The
net production rate is thus R
i
= r
i
−d
i
. Note that R
i
may be negative or positive. Let
X(t) be the inventory level at time t. Then as long as Z(t) = i, {X(t), t ≥ 0} changes
at rate R
i
. When X(t) reaches zero we place an order of size q > 0 from an external
supplier who delivers it instantaneously.
Let
π = [π
1
, π
2
, · · · , π
n
] (2.1)
be the limiting distribution of the CTMC, i.e., it is the unique solution to
πQ = 0, π · e = 1,
where e = [1, ..., 1]
t
is an n-dimensional column vector of ones. The system is stable if
the expected net input rate
n
¸
i=0
π
i
R
i
< 0. Let R = diag(R
1
, ..., R
n
). Then the stability
condition can be written in matrix form as follows
πRe < 0. (2.2)
We assume that this stability condition holds for the rest of this paper.
Next we consider costs to operate the system. The total cost consists of three parts:
holding cost, ordering cost, and production cost. We need the following notation to
describe the costs in subsequent sections:
h: cost to hold one item in inventory for one unit of time;
k: fixed set-up cost whenever an order is placed;
p
1
: cost to purchase one item from the external supplier;
p
2
: cost to produce one item.
We are interested in computing the optimal order size q

that minimizes the long-run
total cost per unit of time. We first need to compute the complementary cumulative
distribution function of the inventory level in steady state. We do this in the next
section.
EOQ Analysis under Stochastic Production and Demand Rates 5
3. Limiting behavior of the inventory level process
In this section we analyze the limiting distribution of the inventory level by a system
of differential equations and solve it with a group of boundary conditions.
3.1. Differential equations
Denote
G
j
(t, x) := P{X(t) > x, Z(t) = j}, x ≥ 0, t ≥ 0, j ∈ S.
Assume the stability condition (2.2) thus the following limit exists:
G
j
(x) := lim
t→∞
P{X(t) > x, Z(t) = j}, x ≥ 0, t ≥ 0, j ∈ S.
In this section we show how to compute G
j
(x). The following theorem gives the
differential equations satisfied by
G(x) := (G
1
(x), ..., G
n
(x)) .
We use the notation
G

(x) :=

dG
1
(x)
dx
, ...,
dG
n
(x)
dx

.
Theorem 3.1. The limiting distribution G(x) satisfies
G

(x)R = G(x)Q + β, x ≤ q, (3.1a)
G

(x)R = G(x)Q, x > q, (3.1b)
where the row vector β is given by β := G

(0)R. The boundary conditions are given by
G
j
(q
+
) = G
j
(q

), ∀j : R
j
= 0 (3.2a)
G

j
(0) = 0, ∀j : R
j
> 0, (3.2b)
G(0)e = 1. (3.2c)
Proof. The differential equations follow from the standard derivation of Chapman
Kolmogorov equations for Markov processes, and hence we omit the details. See [6].
The boundary condition (3.2b) holds because 1/G

j
(0) can be seen to be the expected
time between two consecutive visits by the {(X(t), Z(t)), t ≥ 0} process to the state
(0, j). If R
j
> 0, this mean time is infinity. Hence G

j
(0) = 0 when R
j
> 0.
6 V. Kulkarni and K. Yan
The boundary condition G
j
(q
+
) = G
j
(q

) for all state j with R
j
= 0 is obvious
from the fact that there is no probability mass at (q, j) unless R
j
= 0. If R
j
= 0, it is
easy to show that the probability mass P{X = q, Z = j} satisfies
P{X = q, Z = j}
¸
k=j
q
jk
=
¸
i:Ri<0
G

i
(0)R
i
.
Thus the boundary condition (3.2a) does not hold for state j if R
j
= 0.
3.2. Solution to the differential equations
In this section we derive the solution to the differential equations given in Theorem
3.1. First consider the homogeneous equations G

(x)R = G(x)Q. Let (λ, φ) be an
(eigenvalue, eigenvector) pair that solves
φQ = λφR. (3.3)
Let
m = |{i : R
i
= 0}|.
Then it is known that there are exactly m pairs (λ
i
, φ
i
), 1 ≤ i ≤ m, that satisfy
Equation (3.3). Assume that they are distinct. Exactly one of the eigenvalues is zero,
and the eigenvector corresponding to this eigenvalue is π, the stationary vector of Q
[7]. Assume that λ
1
= 0 and φ
1
= π.
We need the following matrix notation
Λ := diag(λ
1
, λ
2
, ..., λ
m
), (3.4)
and
Φ :=

φ
T
1
, φ
T
2
, ..., φ
T
m

T
. (3.5)
Lemma 3.1. The general solution to the homogeneous equations G

(x)R = G(x)Q
is given by G(x) = ce
Λx
Φ, where c = (c
1
, c
2
, ..., c
m
) is a constant row vector to be
determined by boundary conditions.
Proof. See [7].
Now take into consideration the non-homogeneous equations G

(x)R = G(x)Q+β.
We have the following theorem.
EOQ Analysis under Stochastic Production and Demand Rates 7
Theorem 3.2. The solution to the differential equations in Theorem 3.1 is given by
G(x) = ce
Λx
Φ + sxπ + d, if x ≤ q,
G(x) = ae
Λx
Φ, if x > q,
where the m dimensional row vectors a, c, the n dimensional row vectors d and the
scalar s are the unique solution to the following system of linear equations:
cΛΦR + sπR + dQ = 0, (3.6a)
(ae
Λq
Φ −ce
Λq
Φ −sqπ −d)I
R=0
= 0, (3.6b)
a
i
= 0, ∀i : λ
i
≥ 0, (3.6c)
m
¸
i=0
c
i
λ
i
φ
ij
+ sπ
j
= 0, ∀j : R
j
> 0, (3.6d)
(cΦ + d)e = 1, (3.6e)
where a
i
(c
i
) is the i-th entry of the vector a (c), and I
R=0
is the modified identity
matrix that has 1 as its j-th diagonal entry if R
j
= 0, and 0 otherwise.
Proof. According to Lemma 3.1, the homogenous equations (3.1b) have solutions of
form G(x) = ce
Λx
Φ. It can be shown that the nonhomogeneous equations (3.1a) have
solutions of form
G(x) = ce
Λx
Φ + sxπ + d (3.7)
if and only if sxπ +d is a particular solution to (3.1a). Thus, using sxπ +d into (3.1a),
we get
sπR = sxπQ + dQ + β
= dQ + β.
(3.8)
The last equation holds because πQ = 0.
Since
β = G

(0)R
= (cΛΦ + sπ)R,
using β into (3.8) we obtain
sπR = dQ + (cΛΦ + sπ)R,
which can be rearranged to get Equation (3.6a).
8 V. Kulkarni and K. Yan
Suppose when x > q, G(x) has a solution of the form G(x) = ae
Λx
Φ, where a is
another constant vector.
The boundary condition in Equation (3.2a) reduces to
(ae
Λq
Φ)I
R=0
= (ce
Λq
Φ + sqπ + d)I
R=0
. (3.9)
Rearranging (3.9) we get (3.6b).
Furthermore, boundedness of G(x) as x → ∞ implies Equation (3.6c). Equation
(3.6d) and (3.6e) follow directly from boundary conditions (3.2b) and (3.2c).
The total number of unknown coefficients (a, c, s, d) is 2m + n + 1. Notice that
number of nontrivial equations in (3.6b) is m, and the sum of the number of nontrivial
equations in (3.6c) and (3.6d) is m [7]. Hence we have 2m+n+1 nontrivial equations
to determine a unique solution for the unknown coefficients.
3.3. Examples
Next we study some special cases in which the limiting distributions are interesting.
3.3.1. R ≤ 0 We consider a special case when the production rate never exceeds
the demand rate, and hence the inventory never increases between replenishments.
Without loss of generality assume that X(0) = q. Then it is clear that X(t) ∈ [0, q]
for all t ≥ 0. The next theorem gives the steady-state distribution of X(t).
Theorem 3.3. When R ≤ 0,
G(x) = (1 −
x
q
)π, 0 ≤ x ≤ q. (3.10)
Proof. This is a special case of the model in section 3. The inventory level is always
in [0, q] thus the differential equations reduce to
G

(x)R = G(x)Q + β, (3.11)
where
β = G

(0)R,
with boundary conditions:
G(q) = 0 (3.12a)
G(0)e = 1. (3.12b)
EOQ Analysis under Stochastic Production and Demand Rates 9
It is easy to verify that (3.10) is the solution to the differential equation system (3.11)
with boundary conditions (3.12a) and (3.12b).
Remark 1. Theorem 3.3 implies that in steady state, the inventory level is uniformly
distributed on [0, q], and is independent of Z. This is indeed an unusual and interesting
result. The fact that X is U(0, q) is consistent with the results in [3].
3.3.2. A two-state example Consider a machine shop with only one machine. The
production rate is r when the machine is up, and it fails after an exp(µ) amount of
time. When it is down, there is no production, and it takes exp(λ) amount of time to
fix it. The demand occurs at a constant rate d = r no matter whether the machine
is up or down. When the inventory reaches zero, an external supply of amount q is
ordered and arrives instantaneously.
This is a special case with the following parameters:
Q =

¸
−λ λ
µ −µ
¸

, R =

¸
−d 0
0 r −d
¸

.
The matrices Λ (Equation 3.4), Φ (Equation 3.5) and π (Equation 2.1) are given by
Λ =

¸
0 0
0 θ
¸

, Φ =

¸
µ λ
r −d d
¸

,
π = (π
1
, π
2
)
=

µ
λ+µ
,
λ
λ+µ

,
where
θ =
λ(d −r) + dµ
d −r
is the only nonzero eigenvalue.
The stability condition of Equation (2.2) reduces to
λ(r −d) −µd < 0.
Note that θ < 0 if the system is stable. We consider two cases.
Case 1: r > d. In this case, when the machine is up the production rate is greater
than the demand rate. Thus the inventory level hits zero only when the machine is
10 V. Kulkarni and K. Yan
down. We give explicit expressions for the limiting distributions.
G
down
(x) =

1
θ
π
1
(e
θx

1
q
e
θ(x−q)
) x > q,
π1(r−d)
qθd
e
θx
−π
1
x +
π1
q
(1 −
r−d
λ+µ

rπ2
θd
) 0 ≤ x ≤ q,
G
up
(x) =

r−d
θd
π
1
(e
θx

1
q
e
θ(x−q)
) x > q,
π1

e
θx
−π
2
x +
π1
q
(
π1
π2
+
r−d
λ+µ

rπ1
θd
) 0 ≤ x ≤ q.
Case 2: r < d. In this case the inventory level can hit zero when the machine is either
up or down. This is a special case of the model in section 3.3.1. The solution is given
by
G
down
(x) =

1 −
1
q
x

µ
λ + µ
, 0 ≤ x ≤ q,
G
up
(x) =

1 −
1
q
x

λ
λ + µ
, 0 ≤ x ≤ q.
4. Cost rate calculations
In this section we consider the costs to operate the above system and calculate the
long-run average cost per unit of time.
Let c
h
(q), c
o
(q) and c
p
(q) be the steady-state holding, ordering and production cost
rates respectively as functions of the order quantity q. The total cost rate c(q) is hence
given by
c(q) = c
h
(q) + c
o
(q) + c
p
(q). (4.1)
The next theorem shows how to compute these cost rates in terms of the limiting
distribution G(x). Let
˜
R := diag(r
1
, r
2
, ..., r
n
),
and
˜
Λ = diag(0,
1
λ
2
, · · · ,
1
λ
n
).
Theorem 4.1. The steady-state cost rates are given by
c
h
(q) = h

(c −a)
˜
Λe
Λq
Φ +
s
2
πq
2
+ (d + c
1
π)q −c
˜
ΛΦ

e,
c
o
(q) = (k + p
1
q)(cΛΦ + sπ)Re,
c
p
(q) = p
2
(cΦ + d)
˜
Re.
EOQ Analysis under Stochastic Production and Demand Rates 11
Proof. (1) Holding cost rate.
c
h
(q) =h
¸
i


x=0
G
i
(x)dx
=h
¸
q
x=0
(ce
Λx
Φ + sπx + d)dx +


x=q
ae
Λx
Φdx

e
=h

(c −a)
˜
Λe
Λq
Φ +
s
2
πq
2
+ (d + c
1
π)q −c
˜
ΛΦ

e.
(2) Ordering cost. First consider the number of jumps of the inventory level from 0 to
q during a small time interval (t, t + δ). Notice that when the {Z(t), t ≥ 0} process is
in a state i with negative R
i
and X(t) < −R
i
δ, the number of jumps is 1; otherwise,
it is zero. Thus we have
E( number of jumps in[t, t + δ]) =
¸
i
P{X(t) ≤ −R
i
δ, Z(t) = i}
=
¸
i
(G
i
(0) −G
i
(−R
i
δ)).
Hence
lim
t→∞
lim
δ→0
1
δ
E( number of jumps in[t, t + δ]) =
¸
i
lim
δ→0
Gi(0)−Gi(−Riδ)
δ
=
¸
i
R
i
G

i
(0)
= G

(0)Re.
Thus the ordering cost rate is
c
o
(q) = (k + p
1
q) lim
t→∞
lim
δ→0
1
δ
E( number of jumps in[t, t + δ])
= (k + p
1
q)G

(0)Re
= (k + p
1
q)(cΛΦ + sπ)Re.
(3) Production cost rate. In steady state, the probability that the environmental
process is in state i is given by G
i
(0). The production cost rate is p
2
r
i
when the
environmental process is in state i. Thus the production cost rate is given by
c
p
(q) =
¸
i∈S
p
2
r
i
G
i
(0)
= p
2
G(0)
˜
Re
= p
2
(cΦ + d)
˜
Re.
5. Optimal order size
In this section, we demonstrate the primary result of this paper. We use sample
path method to show that the total cost rate c(q) is a convex function of q and that the
12 V. Kulkarni and K. Yan
equivalent of the classical deterministic EOQ formula remains optimal in this stochastic
environment. In order to prove this, we decompose the {X(t), t ≥ 0} process into two
components. Let S
0
= 0, X(0) = q and S
i
be the ith order point (i ≥ 1). Define
X
1
(t) = min
Sn≤u≤t
{X(u)}, S
n
≤ t < S
n+1
and
X
2
(t) = X(t) −X
1
(t).
Figure 2 illustrates the sample paths of the original {X(t), t ≥ 0} process and the
resulting two processes {X
1
(t), t ≥ 0} and {X
2
(t), t ≥ 0}.
Figure 2: Decomposition of the X(t) process.
The following two lemmas state some important properties of these component
processes {X
1
(t), t ≥ 0} and {X
2
(t), t ≥ 0}. They are proved in appendices.
EOQ Analysis under Stochastic Production and Demand Rates 13
Lemma 5.1. The process {X
2
(t), t ≥ 0} is independent of q.
Lemma 5.2. The limiting distribution of the process {X
1
(t), t ≥ 0} is uniform over
[0, q].
Now with these two lammas, we are ready to give the main result of this section.
Theorem 5.1. Let ∆ be the expected net demand rate (i.e., demand rate -production
rate) in steady state, given by
∆ = −
¸
i
π
i
R
i
. (5.1)
Suppose ∆ > 0. Then the optimal order size q

that minimizes the total cost rate c(q)
is given by
q

=

2k∆
h
. (5.2)
Proof. From Equation 4.1 the total cost rate is given by
c(q) = c
h
(q) + c
o
(q) + c
p
(q)
= hE(X) +
k + p
1
q
E(S
i
−S
i−1
)
+ c
p
(q).
First we calculate c
h
(q).
c
h
(q) = hE(X)
= h(E(X
1
) + E(X
2
)).
According to Lemma 5.2, {X
1
(t), t ≥ 0} is uniformly distributed on [0, q] in steady
state. Thus E(X
1
) =
q
2
. Also, according to Lemma 5.1, we know that {X
2
(t), t ≥ 0} is
independent of q. Since we have assumed the stability of {X(t), t ≥ 0}, it is clear that
{X
2
(t), t ≥ 0} has a limiting distribution and it is independent of q. Hence E(X
2
) is
independent of q.
Next we calculate c
o
(q). From the results on renewal reward processes we get
c
o
(q) =
k + p
1
q
E
i
(S
i
−S
i−1
)
.
In steady state, the average net demand during a cycle time (S
i
, S
i−1
) has to be equal
to the amount of the external supply. Hence we have
E
i
(S
i
−S
i−1
)∆ = q.
14 V. Kulkarni and K. Yan
Thus
c
o
(q) =
(k + p
1
q)∆
q
=
k∆
q
+ p
1
∆.
Since c
p
(q) = p
i
¸
π
i
r
i
, it is independent of q.
Thus the total cost rate is
c(q) =
hq
2
+
k∆
q
+ C,
where C = hE(X
2
)+p
1
∆+c
p
(q) is independent of q. Clearly, C(q) is a convex function
of q, and it is minimized at q

given by 5.2.
Remark 2. The optimal order quantity q

of Equation (5.2) is the classical EOQ
formula with the deterministic demand rate replaced by the steady-state expected net
demand rate.
A machine shop example. Consider a machine shop that has n independent and
identical machines, each behaving as described in section 3.3.2. Each machine has its
own repair person. Let Z(t) be the number of working machines at time t. Thus the
CTMC {Z(t), t ≥ 0} has n+1 states, i.e., S = {0, 1, ..., n}. Suppose the demand rate is
directly proportional to the number of machines. Thus we have d
i
= n and r
i
= i · r for
all i ∈ S, where r is the production rate of one working machine. Next we investigate
the effect of the production rate increases on the optimal order size q

. Consider a
system with λ = 1, µ = 2, h = 10, k = 2, p
1
= 8 and p
2
= 5. We plot the optimal
values of q

in Figure 3 for 1 ≤ n ≤ 5 and r varying in (0, 3).
EOQ Analysis under Stochastic Production and Demand Rates 15
Figure 3: The optimal order size against production rate.
Note that for a fixed n, q

decreases with r. This makes intuitive sense because as the
production increases the net demand rate decreases. Note that q

reaches zero when
r increases to 3. This is because the system is unstable for r ≥ 3 and hence we do not
need to order from the external supplier. Finally, for a fixed r, q

increases with n,
but sublinearly. This is a consequence of the pooling effect of the production from the
n machines.
6. Backlogging systems
In the previous sections we considered a model where we place an order as soon as
the inventory level hits zero. In reality, many businesses do operate with substantial
backlogs. In this section we consider the same system as in section 2, but allow
backlogging.
Let X(t) be the net inventory level at time t. (i.e., the inventory on hand at time t -
backorders at time t). We always use any inventory on hand to fill demands; backorders
accumulate only when we run of of stock entirely. Thus if X(t) is positive, it represents
the amount of inventory on hand. If it is negative, it represents the negative of the
amount of backorders at time t. We consider a policy under which we place an order
of size q when the amount of backorders accumulates to a predetermined level l > 0.
Clearly an optimal policy should have q > l and hence the net inventory level is in
16 V. Kulkarni and K. Yan
(−l, ∞). Figure 4 illustrates a typical sample path of the {X(t), t ≥ 0} process.
Figure 4: The inventory level process when allowing backlogging.
Note that under this policy, the stability condition is the same as in (2.2).
Let
H
j
(x) = lim
t→∞
P(X(t) > x, Z(t) = j).
The next theorem shows how to compute H(x) = [H
1
(x), H
2
(x), . . . , H
n
(x)].
Theorem 6.1. Let G(x)(x ≥ 0) be as in Theorem 3.2. Thus
H(x) = G(x + l), x ≥ −l.
Proof. Follows from the fact that the sample path of the inventory level process
with backorder level l is identical to that without the backorder shifted down by l.
Now suppose it costs b to backlog one unit of demand for one unit of time. Let
c
b
(q, l), c
h
(q, l), c
o
(q, l) and c
p
(q, l) be the steady state backlogging, holding, ordering
and producing cost rates respectively as functions of the order quantity q and reorder
level l. The total cost rate c(q, l) is thus given by
c(q, l) = c
b
(q, l) + c
h
(q, l) + c
o
(q, l) + c
p
(q, l).
The next theorem shows how to compute the cost rates.
Theorem 6.2. The steady-state cost rates are given by
c
b
(q, l) = b

c
˜
Λ(I −e
Λ(−l)
)Φ −
s
2
πl
2
+ (d + c
1
π)l

e,
c
h
(q, l) = h

(c
˜
Λ −a
˜
Λ)e
Λ(q−l)
Φ −
s
2
π(q −l)
2
+ (d + c
1
π)(q −l) −c
˜
ΛΦ

e,
c
o
(q, l) = (k + p
1
q)(cΛΦ + sπ)Re,
c
p
(q, l) = p
2
(cΦ + d)Re,
EOQ Analysis under Stochastic Production and Demand Rates 17
where a, c, s and d are the coefficients in the expression of H(x) corresponding to
Theorem 3.2.
Proof. Follow the same steps in the proof of Theorem 4.1.
The next theorem gives the stochastic version of the EOQ formula with backloggings.
Theorem 6.3. Let ∆ be as in Equation (5.1), ∆ > 0. Then the optimal order size q

and reorder position l

are given by
q

=

2k(b + h)∆
hb
(6.1)
l

=

h
b + h

q

. (6.2)
Proof. Follow the same analysis as in Theorem 5.1.
In particular, when R ≤ 0, {X(t), t ≥ 0} has uniform distribution on (−l, q − l) in
steady state, and is independent of Z, thus
H(x) =

q −l −x
q

π, −l ≤ x ≤ q −l,
and the long-run average cost is
c(q, l) =
¸
(b + h)l
2
2q
+
hq
2
−hl

π −
k
q
πR −p
1
πR + p
2
π
˜
R

e.
This is consistent with the results in deterministic models [13].
7. Conclusion and future work
We have studied a type of inventory models with or without backlogging having
production and demand rates modulated by a background stochastic process. External
replenishment orders are placed at appropriate times and arrive instantaneously. In
this paper we have modeled this system as a bivariate Markovian stochastic process
and derived the limiting distribution of the inventory level. We have established a
stochastic EOQ theorem that shows the optimality of the classical EOQ policy in this
stochastic environment.
We can study three extensions of this system. In the current analysis, the order size
is not allowed to depend on the state of the CTMC when the inventory level hits zero.
18 V. Kulkarni and K. Yan
Clearly, if that information is available, it would lower the costs if the order size can
be made dependent on that information. Berman, Stadje, and Perry have studied such
a two state system [2]. However, more work on deriving the optimal scenario in more
general systems is needed.
Secondly, in this paper we have assumed zero lead times. This assumption is
reasonable when lead times are short enough to be neglected. It would be interesting
to study this system with nonzero lead times. We feel that iid exponential lead times
may lead to tractable analysis.
Clearly, the results of this paper remain valid if the background process is a semi-
Markov process with Phase-type distributions [10]. This can be shown by constructing
an appropriate larger CTMC. Since Phase-type distributions are dense in the set of all
continuous distributions on [0, ∞), it follows that the results hold for a semi-Markov
background process with continuous sojourn times. We believe that the results hold for
more general semi-Markov processes as long as the sample paths of the {X(t), t ≥ 0}
process are not periodic with probability one. Rigorous proof of this remains to be
shown.
Appendix A. Proof of lemma 5.1
Proof. Let S
+
and S

be two subsets of S defined as S
+
= {i ∈ S : R
i
≥ 0}, and
S

= {i ∈ S : R
i
< 0}. Assume that Z(0) ∈ S

and define
T
1
= min{t ≥ 0 : Z(t) ∈ S
+
}.
Regardless of the value of q, X(t) always decreases over (0, T
1
), except for possible
jumps of size q when it hits zero. Thus X
2
(t) is zero over (0, T
1
). T
1
is independent of
q and hence {X
2
(t), t ∈ [0, T
1
)} is independent of q.
Now define
T
2
= min{t > T
1
: X(t) = X(T
1
)}.
Note that T
2
is also independent of q, X
2
(T
1
) = X
2
(T
2
) = 0 and X
2
(t) > 0 for
t ∈ (T
1
, T
2
). The sample path of {X(t), t ∈ (T
1
, T
2
)} is independent of q, since X(t)
never reaches 0 for any t ∈ (T
1
, T
2
). Thus the sample path of {X
2
(t), t ∈ (T
1
, T
2
)} is
independent of q.
EOQ Analysis under Stochastic Production and Demand Rates 19
Define
T
2n+1
= min{t ≥ T
2n
: Z(t) ∈ S
+
},
and
T
2n+2
= min{t ≥ T
2n+1
: X(t) = X(T
2n+1
)}.
Since {X
2
(t), t ≥ 0} goes through these two cycles alternately over (T
2n
, T
2n+1
) and
(T
2n+1
, T
2n+2
) independently, it is clear that {X
2
(t), t ≥ 0} is independent of q.
Appendix B. Proof of lemma 5.2
Proof. First note that the sample paths of {X
1
(t), t ≥ 0} have right derivative
everywhere. Define I(t) = 0 if the right derivative of X
1
(t) is strictly negative at t,
and I(t) = 1 if the right derivative of X
1
(t) is zero at t. Now
lim
t→∞
P(X
1
(t) ≤ x)
= lim
t→∞
P(X
1
(t) ≤ x)|I(t) = 0)P(I(t) = 0) + lim
t→∞
P(X
1
(t) ≤ x)|I(t) = 1)P(I(t) = 1).
(B.1)
Next we will show that
lim
t→∞
P(X
1
(t) ≤ x|I(t) = ζ) = x/q, ζ ∈ {0, 1}. (B.2)
First we construct two new processes {Y
0
(t), t ≥ 0} and {Z
0
(t), t ≥ 0} by eliminating
the segments of the sample paths of {X
1
(t), t ≥ 0} and {Z(t), t ≥ 0} over the time
intervals (T
2n+1
, T
2n+2
] for all n ≥ 0. The sample paths of the {Y
0
(t), t ≥ 0} and
{Z
0
(t), t ≥ 0} processes corresponding to the sample paths of {X
1
(t), t ≥ 0} and
{Z(t), t ≥ 0} are shown in Figure 5. From Figure 5 we can see that {Y
0
, t ≥ 0} can be
thought of as a fluid model modulated by the stochastic process {Z
0
(t), t ≥ 0} with
state space S

. It can be seen that {Z
0
(t), t ≥ 0} is a CTMC with generator matrix
ˆ
Q = [ˆ q
ij
], (i, j ∈ S

) given by
ˆ q
ij
= q
ij
+
¸
k∈S+
q
ik
η
kj
, i, j ∈ S

,
where
η
kj
= P(Z(T
2n+2
) = j|Z(T
2n+1
) = k), k ∈ S
+
, j ∈ S

.
20 V. Kulkarni and K. Yan
Figure 5: Correspondence of the processes X(t), Z(t), X1(t), Y0(t) and Z0(t).
EOQ Analysis under Stochastic Production and Demand Rates 21
Thus the {(Y
0
(t), Z
0
(t)), t ≥ 0} process satisfies the hypothesis of Theorem 3.3. Hence
it follows that
lim
t→∞
P(Y
0
(t) ≤ x, Z
0
(t) = i) =
x
q
ˆ π
i
, (B.3)
where ˆ π
i
is the limiting distribution of the CTMC with generator matrix
ˆ
Q. However,
our construction of the Y
0
process implies that
lim
t→∞
P(Y
0
(t) ≤ x, Z
0
(t) = i) = lim
t →∞
P(X
1
(t) ≤ x|I(t) = 0).
This proves Equation (B.2) for ζ = 0.
Now for ζ = 1, we define Y
1,n
= X
1
(T
+
2n+1
) and Z
1,n
= Z(T
+
2n+1
), for n ≥ 0.
Now construct a semi-Markov process (SMP) {(Z
1
(t), Y
1
(t)), t ≥ 0} with embedded
DTMC {(Z
1,n
, Y
1,n
), n ≥ 0}, so that the n-th sojourn time of this SMP is given by
T
2n+2
−T
2n+1
. Clearly the sample path of {Y
1
(t), t ≥ 0} is identical to the one obtained
by eliminating the segments of the sample path of {X
1
(t), t ≥ 0} over the intervals
(T
2n
, T
2n+1
] for all n ≥ 0. Figure 6 illustrates the sample paths of the {Y
1
(t), t ≥ 0}
and {Z
1
(t), t ≥ 0} processes corresponding to the sample paths of {X
1
(t), t ≥ 0} and
{Z(t), t ≥ 0} processes.
Define
f(j, x)dx = lim
t→∞
P{Z(t) = j, x ≤ Y
1
(t) ≤ x + dx}.
According to the theory of SMP [6],
f(j, x)dx =
π(j, x)u(j, x)dx
¸
k∈S+

q
y=0
π(k, y)u(k, y)dy
, (B.4)
where
π(j, x)dx = lim
n→∞
P{Z
1,n
= j, x ≤ Y
1,n
≤ x + dx},
and u(j, x) is the expected sojourn time of the SMP in state (j, x). Clearly u(j, x) is
independent of x, hence we denote u(j, x) as τ
j
for all x.
Let
g(i, x)dx = lim
t→∞
P{Z(t) = i, x ≤ Y
0
(t) ≤ x + dx}, (i ∈ S

).
From Equation (B.3), we see that
g(i, x) =
ˆ π
i
q
, (i ∈ S

). (B.5)
22 V. Kulkarni and K. Yan
Figure 6: Correspondence of the processes X(t), Z(t), X1(t), Y1(t) and Z1(t).
EOQ Analysis under Stochastic Production and Demand Rates 23
Hence using Equation (B.5),
π(j, x) =
¸
i∈S−
g(i, x)q
ij
=
1
q
¸
i∈S−
ˆ π
i
q
ij
. (B.6)
Substituting Equation (B.6) into (B.4), we have
f(j, x) =
1
q
¸
i∈S−
ˆ π
i
q
ij
τ
j
¸
k∈S+
q

y=0
1
q
¸
i∈S−
ˆ π
i
q
ik
τ
k
dy
=
1
q
·
¸
i∈S−
ˆ π
i
q
ij
τ
j
¸
k∈S+
¸
i∈S−
ˆ π
i
q
ik
τ
k
.
Thus the limiting probability density function of {Y
1
(t), t ≥ 0} process is given by
f(x) =
¸
j∈S+
f(j, x) (B.7)
=
1
q
·
¸
j∈S+
¸
i∈S−
ˆ πˆ q
ij
τ
j
¸
k∈S+
¸
i∈S−
ˆ πˆ q
ik
τ
k
(B.8)
=
1
q
. (B.9)
Equation (B.9) indicates the limiting distribution of {Y
1
(t), t ≥ 0} is uniform over
[0, q]. This proves Equation (B.2) for j = 1. Hence from (B.1)
lim
t→∞
P(X
1
(t) ≤ x) =
x
q
.
This proves Lemma 5.2.
References
[1] Berman, O. and Perry, D. (2004). An EOQ model with state dependent demand rate. Eur.
J. Operat. Res. In Press.
[2] Berman, O., Stadje, W. and Perry, D. (2005). A Fluid EOQ Model with a Two-State Random
Environment.
[3] Browne, S. and Zipkin, P. (1991). Inventory Models with Continuous Stochastic Demands.
Ann. Appl. Prob. 1, 419–435.
[4] El-Taha, M. (2002). A Sample-Path Condition for the Asymptotic Uniform Distribution of
Clearing Processes. Optimization. 51, 965–975.
24 V. Kulkarni and K. Yan
[5] Kella, O. and Whitt, W. (1992). A storage model with a two state random environment.
Operat. Res. 40, 257–262.
[6] Kulkarni, V. G. (1995). Modeling and Analysis of Stochastic Systems. Chapman and Hall,
London.
[7] Kulkarni, V. G. (1997). Fluid models for single buffer systems, in: J. H. Dshalalow
(Ed.), Frontiers in Queueing, Models and Applications in Science and Engineering, Ed. J.H.
Dshalalow, CRC Press, Boca Raton, FL, 321–338.
[8] Kulkarni, V. G. and Tzenova, E. (2002). Mean first passage times in fluid queues. Operat.
Res. Lett. 30, 308–318.
[9] Mitra, D. (1988). Stochastic theory of a fluid model of producers and consumers coupled by a
buffer. Adv. Appl. Prob. 20, 646–676.
[10] Neuts, M. F. (1981). Matrix Geometric Solutions in Stochastic Models. Johns Hopkins
University Press, Baltimore, MD.
[11] Serfozo, R. and Stidham, S. (1978). Semi-stationary clearing processes. Stoch. Proc. Appl. 6,
165–178.
[12] Whitt, W. (1981). The stationary distribution of a stochastic clearing process. Operat. Res. 29,
294–308.
[13] Zipkin, P. (2000). Foundations of Inventory Management. McGraw-Hill, Boston.

2

V. Kulkarni and K. Yan

replenishment order of size q is placed. We assume that the lead time is zero, i.e., the replenishment order is delivered instantaneously, thus the inventory level jumps from 0 to q instantaneously. Figure 1 illustrates a sample path of the inventory level.

Figure 1: A sample path of the inventory level.

This model reflects situations in which the production and demand rates undergo recurring changes in a stochastic fashion. For instance, the demand rates can change seasonally; or, in a machine shop the production rate can change according to the number of working machines, etc. We assume the order size is q regardless of the state of the CTMC in which the inventory level hits zero. This is an appropriate model when we can base our inventory replenishment decisions only on the inventory level and not on the state of the CTMC. This may be because knowledge of the state of the background CTMC is unavailable, or to simplify the ordering policies. We shall study the ordering policies based on the state of the CTMC in a subsequent paper. The objective is to find the replenishment order size q that minimizes the long-run average cost. The total cost includes costs to hold products in inventory, to purchase and to produce. There is also a fixed set-up cost every time an order is placed with an external supplier. To begin with, we assume backlogging is not allowed. (We treat the backlogging case in Section 6.) Since we assume zero lead time, it is optimal to place an order only when the inventory reaches zero. In this paper, we establish the stochastic EOQ theorem that shows the standard deterministic EOQ formula remains optimal if we replace deterministic demand rate by the expected net demand rate in steady state. In the literature, a large variety of inventory models is studied, although many of them are deterministic [13]. For stochastic models, fluid models are widely used as one type of approximations [9]. There are several papers concerning fluid models when

which can be regarded as a special case of the model in this paper. In Section 3 we derive a system of differential equations for the joint distribution function of the inventory level and environmental state in steady state and then solve the equations. we present the optimal order size q ∗ that minimizes this cost. In section 4 we compute the long-run average cost. In the general n-state systems. We give explicit expressions for two special examples: one is when the production rate is always less than or equal to the demand rate for every state. Browne and Zipkin studied a model with continuous demand driven by a Markov process [3]. As for the cases when the production and demand rates are determined by the system environmental state. The rest of this paper is organized as follows. In that case we show that the inventory level in steady state is uniformly distributed. However. Berman. Stadje. . Similarly. In section 5. They consider order sizes that depend on the state of the environmental state and derived the EOQ policy to optimize the system revenue/cost. we describe the model mathematically. in section 6 we extend the results to a more general case that allows backorders. the clearing processes [11] can be regarded as the reverse of the inventory level process in a special case when the production rate is always less than or equal to the demand rate. In section 2.EOQ Analysis under Stochastic Production and Demand Rates 3 the production and demand rates depend on the inventory level [1]. and Perry recently studied an EOQ model with a two-state random environment [2]. the stationary distribution of the content in a clearing system has been proved to be uniform in [4] only under certain conditions. We show that all the results for the limiting behaviors and expected costs hold in this new model. or not. Furthermore. The other is a two-state model and we consider two cases: when the production rate is less than or equal to the demand rate. We show the optimality of a stochastic version of the classical EOQ policy. These conditions are too restrictive and are not satisfied by our model. In [11] and [12] the authors show that the limiting distribution of the content in a clearing system is almost never uniform. and the optimal policy is equivalent to the classical backlogging EOQ in deterministic models under certain conditions. In this context our result about asymptotically uniform distribution is even more unexpected.

. πn ] be the limiting distribution of the CTMC. · · · .. the production occurs continuously at rate ri .. (2. i. We assume that this stability condition holds for the rest of this paper..1) where e = [1.2) .. The model Consider a production-inventory system that is modulated by a stochastic process {Z(t). . t ≥ 0} changes at rate Ri . Rn ). p2 : cost to produce one item. Let π = [π1 . Next we consider costs to operate the system. Note that Ri may be negative or positive.. and production cost. k: fixed set-up cost whenever an order is placed. (2.4 V. The system is stable if n the expected net input rate i=0 πi Ri < 0. {X(t). ordering cost.. π2 .. When X(t) reaches zero we place an order of size q > 0 from an external supplier who delivers it instantaneously. 1]t is an n-dimensional column vector of ones.. When Z(t) is in state i. 2. it is the unique solution to πQ = 0. We are interested in computing the optimal order size q ∗ that minimizes the long-run total cost per unit of time. 2. The total cost consists of three parts: holding cost. Let R = diag(R1 . Let X(t) be the inventory level at time t. We do this in the next section. The net production rate is thus Ri = ri − di . . Then the stability condition can be written in matrix form as follows πRe < 0. t ≥ 0} is an irreducible CTMC on {1. Then as long as Z(t) = i.. We assume that {Z(t).. Kulkarni and K. p1 : cost to purchase one item from the external supplier. We need the following notation to describe the costs in subsequent sections: h: cost to hold one item in inventory for one unit of time. . We first need to compute the complementary cumulative distribution function of the inventory level in steady state.. t ≥ 0} on state space S = {1.. n} with rate matrix Q = [qij ]. and there is a demand at rate di . π · e = 1.e. . n}. Yan 2.

t ≥ 0} process to the state (0. Gj (0) = 0. x) := P {X(t) > x. . dx dx . Hence Gj (0) = 0 when Rj > 0. t ≥ 0. The following theorem gives the differential equations satisfied by G(x) := (G1 (x).2b) (3. The boundary conditions are given by Gj (q + ) = Gj (q − ). 3. this mean time is infinity. (3.2) thus the following limit exists: Gj (x) := lim P {X(t) > x. ∀j : Rj = 0 ∀j : Rj > 0. Assume the stability condition (2. (3.2c) Proof.1. The boundary condition (3. Z(t)). In this section we show how to compute Gj (x). j ∈ S. The differential equations follow from the standard derivation of Chapman Kolmogorov equations for Markov processes. Gn (x)) . and hence we omit the details. Z(t) = j}. j ∈ S.. If Rj > 0.. The limiting distribution G(x) satisfies G (x)R = G(x)Q + β. G(0)e = 1. t ≥ 0..1a) (3. . We use the notation G (x) := dG1 (x) dGn (x) . t→∞ x ≥ 0.2a) (3. . Limiting behavior of the inventory level process In this section we analyze the limiting distribution of the inventory level by a system of differential equations and solve it with a group of boundary conditions.. Differential equations Denote Gj (t.. G (x)R = G(x)Q.1. Theorem 3.2b) holds because 1/Gj (0) can be seen to be the expected time between two consecutive visits by the {(X(t). x ≤ q.1b) where the row vector β is given by β := G (0)R. Z(t) = j}.. See [6]. j). x > q. x ≥ 0.EOQ Analysis under Stochastic Production and Demand Rates 5 3.

λ2 . j) unless Rj = 0. We need the following matrix notation Λ := diag(λ1 .. Exactly one of the eigenvalues is zero.. λm ). (3..5) Lemma 3. Z = j} k=j qjk = i:Ri <0 Gi (0)Ri . φT . Z = j} satisfies P {X = q. . 1 ≤ i ≤ m. and the eigenvector corresponding to this eigenvalue is π. Assume that they are distinct. Let (λ. where c = (c1 . φ) be an (eigenvalue. the stationary vector of Q [7].. c2 ..2.1. Solution to the differential equations In this section we derive the solution to the differential equations given in Theorem 3. .2a) does not hold for state j if Rj = 0. Now take into consideration the non-homogeneous equations G (x)R = G(x)Q + β. First consider the homogeneous equations G (x)R = G(x)Q... eigenvector) pair that solves φQ = λφR. and Φ := φT . Let m = |{i : Ri = 0}|. Kulkarni and K.6 V. 3. See [7]. The general solution to the homogeneous equations G (x)R = G(x)Q is given by G(x) = ceΛx Φ. We have the following theorem.4) ..3). Assume that λ1 = 0 and φ1 = π.1. Proof. it is easy to show that the probability mass P {X = q. .. cm ) is a constant row vector to be determined by boundary conditions.3) Equation (3. φT 1 2 m T (3. Then it is known that there are exactly m pairs (λi . that satisfy (3. If Rj = 0. Thus the boundary condition (3. φi ). Yan The boundary condition Gj (q + ) = Gj (q − ) for all state j with Rj = 0 is obvious from the fact that there is no probability mass at (q. .

which can be rearranged to get Equation (3. using sxπ + d into (3. the n dimensional row vectors d and the scalar s are the unique solution to the following system of linear equations: cΛΦR + sπR + dQ = 0. we get sπR = sxπQ + dQ + β = dQ + β. if x ≤ q. According to Lemma 3.1a) have solutions of form G(x) = ceΛx Φ + sxπ + d (3. ∀j : Rj > 0. i=0 (cΦ + d)e = 1.EOQ Analysis under Stochastic Production and Demand Rates 7 Theorem 3.8) we obtain sπR = dQ + (cΛΦ + sπ)R.6e) ci λi φij + sπj = 0. and IR=0 is the modified identity matrix that has 1 as its j-th diagonal entry if Rj = 0. m (3.6a) (3. It can be shown that the nonhomogeneous equations (3.6a). if x > q. G(x) = aeΛx Φ.2. Thus. where ai (ci ) is the i-th entry of the vector a (c). The last equation holds because πQ = 0.6d) (3. using β into (3.8) . (3. c.1. (3. and 0 otherwise.1a).1b) have solutions of form G(x) = ceΛx Φ.6c) (3. the homogenous equations (3. Proof.6b) ∀i : λi ≥ 0.7) if and only if sxπ + d is a particular solution to (3. where the m dimensional row vectors a. ai = 0.1a).1 is given by G(x) = ceΛx Φ + sxπ + d. (aeΛq Φ − ceΛq Φ − sqπ − d)IR=0 = 0. Since β = G (0)R = (cΛΦ + sπ)R. The solution to the differential equations in Theorem 3.

6e) follow directly from boundary conditions (3. This is a special case of the model in section 3.1. and hence the inventory never increases between replenishments. q] thus the differential equations reduce to G (x)R = G(x)Q + β.2a) reduces to (aeΛq Φ)IR=0 = (ceΛq Φ + sqπ + d)IR=0 .12a) (3.8 V. The boundary condition in Equation (3. (3.3.6b) is m.6d) is m [7].10) (3. Notice that number of nontrivial equations in (3. d) is 2m + n + 1. with boundary conditions: G(q) = 0 G(0)e = 1.2b) and (3. The inventory level is always in [0.11) . G(x) has a solution of the form G(x) = aeΛx Φ. Rearranging (3. 3. Yan Suppose when x > q. 3. Theorem 3.6b). where a is another constant vector.2c). (3.6c). Hence we have 2m + n + 1 nontrivial equations to determine a unique solution for the unknown coefficients. where β = G (0)R. and the sum of the number of nontrivial equations in (3. Equation (3. q] for all t ≥ 0.6d) and (3. The total number of unknown coefficients (a. R ≤ 0 We consider a special case when the production rate never exceeds the demand rate. Examples Next we study some special cases in which the limiting distributions are interesting. s. boundedness of G(x) as x → ∞ implies Equation (3. G(x) = (1 − x )π. q 0 ≤ x ≤ q.9) we get (3. The next theorem gives the steady-state distribution of X(t).9) Proof.12b) (3. When R ≤ 0.3. c. Without loss of generality assume that X(0) = q.6c) and (3.3. Furthermore. Then it is clear that X(t) ∈ [0. Kulkarni and K.

1) are given by  Λ= 0 0 0 θ  . and it fails after an exp(µ) amount of time. This is a special case with the following parameters:  Q= −λ µ λ −µ  . The production rate is r when the machine is up.12a) and (3. 3. where θ= is the only nonzero eigenvalue. Thus the inventory level hits zero only when the machine is λ(d − r) + dµ d−r .3 implies that in steady state.4). The stability condition of Equation (2. π = = (π1 . The matrices Λ (Equation 3.5) and π (Equation 2. Note that θ < 0 if the system is stable. π2 ) µ λ λ+µ .11) with boundary conditions (3. Case 1: r > d. λ+µ . The fact that X is U (0.2.EOQ Analysis under Stochastic Production and Demand Rates 9 It is easy to verify that (3.10) is the solution to the differential equation system (3. Φ (Equation 3. This is indeed an unusual and interesting result. an external supply of amount q is ordered and arrives instantaneously.3. there is no production. In this case. We consider two cases. q) is consistent with the results in [3]. Theorem 3. and is independent of Z. the inventory level is uniformly distributed on [0. The demand occurs at a constant rate d = r no matter whether the machine is up or down. and it takes exp(λ) amount of time to fix it.  Φ= µ r−d λ d  .  R= −d 0 0 r−d  . When the inventory reaches zero.2) reduces to λ(r − d) − µd < 0. When it is down. A two-state example Consider a machine shop with only one machine. Remark 1. q]. when the machine is up the production rate is greater than the demand rate.12b).

Yan down.1) The next theorem shows how to compute these cost rates in terms of the limiting distribution G(x). λ+µ λ .3. rn ). Kulkarni and K. .   1 π (eθx − 1 eθ(x−q) ) x > q. Let ch (q). 2 qθ q π2 λ+µ θd Case 2: r < d. We give explicit expressions for the limiting distributions. The solution is given by Gdown (x) = Gup (x) = 1 1− x q 1 1− x q µ .10 V. and 1 1 ˜ Λ = diag(0. θ 1 q Gdown (x) =  π1 (r−d) eθx − π x + π1 (1 − r−d − rπ2 ) 0 ≤ x ≤ q. This is a special case of the model in section 3.. λ+µ 0 ≤ x ≤ q. ordering and production cost rates respectively as functions of the order quantity q. .1. · · · . . (4.1. θd 1 q Gup (x) =  π1 eθx − π x + π1 ( π1 + r−d − rπ1 ) 0 ≤ x ≤ q. 1 qθd q λ+µ θd   r−d π (eθx − 1 eθ(x−q) ) x > q. r2 .. 4.. Cost rate calculations In this section we consider the costs to operate the above system and calculate the long-run average cost per unit of time. ). In this case the inventory level can hit zero when the machine is either up or down. co (q) and cp (q) be the steady-state holding. The total cost rate c(q) is hence given by c(q) = ch (q) + co (q) + cp (q). 2 co (q) = (k + p1 q)(cΛΦ + sπ)Re. λ2 λn Theorem 4. Let ˜ R := diag(r1 . The steady-state cost rates are given by s ˜ ˜ ch (q) = h (c − a)ΛeΛq Φ + πq 2 + (d + c1 π)q − cΛΦ e. ˜ cp (q) = p2 (cΦ + d)Re. 0 ≤ x ≤ q.

t ≥ 0} process is in a state i with negative Ri and X(t) < −Ri δ. t + δ]) = = i δ→0 i lim Gi (0)−Gi (−Ri δ) δ Ri Gi (0) = G (0)Re. Optimal order size In this section. Thus we have E( number of jumps in[t. the number of jumps is 1. (3) Production cost rate. Notice that when the {Z(t). otherwise. Z(t) = i} (Gi (0) − Gi (−Ri δ)). t + δ]) = i P {X(t) ≤ −Ri δ. ∞ ch (q) =h i q x=0 Gi (x)dx ∞ =h x=0 (ceΛx Φ + sπx + d)dx + x=q aeΛx Φdx e s ˜ ˜ =h (c − a)ΛeΛq Φ + πq 2 + (d + c1 π)q − cΛΦ e. We use sample path method to show that the total cost rate c(q) is a convex function of q and that the .EOQ Analysis under Stochastic Production and Demand Rates 11 Proof. it is zero. we demonstrate the primary result of this paper. 2 (2) Ordering cost. t + δ]) δ t→∞ δ→0 = (k + p1 q)G (0)Re = (k + p1 q)(cΛΦ + sπ)Re. i = Hence lim lim 1 E( t→∞ δ→0 δ number of jumps in[t. the probability that the environmental process is in state i is given by Gi (0). Thus the ordering cost rate is co (q) = (k + p1 q) lim lim 1 E( number of jumps in[t. (1) Holding cost rate. 5. t + δ). In steady state. First consider the number of jumps of the inventory level from 0 to q during a small time interval (t. The production cost rate is p2 ri when the environmental process is in state i. Thus the production cost rate is given by cp (q) = i∈S p2 ri Gi (0) ˜ = p2 G(0)Re ˜ = p2 (cΦ + d)Re.

X(0) = q and Si be the ith order point (i ≥ 1). . t ≥ 0}. t ≥ 0} process into two components.12 V. Yan equivalent of the classical deterministic EOQ formula remains optimal in this stochastic environment. t ≥ 0} and {X2 (t). t ≥ 0} process and the resulting two processes {X1 (t). Define X1 (t) = min {X(u)}. They are proved in appendices. Kulkarni and K. t ≥ 0} and {X2 (t). Let S0 = 0. Sn ≤u≤t Sn ≤ t < Sn+1 and X2 (t) = X(t) − X1 (t). The following two lemmas state some important properties of these component processes {X1 (t). we decompose the {X(t). Figure 2: Decomposition of the X(t) process. t ≥ 0}. In order to prove this. Figure 2 illustrates the sample paths of the original {X(t).

t ≥ 0} is k + p1 q + cp (q). we are ready to give the main result of this section. Also.1. {X1 (t).1.1 the total cost rate is given by c(q) = ch (q) + co (q) + cp (q) = hE(X) + First we calculate ch (q).2. given by ∆=− i πi R i . (5.EOQ Analysis under Stochastic Production and Demand Rates 13 Lemma 5.2) Proof. q].. Since we have assumed the stability of {X(t). E(Si − Si−1 ) independent of q. Now with these two lammas.1. t ≥ 0} is independent of q.1) Suppose ∆ > 0. it is clear that {X2 (t). Theorem 5. The limiting distribution of the process {X1 (t). the average net demand during a cycle time (Si . From Equation 4. Hence we have Ei (Si − Si−1 )∆ = q. The process {X2 (t).2. ch (q) = hE(X) = h(E(X1 ) + E(X2 )). q] in steady q state. Hence E(X2 ) is independent of q. t ≥ 0} has a limiting distribution and it is independent of q. Ei (Si − Si−1 ) In steady state. demand rate -production rate) in steady state. Si−1 ) has to be equal to the amount of the external supply. From the results on renewal reward processes we get co (q) = k + p1 q . Let ∆ be the expected net demand rate (i. Thus E(X1 ) = 2 . according to Lemma 5. Lemma 5. t ≥ 0}. h (5. Then the optimal order size q ∗ that minimizes the total cost rate c(q) is given by q∗ = 2k∆ . t ≥ 0} is uniform over [0. Next we calculate co (q).e. t ≥ 0} is uniformly distributed on [0. According to Lemma 5. . we know that {X2 (t).

14 V. q Since cp (q) = pi πi ri . Consider a system with λ = 1. Each machine has its own repair person. n}.. µ = 2..2) is the classical EOQ formula with the deterministic demand rate replaced by the steady-state expected net demand rate. Consider a machine shop that has n independent and identical machines.. C(q) is a convex function of q. it is independent of q. each behaving as described in section 3. Clearly. i. Next we investigate the effect of the production rate increases on the optimal order size q ∗ . t ≥ 0} has n+1 states. Thus the CTMC {Z(t). Suppose the demand rate is directly proportional to the number of machines. and it is minimized at q ∗ given by 5. The optimal order quantity q ∗ of Equation (5.3. Kulkarni and K. Yan Thus co (q) = (k + p1 q)∆ q k∆ = + p1 ∆. p1 = 8 and p2 = 5. A machine shop example. 1. Thus the total cost rate is c(q) = hq k∆ + + C. Remark 2. Thus we have di = n and ri = i · r for all i ∈ S. . k = 2. S = {0. We plot the optimal values of q ∗ in Figure 3 for 1 ≤ n ≤ 5 and r varying in (0.2. 3)..e. . Let Z(t) be the number of working machines at time t.2. 2 q where C = hE(X2 )+p1 ∆+cp (q) is independent of q. h = 10. where r is the production rate of one working machine.

q ∗ decreases with r. This makes intuitive sense because as the production increases the net demand rate decreases. This is because the system is unstable for r ≥ 3 and hence we do not need to order from the external supplier. Backlogging systems In the previous sections we considered a model where we place an order as soon as the inventory level hits zero. Note that q ∗ reaches zero when r increases to 3. Let X(t) be the net inventory level at time t. Finally.e.EOQ Analysis under Stochastic Production and Demand Rates 15 Figure 3: The optimal order size against production rate. In this section we consider the same system as in section 2. We consider a policy under which we place an order of size q when the amount of backorders accumulates to a predetermined level l > 0. but allow backlogging. it represents the negative of the amount of backorders at time t. In reality. it represents the amount of inventory on hand. q ∗ increases with n.. many businesses do operate with substantial backlogs. This is a consequence of the pooling effect of the production from the n machines. (i. the inventory on hand at time t backorders at time t). for a fixed r. We always use any inventory on hand to fill demands. Note that for a fixed n. If it is negative. Clearly an optimal policy should have q > l and hence the net inventory level is in . Thus if X(t) is positive. but sublinearly. 6. backorders accumulate only when we run of of stock entirely.

∞). . The total cost rate c(q. The steady-state cost rates are given by s ˜ cb (q. Theorem 6. l).2. t ≥ 0} process. Follows from the fact that the sample path of the inventory level process with backorder level l is identical to that without the backorder shifted down by l. the stability condition is the same as in (2. ordering and producing cost rates respectively as functions of the order quantity q and reorder level l. l) + co (q. l) = b cΛ(I − eΛ(−l) )Φ − πl2 + (d + c1 π)l e. . l). l) be the steady state backlogging.2). . Theorem 6. Let G(x)(x ≥ 0) be as in Theorem 3. Hn (x)]. l) = p2 (cΦ + d)Re. Yan (−l. cp (q. Now suppose it costs b to backlog one unit of demand for one unit of time. Figure 4: The inventory level process when allowing backlogging. Kulkarni and K. 2 ˜ − aΛ)eΛ(q−l) Φ − s π(q − l)2 + (d + c1 π)(q − l) − cΛΦ e. Figure 4 illustrates a typical sample path of the {X(t). . l) is thus given by c(q.1. Note that under this policy. t→∞ The next theorem shows how to compute H(x) = [H1 (x). Z(t) = j). H2 (x). Thus H(x) = G(x + l). Let Hj (x) = lim P (X(t) > x. Proof. l) = (k + p1 q)(cΛΦ + sπ)Re.2. x ≥ −l. l) + cp (q. l) = cb (q. ˜ ˜ ch (q. ch (q. l). The next theorem shows how to compute the cost rates. Let cb (q. l) = h (cΛ 2 co (q. .16 V. l) + ch (q. l) and cp (q. holding. co (q.

the order size is not allowed to depend on the state of the CTMC when the inventory level hits zero. Theorem 6.EOQ Analysis under Stochastic Production and Demand Rates 17 where a.1. This is consistent with the results in deterministic models [13]. In the current analysis. when R ≤ 0. The next theorem gives the stochastic version of the EOQ formula with backloggings.1). {X(t). Conclusion and future work We have studied a type of inventory models with or without backlogging having production and demand rates modulated by a background stochastic process.2.2) Proof. . t ≥ 0} has uniform distribution on (−l. In particular. Then the optimal order size q ∗ and reorder position l∗ are given by q∗ = l∗ = 2k(b + h)∆ hb h q∗ .3. −l ≤ x ≤ q − l. Follow the same steps in the proof of Theorem 4. and is independent of Z. We have established a stochastic EOQ theorem that shows the optimality of the classical EOQ policy in this stochastic environment. We can study three extensions of this system. ∆ > 0. q − l) in steady state. 2q 2 q q−l−x q π.1. l) = hq k (b + h)l2 ˜ + − hl π − πR − p1 πR + p2 π R e. In this paper we have modeled this system as a bivariate Markovian stochastic process and derived the limiting distribution of the inventory level. c. 7. s and d are the coefficients in the expression of H(x) corresponding to Theorem 3. Proof. b+h (6. thus H(x) = and the long-run average cost is c(q. Let ∆ be as in Equation (5. Follow the same analysis as in Theorem 5.1) (6. External replenishment orders are placed at appropriate times and arrive instantaneously.

Regardless of the value of q. T1 is independent of q and hence {X2 (t). Secondly. We feel that iid exponential lead times may lead to tractable analysis. T1 ). However. T2 )} is independent of q. Since Phase-type distributions are dense in the set of all continuous distributions on [0. Let S+ and S− be two subsets of S defined as S+ = {i ∈ S : Ri ≥ 0}. T2 ). We believe that the results hold for more general semi-Markov processes as long as the sample paths of the {X(t). The sample path of {X(t). Note that T2 is also independent of q. more work on deriving the optimal scenario in more general systems is needed. it would lower the costs if the order size can be made dependent on that information. if that information is available. t ∈ (T1 . Kulkarni and K. Rigorous proof of this remains to be shown. t ≥ 0} process are not periodic with probability one. and S− = {i ∈ S : Ri < 0}. it follows that the results hold for a semi-Markov background process with continuous sojourn times. T1 )} is independent of q. and Perry have studied such a two state system [2]. since X(t) never reaches 0 for any t ∈ (T1 . Yan Clearly. T1 ). Berman. Proof of lemma 5. This assumption is reasonable when lead times are short enough to be neglected. ∞). Assume that Z(0) ∈ S− and define T1 = min{t ≥ 0 : Z(t) ∈ S+ }. Appendix A. X2 (T1 ) = X2 (T2 ) = 0 and X2 (t) > 0 for t ∈ (T1 .1 Proof. Now define T2 = min{t > T1 : X(t) = X(T1 )}. Clearly. . t ∈ (T1 . Stadje. in this paper we have assumed zero lead times. X(t) always decreases over (0. This can be shown by constructing an appropriate larger CTMC. Thus the sample path of {X2 (t).18 V. T2 )} is independent of q. the results of this paper remain valid if the background process is a semiMarkov process with Phase-type distributions [10]. except for possible jumps of size q when it hits zero. It would be interesting to study this system with nonzero lead times. Thus X2 (t) is zero over (0. T2 ). t ∈ [0.

i. (B.EOQ Analysis under Stochastic Production and Demand Rates 19 Define T2n+1 = min{t ≥ T2n : Z(t) ∈ S+ }. t ≥ 0} are shown in Figure 5. It can be seen that {Z0 (t). t ≥ 0} have right derivative everywhere. t ≥ 0} and {Z0 (t). . j ∈ S− . where ηkj = P (Z(T2n+2 ) = j|Z(T2n+1 ) = k). From Figure 5 we can see that {Y0 . Since {X2 (t). t ≥ 0} can be thought of as a fluid model modulated by the stochastic process {Z0 (t). t ≥ 0} over the time intervals (T2n+1 . Now lim P (X1 (t) ≤ x) t→∞ t→∞ t→∞ = lim P (X1 (t) ≤ x)|I(t) = 0)P (I(t) = 0) + lim P (X1 (t) ≤ x)|I(t) = 1)P (I(t) = 1). T2n+2 ] for all n ≥ 0. The sample paths of the {Y0 (t). k ∈ S+ . Appendix B. ζ ∈ {0. t ≥ 0} and {Z(t). and T2n+2 = min{t ≥ T2n+1 : X(t) = X(T2n+1 )}. T2n+2 ) independently. t ≥ 0} is a CTMC with generator matrix ˆ Q = [ˆij ]. t ≥ 0} with state space S− .1) Next we will show that lim P (X1 (t) ≤ x|I(t) = ζ) = x/q. (i. t ≥ 0} processes corresponding to the sample paths of {X1 (t). t ≥ 0} goes through these two cycles alternately over (T2n . j ∈ S− . and I(t) = 1 if the right derivative of X1 (t) is zero at t. 1}. t ≥ 0} and {Z0 (t). T2n+1 ) and (T2n+1 . t ≥ 0} is independent of q. (B. First note that the sample paths of {X1 (t). t ≥ 0} by eliminating the segments of the sample paths of {X1 (t). t ≥ 0} and {Z(t). j ∈ S− ) given by q qij = qij + ˆ k∈S+ qik ηkj . it is clear that {X2 (t).2) t→∞ First we construct two new processes {Y0 (t).2 Proof. Define I(t) = 0 if the right derivative of X1 (t) is strictly negative at t. Proof of lemma 5.

Kulkarni and K. Z(t). . Y0 (t) and Z0 (t).20 V. Yan Figure 5: Correspondence of the processes X(t). X1 (t).

x)dx .2) for ζ = 0. y)dy y=0 (B. t ≥ 0} with embedded DTMC {(Z1. q π(k.4) where π(j. so that the n-th sojourn time of this SMP is given by T2n+2 −T2n+1 .n = Z(T2n+1 ).3. x) is the expected sojourn time of the SMP in state (j. x). x ≤ Y1 (t) ≤ x + dx}. f (j. T2n+1 ] for all n ≥ 0. Z0 (t) = i) = lim P (X1 (t) ≤ x|I(t) = 0). ˆ q (B.5) . (B.3) ˆ where πi is the limiting distribution of the CTMC with generator matrix Q.n ≤ x + dx}. x)dx = lim P {Z(t) = i. Z0 (t) = i) = x πi . ˆ our construction of the Y0 process implies that lim P (Y0 (t) ≤ x. Hence it follows that t→∞ lim P (Y0 (t) ≤ x. Now construct a semi-Markov process (SMP) {(Z1 (t). Define f (j.EOQ Analysis under Stochastic Production and Demand Rates 21 Thus the {(Y0 (t). Y1 (t)). x) is independent of x. t ≥ 0} processes. t ≥ 0} and {Z1 (t). From Equation (B. t ≥ 0} process satisfies the hypothesis of Theorem 3. for n ≥ 0. t→∞ According to the theory of SMP [6]. we see that g(i. Clearly u(j. Clearly the sample path of {Y1 (t). Z0 (t)). x ≤ Y1. x)dx = lim P {Z1. q (i ∈ S− ). t→∞ (i ∈ S− ). t ≥ 0} is identical to the one obtained by eliminating the segments of the sample path of {X1 (t). x)dx = k∈S+ π(j. t ≥ 0} and {Z(t).3). x)u(j.n . Figure 6 illustrates the sample paths of the {Y1 (t). t ≥ 0} over the intervals (T2n . However. x)dx = lim P {Z(t) = j. t ≥ 0} processes corresponding to the sample paths of {X1 (t). n ≥ 0}. Y1. x) as τj for all x. x) = πi ˆ . hence we denote u(j. x ≤ Y0 (t) ≤ x + dx}. + + Now for ζ = 1.n ). y)u(k.n = X1 (T2n+1 ) and Z1. n→∞ and u(j.n = j. we define Y1. t →∞ t→∞ This proves Equation (B. Let g(i.

X1 (t). Kulkarni and K.22 V. Yan Figure 6: Correspondence of the processes X(t). . Y1 (t) and Z1 (t). Z(t).

x) = i∈S− g(i.1) lim P (X1 (t) ≤ x) = x . (2005).2. [4] El-Taha. π(j. Res. ˆ i∈S− (B.5). S. Optimization. Inventory Models with Continuous Stochastic Demands. Appl. and Perry. t ≥ 0} process is given by f (x) = j∈S+ f (j. D. Operat. x) = q πi qik τk dy ˆ i∈S− k∈S+ y=0 1 = · q πi qij τj ˆ i∈S− πi qik τk ˆ k∈S+ i∈S− . A Sample-Path Condition for the Asymptotic Uniform Distribution of Clearing Processes. An EOQ model with state dependent demand rate. P. References [1] Berman. 1. This proves Equation (B. M.7) 1 = · q 1 = . O. Ann. 965–975.6) into (B.2) for j = 1. and Zipkin. (2002).8) (B. W.4). A Fluid EOQ Model with a Two-State Random Environment.6) Substituting Equation (B. Stadje. [3] Browne. J.9) Equation (B. [2] Berman. t ≥ 0} is uniform over [0. Thus the limiting probability density function of {Y1 (t). 419–435. Prob. x)qij = 1 q πi qij . Hence from (B. 51. (1991). and Perry. we have 1 q πi qij τj ˆ i∈S− 1 q f (j. q π qik τk ˆˆ k∈S+ i∈S− (B.EOQ Analysis under Stochastic Production and Demand Rates 23 Hence using Equation (B. q t→∞ This proves Lemma 5. (2004). D. Eur.. .9) indicates the limiting distribution of {Y1 (t). q]. O. x) π qij τj ˆˆ j∈S+ i∈S− (B. In Press.

[6] Kulkarni. H. [7] Kulkarni. (2000). 29. Yan [5] Kella. [13] Zipkin. (1981). Appl. Mean first passage times in fluid queues. 257–262. Fluid models for single buffer systems. 308–318. V. Appl. London. O. CRC Press. 165–178. Prob. 321–338. R. 20. Res. W. Models and Applications in Science and Engineering. F. (1992). J.). . A storage model with a two state random environment. E. and Stidham. [10] Neuts. Kulkarni and K. The stationary distribution of a stochastic clearing process. W. Modeling and Analysis of Stochastic Systems. in: J. FL. S. G. Operat. M. (1988). Baltimore. (1978). and Tzenova.24 V. Frontiers in Queueing. Boston. Res. G. V. 294–308. P. Matrix Geometric Solutions in Stochastic Models. (1995). Lett. [12] Whitt. Stochastic theory of a fluid model of producers and consumers coupled by a buffer.H. Adv. Chapman and Hall. [9] Mitra. MD. [11] Serfozo. (1997). Dshalalow. Ed. (2002). G. Operat. 6. (1981). Boca Raton. Johns Hopkins University Press. D. Proc. Dshalalow (Ed. Stoch. Foundations of Inventory Management. 30. 40. Operat. Semi-stationary clearing processes. [8] Kulkarni. Res. McGraw-Hill. and Whitt. 646–676. V.