ARTICLE IN PRESS

Int. J. Production Economics 119 (2009) 101–111

Contents lists available at ScienceDirect

Int. J. Production Economics

journal homepage: www.elsevier.com/locate/ijpe

Production and preventive maintenance control in a stochastic

manufacturing system
Dong-Ping Song 
International Shipping and Logistics Group, Business School, University of Plymouth, Cookworthy Building, Drake Circus, Plymouth PL4 8AA, UK

a r t i c l e in fo abstract

Article history: This paper considers the problem of production and preventive maintenance control in a
Received 11 November 2004 stochastic manufacturing system. The system is subject to multiple uncertainties such
Accepted 19 January 2009 as random customer demands, machine failure and repair, and stochastic processing
Available online 7 February 2009
times. A threshold-type policy is proposed to control the production rate and the
Keywords: preventive maintenance operation simultaneously. The explicit form of the stationary
Production control distribution of the system state is derived analytically and used to produce the formula
Preventive maintenance for various steady-state performance measures. The optimal threshold values can then
Stationary distribution be obtained by optimising the relevant formula. Numerical examples demonstrate that
Threshold policy
the proposed threshold policy is indeed optimal or extremely close to the optimal policy
Average cost
in a range of scenarios.
& 2009 Elsevier B.V. All rights reserved.

1. Introduction In practice, many performance measures may be of

The failure-prone manufacturing systems have at-
tracted much attention recently. An important problem
is how to control the system so that the exogenous
demand process can be satisfied with the minimum cost
subject to a variety of uncertainties, e.g. random demands,
stochastic processing times, machine failures and repairs.
By properly controlling the production rate at machine’s
operational mode, a positive inventory level can be
maintained to hedge against the effect of uncertainties.
On the other hand, by executing preventive maintenance
the reliability of the machine can be improved and
therefore the disruptions caused by the machine failures
are reduced. There are two types of preventive main-
tenance operations: a tool change, which necessitates a
machine stoppage; and a control, which does not
necessitate a machine stoppage (Cavory et al., 2001). This
paper focuses on the second type of maintenance. In order
to achieve the best performance, an integrated control
policy for both production and maintenance is preferable.
 Tel.: +44 1752 585630; fax: +44 1752 585713.
E-mail address: Dongping.Song@plymouth.ac.uk
interest, e.g. the long-run average cost (which consists of
the inventory cost, the backlog cost and the maintenance
cost), stock-out probabilities (SP) (representing the custo-
mer service level), average backlog level (BL) (representing
the degree of backlogs), average inventory level (IL), and
machine utilisation.
Literature in failure-prone manufacturing systems (or
deteriorating production systems) can be classified into
two groups (Buzacott and Shathikumar, 1993). The first is
based on ‘‘fluid flow models’’, where the production is
modelled as a continuous production process. The second
is ‘‘discrete part manufacturing systems’’, where parts are
produced in discrete mode.
The first group with the maintenance decisions
included can be referred to Boukas and Haurie (1990),
Boukas (1998), Gharbi and Kenne (2000), Boukas and Liu
(2001), Kenne and Nkeungoue (2008) and the references
there. In Boukas and Haurie (1990), the probability of
machine failure is an increasing function of its age, while
in Boukas and Liu (2001) the machine has multiple
operational states and the machine failure rates are
state-dependent. With the assumptions of a constant
demand rate and a discounted linear cost function, they
established the optimal production and maintenance

0925-5273/$ - see front matter & 2009 Elsevier B.V. All rights reserved.
doi:10.1016/j.ijpe.2009.01.009
 ARTICLE IN PRESS
102 D.-P. Song / Int. J. Production Economics 119 (2009) 101–111
policies using stochastic dynamic programming. Gharbi
and Kenne (2000) considered a multiple-identical-
machine system and investigated the modified hedging
point policy for maintenance and production using
simulation-based approach. Kenne and Nkeungoue
(2008) considered the control of corrective and preventive
maintenance rates in a manufacturing system in which
the failure rate of a machine depends on its age.
The second group often takes into account the
randomness in both production process and demand
process. Sharafali (1984) studied the effects of a machine
failure on the performance measure in which the produc-
tion is controlled by an (s, S) policy. Srinivasan and Lee
(1996) extended the above work by adding preventive
maintenance option to the (s, S) policy. It is assumed that
when the inventory level reaches S, the preventive
maintenance is undertaken and the machine becomes as
good as new. They obtained the optimal parameters s and
S in order to minimise the total average cost. In their
model, preventive maintenance and production are
executed separately and the preventive maintenance is
impliedly determined by the production policy.
Van der Duyn Schouten and Vanneste (1995) consid-
ered a single machine production system with an
exogenously given finite capacity buffer. The production
policy is fixed and determined by the buffer level and
capacity. They proposed a suboptimal preventive main-
tenance policy which is based on the age of the machine
and the buffer level. Das and Sarkar (1999) studied a
production inventory system where the unit production
time, repair time and maintenance time have general
probability distributions. They presented a preventive
maintenance policy based on the inventory level and the
number of items produced since the last repair or
maintenance operation. The production is governed by
the (s, S) policy where the parameters s and S are
exogenously given. Iravani and Duenyas (2002) considered
a single machine system with multiple operational states
and investigated how the structure of the integrated
production and maintenance policy changes as the system
enters different operating states. They introduced a
double-threshold policy to characterise the decisions of
production and maintenance and developed algorithms to
implement the policy. The above three papers assumed
that the unmet demands are not backordered (or
partially) and therefore the system state space is finite.
Yao et al. (2005) investigated the structural properties of
the optimal joint preventive maintenance and production
policy for an unreliable production-inventory system with
constant demands and time-dependent failures.
This paper considers a discrete part manufacturing
system. The system produces a single product type with
stochastic processing times to satisfy an exogenous
demand process. Unmet demands are backordered. The
machine may break down and is repaired afterwards. At
machine’s operational states, both production rate and
preventive maintenance rate are simultaneously control-
lable. The preventive maintenance does not require a
machine stoppage. Such preventive maintenance includes
lubrication, cleaning and adjustment. On one hand,
preventive maintenance can improve the machine’s
reliability to some extent; on the other hand, it will
slightly disrupt the production and therefore reduce the
production capacity. The objective is to find good produc-
tion and maintenance policies to optimise a steady-state
performance measure such as the long-run average cost.
The assumption that unmet demands are backordered
makes the state space infinite. Three random processes
(i.e. production process, demand arrival process and
failure/repair process) together with the production and
preventive maintenance policy determine the behaviour
of the system. It is usually the stationary distribution of
the system state that we need to compute the perfor-
mance measures. We apply a threshold-type policy to
control the production and preventive maintenance. This
is based on the facts that threshold policies are simple,
easy to operate and indeed optimal in many cases without
preventive maintenance (e.g. Sethi and Zhang, 1994; Song
and Sun, 1999; Feng and Xiao, 2002). In the context of
production and maintenance control, Boukas and Haurie
(1990) showed the existence of hedging surfaces, in which
the preventive maintenance actions are triggered by
switching curves. Boukas (1998) studied the effectiveness
of a hedging point policy in a production and corrective
maintenance control problem. Yao et al. (2005) demon-
strated that the optimal production and maintenance
policies have the control-limit structure and the optimal
actions on the entire state space are divided into regions.
The hedging point policy can be regarded as a special type
of threshold policies and the switching-curve or switch-
ing-region policy is an extension of threshold policies.
Moreover, the (s, S) type of policies (e.g. Srinivasan and
Lee, 1996; Das and Sarkar, 1999) are typical examples of
threshold policies. Therefore, threshold policies deserve
more studies in production and maintenance control
problems. In fact, we will show later on through numerical
examples that the threshold policies we proposed are
actually optimal in many cases (cf. Section 5). Our main
task is to derive the explicit form of the stationary
distribution under the proposed threshold policy, evaluate
various steady-state performance measures, find the
optimal threshold values in order to implement the policy,
and demonstrate its effectiveness.
The rest of the paper is organised as follows. The
problem is formulated into an event-driven Markov
decision process in Section 2. In Section 3, a threshold
policy is presented and the stationary distribution under
the threshold control is derived using the characteristic
equation approach. In Section 4, various performance
measures are evaluated and the computation of optimal
threshold values are addressed. Numerical examples are
given in Section 5 and conclusions are made in Section 6.
2. Problem formulation
Consider a manufacturing system producing one part-
type with two modes: operational mode and failure mode.
We first introduce the following basic notation:
d the demand rate. The customer demand follows a
homogeneous Poisson flow with rate d.
l the production rate, which is a controllable variable that takes
a value in [0, r1]. It is assumed that the production time is
 ARTICLE IN PRESS
D.-P. Song / Int. J. Production Economics 119 (2009) 101–111
exponentially distributed with a rate l at machine’s
operational mode.
r1 the maximum production rate at machine’s operational mode
without performing preventive maintenance.
r0 the maximum production rate at machine’s operational mode
with preventive maintenance being performed. We have r0pr1
due to the disruption of the maintenance action.
x the machine failure rate at its operational mode, which is
partially controllable subject to xA[x0, x1]. If x ¼ x1, it means that
no preventive maintenance action is taken; if x ¼ x0, it means
that the maximum preventive maintenance action is taken.
Z the machine repair rate, i.e. the transition rate from failure
mode to operational mode.
x(t) the difference of the cumulative finished parts and the
cumulative demands at time t, which is also termed as the
inventory on hand, but it could take negative values
representing backordered demands.
(l(k), the production rate and preventive maintenance action at
x(k)) system state (1, k), where (1, k) indicates that the machine is at
operational mode and the inventory on hand x(t) equals k.
Let X ¼ (i, k)|i ¼ 0,1 and k ¼ x(t)AZ be the system state
space, here i ¼ 0 represents the failure mode and i ¼ 1
represents the operational mode. Let O ¼ (l(k), x(k))|x(k)
A[x0, x1], l(k)A[0, r1] if x(k) ¼ x1 and l(k)A[0, r0] if
x(k)ox1, (1, k)AX be the admissible set of stationary
control policies. It is assumed that the machine failure
time and repair time are exponentially distributed. At
operational mode, a preventive maintenance action may
be taken, in which case the machine failure rate (i.e. x)
becomes lower and the production capacity is reduced to
be r0 (i.e. r0pr1). It should be pointed out that if the
machine produces nothing (i.e. l ¼ 0) at operational mode,
it will not break down and no preventive maintenance is
required. In other words, the machine failure is operation-
dependent rather than time-dependent. In addition, it is
assumed that the machine failure rate does not depend on
the age of the machine. Instead, the failure rate is
dependent on the machine state and the control actions,
e.g. whether it is operating and whether the preventive
maintenance is being performed. In that sense, the failure
process is similar to Boukas and Liu (2001).
The evolution of the system state is driven by four types of
events, i.e. demand arrival, production completion, machine
failure, machine repaired. Due to the memoryless properties
of Poisson process and exponential distribution, unfinished
production interrupted by an event is statistically equivalent
to that of restarting. The system state changes if and only if
one of the above events occurs. By the uniformisation
technique (Bertsekas, 1987; Puterman, 1994), the problem
can be formulated into a Markov decision process. Let
n ¼ d+r1+Z+x1 be the uniform transition rate. The one-step
transition probability functions at (1, k) and (0, k) are given by
Pðð1; k  1Þjð1; kÞÞ ¼ d=v,
Pðð1; k þ 1Þjð1; kÞÞ ¼ lðkÞ=v,
Pðð0; kÞjð1; kÞÞ ¼ xðkÞ=v  IflðkÞ40g,
Pðð1; kÞjð1; kÞÞ ¼ ðv  d  lðkÞ  xðkÞ  IflðkÞ40g=v,
Pðð0; k  1Þjð0; kÞÞ ¼ d=v,
Pðð1; kÞjð0; kÞÞ ¼ Z=v,
Pðð0; kÞjð0; kÞÞ ¼ ðv  d  ZÞ=v,
103
where I{condition} is an indicator function. It takes 1 if the
condition is true and 0 otherwise.
In reality, it is often the steady-state performance
measures that we are interested in. Therefore, it is
necessary to ensure that the system is stable. A system
is called stable if a stationary distribution of the system
state exists for some uAO. Note that the machine failure
time and repair time are exponentially distributed.
Clearly, if preventive maintenance is never performed,
the long-run maximum production rate is r1Z/(Z+x1) since
the machine produces nothing during breakdown time,
the average machine up time is Z/(Z+x1), and the
maximum machine production rate is r1 during its up
time. Similarly, if preventive maintenance is always
performed at operational mode, the long-run maximum
production rate is r0Z/(Z+x0). Intuitively, the system is
stable if the long-run maximum production rate is greater
than the demand rate, i.e. r1Z/(Z+x1)4d or r0Z/(Z+x0)4d.
In other words, if one of the above two inequality holds,
there exist feasible policies which ensure that the system
will not increasingly backorder customer demands.
The problem is to seek good control policies of
production and maintenance for a specified performance
measure, e.g. the long-run average cost. Threshold policies
have attracted much attention due to its simplicity, easy
to operate and optimality or asymptotic optimality (Sethi
and Zhang, 1994; Srinivasan and Lee, 1996; Das and Sarkar,
1999; Song and Sun, 1999; Feng and Xiao, 2002; Song,
2006). In this paper, we use a threshold-type policy to
control the production and the preventive maintenance.
Our main task is to derive the explicit form of the
stationary distribution, evaluate the various performance
measures, find the optimal threshold parameters, and
demonstrate its effectiveness.
Intuitively, if the system is heavily backlogged, the
maximum production capacity should be utilised in order
to meet the backordered demands as soon as possible.
Namely, if r1Z/(Z+x1)Xr0Z/(Z+x0), i.e. the long-run max-
imum production rate without performing preventive
maintenance is not less than the long-run maximum
production rate with preventive maintenance being
performed, we tend not to perform preventive mainte-
nance operation when x(t)50. On the other hand, if r1Z/
(Z+x1)or0Z/(Z+x0), we prefer to perform maintenance
operation when x(t)50. Therefore, we define two types of
threshold control policies as follows:
(
xðkÞ ¼ x1 if kol
ðIÞ Type-one threshold policy :
xðkÞ ¼ x0 if kXl
8
>
> lðkÞ ¼ r 1 if kol
<
and lðkÞ ¼ r0 if lpkoh
>
>
: lðkÞ ¼ 0 if kXh
(
xðkÞ ¼ x0 if kol
ðIIÞ Type-two threshold policy :
xðkÞ ¼ x1 if kXl
8
>
> lðkÞ ¼ r0 if kol
<
and lðkÞ ¼ r1 if lpkoh
>
>
: lðkÞ ¼ 0 if kXh
 ARTICLE IN PRESS
104 D.-P. Song / Int. J. Production Economics 119 (2009) 101–111

The above threshold policy is characterised by two The flow balance equations are given as follows
integers l and h. Let ul,h denote the type-one threshold
dp1 ðk þ 1Þ þ r 1 p1 ðk  1Þ þ Zp0 ðkÞ ¼ ðd þ r 1 þ x1 Þp1 ðkÞ,
policy and u0l;h denote the type-two threshold policy. The
type-one threshold policy states that the machine should for kol, (1)
produces at full speed if the inventory level is less than h
dp0 ðk þ 1Þ þ x1 p1 ðkÞ ¼ ðd þ ZÞp0 ðkÞ,
and produces nothing otherwise; preventive maintenance
should not be performed if the inventory level is less than for kol, (2)
l, and the maximum preventive maintenance is executed
otherwise. It should be pointed out that the above policy dp1 ðk þ 1Þ þ r 0 p1 ðk  1Þ þ Zp0 ðkÞ ¼ ðd þ r 0 þ x0 Þp1 ðkÞ,
is executed according to the system state, i.e. machine for lokoh, (3)
operational mode and inventory on hand level. It does not
depend on the age of the machine. dp0 ðk þ 1Þ þ x0 p1 ðkÞ ¼ ðd þ ZÞp0 ðkÞ; for lpkoh  1,
The ensuing arguments focus on the type-one thresh- (4)
old policy and we assume r1Z/(Z+x1)4d. For the type-two
threshold policy, the arguments are very similar. We have dp1 ðl þ 1Þ þ r 1 p1 ðl  1Þ þ Zp0 ðlÞ ¼ ðd þ r 0 þ x0 Þp1 ðlÞ (5)
performed a number of numerical experiments, which
r 0 p1 ðh  1Þ ¼ dp1 ðhÞ (6)
confirm that the type-one policy is preferable if r1Z/
(Z+x1)Xr0Z/(Z+x0), while the type-two policy is preferable
x0 p1 ðh  1Þ ¼ ðd þ ZÞp0 ðh  1Þ (7)
if r1Z/(Z+x1)or0Z/(Z+x0). For example, type-one policy
is actually the same as the optimal policy in the The normalisation condition is
cases reported in Tables 2 and 3, whereas type-two X
h1
policy becomes optimal in the cases reported in Table 5 ðp1 ðkÞ þ p0 ðkÞÞ þ p1 ðhÞ ¼ 1 (8)
(cf. Section 5). 1

It is clear that under the above threshold policy the The linear recursive Eqs. (1)–(7) contain infinite number
inventory position will never exceed h as soon as it enters of variables. Define a left-shifting operator D: D pi(k)
a level not greater than h. When the inventory position is : ¼ pi(k1), for i ¼ 0,1.
at h, the machine is idle and will not break down (i.e. no For kol, we have
preventive maintenance is required). Therefore, ul,huh,h 0 1 !
for l4h. Without any loss of the generality, we assume r 1 D2  ðd þ r 1 þ x1 ÞD þ d ZD p1 ðk þ 1Þ
@ A
that lph. x1 D ðd þ ZÞD þ d p0 ðk þ 1Þ
The system state transition map under the type-one
¼0
threshold policy is partially shown in Fig. 1 (where the
self-transition is omitted). It is clear that the induced The characteristic equation for kol is
system forms an irreducible, aperiodic and homogeneous 2
Markov chain. Let {pi(k)} ¼ Prob{state (i, k)} be the ðD  1Þ½r 1 ðd þ ZÞD2  dðd þ r 1 þ Z þ x1 ÞD þ d  ¼ 0 (9)
limiting state probability. From the probability theory, Similarly, the characteristic equation for lokoh1 is
the limiting state probability in an irreducible aperiodic 2
homogeneous Markov chain always exists and it is ðD  1Þ½r 0 ðd þ ZÞD2  dðd þ r 0 þ Z þ x0 ÞD þ d  ¼ 0 (10)
independent of the initial state probability distribution. From the characteristic Eqs. (9) and (10), we can
In addition, we have that either pi(k)0, in which case all construct the solution {pi(k)} using undetermined coeffi-
states are transient or null recurrence; or pi(k)40, in cient method and then find the undetermined coefficients
which case all states are positive recurrence and {pi(k)} based on the boundary conditions and the normalisation
forms a stationary distribution (i.e. steady-state prob- condition. The applications of the characteristics equation
ability distribution). approach can be referred to Feng and Xiao (2002) and
Song (2006).
3. Stationary distribution under a threshold policy
Lemma 1. The solution {pi(k), kpl and i ¼ 0,1} must satisfy

This section aims to derive the stationary distribution p1 ðkÞ ¼ A1 xk k

1 þ B1 x 2 (11)
{pi(k)} under the threshold control policy ul,h. Note that
the stationary distribution of an irreducible aperiodic p0 ðkÞ ¼ A2 xk k
1 þ B2 x 2 (12)
homogeneous Markov chain may be found by solving a set
of flow balance equations and a normalisation condition. A2 ¼ A1 ðr 1 x1  dÞ=d; and B2 ¼ B1 ðr 1 x2  dÞ=d (13)
where x1 and x2 are the roots of equation r1(d+Z)x2
d(d+r1+Z+x1)x+d2 ¼ 0, A1 and B1 are undetermined con-
stants.

Proof. Firstly we show that {pi(k), kol and i ¼ 0,1}

satisfies (11)–(13). Let f(x) : ¼ r1(d+Z) x2d(d+r1+Z+
x1)x+d2. It is clear that f(0)40, f(d/(d+Z))o0, and 0od/
(d+Z)o1. We also have f(1)40 due to r1Z/(Z+x1)4d. It
Fig. 1. State transition map under a threshold control policy. follows that the equation f(x) ¼ 0 has two different roots
 ARTICLE IN PRESS
D.-P. Song / Int. J. Production Economics 119 (2009) 101–111 105

in the interval (0,1). Therefore, the characteristic Eq. (9) (ii) For k ¼ l. Firstly, let k ¼ l+1 in (3), we have:
has three roots, 1, x1 and x2, where x1, x2 A(0,1). Note that dp1(l+2)+r0p1(l)+Zp0(l+1) ¼ (d+r0+x0)p1(l+1). Substi-
the state space is infinite with kol, the root 1 cannot be tute p1(l+2), p0(l+1) and p1(l+1) using (17) and (18),
used to construct the general solution for {pi(k)|kol} due together with (19) and the definition of y1 and y2, it
to the requirement of the normalisation condition (8). can be shown that p1(l) takes the form of (17).
That means, pi(k) for kol can be constructed in the forms Secondly, let k ¼ l in (4). We have: dp0(l+1)+
of (11) and (12) with undetermined constants A1, B1, A2 x0p1(l) ¼ (d+Z)p0(l). Plug p0(l+1) and p1(l) in the
and B2. Substitute two expressions (11) and (12) into (1) forms of (18) and (17), similarly we can show that
and (2), add them together and compare the coefficients p0(l) takes the form of (18).
of xk k
1 and x2 on both sides. We have: A2 ¼ A1(r1x1d)/d (iii) To show E1 ¼ 0. Plug p1(l+1) in the form of (17),
and B2 ¼ B1(r1x2d)/d. p1(l–1), p0(l) and p1(l) in the forms of (11) and (12)
Secondly, we show that {pi(k), k ¼ l and i ¼ 0,1} also into Eq. (5), we have:
satisfies (11)–(13). Let k ¼ l1 in Eq. (1), together with the
dðC 1 yl1
1 þ D1 yl1
2 þ E1 Þ þ r 1 ðA1 xlþ1
1 þ B1 xlþ1
2 Þ
result that {pi(k)|kol} satisfies (11)–(13), we have:
þ ZðA2 xl l l l
1 þ B2 x2 Þ ¼ ðd þ r 0 þ x0 ÞðA1 x1 þ B1 x2 Þ
dp1 ðlÞ ¼ ðd þ r 1 þ x1 Þp1 ðl  1Þ  r 1 p1 ðl  2Þ  Zp0 ðl  1Þ (20)
¼ ðd þ r 1 þ x1 Þ½A1 xlþ1
1 þ B1 xlþ1
2  In addition, let k ¼ l in (4), substitute p0(l+1) using
 r 1 ½A1 xlþ2
1 þ B1 xlþ2
2   Z½A2 xlþ1
1 þ B2 xlþ1
2  the form of (18), p1(l) and p0(l) using the forms of
¼ ½ðd þ r1 þ x1 Þx1  r 1 x21  Zðr 1 x1  dÞx1 =dA1 xl (11) and (12), it follows
1

þ ½ðd þ r 1 þ x1 Þx2  r 1 x22  Zðr 1 x2  dÞx2 =dB1 xl

2 dðC 2 yl1
1 þ D2 yl1
2 þ E2 Þ
¼ ½r1 ðd þ ZÞx21 þ ðd þ r 1 þ x1 þ Þx1 A1 xl
Z 1 =d þ x0 ðA1 xl l l l
1 þ B1 x2 Þ ¼ ðd þ ZÞðA2 x1 þ B2 x2 Þ
2
þ ½r 1 ðd þ Þx2 þ ðd þ r 1 þ x1 þ Þx2 B1 xl
Z Z 2 =d Plug (19) into the above equation, separate the right-
¼ dA1 xl
1 þ dB 1 xl
2 . hand side into two terms and substitute A2 and B2
using (13) in one term, it yields
The last equation follows from the definitions of x1 and x2,
i.e. f(x1) ¼ 0 and f(x2) ¼ 0. Therefore, p1(l) takes the form r 0 ðC 1 yl l l1
1 þ D1 y2 þ E1 Þ  dðC 1 y1 þ D1 yl1
2 þ E1 Þ
of (11). Similarly, let k ¼ l1 in Eq. (2), it can be shown þ ðdx0 þ dZ  r0 ZÞE1 =Z þ x0 ðA1 xl l
1 þ B1 x2 Þ
that p0(l) has the same forms as (12) and (13). This ¼ ZðA2 xl l l l
1 þ B2 x2 Þ þ ðr 1 x1  dÞA1 x1 þ ðr 1 x2  dÞB1 x2
completes the proof. &
(21)
Lemma 2. The solution {pi(k), kXl and i ¼ 0, 1} must satisfy Add (20) and (21), it follows:

p1 ðkÞ ¼ C 1 yk k
1 þ D1 y2 ; lpkph (14) r 0 ðC 1 yl l
1 þ D1 y2 þ E1 Þ þ ðdx0 þ dZ  r 0 ZÞE1 =Z

¼ r 0 ðA1 xl l
1 þ B1 x2 Þ.
p0 ðkÞ ¼ C 2 yk k
1 þ D2 y2 ; lpkoh (15)
On the other hand, from Lemma 1 and the above
C 2 ¼ C 1 ðr 0 y1  dÞ=d; and D2 ¼ D1 ðr 0 y2  dÞ=d (16) proof (ii), we know that p1(l) takes both forms of (11)
and (17), namely
where y1 and y2 are the roots of equation r 0 ðd þ ZÞy2 
2
dðd þ r 0 þ Z þ x0 Þy þ d ¼ 0, C 1 and D1 are undetermined p1 ðlÞ ¼ A1 xl l l l
1 þ B1 x2 ¼ C 1 y1 þ D1 y2 þ E1 (22)
constants. The above two equations yield
Proof. We consider the solution pi(k) in four parts: (i) ðdx0 þ dZ  r 0 ZÞE1 =Z ¼ 0 (23)
lokoh1; (ii) k ¼ l; (iii) k ¼ h1; (iv) k ¼ h and i ¼ 1.
If dx0+dZr0Z ¼ 0, it is easy to show that y2 ¼ 1.
Thus, we have E1 ¼ 0.
(i) For lokoh1, from the characteristic Eq. (10), a (iv) For k ¼ h1. Let k ¼ h2 in (3) and (4), it is easy to
general solution to (3) and (4) can be expressed by show that p1(h1) and p0(h1) must have the same
p1 ðkÞ ¼ C 1 yk k forms as (17) and (18).
1 þ D1 y2 þ E1 (17)
(v) For k ¼ h and i ¼ 1. Similarly, let k ¼ h1 in (3), it
p0 ðkÞ ¼ C 2 yk k yields that p1(h) has the form of (17). This completes
1 þ D2 y2 þ E2 (18)
the proof. &
Let y1 denote the smaller root of r0(d+Z)y2–
d(d+r0+Z+x0)y+d2 ¼ 0. It is easy to show that Lemma 3. The relationships between constants A1, B1, C1
0oy1o1. Note that if y2 ¼ 1, then yk 2  1 and the
and D1 are given by
third term in (17) and (18) is redundant. Therefore,
yl l
1 ðr 0 y1  r 1 x2 ÞC 1 þ y2 ðr 0 y2  r 1 x2 ÞD1
we assume that E1 ¼ E2 ¼ 0 in case y2 ¼ 1. Substitute A1 ¼ (24)
(17) and (18) into (3) and (4), we have r 1 xl
1 ðx1  x2 Þ
C 2 ¼ C 1 ðr 0 y1  dÞ=d; D2 ¼ D1 ðr 0 y2  dÞ=d; E2 ¼ E1 x0 =Z
(19) yl l
1 ðr 0 y1  r 1 x1 ÞC 1 þ y2 ðr 0 y2  r 1 x1 ÞD1
B1 ¼ l
(25)
Later on, we will show that E1 ¼ 0. r 1 x2 ðx2  x1 Þ
 ARTICLE IN PRESS
106 D.-P. Song / Int. J. Production Economics 119 (2009) 101–111

ðd  r 0 y1 Þyh2 Using (26) and the normalisation condition (8), we obtain

D1 ¼ C1 (26)
ðr 0 y2  dÞyh1 (29)–(31). This completes the proof. &

Proof. Consider the boundary conditions at states with Proposition 1. The stationary distribution {pi(k)} under the
k ¼ l. Note that both p1(l) and p0(l) have the forms in threshold policy ul,h can be explicitly given by (11), (12), (14)
Lemmas 1 and 2, i.e. and (15), where the undetermined constants are given in
Lemmas 1–4.
p0 ðlÞ ¼ A2 xl l l l
1 þ B2 x2 ¼ C 2 y1 þ D2 y2 In case r1Z/(Z+x1)Xr0Z/(Z+x0), the inequality r1Z/
Substitute (13) and (16) into the above equation, it follows (Z+x1)4d is actually a sufficient and necessary condition
for the stability of the system under the threshold control
ðr 1 x1  dÞA1 xl l l
1 þ ðr 1 x2  dÞB1 x2 ¼ ðr 0 y1  dÞC 1 y1
policy ul,h. On one hand, since we have obtained the explicit
þ ðr0 y2  dÞD1 yl
2 (27) stationary distribution under the threshold policy with the
Together with the Eq. (22), it leads to (24) and (25). assumption r1Z/(Z+x1)4d, it is sufficient. On the other
Now consider the boundary condition (6), from Lemma 2 hand, suppose r1Z/(Z+x1)pd. With the similar argument in
we have Lemma 1, it is easy to show that there is only one root (say
x1) to the Eq. (9) that is less than 1. Therefore, we have:
r 0 ðC 1 yhþ1
1 þ D1 yhþ1
2 Þ ¼ dðC 1 yh h
1 þ D1 y2 Þ (28)
p1(k) ¼ A1xk k
1 and p0(k) ¼ A2x1 for kpl. In addition, Lemma
The above equation leads to (26). 2 holds. From the boundary conditions at k ¼ l, it is easy to
Lemma 4. The constant C1 is determined by show that C1 ¼ 0. Hence, the only solution to (1)–(6) is
{pi(k)0}, which does not form a stationary distribution.
1 Therefore, r1Z/(Z+x1)4d is necessary for the stability of the
C1 ¼ (29)
F1 þ F2
system.
where
Proposition 2. If r0Z/(Z+x0)4d, then the stationary dis-
ðr 1 þ Z þ x1 Þr 0 y1  dr 1 l r 0 Xh1
1 tribution {pi(k)} under the type-two threshold policy ul,h0
F1 ¼  y1 þ ykþ1 þ h (30)
r 1 Z  dðx1 þ ZÞ d k¼l 1 y1 exists and can be explicitly given by (11), (12), (14) and (15)
by swapping r1 with r0, x1 with x0, and xi with yi.
!
ðr 1 þ Z þ x1 Þr 0 y2  dr 1 l r 0 Xh1
F2 ¼  y2 þ ykþ1 þ yh
2 4. Performance measures and the optimal threshold
r 1 Z  dðx1 þ ZÞ d k¼l 2
values
ðd  r 0 y1 Þyh2
 (31) This section produces formula to compute system
ðr 0 y2  dÞyh1
steady-state performance measures including long-run
Proof. From Lemmas 1 and 3, we have average cost, SP, BL, IL and machine utilisation. The

X
l1
r1 Xl1
ðp1 ðkÞ þ p0 ðkÞÞ ¼ ðA1 xkþ1
1 þ B1 xkþ1
2 Þ
k¼1
d k¼1
!
r 1 xlþ2
1 xlþ2
¼ A1 þ 2 B1
d 1  x1 1  x2
r 1 ðr 0 y1 ðx1 þ x2 Þ  ðr 0 y1 þ r 1 Þx1 x2 Þyl l
1 C 1 þ ðr 0 y2 ðx1 þ x2 Þ  ðr 0 y2 þ r 1 Þx1 x2 Þy2 D1
¼ 
d r 1 ð1  x1 Þð1  x2 Þ

Note that x1 and x2 are the roots of r1(d+Z)x2d(d+r1+ optimal threshold values can then be obtained by
Z+x1)x+d2 ¼ 0, which implies that: x1x2 ¼ d2/r1/(d+Z) and minimising a specific performance measure.
(x1+x2) ¼ d(d+r1+x1+Z))/r1/(d+Z). These lead to
4.1. Long-run average cost
X
l1
ðr 1 þ Z þ x1 Þr 0 y1  dr 1 l
ðp1 ðkÞ þ p0 ðkÞÞ ¼ y1 C 1
k¼1
r 1 Z  dðx1 þ ZÞ The long-run average cost under a control policy u is
ðr 1 þ Z þ x1 Þr 0 y2  dr 1 l defined as
þ y2 D 1 X
r 1 Z  dðx1 þ ZÞ JðuÞ :¼ ½GðkÞðp1 ðkÞ þ p0 ðkÞÞ þ HðkÞp1 ðkÞ (32)
k
In addition, from Lemma 2, we have where G(k) represents the inventory and backlog cost and
X
h1
r0 X
h1 H(k) represents the preventive maintenance cost. A commonly
ðp1 ðkÞ þ p0 ðkÞÞ þ p1 ðhÞ ¼ C 1 ykþ1
1 used function for G(k) is
k¼l
d k¼l

r0 X
h1
GðkÞ ¼ cþ maxðk; 0Þ þ c maxðk; 0Þ (33)
þ D1 ykþ1
2
d k¼l
where c+ and c are the holding cost and the backlog cost
þ C 1 yh h
1 þ D1 y2 per unit of product over per unit of time. The preventive
 ARTICLE IN PRESS
D.-P. Song / Int. J. Production Economics 119 (2009) 101–111 107

P
maintenance cost may be defined as MUðuÞ :¼ lðkÞ40 p1 ðkÞ. For the case lX0, we have,

xlþ1
1 xlþ1 1  yhl
HðkÞ ¼ q  ðx1  xðkÞÞ  IflðkÞ40g (34) MUðul;h Þ ¼ A1 þ 2 B1 þ 1
yhþ1 C 1
1  x1 1  x2 1  y1 1
where qX0 is the cost coefficient for the preventive 1  yhl
2
þ yhþ1 D1
maintenance. 1  y2 2
Under the threshold policy ul,h, if G(k) and H(k) take the Since the system is stable, all the demands must be
forms of (33) and (34), then for the case lX0 the average satisfied eventually. Under the threshold policy, the
cost can be explicitly given by machine’s effective production rate is the same as the
! demand rate d.
rc A1 x21 B1 x22
Jðul;h Þ ¼ þ In the remainder of this section, we present some
d ð1  x1 Þ2 ð1  x2 Þ2
results about the optimal threshold values by minimising
X
l1
rcþ k the long-run average cost.
þ ðA1 xkþ1
1 þ B1 xkþ1
2 Þ
d
k¼1 Proposition 3. To minimise the average cost, the optimal
X
h1
rcþ k threshold value h* must be non-negative, i.e. h*X0.
þ ðC 1 ykþ1
1 þ D1 ykþ1
2 Þ
d
k¼l Proof. It suffices to show that J(ul,1)XJ(ul+1,0). Let {pi(k)}
þ c hðC 1 yh
þ h
1 þ D1 y2 Þ
and {pi0 (k)} be the stationary distributions under the
X
h1 threshold policies ul,1 and ul+1,0, respectively. Clearly,
qðx1  x0 Þ
þ ðC 1 yk k
1 þ D1 y2 Þ (35) pi0 (k+1) ¼ pi(k) for any i and k. To simplify the notation, let
d
k¼l p0(h) ¼ p0(1) : ¼ 0 (because the machine never breaks
For the case lo0, the result is similar and omitted. down at the state h). From the definition of the average
cost,
X
1 X
1
4.2. Stock-out probability Jðul;1 Þ ¼ GðkÞðp1 ðkÞ þ p0 ðkÞÞ þ HðkÞp1 ðkÞ
1 l
The SP is defined as the probability that the system has
P
backordered demands, i.e. SPðuÞ :¼ ko0 ðp1 ðkÞ þ p0 ðkÞÞ. As X
0
Jðulþ1;0 Þ ¼ GðkÞðp1 ðk  1Þ þ p0 ðk  1ÞÞ
an example, for the case lX0, we have 1

rA1 x21 rB1 x22 X

0
SPðul;h Þ ¼ þ þ HðkÞp1 ðk  1Þ
dð1  x1 Þ dð1  x2 Þ lþ1

A related measure is the service level, which is defined Note that H(k) ¼ q(x0x1) for koh, it follows
as the percentage of customer demands that are satisfied
P X
0
from the inventory directly, i.e. ko0 ðp1 ðkÞ þ p0 ðkÞÞ. Jðul;1 Þ  Jðulþ1;0 Þ ¼ ðGðk  1Þ  GðkÞÞðp1 ðk  1Þ
1
þ p0 ðk  1ÞÞ
4.3. BL and IL
The right hand side of the above equation is apparently
P
The BL is defined as: BLðuÞ :¼ ko0 ðkÞ  p1 ðkÞ þ p0 ðkÞÞ non-negative since more backordered demands incur
P
and the IL is defines as: ILðul;h Þ :¼ k40 k  ðp1 ðkÞ þ p0 ðkÞÞ. more penalties in reality. This completes the proof. &
For the case lX0, we have
Since we have obtained the explicit form of the long-run
rA1 x21 rB1 x22 average cost J(ul,h) in (35), the optimal threshold values
BLðul;h Þ ¼ þ (l*, h*) can be obtained through numerical optimisation
dð1  x1 Þ2 dð1  x2 Þ2
approaches, i.e. (l*, h*) ¼ arg minlph;hX0 Jðul;h Þ. Physically,
the optimal threshold value h* is helpful to design the
X
l1
rk
ILðul;h Þ ¼ ðA1 xkþ1 þ B1 xkþ1 Þ buffer size for finished goods since the inventory level will
1 2
k¼1
d never exceed h* under such threshold policy. On the other
X
h1 hand, together with the IL, it yields the utilisation of the
rk
þ ðC 1 ykþ1
1 þ D1 ykþ1
2 Þ þ hðC 1 yh h
1 þ D1 y2 Þ buffer.
d
k¼l In some circumstances, we may aim to find the optimal
The BL indicates how many customer demands are threshold policy to minimise the average cost subject to SP
backordered on average at any time. The IL is useful to (or service level) requirements, e.g. SP(ul,h)oa, where a is
calculate the buffer utilisation. a pre-specified target. This can be easily done by imposing
constraints to the numerical optimisation procedure in
search for the optimal threshold parameters.
4.4. Machine utilisation If the production process is operating in just-in-time
mode (also called make to order), no inventory should be
The machine utilisation is defined as the fraction of maintained. In such case, we only need to optimise the
time that the machine is in operation, i.e. preventive maintenance threshold value. The optimal
 ARTICLE IN PRESS
108 D.-P. Song / Int. J. Production Economics 119 (2009) 101–111
preventive maintenance threshold value l* is determined

by: l ¼ arg minlp0 Jðul;0 Þ, which is a straightforward
single parameter optimisation problem.
5. Numerical examples
This section provides numerical examples to demon-
strate the results. The purpose is three-fold: (i) to show
the effectiveness of the threshold policy in a range of
scenarios using a full factorial experiment; (ii) to
investigate the sensitivity of the steady-state performance
measures in response to some specific system parameters;
(iii) to illustrate the structural properties of the optimal
policy in comparison with the proposed threshold policy.
In terms of minimising the long-run average cost, the
problem can be formulated into a Markov decision process
using stochastic dynamic programming and the optimal
policy can be found numerically by the value iteration
algorithm (e.g. Bertsekas, 1987; Puterman, 1994). We
compare the proposed threshold policies (type-one and
type-two) with three other policies: the optimal inte-
grated production and preventive maintenance policy,
the optimal always-maintenance production policy, and
the optimal non-maintenance production policy. Under
the always-maintenance policy, the machine has the
maximum production rate r0 and a constant failure rate
x0; while under the non-maintenance policy the machine
has the maximum production rate r1 and a constant
failure rate x1. Therefore, we only need to optimise the
production rate for these two policies since the main-
tenance action is implied. The average costs for the above
last three policies (denoted as J*, Ja, and Jn, respectively)
are evaluated using the value iteration algorithm with
1500 iterations and a truncated state space 50ox(t)o50,
while the performance measures for the type-one and
type-two threshold policies are calculated analytically
based on Propositions 1 and 2. It should be pointed out
that the value iteration approach cannot provide other
steady-state performance measures.
From the assumptions of exponential distributions, we
know that 1/Z represents the average machine down time
and 1/x represents the average machine up time, it is
therefore reasonable to assume that Z is greater than x
(with or without preventive maintenance). The machine
production rate should exceed the demand rate. The
backlog cost is greater than the inventory cost and the
maintenance cost. More specifically, system parameters
should generally follow: Z4x14x0, r1Xr04d, c4c+ , and
c–Xq.
With these guidelines, a full factorial experiment is
designed to compare the cost performance under different
policies in a range of parameter values. Six factors
are considered. Each factor takes three or two levels, i.e.
Z (0.7, 0.8, 0.9); x1 (0.3, 0.4); x0 (0.1, 0.2); r1 (2.0, 2.5, 3.0);
r0 (1.5, 1.8, 2.0); and q (1, 5, 10), which gives total
3 * 2 * 2 * 3 * 3 * 3 ¼ 324 different scenarios. Other para-
meters are fixed, e.g. d ¼ 1.0, c+ ¼ 1, and c ¼ 10. The
statistics corresponding to the relative differences of the
costs between a specific policy (e.g. type-one threshold
policy, type-two threshold policy, the minimum of two
threshold policies, the optimal always-maintenance pol-
icy, the optimal non-maintenance policy) and the optimal
policy are given in Table 1. Take the type-one threshold
policy (ul,h) as an example, the relative difference is
defined as (J(ul,h)J*)/J*  100%. These statistics include the
average value (i.e. mean), standard deviation, minimum
value, and maximum value over total 324 scenarios for
each policy.
It can be observed from Table 1 that both types of
threshold policies perform very well for the experimented
324 scenarios (less than 1% away from the optimal policy
on average). More importantly, the minimum of these two
threshold policies is extremely close to the optimal policy
and much better than each individual type. Even in the
worst scenario, it is only 2.21% above the optimal cost.
This demonstrates the effectiveness of the proposed
threshold policies and also reflects that the fact that both
types have their relative merit. As for the always-
maintenance policy and the optimal non-maintenance
policy (in the last two columns of Table 1), they perform
significantly worse than the optimal policy with large
standard deviations. Although in some scenarios they
could be as good as the optimal policy (indicated by ‘Min’
row of Table 1), in other scenarios they are up to 312% and
183% worse than the optimal policy. This full factorial
experiment shows that the proposed threshold policies
are very effective in a reasonable range of scenarios.
Next we want to examine the sensitivity of the steady-
state performance measures in response to some specific
system parameters (e.g. demand rate d, production
capacity r0, maintenance cost q), and make a detailed
comparison between control policies. Consider a system
with parameter settings as follows: r1 ¼ 2.0, Z ¼ 0.9,
x1 ¼ 0.3, x0 ¼ 0.1, c+ ¼ 1, and c ¼ 10. Four specific groups
of cases are investigated:
(A) r0 ¼ 1.6, q ¼ 1.0 with different demand rate d;
(B) d ¼ 1.0, q ¼ 1.0 with different production capacity r0
during performing maintenance;
(C) r0 ¼ 1.6, d ¼ 1.0 with different maintenance cost q;
(D) r0 ¼ 1.9, d ¼ 1.0 with different maintenance cost q.
More specifically, in group A, the demand rate d varies
to represent different traffic scenarios. When d is
approaching to the production rate r0, the system tends
to be unstable. In group B, the production rate r0 varies to
represent the different degrees of maintenance disruption
to the production. In group C, the preventive maintenance
cost q is varying. In the last group, the type-two threshold
policy is examined.
Table 1
Relative difference of the costs between specific policies and the optimal
policy.
J(ul,h) Jðu0l;h Þ Min(J(u), J(u0 )) Ja Jn
Mean (%) 0.21 0.71 0.05 48.76 15.85
Std. dev. 0.68 1.80 0.22 66.35 30.09
Min (%) 0.00 0.00 0.00 0.00 0.00
Max (%) 5.21 12.66 2.21 312.48 183.21
 ARTICLE IN PRESS
D.-P. Song / Int. J. Production Economics 119 (2009) 101–111 109

Tables 2–5 give the optimal threshold values (l*, h*) non-maintenance policy is about 10% worse than the
and the average cost J(ul,h) using the analytical results in optimal threshold policy. As the demand rate increases,
Propositions 1 and 2 associated with a numerical the non-maintenance policy is getting better than the
optimisation procedure. Other steady-state performance always-maintenance policy because in this group the
measures such as SP, IL, BL, and machine utilisation (MU) effective production capacity under the non-maintenance
are also provided. In addition, the costs corresponding to policy is greater than that under the always-maintenance
the optimal policy (J*) and the optimal always-mainte- policy. (ii) The threshold value h*, the IL, the BL and the
nance policy (Ja), and the optimal non-maintenance policy machine utilisation are increasing as the demand rate d
(Jn) are given in the last three columns. increases. This is in agreement with the intuition that
It can be seen from Table 2 that: (i) the optimal type- higher demand rates require higher inventory levels and
one threshold policy is indeed the same as the optimal higher machine utilisation.
policy in this group. In terms of the average cost, the Table 3 also shows that the type-one threshold policy
optimal always-maintenance policy is 1.2–8.1% worse is the same as the optimal policy in terms of minimising
than the optimal threshold policy, whereas the optimal the average cost. The always-maintenance policy per-
forms poorly when r0 is small (e.g. r0o1.6), but is
approaching to the optimal policy when r0 becomes
Table 2 adequately large (e.g. r0X1.7). This indicates that the
Steady-state performance measures with different demand rate d. benefit of machine reliability improvement cannot offset
the capacity reduction due to the disruption of performing
d (l*, h*) J(ul,h) SP IL BL MU J* Ja Jn
maintenance if r0 is too small. The non-maintenance
0.80 (1, 4) 4.682 0.066 2.830 0.176 0.493 4.682 4.736 5.139 policy does not change for different r0. As r0 increases, the
0.90 (2, 5) 5.901 0.077 3.426 0.237 0.557 5.901 6.004 6.513 benefit of performing preventive maintenance is increas-
1.00 (1, 7) 7.611 0.070 4.795 0.270 0.616 7.611 7.793 8.306
ing substantially (up to 52%). This reflects the fact that the
1.10 (1, 9) 10.147 0.084 5.897 0.412 0.676 10.147 10.585 10.971
1.20 (0, 13) 14.464 0.089 8.261 0.607 0.734 14.464 15.637 15.341
less the production capacity is sacrificed for a fixed degree
of improvement of machine reliability, the more benefit
can be achieved. If r0 is greater than 1.70, we have
l* ¼ 50, which means always performing preventive
Table 3 maintenance is a better choice. In fact, when r0X1.70,
Steady-state performance measures with different r0. we have r1Z/(Z+x1)or0Z/(Z+x0). Namely, the average
production rate with maintenance is larger than the
r0 (l*, h*) J(ul,h) SP IL BL MU J* Ja Jn
average production rate without maintenance. As ex-
1.40 (4, 7) 8.204 0.086 4.558 0.356 0.629 8.204 12.056 8.306 pected, in this situation performing maintenance is
1.50 (2, 7) 8.017 0.081 4.599 0.331 0.632 8.017 9.413 8.306 preferred when x(t)50 and the type-two threshold policy
1.60 (1, 7) 7.611 0.070 4.795 0.270 0.616 7.611 7.793 8.306
is recommended.
1.70 (50,6) 6.734 0.072 4.150 0.247 0.588 6.734 6.734 8.306
1.80 (50,5) 5.964 0.077 3.448 0.240 0.556 5.964 5.964 8.306
Table 4 shows that the type-one threshold policy is
1.90 (50,4) 5.437 0.091 2.698 0.263 0.526 5.437 5.437 8.306 optimal when the maintenance cost q is small (e.g.
qo4.0), but it is gradually deviating from the optimal
policy and converges to the optimal non-maintenance
policy as q increases. This reflects the intuition that if the
Table 4 preventive maintenance incurs too much cost, it is
Steady-state performance measures with r0 ¼ 1.6 and different main- preferable not to perform it. However, the results reveal
tenance cost q.
that the threshold policy is still very close to the optimal
q (l*, h*) J(ul,h) SP IL BL MU J* Ja Jn policy (e.g. only 0–1.5% worse than the optimal policy in
Table 3). The always-maintenance policy outperforms the
1.00 (1, 7) 7.611 0.070 4.795 0.270 0.616 7.611 7.793 8.306 non-maintenance policy when q is small (e.g. qp4.0),
2.00 (1, 7) 7.727 0.070 4.795 0.270 0.616 7.727 7.918 8.306
while the non-maintenance policy performs better when
4.00 (1, 7) 7.959 0.070 4.795 0.270 0.616 7.946 8.168 8.306
6.00 (1, 7) 8.192 0.070 4.795 0.270 0.616 8.085 8.418 8.306
q is large (e.g. qX6.0).
8.00 (7, 7) 8.306 0.085 4.795 0.351 0.500 8.180 8.668 8.306 As mentioned in group B, when r0 ¼ 1.90, we have r1Z/
10.0 (7, 7) 8.306 0.085 4.795 0.351 0.500 8.240 8.918 8.306 (Z+x1)or0Z/(Z+x0) and the type-two threshold policy u0l;h
is preferred. Table 5 gives the optimal threshold values,
the average costs and the corresponding steady-state
Table 5 performance measures under u0l;h. In addition, the average
Steady-state performance measures for u0l;h with r0 ¼ 1.9 and different q. costs under the optimal policy (J*), the always-mainte-
nance policy (Ja) and the non-maintenance policy (Jn) are
q (l*, h*) Jðu0l;h Þ SP IL BL MU J* Ja Jn
also provided. It can be observed that in this group the
1.00 (4, 4) 5.437 0.091 2.698 0.263 0.526 5.437 5.437 8.306 type-two threshold policy u0l;h is indeed the same as the
2.00 (4, 4) 5.542 0.091 2.698 0.263 0.526 5.542 5.542 8.306 optimal policy, which is 33–52% better than the optimal
4.00 (4, 4) 5.753 0.091 2.698 0.263 0.526 5.753 5.753 8.306 non-maintenance policy. The always-maintenance policy
6.00 (4, 5) 5.960 0.072 3.459 0.211 0.516 5.960 5.963 8.306 performs fairly well. It is the same as the optimal policy
8.00 (4, 5) 6.090 0.072 3.459 0.211 0.516 6.090 6.174 8.306
10.00 (4, 5) 6.220 0.072 3.459 0.211 0.516 6.220 6.384 8.306
when q is small (e.g. qp4.0), while it is deviating from the
optimal policy as q increases. The threshold value h* in
 ARTICLE IN PRESS
110 D.-P. Song / Int. J. Production Economics 119 (2009) 101–111

Table 5 is lower than that in Table 4, which reflects the fact three-parameter threshold policy is more demanding. In
that the production rate r0 is larger in group D. addition, it is unknown whether such structural property
Tables 2–5 show that the IL and the BL are much more is true for more general cases. Nevertheless, it can be
sensitive to d than to r0 or q. In all cases, the SP are low, observed that the proposed threshold policy is fairly close
between 6% and 9%. This is due to the parameter setting in to the optimal policy.
which the backlog cost is ten times of the inventory cost.
From the above specific numerical examples, it can be 6. Conclusions
seen that in most cases the optimal policy takes either the
type-one threshold policy (group A, group B, and the first This paper investigates the production and preventive
two cases in group C), or the type-two threshold policy maintenance control problem in a discrete part manufactur-
(group D). For some cases in group C with q ¼ 4.0, 6.0, 8.0 ing system. Three random processes, i.e. the production
and 10.0, the optimal policy is different from the threshold process, the demand arriving process and the machine
policy. It is worthwhile to explore the structural proper- failure/repair process, are considered. The unmet demands
ties of the optimal policy when it is different from the are fully backordered, which makes the system state space
threshold policy. As an example, the optimal policies for infinite. Two types of threshold policies are suggested to
the fourth and fifth cases (q ¼ 6.0 and 8.0) in group C, control the production and the preventive maintenance
denoted as case C(4) and case C(5), are shown in Figs. 2 simultaneously. We derive an explicit form of the stationary
and 3, respectively. distribution under such policies. The performance measures
It can be seen that in case C(4) the optimal policy and are then calculated and the optimal threshold parameters are
the type-one threshold policy only differ at states k ¼ 5 obtained. Since the stationary distribution is derived inde-
and 6, while in case C(5) they differ at states k ¼ 0–3. pendent of the form of the performance measures, the result
Physically, x(k) ¼ x1 indicates that no preventive main- is applicable in a wide range of situations (e.g. non-linear cost
tenance is performed, x(k) ¼ x0 indicates that a preventive functions, stock-out probability (SP) requirements, buffer
maintenance action is taken, and l(k) ¼ 0 means that the utilisation, multiple objective functions). In terms of mini-
machine is not producing and therefore will not break mising the long-run average cost, numerical examples
down. It appears that the optimal policy can be char- showed that the threshold policies are indeed optimal or
acterized by three threshold parameters instead of two. very close to the optimal policy, and are significantly better
However, the derivation of the stationary distribution for a than the always-maintenance policy or the non-maintenance
policy in a range of scenarios. The results also confirmed the
intuition about which type of the threshold policies is
λ(k)=r1 λ(k)=r0 λ(k)=r1 λ(k)=0 preferred in which situation.
ξ(k)=ξ1 ξ(k)=ξ0 ξ(k)=ξ1 ξ(k)=0 The research in this paper may be interpreted in
another way. In practice, we may face a problem to make
decisions on whether to select a more reliable machine
k with less production capacity or a less reliable machine
-2 -1 0 4 5 6 7
with more production capacity. The proposed two types of
threshold policies can be easily applied to such situations.
λ(k)=r1 λ(k)=r0 λ(k)=0 One limitation of the research is the assumption that the
ξ(k)=ξ1 ξ(k)=ξ0 ξ(k)=0 preventive maintenance actions are performed without a
machine stoppage. Machine stoppage maintenance is an
important type of preventive maintenance and deserves
k further research. The model may be extended to this type of
-2 -1 0 6 7
maintenance by including additional machine modes, e.g.
Fig. 2. Optimal policy vs. type-one threshold policy with q ¼ 6.0. stoppage mode for preventive maintenance, more reliable
operational mode, less reliable operational mode, and a
failure mode. The machine may transition from the reliable
mode into the less reliable mode. This problem can be
λ(k)=r1 λ(k)=r0 λ(k)=r1 λ(k)=0
formulated similarly. However, the derivation of the sta-
ξ(k)=ξ1 ξ(k)=ξ0 ξ(k)=ξ1 ξ(k)=0 tionary distribution requires further research since the state
space has to be augmented to include more states. Never-
k theless, similar threshold policies can be constructed and
-1 0 3 4 6 7 evaluated at least numerically if not analytically. Moreover,
extending the model to multi-machine or multi-product
systems is also interesting.
λ(k)=r1 λ(k)=0
ξ(k)=ξ1 ξ(k)=0
Acknowledgement
k
The author would like to thank two anonymous
-1 0 6 7
referees for their helpful comments that improved the
Fig. 3. Optimal policy vs. type-one threshold policy with q ¼ 8.0. presentation of the paper.
 ARTICLE IN PRESS
D.-P. Song / Int. J. Production Economics 119 (2009) 101–111
References
Bertsekas, D.P., 1987. Dynamic Programming: Deterministic and Stochas-
tic Models. Prentice-Hall, Englewood Cliffs, NJ.
Boukas, E.K., 1998. Hedging point policy improvement. Journal of
Optimization Theory and Applications 97 (1), 47–70.
Boukas, E.K., Haurie, A., 1990. Manufacturing flow control and preventive
maintenance: a stochastic control approach. IEEE Transactions on
Automatic Control 35 (9), 1024–1031.
Boukas, E.K., Liu, Z.K., 2001. Production and maintenance control for
manufacturing systems. IEEE Transactions on Automatic Control 46
(9), 1455–1460.
Buzacott, J.A., Shathikumar, J.G., 1993. Stochastic Models of Manufactur-
ing Systems. Prentice-Hall, Englewood Cliffs, NJ.
Cavory, G., Dupas, R., Goncalves, G., 2001. A genetic approach to the
scheduling of preventive maintenance tasks on a single product
manufacturing production line. Internal Journal of Production
Economics 74, 135–146.
Das, T.K., Sarkar, S., 1999. Optimal preventive maintenance in a
production inventory system. IIE Transactions 31, 537–551.
Feng, Y.Y., Xiao, B.C., 2002. Optimal threshold control in discrete failure-
prone manufacturing systems. IEEE Transactions on Automatic
Control 47 (7), 1167–1174.
Gharbi, A., Kenne, J.P., 2000. Production and preventive maintenance
rates control for a manufacturing system: an experimental design
approach. Internal Journal of Production Economics 65, 275–287.
111
Iravani, S., Duenyas, I., 2002. Integrated maintenance and production
control of a deteriorating production system. IIE Transactions 34,
423–435.
Kenne, J.P., Nkeungoue, L.J., 2008. Simultaneous control of production,
preventive and corrective maintenance rates of a failure-prone
manufacturing system. Applied Numerical Mathematics 58 (2),
180–194.
Puterman, M.L., 1994. Markov Decision Processes: Discrete Stochastic
Dynamic Programming. Wiley, New York.
Sethi, S.P., Zhang, Q., 1994. Hierarchical Decision Making in Stochastic
Manufacturing Systems. Birkhauser, Boston, MA.
Sharafali, M., 1984. On a continuous review production–inventory
problem. Operations Research Letters 3, 199–204.
Song, D.P., 2006. Optimal production and backordering policy in failure-
prone manufacturing systems. IEEE Transactions on Automatic
Control 51 (5), 906–911.
Song, D.P., Sun, Y.X., 1999. Optimal control structure of an unreliable
manufacturing system with random demands. IEEE Transactions on
Automatic Control 44 (3), 619–622.
Srinivasan, M.M., Lee, H.S., 1996. Production/inventory system with
preventive maintenance. IIE Transactions 28, 879–890.
Van der Duyn Schouten, F.A., Vanneste, S.G., 1995. Maintenance
optimization of a production system with buffer capacity. European
Journal of Operational Research 82, 323–338.
Yao, X.D., Xie, X.L., Fu, M.C., Marcus, S.I., 2005. Optimal joint preventive
maintenance and production policies. Naval Research Logistics 52,
668–681.