European Journal of Operational Research 125 (2000) 398±409

www.elsevier.com/locate/dsw

Theory and Methodology

A discrete semi-Markov decision model to determine the optimal

repair/replacement policy under general repairs
a,*,1
C.E. Love , Z.G. Zhang b, M.A. Zitron c, R. Guo d

a
Faculty of Business Administration, Simon Fraser University, Burnaby, BC, Canada V5A 1S6
b
University College of the Fraser Valley, Abbotsford, BC, Canada
c
Faculty of Business Administration, Simon Fraser University, Burnaby, Canada
d
Department of Statistical Sciences, University of Cape Town, Cape Town, South Africa
Received 1 July 1997; accepted 1 October 1998

Abstract

The state of a machine (system) that may experience failures is characterized by the real age of the machine and the
number of failures incurred to date. On failure, the unit may undergo a repair which can partially reset the failure
intensity of the unit. In this paper, it is assumed that repairs can, at most, shift back the failure intensity so as to remove
the most recent run-time sojourn. The other alternative at a failure is to conduct a major overhaul that serves to refresh
the failure intensity of the unit. General cost structures, depending upon both real age and number of failures are
permitted. The decision, on failure to repair or renew is formulated as a discrete semi-Markov decision process with real
age and number of failures as the state space. Optimal decisions are of the threshold type. That is, on the nth failure, if
the real age is above a predetermined threshold value, refresh; otherwise conduct an imperfect repair. Ó 2000 Elsevier
Science B.V. All rights reserved.

Keywords: Markov processes; Semi-Markov; General repair; Renewals

1. Introduction replaced by a new identical one. In general, a

We consider a machine (system) that is subject
to failure (breakdown). At any failure instant,
there are two types of actions to be taken. Upon
breakdown, the machine can be repaired or can be
*
Corresponding author. Tel.: +1 604 291 3708; fax: +1 604
291 4920.
1
This research supported by NSERC Grant #1228.
repair can bring the state of a failed machine to a
level which is somewhere between new and prior to
failure (assuming the repair does not damage the
machine). The minimal repair (i.e. bad as old) and
perfect repair (good as new) are two special (ex-
treme) cases of this imperfect repair model. Kijima
[4] proposed that the state of the machine just after
repair can be described by its the so-called virtual
age which is smaller (younger) than the real age. In
his framework, the failure rate depends on the

0377-2217/00/$ ± see front matter Ó 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 7 - 2 2 1 7 ( 9 9 ) 0 0 0 0 9 - 0
 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409
virtual age of the system. The repair, following a
failure, serves us to reset the virtual age which
determines the revised failure intensity. In his
original paper, Kijima proposed two repair e€ect
models although in this paper we will focus only
his Type I imperfect repair model leaving attention
to his Type II imperfect repair model (and exten-
sions) to a subsequent paper.
Consider a system that has experienced its
…n ÿ 1†th failure and has been repaired. After the
repair, denote its virtual age as v‡ …tnÿ1 †, where
the ‡ denotes virtual age at failure but after
repair. Upon restart, if the nth system failure
occurs after a sojourn of xn time units, then the
virtual age before repair at the nth failure can be
denoted as
vÿ …tn † ˆ v‡ …tnÿ1 † ‡ xn ; …
where tn is the real age of the system at the nth
repair, and ÿ denotes virtual age at failure but
before repair.
In Kijima's Type I imperfect repair model, he
suggested that upon failure, the repair undertaken
could serve to reset the intensity only as far back
as the virtual age at the start of the last sojourn.
That is:
v‡ …tn † ˆ v‡ …tnÿ1 † ‡ hn xn : …
Here, hn re¯ects the impact of the nth repair
…0 6 hn 6 1†. If hn ˆ 1, then we have a minimal
repair. If hn ˆ 0, then there has been no aging from
the …n ÿ 1†th repair. It is obvious from the above
equation that, since a repair can reset, at a maxi-
mum, the most recent sojourn time, the post-repair
virtual age is non-decreasing as is the real age.
Furthermore, if hnP ˆ h; 8n; one can immediately
n
see that v‡ …tn † ˆ h jˆ1 xj ˆ htn : Thus, in this case,
there is a one-to-one correspondence between real
age and post-repair virtual age.
The purpose of the original paper of Kijima [4]
was to provide a procedure whereby bounds on the
expected real age of a system could be determined
when repairs follow Type I or Type II processes. In
a subsequent paper Kijima [5] sought to determine
the optimal cycle time between replacements for a
system subject to a sequence of Type I repairs. To
solve this problem, utilizing his notion of virtual
aging, Kijima [5] formulated the sequence of fail-
399
ure/Type I repairs as a g-renewal density.
Integrating this density up to an arbitrary time, T,
yields a g-renewal function. The concept of a
g-renewal density and its integral, the g-renewal
function follows directly from the well-known or-
dinary renewal functions of renewal processes (see
Refs. [1,2]). Determination of the g-renewal func-
tion provides a direct estimate of the expected
number of failures from time 0 to time T and hence
an optimal cost rate C…T † can be found by mini-
mizing the function 1=T …C0 ‡ C1 H …T ††, with re-
spect to T; where C0 is the cost of a single
replacement, C1 is the cost per repair and H …T † is
the expected number of failures in …0; T Š (i.e. the
g-renewal function).
To determine the g-renewal function, Kijima
proposed a numerical approximation procedure
1† and applied the scheme to an assumed gamma
failure process with Type I repairs. We return to
his numerical work later in this paper by way of
comparing his results to the semi-Markov decision
analysis developed in this paper.
Makis and Jardine [7] formulated this failure/
Type I repair process as a semi-Markov decision
process (for a thorough discussion of such struc-
tures, see Ref. [11]) and demonstrated that, under
suitable conditions, an optimal stationary policy
2†
exists. A two dimensional state space was utilized
in their formulation, de®ned as (number of failures,
real age). This provides considerable ¯exibility in
modelling such problems since, in addition to the
failure process itself, the costs of repair and re-
placement can depend upon this state space. A
further distinction between the Kijima [5] analysis
and that of Makis and Jardine [7] is the assump-
tion regarding cycle lengths. Kijima assumed that
if the prescribed time T had been reached then the
system would be immediately shut down and re-
placed. We will refer to this as a ®xed-T policy.
Makis and Jardine in contrast assumed that re-
placement occurs at the ®rst failure after time T.
This latter assumption follows from the observa-
tion of Phelps [9,10] following work of Muth [8]
that, if replacements upon failure cost the same as
replacements at schedule times, then it is always
optimal to wait until the ®rst failure following T
to carry out the replacement. Such a policy we
will refer to as a T-plus policy, in contrast to the
400 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409
®xed-T policy utilized by Kijima. Up to T, with
Phelps, failures received minimal repairs whereas
the proposal of Makis and Jardine was that they
should receive Type I repairs.
To generate a minimum expected average cost
per unit time following a T-plus policy with Type I
imperfect repairs, Makis and Jardine reformulated
their cost function in terms of Kijima's g-renewal
function and thereby generated a numerical solu-
tion for any set of presumed data. In this way they
were able to identify the optimum stationary
solution of their semi-Markov model.
Utilizing a g-renewal function approach, while
providing a straightforward numerical approxi-
mation procedure, does limit the range of
structures that can be analyzed. A g-renewal
approach limits the analysis to purely time-de-
pendent failure densities. Furthermore the cost
structures generating unique minimum expected
average costs must be monotonically increasing
and thus cannot depend upon states other than
time itself.
Developing repair/replacement policies directly
from a semi-Markov decision framework however
does provide a method of analysis that allows for
considerable more ¯exibility in accommodating
structures that might occur in practice. Our spe-
ci®c purpose in this paper is to develop a discrete
semi-Markov decision structure and to propose a
numerical search procedure that will generate
optimal repair/replacement policies for stochasti-
cally failing systems subject to Kijima Type I
imperfect repairs. We will utilize the two dimen-
sional state space proposed by Makis and Jardine
in order to permit state dependent cost structures
and, if needed, state dependent failure rates as
well.
The remainder of this paper is as follows. In
Section 2, we will formulate a discrete SMDP for
such failing systems including an appropriate state
space de®nition as well as the necessary transition
structures. In Section 3 we will present a search
algorithm in order to identify optimal control-limit
type policies. In Section 4 we will present the re-
sults of some numerical analysis in order to com-
pare this solution procedure with results of Makis
and Jardine, Kijima as well as Phelps when uti-
lizing renewal functions.
2. Model formulation
2.1. State space
We note in Type I imperfect repairs, that the
failure intensity of the system is identi®ed uniquely
by its virtual age. Furthermore, we assume that
repair costs, in general, depend upon failure
number and real age. We will thus describe the
state of the system in terms of two variables …n; tn †,
where n denotes the nth failure and tn the real age
at that instant. The state space then is S ˆ
f…n; tn † j n ˆ 1; 2; . . . ; 0 6 tn 6 1g. The machine
failure instants are decision epochs. At each deci-
sion epoch, there are two actions, repair …a ˆ 1† or
replacement …a ˆ 0†. Although not necessary, for
consistency with Kijima [5] as well as Makis and
Jardine [7] the replacement cost is considered here
to be ®xed at C0 : On the other hand, again for
consistency with previous authors, we treat the
repair cost as a bounded non-decreasing function
of n and tn (i.e. C1 …n; tn † 6 K ; n P 1; tn P 0). Note
that the state space of this problem is a two di-
mensional in®nite state space with one discrete
state variable (n) and one continuous state vari-
able …tn †.
In this paper, we utilize a discrete semi-Markov
process (SMDP) formulation to determine the
optimal replacement policy. In order to treat the
in®nite and continuous state space of …tn † we divide
the real age axis into a set of equally spaced age
slices and regard that the nth failure occurs at age
slice in . To relate time slice in to real time tn , we
introduce a scaling parameter, n, de®ned as the
number of time slices in a real time unit (i.e.
in =n 6 tn < …in ‡ 1†=n). Notice then that the state at
the instant tn is …n; in † and the failure occurring in
this range is assumed to have taken place precisely
at in . In doing so, the state space can be discretized.
While this remains an approximation to the real
age at this epoch, nonetheless, if need be we can
increase the accuracy of this approximation by
increasing the value of n (and thereby increasing
the number of time slices in ). In Appendix A we
indicate how this can be accomplished for speci®c
failure models.
Utilizing the main theorem of Makis and Jar-
dine [7], the optimal replacement policy for this
 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409
type of system is of the control-limit form
(threshold type). That is, for each n given an im-
perfect repair level h (assumed here to be a con-
stant), there exists a threshold number sn …h† such
that i€ in P sn …h† the optimal action is to replace at
the nth failure. For convenience henceforth, we
will suppress the parameter h, in reference to these
threshold numbers since a constant h is implicit in
the development of any threshold policy. Fur-
thermore, to simplify the notation, we will also
suppress the subscript (n) on the age slices and
simply refer to the age slice as i. Because C1 …n; i†
(the repair cost at the discretized points) is a non-
decreasing convex function in n, it is reasonable to
assume a maximum number of failures before re-
placement, N. (This value can be determined nu-
merically.) Hence, given that C1 …n; i† is non-
decreasing in n, then, as established by Makis and
Jardine [7], the optimal replacement policy can be
characterized by a set of positive integers
…s1 P s2 P


P sN †. This means that if at the nth
failure the real age i P sn , the machine should
be replaced. Since the control limit is a ®nite value,
we can choose an upper bound on the age,
M …>> s1 †, as illustrated in Fig. 1, beyond which
the system is automatically replaced. Bounding the
state space in this way, we can reasonably assume
a ®nite, discrete state space I ˆ f…n; i† j 1 6 n 6 N ;
0 6 i 6 Mg. The form of a typical control-limit
policy being sought is sketched in Fig. 1.
2.2. Transition probabilities and times
Since decision epochs are limited to instances of
machine failure, only policies of a control limit
form need be considered Ref. [7]. Notice that this
eliminates the type of policy analyzed in Kijima [5]
wherein, the system is replaced at T, not the ®rst
failure after T. The state transition graph in Fig. 1
illustrates the possible transitions among the
states. We can identify the action to be undertaken
by the location of the system at failure. For any
state …n; i†:
Failure in Region 1: …n < N ; i < sn †. Action:
a ˆ 1 (repair) Transition: …n; i† ! …n ‡ 1; j† where
i 6 j 6 M. Here we are below the maximum num-
ber of failures, N, and below the control limit.
401
Failure in Region 2: …n < N ; i P sn †. Action:
a ˆ 0 (replace) Transition: …n; i† ! …1; j† where
0 6 j 6 M. Here we are below the maximum
number of failures but above the control limit.
Failure in Region 3: …N ; i†. Action: a ˆ 0 (re-
place) Transition: …N ; i† ! …1; j† where 0 6 j 6 M.
Here we are have reached the maximum number of
failures.
In Fig. 1, the control limit policy we seek to
establish is marked by the heavy line, sn . Region 1
denotes those states in which, if a failure occurs,
we conduct a Type I imperfect repair. A transition
that moves the machine into Region 2 (where it
fails) results in a renewal. Finally transitions oc-
curing in Region 3 denote where an upper bound
on number of failures has been reached and again
result in a renewal (replacement). It is of course
possible for a machine to make a transition from
state …0; 0†, a renewed state, and reach age M
without incurring a failure. However, in our search
algorithm below, we would expand the state space
(in the time dimension) such that the probability of
such an event occurring is e€ectively 0. (That is, we
ensure M >> s1 P s2 P


P sN .)
A typical sample path is marked in Fig. 1.
Notice that two lines are placed in Fig. 1. The
dotted line is the sample path in terms of (real age,
number of failures). The solid line below it is the
sample path in terms of (virtual age, number of
failures). A description of how such a sample path
is generated is included in the ®gure.
To construct an SMDP model, let xn represent
the discrete sojourn time between the …n ÿ 1†th
and the nth failure. We will assume here that the
distribution of xn depends only upon the failure
rate function and the imperfect repair model used,
noting that our structure would permit failure
rates to be state dependent if we desired a more
general process. For Type I imperfect repairs, the
probability density function (p.d.f.) of xn‡1 is the
conditional p.d.f. of the time to ®rst failure in a
replacement cycle. The condition is on the virtual
age after the repair which is vn ˆ htn where tn is the
real age (chronological time) at nth failure. In our
discrete Markovian model, we use vn ˆ hi=n. (If
the determination of the virtual age of the system
dependsPn upon the complete history, as in
fvn ˆ jˆ1 hj ij =ng, then, of course, we lose the
402 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409

Fig. 1. Control limit policy and sample failure/repair path.

 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409 403

Markovian property of the system and the di- ®rst failure after the maximum age (M). The
mensionally of the problem would become un- probability of making such as transition is
manageable.)
Z1
If at state …n; i† a repair action …a ˆ 1† is taken,
P…n;i†…1;M† …a ˆ 0† ˆ f …x† dx: …7†
the (discretized) p.d.f. of xn‡1 can be written as
M

f …x ‡ hi=n† The expected one-step transition times (mean

f…hi†=n …x† ˆ ; …3†
F …hi=n† durations) for the two possible actions (repair,
replacement) are:
where f …x† is the p.d.f. of the time to ®rst failure, Z1
and F …x† is the survival function. The transition
s…n;i† …a ˆ 1† ˆ xf……hi†=n† …x† dx; …8†
probabilities generated when the system begins
in each of the three regions are de®ned as 0

follows. First we have transitions which begin

Z1
and strictly end within the state space de®ned for
the analysis. s…n;i† …a ˆ 0† ˆ xf …x† dx: …9†
0
P…n;i†…n‡1;j† …a ˆ 1†
In Section 3, a numerical search algorithm to
Z
jÿi‡1
determine the optimal control-limit policy is
ˆ f……hi†=n† …x† dx; n 6 N ÿ 1; i 6 j 6 M ÿ 1: developed.
jÿi

We also have transitions arising when the sys-
tem, following a repair, restarts in Region 1 but
next fails on or beyond the maximum age M.
These are de®ned by
Z1
P…n;i†…n‡1;M† …a ˆ 1† ˆ f……hi†=n† …x† dx;
Mÿi
Notice here that we have consolidated all cases
where j P M into one of the states (n; M), such that
the failures occuring beyond M are put at M.
Next we have a machine that has traversed into
Regions 2 or 3 and fails. This initiates a replace-
ment. One case is that the machine on replace-
…4†
3. A search algorithm
Utilizing state transitions de®ned by Eqs. (4)±
(7) and mean durations de®ned by Eqs. (8) and
(9), it is clear that we have a semi-Markov state
transition process. Since in each state, a decision to
either repair or replace must be taken, the resul-
n 6 N ÿ 1:
tant structure is a uni-chain, SMDP. Chose an
initial set of sn values. De®ne this set as an initial
…5† policy (R). For this arbitrary initial feasible policy
(R), solving the following set of simultaneous
equations will yield the value of such a policy in
terms of the cost per unit time (real age):
w…n; i† ˆ C1 …n; i† ÿ g…R†s…n;i† …1†
X
M

ment, has its ®rst failure before the maximum age ‡ P…n;i†…n‡1;k† …1†w…n ‡ 1; k†; …10†
(M) is reached. This probability is kˆi

Z
j‡1 where i 6 sn ÿ 1; 1 6 n 6 N ÿ 1;
P…n;i†…1;j† …a ˆ 0† ˆ f …x† dx; 0 6 j 6 M ÿ 1: …6† w…n; i† ˆ C0 ÿ g…R†s…n;i† …0†
j
X
M
The other situation, again beginning with a ‡ P…n;i†…1;k† …0†w…1; k†; …11†
machine that has traversed into Regions 2 or 3 and kˆ0

failed, is that the machine on replacement has its with i P sn ; 1 6 n 6 N ÿ 1;

404 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409
w…N ; i† ˆ C0 ÿ g…R†s…N ;i† …0†
X
M ®nd if:
‡ P…N ;i†…1;k† …0†w…1; k†; …
kˆ0
w…1; 0† ˆ 0; …
where g…R† represents the gain or cost per unit
time of this policy and w…n; i† represents the value,
(relative to state …1; 0†), of beginning in state …n; i†
and following this policy for an in®nite amount of
time.
With this as an initial policy (over all states), we
can proceed to determine a policy improvement
algorithm to identify improved policies. One ap-
proach would be to proceed by determining, for
each state, whether it is optimal to repair (a ˆ 1)
or replace (a ˆ 0). However since optimal policies
must be of the control-limit (threshold) type, it is
clear that one can search directly for optimal
control limits (sn ). The following search algorithm
can be used to ®nd the parameter values of the
optimal control-limit policy.
3.1. Algorithm
Step 0: Choose an initial control limit policy R
(i.e. parameter values s1 P s2 P s3 P


P sN ÿ1 ).
Select a bound for the control limits, ensuring that
M >> s1 , and a limit on number of repairs, N.
(Note, we could choose separate bounds M1 ; M2 ;
M3 ; . . . ; MN for each repair number, although the
added complexity does not appear to be justi®ed
by any computational savings.)
Step 1: For the current control limit policy R,
compute g…R† and w…n; i† by solving the linear
equation system Eqs. (10)±(13). Set n ˆ 1.
Step 2: Test for state …n; sn †, where sn ˆ sn ÿ 1;
to ®nd if:
C0 ÿ g…R†s…n;sn † …0†
X
M C0 and the Gamma parameter, k:
‡ P…n;sn †…1;k† …0†w…1; k† < w…n; sn † …
kˆ0
provided sn 6ˆ 0. If sn ˆ 0 or Eq. (14) does not
apply for …n; sn †, then go to Step 3. Otherwise,
determine the smallest integer sn with 0 6 sn < sn
such that Eq. (14) holds for sn < sn and go to
Step 4.
Step 3: Test for state …n; sn †; where sn ˆ sn , to
12†
C1 …n; sn † ÿ g…R†s…n;sn † …1†
13† X M
‡ P…n;sn †…n‡1;k† …1†w…n ‡ 1; k† < w…n; sn †: …15†
kˆsn
If Eq. (15) does not apply for …n; sn †, then go to
Step 4. Otherwise, determine the largest sn with
sn < sn 6 M such that Eq. (15) holds. (If sn ˆ M,
enlarge the bound M and retest.)
Step 4: n ˆ n ‡ 1, if n 6 N ÿ 1 go to Step 2,
otherwise go to Step 5.
Step 5: The new control limit rule R consists of
 the
the parameters values s1 ; s2 ; . . . ; sN ÿ1 . If R ˆ R,
algorithm is stopped. Otherwise, go to Step 1 with
R replaced by R. (If sN ÿ1 6ˆ 0, enlarge the bound
N.)
With this algorithm, we can numerically deter-
mine the optimal control-limit (threshold-type)
policy for the replacement problem under general
repair. In Fig. 2, we present a ¯owchart of this
algorithm.
4. Numerical analysis
Makis and Jardine [7] by way of comparison
with analyses previously presented by Kijima [5]
assumed that the lifetime distribution of a new
system followed a Gamma distribution with
density, f …t† ˆ ka tRaÿ1 =C…a† exp …ÿkt†, distribution
kt
function, 1=C…a† 0 exp …ÿu† uaÿ1 du, and ®rst
mean passage time, a=k; where a is a shape pa-
rameter and k is a scale parameter. C…a† is the
gamma function with parameter a. Kijima, as well
as Makis and Jardine, assumed for the purposes of
their analyses that a ˆ k: They further assumed
C1 …n; i† ˆ 1 for all states …i P 0; n P 1†: They pre-
sented analyses varying both the replacement cost,
14† In order to structure this problem as a discrete
SMDP, both the transition probabilities (Eqs. (4)±
(7)) as well as the mean duration times (Eqs. (8)
and (9)) must be determined. In Appendix A we
develop these equations for the Gamma (as well as
the Weibull) failure process subject to Type I
imperfect repairs.
 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409 405

Fig. 2. Algorithm ¯owchart.

Following the parameter assumptions of of 1 (i.e. repair costs do not vary with n or i). In
Makis and Jardine as well as Kijima, we present Table 1, we provide a comparison of our discrete
analyses for the case of k ˆ 3, a ®xed replacement SMDP model with the g-renewal approach of
cost (C0 ) of 2 and a constant repair cost (C1 …n; i†) Makis and Jardine. These restrictions on repair
406 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409

Table 1 we assume a repair cost of the form …C1 …n; i† ˆ

a ˆ k ˆ 3; C0 ˆ 2; C1 …n; i† ˆ 1 1:5
p…i=n† ‡ 1†. Here p is a parameter to provide a
h Th W…Th † sn g…R† family of increasing cost curves. We ®x the re-
0.0 n/a n/a 30 1.036 pair e€ect at 0.8 …h ˆ 0:8† and to see the impact
0.1 3.69 1.399 3.6 1.391 of a rising cost curve on the optimal solution, we
0.2 2.52 1.529 2.45 1.518
select three values of p (0.05, 0.2, 0.5).
0.3 2.01 1.615 1.9 1.601
0.4 1.71 1.679 1.65 1.673 As in Table 1, for each case, a constant
0.5 1.51 1.729 1.50 1.721 threshold value (sn ) results, over all values of n
0.6 1.35 1.769 1.35 1.760 since the repair cost is a constant with respect to n.
0.7 1.23 1.802 1.225 1.792 For the second case …p ˆ 0:2†, the threshold values
0.8 1.13 1.829 1.15 1.819
switch between 0.75 and 0.8 due to end e€ects.
0.9 1.04 1.851 1.05 1.842
1.0 0.97 1.870 0.95 1.859 Running with a larger number of time slices would
no doubt eliminate such switching.
In Table 3 we assume a repair cost increasing
with the number of failures, of the form,
cost imply we are seeking a T-optimal policy (C1 …n; i† ˆ 0:1nr ‡ 1). The repair e€ect, h, is again
(Refs. [9,7]). set to 0.8. Results for two values of r (1.05 and 1.2)
Th is de®ned in the Makis and Jardine paper are presented.
as the optimum replacement time given a par- As expected now sn is not longer a constant but
ticular repair e€ect …h†. W…Th † is the minimum decreases towards 0 as n increases. Notice at low
cost per unit time determined via their procedure. failure numbers the threshold values are quite
For a h of 0 in Table 1, the threshold value of 30 similar because the repair cost is quite similar in
was selected (i.e. sn …0† ˆ 30† as the largest age this range. For higher failure numbers, n, a sepa-
used in the calculations. This e€ectively means ration in threshold values is apparent.
that if each repair is a renewal (h ˆ 0) then the
control limit is sn …0† ˆ 1. Such a renewal model
has an average cost per unit time, g…R†, of 1.036. 5. Conclusions
Makis and Jardine did not report a result for this
case. For a machine (system) subject to failures and
Note in Table 1, for each value of h, a constant which may either be replaced or receive a (Kijima)
threshold value, sn …h†, results, over all values of n Type I imperfect repair, we have proposed the use
since the repair cost is a constant. From Table 1 it of a discrete SMDP to determine optimal policies.
is clear that our discrete semi-Markov approach The structure of the decision model utilizes two
agrees quite closely with the results of Makis and state variables (real age, number of failures in-
Jardine. curred to date). Should a machine be repaired, the
An important advantage in using this SMDP time to next failure given any real age is a function
approach of course is to explore the situation of the virtual age of the unit, which can be de-
when the repair cost is not constant. In Table 2 termined from the knowledge of the unit's real age

Table 2
a ˆ k ˆ 3; C0 ˆ 2; C1 …n; i† ˆ p…i=n†1:5 ‡ 1
p s1 s2 s3 s4 s5 s6 s7 sa8 g…R†
0.05 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0 1.836
0.2 0.75 0.75 0.75 0.80 0.80 0.80 0.80 0 1.869
0.5 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0 1.903
a
With the assumed data, the largest failure number, N, needed was 8. Thus on the 8th failure the threshold real time is 0, implying
replacement.
 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409 407

Table 3
a ˆ k ˆ 3; C0 ˆ 2; C1 …n; i† ˆ 0:1nr ‡ 1
r s1 s2 s3 s4 s5 s6 s7 sa8 g…R†
1.05 1.15 0.80 0.55 0.35 0.20 0.10 0 0 1.839
1.2 1.15 0.80 0.60 0.45 0.30 0.20 0.15 0 1.836
a
With the assumed data, the largest failure number, N, needed was 8.

and the degree of repair …h†. We see that, in gen-
eral, the form of the optimal (repair/replacement)
policy is of the threshold type such that at the nth
failure, if the real age is above a critical value,
replace, otherwise undertake an imperfect repair.
We also see that this discrete SMDP is very ¯ex-
ible, permitting state-dependent cost structures for
both repair and replacement costs. In this paper
we have assumed the failure intensity depends
only on the virtual age of the machine. Although
we have not explored the issue in this paper, with
few changes, the failure intensity could be made to
depend on the state (age, number of failures) as
well.
We have also limited this paper to ®nding op-
timal T-plus policies [9]. It is clear however that
with few modi®cations to the transition probabil-
ities and duration times (Eqs. (4)±(9)) we could
determine optimal ®xed-T policies as well.
We have only addressed the case of Type I
imperfect repairs. Kijima also proposed a Type II
imperfect repair model in which the repair e€ect
served to operate on the full virtual age of the
system up to that time. Other virtual age models
are also possible and may be important in par-
ticular situations (see Refs. [3,6]). It is clear that
for Type II repairs, the semi-Markov construc-
Appendix A. Transition structure for failure pro-
cesses with Type I repairs
We assume that repairs are of Type I such that
the virtual age of the system following the nth re-
pair is htn : In order to discretized our process we
introduce a parameter n to represent the number
of time slices in a unit of time. With this parame-
ter, real age, tn is discretized into in non-overlap-
ping ages such that the discretized virtual age after
repair is denoted as hi=n (where here we have
suppressed the failure number, n for ease of pre-
sentation). Clearly then, increasing n allows us to
increase the state space along the time dimension
(in ) and to achieve any level of accuracy desired. Of
course there is no need to discretize n (the number
of failures to date) since it is a discrete valued
function.
Our numerical work in Section 4 utilizes the
Gamma distribution. Utilizing this n discretization
approach, Eq. (3) for the Gamma distribution,
becomes
f…hi=n† …x†
a 1 aÿ1
k
1 k …n …hi ‡ x†† exp …ÿ n …x ‡ hi††
ˆ R khi=n :
C…a† 1 ÿ C…a† exp …ÿu† uaÿ1 du
0

tion would require the introduction of a virtual The transition probabilities (Eqs. (4)±(7))
age variable into the state space. The general become:
formulation here then would require either two P…n;i†;…n‡1;j† …1† ˆ
state variables; real age and virtual age and
possibly a third state variable (number of failures R…a; …k=n†…j ‡ 1 ÿ …1 ÿ h†i†† ÿ R…a; …k=n†…j ÿ …1 ÿ h†i††
;
to date) in order to properly characterize the 1 ÿ R…a; …khi=n††
system. Kijima Type II imperfect repairs and
where R…a; x† is the
R x incomplete Gamma function
extensions appear to have considerable potential
de®ned by 1=C…a† 0 exp…ÿu† uaÿ1 dt;
in capturing repair e€ects. Research into the
construction and computation demands of ap- P…n;i†;…n‡1;M† …1†
propriate discrete SMDP models for this case are
1 ÿ R…a; kn …M ÿ …1 ÿ h†i†† R…a; khi
n
†
ongoing and the results will be presented in a ˆ ‡ ;
subsequent paper. 1 ÿ R…a; khi
n
† 1 ÿ R…a; khi
n
†
408 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409
   
k kj P…n;i†;…n‡1;j† …1†
P…n;i†;…1;j† …0† ˆ R a; …j ‡ 1† ÿ R a; ;
n n  a    a 
i …j ÿ i ‡ hi†
  ˆ exp kh exp ÿ k
kM n n
P…n;i†;…1;M† …0† ˆ 1 ÿ R a; :   a 
n …j ÿ i ‡ 1 ‡ hi†
ÿ exp ÿ k ;
n
The mean duration times (Eqs. (8) and (9)) are:
P…n;i†;…n‡1;M† …1†
2 a
khi
exp …ÿ khi†  a   a 
n 4 n n i …M ÿ i ‡ hi†
s…n;i† …1† ˆ ˆ exp kh exp ÿ k ;
1 ÿ R…a; khi
n
† kC…a† n n

3 P…n;i†;…1;j† …0†
       a    a 
a hi khi 5; i …j ‡ 1†
‡ ÿ  1 ÿ R a; ˆ exp ÿ k ÿ exp ÿ k ;
k n n n n
  a 
M
an P…n;i†;…1;M† …0† ˆ exp ÿ k :
s…n;i† …0† ˆ : n
k
Finally, the mean duration times (Eqs. (8) and
These transition probabilities and times for (9)) are:
the Gamma distribution can be used directly in
our search algorithm of Section 3 and are utilized s…n;i† …1†
a a
in Section 4 of the paper to determine optimal ˆ exp …khi=n† …n=ka†C…1=a†…1 ÿ R…1=a; …khi=n† ††
policies and optimal cost per unit time for the and
system.
One sees very often in the published literature s…n;i† …1†
the use of the Weibull distribution in reliabilities a a
ˆ exp …khi=n† …n=ka†C…1=a†…1 ÿ R…1=a; …khi=n† ††
studies. For example, Phelps [9] utilized this dis-
tribution in his comparisons of various repair/re- respectively.
placement policies under the assumption of
minimal repairs. Given its frequent application, in
the interests of completeness, we brie¯y present References
here the necessary transition structure for this
distribution as well. We assume the lifetime dis- [1] H. Ascher, H. Feingold, Repairable Systems Reliability:
Lecture Notes in Statistics, Marcel Dekker, New York,
tribution of a new system following a Weibull 1984, pp. 33±34.
a
distribution has density f …t† ˆ ka ataÿ1 exp …ÿ…kt† †; [2] C.L. Chaing, An Introduction to Stochastic Processes and
a
distribution function F …t† ˆ 1 ÿ exp…ÿ…kt† † and their Applications, Krieger Publishing, New York, 1980,
mean ®rst passage time of …1=k†…C…a ‡ 1†=a† pp. 172±206.
where k is the scale parameter of the Weibull, a is [3] R. Guo, C.E. Love, Simulating non-homogeneous Poisson
processes with proportional intensities, Naval Research
the shape parameter and again, C…† is the Gamma Logistics Quarterly 41 (1994) 507±522.
function. [4] M. Kijima, Some results for repairable systems with
Utilizing the same discretization approach, general repair, Journal of Applied Probability 26 (1989)
Eq. (3) for the Weibull distribution, becomes, 89±102.
aÿ1 a [5] M. Kijima, H. Morimura, Y. Suzuki, Periodical replace-
f…hi=n† …x† ˆ ka…k=n…x ‡ hi†† exp …ÿ…k=n…x ‡ hi††
a ment problem without assuming minimal repair, European
‡…khi=n† †. Journal of Operational Research 37 (1988) 194±203.
The transition probabilities equivalent to [6] C.E. Love, R. Guo, Simulation strategies to identify the
Eqs. (4)±(7) are: failure parameters of repairable systems under the in¯u-
 C.E. Love et al. / European Journal of Operational Research 125 (2000) 398±409 409

ence of general repair, Quality and Reliability Engineering
International 10 (1994) 37±47.
[7] V. Makis, A.K.S. Jardine, A note on optimal replacement
policy under general repair, European Journal of Opera-
tional Research 69 (1993) 75±82.
[8] E.J. Muth, An optimal decision rule for repair vs
replacement, IEEE Transactions on Reliability R26
(1977) 179±181.
[9] R.I. Phelps, Replacement policies under minimal repair,
Journal of the Operational Research Society 32 (1981)
549±554.
[10] R.I. Phelps, Optimal policy for minimal repair, Journal of
the Operational Research Society 34 (1983) 425±427.
[11] H.C. Tjims, Stochastic Models: An Algorithmic
Approach, Wiley, New York, 1994, pp. 218±248.