2018 IEEE 14th International Conference on Automation Science and Engineering (CASE)
Munich, Germany, August 20-24, 2018
Real-time Control of Maintenance on Deteriorating Manufacturing
System
Jing Huang, Student Member, IEEE, Qing Chang*, Member, IEEE, Jing Zou, Student Member, IEEE,
Jorge Arinez, Member, IEEE, and Guoxian Xiao, Member, IEEE
Abstract—In modern manufacturing systems, maintenance
operations are the key to improve machines’ reliability and
availability, and hence to improve system productivity and
quality. Maintenance can be roughly categorized to Corrective
Maintenance (CM) and Preventive Maintenance (PM). Since the
production system is highly stochastic and maintenance actions
could either be perfect or imperfect, it is a complex decision to
make on when, where, and which type of maintenance action
should be taken. In this paper, a maintenance control law is
proposed to schedule cost-effective maintenance, either CM or
PM, in a real-time fashion. The control cost consists of a resource
cost, and the immediate and potential production losses due to
the stoppage caused by the maintenance action. A data-driven
model is used to evaluate the production losses. A case study is
performed to demonstrate the effectiveness of the proposed
control method by comparing three different maintenance
policies.
Index Terms—Maintenance management, Deteriorating
system, Real-time control, Opportunity window, Data-driven
modeling
I. INTRODUCTION
Most of the manufacturing systems are subject to
deteriorations due to usage and aging of machines and
operations in real industry practices. Upon random failures,
machines will stop working unexpectedly, forcing the plant
management to react to the disruption passively. Such reactive
action is corrective maintenance (CM). The outcomes and
costs of such unexpected failures are very hard to predict and
sometimes could be far beyond control. To avoid such
uncontrollable situations, it is essential to conduct preventive
maintenance (PM) before the failure actually happens.
Effective and timely maintenance decision making is
definitely not a trivial matter in improving the competitiveness
of a manufacturing organization.
Optimal maintenance policies have been intensively
investigated during the past decades. Researchers have found
that in reality, the maintenance action, either CM or PM, is
not necessarily a complete replacement [1]. Three different
maintenance types are defined according to the improvement
Research supported by the National Science Foundation Grants CMMI
1351160 and 1435534.
J. Huang, Q. Chang* and J. Zou are with the Department of Mechanical
Engineering, Stony Brook University, Stony Brook, NY 11794, USA
(email: jing.huang.3@stonybrook.edu, qing.chang@stonybrook.edu;
jing.zou@stonybrook.edu)
J. Arinez and G. Xiao are with General Motors Research and
Development Center, Warren, MI 48090, USA (email:
jorge.arinez@gm.com; guoxiao.xiao@gm.com)
*Corresponding author
978-1-5386-3593-3/18/$31.00 ©2018 IEEE
degrees. A perfect maintenance action recovers the machine
to the status of “as good as new”. While a minimal
maintenance action only resumes the functionality of the
machine without changing its aging status. An imperfect
maintenance recovers the machine to an extent between the
two. Based on these maintenance types, a lot of maintenance
policies have been proposed for a single machine or a single-
unit system [2]. For instance, Liao et al. [3] used a search
algorithm to determine a reliability threshold, at which a
preventive maintenance action will be taken, thus maximizing
the availability of the machine. These studies provide us with
great insights about the machine reliability analysis.
However, as modern manufacturing systems are
characterized by their complex structures of strongly
interconnected machines and stochastic dynamics, existing
single-machine maintenance policies cannot be directly
applied to machines in a multi-stage production line. These
machines are interconnected with each other, and both the
upstream and downstream impacts need to be fully examined
[4]. As a result, any maintenance decision, either CM or PM,
should consider not only its effect on the single machine’s
degradation, but also the impact on the overall system.
To address the complexity, some simulation-based studies
have been conducted to find the optimal maintenance
schedule, especially for PM [5]. Based on simulation, Roux et
al. [6] optimized the interval of periodic PM to ensure a low
level of failures and minimize the unavailability of the system.
One drawback of this approach is that in multi-stage
manufacturing systems, the temporary stoppage of one
machine doesn’t necessarily lead to production losses. Arab
et al. [7] addressed this issue by incorporating remaining
reliability of machines and work-in-process inventories into
the simulation model to search for the optimal maintenance
schedule. However, intensive computing resource is required
to search in the huge solution space, especially when the
system is scaled up. Changes on the process and equipment,
which are norm in today’s manufacturing industry, will lead
to corresponding changes and reconstructions in the
simulation models, and subsequently a higher resource
consumption. Therefore, a systematic method is desired to
manage both CM and PM, through careful analysis of the
maintenance function and production dynamics [8].
However, in respect of the production analysis, most of the
existing methods are focused on steady-state performance of
the system, which could not be used to guide maintenance
decision-making in a real-time situation. Consequently, in
industrial practice, the plant management has to schedule PM
211
during non-production shifts, breaks, or other scheduled
downtime, which might incur unnecessary extra labor and
A. Notations and assumptions
overhead costs. The PM for machines, on the other hand, will
not be available until these breaks, resulting in a probability
of missing the best timing for maintenance and the existence
of under-maintained or over-maintained machines.
In order to resolve these issues discussed above, recently
researchers explored the hidden opportunities for preventive
maintenance during normal production time. Some studies
omitted the buffers between machines and proposed that a
downtime caused by one machine could be used to perform
preventive maintenance on the other non-failed machines [9],
[10]. Gu et al. [11] proposed that failure-induced starvation or
blockage time could be an opportunity for maintenance.
These opinions were originated from direct observations
rather than the underlying characteristics of the system, and
such opportunities could be very limited, especially in a well-
maintained system. Chang et al. [12] found that a downtime
event brings permanent production losses only when it
impedes (blocks or starves) the slowest machine in the line
and opportunity window of one machine is the largest
possible stoppage time that will not incur this kind of
permanent production losses. With the available real-time
data collected by distributed sensors, Zou et al. [13] was able
to develop a data-driven model for production system analysis,
in which the cost analysis of downtime events was fully
established. Further, this modeling method and concept of
opportunity window have been developed to address issues in
manufacturing systems, including energy saving and gantry
assignment [14]–[16]. However, these works placed their
emphasis on production dynamics modeling. There is still a
notable gap between maintenance decision-making and
production system analysis.
This paper is devoted to bridge the gap between machines’
reliability and production system dynamics, and facilitating
the real-time integrated production and maintenance decision-
making in complex manufacturing systems. In this paper, a
novel real-time maintenance control law is developed by
integrally considering real-time production status and
maintenance cost of various maintenance types. The
machines’ reliability and availability vary with different types
of maintenance such as perfect maintenance, imperfect
maintenance or minimal maintenance. Such variance will in
turn impact production system real-time status and is also
considered in the overall control function.
The remainder of this paper is organized as follows. In
Section II, we illustrate system assumptions and introduce
production system modeling and maintenance modeling. In
Section III, a control cost function is derived through the
analysis of maintenance action and its impact on the overall
system. In Section IV, the real-time maintenance control law
is proposed. A case study is implemented and demonstrated in
Section V. Conclusions and future works are presented in
Section VI.
212
II. SYSTEM MODELING
The following notations are used in this paper.
 𝑆𝑖 denotes the 𝑖 𝑡ℎ machine, where 1 ≤ 𝑖 ≤ 𝑀
 𝐵𝑖 denotes the 𝑖 𝑡ℎ buffer, where 2 ≤ 𝑖 ≤ 𝑀 . With
abuse of notations, 𝐵𝑖 is also used to denote the
maximum capacity of the each buffer.
 𝑇𝑖 denotes the cycle time of machine 𝑆𝑖
 𝑀∗ denotes the index of last slowest machine in the line
 𝑏𝑖 (𝑡) denotes the buffer level of buffer 𝐵𝑖 at time t
 𝑂𝑊𝑖 (𝑡) in the opportunity window of machine 𝑆𝑖 at
time 𝑡
 𝑐𝑝 is the profit per part
 𝑚 = 1𝑝, 2𝑝, 1𝑐, 2𝑐, 3𝑐 denotes the maintenance type,
where 1𝑝 and 2𝑝 denote perfect and imperfect
preventive maintenance respectively; 1𝑐, 2𝑐 𝑎𝑛𝑑 3𝑐
denote perfect, imperfect, and minimal corrective
maintenance respectively
 𝑒⃗𝑖𝑗 = (𝑖, 𝑚𝑖𝑗 , 𝑡𝑖𝑗 , 𝑑𝑖𝑗 ) , 𝑖 = 1, 2, … 𝑀 , , 𝑗 = 1, 2, … ,
denotes the 𝑗𝑡ℎ maintenance action on machine 𝑆𝑖 ,
which starts from time 𝑡𝑖𝑗 and lasts for 𝑑𝑖𝑗 units of time
and the maintenance type is 𝑚𝑖𝑗
 𝑃𝐿𝑖𝑗 and 𝑃𝑃𝐿𝑖𝑗 are the immediate production loss and
potential production loss incurred by the maintenance
action 𝑒⃗𝑖𝑗 respectively
 𝐹𝑃𝐿𝑖𝑗 is the production loss of first downtime event
after maintenance action 𝑒⃗𝑖𝑗
 𝜂 is a discount rate that relates 𝑃𝑃𝐿𝑖𝑗 to 𝐹𝑃𝐿𝑖𝑗
 Δ𝑡 is a look-ahead window used to predict the
production loss of the first subsequent downtime event
 𝐶𝑟𝑒𝑠
𝑚
, 𝑚 = 1𝑝, 2𝑝, 1𝑐, 2𝑐, 3𝑐 denotes the resource cost
of a maintenance action of each type
 𝑑𝑚 , 𝑚 = 1𝑝, 2𝑝, 1𝑐, 2𝑐, 3𝑐 denotes the duration of
maintenance action of each type
 𝑞𝑚 , 𝑚 = 1𝑝, 2𝑝, 1𝑐, 2𝑐, 3𝑐 denotes the improvement
factor of each maintenance type
 𝛼𝑖 and 𝛽𝑖 , 𝑖 = 1, 2, … , 𝑀 denotes the Weibull
distribution parameters of machine 𝑆𝑖
 𝑍𝑖𝑗 denotes the lifetime of machine 𝑆𝑖 after 𝑗𝑡ℎ
maintenance action
 𝑣𝑖𝑗 denotes the virtual age of machine 𝑆𝑖 immediately
after its 𝑗𝑡ℎ maintenance action
 𝑔𝑖 (𝑡) denotes the age of machine 𝑆𝑖 at time 𝑡
We make the following assumptions in this paper:
1) Each machine in the production line is a single
deteriorating unit and the failure time follows Weibull
 distribution. The failure rates of the machines are
mutually independent and increasing with time.
2) Corrective maintenance (CM) is taken upon failure;
preventive maintenance (PM) is taken when the
machine is still operational to prevent future failures.
3) A perfect maintenance action will recover the machine
“as good as new”. A minimal maintenance action will
only resume the machine without changing its aging
status.
4) An imperfect maintenance is recovering the machine
to somewhere between old and new. The recovery
effect is not stochastic and can be described by a
deterministic improvement factor.
5) The duration of each maintenance type is assumed to
be deterministic.
6) CM could either be perfect, imperfect or minimal; PM
could either be perfect or imperfect.
B. Production system model
A serial production line consists of 𝑀 machines and 𝑀 − 1
buffers (Fig. 1). Each machine has a rated speed 1/𝑇𝑖 and
each buffer has a finite capacity.
Figure 1. System structure of the serial production line
A data-driven modeling method for the manufacturing
system is used in this paper[14]. The manufacturing system is
modeled as a stochastic dynamic system:
𝑿̇ = 𝑭(𝑿(𝑡), 𝑼(𝑡), 𝑾(𝑡)) (1)
where the state 𝑿(𝑡) ∈ 𝑅𝑀 is the production count of each
machine up to time 𝑡 , 𝑼(t) is the control input, i.e. the
actively shutdown to perform preventive maintenance, and
𝑾(𝑡) is the random failures of the machines. Given the
control input and random failures of the system, the system
state can be calculated recursively based on our previous
studies[14]. In addition, the buffer level of a certain buffer
𝐵𝑖+1 at any time 𝑡 can be easily evaluated as
𝑏𝑖+1 (𝑡) = 𝑋𝑖 (𝑡) − 𝑋𝑖+1 (𝑡) + 𝑏𝑖+1 (0) (2)
Thus the buffer levels of the production line at time 𝑡 ,
denoted as 𝒃(𝑡) = [𝑏2 (𝑡), 𝑏3 (𝑡), … , 𝑏𝑀 (𝑡)] are obtained.
C. Maintenance modeling
The reliability of machine 𝑆𝑖 is described by a Weibull
model with parameter 𝛼𝑖 and 𝛽𝑖 . 𝛼𝑖 is known as the
characteristic lifetime or scale parameter, while 𝛽𝑖 is known
as the shape parameter. The shape parameter 𝛽𝑖 determines
the aging mode of the machine. When 𝛽𝑖 = 1, the machine is
not aging with time at all, since the distribution is simplified
as an exponential distribution. When 𝛽𝑖 > 1, the machine is
aging with time, i.e. the machine is more likely to fail as time
increasing. The latter case is of interest in this paper.
The failure rate of machine 𝑆𝑖 is continuously increasing
213
with time until it receives a preventive maintenance or breaks
down and a corrective maintenance has to be imposed. As Fig.
2 shows, the machine receives its 𝑗𝑡ℎ maintenance (either CM
or PM) at time 𝑡𝑖𝑗 with a duration of 𝑑𝑖𝑗 , and at time 𝑡𝑖𝑗 + 𝑑𝑖𝑗
the machine resumed operation.
Figure 2. Maintenance actions on machine 𝑆𝑖
Immediately after the 𝑗𝑡ℎ maintenance action, the machine
gains a virtual age 𝑣𝑖𝑗 depending on both the maintenance
type and the age before the maintenance action[1]. Let 𝑞𝑚𝑖𝑗
be the improvement factor of the 𝑗𝑡ℎ maintenance action, then
𝑣𝑖𝑗 = 𝑞𝑚𝑖𝑗 (𝑣𝑖(𝑗−1) + 𝑍𝑖(𝑗−1) ) (3)
where 𝑣𝑖(𝑗−1) is the virtual age of machine 𝑆𝑖 immediately
after (𝑗 − 1)𝑡ℎ maintenance action, 𝑍𝑖(𝑗−1) is its survival time
after (𝑗 − 1)𝑡ℎ maintenance action, and the summation
𝑣𝑖(𝑗−1) + 𝑍𝑖(𝑗−1) is the age of machine 𝑆𝑖 when it is about to
receive 𝑗𝑡ℎ action. Obviously, 𝑞𝑚𝑖𝑗 = 0 corresponds to a
perfect maintenance since the virtual age is reduced to zero,
𝑞𝑚𝑖𝑗 = 1 corresponds to a minimal maintenance and 0 <
𝑞𝑚𝑖𝑗 < 1 relates to an imperfect maintenance.
At any time 𝑡 after the maintenance action, the age 𝑔𝑖 (𝑡) of
machine 𝑆𝑖 is a summation of its virtual age and actual
surviving time.
𝑔𝑖 (𝑡) = 𝑣𝑖𝑗 + [𝑡 − (𝑡𝑖𝑗 + 𝑑𝑖𝑗 )], 𝑡 ≥ 𝑡𝑖𝑗 + 𝑑𝑖𝑗 (4)
∗
Let 𝑝𝑖 (𝑡, 𝑡 ) denotes the probability density function of the
time to failure 𝑡 ∗ of machine 𝑆𝑖 evaluated at time 𝑡. It follows
Weibull distribution conditioning on the age of machine 𝑆𝑖 .
𝑝𝑖 (𝑡, 𝑡 ∗ ) = 𝑝𝑖 (𝑡 ∗ |𝑔𝑖 (𝑡)), 𝑡 ≥ 𝑡𝑖𝑗 + 𝑑𝑖𝑗 , 𝑡 ∗ ≥ 0 (5)
More specifically
𝛽𝑖 𝑡 ∗ +𝑔𝑖 (𝑡) 𝛽𝑖 −1 𝑡 ∗ +𝑔𝑖 (𝑡) 𝛽𝑖
𝑝𝑖 (𝑡, 𝑡 ∗ ) = ( ) exp (− ( ) +
𝛼𝑖 𝛼𝑖 𝛼𝑖
𝑔𝑖 (𝑡) 𝛽𝑖
( ) ) , 𝑡 ≥ 𝑡𝑖𝑗 + 𝑑𝑖𝑗 , 𝑡 ∗ ≥ 0 (6)
𝛼𝑖
III. CONTROL COST FUNCTION
A. Control cost function
To determine when, where and what type of maintenance
actions should be take on to minimize the overall system cost,
a control cost function is desirable for maintenance control.
In this paper, we focus on maintenance action, and the cost of
𝑗𝑡ℎ maintenance action on machine 𝑆𝑖 is given as
 𝑚𝑖𝑗
𝐶𝑖𝑗 = 𝐶𝑟𝑒𝑠 + 𝑐𝑝 (𝑃𝐿𝑖𝑗 + 𝑃𝑃𝐿𝑖𝑗 )
where
𝑚𝑖𝑗
𝐶𝑟𝑒𝑠 is the cost of resource used during maintenance,
including part replacement and other consumable expenses. It
varies with the maintenance type. Generally, the resource cost
of a perfect maintenance is larger than an imperfect
maintenance, and the resource cost of an imperfect
maintenance is larger than that of a minimal one. The second
term is the profit loss due to the maintenance action that
causes unavailability of the machine. 𝑃𝐿𝑖𝑗 and 𝑃𝑃𝐿𝑖𝑗 are the
immediate production loss and future potential production
loss respectively, and they will be discussed in following
sections.
Since the cost is one-time cost, it is unfair to compare the
costs between different maintenance type options. The cost of
a perfect maintenance action might be much higher than an
imperfect maintenance action, but the former one will ensure
the system to operate a longer period of time before failure.
Hence a cost rate function 𝐶𝑅𝑖 is given by unifying the cost
along the expected value of lifetime 𝑍𝑖𝑗 after the action, i.e.
𝑚𝑖𝑗
𝐶𝑟𝑒𝑠 +𝑐𝑝 (𝑃𝐿𝑖𝑗 +𝑃𝑃𝐿𝑖𝑗 )
𝐶𝑅𝑖 (𝑡) = , 𝑡 ∈ [𝑡𝑖𝑗 , 𝑡𝑖(𝑗+1) )
𝐸[𝑍𝑖𝑗 ]
where 𝐸[𝑍𝑖𝑗 ] can be evaluated by the expectation of
(conditional) Weibull distribution.
𝑔𝑖𝑗 𝛽𝑖 ∞ 𝑡 ∗ +𝑔𝑖𝑗 𝛽𝑖
𝐸[𝑍𝑖𝑗 ] = exp (( ) ) ∫0 exp (( ) ) 𝑑𝑡 ∗
𝛼𝑖 𝛼𝑖
The real-time cost rate of the whole production line is
𝐶𝑅(𝑡) = ∑𝑀 (10)
𝑖=1 𝐶𝑅𝑖 (𝑡)
This cost rate function is used as control cost function to
guide maintenance decision making.
B. Immediate production loss evaluation
Any maintenance action taken on a machine will require
the specific machine to stop operating for a period of time. A
stoppage is directly counted toward unavailability time and
integrated into cost function in most related works. However,
Chang et al. found that the stoppage incurs permanent
production loss only if it impedes (blocks or starves) the last
slowest machine in the line[12]. In other words, not all the
stoppages cause permanent production loss.
The largest possible stoppage time of machine 𝑆𝑖 that
won’t lead to permanent production loss is referred to as
opportunity window, denoted as 𝑂𝑊𝑖 .
𝑇 The probability associated with 𝑃𝐿𝑘∗
𝑂𝑊𝑖 (𝑇𝑑 ) = sup {𝑑 ≥ 0: 𝑠. 𝑡. ∃𝑇 ∗ (𝑑), ∫0 𝑠𝑀 (𝑡)𝑑𝑡 =
𝑇
∫0 𝑠̃ (𝑡; 𝑒⃗)𝑑𝑡 , ∀𝑇 ≥ 𝑇 ∗ (𝑑)}
𝑇 𝑇
where ∫0 𝑠̃𝑀 (𝑡; 𝑒⃗)𝑑𝑡 and are the production
∫0 𝑠𝑀 (𝑡)𝑑𝑡
volume of the end-of-line machine 𝑆𝑀 at time 𝑇, with and
without disruption event 𝑒⃗ = (𝑖, 𝑚, 𝑡, 𝑑), respectively. 𝑇 ∗ (𝑑)
signifies the potential dependency of 𝑇 ∗ on 𝑑.
Given a maintenance action 𝑒⃗𝑖𝑗 = (𝑖, 𝑚𝑖𝑗 , 𝑡𝑖𝑗 , 𝑑𝑖𝑗 ) taken on
machine 𝑆𝑖 , the immediate permanent production loss is
𝑑𝑖𝑗 −𝑂𝑊𝑖 (𝑡𝑖𝑗)
𝑃𝐿𝑖𝑗 = max { , 0}
𝑇 𝑀∗
where 𝑑𝑖𝑗 is the duration of the maintenance action, 𝑂𝑊(𝑡𝑖𝑗 )
(7) is the opportunity window evaluated at time 𝑡𝑖𝑗 when action
is taken and 𝑇𝑀∗ is the cycle time of the slowest machine 𝑆𝑀∗ .
C. Potential production loss evaluation
The size of opportunity window heavily relies on the buffer
levels (if upstream) or buffer vacancies (if downstream)
between the machine of interest and the slowest machine. The
opportunity window accumulates when the machine operates
faster than the slowest machine, and shrinks when the
machine stops while the slowest machine operates.
Given a maintenance action 𝑒⃗𝑖𝑗 imposed on machine 𝑆𝑖 , the
stoppage will propagate to nearby machines sequentially
since it will cause blockage or starvation. The opportunity
windows of nearby machines gradually shrink. Considering
subsequent random failures on these machines, the permanent
production losses may be amplified by the reduced
opportunity windows. In principle, these losses cannot be
directly attributed to the initial action 𝑒⃗𝑖𝑗 , but 𝑒⃗𝑖𝑗 does
indirectly contribute to the losses of subsequent failures.
Therefore, a maintenance action may alter the health status of
the overall system to some extent. To incorporate this impact,
(8)
we refer the difference of future production losses with and
without 𝑒⃗𝑖𝑗 as potential production loss, denoted as 𝑃𝑃𝐿𝑖𝑗 .
As the manufacturing system is highly nonlinear and
stochastic, it is nearly impossible to have a precise expression
(9) for the potential production loss. Referring to Zou et al.[17],
the production losses of the very first random failure event
within a short look ahead window Δ𝑡 is taken as an indicator.
𝑃𝑃𝐿𝑖𝑗 = 𝜂 × 𝐹𝑃𝐿𝑖𝑗 (13)
where 𝐹𝑃𝐿𝑖𝑗 is the production loss incurred by the very first
random failure of the whole line after the maintenance action
𝑒⃗𝑖𝑗 . A proper discount ratio 𝜂 can be given based on past
experience.
Suppose that after time 𝑡𝑖𝑗 , the very first random failure 𝑒⃗𝑘∗
occurs on machine 𝑆𝑘 (𝑘 = 1, 2, … , 𝑀) at time 𝑡 ∗ and a
perfect corrective maintenance is taken. Since there is no
other random failure between time 𝑒⃗𝑖𝑗 and 𝑒⃗𝑘∗ , the buffer
levels 𝒃(𝑡𝑖𝑗 + 𝑡 ∗ ) can be exactly computed and 𝑂𝑊𝑘 (𝑡𝑖𝑗 +
𝑡 ∗ ) can be computed. Let 𝑃𝐿𝑘∗ (𝑡 ∗ ) denotes the production
loss incurred by 𝑒⃗𝑘∗ , then
𝑑1𝑐 −𝑂𝑊𝑘 (𝑡𝑖𝑗+𝑡 ∗ )
𝑃𝐿𝑘∗ (𝑡 ∗ ) = max { , 0} (14)
𝑇 𝑀∗
(𝑡 ∗ )
is the probability
∗
𝑝(𝑘, 𝑡𝑖𝑗 , 𝑡 ) that the very first random failure occurs on
(11) machine 𝑆𝑘 at time 𝑡 ∗ from time 𝑡𝑖𝑗 . The machines are
independent with each other regarding reliability. Therefore
𝑝(𝑘, 𝑡𝑖𝑗 , 𝑡 ∗ ) is the joint probability of 𝑀 machines, i.e.
𝑡∗
𝑝(𝑘, 𝑡𝑖𝑗 , 𝑡 ∗ ) = 𝑝𝑘 (𝑡𝑖𝑗 , 𝑡 ∗ ) ∏𝑀
𝑙=1,𝑙≠𝑘 [1 − ∫0 𝑝𝑙 (𝑡𝑖𝑗 , 𝜏)𝑑𝜏 ] (15)
Finally, 𝐹𝑃𝐿𝑖𝑗 can be evaluated by the expected production
losses.
Δt
𝐹𝑃𝐿𝑖𝑗 = ∑𝑀 ∗ ∗
𝑘=1 ∫0 𝑃𝐿𝑘∗ (𝑡 )𝑝(𝑘, 𝑡𝑖𝑗 , 𝑡 )𝑑𝑡
∗
(16)
(12) where Δ𝑡 is the look-ahead window.
214
 IV. REAL-TIME MAINTENANCE CONTROL LAW machines and 5 buffers (Fig. 2).
The maintenance cost heavily depends on the real-time
state of the production line. However, it is nearly impossible
to completely predict the state at any future time. It is
extremely difficult to find an optimal maintenance control
policy for a global time horizon. A feasible proposal is to
develop a control scheme in real-time fashion to optimize
maintenance cost rate, thus obtaining a near optimal
maintenance schedule. Our control objective is to minimize
the real-time cost rate, i.e. Figure 2. System structure of the serial production line
min ∑𝑁 𝑖 𝐶𝑅𝑖 (𝑡) (17)
𝑚
Let 𝐶𝑅𝑖 (𝑡), 𝑚 = 1𝑝, 2𝑝, 1𝑐, 2𝑐, 3𝑐 denotes the cost rate The system parameters are shown in Table I. The profit per
assuming a maintenance action of type 𝑚 is taken on machine part is assumed to be 𝑐𝑝 = $300/𝑝𝑎𝑟𝑡. In this case study, the
𝑆𝑖 at time 𝑡. Then 𝐶𝑅𝑖𝑚 (𝑡) can be evaluated by inserting a look ahead window for future potential production loss
corresponding maintenance action 𝑒⃗𝑖∗ = (𝑖, 𝑚, 𝑡, 𝑑𝑚 ) into Eq. estimation is Δ𝑡 = 100 𝑚𝑖𝑛 and discount ratio is selected as
(8). The control law incorporates both CM and PM control 𝜂 = 0.3. The time step for deciding preventive maintenance
procedures. is 𝜎 = 25 𝑚𝑖𝑛. Maintenance parameters are given in Table II.
1) CM control procedure The case study is carried out in simulation, and the duration
If machine 𝑆𝑖 fails at time 𝑡, a CM action has to be imposed is four weeks, i.e. 𝑇 = 40320 𝑚𝑖𝑛 . The maintenance
in order to resume the machine. The eligible maintenance parameters are presented in Table II.
types are 1𝑐, 2𝑐, and 3𝑐. Then the maintenance type chosen
is the one that minimize the real-time cost rate of machine 𝑆𝑖 , TABLE I. PARAMETERS FOR THE PRODUCTION LINE
i.e. 𝑆1 𝑆2 𝑆3 𝑆4 𝑆5 𝑆6
𝑚𝑖 = arg min{𝐶𝑅𝑖𝑚 (𝑡), 𝑚 = 1𝑐, 2𝑐, 3𝑐} (18) Cycle time 𝑇𝑖 (𝑚𝑖𝑛) 0.92 0.88 1.0 0.87 0.87 0.92
𝑚
Initial age 𝑣𝑖 (𝑚𝑖𝑛) 100 400 20 300 0 50
2) PM control procedure
A small time step 𝜎 is chosen, which is much smaller than Characteristic life 𝛼𝑖 (𝑚𝑖𝑛) 680 720 900 800 700 750
the lifetimes of machines. If machine 𝑆𝑖 is operational at Shape parameter 𝛽𝑖 2 2 2 2 2 2
current step, then machine 𝑆𝑖 is eligible for a PM. The cost 𝐵2 𝐵3 𝐵4 𝐵5 𝐵6
rate of PM is 𝐶𝑅𝑖𝑚 (𝑡), 𝑚 = 1𝑝, 2𝑝. Buffer capacity 𝐵𝑖 20 20 20 20 20
We can also decide not to take any action on the machine. Initial buffer level 𝑏𝑖 (0) 2 3 5 8 2
Then the machine is threatened to fail at next step with
probability 𝑃𝑖 (𝑡, 𝜎).
𝜎
𝑃𝑖 (𝑡, 𝜎) = ∫0 𝑝𝑖 (𝑡, 𝑡 ∗ )𝑑𝑡 ∗ (19) TABLE II. PARAMETERS FOR MAINTENANCE ACTIONS
The future system state at time 𝑡 + 𝜎 is uncertain. In order 1𝑝 2𝑝 1𝑐 2𝑐 3𝑐
to evaluate the production losses at that time, one can use the Resource cost 𝐶𝑟𝑒𝑠 (𝑈𝑆 𝑑𝑜𝑙𝑙𝑎𝑟) 120 70 200 100 50
worst estimate of opportunity window of machine 𝑆𝑖 at time Duration 𝑑𝑚 (𝑚𝑖𝑛) 15 10 35 20 10
𝑡 + 𝜎 , which is the situation where machine 𝑆𝑖 stops Improvement factor 𝑞𝑚 0 0.7 0 0.7 1
operating for 𝜎 units of time. Thus upon the potential failure
at next time step, we can find the minimum 𝐶𝑅𝑖𝑚 (𝑡 + 𝜎), 𝑚 =
Three policies are compared using simulation, i.e.
1𝑐, 2𝑐, 3𝑐 . Specially, let 𝐶𝑅𝑖0 (𝑡) denotes the expected CM Policy 1: failure limit policy [18]
cost rate if we take no action at current step and machine 𝑆𝑖 A perfect PM action ( 1𝑝 ) will be imposed once the
fails at next step, then machine reaches a pre-determined failure rate threshold 𝜆,
𝐶𝑅𝑖0 (𝑡) = 𝑃𝑖 (𝑡, 𝜎) ⋅ min{𝐶𝑅𝑖𝑚 (𝑡 + 𝜎), 𝑚 = 1𝑐, 2𝑐 ,3𝑐} (20) and failures before that will be corrected with minimal
The maintenance decision 𝑚𝑖 at current step is CM (3𝑐).The threshold is chosen as 𝜆 = 1/400.
𝑚𝑖 = arg min{𝐶𝑅𝑖𝑚 (𝑡), 𝑚 = 1𝑝, 2𝑝, 0} (21) Policy 2: empirical policy
𝑚
Policy 2 is to mimic the common practice in current
V. CASE STUDY industry, where preventive maintenance will only be
In order to demonstrate the effectiveness of the real-time conducted between shifts. Given a failure rate threshold
maintenance control law, a case study is presented to compare 𝜆 = 1/400, a machine will receive perfect PM (1𝑝) if it
reaches the failure limit between shifts. If the machine
the overall system profit under three maintenance policies.
fails during shift, the maintenance action is perfect CM
The overall profit over the whole production horizon 𝑇 is
(1𝑐) if the machine exceeds the failure limit threshold or
roughly calculated as:
minimal CM (3𝑐) if it is under the threshold. The shift
𝑂𝑣𝑒𝑟𝑎𝑙𝑙 𝑃𝑟𝑜𝑓𝑖𝑡 = 𝑐𝑝 ⋅ 𝑋𝑀 (𝑇) − 𝑀𝑎𝑖𝑛𝑡𝑒𝑛𝑎𝑛𝑐𝑒 𝐶𝑜𝑠𝑡𝑠 (22) length is 8 hours and duration of the PM break between
The production line used in this case study consists of 6 shifts is 30 𝑚𝑖𝑛.

215
 Policy 3: control law proposed in this paper.
Since PM is conducted during production time under
Policy 3, there will be no breaks between shifts during the
simulation. This is also one of the major benefits of the control
scheme proposed in this paper.
The final results of the overall profit from three policies are
shown in Table III. It is clear that the overall profit of the
production system is well increased with the control proposed
in this paper. Comparing with Policy 1 and Policy 2, the
overall profit has been improved by 25.8% and 15.5%
respectively. The control law proposed for both CM and PM
in deteriorating production line has increased the overall
profit significantly.
TABLE III. RESULTS FOR MAINTENANCE ACTIONS
Policy 1 Policy 2 Policy 3
Overall profit
average 9,156,117.92. 9,980,446.82 11,522,315.49
(𝑈𝑆 𝑑𝑜𝑙𝑙𝑎𝑟)
Overall profit
[8988002.12, [9,841,137.85, [11,520,036.96,
95% CI
9303020.05] 10,095,282.76] 11,525,556.29]
(𝑈𝑠 𝑑𝑜𝑙𝑙𝑎𝑟)
VI. CONCLUSION
In this paper, a real-time control scheme is developed
regarding the maintenance decision-making in complex
manufacturing systems. The cost rate function of a
maintenance action is established by incorporating both
resource cost and production losses due to machine stoppage.
The CM decision is made through finding the minimum real-
time cost rate. The PM decision is made through comparing
the cost rate of PM and expected CM cost rate when
responding to the potential failure at next time step. This
control law is proved to be effective through case study.
In the future, we will work to consider the stochastic
durations and effects of maintenance actions. Future work
will take constraints from maintenance staff number and skill
levels into consideration.
ACKNOWLEDGMENT
This work was supported by the U.S National Science
Foundation (NSF) Grant No. CMMI 1351160 and 1435534.
REFERENCES
[1] H. Pham and H. Wang, “Imperfect maintenance,” Eur. J. Oper. Res.,
vol. 94, no. 3, pp. 425–438, 1996.
[2] H. Wang, “A survey of maintenance policies of deteriorating
systems,” Eur. J. Oper. Res., vol. 139, no. 3, pp. 469–489, Jun. 2002.
[3] H. Liao, E. A. Elsayed, and L.-Y. Chan, “Maintenance of
continuously monitored degrading systems,” Eur. J. Oper. Res., vol.
175, no. 2, pp. 821–835, Dec. 2006.
[4] J. Zou, Q. Chang, Y. Lei, G. Xiao, and J. Arinez, “Stochastic
Maintenance Opportunity Windows for Serial Production Line,” in
Volume 2: Materials; Biomanufacturing; Properties, Applications
and Systems; Sustainable Manufacturing, 2015, p. V002T04A007.
[5] A. Alrabghi and A. Tiwari, “State of the art in simulation-based
optimisation for maintenance systems,” Comput. Ind. Eng., vol. 82,
pp. 167–182, Apr. 2015.
[6] O. Roux, D. Duvivier, G. Quesnel, and E. Ramat, “Optimization of
preventive maintenance through a combined maintenance-production
216
simulation model,” Int. J. Prod. Econ., vol. 143, no. 1, pp. 3–12, May
2013.
[7] A. Arab, N. Ismail, and L. S. Lee, “Maintenance scheduling
incorporating dynamics of production system and real-time
information from workstations,” J. Intell. Manuf., vol. 24, no. 4, pp.
695–705, 2013.
[8] P. Muchiri, L. Pintelon, L. Gelders, and H. Martin, “Development of
maintenance function performance measurement framework and
indicators,” Int. J. Prod. Econ., vol. 131, no. 1, pp. 295–302, May
2011.
[9] T. Xia, L. Xi, X. Zhou, and J. Lee, “Dynamic maintenance decision-
making for series-parallel manufacturing system based on MAM-
MTW methodology,” Eur. J. Oper. Res., vol. 221, no. 1, pp. 231–
240, 2012.
[10] N. Chalabi, M. Dahane, B. Beldjilali, and A. Neki, “Optimisation of
preventive maintenance grouping strategy for multi-component series
systems: Particle swarm based approach,” Comput. Ind. Eng., vol.
102, pp. 440–451, 2016.
[11] X. Gu, X. Jin, and J. Ni, “Prediction of Passive Maintenance
Opportunity Windows on Bottleneck Machines in Complex
Manufacturing Systems.,” J. Manuf. Sci. Eng., vol. 137, no. 3, pp.
31017–310179, 2015.
[12] Q. Chang, J. Ni, P. Bandyopadhyay, S. Biller, and G. Xiao,
“Maintenance Opportunity Planning System,” J. Manuf. Sci. Eng.,
vol. 129, no. 3, p. 661, 2007.
[13] J. Zou, Q. Chang, Y. Lei, and J. Arinez, “Production System
Performance Identification Using Sensor Data,” IEEE Trans. Syst.
Man, Cybern. Syst., vol. 48, no. 2, pp. 255–264, Feb. 2018.
[14] J. Zou, Q. Chang, J. Arinez, and G. Xiao, “Data-driven modeling and
real-time distributed control for energy efficient manufacturing
systems,” Energy, vol. 127, pp. 247–257, 2017.
[15] J. Arinez, X. Ou, and Q. Chang, “Gantry Scheduling for Two-
Machine One-Buffer Composite Work Cell by Reinforcement
Learning,” in Volume 4: Bio and Sustainable Manufacturing, 2017,
p. V004T05A025.
[16] X. Ou, Q. Chang, N. Chakraborty, and J. Wang, “Gantry Scheduling
for Multi-Gantry Production System by Online Task Allocation
Method,” IEEE Robot. Autom. Lett., vol. 2, no. 4, pp. 1848–1855,
Oct. 2017.
[17] J. Zou, Q. Chang, and Y. Lei, “Production Loss Diagnosis and
Prognosis Using Model-based Data-driven Method,” IFAC-
PapersOnLine, vol. 49, no. 12, pp. 1585–1590, 2016.
[18] Y. H. Lie, Chang Hoon; Chun, “An Algorithm for Preventive
Maintenance Policy,” IEEE Trans. Reliab., vol. 35, no. 1, pp. 71–75,
1986.