620

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 33,

NO. 7, JULY

1988

Production Control of a Manufacturing System with Multiple Machine States

ALI SHARIFNIA

Abstract-The production control of a single-product manufacturing system with arbitrary number of machine states (failure modes) is discussed. The objective is to find a production policy that would meet the demand for the product with minimum average inventory or backlog cost. The optimal production policy has a special structure and is called a hedging point policy. If the hedging points are known, the optimal production rate is readily specified. Assuming a set of tentative hedging points, we utilize the simple structure of the optimal policy to find the steady-state probability distribution of the surplus (inventory or backlog). Once this function is determined the average surplus cost is easily calculated in terms of the values of the hedging points. This average cost is then minimized to find the optimum hedging points.

I. INTRODUCTION HIS paper describes a method for solving the short-term production control problem of a failure prone manufacturing system. The system produces a single product and is characterized by a number of possible machine states. Each machine state corresponds to a failure mode of the system (some of the machines failed) and has a production possibility set. The objective is to determine an optimum production policy under each machine state, so that the demand is met with minimum surplus costs (surplus may be negative representing backlog). In their pioneering work Kimemia and Gershwin [l] consider manufacturing systems with an arbitrary number of machine states and product types. Formulating this problem as a dynamic programming problem they show that solving the HamiltonJacobi-Bellman equation for the optimum production policy in each machine state can be reduced to the solution of linear programs, provided the minimum cost-to-go function is known. The linear program has the form minimize V J ( x , a )

Gershwin, Akella, and Choong [2] apply the method to scheduling of a flexible manufacturing system using approximate values of the hedging points and the cost-to-go function. However, one difficulty with the implementation is the determination of the minimum cost-to-go and the hedging points. Akella and Kumar [3] find an exact solution for the case involving a single product and two machine states (up and down). Their analysis is quite complicated even for this simple case. In a recent work Bielecki and Kumar [4] use another approach for solving the single-product two-machine-state problem. Assuming that the hedging point (corresponding to the up state) is known, they calculate the steady-state probability density function of the surplus and the probability mass at the hedging point. The optimum hedging point is then found by minimizing the average cost per unit time over all possible values of the hedging point. This approach is simpler and more promising and is adopted in our work. In this paper we solve the single product problem with an arbitrary number of machine states. We use sample path analysis and time averaging to derive the probability density function (PDF) of the surplus. The ergodic hypothesis holds since state transitions are generated by a finite irreducible Markov chain. The single product case is of interest for its own sake, and also because it is a first step towards understanding the more general multiple product case in flexible manufacturing systems. A method has already been suggested by Kimemia and Gershwin [11 for decomposition of the multiple product case into single product problems. In Section I1 we give some definitions and discuss the basic concepts used in the paper. The steady-state probability density and mass functions of the surplus are derived in Section 111. Section IV describes how the average cost function is calculated and the optimum hedging points are determined. An example is given in Section V and we conclude in Section VI.

.U

subject to

E Q(a).

(1)

11. DEFINITIONS AND BASIC CONCEPTS

We consider a manufacturing system that produces a single product. The machines can be in a number of different failure 0 . * - ) and we assume that the machine state changes in modes (a, continuous time according to an irreducible Markov chain with known transition rates Aao. The surplus at time I is a random variable X ( t ) ,and there is a constant demand d, per unit time, for the product. Let g ( x ) denote the cost of maintaining a surplus x for one time unit. The objective is to find the optimum production rate U , for each machine state a and surplus level x , so as to minimize the average surplus cost per unit time
(2)

Here x is the inventory level, a is the machine state, V J ( x , a ) is the gradient of the minimum cost-to-go function, U is the is the set of production rate possibilities production rate, and Q(a) in machine state a.The approach provides production rates which are then translated into loading times of the parts into the system. A very interesting and useful result of [l] is the concept of hedging points. It is shown that for each feasible machine state (in which enough can be produced to keep up with demand), the optimum production policy will move the inventory level toward a specific target (the hedging point) as quickly as possible, and when at this point, it will keep it there as long as the machine state remains unchanged.
Manuscript received March 24, 1987; revised December 3, 1987 and January 21, 1988. This work was supported by the National Science Foundation under Grant DMC-8615560. Paper recommended by Past Associate Editor, S. G. Gershwin. The author is with the College of Engineering, Boston University, Boston, MA 02215. IEEE Log Number 8821357.

The machine states which are capable of producing the demand are called feasible machine states. We assume that the system is strictly feasible in the long run, in the sense that if we produce at maximum possible rate in each machine state, then the average production rate will exceed the demand rate d.

OO18-9286/88/07OO-0620$01.OO 0 1988 IEEE

11---

SHARIFNIA: MANUFACTURING SYSTEM WITH MULTIPLE MACHINE STATES

62 1

The solution of the problem defined above can be obtained from the linear program (1). Since we are using the average cost criterion ( J * ) , the functions J(x, a ) in ( 1 ) will be the value functions (differential costs). Since X i s scalar in our case, Q(a) is always the positive segment of the real line bounded by the maximum production rate in machine state a , denoted 6 ,
Q(a) = [O, ti,].

For each machine state a the value function J ( x , a ) reaches its minimum at a hedging point x = z,. The partial derivative U ( x , a ) / d x is positive for x > z, and is negative for x < zn. It follows that the optimum production rate for machine state a is u*(x, a ) =

ti, d 0

ifx<z,, ifx=z,, ifx>z,.

(3)

From the preceding discussion it follows that if the optimum hedging points (HPs) z, are known, the optimum policy is completely determined. To find the optimum HPs we follow the approach of Bielecki and Kumar [4]. First we find the distribution of X given tentative values of the HPs, then we calculate the average cost per period J* as a function of these tentative values, and finally we find the optimum HPs by minimizing the function J*. Once the optimum production policy is known we can derive the probability distribution P of X in steady state. For the density of the continuous part of P we write f ( x ) = CJ(x, a ) = C,aJ(xla). Here a , is the stationary probability of machine state a,f ( x l a )is the conditional density of X given that the machine state is a , andf(x, a ) is the joint probability distribution of x and a. The distribution P of X also has a finite number of point masses P(z,) located at the hedging points za of the feasible machine states. To find the steady-state probability distribution P we use time averaging of the sample path X ( t )over a long period of time. This approach turns out to be much simpler than calculating the probabilities in the ensemble space. The analysis of the sample paths is also found very useful in obtaining the boundary conditions needed to determine the density f ( x ) in regions between successive hedging points, and the probability masses at the hedging points which separate regions. At the heart of our derivation of the distribution P are the functions m(x, CY), which are defined as follows. Let us consider a time interval of length T and let M(x, a , T ) denote the number of times X ( t ) crosses the line X = x while the machine state is a (Fig. 1). We define
(4)

Fig. 1. A sample path of X ( t )

We can find f ( x , a ) in terms of m ( x , a ) , again by time averaging. Indeed, suppose the inventory level X ( t ) is observed over a long period of time T , and ~ ( x a , ) designates the accumulated sum of the short intervals during which X falls in (x, x + dx) and the machine state is a. Then

But ~ ( xa,) = M ( x , a , T ) .ldtl where dt is the amount of time necessary for the surplus X to sweep through the interval (x, x + d x ) while the machine state is a (see Fig. 1). Substituting in (5) and using (4) we obtain

f ( x , a)IdxI=m(x, a)ldtI
or

(6)

(7)

Here dx/dt is the velocity with which the surplus X ( t ) is changing when the machine state is a and the surplus is x. This velocity is denoted u(x, a) and is determined by the hedging point policy
dx -=u(x,
~ ) = U * ( X ,a ) - d =

The time average on the right-hand side converges with probability one to a unique limit, namely the expected frequency with which the surplus X ( t ) crosses the level X = x and the machine state is a , provided that X ( t ) is ergodic. The process X ( t ) is ergodic once the machine states form an irreducible Markov chain and the production policy is such that X ( t ) is stationary. For the hedging point policy X ( t ) will be stationary if the system is strictly feasible in the long run, i.e., a,ti,>d.
a

dt

ti,-d 0 -d

ifx<z,, if x=z, if x>z,.

The optimum velocity is constant as long as the machine state a remains unchanged and the surplus X falls in a region between successive hedging points. We write R for the region in between ith and (i + 1)st hedging points, and U: for the optimum velocity in region R. We find that
x E Ri.

The density f ( x ) is obtained by summing f ( x , a ) over all a

This condition implies that if we produce at maximum rate in each machine state, then there will be a strictly positive long term trend in X ( t ) . It is then clear that the smallest hedging point will be recurrent. However, since the hedging points are finitely separated, it follows that all the hedging points will be recurrent, and hence X ( t ) will be stationary.

In the next section we shall see that m(x, a ) is more tangible and easier to work with than f ( x , a). This is particularly true with

622
regard to the boundary conditions required for solving the problem.
OF THE SURPLUS LEVEL 111. DISTRIBUTION

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 33, NO. 7, JULY 1988

we obtain the system of linear differential equations

In this section we derive an equation for m(x, a) in each region. We also derive boundary conditions for these functions at the hedging points which delimit regions. The PDF of X will then be determined by (9). The machine failure states evolve according to a continuoustime Markov chain which is assumed irreducible. Let has be the rate of machine state transitions from a to # a,and let h, = ,A , = CP+,holP. Thus, ha-' is the mean holding time in state a. We calculate the frequency with which the inventory X crosses a given level x in the interior of region R ' while the machine state is C Y , as follows. Let Ax > 0 be sufficiently small so that x k Ax is contained in region R', and let us define a; and ag(p # a) as follows:
a; = sgn

In terms of the matrix G' = [gLJ and the row vector m(x) = [m(x, a)], this system of differential equations can be written in matrix form as
dx

d -m(x)=m(x)

G',

x E R'.

(11)

The solution is m ( x ) = m(xo)eG'(x-xd, x E R'. In particular, if the matrix G' is diagonalizable with eigenvalues s i and corresponding eigenvectors U:, then there exist constants C ; such that

ut=

+I
-1

ifuL>O, if u t < O .

Once the m(x, a) are known we can immediately compute the f ( x , a). Indeed, the differential equation (10) for m(x, a) and relation (8) between m (x, a)and f ( x , a)imply that

d x f ( x , a ) = E f ( x ,p)
P

%,
01

x E R'.

(13)

One way to cross the level x while being in state a is to pass through level x - u;Ax while being in state a,and to make no state transition for a duration At = A x / \U; I. The state transition fails to occur with probability
1 - h,At+ O(At)= 1 ,A ,

If we introduce the matrix

Hi= [ hh,]

where hh, =

%,

Ax -+o(Ax).

Iutl

then the row vectorf(x) = [ f ( x , a)]satisfies the linear system of differential equations

Another way to reach level x while being in state a is to start at level x - aLAx in state /3 # a and to make a transition from state /3 to state a during a time interval At = Ax/luL[ o(Ax). Such a transition occurs with probability

-f ( x )= f ( x ) . Hi.
dx

AP,At

+ o ( A t )= Xg,

+ AX).

Ax

The solution isf(x) = ~ ( x ~ ) e " ( x - dx, E R'. It is easy to show that G' and N' are similar matrices. Indeed,

I q

We may neglect the possibility that level x is reached after several state transitions have taken place while X was in the range x Ax, since the probability of such events is o(Ax). The following equation holds for m(x, a), which is the frequency with which X ( t ) crosses level x while the machine state is a:

= (vi) G'.( V i ) - ]where Vi is the diagonal matrix and hence H i with diagonal entries Iubi. Hi and G i are also similar to the matrix that appears in the Hamilton-Jacobi-Bellman equation for the system. This equation is (cf. [l])

m(x, a)= m(x- U ~ A X a) , 1 + X,,

>I;:,

Since u t = U,* - d is the optimum control in region R', this equation reduces to

Simplification gives m(x, a ) - m ( x - o b A x , a) Ax Letting Ax

+

x E R', all a.

=Em(x-a;Ax, @) &+o(Ax). b;I

0

0 we obtain

Now let J ( x ) denote the column vector [J(x, a),J(x, /3), * * , * , 11 T , V' the diagonal e the column vector [ 1, 1, matrix [U;], and A' the generator matrix [haP/u;]. Then we can express (16)in matrix form

J(x, y)]

- J ( x ) = -A'J(x)-g(x)(V')-'e,
Writing

d dx

x E R'.

(17)

The matrix N'is in fact V'A'( V')- I . Notice that G' and N' depend on the ha, and u t , but not on the location of the hedging points. Therefore, the functional form of the densityf(x) = C&x, a)in each region R'does not depend on

SHARIFNIA- MANUFACTURING SYSTEM WITH MULTIPLE MACHINE STATES

623

the hedging points. This property facilitates solution of the problem if the hedging points are changed.
\ /

Boundary Conditions
To determine the constants C ; in (12) we write the boundary conditions at the hedging points, Consider any hedging point zg, except the largest one, and a machine state a # /3 for which X will be increasing at zg, i.e., U&, a ) > 0 (Fig. 2). Letting z,J = zg + E and zp = zo - t where E > 0 is infinitesimally small we may write
m(z,,!, a ) = m ( z ; , a ) + [ m ( z ; , P ) + m ( z i ,

zE

Fig. 2 .

Possible sample paths reaching or leaving a hedging point.

\ \

zY

\a
\

\
\

P)1 - ,
A ,

Fig. 3 .

Sample paths reaching or leaving largest hedging point.

if

u ( z 0 , a)>O.

(18)

This equation states that m(z;, a ) and m(zi, a ) would be equal were it not for the transitions from state p to state a which will add to m(z;, a). If X is decreasing at zd in machine state a (i.e., if U&, a ) < 0), then the boundary equation will be

This condition gives us M more equations. Since we have NM M equations in total we can find all the unknowns.
IV. OPTIMUM HEDGING POINTS

For tentative hedging points z,, zg, . . . , z, we can calculate the steady-state PDF of the surplus X in each region and also the probability masses at the hedging points. The average cost per unit time is then given by

if For a
=

u(zg,

a)<O. (19)

p the boundary condition is

[ m ( z ; , P ) + m ( z i , , P)1 -=P(zg).

(20)
(25)

The expression inside the bracket on the left-hand side of (20) is the average number of times (per unit time) that the inventory reaches zg while the machine state is p. Since the machine transition times have exponential distribution and have no memory, the expected time that X remains at zo, given that it has reached zg, is nothing but the expected holding time l / X g of state 0. Therefore, the left-hand side of (20) is the probability P ( z & of being at hedging point zd. For the largest hedging point zT the boundary conditions have a slightly different form, namely (see Fig. 3)

Equation (21) is equivalent to (18) and (19), and (22) is equivalent to (20). Suppose there are N machine states, out of which M are feasible. Then there will be M hedging points. The boundary conditions (18)-(22) give us N equations for each hedging point, or a total of N . M equations. We have N * M + M unknowns, namely Cks and P(z,)s. To find additional conditions we observe that
ah + P(z,,)= acx,
I

Here g ( x ) is the cost (per unit time) of maintaining the surplus at level x . The function J * ( z a , z g , . . . , z,) can be minimized over z,, zg, * . ; , z, to find the optimum values of the hedging points. The minimization must be done subject to constraints of the form z, 2 zg, to impose the order presumed for the hedging points. The function J*(zu, zg, * . . , z,) is in general nonlinear and the constraints are linear with a special structure. An appropriate algorithm for solving this problem would be the Frank-Wolfe algorithm. Clearly, the solution obtained depends on the order assumed for the tentative HPs z,, zg, . . ., z,, and may not be the true optimum. One may solve the problem for all plausible HP orders to find the true optimum. It may be possible, however, to use the solution obtained for a given order to choose the next HP order in such a way that the true optimum is found more efficiently. In particular, if it were assumed that z, 2 zg and in the optimum solution z, = z g , this would be an indication that the right order is probably zo 2 z,.
V . EXAMPLE

We apply the results derived in the previous sections to an example with three machine states (Fig. 4). The transition rate matrix is
A=(-:

a feasible

(23)

-1 -[).
li3=0,

where is the steady-state probability of machine state a and a i is the steady-state probability of being in region R and machine state a . But (8) implies that

The maximum production rate for the three states and the demand rate are:
li,=2,

liz=2,

d= I .

Substituting a:rfrom this equation in (23) we find

c&
,

JR,

m(x, a ) dx+ P(z,)= a , ,

a feasible.

(24)

Thus, states 1 and 2 are feasible and state 3 is infeasible. The steady-state probability of the states is found to be a, = 5/19, a2 = 6/19, r3 = 8/19. The overall system is strictly feasible, since a,(t?,- d ) = 3/19 is positive. There are two hedging points, z1 and zz, corresponding to states 1 and 2, and we assume that z1 > zz. This assumption is plausible

624

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 33, NO. 7, JULY 1988

Fig. 4. The continuous-time Markov chain model of the production system in the example. Numbers on arcs are transition rates. Fig. 5 . PDF of the surplus X in the example

because state 1 is more susceptible to failure than state 2 (AI3 > The PDF of X is obtained by substituting these functions in (9). A23), and the repair rate is the same for both states (A31 = A32). Including the probability masses at z1and z2as impulses 6(x - zl) There are two regions and 6(x - z2)inf(x) we have R = { x I 21 >x > 22 } ,

R 2 ={ x ( z ~ > x } .
The optimum production strategy for each machine state is to produce at capacity if X is below the corresponding hedging point, and to produce nothing if it is above it. Therefore,
u ; = f i l - d =1 ,
v'=O-d=
-

I
f(XI =

+0.1266(x-zl)1

if z12x>Zz

u;=fil-d=1, ui=fi2-d= 1,
u32 = O - d =
-

0.398e2.546 [e~54(X-Z2)+0.1816(~-~2)] e2,54*-0.13

if x 5 z 2 .

1,

U;=O-d= - 1 '

1.

The matrices G 1 and G2 corresponding to regions RI and R 2 are -4 -2 -2

A discrete-event simulation was carried out for this example to confirm the validity of the results. In this simulation the probability of X being at the hedging points zI and z2as well as the probability of X being in regions RI and R 2 were measured by time averaging over 10 OOO machine transitions. The same quantities were calculated from the equations obtained above and are compared to the simulation results (for 3 values of 6 = zl 22) in Table I. The results from the two methods are very close. Assuming the cost function is linear with slopes c+ > 0 and c> 0 for inventory and backlog

c+x GI has eigenvalues 0, 3.54, and - 2.54, and the eigenvalues of -c-x G2 are 0, - 5.54, and 0.54. After writing (12) for ml(x), W Z ~ ( X ) , and m3(x) and solving the boundary equations (18)-(22) to We calculate J * ( z l , z2)as calculate the constants, we find, with 6 = zI - 22, that (0.0016e3.54(X-21) + 0.234e-2W-21))
if x E RI,
= - (c+

ifxr0, ifxs0.

+c-)

J -a

io xf(x) dx+ c+
a

J -a

xf(x) dx

= -(c+ + c - )

ifx E

so

xf(x) d x + c + E { X }

where E { X } is the expectation of the surplus X . Evaluating the integrals and simplifying, we get

ifx E R I ,

. e2~546-0.126(z2+6)-0.073 .
ifx E
f
~ 2 ,

We ignore the constraint z1 Iz2 and minimize J*(zI, z2)with no constraints. We will see that z1 Iz2 will be met automatically. We set aJ*laz2 and aJ*la6 equal to zero to find optimum values for z2 and 6. These equations yield
~ 2 1.85 =

log

+(c-/c+)

1.35 -0.17e-2.546

e2~546(0.326-0.024z2-0.129)+0.097=0. Optimum values of z2 and zI = z2 6 depend on the ratio ( c - / c + ) . If cc + , zI and z2 increase with log (1 + c-/c+) as shown in Fig. 6.

SHARIFNIA: MANUFACTURING SYSTEM WITH MULTIPLE MACHINE STATES TABLE I ANALYTICAL AND SIMULATION RESULTS FOR THE EXAMPLE IN SECTION V A = ANALYTIC, s = SIMULATION

625
manuscript. He is also indebted to his colleague P. H. Algoet for suggestions that led to substantial improvements in the presentation of Section 1 1 1 . Finally he thanks M. Caramanis for his valuable comments.
REFERENCES
J. G.. Kimemia and S. B. Gershwin, An algorithm for the computer control of production in flexible manufacturing systems, IIE Trans., vol. AC-15, pp. 353-362, Dec. 1983. S. B. Gershwin, R. Akella, and Y. F. Choong, Short-term production scheduling of an automated manufacturing facility, IBM J. Research Development, vol. 29, no. 4, July 1985. R. Akella and P. R. Kumar, Optimal control of production rate in a failure prone manufacturing system, IEEE Trans. Automat. Confr., vol. AC-31, no. 2, Feb. 1986. T. Bielecki and P. R. Kumar, Optimality of zero-inventory policies for unreliable manufacturing systems, Operat. Res. (Special Issue in Manufacturing Systems), 1987. R. Akella, Y. Choong, and S. B. Gershwin, Real-time production scheduling of an automated cardline, Ann. Operat. Res., vol. 3, pp. 403-425. 1985.

0.5 1.0 1.5

0.138 0.132 0.770 0.172 0.171 0.769 0.181 0.182 0.743

0.777 0.017 0.016 0.074 0.748 0.005 0.005 0.072 0.741 0.001 0.001 0.072

0.076 0.076 0.076

+ * = 1 =2

c-/c+

Fig. 6 .

Optimum HPs as functions of c - / c .

VI. CONCLUSION

In this paper we have derived equations for the steady-state probability distribution of the surplus level in a single product manufacturing system with machine failures, when a hedging point control policy is used. The results make it possible to solve the optimum production scheduling of such systems with arbitrary number of failure states. The results are expected to be very valuable for solving the scheduling problem of the multiproduct case in flexible manufacturing systems.
ACKNOWLEDGMENT

The author is grateful to S. B. Gershwin for introducing him to the problem studied here and for his valuable comments on the

Ali Sharifnia was born in Iran in 1948. He received the B.S. degree in electrical engineering from Tehran University of Technology, Tehran, Iran, in 1970, and the M.S. degree in industrial engineering, and the Ph.D. degree in engineering-economic systems, both from Stanford University, Stanford, CA. He is currently an Assistant Professor of Manufacturing Engineering at Boston University, Boston, MA. His research interests are in capacity planning, scheduling, and control of manufacturing systems.