6802

Ind. Eng. Chem. Res. 2003, 42, 6802-6814

Product Quality Trajectory Tracking in Batch Processes Using Iterative Learning Control Based on Time-Varying Perturbation Models
Zhihua Xiong and Jie Zhang*
Centre for Process Analytics and Control Technology, School of Chemical Engineering and Advanced Materials, University of Newcastle, Newcastle upon Tyne, NE1 7RU U.K.

A run-to-run model-based iterative learning control (ILC) strategy for the tracking control of product quality in batch processes is proposed. A linear perturbation model for product quality, linearized around the nominal trajectories, is identified from process operating data using linear regression. To address the problem of model-plant mismatches, model prediction errors in the previous batch run are added to the model predictions for the current batch run. On the basis of the modified predictions, an ILC law with direct error feedback can be explicitly obtained. The convergence of tracking error under ILC is analyzed. To overcome the detrimental effects of unmeasured disturbances and process variations, it is proposed in this paper that the perturbation model should be updated in a batchwise manner. After the completion of each batch, a batchwise perturbation model, linearized around the control trajectory for that batch, is identified. A forgetting factor is introduced so that data from the more recent batch runs are weighted more than data from earlier batch runs. The proposed technique is successfully applied to a simulated batch reactor and a simulated batch polymerization process.
1. Introduction Run-to-run control of operating conditions for improving the final product quality in batch processes has generated a challenging area of research. Run-to-run control exploits the repetitive nature of batch processes to refine the operating policy. The general idea of runto-run control is to use process knowledge obtained from previous batch runs to update the operating strategy of the current batch run.1 Various run-to-run control strategies for product quality have been proposed in the literature. The operating policy can be optimized by runto-run optimizing control for the final product quality to address the problems of model-plant mismatches and/or unmeasured disturbances in batch processes.2-5 Recently, iterative learning control (ILC) has been used in the run-to-run control of batch processes to directly update the input trajectory.6-10 The basic idea of ILC is to update the control trajectory for a new batch run using the information from previous batch runs so that the output trajectory converges asymptotically to the desired reference trajectory. Refinement of control signals on the basis of ILC can significantly enhance the performance of tracking control systems. Campbell et al.11 presents a brief survey of run-to-run control algorithms based on linear models for batch processes. Lee and co-workers in several related articles7,9,12 proposed the Q-ILC approach with a quadratic criterion for the temperature tracking control of batch processes. Because of the limited availability of robust on-line sensors in the industrial practice of batch process operations, typically only off-line quality measurements are available. Off-line quality measurements usually include measurements taken at the end of the batch
* To whom correspondence should be addressed. Tel.: +44-191-2227240. Fax: +44-191-2225292. E-mail: jie.zhang@ newcastle.ac.uk.

(batch-end measurements) and off-line analysis of samples taken during the batch. These measurements are most common in industrial practice.5 A common approach to final product quality control in the current industrial practice is allowing the directly measured variables, such as temperatures and feed rate, to track prespecified trajectories. Once the recipes of temperatures or feed rates are fixed, the same time-varying trajectories tend to be used batch after batch until a different product needs to be made, which necessitates a change in the operating procedure. This strategy would work well if all of the disturbances affected the controlled process variables first and then affected the product quality through kinetics, such as the heattransfer disturbances affecting the temperature and thereupon the product quality through kinetics.7 In this strategy, the final product quality would depend completely on the recipe of temperatures and/or feed rates. However, product quality is often determined by factors other than the temperature trajectory, such as the feedback condition.12 Practical evidence also suggests that batch-to-batch variations in batch ingredients, such as raw material properties, impurities, and catalyst activities, can be significant during a number of batch operations.13,14 Thus maintaining consistent trajectories of measured variables alone does not render consistent product quality. Trajectories of those directly measured variables should be updated. An obvious way to do this is to reoptimize the trajectories of the measured variables (as input variables) by minimizing the performance index about the final product quality as in run-to-run optimizing control.5 On the other hand, off-line quality measurements taken during the batch enable the setup of a run-to-run control approach for product quality. When the desired trajectories of product quality variables can be set reasonably, the ILC strategy for the tracking control of measured variables can be used straightforwardly to the tracking control of product

10.1021/ie034006j CCC: $25.00 2003 American Chemical Society Published on Web 11/12/2003

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6803

quality. The final product quality will be improved gradually under this strategy, as the whole trajectory of product quality converges asymptotically to the desired trajectory. This paper focuses on how the control trajectory of a batch reactor is updated by tracking the desired trajectories of product quality variables under the run-to-run control scheme. Because ILC theory is well-developed for linear timeinvariant and linear time-varying (LTV) systems,9,15-19 developing linear models for product quality control from nonlinear process mechanics or operating data of industrial plants has become a common practice. To address the problem of nonlinearites in batch processes, it is very useful to look at the perturbation variables, which are changes of variables away from their nominal trajectories, rather than the actual process variables themselves. Usually the input and output trajectories in batch processes are intrinsically nonlinear and timevarying. In many cases, subtracting the time-varying nominal trajectories from the batch operation trajectories removes the majority of the process nonlinearity and allows linear modeling methods to perform well on the resulting perturbation variables.20 Thus, a linear perturbation model of product quality can be obtained by linearizing a nonlinear model with respect to the nominal trajectories, and the developed ILC strategy can be used straightforwardly for the tracking control of product quality. Because of modeling errors and unmeasured disturbances, model prediction offsets always exist. However, the perturbation model predictions of the current batch run can be corrected by adding the corresponding model prediction residuals of the previous run. Doyle et al.4 introduces a strategy to add the filtered model prediction residuals obtained from the immediately previous run to the current batch run and shows that this can reduce batch-to-batch variability in model predictions. In this study, we utilize the model errors of the previous batch run to modify the predictions for the current batch. On the basis of the modified predictions, an ILC law is obtained explicitly. During a number of batch operations, process variations usually exist, such as in raw material properties, impurities, and catalyst activities, so there are often some plant parametric uncertainties.13,14 Therefore, the perturbation model should be updated to capture the changed behavior of the batch process. A new perturbation model can be approximated reasonably well through linearization of the nonlinear model around the control trajectory in the immediately previous batch run. It is proposed here that a new linear model should be identified at each batch run by using the immediately previous run as the nominal batch. This paper differs from earlier works in this field, such as those of Lee et al.9 and Doyle et al.,4 in several respects. First, this paper uses linearized perturbation models, which model the relationship between deviations of the quality and control variables from their nominal trajectories. On the other hand, Lee et al.9 used a linear time-varying model among input and output variables directly, and Doyle et al.4 used a hybrid model of particle size distribution in emulsion polymerization that was built by combining a simple mechanical model with a partial least-squares regression model. Second, this paper proposes that the linearized model be identified from process operating data using linear regression and updated after each batch run to address the problems of unknown disturbances. A forgetting factor

is also introduced so that data from the more recent batch runs are weighted more heavily than data from earlier batch runs. To our knowledge, combining an updated model with batch-to-batch iterative learning control has not previously been reported. Most of the earlier works on batch process optimal control generally utilize nonlinear models such as mechanistic models, neural network models, and hybrid models.2,4,13,14 The rest of this paper is organized as follows: Section 2 presents the linear time-varying perturbation model and its identification from process operating data. An optimal ILC strategy based on the modified predictions of an LTV perturbation model is proposed in section 3. The method of handling plant parametric uncertainties through updating of the perturbation model from batch to batch is presented in section 4. Applications of this control strategy to a simulated batch reactor and a simulated batch polymerization process are given in section 5. Finally, section 6 contains some concluding remarks. 2. Linear Time-Varying Perturbation Models for Batch Processes 2.1. Nonlinear Representations of Batch Processes. In this study, we consider batch processes where the batch run length (tf) is fixed and consists of N sampling intervals (i.e., N ) tf/h, where h is the sampling time) and all batches run from the same initial condition. The control problem is to manipulate the control policy subject to given constraints so that the product quality variables follow specific desired reference trajectories. The product quality variables (outputs), y Rn (n g 1), can be obtained off-line by analyzing the samples taken during the batch run, and the manipulated variable, u Rm (m ) 1 in this work), can be measured at each sampling time on-line. The product quality and control trajectories are defined, respectively, as

Yk ) [ykT(1), ykT(2), ..., ykT(N)]T Uk ) [uk(0), uk(1), ..., uk(N-1)]T

(1) (2)

where the subscript k denotes the batch index. It is assumed here that the desired reference trajectories of product quality can be set reasonably and are defined as

Yd ) [ydT(1), ydT(2), ..., ydT(N)]T

(3)

A batch operation is typically modeled with a dynamic model, but it would be convenient to consider a static function relating the control sequence to the product quality sequences over the whole batch duration.12 Considering the causality, a product quality variable at time t, yk(t), is a nonlinear function of all control actions up to time t, Uk(t) ) [uk(0), uk(1), ..., uk(t-1)]T, i.e.

yk(t) ) ft(Uk(t)) + vk(t),

t ) 1, 2, ..., N; yk(0) ) y0 (4)

where ft() represents the nonlinear function between Uk(t) and yk(t) and vk(t) is the measurement noise at time t. Equation 4 can be rewritten in matrix form as

Yk ) F(Uk) + vk

(5)

6804 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003

where F() represents the nonlinear static functions between Uk(t) and yk(t) at different sampling times and vk ) [vkT(0), vkT(1), ..., vkT(N-1)]T is a vector of measurement noises. 2.2. Linear Time-Varying Perturbation Models. In batch process operation practice, it is quite useful to look at the changes of variables away from their nominal trajectories rather than the actual values of the variables. Because the output and input variables, R ) {Y, U}, are functions of time t, their perturbation variables, R h ) R - Rs ) {Y h, U h }, which are departures from the corresponding nominal trajectories, will also be functions of time t, where Rs represents the nominal trajectory. In many batch processes, subtracting the time-varying nominal trajectories from the process operation trajectories removes the majority of the process nonlinearity and allows linear modeling methods to perform well on the resulting perturbation variables.20 The linearization of nonlinear batch processes around the nominal trajectory provides a good approximation to the real processes as the input change is typically small.17 In an LTV model, Y ) GU, the operator G can be used to describe the linear function between the input and output trajectories for an entire batch run. For product quality control, an LTV perturbation model, Y h ) GU h , can be obtained by linearizing a nonlinear model with respect to the nominal (mean or reference) trajectories. Here, the term time varying refers to the fact that G varies with the nominal trajectory that is typically updated from batch to batch in practical batch process operations. In the case of continuous reactors, linearization is more or less easily achieved around the normal operating steady state of the reactor. However, in the case of batch process operation, because the process is operated in transient modes, the reactor states change continuously, and there is no steady state. Hence, linearization is carried out around some nominal process operation trajectories.21 For example, in the practical operations of batch processes, the common operation control profile or the mean profile of all previous control trajectories can be selected as the nominal control trajectory, and its corresponding output trajectories can be selected as the nominal output trajectories. Such a nominal control trajectory could also be obtained as a result of developing an optimal control policy for the economic operation of batch processes. Let the nominal control trajectory and its corresponding product quality trajectory be defined as

around the nominal trajectories (Us, Ys), the following expression can be obtained

Y k ) Ys +

F(Uk) | (U - Us) + wk + vk Uk Us k

(10)

where wk ) [wkT(1), wkT(2), ..., wkT(N)]T is a sequence of model errors due to the linearization (i.e., due to neglecting the higher-order terms) and vk represents the effects of noise and unmeasured disturbances. Define the LTV oprerator Gs as

Gs )

F(Uk) | Uk Us

(11)

and the structure of Gs is restricted to the following lower-block-triangular form because of the causality

g10 g Gs ) 20 l gN0

0 g21 l gN1

0 0 l gNN-1

(12)

where gij Rn. In eq 11, Gs is LTV in the sense that it varies with Us, which usually varies from batch to batch. Then, the linearized time-varying perturbation model can be obtained from eq 10 as

Y h k ) GsU h k + dk

(13)

where dk is defined as the model disturbance sequence

dk ) wk + vk

(14)

and is supposed to be bounded by a certain small positive constant Bd such that

|dk| < Bd

(15)

Us ) [us(0), us(1), ..., us(N-1)]T Ys ) [ysT(1), ysT(2), ..., ysT(N)]T

(6) (7)

where the subscript s denotes the nominal batch and ys(0) ) y0. Then, the perturbation variables of the control and product quality variables are defined, respectively, as

U h k ) Uk - Us Y h k ) Yk - Ys

(8) (9)

where y jk(0) ) 0. By linearizing the nonlinear batch process model described by eq 5 with respect to Us

Remark. Although this study assumes that the same number of samples are collected for quality variables and process variables (see eqs 6 and 7) for simplicy in presenting the method, the situation where the quality variable is sampled at a lower rate than the process variables can be handeled by adding more columns to Gs in eq 13. 2.3. Identification of the Perturbation Model Gs. The LTV model Gs can be found by linearizing a nonlinear model along the nominal trajectories or through direct identification from process operating data. If the fundamental mechanistic model of a batch process is available, one can linearize the model around the nominal batch operation trajectories to build the perturbation model. In many cases, however, developing detailed mechanistic models of batch processes is quite difficult and time-consuming. Empirical models based on process operating data can provide useful alternatives. Available methods for identifying the perturbation model range from simple static linear regression, such as the least-squares method and its variants (e.g., partial least squares, PLS), to more elaborate optimal dynamic estimation methods such as Kalman filtering. Russell et al.20 proposed recursive data-based prediction of batch product quality based on a least-squares regression model. Multiway PCA and PLS methods have also been introduced to model batch and semibatch processes.2 Chin et al.7 used PLS to build a linear

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6805

correlation model to predict the final product quality. In this study, the least-squares regression method is used to estimate the parameters of the perturbation model on the basis of the process operating data. From the process input-output data set, the parameters of Gs in eq 13 can be identified using the leastsquares method. The historical batch process operating data are defined as

problems in H0i, the ordinary inverse in eq 22 can be replaced by a generalized inverse ()+, for example, defined by PLS or PCR (principal component regression). 3. Optimal Iterative Learning Control of Batch Processes 3.1. Perturbation Model Prediction Modification. As batch process dynamics are nonlinear and the perturbation model is linearized around the nominal operation trajectories of a batch process, offsets always occur as a result of modeling errors and unmeasured disturbances. The perturbation model predictions of the current batch run can be corrected by adding the model prediction residuals of previous batch runs. Crowley et al. introduced a strategy to add the filtered model prediction residuals obtained from the immediately previous run to the current batch run.4 In this study, we utilize the model errors of the immediately previous batch run to modify predictions of the perturbation model. We define the prediction of the perturbation model as

0 ) [Y1, Y2, ..., YL] , 0 ) [U1, U2, ..., UL]

(16)

where L is the number of batches in the historical batch process operating data. To identify the parameters of Gs, Gs is partitioned as a block column matrix according to the time index of the output trajectory

Gs ) [g1T, g2T, ..., gNT]T

where

(17)

j i (i ) 1, 2, On the basis of the data matrices (0y, 0u), g ..., N) can be identified using the least-squares method. From eq 4, it can be seen that the product quality variable yk(t) at time t is only related to all manipulated variables up to time t, Uk(t) ) [uk(0), uk(1), ..., uk(t-1)]T. Thus, g j i can be estimated independently from the data of yk(i) and Uk(i). Therefore, we need only a few more than N batches of process operating data to identify all of the parameters of Gs. In practice, because the number of sample intervals for the product quality is usually not very large, e.g., N ) 10 or less, there would always be enough process operating data for the parameter identification. The identification of g j i is given next. From eq 13, we have

k ) G Y h sU hk
and the absolute model prediction as

(24)

k ) Ys + G h sU hk Y k ) Ys + Y

(25)

After completion of the kth batch run, prediction errors between off-line-measured or -analyzed product qualities and their model predictions can be calculated as
k

k ) Yk - Y k ) Y hk - Y h

(26)

y lT(i)) hlT(i)g i,
where

i ) 1, 2 , ..., N; l ) 1, 2, ..., L (19)

On the basis of the prediction errors of the kth batch run, the modified prediction of the perturbation model in the (k + 1)th batch run is obtained as

k+1 ) Y k+1 + h Y h y jl(i) ) yl(i) - ys(i) j l(0),u j l(1), ..., u j l(i - 1)]T hl(i) ) [u u j l(t) ) ul(t) - us(t), t ) 0, 1, ..., i - 1 (20)

(27)

The absolute modified model prediction is defined as

Y k+1 ) Y k+1 +

k+1 + ) Ys + Y h

(28)

and the modified prediction error as

If we define

y j1T(i) y j T(i) , Z0i ) 2 l y jLT(i)

then g j i can be estimated by the least-squares method as

[] [ ]
h1T(i) h T(i) H0i ) 2 l hLT(i)

k+1 k+1 ) Y h k+1 - Y h k+1 ) Yk+1 - Y

From the definitions in eqs 26 and 27, we have

(29)

(21)

k+1 )

k+1

(30)

In this work, we assume that the prediction error of the perturbation model is bounded by a certain small positive constant Bm such that

g i ) (H0iTH0i)-1H0iTZ0i
and g i and G s are obtained as

(22)

| k| < Bm

(31)

The prediction error bound Bm is a measure used to kfrom Y h k or Y k from Yk. represent the deviation of Y h The higher the value of Bm is, the poorer the identified model is. The modified prediction error is bounded by 2Bm as follows

To account for the possible existence of collinearity

| k| < | k| + |

k-1|

< 2Bm

(32)

6806 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003

Figure 1. Structure of modified prediction-based optimal ILC.

3.2. Derivation of the ILC Law. As shown in Figure 1, the procedure of run-to-run-modified prediction-based optimal ILC is outlined as follows: At the current kth batch run, the control trajectory Uk is implemented. After the completion of the batch run, the product qualities Yk are obtained by off-line analysis of samples taken during the batch run. The model prediction errors k are calculated and used to correct the model predictions for the next batch. On the basis of the modified k+1, a new control policy Uk+1 for the next predictions Y h batch is calculated using the ILC law. At the next batch, this procedure is repeated. Considering that the objective of ILC is to track the desired reference trajectories of product quality, we define tracking errors of the process and of the perturbation model, respectively, as

Upon substitution of eqs 37 and 39 into eq 38, we have

e k+1 ) e k+1 - (e k - ek) ) ek - G sU h k+1

On the other hand, eq 37 can be rewritten as

(42)

ek ) e k -

(43)

From eqs 43 and 39, an iterative relationship for ek along the batch index k can also be obtained as

sU h k+1 - k+1 ek+1 ) ek - G

(44)

ek ) Yd - Yk ) Y hd - Y hk k e k ) Yd - Y k ) Y hd - Y h

(33) (34)

Given the error transition model in the form of eqs 42 and 44, the objective of ILC is to design a learning algorithm to manipulate the control policy so that the product qualities follow the specific desired reference trajectories. It is required that the learning algorithm have the following property9

where Y h d is the deviated desired trajectory, defined as

lim||ek||Q2 f min||e||Q2
kf U

(45)

Y h d ) Yd - Ys

(35)

The tracking error of the modified prediction of the perturbation model is defined as

By the certainty-equivalence principle,22 we consider solving the following quadratic objective function using the modified prediction errors upon completion of the kth batch run to update the input trajectory for the (k + 1)th batch run

k e k ) Yd - Y k ) Y hd - Y h

(36)

1 Jk+1 ) min [e k+1TQe k+1 + U h k+1TRU h k+1] U h k+12

(46)

From the definitions in eqs 26, 33, and 36, the following relationships among these three tracking errors can be obtained
k

)e k - ek
k-1

(37) (38)

k e k ) e

From eqs 34 and 24, an iterative relationship for e k along the batch index k can be obtained as

k - G sU h k+1 e k+1 ) e
where U h k+1 is defined as

(39)

h k+1 - U hk U h k+1 ) U

(40)

From the definition of the perturbation variables, eq 8, we can have

where Q and R are positive-definitive matrices. Note that the objective function, eq 46, has a penalty term on the input change U h k+1 between two adjacent batch runs, and the algorithm has an integral action with respect to the batch index k.9 The weighting matrices Q and R should be selected carefully. A larger weight on the input change will lead to more conservative adjustments and slower convergence. There are also other variants of the objective function. For example, the weighting matrices Q and R can be set as Q ) diag{Q(1), Q(2), ..., Q(N)}, R ) diag{R(0), R(1), ..., R(N1)}, where Q(i) and R(j) increase with respect to the time intervals t in proportion to the effect on the final product quality. For the sake of simplicity, Q and R are selected in this study as Q ) qIN and R ) rIN. By finding the partial derivative of the quadratic objective function eq 46 with respect to the input change U h k+1 and performing straightforward manipulations, the following ILC law can be obtained

U h k+1 ) U h k+1 - U h k ) Uk+1 - Uk

(41)

ek U h k+1 ) K

(47)

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6807

where K is defined as the learning rate

K ) [G sTQG s + R]-1G sTQ

(48)

error will change to |ek| f as k f , where is a small positive constant. Upon substitution of the ILC update law eq 47 into eq 44, we have

From eqs 41 and 47, the ILC law for the control trajectory can be written as

ek+1 ) [I - G sK ]ek - k+1

(57)

ek Uk+1 ) Uk + K

(49)

3.3. Comparison with Lee et al.s Algorithm. For the temperature tracking control of batch processes, Lee et al.9 proposed a direct error transition model for the deterministic case in their Q-ILC approach

h k + dk, On the other hand, according to eq 13, Y h k ) GsU k ) G sU h k, so we can obtain and according to eq 24, Y h
k

k ) (Gs - G )Y hk - Y h s)U h k + dk

(58)

Thus, eq 30 can be rewritten as

ek+1 ) ek - GUk+1

(50)

k+1 )

k+1

) (Gs - G s)U h k+1 + dk+1 (59)

where ek is the tracking error and G is the LTV model. They also proposed the observer-based model as

where dk+1 ) dk+1-dk is the deviation of the model disturbance. Substituting eq 59 into eq 57 gives

k|k - GUk+1 e k+1|k ) e e k+1|k+1 ) e k+1|k + K(ek - e k+1|k)

(51) (52)

ek+1 ) [I - GsK ]ek - dk+1

(60)

where e k+1|k+1 is the estimate of e j k based on the measurement ek. More details can be found in the literature.7,9 In this work, we focus on the perturbation variables of the product quality of batch processes rather than the absolute variables. Thus, our perturbation model is different from model eq 50 proposed by Lee et al. Compared to model eq 50, here, we similarly obtain the recursive relationship of model prediction tracking error e k as

Assuming that dk+1 is constant, ek+1 will converge if I has all its eigenvalues inside the unit circle, i.e. - GsK

|| < 1 ||I - GsK

(61)

k - G sU h k+1 e k+1 ) e

(53)

By introducing the prediction errors in the kth batch run into the (k + 1)th batch run, we obtain the transition model of process tracking error ek as

sU h k+1 - k+1 ek+1 ) ek - G

(54)

This model is different from eq 50 because of the term including modified prediction errors k+1, so that it focuses on the perturbation variables rather than the absolute variables. Although the models are different, using the above process error transition model eq 54, the ILC law in eqs 47 and 48 can also be obtained and is the same as that proposed by Lee et al.9 Combining eqs 39 and 42, it can be seen that the modified prediction tracking error e k+1 is estimated by

Because no available off-line measurements can be used to improve the current batch run but only subsequent ones, random disturbances vk within the current batch run are not taken into account, and the existence of vk will prevent convergence to zero tracking error.5 According to eq 60, when vk+1 ) vk ) 0 and wk+1 ) 0, i.e., dk+1 ) 0, one can be obtain ek f 0 as k f if eq 61 is satisfied. However, in practice, linearized modeling error is always unavoidable, i.e., wk * 0 and wk+1 * 0 (causing dk+1 * 0), so the tracking error will change to |ek| f as k f , where is a small positive constant. This means that the tracking performance depends on the accuaracy of the model Gs. Because Gs cannot be known precisely a priori, the condition in eq 61 cannot be checked explicitly. However, s to determine whether the ILC we can replace Gs with G algorithm is nominally convergent. We find that the ILC algorithm is nominally convergent if

|| < 1 ||I - G sK

(62)

4. Handling Process Parametric Uncertainties through Model Updating During a number of batch operations, batch-to-batch variations always occur in, for example, raw material properties, impurities, and catalyst activities, so there are often some process parametric uncertainties.13,14 In this case, the perturbation model Gs, which is identified on the basis of the old input-output data matrices 0u and 0y, will give inaccurate predictions. Thus, the perturbation model Gs should be updated in a batchwise manner to capture the changed dynamics of the batch process. It is proposed in this paper that, after the completion of the kth batch, the control trajectory Uk be used as the nominal trajectory for updating the perturbation model. Because of the process parametric uncertainties, it is very useful to identify the parameters of Gk by the leastsquares method with a forgetting factor. By introducing the forgetting factor into the identification data set, data from the more recent batches will be weighted more than data from earlier batches, and thus, the estimated

k - G sU h k+1 e k+1 ) e e k+1 ) e k+1 + (ek - e k)

(55) (56)

Comparing these expressions with observer-based model eqs 51 and 52, the above model is the same as the observer-based model when K ) I. 3.4. Convergence Analysis. The convergence of the ILC alogrithm can be obtained, and its proof can directly be derived from the convergence theorems of the observer-based model in the literature.9 However, in this study, the tracking performance will depend on the accuracy of the model Gs. Under the condition that dk+1 ) dk+1-dk ) 0, i.e., dk is the same for all values of batch index k, perfect tracking will be obtained, i.e., ek f 0 and U h k+1 f 0 as k f . When modeling error cannot be neglected, i.e., dk+1 * 0, then the tracking

6808 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003

parameters will capture the up-to-date characteristics of the process. The identification of this batchwise perturbation model can be obtained straightforwardly from the method described in section 2.2 with minor modifications. 4.1. Parameter Identification for the Batchwise Perturbation Model. The control for the kth batch, Uk, is selected as the nominal trajectory, and the batchwise perturbation model is obtained by linearizing the nonlinear function around Uk and its corresponding output sequence Yk. For the identification of Gk, Gk is partitioned into the following block form

respectively, in the new runs; and

y jl0(i) ) yl0(i) - yk(i) j l0(0), u j l0(1), ..., u j l0(i-1)]T hl0(i) ) [u u j l0(t) ) ul0(t) - uk(t) y jj(i) ) yj(i) - yk(i) j j(0), u j j(1), ..., u j j(i - 1)]T hj(i) ) [u u j j(t) ) uj(t) - uk(t)
with t ) 0, 1, ..., i - 1; l ) 1, 2, ..., L; and j ) 1, 2, ..., k - 1. It can be seen that the old data will have decreased effects on the identified model. Then, g j k,i can be estimated by the least-squares method as

(67)

(68)

Gk ) [gk,1T, gk,2T, ..., gk,NT]T

where

(63)

g k,i) (HkiTHki)-1HkiTZki
After the completion of the kth batch run, new input and output data from the kth batch are added to the database as

(69)

After all g k,i(i ) 1, 2, ..., N) values have been k can be constructed as estimated, g k,i and G

ky ) [k-1yT, Yk]T, ku ) [k-1uT, Uk]T

(65)
4.2. Summary of the ILC Algorithm Based on the Batchwise Perturbation Model. The run-to-run optimal ILC based on the batchwise perturbation model is outlined as follows: Step 1. Set k ) 1. Using the old data matrices 0u and 0y, select the nominal input and output trajectories, Us and Ys, and set Uk ) Us. Step 2. Apply Uk to the batch process. After the completion of the kth batch run, the input and output sequences, Uk and Yk, are obtained or measured, and these data are added to the data matrices ku and ky according to eq 65. Step 3. Update G s. (i) Set i ) 1 and estimate g j k,i. (ii) Generate new data matrices, Zki and Hki, from ky and ku according to eq 66. (iii) Estimate g j k,i according to eq 69. (iv) Set i ) i + 1 and return to step ii until i ) N. k according to eq 70 and let (v) Construct g k,i and G k. G s ) G Step 4. Calculate control actions for the (k + 1)th batch using ILC

where Yk and Uk are the output and input sequences obtained after the kth batch run and 0y and 0u are the initial historical batch process operating data sets. To capture the changed dynamics of batch processes due to process variations, a forgetting factor is introduced to the data sets ky and ku. The new data Zki and Hki for identifying g j k,i are generated by deviating around Yk and Uk at time i

K k ) [G sTQG s + R]-1G sTQ kek Uk+1 ) Uk + K

Step 5. Set k ) k + 1 and return to step 2. 5. Case Studies

(71) (72)

where (0.9 e e 1) is the forgetting factor; L is the number of batches in matrices 0y and 0u; yl0(i) and ul0(t) (l ) 1, 2, ..., L) are output and input data, respectively, in the old runs (i.e., in 0y and 0u ); yj(i) and uj(t) (j ) 1, 2, ..., k-1) are output and input data,

5.1. Case 1: Typical Batch Reactor. This case study is a typical nonlinear batch reactor with temperature as the control variable.23,24 The operating objective of the reactor is to maximize the product (B) after a fixed reaction time. The reaction scheme is

8B9 8C A9

k1

k2

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6809
Table 1. Parameter Values for the Batch Reactor parameter k1 k2 E1 E2 Tref value 4.0 103 6.2 105 2.5 103 5.0 103 348 K

and the equations describing the batch process are

dx1 ) -k1 exp(-E1/uTref)x12 dt

(73)

dx2 ) k1 exp(-E1/uTref)x12 - k2 exp(-E2/uTref)x2 dt (74)

where x1 and x2 represent the dimensionless concentrations of A and B, respectively; u ) T/Tref is the dimensionless temperature of the reactor; and Tref is the reference temperature. The final time tf is fixed to be 1.0 h, and values of k1, k2, E1, E2, and Tref are given in Table 1. The initial conditions are x1(0) ) 1 and x2(0) ) 0, and the reactor temperature is constrained to the interval 298 K e T e 398 K. In this study, the above mechanistic model, eqs 73 and 74, is assumed to be unavailable. Because the objective of the reactor is to maximize the product (B), an LTV perturbation model is built to model the relationship between y ) x2 and u. Here, the batch length is divided into N equal stages, and two values of N are studied, N ) 10 and N ) 5. The desired product reference trajectory, Yd, was selected from the literature.24 Twelve batches of process operations under different temperature profiles were simulated from the mechanistic model and used as the historical process data sets. These 12 temperature profiles were generated randomly by deviating from the nominal temperature trajectory Us. Then, using these historical process data sets and the selected Us and Ys, the parameters of Gs were identified according to eqs 21-23. The parameters of ILC are set as follows: Q ) I and R ) 0.01I. The nominal case without any uncertainties (unknown disturbances) in the batch reactor was first considered. After Gs had been identified, Gs was kept fixed throughout the successive batch runs. Figure 2 shows the tracking performance of ILC. It can be seen from Figure 2 that the total RMSE of ek and the tracking error at the batch end, ek(tf), have almost converged after about five batch runs. The performance with N ) 10 is slightly better than that with N ) 5. Thus, the proposed method still works well when the number of quality variable samples within a batch is reduced from 10 to 5. The selection of N is a tradeoff between improved control performance and the cost of collecting more quality variable samples and can be based on the particular circumstance of the process

Figure 2. Tracking performance in the nominal case using fixed Gs: (a) RMSE of ek, (b) ek(tf).

considered. In this nominal case, the results of using a fixed Gs were compared with those of using an updated Gs. According to the ILC algorithm based on the batchwise perturbation model, the model Gs was updated by using new process data after the completion of each batch run. The forgetting factor, , was selected as 0.98. Table 2 presents the control performance in the 15th batch run. From Table 2, it can be seen that the performance based on an updated Gs improves but not by much because there are no process variations (disturbances) in this nominal case. To investigate the performance of the proposed control strategy under the presence of process variations (disturbances), two additional cases were considered: disturbance case 1, batch-to-batch parametric changes; disturbance case 2, fixed parametric changes. In disturbance case 1, it is assumed that batch-to-batch variations are caused by the variations in reactor conditions, such as impurities, raw material conditions, and so on. This case is simulated by batch-to-batch

Table 2. Performance in the 15th Batch Run of All Cases Obtained by Using a Fixed Gs and an Updated Gs with disturbances without disturbances fixed Gs Yk(tf)a RMSE of ek
a

case 1 fixed Gs 0.6236 0.6222 0.0058 0.0049 updated Gs 0.6163 0.6155 0.0028 0.0023 fixed Gs 0.5917 0.5922 0.0120 0.0115

case 2 updated Gs 0.5951 0.5991 0.0099 0.0095

updated Gs 0.6088 0.6090 0.0015 0.0011

N)5 N ) 10 N)5 N ) 10

0.6087 0.6088 0.0018 0.0013

Yd(tf) ) 0.61.

6810 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003

Figure 3. Comparison of tracking performance in disturbance case 1: (a) RMSE of ek, (b) ek(tf).

changes of parameter k2 as

k2(k) ) k20[0.8 + 0.6 exp(-k/3)]

(75)

where k20 is the nominal value of parameter k2, as shown in Table 1. Equation 75 indicates that k2(k) varies from 1.23k20 to 0.8k20 as the batch index k increases from 1 to . In disturbance case 2, parameter k1 is simulated as k1(k) ) 0.85k10, where k10 is the nominal value of parameter k1. In these two cases, the results of using a fixed Gs are compared with those of using an updated Gs, and the results are given in Table 2. In disturbance case 1, the tracking performance is shown in Figure 3. In Figure 3a, when a fixed Gs is used, the RMSE of ek is reduced significantly in the first four batch runs, but after the fifth run, it gradually degrades because of the increased model-process mismatch. However, with an updated Gs, the model-process mismatch does not increasebecause of model updating, so the RMSE of ek in this case still converges after five batch runs. It is also demonstrated in Figure 3b that, by updating Gs, the endpoint tracking error ek(tf) converges more quickly than than when a fixed Gs is used, and in the case of N ) 10, the value of Yk(tf) of the last batch obtained by using an updated Gs is 0.6155, which is lower than the value 0.6222 obtained by using a fixed Gs, as shown in Table 2. The product profiles, Yk, and temperature profiles, Uk, of the 1st, 3rd, and 15th batch runs in the case of N ) 10 are shown in Figure 4 for a fixed Gs and in Figure 5 for an updated Gs. It can be observed that Yk of the 15th batch run under ILC with the fixed Gs is far from the desired Yd, whereas Yk of the 15th batch run under ILC with the

Figure 4. Output and input trajectories in disturbance case 1 using fixed Gs (N ) 10): (a) output Yk, (b) input Uk.

updated Gs is very close to Yd. Using these two different models, Uk converges to different trajectories, as shown in Figures 4b and 5b. Figure 6 shows the convergence of tracking errors in disturbance case 2 when N ) 10. It can be seen that although the tracking errors almost converge after 5 batch runs by using either fixed Gs or updated Gs, the performance of using updated Gs is better than that of using fixed Gs. The RMSE of ek reduces to 0.0095 by using updated Gs from the value, 0.0115, by using fixed Gs. The endpoint product quality Yk(tf) of the 15th batch is increased to 0.5991 by using updated Gs from the value, 0.5922, by using fixed Gs, as shown in Table 2. Parts a and b of Figure 7, respectively, show the product profiles Yk of the 1st, 3rd, and 15th batch runs based on fixed Gs and updated Gs in disturbance case 2 with N ) 10. Because there is a large amount of modelprocess mismatch between the fixed preestimated model Gs and the process, Yk of the 15th run is still quite far from the desired Yd in the later stage of a batch as shown in Figure 7a. However, when the updated Gs is used, Yk of the 15th run is improved to be closer to Yd than in the case of using fixed Gs, as can be seen from Figure 7b. 5.2. Case 2: Batch Polymerization Process. This example involves the thermally initiated bulk polymerization of styrene in a batch reactor. The differential equations describing the polymerization process were

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6811

Figure 6. Comparison of tracking performance in disturbance case 2 (N ) 10): (a) RMSE of ek, (b) ek(tf).

Figure 5. Output and input trajectories in disturbance case 1 using updated Gs (N ) 10): (a) output Yk, (b) input Uk.

given by Kwon and Evans through reaction mechanism analysis and laboratory testing.25 Gattu and Zafiriou reported the parameter values of the first-principles model.26 Dong et al. also used this model to demonstrate batch-to-batch optimization.2 The differential equations for this process are as follows

dx1 ) f1 ) dt Em F02F (1 - x1)2 exp(2x1 + 2x12)Am exp (76) Mm uTref f1x2 1400x2 dx 2 ) f2 ) 1dt 1 + x1 Aw exp(B/uTref)

f1 Aw exp(B/uTref) dx3 ) f3 ) - x3 dt 1 + x1 1500

with

(77) (78)

F)

x1 1 - x1 + r1 + r2Tc r3 + r4Tc F0 ) r1 + r2Tc (79)

Tc ) uTref - 273.15

where x1 is the conversion; x2 ) xn/xnf and x3 ) xw/xwf are the dimensionless number-average and weightaverage chain lengths (NACL and WACL), respectively; u ) T/Tref is the control variable; T is the absolute temperature of the reactor; Tc is the temperature in degrees Celsius; Aw and B are coefficients in the relation between the WACL and temperature obtained from experiments; Am and Em are the frequency factor and activation energy, respectively, of the overall monomer reaction; the constants r1-r4 are density-temperature corrections, and Mm and are the monomer molecular weight and polymer-monomer interaction parameter, respectively. Table 3 lists the reference values used to obtain the dimensionless variables, as well as the values of the reactor parameters. The final time, tf, was fixed to be 313 min.26 The initial values of the states used were x1(0) ) 0, x2(0) ) 1, and x3(0) ) 1. In this study, the above mechanistic model, eqs 7679, is assumed to be unavailable, and an LTV perturbation model is utilized to build the relationship between u and y ) [x1 x2 x3]T. Here, the batch length is divided into N equal stages, and two values of N are studied, N ) 10 and N ) 5. The desired product reference trajectory, Yd, was taken from the literature.26 Thirteen batches of process operation under different temperature profiles were simulated with the mechanistic model and used as the historical process data sets for building an LTV model. The model Gs were identified using eqs 21-23. The temperature Tc is constrained within the range 100 e Tc e 200. The parameters in ILC were set as Q

6812 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003

Figure 8. Convergence of RMSE of ek in the nominal case using fixed Gs.

Figure 7. Input and output trajectories in disturbance case 2 (N ) 10): (a) using fixed Gs, (b) using updated Gs. Table 3. Parameter Values for the Batch Polymerization Process parameter Am Aw B Em Mm r1 r3 r2 r4 Tref tf xnf xwf value 4.266 105 m3/(kmol s) 0.033 454 4364 K 10 103.5 K 104 kg/kmol 0.9328 103 kg/m3 1.0902 103 kg/m3 -0.879 02 kg/(m3 C) -0.59 kg/(m3 C) 399.15 K 313 min 700 1500 0.33

Figure 9. Convergence of ek(tf) in the nominal case using fixed Gs.

The corresponding control trajectories are shown in Figure 11. In this nominal case, the results using a fixed Gs are compared with those obtained using an updated Gs. The forgetting factor, , for updating Gs was selected as 0.98. Table 4 shows the performance for the 15th batch run. From Table 4, it can be seen that the performance based on the updated Gs is better than that based on the fixed Gs. In this example, we consider one disturbance case with batch-to-batch parametric changes simulated as

Am(k) ) Am0[1.2 - 0.25 exp(-k/3)]

(80)

) I and R ) 0.05I. The nominal case without any uncertainties in the batch reactor was initially considered. Figures 8 and 9 show, respectively, the RMSE of ek and the tracking errors at the batch endpoint ek(tf) in the nominal case using a fixed Gs. It can be seen from these figures that the RMSE of ek and ek(tf) have almost converged after about five batch runs for both cases. The results for N ) 10 are slightly better than those for N ) 5. Figure 10 shows the product profiles Yk of the 1st, 2nd, 5th, and 15th batch runs when N ) 10. It can be seen that Yk has almost converged to the desired reference Yd after only five batch runs under the ILC.

where Am0 is the nominal value of parameter Am given in Table 3. This equation represents the variation of Am(k) from 1.02Am0 to 1.2Am0 as the batch index k increases from 1 to . In this case with batch-to-batch process parameter variations, the results obtained using a fixed Gs were also compared with those obtained using an updated Gs, and the comparison results are reported in Table 4. Figure 12 shows the RMSE of ek. It can be observed from Figure 12 that, when a fixed Gs is used, the RMSE of ek reduces significantly in the second batch run, but after that, it is gradually degraded because of the increased model-process mismatch. However, when the model Gs is updated through the introduction of new process data, Gs is improved to match the process, so the RMSE of ek gradually decreases after five batch

Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003 6813

Figure 11. Convergence of Uk in the nominal case using fixed Gs (N ) 10). Table 4. Performance in the 15th Batch Run of All Cases Obtained by Using a Fixed Gs and an Updated Gs
without disturbances Yk(tf)a x1 x2 x3 RMSE of ek
a

with disturbances 0.8130 0.8008 1.0172 1.0181 1.0258 1.0228 0.0171 0.0146 0.8012 0.7965 1.0138 1.0142 1.0117 1.0125 0.0109 0.0097

fixed Gs updated Gs fixed Gs updated Gs N)5 N ) 10 N)5 N ) 10 N)5 N ) 10 N)5 N ) 10 0.7933 0.7938 1.0041 1.0010 1.0026 1.0005 0.0017 0.0006 0.7926 0.7919 1.0026 1.0028 1.0018 1.0014 0.0009 0.0003

Yd(tf) ) [0.792 1.003 1.001]T.

Figure 12. Comparison of RMSE of ek in the disturbance case.

Figure 10. Trajectories of quality variables in the nominal case using fixed Gs (N ) 10): (a) x1, (b) x2, (c) x3.

runs. Figure 13 shows that, by updating Gs, the endpoint tracking errors ek(tf) of all three product variables converge much closer to zero than those obtained using a fixed Gs. The endpoint product qualities of all three variables are improved, as shown in Table 4. 6. Conclusions A run-to-run model-based iterative learning control strategy for the tracking control of product quality in

batch processes is proposed. To address the problem of nonlinearities in batch processes, a time-varying perturbation model for product quality prediction, linearized around the nominal batch trajectories, is identified from process operating data using multivariate linear regression. To address the problem of model-plant mismatches, predictions of the LTV perturbation model for the current batch run are modified by the addition of the corresponding prediction errors of the immediately previous batch run. On the basis of the modified model predictions, an ILC law with direct error feedback is obtained explicitly. The nominal condition for the convergence of the tracking error under ILC is obtained. To address the problems of unmeasured disturbances and process variations, the perturbation model should be updated in a batchwise manner to capture the

6814 Ind. Eng. Chem. Res., Vol. 42, No. 26, 2003
(5) Srinivasan, B.; Bonvin, D.; Visser, E.; Palanke, S. Dynamic Optimization of Batch Processes II. Role of Measurements in Handling Uncertainty. Comput. Chem. Eng. 2003, 27, 27-44. (6) Gao, F.; Yang, Y.; Shao, C. Robust Iterative Learning Control with Applications to Injection Molding Process. Chem. Eng. Sci. 2001, 56, 7025-7034. (7) Chin, I. S.; Lee, K. S.; Lee, J. H. A Technique for Integrated Quality Control, Profile Control, and Constraint Handling for Batch Processes. Ind. Eng. Chem. Res. 2000, 39, 693-705. (8) Lee, K. S.; Lee, J. H. Convergence of Constrained Modelbased Predictive Control for Batch Processes. IEEE Trans. Autom. Control 2000, 45, 1928-1932. (9) Lee, J. H.; Lee, K. S.; Kim, W. C. Model-based Iterative Learning Control with a Quadratic Criterion for Time-varying Linear Systems. Automatica 2000, 36, 641-657. (10) Xu, J. X.; Hu, Q.; Lee, T. H.; Yamamoto, S. Iterative Learning Control with Smith Time Delay Compensator for Batch Process. J. Process Control 2001, 11, 321-328. (11) Campbell, W. J.; Firth, S. K.; Toprac, A. J.; Edgar T. F. A Comparison of Run-to-run Control Algorithms. In Proceedings of the 2002 American Control Conference; IEEE: Piscataway, NJ, 2002; pp 2150-2155. (12) Lee, K. S.; Chin, I. S.; Lee, H. J.; Lee, J. H. Model Predictive Control Technique Combined with Iterative Learning Control for Batch Processes. AIChE J. 1999, 45, 2175-2187. (13) Terwiesch, P.; Argarwal, M.; Rippin, D. W. T. Batch Unit Optimization with Imperfect Modelling: A Survey. J. Process Control 1994, 4, 238-258. (14) Tian, Y.; Zhang, J.; Morris, A. J. Optimal Control of a Batch Emulsion Copolymerisation Reactor based on Recurrent Neural Network Models. Chem. Eng. Process. 2002, 41, 531-538. (15) Moore, K. L. Iterative Learning Control for Deterministic SystemssAdvances in Industrial Control; Springer-Verlag: London, 1993. (16) Saab, S. A Discrete-time Learning Control Algorithm for a Class of Linear Time-invariant Systems. IEEE Trans. Autom. Control 1995, 40, 1138-1142. (17) Amann, N.; Owens, D. H.; Rogers, E. Iterative Learning Control for Discrete-time System with Exponential Rate of Convergence. IEE Proc. D: Control Theory Appl. 1996, 143, 217-224. (18) Amann, N.; Owens, D. H.; Rogers, E. Predictive Optimal Iterative Learning Control. Int. J. Control 1998, 69, 203-226. (19) Owens, D. H.; Munde, G. Error Convergence in an Adaptive Iterative Learning Controller. Int. J. Control 2000, 73, 851857. (20) Russell, S. A.; Kesavan, P.; Lee, J. H. Recursive Data-based Prediction and Control of Batch Product Quality. AIChE J. 1998, 44, 2442-2458. (21) Penlidis, A.; Ponnuswarmy, S. R.; Kiparissides, C.; ODriscoll, K. F., Polymer Reaction Engineering: Modeling Considerations for Control Studies. Chem. Eng. J. 1992, 50, 95-107. (22) Lee, K. S.; Lee, J. H. Model Predictive Control for Nonlinear Batch Processes with Asymptotically Perfect Tracking. Comput. Chem. Eng. 1997, 21, s873-s879. (23) Ray, W. H. Advanced Process Control; McGraw-Hill: New York, 1981. (24) Logsdon J. S.; Biegler L. T. Accurate Solution of Differential-Algebraic Optimization Problems. Ind. Eng. Chem. Res. 1989, 28, 1628-1639. (25) Kwon, Y. D.; Evans, L. B. A Coordinate Transformation Method for the Numerical Solution of Nonlinear Minimum-Time Control Problems. AIChE J. 1975, 21, 1158-1164. (26) Gattu, G.; Zafiriou, E. A Methodology for On-line Setpoint Modification for Batch Reactor Control in the Presence of Modeling Error. Chem. Eng. J. 1999, 75, 21-29.

Figure 13. Comparison of ek(tf) in the disturbance case (N ) 10).

changed batch process dynamics. It is proposed in this paper that a batchwise perturbation model be identified after each batch run, with the control trajectory for that batch being used as the nominal trajectory. A forgetting factor is introduced so that data from the more recent batches contribute more to the model identification than data from the earlier batches. The proposed technique is illustrated on a simulated batch reactor and a simulated batch polymerization process. The results demonstrate that the tracking errors can be significantly reduced by ILC based on the identified LTV perturbation model and the problems of model-plant mismatches and process varaitions can be effectively addressed through model updating. The proposed method assumes that all batch runs are of the same length. Further work is required to extend this approach to address situations where the batch runs can be of different lengths. Acknowledgment This work was supported by the UK EPSRC through Grant GR/N13319, Nonlinear Optimizing Control in Agile Batch Manufacturing, and Grant GR/R10875, Intelligent Computer Integrated Batch Manufacturing. Literature Cited
(1) Bonvin, D. Optimal Operation of Batch ReactorssA Personal View. J. Process Control 1998, 8, 355-368. (2) Dong, D.; McAvoy, T. J.; Zafiriou, E. Batch-to-Batch Optimization Using Neural Network Models. Ind. Eng. Chem. Res. 1996, 35, 2269-2276. (3) Clarke-Pringle, T. L.; MacGregor, J. F. Optimization of Molecular-Weight Distribution Using Batch-to-Batch Adjustments. Ind. Eng. Chem. Res. 1998, 37, 3660-3669. (4) Doyle, F. J., III; Harrison, C. A.; Crowley. T. J. Hybrid Model-based Approach to Batch-to-batch Control of Particle Size Distribution in Emulsion Polymerisation. Comput. Chem. Eng. 2003, 27, 1153-1163.

Received for review July 16, 2003 Revised manuscript received October 4, 2003 Accepted October 7, 2003 IE034006J