Automatica 36 (2000) 641}657

Model-based iterative learning control with a quadratic criterion for

time-varying linear systemsq
Jay H. Lee!,*, Kwang S. Lee", Won C. Kim"
!School of of Chemical Engineering, Purdue University, West Lafayette, IN 47907, USA
"Department of Chem. Eng., Sogang Univ. Shinsoo-1, Mapogu, Seoul 121-742, South Korea
Received 24 February 1997; revised 5 August 1998; received in "nal form 26 August 1999

Abstract
In this paper, iterative learning control (ILC) based on a quadratic performance criterion is revisited and generalized for
time-varying linear constrained systems with deterministic, stochastic disturbances and noises. The main intended area of application
for this generalized method is chemical process control, where excessive input movements are undesirable and many process variables
are subject to hard constraints. It is shown that, within the framework of the quadratic-criterion-based ILC (Q-ILC), various practical
issues such as constraints, disturbances, measurement noises, and model errors can be considered in a rigorous and systematic
manner. Algorithms for the deterministic case, the stochastic case, and the case with bounded parameter uncertainties are developed
and relevant properties such as the asymptotic convergence are established under some mild assumptions. Numerical examples are
provided to demonstrate the performance of the proposed algorithms. ( 2000 Elsevier Science Ltd. All rights reserved.
Keywords: Learning control; Predictive control; Robust control; Constraint satisfaction

1. Introduction
Iterative learning control (ILC) was originally proposed in the robotics community (Arimoto, Kawamura
& Miyazaki, 1984) as an intelligent teaching mechanism
for robot manipulators. The basic idea of ILC is to
improve the control signal for the present operation cycle
by feeding back the control error in the previous cycle.
Even though the mainstream ILC research has thus far
been carried out with mechanical systems in mind, chemical and other manufacturing processes could also bene"t
signi"cantly from it. Batch chemical processes such as the
batch reactor, batch distillation, and heat treatment processes for metallic or ceramic products are good examples. Traditionally, operations of these processes have
relied exclusively on PID feedback and logic-based controllers. Re"nement of input bias signals based on the
general concept of ILC can potentially enhance the

This paper was presented in '96 AIChE Annual Meeting in Chicago.

This paper was recommended for publication in revised form by Associate Editor P.J. Fleming under the direction of Editor S. Skogestad.
* Corresponding author. Tel.: 765-494-4088; fax: 765-494-0805.
E-mail address: jhl@ecn.purdue.edu (J.H. Lee)

performance of tracking control systems signi"cantly.

Diversi"cation of ILC applications to the above-mentioned problems is already starting to take place, as
evidenced by the comprehensive lists of recent ILC papers compiled by Chen (1998).
The classical formulation of ILC design problem has
been as follows: Find an update mechanism for the input
trajectory of a new cycle based on the information from
previous cycles so that the output trajectory converges
asymptotically to the desired reference trajectory.
The "rst-order ILC algorithms update the input trajectory u (de"ned over the same time interval) in the
following way (Moore, 1993):
u
"u #He .
(1)
k`1
k
k
In the above, eOy !y where y and y denote the output
$
$
and output reference trajectories which can be either
continuous or discrete signals de"ned over a "nite time
interval of [0, ]. The subscript k here represents the
batch/cycle index.
In the above, H called `learning "ltera is an operator
that maps the error signal e to the input update
k
signal u
!u . Within this somewhat restrictive
k`1
k
problem setup, the ILC design is reduced to choosing the
operator H.

0005-1098/00/$ - see front matter ( 2000 Elsevier Science Ltd. All rights reserved.
PII: S 0 0 0 5 - 1 0 9 8 ( 9 9 ) 0 0 1 9 4 - 6

642

J.H. Lee et al. / Automatica 36 (2000) 641}657

The prevalent approach thus far has been to assume

a simple structure for the learning "lter H and tune the
parameters to achieve the desired learning properties.
Examples of this type include D-type (Arimoto et al.,
1984), PID-type (Bondi, Casalino & Gambradella, 1988),
and their variants. As a straightforward extension of the
"rst-order algorithms, higher-order algorithms have
been proposed too (Bien & Huh, 1989). This line of
approaches, however, could yield only limited results for
general multivariable systems.
Model-based algorithms have also been proposed.
However, most algorithms proposed were based on the
notion of direct model inversion (Togai & Yamano, 1985;
Oh, Bien & Suh, 1988; Lucibello, 1992; Moore, 1993; Lee,
Bang & Chang, 1994b; Yamada, Watanabe, Tsuchiya
& Kaneko, 1994), that is, H"G~1 where G represents
the input}output map of the process. Since G~1 would
contain a di!erentiator(s) (in the continuous-time case),
the learning "lter based on the model inverse becomes
hyper-sensitive to high-frequency components in e .
k
Since, in most process control applications, smooth manipulation of actuators is at least as important as precise
control of outputs, these approaches cannot be used
directly.
Furthermore, the zero tracking error objective cannot
be satis"ed for general nonsquare MIMO processes.
Since it is not uncommon for industrial batch processes
to render a nonsquare problem for which zero tracking
error for all the output variables is impossible, a more
general objective appropriate for nonsquare processes is
needed.
There are certain additional traits and requirements
found in prototypical process control problems that motivate a more general (but perhaps more computationally
intensive) approach. First, most process variables are
subject to certain constraints that are set by physical or
safety considerations. Hence, it is desirable to have algorithms that incorporate the constraint information explicitly into the calculation. Second, dynamics of almost
all chemical processes are intrinsically nonlinear, and the
nonlinearities become exposed when the processes are
operated over a wide range of conditions, as in typical
industrial batch operations. For this reason, it is necessary to derive ILC algorithms that can accommodate
nonlinear system models, when available. Third, disturbances and noises are integral aspects of most process
control problems and must be dealt with in a systematic
fashion. Some disturbances, once they occur, tend to
repeat themselves in subsequent batches, while others
tend to be more speci"c to a particular batch. Most
disturbances exhibit signi"cant time correlation that
must be exploited for e$cient rejection. Finally, chemical
processes have a quite long interval allowed between two
adjacent batches and sample times can be chosen relatively large in relation to the total cycle time. These traits
should allow us to implement numerically more intensive

algorithms, such as those based on mathematical programming techniques.

Some of the aforementioned generalizations have already appeared in the literature. For example, to accommodate the nonsquare MIMO systems, the zero-tracking
error requirement has been relaxed to `minimum possible error in the least-squares sensea. This type of approach has been studied by Togai and Yamano (1985)
and also by Moore (1993). For the purpose of reducing
the noise sensitivity, Tao, Kosut and Aral (1994) proposed a discrete-time ILC algorithm based on the following least-squares objective with an input penalty term:
DDe DD2Q #DDu DD2R Pmin(DDeDD2Q #DDuDD2R ) as kPR.
(2)
k
k
A similar objective has also been considered by Sogo and
Adachi (1994) but in the continuous-time domain. These
algorithms can accommodate nonsquare MIMO systems
and mitigate the noise sensitivity by using the input
penalty term. However, by adding the quadratic penalty
term on the inputs directly, o!sets result, i.e., the algorithms fail to attain the minimum achievable error in the
limit. In addition, it is unclear how to best trade o! the
noise sensitivity against the speed of convergence and
output o!set, using the input weight matrix.
Recently, Amann, Owens and Rogers (1996) and Lee,
Kim and Lee (1996) have independently proposed to use
the following objective:
min [J "MDDe DD2Q #DDu !u
DD2 N].
k
k
k
k~1 R
u

(3)

Because the input change is penalized instead of the

input, the algorithm has an integral action (with respect
to the batch index) and achieves the minimum achievable
error in the limit. In the unconstrained, deterministic
setting, Amann et al. (1996) derived a noncausal input
updating law
u "u
#R~1GTQe
(4)
k
k~1
k
from LJ /Lu "0, while Lee et al. (1996) obtained
k k
u "u
#(GTQG#R)~1GTQe
(5)
k
k~1
k~1
which is indeed a rephrasing of (4) in a pure learning
form. Amann et al. (1996) transformed (4) to a causal
form by borrowing the idea from the solution of the
"nite-time quadratic optimal tracking problem. The resulting algorithm is a combination of a state feedback law
and a feedforward signal based on the error signal of the
previous cycle. In addition to signi"cant reduction in the
computational load, the feedback implementation gives
some robustness to disturbances and model errors. However, their algorithm is developed entirely in a deterministic setting (without direct references to disturbances)
and hence deserves further investigation. In a similar
spirit, Lee and Lee (1997) also showed that the Q-ILC
algorithm can be implemented as an output feedback

J.H. Lee et al. / Automatica 36 (2000) 641}657

algorithm, thus improving the robustness. Their realtime algorithm can be viewed as a combination of the
popular model predictive control (Lee, Morari & Garcia,
1994a) and the iterative learning control.
The objective of this paper is to provide a more general
and comprehensive framework for quadratic-criterionbased ILCs that is capable of addressing all the issues
that were mentioned to be important for process control
applications. We focus on the iterative learning control
implementation rather than the feedback implementation, keeping in mind the fact that the conversion to the
latter type of implementation can always be done in
a straightforward manner. We "rst introduce an error
transition model that represents the transition of tracking error trajectories between two adjacent batches. We
also discuss how the e!ects of disturbances of various
types can be integrated into the transition model. Based
on this model, one-batch-ahead quadratic optimal control algorithms are derived for both the unconstrained
and constrained cases. In addition, a robust ILC algorithm that minimizes the worst-case tracking error for
the next batch is proposed. For each algorithm, relevant
mathematical properties such as the convergence, robustness, and noise sensitivity are investigated.
The rest of the paper is organized as follows: In
Section 2, the static gain representation of the dynamic
system is introduced and is converted into an error
transition model. The ILC design objective is de"ned
based on the model description. In Section 3, the quadratic-criterion-based iterative learning control (Q-ILC)
algorithm is derived for the unconstrained case, and the
analysis of the relevant properties such as the convergence, noise sensitivity, and robustness follows. A realtime output feedback implementation of the algorithm is
discussed and a comprehensive comparison with Amann
et al.'s state-feedback/error-feedforward algorithm is
made. The constrained Q-ILC algorithm is presented
with the convergence proof in Section 4. In Section 5, the
robust Q-ILC algorithm is proposed with the convergence proof. Numerical examples are given in Section 6
and conclusions are drawn in Section 7.
2. Process description and problem statement
We assume that the underlying system is described by
the following static map between input and output sequences de"ned over the time domain of a batch operation of interest:
y "Gu #G d #b,
k
k
$ k
where
yT"[yT(1)
k
k
uT"[uT(0)
k
k
dT"[dT(1)
k
k

yT(2)2yT(N)],
k
k
uT(1)2uT(N!1)],
k
k
dT(2)2dT(N)].
k
k

(6)

(7)

643

In the above, y, u and d represent the output, input and

disturbance signals repspectively. Subscript k denotes the
batch index. b is a constant vector. Eq. (6) is linear a$ne
and is assumed to be invariant with respect to the batch
index. Since the operators G and G relate the output
$
trajectory to the input trajectories for an entire batch run,
the above formulation is general enough to include linear
time-varying models (which may be obtained, for instance, by linearizing a nonlinear system model along
a reference trajectory).
Let y denote the reference (desired) output sequence
$
and u the corresponding input sequence such that
$
y "Gu #b
(8)
$
$
Combining Eq. (6) with Eq. (8), we can write the equation
for the error trajectory e as
e Oy !y "G(u !u )!G d .
(9)
k
$
k
$
k
$ k
Generally, the exact input sequence that produces the
desired output sequence would not be known, even in the
absence of disturbances. Hence, we assume that u( , an
$
estimate for u , is given instead. Then, we have
$
e "G(u( !u )#G(u !u( )!G d ,
(10)
k
$
k
$
$
$ k
hgggigggj
e$
k

e$ includes the error in the input bias trajectory,

(u !u( ), as well as the e!ect of disturbances. Hence, it
$
$
has both deterministic and stochastic components in it.
For the ILC design, it is convenient to express e$ as
output of a linear stochastic system:
x% "Ax% #Bw ,
k
k
k`1

(11)
e$"Cx% #t ,
k
k
k
w and t are zero-mean (batch-index-wise) independent
k
k
identically distributed random vector variable sequences.
In most cases, it su$ces to work with the particular
model of
e6 $ "e6 $#w ,
k`1
k
k

(12)
e$"e6 $#t .
k
k
k
Note that e6 $ can be interpreted as the part of e$ that will
tend to repeat itself in the subsequent batches (e.g., the
error due to the bias in the input trajectory) while t can
be thought of as the part that is speci"c to the particular
batch. This simpli"ed representation makes sense since
the two types have entirely di!erent rami"cations in
iterative learning control.
Similarly, we can de"ne e6 as [e6 $#G(u( !u)], which
$
represents the part of the error trajectory e that will repeat
itself in the next batch with no further change in the disturbance or input trajectory. Writing the expression for e6
for two consecutive time steps and di!erencing yields

644

J.H. Lee et al. / Automatica 36 (2000) 641}657

the following transition model for the tracking error

trajectory:

3.1. Derivation of algorithms

e6
"G(u( !u
)#e6 $ ,
k`1
k`1
$
k`1

(13)

e6 "G(u( !u )#e6 $ ,
k
$
k
k

(14)

q
e6
"e6 !G*u
#w ,
k`1
k
k`1
k
e "e6 #t ,
k
k
k

(15)

where *u
"u
!u .
k`1
k`1
k
Given the model in the form of (15), our objective is to
design a learning algorithm in the form of *u
"f (I ),
k`1
k
where I denotes the information available after the
k
k batch. We require that the learning algorithm has the
5)
following property:
eTQe Pmin eTQe
k k
u
as kPR when w "t "0 k.
k
k

(16)

Remark.
f When G does not have a full row-rank, the
above system has uncontrollable modes at (1,0). This
means that it is impossible to make eTQe zero in
general. In this case, the requirement of (16) is equivalent to
eTQ1@2U UTQ1@2e P0 as kPR
k
k
# #

3. Unconstrained Q-ILC

(17)

where UT is an orthogonal projection matrix onto the

#
image space of Q1@2G. The matrix UTQ1@2G has a full
#
row-rank.
f By causality, G has a block lower-triangular shape
with time-varying pulse response coe$cient matrices
as its elements. For the standard case, we assume that
G is known exactly. We will, however, consider various forms of uncertainty throughout the paper.
f The error transition of a nonlinear batch process
can be approximated reasonably by (15) through
linearization of the nonlinear map around u . In this
k
case, G is replaced by G
. G
can be assumed
k`1
k`1
to be available after the k batch run by some
5)
means (e.g., model linearization). All the mathematical developments in the subsequent sections hold
for this batch-wise varying model, too, with minor
modi"cation.
f Both t and w are assumed zero-mean i.i.d. sequences
k
k
with respect to the batch index. For a particular k,
however, the elements of these vectors can be correlated. Hence, knowledge about the temporal correlation in the disturbance e!ects can be re#ected in the
choice of covariance matrices for them.

We consider solving the following quadratic subproblem upon the completion of the k batch to update the
5)
input trajectory for the k#1 batch:
5)
1
N,
(18)
min MeT Qe
#*uT R*u
k`1
k`1
2 k`1 k`1
u
* k`1
where Q and R are PD (positive-de"nite) matrices. We
may need an expectation operator for the quadratic cost
function depending on whether we are dealing with the
deterministic case or the stochastic case. Note the cost
function has a penalty term on the input change between
two adjacent batches. This term does not cause output
o!set since it vanishes if the input trajectory converges to
a limit.
3.1.1. Direct-error-based algorithm
For the deterministic case (for which w "t "0, i),
i
i
(15) simpli"es to
e "e !G*u
.
(19)
k`1
k
k`1
By solving a standard least-squares problem, we obtain
the following quadratic-criterion-based (Q-ILC) algorithm with direct error feedback:
u
"u #HQe ,
k`1
k
k
where

(20)

HQ"(GTQG#R)~1GTQ.

(21)

Through straightforward manipulation, (20) can be

converted to
u
"u #R~1GTQe
.
(22)
k`1
k
k`1
Amann et al. (1996) investigated mathematical properties
of their ILC algorithm based on this form.
This algorithm should work well, but can lead to
over-compensation when batch-speci"c disturbances and
noises (represented by t in our model) are signi"cant.
Disturbances and noises must be "ltered indirectly by
adjusting the input weighting matrix, which can be a
di$cult task.
3.1.2. Observer-based algorithm
When batch-speci"c disturbances and noises are
signi"cant, the following observer-based algorithm may
be considered:
u
"u #HQe( ,
(23)
k`1
k
k@k
where e( is the estimate of e6 based on the measurement
k@k
k
e . e(
is obtained from the following observer:
k k@k
e(
"e(
!G*u ,
k@k~1
k~1@k~1
k
e( "e(
#K(e !e(
),
(24)
k@k
k@k~1
k
k@k~1

J.H. Lee et al. / Automatica 36 (2000) 641}657

K is the "lter gain matrix and can be obtained through

various means. For instance, when the statistical information on w and t (e.g., their covariances) are available, we can derive the optimal "lter gain matrix using
the Kalman "ltering technique. It can also be chosen
based on the pole placement consideration. For instance,
choosing K"f I, 0(f)1, gives a stable observer with
a single tuning knob.
If w and t are i.i.d. Gaussian sequences and K is chosen
as the Kalman "lter gain matrix, the observer-based
Q-ILC algorithm yields an input trajectory that is optimal in the following sense:
D I N.
#*uT R*u
min 1EMeT Qe
k`1
k`1 k
k`1 k`1
2
u
* k`1

(25)

3.2. Real-time output feedback implementation

Even though the focus of this article is placed on
iterative learning control, it is worth noting that the
proposed algorithm can be implemented as a real-time
output feedback algorithm for enhanced robustness
against disturbances and model errors. For this, it can be
shown "rst that the batch-wise transition model (15) can
be converted into a time-wise transition model, described
by the following periodically time-varying state-space
system (Lee & Lee, 1997):

D C DC D C D

e6 (t#1)
I 0
k
"
e (t#1)
0 I
k

e6 (t)
G(t)
k
!
*u (t),
k
e (t)
G(t)
k

C D

e6 (t)
e (t)"[0 H(t)] k
, t"0,2, N!1,
k
e (t),
k

(26)

where e (t)Oe when *u (t)"2"*u (N!1)"0 and

k
k
k
k
e6 (t) is similarly de"ned. G(t) is the sub-block of G correk
sponding to the input at time t, and
H(t)"[

I
0
].
0
hij hij hij
ny C(t~1)>ny ny Cny ny C(N~t)>ny

(27)

The transition from the end of one batch to the start of

next batch is described by

e6
e

k`1
k`1

D C DC D C D

(0)

e6 (N)
I
k
# w
e (N)
I k
k

I 0

"

(0)

I 0

CD
0

k`1

(28)

Eqs. (26)}(28) de"ne a periodically time-varying system,

for which an optimal output feedback control law based
on the quadratic criterion of (18) can be derived in
a straightforward manner. For details, see Lee and Lee
(1997).

645

3.3. Comparison with Amann et al.'s algorithm

Amann et al. (1996) also proposed a state feedback
algorithm with a feedforward learning signal. They base
their algorithm on the following state-space model:
*x
(t#1)"A*x
(t)#B*u
(t),
k`1
k`1
k`1

(29)
e (t)"e (t)!C*x
(t), t"0,2, N!1.
k`1
k
k`1
The optimal state feedback law for the quadratic objective (18) can easily be derived by the application of
dynamic programming. The familiar Riccati recursion
for solving a "nite-horizon LQ tracking problem results.
e (t) is a given signal (just like the reference signal in the
k
LQ tracking problem) and does not complicate the LQ
controller design.
In comparing their algorithm with ours, we note the
following:
f In their algorithm, the e!ect of input represented by
y"Gu#b in our formulation is expressed through
a time-invariant state-space system,
x(t#1)"Ax(t)#Bu(t),
(30)
y(t)"Cx(t)#b.
Note that, if the model is indeed given in the above
form, it can easily be converted into the equivalent
choice of G simply by computing the pulse response
coe$cients. G can, in fact, be chosen to match any
time-varying linear state-space system, which would be
more appropriate for chemical batch processes.
f In their algorithm, they feed-forward the error signal
for the previous cycle directly. Our algorithm works
with the "ltered error signal e6 . This makes no di!erk
ence in the purely deterministic case or the case when
t "0, i. However, there invariably are disturbances
i
(such as measurement noises) that are speci"c to each
batch and will not repeat themselves in the future
cycles. Hence, the direct use of the error signal can
result in an over-compensation. The feedback will be
able to get rid of the e!ect of this over-compensation
only to a limited degree.
f Their algorithm, when implemented as a real-time
state feedback, certainly gives some extra robustness
against disturbances when compared to the pure
learning algorithms given in Section 3.1. However, the
disturbance model implicitly assumed in their deterministic algorithm, while it may be "ne for most mechanical systems, is thought to be too limiting for most
chemical process applications. It is straightforward to
see that their state feedback algorithm would be equivalent to our real-time output feedback algorithm (presented in Section 3.2) if the disturbance ewects can be
described through white noise added to the state variables in the state space model of (29). However, this

646

J.H. Lee et al. / Automatica 36 (2000) 641}657

hardly is the case in most chemical processes. Indeed, it

is often necessary to create extra states in order to
model the disturbance e!ects correctly (Morari &
Stephanopoulos, 1980). Furthermore, in most applications, state variables are not directly measurable and
must be estimated. The underlying disturbance model
for our algorithm can be much more general. In fact, it
does not even require the dynamic states to be de"ned
since the e!ect of inputs and disturbances are to be
modeled directly in the output space; the underlying
system can be a time-varying state-space system of any
order.
f A potential problem in implementing our algorithm
has to do with the large dimensionality of the matrix
G. The large dimensionality can lead to numerical
di$culties in computing the optimal learning gain
matrix HQ. The state-space model and the stage-wise
solution adopted in their work yields a Riccati recursion of relatively low dimension, which would be more
attractive from the computational viewpoint. However, in most process control applications, the sample
times can be chosen relatively large in relation to the
total batch time. Hence, the dimension of G, albeit
large, is not expected to be excessive from the viewpoint of matrix computation. In addition, in the case of
pure iterative learning implementation, there almost
always is su$cient time allowed between batches to
carry out most intensive computation. Furthermore, if
one chooses to go with the real-time output feedback
implementation discussed in Section 3.2, the computational load can be reduced signi"cantly by adopting
strategies such as the receding horizon control, where
only a "xed number of input moves are computed at
each time step (Lee et al., 1994a). The ultimate reason
for adopting the less elegant approach, however, is that
the stage-wise solution procedure cannot be applied to
the constrained case, which is the main interest for
process control applications.
3.4. Properties of the Q-ILC algorithms
3.4.1. Convergence
It is a standard result that a linear feedback system
with an observer is stable if the observer and the feedback
control system are individually stable. Since (15) is completely observable, the observer can always be designed
to be stable. Hence, it is su$cient to investigate
the convergence behavior of the direct error feedback
algorithm.
We allow G not to have a full row-rank in order to
demonstrate how the orthogonal projection to the image
space of Q1@2G is applied.
From (19) and (20), we obtain
e "(I!L)e ,
k`1
k
where LOGHQ.

(31)

Now we introduce an orthogonal matrix UT which

separates the controllable subspace (i.e., the image space
of Q1@2G) from the uncontrollable subspace.

C D

C D

UT
# Q1@2G
# OUTQ1@2G"
UT
0
6#
with U~1"UT.

(32)

Also de"ne

C D
e

# OUTQ1@2e.
6#

(33)

By the above transformation, (31) can be separated

into the dynamics of e and e . The following exponen#
6#
tial convergence of the controllable output has been
established by Lee et al. (1996) and also by Amann et al.
(1996).
Theorem 1. Under the condition that t "w "0 k, the
k
k
controllable part of e6 from system (15) converges exponenk
tially to the origin under the observer-based Q-ILC of (24).
At the same time, *u P0 as kPR.
k
3.4.2. Sensitivity to high-frequency errors
Sensitivity of HQ to high-frequency components in
e can be assessed from the learning "lter gain. It can be
k
readily seen that the following inequality holds:
DDHQDD "DD(GTQG#R)~1GTQDD
=
=
p (G)p (Q)
p (G)p (Q)
.!9
.!9
4 .!9
4 .!9
,
p (GTQG#R)
p (R)
.*/
.*/

(34)

where p
and p
denote maximum and minimum
.!9
.*/
singular values, respectively.
On the other hand, the "lter gain of the inverse modelbased ILC (I-ILC) algorithm represented by
u
"u #HIe where HI"G~1
k`1
k
k
(assuming G is invertible) is

(35)

DDHIDD "DDG~1DD "1/p (G).

(36)
=
=
.*/
In the above, p (G) is the gain for the input direction
.!9
with the largest ampli"cation, which corresponds to the
steady-state gain for an overdamped SISO system. From
this fact, we can see that the upper bound of DDHQDD
=
remains constant irrespective of the sampling period
once Q and R are "xed. Contrary to this, p (G) corre.*/
sponds to the gain of the associated transfer function
at the Nyquist frequency. As a consequence, DDHIDD in=
creases inde"nitely as the sampling period is decreased.
This indicates that I-ILC may show extreme sensitivity to
high-frequency components of the output error for short
sampling periods.

J.H. Lee et al. / Automatica 36 (2000) 641}657

3.4.3. Disturbance ewects

E!ects of disturbances can be investigated by examining the transfer functions between the disturbances and
the output error. By combining (15), (23), and (24), we can
derive the following relationship:
e6 "![(1!q~1)I#L(qI!(I!K)(I!L))~1K]~1
k
]L(qI!(I!K)(I!L))~1Kt
k
![(1!q~1)I#L(qI!(I!K)(I!L))~1K]~1w ,
k
(37)
where q~1 represents the backward shift operator with
respect to the batch index.
From (37), we "rst note that K and L appear symmetrically. In fact, when K"I (direct error feedback or no
"ltering on t ), (37) is reduced to
k
e6 "!(qI!(I!L))~1Lt !q(qI!(I!L))~1w . (38)
k
k
k
When L"I (R"0 for invertible processes), (37) is simpli"ed to
e6 "!(qI!(I!K))~1Kt !q(qI!(I!K))~1w . (39)
k
k
k
The above relationships tell us that the learning gain and
the "lter gain essentially play the same role in shaping the
disturbance rejection property, which means I-ILC combined with an observer can be made to have the same
disturbance rejection property as direct-error-feedback
Q-ILC does. However, from the viewpoint of design, it is
convenient to have the #exibility of choosing K. For
example, if the statistics of the disturbance signals (e.g.,
their covariances) are given, it is more straightforward to
use the optimal state estimation theory to obtain the
proper choice for K. Tuning the input weighting matrix
to obtain the same e!ect must be done on a trial-anderror basis and can be very di$cult.
Eqs. (38) and (39) indicate how the tuning of L or
K a!ect the rejection of batch-to-batch random disturbance, t , and batch-wise correlated disturbances, which
k
is caused by w .
k
3.4.4. Robustness
Another advantage of Q-ILC over I-ILC is the availability of tuning parameters like the input weighting
matrix and "lter gain matrix that can be used to enhance
the robustness against model uncertainty. Let us demonstrate this fact by considering a simple scenario where
there is a zero-frequency gain error.
Gr"(1#a)G,

(40)

where a is a scalar factor representing structured multiplicative uncertainty. The error evolution equation for
the true system is written as
e "(I!GrHQ)e "(I!L!aL)e .
(41)
k`1
k
k
For simplicity, let us also assume that Gr has a full
row-rank.

647

Now let the SVD of G be G"U diag(g )VT and supi

pose R is chosen as
R"V diag(r2)VT.
i
Then, we have

(42)

I!L!aL
g2
i . (43)
"U diag(l )UT where l "1!(1#a)
i
i
g2#r2
i
i
For convergence, Dl D(1 i, which is satis"ed when
i
!1(a(J1#2(r2/g2) i.
i i
It is interesting to note the lower bound of a is "xed at
!1 while the upper bound can be increased inde"nitely
by increasing r2. Even when a is less than !1, the rate of
i
divergence can be controlled through r2. On the other
i
hand, I-ILC does not o!er this #exibility ("xing r to be
i
0 (as in I-ILC) "xes the upper bound).
The above observation may remain valid for more
general uncertainty structures. Generalization of the
above analysis to more complex uncertainty structures,
however, appears to be a formidable task. See Section 5
for a more rigorous way of handling known model uncertainties at the design stage.

4. Constrained Q-ILC
4.1. Derivation of algorithms
4.1.1. Direct-error-feedback algorithm
In many industrial process control applications,
certain restrictions need to be placed on the process
variables in order to ensure safe, smooth operations.
Commonly, constraints imposed on the inputs (its raw
values as well as changes with respect to the time and
batch index) and the outputs. These constraints are often
expressed in the form of mathematical inequalities.
f For the raw values of inputs,
u-084u
4u)*.
(44)
k`1
f For the rate of input changes with respect to the time
index,
du-084du 4du)*,
(45)
k`1
where d denotes the di!erence along the discrete-time
index.
f For the rate of input changes with respect to the batch
index,
*u-084*u
4*u)*.
(46)
k`1
The above constraint may be useful to con"ne
the input adjustments to the linear regime when a
successive linear approximation (around u ) is used
k
to represent a nonlinear batch process.

648

J.H. Lee et al. / Automatica 36 (2000) 641}657

f For outputs, the following soft constraints are often

used to avoid the problem with potential infeasibility
(Za"riou & Chiou, 1993):
y-08!e
4y
4y)*#e
,
e
'0. (47)
k`1
k`1
k`1
k`1
All the above constraint equations can be rearranged as
linear inequalities with respect to *u
. First, the input
k`1
constraints (44) becomes

where

CD C D
I

!I

Cu"

!J
G

!G

u-08!u 4*u
4u)*!u .
(48)
k
k`1
k
Under the assumption that du(0)"u(0), du
can be
k`1
written as

u
(0)
k`1
u
(1)!u
(0)
k`1
k`1
F

du "
k`1

"Ju
,
k`1

(49)

u
(N!1)!u
(N!2)
k`1
k`1

where

!I

J"

2
!I }

0 .

2 !I

(50)

Hence, the input rate constraints with respect to the time

index become
du-08!Ju 4J*u
4du)*!Ju .
k
k`1
k
The output constraints can also be rearranged as

(51)

and

*u-08H
k
!*u)*H
k
du-08!Ju
k
C
"
k`1
!du)*#Ju
k
y-08!y !e
k
k`1
!y)*#y !e
k
k`1

*u-08H"max(u-08!u , *u-08),
k
k
(57)
*u)*H"min(u)*!u , *u)*).
k
k
Now by substituting (19) into (18) and adding the penalty
term for the slack variable in the soft constraints, we
obtain
1
min
M*uT (GTQG#R)*u
k`1
k`1
2
u
k`1
k`1
* ,e
N.
(58)
#eT Se
!2eTQG*u
k`1 k`1
k
k`1
Since the feasible region is de"ned by the linear inequalities, the above minimization together with the constraints equation of (55) constitutes a standard quadratic
programming (QP) problem.
4.1.2. Observer-based algorithm
In the observer-based algorithm, we need to replace
e in (58) with e( and y in (55) with y( "y !e( . The
k
k@k
k
k@k
$
k@k
algorithm is summarized as follows:

y-08!y !e
4G*u
4y)*!y #e
.
(52)
k
k`1
k`1
k
k`1
The above constraints can be combined together into the
following linear inequality:

1
min
M*uT (GTQG#R)*u
k`1
k`1
2
*uk`1 ,ek`1
!2e( T QG*u
#eT Se
N
k@k
k`1
k`1 k`1
subject to

CM u*u
5CM
,
k`1
k`1
where

C6*u
5C
,e
50,
k`1
k`1@k k`1
where

e
50,
k`1

(53)

CD C D
I

!I
J

CM u"

!J
I

CM
"
k`1

!I

u-08!u
k
!u)*#u
k
du-08!Ju
k
!du)*#Ju
k
*u-08

!I

Cu"

!*u)*

!J

!G

y-08!y !e
k
k`1
!G
!y)*#y !e
k
k`1
The constraints on the input values and the rate of input
changes with respect to the batch index can be easily
combined together:
e
50,
k`1

Cu*u
5C
,
k`1
k`1

CD C
I

(54)

(55)

(56)

(59)

(60)

*u-08H
k
!*u)*H
k
du-08!Ju
k
C
"
.
k`1@k
!du)*#Ju
k
y-08!y #e( !e
$
k@k
k`1
!y)*#y !e( !e
$
k@k
k`1
(61)

4.2. Computational issues

Quadratic programming is convex and is therefore a fundamentally tractable optimization problem.
However, the large dimensions of the hessian, gradient as

J.H. Lee et al. / Automatica 36 (2000) 641}657

well as the constraint vector can demand excessive computation time and lead to numerical complications. In
ILC, since the computation is performed between two
batches, there is su$cient time for carrying out numerically intensive computations such as the QP formulated
above. On the other hand, if the real-time output feedback implementation is desired, one would probably
have to reduce the computational load by adopting strategies such as the receding horizon control (Lee et al.,
1994a). Such on-line optimization based control strategies have already been used quite successfully in vast
number of continuous process applications (Qin &
Badgwell, 1997). On the other hand, these types of algorithms would not be suitable for applications involving
mechanical systems, where sampling intervals tend to be
very short and the breaks between the cycles are also
relatively insigni"cant.
4.3. Convergence proofs
In proving the asymptotic zeroing of the tracking error
under the constrained Q-ILC algorithm, we make the
following assumptions:
A.1. G has a full row-rank.
A.2. Input constraints, reference trajectory y , and
$
steady-state disturbance d are such that the zero=
ing of the error is possible with an input in the
feasible set, i.e., &u= such that y !Gu=!
$
G d #b"0, u-084u=4u)*
and
du-084
$ =
du=4du)*. In addition, *u-08(0 and *u)*'0.
A.3. Output constraints (with e"0) are satis"ed when
e"0.
A.4. Q, R, and S are positive de"nite.
The "rst assumption is made for simplicity. If G is rowrank de"cient, we can introduce the projection operator
UT and consider the convergence of the controllable part
of e as we did in Section 3.4.1. The assumptions A.2
k
and A.3 are clearly necessary in order for any algorithm to achieve zero tracking error while satisfying
the constraints. The last assumption is quite reasonable.
The convergence of the constrained Q-ILC is shown for
the direct-error-feedback algorithm and then for the observer-based algorithm.
Theorem 2. Under assumptions A.1}A.4 and t "w "0,
k
k
system (19) converges to the origin under the constrained
Q-ILC of (55), (58), i.e., e P0 and *u P0 as kPR.
k
k
Proof. Let us de"ne
J(e , u )
k k

1
"
min
'
O MeT Qe
k`1
2 k`1 k`1
(ek`1 ,*uk`1 ,ek`1 )|)k`1

N 50,
#eT Se
#*uT R*u
k`1 k`1
k`1
k`1

(62)

649

where )
is a convex set de"ned by the constraints in
k`1
(55) as well as the model equation e "e !G*u
.
k`1
k
k`1
Note that (e , 0, e )3)
since e
"e and e
"e
k
k
k`1
k`1
k
k`1
k
when we "x *u
"0. The cost at (e , 0, e ) is always
k`1
k
k
greater than or equal to the optimal cost, which implies
J(e , u )4'
D
k k
k`1 (ek ,0,ek )
"J(e
,u
)!1*uTR*u .
2 k
k
k~1 k~1
From this, it follows that

(63)

k 1
04J(e , u )# + *uTR*u 4J(e , u )(R.
(64)
k k
i
0 0
2 i
i/1
Hence, *u P0 as kPR.
k
Since *u P0, *u-08(0 and *u)*'0, the constraint
k
*u-084*u
4*u)* becomes inactive for large enough
k`1
k. Then, from the assumptions, (0, *u=, 0), where
k
*u="u=!u , is a feasible point. Moreover, it is clear
k
k
that the optimal solution (e
, *u
,e
) of J(e , u ) is
k`1
k`1 k`1
k k
also a feasible point. By the convexity of )
, any point
k`1
that lies between these two feasible points is feasible.
Now consider the directional derivative of cost function
'
from (e
, *u
,e
) to (0, *u=, 0).
k
k`1
k`1
k`1 k`1
!e
k`1
+'T D ek`1 uk`1 k`1 *u=!*u
k`1 ( ,* ,e )
k
k`1
!e
k`1

!e
k`1
"[eT Q *uT R eT S] *u=!*u
k`1
k`1
k
k`1
k`1
!e
k`1
)
"!eT Qe !eT Se #*uT R(*u=!*u
k
k`1
k`1
k`1 k`1
k`1 k`1
!*uT R*u
!eT Se
"!eT Qe
k`1
k`1
k`1 k`1
k`1 k`1
(65)
#*uT R*u=50.
k
k`1
The last inequality comes from the optimality of
(e
, *u
,e
). It follows that
k`1
k`1 k`1
*uT R*u 5eT Qe
#eT Se
k`1
=
k`1 k`1
k`1 k`1
50.
(66)
#*uT R*u
k`1
k`1
Since *u P0 as kPR, e P0 as kPR follows. h
k
k
In the operation of batch processes, time-varying constraints may be given to prevent abnormal process action
or to maintain the process within linear region. The
above result still holds for this more general case, given
that the above assumptions are satis"ed.
The theorem can be extended to the constrained observer-based algorithm. If we de"ne e8
Oe6
!
k`1
k`1
e(
, the dynamics of the observer error becomes
k`1@k
e8 "(I!K)e8
.
(67)
k
k~1

650

J.H. Lee et al. / Automatica 36 (2000) 641}657

Since the dynamics of e8 can be assumed to be exponentik

ally stable, there exist D'0 and o3(0,1) such that
DDe8 DD4DokDDe8 DD
(68)
k
0
for any compatible norm. For notational simplicity, we
also de"ne
m Oe( !e(
"Ke8 .
(69)
k
k@k
k@k~1
k
In the proof of the theorem, the following inequality
plays an important role:

4(JJ(e(
,u
)!1*uTR*u
k~1@k~1 k~1
2 k
k
#J1mTQm #J1gTSg )2.
2 k k
2 k k

(72)

Then
JJ(e( , u )4JJ(e(
,u
)#J1mTQm
2 k k
k@k k
k~1@k~1 k~1
#J1gTSg
2 k k
4JJ(e(
,u
)#aDDKDD DDe8 DD,
k~1@k~1 k~1
i k

(73)

where a"Jj (Q)/2#Jj (S)/2 and DD ) DD denotes

.!9
.!9
i
an induced operator norm. This implies

Fact.
1
[(a#b)TQ(a#b)#(c#d)TS(c#d)]
2
4[J1(aTQa#cTSc)#J1bTQb#J1dTSd]2, (70)
2
2
2
where Q and S are positive dexnite. Application of the
Schwarz inequality after expansion of the left-hand side
yields the above.
Theorem 3. Under assumptions A.1}A.4 and t "w "0,
k
k
system (15) converges to the origin under the observerbased constrained Q-ILC algorithm of (59), (60), i.e., e P0
k
and *u P0 as kPR.
k
Proof. Let
J(e( , u )
k@k k
"
min
['K
O1
k`1 2
e(
u
K
k`1@k
k`1
k`1
k`1
,* ,e )|)
(
N]
Qe(
#*uT R*u #eT Se
]Me( T
k`1 k`1
k`1
k`1
k`1@k k`1@k
50,
(71)
where )K
is a convex set de"ned by the constraints in
k`1
(60) as well as the observer equation in (24).
When *u
"0, e(
"e( . From e( "e(
#
k`1
k`1@k
k@k
k@k
k@k~1
m , we can see (e(
#m , 0, e #g ) is a feasible soluk
k@k~1
k
k
k
tion of (71) where

m
k
g O
k
!m

when m '0
k
elementwise.
when m 40
k
k
Through the same reasoning as in the proof of
Theorem 2 and using (70), we have
J(e( , u )4'K
D
k@k k
k`1 (e( k@k~1 `mk ,0,ek `gk )
1
#m )TQ(e(
#m )
" [(e(
k
k@k~1
k
2 k@k~1
#(e #g )TS(e #g )]
k
k
k
k
Qe(
#eTSe ]#J1mTQm
4(J1[e( T
k k
2 k@k~1 k@k~1
2 k k
#J1gTSg )2
2 k k

k
JJ(e( , u )4JJ(e( , u )#aDDKDD + DDe8 DD.
(74)
k@k k
0@0 0
i
j
j/1
From the exponential stability of e8 , there exists J.!9'0
k
such that
J(e( , u )4J.!9(R k
k@k k
Hence, expansion of (72) using the above leads to
J(e( , u )4J(e(
,u
)
k@k k
k~1@k~1 k~1
!1*uTR*u #aDDe8 DD#bDDe8 DD2,
2 k
k
k
k
where

(75)

(76)

a"2aJJ.!9DDKDD ,
i
b"(aDDKDD )2.
i
To this end,
k 1
J(e( , u )# + *uTR*u
k@k k
j
2 j
j/1
k
4J(e( , u )# + [aDojDDe8 DD#(cDoj)2DDe8 DD2](R.
0 0
0
0
j/1
(77)
Hence, *u P0 as kPR. Through the same arguments
k
as in the proof of Theorem 2, this implies e(
P0 as
k@k~1
kPR. Since m P0 as kPR, e( P0 as kPR. Since
k
k@k
e8 P0 as kPR, e P0 as kPR. h
k
k
5. Robust Q-ILC
5.1. Derivation of algorithm
In the previous sections, we have not incorporated
any model uncertainty information explicitly into the
algorithm. In this section, we propose a robust Q-ILC
algorithm that guarantees convergence and provides
optimal performance for a certain class of model
uncertainty.

J.H. Lee et al. / Automatica 36 (2000) 641}657

Let us consider the case where our error update model

is given as
e (h)"e !G(h)*u .
k`1
k
k`1

(78)

In the above, the gain matrix is parameterized in terms of

an uncertain vector h. We discuss the deterministic case
only here, but nonzero w and t can be dealt with by using
an observer as before.
We make the following assumptions:
B.1
B.2
B.3

B.4

h3# where # is a compact and convex set.

G(h) is an a$ne function of h.
G(h) has the same row-rank for every h3#. In
addition, as in Section 4.3, we assume that G(h) has
a full row-rank in order to prove the asymptotic
convergence.
Q, R, and S are positive de"nite.

In the above, B.1 and B.2 are needed for the subsequently
presented robust Q-ILC algorithm to be a convex
program and therefore computationally feasible. In the
robust control literature, this linear a$ne uncertainty
description is quite popular and is not considered to be
very limiting (Morari & Za"riou, 1989; Campo &
Morari, 1987; Lee & Cooley, 1997). Assumption B.3 is
needed in order to prove the convergence (convergence
cannot be achieved without this assumption since the
loss of rank means the loss of controllability), but the
algorithm can be implemented even when this assumption is not satis"ed.
Under the above assumptions, one sensible criterion
for determining *u
is the following one-batch-ahead
k`1
worst-case error minimization:
min
* |

u
U
k`1
k`1

(h)
max M(
O1[eT (h)Qe
k`1
k`1 2 k`1
h|#
]N
#eT Se
#*uT R*u
k`1 k`1
k`1
k`1

(79)

with
e (h)"e !G(h)*u ,
k`1
k
k`1

y-08#e !G(h)*u !y
k
k`1
d
if y !e #G(h)*u
(y-08,
d
k
k`1
0

e
"
k`1
if y-084y !e #G(h)*u
4y)*,
d
k
k`1
!y)*!e #G(h)*u
#y
k
k`1
d
if y !e #G(h)*u
'y)*,
d
k
k`1

651

5.2. Computational issues

The min}max optimization de"ned above is a convex
programming problem. To see this, substitute the model
equation into the objective function to rewrite (
as
k`1
(
"1[*uT (G(h)TQG(h)#R)*u
k`1 2
k`1
k`1
]. (82)
!2eTQG(h)*u #eTQe #eT Se
k`1 k`1
k k
k
k`1
We can do the same for e
. Then, it is easy to see that
k`1
(
is convex in both h (since it is convex in G * see
k`1
assumption B.2) and *u
independently. Now de"ne
k`1

hH(*u
)"arg max (
.
(83)
k`1
k`1
h|#
One immediate consequence of the convexity is that
hH(*u) lies on the boundary of #. If # is a polytope, the
maximum occurs at one of the vertices.
In addition, it is easy to show that max (
is
h|# k`1
a convex function of *u
. Note that, for any
k`1
*u , *u 3U
and a3[0,1], we have
1
2
k`1
a( D u1 H u1 #(1!a)(
D
k`1 (* ,h (* ))
k`1 (*u2 ,hH(*u2 ))
5a(
D
#(1!a)(
D
k`1 (*u1 ,hH(*ua ))
k`1 (*u2 ,hH(*ua ))
,
(84)
5(
D
k`1 (*ua ,hH(*ua ))
where *u "a*u #(1!a)*u . The "rst inequality is
a
1
2
because hH(*u ) and hH(*u ) are the worst-case param1
2
eters. The second is due to the convexity of (
with
k`1
respect to *u. Since U
is a convex set, the min}max
k`1
optimization is a convex programming problem for
which global optimum can be found. There exist e$cient
algorithms for these types of problems, such as the cutting-plane method discussed in Boyd and Barratt (1991).
Once again, with su$cient time allowed between batches,
it is conceivable to use the min}max algorithm to update
the input trajectory for an upcoming batch.

(80)

5.3. Convergence

(81)

Convergence will be proved for the unconstrained case

only. Extension of the proof to the constrained case will
be discussed afterwards. The unconstrained algorithm
can be rewritten as

where U
is a convex set de"ned by the inequality
k`1
constraints in (55) excluding the output constraints. Because the gain matrix depends on h, output prediction
also depends on h. To distinguish the model prediction
from the actual output, we use e(h) for the predicted value
and e for the true value.

min max M(
]N
(h)#*uT R*u
O1[eT (h)Qe
k`1
k`1
k`1
k`1 2 k`1
*uk`1 h|#
(85)
with the model equation constraint e
(h)"e !
k`1
k
G(h)*u .
k`1
Theorem 4. Under assumptions B.1}B.4, system (78) converges to the origin under the robust Q-ILC algorithm of
(85), i.e., e P0 and *u P0 as kPR.
k
k

652

J.H. Lee et al. / Automatica 36 (2000) 641}657

Proof. For the unconstrained case, the objective function

in (85) can be rewritten as

guarantees the existence of an input trajectory leading to

the zero tracking error.

(
"1[*uT (G(h)TQG(h)#R)*u
k`1
k`1
k`1 2
!2eTQG(h)*u #eTQe ].
k k
k
k`1
We de"ne

6. Numerical illustrations

(86)

<(e )"min max (

50.
(87)
k
k`1
u
k`1
*
h|#
Let hH be the maximizer for the min}max problem for the
k
previous (k ) batch. Then, it follows that
5)
1
<(e )4 eTQe since e
"e with *u
"0
k
k`1
k
k`1
2 k k
1
4 eT(hH)Qe (hH) since hH is the maximizer
k k
k
2 k k
1
"<(e
)! *uTR*u
k
k~1
2 k
k 1
(88)
N<(e )# + *uTR*u 4<(e )(R.
j
0
k
2 j
j/1
Hence, *u P0 as kPR. In addition, <(e )P< ,
k
k
=
which together with *u P0 implies that e Pe .
k
k
=
Next, we show the convergence of e to zero by contrak
diction. For this, let us assume that e O0 when
=
*u "0. Then the gradient of ( at *u "0 is
=
=
=
+( D u= 0 "!G(h)Qe .
(89)
=* /
=
Since G(h)Qe is an a$ne function of h and # is a con=
vex, compact set, $"MG(h)Qe : h3#N forms a convex,
=
compact set. In addition, $ does not include the origin,
since Q'0, G(h) has a full row rank, and e O0 by
=
assumption.
This means there exists a nonzero direction d of *u
u
=
that makes angle of more than 903 degrees with $. In
other words,
&d s.t.!eT QGT(h)d (0 h3#.
(90)
u
=
u
This contradicts the result that *u "0 and proves
=
e P0. h
k
Extending the proof to the constrained algorithm
involves some technical complications. The input rate
constraint and the output constraint do not cause any
problem. However, the input magnitude constraint may
become active along the progression, preventing the convergence to zero tracking error. The input magnitude
constraint is not particularly meaningful in the context of
our problem setup. Note that, when h is allowed to vary
from one batch to next, there is no one-to-one correspondence between y and u, even for a "xed sequence of
h. Hence, it is di$cult to state a formal condition that

Performance of the proposed Q-ILC algorithms are

demonstrated through four numerical examples. In the
"rst three examples, we considered batch systems whose
input}output map, G, is derived by sampling the zeroorder-hold (ZOH) equivalent of a continuous-time model
over [0,40]. Sampling period di!ers depending on the
examples.
Example 1 (Unconstrained Q-ILC). It was assumed that
G for the true process is derived from the following
continuous-time model with sampling period of h"0.25:
0.8
G (s)"
.
1
(5s#1)(3s#1)

(91)

Filtered square wave signal corrupted with zero-mean

Gaussian i.i.d. sequence with variance 0.0052 was assumed for w (shown in Fig. 1) and a zero-mean Gaussian
k
i.i.d. sequence with the same variance was considered
for t .
k
The learning controllers were designed based on the
nominal model derived from
1.2
G (s)"
.
.
(6s#1)(2s#1)

(92)

Note that there are signi"cant model errors in the steady

state gain as well as in the dynamic gain.
In Figs. 2 and 3, we compare the performance of I-ILC
and unconstrained direct error feedback Q-ILC. In
Q-ILC, we used Q"I and R"0.02I. We can see that, in
both cases, the output converges to the reference trajectory with similar convergence pattern. The input from
I-ILC, however, shows violent oscillation and spikes due
to its extreme sensitivity to high-frequency errors. In fact,

Fig. 1. Disturbance pattern considered in Example 1.

J.H. Lee et al. / Automatica 36 (2000) 641}657

Fig. 2. Performance of I-ILC for Example 1.

653

Fig. 4. Performance of I-ILC combined with an observer for

Example 1.

To the same system, this time, we applied I-ILC

combined with an observer to investigate how the
asserted similarity (in Section 3.4.3) of the role of K and
L in the closed-loop response is a!ected by model error.
For this purpose, we set K"L"G(GTQG#R)~1GT.
The result is given in Fig. 4. We can notice the results closely resemble those in Fig. 3, which manifests
the similarity of the observer and controller gains in
the closed-loop dynamics even in case of model uncertainty.
Next, we considered the nominal model derived from
1.2
G (s)"
.
6s#1

Fig. 3. Performance of direct error feedback Q-ILC for Example 1.

the I-ILC input shows a tendency to diverge after the "fth

iteration. On the other hand, Q-ILC maintains very
smooth, moderated input signal.

(93)

which has larger model uncertainty at high frequencies

than (92) does. Note that this is the case where I-ILC
diverges. We tried observer-based Q-ILC with
Q"I, R"0.05I and K"0.5I. To enhance high-frequency robustness, we considered a larger R than in the
previous case. Fig. 5 shows the performance. We can see
the designed Q-ILC shows a clear tendency to converge
while rejecting the unknown repetitive disturbance even
when there is signi"cant model uncertainty.
Example 2 (Constrained Q-ILC). In this example, constrained observer-based Q-ILC of the batch process was

654

J.H. Lee et al. / Automatica 36 (2000) 641}657

Fig. 5. Performances of observer-based Q-ILC for Example 1.

derived from
0.8
G (s)"
1
15s2#8s#1

(94)

with sampling interval h"1. Constraints were imposed

on the input magnitude and input change. u
and
.!9
u are shown in Fig. 6. *u
and *u were chosen as
.*/
.!9
.*/
#5 and !5, respectively. Remember that convergence
of the constrained algorithm proved in Section 4.3 was
only for the zero model error case. To demonstrate the
robustness, in this example, we assumed the nominal
model derived from
1.2
G (s)"
.
.
12s2#8s#1

Example 3 (Robust Q-ILC). In robust Q-ILC, we assumed that the true process model is described by

1
0.8e~s
G (s)"
#h
1
(5s#1)(3s#1)
5s#1

with h"1. For !0.854h40.16, G(h) has a full rowrank. Outside this range, there exist h's which lead G(h) to
lose its row-rank. h for the true process was assumed to
vary according to the sequence, M!0.8, 0.1, !0.6, !0.2,
!0.7, !0.5, !0.3, !0.2, !0.7, !0.3N for the "rst 10
batches. In order to reduce computations in the min}max
optimization, h was assumed to have discrete values from
!0.8 to 0.1 with an interval of 0.1.
Fig. 7 shows the performance of robust Q-ILC. In this
"gure, we plot *u instead of u since the process is
k
k
represented as in (96). As has been proved, the output
is observed to converge to the desired trajectory as the
batch number increases.

(95)

Q"I and R"0.02I were used in the simulation. Fig. 6

shows the results of the numerical simulation. As illustrated in the "gure, the output converges to the reference
trajectory notwithstanding the constraints imposed on
input movements.

e (h)"e !G(h)*u ,
k`1
k
k`1
where G(h) is derived from

Fig. 6. Performance of constrained Q-ILC for Example 2.

(96)

(97)

Example 4 (Q-ILC of a nonlinear batch reactor). As the

"nal example, we considered an input re"nement problem for temperature control of a nonlinear chemical
batch reactor where a second-order exothermic reaction
APB takes place. It is assumed that the reactor has
a cooling jacket whose temperature is directly manipulated. The following equations describe the dynamics of
the reactor system:
d
;A
"!
(! )
j
dt
MC
p
(!*H)<
k e~E@RTC2 ,
#
A
0
MC
p

(0)" ,
I

(98)

J.H. Lee et al. / Automatica 36 (2000) 641}657

655

Fig. 8. Test inputs and resulting outputs for identi"cation of the nonlinear batch reactor in Example 4.

Fig. 7. Performances of robust Q-ILC for Example 3.

dC
A "!k e~E@RTC2 ,
0
A
dt

C (0)"C .
A
AI

(99)

The following values were used for the parameters:

;A/MC "0.09(1/min)
p
](!*H)</MC "1.64(K l/mol)
p
k "2.53]1019(l/mol min)E/R"13,550(3K)
0
C "0.9(mol/l)
AI
"25, 23(3C)
I
To obtain a linear model for the Q-ILC design, we
carried out two open-loop tests where the di!erence
between the two inputs is kept constant throughout the
batch. Fig. 8 shows the test inputs and the resulting
output responses. The di!erence between the two output
responses provides the step response from which the
pulse response coe$cients for construction of G are
derived. During identi"cation experiments, was kept
I
at 253C. The time-invariant linear model obtained this
way was used throughout the successive batch runs without further updating. Zero-mean i.i.d. noise with
covariance 0.12 was assumed to corrupt and input
magnitude was constrained as shown in Fig. 9. In this
example, we chose Q"I and R"0.1I.

Fig. 9. Result of Q-ILC of the nonlinear batch reactor in Example 4.

Fig. 9 shows the result of constrained direct error

feedback Q-ILC. During successive Q-ILC runs, we intentionally decreased to 233C as a main source of
I
disturbance. To compensate the initial temperature
change, moves high at initial times up to the imposed
j
constraint. We observed that, except for a small deviation
around the initial time, the reactor temperature approaches the reference trajectory as the iteration continues while producing quite smooth input movements.

656

J.H. Lee et al. / Automatica 36 (2000) 641}657

7. Conclusions
In this paper, it was argued that the existing ILC
algorithms, despite their successes in controlling mechanical systems, are not well-suited for process control applications. Motivated by this, we presented new modelbased iterative learning control algorithms that were
tailored speci"cally for this type of application. The algorithms were based on quadratic performance criteria and
were designed to consider the issues relevant to process
control, such as disturbances, noises, nonlinearities, constraints, and model errors. We proved the convergence of
the error signal under the proposed algorithms. Investigation of other relevant properties such as the noise
sensitivity and robustness along with some numerical
studies indicated that these algorithms should perform as
intended but at the expense of added computational
requirements.

Acknowledgements
The "rst author (JHL) gratefully acknowledges the
"nancial support from the National Science Foundation's Young Investigator Program (USA) under the
Grant CTS d9357827. The second author would like to
acknowledge LG Yonam Foundation (Korea), the Automation Research Center (Korea) at POSTECH and
Korea Science and Engineering Foundation for "nancial
support.

References
Amann, N., Owens, D. H., & Rogers, E. (1996). Iterative
learning control for discrete-time system with exponential rate of
convergence. IEE Proceedings Control Theory Applications, 143(2),
217}224.
Arimoto, S., Kawamura, S., & Miyazaki, F. (1984). Bettering operation of robots by learning. Journal of Robotic Systems, 1(1),
123}140.
Bien, Z., & Huh, K. M. (1989). Higher-order iterative learning control
algorithm. IEE Proceedings Part D, 136(3), 105}112.
Bondi, P., Casalino, G., & Gambradella, L. (1988). On the iterative
learning control theory for robotic manipulators. IEEE Journal of
Robotics Automation, 4(1), 14}22.
Boyd, S. P., & Barratt, C. H. (1991). Linear controller design}limits of
performance. Englewood Cli!s: Prentice-Hall.
Chen, Y. (1998). A bibliographical library for ILC research.
http://shuya.ml.org:888/&yqchen/ILC/ilcref.html.
Campo, P. J., & Morari, M. (1987). Robust model predictive control.
Proceedings of ACC, Minneapolis, MN (pp. 1021}1026).
Lee, J. H., & Cooley, B. (1997). Stable min}max control of statespace systems with bounded input matrix. Proceedings of ACC,
Alberquerque, NM, June (pp. 2945}2949). A full version will appear
in Automatica in 2000.
Lee, J. H., Morari, M., & Garcia, C. E. (1994a). State space
interpretation of model predictive control. Automatica, 30(4),
707}717.

Lee, K. S., Bang, S. H., & Chang, K. S. (1994b). Feedback-assisted

iterative learning control based on an inverse process model. Journal of Process and Control, 4(2), 77}89.
Lee, K. S., Kim, W. C., & Lee, J. H. (1996). Model-based iterative learning control with quadratic criterion for linear batch
processes. Journal of Control Automation Systems Engineering, 2(3),
148}157.
Lee, K. S., & Lee, J. H. (1997). Constrained model-based predictive
control combined with iterative learning for batch or repetitive
processes. Proceedings of ASCC, Seoul, July, 1, 33}36. A full
version will appear in IEEE Transactions on Automatic Control in
2000.
Lucibello, P. (1992). Learning control of linear systems. Proceedings of
ACC, Chicago, June (pp. 1888}1892).
Moore, K. L. (1993). Iterative learning control for deterministic systems.
London: Springer.
Morari, M., & Stephanopoulos, G. (1980). Minimizing unobservability
in inferential control schemes. International Journal of Control,
31, 367.
Morari, M., & Za"riou, E. (1989). Robust process control. Englewood
Cli!s: Prentice-Hall.
Oh, S. R., Bien, Z., & Suh, I. H. (1988). An iterative learning control
method with application for the robot manipulator. IEEE Journal of
Robotics Automation, 4(5), 508}514.
Qin, S. J., & Badgwell, T. A. (1997). An overview of industrial
model predictive control technology. In J. C. Kantor, C. E. Garcia.,
& B. Carnahan, Fifth international conference on chemical process
control (pp. 232}256). CACHE, ALChE.
Sogo, T., & Adachi, N. (1994). A gradient-type learning control
algorithm for linear systems. Proceedings of ASCC, vol. 3
(pp. 227}230). Tokyo, July.
Tao, K. M., Kosut, R. L., & Aral, G. (1994). Learning feedforward
control. In Proceedings of ACC, Baltimore, Maryland, June
(pp. 2575}2579).
Togai, M., & Yamano, O. (1985). Analysis and design of an optimal
learning control scheme for industrial robots: A discrete system approach. In Proceedings of 24th IEEE conference on
decision and control, Ft. Lauderdale, Florida, December (pp.
1399}1404).
Yamada, K., Watanabe, K., Tsuchiya, M., & Kaneko, T. (1994).
Robust control design for repetitive control systems with "ltered
inverse. Proceedings of ASCC, vol. 1 (pp. 243}246). Tokyo, Japan,
July.
Za"riou, E., & Chiou, H. W. (1993). Output constraint softening for
SISO model predictive control. Proceedings of ACC, San Francisco,
CA, June (pp. 372}376).

Jay H. Lee was born in Seoul, Korea, in

1965. He obtained his B.S. degree in
Chemical Engineering from the University
of Washington, Seattle, in 1986, and his
Ph.D. degree in Chemical Engineering
from California Institute of Technology,
Pasadena, in 1991. From 1991 to 1998,
he was with the Department of Chemical
Engineering at Auburn University, AL, as
an Assistant Professor and an Associate
Professor. Since 1998, he has been with
School of Chemical Engineering at Purdue
University, West Lafayette, where he currently holds the rank of Associate Professor. He has held visiting appointments at E. I. Du Pont de
Numours, Wilmington, in 1994 and at Seoul National University,
Seoul, Korea, in 1997. He was a recipient of the National Science
Foundation's Young Investigator Award in 1993. His research interests
are in the areas of system identi"cation, robust control, model predictive control and nonlinear estimation.

J.H. Lee et al. / Automatica 36 (2000) 641}657

Dr. Kwang Soon Lee was born in Korea in
1955. He graduated from the Seoul
National University in 1977 with B.S. in
Chemical Engineering. He then entered
KAIST and obtained Ph.D. in Chemical
Engineering in 1983 in the area of process
control. Since 1983, he joined the faculty of
the Department of Chemical Engineering
at Sogang University, Seoul, Korea, where
he is currently a Professor and the Department Chair. He held visiting appointments
at the University of Waterloo, Canada, in
1986 and at Auburn University, Alabama, in 1995. His research has
covered the topics of batch process control, model predictive control,
model reduction, and system identi"cation.

657
Won Cheol Kim was born in Yeosu, Koea
in 1963. He received MSc and PhD degrees
in Chemical Engineering from Sogang
University, Seoul, Korea, in 1990 and
1997, respectively. He is currently with
Conwell Co.,Ltd. His main research interests are iterative learning control, model
predictive control, and statistical process
control.