Asian Journal of Control, Vol. 4, No. 1, pp.

21-29, March 2002 21

AN ITERATIVE LEARNING CONTROL OF NONLINEAR SYSTEMS

USING NEURAL NETWORK DESIGN
Chiang-Ju Chien and Li-Chen Fu

ABSTRACT

In this paper, a feedforward neural network with sigmoid hidden units is

used to design a neural network based iterative learning controller for nonlin-
ear systems with state dependent input gains. No prior offline training phase
is necessary, and only a single neural network is employed. All the weights of
the neurons are tuned during the iteration process in order to achieve the
desired learning performance. The adaptive laws for the weights of neurons
and the analysis of learning performance are determined via Lyapunov-like
analysis. A projection learning algorithm is used to prevent drifting of weights.
It is shown that the tracking error vector will asymptotically converges to zero
as the iteration goes to infinity, and the all adjustable parameters as well as
internal signals remain bounded.

KeyWords: Iterative learning control, neural network, adaptive control, non-

linear system.

I. INTRODUCTION design adaptive controllers. Furthermore, Gaussian neu-

Iterative learning control (ILC) has become one of
the most effective control strategies for dealing with
repeated tracking control problems or periodic distur-
bance rejection problems for nonlinear systems. The ILC
system improves control by means of some simple self-
tuning processes without using accurate system models
and can be applied in some practical applications, such as
robotics, servo motors, etc. Traditional D-type or P-type
iterative learning controllers [1-8] have been developed
during the past decade for nonlinear systems with
nonlinearities satisfying the global Lipschitz continuous
condition. Recently, ILCs combined with other control
methodologies, such as model reference iterative learning
control [9], adaptive iterative learning control [10], robust
iterative learning control [11] and adaptive robust iterative
learning control [12], have been widely studied. Actually,
for the control of general nonlinear systems, neural net-
work based controllers have also become important in the
past decade. Multilayer feedforward neural networks or
recurrent neural networks [13-16] have been used to
Manuscript received September 11, 2000; accepted February
6, 2001.
Chiang-Ju Chien is with Department of Electronic Engine-
ering, Huafan University, 223, Shihtin, Taipei Hsien, Taiwan.
Li-Chen Fu is with Department of Electrical Engineering,
National Taiwan University, Taipei, Taiwan.
This work is supported by the National Science Council,
R.O.C., under Grant NSC89-2218-E-211-001.
ral networks and fuzzy logic systems have also become
popular tools for adaptive control of nonlinear systems
since they can be expressed as series expansion of basis
functions. Adaptive tracking control architectures have
been presented in [17-20] for Gaussian or fuzzy networks
for which explicit linear parameterization of the
nonlinearity is either unknown or impossible. For the
iterative learning control problem, for example, an adap-
tive nonlinear compensation ILC using a fuzzy approxi-
mation technique can be found in [21]. However, there
have been few works on the design of neural or fuzzy
network based ILCs.
In this paper, a feedforward neural network with
sigmoid hidden units is used to design a neural network
based iterative learning controller for a class of nonlinear
systems. Compared with previous works, which used a
neural network or fuzzy logic system in controller design,
the main advantages of the proposed neural network are
that no prior offline training phase is necessary, and that
only a single neural network is employed. Unlike a
Gaussian network or fuzzy network, in which the center
and width of radial Gaussian function or fuzzy member-
ship function need to be fixed before the controller can be
designed, all the weights of the neurons are tuned during
the iteration process. The proposed iterative learning
controller updates weights in the next iteration in order to
achieve the desired learning performance. The adaptive
laws for the weights of neurons are determined via
Lyapunov-like analysis. A projection learning algorithm
is used to prevent drifting of weights. This neural network
22 Asian Journal of Control, Vol. 4, No. 1, March 2002

based ILC algorithm can be applied to a nonlinear system
with state dependent input gain. Compared with works
dealing with the same case of state dependent input gain
(for example, [17,20] for the adaptive tracking control
problem or [21] for the adaptive iterative learning control
problem), we only require a known lower bound of the
input gain before the controller is designed. It is shown
that the tracking error vector asymptotically converges to
zero as the iteration goes to infinity, and that all adjustable
parameters as well as the internal signals remain bounded.
This paper is organized as follows. In Section 2, we
first introduce the feedforward neural network and then
derive the linear parameterization structure. The plant
description, neural network based adaptive ILC algorithm,
adaptation laws and performance analysis are presented in
Section 3. To illustrate application of the proposed ILC
system, use of an inverted pendulum system in computer
simulation is presented in Section 4. Finally conclusions
are drawn in Section 5.
II. THE FEEDFORWARD NEURAL
NETWORKS
In this paper, a feedforward neural network is used to
design an adaptive iterative learning controller for nonlin-
ear systems. The structure of the neural network is shown
in Appendix A. Let n denote the dimension of input vector
X of nonlinear function f(X), and let ρ denote the number
of neurons; the neural network which acts as an
approximator of the nonlinear function f(X) is shown in
matrix form as follows:
f(X, W, V) = W σ (V X a ),
T T
(1)
where W ∈ Rρ × 1 and V ∈ R(n + 1) × ρ are the output-hidden
weight matrix and hidden-input weight matrix, respectively,
X ∈ Rn × 1 is the input vector of the neural network, and
Xa ≡ [XT, 1]T ∈ R(n + 1) × 1 is the augmented neural input
vector (the –1 term denotes the input bias). The vector
This artificial neural network is a promising tool for
use in control applications because a multilayer feedforward
network with as few as one hidden layer can approximate
any “well-behaved” nonlinear function to any desired
degree of accuracy [22,23]. This universal approximation
theory is stated formally in the following theorem:
Theorem 1. [22] Let X ∈ A (a compact subset of Rn),
f(X) : A → R1 is a continuous function. For an arbitrary
constant : > 0, there exists an integer ρ (the number of
hidden neurons) and real constant optimal weight matrices
W* ∈ Rρ × 1 and V* ∈ R(n + 1) × ρ such that
f(X) = f *(X, W *, V *) + :f(X), (2)
where :f(X) is the approximation error satisfying :f(X) ≤
:, ∀X ∈ A. The optimal approximator can be described as
f *(X, W *, V *) ≡ W *Tσ(V *TXa).
Given the result of Theorem 1, there should exist W
and V as X ∈ A such that
W* ≤ W ∈ R +, V ≤ V ∈ R +,
*
F F
* 2
where ⋅ F
denotes the Frobenius norm, and W F
≡
* 2
tr{W *TW *}, V F ≡ tr{V *TV *}. For this neural network,
the approximation error f can be expressed in a linearly
parameterized form modulo a residual term. This is stated
in the following theorem.
Theorem 2. Define the estimation errors of the weight
matrix as W ≡ W – W *, V ≡ V – V * and the estimation error
of the hidden layer vector as σ ≡ σ(V Xa) – σ(V *TXa); then,
T
the function approximation error f is
f ≡ f(X, W, V) – f(X) = W ⋅ (σ – σ ′V X a )
T T

σ(V Xa) ∈ Rρ × 1 is defined as

T

+ W σ ′V X a + r,
T T
(3)
σ 1(V X a )
T
1
where the residual term r can be bounded by a linear in
σ (V X a ) ≡
T
parameter function, i.e.,
σ ρ(V ρX a )
T

r < θ * TY, (4)

where V = [V 1 , V 2 , …, V ρ ] with V i ∈ R(n + 1) × 1 being the ith where the unknown parameter vector θ* ∈ R4 × 1 is com-
column vector, and posed of optimal weight matrices and some bounded
constants, and the known function vector Y = [1, X a ,
σ i(V i X a ) ≡
T 1 ∈ R , i = 1, …, ρ,
T
1 + exp ( – V i X a ) X a W F, X a V F
]T. Here, ⋅ denotes the 2-norm of
a vector.
is a sigmoid function. For the sake of convenience, we de-
fine σi(V i Xa) ≡ σ i for each i = 1, …, ρ, and σ(V Xa) ≡ σ .
T T
Proof. Please see Appendix B.
 C.J. Chien and L.C. Fu: An Iterative Learning Control for Nonlinear Systems Using Neural Network Design 23

III. THE NEURAL NETWORK BASED mial λn – 1 + cn – 1λn – 2 + … + c1. It is noted that s j(0) = 0 by
ADAPTIVE ITERATIVE assumption (A4). The neural network based adaptive
LEARNING CONTROLLER iterative learning controller is designed as follows:

In this section, we will consider a nonlinear system

u j = U m – sgn (s j) 1 + 1 U m ,
j j
which can perform a given task repeatedly over a finite (7)
bL
time interval [0, T] as follows:

x 1(t)j = x 2(t)j 1 if s j > 0

j
x 2(t)j = x 3(t)j sgn (s ) = 0 if s j = 0 ,
– 1 if s j < 0

x n (t)j = – f(X(t)j) + b(X(t)j)u(t)j, (5) j

where U m is the main control force defined as

where X(t)j = [x1(t)j, …, xn(t)j] ∈ Rn is the state vector of the n –1

system, u(t)j is the control input, and f(X(t)j) and b(X(t)j)

j
U m = – ks j – Σ c ie ji + 1 + x (n)d + U jml + U jmf
i =1
(8)
are the unknown continuous functions of state. Here, j
denotes the index of iteration and t ∈ [0, T]. In order to j j
with k > 0. U ml and U mf , defined as the neural network
simplify the notations, the time index t will be omitted in based learning component and feedback component of the
the following discussion unless otherwise specified. main control force, are given as follows:
The control objective is to force the state vector Xj =
[x 1 , x j2 , …, x jn ]T = [x j1 , x j1 , …, x (n1 – 1), j ]T to follow some
j
U jml = f(X j, W p , V p ) – sgn (s j)θ p Y j,
j j jT
(9)
specified desired trajectory Xd = [xd, x d , …, x (nd – 1)]T for t ∈
[0, T]. In order to achieve the above control objective,
U jmf = – γ ⋅ s j ⋅ (σ j – σ ′ jV p X ja )T(σ j – σ ′ jV p X ja )
jT jT
some assumptions about the nonlinear system and desired
trajectory are given as follows:
– γ ⋅ s j ⋅ tr (X ja W p σ ′ j)T(X ja W p σ ′ j)
jT jT
(A1) The nonlinear functions f(Xj) and b(Xj) are bounded
if Xj is bounded.
– γ ⋅ sj ⋅ Y Y ,
(A2) There exists a positive known constant bL, such that jT j
(10)
0 < bL < b(Xj) for all Xj ∈ Rn.
(A3) The desired state trajectory Xd = [xd, x d , …, x (nd – 1)]T is j j
where γ > 0 and f (Xj, W p , V p ) is the neural network defined
in (1). In this controller, W p ∈ Rρ × 1, V p ∈ R(n + 1) × ρ, θ p ∈
j j
measurable and bounded. j

(A4) Let the state errors e j1 , …, e jn be defined as e j1 = x j1 –
xd, e j2 = x j1 – x d , …, e jn = x (n1 – 1), j – x (nd – 1). The initial
state errors at each iteration are zero, i.e., e j1 (0) = …
= e jn (0) = 0.
Remark. Assumption (A1) is reasonable in most physical
plants. Assumption (A2) needs only the lower bound of
b(Xj), which can be found in most of the related works [17,
20,21] dealing with the case of state dependent input gain.
The final assumption (A4) is a typical requirement in the
R4 × 1 are the control parameters to be tuned via some
suitable adaptive laws between successive iterations. In
order to explain the design ideas behind the proposed ILC
algorithm, we first differentiate sj along the system trajec-
tory (5) with respect to time t as follows:
n –1
sj = Σ c ie ji + 1 – x (n)d – f(X j) + b(X j)u j
i =1
n –1

field of iterative learning control theory for perfect track- = Σ c ie ji + 1 – x (n)d – f(X j) + u j + (b(X j) – 1)u j.
i =1
(11)
ing during time interval [0, T].
Substituting the controller (7), (8) and (9) into (11),
Based on the above assumptions and the neural and using the fact (3), we have
network given in Section 2, we will now propose a neural
network based adaptive iterative learning controller for
s j = – ks j + f(X j, W p , V p ) – sgn (s j)θ p Y j – f(X j)
j j jT
solving the control problem. First, we define a switching
function as follows:
– sgn (s j) 1 + 1 U m + (b(X j) – 1)u j + U jmf
j

s j = c 1e j1 + c 2e j2 + … + c n – 1e jn – 1 + e jn , (6) bL

= – ks j + W p ⋅ (σ j – σ ′ jV p X ja ) + W p σ ′ jV p X ja
jT jT jT jT
where c1, …, cn – 1 are the coefficients of Hurwitz polyno-
24 Asian Journal of Control, Vol. 4, No. 1, March 2002

Now, we are ready to state the main result for this

– sgn (s j)θ p Y + r j – sgn (s j) 1 + 1 b(X ) U m
jT j j j
bL neural network based ILC algorithm.

Theorem 3. Consider the nonlinear system (5) satisfying

+ (b(X j) – 1)U m + U mf.
j j
(12)
assumptions (A1)-(A4). If the neural network based
iterative learning controller is designed as in (6)-(10) with
The design of the controller in (7) and (8) will be
the adaptation laws in (13)-(15), then the state tracking
clear if we choose suitable adaptive laws for the control
j errors e 1j (t), …, e jn (t) will asymptotically converge to zero
parameters. Adaptation algorithms for control gains W p ,
for all t ∈ [0, T] as iteration j → ∞, and all adjustable
V p , θ p are needed since the optimal neural weights are, in
j j
parameters as well as the internal signals will remain
general, unknown. They are chosen as follows in order to
bounded.
achieve the desired learning performance:
Proof. Define the following cost functions of performance:
= W p – γ ⋅ s j ⋅ (σ j – σ ′ jV p X ja ),
j +1 j jT
W (13)
t
J p (t) = 1γ [W p W p + tr{V p V p } + θ p θ p ]dz,
j jT j jT j jT j

= V – γ ⋅ s ⋅ (X W σ ′ ),
j +1 j j j jT j
V p a p (14) 0

t
J j(t) = 1γ [W W + tr{V V } + θ θ ]dz.
jT j jT j
θ = θp + γ ⋅ sj ⋅ Y ,
j +1 j j jT j
(15)
0

and the projection algorithms are given as We can derive the main inequality for convergence
analysis as follows:
) = [proj(W 1 ), …, proj(W ρ )]T,
j +1 j +1 j +1 j +1
Wp = proj(W
J p (t) – J p (t) ≤ J
j +1 j j +1 j
(t) – J p (t)
j +1 j +1
proj(V 1, 1 ) proj(V 1, ρ ) t
= 1γ }+θ θ
j + 1T j +1 j + 1T j +1 j + 1T j +1
V
j +1
= proj(V
j +1
)= , [W W + tr{V V
p 0
j +1 j +1
proj(V n + 1, 1) proj(V n + 1, ρ )
– W p W p – tr{V p V p } – θ p θ p ]dz
jT j jT j jT j

θp = proj(θ ) = [proj(θ 1 ), …, proj(θ 4 )]T,

j +1 j +1 j +1 j +1
t
[ – 2s jW p ⋅ (σ j – σ ′ jV p X ja )
jT jT
=
0
where proj denotes the projection mechanism:
+ γ(s j)2(σ j – σ ′ jV p X ja )T(σ j – σ ′ jV p X ja )
jT jT

z if z j + 1 ≥ z
– 2s jtr{V p X a W p σ ′ j}
jT j jT
proj(z j + 1) = – z if z j + 1 ≤ – z
z j + 1 otherwise
+ γ(s j) tr{(X a W p σ ′ j)T(X a W p σ ′ j)}
2 j jT j jT

with z being the upper bound of z *. According to the + 2 s j θ p Y j + γ(s j)2Y jTY j]dz.
jT
(16)
projection algorithm, the parameter errors between the
control parameters and the desired parameters will be
bounded for all iterations and for all t ∈ [0, T]. Furthermore, On the other hand, using facts (4) and (12), we find
j j
if we define the parameter errors as W p = W p – W *, V p =
j that
V p – V *, θ p = θ p – θ * and W = W – W *, V = V – V *, θ
j j j j j j j j

= θ – θ *, then we have W W ≥ W p W p , tr{V V } ≥ tr

j jT j jT j jT j
d (s j)2 = 2s js j
{V p V p }, θ θ ≥ θ p θ p , and
jT j jT j jT j dt

= 2s j ⋅ ( – ks j + W p ⋅ (σ j – σ ′ jV p X a )
jT jT j

= W – γ ⋅ s ⋅ (σ – σ ′ V X ),
j +1 j j j j jT j
W p p a

+ W p σ ′ jV p X ja – sgn (s j)θ p Y j + r j
jT jT jT

= V – γ ⋅ s ⋅ (X W σ ′ ),
j +1 j j j jT j
V p a p

– sgn (s j) 1 + 1 b(X j) U jm
θ = θ p + γ ⋅ s j ⋅ Y j.
j +1 j
bL
 C.J. Chien and L.C. Fu: An Iterative Learning Control for Nonlinear Systems Using Neural Network Design 25

+ (b(X j) – 1)U jm + U jmf) t

≤
2 2
[ – 2k(s j) ]dz – (s j(t)) .
0

≤ – 2k(s j) + 2s jW p ⋅ (σ j – σ ′ jV p X a )
2 j jT jT

Thus, we have
+ 2s W σ ′ V X – 2 s θ Y
j jT j jT j j jT j
p p a p
(s j(t))2 ≤ J jp (t) – J jp + 1(t) ≤ J 1p (t)

– 2 sj 1 + 1 b(X j) U j
bL m for iteration j ≥ 1. This further implies that (sj(t))2 is
bounded for all t ∈ [0, T], and that lim s j(t) = 0 for all t ∈
j→∞

+ 2 sj 1 + 1 b(X j) U j + 2s jU j [0, T]. Finally, based on the definition of e j1 , e j2 , …, e jn in

j m
b(X )
≤ – 2k(s j) + 2s jW p ⋅ (σ j – σ ′ jV p X a )
2 jT
mf
Assumption (A4) and the switching function sj in (6), it can
be easily concluded that the tracking errors e j1 , e j2 , …, e jn
jT j will be bounded for all iterations and will converge to zero
as iteration j approaches infinity for all t ∈ [0, T]. ■

+ 2s jW p σ ′ jV p X ja – 2 s j θ p Y j + 2s jU jmf.
jT jT jT
(17)
IV. SIMULATION EXAMPLE
If we integrate (17) over time interval [0, t], then In this section, we will use the proposed neural
network based ILC to control an inverted pendulum to
t t
d (s j)2dz ≤ [ – 2k(s j) + 2s jW p ⋅ (σ j – σ ′ jV p X a )
2 jT jT j track a sine-wave trajectory. The dynamic equation of the
0 dz 0 inverted pendulum system is given as [24]

+ 2s jW p σ ′ jV p X ja – 2 s j θ p Y j + 2s jU jmf]dz,
jT jT jT
x j1 = x j2

which implies that 2

m (x j2) cos x j1sin x j1 cos x j1
g sin x j1 – mc + m mc + m
t x j2 = + u j,
[ – 2s jW p (σ j – σ ′ jV p X a )
jT jT j
2 j 2 j
0
4 – m cos x 1 4 – m cos x 1
3 mc + m 3 mc + m
– 2s jW p σ ′ jV p X ja + 2 s j θ p Y j]dz
jT jT jT

where x j1 and x j2 denote the angular displacement and

t velocity of the pole, respectively, g = 9.8 m/s 2 is the
≤
2 j 2 2
[ – 2k(s j) + 2s jU mf]dz – (s j(t)) + (s j(0)) . acceleration due to gravity, mc is the mass of the cart, m is
0
(18) the mass of the pole, is the half-length of the pole, and uj
is the applied force (control). In this simulation, we choose
It is noted that sj(0) = 0 by assumption (A4), and that mc = 1 kg, m = 0.1 kg, and = 1 m. The control objective
tr{V X a W p σ ′ j } = W p σ ′ j V p X a . Now, substituting (18)
jT
j
p
jT jT jT j
is to make the state vector Xj = [x j1 , x j2 ]T track the desired
and (10) into (16) yields trajectory Xd = [sin(t), cos(t)]T for t ∈ [0, 10]. In order to
satisfy assumption (A2), we assume that the angular dis-
j +1 j placement of the pole is limited to x j1 < 1.483 rad (about
J p (t) – J p (t) 85o) so that the input gain b(X(t)j)) will be bounded away
t
from zero. In this simulation, the control process is
[ – 2k(s j) + γ(s j) (σ j – σ ′ jV p X a )T(σ j – σ ′ jV p X a )
jT
≤ assumed to fail if x j1 is greater than 1.483 during the
2 2 j jT j

0
control interval. The neural network based ILC is de-
signed as in (6)-(10) with c1 = 1, k = 1, γ = 10 and bL =
+ γ(s j) tr{(X ja W p σ ′ j)T(X ja W p σ ′ j)}
2 jT jT
0.05. In this neural network, we set W p ∈ R1 × 2 and V p ∈
R2 × 3; i.e., only two neurons are used to construct the neural
+ γ(s j)2Y jTY j + 2s jU jmf]dz – (s j(t))2 network. For simplicity, all the initial conditions of the
adaptation parameters are set to be 0 in the first iteration.
t
≤ [ – 2k(s j) – γ(s j) (σ j – σ ′ jV p X a )T(σ j – σ ′ jV p X a )
2 2 jT j jT j The upper bounds on the optimal neuron weights are all set
0 to be 10. In general, these values are not easy to estimate
for an arbitrary nonlinear function. However, we can
– γ(s j) tr{(X ja W p σ ′ j)T(X ja W p σ ′ j)}
2 jT jT
usually find a suitable estimate for real implementation
based on the physical information of the controlling plant.
– γ(s j) Y Y ]dz – (s j(t))
2 jT j 2 In this example, the system is well learned during the
26 Asian Journal of Control, Vol. 4, No. 1, March 2002

control process, and the control parameters in fact do not pearing in the control input, we can adopt a smooth
reach the upper bounds. The simulation was conducted function (e.g., saturation function) to replace sign function
with the initial states x j1 (0) = 0 rad and x j2 (0) = 1 rad/sec so in a real implementation.
that assumption (A4) could be satisfied. Figure 1 and Fig.
2 show the supremum tracking errors sup e j1(t) and V. CONCLUSION
t ∈ [0, 10]

sup e j2(t) versus iteration j = 0, 1, … 20. The supremum
t ∈ [0, 10]
state tracking errors are less than 0.0008 after the 20th
iteration for sup e 20
1 (t) and 0.0013 for sup
t ∈ [0, 10]
respectively. Finally, the control input at the 20th iteration
u20 is also shown in Fig. 3 to demonstrate the bounded
internal signal.
It is very interesting to note that although a switching
mechanism is used to compensate for uncertainties due to
the state dependent input gain and approximation residual
error, the control input is smooth. Actually, the main
control force coming from the smooth neural network
learning component (9) and neural network based feed-
back component (10) dominates most of the control input
trajectory. However, we can still find some chattering
behavior when the magnitude of u20(t) is small. In order to
keep any possible high frequency components from ap-
For the control of a repeatable nonlinear system with
state dependent input gain, a neural network based itera-
tive learning controller has been proposed in this paper.
e 20 All the weights of the neurons are tuned during the itera-
2 (t) ,
t ∈ [0, 10] tion process so that no prior off-line training phase is
necessary. The adaptive laws for the weights of neurons
and the analysis of learning performance are determined
via Lyapunov-like analysis. In order to prevent drifting of
neuron weights, a projection mechanism has been pro-
vided for designing the learning algorithm. We have
shown that the tracking error vector asymptotically con-
verges to zero as the iteration goes to infinity, and that all
adjustable parameters as as the internal signals will remain
bounded for all iterations.
APPENDIX A
The Multilayer Neural Network Structure

0
10

–1
10

–2
10

–3
10

–4
10
0 2 4 6 8 10 12 14 16 18 20

Fig. 1. Supremum tracking error sup e j1(t) versus iteration j.

t ∈ [0, 10]

0
10

–1
10

–2
10
APPENDIX B
–3
10
0 2 4 6 8 10 12 14 16 18 20 Proof of Theorem 2.
Fig. 2. Supremum tracking error sup e j2(t) versus iteration j.
t ∈ [0, 10] To obtain the linear parameterization structure stated
in Theorem 2, we manipulate the following equalities
20 using (1) and (2):

10
f = f(X, W, V) – f(X)
0
= W σ (V X a ) – W * Tσ (V * TX a ) – : f(X)
T T

10

= W σ (V X a ) – W * Tσ (V X a ) + W * Tσ (V X a )
T T T T
20
0 1 2 3 4 5 6 7 8 9 10

– W Tσ (V TX a ) – : f(X).
* *
Fig. 3. Control input u20(t) at 20th iteration. (B.1)
 C.J. Chien and L.C. Fu: An Iterative Learning Control for Nonlinear Systems Using Neural Network Design 27

Since we define σ ≡ σ(V Xa), if we let σ = σ(V Xa)

T T
≤ W + W F and V ≤V + V , then we can find that
– σ(V*TXa) for simplicity, then (B.1) can be rewritten as
F F
r is bounded by

f = W σ – W Tσ + W Tσ – : f(X)
T * *

r = W σ ′V * TX a – W * To(V X a ) – : f(X)
T T

= W σ + W σ – : f(X)
T *T

≤ W σ′ V
*
= W σ + W * Tσ + W σ – W σ – : f(X)
T T T
F F
X a + W * F(c 1 V F
X a + c 2) + :

= W σ + W σ – W σ – : f(X).
T
≤ W Fc 1V X a + W(c 1 V
T T
(B.2) F
X a + c 2) + :

In order to deal with σ , we use Taylor series ex- ≤ (c 2W + :) + c 1V(W + W F) X a + c 1W(V + V F) X a

pansion:
= (c 2W + :) + (c 1VW + c 1WV) X a
σ (V TX a ) = σ (V X a ) + σ ′(V X a )
* T T

θ 1* θ 2*
T
⋅ (V X a – V X a ) + o(V X a )
*T T

+ (c 1V) W X a + (c 1W) V Xa
≡ σ (V X a ) – σ ′V X a + o(V X a ),
T T T F F
(B.3)
θ 3* θ 4*

where o(⋅) denotes the sum of the high-order terms of the

argument in a Taylor series expansion, and 1
Xa
′ d σ i(Z) = [θ 1* θ 2* θ 3* θ 4*] ⋅
σ ≡ σ (V X a ) = , ∀i = 1, …, ρ,
′ T
i i i Xa W
dZ T
Z = Vi Xa
F

Xa V F
′ ′ ρ×ρ
σ ′ = diag {σ , …, σ } ∈ R
1 ρ .
≡ θ * TY.
Equation (B.3) implies that
This completes the proof. ■
σ = σ (V X a ) – σ (V TX a ) = σ ′V X a – o(V X a ). (B.4)
T * T T

Now, we can rewrite f in (B.2) as
f = W σ + W ⋅ (σ ′V X a – o(V X a ))
T T T T
– W ⋅ (σ ′V X a – o(V X a )) – : f(X)
T T
= W ⋅ (σ – σ ′V X a ) + W σ ′V X a
T T
+ W σ ′V * TX a – W * To(V X a ) – : f(X)
T T
r
For the sigmoid activation function, it can be shown
using (B.4) that the high-order terms are bounded as
o(V X a ) = σ ′V X a – σ ≤ σ ′ V
T T
+ σ ≤ c1 V
where c1, c2 are some bounded constants due to the fact
that the sigmoid function and its derivative are always
bounded by constants. If we use the following facts: W F
REFERENCES
1. Arimoto, S., S. Kawamura and F. Miyazaki, “Better-
ing Operation of Robots by Learning,” J. Rob. Syst.,
Vol. 1, No. 2, pp. 123-140 (1984).
T
2. Hauser, J.E., “Learning Control for a Class of Nonlin-
ear Systems,” Proc. IEEE 26th Conf. Decis. Contr.,
T T
Los Angeles, CA, pp. 859-860 (1987).
3. Kawamura, S., F. Miyazaki and S. Arimoto, “Realiza-
tion of Robot Motion Based on a Learning Method,”
IEEE Trans. Syst. Man Cybern., Vol. 18, No. 1, pp.
126-134 (1988).
4. Heinzinger, G., D. Fenwick, B. Paden and F. Miyazaki,
“Stability of Learning Control with Disturbances and
Uncertain initial Conditions,” IEEE Trans. Automat.
Contr., Vol. 37, No. 1, pp. 110-114 (1992).
F
Xa 5. Kuc, T.Y., J.S. Lee and K. Nam, “An Iterative Learn-
ing Control Theory for a Class of Nonlinear Dynamic
Systems,” Automatica, Vol. 28, No. 6, pp. 1215-1221
X a + c 2,
F (1992).
6. Ahn, H.S., C.H. Choi and K.B. Kim, “Iterative Learn-
ing Control for a Class of Nonlinear Systems,”
Automatica, Vol. 29, pp. 1575-1578 (1993).
7. Chien, C.J. and J.S. Liu, “A P-type Iterative Learning
28 Asian Journal of Control, Vol. 4, No. 1, March 2002
Controller for Robust Output Tracking of Nonlinear
Time-varying Systems,” Int. J. Contr., Vol. 64, No. 2,
pp. 319-334 (1996).
8. Jang, T.J., C.H. Choi and H.S. Ahn, “Iterative Learn-
ing Control in Feedback Systems,” Automatica, Vol.
31, No. 2, pp. 243-248 (1995).
9. Chean, C.C. and D. Wang, “A Model Reference
Learning Control Scheme for a Class of Nonlinear
Systems,” Int. J. Contr., Vol. 66, pp. 271-287 (1997).
10. Park, B.H., T.Y. Kuc and J.S. Lee, “Adaptive Learn-
ing of Uncertain Robotic Systems,” Int. J. Contr., Vol.
65, No. 5, pp. 725-744 (1996).
11. Xu, J.X. and Z. Qu, “Robust Learning Control for a
Class of Nonlinear Systems,” Automatica, Vol. 34,
No. 8, pp. 983-988 (1998).
12. Xu, J.X. and B. Viswanathan, “Adaptive Robust Itera-
tive Learning Control with Dead Zone Scheme,”
Automatica, Vol. 36, pp. 91-99 (2000).
13. Chen, F.C. and C.C. Liu, “Adaptively Controlling
Nonlinear Continuous-time Systems Using Multi-
layer Neural Networks,” IEEE Trans. Automat. Contr.,
Vol. 39, No. 6, pp. 1306-1310 (1994).
14. Chen, F.C. and H.K. Khalil, “Adaptive Control of a
Class of Nonlinear Discrete-time Systems Using Neu-
ral Networks,” IEEE Trans. Automat. Contr., Vol. 40,
No. 5, pp. 791-801 (1995).
15. Lewis, F.L., A. Yesildirek and K. Liu, “Neural Net
Robot Controller: Structure and Stability Proofs,”
IEEE Conf. Decis. Contr., San Antonio, TX, pp. 2785-
2791 (1993).
16. Rovithakis, G.A. and M.A. Christodoulou, “Adaptive
Control of Unknown Plants Using Dynamical Neural
Networks,” IEEE Trans. Syst. Man Cybern., Vol. 24,
No. 3, pp. 400-412 (1994).
17. Sanner, M. and J.J.E. Slotine, “Gaussin Networks for
Direct Adaptive Control,” IEEE Trans. Neural
Networks, No. 3, pp. 837-863 (1992).
18. Su C.Y. and Y. Stepanenko, “Adaptive Control of a
Class of Nonlinear Systems with Fuzzy Logic,” IEEE
Trans. Fuzzy Syst., Vol. 2, No. 4, pp. 285-294 (1994).
19. Wang, L.X., Adaptive Fuzzy Systems and Control,
Prentice Hall, Englewood Cliffs, NJ, pp. 140-154
(1994).
20. Sponner, J.T. and K.M. Passino, “Stable Adaptive
Control Using Fuzzy Systems and Neural
Networks,” IEEE Trans. Fuzzy Syst., Vol. 4, No. 3, pp.
339-359, 1996.
21. Park, B.H., J.S. Lee and T.Y. Kuc, “Adaptive Learn-
ing Control of Robotic Systems and Its Extension to a
Class of Nonlinear Systems,” Chapter 13 of Iterative
Learning Control - Analysis, Design Integration and
Application, Kluwer Academic Publishers, The
Netherland, pp. 239-259 (1998).
22. K.I. Funahashi, K.I., “On the Approximate Realiza-
tion of Continuous Mappings by Neural Networks,”
Neural Networks, No. 2, pp. 183-192 (1989).
23. Hornik, K., M. Stinchcombe and H. White, “Multi-
layer Feedforward Networks are Universal
Approximators,” Neural Networks, Vol. 2, pp. 359-
366 (1989).
24. Slotine, J.E. and W. Li, Applied Nonlinear Control,
Prentice-Hall, Englewood Cliffs, NJ (1991).
Chiang-Ju Chien was born in
Taiwan, Republic of China in 1963.
He received the B.S. degree in Nuclear
Engineering from National Tsing Hua
University, Hsinchu, Taiwan, in 1985
and Ph.D degree in Electrical Engi-
neering from National Taiwan
University, Taipei, Taiwan, in 1992. Since 1993, he was
with the Department of Electronic Engineering, Huafan
University, Taipei, Taiwan, where he is currently an asso-
ciated professor and the head of the Department. His
present research interests are mainly in iterative learning
control, adaptive control and fuzzy-neuro systems. He is
now a member of the Chinese Automatic Control Society.
Li-Chen Fu was born in Taipei,
Taiwan, Republic of China in 1959.
He received the B.S. degree from
National Taiwan University in 1981,
and the M.S. and Ph.D. degrees from
the University of California,
Berkeley, in 1985 and 1987,
respectively. Since 1987 till now, he
has been on the faculty and currently is a professor of both
Department of Electrical Engineering and Department of
Computer Science & Information Engineering of National
Taiwan University. From 1998 to 2001, he also served as
the Deputy Director of Tjing Ling Industrial Research
Institute of National Taiwan University. His areas of
research interest include adaptive control, nonlinear control,
induction motor control, visual tracking, control of robots,
FMS scheduling, and shop floor control. He has authored
43 peer-reviewed journal papers and 135 international
conference papers.
He is now a senior member in both Robotics and
Automation Society and Automatic Control Society of
IEEE, and is also board members of Chinese Automatic
Control Society and Chinese Institute of Automation
Engineers. During 1996-1998 and 2000, he was appointed
a member of AdCom of IEEE Robotics and Automation
Society, and will serve as the Program Chair of 2003 IEEE
International Conference on Robotics and Automation.
He has been the Editor of Journal of Control and Systems
Technology and Associate Editor of Automatica. Starting
from 1999, he became Editor-in-Chief of a new control
 C.J. Chien and L.C. Fu: An Iterative Learning Control for Nonlinear Systems Using Neural Network Design 29

journal, called Asian Journal of Control.
Dr. Fu received the Excellent Research Award in the
period of 1990-1993 and Outstanding Research Awards in
the years of 1995, 1998, and 2000 from National Science
Council, R.O.C., respectively, the Outstanding Youth
Medal in 1991, the Outstanding Engineering Professor
Award in 1995, the Best Teaching Award in 1994 from
Ministry of Education, The Ten Outstanding Young Per-
sons Award in 1999 of R.O.C., the Outstanding Control
Engineering Award from Chinese Automatic Control So-
ciety in 2000, and the Lee Kuo-Ding Medal from Chinese
Institute of Information and Computing Machinery in
2000.