2244 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO.
12, DECEMBER 2004
REFERENCES
[1] B. Hassibi, A. H. Sayed, and T. Kailath, Indefinite Quadratic
Estimation and Control: A Unified Approach to H and H Theo-
ries. Philadelphia, PA: SIAM, 1998.
[2] A. Kojima and S. Ishijima, “ H performance of preview control sys-
tems,” Automatica, vol. 39, pp. 693–701, 2003.
[3] ,“ H preview tracking in output feedback setting,” Int. J. Robust
Nonlinear Control, vol. 14, pp. 627–641, 2004.
[4] L. Mirkin and G. Meinsma, “When does the H fixed-lag smoothing
performance saturate?,” presented at the IFAC World Congr., Barcelona,
Spain, 2002.
[5] L. Mirkin, “On the H fixed-lag smoothing: How to exploit the infor-
mation preview,” Automatica, vol. 39, no. 8, 2003.
[6] K. M. Nagpal and R. Ravi, “ H control and estimation problems with
delayed measurements: state-space solutions,” SIAM J. Control Optim.,
vol. 35, no. 4, pp. 1217–1243, 1997.
[7] U. Shaked, “ H optimal estimation—old and new result,” presented
at the 21st Brazilian Automatic Control Conf., Uberlandia, MG, Brasil,
Sept. 1998.
[8] H. Zhang and D. Zhang, “Finite horizon H fixed-lag smoothing for
time-varying continuous systems,” IEEE Trans. Circuits Syst. II, vol.
51, pp. 496–499, Sept. 2004.
[9] H. Zhang, L. Xie, and Y. C. Soh, “A unified approach to linear estimation
for discrete-time systems—part II: H estimation,” in Proc. IEEE Conf.
Decision Control, Dec. 2001, pp. 2923–2928.
[10] H. Zhang, D. Zhang, and L. Xie, “Fixed-lag smoothing and prediction
for linear continuous-time systems,” presented at the Amer. Control
Conf., Denver, CO, 2003.
Nonlinear Internal Models for Output Regulation
C. I. Byrnes and A. Isidori
Abstract—In this note, we show how nonlinear internal models can be
effectively used in the design of output regulators for nonlinear systems.
This result provides a significant enhancement of the nonequilibrium
theory for output regulation, which we have presented in a recent paper.
Index Terms—Internal model, limit sets, nonlinear control, regulation,
zero dynamics.
I. INTRODUCTION
The problem of nonlinear output regulation has attracted a good deal
of interest in the last few years. Major recent advances have dealt with
the design of nonlinear internal models. Specifically, nonlinear internal
models have been used in [7], as adaptive (thus, nonlinear) versions
of linear internal models, and in the new design methods for robust
regulation presented in [3] and [4]. A systematic method for the design
of a special class of nonlinear internal models has also been described
in [5].
Manuscript received October 23, 2003; revised April 30, 2004. Recom-
mended by Associate Editor Z.-P. Jiang. This work was supported in part
by the National Science Foundation under Grant ECS-0314004, by the
Air Force Office of Scientific Research under Grant F49620-01-10039, by
the Office of Naval Research under Grant N00014-03-1-0314, and by the
Boeing–McDonnell Douglas Foundation.
C. I. Byrnes is with the Department of Electrical and Systems Engineering,
Washington University, St. Louis, MO 63130 USA.
A. Isidori is with the Dipartimento di Informatica e Sistemistica, Università
di Roma “La Sapienza,” 00184 Rome, Italy.
Digital Object Identifier 10.1109/TAC.2004.838492
0018-9286/04$20.00 © 2004 IEEE
Another advance in the theory of nonlinear output regulation has
been the recent work [1], in which we have proposed the a new
“nonequilibrium” approach to the problem, given a more general
(nonequilibrium) formulation, derived necessary conditions for the
existence of solutions, and described how the necessary conditions
thus found, complemented with an additional set of appropriate
hypotheses, can be used in the design of output regulators.
We recall that, generally speaking, the problem of output regula-
tion is to have the regulated variables of a given controlled plant to
asymptotically track (or reject) all desired trajectories (or disturbances)
generated by some fixed autonomous system, known as exosystem.
The hypotheses assumed in [1] for the design of output regulators no
longer include the assumption, common to all earlier literature, that
the zero-dynamics of the controlled plant have a globally asymptoti-
cally stable equilibrium. Rather, this assumption is replaced with the
(substantially weaker) hypothesis that the zero dynamics of the plant
“augmented by exsosystem” have a compact attractor. In [1], though,
we have retained the (rather strong) assumption, itself also common to
all earlier literature, that the set of all “feedforward inputs capable to
secure perfect tracking” is a subset of the set of solutions of a suitable
linear differential equation. In this note we show that, within the new
framework, the assumption of linearity can also be dropped.
Since this technical note is to be viewed as a continuation of the
work [1], we retain the same notation and—to avoid duplication—we
refer the reader to a number key concepts introduced and/or described
in that paper, among which the notion of omega limit set !(B ) of a
set B , plays a major role. This concept is a deep generalization of the
classical concept, due to Birkhoff, of omega limit set of a point and
provides a rigorous definition of steady-state response in a nonlinear
system (see [1] for details).
II. PROBLEM STATEMENT
In [1], as an illustration of how the new nonequilibrium concepts can
be applied to the design of regulators, we have shown how the problem
of output regulation can be solved, under appropriate assumptions, for
a system which can be put in the form
z_ = f0 (z; w ) + f1 (z; ; w )
_ = q (z; ; w ) + u
e=
y= (1)
with state (z;  ) 2 2 , control input u 2 , regulated output
n
e 2 r, measured output y 2 and exogenous (disturbance) input
w 2 generated by an exosystem
w_ = s(w ): (2)
The functions f0 (z; w); f1 (z; ; w); q (z; ; w), and s(w) are C k func-
tions (for some large k ) of their arguments.
Remark: System (1) may look very particular, as it has relative de-
gree 1 between control input u and regulated output e. However, the de-
sign methodology described in [1], and pursued in what follows under
much weaker hypotheses, lends itself to a straightforward extension to
systems with higher relative degree, namely systems having the form
of [1, eq. (33)]. Details are somewhat lengthy and for this reason they
are not included here. /
Remark: Note that regulated variable e of (1) is not necessarily
the physical “controlled” output of the plant, but rather represents a
tracking error, namely the difference between a physical “controlled”
output and its “reference” behavior, the latter assigned as a function
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 12, DECEMBER 2004 2245

of the state w of the exosystem. Thus, the problem under considera- Let fc : d ! be any locally Lipschitz function of compact
tion includes problems of tracking as well as problems of disturbance support which agrees on  (A0 ) with the function f defined in (4), i.e.,
attenuation. / a function such that, for some compact superset S of  (A0 ) satisfies
The analysis in [1] was based on three standing hypotheses. The first
of these hypotheses is that the exosystem is Poisson stable, namely, the fc ( ) = 0; for all  62 S
following. f c ( ) = f ( ); for all  2  (A ):
Assumption 0: The set W  r of admissible initial conditions for
0

the exosystem (2) is compact and W = w2W !(w). / Note that, by definition, there is a number C > 0 such that
Letting Z 2 E  n 2 be the compact set of initial states of (1) for jf ()j  C; 2 d: for all  (5)
c
which the problem of output regulation is to be solved, the second hy-
pothesis is that the trajectories of the zero dynamics of (1), augmented Let now 8 ( 1 ) be the smooth vector field of d defined as
c
with (2), are bounded, namely, the following.
Assumption 1: The positive orbit of Z 2 W under the flow of
2
3
z_
= f0 (z; w) 8 c ( ) = 111
w_ = s(w ) (3) d

has a compact closure, and ! (Z 2 W )  int(Z ) 2 W . /

0 f ( ;  ; . . . ;  d )
c 1 2

It follows from this assumption that the set and let 0 be the 1 2d matrix
A := !(Z 2 W )
0 0 = (1 0 1 1 1 0):
i.e., the ! -limit set—under the flow of (3)—of the set Z 2 W , is a Trivially, for any (z0 ; w0 ) 2A the function  (z (t); w(t)) is such
0
nonempty, compact, invariant set which is stable in the sense of Lya-
punov and uniformly attracts Z 2 W .
that
d  (z (t); w(t)) =  (z (t); w(t))
dt i i+1
The third assumption was the following.
Assumption 2: There exist an integer d and real numbers
a0 ; a1 ; . . . ; ad01 such that, for any (z0 ; w0 ) 2 A0 , the solution
for i = 1; . . . ; d 0 1. If, in addition, Assumption 2-nl holds, then
(z (t); w(t)) of (3) passing through (z0 ; w0 ) at t = 0 is such that the
function '(t) := 0q (z (t); 0; w(t)) satisfies d  (z (t); w(t)) = 0f ( (z (t); w(t)))
d + ad01 '(d01) + 1 1 1 + a1 '(1) + a0 ' = 0: dt d
= 0fc ( (z (t); w(t))):
( )
'

It is well-known that the function '(t) considered above is the input

/
Thus, if Assumption 2-nl holds, for any (z0 ; w0 ) 2A 0 the function
 (z (t); w (t)) satisfies
needed to keep the regulated output e(t) identically zero (so long as
e(0) = 0). The assumption above calls for the existence of a linear d  (z (t); w(t)) = 8 ( (z (t); w(t))):
differential equation of which '(t) be a solution. In this note, we dras- dt c (6)
tically weaken this assumption, by simply calling for the existence of Remark: In other words, the restriction of (3) to A , which is in-
0
a nonlinear differential equation of which '(t) be a solution, namely, variant under the flow of (3), and the restriction of
the following.
Assumption 2-nl: There exists an integer d and a locally Lipschitz _ = 8 c ( ) (7)
function f : d ! such that, for any (z0 ; w0 ) 2 A0 , the solution
to  (A0 ), which is invariant under the flow of (7), are  -related
(z (t); w(t)) of (3) passing through (z0 ; w0 ) at t = 0 is such that the
function '(t) := 0q (z (t); 0; w(t)) satisfies
systems. /

'
d + f ('; '(1) ; . . . ; '(d01) ) = 0:
( )
(4)
Moreover

q (z; 0; w ) = 00 (z; w); for all (z; w) 2 A0 : (8)

/
Consider now a control law of the form

_ = 8c ( ) + Gv
III. OUTPUT REGULATION VIA NONLINEAR INTERNAL MODELS
We proceed now with the construction of a controller which solves u = 0 + v (9)
the problem of output regulation for (1). To this end, consider the se-
quence of functions recursively defined as with

1 (z; w ) = 0q(z; 0; w); . . . ; i (z; w)

+1 v = 0ky: (10)
= @i f (z; w) + @i s(w)
0
@z @w A simple check shows that, if Assumptions 0, 1, and 2-nl hold, this
for i = 1; . . . ; d 0 1, and consider the map controller does have the internal-model property (see [1]), relative to
n2 r ! d the set A0 .
 : The control of (1) by means of (9) results in a system
(z; w) 7! col(1 (z; w); 2 (z; w); . . . ; d (z; w)):
z_ = f0 (z; w) + f1 (z; ; w)
_
If k , the degree of continuous differentiability of the functions in (1), is
= q(z; ; w) + 0 + v
large enough, the map  is well defined and C 1 . In particular  (A0 ),
the image of A0 under  is a compact subset of d , because A0 is a _ = 8c ( ) + Gv
compact subset of n 2 r . w_ = s(w ) (11)
2246 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 12, DECEMBER 2004

which, viewing v as input and  as output, has relative degree 1 and a of W; Z , and 4 and of the parameter . This concludes the proof that
zero-dynamics the positive orbit of B is bounded.
To find !(B), consider now the (closed) solid cylinder C = A0 2
z_ = f0 (z; w) d . A simple argument shows that !(B)  C . In fact, for any point
_ = 8c () + G(0q(z; 0; w) 0 0) p = (z; w; ) 2 !(B), there must exist sequences pk = (zk ; wk ; k )
w_ = s(w): and tk , with tk ! 1 as k ! 1, such that limk!1 (tk ; pk ) =
p. Because of the triangular structure of (12), the point (z; w) is by
The asymptotic properties of the latter are summarized in the fol- definition in A0 and, hence, (z; w; ) 2 C .
lowing results. It is immediate to see that graph( jA ) is invariant for (12). In fact,
Lemma 1: Suppose Assumptions 0, 1, and 2-nl hold. Consider the observe that, for any (z0 ; w0 ) 2 A0 , the solution (z(t); w(t)) of the top
triangular system two equations of (12) passing through (z0 ; w0 ) at time t = 0 remains
z_ = f0 (z; w) in A0 for all t 2 , because the latter is invariant for (3). Thus, bearing
in mind (8), for all such trajectories the bottom equation of (12) can be
w_ = s(w) rewritten as
_ = 8c () + G(0q(z; 0; w) 0 0): (12)
_ = 8c () + G(0(z; w) 0 0):
Let 4  d be a bounded subset such that (A0 )  int(4). Choose
numbers c0 ; c1 ; . . . ; cd01 such that the polynomial Set (t) = (t) 0 (z(t); w(t)). Then, using (6), we have
p() = d + cd01 d01 + 1 1 1 c1  + c0 _ = 8c ( + (z; w)) 0 8c ((z; w)) 0 G0: (14)
is Hurwitz, set The point  = 0 is an equilibrium of this equation and, therefore,
G0 = col(cd01 ; cd02 ; . . . ; c0 ) graph( jA ) is invariant for (12). In particular, for any point p 2
graph( jA ) it is possible to find sequences pk 2 graph( jA ) and
D = diag(; 2 ; . . . ; d ) tk , with tk ! 1 as k ! 1, such that limk!1 (tk ; pk ) = p. This
and shows that graph( jA )  !(B).
G = D  G0 : To complete the proof, we need to show that no point in
C n graph( jA ) can be a point of !(B). For, pick any point
Then, there is a number 3 such that, if   3 p0 2 C n graph( jA ), i.e., a point p0 = (z0 ; w0 ; 0 ) in which
graph ( jA ) = !(Z 2 W 2 4) (z0 ; w0 ) 2 A0 and 0 =6 (z0 ; w0 ). The complete orbit of (12) through
p0 is in C , because A0 is invariant under (3). Thus, the  -component of
under the flow of (12). In particular, graph( jA ) is a compact in- the corresponding trajectory is such that (t) = (t) 0 (z(t); w(t))
variant set which uniformly attracts Z 2 W 2 4. obeys (14), with (0) = 6 0.
Proof: Set, for convenience, B = Z 2 W 2 4. Set
First of all, we show that the positive orbit of B is bounded, so that
!(B) is nonempty, compact, invariant and uniformly attracts B (see  = D01 
[1, Lemma 2.1]). To this end, set
 = D01 
and observe that the variable, thus, defined obeys
_ = (A 0 G0 0) 0 B1(; (z; w); ) (15)
and observe that the variable thus defined obeys
_ = (A 0 G0 0) 0 D01 Bfc (D )
with
 0 G0 q(z; 0; w)
1(; (z; w); )) = 1d [fc (D  + (z; w)) 0 fc ((z; w))]:
(13)
in which 
111
0 1 0 0 0 Since the function fc ( 1 ) is locally Lipschitz, and bounded, there
111
0 0 0 0 0 exists a number L such that, for any 1 ; 2 2 d
A = 1 1 111 1 1 B= 1 : jfc (1 + 2 ) 0 fc (2 )j  Lj1j:
0 0 111 0 1 0
0 0 111 0 0 1 Thus, for any  > 1

It is seen from (5) that, for any  > 1 (which we can assume without j1(; (z; w); ))j  1d LjDj  Ljj:
loss of generality) and for any  2 d
On the other hand, the matrix (A 0 G0 0) is Hurwitz by construction
D01 Bfc (D )
  C: and, hence, there exists a positive–definite symmetric matrix P such
Thus, the integral curves of (13) are maximally defined on the entire that
real axis. Knowing that (t) exists for any t > 0, and setting
P (A 0 G0 0) + (A 0 G0 0)T P = 0I:
u(t) = D01 Bfc (D (t))
  + G0 q(z(t); 0; w(t))
From this, it is immediately seen that the function V () = T P 
system (13) can be rewritten in the form satisfies, along the trajectories of (15)
_ = (A 0 G0 0) 0 u: V_ ()  0( 0 2LjP j)jj2 :
Since the matrix (A 0 G0 0) is Hurwitz by construction, and u(t) is If  > 2LjP j, there exist numbers M > 0 and  > 0, both depending
bounded (by some fixed number which only depends on the choice on , such that
of W and Z ), it is concluded that also j(t)
 j is bounded. As a conse-
quence, (t) itself is bounded, by number which depends on the choice j(t)j  Me0t j(0)j: (16)
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 12, DECEMBER 2004 2247

This property can be used to show that no point of C n graph( jA )
can be a point of ! (B ). For, suppose by contradiction that a point p0 =
(z0 ; w0 ; 0 ), with (z0 ; w0 ) 2 A0 and 0 6=  (z0 ; w0 ), be a point of
! (B ). Set
d0 = j0 0  (z0 ; w0 )j: (17)
As the trajectory of (12) through this point is defined for all t 2 and
bounded, there exists a number K such that j(t)j  K for all t < 0.
Pick now a number T > 0 such that
M e0T K  0: 5 d 0 :
Then, using (16), we obtain
j0 0  (z0 ; w0 )j  M e0T j(0T ) 0  (z(0T ); w(0T ))j
 M e0T K  0:5 d0
/.
which contradicts (17). This concludes the proof of the Lemma.
Lemma 2: Suppose Assumptions 0, 1, and 2-nl hold. Choose G as
indicated in Lemma 1, with   3 . If A0 is locally exponentially
attractive for (3), then graph( jA ) is locally exponentially attractive
for (12).
Proof: To say that A0 is locally exponentially attractive for (3)
is to say that there exists positive numbers ; A;  such that, for any
(z0 ; w0 ) in the (closed) set
from which, using again the same Lyapunov function V () introduced
before and bearing in mind (19), standard arguments show that, if  >
maxf2LjP j; 1g, an estimate of the form
j(t)j  M^ e0^ t (j(0)j + dist((z0 ; w0 ); A0 ))
for every (0) and for every (z0 ; w0 )
holds, satisfying
dist((z0 ; w0 ); A0 )  0 . From this, the result follows. /
As shown in [1], the properties indicated in these two Lemma (which
in [1] were proven to hold under the much stronger Assumption 2)
guarantee that, if the input v of (11) is chosen as in (10), the resulting
closed-loop system has the desired asymptotic properties.
As a matter of fact, the following conclusion holds.
Proposition 1: Consider (11) controlled by (10). Let Z; W be fixed
compact sets of initial conditions and suppose Assumptions 0, 1, and
2-nl hold. Pick any compact set 4 as indicated in Lemma 1 and any
closed interval E of . Choose G as indicated in Lemma 1, with 
large enough so that the conclusions of Lemma 1 hold. Then, for every
" > 0, there exists a number k3 such that, if k  k3 , the positive orbit
of Z 2 W 2 4 2 E is bounded and there exists t such that je(t)j  "
for all t  t. If, in addition, A0 is locally exponentially attractive for
(3), then e(t) ! 0 as t ! 1.
The proof of this proposition follows a standard paradigm. Changing
the variable  into  =  0 Ge and setting k  = k 0 0G yields a system
of the form

A = f(z; w) 2 n2W : dist((z; w); A0 )  g z_ = f0 (z; w) + f1 (z; e; w)e

the solution (z (t); w(t)) of (3) passing through (z0 ; w0 ) at t
such that
dist((z (t); w(t)); A0 )  Ae0t dist((z0 ; w0 ); A0 ):
Set, as in the proof of the previous Lemma, (t) =  ( t)
 (z (t); w(t)). Then
_ = 8c ( +  (z; w)) + G(0q (z; 0; w)
0 0( +  (z; w)) 0 @ f0 (z; w) 0
@
s (w )
@z @w
= 8c ( +  (z; w)) 0 8c ( (z; w))
0 G0 + Gp0 (z; w) + p1 (z; w)
w_ = s(w)
= 0 is
_ = 8c ( + Ge) 0 G0( + Ge) 0 Gq (z; e; w)
e_ = q (z; e; w) + 0 0 ke:
 (20)
0 This can be viewed as interconnection of two subsystems, one with
state (z; w;  ) and input e, the other with state e and input (z; w;  ).
As shown in [2], the former subsystem is input-to-state stable (relative
to the set ! (B )), with a gain function which is linear if the assumptions
of Lemma 2 hold. Then, an easy extension (to the case of systems which
are input-to-state stable relative to compact attractors) of the small-gain
theorem of [6] can be invoked to show that, if k  is large enough, the
results of the proposition hold. Further details can be found in [2].
(18)

in which

p0 (z; w) = 00 (z; w) 0 q (z; 0; w) REFERENCES

@ @
p1 (z; w) = 8c ( (z; w)) 0 f0 (z; w) 0 s (w ):
@z @w [1] C. I. Byrnes and A. Isidori, “Limit sets, zero dynamics and internal
models in the problem of nonlinear output regulation,” IEEE Trans. Au-
Using (8) and (6), it is seen that the functions p0 (z; w) and p1 (z; w) tomat. Contr., vol. 48, pp. 1712–1723, Sept. 2003.
vanish on A0 . Since the functions in question are C 1 and A is com- [2] C. I. Byrnes, A. Isidori, and L. Praly, On the Asymptotic Properties
pact, there exist a number N > 0 such that of a System Arising in Non-Equilibrium Theory of Output Regula-
tion. Stockholm, Sweden: Mittag-Leffler Institute, 2002–2003.

jpi (z; w)j  N dist((z; w); A0 ); for all (z; w) 2 A

and, therefore, if dist((z0 ; w0 ); A0 )  0 , with 0 = =A
jpi (z(t); w(t))j  N Ae0t dist((z0 ; w0 ); A0 ):
[3] Z. Chen and J. Huang, “Nonlinear internal model and the robust output
regulation problem,” in Proc. 2003 Amer. Control Conf., Denver, CO,
June 2003, pp. 1560–1565.
[4] , “A general formulation and solvability of the global robust output
regulation problem,” in Proc. 42nd IEEE Conf. Decision Control, Maui,
(19)
HI, Dec. 2003, pp. 1017–1079.
[5] F. D. Priscoli, “Output regulation with nonlinear internal models,” Syst.

D01  in (18) yields (see the proof of the

Control Lett., 2004, to be published.

Changing  into  = [6] Z. P. Jiang, A. Teel, and L. Praly, “Small-gain theorem for ISS sys-
previous lemma) tems and applications,” Math. Control, Signals, Syst., vol. 7, pp. 95–120,
1994.
_ = (A 0 G0 0) 0 B 1(;  (z; w); )
 [7] A. Serrani, A. Isidori, and L. Marconi, “Semiglobal nonlinear output

+G0 p0 (z; w) + D01 p1 (z; w)

regulation with adaptive internal model,” IEEE Trans. Automat. Contr.,
vol. 46, pp. 1178–1194, July 2001.