IEEE WorkShop Slides Lavretsky

Robust and Adaptive Control Workshop Adaptive Control: Introduction, Overview, and Applications
Adaptive Control: Introduction,

Overview, and Applications
Eugene Lavretsky, Ph.D.
E-mail: eugene.lavretsky@boeing.com
Phone: 714-235-7736
E. Lavretsky
Course Overview
• Motivating Example
• Review of Lyapunov Stability Theory
– Nonlinear systems and equilibrium points
– Linearization
– Lyapunov’s direct method
– Barbalat’s Lemma, Lyapunov-like Lemma, Bounded Stability
• Model Reference Adaptive Control
– Basic concepts
– 1st order systems
– nth order systems
– Robustness to Parametric / Non-Parametric Uncertainties
• Neural Networks, (NN)
– Architectures
– Using sigmoids
– Using Radial Basis Functions, (RBF)
• Adaptive NeuroControl
• Design Example: Adaptive Reconfigurable Flight Control using
2
RBF NN-s
E. Lavretsky
References
• J-J. E. Slotine and W. Li, Applied Nonlinear Control,
Prentice-Hall, New Jersey, 1991
• S. Haykin, Neural Networks: A Comprehensive
Foundation, 2nd edition, Prentice-Hall, New Jersey, 1999
• H. K., Khalil, Nonlinear Systems, 2nd edition, Prentice-
Hall, New Jersey, 2002
• Recent Journal / Conference Publications, (available
upon request)
3
E. Lavretsky
Motivating Example: Roll Dynamics

(Model Reference Adaptive Control)
• Uncertain Roll dynamics: p = Lp p + Lδ ail δ ail

– p is roll rate,
– δ ail is aileron position
( )
– L p , Lδ ail are unknown damping, aileron effectiveness
• Flying Qualities Model: pm = Lmp pm + Lmδ δ ( t )
– ( Lp , Lδ ) are desired damping, control effectiveness
m m
– δ ( t ) is a reference input, (pilot stick, guidance command)

– roll rate tracking error: e p ( t ) = ( p ( t ) − pm ( t ) ) → 0
• Adaptive Roll Control: δ ail = Kˆ p p + Kˆ δ δ
⎧ Kˆ = −γ p ( p − p )
⎪ p
, (γ p , γ δ )>0
p m
⎨
⎪⎩ Kˆ δ = −γ δ δ ( t )( p − pm )
ail
4
ail parameter adaptation laws
E. Lavretsky
Motivating Example: Roll Dynamics

(Block-Diagram)
desired flying qualities model roll tracking

Lmδ pm
error
s + Lmp
δ (t )
unknown plant
ep → 0
Lδ ail p
K̂δ
s + Lp
Kˆ p
parameter adaptation loop
• Adaptive control provides Lyapunov stability

• Design is based on Lyapunov Theorem (2nd method)
• Yields closed-loop asymptotic tracking with all remaining
signals bounded in the presence of system uncertainties5
E. Lavretsky
Lyapunov Stability Theory

Alexander Michailovich Lyapunov

1857-1918
• Russian mathematician and engineer who
laid out the foundation of the Stability
Theory
• Results published in 1892, Russia
• Translated into French, 1907
• Reprinted by Princeton University, 1947
• American Control Engineering Community
Interest, 1960’s
7
E. Lavretsky
Nonlinear Dynamic Systems and

Equilibrium Points
• A nonlinear dynamic system can usually be
represented by a set of n differential equations
in the form: x = f ( x, t ) , with x ∈ R n , t ∈ R
– x is the state of the system
– t is time
• If f does not depend explicitly on time then the
system is said to be autonomous: x = f ( x )
• A state xe is an equilibrium if once x(t) = xe, it
remains equal to xe for all future times: 0 = f ( x )
8
E. Lavretsky
Example: Equilibrium Points of a

Pendulum
• System dynamics: M R 2 θ + bθ + M g R sin (θ ) = 0
• State space representation, ( x 1 = θ , x2 = θ )

x1 = x2
b g R
x2 = − x − sin ( x1 )
2 2
MR R
θ
• Equilibrium points:
0 = x2 M
b g x2 = 0, sin ( x1 ) = 0
0=− x
2 2
− sin ( x1 )
MR R
x2 ⎛π k ⎞
xe = ⎜ ⎟, ( k = 0, ± 1, ± 2,…)
x1 ⎝ 0 ⎠ 9
E. Lavretsky
Example: Linear Time-Invariant

(LTI) Systems
• LTI system dynamics: x = Ax
– has a single equilibrium point (the origin 0) if

A is nonsingular
– has an infinity of equilibrium points in the null-
space of A: A xe = 0
• LTI system trajectories: x ( t ) = exp ( A ( t − t0 ) ) x ( t0 )
• If A has all its eigenvalues in the left half
plane then the system trajectories
converge to the origin exponentially fast
10
E. Lavretsky
State Transformation
• Suppose that xe is an equilibrium point
• Introduce a new variable: y = x - xe
• Substituting for x = y + xe into x = f ( x )
• New system dynamics: y = f ( y + x ) e
• New equilibrium: y = 0, (since f(xe) = 0)

• Conclusion: study the behavior of the new
system in the neighborhood of the origin
11
E. Lavretsky
Nominal Motion
• Let x*(t) be the solution of x = f ( x)
– the nominal motion trajectory corresponding to initial
conditions x*(0) = x0
• Perturb the initial condition x ( 0 ) = x0 + δ x0
• Study the stability of the motion error: e ( t ) = x ( t ) − x∗ ( t )
• The error dynamics:

( ) ( )
e = f x∗ ( t ) + e ( t ) − f x∗ ( t ) = g ( e, t )
e ( 0 ) = δ x0
– non-autonomous!
• Conclusion: Instead of studying stability of the
nominal motion, study stability of the error
12
dynamics w.r.t. the origin
E. Lavretsky
Lyapunov Stability
• Definition: The equilibrium state x = 0 of
autonomous nonlinear dynamic system is said to
be stable if:
∀ R > 0, ∃ r > 0, { x ( 0 ) < r} ⇒ {∀ t ≥ 0, x ( t ) < R}
• Lyapunov Stability means that the system

trajectory can be kept arbitrary close to the origin
by starting sufficiently close to it
x ( 0) x ( 0)
Stable 0 Unstable 0
R r R r
13
E. Lavretsky
Asymptotic Stability
• Definition: An equilibrium point 0 is

asymptotically stable if it is stable and if in
addition: ∃ r > 0, { x ( 0 ) < r} ⇒ {lim
t →∞
x ( t ) = 0}
• Asymptotic stability means that the equilibrium is

stable, and that in addition, states started close
to 0 actually converge to 0 as time t goes to
infinity
• Equilibrium point that is stable but not
asymptotically stable is called marginally stable 14
E. Lavretsky
Exponential Stability
• Definition: An equilibrium point 0 is
exponentially stable if:
{ }
∃ r , α , λ > 0, ∀ x ( 0 ) < r ∧ t > 0 : x ( t ) ≤ α x ( 0 ) e− λ t ,
• The state vector of an exponentially stable

system converges to the origin faster than
an exponential function
• Exponential stability implies asymptotic
stability
15
E. Lavretsky
Local and Global Stability

• Definition: If asymptotic (exponential) stability
holds for any initial states, the equilibrium point
is called globally asymptotically (exponentially)
stable.
• Linear time-invariant (LTI) systems are either
exponentially stable, marginally stable, or
unstable. Stability is always global.
• Local stability notion is needed only for nonlinear
systems.
• Warning: State convergence does not imply
stability! 16
E. Lavretsky
Lyapunov’s 1st Method

• Consider autonomous nonlinear dynamic
system: x = f ( x )
• Assume that f(x) is continuously differentiable
• Perform linearization: x = ⎜⎛ ∂ f ( x ) ⎟⎞ x + f h.o.t . ( x ) ≅ A x
⎝ ∂ x ⎠ x =0
• Theorem A
higher-order terms
– If A is Hurwitz then the equilibrium is asymptotically

stable, (locally!)
– If A has at least one eigenvalue in right-half complex
plane then the equilibrium is unstable
– If A has at least one eigenvalue on the imaginary axis
then one cannot conclude anything from the linear
17
approximation
E. Lavretsky
Lyapunov’s Direct (2nd) Method

• Fundamental Physical Observation
– If the total energy of a mechanical (or
electrical) system is continuously dissipated,
then the system, whether linear or nonlinear,
must eventually settle down to an equilibrium
point.
• Main Idea
– Analyze stability of an n-dimensional dynamic
system by examining the variation of a single
scalar function, (system energy).
18
E. Lavretsky
Lyapunov’s Direct Method

(Motivating Example)
• Nonlinear mass-spring-damper system
m
m x + b x x + k0 x + k1 x3 = 0
damping spring term
x
• Question: If the mass is pulled away and

then released, will the resulting motion be
stable?
– Stability definitions are hard to verify
– Linearization method fails, (linear system is
only marginally stable 19
E. Lavretsky

(Motivating Example, continued)
• Total mechanical energy
x
1
2
2 3
( 1
2
2 1
2
2 1
4
)
V ( x ) = m x + ∫ k0 x + k1 x dx = m x + k0 x + k1 x 4
0
kinetic
potential
• Total energy rate of change along the

system’s motion:
V ( x ) = m x x + ( k x + k x ) x = x ( −b x x ) = − b x ≤ 0
3 3
0 1
• Conclusion: Energy of the system is

dissipated until the mass settles down: x = 0
20
E. Lavretsky

(Overview)
• Method
– based on generalization of energy concepts
• Procedure
– generate a scalar “energy-like function
(Lyapunov function) for the dynamic system,
and examine its variation in time, (derivative
along the system trajectories)
– if energy is dissipated (derivative of the
Lyapunov function is non-positive) then
conclusions about system stability may be
drawn 21
E. Lavretsky
Positive Definite Functions

• Definition: A scalar continuous function
V(x) is called locally positive definite if
V ( 0 ) = 0 ∧ {∀ x ≠ 0 ∧ x < R} ⇒ V ( x ) > 0
• If V ( 0 ) = 0 ∧ {∀ x ≠ 0} ⇒ V ( x ) > 0 then V(x) is

globally positive definite
• Remarks
– a positive definite function must have a
unique minimum min V ( x ) = V ( xmin ) = Vmin
x∈B R
– if Vmin =/= 0 or xmin =/= 0 then use

W ( x ) = V ( x − xmin ) − Vmin 22
E. Lavretsky
Lyapunov Functions
• Definition: If in a ball BR the function V(x)
is positive definite, has continuous partial
derivatives, and if its time derivative along
any state trajectory of the system x = f ( x ) is
negative semi-definite, i.e., V ( x ) ≤ 0 then V(x)
is said to be a Lyapunov function for the
system.
• Time derivative of the Lyapunov function
⎛ ∂V ( x) ∂V ( x) ⎞
V ( x ) = ∇V ( x ) f ( x ) ≤ 0, ∇V ( x ) = ⎜ … ⎟∈ R
n
⎝ ∂ x1 ∂ xn ⎠ 23
E. Lavretsky
Lyapunov Function
(Geometric Interpretation)
x2
V = V1
V = V2
V ( x (t )) V = V3 x1
x2
x (t )
0 x1
V1 > V2 > V3
x (t )
• Lyapunov function is a bowl, (locally)
• V(x(t)) always moves down the bowl
• System state moves across contour
curves of the bowl towards the origin 24
E. Lavretsky
Lyapunov Stability Theorem

• If in a ball BR there exists a scalar function
V(x) with continuous partial derivatives
such that ∀x ∈ B : V ( x ) > 0 ∧ V ( x ) ≤ 0 then the
R
equilibrium point 0 is stable

– If the time derivative is locally negative
definite V ( x ) < 0 then the stability is asymptotic
• If V(x) is radially unbounded, i.e., lim V ( x) = ∞ ,
x →∞
then the origin is globally asymptotically stable
• V(x) is called the Lyapunov function of the
system
25
E. Lavretsky
Example: Local Stability

• Pendulum with viscous damping: θ + θ + sin θ = 0
• State vector: x = (θ θ )
T
θ 2
• Lyapunov function candidate: ( ) (

V x = 1 − cos θ ) +
2
– represents the total energy of the pendulum
– locally positive definite
– time-derivative is negative semi-definite
∂V ( x) ∂V ( x)
V ( x) = θ+ θ = θ sin θ + θ θ = −θ 2 ≤ 0
∂θ ∂θ −θ −sin θ
• Conclusion: System is locally stable

26
E. Lavretsky
Example: Asymptotic Stability

( )
x1 = x1 x12 + x22 − 2 − 4 x1 x22
• System Dynamics: x = x x
( 2 2
2
1 + x22 − 2) + 4 x 2
1 x2
• Lyapunov function candidate: V ( x1 , x2 ) = x12 + x22
– positive definite
– time-derivative is negative definite in the 2-
dimensional ball defined by 1 2 < 2
+
2 2
x x
V ( x1 , x2 ) = 2 ( x12 + x22 )( x12 + x22 − 2 ) < 0
• Conclusion: System is locally

asymptotically stable 27
E. Lavretsky
Example: Global Asymptotic

Stability
c ( x)
• Nonlinear 1st order system
x
x = −c ( x ) , where: x c ( x ) > 0
• Lyapunov function candidate: V ( x ) = x2
– globally positive definite

– time-derivative is negative definite
V ( x ) = 2 x x = −2 x c ( x ) < 0
• Conclusion: System is globally

asymptotically stable
• Remark: Trajectories of a 1st order system are
28
monotonic functions of time, (why?)
E. Lavretsky
La Salle’s Invariant Set Theorems

• It often happens that the time-derivative of
the Lyapunov function is only negative
semi-definite
• It is still possible to draw conclusions on
the asymptotic stability
• Invariant Set Theorems (attributed to La
Salle) extend the concept of Lyapunov
function
29
E. Lavretsky
Example: 2nd Order Nonlinear

System
• System dynamics: x + b ( x) + c ( x) = 0
– where b(x) and c(x) are continuous functions verifying
the sign conditions: x b ( x ) > 0, for x ≠ 0
x c ( x ) > 0, for x ≠ 0
• Lyapunov function candidate: 1 2
x
V ( x, x ) = x + ∫ c ( y ) dy
– positive definite 2 0
– time-derivative is negative semi-definite

V = x x + c ( x) x = −x b ( x) ≤ 0
• system energy is dissipated
x b ( x ) = 0 ⇔ x = 0 ⇒ x = −c ( x ) ⇒ xe = 0
• system cannot get “stuck” at a non-zero equilibrium
30
• Conclusion: Origin is globally asymptotically stable
E. Lavretsky
Lyapunov Functions for LTI

Systems
• LTI system dynamics: x = A x
• Lyapunov function candidate: V ( x ) = xT P x
– where P is symmetric positive definite matrix
– function V(x) is positive definite
• Time-derivative of V(x(t)) along the system
trajectories: V ( x ) = xT P x + xT P x = xT ( AT P + P A) x = − xT Q x < 0
−Q
– where Q is symmetric positive definite matrix

– Lyapunov equation: AT P + P A = −Q
• Stability analysis procedure:
– choose a symmetric positive definite Q
– solve the Lyapunov equation for P 31
E. Lavretsky – check whether P is positive definite
Stability of LTI Systems

• Theorem
– An LTI system is stable (globally
exponentially) if and only if for any symmetric
positive definite matrix Q, the unique matrix
solution P of the Lyapunov equation is
symmetric and positive definite
• Remark: In most practical cases Q is
chosen to be a diagonal matrix with
positive diagonal elements
32
E. Lavretsky
Barbalat’s Lemma: Preliminaries
• Invariant set theorems of La Salle provide

asymptotic stability analysis tools for
autonomous systems with a negative
semi-definite time-derivative of a
Lyapunov function
• Barbalat’s Lemma extends Lyapunov
stability analysis to non-autonomous
systems, (such as adaptive model
reference control)
33
E. Lavretsky
Barbalat’s Lemma
• Lemma
– If a differentiable function f(t) has a finite limit as t → ∞
and if f ( t ) is uniformly continuous, then lim f ( t ) = 0
t →∞
• Remarks
– uniform continuity of a function is difficult to verify
directly
– simple sufficient condition:
• if derivative is bounded then function is uniformly continuous
– The fact that derivative goes to zero does not imply
that the function has a limit, as t tends to infinity. The
converse is also not true, (in general)
– Uniform continuity condition is very important! 34
E. Lavretsky
Example: LTI System

• Statement: Output of a stable LTI system is
uniformly continuous in time
– System dynamics: x = A x + B u
– Control input u is bounded
– System output: y=Cx
• Proof: Since u is bounded and the system is

stable then x is bounded. Consequently, the
output time-derivative y = C x = C ( A x + B u ) is
bounded. Thus, (using Barbalat’s Lemma), we
conclude that the output y is uniformly
continuous in time. 35
E. Lavretsky
Lyapunov-Like Lemma
• If a scalar function V(x,t) satisfies the
following conditions
– function is lower bounded
– its time-derivative along the system
trajectories is negative semi-definite and
uniformly continuous in time
• Then: lim V ( x, t ) = 0
t →∞
• Question: Why is this fact so important?

• Answer: It provides theoretical foundations
for stable adaptive control design 36
E. Lavretsky
Example: Stable Adaptation

• Closed-loop error dynamics of an adaptive
system e = −e + θ w ( t ) , θ = −e w ( t )
– where e is the tracking error, θ is the parameter error,
and w(t) is a bounded continuous function
• Stability Analysis
– Consider Lyapunov function candidate: V ( e,θ ) = e2 + θ 2
• it is positive definite
• its time-derivative is negative semi-definite
V ( e,θ ) = 2 e ( −e + θ w ) + 2θ ( −e w ) = −2 e 2 ≤ 0
• consequently, e and θ are bounded
• since V ( e,θ ) = −4 e ( −e + θ w ) is bounded, V ( e,θ ) is
uniformly continuous
t →∞
( )
• hence: lim −2 e 2 = lim V ( e,θ ) = 0 ⇒ lim e ( t )
t →∞ t →∞
37
E. Lavretsky
Uniform Ultimate Boundedness

• Definition: The solutions of x = f ( x, t ) starting at
x ( t0 ) = x0 are Uniformly Ultimately Bounded (UUB)
with ultimate bound B if:
∃ C0 > 0, T = T ( C0 , B ) > 0: ( x (t )
0 ) (
≤ C0 ⇒ x ( t ) ≤ B, ∀t ≥ t0 + T )
• Lyapunov analysis can be used to show UUB
V ( x ) = VB < V0 V ( x ) = V0
x =B
x0 V ( x ) < 0, ∀ C ≤ x ≤ C0
x = C < C0 x = C0
38
All trajectories starting in large ellipse enter small ellipse within finite time T(C0 ,B)
E. Lavretsky
UUB Example : 1st Order System

• The equilibrium point xe is UUB if there
exists a constant C0 such that for every
initial state x(t0) in an interval x ( t ) ≤ C there 0 0
exists a bound B and a time T ( B, x ( t ) ) such 0
that x ( t ) − x ≤ B for all t ≥ t + T

e 0
xe + C0
x ( t0 ) bound B
xe + B
xe x (t )
t
t0 t0 + T
xe − B
T
xe − C0 39
E. Lavretsky
UUB by Lyapunov Extension

• Milder form of stability than SISL
• More useful for controller design in practical
systems with unknown bounded disturbances:
x = f ( x) + d ( x)
• Theorem: Suppose that there exists a function
V(x) with continuous partial derivatives such that
for x in a compact set S ⊂ R n
– V(x) is positive definite: V ( x ) > 0, ∀ x ≠ 0
– time derivative of V(x) is negative definite outside of S:
V ( x ) < 0, ∀ x > R, ( x ≤ R ) ⇒ ( x ∈ S )
– Then the system is UUB and x ( t ) ≤ R, ∀ t ≥ t0 + T 40
E. Lavretsky
Example: UUB by Lyapunov Extension
• System: (
x1 = x1 x22 − x1 x12 + x22 − 9 )
(
x2 = − x12 x2 − x2 x12 + x22 − 9 ) x ( t0 )
R
x2
x (T )
• Lyapunov function candidate:
x1
V ( x1 , x2 ) = x + x
2
1
2
2
• Time derivative:
(
V ( x1 , x2 ) = 2 ( x1 x1 + x2 x2 ) = −2 x12 + x22 )( x 2
1 + x22 − 9 )
• Time derivative negative outside compact set
{
V ( x1 , x2 ) < 0, ∀ x : x12 + x22 > 9 }
• Conclusion: All trajectories enter circle of radius
R = 3, in a finite time 41
E. Lavretsky
Adaptive Control
Introduction
• Basic Ideas in Adaptive Control
– estimate uncertain plant / controller
parameters on-line, while using measured
system signals
– use estimated parameters in control input
computation
• Adaptive controller is a dynamic system
with on-line parameter estimation
– inherently nonlinear
– analysis and design rely on the Lyapunov
2
E. Lavretsky
Stability Theory
Historical Perspective
• Research in adaptive control started in the
early 1950’s
– autopilot design for high-performance aircraft
• Interest diminished due to the crash of a
test flight
– Question: X-?? aircraft tested
• Last decade witnessed the development of
a coherent theory and many practical
applications
3
E. Lavretsky
Concepts
• Why Adaptive Control?
– dealing with complex systems that have unpredictable parameter
deviations and uncertainties
• Basic Objective
– maintain consistent performance of a system in the presence of
uncertainty and variations in plant parameters
• Adaptive control is superior to robust control in dealing
with uncertainties in constant or slow-varying parameters
• Robust control has advantages in dealing with
disturbances, quickly varying parameters, and
unmodeled dynamics
• Solution: Adaptive augmentation of a Robust Baseline
controller 4
E. Lavretsky
Model-Reference Adaptive Control

(MRAC)
ym
reference model
r u y e
controller plant
θˆ
adaptation law
• Plant has a known structure but the parameters are
unknown
• Reference model specifies the ideal (desired) response
ym to the external command r
• Controller is parameterized and provides tracking
• Adaptation is used to adjust parameters in the control law
5
E. Lavretsky
Self-Tuning Controllers (STC)

r y
controller plant
θˆ
estimator
• Combines a controller with an on-line (recursive) plant

parameter estimator
• Reference model can be added
• Performs simultaneous parameter identification and
control
• Uses Certainty Equivalence Principle
– controller parameters are computed from the estimates of the
plant parameters as if they were the true ones
6
E. Lavretsky
Direct vs. Indirect Adaptive Control

• Indirect
– estimate plant parameters
– compute controller parameters
– relies on convergence of the estimated parameters to
their true unknown values
• Direct
– no plant parameter estimation
– estimate controller parameters (gains) only
• MRAC and STC can be designed using both
Direct and Indirect approaches
• We consider Direct MRAC design 7
E. Lavretsky
MRAC Design of 1st Order Systems

• System Dynamics: x = a x + b (u − f ( x))
– a, b are constant unknown parameters

N
– uncertain nonlinear function: f ( x ) = ∑θ i ϕi ( x ) = θ T Φ ( x )
i =1
• vector of constant unknown parameters: θ = (θ1 … θ N )T

• vector of known basis functions: Φ ( x ) = (ϕ1 ( x ) … ϕ N ( x ) )
T
• Stable Reference Model: x = a m m xm + bm r , ( am < 0 )

• Control Goal
– find u such that: lim ( x ( t ) − x ( t ) ) = 0
t →∞
m
8
E. Lavretsky

(continued)
• Control Feedback: u = kˆx x + kˆr r + θˆT Φ ( x )
– (N + 2) parameters to estimate on-line: kˆx , kˆr , θˆ
• Closed-Loop System: ( )
x = a + b kˆx x + b ⎛⎜ kˆr r + θˆ − θ ( ) Φ ( x ) ⎞⎟
T
⎝ ⎠
• Desired Dynamics: x = a x + b r m m m m
• Matching Conditions Assumption a + b k x = am

– there exist ideal gains ( k x , kr ) such that: b kr = bm
– Note: knowledge of the ideal gains is not required,
only their existence is needed
– consequently: a + b kˆx − am = a + b kˆx − a − b k x = b ( kˆx − k x ) = b ∆k x
E. Lavretsky
( )
b kˆr − bm = b kˆr − b kr = b kˆr − kr = b ∆kr 9

(continued)
• Tracking Error: e ( t ) = x ( t ) − x ( t ) m
• Error Dynamics:
⎛ ⎞
( ) ( )
e ( t ) = x ( t ) − xm ( t ) = a + b kˆx x + b ⎜ kˆr r + θˆ − θ Φ ( x ) ⎟ − am xm − bm r ± am x
T
⎜⎜ ⎟⎟
⎝ ∆θ ⎠
( ) ( )
= am ( x − xm ) + a + b kˆx − am x + b kˆr − kr r + b ∆θ T Φ ( x )
(
= am e + b ∆k x x + ∆kr r + ∆θ T Φ ( x ) )
• Lyapunov Function Candidate:
V ( e ( t ) , ∆k ( t ) , ∆k ( t ) , ∆θ ( t ) ) = e + b (γ ∆k + γ
x r
2 −1
x
2
x
−1
r ∆kr2 + ∆θ T Γθ−1 ∆θ )
– where: γ x > 0, γ r > 0, and Γ = ΓT > 0 is symmetric positive 10

E. Lavretsky
definite matrix

(continued)
• Time-derivative of the Lyapunov function
(
V ( e, ∆k x , ∆kr , ∆θ ) = 2 e e + 2 b γ x−1 ∆k x kˆx + γ r−1 ∆kr kˆr + ∆θ T Γθ−1 θˆ )
(
= 2 e am e + b ( ∆k x x + ∆kr r ) + ∆θ T Φ ( x ) )
(
+2 b γ x−1 ∆k x kˆx + γ r−1 ∆kr kˆr + ∆θ T Γθ−1 θˆ )
( )
= 2 am e2 + 2 b ⎛⎜ ∆k x x e sgn ( b ) + γ x−1 kˆx ⎞⎟
⎝ ⎠
⎝ ( ) ⎠ (
+2 b ⎛⎜ ∆kr r e sgn ( b ) + γ r−1 kˆr ⎞⎟ + 2 b ∆θ T Φ ( x ) e sgn ( b ) + Γθ−1 θˆ )
11
E. Lavretsky

(continued)
• Adaptive Control Design Idea
– Choose adaptive laws, (on-line parameter updates)
such that the time-derivative of the Lyapunov function
decreases along the error dynamics trajectories
kˆx = −γ x x e sgn ( b )
kˆr = −γ r r e sgn ( b )
θˆ = −Γθ Φ ( x ) e sgn ( b )
becomes semi-negative definite!
V ( e ( t ) , ∆k x ( t ) , ∆kr ( t ) , ∆θ ( t ) ) = 2 am e ( t ) ≤ 0
2
12
<0
E. Lavretsky

(continued)
• Closed-Loop System Stability Analysis
– Since V ≥ 0 and V ≤ 0 then all the parameter
estimation errors are bounded
– Since the true (unknown) parameters are
constant then all the estimated parameters
are bounded
• Assumption
– reference input r(t) is bounded
• Consequently, xm and xm are bounded
13
E. Lavretsky

(continued)
• Since x = e + xm then x is bounded
• Consequently, the adaptive control
feedback u is bounded
• Thus, x is bounded, and e = x − xm is
bounded, as well
• It immediately follows that V = 4 am e ( t ) e ( t ) is
bounded
• Using Barbalat’s Lemma we conclude that V ( t )
is uniformly continuous function of time 14
E. Lavretsky

(completed)
• Using Lyapunov-like Lemma: lim V ( x, t ) = 0 t →∞
• Since V = 2 a e ( t ) it follows that: lim e ( t ) = 0

m
2
t →∞
• Conclusions
– achieved asymptotic tracking: x ( t ) → xm ( t ) , as t → ∞
– all signals in the closed-loop system are
bounded
15
E. Lavretsky

(Block-Diagram)
bm xm
r s + am
e
b x
kˆr
s+a
f ( x)
fˆ ( x )
kˆx
• Adaptive gains: kˆx ( t ) , kˆr ( t )

N
• On-line function estimation: fˆ ( x ) = θˆ ( t ) Φ ( x ) = ∑θˆi ( t ) ϕ i ( x )

T
i =1 16
E. Lavretsky
Adaptive Dynamic Inversion

(ADI) Control
ADI Design of 1st Order Systems

• System Dynamics: x = a x + b u + f ( x)
– a, b are constant unknown parameters

N
– uncertain nonlinear function: f ( x ) = ∑θ i ϕi ( x ) = θ T Φ ( x )
i =1
• vector of constant unknown parameters: θ = (θ1 … θ N )T

• vector of known basis functions:Φ ( x ) = (ϕ1 ( x ) … ϕ N ( x ) )
T
• Stable Reference Model: x = a m m xm + bm r , ( am < 0 )

• Control Goal
– find u such that: lim ( x ( t ) − x ( t ) ) = 0
t →∞
m
18
E. Lavretsky

(continued)
• Rewrite system dynamics:
( ) (
x = aˆ x + bˆ u + fˆ ( x ) − ( aˆ − a ) x − bˆ − b u − fˆ ( x ) − f ( x ) )
∆a ∆b ∆f ( x )
• Function estimation error:

∆f ( x ) (
fˆ ( x ) − f ( x ) = θˆ − θ ) Φ ( x)
T
∆θ
• On-line estimated parameters: aˆ, bˆ, θˆ
• Parameter estimation errors

∆a aˆ − a, ∆b bˆ − b, ∆θ θˆ − θ 19
E. Lavretsky

(continued)
• ADI Control Feedback: u=
1
bˆ
( ( am − aˆ ) x + bm r ) − θˆT Φ ( x )
– (N + 2) parameters to estimate on-line: aˆ , bˆ, θˆ
– Need to protect b̂ from crossing zero

• Closed-Loop System: x = a x + b r − ∆a x − ∆b u − ∆θ Φ ( x )
m m
• Desired Dynamics: m x = a x +b r
m m m
• Tracking error: e x − x m
• Tracking error dynamics: e = a e − ∆a x − ∆b u − ∆θ Φ ( x ) m
• Lyapunov function candidate

V ( e ( t ) , ∆a ( t ) , ∆b ( t ) , ∆θ ( t ) ) = e + γ ∆a + γ ∆b + ∆θ Γθ ∆θ 20
2 −1
a
2 −1
b
2 T −1
E. Lavretsky

(continued)
(
V ( e, ∆a, ∆b, ∆θ ) = 2 e e + 2 γ a−1 ∆a aˆ + γ b−1 ∆b bˆ + ∆θ T Γθ−1 θˆ )
= 2 e ( am e − ∆a x − ∆b u − ∆θ Φ ( x ) )
(
+2 γ a−1 ∆a aˆ + γ b−1 ∆b bˆ + ∆θ T Γθ−1 θˆ )
( ) ( ) (
= 2 am e2 + ∆a γ a−1 aˆ − x e + ∆b γ b−1 bˆ − u e + ∆θ T Γθ−1 θˆ − Φ ( x ) e )
• Adaptive laws System
energy
aˆ = γ a x e decreases
V ( e, ∆a, ∆b, ∆θ ) = 2 am e2 ≤ 0
bˆ = γ b u e
θˆ = Γθ Φ ( x ) e
21
E. Lavretsky

(stability analysis)
• Similar to MRAC
• Using Barbalat’s Lemma and Lyapunov-
like Lemma: limV ( x, t ) = lim ⎡⎣2 a e ( t ) ⎤⎦ = 0
2
m
t →∞ t →∞
• Consequently: lim e ( t ) = 0 x ( t ) → xm ( t ) , as t → ∞
t →∞
• Conclusions
– asymptotic tracking
bounded
22
E. Lavretsky
Parameter Convergence ?
• Convergence of adaptive (on-line

estimated) parameters to their true
unknown values depends on the reference
signal r(t)
• If r(t) is very simply, (zero or constant), it is
possible to have non-ideal controller
parameters that would drive the tracking
error to zero
• Need conditions for parameter convergence
23
E. Lavretsky
Persistency of Excitation (PE)

• Tracking error dynamics is a stable filter
(
e ( t ) = am e + b ∆k x x + ∆kr r + ∆θ T Φ ( x ) )
Input
• Since the filter input signal is uniformly

continuous and the tracking error
asymptotically converges to zero, then
when time t is large: ∆k x x + ∆kr r + ∆θ T Φ ( x ) ≅ 0
⎛ ∆k x ⎞
• Using vector form: ⎜ ⎟
( )
x r ΦT ( x ) ⎜ ∆kr ⎟ ≅ 0
⎜ ∆θ ⎟ 24
⎝ ⎠
E. Lavretsky
Persistency of Excitation (PE)

(completed)
• If r(t) is such that ( ) satisfies
v = x r ΦT ( x )
T
the so-called “persistent excitation”

conditions, then the adaptive parameter
convergence will take place t +T
– PE Condition: ∃α > 0 ∀ t ∃T > 0 ∫ v (τ ) v (τ ) dτ > α I T

N +2
t
• PE Condition implies that parameter errors

converge to zero
– for linear systems: m - sinusoids ensure convergence
of (2 m) - parameters
25
– not known for nonlinear systems
E. Lavretsky
ADI vs. MRAC

• No knowledge about sgn b
• Adaptive laws are similar
• Both methods yield asymptotic tracking that
does not rely on Persistency of Excitation (PE)
conditions
• ADI needs protection against b̂ crossing zero
– If PE takes place and initial parameter bˆ ( 0 ) has wrong
sign then a control singularity may occur
• Regressor vector Φ ( x ) must have bounded
components, (needed for stability proof)
26
E. Lavretsky
Example: MRAC of a 1st-Order

Linear System
• Unstable Dynamics: x = x + 3 u, x ( 0 ) = 0
– plant parameters a = 1, b = 3 are unknown to the

adaptive controller
• Reference Model: x = −4 x + 4 r ( t ) ,
m m xm ( 0 ) = 0
• Adaptive Control: u = kˆ x + kˆ rx r
• Parameter Adaptation: kˆ = −2 x e, x kˆx ( 0 ) = 0
kˆr = −2 r e, kˆr ( 0 ) = 0
• Two Reference Inputs: r ( t ) = 4

r ( t ) = 4sin ( 3 t ) 27
E. Lavretsky
1st-Order Linear System

MRAC Simulation w/o PE: r(t) = 4
Tracking Error Converges to Zero

28
E. Lavretsky
Parameter Errors don’t Converge to Zero
1st-Order Linear System

MRAC Simulation with PE: r(t) = 4 sin(3 t)
Tracking and Parameter Errors Converge to Zero29

E. Lavretsky
Example: MRAC of a 1st-Order

Nonlinear System
• Unstable Dynamics: x = x + 3 (u − f ( x ) ) , x ( 0) = 0
– plant parameters a = 1, b = 3 are unknown
– nonlinearity: f ( x ) = θ Φ ( x )
T
( )
T
• known basis functions: Φ ( x ) = x e sin ( 2 x )
−( x + 0.5 ) 10 − ( x − 0.5 ) 10
2 2
3
e
• unknown parameters: θ = ( 0.01 −1 1 0.5 )
T
• Reference Model: xm = −4 xm + 4 r ( t ) , xm ( 0 ) = 0
• Adaptive Control: u = kˆx x + kˆr r + θˆT Φ ( x ) kˆx = −2 x e, kˆx ( 0 ) = 0
• Parameter Adaptation: kˆr = −2 r e, kˆr ( 0 ) = 0
θˆ = −2 Φ ( x ) e, θˆ ( 0 ) = 04
• Reference Input: r ( t ) = sin ( 3 t ) + sin ⎛⎜

3t ⎞ ⎛ 3t ⎞ ⎛ 3t ⎞
⎟ + sin ⎜ ⎟ + sin ⎜ ⎟ 30
E. Lavretsky
⎝ ⎠
2 ⎝ ⎠
4 ⎝8⎠
1st-Order Nonlinear System

MRAC Simulation
Good Tracking & Poor Parameter Estimation 31

E. Lavretsky

MRAC Simulation, (continued)
Nonlinearity: Poor Parameter Estimation 32

E. Lavretsky

MRAC Simulation, (completed)
Nonlinearity: Poor Estimation 33

E. Lavretsky
Example: MRAC of a 1st-Order Nonlinear

System with Local Nonlinearity
• Unstable Dynamics: x = x + 3 ( u − f ( x ) ) , x ( 0) = 0
– plant parameters a = 1, b = 3 are unknown
– nonlinearity: f ( x ) = θ T Φ ( x )
( )
T
• known basis functions: Φ ( x ) = x sin ( 2 x )

−( x + 0.5 ) 10 − ( x − 0.5 ) 10
2 2
3
e e
θ = ( 0 −1 1 0 )
T
• unknown parameters:
• Reference Model: xm = −4 xm + 4 r ( t ) , xm ( 0 ) = 0
• Adaptive Control: u = kˆx x + kˆr r + θˆT Φ ( x ) kˆx = −2 x e, kˆx ( 0 ) = 0

• Parameter Adaptation: kˆr = −2 r e, kˆr ( 0 ) = 0
θˆ = −2 Φ ( x ) e, θˆ ( 0 ) = 04
⎛ 3t ⎞ ⎛ 3t ⎞ ⎛ 3t ⎞
• Reference Input: r ( t ) = sin ( 3 t ) + sin ⎜ ⎟ + sin ⎜ ⎟ + sin ⎜ ⎟ 34
E. Lavretsky
⎝ 2⎠ ⎝ 4⎠ ⎝8⎠
1st-Order Nonlinear System with Local

Nonlinearity: MRAC Simulation
Good Tracking & Parameter Estimation 35

E. Lavretsky

Nonlinearity: MRAC Simulation, (continued)
Nonlinearity: Good Parameter Estimation 36

E. Lavretsky

Nonlinearity: MRAC Simulation, (completed)
Nonlinearity: Good Function Approximation 37

E. Lavretsky
MRAC of a 1st-Order Nonlinear System

Conclusions & Observations
• Direct MRAC provides good tracking in spite of
unknown parameters and nonlinear uncertainties
in the system dynamics
• Parameter convergence IS NOT guaranteed
• Sufficient Condition for Parameter Convergence
– Reference input r(t) satisfies Persistency of Excitation
• PE is hard to verify / compute
– Enforced for linear systems with local nonlinearities
• A control strategy that depends on parameter
convergence, (such as indirect MRAC), is
unreliable, unless PE condition takes place 38
E. Lavretsky
MRAC Design of nth Order Systems

• System Dynamics: x = A x + B Λ ( u − f ( x ) ) , x ∈ R , u ∈ R
n m
– A ∈ R n×n , Λ = diag ( λ1 … λm ) ∈ R m×m are constant unknown

matrices
– B ∈ R n×m is known constant matrix
– ∀ i = 1,… , m sgn ( λi ) is known
– uncertain matched nonlinear function: f ( x ) = Θ Φ ( x ) ∈ R
T m
m× N
• matrix of constant unknown parameters: Θ ∈ R
• vector of N known basis functions: Φ ( x ) = (ϕ1 ( x ) … ϕ N ( x ) )
T
• Stable Reference Model: xm = Am xm + Bm r , ( Am is Hurwitz )

• Control Goal r ∈ Rm , Am ∈ R n×n , Bm ∈ R n×m
– find u such that:
lim x ( t ) − xm ( t ) = 0 39
t →∞
E. Lavretsky

(continued)
• Control Feedback: ˆ T Φ ( x)
u = Kˆ xT x + Kˆ rT r + Θ
– (m n + m2 + m N) - parameters to estimate: Kˆ x , Kˆ r , Θˆ
• Closed-Loop System: ( ) (
ˆ − Θ Φ ( x)
x = A + B Λ Kˆ xT x + BΛ Kˆ rT r + Θ ( )
T
)
• Desired Dynamics: xm = Am xm + Bm r
• Model Matching Conditions

A + B Λ K xT = Am
– there exist ideal gains ( K x , K r ) such that:
B Λ K rT = Bm
– Note: knowledge of the ideal gains is not required
( )
T
A + B Λ Kˆ xT − Am = A + B Λ Kˆ xT − A − B Λ K xT = B Λ Kˆ x − K x = B Λ ∆K xT
( )
T
B Λ Kˆ rT − Bm = B Λ Kˆ rT − B Λ K rT = B Λ Kˆ r − K r = B Λ ∆Kˆ rT 40
E. Lavretsky

(continued)
• Tracking Error: e ( t ) = x ( t ) − xm ( t )
• Error Dynamics:
e ( t ) = x ( t ) − xm ( t ) =
⎛ ⎞
( A + B Λ Kˆ ) (
x + B Λ ⎜ Kˆ rT r + Θ )
ˆ − Θ Φ ( x) ⎟ − A x − B r ± A x
T
T
x
⎜⎜ ⎟⎟ m m m m
⎝ ∆Θ ⎠
( )
= Am ( x − xm ) + A + B Λ Kˆ xT − Am x + B Λ Kˆ r − K r( ) r + B Λ ∆ΘT Φ ( x )
T
(
= Am e + B Λ ∆K xT x + ∆K rT r + ∆ΘT Φ ( x ) )
41
E. Lavretsky

(continued)
• Lyapunov Function Candidate
V ( e, ∆K x , ∆K r , ∆Θ ) = eT P e
+ trace ( ∆K xT Γ −x 1 ∆K x Λ ) + trace ( ∆K rT Γ −r 1 ∆K r Λ ) + trace ( ∆ΘT Γ Θ−1 ∆Θ Λ )
– where: trace ( S ) ∑ sii

– Λ diag ( λ1 … λm ) is diagonal matrix with positive
i
elements
– Γ x = ΓTx > 0, Γ r = ΓTr > 0, Γ Θ = ΓTΘ > 0 are symmetric
positive definite matrices
– P = PT > 0 is unique symmetric positive definite
solution of the algebraic Lyapunov equation P Am + Am P = −Q
T
• Q = Q > 0 is any symmetric positive definite matrix

T
42
E. Lavretsky

(continued)
• Adaptive Control Design
– Choose adaptive laws, (on-line parameter updates)
such that the time-derivative of the Lyapunov function
decreases along the error dynamics trajectories
Kˆ x = −Γ x x eT P B sgn ( Λ )
Kˆ r = −Γ r r eT P B sgn ( Λ )
ˆ = −Γ Φ ( x ) eT P B sgn ( Λ )
Θ Θ

becomes semi-negative definite!
V ( e ( t ) , ∆K x ( t ) , ∆K r ( t ) , ∆Θ ( t ) ) = −eT ( t ) Q e ( t ) ≤ 0 43
E. Lavretsky

(completed)
• Using Barbalat’s and Lyapunov-like
Lemmas: lim V ( x, t ) = 0 t →∞
• Since V = −e ( t ) Q e ( t ) it follows that: lim e ( t )

T T
t →∞
=0
• Conclusions
– achieved asymptotic tracking: x ( t ) → xm ( t ) , as t → ∞
bounded
• Remark
– Parameter convergence IS NOT guaranteed
44
E. Lavretsky
Robustness of Adaptive Control

• Adaptive controllers are designed to
control real physical systems
– non-parametric uncertainties may lead to
performance degradation and / or instability
• low-frequency unmodeled dynamics, (structural
vibrations)
• low-frequency unmodeled dynamics, (Coulomb
friction)
• measurement noise
• computation round-off errors and sampling delays
• Need to enforce robustness of MRAC 45
E. Lavretsky
Parameter Drift in MRAC

• When r(t) is persistently exciting the system,
both simulation and analysis indicate that MRAC
systems are robust w.r.t non-parametric
uncertainties
• When r(t) IS NOT persistently exciting even
small uncertainties may lead to severe problems
– estimated parameters drift slowly as time goes on,
and suddenly diverge sharply
– reference input contains insufficient parameter
information
– adaptation has difficulty distinguishing parameter
information from noise 46
E. Lavretsky
Parameter Drift in MRAC: Summary
• Occurs when signals are not persistently

exciting
• Mainly caused by measurement noise and
disturbances
• Does not effect tracking accuracy until the
instability occurs
• Leads to sudden failure
47
E. Lavretsky
Dead-Zone Modification
• Method is based on the observation that small
tracking errors contain mostly noise and
disturbance
• Solution
– Turn off the adaptation process for “small” tracking
errors ⎧⎪−Γ x x eT P B sgn ( Λ ) , e > ε
Kˆ x = ⎨
– MRAC using Dead-Zone ⎪⎩0, e ≤ ε
– ε is the size of the dead-zone ⎧⎪−Γ r r eT P B sgn ( Λ ) , e > ε
ˆ Kr = ⎨
• Outcome ⎪⎩0, e ≤ ε
– Bounded Tracking ⎧⎪−Γ Θ Φ ( x ) eT P B sgn ( Λ ) , e >ε
Θ
ˆ =⎨
⎪⎩0, e ≤ ε
48
E. Lavretsky
1st-Order Linear System with Noise

MRAC w/o Dead-Zone: r(t) = 4
• Satisfactory Tracking
49
E. Lavretsky
• Parameter Drift due to measurement noise
1st-Order Linear System with Noise

MRAC with Dead-Zone: r(t) = 4
• Satisfactory Tracking
50
E. Lavretsky
• No Parameter Drift
Parametric and Non-Parametric

Uncertainties
• Parametric Uncertainties are often easy to
characterize
– Example: m x = u
• uncertainty in mass m is parametric
• neglected motor dynamics, measurement noise, sensor
dynamics are non-parametric uncertainties
• Both Parametric and Non-Parametric
Uncertainties occur during Function
Approximation N
fˆ ( x ) = ∑θ i ϕ i ( x ) + ε ( x )
i =1
parametric non-parametric 51
E. Lavretsky
Enforcing Robustness in MRAC

Systems
• Non-Parametric Uncertainty
– Dead-Zone modification
– Others ?
• Parametric Uncertainty
– Need a set of basis functions that can approximate a
large class of functions within a given tolerance
• Fourier series
• Splines
• Polynomials
• Artificial Neural Networks
– sigmoidal
– RBF
52
E. Lavretsky
Artificial Neural Networks
Σ
x
NN Architectures
• Artificial Neural Networks are multi-input-multi-
output systems composed of many inter-
connected nonlinear processing elements
(neurons) operating in parallel
1
1 4
Network Input Network Output Network Input Network Output

2 5 2
Input Node 3 6 Output

Input Node 3 Output
First Hidden Second Hidden

1
Two-Hidden-Layers Neural Neurons

Network 2
Ordered Neural Network
3
5 2
Single-Hidden-Layer Neural Network
E. Lavretsky
Single Hidden Layer (SHL)

Feedforward Neural Networks (FNN)
• Three distinct characteristics
– model of each neuron includes a nonlinear
activation function
1
• sigmoid σ (s) = −s
1+ e x−r
2
−
• radial basis function ϕ ( x) = e 2σ 2
– a single layer of N hidden neurons

– feedforward connectivity
3
E. Lavretsky
SHL FNN Architecture
Hidden Layer of N neurons

1
Input 2 Output
x y
1
Threshold
N
1
Output Bias 4
E. Lavretsky
SHL FNN Function

• Maps n - dimensional input into m -
dimensional output: x → NN ( x ) , x ∈ R n
, NN ( x ) ∈ R m
• Functional Dependence
– sigmoidal: NN ( x ) = W T σ (V T x + θ ) + b
– RBF:
⎛ ϕ ( x − C1 ) ⎞⎟
⎜
NN ( x ) = W ⎜
T
⎟ + b = W T
Φ ( x) + b
⎜⎜ ϕ x − C
⎝
( N ) ⎟⎟⎠
Φ( x )
5
E. Lavretsky
Sigmoidal NN
⎛ x ⎞
• Matrix form: NN ( x ) = W T σ ⎜ V T ⎛⎜ ⎞⎟ ⎟ + c
⎝ ⎝ 1⎠⎠
• Vector of hidden layer sigmoids:

(
σ (V x + θ ) = σ ( v x + θ1 ) … σ ( v x + θ N ) )
T
T T T
1 N
• Matrix of inner-layer weights: V = ( v1 … vN ) ∈ R n× N

• Matrix of output-layer weights: W = ( w1 … wm ) ∈ R N ×m
• Vector of output biases c ∈ R and thresholds θ ∈ R N
m
• kth output:
⎛ n ⎞
NN k ( x ) = w σ ( v x + θ k ) + ck = ∑ w j k σ ⎜ ∑ vi k xi + θ k ⎟ + ck
N
T T
k k
j =1 ⎝ i =1 ⎠ 6
E. Lavretsky
Sigmoidal NN, (continued)

• Universal Approximation Property
– large class of functions can be approximated
by sigmoidal SHL NN-s within any given
tolerance, on compacted domains
∀ f ( x ) : R n → R m ∀ ε > 0 ∃ N , W , b, V , θ ∀ x ∈ X ⊂ R n
⎛ T ⎛ x⎞⎞ ⎛ 1 ⎞
f ( x ) − W σ ⎜V ⎜ ⎟ ⎟ − b ≤ ε = O ⎜
T
⎟
⎝ ⎝ ⎠⎠1 ⎝ N⎠
T T ⎡σ ⎤ ⎡ x⎤
• Introduce: W ⎡⎣W T
b ⎤⎦ , V ⎡⎣V T
θ ⎤⎦ , σ ⎢1⎥, µ ⎢1 ⎥
⎣ ⎦ ⎣ ⎦
• Then: NN ( x ) = W T σ (V T µ ) 7
E. Lavretsky
Sigmoidal SHL NN: Summary

• A very large class of functions can be
approximated using linear combinations of
shifted and scaled sigmoids
• NN approximation error decreases as the
number of hidden-layer neurons N increases:
⎛ − 12 ⎞
f ( x ) − NN ( x ) = O ⎜ N ⎟
⎝ ⎠
• Inclusion of biases and thresholds into NN
weight matrices simplifies bookkeeping
NN ( x ) = W T σ (V T µ )
• Function approximation using sigmoidal NN
8
means finding connection weights W and V
E. Lavretsky
RBF NN
• Matrix form: NN ( x ) = W Φ ( x ) + b
T
T
⎛ ⎞
2 2
− x −C1 − x −CN
• Vector of RBF-s: Φ ( x) = ⎜ e 2σ12
… e 2σ N2 ⎟
⎜ ⎟
⎝ ⎠
• Matrix of RBF centers: C ⎡C1 … CN ⎤ ∈ R n× N
⎣ ⎦
σ (σ 1 … σ N ) ∈ RN
T
• Vector of RBF widths:
• Matrix of output weights: W = ( w1 … wm ) ∈ R N ×m
• Vector of output biases: b ∈ R m
2
− x −C j
• kth output: N
NN k ( x ) = w Φ ( x ) + bk = ∑ w j k e
2σ 2j
T
k + bk 9
j =1
E. Lavretsky
RBF NN, (continued)

• Universal Approximation Property
– large class of functions can be approximated
by RBF NN-s within any given tolerance, on
compacted domains
∀ f ( x ) : Rn → Rm ∀ ε > 0 ∃ N , W , C,σ ∀ x ∈ X ⊂ Rn
⎛ − ⎞
1
f ( x) −W T Φ ( x) − b ≤ ε = O ⎜ N n ⎟
⎝ ⎠
⎡Φ ( x ) ⎤
• Introduce: W [W b] , Φ ( x ) ⎢ ⎥
⎣ 1 ⎦
• Then: NN ( x ) = W T Φ ( x ) 10
E. Lavretsky
RBF NN: Summary

• A very large class of functions can be
approximated using linear combinations of
shifted and scaled gaussians
• NN approximation error decreases as the
number of hidden-layer neurons N increases:
⎛ − 1n ⎞
f ( x ) − NN ( x ) = O ⎜ N ⎟
⎝ ⎠
• Inclusion of biases into NN output weight matrix
simplifies bookkeeping
NN ( x ) = W T Φ ( x )
• Function approximation using RBF NN means
finding output weights W, centers C, and widths σ
E. Lavretsky
11
What is Next?
• Use SHL FNN-s in the context of MRAC
systems
– off-line / on-line approximation of uncertain
nonlinearities in system dynamics
• modeling errors, (aerodynamics)
• battle damage
• control failures
• Start with fixed widths RBF NN architectures,
(linear in unknown parameters)
• Generalize to using sigmoidal NN-s
12
E. Lavretsky
Adaptive NeuroControl
nth Order Systems with Matched

Uncertainties
• System Dynamics: x = A x + B Λ (u − f ( x ) ) , x ∈ Rn , u ∈ Rm
– A ∈ R n×n , Λ = diag ( λ1 … λm ) ∈ R m×m are constant unknown
matrices
– B ∈ R n×m is known constant matrix, and
• Approximation of uncertainty: f ( x ) = ΘT Φ ( x ) + ε f ( x )
– matrix of constant unknown parameters: Θ ∈ R m× N

– vector of N fixed RBF-s: Φ ( x ) = (ϕ1 ( x ) … ϕ N ( x ) )
T
– function approximation tolerance: ε f ( x ) ∈ R m
14
E. Lavretsky

Uncertainties, (continued)
• Assumption: Number of RBF-s, true
(unknown) output weights W and widths σ
are such that RBF NN approximates the
nonlinearity within given tolerance:
ε f ( x ) = f ( x ) − ΘT Φ ( x ) ≤ ε , ∀ x ∈ X ⊂ R n
• RBF NN estimator: fˆ ( x ) = Θˆ Φ ( x ) T
• Estimation error:
NN ( x ) − f ( x ) = Θ ( )
ˆ − Θ Φ ( x ) − ε ( x ) = ∆ΘT Φ ( x ) − ε ( x )
T
f f
∆Θ
15
E. Lavretsky

• Stable Reference Model: x = A m m xm + Bm r , ( Am is Hurwitz )
• Control Goal r∈R , m

Am ∈ R n×n , Bm ∈ R m×m
– bounded tracking: lim x ( t ) − xm ( t ) ≤ εx

t →∞
• MRAC Design Process

– choose N and vector of widths σ
• can be performed off-line in order to incorporate
any prior knowledge about the uncertainty
– design MRAC and evaluate closed-loop
system performance
– repeat previous two steps, if required 16
E. Lavretsky

• Control Feedback: u = Kˆ xT x + Kˆ rT r + Θˆ T Φ ( x )
– (m n + m2 + m N) - parameters to estimate:Kˆ x , Kˆ r , Θˆ
• Closed-Loop: ( ) (
x = A + B Λ Kˆ xT x + BΛ Kˆ rT r + ∆ΘT Φ ( x ) − ε f ( x ) )
• Desired Dynamics: xm = Am xm + Bm r
• Model Matching Conditions

A + B Λ K xT = Am
– there exist ideal gains ( K x , K r ) such that:
B Λ K rT = Bm
– Note: knowledge of the ideal gains is not required
( )
T
A + B Λ Kˆ xT − Am = A + B Λ Kˆ xT − A − B Λ K xT = B Λ Kˆ x − K x = B Λ ∆K xT
( )
T
B Λ Kˆ rT − Bm = B Λ Kˆ rT − B Λ K rT = B Λ Kˆ r − K r = B Λ ∆Kˆ rT 17
E. Lavretsky

• Tracking Error: e ( t ) = x ( t ) − xm ( t )
• Error Dynamics:
e ( t ) = x ( t ) − xm ( t ) =
( A + B Λ Kˆ ) x + B Λ ( Kˆ r + ∆Θ Φ ( x ) − ε ( x ) ) − A x − B r ± A x
T
x
T
r
T
f m m m m
= A ( x − x ) + ( A + B Λ Kˆ − A ) x + B Λ ( Kˆ − K ) r + B Λ ( ∆Θ Φ ( x ) − ε ( x))
T
T T
m m x m r r f
(
= Am e + B Λ ∆K xT x + ∆K rT r + ∆ΘT Φ ( x ) − ε f ( x ) )
• Remarks
– estimation error ε f ( x ) is bounded, as long as x ∈ X
– need to keep x within X 18
E. Lavretsky

• Lyapunov Function Candidate
V ( e, ∆K x , ∆K r , ∆Θ ) = eT P e
( ) ( ) (
+ trace ∆K xT Γ −x 1 ∆K x Λ + trace ∆K rT Γ −r 1 ∆K r Λ + trace ∆ΘT Γ Θ−1 ∆Θ Λ )
– where: trace ( S ) ∑ sii
– Λ diag ( λ1 … λm ) is diagonal matrix with positive
i
elements
– Γ x = ΓTx > 0, Γ r = ΓTr > 0, Γ Θ = ΓTΘ > 0 are symmetric
positive definite matrices
– P = PT > 0 is unique symmetric positive definite
solution of the algebraic Lyapunov equation P A + AT P = −Q
• Q = Q > 0 is any symmetric positive definite matrix
T
19
E. Lavretsky

V = eT P e + eT P e
( ) ( ) (
+2 trace ∆K xT Γ −x1 Kˆ x Λ + 2 trace ∆K rT Γ −r 1 Kˆ r Λ + 2 trace ∆ΘT Γ Θ−1 Θ
ˆ Λ
)
( (
= Am e + B Λ ∆K x + ∆K r + ∆Θ Φ ( x ) − ε f ( x ) ))
T
T T T
x r Pe
( (
+eT P Am e + B Λ ∆K xT x + ∆K rT r + ∆ΘT Φ ( x ) − ε f ( x ) ))
( ) ( ) (
ˆ Λ
)
= eT ( Am P + P Am ) e
(
+2 eT P B Λ ∆K xT x + ∆K rT r + ∆ΘT Φ ( x ) − ε f ( x ) )
( ) ( ) (
ˆ Λ
20 )
E. Lavretsky

V = −eT Q e − 2 eT P B Λ ε f ( x )
(
+2 eT P B Λ ∆K xT x + 2 trace ∆K xT Γ −x1 Kˆ x Λ )
+2 eT P B Λ ∆K rT r + 2 trace ( ∆K T
r Γ r−1 Kˆ r Λ)
(
+2 eT P B Λ ∆ΘT Φ ( x ) + 2 trace ∆ΘT Γ Θ−1 Θ
ˆ Λ
)
• Using trace identity: a T
b = trace b a T
( )
⎛ ⎞
• Example: e P B Λ ∆K x = trace ⎜ ∆K x x e P B Λ ⎟
T T T T
⎜ ⎟
x
aT b ⎝ b aT ⎠ 21
E. Lavretsky

V = −eT Q e − 2 eT P B Λ ε f ( x )
{ }
+2 trace ⎛⎜ ∆K xT Γ −x1 Kˆ x + x eT P B sgn ( Λ ) Λ ⎞⎟
⎝ ⎠
+2 trace ⎛⎜ ∆K {Γ
T −1
Kˆ + r e P B sgn ( Λ )} Λ ⎞⎟
T
⎝ ⎠
r r r
+2 trace ⎛⎜ ∆Θ {Γ
T −1
Θ Θ T
} ⎠
ˆ + Φ ( x ) e P B sgn ( Λ ) Λ ⎞⎟
⎝
• Problem
– choose adaptive parameters Kˆ x , Kˆ r , Θˆ such that time-
derivative V becomes negative definite outside of a
compact set in the error state space, and all
parameters remain bounded for all future times 22
E. Lavretsky

• Suppose that we choose adaptive laws:
Kˆ x = −Γ x x eT P B sgn ( Λ )
Kˆ r = −Γ r r eT P B sgn ( Λ )
ˆ = −Γ Φ ( x ) eT P B sgn ( Λ )
Θ Θ
• Then we get:
V = −e Q e − 2 e P B Λ ε f ( x ) ≤ −λmin ( Q ) e + 2 e P B λmax ( Λ ) ε
T T 2
• Consequently, V < 0 outside of the compact set

⎧⎪ 2 P B λmax ( Λ ) ε ⎫⎪
E ⎨e : e ≤ ⎬
⎪⎩ λmin ( )
Q ⎭⎪
• Unfortunately, inside E parameter errors may grow out of
bounds, (for e ∈ E , V IS NOT necessarily negative!) 23
E. Lavretsky
How to Keep Adaptive Parameters

Bounded?
• σ - modification: ( )
Kˆ x = −Γ x x eT P B + σ x Kˆ x sgn ( Λ )
Kˆ = −Γ ( r e P B + σ Kˆ ) sgn ( Λ )
r r
T
r r
Θˆ = −Γ ( Φ ( x ) e P B + σ Θ ˆ ) sgn ( Λ )
T
Θ Θ
• e - modification: Kˆ = −Γ ( x e P B + σ e P B Kˆ ) sgn ( Λ )
x x
T
x
T
x
Kˆ = −Γ ( r e P B + σ e P B Kˆ ) sgn ( Λ )
r r
T
r
T
r
Θˆ = −Γ ( Φ ( x ) e P B + σ e P B Θ
T ˆ ) sgn ( Λ )
T
Θ Θ
• Modifications add damping to adaptive laws

– damping controlled by choosing σ x , σ r , σ Θ > 0
– there is a trade off between adaptation rate and 24
E. Lavretsky
damping
Introducing Projection Operator

• Requires no damping terms
• Designed to keep NN weights within pre-
specified bounds
• Maintains negative values of the Lyapunov
function time-derivative outside of compact
subset: E ⎧⎪⎨e : e ≤ 2 P B λ ( Λ ) ε ⎫⎪⎬
max
⎪⎩ λmin ( Q ) ⎭⎪
– the size of E defines tracking tolerance

– the size of E can be controlled!
25
E. Lavretsky
Projection Operator
• Function f (θ ) defines pre- y
specified parameter domain Proj (θ , y ) ∇f (θ )
boundary {θ }
f (θ ) = 1
θ
• Example:
θ∗
θ −θ
2 2
f (θ ) = max
εθ θ max
2
{θ f (θ ) ≤ 0}
{ f (θ ) ≤ 0} ⇒ { θ ≤ θ } ⇒ θ is within bounds
max
{0 < f (θ ) ≤ 1} ⇒ { θ ≤ 1 + εθ θ } ⇒ θ is within ( 1 + εθ ) % of bounds

max
{ f (θ ) > 1} ⇒ { θ > 1 + εθ θ } ⇒ θ is outside of bounds

max
• θ maxspecifies boundary
• εθ specifies boundary tolerance 26
E. Lavretsky
Projection Operator, (continued)

⎧ ( ) ( ( ) )
T
• Definition: ∇ f θ ∇ f θ
⎪⎪ y − y f (θ ) , if f (θ ) > 0 and y T
∇f (θ ) > 0
Proj (θ , y ) = ⎨ ∇f (θ )
2
⎪
⎪⎩ y, if not
• Depends on (θ , y ) y
• Does not alter y if θ is within the pre- Proj (θ , y ) ∇f (θ )
specified bounds: θ ≤ θ max {θ f (θ ) = 1}
2θ θ
• Gradient: ∇f (θ ) =
εθ θ max
2
θ∗
• {
In 0 ≤ f (θ ) ≤ 1 } the operator subtracts
gradient vector ∇f (θ ) (normal to the
boundary) from y {θ f (θ ) ≤ 0}
– get a smooth transition from y for λ = 0 to
a tangent vector field for λ =1
(θ − θ ∗ ) ( Proj (θ , y ) − y ) ≤ 0
T
• Important Property 27
E. Lavretsky
Lyapunov Function Time-Derivative with

Projection Operator
• Make trace terms semi-negative AND keep parameters
bounded: V = −eT Q e − 2 eT P B Λ ε x
f ( )
⎛ ⎧ ⎫ ⎞
⎜ ⎪ ⎪ ⎟
+2 trace ⎜ ∆K xT ⎨ Γ −x 1 Kˆ x + x eT P B sgn ( Λ ) ⎬ Λ ⎟
⎜ ⎪ Proj( Kˆ x , y ) ⎪ ⎟
⎝ ⎩ −y
⎭ ⎠
⎛ ⎧ ⎫ ⎞
⎜ T ⎪ −1 ˆ ⎪ ⎟
+2 trace ⎜ ∆K r ⎨ Γ r K r + r e P B sgn ( Λ ) ⎬ Λ ⎟
T
⎜ ⎪ Proj( Kˆ r , y ) ⎪ ⎟
⎝ ⎩ −y
⎭ ⎠
⎛ ⎧ ⎫ ⎞
⎜ ⎪ ˆ + Φ ( x ) eT P B sgn ( Λ ) ⎬⎪ Λ ⎟
+2 trace ⎜ ∆ΘT ⎨ Γ Θ−1 Θ ⎟
⎜ ⎪ Proj( Θˆ , y ) ⎪ ⎟
⎝ ⎩ −y
⎭ ⎠ 28
E. Lavretsky
Adaptation with Projection
( )
• Modified adaptive laws:
Kˆ x =Γ x Proj Kˆ x , − x eT P B sgn ( Λ )
Kˆ =Γ Proj ( Kˆ , − r e P B sgn ( Λ ) )
r r r
T
Θˆ =Γ Proj ( Θˆ , − Φ ( x ) e P B sgn ( Λ ) )
T
Θ
• Projection Operator, its bounds and tolerances are defined column-

wise
• Lyapunov function time-derivative:
V ≤ −eT Q e − 2 eT P B Λ ε f ( x ) ≤ −λmin ( Q ) e + 2 e P B λmax ( Λ ) ε
2
• Adaptive parameters stay within the pre-specified bounds, while V < 0

outside of the compact set: ⎧⎪ 2 P B λmax ( Λ ) ε ⎫⎪
E ⎨e : e ≤ ⎬
⎪⎩ λmin ( )
Q ⎭⎪ 29
E. Lavretsky
Example: Projection Operator, (scalar case)
(
• Scalar adaptive gain: k =γ Proj k , − x e sgn ( b )
ˆ ˆ )
• Pre-specified parameter domain boundary:
() −
() ()
– using function: ˆ
k 2
k 2
2 ˆ
k
f kˆ = max
∇f kˆ = f ′ kˆ = 2
ε kmax
2
ε kmax
{ f ( kˆ ) ≤ 0} ⇒ { kˆ ≤ k } ⇒ kˆ is within bounds
max
{0 < f ( kˆ ) ≤ 1} ⇒ { kˆ ≤ 1 + ε k } ⇒ kˆ is within ( 1 + ε ) % of bounds

max
{ f ( kˆ ) > 1} ⇒ { kˆ > 1 + ε k } ⇒ kˆ is outside of bounds

max
– Projection Operator:
y = − x e sgn ( b ) ( ) ( ( )) ()
⎧ y 1 − f kˆ , if f kˆ > 0 and y f ′ kˆ > 0
⎪
Proj k , y = ⎨
ˆ ()
⎪⎩ y, if not 30
E. Lavretsky
Example: Projection Operator, (scalar case)

(continued)
• Adaptive Law, (b > 0):
k=⎨
ˆ ⎪ ( ( ))
⎧− x e 1 − f kˆ , if ⎡ f kˆ > 0 and
⎣ () ()
x e f ′ kˆ ⎤ < 0
⎦
⎪⎩− x e, if not
()
kˆ 2 − kmax
2
where: f k = ˆ
ε kmax
2
• Geometric Interpretation
– adaptive parameter k̂ ( t ) changes within the pre-specified interval
– interval bound: kmax
– Bound tolerance: ε k̂ ( t )
− kmax 1 + ε kmax 1 + ε
x
−kmax 0 kmax 31
E. Lavretsky
Adaptive Augmentation Design

• Nominal Control: unom = FxT x + FrT r
• ˆ T Φ ( x)
Adaptive Control: u = Kˆ xT x + Kˆ rT r + Θ
• Augmentation: ˆ T Φ ( x) ± u
u = Kˆ xT x + Kˆ rT r + Θ nom
( ) ( ) ˆ T Φ ( x)
T T
= unom + Kˆ x − Fx x + Kˆ r − Fr r+Θ
Dˆ x Dˆ r
ˆ T Φ ( x)
= unom + Dˆ xT x + Dˆ rT x + Θ
• Incremental Adaptation:
( )
Dˆ x =Γ x Proj Dˆ x , − x eT P B sgn ( Λ ) , Dˆ x = 0n×m
Dˆ =Γ Proj ( Dˆ , − r e P B sgn ( Λ ) ) , Dˆ = 0
r r r
T
r m×m
Θˆ =Γ Proj ( Θ
ˆ , − Φ ( x ) e P B sgn ( Λ ) ) , Θ
T ˆ =0 32
Θ N ×m
E. Lavretsky
Adaptive Augmentation Block-Diagram

Ref.
xref x
Model -
e
Adaptive
Control
+ Allocation
Actuators A/C Sensors x
Baseline Inner-Loop
Controller
ReferencerInput, r(t)
• Reference Model provides desired response

• Nominal Baseline Controller
• Adaptive Augmentation
– Dead-Zone modification prevents adaptation from changing nominal
closed-loop dynamics
– Projection Operator bounds adaptation parameters / gains 33
E. Lavretsky
Adaptive Control using Sigmoidal NN

• System Dynamics: x = A x + B Λ ( u − f ( x ) ) , x ∈ Rn , u ∈ Rm
– A ∈ R n×n , Λ = diag ( λ1 … λm ) ∈ R m×m are constant unknown matrices
– B ∈ R M ×mis known constant matrix, and M ≥ m
• Approximation of uncertainty:
f ( x ) = W T σ (V T µ ) + ε f ( x ) , µ = ( xT 1) , ε f ( x ) ∈ R m
T
– matrix of constant unknown Inner-Layer weights:

⎡ v … vN ⎤ ( n +1)× N
V =⎢ 1 ⎥ ∈ R
⎣θ1 … θ N ⎦
– matrix of constant unknown Outer-Layer weights: ⎡ w … wm ⎤
∈ R( )
N +1 ×m
W =⎢ 1 ⎥
⎣ c1 … cm ⎦
– vector of N sigmoids and a unity:
( )
σ (V T µ ) = σ ( v1T x + θ1 ) … σ ( vNT x + θ N ) 1 , where: σ ( s ) =
1
T
1 + e− s 34
E. Lavretsky
Adaptive Control using Sigmoidal NN
• Control Feedback: u = Kˆ xT x + Kˆ rT r + Wˆ T σ Vˆ T µ( )
– (m n + m2 + (n + 1) N + (N + 1) m) - parameters to
estimate: Kˆ x , Kˆ r , Wˆ , Vˆ
• Adaptation with Projection, ( Λ > 0 ) :

⎪ x x (
⎧ Kˆ = Γ Proj Kˆ , − x eT P B
x )
⎪ˆ
⎪⎪ uK = Γ u Proj (
ˆ , − r eT P B
K u )
⎨
( ( ( ) ( ) )
⎪Wˆ = ΓW Proj Wˆ , σ Vˆ T µ − σ ′ Vˆ T µ Vˆ T µ eT P B
⎪
)
( ( ))
⎪Vˆ = Γ Proj Vˆ , µ eT P BWˆ T σ ′ Vˆ T µ
⎪⎩ V
• Provides bounded tracking

35
E. Lavretsky
Design Example
Adaptive Reconfigurable Flight Control using
RBF NN-s
Aircraft Model
• Flight Dynamics Approximation, (constant speed):
( ) (
x p = Ap x p + B G Λ δ + K 0 ( x p ) = Ap x p + B p Λ δ + K 0 ( x p ) )
Bp
x p = (α β p q r)
T
– State:
– Control allocation matrix G
– Virtual Control Input: δ ∈ R
3
– Modeling control uncertainty / failures by Λ ∈ R 3×3 diagonal matrix

with positive elements
– Vector of actual control inputs:
G Λ δ = (δ LOB δ LMB δ LIB δ RIB δ RMB δ ROB δ Tvec ) ∈ R 7
T
– Ap, Bp are known matrices

• represent nominal system dynamics
– Matched unknown nonlinear effects: K 0 ( x p ) ∈ R
3 37
E. Lavretsky
Baseline Inner-Loop Controller

• Dynamics: xc = Ac xc + B1 c x p + B2 c u
• States: x = ( q p r r ) ∈ R
T 4
c I I I w
• Inner-loop commands, (reference input):

( )
T
u= a cmd
z β cmd
p cmd
r cmd
• System output: a = C x + D G Λ (δ + K ( x ) ) = C x + D Λ (δ + K ( x ) )
z p p
Dp
0 p p p p 0 p
• Augmented system dynamics:

⎛x ⎞ ⎛ A 0 ⎞⎛ x ⎞ ⎛ B ⎞ ⎛ 0 ⎞
⎜ ⎟ =⎜B ⎟ ⎜ ⎟ + ⎜ ⎟ Λ (δ + K ( x ) ) + ⎜
p p p p
⎟u 0 p
⎝ ⎠
x A ⎝ ⎠ ⎝ ⎠
x 0 c ⎝ 1cB c ⎠ c ⎝ 2c ⎠
x A x B1 B2
x = A x + B1 Λ δ + K 0 ( x p ) + B2 u ( )
• Inner-Loop Control: δ L = K xT x + KuT u 38
E. Lavretsky
Reference Model
• Assuming nominal data, ( Λ = I 3×3 , K 0 ( x p ) = 03×1 , ) and

using baseline controller:
( ) ( )
xref = A + B1 K xT xref + B2 + B1 K uT u = Aref xref + Bref u
Aref Bref
• Assumption: Reference model matrix A is ref
Hurwitz, (i.e., baseline controller stabilizes

nominal system)
39
E. Lavretsky
Inner-Loop Control Objective

(Bounded Tracking)
• Design virtual control input such that,

despite system uncertainties, the system
state tracks the state of the reference
model, while all closed-loop signals remain
bounded
• Solution
– Incremental, (i.e., adaptive augmentation),
MRAC system with RBF NN, Dead-Zone, and
Projection Operator
40
E. Lavretsky
Adaptive Augmentation
• Total control input: δ = Kˆ xT x + Kˆ uT u − Kˆ 0 ( x p ) ± δ L ( x, u )
Total Adaptive Control Nominal Baseline
= δ L ( x, u ) + Kˆ x − K x( ) ( ) u − Kˆ 0 ( x p )
T T
x + Kˆ u − K u
δ L ( x,u ) kˆx kˆu Θ ( )
ˆT Φ x
p
= δ L ( x p , xc , u ) + ∆Kˆ xT x + ∆Kˆ uT u − Θ
ˆ T Φ(x )
p
Nominal Baseline Incremental Adaptive Control
• Incremental adaptation with projection:

⎪ x x (
⎧∆Kˆ = Γ Proj ∆Kˆ , − x eT P B ,
x 1 ) ∆Kˆ x ( 0 ) = 0n×3
⎪ ˆ
(
⎨∆K u = Γu Proj ∆Kˆ u , − u e P B1 ,
T
) ∆Kˆ u ( 0 ) = 0n×4
⎪
⎪Θ
⎩
ˆ = Γ Proj Θ
Θ (
ˆ , Φ ( x ) eT P B ,
p 1 ) ˆ (0) = 0
Θ N ×m
41
E. Lavretsky
Inner-Loop Block-Diagram
x = (α β rw )
T
Ref.
xref p q r qI pI rI
Model -
e
Adaptive
Control
+ Allocation
Actuators A/C Sensors x
Baseline Inner-Loop
Controller
(
u = azcmd β cmd pcmd rcmd )
• Reference Model provides desired response
• Nominal Baseline Inner-Loop Controller
• Adaptive Augmentation
– Dead-Zone modification prevents adaptation from changing nominal
closed-loop dynamics
– Projection Operator bounds adaptation parameters / gains 42
E. Lavretsky
Adaptive Backstepping
E. Lavretsky
Why?
• MRAC requires model matching conditions
A + B Λ K xT = Am
B Λ K rT = Bm
• Example that violates matching

– System: ⎛ x1 ⎞ ⎛ 0 1 ⎞ ⎛ x1 ⎞ ⎛ 0 ⎞
⎜ ⎟=⎜ ⎟⎜ ⎟ + ⎜ ⎟u
⎝ x2 ⎠ ⎝
0 0 ⎠ ⎝ x2 ⎠ ⎝ 1 ⎠
N ⎛ x1m ⎞ ⎛ −1 1 ⎞ ⎛ x1m ⎞ ⎛ 0 ⎞
A b
⎜ m⎟=⎜ ⎟⎜ m ⎟ + ⎜ ⎟ r
– Reference model: ⎝ x2 ⎠ ⎝0 −2 ⎠ ⎝ x2 ⎠ ⎝ 1 ⎠

Am
Matching ⎛1 0⎞ ⎛ 0 0⎞
conditions A − Am = ⎜ ⎟ ≠ b kx = ⎜
T
⎟
don’t hold ⎝ 0 2 ⎠ ⎝ ∗ ∗ ⎠
E. Lavretsky
Control Tracking Problem

• Consider 2nd order cascaded system
x1 = f1 ( x1 ) + g1 ( x1 ) x2
x2 = f 2 ( x1 , x2 ) + g 2 ( x1 , x2 ) u
• Control goal
– Choose u such that: 1 ( )
x t → x1
com
( t ) , as t → ∞
• Assumptions
⎧ P
x1
P
x2
– All functions are known ⎪α = − Lα (α ) α + 1N q
⎪
g1
– gi ≠ 0 does not cross zero ⎨ f1
⎪q = M 0 (α , q ) + 1N qcmd
• Example: AOA tracking ⎪
g2 N
⎩ f2 u
E. Lavretsky
Backstepping Design
• Introduce pseudo control: x2com = x2com ( t )
• Rewrite the 1st equation:
x1 = f1 ( x1 ) + g1 ( x1 ) x2com + g1 ( x1 ) ( x2 − x2com )

∆x2
• Dynamic inversion using pseudo control:

x com
2 =
1
g1 ( x1 )
( x1com − f1 ( x1 ) − k1 ∆x1 )
• 1st state error dynamics: ∆x1 = − k1 ∆x1 + g1 ( x1 ) ∆x2
E. Lavretsky
Backstepping Design
(continued)
• Dynamic inversion using actual control
u=
1
g 2 ( x1 , x2 )
(
x com
2 − f 2 ( x1 , x2 ) − k2 ∆x2 − g1 ( x1 ) ∆x1 )
• 2nd state error dynamics

∆x2 = − k2 ∆x2 − g1 ( x1 ) ∆x1
• Asymptotically stable error dynamics
⎛ ∆x1 ⎞ ⎛ −k1 g1 ( x1 ) ⎞ ⎛ ∆x1 ⎞ nonlinear system
⎜ ⎟=⎜ ⎟⎜ ⎟
⎝ 2 ⎠ ⎝ 1 ( 1)
∆x − g x − k 2 ⎠⎝ ∆x2⎠
• Conclusion: xi ( t ) → xicom ( t ) , as t → ∞
E. Lavretsky
Adaptive Backstepping Design

• 1st state dynamics: x1 = fˆ1 + gˆ1 x2com + gˆ1 ∆x2 − ∆f1 − ∆g1 u
– Function estimation errors:
∆f1 fˆ1 − f1 , ∆g1 gˆ1 − g1
• Dynamic inversion using pseudo control
and estimated functions:
x com
2 =
1
gˆ1 ( x1 )
(x1com − fˆ1 ( x1 ) − k1 ∆x1 )
• 1st state error dynamics:
∆x1 = −k1 ∆x1 + gˆ1 ∆x2 − ∆f1 − ∆g1 u
E. Lavretsky

(continued)
• 2nd state dynamics: x2 = fˆ2 + gˆ 2 u − ∆f 2 − ∆g 2 u
– Function estimation errors:
∆f 2 fˆ2 − f 2 , ∆g 2 gˆ 2 − g 2
• Dynamic inversion using actual control
and estimated functions:
u=
1
gˆ 2 ( x1 , x2 )
(
x2com − fˆ2 ( x1 , x2 ) − k2 ∆x2 − gˆ1 ( x1 ) x1 )
• 2nd state error dynamics:
∆x2 = − k2 ∆x2 − gˆ1 ∆x1 − ∆f 2 − ∆g 2 u
E. Lavretsky

(continued)
• Combined error dynamics:
⎛ ∆x1 ⎞ ⎛ −k1 gˆ1 ( x1 ) ⎞ ⎛ ∆x1 ⎞ ⎛ −∆f1 − ∆g1 u ⎞
⎜ ⎟=⎜ ˆ ⎟⎜ ⎟+⎜ ⎟
⎝ 2 ⎠ ⎝ 1 ( 1)
∆x − g x − k 2 ⎠⎝ ∆x2⎠ ⎝ −∆ f 2 − ∆g 2 ⎠
u
• Uncertainty parameterization, function and

parameter estimation errors:
∆fi = ∆θ Tfi Φ f ( x1 , x2 ) − ε fi ∆θ fi θˆfi − θ fi
∆gi = ∆θ gTi Φ g ( x1 , x2 ) − ε gi ∆θ gi θˆgi − θ gi
E. Lavretsky

(continued)
• Tracking error dynamics:
⎛ ∆x1 ⎞ ⎛ −k1 gˆ1 ⎞ ⎛ ∆x1 ⎞ ⎛ ∆θ f1 ∆θ g1 ⎞ ⎛ Φ f ⎞ ⎛ ε f1 + ε g1 u ⎞
T T
⎜ ⎟=⎜ ˆ ⎟⎜ ⎟ − ⎜⎜ ∆θ T ∆θ T ⎟⎟ ⎜ Φ u ⎟ + ⎜ ε + ε u ⎟
⎝N∆x2 ⎠ ⎝ − g1 −k2 ⎠ ⎝ ∆x2 ⎠ ⎝ f2 g2 ⎠ ⎝ ⎠ ⎝ ⎠

N

g f2

g2
e A e Φ ε
∆ΘT
e = Ae − ∆ΘT Φ + ε
• Stable robust adaptive laws:
ˆ
(
Θ = Γ Proj Θ
ˆ , Φ eT )
• Conclusion: Bounded tracking
E. Lavretsky
Adaptive Control in the Presence of

Actuator Constraints*
*
E. Lavretsky and N. Hovakimyan, “Positive µ – modification for stable adaptation in the presence of input
constraints,” ACC, 2004.
Overview
• Problem: Assure stability of an adaptive control system in the
presence of actuator position / rate saturation constraints.
• Solutions
– Ad-hoc
– Proof-by-simulation not acceptable
– Lyapunov based preferred

• limited class of systems
• sufficient conditions are often hard-to-verify
• Need: Theoretically justified and verifiable conditions for stable
adaptation and control design with a possibility of avoiding actuator
saturation phenomenon.
• Design Solutions include modifications, (adaptive / fixed gain) to:
– control input
– tracking error
– reference model
2
Known Design Solutions

• R. Monopoli, (1975)
– adaptive modifications: tracking error and reference input
– no theoretical stability proof
• S.P. Karason, A.M. Annaswamy, (1994)
– adaptive modifications: reference input
– rigorous stability proof
• E.N. Johnson, A.J. Calise, (2003)
– pseudo control hedging (PCH)
• fixed gain modification of reference input
• E. Lavretsky, N. Hovakimyan, (2004)
– positive µ – modification
• adaptive modification of control and reference inputs
• rigorous stability proof and verifiable sufficient conditions
• capability to completely avoid control saturation 3
Adaptive Control in the Presence of Input Constraints:

Problem Formulation
battle damage control failures
• System dynamics: x ( t ) = A x ( t ) + b λ u ( t ) , x ∈ R , u ∈ R
n
– A is unknown matrix, (emulates battle damage)

– b is known control direction
– λ > 0 is unknown positive constant, (control failures)
• Static actuator amplitude saturation
⎛ uc ⎞ ⎧⎪uc ( t ) , uc ( t ) ≤ umax
u ( t ) = umax sat ⎜ ⎟=⎨
⎝ umax ⎠ ⎪⎩umax sgn ( uc ( t ) ) , uc ( t ) ≥ umax
• Ideal Reference model dynamics: commanded input
xm* ( t ) = Am xm* ( t ) + bm r ( t ) , xm* ∈ R n , r ∈ R
Hurwitz bounded reference input 4

Preliminaries
safety zone
δ
• Define: umax = umax − δ , where: 0 < δ < umax
δ⎛ uc ⎞
• Commanded control deficiency: ∆uc = umax sat ⎜ δ ⎟ − uc
⎝ umax ⎠
• Adaptive control with µ– modification, (implicit
form):
u c = k x x + k r r + µ ∆u c
T

ulin
linear feedback /
control deficiency feedback
feedforward component
• Need explicit form of uc

5
Positive µ – modification
• Adaptive control with µ – mod is given by
⎛ ulin ⎞ δ
convex combination of ulin and umax sat ⎜ δ ⎟
⎝ umax ⎠
⎧
⎪u , u ≤ u δ
⎪ lin lin max
1 ⎛ ⎛ ulin ⎞⎞ ⎪ 1
uc = ⎜⎜ ulin + µ umax sat ⎜ δ
1+ µ ⎝
δ
=
⎟ ⎟⎟ ⎨
+ µ
( lin
u + µ u max )
δ
, u lin > u δ
max
⎝ umax ⎠⎠ ⎪ 1
⎪ 1
⎪
+ µ
( lin
u − µ u max )
δ
, ulin < − u δ
max
continuous in time but not ⎩1
continuously differentiable
⎛ ulin ⎞
δ = 0 ∧ ( µ = 0 ∨ µ = ∞ ) ⇒ u = umax sat ⎜ ⎟
⎝ max ⎠
u 6
Closed-Loop Dynamics
• µ – mod control: uc = ulin + µ ∆uc
∆u

• System dynamics: x = A x + b λ uc + b λ ( u − uc )
• Closed-loop system:
linear control deficiency
∆ulin =u −ulin

x = A x + b λ ulin + b λ ( µ ∆uc + ∆u )
⎛ uc ⎞
where: ∆ulin = umax sat ⎜ ⎟ − ulin
⎝ umax ⎠
does not depend on µ explicitly
x = ( A + b λ k xT ) x + b λ ( kr r + ∆ulin ) 7
Adaptive Reference Model Modification

• Closed-loop system:
x = ( A + b λ k xT ) x + b λ ( kr r + ∆ulin )
• Leads to consideration of adaptive

reference model: adaptive augmentation
(
xm = Am xm + bm r ( t ) + ku ∆ulin , ) r ( t ) ≤ rmax
reference input
• Matching conditions: ⎧ A + b λ ( k T )* = A
⎪⎪ x m
∀ λ > 0 ∃ ( k x∗ ∈ R n , kr∗ ∈ R, ku∗ ∈ R ) ⎨b λ kr* = bm ⎫⎪

⎪ *⎬
⇒ k u kr = 1
* *
⎪⎩b λ = bm ku ⎭⎪ 8
Adaptive Laws Derivation
• Tracking error: e = x − xm ⎧∆k x = k x − k x∗

⎪ ∗
• Parameter errors: ∆
⎨ rk = k r − k r
⎪ ∗
∆
⎩ uk = k u − k u
• Tracking error dynamics:

e = Am e + b λ ( ∆k xT x + ∆kr r ) − bm ∆ku ∆ulin
• Lyapunov function:
V ( e, ∆k x , ∆kr , ∆ku ) = eT P e + λ ( ∆k xT Γ −x 1 ∆k x + γ r−1 ∆kr2 + γ u−1 ∆ku2 )
where: P Am + Am P = −Q < 0
9
Stable Parameter Adaptation

• Adaptive laws derived to yield stability:
kx = −Γ x x eT P b
kr = −γ r r ( t ) eT P b ⇔ V = −eT Q e < 0 ⇒ V ( e, ∆k x , ∆kr , ∆ku ) ≤ 0
k = γ ∆u eT P b
u u lin m
• For open-loop stable systems – global result

• For open-loop unstable systems verifiable
sufficient conditions established:
• upper bound on rmax
• lower bound on µ
• upper bounds on initial conditions x(0) and Lyapunov function
V(0) 10
µ – mod Design Steps

• Choose “safety zone” 0 < δ < umax and sufficiently large µ > 0
δ
• Define virtual constraint: umax = umax − δ
= x x + kr r ( t )
T
u
• Linear component of adaptive control signal: lin k
• Total adaptive control with µ – mod: adaptive laws
⎧kx = −Γ x x eT P b
1 ⎛ ⎛ ulin ⎞⎞ ⎪⎪
⎜⎜ ulin + µ umax sat ⎜ δ
δ
uc = ⎟ ⎟⎟
⎨ r
k = − γ r ( t ) e T
Pb
1+ µ ⎝ ⎝ umax ⎠⎠ ⎪
r
⎪⎩ u
k = γ u ∆ u lin e T
P bm
⎛ ⎞
⎜ ⎛ ⎛ uc ⎞ ⎞⎟
xm = Am xm + bm ⎜ r + ku ⎜⎜ umax sat ⎜ ⎟ − ulin ⎟⎟ ⎟
modified reference
model
⎜ ⎝
⎝ umax ⎠ ⎠⎟
⎜ ⎟ 11
⎝ ∆ulin ⎠
Simulation Example
• Unstable open-loop system:
⎛ uc ⎞
x = a x + b umax sat ⎜ ⎟ , where: a = 0.5, b = 2, umax = 0.47
⎝ umax ⎠
• Choose: δ = 0.2umax
δ
umax = umax − δ = 0.8 umax
• Ideal reference model: xm = −6 ( xm − r ( t ) )

• Reference input: r ( t ) = 0.7 ( sin ( 2 t ) + sin ( 0.4 t ) )
• Adaptation rates set to unity
• System and reference model start at
zero 12
Simulation Data
µ =0 µ =1
13
µ = 10 µ = 100
µ – mod Design Summary

• Lyapunov based
• Provides closed-loop stability and bounded
tracking
– convex combination of linear adaptive control and
δ
u
its max – limited value
– adaptive reference model modification
• Verifiable sufficient conditions
• Future Work
– MIMO systems
– Dynamic actuators
– Nonaffine-in-control dynamics
14
– Flight control applications
Adaptive Flight Control Applications,

Open Problems, and Future Work
Autonomous Formation Flight, (AFF)
References:
• Lavretsky, E. “F/A-18 Autonomous Formation Flight Control System Design”, AIAA GN&C
Conference, Monterey, CA, 2002.
• Lavretsky, E., Hovakimyan, N., Calise, A., Stepanyan, V. “Adaptive Vortex Seeking Formation
Flight Neurocontrol”, AIAA-2002-4757, AIAA GN&C Conference,St. Antonio, TX, 2003.
AFF: Program Overview
• Program participants:
– NASA Dryden
– Boeing - Phantom Works
– UCLA
• Flight test program
– Completed in December of 2001
– 2 F/A-18 Hornets, 45 flights
– Demonstrated up to 20% induced
aerodynamic drag reduction
• AFF Autopilot
– Baseline linear classical design to meet
stability margins
– Adaptive incremental system to
counteract unknown vortex effects and
environmental disturbances
– On-line extremum seeking command
generation 3
AFF: Lead Aircraft Wingtip Vortex Effects

Induced Drag Ratio & Rolling Moment Coefficient
Drag Reduction (φ = 0) Induced Rolling Moment (φ = 0)

3.00 0.03
∆Z ∆Z
+280
0.02
+280
2.50 +240 +240
+200 +200
+160 0.01 +160
+120 +120
CD form / CD
2.00 +080 +080

+040 0.00 +040
+020 +020
∆Cl
+010 +010
1.50 +005 -0.01 +005
+000 +000
-005 -005
-010 -0.02 -010
1.00 -020 -020
-040 -040
-080 -0.03 -080
-120 -120
0.50 -160 -160
-200 -0.04 -200
-240 -240
-280 -280
0.00
-0.05
0 100 200 300 400 500 600 700 800 900 1000
0 100 200 300 400 500 600 700 800 900 1000
∆Y ∆Y
close to vortex
“sweet” spot induced roll
reversal
4
AFF: Trailing Aircraft Dynamics in Formation
• Trailing Aircraft:
Longitudinal Dynamics Lateral Dynamics
⎧l = V y = − g nz sin φ
⎧
⎪ ⎪
⎨ ρV 2 ⎨φ = p
(
⎪V = − g sin γ + 2 m S CTδT δ T − CD ( M , α )η ( y, φ ) ) ⎪ p = − a p + b δ + ξ y, φ
⎩
⎩ p δa a ( )
Throttle Input
Aileron Input
z Trailing Aircraft Modeling Assumptions
st
¾ SCAS yields 1 order roll dynamics & turn coordination
¾ a p , bδ , CT are unknown positive constants

a δT
¾ C D ( M , α ) , η ( y , φ ) , ξ ( y , φ ) are unknown bounded functions of
known arguments and shapes

z Lead aircraft trimmed for level flight
5
AFF: Vortex Seeking Formation Flight Control
• Problem: Using throttle and aileron inputs

– Track desired longitudinal displacement command lc
– Generate on-line and track lateral separation command yc in order to:
• Minimize unknown vortex induced drag coefficient η ( y, φ ) with
respect to its 1st argument, (lateral separation)
ρ
( )
2
V
V = − g sin γ + S CTδ δ T − CD ( M , α )η ( y, φ )
2m T
• Remarks:
– Aileron controls lateral separation
– Throttle controls longitudinal separation
• depends on lateral separation through unknown function η ( y , φ )
• Solution
– Using Direct Adaptive Model Reference Control
– Radial Basis Functions for approximation of uncertainties
∂ηˆ ( y, φ )
– Extremum Seeking Command Generation y r = −γ , γ > 06
∂y y = yr
AFF: Simulation Data
7
Open Problems and Future Work

Task 1: Validation & Verification (V&V) of

Adaptive Systems
• Significant industry effort going into development of adaptive

/ reconfigurable GN&C systems
• Methods to test and certify flight critical systems are not
readily available
• There exists a necessity to develop V&V methods and
certification tools that are similar to and extend the current
process for conventional, non-adaptive GN&C systems
• Theoretically justified V&V technologies are needed to:
– provide a standard process against which adaptive GN&C systems can
be certified
– offer certification guidelines during the early design cycle of such
systems
9
Task 1: V&V of Adaptive Systems

Road Map to Solution (Issue Paper)
Provide Evidence
Preliminary V&V of Compliance
Design Concept Certification
Process
Certification of V&V
Intelligent System Tool and Process
Certification
Working Group Verification
Certification
Plan/Conduct
Authorities
Certifiable Verification
(FAA/DoD/
• Design Events
NASA) Certification
• Hardware (Test Plan)
Guidelines
Definition of • Software
Intelligent System Design and Build
Requirements
Certification Implementation
Requirement Requirements
(Cert. Plan) 2T049047.3
Figure C.3-16. Process for Certifying the Validation and Verification (V&V) Software and
• Goal: Provide theoretically
Developing justified
the Certification Two Major
z Intelligent
Process for Tasks
Adaptive Systems
V&V method and a process-based ¾ Stability Margins / Robustness
acceptance procedure to certify Analysis
current and future intelligent / ¾ S/W V&V Procedures
adaptive GN&C flight critical systems
10
Task 1: V&V of Adaptive Systems

Subtask: Theoretical Stability / Robustness Analysis
• Establish adaptive control design guidelines
– Define rates of adaptation
– Calculate stability / robustness margins
– Determine bounds on control parameters that correspond to
stability / robustness margins
• Perform system validation using the derived margins
• Incorporate modifications that lead to improvement (if
required) in the stability / robustness margins
• Validate closed-loop system tracking performance
11
Task 2: Integrated Vehicle Health Management

(IVHM) and Composite Adaptation
• Aerodynamic parameters are of paramount importance

to IVHM system functionality
• Examine different sources of on-line aerodynamic
parameter estimation
– Tracking errors
– Prediction errors
• Composite Adaptive Flight Control = (Indirect + Direct)
MRAC
12
Task 3: Persistency of Excitation in Flight

Mechanics
• Information content from adaptation / estimation processes depends

on parameter convergence
– Requires persistent excitation (PE) of control inputs
• Need numerically stable / on-line verifiable PE conditions for flight
mechanics and control
• Aircraft Example: Longitudinal dynamics
⎧ T cos α − D
− g sin (θ − α ) ⎧T = q S CT ≅ q S CTδT δ T Control Inputs
⎪V = m ⎪
⎪ ⎪⎪ L = q S CL ≅ q S CL (α , q )
⎪α = q − T sin α + L + g cos (θ − α ) ⎨D = q S C ≅ q S C α , q
D( )
⎪
⎨ mV ⎪ D
⎪
⎪
q =
M
Iy
⎪
(
⎪⎩ M = q S c CM ≅ q S c CM 0 (α , q ) + CM δe (α , δ e ) δ e )
⎪
⎪θ = q
⎩
Problem:
¾Estimate on-line unknown aerodynamic coefficients
¾Find sufficient conditions (PE) that yield convergence of the estimated 13
parameters to their corresponding true (unknown) values
Design Example:
F-16 Adaptive Pitch Rate Tracker
Aircraft Data
Short-Period Dynamics
• Trim conditions
– CG = 35%, Alt = 0 ft, QBAR = 300 psf, VT = 502 fps, AOA = 2.1 deg
• Nominal system
– statically unstable
– open-loop dynamically stable, (2 real negative eigenvalues)
• Control architecture
– baseline / nominal controller
• LQR pitch tracking design
– direct adaptive model following augmentation
• Simulated failures
– elevator control effectiveness: 50% reduction
– battle damage instability
• static instability: 150% increase
• pitch damping: 80% reduction
– pitching moment modeling nonlinear uncertainty 2
LQR PI Baseline Controller

• Using LQR PI state feedback design
– nominal values for stability & control
derivatives
– pitch rate step-input command
– no uncertainties, no control failures
⎧e = q − q I cmd
– system dynamics: ⎪
q
⎪ Z Z
α = α + + α δ
δe
– “wiggle” system in matrix form ⎨⎪ V q
V
⎛0 0 1 ⎞ ⎛ 0 ⎞ ⎪q = M α α + M q q + M δ δ e
⎛ eq ⎞ ⎜ ⎛ ⎞ ⎩
⎟ q ⎜
e ⎟
⎜ ⎟ ⎜ Zα ⎜ ⎟ Z
⎜α ⎟ = ⎜0 V 1 ⎟ ⎜ α ⎟ + ⎜ δ ⎟ δe ⇔ x = A x + B u
⎜ q ⎟ ⎟⎜ ⎟ ⎜ V ⎟ N
⎝N⎠ ⎜ 0 M ⎟ ⎝ q ⎠ ⎜ ⎟
u
M q ⎠ N ⎝ Mδ ⎠
⎝
α
x
x N 3
A B
LQR PI Baseline Controller

(continued)
• LQR design for the “wiggle” system
– Optimal feedback solution: u = − K x
– Using original states:
⎛ eq ⎞
⎜ ⎟
δebl = − ( K qI Kα K q ) ⎜ α ⎟ = − K qI eq − Kα α − K q q
⎜ q ⎟
⎝ ⎠ K
P
δ
– Integration yields LQR PI feedback: e
bl
= − Kx x
δ ebl = − K qI eqI − Kα α − K q q δ ebl = −10 eqI − 3.2433α − 10.7432 q
closed-loop
eigenvalues
⎛0 0 1 ⎞ ⎛ 0 ⎞ ⎛ 100 0 0 ⎞ Eigenvalue Damping Freq. (rad/s)

⎜ ⎟ ⎜ ⎟ ⎜ ⎟
A = ⎜ 0 -1.0189 0.9051 ⎟ , B = ⎜ -0.0022 ⎟ , Q = ⎜ 0 1 0 ⎟
-7.97e-001 + 3.45e-001i 9.18e-001 8.68e-001
⎜ 0 0.8223 -1.0774 ⎟ ⎜ -0.1756 ⎟ ⎜ 0 0 100 ⎟ -7.97e-001 - 3.45e-001i 9.18e-001 8.68e-001

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ -2.39e+000 1.00e+000 2.39e+000 4
Short-Period Dynamics with

Uncertainties
⎛0 0 1 ⎞ I ⎛ 0 ⎞
• System: ⎛ eqI ⎞ ⎜ ⎛
⎟ q ⎜
e ⎞ ⎟ ⎛ −1⎞
⎜ ⎟ ⎜ Zα ⎟ ⎜ α ⎟ + ⎜ Zδ ⎟ Λ ( δ e + K 0 (α , q ) ) + ⎜ 0 ⎟ q cmd
α
⎜ ⎟ ⎜ = 0 1
V ⎟⎜ ⎟ ⎜ V ⎟ ⎜ ⎟
⎜0⎟
⎜ q ⎟ ⎜ ⎜q⎟ ⎜
⎝N⎠ 0 M α M q ⎟ ⎝N ⎠ Mδ ⎟ ⎝N⎠
⎝

x⎠ ⎝
N ⎠
x B2
A B1 = B
• Reference model: x = A x + B1 Λ ( δ e + K 0 (α , q ) ) + B2 q cmd

– no uncertainties
– (Plant + Baseline LQR PI)
xref = ( A + B1 K xT ) xref + B2 q cmd = Aref xref + Bref q cmd

N
B
ref
Aref
tracking error vector
• Control Goal
– Model following pitch rate tracking: x − xref → 0 5

• Total elevator deflection:
K 0 (α , q )
ˆ

δ e = δ ebl + δ ead ˆ T Φ (α , q )
= K qI eqI + Kα α + K q q + kˆqI eqI + kˆα α + kˆq q − Θ

δ ebl δ ead
( ) ˆ T Φ (α , q )
T
δ e = K x + kˆx x−Θ
⎧⎛ kˆ ⎞ ⎛ ⎛ kˆ ⎞ ⎛ 0 ⎞⎞
• Adaptive laws: ⎪⎜ α ⎟ ⎜⎜ α ⎟ ⎛ α ⎞
⎪⎜ kˆ ⎟ = Γ Proj ⎜ ⎜ kˆ ⎟ , − ⎜ q ⎟ q − q ref α − α
⎜ ⎟⎟
⎪⎜ q ⎟ x
⎜ ⎜ q ⎟ ⎜⎜ ⎟⎟ I
( I ref q − qref ) P ⎜ Zδ ⎟ ⎟
⎜ V ⎟⎟
⎪⎜ ˆ I ⎟ ⎜ ⎜ ⎟ ⎜ M ⎟⎟
( )
ˆ ⎝ I⎠
q
⎧kˆ = Γ Proj kˆ , − x eT P B
I
⎪⎜⎝ kq ⎟⎠
k
⎪ x x x 1 ⎝⎝ q ⎠ ⎝ δ ⎠⎠
⎨ ⎨
⎪Θ
⎩
ˆ = Γ Proj Θ
Θ (
ˆ , Φ ( x ) eT P B
p 1 ) ⎪
⎪
⎛
⎜
⎛ 0 ⎞⎞
⎜ ⎟⎟
⎪Θ ˆ , Φ (α , q ) ( q − q ref α − α
ˆ = Γ Proj ⎜ Θ q − qref ) P ⎜ Zδ ⎟ ⎟
Θ
⎪ ⎜ I I ref
⎜ V ⎟⎟
⎪ ⎜ ⎜ M ⎟⎟ 6
⎩ ⎝ ⎝ δ ⎠⎠

(continued)
• Free design parameters
– symmetric positive definite matrices: ( Q, Γ x , ΓΘ )
• Need to solve algebraic Lyapunov equation

P Aref + Aref
T
P = −Q
• Using Dead-Zone modification and Projection

Operator
x = ( qI α q)
T
xref
Ref.
Model -
e
Adaptive
δ ead δe Control
+ Allocation
Actuators F-16 Sensors x
δ ebl
Baseline Inner-Loop
Controller
qcmd 7
Adaptive Design Data

• Design parameters (α −αi )2
−
– using 11 RBF functions: φi = e σ2
, αi ∈[ −10:.1:10]
– Rates of adaptation:
Γx = 0, ΓΘ = 1
– Solving Lyapunov equation with: Q = diag ([ 0 1 800])
• Zero initial conditions
• Pitch rate command input
• System Uncertainties
– 50% elevator effectiveness failure,( 0.5* Mδbl )
– 50% increase in static instability, (1.5* Mα )
bl M (α )
– 80% decrease in pitch damping, ( 0.2* M qbl )
– nonlinear pitching moment
⎛ 2π ⎞
2
⎜α − ⎟
−⎝
180 ⎠
M (α ) = 1.5* Mαbl + e 0.01162 8
LQR PI: Tracking Step-Input

Command
Unstable Dynamics due to Uncertainties 9

LQR PI + Adaptive: Tracking Step-

Input Command
Adaptive Augmentation yields Bounded Stable Tracking 10

in the Presence of Uncertainties
LQR PI: Tracking Sinusoidal Input

with Uncertainties
LQR PI Tracking Performance Degradation in the

11
Presence of Uncertainties
LQR PI + Adaptive: Tracking

Sinusoidal Input with Uncertainties
Adaptive Augmentation Recovers Target Tracking 12

Dynamics in the Presence of Uncertainties
Model Following Tracking Error

Comparison
Adaptive Augmentation yields Significant Reduction in 13

Tracking Error Magnitude
Adaptive Design Comments

• RBF NN adaptation dynamics
ˆ
Θ i = ( Γ )
Θ ii Φ i (α , q ) ( k1i ( q I − q I ) + k 2 i (α − α ref ) + k3 i ( q − qref ) )
ref
• Fixed RBF NN gains

⎛ 0 ⎞
– simulation data ⎛ k1i ⎞ ⎜ ⎟
⎜ ⎟ Z
k1i = 0, k2 i = −1.1266, k3i = −24.0516 k
⎜ 2i ⎟ = P ⎜ δ ⎟
( ΓΘ )ii
⎜ k3 i ⎟ ⎜ V ⎟
⎝ ⎠ ⎜M ⎟
• Projection Operator ⎝ δ⎠
– keeps parameters bounded
– nonlinear extension of anti-windup integrator logic
• Dead-Zone modification dead-zone tolerance
– freezes adaptation process if: x − xref ≤ ε

– separates adaptive augmentation from baseline 14
controller

IEEE WorkShop Slides Lavretsky

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IEEE WorkShop Slides Lavretsky

Uploaded by

Copyright:

Available Formats

Robust and Adaptive Control Workshop Adaptive Control: Introduction, Overview, and Applications

Adaptive Control: Introduction,

Eugene Lavretsky, Ph.D.

Motivating Example: Roll Dynamics

• Uncertain Roll dynamics: p = Lp p + Lδ ail δ ail

– δ ( t ) is a reference input, (pilot stick, guidance command)

Motivating Example: Roll Dynamics

desired flying qualities model roll tracking

parameter adaptation loop

• Adaptive control provides Lyapunov stability

Lyapunov Stability Theory

Alexander Michailovich Lyapunov

Nonlinear Dynamic Systems and

Example: Equilibrium Points of a

• State space representation, ( x 1 = θ , x2 = θ )

Example: Linear Time-Invariant

– has a single equilibrium point (the origin 0) if

• New equilibrium: y = 0, (since f(xe) = 0)

• The error dynamics:

• Lyapunov Stability means that the system

• Definition: An equilibrium point 0 is

• Asymptotic stability means that the equilibrium is

• The state vector of an exponentially stable

Local and Global Stability

Lyapunov’s 1st Method

– If A is Hurwitz then the equilibrium is asymptotically

Lyapunov’s Direct (2nd) Method

Lyapunov’s Direct Method

• Question: If the mass is pulled away and

Lyapunov’s Direct Method

• Total energy rate of change along the

• Conclusion: Energy of the system is

Lyapunov’s Direct Method

Positive Definite Functions

• If V ( 0 ) = 0 ∧ {∀ x ≠ 0} ⇒ V ( x ) > 0 then V(x) is

– if Vmin =/= 0 or xmin =/= 0 then use

Lyapunov Stability Theorem

equilibrium point 0 is stable

Example: Local Stability

• Lyapunov function candidate: ( ) (

• Conclusion: System is locally stable

Example: Asymptotic Stability

• Lyapunov function candidate: V ( x1 , x2 ) = x12 + x22

V ( x1 , x2 ) = 2 ( x12 + x22 )( x12 + x22 − 2 ) < 0

• Conclusion: System is locally

Example: Global Asymptotic

• Lyapunov function candidate: V ( x ) = x2

– globally positive definite

• Conclusion: System is globally

La Salle’s Invariant Set Theorems

Example: 2nd Order Nonlinear

– time-derivative is negative semi-definite

Lyapunov Functions for LTI

– where Q is symmetric positive definite matrix

Stability of LTI Systems

Barbalat’s Lemma: Preliminaries

• Invariant set theorems of La Salle provide

Example: LTI System

• Proof: Since u is bounded and the system is

• Question: Why is this fact so important?

Example: Stable Adaptation

Uniform Ultimate Boundedness

UUB Example : 1st Order System

exists a bound B and a time T ( B, x ( t ) ) such 0