Adaptive Volterra Filters 265

10.3. Compute the gradients of the mean square error function, E{e 2 [n]} , with
respect to h0, a and B, for the second-order Volterra filter in the case when
the data are complex.

10.4. What would the LMS weight update equations be for Problem 10.3?
11. Adaptive Control Systems

Classical control theory has mostly been concerned with the design of feedback
systems for time-invariant plants with known transfer functions (Levine 1996).
However, the assumptions of known mathematical models and time-invariance are
not valid for many modern control problems. For example, in robotics the dynamic
models vary with robot attitude and load variations. Chemical reactor transfer
functions vary as a function of reagent mix, catalyst and time. These types of
problems might be solved using a classical approach by designing a robust fixed
controller that ensures stability for all possible plant dynamics. However, this
approach may often be at the expense of suboptimal control behaviour. The other
approach is to use adaptive control algorithms that can learn from the plant input-
output behaviour and thereby develop on-line self-tuning controllers to improve the
closed loop performance. There are typically two main themes found in relation to
learning, or adaptive controllers. Systems may have unknown but constant
dynamics or the dynamics may be time-varying.
Many of the past algorithms and approaches in adaptive control have often been
somewhat ad hoc, lacking good systematic methods. They used and applied
methods gathered from a wide range of areas including nonlinear system theory,
stability theory, singular perturbations and averaging theory, stochastic control
theory, parameter estimation theory, and optimisation theory. Nevertheless, useful
adaptive control techniques are beginning to emerge after a long period of research
and experimentation. There is still much more work required on stability issues but
some important theoretical results have already been established. The field is now
sufficiently mature to have a number of adaptive regulator products appearing in
the market place.
In adaptive systems design it is desirable to find the simplest possible parameter
adjustment rules. However, these rules must generally be nonlinear rules. There are
two main methods for adaptive control depending on the parameter adjustment
rules, the direct and indirect methods. The direct methods have adjustment rules
that tell how the regulator parameters should be updated. Indirect methods, on the
other hand, update the process parameters and then the regulator parameters are
obtained from the solution of a design problem. One most important direct method
is the Model-Reference Adaptive System (MRAS) and one important indirect
method is the Self-Tuning Regulator (STR). Although different in detail these two
methods are closely related in principle.
268 Principles of Adaptive Filters and Self-learning Systems

11.1 Main Theoretical Issues

Because adaptive control systems are inherently nonlinear in their operation they
are complex and difficult to analyse. Because of this complexity it is necessary to
invoke various theories to achieve adequate design and analysis. These theories
include nonlinear systems, stability, recursive parameter estimation, system
identification, optimal control, and stochastic control theories.
Although it is possible to establish that a particular nonlinear system solution is
stable the same solution applied to other cases may not be stable. Since an adaptive
system is seeking and finding new solutions on an ongoing basis it is only in very
special cases that it is possible to speak of a “stable adaptive system” in a global
sense. Often the best that can be done is to find the stable equilibrium solutions and
then determining the local behaviours by linearization techniques.
A typical problem in adaptive control is to design a parameter adjustment rule
that is guaranteed to result in a stable closed loop system using a range of
theoretical considerations. There are two separate problems to consider. In a tuning
problem it is assumed that the process to be controlled is constant but with
unknown parameters. In an adaptation problem the parameters are changing. The
tuning problem is much easier to deal with since the parameter convergence has a
final endpoint, while the adaptation problem has not. The estimation algorithms for
tuning and adaptation are similar in form but they are applied differently. A
common parameter estimation algorithm form is defined by Equation 11.1.

θˆ(t + 1) = θˆ(t ) + P (t )ϕ (t )( y (t + 1) − ϕ T (t )θˆ(t )) (11.1)

where:
θˆ(t ) = estimate of the parameter vector at time t.
ϕ (t ) = vector of functions of measured signals in the system (regressors).
y (t + 1) = measurement signal at time (t+1).
P (t ) = gain matrix (also governed by a difference equation).

In the tuning case the gain matrix P(t) goes to zero as t increases, whereas in the
adaptation case it is not allowed to converge to zero.
An important theoretical consideration for parameter convergence is to establish
the conditions under which the recursive parameter estimation process will work.
The conditions must provide a persistent excitation or sufficient richness in the
input signal to ensure that the process dynamics can be captured.
Many adaptive algorithms rely on the fact that the parameters change more
slowly than the state variables of the system. When investigating the behaviour of
the states the parameters are seen as constants. Therefore they are often replaced
with their mean values. In Equation 11.1, this is the same as replacing the term
P(t )ϕ (t )( y (t + 1) − ϕ T (t )θˆ(t )) with its mean value. The rate of adaptation of the
parameters can be controlled by the selection of a gain constant. The averaging
method works best when a small adaptation gain is used. In many cases the
 Adaptive Control Systems 269

difference between the true and averaged equations is proportional to the

adaptation gain. It is believed that these types of averaging methods may eventually
lead to a unification of analysis of adaptive systems.
It is possible to consider a unified theoretical structure for adaptive systems by
using nonlinear stochastic theory in which the system and its environment are
described by a stochastic model. In this structure the parameters are introduced as
state variables and their uncertainty is modelled by stochastic models. According to
this model an unknown parameter constant can be modelled by the differential
equation dθ / dt = 0 or the equivalent difference equation θ (t + 1) = θ (t ) with an
initial probability distribution that models the parameter uncertainty. Parameter
drift is described by simply adding random variables to the right sides of these two
equations. Next, a rule is developed to minimise the expected value of a loss
function, which is made to be a scalar function of the states and controls.
It is difficult to find a control that minimises the expected loss function. If it can
be assumed that a solution exists it is possible to formulate the optimal loss
function by using dynamic programming. Dynamic programming involves solving a
functional equation called the Bellman equation. The optimal regulator formed
from stochastic control theory can be represented by Figure 11.1 (Åström and
Wittenmark 1995).

Hyperstate

Calculation of
Hyperstate

Estimator

uc Nonlinear Nonlinear Output y

Control Law u Process

Regulator

Figure 11.1. Adaptive Regulator

The controller is composed of a nonlinear estimator and a feedback regulator. The

conditional probability distribution of the state (the hyperstate) is generated from
the measurements by the estimator. The nonlinear feedback regulator maps the
hyperstate into the space of control variables. The regulator’s nonlinear function
can be computed off-line because it changes more slowly, whereas the hyperstate
must be up-dated on-line. The hyperstate is usually a high dimensional quantity,
which allows a structurally simple control solution. However the disadvantage is
that updating of the hyperspace can require the solution of a complex nonlinear
filtering problem. The advantage of this approach is that there is no distinction
between parameters and the other state variables, which means that the regulator
can handle very rapid parameter changes. The control attempts to drive the output
to its desired value and it also introduces probing perturbations when the
parameters are uncertain. Thus, the optimal control automatically gives a good
270 Principles of Adaptive Filters and Self-learning Systems

balance between maintaining adequate control and small control errors. This
control property is called dual control, which inherently improves the quality of the
estimates and the future controls.
Most work on stability has been done in relation to the MRAS. A typical system
may be composed of a linear system with a nonlinear feedback because this is a
classical configuration for which stability results are available. Here, it can be said
that the closed loop system is stable if the linear part is strictly positive real and the
nonlinear part is passive.

11.2 Introduction to Model-reference Adaptive Systems

The MRAS is one of the main adaptive control approaches. When the system
specifications are given in terms of a reference model that tells how the process
output should ideally respond to command signals it is then possible to use the
MRAS. Figure 11.2 shows the block diagram of Whitaker’s original MRAS
(Whitaker et al 1958). The original MRAS was derived for the servo problem in
deterministically continuous-time systems. Since then the theory has been extended
to discrete-time systems and systems with stochastic disturbances.

Model

y m is the Model Output

Adjustments
Outer Control Loop

Regulator
Parameters

uc
Regulator Plant Output y
u

Inner Control Loop

Figure 11.2. Model-Reference Adaptive System

The MRAS regulator has an inner and outer loop. The inner loop is an ordinary
feedback loop composed of the process and the regulator. The outer loop is also a
regulator loop, it adjusts the parameters in such a way as to make the error e
between the process output y and the model output ym small. It is a nontrivial
problem to formulate a stable adjustment mechanism for the MRAS that is
guaranteed to reduce the error to zero.
There are three basic approaches to the analysis and design of a MRAS. These
are the gradient method, Lyapunov functions and passivity theory. Whitaker’s