454
3, MARCH 2007
Numerical Optimization-Based Extremum Seeking
Control With Application to ABS Design
Chunlei Zhang, Member, IEEE, and Raúl Ordóñez, Member, IEEE
Abstract—Extremum seeking control (ESC) schemes based on
numerical optimization are proposed in this paper. The extremum
seeking problem is treated as an optimization with dynamic system
constraints. The numerical optimization-based extremum seeking
control scheme is first applied to lineat time-invariant (LTI) sys-
tems, then it is extended to a class of feedback linearizable systems.
The convergence of the ESC scheme is guaranteed by the numer-
ical optimization algorithm and state regulation. The robustness of
line search methods and trust region methods is studied, which pro-
vides further flexibility for the design of robust extremum seeking
controller. Simulation study of antilock braking systems (ABS) de-
sign via extremum seeking control is addressed.
Index Terms—Antilock braking systems (ABS), extremum
seeking, numerical optimization, state regulation.
I. INTRODUCTION
RADITIONAL automatic control deals with the problem
T of stabilization of a known reference trajectory or set point,
that is, so called “tracking” and “regulation” problem. The ref-
erence is often easily determined. However, in some occasions
it can be very difficult to find a suitable reference value. For in-
stance, the friction force coefficient of the wheel in a car depends
on the slip. In order to maximize the friction force, it is neces-
sary to apply a suitable braking torque to maintain the optimal
slip, which changes as different road conditions and weather.
Tracking a varying maximum or minimum (extremum) of a
performance (output, cost) function is called extremum seeking
control [1], which has two layers of meaning: first, we need to
seek an extremum of the performance function; second, we need
to be able to control (stabilize) the system and drive the per-
formance output to that extremum. The goal of the extremum
seeking control is to operate at a set point that optimizes the
performance function.
Extremum seeking control is related to optimization; many
of the ideas used in extremum seeking control have been
transferred from numerical optimization. Developments in
computers and optimization have led to a renewed interest in
extremum seeking control. Many investigations of extremum
seeking control systems assume that the system is static, which
can be justified if the time between the changes in the optimal
Manuscript received November 7, 2005; revised June 23, 2006. Recom-
mended by Associate Editor A. Hansson. This work was supported by the
Dayton Area Graduate Studies Institute (DAGSI).
C. Zhang is with the Etch Engineering Technology, Applied Materials, Sun-
nyvale, CA 94085 USA (e-mail: Chunlei_Zhang@amat.com).
R. Ordóñez is with the Department of Electrical and Computer Engineering,
University of Dayton, Dayton, OH 45469 USA (e-mail: ordonez@ieee.org).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TAC.2007.892389
0018-9286/$25.00 © 2007 IEEE
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO.
reference is sufficiently long. Then, the extremum seeking is
reduced to an optimization problem in some sense. A mathe-
matical interpretation of the process is where
is a static output function, is a vector of parameters that may
change with time. The goal to optimize the output function
is essentially a numerical optimization problem given only
the knowledge of the output measurements and its derivatives
if available. The parameters can be the parameters of the
controller or the controller itself. Thus the extremum seeking
control of static systems is generally solved in two ways, one
by tuning the parameters of the feedback controller, the other
by tuning the control input directly [2]. A few references (for
instance, see [3] and [4]) have approached problems where the
plant is a cascade of a nonlinear static map and a linear dynamic
system (the so-called Hammerstein and Wiener model).
The first rigorous proof [5] of local stability of perturbation-
based extremum seeking control scheme uses averaging anal-
ysis and singular perturbation, where a high-pass filter and slow
perturbation signal are employed to derive the gradient infor-
mation. Reference [6] presents a systematic description of the
perturbation-based extremum seeking control and its applica-
tions. New progress in semiglobal stability appears in [7]. Gra-
dient estimation-based extremum seeking control is studied in
[8], nonlinear programming-based methods can be found in [9].
Extremum seeking control based on sliding mode are studied in
[10]–[12], where time delay, excessive oscillation and perfor-
mance improvement issues are addressed. Extremum seeking
via continuous time nonderivative optimizers is proposed in [4].
An extremum seeking control problem is proposed and solved
in [13] for a class of nonlinear systems with unknown parame-
ters, where an explicit structure information for the performance
function to be maximized is required. Moreover, many applica-
tions of extremum seeking have been studied recently, such as
in optimizing bioreactor [14], combustion instability [15], an-
tilock braking systems (ABS) design [16]–[18], electromechan-
ical valve actuator [19], thermoacoustic cooler [20], and human
exercise machine [21].
In this paper, we combine numerical optimization and state
regulator to form an extremum seeking control scheme, where a
numerical optimization algorithm provides search candidates of
the unknown extremum and a state regulator is designed to reg-
ulate the state to where the search sequence leads to. A general
problem statement is given in Section II. Two main types of nu-
merical optimization algorithms, line search methods and trust
region methods, are reviewed in Section III. In Section IV, a nu-
merical optimization-based extremum seeking control scheme
is proposed first to the LTI system, then it is extended to a
class of feedback linearizable systems, where the convergence
of proposed schemes is analyzed. Then, the robustness issues
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN
Fig. 1. Numerical optimization-based extremum seeking control.
are studied in Section V. Application to ABS design is consid-
ered in Section VI and finally some conclusions can be found in
Section VII.
II. PROBLEM STATEMENT
In general, we consider the nonlinear model
where is the state, is the input, is the
performance output, and and
sufficiently smooth functions on . Without loss of generality,
we consider the minimization of the performance function (2).
For simplicity, we assume in this paper. The explicit
form of is unknown and hence not available for the design.
The goal of extremum seeking control is to design a controller
based on the output measurements and state measurements to
regulate the state to an unknown minimum of , and there-
fore minimize the performance output. Unlike optimal control,
we do not suppose the knowledge of the performance function
(2), therefore we cannot explicitly find the solution of the op-
timal condition and design an open-loop controller to operate
the system at the optimal set point.
A block diagram of numerical optimization-based extremum
seeking control can be found in Fig. 1, where the nonlinear
system is modeled as (1) and the performance function is (2).
Based on the measurements of the state, function values
or gradient , the extremum seeking loop is expected to
regulate the state as guided by the search sequence , and
eventually minimizes the performance output. The following as-
sumption is generally needed for the fulfillment of the extremum
seeking purpose:
Assumption 2.1 (Existence of the Minimum): The perfor-
mance function is continuous on the compact level sets
for all in .
Assumption 2.1 guarantees the existence of the minimum,
and any numerical optimization algorithms with first-order
global convergence property will produce a sequence
converging to a minimizer (strictly speaking, first-order sta-
tionary point) of the performance function. Moreover, from
the perspective of optimization, extremum seeking control can
be considered as a kind of constrained optimization problem,
455
whose constraint is the differential equation (1) as compared
to the traditional algebraic constraints, and the manipulation
of has to be done indirectly through the control input . The
extremum seeking control problem then can be stated as:
subject to
Now the state is feasible if it is a solution of the dynamic
system. In the case when (1) is controllable, there always exists
an input that transfers to any where in in a finite time.
Although controllable dynamic system constraints do allow to
be anywhere in the state space where the numerical optimizer
wants, the way in which reaches the particular place is de-
termined by the dynamic system and the state regulator to be
designed.
III. UNCONSTRAINED OPTIMIZATION
Two main methods for unconstrained optimization problem
are reviewed in this section.
A. Line Search Methods
(1) Each iteration of a line search method computes a search di-
rection and then decides how far to move along that
(2)
direction. The iteration is given by
are (3)
where the positive scalar is called the step length and requires
to be a descent direction (one for which ),
because this property guarantees that the function can be re-
duced along this direction. The steepest descent direction
is the most obvious choice for search direction.
Given a descent direction , we face a tradeoff in choosing
step length that gives a substantial reduction of and not
spending too much time making the choice. It can be found
by approximately solving the following one-dimensional mini-
mization problem:
(4)
An exact line search to find the optimal step length needs
the minimization of , which is expensive and sometimes
,
unnecessary. More practical strategies perform an inexact line
search to identify a step length that achieves adequate reduction
of with a small computational cost. In particular, the Armijo
condition
(5)
prevents steps that are too long via a sufficient decrease crite-
rion, while the Wolfe condition
(6)
prevents steps that are too short via a curvature criterion, with
. The restriction ensures that ac-
ceptable points exist. Moreover, in order to avoid a poor choice
of descent directions, an angle condition is set up to enforce
456 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007

a uniform lower bound on the angle between vector and model, the trial point is accepted as the new iterate and the trust
, that is region is centered at the new point and possibly enlarged. On
the other hand, if the achieved reduction is poor compared with
(7) the predicted one, the current iterate is typically unchanged and
trust region is reduced. This process is then repeated until con-
vergence occurs. Define the ratio
where is independent of . The following is the first-order
global convergence1 result for line search methods [22], [23].
Theorem 3.1: Let be continuously differentiable (9)
on and be bounded below. Suppose that is Lipschitz
continuous with constant ; that is, The following algorithm [22] describes the process.
for all . If the sequence satisfies conditions Trust Region Algorithm
(5), (6) and (7), then Step 0 Given , initialize the trust region
, and , set .
Step 1 Approximately solve the trust region problem (8) to
obtain .
The following lemma will be used in the robustness analysis Step 2 Evaluate from (9).
for line search methods. Step 3 If ; if
Lemma 3.2: Let be continuously differentiable and ; else
on . Suppose that is Lipschitz continuous with constant .
. Let be the step length and descent direction, then Step 4 If , else . Set
. Go to step 1.
Quadratic approximations of are often used for constructing
. In this case, in (8) can be formed as

where for exact line search, and for in- (10)

exact line search satisfying conditions (5) and (6), represents
the angle between vector and . The vector
Proof: See Appendix I.
Since is required to ensure the feasibility
of inexact line search, we will have
This observation is consistent for the upper bound results in the
previous lemma. That is, we always expect that the exact line
search achieves more decrease along the search direction than
the inexact line search.
B. Trust Region Methods
At each iteration of a trust region method, we consider the
minimization of a model function instead of the objective
function at current iterate . Because the model function may
not be a good approximation of when is far away from ,
we have to restrict the search for a minimizer of to a local
region involving . Such a region is called a trust region. A
trust region method is defined as
is either the gradient or an approxi-
mation of it, and the matrix is either the Hessian matrix
or an approximation of it. Thus, such construction of
. still requires gradient information. However, the trust re-
gion framework provides large flexibility in designing deriva-
tive free optimization methods. This compares very favorable
with most line search methods which do require gradient mea-
surements of the objective function. Derivative free trust region
algorithms proposed in [24]–[26] use multivariate interpolation
to construct the model function , where only an interpola-
tion set containing the interpolating nodes and their objec-
tive function values are needed. Overall, trust-region methods
retain the quadratic convergence rate while being globally con-
vergent. The following is a global convergence result for trust
region methods [22].
Theorem 3.3: Let be Lipschitz continuously dif-
ferentiable and bounded below on level set .
Further, let in the trust region algorithm. Suppose that
for some constant , and that all approximate solu-
tions of (8) satisfy the inequality
(8)

Let be the minimizer obtained. The current iterate is then

updated to be , if the achieved objective function reduc-
tion is sufficiently compared with the reduction predicted by the
1In the optimization community, first-order convergence of an optimization
for some constant , and for some con-
algorithm means that one (or some, or all) of the limit points of the iterate
sequence is a stationary point of J (x). Global convergence is used to mean stant . Then
first-order convergence from an arbitrary starting point. In contrast, local con-
vergence is used to mean convergence when the initial point is close enough to
a stationary point.
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN 457

IV. EXTREMUM SEEKING CONTROL needed every time, therefore, we can estimate the gradient
by collecting enough measurements of during the time.
A. ESC of LTI Systems Remark 4.4: The stopping criterion is
Here, we consider a single-input–single-output (SISO) linear used only for simplicity, where is a predefined small positive
time-invariant (LTI) system constant. In case gradient information is not available, there are
other stopping criteria only based on the difference of function
values [22].
(11) Now, we present the convergence result of the extremum
seeking control scheme that follows.
with the performance function defined in (2), where is Theorem 4.5: Consider the LTI system (11) with perfor-
the state, is the input. The matrices are given as a mance output (2), suppose the LTI system (11) satisfies 4.1 and
model of a real system. However, the explicit form of the per- the performance function (2) satisfies Assumption 2.1; more-
formance function and its minimum are not known, where over, if the extremum seeking control scheme above is applied,
we can only measure the function value or its derivatives. We where the optimization algorithm used is of first-order global
need the following assumption to ensure the feasibility of ex- convergence, then the state will globally asymptotically
tremum seeking control for the LTI system (11). converge to the first-order stationary point of the performance
Assumption 4.1: The LTI system (11) is controllable. function (2).
Now, we can combine an optimization algorithm and a state reg- Proof: See Appendix II.
ulator originated from the controllability proof in [27] to form Remark 4.6: The convergence result for the extremum
an extremum seeking control scheme. seeking control scheme is global since the numerical optimiza-
ESC Scheme for LTI Systems tion algorithm used is of first-order global convergence.
Step 0 Given , set and . Remark 4.7: Additional assumptions about the performance
Step 1 Use an optimization algorithm with first-order function may be required to guarantee an arbitrary optimiza-
global convergence to produce based on current tion algorithm with first-order global convergence indeed con-
state , the measurement of or . verges to the stationary point of (2). For example, according to
Denote Theorem 3.1, we need assume is continuous differentiable and
is Lipschitz continuous for line search methods; also
we will assume is Lipschitz continuously differentiable and
bounded below on level set when using trust region methods.
Step 2 Choose a regulation time , let , Remark 4.8: The design of controller (12) is not limited to
and design the control input during to be single input system and just one way to fulfill the state regula-
tion task. It is an open loop controller during the time without
considering the change of , which is fed back from the opti-
(12) mization algorithm. Such design has the advantage of achieving
where regulation in a finite time, but it relies on the precise knowledge
of and matrices, and is very difficult to robustify. Later on,
(13) we can relax the state regulation design criterion from perfect
regulation to regulation within certain neighborhood of ,
which provides further flexibility of using other designs of state
Step 3 If , then stop. Otherwise, set regulator to deal with input disturbance or unmodeled plant dy-
. Go to Step 1. namics.
Remark 4.2: The above extremum seeking control scheme Remark 4.9: The only requirement for the feasibility of state
can be derivative free if the optimization algorithm used in regulator (12) is that the LTI system is controllable ( of
Step 1 does not require gradient information. For example, (13) is nonsingular), which means the extremum seeking control
we can use derivative free trust region methods [24], [26] or scheme works for both stable and unstable systems. However,
direct search [23]. Some modifications of the above scheme we would like to first stabilize the unstable system by pole place-
are required in order to use the trust region methods due to the ment, then design a state regulator on the stabilized LTI system
need to obtain the ratio in (9), where we may need additional since the unstable LTI system will amplify the regulation error
regulation time to drive the state back to if . Please resulting from an input disturbance, for example.
refer to [28] for the details of trust region-based extremum Remark 4.10: The performance of the extremum seeking
seeking control. control design largely depends on the performance function to
Remark 4.3: If steepest descent method is used in Step 1, we be optimized, the optimization algorithm used, and a robust
will have and efficient state regulator.
Remark 4.11: If the performance function (2) is differentiable
Even though it requires gradient measurement and convex, then the convergence to stationary point becomes
by using steepest descent method, such measurement is only convergence to the global minimum [22].
458 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007

The banana function (15) has its minimizer at , and

. The explicit form of the function and its minimum
are both unknown to the designer. Extremum seeking control
scheme based on steepest descent algorithm with inexact line
search is applied, the simulation results with the first 15 steps
are shown in Fig. 2, where . Hence, we
only require the gradient measurement at every time to
continue the optimization algorithm. The performance output
[Fig. 2(a)] approaches its minimum at and the
state [Fig. 2(b)] accordingly converges to the minimizer .
The steepest descent algorithm produces a sequence as
a guideline for the controller. The trajectory between and
is shaped by the dynamical system (11) and the regulator
(12). This can be viewed clearly in Fig. 2(d),2 where the blue
circle represents the and the red dashed line represents the
state trajectory. The choice of is rather heuristic in this ex-
ample. However, in practice this is an important design factor.
We can see that a smaller yields a larger control force to sat-
isfy the short regulation time. Thus, should be chosen such
that the control force does not exceed the practical limits. There
is always a tradeoff between the extremum seeking time and the
control gain.

B. ESC of State Feedback Linearizable Systems

Now, we consider a SISO nonlinear affine system

(16)

with the performance function defined in (2), where is

the state, is the input, are smooth func-
tions on . We have the following assumption for the nonlinear
affine system.
Assumption 4.12: The nonlinear affine system is state feed-
back linearizable on the domain .
Upon this assumption, we can always put the system in the
controllable canonical form [29]. That is, there exists a diffeo-
morphism such that contains the
origin and the change of variables transforms the
system (16) into the form

(17)

with controllable and nonsingular for all .

Then, we can easily extend the results on LTI systems to outline
the extremum seeking control scheme for the state feedback lin-
earizable systems (16), where we still assume for sim-
plicity.
ESC Scheme for State Feedback Linearizable Systems
Fig. 2. Extremum seeking control for a LTI system. (a) Performance output.
(b) State. (c) Control input. (d) Phase portrait, steepest descent sequence fx g Step 0 Given , set and .
over the contour of the performance function. Step 1 .
Step 2 .
Step 3 Choose a regulation time , let ,
Now, consider a second-order stable LTI system in its con-
and design the control input during to be
trollable canonical form. Let ,
(18)
(14) 2Since it will take thousands of steps for the steepest descent algorithm to
converge to the minimizer of the banana function, we only simulate the first 15
(15) steps for illustration purpose.
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN
where
(19)
Step 4 If , then stop. Otherwise, set
. Go to step 1.
Theorem 4.13: Consider the nonlinear affine system (16)
with performance output (2), suppose the system (16) satisfies
Assumption 4.12 and the performance function (2) satisfies
Assumption 2.1; moreover, if the extremum seeking control
scheme above is applied, where the optimization algorithm
used is of first-order global convergence, then the state will
globally asymptotically converge to the first-order stationary
point of the performance function (2).
The proof mainly follows the proof of Theorem 4.5. The fea-
sibility of the controller defined in (18) is guaranteed by As-
sumption 4.12. Similar remarks like 4.2–4.11 also apply here.
C. ESC of Input-Output Feedback Linearizable Systems
We can extend the previous results on state feedback lin-
earizable systems to input-output feedback linearizable systems
given some minor modifications. We will have the following as-
sumption.
Assumption 4.14: The nonlinear affine system (16) is
input–output feedback linearizable from input to output on
. We define
(20)
where is sufficiently smooth in the domain . The
motivation of defining a new output is to retain the claim that
in the extremum seeking control we do not have the knowledge
of the performance function, while we do need the knowledge
of some suitable output to perform input–output lin-
earization. Let be the relative degree of the nonlinear system
(20), then for every , a neighborhood of and a
diffeomorphism exists such that the change of
variables , transforms the system (20) into
the form
(21)
(22)
(23)
where is controllable and is
nonsingular in . Since is uncontrollable, therefore, in order
to fulfill the extremum seeking of the performance function, two
more assumptions are proposed.
Assumption 4.15: The performance function is not dependent
on the state of the internal dynamics.
Assumption 4.16: The state of the internal dynamics (21)
will be bounded given bounded and any initial state .
The Assumption 4.15 puts the performance function as
459
It is a reasonable assumption since we have to be able to con-
trol the variables of the performance function to achieve the ex-
tremum seeking. Moreover, Assumption 4.16 simply means that
the internal dynamics are well behaved such that the linearizing
controller will not grow unbounded due to the existence of the
uncontrollable state . Note that assuming input-to-state sta-
bility of the internal dynamics is stronger than Assumption 4.16.
However, simply assuming the zero dynamics are
asymptotically stable is also not enough since the state may
grow unbounded given bounded input . Thus, the same anal-
ysis for the state feedback linearizable systems holds here given
the extremum seeking scheme in Section IV-B with minor mod-
ifications, where we replace with . The following theorem is
a straightforward extension of Theorem 4.13.
Theorem 4.17: Consider the nonlinear affine system (16)
with performance output (2), suppose the system (16) satisfies
Assumptions 4.14 and 4.16, and the performance function (2)
satisfies Assumptions 2.1 and 4.15; moreover, if the extremum
seeking control scheme of Section IV-B is applied with
replaced by , then the state will globally asymptotically
converge to the first-order stationary point of the performance
function (2).
The proof mainly follows the proof of Theorem 4.5. The fea-
sibility of the controller defined in (18) is guaranteed by As-
sumptions 4.14 and 4.16. Similar remarks like 4.2–4.11 also
apply here.
V. ROBUSTNESS ISSUES
The main restriction of Theorems 4.5, 4.13, and 4.17 is the
requirement of perfect state regulation to guarantee the conver-
gence. That is, at each iteration, the controller needs to regulate
the state precisely to the desired set point , which is offered
by the iterative optimization algorithm based on the current state
. In practical applications, noisy output or state measure-
ments, input disturbance, saturation and time delay, unmodeled
plant dynamics and computational error will be detrimental to
the theoretical result. In fact, we are only able to regulate the
state to the neighborhood of the set point .
For example, let us consider an LTI system with input dis-
turbance. Let , where is given as
in (12), then at time , the state and
, then at time , we will have
where . However, the error
will not accumulate since the optimization algorithm
will generate the next destination based on current state
. That is, by including the numerical optimizer in
the extremum seeking loop, it offers a feedback mecha-
nism to robustify the extremum seeking scheme. Denote
, then by induction,
460
we will have the controller interpolates precisely between the
sequence , where
will be bounded if is bounded and the input disturbance
is bounded. Moreover, for stable systems, we will have matrix
with negative eigenvalues and therefore the transition matrix has
exponentials with negative real parts. Then, will asymp-
totically converge to some constant as approaches infinity,
that is, even if is unbounded, we will still have a bounded
given bounded input disturbance. Moreover, the more neg-
ative the eigenvalues are, the smaller will be. On the con-
trary, for unstable systems, the transition matrix will amplify
even a small input disturbance , and will grow as
increases. Therefore, we wish to have a stable LTI system and a
short regulation time. The latter amounts to a need of high gain
controller to deal with disturbance. Consider extremum seeking
control of an unstable but still controllable LTI system, we will
perform pole placement to transform the unstable LTI system to
a desired stable LTI system, and then design the state regulator
on the stabilized LTI system.
Similarly, for state feedback linearizable systems given the
knowledge of which are in the controllable canonical form,
the controller will be implementing functions and , Proof: See Appendix IV.
approximations of , and , that is
where is defined in (19). Then the closed-loop systems be-
comes
where . search.
Then, the imperfect modeling is equivalent to having input dis-
turbance in the resulting linear system, therefore we need to deal
with imperfect regulation as well. Overall, we will hope that a
well designed optimization algorithm will convey its robustness
to the extremum seeking scheme. That is, if the new sequence
is able to converge to the minimum or its small neighbor-
hood given the error is bounded, then the extremum seeking
control schemes in Section IV will converge to the minimum or
its small neighborhood as well. In the following, we will present
the robustness analysis of two types of optimization algorithms.
A. Robustness of Line Search Methods
Theorem 5.1: Let be continuously differentiable
on and be bounded below. Suppose that is Lipschitz
continuous with constant . A line search method starting from
is used but with bounded error at each iteration, i.e.,
and . The new sequence
is a descent sequence, that is
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007
given
(24)
where for exact line search and for
inexact line search satisfying conditions (5) and (6).
Proof: See Appendix III.
Although the bound (24) is very conservative, it can give us
some insights into the robustness of line search methods. First,
the exact line search allows a larger error bound than the inexact
line search. Second, we can see that the bound is an increasing
function of . That is, when is far away from the
minimizer of the performance function, we will expect the gra-
dient to be large and thus the error the algorithm can tolerate
is also large. This observation implies that line search methods
will be very robust until the gradient converges to some invariant
set, which is illustrated in the following corollary.
Corollary 5.2: Let be continuously differ-
entiable on and be bounded below. Suppose is Lips-
chitz continuous with constant . A steepest descent algorithm
is used with bounded error at each iteration. If assuming
for some constant , then we will have the gradient of the
sequence converges to the invariant set satisfying
(25)
where for exact line search and for
inexact line search satisfying conditions (5) and (6).
Observed from inequality (25), a diminishing step
length is preferred later on to decrease the bound of
the invariant set. As , the bound converges to
. This is again coincident
with theory of numerical optimization, where generally a di-
minishing step length is required for the algorithms to converge
to a minimum. And if there is no error between and , we
will see that the gradient converges to zero. Moreover, exact
line search can achieve a smaller bound than the inexact line
Now, we continue the simulation in Section IV-A on the LTI
system (14) with performance function (15), a random distur-
bance uniformly distributed with amplitude 0.8 is introduced to
the input. The simulation results are shown in Fig. 3 with
. The controller (12) is not able to regulate
the state to the desired set point precisely. For example, at the
first step, the controller cannot transfer the state to the desired
destination , instead it arrives at
due to the input disturbance. Then,
, that is, the line search
method tries to amend the deviated path towards the minimum.
Again, the state only arrives at , there-
fore, eventually we will still have a descent sequence as
long as the error satisfies the bound (24). The comparison of
and can be seen in Fig. 3(d), where the blue circle
represents the , magenta square denotes the and the
red dashed line is the state trajectory. Interestingly, we find that
the disturbance actually helps the algorithm to achieve more re-
duction in function values in the first few steps by comparing
with the ideal case [see Figs. 2(d) and 3(d)]. However, as shown
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN 461

we can do given the current design of state regulator (12) with

input disturbance.

B. Robustness of Trust Region Method

For trust region methods, the ratio in (9) is required at each
step to determine . Let us call th step a successful step if
, and unsuccessful step if . However, in order to
obtain , we need the measurement of . Then, in
the implementation of extremum seeking control based on trust
region method, this means we need to regulate the state to
to obtain measurement of even if it is not
the next iterate . For example, at th step, the controller
needs to regulate the state to to obtain the ratio
. If , then it is a successful step and we are done. Oth-
erwise, if , then we need to regulate the state back to
since it is an unsuccessful step. However, for practical applica-
tions, the ideal ratio cannot be obtained due to the imperfect
regulation. The following theorem provides one possible quan-
titative analysis about the convergence of trust region methods
under certain bounded error.
Theorem 5.3: Let be Lipschitz continuously dif-
ferentiable and bounded below on level set .
Suppose that is Lipschitz continuous with constant . A
trust region method starting from is used but with
bounded error at each iteration, that is,
for a successful step or for an unsuccessful
step. If, for every successful step

(26)

and for every unsuccessful step

(27)

where

and , then there

exists a subsequence of converges to the first-order sta-
tionary point of .
Proof: See Appendix V.

C. Design of Robust Extremum Seeking Control Scheme

Fig. 3. Extremum seeking control for a LTI system with input disturbance. (a)
Performance output. (b) State. (c) Control input. (d) Phase portrait, steepest de-
scent sequence fx g over the contour of the performance function.
in Fig. 3(a), the performance output becomes diverging eventu-
ally since the state regulator design (12) with input disturbance
cannot regulate the state even into the desired neighborhood (24)
of as is very small. Therefore, in such cases, we
will prefer using (25) as the stopping criterion, which is the best
The importance of Theorems 5.1 and 5.3 is that they relax
the design criterion for the state regulator, where now no per-
fect state regulation to is needed. Robust design of state
regulator to deal with disturbance, unmodeled plant dynamics
via adaptive control [30] or sliding mode control can be made
easier by regulating state to a neighborhood of , which
will be presented in our future work. For extremum seeking con-
trol based on line search methods, as long as we can design a
regulator satisfying (24), then the extremum seeking will con-
tinue to decrease the performance output.
For extremum seeking control based on trust region methods,
at time , we do not know whether it will be a successful
step or not in advance since we do not have the knowledge of .
462 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007

Fig. 4. Design criterion for the robust regulator to satisfy (26) and (27) of the trust region-based extremum seeking control.

Then, one way to guarantee the convergence under input distur- optimal slip. Consider only one wheel model, where the tire
bance is the following. The state regulator for set point dynamics are described as in [16]
needs to be designed to satisfy (26). If it is a successful step,
then the design will guarantee the step still to be a successful (28)
step since (see (34) in Appendix V). If it is not a suc- (29)
cessful step, that is , then there is no guarantee that where is the linear velocity and is the angular velocity of
will be greater than . So if , then we are luck in the sense the wheel, the mass, the weight of the wheel,
that we are on the right way due to the disturbance. If not, then the radius of the wheel, the moment of inertia of the wheel,
we need to regulate the state back to , that means we need to the braking friction torque, the braking torque, is
design the regulator for to satisfy (27), which means we can the friction force coefficient and the wheel slip is defined as
return to the right path as inferred from previous steps. Both in-
equalities (26) and (27) define similar criteria for the design of (30)
state regulators. A pictorial illustration of such criteria in is
for . There exists a maximum for the friction force
shown in Fig. 4, where we let the set point, in (26) and
coefficient at the optimal slip , but will change
in (27), to be and its gradient to be , the Lip-
as the road conditions changes. Now, the purpose of the ABS
schitz constant . The blue star is the desired destination,
design is to generate a control input such that the friction force
and the dotted area depicts the acceptable region satisfying in-
coefficient is maximized.
equalities (26) and (27). The size of the region is proportional to
By observing (28), we find that is not controllable from
the gradient of the set point, and the reciprocal of the Lipschitz
given fixed. Fortunately, the friction force coefficient
constant.
is only dependent on , which is a function of and ,
and it may be controllable by . Therefore, we try to linearize
VI. ABS DESIGN VIA EXTREMUM SEEKING CONTROL the system from the input to output . We define the change
of variables that transforms the
In this section, we apply the extremum seeking control to system (28), (29) into the form
solve the ABS design problem [6]. It is well known that the ABS
is used to deal with braking on slippery surfaces. The friction (31)
force has a maximum for a low wheel slip and decreases as
the slip increases. Our goal is to design a control algorithm to
achieve maximal friction force without prior knowledge of the (32)
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN 463

Fig. 5. ABS design via numerical optimization-based extremum seeking control.

Since is the linear velocity, it will be bounded all the time due coefficient is only regulated to the neighborhood of the max-
to physical restrictions. Moreover, even though the knowledge imum since no robust controllers are used to guarantee the con-
of is not available, we are able to obtain the measurement vergence. In both cases, the wheel is stopped within the same
of from (28) given the linear acceleration is measurable time. Similarly, the delay introduced by hydraulic or pneumatic
via an accelerometer. Then, given , let the breaking line dynamics may result in regulation error as well and, there-
torque be fore, a robust term is needed in the state regulator to address
the delay problem, which is much easier than robustifying per-
(33) turbation-based or sliding mode-based extremum seeking de-
sign. Moreover, oscillation is successfully avoided compared
Then (32) becomes with those methods because no perturbation signal or sliding
mode function is introduced. However, ABS design via pertur-
bation-based or sliding mode-based extremum seeking control
are much easier to tune. Extremum seeking control based on line
where is the regulator designed according to (19) search methods achieves comparable performance.

VII. CONCLUSION
and is offered by the optimization algorithm used in the In this paper, we successfully incorporate numerical opti-
extremum seeking loop. A block diagram of extremum seeking mization algorithms into the set up of an extremum seeking
scheme for the wheel model can be found in Fig. 5. That is, by control scheme. The convergence of the proposed extremum
designing the control torque as in (33), we are able to adjust seeking control scheme is guaranteed if the optimization algo-
the slip to maximize the friction force coefficient. For simula- rithm is globally convergent and with appropriate state regula-
tion purpose, we postulate a simple function which qualitatively tion. We also analyze the robustness of line search methods and
matches as in [6] trust region methods, which relaxes the design requirement for
the state regulator and provides further flexibility in designing
the robust extremum seeking control scheme. Numerical exam-
ples are given to illustrate the analysis results. Furthermore, an
where and . The parameters of the wheel application of ABS design via extremum seeking control is used
are chosen as: kg, m. Initial to illustrate the feasibility of the proposed scheme.
conditions are m/s, , which The current setting of the proposed extremum seeking control
makes . The parameters of extremum seeking loop: scheme is one attractive way to use the numerical optimization
. The simulation results of extremum seeking for the purpose of real time optimization, as it retains the global
control based on trust region method are shown in Figs. 6 and convergence property of the numerical optimization algorithm.
7, the latter where the input is disturbed by a uniformly dis- It allows us to utilize the research results from the optimiza-
tributed noise with amplitude 2. Without disturbance, the ex- tion community. For example, when , it becomes a
tremum seeking scheme maximizes the friction force coeffi- constrained extremum seeking control problem, and generally
cient. Due to the existence of the disturbance, the friction force we need to first resort to constrained optimization algorithms
464
Fig. 6. ABS design via trust region-based extremum seeking control. (a) Fric-
tion force coefficient. (b) Slip. (c) Linear velocity and angular velocity. (d)
Braking torque.
and the state regulator design will be more challenging in en-
suring the state will not violate the constraints during the tran-
sient. In the future, the exploration of more robust numerical op-
timization algorithms and the design of robust state regulators
will be two ways to enhance the robustness of the extremum
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007
Fig. 7. ABS design via trust region-based extremum seeking control with input
disturbance. (a) Friction force coefficient. (b) Slip. (c) Linear velocity and an-
gular velocity. (d) Braking torque.
seeking control schemes. Moreover, the design of the regula-
tion time needs to be further studied to deal with various re-
quirements originated from practical applications. Current de-
sign also wastes a lot of information by setting the numerical
optimizer to work discretely. Similar to the perturbation-based
[6] or sliding mode-based [12] extremum seeking control, other
analog optimization algorithms are in need of attention.
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN 465

APPENDIX I By induction, at , we suppose the state ,

PROOF OF LEMMA 3.2 then we obtain . At time
We first present the following result. , we will have
Lemma 1.1 (Descent Lemma [31]): Let be
continuously differentiable on . Suppose that is Lipschitz
continuous with constant . Then, for
where is defined in (12). Thus, the controller (12) interpo-
lates between the precisely within finite time . There-
fore, the state of the system will globally asymptotically con-
We now proceed to establish Lemma 3.2. First, for exact line verge to the first-order stationary point of performance function
search, is the solution of (4). From descent Lemma 1.1, we (2).
have
valid for all . Letting ,
then
APPENDIX III
PROOF OF THEOREM 5.1
Now, for line search method at step , we have

Second, for inexact line search, satisfies conditions (5)

and (6). From the Lipschitz condition, we have

. Then, from (6), we have

.
That is, . Finally,
from (5)

where the second row is obtained via Descent Lemmas 1.1, and
the third row is achieved based on Lemma 3.2. Then, if we have
where .

APPENDIX II
PROOF OF THEOREM 4.5
First, given the performance function (2) satisfies Assump-
tion (2.1), an optimization algorithm with first-order global con- we can obtain .
vergence will produce a search sequence that globally
asymptotically converges to the first-order stationary point of
the performance function (2). APPENDIX IV
By assuming the LTI system (11) is controllable, the con- PROOF OF COROLLARY 5.2
troller (12) is feasible since is nonsingular. First, at
, the state , we have . We have for steepest descent algorithm, and from
Then, at time , we will have inequality (24), as long as

where the control input during is we will always have . So we can find a con-
servative bound on given the error bound .
For steepest descent method, we have
. Then, from Lipschitz condition
466 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 52, NO. 3, MARCH 2007

Since is always positive, we need

Now, the bound can be found via

Now, define to be the angle between and
, that is
. Then

Case II, for an unsuccessful step, , then we do not

need to care about the regulation error to the set point .
Thus, we will have the gradient of the sequence converges If the resulting , then it regulates the state to a place
to the invariant set satisfying with lower performance function value, which actually helps
the optimization process. If , then we need to regulate
the state back to . Here, we redefine the regulation error
as an error from this second state regulation, i.e.,
. Therefore, we wish to be a successful step
compared with the previous one , i.e.,
APPENDIX V
PROOF OF THEOREM 5.3
At iteration and , we obtain approximately solving
the trust region subproblem (8). And ideally we would have the
regulator such as (12) to drive the state from to
to obtain an ideal ratio (9). However, due to the input
disturbance or model uncertainty, such controller can only reg-
ulate the state to and end up with a practical
ratio

(34)

Let be a subsequence of
such that represents the index set of successful steps Now, define to be the angle between and , that
up to , that is and is, . Then

(35)

which means is a successful step if started from . Two

cases need to be analyzed to guarantee the global convergence
of the sequence . Thus, if (26) and (27) are satisfied, we will have a descent sub-
Case I, for a successful step, , then we want to sequence of globally converges to the first-order
guarantee a successful step in the presence of , that is we stationary point of .
wish
ACKNOWLEDGMENT
The authors would like Prof. R. Liu and Prof. M. Maggiore
for their valuable suggestions.

REFERENCES
[1] K. J. Åström and B. Wittenmark, Adaptive Control. Boston, MA:
Wiley, 1994.
[2] Y. Zhang, “Stability and performance tradeoff with discrete time trian-
gular search minimum seeking,” in Proc. Amer. Control Conf., 2000,
vol. 1, pp. 423–427.
ZHANG AND ORDÓÑEZ: NUMERICAL OPTIMIZATION-BASED EXTREMUM SEEKING CONTROL WITH APPLICATION TO ABS DESIGN
[3] J.-Y. Choi, M. Krstić, K. B. Ariyur, and J. S. Lee, “Extremum seeking
control for discrete-time systems,” IEEE Trans. Autom. Control, vol.
47, no. 2, pp. 318–323, Feb. 2002.
[4] Nusawardhana and S. H. Żak, “Extremum seeking using analog non-
derivative optimizers,” in Proc. Amer. Control Conf., 2003, vol. 4, pp.
3242–3247.
[5] M. Krstić and H.-H. Wang, “Design and stability analysis of extremum
seeking feedback for general nonlinear systems,” Automatica, vol. 36,
no. 2, pp. 595–601, 2000.
[6] K. B. Ariyur and M. Krstić, Real-Time Optimization by Extremum-
Seeking Control. Hoboken, NJ: Wiley-Interscience, 2003.
[7] Y. Tan, D. Nesic, and I. Mareels, “On non-local stability properties of
extremum seeking control,” in Proc. 16th IFAC World Congr., 2005,
pp. 1–6.
[8] R. N. Banavar, D. F. Chichka, and J. L. Speyer, “Convergence and
synthesis issues in extremum seeking control,” in Proc. Amer. Control
Conf., 2000, vol. 1, pp. 438–443.
[9] A. R. Teel and D. Popović, “Solving smooth and nonsmooth multivari-
able extremum seeking problems by the methods of nonlinear program-
ming,” in Proc. Amer. Control Conf., 2001, vol. 3, pp. 2394–2399.
[10] H. Yu and Ü. Özgüner, “Extremum-seeking control via sliding mode
with periodic search signals,” in Proc. 41st Conf. Decision Control,
2002, vol. 1, pp. 323–328.
[11] H. Yu and Ü. Özgüner, “Smooth extremum-seeking control via second
order sliding mode,” in Proc. Amer. Control Conf., 2003, vol. 4, pp.
3248–3253.
[12] Y. Pan, Ü. Özgüner, and T. Acarman, “Stability and performance im-
provement of extremum seeking control with sliding mode,” Int. J.
Control, vol. 76, no. 9/10, pp. 968–985, 2003.
[13] M. Guay and T. Zhang, “Adaptive extremum seeking control of non-
linear dynamic systems with parametric uncertainties,” Automatica,
vol. 39, no. 7, pp. 1283–1293, 2003.
[14] M. Guay, D. Dochain, and M. Perrier, “Adaptive extremum seeking
control of continuous stirred tank bioreactors with unknown growth
kinetics,” Automatica, vol. 40, no. 5, pp. 881–888, 2004.
[15] A. Banaszuk, K. B. Ariyur, M. Krstić, and C. A. Jacobson, “An adaptive
algorithm for control of combustion instabilityand application to com-
pressor instability control,” Automatica, vol. 40, no. 11, pp. 1965–1972,
2004.
[16] S. Drakunov, Ü. Özgüner, P. Dix, and B. Ashrafi, “Abs control
using optimum search via sliding modes,” IEEE Trans. Control Syst.
Technol., vol. 3, no. 1, pp. 79–85, Jan. 1995.
[17] H. Yu and Ü. Özgüner, “Extremum-seeking control strategy for ABS
system with time delay,” in Proc. Amer. Control Conf., 2002, vol. 5,
pp. 3753–3758.
[18] İ. Tunay, “Antiskid control for aircraft via extremum-seeking,” in Proc.
Amer. Control Conf., 2001, vol. 2, pp. 665–670.
[19] K. S. Peterson and A. G. Stefanopoulou, “Extremum seeking control
for soft landing of an electromechanical valve actuator,” Automatica,
vol. 40, no. 6, pp. 1063–1069, 2004.
[20] Y. Li, M. A. Rotea, G. T.-C. Chiu, L. G. Mongeau, and I.-S. Paek, “Ex-
etremum seeking control of tunable thermoacoustic cooler,” in Proc.
Amer. Control Conf., 2004, pp. 2033–2038.
[21] X. T. Zhang, D. M. Dawson, W. E. Dixon, and B. Xian, “Extremum
seeking nonlinear controllers for a human exercise machine,” in Proc.
43rd Conf. Decision Control, 2004, pp. 3950–3955.
[22] J. Nocedal and S. Wright, Numerical Optimization. New York:
Springer-Verlag, 1999.
[23] T. G. Kolda, R. M. Lewis, and V. Torczon, “Optimization by direct
search: New perspectives on some classical and modern methods,”
SIAM Rev., vol. 45, no. 3, pp. 385–482, 2003.
467
[24] A. R. Conn, K. Scheinberg, and P. L. Toint, “Recent progress in uncon-
strained nonlinear optimization without derivatives,” Math. Program.,
vol. 79, pp. 397–414, 1997.
[25] A. R. Conn, K. Scheinberg, and P. L. Toint, “On the convergence
of derivative-free methods for unconstrained optimization,” Approx.
Theory Optim.: Tributes to M. J. D. Powell, pp. 83–108, 1997.
[26] M. J. D. Powell, Uobyqa: Unconstrained Optimization by Quadratic
Approximation Univ. Cambridge, Cambridge, U.K., Tech. Rep.
NA2000/14, DAMTP, 2000.
[27] C.-T. Chen, Linear System Theory and Design. New York: Oxford
Univ. Press, 1999.
[28] C. Zhang and R. Ordóñez, “Non-gradient extremum seeking control of
feedback linearizable systems with application to abs design,” in Proc.
Conf. Decision Control, 2006, pp. 6666–6671.
[29] H. K. Khalil, Nonlinear Systems. Upper Saddle River, NJ: Prentice-
Hall, 2001.
[30] J. T. Spooner, M. Maggiore, R. Ordóñez, and K. M. Passino, Stable
Adaptive Control and Estimation for Nonlinear Systems, Neural and
Fuzzy Approximator Techniques. New York: Wiley, 2002.
[31] D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena
Scientific, 1995.
Chunlei Zhang (M’01) received the B.S. degree
from the Nanjing University of Science and Tech-
nology, Nanjing, China, in 1999, the M.S. degree in
electrical engineering and the M.S. degree in applied
mathematics from the University of Dayton, Dayton,
OH, in 2003 and 2005, respectively, and the Ph.D.
degree in electrical engineering from the University
of Dayton, in 2006.
He joined the Etch Control Engineering Group
at Applied Materials, Sunnyvale, CA, in 2006, as
a System Design Engineer. His research interests
include extremum seeking control and its applications, cooperative control of
multiple autonomous agents and control systems design, and implementation
in semiconductor industry.
Dr. Zhang is a member of the IEEE Control Systems Society (CSS).
Raúl Ordonez (S’96–M’99) received the M.S. and
Ph.D. degrees in electrical engineering from the Ohio
State University, Columbus, in 1996 and 1999, re-
spectively, specializing in controls research, and mi-
noring in mathematics and signal processing.
Prior to joining the University of Dayton, Dayton,
OH, where he is currently an Associate Professor of
Electrical and Computer Engineering, he served for
two years as a Faculty Member at Rowan University,
Glassboro, NJ. He is a coauthor (with J. T. Spooner,
M. Maggiore, K. Passino) of the book Stable Adap-
tive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approx-
imator Techniques (New York: Wiley, 2002).
Dr. Ordonez is a member of the IEEE Control Systems Society (CSS), where
he serves as Associate Editor for Conference Activities. He was Publicity
Chair for the 2001 IEEE International Symposium on Intelligent Control,
and a member of the Program Committee for the 2001 IEEE Conference
on Decision and Control. He is currently serving as Associate Editor for the
journal Automatica.