John G. Webster Editor 04.automatic Control 4 1999

04 Automatic Control
Absolute Stability Abstract | Full Text: PDF (196K) Adaptive Control Abstract | Full Text: PDF (336K) Bilinear Systems Abstract | Full Text: PDF (233K) Chaos, Bifurcations, and Their Control Abstract | Full Text: PDF (471K) Controllability and Observability Abstract | Full Text: PDF (265K) Control System Design, Continuous-Time Abstract | Full Text: PDF (282K) Convex Optimization Abstract | Full Text: PDF (264K) Abstract | Full Text: PDF (173K) Digital Control Abstract | Full Text: PDF (241K) Discrete Event Dynamical Systems Abstract | Full Text: PDF (223K) Discrete Time Systems Design Methods Abstract | Full Text: PDF (308K) Filtering and Estimation, Nonlinear Abstract | Full Text: PDF (209K) Gain Scheduling Abstract | Full Text: PDF (167K) H Infinity Control Abstract | Full Text: PDF (345K) Input-Output Stability Abstract | Full Text: PDF (396K) Intelligent Control Abstract | Full Text: PDF (115K) Jet Engine Control, Implementations Abstract | Full Text: PDF (273K)
file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20...RONICS%20ENGINEERING/04.Automatic%20Control.htm (1 of 3)18.06.2008 19:59:52
Kalman Filters and Observers Abstract | Full Text: PDF (1787K) Large-Scale and Decentralized Systems Abstract | Full Text: PDF (267K) Linear Dynamical Systems, Approximation Abstract | Full Text: PDF (397K) Model Reference Adaptive Control Abstract | Full Text: PDF (132K) Multivariable Systems Abstract | Full Text: PDF (174K) Nonlinear Control Systems, Analytical Methods Abstract | Full Text: PDF (1141K) Nonlinear Control Systems, Design Methods Abstract | Full Text: PDF (268K) Nyquist Criterion, Diagrams, and Stability Abstract | Full Text: PDF (222K) Open-Loop Oscillatory Control Abstract | Full Text: PDF (298K) Optimal Control Abstract | Full Text: PDF (291K) Periodic Control Abstract | Full Text: PDF (219K) PID Control Abstract | Full Text: PDF (182K) Position Control Abstract | Full Text: PDF (651K) Process Control Abstract | Full Text: PDF (223K) Robust Control Abstract | Full Text: PDF (363K) Robust Control Analysis Abstract | Full Text: PDF (247K) Self-Tuning Regulators Abstract | Full Text: PDF (233K) Singularly Perturbed Systems Abstract | Full Text: PDF (225K) Software for Control System Analysis and Design, Singular Value Decomposition Abstract | Full Text: PDF (198K)
Software for Control System Analysis and Design: Symbol Manipulation Abstract | Full Text: PDF (438K) Stability Theory, Asymptotic Abstract | Full Text: PDF (306K) Stability Theory, Nonlinear Abstract | Full Text: PDF (270K) Stability Theory: Including Saturation Effects Abstract | Full Text: PDF (444K) Stochastic Optimization, Stochastic Approximation and Simulated Annealing Abstract | Full Text: PDF (206K) Stochastic Systems Abstract | Full Text: PDF (210K) System Identification Abstract | Full Text: PDF (287K) Voltage-to-Frequency Converters Abstract | Full Text: PDF (1080K)
file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20ELECTRICAL...TRONICS%20ENGINEERING/04.%20automatic%20control/W1001.htm
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
Home / Engineering / Electrical and Electronics Engineering
HELP
G
Wiley Encyclopedia of Electrical and Electronics Engineering

Absolute Stability Standard Article Anders Rantzer1 1Lund Institute of Technology, Lund, Sweden Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1001 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (196K)
G G
Recommend to Your Librarian Save title to My Profile Email this page Print this page
Browse this title

G
Search this title Enter words or phrases

H
H H
Advanced Product Search Search All Content Acronym Finder
Abstract
The sections in this article are The Circle Criterion The Popov Criterion Gain and Passivity Multipliers and Integral Quadratic Constraints About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20ELE...%20ENGINEERING/04.%20automatic%20control/W1001.htm18.06.2008 21:00:14
then also the feedback system x (t ) = Ax(t ) B [Cx(t ), t ] (4)
ABSOLUTE STABILITY
Analysis of dynamical systems and circuits is mostly done under the assumption that the system is linear and time-invariant. Powerful mathematical techniques are then available for analysis of stability and performance of the system, for example, superposition and frequency domain analysis. In fact, even if the system is nonlinear and time-varying, such assumptions can often be used to get a rst estimate of the system properties. The purpose of absolute stability theory is to carry the analysis one step further and get a bound on the possible inuence of the nonlinear or time-varying components. The approach was suggested by the Russian mathematician Lure in the 1940s and has, since then, developed into a cornerstone of nonlinear systems theory. The basic setup is illustrated in Fig. 1, where the linear time-invariant part is represented by the transfer function G(s) and the nonlinear parts of the system are represented by a feedback loop w (v, t). The analysis of the system is based on conic bounds on the nonlinearity (Fig. 2): (v, t )/v for all v = 0 (1)
is exponentially stable. The name of this result comes from its graphical interpretation. The Nyquist plot, that is, the plot of G(i) in the complex plane as R, must not cross or circumscribe the circle centered on the real axis and passing through 1/ and 1/ (Fig. 3). An important aspect of the circle criterion is that it demonstrates how frequency domain properties can be used in a nonlinear setting. It is instructive to compare with the Nyquist criterion, which states that the closed-loop system with linear feedback w(t) kv(t) is stable for all k [, ], provided that G(i) does not intersect the real axis outside the interval [1/ , 1/ ]. The circle criterion replaces the interval condition with a circular disk. As a consequence, the stability assertion is extended from constant feedback to nonlinear and time-varying feedback. The proof of the circle criterion is based on a quadratic Lyapunov function of the form V (x ) = x Px where the matrix P is positive denite. It can be veried that V(x) is decreasing along all possible trajectories of the system, provided that the frequency condition [Eq. (3)] holds. As a consequence, the state must approach zero, regardless of the initial conditions. THE POPOV CRITERION In the case that has no explicit time dependence, the circle criterion can be improved. For simplicity, let 0 and hence 0 (v )/v for all v = 0 (5)
This problem was studied in Russia during the 1950s. In particular, a conjecture by Aiserman was discussed, hoping that the system would be stable for all continuous functions in the cone [Eq. (1)], if and only if it was stable for all linear functions in the cone. This conjecture was nally proved to be false, and it was not until the early 1960s that a major breakthrough was achieved by Popov (1).
THE CIRCLE CRITERION Popov and his colleagues made their problem statements in terms of differential equations. The linear part then has the form
The Popov criterion can then be stated as follows. Theorem 2 (Popov criterion). Suppose that : R R is Lipschitz continuous and satises Eq. (5). Suppose the system x Ax is exponentially stable and let G(i) C(iI A)1B. If there exists R such that R e[(1 + i)G(i)] > then the system x (t ) = Ax(t ) B [Cx(t )] is exponentially stable. Note that the circle criterion is recovered with 0. Also the Popov criterion can be illustrated graphically. Introduce the Popov plot, where ImG(i) is plotted versus ReG(i). Then
1
x = Ax Bw v = Cx
and the corresponding transfer function is G(s ) = C(sI A ) 1 B
(2)
(6)
Theorem 1 (Circle criterion). Suppose that the system x Ax is exponentially stable and that : R2 R is Lipschitz continuous and satises Eq. (1). If G(i) + 1 0 < Re G(i) + 1 R (3)
(7)
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
ABSOLUTE STABILITY
(, t)
w v
15 10 5
Im
G(s)
0 5 10
1/
1/
Figure 1. Absolute stability deals with linear time-invariant systems in interconnection with nonlinear functions.
G(i)
stability can be concluded from the Popov criterion if and only if there exists a straight line separating the plot from the point 1/ . The slope of the line corresponds to the parameter . See the following example. Example. To apply the Popov criterion to the system x 1 = 5x1 4x2 + (x1 ) x 2 = x1 2x2 + 21(x1 ) let
15 10 0 Re (a) 10
15 10 5
Im
s + 82 G ( s ) = C ( i A ) 1 B = 2 s + 7s + 6 The plot in Fig. 4 then shows that the Popov criterion is satised for 1. The theory of absolute stability had a strong development in the 1960s, and various improvements to the circle and Popov criteria were generated, for example, by Yakubovich. Many types of nonlinearities were considered and stronger criteria were obtained in several special cases (24). Important aspects of the theory were summarized by Willems (5), using the notions of dissipativity and storage function. GAIN AND PASSIVITY In the results described so far, the results were stated in terms of differential equations. A parallel theory was developed by Zames (6) avoiding the state-space structure and studying stability purely in terms of inputoutput relations.
0 5 10 15
1/
1/
G(i)
10
0 Re (b)
10
Figure 3. The circle criterion proves stability as long as the Nyquist plot does not cross or circumscribe the circle corresponding to the conic bounds on the nonlinearity. (a) 0 . (b) 0 .
2 0 2 1/ +
w w= (v)
Im G(i)
4 6 8
v
v
10 12 5 0 5 Re G(i) 10 15
v
Figure 2. The nonlinearity is bounded by linear functions.
Figure 4. The Popov criterion can be applied when there exists a straight line separating the Popov plot from the point 1/ .
ABSOLUTE STABILITY
For this purpose, a dynamical system is viewed as a map F from the input u to the output Fu. The map F is said to be bounded if there exists C 0 such that
T T
The only property of saturation that would be exploited by the Popov criterion is that 0 sat(v)/ v 1 and, consequently,
T
0 |u|2 dt
0
w(t )[v(t ) w(t )] dt
|Fu|2 dt C2
0 0
for all T 0. The gain of F is denoted F and dened as the minimal such constant C. The map is said to be causal if two inputs that are identical until time T will generate outputs that are also identical until time T. This makes it possible to state the following well-known result: Theorem 3 (Small gain theorem). Suppose that the input output maps F and G are bounded and casual. If F G <1 then the feedback equations v = Gw + f w = Fv + e dene a bounded causal map from the inputs (e, f ) to the outputs (v, w). It is worthwhile to make a comparison with the circle criterion. Consider the case when . Then the condition [Eq. (3)] becomes |G(i)| < 1 [0, ] (8)
for all w sat(v), T 0. However, the inequality will remain valid even if some perturbation of amplitude smaller than one is added to the factor w in the product w(v w). One way to do this is to introduce a function h(t) with the property h(t)dt 1 and replace the previous expression by (w h w) (v w), where h w is a convolution. The integral inequality then becomes
T
0
0
(w + h w )(v w ) dt
(9)
Using this inequality, the Popov criterion [Eq. (6)] can be replaced by the condition Re[(1 + i + H (i))(G(i) + 1 )]] > 0
wR
(10)
This is the same as Eq. (8), since is the gain of and maxG(i) is the gain of the linear part [Eq. (2)]. Another important notion closely related to gain is passivity. The inputoutput map F is said to be passive if
T
where H(i) ei h(t)dt. The factor 1 i H(i) is called multiplier. The theory and applications of absolute stability have recently had a revival since new computer algorithms make it possible to optimize multipliers numerically and to address applications of much higher complexity than previously. The inequality [Eq. (9)] is a special case of what is called an integral quadratic constraint, IQC. Such constraints have been veried for a large number of different model components such as relays, various forms of hysteresis, time delays, time variations, and rate limiters. In principle, all such constraints can be used computationally to improve the accuracy in stability and performance analysis. A unifying theory for this purpose has been developed by Megretski and Rantzer (8), while several other authors have contributed with new IQCs that are ready to be included in a computer library for system analysis. BIBLIOGRAPHY
1. V. M. Popov, Absolute stability of nonlinear systems of automatic control, Autom. Remote Control, 22: 857875, 1962. (Original in Russian, August, 1961.) 2. V. A. Yakubovich, Absolute stability of nonlinear control systems in critical cases, parts 13, Avtomaika i Telemechanika, 24 (3): 293302; 24 (6): 717731, 1963; 25 (25): 601612, 1964. (English translation in Autom. Remote Control.) 3. V. A. Yakubovich, On an abstract theory of absolute stability of nonlinear systems, Vestnik Leningrad Univ. Math., 10: 341361, 1982. (Original in Russian, 1977.) 4. S. Lefschetz, Stability of Nonlinear Control Systems, New York: Academic Press, 1963. 5. J. C. Willems, Dissipative dynamical systems, part 1, General theory; part 2, Linear systems with quadratic supply rates, Arch. Rational Mech. Anal., 45 (5): 321393, 1972. 6. G. Zames, On the inputoutput stability of nonlinear time-varying feedback systemspart 1, Conditions derived using concepts of loop gain; part 2, Conditions involving circles in the frequency plane and sector nonlinearities, IEEE Trans. Autom. Control, 11: 228238, 1966.
0
0
u(t )y(t ) dt
for all T and all y = Fu
For example, if the input is a voltage and the output is a current, then passivity property means that the system only can consume electrical power, not produce it. Stability criteria can also be stated in terms of passivity. For example, the circle criterion can be interpreted this way, if 0 and is large. MULTIPLIERS AND INTEGRAL QUADRATIC CONSTRAINTS Less conservative stability criteria can often be obtained by exploiting more information about the nonlinearity. One way to do this is to introduce so-called multipliers. Consider, for example, a system with a saturation nonlinearity:
Ax + B x = Ax Bsat(Cx ) = Ax + BCx Ax B
if Cx 1 if |Cx| < 1 if Cx 1
ABSTRACT DATA TYPES
7. G. Zames and P. L. Falb, Stability conditions for systems with monotone and slope-restricted nonlinearities, SIAM J. Control, 6 (1): 89108, 1968. 8. A. Megretski and A. Rantzer, System analysis via Integral Quadratic Constraints, IEEE Trans. Autom. Control, 47: 819830, 1997.
ANDERS RANTZER
Lund Institute of Technology
ABSORBER. See ELECTROMAGNETIC FERRITE TILE ABSORBER. ABSORPTION MODULATION. See ELECTROABSORPTION.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Adaptive Control Standard Article Petros A. Ioannou1 and Elias B. Kosmatopoulos1 1University of Southern California, Los Angeles, CA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1002 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (336K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are On-Line Parameter Estimation Model Reference Adaptive Control Adaptive Pole Placement Control Adaptive Control of Nonlinear Systems Acknowledgment About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering Copyright c 1999 John Wiley & Sons, Inc.
ADAPTIVE CONTROL
According to Websters dictionary, to adapt means to change (oneself) so that ones behavior will conform to new or changed circumstances. The words adaptive systems and adaptive control were used as early as 1950 (1). The design of autopilots for high-performance aircraft was one of the primary motivations for active research on adaptive control in the early 1950s. Aircraft operate over a wide range of speeds and altitudes, and their dynamics are nonlinear and conceptually time-varying. For a given operating point, specied by the aircraft speed (Mach number) and altitude, the complex aircraft dynamics can be approximated by a linear model of the form (2)
where Ai , Bi , Ci , and Di are functions of the operating point i. As the aircraft goes through different ight conditions, the operating point changes, leading to different values for Ai , Bi , Ci , and Di . Because the output response y(t) carries information about the state x as well as the parameters, one may argue that in principle, a sophisticated feedback controller should be able to learn about parameter changes by processing y(t) and use the appropriate gains to accommodate them. This argument led to a feedback control structure on which adaptive control is based. The controller structure consists of a feedback loop and a controller with adjustable gains as shown in Fig. 1. The way the controller parameters are adjusted on line leads to different adaptive control schemes. Gain Scheduling The approach of gain scheduling is illustrated in Fig. 2. The gain scheduler consists of a lookup table and the appropriate logic for detecting the operating point and choosing the corresponding value of the control parameter vector . For example, let us consider the aircraft model in Eq. (1) where for each operating point i, i = 1, 2, . . ., N , the parameters Ai , Bi , Ci , and Di are known. For each operating point i, a feedback controller with constant gains, say i , is designed to meet the performance requirements for the corresponding linear model. This leads to a controller, say C(), with a set of gains {1 , 2 , . . ., i , . . ., N } covering N operating points. Once the operating point, say i, is detected, the controller gains can be changed to the appropriate value of i obtained from the precomputed gain set. Transitions between different operating points that lead to signicant parameter changes may be handled by interpolation or by increasing the number of operating points. The two elements that are essential in implementing this approach are a lookup table to store the values of i and the plant auxiliary measurements that correlate well with changes in the operating points. Direct and Indirect Adaptive Control A wide class of adaptive controllers is formed by combining an on-line parameter estimator, which provides estimates of unknown parameters at each time instant, with a control law that is motivated from the knownparameter case. The way the parameter estimator, also referred to as the adaptive law, is combined with the control law gives rise to two different approaches. In the rst approach, referred to as indirect adaptive control
1
ADAPTIVE CONTROL
Fig. 1. Controller structure with adjustable controller gains.
Fig. 2. Gain scheduling.
(shown in Fig. 3), the plant parameters are estimated online and used to calculate the controller parameters. This approach has also been referred to as explicit adaptive control, because the design is based on an explicit plant model. In the second approach, referred to as direct adaptive control (shown in Fig. 4), the plant model is parametrized in terms of the controller parameters, which are estimated directly without intermediate calculations involving plant parameter estimates. This approach has also been referred to as implicit adaptive control, because the design is based on the estimation of an implicit plant model. The principle behind the design of direct and indirect adaptive control shown in Figs. 3 and 4 is conceptually simple. The design of C(c ) treats the estimates c (t) (in the case of direct adaptive control) or the estimates (t) (in the case of indirect adaptive control) as if they were the true parameters. This design approach is called certainty equivalence and can be used to generate a wide class of adaptive control schemes by combining different on-line parameter estimators with different control laws. The idea behind the certainty equivalence approach is that as the parameter estimates c (t) and (t) converge to the true ones c and , respectively, the performance of the adaptive controller C(c ) tends to that achieved by C( c ) in the case of known parameters. Model Reference Adaptive Control Model reference adaptive control (MRAC) is derived from the model-following problem or model reference control (MRC) problem. The structure of an MRC scheme for a LTI, single-input single-output (SISO) plant is shown in Fig. 5. The transfer function W m (s) of the reference model is designed so that for a given reference input signal r(t) the output ym (t) of the reference model represents the desired response the plant output y(t) should follow. The feedback controller, denoted by C( c ), is designed so that all signals are bounded and the closed-loop plant transfer function from r to y is equal to W m (s). This transfer function matching guarantees y ym , which represents the deviation of the that for any given reference input r(t), the tracking error e1
ADAPTIVE CONTROL
Fig. 3. Indirect adaptive control.
Fig. 4. Direct adaptive control.
plant output from the desired trajectory ym , converges to zero with time. The transfer function matching is achieved by canceling the zeros of the plant transfer function G(s) and replacing them with those of W m (s) through the use of the feedback controller C( c ). The cancellation of the plant zeros puts a restriction on the plant to be minimum-phase, that is, have stable zeros. If any plant zero is unstable, its cancellation may easily lead to unbounded signals. The design of C( c ) requires the knowledge of the coefcients of the plant transfer G(s). When is unknown we use the certainty equivalence approach to replace the unknown c in the control law with its estimate c (t) obtained using the direct or the indirect approach. The resulting control schemes are known as MRAC and can be classied as indirect MRAC of the structure shown in Fig. 3 and direct MRAC of the structure shown in Fig. 4.
ADAPTIVE CONTROL
Fig. 5. Model reference control.
Adaptive Pole Placement Control Adaptive pole placement control (APPC) is derived from the pole placement control (PPC) and regulation problems used in the case of LTI plants with known parameters. In PPC, the performance requirements are translated into desired locations of the poles of the closed-loop plant. A feedback control law is then developed that places the poles of the closed-loop plant at the desired locations. The structure of the controller C( c ) and the parameter vector c are chosen so that the poles of the closed-loop plant transfer function from r to y are equal to the desired ones. The vector c is usually calculated using an algebraic equation of the form
where is a vector with the coefcients of the plant transfer function G(s). As in the case of MRC, we can deal with the unknown-parameter case by using the certainty equivalence approach to replace the unknown vector c with its estimate c (t). The resulting scheme is referred to as adaptive pole placement control (APPC). If c (t) is updated directly using an on-line parameter estimator, the scheme is referred to as direct APPC. If c (t) is calculated using the equation
where (t) is the estimate of generated by an on-line estimator, the scheme is referred to as indirect APPC. The structure of direct and indirect APPC is the same as that shown in Figs. 3 and 4, respectively, for the general case. The design of APPC schemes is very exible with respect to the choice of the form of the controller C(c ) and of the on-line parameter estimator. Design of On-Line Parameter Estimators As we mentioned in the previous sections, an adaptive controller may be considered as a combination of an on-line parameter estimator with a control law that is derived from the known-parameter case. The way this combination occurs and the type of estimator and control law used give rise to a wide class of different adaptive controllers with different properties. In the literature of adaptive control the on-line parameter estimator has often been referred to as the adaptive law, update law, or adjustment mechanism. Some of the basic methods used to design adaptive laws are (1) Sensitivity methods (2) Positivity and Lyapunov design (3) Gradient method and least-squares methods based on estimation error cost criteria The sensitivity method is one of the oldest methods used in the design of adaptive laws and is briey explained below.
ADAPTIVE CONTROL
Sensitivity Method. This method became very popular in the 1960s (3), and it is still used in many industrial applications for controlling plants with uncertainties. In adaptive control, the sensitivity method is used to design the adaptive law so that the estimated parameters are adjusted in a direction that minimizes a certain performance function. The adaptive law is driven by the partial derivative of the performance function with respect to the estimated parameters multiplied by an error signal that characterizes the mismatch between the actual and desired behavior. This derivative is called the sensitivity function, and if it can be generated online, then the adaptive law is implementable. In most earlier formulations of adaptive control, the sensitivity function cannot be generated online, and this constitutes one of the main drawbacks of the method. The use of approximate sensitivity functions that are implementable leads to adaptive control schemes whose stability properties are either weak or cannot be established. Positivity and Lyapunov Design. This method of developing adaptive laws is based on the direct method of Lyapunov and its relationship to positive real functions. In this approach, the problem of designing an adaptive law is formulated as a stability problem where the differential equation of the adaptive law is chosen so that certain stability conditions based on Lyapunov theory are satised. The adaptive law developed is very similar to that based on the sensitivity method. The only difference is that the sensitivity functions in the approach are replaced with ones that can be generated online. In addition, the Lyapunov-based adaptive control schemes have none of the drawbacks of the MIT rule-based schemes. The design of adaptive laws using Lyapunovs direct method was suggested by Grayson (4) and Parks (5) in the early 1960s. The method was subsequently advanced and generalized to a wider class of plants by Phillipson (6), Monopoli (7), and others (8,9,10,11). Gradient and Least-Squares Methods Based on Estimation Error Cost Criteria. The main drawback of the sensitivity methods used in the 1960s is that the minimization of the performance cost function led to sensitivity functions that are not implementable. One way to avoid this drawback is to choose a cost function criterion that leads to sensitivity functions that are available for measurement. A class of such cost criteria is based on an error, referred to as the estimation error (12), that provides a measure of the discrepancy between the estimated and actual parameters. The relationship of the estimation error with the estimated parameters is chosen so that the cost function is convex, and its gradient with respect to the estimated parameters is implementable. Several different cost criteria may be used, and methods such as the gradient and least-squares may be adopted to generate the appropriate sensitivity functions.
On-Line Parameter Estimation

The purpose of this section is to present the design and analysis of a wide class of schemes that can be used for on-line parameter estimation. The essential idea behind on-line estimation is the comparison of the observed system response y(t) with the output of a parametrized model y (,t) whose structure is the same as that of the plant model. The parameter vector (t) is adjusted continuously so that y (,t) approaches y(t) as t increases. The on-line estimation procedure involves three steps: In the rst step, an appropriate parametrization of the plant model is selected. The second step involves the selection of the adjustment law, referred to as the adaptive law, for generating or updating (t). The third and nal step is the design of the plant input so that the properties of the adaptive law imply that (t) approaches the unknown plant parameter vector as t . We start by considering the SISO plant
ADAPTIVE CONTROL
where x Rn and only y, u are available for measurement. Note that the plant equation can also be written as an nth-order differential equation
The constants ai , bi are the plant parameters. We can express yn as
where
We can avoid the use of differentiators by ltering with our nth-order stable lter 1/ (s) to obtain
where
In a similar way we can express the plant dynamics as follows:
where W (s) is an appropriate proper stable transfer function and , are dened similarly to , . In Eqs. (5), (6) the unknown vectors , appear linearly in equations where all the other terms are known a priori or can be measured online. We use these parametric models to estimate the unknown vectors or by using the following approaches. SPRLyapunov Design Approach. We start by rewriting Eq. (6) as follows (for simplicity we drop the subscript ):
where = L 1 (s)and L(s) is chosen so that L 1 (s) is a proper transfer function and W (s)L(s) is a proper strictly positive real (SPR) transfer function. Let (t) denote the estimate of at time t. Then the estimated value of z
ADAPTIVE CONTROL based on (t) is given by
and the estimation error is given by
Let
denote the normalized estimation error, where n2 s is the normalizing signal, which we design to satisfy
Typical choices for ns that satisfy this condition are n2 s = T P for any P = PT > 0, and the like. When L , the condition is satised with m = 1, that is, ns = 0, in which case = 1 . We express in terms of the parameter = : error
For simplicity, let us assume that L(s) is chosen so that WL is strictly proper and consider the following state space representation of Eq. (8):
where Ac , Bc , and Cc are the matrices associated with a state-space representation that has a transfer function W (s)L(s) = CT c (sI Ac ) 1 Bc . Let us now consider the following Lyapunov-like function for the differential equation (9):
where
> 0 is a constant matrix and Pc = PT c > 0 satises the algebraic equations
for some vector q, matrix Lc = LT c > 0, and small constant > 0. The existence of Pc = PT c > 0 satisfying the above equations is guaranteed by the SPR property (12) of W (s)L(s) = CT c (sI Ac ) 1 Bc . The time derivative of
ADAPTIVE CONTROL
V is given by
= as a function of signals that can be measured so that the indenite terms in V are We now need to choose cannot depend on e explicitly. canceled out. Because e is not available for measurement, We know that Pc Bc = Cc , which implies that eT Pc Bc = eT Cc = . Therefore
= to make V 0 is now obvious, namely, for The choice for
we have
Using the above inequality, we can prove the following theorem. Theorem 1. The adaptive law in Eq. (4) guarantees that: (i) , L . L 2 , independent of the boundedness of . (ii) , ns , L , and is perstistently exciting (PE)that is, there exist positive constant 1 , 0 , T 0 such (iii) If ns , , that
then (t) exponentially fast. The proof of the theorem can be found in Ref. 12.
Gradient Method. In this method, we consider the parametric model in Eq. (5). Similar to the previous
subsection, we dene (t) to be the on line estimate of and the normalized estimation error as
where z = T (t) and m2 = 1 + n2 s and n2 s is chosen so that /m L . The adaptive law is designed to minimize the performance index J (), i.e.,
ADAPTIVE CONTROL which gives
Different choices for the performance index lead to different adaptive laws. Let us consider the simple quadratic cost function (instantaneous cost function)
Applying the gradient method, the minimizing trajectory (t) is generated by the differential equation
where
> 0 is a scaling matrix that we refer to as the adaptive gain. We have
The following theorem holds: Theorem 2. The adaptive law in Eq. (18) guarantees that: L . (i) , ns , (ii) , ns , L 2 , independent of the boundedness of . (iii) If ns , L and is PE, then (t) exponentially fast. The proof of the theorem can be found in Ref. 12.
Least Squares. Let (t), , z be dened as above, and let m2 = 1 + n2 s , (t) be the estimate of at time
t, and m satisfy /m L . We consider the following cost function:
where Q0 = QT 0 > 0, 0, 0 = (0), which includes discounting of past data and a penalty on the initial estimate 0 of . Because z/m, /m L , we have that J () is a convex function of over Rn at each time t. Hence, any local minimum is also global and satises
which yields the so-called nonrecursive least-squares algorithm
10 where
ADAPTIVE CONTROL
We can show that P, satisfy the differential equations
We refer to Eqs. (22) and (23) as the continuous-time recursive least-squares algorithm with forgetting factor. The stability properties of the least-squares algorithm depend on the value of the forgetting factor . In the identication literature, Eqs. (22) and (23) with = 0 are referred to as the pure least-squares algorithm and have a very similar form to the Kalman lter. For this reason, the matrix P is usually called the covariance matrix. The pure least-squares algorithm has the unique property of guaranteeing parameter convergence to constant values as described by the following theorem: Theorem 3. The pure least-squares algorithm guarantees that: (i) (ii) (iii) (iv) , P L . , ns , , L 2. , ns , , where is a constant vector. limt (t) = If ns , L and is PE, then (t) converges to as t . The proof of the theorem can be found in 12.
Bilinear Parametric Model. As will be shown in the next sections, a certain class of plants can be
parametrized in terms of their desired controller parameters, which are related to the plant parameters via a Diophantine equation. Such parametrizations and their related estimation problem arise in direct adaptive control and, in particular, direct MRAC, which is discussed in the next section. In these cases appears in the form
where is an unknown constant, z, , z0 are signals that can be measured, and W (s) is a known proper transfer function with stable poles. Because the unknown parameters , appear in a special bilinear form, we refer to Eq. (24) as the bilinear parametric model. Known Sign of . The SPRLyapunov design approach and the gradient method with an instantaneous cost function discussed in the linear parametric case extend to the bilinear one in a rather straightforward manner. Let us start with the SPRLyapunov design approach. We rewrite Eq. (24) in the form
ADAPTIVE CONTROL
11
where z1 = L 1 (s)z0 , = L 1 (s) , and L(s) is chosen so that L 1 (s) is proper and stable and WL is proper and SPR. The estimate z of z and the normalized estimation error are generated as
where ns is designed to satisfy
and (t), (t) are the estimates of , at time t, respectively. Letting Eqs. (25) to (27) that
, it follows from
T Now T T = T T + T T = T , and therefore,
By choosing
we can see that the following theorem holds. Theorem 4. The adaptive law in Eq. (30) guarantees that: (i) (ii) (iii) (iv) , , L . , , ns , L 2. L , is PE, and L 2 , then (t) converges to as t . If , If L 2 , the estimate converges to a constant independent of the properties of .
The proof of the theorem can be found in Ref. 12. The case where the sign of is unknown is also given in Ref. 12.
Model Reference Adaptive Control Problem Statement. Consider the SISO LTI plant described by
12
ADAPTIVE CONTROL
where Gp (s) is expressed in the form
where Zp , Rp are monic polynomials and kp is a constant referred to as the high-frequency gain. The reference model, selected by the designer to describe the desired characteristics of the plant, is described by
which is expressed in the same form as Eq. (32), that is,
where Zm (s), Rm (s) are monic polynomials and km is a constant. The MRC objective is to determine the plant input up so that all signals are bounded and the plant output yp tracks the reference model output ym as close as possible for any given reference input r(t) of the class dened above. We refer to the problem of nding the desired up to meet the control objective as the MRC problem. In order to meet the MRC objective with a control law that is implementable (i.e., a control law that is free of differentiators and uses only measurable signals), we assume that the plant and reference model satisfy the following assumptions:
Plant Assumptions.
P1. Zp (s) is a monic Hurwitz polynomial of degree mp . P2. An upper bound n on the degree np of Rp (s) is known. P3. The relative degree n = np mp of Gp (s) is known. P4. The sign of the high-frequency gain kp is known.
Reference Model Assumptions.

M1. Zm (s), Rm (s) are monic Hurwitz polynomials of degree qm , pm , respectively, where pm n. M2. The relative degree n m = pm qm of W m (s) is the same as that of Gp (s), that is, n m = n.
MRC Schemes: Known Plant Parameters. In addition to assumptions P1 to P4 and M1, M2, let us also assume that the plant parameters, that is, the coefcients of Gp (s), are known exactly. Because the plant is LTI and known, the design of the MRC scheme is achieved using linear system theory. We consider the feedback control law
ADAPTIVE CONTROL where
13
c 0 , 3 R1 , 1 , 2 Rn 1 are constant parameters to be designed, and (s) is an arbitrary monic Hurwitz polynomial of degree n 1 that contains Zm (s) as a factor, i.e., (s) = 0 (s)Zm (s), which implies that 0 (s) is monic, Hurwitz, and of degree n0 = n 1 qm . The controller parameter vector
is to be chosen so that the transfer function from r to yp is equal to W m (s). The inputoutput properties of the closed-loop plant are described by the transfer function equation
where
We can now meet the control objective if we select the controller parameters 1 , 2 , 3 , c 0 so that the closed-loop poles are stable and the closed-loop transfer function Gc (s) = W m (s) is satised for all s C. Choosing
and using
(s) =
0 (s)Zm (s),
the matching equation Gc (s) = W m (s) becomes
Equating the coefcients of the powers of s on both sides of Eq. (38), we can express it in terms of the algebraic equation
= [T 1 , T 2 , 3 ]T ; S is an (n + np 1) (2n 1) matrix that depends on the coefcients of Rp , kp Zp , where to satisfy Eq. (39) and and p is an n + np 1 vector with the coefcients of Rp Zp 0 Rm . The existence of and, therefore, Eq. (38) will very much depend on the properties of the matrix S. For example, if n > np , more will satisfy Eq. (39), whereas if n = np and S is nonsingular, Eq. (39) will have only one solution. than one Lemma 1. Let the degrees of Rp , Zp , ,
0
and Rm be as specied in Eq. (34). Then
of Eq. (38) or (39) always exists. (i) The solution is unique. (ii) In addition, if Rp , Zp are coprime and n = np , then the solution
14
ADAPTIVE CONTROL The proof of the lemma can be found in Ref. 12.
MRAC for SISO Plants. The design of MRAC schemes for the plant in Eq. (31) with unknown parameters is based on the certainty equivalence approach and is conceptually simple. With this approach, we develop a wide class of MRAC schemes by combining the MRC law, where is replaced by its estimate (t), with different adaptive laws for generating (t) online. We design the adaptive laws by rst developing appropriate parametric models for , which we then use to pick up the adaptive law of our choice from the preceding section. Let us start with the control law
whose state-space realization is given by
where = [T 1 , T 2 , 3 , c0 ]T and = [T 1 , T 2 , yp , r]T , and search for an adaptive law to generate (t), the estimate of the desired parameter vector . It can be seen that under the above control law, the tracking error satises
where = 1/c 0 , = [T 1 , T 2 , 3 , c 0 ]T . The above parametric model may be developed by using the matching Eq. (38) to substitute for the unknown plant polynomial Rp (s) in the plant equation and by canceling the Hurwitz polynomial Zp (s). The parametric model in Eq. (42) holds for any relative degree of the plant transfer function. A linear parametric model for may also be developed from Eq. (42). Such a model takes the form
where
The main equations of several MRAC schemes formed by combining Eq. (41) with an adaptive law based on Eq. (42) or (43). The following theorem gives the stability properties of the MRAC scheme. Theorem 5. The closed-loop MRAC scheme in Eq. (41), with (t) adjusted with any adaptive law with normalization based on the model in Eq. (42) or (43) as described in the preceding section, has the following properties: (i) All signals are uniformly bounded. (ii) The tracking error e1 = yp ym converges to zero as t .
ADAPTIVE CONTROL
15
(iii) If the reference input signal r is sufciently rich of order 2n, r L , and Rp , Zp are coprime, then the = converge to zero for the adaptive law with known sgn(kp ). tracking error e1 and parameter error The proof of the theorem can be found in Ref. 12.
Adaptive Pole Placement Control

In the preceding section we considered the design of a wide class of MRAC schemes for LTI plants with stable zeros. The assumption that the plant is minimum-phase, that is, it has stable zeros, is rather restrictive in many applications. A class of control schemes that is popular in the known-parameter case are those that change the poles of the plant and do not involve plant zeropole cancellations. These schemes are referred to as pole placement schemes and are applicable to both minimum- and nonminimum-phase LTI plants. The combination of a pole placement control law with a parameter estimator or an adaptive law leads to an adaptive pole placement control (APPC) scheme that can be used to control a wide class of LTI plants with unknown parameters. Problem Statement. Consider the SISO LTI plant
where Gp (s) is proper and Rp (s) is a monic polynomial. The control objective is to choose the plant input up so that the closed-loop poles are assigned to those of a given monic Hurwitz polynomial A(s). The polynomial A(s), referred to as the desired closed-loop characteristic polynomial, is chosen according to the closed-loop performance requirements. To meet the control objective, we make the following assumptions about the plant: P1. Rp (s) is a monic polynomial whose degreen n is known. P2. Zp (s), Rp (s) are coprime, and degree(Zp ) < n. Assumptions P1 and P2 allow Zp , Rp to be non-Hurwitz, in contrast to the MRC case, where Zp is required to be Hurwitz. If, however, Zp is Hurwitz, the MRC problem is a special case of the general pole placement problem dened above with A(s) restricted to have Zp as a factor. In general, by assigning the closed-loop poles to those of A(s), we can guarantee closed-loop stability and convergence of the plant output yp to zero provided there is no external input. We can also extend the PPC objective to include tracking, where yp is required to follow a certain class of reference signals ym , by using the internal model principle as fol8lows: The reference signal ym L is assumed to satisfy
where Qm (s), the internal model of ym , is a known monic polynomial of degree q with nonrepeated roots on the j axis and satises P3. Qm (s), Zp (s) are coprime. For example, if yp is required to track the reference signal ym = 2 + sin 2t, then Qm (s) = s(s2 + 4) and therefore, according to assumption P3, Zp (s) should not have s or s2 + 4 as a factor.
16
ADAPTIVE CONTROL
In addition to assumptions P1 to P3, let us also assume that the coefcients of Zp (s), Rp (s), i.e., the plant parameters are known exactly. We consider the control law
where P(s), L(s), M (s) are polynomials [with L(s) monic] of degree q + n 1, n 1, q + n 1, respectively, to be found, and Qm (s) satises Eq. (45) and assumption P3. Applying Eq. (46) to the plant in Eq. (44), we obtain the closed-loop plant equation
whose characteristic equation
has order 2n + q 1. The objective now is to choose P, L such that
is satised for a given monic Hurwitz polynomial A(s) of degree 2n + q 1. It can be seen that assumptions P2 and P3 guarantee that L, P satisfying Eq. (49) exist and are unique. The solution for the coefcients of L(s), P(s) of Eq. (49) may be obtained by solving the algebraic equation
where Sl is the Sylvester matrix of Qm Rp , Zp of dimension 2(n + q) 2(n + q),
and l, p, a are the vectors whose entries are the coefcients of L(s), P(s) and A(s), respectively. The coprimeness of Qm Rp , Zp guarantees that Sl is nonsingular; therefore, the coefcients of L(s), P(s) may be computed from the equation
The tracking error e1 = yp ym is given by
ADAPTIVE CONTROL
17
For zero tracking error, Eq. (51) suggests the choice of M (s) = P(s) to null the rst term. The second term in Eq. (51) is nulled by using Qm ym = 0. Therefore, the pole placement and tracking objective are achieved by using the control law
which is implemented using n + q 1 integrators to realize C(s) = P(s)/Qm (s)L(s). Because L(s) is not necessarily Hurwitz, the realization of Eq. (52) with n + q 1 integrators may have a transfer function, namely C(s), with poles outside C . An alternative realization of Eq. (52) is obtained by rewriting it as
where is any monic Hurwitz polynomial of degree n + q 1. The control law (53) is implemented using 2(n + q 1) integrators to realize the proper stable transfer functions ( LQm )/ , P/ . APPC. The APPC scheme that meets the control objective for the unknown plant is formed by combining the control law in Eq. (53) with an adaptive law based on the parametric model in Eq. (5). The adaptive law generates on-line estimates a , b of the coefcient vectors a of Rp (s) = sn + T a n 1 (s) and b p (s, t) = sn + T a n 1 (s), Z p (s, of Zp (s) = T b n 1 (s), respectively, to form the estimated plant polynomials R T (s, t) = b n 1 (s). The estimated plant polynomials are used to compute the estimated controller polynomials L (s, t) by solving the Diophantine equation t), P
, P pointwise in time, or the algebraic equation for L
l , where S l is the Sylvester matrix of R l contains the coefcients of L p Qm , Z p; , P ; and l contains the for coefcients of A(s). The control law in the unknown-parameter case is then formed as
Because different adaptive laws may be picked up from the section On-Line Parameter Estimation above, a wide class of APPC schemes may be developed. , P or The implementation of the APPC scheme requires that the solution of the polynomial Eq. (54) for L of the algebraic Eq. (55) for l exists at each time. The existence of this solution is guaranteed provided that l (t) is nonsingular at each time t. p (s, t) are coprime at each time t, that is, the Sylvester matrix S p (s, t)Qm (s), Z R p Qm , Z p are strongly coprime at each time t. Then Theorem 6. Assume that the estimated plant polynomials R all the signals in the closed-loop APPC scheme are u.b., and the tracking error converges to zero asymptotically with time. The proof of the theorem can be found in Ref. 12. The assumption that the estimated polynomials are strongly coprime at each time t is restrictive and cannot be guaranteed by the adaptive law. Methods that relax this assumption are given in Ref. 12.
18
ADAPTIVE CONTROL
Adaptive Control of Nonlinear Systems

In the previous sections we dealt with the problem of designing controllers for linear time-invariant systems. In this section, we show how the techniques of adaptive control of linear systems can be extended or modied for nonlinear systems. Although the techniques presented can be applied to a variety of nonlinear systems, we will concentrate our attention on adaptive state feedback control of SISO feedback-linearizable systems. Feedback-Linearizable Systems in Canonical Form. We start with an nth-order SISO feedbacklinearizable system in canonical form, whose dynamics are as follows:
where y, u R are the scalar system output and input, respectively, f , g are smooth vector elds, and x [x1 , x2 , . . ., xn ]T is the state vector of the system. In order for the system in Eq. (57) to be controllable and feedback-linearizable we assume that A1. A lower bound for g(x) [i.e., |g(x)| > > 0 x Rn ] and the sign of g(x) are known. The control objective is to nd the control input u that guarantees signal boundedness and forces y to follow the output ym of the reference model
where A is a Hurwitz n n matrix, r L , and therefore xm L . In order to have a well-posed problem, it is assumed that the relative degree of the reference model is equal to n. If e ym y is the tracking error, then its nth time derivative satises
(s) = sn + k1 sn 1 + + kn be a Hurwitz polynomial (here s denotes the d/dt operator). Also let [e, Let h e , . . ., e(n 1) ]T . Under assumption A1, the system Eq. (57) is a feedback-linearizable system. Therefore, if we know the vector elds f and g, we can apply the static feedback
where k
[kn , kn 1 , . . ., k1 ]T . Then the error system in Eq. (59) becomes
which implies that e, L and therefore all closed-loop signals are bounded, and limt e(t) = 0.
ADAPTIVE CONTROL
19
In many cases, the vector elds f and g are not completely known and thus adaptive versions of the feedback law (60) have to be applied. For instance, using the usual assumption of linear parametrization, if the vector elds f and g are of the form
where i , i = 1, 2, are vectors with unknown constant parameters, one may replace the feedback law in Eq. (60) with the certainty-equivalent one [the certainty-equivalent feedback-linearizing (CEFL) controller]
i , i = 1, 2, are the estimates of the unknown parameter vectors i , i = 1, 2. These estimates are generated where i: by an on-line adaptive law. We next propose the following adaptive laws for updating
1 = bc f , 2 = ubc g , bc = [0, . . ., 0, 1] . The where i , i = 1, 2, are symmetric positive denite matrices and next theorem summarizes the properties of the proposed control law. Theorem 7. Consider the system in Eq. (57) and the feedback control law in Eqs. (62) and (63). Let assumption T 2 (t)g (x(t)) = 0 for all t, all the closed-loop signals are bounded and the tracking error A1 hold. Then, if converges to asymptotically to zero.
Parametric-Pure-Feedback Systems. Let us now try to extend the results of the previous section to
nonlinear systems that take the form
where u, zi R, f ij , gnj are smooth functions, and Rp is the vector of constant but unknown system parameters. Let us rewrite Eq. (64) as
where f i1 () = f i0 () zi+1 . Systems of the form in Eq. (65) are called parametric-pure-feedback (PPF) systems (13,14). Note that the above class of systems includes as a special case the system in Eq. (57) of the previous section.
20
ADAPTIVE CONTROL
The control objective is to force the system output y to asymptotically track a reference signal ym . We assume that the rst n 1 time derivatives of ym are known. Also it will be assumed that ym and its rst n 1 time derivatives are bounded and smooth signals. Let us now assume that the parameter vector is known and construct a control law that meets the control objectives. Before we design the feedback law, we will transform the system in Eq. (64) into a suitable form. The procedure we will follow is based on the backstepping integrator principle (13). Step 0. Let 1 z1 ym . Let also c1 , . . ., cn be positive constants to be chosen later. Step 1. Using the chain-of-integrators method, we see that, if z2 were the control input in the z1 part of Eq. (65) and were known, then the control law
would result in a globally asymptotically stable tracking, since such a control law would transform the z1 part of Eq. (65) as follows:
However, the state z2 is not the control. We therefore dene 2 to be the difference between the actual z2 and its desired expression in Eq. (66):
Step 2. Using the above denition of 2 , the denition of 1 , and the z1 part of Eq. (65), we nd that
Step 2. Using the above denitions of 1 , 2 , we have that
where (2) is a (p + p2 )-dimensional vector that consists of all elements that are either of the form 2,i or of the form 2,i j , where by 2,i we denote the ith entry of the vector . In the system (69) we will think of z3 as our control input. Therefore, as in step 1, we dene the new state 3 as
ADAPTIVE CONTROL Substituting Eq. (70) into Eq. (69) yields
21
Step i (2 < i n 1). Using the denitions of 1 , . . ., i and working as in the previous steps, we may express the derivative of i as
where the vector (i) contains all the terms of the form i1 i2 i with 1 j i. Dening now i+1 as
we obtain that
Step n.Using the denitions of 1 , . . ., n 1 and working as in the previous steps, we may express the derivative of n as follows:
where the vector contains all the terms of the form i1 i2 ij with 1 j n, Y m y(n 1) m ]T , and [ 0 (z1 , . . ., zn ) + T 1 (z1 , . . ., zn )] is given by
[ym , y m, y m , . . .,
Using the denitions of 1 , . . ., n , and rearranging terms, we may rewrite Eq. (75) as follows:
22
ADAPTIVE CONTROL Using the above methodology, we have therefore transformed the system in Eq. (65) into the following
one:
The above system is feedback-linearizable if the following assumption holds. A1. 0 (z) + T 1 (z) = 0 for all z. Note now that in the case where (and thus ) is known, a controller that meets the control objective is the controller of the form
Under the above control law, the closed-loop dynamics become
It can be shown that the matrix A0 is a stability matrix, provided that ci > 2. Theorem 8. The control law in Eq. (79) guarantees that all the closed-loop signals are bounded and that the tracking error converges to zero exponentially fast, provided that the design constants ci satisfy ci > 2. In the case where the vector is not known, the certainty-equivalence principle can be employed in order to design an adaptive controller for the system. However, the problem of designing parameter estimators for the unknown parameters is not as easy as it was in the linear case. This can be seen from the fact that the states i , i > 1, are not available for measurement, since they depend on the unknown parameters. To overcome this problem a recursive design approach similar to the approach above can be constructed. The difference between this approach [called adaptive backstepping (13)] and the approach presented above is the following: in the approach presented above the states i , i > 1, depend on the unknown vector of parameters . ; in the new approach they are appropriately redened so they depend on the parameter estimate vector
ADAPTIVE CONTROL
23
. In order to overcome Then the derivatives of i , i > 1, depend on the derivatives of the parameter estimates this problem, the adaptive backstepping approach makes use of the so-called tuning functions (13). Next we present the adaptive controller that results from applying the adaptive backstepping procedure to the system in Eq. (65) for the case where A2. T f i2 () are independent of zi+1 and T gn2 = 0. Also for simplicity, and without loss of generality, we will assume that f i1 () = 0. The case where assumption A2 is not valid will be treated in the next subsection. The adaptive controller that results from applying the adaptive backstepping procedure is recursively dened as follows: Control law:
Parameter update law:
Tuning functions:
Regressor vectors:
Here ci > 2, i are positive design constants, and = T > 0 is a positive denite design matrix. The next theorem summarizes the properties of the above control law. Theorem 9. Suppose that assumptions A1 , A2 hold. Then the above adaptive control law guarantees that all the closed-loop signals are bounded and that the tracking error converges asymptotically to zero. The proof of the theorem can be found in Ref. 13.
24
ADAPTIVE CONTROL
A Control Law That Overcomes the Loss-Of-Controllability Problem. A signicant problem that arises in adaptive control of linear-in-the-parameters feedback-linearizable systems is the computation of the feedback control law when the identication model becomes uncontrollable although the actual system is controllable; so far, there is no known solution to this problem. For instance, for the case of the system in Eq. (57) the parameter estimation techniques used in adaptive control cannot guarantee, in general, that 2 (t)T g (x(t))| > 0 for each time t, that is, they cannot guarantee that the identication model is controllable. | Also, for the case of PPF systems presented in the previous subsection, the adaptive backstepping techniques guarantee global stability only in the case where assumption A2 is valid. Such restrictions are made because the computation of the adaptive control law depends on the existence of the inverse of the matrix that consists of the estimated input vector elds (or the Lie derivatives of the output functions along those vector elds). Even in the case of known parameters where the inverse of the corresponding matrix exists (this is trivially satised for feedback-linearizable systems), the inverse of the estimate of this matrix might not exist at each time due to insufciently rich regressor signals, large initial parameter estimation errors, and so on. We next show how one can overcome the problem where the estimation model becomes uncontrollable, by appropriately using switching adaptive control. We will apply the switching adaptive control methodology to the PPF system of the previous subsection, by removing assumption A2. Consider the Lyapunov function for the PPF system of the previous subsection.
By differentiating V with respect to time, we obtain that
Let us dene
Note now that, using the denition of i , we can rewrite the i s as follows:
where i and w i are appropriately dened known functions. Therefore, we have that
is dened to be the vector whose entries are the elements i j , and 0 , 1 areappropriately dened where known functions. We are now ready to present the proposed controller. The control input is chosen as follows:
ADAPTIVE CONTROL where: One has
25
denotes the estimate of . where k() is a positive design function satisfying k(1 , ) = 0 iff 1 = 0 and
is a continuous-switching signal that is used to switch from control u1 to control u2 and vice versa:
s is hysteresis-switching variable dened as follows:
where s (t)
lim t s(), where t.
are updated using the following smooth projection update law (15) The parameter estimates
where
is a symmetric positive denite design matrix and PC is dened as follows (15):
26 where
ADAPTIVE CONTROL
where 0 < < 1, q 2, and j are positive design constants. The following theorem summarizes the properties of the control law in Eq. (83, 84, 85, 86, 87, 88, 89). Theorem 10. Consider the system in Eq. (65) and the control law in Eqs. (83, 84, 85, 86, 87, 88, 89). Let assumption A1 hold. Moreover assume that the following hold: C1. K > 1; k() satises Eq. (85). (0) C, where C2. j are sufciently small. Moreover,
Then for any compact X 0 Rn and for any positive constant c the following holds: there exist a positive constant K such that, for any initial state x0 X 0 , the control law in Eqs. (83, 84, 85, 86, 87, 88, 89) with K > K guarantees that all the closed-loop signals are bounded and, moreover, that the tracking error 1 converges in nite time to the residual set
The idea of using switching adaptive controllers of the form presented above was rst introduced in Ref. 16, where the proposed methodology was applied to systems of the form in Eq. (57). The controller of Ref. 16 was extended in Ref. 17 for PPF systems of the form in Eq. (65).
Acknowledgment
This article was supported by NASA grant NAGW-4103.
BIBLIOGRAPHY
1. J. A. Aseltine, A. R. Manchini, and C. W. Sartune, A survey of adaptive control systems, IRE Trans. Automat. Control, 6 (3): 1958. 2. D. McRuer, I. Ashkenas, D. Graham, Aircraft Dynamics and Automatic Control, Princeton, NJ: Princeton Univ. Press, 1973. 3. J. B. Cruz, Jr., System Sensitivity Analysis, Stroudsburg, PA: Dowden, Hutchinson & Ross, 1973. 4. L. P. Grayson, Design via Lyapunov second method, Proc. 4th JACC, 1963. 5. P. C. Parks, Lyapunov redesign of model reference control systems, IEEE Trans. Autom. Control, 11: 1966. 6. P. H. Phillipson, Design methods for model reference adaptive systems, Proc. Inst. Mech. Eng., 183 (35):, 695706, 1968. 7. R. V. Monopoli, Lyapunovs method for adaptive control design, IEEE Trans. Autom. Control, 3: 1967. 8. I. D. Landau, Adaptive Control: The Model Reference Approach, New York: Marcel Dekker, 1979.
ADAPTIVE CONTROL
27
9. K. S. Narendra, A. M. Annaswamy, Stable Adaptive Systems, Englewood Cliffs, NJ: Prentice-Hall, 1989. 10. S. Sastry, M. Bodson, Adaptive Control: Stability, Convergence and Robustness, Englewood Cliffs, NJ: Prentice-Hall, 1989. 11. K. J. Astrom, B. Wittenmark, Adaptive Control, Reading, MA: Addison-Wesley, 1989. 12. P. A. Ioannou, J. Sun, Robust Adaptive Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 13. M. Krstic, M. I. Kanellakopoulos, and R. Kokotovic, Nonlinear and Adaptive Control Design, New York: Wiley, 1995. 14. D. Seto, A. M. Annaswamy, J. Baillieul, Adaptive control of nonlinear systems with a triangular structure, IEEE Trans. Autom. Control, 7: 1994. 15. J.-B. Pomet, L. Praly, Adaptive nonlinear stabilization: Estimation from the Lyapunov equation, IEEE Trans. Autom. Control, 37: 729740, 1992. 16. E. B. Kosmatopoulos and P. A. Ioannou, A switching adaptive controller for feedback linearizable systems, IEEE Trans. Autom. Control, to be published. 17. E. B. Kosmatopoulos and P. A. Ioannou, Robust switching adaptive control using control Lyapunov functions, Proc. IEEE Conf. Decision and Control, 1997.
PETROS A. IOANNOU ELIAS B. KOSMATOPOULOS University of Southern California
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Bilinear Systems Standard Article David L. Elliott1 1University of Maryland Copyright 2007 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1003.pub2 Article Online Posting Date: July 13, 2007 Abstract | Full Text: HTML PDF (233K)
G G
Browse this title

G

H
H H
Abstract
This article contains necessary background on linear control systems; the definition of bilinear systems and some sources of BLS in engineering; a guide to some useful system theoretic methods; the application of these methods to problems such as stabilization, observability, and controllability; and brief accounts of areas where BLS have made a major contribution to system theory. A few examples and applications are given where appropriate. As much as possible, the notation and terminology are usual in control system work; the state-space approach will be used, and the articleson Linear algebra and Stability theory, nonlinear may be helpful. Some suggested textbooks and other recom mendationsare provided in the Reading list at the end of this article. Keywords: linear control system; state-space approach; discrete time; stabilization; controllability; piecewise; contart control; observability About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
BILINEAR SYSTEMS
with interaction matrix F and a linear output function. Here is a single-input single-output example, in which the coefcient vector g describes the control transducer, the output transducer is described by the row vector h , and v = v(t ) is a control signal: = F x + vg, y = h x x or written out in full, dxi = dt
n n
An electrical circuit or other engineering system often communicates with its external environment by input signals that control its behavior and output signals; it is then called a control system. If the components of a control system all obey Ohms Law or one of its analogs, such as Hookes Law, the system is called linear. In linear control systems, the effect of the controls is additive and the output measurement is linear. They are discussed in the article in this encyclopedia on MULTIVARIABLE SYSTEMS. What distinguishes a bilinear system (BLS) is that although it is linear in its state variables, some control signal u(t) exerts its effect multiplicatively. BLS may be given as mathematical models of circuits or plants or may be chosen by a designer to obtain better system response than is possible with a linear system. Their study is a rst step toward nonlinear control theory. Industrial process control, economics, and biology provide examples of BLS with multiplicative controls such as valve settings, interest rates, and neural signals, respectively. This topic of research began in the early 1960s with independent work in the USSR and in the USA; see the surveys of Bruni et al. (1) and Mohler (2, 3) for historical development and reviews of the early literature.
(1)
(Fi j x j ) + vgi , i = 1, . . . , n; y =
j =1 i=1
h i xi
(1)
(It is customary to suppress in the notation the timedependence of x, y and often the control v .) As written, equation 1 has constant coefcients, and such a control system is called time-invariant , which means that its behavior does not depend on where we choose the origin of the independent variable t ; the system can be initialized and control v exerted at any time. When the coefcients are not constant, that is, made explicit in the notation, e.g., = F (t )x + vg(t ), y = h (t )x x which is called a time-variant linear system. For both linear and bilinear systems, we will need the = F x, which for a given initial condition x(0) solution of x is x(t ) = e Ft x(0). The matrix exponential function is dened by eFt = I + Ft + F k tk F 2t2 + ... + + ... 2 k! (2)
Notation in This Article. The symbol R means the real numbers and Rn the n -dimensional real linear space; C means the complex plane, with R(s) the real part of s C. The bold symbols ah, x, z will represent elements (col (x transposed) is a row vector; x . umn vectors) of Rn ; x is the complex conjugate transpose of x. Given a vector function x(t ) = col[x1 (t ), . . . , xn (t )], its time derivative is def (t ) = d x(t )/dt . Capital letters A , B , F , X are square matrix ces and I the identity matrix diag (1, 1, . . . , 1). The trace of matrix A is the sum of its diagonal elements, written tr(A ); det(A ) is its determinant. i n are integers; r , s , t are real scalars, as are lowercase Greek letter quantities. German type g, sl, . . . will be used for Lie algebras. Control Systems: Facts and Terminology The following discussion is a brief reminder of state-space methods; see Sontag (4). The state variables in an electrical circuit are currents through inductors and voltages across capacitors; in mechanics, they are generalized positions and momenta; and in chemistry, they are concentrations of molecules. The state variables for a given plant constitute a vector function depending on time x(t ). Knowledge of an initial state x(0), of future external inputs, and the rstorder vector differential equation that describes the plant determine the trajectory {x(t ), t 0}. For the moment, we will suppose that the plant elements, such as capacitors, inductors, and resistors, are linear (Ohms Law) and constant in value. The circuit equations can usually be combined into a single rst-order vector differential equation,
it is the inverse Laplace transform of the matrix (sI F )1 and can be computed by numerical methods described by Golub and Van Loan (5). Its most familiar use in electrical engineering is to solve equation 1
t
x(t ) = eFt x(0) +

0
e(t r)F v(r )gdr
The polynomial PF (s) = det(sI F ) = (s 1 )(s 2 ) (s n ) is called the characteristic polynomial of F , and its roots are called the eigenvalues of F . The entries in the matrix e Ft are linear combinations of terms like t mi ei t , i = 1, . . . , n If the eigenvalues are distinct, the integers m i vanish, but if i is a multiple eigenvalue, m i may be positive. For a given F , the matrices {exp(Ft ), t R} are a group under matrix multiplication: exp((r + t )F ) = exp(rF )exp(tF ), (exp(tF ))1 = exp(tF ). For different applications, different restrictions may be placed on the class U of admissible control signals . In this article, U will usually be the class of piecewise constant (PWC) signals. The value of a control at an instant of transition between pieces need not be dened; it makes no difference. If there is a single number > 0 that is an upper bound for all admissible controls, call the class U as a reminder.
def
J. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright 2007 John Wiley & Sons, Inc.
Bilinear Systems
k
A control system, linear or not, is said to be controllable , x , a time on its state space if for any two states x T > 0 exists and a control in U for which the trajectory ends at x(T ) = x . For time-invariant starting at x(0) = |x linear systems, there is a simple and useful condition necessary for controllability. Stated for equation 1, this Kalman rank condition is rank(g, Fg, . . . , F n1 g) = n If it is satised, the matrix-vector pair {F, g} is called a controllable pair . If the controls are not bounded, this condition is sufcient for controllability, but if they are in some U , the control may be small compared with F x for large x and have an insufcient effect. BILINEAR SYSTEMS: WHAT, WHY, WHERE? Precisely what are BLS and why should one use bilinear systems in control engineering? This section will give some answers to those questions, starting with a formal denition, and give some examples.
= Ax + x
j =1
u jB jx
(5)
(We will see later how to recover facts about equation 3 from the homogeneous case.) If A = 0 in equations 4 or 5, the BLS is called symmetric :
k
x =
j =1
B jx
(5)
Denition. A bilinear system is a rst-order vector dif = p(x, u) in which x is a state vector; ferential equation x u is a control signal (scalar or vector); and the components of p (x, u ) are polynomials in (x, u ) that are linear in x, u separately, but jointly quadratic, with constant real coefcients. Restating that, for any real numbers , p(x, u) p(x, u) + p(x, 0) + p(0, u) The (optional) output is linear, y = h x. To begin with, BLS are simpler and better understood than most other nonlinear systems. Their study involves a constant interplay between two protable viewpoints: looking at BLS as time-invariant nonlinear control systems and as time-variant linear systems. Another answer is that BLS are useful in designing control systems that use a very small control signal to modulate a large current of electricity of uid, apply brakes, or change rates of growth. A third answer is that the usual linearization of a nonlinear control system near an equilibrium point can be improved by using a BLS approximation; thus, a(x) + ug(x), with a(xe ) = 0; let a q |x=xe , b = q(xe ), B = |x=xe x x Translating the origin so that xe = 0, to rst order in x and u separately: x A = = Ax + u(Bx + b), y = h x x (3) =
As a control system, equation 4 is time-invariant, and we need to use controls that can start and stop when we wish. The use of PWC controls not only is appropriate for that reason but also allows us to consider switched linear systems as BLS, and in that case, only a discrete set of control values such as {1, 1} or {0, 1} is used. Later we will be concerned with state-dependent feedback controls, u = u(x), which may have to satisfy condi = tions that guarantee that differential equations like x Ax + u(x)Bx have well-behaved solutions. Discrete time bilinear systems (DBLS) are described by difference equations, rather than ordinary differential equations. DBLS applications have come from the discretetime dynamics common in economic and nancial modeling, in which the control is often an interest rate. DBLS also are used for digital computer simulation of continuous time systems like equation 3: Using Eulers point-slope method with a time-step , for times k = 0, 1, 2, . . . , the discrete-time system is x(k + 1) = (I + A)x(k) + u(k)(Bx(k) + b) (6)
with output y(k) = h x(k). DBLS will be discussed briey at appropriate places below. Their solutions are obtained by recursion from their initial conditions using their difference equations. Some Application Areas Reference (2) lists early applications of BLS to nuclear reactors, immunological systems, population growth, and compartmental models in physiology. For a recently analyzed BLS from a controlled compartmental model, see the work on cancer chemotherapy by Ladzewicz and Schattler (6) and its references. Using linear feedback u = Kx in a BLS results in a quadratic autonomous system. Recently some scientically interesting quadratic systems, exhibiting chaotic behavior, have been studied by decomposing them into BLS of this type. Celikovsk y and Van ec ek (7) have studied the thirdorder Lorenz system as a BLS with output feedback: = Ax + uBx, u(x) = x1 , with > 0, > 0, > 0, and x 0 0 0 0 , B = 0 0 1 1 0 0 0 0 1 0 For small all eigenvalues of A are negative, but for > 1 one becomes positive and B generates a rotation. This description seems to be characteristic of several such examples of strange attractors and bounded chaos, including the R ossler attractor. A=
Although some results will be stated for equation 3, usually we will suppose that b = 0; such BLS are called homogeneous bilinear systems , and for later reference, they are given here in both their single-input and k -input versions = Ax + uBx x (4)
Bilinear Systems
In electrical engineering, BLS viewpoints can be used to discuss switched and clocked circuits, in which the control takes on only a discrete set of values like {0, 1} and the plant is linear in each switched condition. The solutions are then easy to compute numerically. Sometimes the duty cycle of a switch is under control, as in motors and DC-to-DC power conversion systems. A simple example is a conventional ignition sparking system for automobiles, in which the primary circuit can be modeled by assigning x 1 to voltage across capacitor of value C , x 2 to current in the primary coil of inductance L . The control is a distributor or electronic switch, either open (innite resistance) or closed (small resistance R ) with duty cycle specied by the crankshaft rotation and timing controls. Then with a battery of emf V , 1 u x 1 = x2 (x1 V ), C C 1 x 2 = x1 , L u= 1/R, closed 0, open
stabilize, the BLS is somewhat difcult, but for small n one can nd PA+B () and test possible values of by the RouthHurwitz stability test for polynomials (see STABILITY THEORY, ASYMPTOTIC). For n = 2 PA+B () = 2 (tr(A) + tr(B)) + det(A + B), so tr(A) + tr(B) < 0 and det (A + B) > 0 (7)
Other automotive BLS include mechanical brakes and controlled suspension systems, among the many applications discussed in Reference 3. One advantage of piecewise constant control in BLS is that the solutions, being piecewise linear, are readily computed. For that reason, in aerospace and process engineering, control designs with gain scheduling (see GAIN SCHEDULING) are an area where BLS methods should be useful; such control schemes change the F matrix and equilibrium point to permit locally linear control. Recent work on hybrid systems (nite-state machines interacting with continuous plants) also falls into the category of switched linear systems. STABILIZATION I: CONSTANT CONTROLS This section will introduce an important engineering design goal, stability, and the beginning of a running discussion of stabilization. Stabilization is an active area of research, in an effort to nd good design principles. A matrix F is called a Hurwitz matrix if all n of the eigenvalues of F lie in the left half of the com = F x is said to be plex plane; i.e., R(i ) < < 0. Then x exponentially stable (ES); as time increases, all solutions are bounded and ||x(t )|| < ||x(0)||et . If even one eigenvalue lies in the right half plane, almost all solutions will grow unboundedly and the system is called unstable . Multiple imaginary eigenvalues = j can give t m cos(t ) (resonance) terms that also are unstable. Warning: Even if the time-varying eigenvalues of a time-variant linear differential equation all lie in the left half plane, that does not guarantee stability! If A is a Hurwitz matrix, equation 4 is ES when u = 0. Suppose that A is not Hurwitz or that is too small; nding a feedback control u such that equation 4 is ES with a desirable is called stabilization . The problem of stabilization of BLS and other nonlinear control systems is still an active area of engineering research. In this section, we consider only the use of constant controls u = in equation 4; the result of applying this feedback is a linear dynamical system = (A + B)x. To nd a range of values for that will x
guarantee stability. Graphing these two expressions against is an appropriate method for nding good values of . A complete set of conditions A and B that are necessary and sufcient for stabilizability of second-order BLS with constant feedback were given by Chabour et al. (8). Other criteria for stabilization by constant control have been found, such as this one from Luesink and Nijmeijer (9). Suppose the eigenvalues of A are i , i = 1,. . . , n , and the i . If there is some nonsingular matrix eigenvalues of B are P , real or complex, for which P 1 AP and P 1 BP are simultaneously upper triangular, then the eigenvalues of A + B i , i = 1, . . . , n. If some real satises the n linear are i + i ) < 0, it will be the desired constant inequalities R(i + control. For more about such triangularizable BLS, see the section below on The Lie Algebra of a BLS. SOLUTIONS OF BILINEAR SYSTEMS From one viewpoint a BLS with a specic nonconstant control history {u(t ), t 0} should be thought of as a timevariant linear differential equation. We will use the singleinput case of equation 4 as an example, with A + u(t )B as time-variant matrix. The solution depends on the initial time t 0 at which the state is x(t0 ), and is of the form x(t ) = (t, t0 )x(t0 ), where (t, t0 ) is called a transition matrix . For more about the general theory of these matrices, see MULTIVARIABLE SYSTEMS, Chap. 9 of Kailath 10, or vol. 1, Chap. 4 of Reference 3. Once having written that expression for x(t ), it can be observed that must satisfy the matrix differential equation = (A + u(t )B) , (t0 , t0 ) = I
It has the composition property, also called the semigroup property (t, t1 ) (t1 , t0 ) = (t, t0 ). However, as a control system , equation 4 is timeinvariant, by denition. Then the most convenient families of admissible controls for BLS are the PWC and other piecewise-dened controls; such a control can be specied by its values on an interval of denition of duration , for instance {u(t ), t (t0 , t0 + )}. From the time-invariance of the system, a basic interval of denition, (0, ), can be used without any loss. Given a particular control signal u on (0, ), its time shift by can be denoted u (t ) = u(t ), on (, + ), as is usual in system theory. The concatenation of two controls u and v with respective durations 1 and 2 is written u o v and is another admissible control with duration 1 + 2 : (u v)(t ) = u(t ), t [0, 1 ) v1 (t ), t [1 < t 2 ]
For the general multi-input BLS of equation 5, the control is a k -component vector u = [u1 , . . . , uk ], and the transition k matrix satises = (A + u B ) . Concatenation is j =1 j j
Bilinear Systems
dened in the same way as for scalar controls: u o v is u followed by the translate of v. The time-invariance of the BLS leads to useful properties of the transition matrices. The transition matrix depends on the control u and its starting time, so the matrix should be labeled accordingly as (u; t, t0 ), and the state trajectory corresponding to u is x(t ) = (u; t, t0 )x(0)
u given by m constant pieces {u(t ) = u(k1 ), k1 t < k } on intervals that partition {0 t < m = T }, the transition = (A + uB)X is clearly matrix for X (u; T, 0) =
m k =1
e(A+u(k1 )B)k
(9)
Given two controls and their basic intervals {u, 0 < t < } and {v, 0 < t < }, the composition property for BLS transition matrices can be written in a nice form that illustrates concatenation (u followed by the translate of v) (v ; , ) (u; , 0) = (u v; , 0) (8a)
A transition matrix always has an inverse, but it is not always a transition matrix for the BLS. However, if the BLS is symmetric (5) and the admissible controls U are signsymmetric (i.e., if u U, then u U), the transition matrix (u; , 0) resulting from control history {u(t ), 0 t } has an inverse that is again a transition matrix, obtained by using the control that reverses what has been done before, u (t ) = u( t ), t 2 ; (u ; 2, ) (u; , 0) = I (8b)
This idea can be carried much further with more analysis: More general (measurable) inputs can be approximated by PWC controls, and in analogy to the denition of an integral as a limit of sums, the solution to equation 4 for measurable inputs can be written as (in an appropriate sense) the limit of products like equation 9, called a product-integral. The representation given by equation 9 generalizes to the multi-input BLS equation 5 in the obvious way. With equation 9, one can also easily verify the composition and (for A = 0) inverse properties. To emphasize that exponential formulas for noncommuting matrices have surprising behavior, here is a standard example in which you should notice that A 2 = 0 and B 2 = 0. 0 1 0 0 1 0 A= and B = ; AB BA = ; 0 0 1 0 0 1 eAt eBt = 1 0 t 1 0 t 1 t t 0 0 1 = = 1 + t2 t 1 , but 1
e(A+B)t = exp
cosh(t ) sinh(t )
sinh(t ) cosh(t )
= (A + u1 B + + uk Bk )x, is like a An asymmetric BLS, x symmetric one in which one of the matrices B0 = A has a constant control u0 1, whose sign cannot be changed. The controllability problem for asymmetric BLS involves nding ways around this obstacle by getting to I some other way. From a mathematical viewpoint, the set of transition matrices for equation 4 is a matrix semigroup with identity element I . See the last section of this article. MORE ABOUT TRANSITION MATRICES Transition matrices for BLS have some additional properties worth mentioning. For instance, in the rare situation that A , B 1 ,. . . , B k all commute, the transition matrix has a comforting formula (u; , 0) = e
At +
t 0 k i=1
OBSERVABILITY AND OBSERVERS This section is concerned with BLS that have an output measurement y with m < n components, so that x (t ) is not directly available. For many purposes in control (stabilization, prediction of future outputs, and optimization), it is necessary 1) to ensure that different states can be distinguished and 2) to obtain estimates of x(t ) from available information. An important question is whether an inputoutput history HT = {u(t ), y(t )|0 t T } will uniquely determine the initial or nal state. Let C be an m n matrix (think of it as m row vectors); its null-space {x|Cx = 0} is denoted by C . The BLS is the m -output system given by = (A + uB)x, y = Cx. x (10)
ui (s)Bi ds
Warning: If the matrices do not commute, this formula is invalid! The solution of a single-input inhomogeneous BLS like = Ax + u(Bx + b), is much like the solution equation 3, x of a linear system. If (u; t, 0) is the solution of the homogeneous matrix system =A + uB , (0) = I
then for equation 3 with initial condition x(0),

t
x(t ) =
(u; t, 0)x(0) +
0
(u; t, s)u(s)bds
The initial state is not known, only the history HT . Rn Suppose that u(t ), t 0, is given. Call two states x, x u -indistinguishable on the interval (0, T ) if the two corresponding outputs are equal on that interval, i.e., if , 0 t T . This relation, written C (u; t, 0)x = C (u; t, 0)x , is transitive, reexive, and symmetric (an equivax ux lence relation) so it partitions the state space into disjoint sets (see Chap. 5 of Reference 4); it is also linear in the state. Therefore we need only be concerned with the set of states u -indistinguishable from the origin, namely, Iu = {x|x u 0} = {x|C (u; t, 0)x = 0, 0 t T } which is a linear subspace called the u -unobservable subspace of the BLS; the u -observable subspace is the quotient
def
One advantage of using piecewise constant controls is that they not only approximate other signals, but suggest a construction of the transition matrix. For a PWC control
Bilinear Systems
space Ou = Rn /Iu . That can be rephrased as Rn = Iu Ou . If Iu = 0, we say that the given system is u -observable. In Grasselli and Isidori (11) u -observability is given the name observability under single experiment. If two states x, z are u -indistinguishable for all admissible u , we say they are indistinguishable and write x z. The set of unobservable states is the linear subspace I = {x|C (u; t, 0)x = 0 for all u U } Its quotient subspace (complement) O = R /I is called the observable subspace, and the system is called observable if I = 0. The unobservable subspace is invariant for the BLS; trajectories that begin in I remain there. If the largest invariant linear subspace of C is 0, the BLS is observable ; this is also called observability under multiple experiments, because to test it one would have to have duplicate systems, each initialized at x(0) but using its own control u . Theorem 1 of Reference 11 states that for PWC controls and piecewise continuous controls, the BLS is observable if and only if a u exists for which it is u -observable. The proof constructs a universal input u that distinguishes all states from 0 by concatenating at most n + 1 inputs: u = u0 o o un . At the k th stage in the construction, the set of states indistinguishable from 0 is reduced in dimension by a well-chosen u k . The test for observability that comes out of this analysis is that the rank of the matrix (C ; A , B ) is n , where
n
State Observers Given a u -observable system with A , B known, it is possible to estimate the initial state (or current state) from the history HT . The theory of time-variant linear systems (see Reference 10 or vol. 1 of Reference 3) shows that Ou is the range of the time-variant version of the observability Gramian,
T
WT =
def 0
(u; t, 0)C C (u; t, 0)
If rank(WT ) = n, the initial state can be recovered; in our notation

T 1 x(0) = WT (T ) 0
(u; t, 0)C y(t )dt
(C; A, B) = col[C, CA, CB, CA2 , CAB, CBA, CB2 , . . .] That is, (C ; A , B ) contains C and all matrices obtained by repeated multiplications on the right by A and B . This is the rst theorem on the existence of universal inputs, and the idea has been extended to other nonlinear systems by Sontag and Sussmann. The simplest situation in which to look for observability criteria is for a system with input zero, an autonomous = Ax, y = Cx. It is no surtime-invariant linear system x prise that the Kalman rank criterion for observability is appropriate for such systems. The (time-invariant) observability Gramian is W = col[C, CA, . . . , C(A)n1 ]; we say {C , A } is an observable pair if rank(W ) = n , and that is both necessary and sufcient for linear system observability. How can we extend this to the case where the input is unknown? To derive the answer, from williamson (12), choose our admissible controls to be polynomials in t of degree n on any xed time interval. Assume x(0) = 0. It is still necessary that rank(W ) = n , to preserve observability when u = 0. Repeatedly differentiate y = Cx at t = 0 (not that one would do this in practice) to obtain def (n1) Y = col{y0 , y 0 , . . . , y0 }. If y 0 = CAx(0) + u(0)CBx(0) = 0 for some u (0), the information from y 0 would be lost; this gives a necessary condition that CB = 0; continuing this way, necessarily CAB = 0, and so on. All the necessary conditions for observability can be summarized as rank(W ) = n and c Ak B = 0, 0 k n 2 (11)
The current state can be obtained from (u; t, 0)x(0) or by more efcient means. Even though observability may fail for any constant control, it still may be possible, using some piecewise constant control u , to achieve u -observability. Frelek and Elliott (13) pointed out that the Gramian could be optimized in various ways (e.g., by minimizing its condition number) with PWC controls, using a nite sequence of u values, to permit accurate recovery of the entire state or certain preferred state variables. The larger the linear span of the trajectory, the more information is acquired about x(0). One recursive estimator of the current state is an asymptotic state observer. There are many variations on this idea; see KALMAN FILTERS AND OBSERVERS. A state observer can be regarded as a simplication of the Kalman lter in which no assumptions about noise statistics are made, nor is a Riccati equation used, and so can be extended to nonlinear systems for which no Kalman lter can be found. The asymptotic state observer to be described is sufciently general for BLS. For a given control system, it is a model of the plant to be observed, with state vector denoted by z and an input proportional to the output error. (Grasselli and Isidori (14) showed that there is nothing to be gained by more general ways of introducing an error term.) To show how this works, we generalize slightly to allow an inhomoge-neous BLS. Here are the plant, observer, and error equations; K is an n m gain matrix at our disposal. = Ax + u(Bx + b), y = Cx x z = Az + u(Bz + b) + uK(y Cz); let e = z x = (A + uB uKC)e e (12a) (12b) (12c)
To show the sufciency of these conditions for observability, just note that no matter what control u is used, it and its derivatives do not appear in any of the output derivatives, so Y = W x(0) and x(0) = W 1 Y from the rank condition.
Observer design for linear systems is concerned with nding K for which the observer is convergent , meaning that e(t ) 0, under some assumptions about u . (Observability is more than what is needed for the error to die out; a weaker concept, detectability, will do. Roughly put, a system is detectable if what cannot be observed is asymptotically stable.) At least three design problems can be posed for equation 12.
Bilinear Systems
1. Design an observer that will converge for all choices of (unbounded) u in U. This requires only the conditions of equation 11, replacing B with B KC, but then the convergence depends only on the eigenvalues of A ; using the input has gained us no information. 2. Assume that u is known and xed; in which case, the methods of nding K for observers of time-variant linear systems are employed, such as Riccati-type equations. There is no advantage to the BLS form in this problem. 3. Design a control and simultaneously choose K to get best convergence. Currently, this is a difcult nonlinear programming problem, although Sen (15) shows that a random choice of the values of PWC u should sufce; from the invariance of the problem under dilation (zooming in toward the origin), it seems likely that a periodic control would be a good choice. This observer design problem is much like an identication problem for a linear system, but identication algorithms are typically not bilinear. Using digital computer control, one is likely to have not a continuous history H but the history of a PWC input u d and a sampled output Hd = {(ud (t1 ), y(t1 )), (ud (t2 ), y(t2 )), . . .} and the BLS can be dealt with as a discrete-time BLS. Using only a, nite sequence of N > n values, the initial state z [or current state x(t N )] can be estimated by the leastsquares method, which at best projects z onto the closest z in the observable subspace:
N
{0, }, {, 2 }, {2, 3 }, {3, 4 }, respectively
(13c)
The nal state is (using the Taylor series of Eq. (7) for the exponential and keeping only terms up to degree 2) x(4 ) = 7eB eA eB eA x(0) = (I + 2 [A, B] + 3 (higher order terms) + )x(0) (13d) The Lie Algebra of a BLS In much the same way that controllability of the linear system equation 1 is related to the linear span of {g, Fg, , F n1 g}, controllability of BLS is related to a linear space of matrices generated from A and B by repeated bracketing, called the Lie algebra of the BLS. A survey of Lie algebra facts is given in Belinfante and Kolman (16). The primary concern of this subsection is homogeneous BLS of the types given in equations 4 and 5. To make clear what is being discussed, we need a denition. Denition. A linear space g over a eld K (usually the real or complex numbers) with a multiplication g g g: {X, Y } [X, Y ] g will be called a Lie algebra if it satises the properties 1. [X, Y ] = [X, Y ] = [X, Y ], K. 2. [X, Y ] + [X, X] = 0. 3. [X, [Y, Z]] + [Y, [Z, X]] + [Z, [X, Y ]] = 0 (Jacobi identity). In mathematical writing, Lie algebras are usually given abstractly, by relations among their elements, and only then does one represent the elements by matrices acting on some vector space. In contrast, the Lie algebras of control theory have specied generators. (In this article, the Lie algebras are all matrix Lie algebras; for nonlinear vector eld Lie algebras, see CONTROLLABILITY AND OBSERVABILITY.) The Lie algebra generated by a BLS is constructed as follows. Let Mn denote the linear space of all n n real matrices; its dimension is r 2 . A real linear subspace g Mn that is closed under the bracket [X, Y ] = XY YX is called a (real) matrix Lie algebra . The space Mn itself can be identied with a Lie algebra of dimension n 2 over R called the general linear algebra gl(n, R). are said to be equivalent if Two matrix Lie algebras g, g their elements are related by a common similarity transfor = P 1 XP . This denition is justied by the idenmation X tity P 1 [A, B]P = [P 1 AP, P 1 BP ] For homogeneous BLS with system matrices A and B as in equation 4, we generate a Lie algebra in the following way, based on the theory of free Lie algebras. {A, B}LA is the subspace of Mn containing A and B and closed under the Lie bracket and linear span operations. It is not hard to compute {A, B}LA , using the properties 13 of Lie brackets. Start with two generators A and B , we write down a tree
z = arg min
z k =1
y(tk ) C (ud ; tk , 0)z
CONSEQUENCES OF NONCOMMUTIVITY The noncommutativity of the coefcient matrices A , B of a BLS is crucial to its controllability, raising questions that suggest for their answers some interesting mathematical tools: Lie algebras and Lie groups, named for Norwegian mathematician Sophus Lie, pronounced lee. See the Reading List at the end of this article for books about them. Lie Brackets If A and B do not commute, then solving equation 4 is more interesting and difcult. In addition to A , B , we will need AB BA , which is written [A , B ] and is called the Lie bracket of A and B . To see how this matrix might come into the picture, and obtain a geometric interpretation of the Lie bracket, consider the two-input BLS with piecewise constant controls = uAx + vBx; u, v {1, 0, 1} x (13a)
Starting at any x (0), use a control with four control segments (u , v ), each of small duration > 0: {1, 0}, {0, 1}, {1, 0}, {0, 1} on intervals (13b)
Bilinear Systems
of brackets as a data structure T (A, B):

BA [A, B] [A, [A, B]] [B, [B, A]] [A, [A, [A, B]]] [B, [A, [A, B]]] [A, [B, [B, A]]] [B, [B, [B, A]]] . . . . . . . . . . . .
The trees indicated structure depends only on the denition of a Lie algebra; T (A, B) is built level by level, bracketing each terminal leaf by A and by B ; as shown it has already been pruned of obviously linearly dependent leaves by the identities [X, Y ] = [Y, X] and [X, X] = 0. By using the Jacobi identity, more leaves can be pruned; e.g., note that [B, [A, [A, B]]] = [A, [B, [B, A]]] at level 4 before building level 5, and so forth. It is a known feature of Lie algebras that all higher order Lie brackets generated by A and B can be obtained from those in this tree using the Jacobi identity. The linear span of the members of T (A, B) is the desired Lie algebra g = {A, B}LA . As our matrices are in Mn there can be no more than n 2 of them that are linearly independent; working with the specic matrices A , B more dependencies can be found (the Cayley-Hamilton Theorem provides them) so the construction of the tree stops when the matrices at some level are linearly dependent on their ancestors. In this process, we nd enough matrices to obtain the dimension l and a basis B for the linear space g; we shall write it as an array B = {C1 , . . . , Cl }. As we assume A and B are not linearly dependent, it is convenient to take C1 = A, C2 = B, C3 = [A, B]. If the entries of A and B are rational numbers, symbolic algebra computer programs can generate T (A, B) and produce B. It is known that generic pairs A , B will generate a tree of matrices whose span is Mn . In other words, if you ll out the entries with randomly chosen real numbers, then for almost all sample pairs, {A, B}LA = gl(n, R). However, control systems have structural relationships among their components that may lead to smaller Lie algebras. For example, here are some properties of generators that are preserved under bracketing and linear combination, and the name of the Lie subalgebra g gl(n, R) with the given property and largest dimension d . Commutativity: g is called abelian and has d = 2. Zero trace: the special linear algebra sl(n, R) has d = n2 1. Skew-symmetry: the orthogonal algebra so(n, R) has d = n(n 1)/2. Simultaneously triangularizable over R or C: solvable Lie algebras have d n(n + 1)/2. In the stabilizability criterion of Reference 9 discussed above, solvable Lie algebras were mentioned; they are characterized by the property that the sequence of Lie algebras g1 = [g, g], fg2 = [g, g1 ], . . . , g j+1 = [g, g j ], . . . terminates at the trivial Lie algebra {0}. For multi-input BLS like equation 5 or symmetric systems (5), a similar (if harder to diagram) tree construction
can be carried out. If the dimension n is low and there are many independent matrices A, B1 , . . . , Bk , only a few brackets may be needed to obtain the basis B of the Lie algebra. One of the more useful ideas in computing with Lie brackets is to notice that [A , X ] is a linear operation on def the matrix X ; dene adA (X) = [A, X] and powers of adA 0 k 1 recursively: adA (X) = X, adA (X) = [A, adk A (X)], k > 0. This simplies the discussion of an important part of the tree T(A, B) because its leftmost leaves adk A (B) are used in an important rank condition (the ad-condition, discussed in the Controllabity Conditions section below). There are many useful formulas involving adA , such as et adA (B) = etA BetA Accessibility and the Lie Rank Condition. What state space is most appropriate for a bilinear system? For inhomogeneous systems, Rn is appropriate. However, for homogeneous BLS, a trajectory starting at 0 can never leave it, and 0 can never be reached in nite time; the state space may as well be punctured at 0. This punctured n -space is denoted by Rn \0 or Rn 0 , and it is of interest in understanding controllability. For n = 1, it is the union of two open half-lines; in the scalar BLS = u , the state can never change sign. For n = 2, the punctured plane is not simply connected. For n 3 the puncture has negligible effect. In some applications, other unusual state spaces may be appropriate. There is an n -dimensional generalization of our scalar example, the diagonalizable BLS, which are (up to similarity) given by x i = ui xi , i = 1, . . . , n. If the initial state is on a coordinate half-axis, a quadrant of a coordinate plane, . . . , or on one of the 2n orthants, the state must stay there forever. Other BLS that live this way on positive orthants occur rather often, and their controllability properties will be discussed in the Positive Systems section below. In economics, chemistry, ecology, and probability applications, the state variables are usually positive and the dynamical model must respect that; bilinear systems and quadratic systems are the simplest models needed in such applications. Sometimes all we need to establish or can establish will be a property weaker than controllability but still very useful: if the set of states { (u; t, 0)x, t > 0} has an open interior, the BLS is said to satisfy the accessibility condition at x. If that condition is satised for all initial states, we say the BLS has the accessibility property on its state space. Controllable systems have this property, always, but it is not enough to ensure controllability, as is shown in our next example, which will also motivate the concept of strong accessibility , which means that { (u; t, 0)x} has an open interior at each t . Example I. On R2 0 consider a BLS = (I + uJ )x, I = x 1 0 0 , J= 1 0 1 1 0
=u In polar coordinates, this system becomes r = r, and its trajectories are, for constant control values, expanding spirals, clockwise or counterclockwise. Starting at r0 , 0
Bilinear Systems
at the set of states that can be reached at , any xed positive time, it is a circle with radius r( ) = e r0 , which is not an open set, so the BLS does not have the strong accessibility property. No state with r( ) < r0 can be reached, which establishes that the system is not controllable. The set {r(t ), (t )|0 < t < } is the open annulus {(r, )|r0 < r < r0 e }, so the control system does have the accessibility property. Reverse the roles of the two generators. The BLS be = 1. At = (J + uI )x; in polar coordinates r comes x = ur , later time T , the radius is arbitrary but (T ) = 2T + 0 . This system has the accessibility property (but not strong accessibility) and is controllable . If the trajectory misses a target state, it takes 2 seconds before it has a second chance. This peculiar state of affairs reects the fact that our state space is punctured at the origin, and as we remarked before, it is topologically a cylinder. Sometimes it is easy to see from the structure or known symmetries of generators that a given system cannot have the accessibility property; the most obvious of these, for equation 5, are 1) the dimension of {A, B}LA is less than n2 1 or 2) A and B are simultaneously triangularizable. How can we guarantee accessibility on Rn 0 ? For a given homogeneous BLS, e.g., equation 4 whose Lie algebra is g = {A, B}LA , construct the n l matrix def Bx = [C1 x, C2 x, . . . , Cl x], where l is the dimension of g. Dene the Lie rank (x) as the dimension of the linear span of {Xx |X g}, or more constructively, (x) = rank(Bx). For homogeneous BLS, a necessary and sufcient condition for the accessibility property is the Lie rank condition introduced by J. Ku cera (17): (x) = n, for all x = 0 (14)
every state x the quotient space of its isotropy subalgebra hx in g satises g/hx Rn 0. Returning once more to Example I, g = {I + J |, R}, so Bx = {I x, Jx} = x1 x2 x2 x1
2 2 , det(Bx) = x1 + x2
2 2 As, x1 + x2 = 0, the equation I x + J x = 0 has the unique solution = 0, = 0; therefore, the isotropy algebra is {0}; on R2 0 the Lie rank is (x) = 2. Those interested in computability will notice that the Lie rank criterion (Eq. 14) is computationally of exponential complexity as n increases, but as a negative criterion, it can be easily checked by choosing a random state x Rn 0 ; if (x) < n, the system cannot have the accessibility property nor controllability. However, if (x) = n, there still may be states, as in our diagonal-system example, which cannot be reached from x. Proving accessibility or establishing useful tests was made easier through the work of Boothby and Wilson (18), which lists, for each state-space dimension n , the (transitive Lie algebras ) and provides a rational algorithm to determine whether a homogeneous BLS has a Lie algebra on the list. This list was completed recently by the independent work of Kramer (19) with an additional Lie algebra known as spin (9, 1).
Example II. Another Lie algebra helps relate linear and bilinear systems; its system theoretic interpretation was pointed out by Brockett (20). aff (n, R) = X M n+1 |X = X 0 x , X Mn , x Rn 0
If (x(0)) = k, then (x(t )) = k, t R. If at some point x the BLS can move in (x) directions, this must remain true at all points that the trajectories can reach from x. (Consider the diagonal systems mentioned above, for example.) Due to the radial scaling properties of BLS, the Lie rank actually needs to be checked only on the unit sphere and is the same at antipodal points. If g satises the condition in equation 14, it is called transitive on Rn 0 ; see the Matrix Groups section to see why the word transitive is used. To check a BLS for transitivity, nd the n n minors of Bx; these are n th degree polynomials. If 0 is their only common zero, the Lie rank is n . This algebraic task is performed by symbolic algebra; see the book by Elliott in the Reading List. For symmetric systems (Eq. 5), transitivity of the Lie algebra generated by {B1 , . . . , Bk } is necessary and sufcient for controllability on Rn 0 ; the Matrix Groups section will explain why. For asymmetric systems (Eq. 4) and (Eq. 5) transitivity is necessary but far from sufcient; that can be seen from Example I. Its Lie algebra is the span of I 2 2 and J , det(x, J x) = x1 + x2 so (x) = 2; but all paths have t x(t ) = e x(0) . def At each state x Rn 0 , the set hx = {X g|Xx = 0} is a linear subspace of g and contains the Lie bracket of any two of its elements, so it is called the isotropy Lie algebra at x; it is the same, up to similarity equivalence, at all points reachable from x by trajectories of the BLS. Transitivity of the Lie algebra g on state space Rn 0 also means that for
The appropriate state space will be an n -dimensional hyperplane P = {z Rn+1 |zn+1 = 1} To see what is happening, consider rst a linear system on = Ax + u(t )b. Now on Rn let Rn , x x1 . z=. . zn + 1 with z n+1 = 0 and zn+1 (0) = 1. On Rn+1 , the bilinear system = Ax + u(t )Bx has aff (n, R) for its Lie algebra but on P z and is equivalent to the linear system we started with. Note that adA (B) = 0 0 Ab 2 , adA (B) = 0 0 0 A2 b , etc. 0
, A = A 0 , B = 0
0 0 0
b 0
Brackets containing the factor B twice will vanish. The hyperplane zn+1 = 1 is invariant, and the BLS is controllable on that hyperplane under the usual Kalman rank condition that rank{b, Ab, . . .} = n. The controllability properties of inhomogeneous BLS = Ax + u(Bx + b) can be studied using like equation 3 x the idea and notation of Example II. This BLS system can, using n + 1 coordinates, be given as a homogeneous BLS with an invariant hyperplane, = Az + u(t )Bz, where A = z A 0 0 , B= 0 B 0 b 0
Bilinear Systems
For equation 4, in Cheng et al. (24), the ad-condition A Left Inverse System. A problem closely related to observability can be posed at this point: left invertibility of = Ax + uBx, y = c x. a single-input single-output BLS x In the terminology of Hirschorn (21), this system is called left invertible if the output history on some interval [0, T ) uniquely determines the input history on [0,T ). The relative order of the BLS is the least positive integer k such that c adk A (B) = 0 (15)
n 1 rank(Ax, Bx, adA (B)x, . . . , adA (B)x) = n on Rn 0
2
(16)
1 or = if all c adk A B = 0, k > 0. The BLS is invertible if 1 < and c adk B x (0) = 0. Invertibility fails if and only if A every control results in the same output. The conditions of Hirschorn (Eq. 15) and Williamson (Eq. 11) are interestingly related when one takes into account that in the system-inverse problem x(0) is known. Assume that the Williamson condition holds, i.e., the rank of W is n and c Ak B = 0, 0 k n 2. They do not force B = 0, because we can have c An1 B = 0. Now evaluating the Hirschorn conditions, in turn
c B = 0; c [A, B] = c AB = 0; c [A, [A, B]] = c A2 B = 0; . . .

1 n1 B = 0; so the relative order is = n. but c adn A B = c A Reference 21 points out, in this case the inverse system becomes an observer, tracking x(t ) when y() is supplied as its input.
CONTROLLABILITY PROBLEMS Criteria for Controllability. The necessary conditions for controllability of homogeneous BLS begin with the Lie rank condition (Eq. 14). Using it is made easier by the list of transitive Lie algebras in References 1819. A necessary condition for controllability given by Elliott (22) is that controllable homogeneous BLS on Rn 0 , n > 2 have the strong accessibility property. (The peculiarity of Example I occurs because the punctured plane is not simply connected.) For inhomogeneous BLS like equation 3, at x = 0, the Kalman rank condition on {b, Ab, . . .} is a sufcient condition for local controllability with unbounded controls; as that rank condition is an open one in the space of coefcients, for sufciently small x, the B x term does no harm. There is usually a family of possible equilibria corresponding to solutions of (A + B)x = b and at each of these one can make such a test. In the rst paper on BLS, Rink and Mohler (23) assumed such local controllability conditions to show controllability with bounded control u of
m
plus the hypothesis that A is similar to a skew-symmetric matrix are shown sufcient for controllability with bounded control. A partial converse is that controllability of equation 4 for arbitrarily small bounds on the controls implies that all eigenvalues of A are imaginary. Many papers on controllability and stabilization cite Jurdjevic and Quinn (25). For BLS their condition specializes as follows, for piecewise continuous signals. If A has eigenvaiues that are purely imaginary and distinct and the Lie = Ax + uBx is controlrank condition is satised, then x lable. This extends to multiple-input BLS immediately. A stabilization result using the ad-condition is also obtained. For symmetric systems, the Lie rank condition is a sufcient condition for controllability, which is connected to the fact that in that case the transition matrices constitute a Lie group (discussed below). For asymmetric BLS, the set of transition matrices is only a semigroup, meaning that it may not contain the inverses of some transition matrices. A broad principle of controllability theory is that you must be able to get back to an open neighborhood of the state where you started, somehow. Several conditions sufcient for controllability of BLS were given by Jurdjevic and Kupka (26), for unbounded controls. Here is a sample. Assume that the eigenvalues of B are simple and real, in the order q1 > q2 . . . > qn ; choose coordinates so that B = diag(q1 , . . . , qn ). Then if the numbers qi q j are all distinct, if the elements of A satisfy Ai j = 0 for all i , j such that |i j | = 1, and if A1n An1 < 0, = (A + uB)x is controllable on Rn \0. Jurdjevic and then x Sallet (27) and others have extended the approach of Reference 24 to nonhomogeneous BLS like equation 3, in the m -input case. Positive Systems. There are several ways in which BLS models arise that involve positive variables. As a simple example, diagonal systems x i = ui xi , i = 1, . . . , n, xi > 0 on the orthant
n Rn + = {x R |xi > 0, i = 1, . . . , n}
can obviously be transformed by the substitution xi = exp(zi ) to the system of coordinate-translations z i = ui . Controllability for systems of a more interesting nature that have positive orthants as their natural state spaces was studied by Boothby (28). Here the BLS is = Ax + uBx on Rn x + (17)
= Ax + x
k =1
(Bk x + bk )uk
provided that the equilibrium set, for admissible u , is conm nected and all eigenvalues of A + k=1 uk Bk can be made strictly stable and strictly unstable using admissible constant values of u . This use of bilinear terms to make up for the deciencies of linear control methods, when controls are bounded, is emphasized by Mohler.
under the hypothesis that the n n matrix A is essentially positive : ai j > 0, i = j , written A > 0. It is well known and easy to show that, for A > 0, if x(0) Rn + , then x(t ) Rn + , t > 0. The conditions on B used in Reference 28 are that B is nonsingular and diagonal; B = diag[1 , . . . , n ]; and for all i = j , i j = 0, which is an invertibility condition for adB . If none of the eigenvalue differences is repeated, {A, B}LA is sl(n, R) if the trace of A is zero, gl(n, R) otherwise, so the
10
Bilinear Systems
Lie rank condition is satised. Controllability and noncontrollability results are established for several families of A , B pairs, especially for n = 2. Bacciotti (29) completed the study for n = 2. He assumes the same conditions: A > 0; B is diagonal and nonsingular, with no repeated eigenvalues; and 2 > 0 (if not, reverse the = Ax + uBx: sign of the control). Then for the BLS x 1. If 1 > 0, the BLS is completely controllable on Rn +. 2. If 1 < 0 but = (2 a11 1 a22 )2 + 41 2 a12 a21 > 0 and 1 a22 2 a11 > 0, then the BLS is completely controllable on Rn + . In any other case, controllability fails. Sachkov (30) gives answers for m -input problems (Eq. 5) with m = n 1 or m = n 2, using an idea that has been successful in other controllability problems: if the m = i=1 ui Bi x is controllable on hypersymmetric system x surfaces V (x) = that ll up the (simply connected) state space Rn + , and if tne zero-control trajectories of equation 5 can cross all the hypersurfaces in both directions, then equation 5 is globally controllable on Rn +. Stabilization II. At this point it is appropriate to look at stabilization by state feedback controls that are not constant. For BLS the usual way of attacking this has been by quadratic Lyapunov functions. Given a vector differ = f(x), f (0) = 0, the basic idea behind ential equation x Lyapunovs Direct Method is to nd a family of nested smooth hypersurfaces around 0, such as concentric ellipsoids, which the trajectories enter and never leave. For example, let us start with a test for stability of the differen = Ax. Choose a symmetric matrix Q that is tial equation x positive denite , which means that all its eigenvalues are positive and is easy to check by the criterion that each of the n leading minor determinants of Q is positive: Q11 > 0, Q11 Q12 Q12 > 0, . . . , det(Q) > 0 Q22
Returning to equation 4, suppose A has all its eigenvalues on the imaginary axis. Then there is a positive denite Q for which QA + A Q = 0; by a change of variables, Q = I . That is the assumption in Reference 25, which therefore uses the Lyapunov function V (x) = x x; the the adcondition on A and B (Eq. 16) is assumed to hold. The proposed feedback control is u(x) = x Bx; so along the = Ax (x Bx)Bx, trajectories of the closed-loop system x = (x Bx)2 0 x = 0. From the ad-condition, we have V trajectories cannot remain in the set {x Bx = 0|x = 0}, and V (x(t )) 0 as t . However, looking at the onedimensional case, one sees that for the differential equa = 3 , the approach to the origin is of order t 1/2 , tion and that this is also true for the n -dimensional problem using quadratic feedback. Using the same hypotheses and nearly the same short proof as in Reference 25, the reader can verify that the bounded feedback u = x Bx/x x provides exponential stability. In linear system work, the choice of feedback control = Ax + uBx, this reis linear, u = c x. Applied to BLS x sults in a quadratic system, such as the Lorenz system, and is the topic of References 6 and 7. Reference 7 is concerned with linear feedback for a class of BLS in which B generates a rotation and, for some constant , A + B has real eigenvalues of mixed sign. For n = 2, after a similarity transformation A + B = diag(1 , 2 ); 1 > 0, 2 < 0, B = J Using a constant control > , the BLS can be globally practically stabilized; i.e., the trajectory eventually enters a ball of radius of order 1/ . For n = 3, the same type of system [now with 3 < 0, B so (3)] can be globally asymptotically stabilized given certain polynomial inequalities in the i and B . Given some simpler inequalities, such as tr(A) < 0 but allowing 1 > 0, the system is practically stabilized by a family of linear feedbacks with parameter . Stabilization of homogeneous BLS in the plane has been fully analyzed; it is not necessary that A have imaginary eigenvalues. Bacciotti and Boieri (32) using constant, linear, and quadratic feedbacks, and Reference 8 using feedbacks differentiable except perhaps at 0, have given complete classications of the possibilities for stabilizability of equation 4 on R2 0 . The methods of analysis in these papers include Lyapunov functions, center manifolds, and properties of plane curves, depending on the various cases. These cases are specied by the determinants and traces of A and B , diagonalizability, and a few other structural features. In Reference 8, a feedback control, useful even when A is not stable, is u = x Rx/x T x where matrices R , T are found on a case by case basis. These controls are homogeneous of degree zero, and if not constant are discontinuous at 0 but differentiate for x = 0. The stabilized trajectories typically approach the origin in a spiraling fashion, often with large excursions; over each revolution the distance to 0 decreases in a constant ratio. In that sense they generalize the constant controls; the signum controls are essentially homogeneous of degree zero, and there may be other possibilities. Recent work on more general problems of nonlinear system stabilization suggests that a time-periodic feedback
Let V (x) = x Qx, whose level surfaces are ellipsoids; then = x (QA + along trajectories of the differential equation, V A Q)x. If (QA + A Q) is negative denite (all eigenvalues < 0, which is what we wanted. There are negative), then V various recipes for choosing Q , such as solving a linear Lyapunov equation QA + A Q = I for Q and testing for positive deniteness afterward. One of the applications of BLS is to controlled switched linear systems x = Au x, Au A where A is (in the simplest version) a nite family of n n matrices indexed by an integer valued control u (t ) that can be assigned in any way that enforces some delay between different values. A basic question is to nd conditions are needed to ensure that the switched system will be ES for arbitrary control sequences. Necessarily all the A u are Hurwitz; otherwise a xed u would not provide ES. Agrachev and Liberzon (31) showed that it is sufcient to also impose the condition that the Lie algebra generated by the Au A is solvable, as in Reference 9. In the coordinates (real or complex) in which the matrices are triangular, their common Lyapunov function is x x.
Bilinear Systems
11
will be needed for three or more dimensions, in order to bring into play the higher order Lie brackets of A and B . A Note on Optimal Control. At the time that work began on BLS one of the motivations for such studies (2, 17) was their application to optimal control; see OPTIMAL CONTROL. Reference 9 treats optimal control of a BLS arising in cancer chemotherapy. Case studies for vehicle control and nuclear reactor control were summarized in Reference 3, vol. 2; its bibliography lists some studies of biological and ecological systems with BLS models, in which there is some evidence that certain biosystems switch behaviors optimally; see the paper of Oster (33) on bees. The most studied cost was the time-to-target; that is, given an initial state and a target (a state or closed set), use controls in some admissible class of bounded controls U to nd the trajectory connecting the initial state and the target in least elapsed time [See Jurdjevics book (34)]. In linear system versions of this problem, it was known that the set accessible from a given state was the same for U as for the set of PWC controls with values in {, }, called bang-bang controls. One formulation of the bangbang principle is the set attainable for bounded controls can be obtained by using only the extreme values of the controls. It holds true for BLS in which all matrices commute. The computation of time-optimal controls, as well as the rst attempts at studying controllability (17), assumed that one could simplify their solution to the problem of nding optimal switching times (open-loop) or nding hypersurfaces on which the control would switch values. However, early on it was discovered by Sussmann (35) that the bang-bang principle did not apply to bilinear systems in general. A value between the extremes may be required. There are examples of simply formulated optimal control problems for which the control may have to switch innitely often in a nite time interval. MATRIX GROUPS The set of all n n nonsingular matrices equipped with the usual matrix multiplication, identity I , and inverse constitutes a group, called GL(n , R) where GL stands for general linear. A subgroup of GL(n ,R) is called a matrix group ; the matrix groups we need for BLS are matrix Lie groups . The derivation of the results of References 17 and 23 other nonlocal results for BLS depends not only on Lie algebras, but also the corresponding matrix Lie groups. Begin by considering the k -input symmetric homogeneous BLS of equation 5, where it is assumed that the matrices B i are linearly independent. Again write u = (u1 , . . . , uk ). The Lie algebra of this BLS is {B1 , . . . , Bk }LA . We let { } be the set of transition matrices for this BLS, i.e., the solutions of the matrix system
k
positive determinants, { } actually lies in GL+ (n, R). On the other hand, corresponding to any matrix Lie algebra g, a Lie group we shall call G (g) can be constructed, consisting of all products of exponentials of matrices in g; see Reference 16 or Rossmanns book in the Reading List. If a basis of g is {C1 , . . . , Cl }, in a neighborhood of the identity G (g) has coordinates {exp(C1 t1 ), . . . , exp(C1 tl )} (t1 , . . . , tl ). (Thus, g is the tangent space to G at I .) Using elements of G to translate this coordinate patch anywhere on the group, it can be observed that G (g) has an atlas of coordinate charts that are related by differentiable transformations where they overlap, like a geographic atlas, and on which the group multiplication and inverse are differentiable functions. Lie groups occur in many applications of mathematics to classical mechanics, quantum mechanics, chemistry, and the control of robots and aerospace vehicles. At this point we can note that the mathematics of controllability for symmetric BLS is rather simple. If a matrix Lie group G has the property that given any two states x and z in Rn 0 , X G exists such that Xx = z, then G is called transitive on Rn 0 and g is a transitive Lie algebra. In Reference 20, it was shown that G({B1 , . . . , Bk }LA ) = { }; that is, all matrices in the Lie group can be obtained as transition matrices. This is a simple version of the Chow Theorem of nonlinear control. Its meanings for BLS are that, once the Lie algebra has been identied, the structure of the group of transition matrices is completely known; and that any matrix M G({B1 , . . . , Bk }LA ) can be written as a product M = et1 B1 es1 B2 . . . etk B1 esk B2 for some nite sequence of reals {t1 , s1 , . . . , tk , sk }. Thus, a few generators are as good as l of them. Some examples of Lie groups are useful; in some of those listed below, the identity det(eA ) = etr(A) is relevant: Lie algebra g gl(n, R) = Mn sl(n, R) = {X|tr(X) = 0} so(n) = {X|X = X} G GL(n, R) GL+ (n, R) = {Q|det(Q) > 0} SL(n, R) = {Q|det(Q) = 1} SO(n) = {Q|Q Q = I }
and if J 2 = I and is irrational, GL(4, R) {X|X = diag {J, J } M4 } R is a densely wound curve that lls up a 2-torus, where R so it is not a closed (Lie) subgroup of GL (4, R). Any element Z G is a product of exponentials of elements of the corresponding Lie algebra g but not necessarily an exponential of any such element; the standard counterexample is Z = diag(e, e1 ) SL(2, R). Here I = exp(J ), diag(e, e1 ) = exp(diag(1, 1)), but their product Z is not the exponential of any real matrix. = (A + uB)x, in GL(n, R) the For asymmetric systems x set of transition matrices S = { (u; t, 0)|u U, t 0} contains I and is closed under multiplication but not under matrix inversion; it is a matrix semigroup . Example I showed an uncontrollable system for which S is a semigroup but with the accessibility property. Here is another two-dimensional example with bounded controls, showing that controllability and stabilizability are possible for semigroups acting on R2 0.
=
1
ui Bi ) ,
(0, 0
=I
(18)
As { } contains I and is closed under composition [eq. 8a] and inverse [eq. 8b], we see that it constitutes a group, in fact a subgroup of GL(n ,R). As all matrices in { } have
12
Bilinear Systems
Figure 1. Phase portrait and path from p to q for Example III. From p use u = 1 to reach the x 2 axis; then use u = 1; switch to u = 1 at the arc of the spiral that leads to q.
Example III. Consider this BLS with u = 1 and 0.5 > > 0:
= Ax + uBx; A = x AB = 1 + 1 1 1 , , B= 1 1 ; 1 1 2 A+B = 2
( ) is not of the form 2ki for any pair of eigenvalues , of A and integer k . However, Sontag (36) has shown that for a controllable BLS (eq. 4), the sampled system (eq. 20) will be controllabile if this condition is imposed: ( + ) = 2k, k = 0 for any four eigenvalues of A . The discretization of an uncontrollable BLS can be artifactually controllable,. depending on the BLS and the numerical method used. For the Euler discretization shown in equation 6, here is a two-dimensional example. The BLS = u(2J + I )x, which has A = 0, B = 2J + I , and is not is x controllable. The system can move back and forth along one spiral trajectory through x(0). The discrete-time approxi2 mation is xs = x + uBx. This DBLS is controllable on R0 ; the trajectories move on the tangent lines to the spiral. Control and optimal control problems for DBLS xs = Ax + Bxu are investigated in Swamy and Tarn (37); in the optimal control area, the case that B has rank one is notable, because it can be reduced to a linear system problem by the factorization B = bc . Perform the optimization, with whatever cost, for the linear system xs = Ax + vb and obtain the optimal control v then in the DBLS, let u = v /(c x), so xs = Ax + bc x v cx
2+ 0 0 2
(19)
As tr(A B) = 2, for either choice of control this is not a stable system. Paths with u = 1 are spirals that provide rotations. With u = 1 the paths are hyperbolas, which near the coordinate axes permit movement toward the origin x2 -axis) or away from it (x1 -axis). System l9 is controllable, because dilations and rotations of arbitrary extent are possible. Furthermore, system 19 satises the conditions tr B = 0, tr A > 0, and tr AB = 0 of Th. 2.2.3 of Reference 33, so it can be stabilized with a feedback control u(x) = x Rx/x x. Other Aspects of Bilinear System Theory. Discrete-Time Systems. There is a large literature on DBLS on most of the topics we have covered, sometimes employing very different mathematical tools and sometimes highly parallel to continuous-time systems. To emphasize this we use a successor (time-shift) notation; instead of x(t + 1) = (A + uB)x(t ), for DBLS, we use xs = (A + uB)x. The observability theory of discrete-time bilinear systems is very much like that of the continuous version, with the same criteria, e.g., equation 11 to assure observability for all u . Inverting an observaole discrete-time system requires no time-derivatives of the output y = c x. When the DBLS is obtained by sample-and-hold operations with interval on a continuous-time system like equation 4, the sampled system is xs = e(A+uB) (20)
when c x = 0, and perturb the computation slightly to avoid its vanishing. This division controller reects the fact that the discrete motion can avoid the hyperplane c x = 0. Controllability with bounded controls is associated (as it is in continuous-time systems) with zero-control motion on an ellipsoid and an accessibility condition. For the DBLS xs = Ax + uBx, Reference 24 has a DBLS version of the adcondition obtained from the Jacobian of the map from the input history to output history rank{Ax, Am Bx, Am1 BAx, . . . , ABAm1 x, BAm x} = n
2 on Rn 0 , where m need be no larger than n 1. When the rank condition holds, the condition for controllability with controls bounded by any positive constant is that A is similar to an orthogonal matrix. As a partial converse, it was shown that requiring controllability for inputs with arbitrarily small bounds implies the spectrum of A lies on the unit circle, and that the DBLS ad-condition is sufcient for controllability when some power of A is orthogonal. The solution of a DBLS xs = (A + uB)x is given by the right-to-left ordered product
(u; t, 0)x(0) = (
0 k = t (A
+ u(k)B))x(0)
The set of states attainable from x(0) may be pathological. For example take A= cos() sin() , B=I sin() cos()
If the BLS is observable to begin with, observability is preserved when the sampling interval is sufciently small; the condition (the same as for linear systems) is that
For rational, S is a nite set of rays and for irrational, a dense set of rays. For linear systems, the realization of an inputoutput map as a system on a state space is essentially the same in continuous and discrete time, using the concept of Hankel
Bilinear Systems
13
matrix
cb c Ab . . .
c Ab c A2 b . . .
c A2 b c A3 b . . .
. . .
maps is surveyed in Reference 1. Volterra series are used extensively in the identication of nonlinear systems, especially in biological work, because algorithms exist to evaluate the rst few terms purely from inputoutput data when no state-space model is known. The series is y(t ) = W0 (t ) + n=1 0 0 n1 Wn (t, 1 , . . . , n )u(1 ) u(n )d1 dn
t
For discrete-time bilinear systems, realization theory involves a different looking but highly analogous Hankel matrix; Isidori (38), Fliess (39), and Tarn and Nonoyama (40) introduced variations on this idea. The situation differs from linear systems in that the inputputput map is u c (u; k, 0), with the single output a row vector, or it may be multiplied on the right by the initial state. Discretetime bilinear systems were then, like linear systems, used as examples of adjoint systems in category theory, with the Hankel matrix being the fundamental description. A category, loosely speaking, is a collection of objects and maps between them; for instance, the category Lin has linear spaces and linear maps. A machine (adjoint system) in a category has input, state, and output objects; dynamics (iterated maps on the state space) and input and output maps; and Hankel matrices. The category of discrete sets underlies automata theory; Lin leads to discrete-time linear system theory, etc., and discrete-time bilinear systems occur naturally; see Arbib and Manes (41). Continuoustime dynamics does not t into category theory. Transforming Nonlinear Systems to Bilinear Form. As an application of BLS ideas to nonlinear control theory, consider the nonlinear system z 1 z 2 = z1 + (z2 + z2 1 )u = z2 + 3z2 + ( z 2z1 z2 2z3 1 1 1 )u
= Ax + u(t )Bx, as if it were inhoTreat our usual system x mogeneous to get the integral equation
t
x(t ) = eAt x(0) +

0
u(t1 )eA(t t1 ) Bx(t1 )dt1
then the Volterra kernels are easily observed, by iteration, to be = c eAt x(0) W0 (t ) W1 (t, 1 ) = c eA(t 1 ) BeA1 x(0) W2 (t, 1 , 2 ) = c eA(t 1 ) BeA(1 2 ) BeA2 x(0) For bounded u , the series converges on any time interval and represents the solution. The Volterra series has a nite number of terms precisely when the systems Lie algebra g is nilpotent, which means that all brackets with a sufciently high number of factors must vanish. Several approximation theorems for analytic nonlinear systems have been based on this approach. For an account of this theory in its general form for analytic systems, see Isidori (43); the book is also a source for many other topics omitted here, such as Fliess functional expansions and continuous-time realization theory. Also see NONLINEAR CONTROL SYSTEMS: ANALYTICAL METHODS. Systems on Lie Groups; Quantum Systems. The transition matrices for Eq. (14) satisfy
k
This was cooked by using the one-to-one coordinate trans = Ax + uBx formation x1 = z1 , x2 = z2 + z2 1 , from the BLS x where A= 1 0 0 , B= 1 0 1 1 0
=A
+
i= 1
Bi
(20)
The Lie algebra of the BLS is sl(2, R). The matrices A ,B are the Jacobians at 0 of the nonlinear vector functions in the rst system, so the BLS is also the classic linearization of the nonlinear system, but what we have here is not an approximation but a system equivalence. The Lie algebra of nonlinear vector elds (see CONTROLLABILITY AND OBSERVABILITY, Section Nonlinear Finite-Dimensional Systems) generated by our example is also sl(2, R). It has been shown by Sedwick and Elliott (42) that given a family F of real-analytic nonlinear vector elds on Rn that vanish at 0, if F and the family F1 of their linear terms isomorphically generate a transitive Lie algebra, then a realanalytic coordinate transformation exists that transforms F system to its equivalent BLS F1 , and that can be found by solving linear partial differential equations. Of Lie algebras not on that list, the compact and the semisimple ones also permit this linearization, but all others either fail to have differentiable transformations or do not have any, as far as is known; and nonlinear systems axe prone to have innite-dimensional Lie algebras. A Note on Volterra Series. The connection between BLS and the Volterra series representation of inputoutput
with the initial condition (0) = I . It is also worthwhile to look at equation 20 as a control system on GL(n ,R) or on one of its Lie subgroups such as (if A and the B i are skew-symmetric) SO(3), the group of rigid-body rotations. Such control systems are also called bilinear and have inspired much work such as References 20 and 26 and especially Reference 34. The controllability problem for systems on Lie groups is closely related to the study of matrix semigroups; see Lawson (44) for a survey of this area. Typical applications have been to the angular attitude control of spacecraft (satellites) and undersea vehicles. A new application in which electronic engineers have been active is in quantum control; see QUANTUM SYSTEMS. DAlessandro and Dahleh (45) and much subsequent work by DAlessandro on quantum bits has made the study of BLS on complex Lie groups an attractive subject of study. BIBLIOGRAPHY
1. Bruni, C.; DiPillo, G.; Koch, G., Bilinear Systems: An Appealing Class of Nearly Linear Systems in Theory and Application. IEEE Trans. Automat. Contr. 1974, AC-19, pp 334348.
14
Bilinear Systems 26. Jurdjevic, V.; Kupka, I., Control Systems Subordinated to a Group Action: Accessibility. J. Diff. Equations 1981, 39, pp 186211. 27. Jurdjevic, V.; Sallet, G., Controllability Properties of Afne Systems. SIAM J. Contr. Optim. 1984, 22, pp 501508. 28. Boothby, W. M., Some Comments on Positive Orthant Controllability of Bilinear Systems. SIAM J. Contr. Optim. 1982, 20, pp 634644. 29. Bacciotti, A., On the Positive Orthant Controllability of TwoDimensional Bilinear Systems. Syst. Contr. Lett. 1983, 3, pp 5355. 30. Sachkov, Y. L., On Positive Orthant Controllability of Bilinear Systems in Small Codimensions. SIAM J. Contr. Optim. 1974, 35, pp 2935. 31. Agrachev, A. A.; Liberzon, D., Lie-Algebraic Stability Criteria for Switched Systems. SIAM J.Contr. Optim 2001, 40, pp 253259. 32. Bacciotti, A.; Boieri, P., A Characterization of Single-Input Planar Bilinear Systems which Admit a Smooth Stabilizer. Syst. Contr. Lett. 1991, 16, pp 139144. 33. Oster, G., Bilinear Models in Ecology. In Recent Developments in Variable Structure Systems, Economics and Biology;Mohler, R. R.;Ruberti, A., Eds., Springer-Verlag: Berlin, 1978; pp 260271. 34. Jurdjevic, V., Geometric Control Theory; Cambridge University Press: New York, 1996. 35. Sussmann, H. J., The Bang-Bang Problem for Certain Control Systems in GL (n,R). SIAM J. Contr. Optim. 1972, 10, pp 470476. 36. Sontag, E. D., A Chow Property for Sampled Bilinear Systems. In Analysis and Control of Nonlinear Systems;Byrnes, C. I.;Martin, C. F.;Sacks, R., Eds.; North Holland: Amsterdam, 1988, pp 205211. 37. Swamy, K. N.; Tarn, T.-J., Deterministic and Stochastic Control of Discrete-Time Bilinear Systems. AutomaticaJ. IFAC 1979, 15, pp 677682. 38. Isidori, A., Direct Construction of Minimal Realizations from Nonlinear Input-Output Maps. IEEE Trans. Autom. Contr. 1973, AC-18, pp 626631. 39. Fliess, M., Matrices de Hankel. J. Math. Pures Appl. 1974, 53, pp 197224. 40. Tarn, T.-J.; Nonoyama, S., Realization of Discrete-Time Internally Bilinear Systems; Proc. IEEE Conf. Decision and Control; Clearwater, FL, 1977, pp 125133. 41. Arbib, M. A.; Manes, E. G., Foundations of System Theory: The Hankel Matrix. J. Comp. Sys. Sci. 1980, 20, pp 330378. 42. Sedwick, J. L.; Elliott, D. L., Linearization of Analytic Vector Fields in the Transitive Case. J. Diff. Equations 1977, 25, pp 377390. 43. Isidori, A., Nonlinear Control Systems, 3rd ed.; SpringerVerlag: London, 1995. 44. Lawson, J., Germetric Control and Lie Semigroup Theory. In Differential Geometry and Control, (Proc. Symp. Pure Math. Vol. 64);Ferreyra, G.;Gardner, R.;Hermes, H.;Sussmann, H., Eds.; Amer. Math. Soc.: Providence, RI, 1999. 45. DAlessandro, D.; Dahleh, M., Optimal Control of Two Level Quantum Systems. IEEE Trans. Autom. Contr. 2001, 46, pp 866876.
2. Mohler, R. R., Bilinear Control Processes; Academic: New York, 1973. 3. Mohler, R. R., Nonlinear Systems: Volume 1, Dynamics and Control, Volume II, Applications to Bilinear Control; PrenticeHall, Englewood Cliffs, NJ: 1991. 4. Sontag, E. D., Mathematical Control Theory; Springer-Verlag: New York; 1990. 5. Golub, G. H.; Van Loan, C. F., Matrix Computations, 2nd ed.; Baltimore, NJ, Johns Hopkins: 1989. 6. Ledzewicz, U.; Schattler, H., Optimal Bang-Bang Controls for a Two Compartment Model in Cancer Chemotherapy. J. Optim. Theory Appl. 2002, 114, pp 609637. 7. Celikovsk y, S.; Van ec ek, A., Bilinear Systems and Chaos. Kybernetika (Prague) 1994, 30, pp 403424. 8. Chabour, R.; Sallet, G.; Vivalda, J. C., Stabilization of Nonlinear Systems: A Bilinear Approach. Math. Contr. Signals Syst. 1993, 6, pp 224246. 9. Luesink, R.; Nijmeijer, H., On the Stabilization of Bilinear Systems via Constant Feedback. Linear Algebra Appl. 1989, 122124, pp 457474. 10. Kailath, T., Linear Systems; Prentice-Hall: Englewood Cliffs, NJ, 1980. 11. Grasselli, O. M.; Isidori, A., Deterministic State Reconstruction and Reachability of Bilinear Control Processes; Proc. Joint Automatic Control Conf. San Francisco, June 2225, 1977 ; IEEE: New York, pp 14231427. 12. Williamson, D., Observation of Bilinear Systems with Application to Biological Control. Automatica: J. IFAC 1977, 13, pp 243254. 13. Frelek, B. A.; Elliott, D. L., Optimal Observation for VariableStructure Systems; Proc. VI IFAC Congress, Boston, Mass. ; vol I, Paper 29.5, 1975. 14. Grasselli, O. M.; Isidori, A., An Existence Theorem for Observers of Bilinear Systems. IEEE Trans. Autom. Contr. 1981, AC-26, pp 12991300;Erratum 1982, AC-27, 284. 15. Sen, P., On the Choice of Input for Observability in Bilinear Systems. IEEE Trans. Autom. Contr. 1981, AC-26, pp 451 454. 16. Belinfante, J. G. F.; Kolman, B., A Survey of Lie Groups and Lie Algebras with Applications and Computational Methods; Philadelphia: SIAM, 1972. 17. Ku cera, J., Solution in Large of Control Problem x = (A(1 u) + Bu)x. Czechoslovak Math. J. 1966, 16, pp 600623. 18. Boothby, W. M.; Wilson, E. N., Determination on the Transitivity of Bilinera System. SIAM J. Contr. Optim. 1979, 17, pp 212221. 19. Kramer, L., Twotransitive Lie groups. J. Reine Angew Math. 2003, 563, pp 83113. 20. Brockett, R. W., System Theory on Group Manifolds and Coset Spaces. SIAM J. Contr. 1972, 10, pp 265284. 21. Hirschorn, R. M., Invertibility of Control Systems on Lie Groups. SIAM J. Contr. Optim. 1977, 15, pp 10341049. 22. Elliott, D. L., A Consequence of Controllability. J. Diff. Equations 1971, 10, pp 364370. 23. Rink, R. E.; Mohler, R. R., Completely Controllable Bilinear Systems. SIAM J. Contr. 1968, 6, pp 477486. 24. Cheng, G.-S. J.; Tarn, T.-J.; Elliott, D. L., Controllability of Bilinear Systems. In Variable Structure Systems with Applications to Economics and Biology;Roberti, A.;Mohler, R. R., Eds.; Springer-Verlag: Berlin, 1975; pp 83100. 25. Jurdjevic, V.; Quinn, J., Controllability and Stability. J. Diff. Equation 1978, 28, pp 381389.
Bilinear Systems
15
Further Reading
The following books and articles are listed more or less in the order of the topics of this article.
Lipschutz, S., Schaums Outline of Linear Algebra, 2nd ed.; McGraw-Hill: New York, 1991. (Fine self-study text for students and as a refresher; the accounts of the CayleyHamilton theorem, the Jordan canonical form, and quadratic forms will be helpful in studying BLS.) Bellman, R., Introduction to Matrix Analysis SIAM: Philadelphia, PA, 1995. (This is a reprint of the 1965 Second Edition, by popular demand, a true classic. Covers exponential matrices, positive matrices, and much more.) Rossmann, W., Lie Groups: An Introduction Through Linear Groups, ser., Oxford Grad. Texts Math., 5; Oxford University Press: Oxford: 2005. (This books emphasis on matrix groups makes it well suited for the study of bilnear control systems.) Varadarajan, V. S., Lie Groups, Lie Algebras, and Their Representations (GTM 102); Springer-Verlag: New York: 1988. (Wellknown and respected graduate text.) Jurdjevic, V.; Sussmann, H. J., Controllability on Lie Groups. J. Diff. Equations 1972, 12, pp 313329.(A foundational paper on modern nonlinear control. State space generalized to groups and group manifolds.) Jurdjevic, V., Geometric Control Theory; Cambridge University Press: New York, 1996. Eilliott, D. L., Bilinear Control Systems; Kluwer Academic Publishers: Dordrecht. In Press.
DAVID L. ELLIOTT
University of Maryland
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Chaos, Bifurcations, and Their Control Standard Article Guanrong Chen1 1University of Houston, Houston, TX Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1007 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (471K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Nonlinear Dynamics Chaos Bifurcations Controlling Chaos Controlling Bifurcations Anticontrol of Chaos About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
194
CHAOS, BIFURCATIONS, AND THEIR CONTROL

NONLINEAR DYNAMICS Unlike linear systems, many nonlinear dynamical systems do not show orderly, regular, and long-term predictable responses to simple inputs. Instead, they display complex, random-like, seemingly irregular, yet well-dened output behaviors. This dynamical phenomenon is known as chaos. The term chaos, originating from the Greek word , was designated as the primeval emptiness of the universe before things came into being of the abyss of Tartarus, the underworld. . . . In the later cosmologies Chaos generally designated the original state of things, however conceived. The modern meaning of the word is derived from Ovid, who saw Chaos as the original disordered and formless mass, from which the maker of the Cosmos produced the ordered universe (1). There also is an interpretation of chaos in ancient Chinese literature, which refers to the spirit existing in the center of the universe (2). In modern scientic terminology, chaos has a fairly precise but rather complicated denition by means of the dynamics of a generally nonlinear system. For example, in theoretical physics, chaos is a type of moderated randomness that, unlike true randomness, contains complex patterns that are mostly unknown (3). Bifurcation, as a twin of chaos, is another prominent phenomenon of nonlinear dynamical systems: Quantitative change of system parameters leads to qualitative change of system properties such as the number and the stability of system response equilibria. Typical bifurcations include transcritical, saddle-node, pitchfork, hysteresis, and Hopf bifurcations. In particular, period-doubling bifurcation is a route to chaos. To introduce the concepts of chaos and bifurcations as well as their control (4,5), some preliminaries on nonlinear dynamical systems are in order. Nonlinear Dynamical Systems A nonlinear system refers to a set of nonlinear equations, which can be algebraic, difference, differential, integral, functional, and abstract operator equations, or a certain combination of these. A nonlinear system is used to describe a physical device or process that otherwise cannot be well dened by a set of linear equations of any kind. The term dynamical system is used as a synonym of mathematical or physical system, in which the output behavior evolves with time and/or other varying system parameters (6). In general, a continuous-time dynamical system can be described by either a differential equation = f (x , t ; p ), x or a map
F : x g ( x , t ; p ), t [t0 , ) (2)
which have explicit formulation for a specied physical system. For the discrete-time setting, a nonlinear dynamical system is described by either a difference equation x k+ 1 = f ( x k , k; p ), or a map F : x k g (x k , k; p ), k = 0, 1, . . . (4) k = 0, 1, . . . (3)
where notation is similarly dened. Repeatedly iterating the discrete map F backward yields x k = F ( x k 1 ) = F (F ( x k 2 )) = = F k ( x 0 ) where the map can also be replaced by a function, f , if the system is given via a difference equation, leading to xk = f f ( x0 ) = f k ( x0 )
k times
where denotes composition operation of functions or mappings. The dynamical system of Eq. (1) is said to be nonautonomous when the time variable, t, appears separately in the system function f (e.g., a system with an external time-varying force input); otherwise, it is said to be autonomous and is expressed as = f (x ; p ), x Classication of Equilibria For illustration, consider a general two-dimensional autonomous system t [t0 , ) (5)
x = f (x, y ) y = g(x, y )
(6)
t [t0 , )
(1)
with given initial conditions (x0, y0), where f and g are two smooth nonlinear functions that together describe the vector eld of the system. The path traveled by a solution of Eq. (6), starting from the initial state (x0, y0), is a solution trajectory, or orbit, of the system and is sometimes denoted by t(x0, y0). For autonomous systems, two different orbits will never cross each other (i.e., never intersect) in the x-y plane. This x-y coordinate plane is called the (generalized) phase plane (phase space in the higher-dimensional case). The orbit family of a general autonomous system, corresponding to all possible initial conditions, is called solution ow in the phase space. Equilibria, or xed points, of Eq. (6), if they exist, are the solutions of two homogeneous equations: f (x, y ) = 0 and g(x, y ) = 0
where x x(t) is the state of the system, p is a vector of variable system parameters, and f and g are continuous (or differentiable) nonlinear functions of comparable dimensions,
An equilibrium is denoted by (x, y). It is stable if all the nearby orbits of the system, starting from any initial conditions, approach it; it is unstable if the nearby orbits are moving away from it. Equilibria can be classied, according to their stabilities, as stable or unstable node or focus, saddle
195
Stable node
Unstable node
Stable focus
x
Figure 1. Classication of two-dimensional equilibria: Stabilities are determined by Jacobian eigenvalues.
Unstable focus
Saddle
Center
point and center, as summarized in Fig. 1. The type of equilibria is determined by the eigenvalues, 1,2, of the system Jacobian
J :=
fx gx
fy gy
with f x : f / x, f y : f / y, and so on, all evaluated at (x, y). If the two Jacobian eigenvalues have real parts R 1,2 0, the equilibrium (x, y) at which the linerization was taken, is said to be hyperbolic. Theorem 1 (Grobman-Hartman) If (x, y) is a hyperbolic equilibrium of the nonlinear dynamical system of Eq. (6), then the dynamical behavior of the nonlinear system is qualitatively the same as (i.e., topologically equivalent to) that of its linearized system,
some constant tp 0. The minimum value of such tp is called the ( fundamental) period of the periodic solution, while the solution is said to be tp-periodic. A limit cycle of a dynamical system is a periodic solution of the system that corresponds to a closed orbit in the phase space and possesses certain attracting (or repelling) properties. Figure 2 shows some typical limit cycles for the two-dimensional case: (a) an inner limit cycle, (b) an outer limit cycle, (c) a stable limit cycle, (d) an unstable limit cycle, and (e) and (f) saddle limit cycles. Limit Sets and Attractors The most basic problem in studying the general nonlinear dynamical system of Eq. (1) is to understand and/or to solve for the system solutions. The asymptotic behavior of a system solution, as t , is called the steady state of the solution, while the solution trajectory between its initial state and the steady state is the transient state. For a given dynamical system, a point x in the state space is an -limit point of the system state orbit x(t) if, for every open neighborhood U of x, the trajectory of x(t) will enter U at a (large enough) value of t. Consequently, x(t) will repeatedly enter U innitely many times, as t . The set of all such -limit points of x(t) is called the -limit set of x(t) and is denoted x. An -limit set of x(t) is attracting if there exists an open neighborhood V of x such that whenever a system orbit enters V then it will approach x as t . The basin of attraction of an attracting set is the union of all such open neighborhoods. An -limit set is repelling if the nearby system orbits always move away from it. An attractor is an limit set having the property that all orbits nearby have it as their -limit sets. For a given map, F, and a given initial state, x0, an -limit set is obtained from the orbit Fk(x0) as k . This -limit set, x, is an invariant set of the map, in the sense that F(x) x. Thus, -limit sets include equilibria and periodic orbits. Maps Poincare Assume that the general n-dimensional nonlinear autonomous system of Eq. (5) has a tp-periodic limit cycle, , and let
x x =J y y
in a neighborhood of the equilibrium (x, y). This theorem guarantees that for the hyperbolic case, one can study the linearized system instead of the original nonlinear system, with regard to the local dynamical behavior of the system within a (small) neighborhood of the equilibrium (x, y). In other words, there exist some homeomorphic maps that transform the orbits of the nonlinear system into orbits of its linearized system in a (small) neighborhood of the equilibrium. Here, a homeomorphic map (or a homeomorphism) is a continuous map whose inverse exists and is also continuous. However, in the nonhyperbolic case the situation is much more complicated, where such local dynamical equivalence does not hold in general. Periodic Orbits and Limit Cycles A solution orbit, x(t), of the nonlinear dynamical system of Eq. (1) is a periodic solution if it satises x(t tp) x(t) for
196
(a)
(b)
(c)
Figure 2. Periodic orbits and limit cycles.
(d)
(e)
(f)
x* be a point on the limit cycle and be an (n 1)-dimensional hyperplane transversal to at x*, as shown in Fig. 3. Here, the transversality of to at x* means that and the tangent line of at x* together span the entire n-dimensional space (hence, this tangent line of cannot be tangent to at x*). Since is tp-periodic, the orbit starting from x* will return to x* in time tp. Any orbit starting from a point, x, in a small neighborhood U of x* on will return and hit at a point, denoted P(x), in the vicinity V of x*. Therefore, a map P : U V can be uniquely dened by , along with the solution ow of the autonomous system. This map is called the Poincare map associated with the system and the cross sec tion . For different choices of the cross section , Poincare map is only maps are similarly dened. Note that a Poincare locally dened and is a diffeomorphism namely, a differentiable map that has an inverse and the inverse is also differentiable. If a cross section is suitably chosen, the orbit will repeatedly return and pass through the section. The Poincare map together with the rst return orbit is particularly impor map. Poincare tant, which is called the rst return Poincare maps can also be dened for nonautonomous systems in a similar way, where, however, each return map depends on the initial time in a nonuniform fashion.
Homoclinic and Heteroclinic Orbits Let x* be a hyperbolic equilibrium of a diffeomorphism P : Rn Rn, which can be of either unstable, center, or saddle type; t(x) be a solution orbit passing through x*; and x* be the -limit set of t(x). The stable manifold of x*, denoted Ms, is the set of such points x* that satisfy t(x*) x* as t ; the unstable manifold of x*, Mu, is the set of points x* satisfying t(x*) x* as t . Suppose that s(x*) and u(x*) are cross sections of the stable and unstable manifolds of t(x), respectively, which intersect at x*. This intersection always includes one constant orbit, t(x) x*. A nonconstant orbit lying in the intersection is called a homoclinic orbit and is illustrated in Fig. 4(a). For two equilibria, x1 x2, of either unstable, center, or saddle type, an orbit lying in s(x1) u(x2), or in u(x1) s(x2), is called a heteroclinic orbit. A heteroclinic orbit is dipicted in Fig. 4(b), which approaches one equilibrium as t but converges to another equilibrium as t . It is known that if a stable and an unstable manifold intersect at a point, x0 x*, then they will do so at innitely many points, denoted xkk counted in both forward and backward directions, which contains x0. This sequence, xk, is a homo-
P (x) x x*
map and cross Figure 3. Schematic illustration of the Poincare section.
x*
x1
x2
(a)
(b)
Figure 4. Schematic illustration of homoclinic and heteroclinic orbits.
197
ilinkov-type homoclinic orbit. Figure 5. Illustration of a S
and, by changing variables if necessary, assume the origin, x 0, is an equilibrium of the system satisfying f (0, t) 0. Lyapunov stability theory concerns various stabilities of the zero equilibrium of Eq. (7). Stability in the Sense of Lyapunov. The equilibrium x 0 of Eq. (7) is said to be stable in the sense of Lyapunov if for any 0 and any initial time t0 0, there exists a constant, (, t0) 0, such that x (t0 ) < x (t ) < , t t0 (8)
clinic orbit in which each xk is called a homoclinic point. This special structure is called a homoclinic structure, in which the two manifolds usually do not intersect transversally. Here, two manifolds are said to intersect transversally if their tangent planes together span the entire space (hence, these two tangent planes cannot coincide at the intersection). This structure is unstable in the sense that the connection can be destroyed by very small perturbations. If they intersect transversally, however, a transversal homoclinic point will imply innitely many other homoclinic points. This eventually leads to a picture of stretching and folding of the two manifolds. Such complex stretching and folding of manifolds are key to chaos, which generally implies the existence of a complicated Smale horseshoe map and is supported by the following mathematical theory (7). Theorem 2 (Smale-Birkhoff) Let P : Rn Rn be a diffeomorphism with a hyperbolic equilibrium x*. If the cross sections of the stable and unstable manifolds, s(x*) and u(x*), intersect transversally at a point other than x*, then P has a horseshoe map embedded within it. For three-dimensional autonomous systems, the case of an equilibrium with one real eigenvalue, , and two complex conjugate eigenvalues, j, is especially interesting. For ex ilnikov type of ample, the case with 0 and 0 gives a S homoclinic orbit, which is illustrated in Fig. 5. ilnikov) Let t be the solution ow of a Theorem 3 (S ilnikovthree-dimensional autonomous system that has a S type homoclinic orbit . If , then t can be extremely , slightly perturbed to t, such that t has a homoclinic orbit map dened by a near , of the same type, and the Poincare , has a countable set of Smale cross section, transversal to horseshoes. Stabilities of Systems and Orbits Stability theory plays a central role in both dynamical systems and automatic control. Conceptually, there are different types of stabilities, among which Lyapunov stabilities and the orbital stability are essential for chaos and bifurcations control. Lyapunov Stabilities. In the following discussion of Lyapunov stabilities for the general nonautonomous system of Eq. (1), the parameters are dropped since they do not affect the concept and consequence. Thus, consider the general nonautonomous nonlinear system x = f (x , t ), x (t0 ) = x 0 (7)
Here and throughout, denotes the standard Euclidean norm of a vector. It should be emphasized that the constant in the preceeding equation generally depends on both and t0. It is particularly important to point out that, unlike autonomous systems, one cannot simply assume the initial time t0 0 for a nonautonomous system in a general situation. The stability is said to be uniform, with respect to the initial time, if this constant, (), is indeed independent of t0 over the entire time interval of interest. Asymptotic Stability. In both theoretical studies and applications, the concept of asymptotic stability is of most importance. The equilibrium x 0 of Eq. (7) is said to be asymptotically stable if there exists a constant, (t0) 0, such that x (t0 ) < x (t ) 0 as t (9)
This asymptotical stability is said to be uniform if the existing constant is independent of t0, and is said to be global if the convergence (x 0) is independent of the starting point x(t0) over the entire domain on which the system is dened (e.g., when ). Orbital Stability. The orbital stability differs from the Lyapunov stabilities in that it concerns the structural stability of a system orbit under perturbation. Let t(x0) be a tp-periodic solution of the autonomous system x (t ) = f (x ), x (t0 ) = x 0 (10)
and let be the closed orbit of t(x0) in the phase space namely, = { y | y = t ( x 0 ), 0 t < tp }
The solution trajectory t(x0) is said to be orbitally stable if for any 0 there exits a () 0, such that for any x 0 satisfying 0 , ) := inf x 0 y < d (x
y
the solution t(x 0) of the autonomous system satises 0 ), ) < , d (t ( x t t0
Lyapunov Stability Theorems. Two cornerstones in the Lyapunov stability theory for dynamical systems are the Lyapunov rst (or indirect) method and the Lyapunov second (or direct) method.
198
The Lyapunov rst method, known also as the Jacobian or local linearization method, is applicable only to autonomous systems. This method is based on the fact that the stability of an autonomous system, in a neighborhood of an equilibrium, is essentially the same as its linearized model operating at the same point, and under certain conditions local system stability behavior is qualitatively the same as that of its linearized model (in some sense, similar to the Grobman-Hartman theorem). The Lyapunov rst method provides a theoretical justication for applying linear analysis and linear feedback controllers to nonlinear autonomous systems in the study of asymptotic stability and stabilization. The Lyapunov second method, on the other hand, which originated from the concept of energy decay (i.e., dissipation) associated with a stable mechanical or electrical system, is applicable to both autonomous and nonautonomous systems. Hence, the second method is more powerful, also more useful, for rigorous stability analysis of various complex dynamical systems. For the general autonomous system of Eq. (10), under the assumption that f : D Rn is continuously differentiable in a neighborhood, D , of the origin in Rn, the following theorem of stability for the Lyapunov rst method is convenient to use. Theorem 4 (Lyapunov First Method) (for continuoustime autonomous systems) In Eq. (10), let J0 = f x
Theorem 6 (Lyapunov Second Method) (for continuoustime autonomous systems) Let x 0 be an equilibrium for the autonomous system of Eq. (10). This zero equilibrium is globally (over the domain D Rn containing the origin) and asymptotically stable if there exists a scalar-valued function V(x) dened on D such that (x) 0 (1) V(0) 0; (2) V(x) 0 for all x 0 in D ; and (3) V for all x 0 in D . In the preceding two theorems, the function V is called a Lyapunov function, which is generally not unique for a given system. Similar stability theorems can be established for discrete-time systems (by properly replacing derivatives with differences). To this end, it is important to remark that the Lyapunov theorems only offer sufcient conditions for determining the asymptotic stability. Yet the power of the Lyapunov second method lies in its generality: It works for all kinds of dynamical systems (linear and nonlinear, continuous-time and discrete-time, autonomous and nonautonomous, time-delayed, functional, etc.), and it does not require any knowledge of the solution formula of the underlying system. In a particular application, the key is to construct a working Lyapunov function for the system, which can be technically difcult if the system is higher-dimensional and structurally complicated. CHAOS
x =x = 0
be the Jacobian of the system at the equilibrium x 0. Then (1) x* 0 is asymptotically stable if all the eigenvalues of J0 have negative real parts; and (2) x 0 is unstable if one of the eigenvalues of J0 has a positive real part. Note that the region of asymptotic stability given in this theorem is local. It is important to emphasize that this theorem cannot be applied to nonautonomous systems in general, not even locally. For the general nonautonomous system of Eq. (7), the following criterion can be used. Let K ={ g(t ) : g(t0 ) = 0, g(t ) is continuous and nondecreasing on [t0 , )}
Nonlinear systems have various complex behaviors that would never have been anticipated in (nite-dimensional) linear systems. Chaos is a typical behavior of this kind. In the development of chaos theory, the rst evidence of physical chaos was Edward Lorenzs discovery in 1963. The rst underlying mechanism within chaos was observed by Mitchell Feigenbaum, who in 1976 found that when an ordered system begins to break down into chaos, a consistent pattern of rate doubling occurs (3). What Is Chaos? There is no unied, universally accepted, rigorous denition of chaos in the current scientic literature. The term chaos was rst formally introduced into mathematics by Li and Yorke (8). Since then, there have been several different but closely related proposals for denitions of chaos, among which Devaneys denition is perhaps the most popular one (9). It states that a map F : S S, where S is a set, is said to be chaotic if 1. F is transitive on S: For any pair of nonempty open sets U and V in S, there is an integer k 0 such that Fk(U) V is nonempty. 2. F has sensitive dependence on initial conditions: There is a real number 0, depending only on F and S, such that in every nonempty open subset of S there is a pair of points whose eventual iterates under F are separated by a distance of at least . 3. The periodic points of F are dense in S. Another denition requires the set S be compact but drops condition 3. There is a belief that only the transitive property is essential in this denition. Although a precise and rigorous
Theorem 5 (Lyapunov Second Method) (for continuoustime nonautonomous systems) Let x 0 be an equilibrium of the nonautonomous system of Eq. (7). The zero equilibrium of the system is globally (over the domain D Rn containing the origin), uniformly (with respect to the initial time), and asymptotically stable if there exist a scalar-valued function V(x, t) dened on D [t0, ) and three functions ( ), ( ), ( ) K , such that (1) V(0, t0) 0; (2) V(x, t) 0 for all x 0 in D and all t t0; (3) (x, t) (x) (x) V(x, t) (x) for all t t0; and (4) V for all t t0. In this theorem, the uniform stability is usually necessary since the solution of a nonautonomous system may depend sensitively on the initial time. As a special case for autonomous systems, the preceding theorem reduces to the following.
199
mathematical denition of chaos does not seem to be available anytime soon, some fundamental features of chaos are well received and can be used to signify or identify chaos in most cases. Features of Chaos A hallmark of chaos is its fundamental property of extreme sensitivity to initial conditions. Other features of chaos include the embedding of a dense set of unstable periodic orbits in its strange attractor, positive leading (maximal) Lyapunov exponent, nite Kolmogorov-Sinai entropy or positive topological entropy, continuous power spectrum, positive algorithmic complexity, ergodicity and mixing (Arnolds cat map), Smale hilnikov, horseshoe map, a statistical-oriented denition of S as well as some unusual limiting properties (4). Extreme Sensitivity to Initial Conditions. The rst hallmark of chaos is its extreme sensitivity to initial conditions, associated with its bounded (or compact) region of orbital patterns. It implies that two sets of slightly different initial conditions can lead to two dramatically different asymptotic states of the system orbit after some time. This is the so-called buttery effect and says that a single ap of a butterys wings in China today may alter the initial conditions of the global weather dynamical system, thereby leading to a signicantly different weather pattern in Argentina at a future time. In other words, for a dynamical system to be chaotic it must have a (large) set of such unstable initial conditions that cause orbital divergence within a bounded region. Positive Leading Lyapunov Exponents. Most sensitive dependence on initial conditions of a chaotic system possesses an exponential growth rate. This exponential growth is related to the existence of at least one positive Lyapunov exponent, usually the leading (largest) one. Among all main characteristics of chaos, the positive leading Lyapunov exponent is perhaps the most convenient one to verify in engineering applications. To introduce this concept, consider an n-dimensional, discrete-time dynamical system described by xk1 f (xk) via a smooth map f . The ith Lyapunov exponent of the orbit xkk 0, generated by the iterations of the map starting from any given initial state x0, is dened to be i (x 0 ) = lim 1 ln |i (Jk ( x k ) . . . J0 (x 0 ))|, k i = 1, . . ., n (11)
pending on the direction (but not the position) of the initial state vector, the n Lyapunov exponents, 1 n, describe different types of attractors. For example, for some nonchaotic attractors (limit sets),
i < 0, i = 1, . . ., n
stable equilibrium stable limit cycle stable two-torus stable m-torus
1 = 0, i < 0, i = 2, . . ., n 1 = = m = 0,
1 = 2 = 0, i < 0, i = 3, . . ., n
i < 0, i = m + 1, . . ., n
Here, a two-torus is a bagel-shaped surface in three-dimensional space, and an m-torus is its geometrical generalization in (m 1)-dimensional space. It is now well known that one and two-dimensional continuous-time autonomous dynamical systems cannot produce chaos. For a three-dimensional continuous-time autonomous system, the only possibility for chaos to exist is that the three Lyapunov exponents are (+, 0, ) := (1 > 0, 2 = 0, 3 < 0 ) and 3 < 1 Intuitively, this means that the system orbit in the phase space expands in one direction but shrinks in another direction, thereby yielding many complex (stretching and folding) dynamical phenomena within a bounded region. The discretetime case is different, however. A prominent example is the one-dimensional logistic map, discussed in more detail later, which is chaotic but has (the only) one positive Lyapunov exponent. For four-dimensional continuous-time autonomous systems, there are only three possibilities for chaos to emerge: 1. (, 0, , ): 1 0, 2 0, 4 3 0; leading to chaos 2. (, , 0, ): 1 2 0, 3 0, 4 0; leading to hyperchaos 3. (, 0, 0, ): 1 0, 2 3 0, 4 0; leading to a chaotic two-torus (this special orbit has not been experimentally observed). Simple Zero of the Melnikov Function. The Melnikov theory of chaotic dynamics deals with the saddle points of Poincare maps of continuous solution ows in the phase space. The Melnikov function provides a measure of the distance between the stable and unstable manifolds near a saddle point. To introduce the Melnikov function, consider a nonlinear oscillator described by the Hamiltonian system
where Ji( ) f ( ) is the Jacobian and i( ) denote the ith eigenvalue of a matrix (numbered in decreasing order of magnitude). In the continuous-time case, x f (x), the leading Lyapunov exponent is dened by ( x 0 ) = lim 1 ln x (t ; x 0 )/x 0 t
H + f1 = p q H q + f2 = p
where f : [f 1(p, q, t), f 2(p, q, t)] has state variables (p(t), q(t)), 0 is small, and H H(p, q) EK EP is the Hamilton function for the undamped, unforced (when 0) oscillator, in which EK and EP are the kinetic and potential energy of the system, respectively. Suppose that the unperturbed (unforced and undamped) oscillator has a saddle-node equilibrium (e.g., the undamped pendulum) and that f is tp-periodic with phase frequency . If the forced motion is described in
which is usually evaluated by numerical computations. All Lyapunov exponents depend on the system initial state x0, and reect the sensitivity with respect to x0. Lyapunov exponents are generalizations of eigenvalues of linear systems and provide a measure for the mean convergence or divergence rate of neighboring orbits of a dynamical system. For an n-dimensional continuous-time system, de-
200
the three-dimensional phase space (p, q, t), then the Melnikov function is dened by F (d ) =

[ H ( p, q )] f dt
(12)
where (p, q) are the solutions of the unperturbed homoclinic orbit starting from the saddle point of the original Hamiltonian system, f * f (p, q, t d*), and H [H / p, H / q]. The variable d* gives a measure of the distance between the stable and unstable manifolds near the saddle-node equilibrium. The Melnikov theory states that chaos is possible if the two manifolds interset, which corresponds to the fact that the Melnikov function has a simple zero: F(d*) 0 at a single point, d*. Strange Attractors. Attractors are typical in nonlinear systems. The most interesting attractors, very closely related to chaos, are the strange attractors. A strange attractor is a bounded attractor that exhibits sensitive dependence on initial conditions but cannot be decomposed into two invariant subsets contained in disjoint open sets. Most chaotic systems have strange attractors; however, not all strange attractors are associated with chaos. Generally speaking, a strange attractor is not any of the stable equilibria or limit cycles, but rather consists of some limit sets associated with Cantor sets and/or fractals. In other words, it has a special and complicated structure that may possess a noninteger dimension (fractals) and has some of the properties of a Cantor set. For instance, a chaotic orbit usually appears to be strange in that the orbit moves toward a certain point (or limit set) for some time but then moves away from it for some other time. Although the orbit repeats this process innitely many times it never settles anywhere. Figure 6 shows a typical Chua circuit attractor (2,4,6) that has such strange behavior. Fractals. An important concept that is related to Lyapunov exponent is the Hausdorff dimension. Let S be a set in Rn and
C be a covering of S by countably many balls of radii d1, d2, , satisfying 0 dk for all k. For a constant 0, consider k 1 dk for different coverings, and let infC dk be the smallest value of the sum over all possible such coverings. In the limit 0, this value will diverge if h but tends to zero if h for some constant h 0 (need not be an integer). This value, h, is called the Hausdorff dimension of the set S. If such a limit exists for h, then the Hausdorff measure of the set S is dened to be h (S ) := lim inf
0 C k=1 dk
There is an interesting conjecture that the Lyapunov exponents k (indicating the dynamics) and the Hausdorff dimension h (indicating the geometry) of a strange attractor have the relation
h=k+
1 k + 1
i ,
i= 1
1 2 n
where k is the largest integer that satises ik1 i 0. This formula has been mathematically proved for large families of three-dimensional continuous-time autonomous systems and of two-dimensional discrete-time systems. A notion that is closely related to the Hausdorff dimension is fractal. Fractal was rst coined and dened by Mandelbrot in the 1970s to be a set with Hausdorff dimension strictly greater than its topological dimension (which is always an integer). Roughly, a fractal is a set that has a fractional Hausdorff dimension and possesses certain self-similarities. An illustration of the concept of self-similarity and fractal is given in Fig. 13. There is a strong connection between fractal and chaos. Chaotic orbits often possess fractal structures in the phase space. For conservative systems, the KolmogorovArnold-Moser (KAM) theorem implies that the boundary between the region of regular motion and that of chaos is fractal. However, some chaotic systems have nonfractal limit sets, and many fractal structures are not chaotic. Finite Kolmogorov-Sinai Entropy. Another important feature of chaos and strange attractors is quantied by the Kolmogorov-Sinai (KS) entropy, a concept based on Shannons information theory. The familiar statistical entropy is dened by E = c
k
Pk ln(Pk )
Figure 6. A typical example of stranger attractor: The double scroll of Chuas circuit response.
where c is a constant and Pk is the probability of the system state being at the stage k of the process. According to Shannons information theory, this entropy is a measure of the amount of information needed to determine the state of the system. This idea can be used to dene a measure for the intensity of a set of system states, which gives the mean loss of information on the state of the system when it evolves with time. To do so, let x(t) be a system orbit and partition its mdimensional phase space into cells of a small volume, m. Let Pk0. . .ki be the joint probability that x(t 0) is in cell k0, x(t ts) is in cell k1, . . ., x(t its) is in cell ki, where ts 0
201
is the sampling time. Then Shannon dened the information index to be I n :=

k 0 ,..., k n
Pk
0 ... k n
ln Pk
0 ... k n
which is proportional to the amount of the information needed to determine the orbit, if the probabilities are known. Consequently, I n1 I n gives additional information for predicting the next state if all preceding states are known. This difference is also the information lost during the process. The KS entropy is then dened by
EKS := lim lim lim
t s 0 0 n
1 nts
n1 i= 0
(Ii + 1 Ii ) (13) Pk
0 ... k n 1
= lim lim lim
t s 0 0 n
1 nts
ln Pk
k 0 ,..., k n 1
0 ... k n 1
This entropy, EKS, quanties the degree of disorder: (1) EKS 0 indicates regular attractors, such as stable equilibria, limit cycles, and tori; (2) EKS implies totally random dynamics (which has no correlations in the phase space); and (3) 0 EKS signies strange attractors and chaos. It is interesting to note that there is a connection between the Lyapunov exponents and the KS entropy: EKS
i
+ i
where i are positive Lyapunov exponents of the same system. Chaos in Control Systems Chaos is ubiquitous. Chaotic behaviors have been found in many typical mathematical maps such as the logistic map, non map, Lozi map, Ikeda map, BerArnolds circle map, He noulli shift; in various physical systems, including the Dufng oscillator, van der Pol oscillator, forced pendula, hopping robot, brushless dc motor, rotor with varying mass, Lorenz model, and Ro ssler system. They are also found in electrical and electronic systems (such as Chuas circuit and electric power systems), digital lters, celestial mechanics (the threebody problem), uid dynamics, lasers, plasmas, solid states, quantum mechanics, nonlinear optics, chemical reactions, neural networks and fuzzy systems, economic and nancial systems, biological systems (heart, brain, and population models), and various Hamiltonian systems (4). Chaos also exists in many engineering processes and, perhaps unexpectedly, in both continuous-time and discrete-time feedback control systems. For instance, in the continuoustime case, chaos has been found in very simple dynamical systems such as a rst-order autonomous feedback system with a time-delay feedback channel, surge tank dynamics under a simple liquid level control system with time-delayed feedback, and several other types of time-delayed feedback control systems. Chaos also exists in automatic gain control loops, which are very popular in industrial applications, such as in many receivers of communication systems. Most fascinating of all, very simple pendula can display complex dynamical phenomena; in particular, pendula subject to linear feedback controls can exhibit even richer bifurcations and chaotic behaviors.
For example, a pendulum controlled by a proportional-derivative controller can behave chaotically when the tracking signal is periodic, with energy dissipation, even for the case of small controller gains. In addition, chaos has been found in many engineering applications, such as design of control circuits for switched-mode power conversion equipment, highperformance digital robot controllers, second-order systems containing a relay with hysteresis, and various biochemical control systems. Chaos also occurs frequently in discrete-time feedback control systems due to sampling, quantization, and roundoff effects. Discrete-time linear control systems with dead-zone nonlinearity have global bifurcations, unstable periodic orbits, scenarios leading to chaotic attractors, and crises of chaotic attractors changing to periodic orbits. Chaos also exists in digitally controlled systems, feedback types of digital ltering systems (either with or without control), and even the linear Kalman lter when numerical truncations are involved. Many adaptive systems are inherently nonlinear, and thus bifurcation and chaos in such systems are often inevitable. The instances of chaos in adaptive control systems usually come from several possible sources: the nonlinearities of the plant and the estimation scheme, external excitation or disturbances, and the adaptation mechanism. Chaos can occur in typical model-referenced adaptive control (MRAC) and selftuning adaptive control (STAC) systems, as well as some other classes of adaptive feedback control systems of arbitrary order that contain unmodeled dynamics and disturbances. In such adaptive control systems, typical failure modes include convergence to undesirable local minima and nonlinear selfoscillation, such as bursting, limit cycling, and chaos. In indirect adaptive control of linear discrete-time plants, strange system behaviors can arise due to unmodeled dynamics (or disturbances), bad combination of parameter estimation and control law, and lack of persistency of excitation. For example, chaos can be found in set-point tracking control of a linear discrete-time system of unknown order, where the adaptive control scheme is either to estimate the order of the plant or to track the reference directly. Chaos also emerges from various types of neural networks. Similar to biological neural networks, most articial neural networks can display complex dynamics, including bifurcations, strange attractors, and chaos. Even a very simple recurrent two-neuron model with only one self-interaction can produce chaos. A simple three-neuron recurrent neural network can also create period-doubling bifurcations leading to chaos. A four-neuron network and multineuron networks, of course, have higher chances of producing complex dynamical patterns such as bifurcations and chaos. A typical example is cellular neural networks, which have very rich complex dynamical behaviors. Chaos has also been experienced in some fuzzy control systems. The fact that fuzzy logic can produce complex dynamics is more or less intuitive, inspired by the nonlinear nature of the fuzzy systems. This has been justied not only experimentally but also both mathematically and logically. Chaos has been observed, for example, from a coupled fuzzy control system. The change in the shapes of the fuzzy membership functions can signicantly alter the dynamical behavior of a fuzzy control system, potentially leading to the occurrence of chaos. Many specic examples of chaos in control systems can be given. Therefore, controlling chaos is not only interesting as
202
a subject for scientic research but also relevant to the objectives of traditional control engineering. Simply, it is not an issue that can be treated with ignorance or neglect. BIFURCATIONS Associated with chaos is bifurcation, another typical phenomenon of nonlinear dynamical systems that quanties the change of system properties (such as the number and the stabilities of the system equilibria) due to the variation of system parameters. Chaos and bifurcation have a very strong connection; often they coexist in a complex dynamical system. Basic Types of Bifurcations To illustrate various bifurcation phenomena, it is convenient to consider the two-dimensional, parametrized, nonlinear dynamical system
0
x2 = p
Figure 8. The saddle-node bifurcation.
has an equilibrium x1 0 at p0 0 and an equilibrium curve x2 p at p 0, where x21 p is stable and x22 p is unstable for p p0 0. This bifurcation, as shown in Fig. 8, is called the saddle-node bifurcation. Pitchfork Bifurcation. The one-dimensional system x = f (x; p ) = px x3
x = f (x, y; p ) y = g(x, y; p )
(14)
where p is a real and variable system parameter. Let (x, y) (x(t; p0), y(t; p0)) be an equilibrium of the system when p p0, at which f (x, y; p0) 0 and g(x, y; p0) 0. If the equilibrium is stable (respectively, unstable) for p p0 but unstable (respectively, stable) for p p0, then p0 is a bifurcation value of p, and (0, 0, p0) is a bifurcation point in the parameter space, (x, y, p). A few examples are given next to distinguish several typical bifurcations. Transcritical Bifurcation. The one-dimensional system x = f (x; p ) = px x2 has two equilibria: x1 0 and x2 p. If p is varied, then there are two equilibrium curves, as shown in Fig. 7. Since the Jacobian at zero for this one-dimensional system is simply J p, it is clear that for p p0 0, the equilibrium x1 0 is stable, but for p p0 0 it changes to be unstable. Thus, (x1, p0) (0, 0) is a bifurcation point. In the gure, the solid curves indicate stable equilibria and the dashed curves, the unstable ones. (x2, p0) is another bifurcation point. This type of bifurcation is called the transcritical bifurcation. Saddle-Node Bifurcation. The one-dimensional system x = f (x; p ) = p x2
has an equilibrium x1 0 at p0 0 and an equilibrium curve x2 p at p 0. Since x1 0 is unstable for p p0 0 and stable for p p0 0, and since the entire equilibrium curve x2 p is stable for all p 0 at which it is dened, this situation, as depicted in Fig. 9, is called the pitchfork bifurcation. Note, however, that not all nonlinear parametrized dynamical systems have bifurcations. A simple example is x = f (x; p ) = p x3 which has an entire stable equilibrium curve x p1/3 and hence does not have any bifurcation. Hysteresis Bifurcation. The dynamical system
x 1 = x1 x 2 = p + x2 x3 2
has equilibria x1 = 0 and p x2 + x3 2 =0
According to different values of p, there are either one or three equilibrium solutions, where the second equation gives
x2 = p
0
p
t
t
Figure 7. The transcritical bifurcation. Figure 9. The pitchfork bifurcation.
203
x2
where R denotes the real part of the complex eigenvalues. Then 1. p p0 is a bifurcation point of the system. 2. For close enough values p p0, the equilibrium x 0 is asymptotically stable. 3. For close enough values p p0, the equilibrium x 0 is unstable. 4. For close enough values p p0, the equilibrium x 0 is surrounded by an emerging limit cycle of magnitude O(p p0). Graphical Hopf Bifurcation Theorem. The Hopf bifurcation can also be analyzed in the frequency-domain setting (10). In this approach, the nonlinear parametrized autonomous system of Eq. (15) is rst rewritten in the following Lure form:
p0
p0
Figure 10. The hysteresis bifurcation.
a bifurcation point at p0 23/9, but three equilibria for p0 23/9. The stabilities of the equilibrium solutions are shown in Fig. 10. This type of bifurcation is called the hysteresis bifurcation. Hopf Bifurcation and Hopf Theorems. In addition to the bifurcations described previously, called static bifurcations, the parametrized dynamical system of Eq. (14) can have another type of bifurcation, the Hopf bifurcation (or dynamical bifurcation). Hopf bifurcation corresponds to the situation where, as the parameter p is varied to pass the critical value p0, the system Jacobian has one pair of complex conjugate eigenvalues moving from the left-half plane to the right, crossing the imaginary axis, while all the other eigenvalues remain to be stable (with negative real parts). At the moment of the crossing, the real parts of the eigenvalue pair are zero, and the stability of the existing equilibrium changes to opposite, as shown in Fig. 11. In the meantime, a limit cycle will emerge. As indicated in the gure, Hopf bifurcation can be classied as supercritical (respectively, subcritical), if the equilibrium is changed from stable to unstable (respectively, from unstable to stable). The same terminology of supercritical and subcritical bifurcations applies to other non-Hopf types of bifurcations. Consider the general nonlinear, parametrized autonomous system = f ( x ; p ), x x (t0 ) = x 0 (15)
x = A( p )x + B( p )u y = C( p ) x u = g ( y ; p )
(16)
where the matrix A(p) is chosen to be invertible for all values of p, and g C4 depends on the chosen matrices A, B, and
p < p0
p = p0
p > p0
where x Rn, p is a real variable parameter, and f is differentiable. The most fundamental result on the Hopf bifurcation of this system is the following theorem, which is stated here for the special two-dimensional case. Theorem 7 (Poincare -Andronov-Hopf) Suppose that the two-dimensional system of Eq. (15) has a zero equilibrium, x 0, and assume that its associate Jacobian A f / xxx0 has a conjugate pair of purely imaginary eigenvalues, (p0) and *(p0) for some p0. Also assume that
p p
p = p0 y x y
d {( p )} dp
>0
p= p 0
Figure 11. Two types of Hopf bifurcations illustrated in the phase plane. (a) supercritical; (b) subcritical.
204

{ ( ;p )} ( ;p ) ( ;p )
C. Assume that this system has an equilibrium solution, y, satisfying y (t ; p ) = H (0; p )g( y (t ; p ); p ) where H (0; p ) = C( p )A 1 ( p )B( p ) ( j; p) be the eigenvalue of Let J(p) g / yyy and let the matrix [H( j; p) J(p)] that satises ( j 0 ; p0 ) = 1 + j 0, j= 1
{ ( ;p )}
P = 1 + 2 1( )
2 1()
Figure 12. The frequency-domain version of the Hopf bifurcation theorem.
Then x p p and let vary. In so doing, a trajectory of the (; p function ), the eigenlocus, can be obtained. This locus traces out from the frequency 0 0. In much the same way, a real zero eigenvalue (a condition for the static bifurcation) is replaced by a characteristic gain locus that crosses the point (1 j 0) at frequency 0 0. For illustration, consider a single-input single-output (SISO) system. In this case, the matrix [H( j; p) J(p)] is merely a scalar, and
n
The graphical Hopf bifurcation theorem (for SISO systems) formulated in the frequency domain, based on the generalized Nyquist criterion, is stated as follows (10). Theorem 8 (Graphical Hopf Bifurcation Theorem) Suppose that when varies, the vector 1( ) 0. Assume also that the half-line, starting from 1 j 0 and pointing to the direction parallel to that of 1( ), rst intersects the locus of ( j ; p the eigenvalue ) at the point = ( P ; p ) = 1 + 1 () 2 at which and the constant ( ) 0, as shown in Fig. 12. Suppose, furthermore, that the preceding intersection is transversalnamely, } I {1 ( j ) } {1 ( j ) d d det =0 ;p ;p ( ) = ( ) = I d d where I is the imaginary part of the complex eigenvalue. Then 1. The nonlinear system of Eq. (16) has a periodic solution (output) y(t) y(t; y). Consequently, there exists a unique limit cycle for the nonlinear equation x f (x), in a ball of radius O(1) centered at the equilibrium x. 2. If the total number of counterclockwise encirclements of 1( the point p1 P ), for a small enough 0, is equal to the number of poles of [H(s; p) J(p)] that have positive real parts, then the limit cycle is stable. Period-Doubling Bifurcations to Chaos There are several routes to chaos from a regular state of a nonlinear system, provided that the system is chaotic in nature. It is known that after three Hopf bifurcations a regular motion can become highly unstable, leading to a strange attractor and, thereafter, chaos. It has also been observed that even pitchfork and saddle-node bifurcations can be routes to chaos under certain circumstances. For motion on a normalized two-torus, if the ratio of the two fundamental frequencies 1 / 2 p / q is rational, then the orbit returns to the same point after a q-cycle; but if the ratio is irrational, the orbit (said to be quasiperiodic) never returns to the starting point. Quasiperiodic motion on a two-torus provides another common route to chaos.
y(t ) y +
yk e jkt
k=0
where y is the equilibrium solution and the complex coefcients, yk, are determined as follows. For the approximation with n 2, rst dene an auxiliary vector 1 () = ; p )] h 1 l T [H ( j l Tr
(17)
where p is the xed value of the parameter p, l and r are the left and right eigenvectors of [H( j ) J(p )], respectively, ; p ( j associated with the eigenvalue ), and ; p 1 1 h 1 = D2 z 02 r + r z 22 + D3 r r r 2 8 in which * denotes the complex conjugate, is the frequency locus and the negative real of the intersection between the axis that is closest to the point (1 j 0), is the tensor product operator, and
D2 =
2 g (y; p ) y2
y=y
) 3g (y; p D3 = y3
y=y
1 )J ( p )] 1 G ( 0 ; p z 02 = [1 + H (0; p )D 2 r r 4 1 ; p )] 1 H ( 2 j z 22 = [1 + H (2 j ; p )D 2 r r 4 y0 = z 02 p p0 y1 = r | p p0
1 /2
y2 = z 22 p p0
205
Period-doubling bifurcation is perhaps the most typical route that leads system dynamics to chaos. Consider, for example, the logistic map xk+ 1 = pxk (1 xk ) (18)
1.00
0.00
1.00
where p 0 is a variable parameter. With 0 p 1, the origin x 0 is stable, so the orbit approaches it as k . However, for 1 p 3, all points converge to another equilibrium, denoted x. The dynamical evolution of the system behavior, as p is gradually increased from 3.0 to 4.0 by small steps, is mostly interesting, which is depicted in Fig. 13. The gure shows that at p 3, a (stable) period-two orbit is bifurcated out of x, which becomes unstable at that moment, and, in addition to 0, there emerge two (stable) equilibria: x 1 , 2 = (1 + p p2 2 p 3 )/(2 p )
2.00
3.00 2.50
2.88
3.25 p
3.63
4.00
Figure 14. Lyapunov exponent versus parameter p for the logistic map. Reprinted from J. Argyris, G. Faust, and M. Haase, An Exploration of Chaos, 1994, Fig. 5.4.8.(b), p. 172, with kind permission from Elsevier ScienceNL, Amsterdam, The Netherlands.
When p continues to increase to the value of 1 6 3.544090 . . ., each of these two points bifurcates to the other two, as can be seen from the gure. As p moves consequently through the values 3.568759 . . ., 3.569891 . . ., , an innite sequence of bifurcations is generated by such period doubling, which eventually leads to chaos: period 1 period 2 period 4 period 2k chaos It is also interesting to note that certain regions (e.g., the three windows magnied in the gure) of the logistic map show self-similarity of the bifurcation diagram of the map, which is a typical fractal structure.
Figure 14 shows the Lyapunov exponent versus the parameter p, in the interval [2.5, 4]. This gure corresponds to the period-doubling diagram shown in Fig. 13. The most signicant discovery about the phenomenon of period-doubling bifurcation route to chaos is Feigenbaums observation in 1978: The convergence of the period-doubling bifurcating parameters has a geometric rate, p pk k, where pk + 1 pk = 4.6692 . . . pk + 2 pk + 1 (k )
This is known as a universal number for a large class of chaotic dynamical systems. Bifurcations in Control Systems Not only chaos but also bifurcations can exist in feedback and adaptive control systems. Generally speaking, local instability and complex dynamical behavior can result from feedback and adaptive mechanisms when adequate process information is not available for feedback transmission or for parameter estimation. In this situation, one or more poles of the linearized closed-loop transfer function may move to cross over the stability boundary, thereby causing signal divergence as the control process continues. However, this sometimes may not lead to global unboundedness, but rather, to self-excited oscillations or self-stabilization, creating very complex dynamical phenomena. Several examples of bifurcations in feedback control systems include the automatic gain control loop system, which has bifurcations transmitting to Smale horseshoe chaos and the common route of period-doubling bifurcations to chaos. Surprisingly enough, in some situations even a single pendulum controlled by a linear proportional-derivative controller can display rich bifurcations in addition to chaos. Adaptive control systems are more likely to produce bifurcations than a simple feedback control system due to changes of stabilities in adaptation. The complex dynamics emerging from an adaptive control system are often caused by estimation instabilities. It is known that certain prototypes of MRAC systems can experience various bifurcations.
xn
1.0
0.5
0.0 2.8 3.0
1+ 6 1+ 8 3.56994 4.0
Figure 13. Period doubling of the logistic system with self-similarity. Reprinted from J. Argyris, G. Faust, and M. Haase, An Exploration of Chaos, 1994, Fig. 9.6.6, p. 66f, with kind permission from Elsevier ScienceNL, Amsterdam, The Netherlands.
206
Bifurcation theory has been employed for analzying complex dynamical systems. For instance, in an MRAC system, a few pathways leading to estimator instability have been identied via bifurcation analysis: 1. A sign change in the adaptation law, leading to a reversal of the gradient direction as well as an innite linear drift. 2. The instability caused by high control gains, leading to global divergence through period-doubling bifurcations. 3. A Hopf bifurcation type of instability, complicated by a number of nonlocal phenomena, leading to parameter drift and bursting in a bounded regime through a sequence of global bifurcations. Both instabilities of types 1 and 2 can be avoided by gain tuning or simple algorithmic modications. The third instability, however, is generally due to the unmodeled dynamics and a poor signal-to-noise ratio, and so cannot be avoided by simple tuning methods. This instability is closely related to the presence of a degenerate set and a period-two attractor. Similarly, in the discrete-time case, a simple adaptive control system can have rich bifurcation phenomena, such as period-doubling bifurcation (due to high adaptive control gains) and Hopf and global bifurcations (due to insufcient excitation). Like the omnipresent chaos, bifurcations exist in many physical systems (4). For instance, power systems generally have various bifurcation phenomena. When consumers demands for power reach peaks, the stability of an electric power network may move to its margin, leading to serious oscillations and stability bifurcations, which may quickly result in voltage collapse. As another example, a typical double pendulum can display bifurcations as well as chaotic motions. Some rotational mechanical systems also have similar behavior. Even a common road vehicle driven by a pilot with driver steering control can have Hopf bifurcation when its stability is lost, which may also develop chaos and even hyperchaos. A hopping robot, or a simple two-degree-of-freedom exible robot arm, can response strange vibrations undergoing period doubling, which eventually lead to chaos. An aircraft stalling for ight below a critical speed or over a critical angle of attack can cause various bifurcations. Dynamics of a ship can exhibit stability bifurcation according to wave frequencies that are close to the natural frequency of the ship, which creates oscillations and chaotic motions leading the ship to capsize. Simple nonlinear circuits are rich sources of different types of bifurcations as well as chaos. Other systems that have bifurcation properties include cellular neural networks, lasers, aeroengine compressors, weather systems, and biological population dynamics, to name just a few.
However, recent research effort has shown that not only (short-term) prediction but also control of chaos are possible. It is now well known that most conventional control methods and many special techniques can be used for controlling chaos (4,11,12). In this pursuit, whether the purpose is to reduce bad chaos or to introduce good ones, numerous control strategies have been proposed, developed, tested, and applied to many case studies. Numerical and experimental simulations have demonstrated that chaotic physical systems respond quite well to these controls. In about the same time, applications are proposed in such diverse elds as biology, medicine, physiology, chemical engineering, laser physics, electric power systems, uid mechanics, aerodynamics, circuits and electronic devices, and signal processing and communication. The fact that researchers from vast scientic and engineering backgrounds are joining together and aiming at one central themebringing order to chaosindicates that the study of nonlinear dynamics and their control has progressed into a new era. Much has been accomplished in the past decade, yet much more remains a challenge for the future. Similar to conventional systems control, the concept of controlling chaos rst means to suppress chaos in the sense of stabilizing chaotic system responses, often unstable periodic outputs. However, controlling chaos has also encompassed many nontraditional tasks, particularly those of creating or enhancing chaos when it is useful. The process of chaos control is now understood as a transition between chaos and order and, sometimes, the transition from chaos to chaos, depending on the application at hand. In fact, the notion of chaos control is neither exclusive of, nor conicting with, the purposes of conventional control systems theory. Rather, it targets at better managing the dynamics of a nonlinear system on a wider scale, with the hope that more benets may be derived from the special features of chaos. Why Chaos Control? There are many practical reasons for controlling or ordering chaos. First, chaotic (messy, irregular, or disordered) system response with little useful information content is unlikely to be desirable. Second, chaos can lead systems to harmful or even catastrophic situations. In these troublesome cases, chaos should be reduced as much as possible or totally suppressed. Traditional engineering design always tries to reduce irregular behaviors of a system and, therefore, completely eliminates chaos. Such overdesign is needed in the aforementioned situations. However, this is usually accomplished at the price of loosing great benets in achieving high performance near the stability boundaries, or at the expense of radically modifying the original system dynamics. Ironically, recent research has shown that chaos can actually be useful under certain circumstances, and there is growing interest in utilizing the very nature of chaos (4). For example, it has been observed (13) that a chaotic attractor typically has embedded within it a dense set of unstable limit cycles. Thus, if any of these limit cycles can be stabilized, it may be desirable to select one that characterizes maximal system performance. In other words, when the design of a dynamical system is intended for multiple uses, purposely building chaotic dynamics into the system may allow for the
CONTROLLING CHAOS Understanding chaos has long been the main focus of research in the eld of nonlinear dynamics. The idea that chaos can be controlled is perhaps counterintuitive. Indeed, the extreme sensitivity of a chaotic system to initial conditions once led to the impression and argument that chaotic motion is in general neither predictable nor controllable.
207
desired exibilities. A control design of this kind is certainly nonconventional. Fluid mixing is a good example in which chaos is not only useful but actually necessary (14). Chaos is desirable in many applications of liquid mixing, where two uids are to be thoroughly mixed while the required energy is minimized. For this purpose, it turns out to be much easier if the dynamics of the particle motion of the two uids are strongly chaotic, since it is difcult to obtain rigorous mixing properties otherwise due to the possibility of invariant two-tori in the ow. This has been one of the main subjects in uid mixing, known as chaotic advection. Chaotic mixing is also important in applications involving heating, such as plasma heating for a nuclear fusion reactor. In such plasma heating, heat waves are injected into the reactor, for which the best result is obtained when the heat convection inside the reactor is chaotic. Within the context of biological systems, the controlled biological chaos seems to be important with the way a human brain executes its tasks. For years, scientists have been trying to unravel how our brains endow us with inference, thoughts, perception, reasoning, and, most fascinating of all, emotions such as happiness and sadness. It has been suggested that the human brain can process massive information in almost no time, for which chaotic dynamics could be a fundamental reason: the controlled chaos of the brain is more than an accidental by-product of the brain complexity, including its myriad connections, but rather, it may be the chief property that makes the brain different from an articial-intelligence machine (15). The idea of anticontrol of chaos has been proposed for solving the problem of driving the system responses of a human brain model away from the stable direction and, hence, away from the stable (saddle-type) equilibrium. As a result, the periodic behavior of neuronal population bursting can be prevented (16). Control tasks of this type are also nontraditional. Other potential applications of chaos control in biological systems have reached out from the brain to elsewhere, particularly to the human heart. In physiology, healthy dynamics has been regarded as regular and predictable, whereas disease, such as fatal arrhythmias, aging, and drug toxicity, is commonly assumed to produce disorder and even chaos. However, recent laboratory studies have seemingly demonstrated that the complex variability of healthy dynamics in a variety of physiological systems has features reminiscent of deterministic chaos, and a wide class of disease processes (including drug toxicities and aging) may actually decrease (yet not completely eliminate) the amount of chaos or complexity in physiological systems (decomplexication). Thus, in contrast to the common belief that healthy heartbeats are completely regular, a normal heart rate may uctuate in a somewhat erratic fashion, even at rest, and may actually be chaotic (17). It has also been observed that, in the heart, the amount of intracellular Ca is closely regulated by coupled processes that cyclically increase or decrease this amount, in a way similar to a system of coupled oscillators. This cyclical uctuation in the amount of intracellular Ca is a cause of afterdepolarizations and triggered activities in the heartthe so-called arrhythmogenic mechanism. Medical evidence reveals that controlling (but not completely eliminating) the chaotic arrhythmia can be a new, safe, and promising approach to regulating heartbeats (18,19).
Chaos Control: An Example To appreciate the challenge of chaos control, consider the onedimensional logistic map of Eq. (18) with the period-doubling bifurcation route to chaos as shown in Fig. 13. Chaos control problems in this situation include, but are not limited to, the following: Is it possible (and, if so, how) to design a simple (e.g., linear) controller, uk, for the given system in the form xk+ 1 = pxk (1 xk ) + uk such that 1. The limiting chaotic behavior of the period-doubling bifurcation process is suppressed? 2. The rst bifurcation is delayed to take place, or some bifurcations are changed either in form or in stability? 3. When the parameter p is currently not in the bifurcating range, the asymptotic behavior of the system becomes chaotic? Many of such nonconventional control problems emerging from chaotic dynamical systems have posed a real challenge to both nonlinear dynamics analysts and control engineers they have become, in effect, motivation and stimuli for the current endeavor devoted to the new research direction in control systems: controlling bifurcations and chaos. Some Distinctive Features of Chaos Control At this point, it is illuminating to highlight some distinctive features of chaos control theory and methodology, in contrast to other conventional approaches regarding such issues as objectives, perspectives, problem formulations, and performance measures. 1. The targets in chaos control are usually unstable periodic orbits (including equilibria and limit cycles), perhaps of high periods. The controller is designed to stabilize some of these unstable orbits or to drive the trajectories of the controlled system to switch from one orbit to another. This interorbit switching can be either chaos order, chaos chaos, order chaos, or order order, depending on the application of interest. Conventional control, on the other hand, does not normally investigate such interorbit switching problems of a dynamical system, especially not those problems that involve guiding the system trajectory to an unstable or chaotic state by any means. 2. A chaotic system typically has embedded within it a dense set of unstable orbits and is extremely sensitive to tiny perturbations in its initial conditions and system parameters. Such a special property, useful for chaos control, is not available in nonchaotic systems and is not utilized in any forms in conventional controls. 3. Most conventional control schemes work within the state space framework. In chaos control, however, one more often deals with the parameter space and phase maps, delay-coordinates embedding, space. Poincare parametric variation, entropy reduction, and bifurca-
208
4.
5.
6.
7.
tion monitoring are some typical but nonconventional tools for design and analysis. In conventional control, the terminal time for the control is usually nite (e.g., the elementary concept of controllability is typically dened using a nite and xed terminal time, at least for linear systems and afne-nonlinear systems). However, the terminal time for chaos control is usually innite to be meaningful and practical, because many nonlinear dynamical behaviors, such as equilibrium states, limit cycles, attractors, and chaos, are asymptotic properties. In addition, in chaos control, a target for tracking is not limited to constant vectors in the state space but often is an unstable periodic orbit of the given system. Depending on different situations or purposes, the performance measure in chaos control can be different from those for conventional controls. Chaos control generally uses criteria like Lyapunov exponents, Kolmogorov-Sinai entropy, power spectra, ergodicity, and bifurcation changes, whereas conventional controls normally emphasize robustness of the system stability or control performance, optimality of control energy or time, ability of disturbances rejection, etc. Chaos control includes a unique taskanticontrol, required by some unusual applications such as those in biomedical engineering mentioned previously. This anticontrol tries to create, maintain, or enhance chaos for improving system performance. Bifurcation control is another example of this kind, where a bifurcation point is expected to be delayed in case it cannot be avoided or stabilized. This delay can signicantly extend the operating time (or system parameter range) for a time-critical process such as chemical reaction, voltage collapse of electric power systems, and compression of stall of gas turbine jet engines. These are in direct contrast to traditional control tasks, such as the textbook problem of stabilizing an equilibrium position of a nonlinear system. Due to the inherent association of chaos and bifurcations with various related issues, the scope of chaos control and the variety of problems that chaos control deals with are quite diverse, including creation and manipulation of self-similarity and symmetry, pattern formation, amplitudes of limit cycles and sizes of attractor basins, and birth and change of bifurcations and limit cycles, in addition to some typical conventional tasks, such as target tracking and system regulation.
dynamical systems, including such complex phenomena as degenerate bifurcations and hyperchaos in the system dynamics when a controller is involved. Unless suppressing complex dynamics in a process is the only purpose for control, understanding and utilizing the rich dynamics of a controlled system are very important for design and applications. Representative Approaches to Chaos Control There are various conventional and nonconventional control methods available for bifurcations and chaos control (4, 11, 12). To introduce a few representative ones, only three categories of methodologies are briey described in this section. Parametric Variation Control. This approach for controlling a chaotic dynamical system, proposed by Ott, Grebogi, and Yorke (13,20) and known as the OGY method, is to stabilize one of its unstable periodic orbits embedded in an existing chaotic attractor, via small time-dependent perturbations of the key system parameter. This methodology utilizes the special feature of chaos that a chaotic attractor typically has embedded within it a dense set of unstable periodic orbits. To introduce this control strategy, consider a general continuous-time parametrized nonlinear autonomous system (t ) = f ( x (t ), p ) x (19)
where, for illustration, x [x y z] denotes the state vector, and p is a system parameter accessible for adjustment. Assume that when p p* the system is chaotic, and it is desired to control the system orbit, x(t), to reach a saddle-type unstable equilibrium (or periodic orbit), . Suppose that within a small neighborhood of p*, that is, p pmax < p < p + pmax (20)
where pmax 0 is the maximum allowable perturbation, both the chaotic attractor and the target orbit do not disappear (i.e., within this small neighborhood of p*, there are no bifurcation points of the periodic orbit ). Then let P be the under map and be a surface of cross section of . lying Poincare For simplicity, assume that this two-dimensional hyperplane is orthogonal to the third axis and thus is given by = {[ ]T R3 : = z0 (a constant)} Moreover, let be the coordinates of the surface of cross section; that is, a vector satisfying k+ 1 = P(k , pk ) where pk = p + pk , pk pmax
It is also worth mentioning an additional distinctive feature of a controlled chaotic system that differs from an uncontrolled chaotic system. The controlled chaotic system is generally nonautonomous and cannot be reformulated as an autonomous system by dening the control input as a new state variable, since the controller is physically not a system state variable and, moreover, it has to be determined via design for performance specications. Hence, a controlled chaotic system is intrinsically much more difcult to design than it appears (e.g., many invariant properties of autonomous systems are no longer valid). This observation raises the question of extending some existing theories and techniques from autonomous system dynamics to nonautonomous, controlled,
At each iteration, p pk is chosen to be a constant. Many distinct unstable periodic orbits within the chaotic map. Suppose attractor can be determined by the Poincare that an unstable period-one orbit * f has been selected, which maximizes certain desired system performance with respect
209
to the dynamical behavior of the system. This target orbit satises

f = P ( f , p )
em ani fold
The iteration of the map near the desired orbit are then observed, and the local properties of this chosen periodic orbit are obtained. To do so, the map is rst locally linearized, yielding a linear approximation of P near * f and p*, as
k + 1 f + Lk (k f ) + v k ( pk p )
Target
(21)
k + 1 iterate without perturbation kth iterate
or k + 1 Lk k + v k pk where (22)
k + 1 iterate with parameter perturbation
Figure 15. Schematic diagram for the parametric variation control method.
k = k f, Lk = P( f,
pk = pk p , P( f, p )/ pk
p )/k , v k =
The stable and unstable eigenvalues, s,k and u,k satisfying s,k 1 u,k, can be calculated from the Jacobian Lk. Let Ms and Mu be the stable and unstable manifolds whose directions are specied by the eigenvectors es,k and eu,k that are associated with s,k and u,k, respectively. If gs,k and gu,k are the basis vectors dened by
T T gs , k e s , k = g u , k e u , k = 1, T T gs ,k eu ,k = gu ,k es ,k = 0
For this case of a saddle-node equilibrium target, this control procedure is illustrated by Fig. 15. Now suppose that k has approached sufciently close to * f , so that Eq. (21) holds. For the next iterate, k1, to fall onto the local stable manifold of * f , the parameter pk p* pk has to be so chosen that
T T gu , k k + 1 = g u , k (k + 1 f ) = 0
This simply means that the direction of the next iteration is perpendicular to the direction of the current local unstable manifold. For this purpose, taking the inner product of Eq. (22) with gu,k and using Eq. (23) lead to
pk = u ,k
then the Jacobian Lk can be expressed as
T T Lk = u ,k eu ,k gu , k + s , k es , k g s , k
T gu , k k T v gu ,k k
St
ab
Uns
tabl
le
an
ifo
ld
(24)
(23)
To start the parametric variation control scheme, one may open a window covering the target equilibrium and wait until the system orbit travels into the window (i.e., until k falls close enough to * f ). Then the nominal value of the parameter pk is adjusted by a small amount pk using a control formula given below in Eq. (24). In so doing, both the location of the orbit and its stable manifold are changed, such that the next iteration, represented by k1 in the surface of cross section, is forced toward the local stable manifold of the original equilibrium. Since the system has been linearized, this control action usually is unable to bring the moving orbit to the target at one iteration. As a result, the controlled orbit will leave the small neighborhood of the equilibrium again and continue to wander chaotically as if there was no control on it at all. However, due to the semi-attractive property of the saddle-nose equilibrium, sooner or later the orbit returns to the window again, but generally is closer to the target due to the control effect. Then the next cycle of iteration starts, with an even smaller control action, to nudge the orbit toward the target.
,kvk 0. This is the control forwhere it is assumed that gu mula for determining the variation of the adjustable system parameter p at each step, k 1, 2, . The controlled orbit thus is expected to approach * f at a geometric rate. Note that this calculated pk is used to adjust the parameter p only if pk pmax. When pk pmax, however, one should set pk 0. Also, when k1 falls on a local stable manifold of * f , one should set pk 0 because the stable manifold might lead the orbit directly to the target. Note also that the preceding derivation is based on the as map, P, always possesses a stable sumption that the Poincare and an unstable direction (saddle-type orbits). This may not be the case in many systems, particularly those with high periodic orbits. Moreover, it is necessary that the number of accessible parameters for control is at least equal to the number of unstable eigenvalues of the periodic orbit to be stabilized. In particular, when some of such key system parameters are unaccessible, the algorithm is not applicable or has to be modied. Also, if a system has multiattractors the system orbit may never return to the opened window but move to another nontarget limit set. In addition, the technique is successful only if the control is applied after the system orbit moves into the small window covering the target, over which the local
210
linear approximation is still valid. In this case, the waiting time can be quite long for some chaotic systems. While this algorithm is effective, it generally requires good knowledge of the equations governing the system, so that computing pk by Eq. (24) is possible. In the case where only time-series data of the system are available, the delay-coordinate technique may be used to construct a faithful dynamical model for control (20,21). Entrainment and Migration Controls. Another representative approach for chaos control is the entrainment and migration control. Originally an open-loop strategy, this approach has lately been equipped with the closed-loop control technique and has been applied to many complex dynamical systems, particularly those with multiattractors. The entrainment and migration control strategy results in a radical but systematic modication of the behavior of the given dynamical system, thereby allowing to introduce a variety of new dynamical motions into the system (22,23). This approach can handle a multiattractor situation effectively, as opposed to the parametric variation control method. Entrainment means that an otherwise chaotic orbit of a system can be purposely entrained so that its dynamics, in both amplitude and phase, asymptotically tends to a prespecied set or region (e.g., a periodic orbit or an attractor). The basic formulation of entrainment control is based on the existence of some convergent regions in the phase space of a dynamical system. For a general smooth discrete-time system, x k+ 1 = f k ( x k ), or continuous-time system, x (t ) = f ( x (t )), x (0 ) = x 0 k = 0, 1, . . .
For simplicity, assume that Gk Ck with the goal orbit gk Gk, k 1, 2, . Let C k 1 Ck, and denote the basin of entrainment for the goal by B = { x 0 Rn : lim x k g k = 0}
k
Once a near entrainment is obtained in the sense that xk gk for some small 0, another form of control can be applied (i.e., to use migration-goal dynamics between different convergent regions, which allows the system trajectory to travel from one attractor to another). This is the entrainment-migration control strategy. To describe the entrainment-goal control more precisely, consider a discrete-time system of the form x k+ 1 = f k ( x k ), x k Rn
Let the goal dynamics be gk and Sk be a switching function dened by Sk 1 at some desired steps k but Sk 0 otherwise. The controlled dynamical system is suggested as x k + 1 = f k ( x k ) + k S k [ g k + 1 f k ( g k )] where 0 k 1 are constant control gains determined by the user. The control is initiated, with Sk 1, if the system state has entered the basin B; that is, when the system state enters the basin B at k kb, the control is turned on for k kb. With k 1, it gives g k + 1 f k ( g k ) = x k + 1 f k ( x k ) = 0, k kb (25)
the convergent regions are dened to be
The desired goal dynamics is then achieved: gk1 f (gk) for all k kb. Clearly, in this approach, u k = k S k [ g k + 1 f k ( g k )] is an open-loop controller, which is directly added to the righthand side of the original system. A meaningful application of the entrainment control is for multiattractor systems, to which the parametric variation control method is not applicable. Another important application is for a system with an asymptotic goal gk g, an equilibrium of the given system. In this case, the basin of entrainment is a convex region in the phase space: B e = { x 0 R n : x 0 g < r ( g )} where r( g ) = max{r : x 0 g < r lim x k g = 0}
r k
C( f k ) = {x Rn : f i ( x )/ x j ij i ( x ) = 0, i ( x ) < 1 for all i = 1, . . ., n}

or
C( f ) = { x Rn : f i ( x )/ x j ij i ( x ) = 0, {i (x )} < 0 for all i = 1, . . ., n}

where i( ) and i( ) are the eigenvalues of the Jacobians of the nonlinear maps f k and f , respectively, and ij 1 if i j but ij 0 if i j. If these convergent regions exist, the system orbitssay, xk in the discrete casecan be forced by a suitably designed external input to approach (a limit set of) the desired goal dynamics, gk, in the sense that
k
lim x k g k = 0
In other words, the system is entrained to the goal dynamics. One advantage of the entrainment control is that the goal dynamics can have any topological characteristics, such as equilibrium, periodic, knotted, and chaotic, provided that the target orbit gk is located in some goal region Gk satisfying G k Ck 0 , where Ck (k 1, 2, ) are convergent regions.
The entrainment-migration control method is straightforward, easily implementable, and exible in design. However, it requires the dynamics of the system be accurately described by either a map or a differential equation. Also, in order for the system state to be entrained to the given equilibrium, the equilibrium must lie in a particular subset of the convergent region. This can be a technical issue, particularly for higherdimensional systems. In addition, due to the open-loop na-
211
ture, process stability is not guaranteed in most cases. The main disadvantage of this approach is that it generally employs sophisticated controllers, which may even be more complicated than the given system. Engineering Feedback Controls. From a control theoretic point of view, if only suppression of chaos is concerned, chaos control may be considered as a special deterministic nonlinear control problem and so may not be much harder than conventional nonlinear systems control. However, this remains to be a technical challenge to conventional controls when a single controller is needed for stabilizing the chaotic trajectory to multiple targets of different periods. A distinctive characteristic of control engineering from other disciplines is that it employs some kind of feedback mechanism. In fact, feedback is pervasive in modern control theories and technologies. For instance, the parametric variation control method discussed previously is a special type of feedback control method. In engineering control systems, conventional feedback controllers are used for nonchaotic systems. In particular, linear feedback controllers are often designed for linear systems. It has been widely experienced that with careful design of various conventional controllers, controlling chaotic systems by feedback strategies is not only possible, but indeed quite successful. One basic reason for this success is that chaotic systems, although nonlinear and sensitive to initial conditions with complex dynamical behaviors, belong to deterministic systems by their very nature. Some Features of Feedback Control. Feedback is one of the most fundamental principles prevalent in the world. The idea of using feedback, originated from Isaac Newton and Gottfried Leibniz some 300 years ago, has been applied in various forms in natural science and modern technology. One basic feature of conventional feedback control is that, while achieving target tracking, it can guarantee the stability of the overall controlled system, even if the original uncontrolled system is unstable. This implies its intrinsic robustness against external disturbances or internal variations to a certain extent, which is desirable and often necessary for good performance of a required control task. The idea of feedback control always consuming strong control energy perhaps led to a false impression that feedback mechanisms may not be suitable for chaos control due to the extreme sensitive nature of chaos. However, feedback control under certain optimality criteria, such as a minimum control energy constraint, can provide the best performance, including the lowest consumption of control energy. This is not only supported by theory but is also conrmed by simulation with comparison. Another advantage of using feedback control is that it normally does not change the structure and parameters of the given system, and so whenever the feedback is disconnected the given system retains the original form and properties without modication. In many engineering applications, the system parameters are not feasible or not allowed for direct tuning or replacement. In such cases, state or output feedback control is a practical and desirable strategy. An additional advantage of feedback control is its automatic fashion in processing control tasks without further human interaction after being designed and implemented. As long as a feedback controller is correctly designed to satisfy the stability criteria and performance specications, it works
on its own. This is important for automation, reducing the dependence on individual operators skills and avoiding human errors in monitoring the control. A shortcoming of feedback control methods that employ tracking errors is the explicit or implicit use of reference signals. This has never been a problem in conventional feedback control of nonchaotic systems, where reference signals are always some designated, well-behaved ones. However, in chaos control, quite often a reference signal is an unstable equilibrium or unstable limit cycle, which is difcult (if not impossible) to be physically implemented as a reference input. This critical issue has stimulated some new research efforts (for instance, to use another auxiliary reference as the input in a self-tuning feedback manner). Engineering feedback control approaches have seen an alluring future in more advanced theories and applications in controlling complex dynamics. Utilization of feedback is among the most inspiring concepts that engineering has ever contributed to modern sciences and advanced technologies. A Typical Feedback Control Problem. A general feedback approach to controlling a dynamical system, not necessarily chaotic nor even nonlinear, can be illustrated by starting from the following general form of an n-dimensional control system: (t ) = f ( x , u , t ), x x (0 ) = x 0 (26)
where, as usual, x is the system state, u is the controller, x0 is a given initial state, and f is a piecewise continuous or smooth nonlinear function satisfying some dening conditions. Given a reference signal, r(t), which can be either a constant (set-point) or a function (time-varying trajectory), the automatic feedback control problem is to design a controller in, say, the state-feedback form u (t ) = g ( x , t ) (27)
where g is generally a piecewise continuous nonlinear function, such that the feedback-controlled system x (t ) = f ( x, g ( x, t ), t ) can achieve the goal of tracking:
t
(28)
lim x (t ) r (t ) = 0
(29)
For discrete-time systems, the problem and notation are similar: For a system x k+1 = f k ( x k , u k ) (30)
with given target trajectory rk and initial state x0, nd a (nonlinear) controller uk = gk ( xk ) to achieve the tracking-control goal:
k
(31)
lim x k r k = 0
(32)
212

r + ex x
f (.)
which yields en(t) : xn(t) yn 0 as t . Overall, it results in a completely controllable linear system, so that the constant control gain kc can be chosen such that x(t) y as t . Another example is that for the control system (t ) = f ( x (t ), t ) + u (t ) x using (t ) + K ( x (t ) y (t )) u (t ) = f ( x (t ), t ) + y
g(.)
Figure 16. Conguration of a general feedback control system.
A closed-loop continuous-time feedback control system has a conguration as shown in Fig. 16, where ex : r g(x), f is the given system, and g is the feedback controller to be designed, in which f and g can be either linear or nonlinear. In particular, it can be a linear system in the state-space form connected with a linear additive state-feedback controller namely, = Ax x + Bu u = Ax x + BKc (r x ) x where Kc is a constant control gain matrix to be determined. The corresponding closed-loop block diagram is shown in Fig. 17. A Control Engineers Perspective. In controllers design, particularly in nding a nonlinear controller for a system, it is important to emphasize that the designed controller should be (much) simpler than the given system to make sense of the world. For instance, suppose that one wants to nd a nonlinear controller, u(t), in the continuous-time setting, to guide the state vector x(t) [x1(t), , xn(t)] of a given nonlinear control system, x 1 (t ) = x2 (t ) x (t ) = x (t )
2 3
. . . x n (t ) = f (x1 (t ), . . ., xn (t )) + u(t )
to a target state, y [y1, . . ., yn] namely, x (t ) y as t It is then mathematically straightforward to use the controller u(t ) = f (x1 (t ), . . ., xn (t )) + kc (xn (t ) yn ) with an arbitrary constant kc 0. This controller leads to x n (t ) = kc (xn (t ) yn )
with a stable constant gain matrix K can drive its trajectory to the target y(t) as t . This kind of design, however, is undesirable, and its practical value is questionable in most cases, because the controller is even more complicated than the given system (it cancels the nonlinearity by using the given nonlinearity which means it removes the given plant and then replaces it by another system). In the discrete-time setting, for a given nonlinear system, xk1 f k(xk) uk, one may also nd a similar nonlinear feedback controller or, even simpler, use u f k(xk) gk(xk) to achieve any desired dynamics satisfying xk1 gk(xk) in just one step. This is certainly not an engineering design, nor a valuable methodology, for any realworld application other than articial computer simulations. Therefore, in designing a feedback controller, it is very important to come out with a simplest possible working controller: If a linear controller can be designed to do the job, use a linear controller; otherwise, try the simplest possible nonlinear controllers (starting, for example, from piecewise linear or quadratic controllers). Whether or not one can nd a simple, physically meaningful, easily implementable, low-cost, and effective controller for a designated control task can be quite technical: It relies on the designers theoretical background and practical experience. A General Approach to Feedback Control of Chaos. To outline the basic idea of a general feedback approach to chaos suppression and tracking control, consider Eq. (26), which is now assumed to be chaotic and possess an unstable periodic orbit (or equilibrium), x, of period tp 0namely, x(t tp) x(t), t0 t . The task is to design a feedback controller in the form of Eq. (27), such that the tracking control goal of Eq. (29), with r x therein, is achieved. Since the target periodic orbit x is itself a solution of the original system, it satises = f ( x , 0, t ) x (33)
Subtracting Eq. (33) from Eq. (26) then yields the error dynamics: x = f e (ex , x , t ) e (34)
r +
ex
Kc
x +
where ex (t ) = x (t ) x (t ), f e (ex , x , t ) = f ( x , g ( x , x , t ), t ) f ( x , 0, t ) Here, it is important to note that in order to perform correct stability analysis later on, in the error dynamical system of Eq. (34) the function f e must not explicitly contain x; if so, x should be replaced by ex x (see Eq. (38) below). This is because Eq. (34) should only contain the dynamics of ex but not
Figure 17. Conguration of a state-space feedback control system.
213
x, while the system may contain x, which merely is a specied time function but not a system variable. Thus, the design problem becomes to determine the controller, u(t), such that
t
the nonlinear controller will be designed to satisfy g(0, kc, t) 0. Then the error dynamics is reduced to e x = A( x, t )ex + h (ex , Kc , k c , t ) where A ( x, t ) = f e (e x , t ) ex (39)
lim ex (t ) = 0
(35)
which implies that the goal of tracking control described by Eq. (29) is achieved. It is clear from Eqs. (34) and (35) that if zero is an equilibrium of the error dynamical system of Eq. (34), then the original control problem has been converted to the asymptotic stability problem for this equilibrium. As a result, Lyapunov stability methods and theorems can be directly applied or modied to obtain rigorous mathematical techniques for controller design (24). This is discussed in more detail next. Chaos Control via Lyapunov Methods. The key in applying the Lyapunov second method to a nonlinear dynamical system is to construct a Lyapunov function that describes some kind of energy and governs the system motion. If this function is constructed appropriately, so that it decays monotonically to zero as time evolves, then the system motion, which falls on the surface of this decaying function, will be asymptotically stabilized to zero. A controller, then, may be designed to force this Lyapunov function of the system, stable or not originally, to decay to zero. As a result, the stability of tracking error equilibrium, and hence the goal of tracking, is achieved. For a chaos control problem with a target trajectory x, typically an unstable periodic solution of the given system, a design can be carried out by determining the controller u(t) via the Lyapunov second method such that the zero equilibrium of the error dynamics, ex 0, is asymptotically stable. In this approach, since a linear feedback controller alone is usually not sufcient for the control of a nonlinear system, particularly a chaotic one, it is desirable to nd some criteria for the design of simple nonlinear feedback controllers. In so doing, consider the feedback controller candidate of the form u (t ) = Kc (x x ) + g ( x x , k c , t ) (36)
ex =0
and h(ex, Kc, kc, t) contains the rest of the Taylor expansion. The design is then to determine both the constant control gains Kc and kc as well as the nonlinear function g( , , t) based on the linearized model of Eq. (39), such that ex 0 as t . When this controller is applied to the original system, the goal of both chaos suppression and target tracking will be achieved. For illustration, two controllability conditions established based on the boundedness of the chaotic attractors as well as the Lyapunov rst and second methods, respectively, are summarized next (24). Suppose that in Eq. (39), h(0, Kc, kc, t) 0 and A(x, t) A is a constant matrix whose eigenvalues all have negative real parts, and let P be the positive denite and symmetric solution of the Lyapunov equation PA + AT P = I where I is the identity matrix. If Kc is designed to satisfy h (ex , Kc , k c , t ) c ex for a constant c max(P) for t0 t , where max(P) is the maximum eigenvalue of P, then the controller u(t), dened in Eq. (36), will drive the trajectory x of the controlled system of Eq. (37) to the target, x, as t . For Eq. (39), since x is tp-periodic, associated with the matrix A(x, t) there always exist a tp-periodic nonsingular matrix M(x, t) and a constant matrix Q such that the fundamental matrix (consisting of n independent solution vectors) has the expression ( x, t ) = M ( x, t )etQ The eigenvalues of the constant matrix etpQ are called the Floquet multipliers of the system matrix A(x, t). In Eq. (39), assume h(0, Kc, kc, t) 0 and h(ex, Kc, kc, t) and h(ex, Kc, kc, t)/ ex are both continuous in a bounded neighborhood of the origin in Rn. Assume also that lim h (ex , Kc , k c , t ) =0 ex
where Kc is a constant matrix, which can be zero, and g is a simple nonlinear function with constant parameters kc, satisfying g(0, kc, t) 0 for all t t0. Both Kc and kc are determined in the design. Adding this controller to the given system gives = f ( x , t ) + u = f ( x , t ) + Kc ( x x ) + g ( x x , k c , t ) x (37)
The controller is required to drive the trajectory of the controlled system of Eq. (37) to approach the target orbit x. The error dynamics of Eq. (34) now takes the form x = f e (ex , t ) + Kc ex + g (ex , k c , t ) e where ex = x x , f e (ex , t ) = f (ex + x , t ) f ( x , t ) (38)
ex 0
uniformly with respect to t [t0, ). If the nonlinear controller of Eq. (36) is so designed that all Floquet multipliers i of the system matrix A(x, t) satisfy i (t ) < 1, i = 1, . . ., n, t [t0 , )
It is clear that f e(0, t) 0 for all t [t0, )namely, ex 0 is an equilibrium of the tracking-error dynamical system of Eq. (38). Next, Taylor expand the right-hand side of the controlled system of Eq. (38) at ex 0 (i.e., at x x) and remember that
then the controller will drive the chaotic orbit x of the controlled system of Eq. (37) to the target orbit, x, as t . Various Feedback Methods for Chaos Control. In addition to the general nonlinear feedback control approach described pre-
214
viously, adaptive and intelligent controls are two large classes of engineering feedback control methods that have been shown to be successful for chaos control. Other effective feedback control methods include optimal control, sliding mode and robust controls, digital controls, and occasionally proportional and time-delayed feedback controls. Linear feedback controls are also useful, but generally for simple chaotic systems. Various variants of classical control methods that have demonstrated great potential for controlling chaos include distortion control, dissipative energy method, absorber as a controller, external weak periodic forcing, Kolmogorov-Sinai entropy reduction, stochastic controls, and chaos ltering (4). Finally, it should be noted that there are indeed many valuable ideas and methodologies that by their nature cannot be well classied into one of the aforementioned three categories, not to mention that many novel approaches are still emerging, improving, and developing as of today (4). CONTROLLING BIFURCATIONS Ordering chaos via bifurcation control has never been a subject in conventional control. This seems to be a unique approach valid only for those nonlinear dynamical systems that possess the special characteristic of a route to chaos from bifurcation. Why Bifurcation Control? Bifurcation and chaos are often twins and, in particular, period-doubling bifurcation is a route to chaos. Hence, by monitoring and manipulating bifurcations, one can expect to achieve certain types of control for chaotic dynamics. Even bifurcation control itself is very important. In some physical systems such as a stressed system, delay of bifurcations offers an opportunity to obtain stable operating conditions for the machine beyond the margin of operability in a normal situation. Also, relocating and ensuring stability of bifurcated limit cycles can be applied to some conventional control problems, such as thermal convection, to obtain better results. Other examples include stabilization of some critical situations for tethered satellites, magnetic bearing systems, voltage dynamics of electric power systems, and compressor stall in gas turbine jet engines (4). Bifurcation control essentially means designing a controller for a system to result in some desired behaviors, such as stabilizing bifurcated dynamics, modifying properties of some bifurcations, or taming chaos via bifurcation control. Typical examples include delaying the onset of an inherent bifurcation, relocating an existing bifurcation, changing the shape or type of a bifurcation, introducing a bifurcation at a desired parameter value, stabilizing (at least locally) a bifurcated periodic orbit, optimizing the performance near a bifurcation point for a system, or a combination of some of these. Such tasks have practical values and great potential in many nontraditional real-world control applications. Bifurcation Control via Feedback Bifurcations can be controlled by different methods, among which the feedback strategy is especially effective. Consider a general discrete-time parametrized nonlinear system x k+ 1 = f ( x k ; p ), k = 0, 1, . . . (40)
where f is assumed to be sufciently smooth with respect to both the state xk Rn and the parameter p R, and has an equilibrium at (x, p) (0, 0). In addition, assume that the system Jacobian of Eq. (40), evaluated at the equilibrium that is the continuous extension of the origin, has an eigenvalue (p) satisfying (0) 1 and (0) 0, while all remaining eigenvalues have magnitude strictly less than one. Under these conditions, the nonlinear function has a Taylor expansion f ( x ; p ) = J ( p ) x + Q( x , x ; p ) + C( x , x , x ; p ) + where J(p) is the parametric Jacobian, and Q and C are quadratic and cubic terms generated by symmetric bilinear and trilinear forms, respectively. This system has the following property (25): A period-doubling orbit can bifurcate from the origin of system of Eq. (40) at p 0; the period-doubling bifurcation is supercritical and stable if 0 but is subcritical and unstable if 0, where
= 2 l T [C0 (r , r , r ; p ) 2Q0 (r , J0 Q0 (r , r ; p ))]
in which l is the left eigenvector and r the right eigenvector of J(0), respectively, both associated with the eigenvalue 1, and
Q 0 = J ( 0 )Q ( x , x ; p ) + Q ( J ( 0 ) x , J ( 0 ) x ; p ) C0 = J (0 )C( x, x , x ; p ) + 2Q(J (0 ) x, Q( x , x ; p )) + C(J (0 ) x, J (0 ) x, Q( x , x; p ); p )

J0 = [J T (0 )J (0 ) + ll T ] 1 J T (0 )
Now consider Eq. (40) with a control input: x k+ 1 = f ( x k ; p, u k ), k = 0, 1, . . .
which is assumed to satisfy the same assumptions when uk 0. If the critical eigenvalue 1 is controllable for the linearized system, then there is a feedback controller, uk(xk), containing only third-order terms in the components of xk, such that the controlled system has a locally stable bifurcated period-two orbit for p near zero. Also, this feedback stabilizes the origin for p 0. If, however, 1 is uncontrollable for the linearized system, then generically there is a feedback controller, uk(xk), containing only second-order terms in the components of xk, such that the controlled system has a locally stable bifurcated period-two orbit for p near 0. This feedback controller also stabilizes the origin for p 0 (25). Bifurcation Control via Harmonic Balance For continuous-time systems, limit cycles in general cannot be expressed in analytic forms, and so limit cycles corresponding to the period-two orbits in the period-doubling bifurcation diagram have to be approximated in applications. In this case, the harmonic balance approximation technique (10) can be applied, which is also useful in controlling bifurcations such as delay and stabilization of the onset of period-doubling bifurcations (26). Consider a feedback control system in the Lure form described by f ( g y + Kc y ) + y = 0

y
215
f (.;p)
and a new parameter, for the period-doubling to occur at the point p* c . Thus, the controller design is completed by choosing a suitable value for kc to satisfy such conditions (26). Controlling Multiple Limit Cycles As indicated by the Hopf bifurcation theorem, limit cycles are frequently associated with bifurcations. In fact, one type of degenerate (or singular) Hopf bifurcations (when some of the conditions stated in the Hopf theorems are not satised) determines the birth of multiple limit cycles under system parameters variation. Hence, the appearance of multiple limit cycles can be controlled by manipulating the corresponding degenerate Hopf bifurcations. This task can be conveniently accomplished in the frequency-domain setting. Again, consider the feedback system of Eq. (16), which can be illustrated by a variant of Fig. 18. For harmonic expansion of the system output, y(t), the rst-order formula is (10) y 1 = r + 3 z 13 + 5 z 15 + where is shown in Fig. 12, r is dened in Eq. (17), and z13, , z1,2m1 are some vectors orthogonal to r, m 1, 2, , given by explicit formulas (10). Observe that for a given value of , dened in the graphical Hopf theorem, the SISO system transfer function satises
Kc g(.;p)
Figure 18. A feedback system in the Lure form.
where and represent the convolution and composition operations, respectively, as shown in Fig. 18. First, suppose that a system S S(f , g) is given as shown in the gure without the feedback controller, Kc. Assume also that two system parameter values, ph and pc, are specied, which dene a Hopf bifurcation and a supercritical predicted period-doubling bifurcation, respectively. Moreover, assume that the system has a family of predicted rst-order limit cycles, stable in the range ph p pc. Under this system setup, the problem for investigation is to design a feedback controller, Kc, added to the system as shown in Fig. 18, such that the controlled system, S* S*(f , g, Kc), has the following properties: 1. S* has a Hopf bifurcation at p* h ph. 2. S* has a supercritical predicted period-doubling bifurcation for p* c pc. 3. S* has a one-parameter family of stable predicted limit cycles for p* h p p* c . 4. S* has the same set of equilibria as S. Only the one-dimensional case is discussed here. First, one can design a washout lter with the transfer function s /(s a), where a 0, such that it preserves the equilibria of the given nonlinear system. Then note that any predicted rstorder limit cycle can be well approximated by y
(1 )
H ( j ) = H (s ) + ( + j )H (s ) + 1 ( + j )2 H (s ) + 2 (41)
where , with being the imaginary part of the bifurcating eigenvalues, and H(s) and H(s) are the rst and second derivatives of H(s), dened in Eq. (16), respectively. On the other hand, with the higher-order approximations, the following equation of harmonic balance can be derived:
m
[H ( j )J + I]
i= 0
z 1,2i+ 1 2i+ 1 = H ( j )
m i= 1
r 1 , 2 i + 1 2 i + 1
(t ) = y0 + y1 sin(t )
In so doing, the controller transfer function becomes Kc (s ) = kc s(s2 + 2 ( ph )) (s + a )3
where z11 r and r1,2m1 hm, m 1, 2, , in which h1 has the formula shown in Eq. (17), and the others also have explicit formulas (10). In a general situation, the following equation has to be solved:
where kc is the constant control gain, and (ph) is the frequency of the limit cycle emerged from the Hopf bifurcation at the point p ph. This controller also preserves the Hopf bifurcation at the same point. More importantly, since a 0, the controller is stable, so by continuity in a small neighborhood of kc the Hopf bifurcation of S* not only remains supercritical but also has a supercritical predicted period-doubling bifurcation (say at pc(kc), close to ph) and a one-parameter family of stable predicted limit cycles for ph p pc(kc). The design is then to determine kc such that the predicted period-doubling bifurcation can be delayed, to a desired parameter value p* c . For this purpose, the harmonic balance approximation method (10) is useful, which leads to a solution of y(1) by obtaining values of y0, y1, and (they are functions of p, depending on kc and a, within the range ph p p* c ). The harmonic balance also yields conditions, in terms of kc, a
[H ( j ) J + I](r + z 13 3 + z 15 5 + ) = H ( j ) [h 1 3 + h 2 5 + ] (42)
In so doing, by substituting Eq. (41) into Eq. (42), one obtains the expansion ( j ) = 1 2 + 2 4 + 3 6 + 4 8 + O( 9 ) (43)
in which all the coefcients i, i 1, 2, 3, 4, can be calculated explicitly (10). Then taking the real part of Eq. (43) gives = 1 2 2 4 3 6 4 8 where i R i are the curvature coefcients of the expansion. To this end, notice that multiple limit cycles will emerge when the curvature coefcients are varied near the value
216
zero, after alternating the signs of the curvature coefcients in increasing (or decreasing) order. For example, to have four limit cycles in the vicinity of a type of degenerate Hopf bifurcation that has 1 2 3 0 but 4 0 at the criticality, the system parameters have to be varied in such a way that, for example, 0, 1 0, 2 0, 3 0, and 4 0. This condition provides a methodology for controlling the birth of multiple limit cycles associated with degenerate Hopf bifurcations. One advantage of this methodology is that there is no need to modify the feedback control path by adding any nonlinear components, to drive the system orbit to a desired region. One can simply modify the system parameters, a kind of parametric variation control, according to the expressions of the curvature coefcients, to achieve the goal of controlling bifurcations and limit cycles. ANTICONTROL OF CHAOS Anticontrol of chaos, in contrast to the main stream of ordering or suppressing chaos, is to make a nonchaotic dynamical system chaotic or to retain/enhance the existing chaos of a chaotic system. Anticontrol of chaos as one of the unique features of chaos control has emerged as a theoretically attractive and potentially useful new subject in systems control theory and some time-critical or energy-critical high-performance applications. Why Anticontrol of Chaos? Chaos has long been considered as a disaster phenomenon and so is very fearsome in benecial applications. However, chaos is dynamics freed from the shackles of order and predictability. Under good conditions or suitable control, it permits systems to randomly explore their every dynamical possibility. It is exciting variety, richness of choice, a cornucopia of opportunities (27). Today, chaos theory has been anticipated to be potentially useful in many novel and time- or energy-critical applications. In addition to those potential utilizations of chaos mentioned earlier in the discussion of chaos control, it is worth mentioning navigation in the multibody planetary system, secure information processing via chaos synchronization, dynamic crisis management, and critical decision making in political, economical, and military events. In particular, it has been observed that a transition of a biological systems state from being chaotic to being pathophysiologically periodic can cause the so-called dynamical disease and so is undesirable. Examples of dynamical diseases include cell counts in hematological disorder; stimulant drug-induced abnormalities in the behavior of brain enzymes and receptors; cardiac interbeat interval patterns in a variety of cardiac disorders; the resting record in a variety of signal sensitive biological systems following desensitization; experimental epilepsy; hormone release patterns correlated with the spontaneous mutation of a neuroendocrine cell to a neoplastic tumor; the prediction of immunologic rejection of heart transplants; the electroencephalographic behavior of the human brain in the presence of neurodegenerative disorder; neuroendocrine, cardiac, and electroencephalographic changes with aging; and imminent ventricular brillation in human subjects (28). Hence, pre-
serving chaos in these cases is important and healthy, which presents a real challenge for creative research on anticontrol of chaos (4). Some Approaches to Anticontrolling Chaos Anticontrol of chaos is a new research direction. Different methods for anticontrolling chaos are possible (4), but only two preliminary approaches are presented here for illustration. Preserving Chaos by Small Control Perturbations. Consider an n-dimensional discrete-time nonlinear system x k+ 1 = f ( x k , p, uk ) xk is the system state, uk is a scalar-valued control input, p is a variable parameter, and f is a locally invertible nonlinear map. Assume that with uk 0 the system orbit behaves chaotically at some value of p, and that when p increases and passes a critical value, pc, inverse bifurcation emerges leading the chaotic state to periodic. Within the biological context, such a bifurcation is often undesirable: There are many cases where loss of complexity and the emergence of periodicity are associated with pathology (dynamical disease). The question, then, is whether it is possible (if so, how) to keep the system state chaotic even if p pc, by using small control inputs, uk. It is known that there are at least three common bifurcations that can lead chaotic motions directly to low-periodic attracting orbits: (1) crises, (2) saddle-node type of intermittency, and (3) inverse period-doubling type of intermittency. Here, crisis refers to sudden changes caused by the collision of an attractor with an unstable periodic orbit; intermittency is a special route to chaos where regular orbital behavior is intermittently interrupted by a nite duration burst in which the orbit behaves in a decidedly different fashion; and inverse period-doubling bifurcation has a diagram in reverse form to that shown in Fig. 13 (i.e., from chaos back to less and less bifurcating points, leading back to a periodic motion) while the parameter remains increasing. In all these cases, one can identify a loss region, G, which has the property that after the orbit falls into G, it is rapidly drawn to the periodic orbit. Thus, a strategy to retain the chaos for p pc is to avoid this from happening by successively iterating G in such a way that
G1 = f 1 (G, p, 0 ), G2 = f 1 (G1 , p, 0 ) = f 2 (G, p, 0 ), . . . Gm = f m (G, p, 0 )

As m increases, the width of Gm in the unstable direction(s) has a general tendency to shrink exponentially. This suggests the following control scheme (28):
Pick a suitable value of m, denoted m0. Assume that the orbit initially starts outside the region Gm01 Gm0 G1 G. If the orbit lands in Gm01 at iterate , the control ul is applied to kick the orbit out of Gm0 at the next iterate. Since Gm0 is thin, this
CHAOS, BIFURCATIONS, AND THEIR CONTROL control can be very small. After the orbit is kicked out of Gm0, it is expected to behave chaotically, until it falls again into Gm01; at that moment another small control is applied, and so on. This procedure can keep the motion chaotic.
217
It turns out that this is possible under a natural condition that all the Jacobians f k (xk) are uniformly bounded: sup
0 k
f k ( xk ) f <
(46)
Anticontrol of Chaos via State Feedback. An approach to anticontrol of discrete-time systems can be made mathematically rigorous by applying the engineering feedback control strategy. This anticontrol technique is rst to make the Lyapunov exponents of the controlled system either strictly positive or arbitrarily assigned (positive, zero, and negative in any desired order), and then apply the simple mod operations (4,29). This task can be accomplished for any given higherdimensional discrete-time dynamical system that could be originally nonchaotic or even asymptotically stable. The argument used is purely algebraic and the design procedure is completely schematic without approximations. Specically, consider a nonlinear dynamical system, not necessarily chaotic nor unstable to start with, in the general form x k+1 = f k ( x k ) (44)
To come up with a design methodology, rst observe that if (ik)in1 are the singular values of the matrix Tk(x0); then (ik) 0 for all i 1, , n and k 0, 1, . Let i 0 for (ik) 0 and i = lim 1 ln i(k ) k (for i(k ) > 0 ), i = 1, . . ., n
Clearly, if (ik) e(k1)i is used in the design, then all (ik) will not be zero for any nite values of i, for all i 1, , n and k 0, 1, . Thus, Tk(x0) is always nonsingular. Consequently, a control-gain sequence Bk can be designed such that the singular values of the matrix Tk(x0) are exactly equal to ekiin1: At the kth step, k 0, 1, 2, , one may simply choose the control gain matrix to be Bk = ( f + ec )In , for all k = 0, 1, 2, . . .
where xk Rn, x0 is given, and f k is assumed to be continuously differentiable, at least locally in the region of interest. The anticontrol problem for this dynamical system is to design a linear state-feedback control sequence, uk Bkxk, with uniformly bounded constant control gain matrices, Bks u , where s is the spectral norm for a matrix, such that the output states of the controlled system x k+ 1 = f k (x k ) + u k behaves chaotically within a bounded region. Here, chaotic behavior is in the mathematical sense of Devaney described previouslynamely, the controlled map (a) is transitive, (b) has sensitive dependence on initial conditions, and (c) has a dense set of periodic solutions (9). In the controlled system x x + 1 = f k ( x k ) + Bkx k let Jk ( x k ) = f k ( x k ) + Bk be the system Jacobian, and let Tk ( x 0 ) = Jk ( x k ) J1 ( x 1 )J0 ( x 0 ), k = 0, 1, 2, . . .
Tk) be the ith eigenvalue of the kth Moreover, let ik i(Tk product matrix [Tk Tk], where i 1, , n and k 0, 1, 2, . The rst attempt is to determine the constant control gain matrices, Bk, such that the Lyapunov exponents of the controlled system are all nite and strictly positive:
where the constants c and f are given in Eqs. (45) and (46), respectively (29). This ensures Eq. (45) to hold. Finally, in conjunction with the previously designed controllerthat is, u k = Bkx k = ( f + ec ) x k anticontrol can be accomplished by imposing the mod operation in the controlled system: x k+1 = f k ( x k ) + u k (mod 1)
This results in the expected chaotic system whose trajectories remain within a bounded region in the phase space and, moreover, satises the aforementioned three basic properties that together dene discrete chaos. This approach yields rigorous anticontrol of chaos for any given discrete-time systems, including all higher-dimensional, linear time-invariant systems; that is, with f k(xk) Axk in Eq. (44), where the constant matrix A can be arbitrary (even asymptotically stable). Although uk Bkxk is a linear state-feedback controller, it uses full-order state variables, and the mod operation is inherently nonlinear. Hence, other types of (simple) feedback controllers are expected to be developed in the near future for rigorous anticontrol of chaos, particularly for continuous-time dynamical systems [which is apparently much more difcult (30), especially if small control input is desired]. BIBLIOGRAPHY
1. The New Encyclopaedia Britannica (Ready Reference and Index), Micropaedia, Vol. 2, Chicago: Encyclopaedia Britannica, Inc., 1984, p. 743. 2. T. Matsumoto, Chaos in electronic circuits, Proceedings of the IEEE, 75: 10331057, 1987.
0 < c i ( x 0 ) < ,
i = 1, . . ., n
(45)
218
CHAOS TIME SERIES ANALYSIS 30. T. S. Parker and L. O. Chua, Practical Numerical Algorithms for Chaotic Systems, New York: Springer-Verlag, 1989.
3. Academic American Encyclopaedia, Vol. 4, Danbury, CT: Grolier, 1986, p. 283. 4. G. Chen and X. Dong, From Chaos to Order: Methodologies, Perspectives, and Applications, Singapore: World Scientic, 1998. 5. G. Chen and J. L. Moiola, An overview of bifurcation, chaos and nonlinear dynamics in control systems, J. Franklin Inst., 331B: 819858, 1994. 6. G. Chen, Representation, approximation, and identication, in W. K. Chen (ed.), The Circuits and Filters Handbook, Boca Raton, FL: CRC Press, 1995, pp. 9731006. 7. S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos, New York: Springer-Verlag, 1990. 8. T. Y. Li and J. A. Yorke, Period three implies chaos, Amer. Math. Monthly, 82: 481485, 1975. 9. R. L. Devaney, An Introduction to Chaotic Dynamical Systems, Reading, MA: Addison-Wesley, 1987. 10. J. L. Moiola and G. Chen, Hopf Bifurcation Analysis: A Frequency Domain Approach, Singapore: World Scientic Pub., 1996. 11. G. Chen and X. Dong, From chaos to orderperspectives and methodologies in controlling chaotic nonlinear dynamical systems, Int. J. Bifur. Chaos, 3: 13631409, 1993. 12. M. J. Ogorzalek, Taming chaospart II: Control. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., 40: 700706, 1993. 13. E. Ott, C. Grebogi, and J. A. Yorke, Controlling chaos, Phys. Rev. Lett., 64: 11961199, 1990. 14. J. M. Ottino, The Kinematics of Mixing: Stretching, Chaos, and Transport, New York: Cambridge Univ. Press, 1989. 15. W. J. Freeman, The physiology of perception, Sci. Amer., 7885, Feb. 1991. 16. S. J. Schiff et al., Controlling chaos in the brain, Nature, 370: 615620, 1994. 17. A. L. Goldberger, Applications of chaos to physiology and medicine, in J. Kim and J. Stringer (eds.), Applied Chaos, New York: Academic Press, 1992, pp. 321331. 18. A. Garnkel et al., Controlling cardiac chaos, Science, 257: 1230 1235, 1992. 19. J. N. Weiss et al., Chaos and chaos control in biology, J. Clinical Invest., 93: 13551360, 1994. 20. T. Shinbrot et al., Using small perturbations to control chaos, Nature, 363: 411417, 1993. 21. D. Auerbach et al., Controlling chaos in high dimensional systems, Phys. Rev. Lett., 69: 34793482, 1992. 22. E. A. Jackson, On the control of complex dynamic systems, Physica D, 50: 341366, 1991. 23. E. A. Jackson, OPCL migration controls between ve attractors of the Chua system, Int. J. Bifur. Chaos, 5: 12551260, 1995. 24. G. Chen, On some controllability conditions for chaotic dynamics control, Chaos, Solitions & Fractals, 8: 14611470, 1997. 25. E. H. Abed, H. O. Wang, and R. C. Chen, Stabilization of period doubling bifurcations and implications for control of chaos. Physica D, 70: 154164, 1994. 26. A. Tesi et al., Harmonic balance analysis of period-doubling bifurcations with implications for control of nonlinear dynamics, Automatica, 32: 12551271, 1996. 27. J. Ford, What is chaos, that we should be mindful of it?, in P. Davies (ed.), The New Physics, New York: Cambridge Univ. Press, 1989, pp. 348372. 28. W. Yang et al., Preserving chaos: Control strategies to preserve complex dynamics with potential relevance to biological disorders. Phys. Rev. E., 51: 102110, 1995. 29. G. Chen and D. Lai, Feedback control of Lyapunov exponents for discrete-time dynamical systems, Int. J. Bifur., Chaos, 6: 1341 1349, 1996.
GUANRONG CHEN
University of Houston
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Controllability and Observability Standard Article A. C. Antoulas1, E. D. Sontag2, Y. Yamamoto3 1Rice University, Houston, TX, 2Rutgers University, New Brunswick, NJ, 3Kyoto University, Kyoto, Japan Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1006 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (265K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Linear Finite-Dimensional Systems Nonlinear Finite-Dimensional Systems Linear Infinite-Dimensional Systems Keywords: reachability; controllability; reachability matrix; reachability grammian; observability; reconstructibility; observability matrix; observability grammian; duality; accessibility; Lie algebra accessibility; topological observability; exact observability About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
264
CONTROLLABILITY AND OBSERVABILITY
one may deduce the evolution of x. We will illustrate these concepts for three different cases: (1) linear nite-dimensional systems, (2) nonlinear systems, and (3) linear innite dimensional systems. We remark that one could also study a more general type of dynamic system, namely one given by implicit equations on a set of abstract variables w: f ( n w, , w, w ) = 0 Two examples are (3)
u w= y
or
u w = x y

In this article, we will consider dynamic systems described by a set of rst order differential or difference equations, together with a set of algebraic equations: d x(t ), t R x = f (x, u ) where ( x )(t ) := dt (1) x(t + 1 ), t Z y = h(x, u ) (2)
where u, x, and y are, as stated, the input, state, and output, respectively. Let B denote the behavior of this system, that is, the set of all time trajectories w which satisfy Eq. (3). This leads to the so-called behavioral theory of dynamic systems (1). Let w and w denote, respectively, the past and future of a given trajectory with respect to some xed time t0 (with w restricted to t t0 and t t0). The system described by Eq. (3) is said to be controllable, if for any w1, w2 belonging to B , any trajectory composed of the concatenation of (w1) and (w2) belongs to B . This approach has actually been worked out for the case where f is a linear and time-invariant function of its arguments. This provides a generalization of state controllability for the case where states are not available, and without distinguishing between inputs and outputs. It turns out that this concept is indeed the generalization of the concept of state controllability when Eq. (3) is Eq. (1). This will be discussed briey in a later section of this article. For reasons of brevity and space, the proofs of the various results are omitted in the sections that follow. The interested reader is referred to the original sources. LINEAR FINITE-DIMENSIONAL SYSTEMS The system described by the state equations Eq. (1) is linear, if f (x, u) Fx Gu: x = Fx + Gu (4)
Here, u is the input or excitation function, and its values u(t) belong to a Euclidean space m. The state at time t is x(t) and, depending on the context, may belong to a nite or an innite dimensional space. The output or measurement function y takes values y(t) p. When u(t) 0, one interprets the equation x f (x, 0) as describing the evolution of the system in the absence of inputs. Observe that, in contrast to the classical study of dynamic systems, where inputs (or forcing functions) are xed, the object of control theory is to study the effect of different input functions on the system , that is, on the solutions of the difference or differential Eq. (1). In the sequel questions of state controllability and state observability will be investigated. Roughly speaking, the former questions have to do with the extent to which the choice of u can inuence the evolution of x, while the latter questions are concerned with the extent to which by observing y and u
where as in Eq. (1), denotes the derivative operator for continuous-time systems and the (backwards) shift operator for discrete-time systems. The input u(t) and state x(t) of the system at time t belong, respectively, to the input space U m and state space X n. Moreover, G : U X, F:X X
are linear maps; the rst is called the input map, while the second describes the dynamics or internal evolution of the system. The output Eq. (2), for both discrete- and continuous-time linear systems, is composed of a set of linear algebraic equations y = Hx + Ju (5)
265
where y(t) is the output (response) at time t, and belongs to the output space Y p; furthermore: H : X Y, J :U Y
then Eqs. (4) and (5) when expressed in terms of the new state x , will become x = TFT 1 x + TG u,
F G
y = HT 1 x + Ju
H
are linear maps; H is called the output map. It describes how the dynamics affect the output, while J describes how the input affects the output directly (i.e., without passing through the state). For simplicity of exposition, we will assume that the systems considered are time-invariant, that is to say, there is no explicit dependence of the system matrices on time. Thus the term linear will be used in this section to denote a linear, time-invariant, continuous- or discrete-time system which is nite-dimensional. Linearity means: U, X, Y are linear spaces, and F, G, H, J are linear maps; nite-dimensional means: U, X, Y are all nite dimensional; time-invariant means: F, G, H, J do not depend on time; their matrix representations are constant n n, n m, p n, p m matrices. We are now ready to give Denition 1. (a) A linear system in state space description is a quadruple of linear maps (matrices)
where J remains unchanged. The corresponding triples are called equivalent. Remark. The material that follows was rst introduced by R. E. Kalman (2), (3); see also Refs. 4 and 5. For a more recent treatment, we refer to the book by Sontag (6); see also Refs. 7,8, and 9. The State Controllability Problem There are two fundamental concepts associated with the state controllability problem: reachability and controllability. These concepts allow us to answer questions concerning the extent to which the state of the system x can be manipulated through the input u. We will rst discuss the concept of state reachability for linear systems. The related concept of controllability will also be discussed. Both concepts involve only the state equations. Consequently, for this subsection, H and J will be ignored. Denition 2. Given = F G
:=
F H
G J
, F Rn n , G Rn m , H R p n , J R p m (6)
The dimension of the system is dened as the dimension of the associated state space: dim =n (7)
(b) is called (asymptotically) stable if the eigenvalues of F have negative real parts or lie strictly inside the unit disk, depending on whether is a continuous-time or a discretetime system. Let (u; x0; t) denote the solution of the state Eq. (4), that is, the state of the system at time t attained starting from the initial state x0 at time t0, under the inuence of the input u. For the continuous-time, time-invariant state equations,
F nxn, G nxm, a state x X is reachable from the zero state iff there exist an input function u(t) and a time T , such that x = (u; 0; T ) The reachable subspace Xreach X of is the set which contains all reachable states of . We will call the system (completely) reachable iff Xreach X. Furthermore Rn (F, G ) := [G FG F 2 G F n 1 G] (11)
(u; x0 ; t ) = eF (t t 0 ) x0 +
t t0
eF (t ) Gu( ) d , t t0
(8)
will be called the reachability matrix of . A useful concept is that of the reachability grammian. Complex conjugation and transposition will be denoted by *. Denition 3. The nite reachability Grammian at time t is dened as follows. For continuous-time systems:
t
while for the discrete-time state equations,
(u; x0 ; t ) = F t t 0 x0 +
t 1 j =t 0
F t 1 j Gu( j ), t t0
(9)
For both discrete- and continuous-time systems the output is given by: y(t ) = H (u; x(0 ); t ) + Ju(t ) = H (0; x0 ; t ) + H (u; 0; t ) + Ju(t ) (10)
P (t ) :=
0
eF GG eF d , t > 0
(12)
while for discrete-time systems

P (t ) := Rt (F, G )Rt (F, G ) = t 1 k=0
F k GG (F )k , t > 0
(13)
If we transform the state under a linear change of coordinates, the corresponding matrices describing the system will change. In particular, if the new state is x := Tx, det T = 0
We will make use of the following input w ,T (t ) := G eF

(T t )
, R n , t , T R, 0 t T
(14)
266
for the continuous-time case, and w ,T (t ) := G (F )T t , Rn , t , T Z, 0 t T (15)
Proposition 1. The reachability Grammians have the following properties: (a) P (t) P *(t) 0, and (b) their columns span the reachability subspace, that is im P (t ) = im Rn (F, G ) This relationship holds for continuous-time systems for all t 0, and for discrete-time systems (at least for t n). Corollary 2
for the discrete-time case. A concept which is closely related to reachability is that of controllability. Here, instead of driving the zero state to a desired state, a given nonzero state is steered to the zero state. More precisely we have: Denition 4. Given = F G
a (nonzero) state x X is controllable to the zero state if and only if there exist an input function u(t) and a time T , such that (u; x; T ) = 0 The controllable subspace Xcontr of is the set of all controllable states. The system is (completely) controllable if and only if Xcontr X. The fundamental result concerning reachability is the following Theorem 1. Given = F G
is reachable if and only if P (t) is positive denite for some t 0. The energy f of the vector function f , dened on an interval I or , is dened as
:= f , f :=
I I
f ( ) f ( ) d , t R
f ( ) f ( ), t Z
The input function w,T dened by Eqs. (14) and (15) has the following property: Proposition 2. Given the reachable state x Xreach, let be any input function which reaches x at time T, that is, u (u ; 0; T) x. There exists n satisfying: x = P (T ) (17)
for both the continuous- and discrete-time case, Xreach is a linear subspace of X, given by the formula X reach = im Rn (F, G ) (16)
It follows that w,T dened by Eqs. (14) and (15) reaches x at time T; moreover this is the minimum energy input which achieves this: u w , T (18)
where imR n denotes the image (span of the columns) of R n. X . (b) is (completely) reachCorollary 1. (a) FX able if, and only if, rank R n(F, G) n. (c) Reachability is basis independent.
reach reach
The minimum energy required to reach the state x at time T is equal to the energy of the input function w,T. If the system is reachable this energy is equal to: w , T = x P ( T ) 1 x (19)
In general, reachability is an analytic concept. The previous theorem, however, shows that for linear, nite-dimensional, time-invariant systems, reachability reduces to an algebraic concept depending only on properties of F, G and in particular on the rank of the reachability matrix R n(F, G), but independent of time and the input function. It is also worthwhile to note that Eq. (16) is valid for both continuousand discrete-time systems. This, together with a similar result on observability [see Eq. (23)], has as a consequence the fact that many tools for studying linear systems are algebraic. It should be noticed however that the physical signicance of F and G is different for the discrete- and continuous-time cases; if for instance we discretize the continuous-time system
From the previous considerations the length of time needed to reach a given reachable state can be derived. Proposition 3. Given is = F G
(a) For discrete-time systems, every reachable state can be reached in at most n time-steps. (b) For continuous-time systems, every reachable state can be reached in any arbitrary positive length of time. The second part of the proposition shows that ideally, in continuous-time linear systems, every reachable state can be reached arbitrarily fast. In a practical situation, the extent to which this is not possible gives a measure of how signicant
d x(t ) = Fcontx(t ) + Gcontu(t ), to dt x(t + 1 ) = Fdiscrx(t ) + Gdiscru(t )

then Fdiscr eFcont.
267
the nonlinearities of the system are. We conclude this subsection by stating a result on various equivalent conditions for reachability. Theorem 2 Reachability Conditions. The following are equivalent: 1. = F G
ability of x1 implies the existence of u1, T1 such that x1 (u1; 0; T1). The function u12 is then the concatenation of u2 with u1, while T12 T1 T2. In general, if x1, x2 are not reachable, there is a trajectory passing through the two points if, and only if, x2 f (F, T )x1 X reach, for some T,
where f (F, T) eFT for continuous-time systems and f (F, T) FT for discrete-time systems. This shows that if we start from a reachable state x1 0 the states that can be attained are also within the reachable subspace. The State Observation Problem In order to be able to modify the dynamics of a system, very often the state x needs to be available. Typically however the state variables are inaccessible and only certain linear combinations y, given by the output Eqs. (5), are known. Thus, the problem of reconstructing the state x(T) from observations y(), where is in some appropriate interval, arises. If [T, T t], we have the state observation problem, while if [T t, T] we have the state reconstruction problem. We will rst discuss the observation problem. Without loss of generality we will assume that T 0. Recall Eqs. (8), (9), and (10). Since the input u is known, the latter two terms in Eq. (10) are also known. Therefore, in determining x(0) we may assume without loss of generality that u( ) 0. Thus, the observation problem reduces to the following: given H(0; x(0); t) for t 0 or t 0, nd x(0). Since G and J are irrelevant, for this subsection = F H , F Rnm , H R pxn
F nxn, G nm is reachable. 2. The rank of the reachability matrix is full: rank R n(F, G) n. 3. The reachability Grammian is positive denite, that is, P (t) 0, for some t 0. 4. No left eigenvector v* of F is in the left kernel of G: v F = v v G = 0 5. rank(In F G) n, for all 6. The polynomial matrices sI F and G are left coprime. The fourth and fth conditions in this theorem are known as the PHB or PopovHautusBelevich tests for reachability. The equivalence of the fth and sixth conditions is a straightforward consequence of the theory of polynomial matrices; it will not be discussed in this article. Remark. Reachability is a generic property. This means intuitively that almost every n n, n m pair of matrices F, G satises rank Rn (F, G ) = n Put in a different way, in the space of all n n, n m pairs of matrices, the unreachable pairs form a hypersurface of measure zero. The next theorem shows that for continuous-time systems, the concepts of reachability and controllability are equivalent while for discrete-time systems the latter is weaker. This is easily seen by considering the system with state equation: x(t 1) 0. Clearly, for this system all states are controllable, while none is reachable. Often for this reason, only the notion of reachability is used. Theorem 3. Given is = F G
Denition 5. A state x X is unobservable iff y(t) H(0; x; t) 0, for all t 0, that is, iff x is indistinguishable from the zero state for all t 0. The unobservable subspace Xunobs of X is the set of all unobservable states of . is (completely) observable iff Xunobs 0. The observability matrix of is H HF HF 2 O n ( H , F ) := (20) . . . HF n 1
Denition 6. Let = F H
(a) For continuous-time systems Xcontr X reach. (b) For discretetime systems X reach. X contr; in particular X contr X reach ker F n. Remark. It follows from the previous results that for any two states x1, x2 X reach there exist u12, T12 such that x1 (u12; x2; T12). Since x2 is reachable it is also controllable; thus there exist u2, T2 such that (u2; x2; T2) 0. Finally, the reach-
The nite observability Grammians at time t are:

t
Q (t ) :=
0
eF H HeF d , t > 0
(21) (22)
Q (t ) := Ot (H , F )Ot (H , F ), t > 0 for continuous- and discrete-time systems, respectively.
268
Denition 7. A state x X is unreconstructible iff y(t) H(0; x; t) 0, for all t 0, that is, iff x is indistinguishable from the zero state for all t 0. The unreconstructible subspace X unrecon of X is the set of all unreconstructible states of . is (completely) reconstructible iff X unrecon 0. We are now ready to state the main theorem Theorem 4. Given = F H
unobs
(a) For t , Xunrecon Xunobs. (b) For t , Xunrecon Xunobs, in particular, Xunrecon Xunobs im Fn. The Duality Principle in Linear Systems The dual of a linear system = F H G J
is dened as follows. Let U*, X*, Y* be the dual spaces of the input U, state X, output Y spaces of . Let F : X X , G : X U , H : Y X , J : Y U is a linbe the dual maps to F, G, H, J. The dual system * of is
for both continuous- and discrete-time systems, X ear subspace of X given by
X unobs = ker On (H , F ) = {x X : HF i 1 x = 0, i > 0}
(23)
:=
F G
H J
R(n + m ) (n + p )
(25)
As an immediate consequence of the last formula we have Corollary 3. (a) The unobservable subspace Xunobs is F-invariant. (b) is observable iff, rank O n(H, F) n. (c) Observability is basis independent. Remark. Given y(t), t 0, let Y0 denote the following np 1 vector:
that is, the input map is given by H*, the output map by G*, and the dynamics are given by F*. Correspondingly the input, state, and output spaces of * are Y*, X*, U*. The matrix representations of F*, H*, G*, J* are the complex conjugate transposes of F, H, G, J, respectively, computed in appropriate dual bases. One may think of the dual system * as the system where the role of the inputs and the outputs has been interchanged, or the ow of causality has been reversed. The main result is the duality principle. Theorem 5. The orthogonal complement of the reachable subspace of is equal to the unobservable subspace of its dual *: (X reach) = X unobs Corollary 4. The system is reachable iff its dual * is observable. It can also be shown that controllability and reconstructibility are dual concepts. We conclude this subsection by stating the dual to theorem 2. Theorem 6. Observability conditions. The following are equivalent: 1. = F H
Y0 := (y (0 ) Dy (0 ) Dn 1 y (0 )) , t R
where D : d / dt. The observability problem reduces to the solution of the linear set of equations On (H , F )x(0 ) = Y0 This set of equations is solvable for all initial conditions x(0), that is, it has a unique solution iff, is observable. Otherwise x(0) can only be determined modulo Xunobs, that is, up to an arbitrary linear combination of unobservable states. It readily follows that ker Q (t) ker O n(H, F). As in the case of reachability, this relationship holds for continuoustime systems for t 0 and for discrete-time systems, at least for t n. The energy of the output function y at time T, generated from the initial state x will be denoted by y. In terms of the observability Grammian this energy can be expressed as y = x Q ( T )x (24)
Y0 := (y (0 ) y (1 ) y (n 1 )) , t Z
We now briey turn our attention to the reconstructibility problem. The main result which follows shows that while for continuous-time systems the concepts of observability and reconstructibility are equivalent, for discrete-time systems the latter is weaker. For this reason, the concept of observability is used most of the time. Proposition 4. Given is = F H
H pn, F nn is observable. 2. The rank of the observability matrix is full: rank O n(H, F) n. 3. The observability Grammian is positive denite Q (t) 0, for some t 0. 4. No right eigenvector v of F is in the right kernel of H: Fv = v Hv = 0 5.
rank
for all
In F H
=n
269
6. The polynomial matrices sI F and H are right coprime. Canonical Forms A nonreachable system can be decomposed in a canonical way into two subsystems; one whose states are all reachable and a second whose states are all unreachable. The precise result is stated next. Lemma 1. Reachable canonical decomposition. Given is = F G
Lemma 2 Observable canonical decomposition. Given is = F H
There exists a basis in X such that F, H have the following matrix representations Fo Foo F = 0 Fo H 0 Ho where
o
There exists a basis in X such that F, G have the following matrix representations:
Fo Ho
Fr = 0
Frr Fr
Gr 0
is observable. The reachable and observable canonical decompositions given in lemmas 1 and 2 can be combined to obtain the following decomposition of the triple (H, F, G): Lemma 3. Reachable-observable canonical decomposition. Given = F H G
(26)
where the subsystem

r
:=
Fr
Gr
is reachable. Thus every system = F G
there exists a basis in X such that F, G, and H have the following matrix representations
= F H G =
can be decomposed in a subsystem

r
Fr
Gr where the triple
Fro 0 0 0 0
F12 Fro 0 0 Hro
F13 0 Fro 0 0
F14 F24 F34 Fro Hro
Gro Gro 0 0
(27)
which is reachable, and in a subsystem

ro r
:=
Fr
Fro Hro
Gro
is both reachable and observable. which is completely unreachable, that is, it cannot be inuenced by outside forces. The interaction between r and r is given by Frr. Since Frr 0, it follows that the unreachable subsystem r inuences the reachable subsystem r but not vice versa. It should be noticed that although the direct complement X of Xreach is not unique, the form of the reachable decomposition of Eq. (26) is unique. Since by duality F G A concept related to, but weaker than reachability, is that of stabilizability. Its dual is detectability. Denition 8. The pair F G
is stabilizable iff in the reachable canonical decomposition, Fr is stable, that is, all its eigenvalues have either negative real parts or are inside the unit disk, depending on whether we are dealing with continuous- or discrete-time systems. F H is detectable iff in the observable canonical decomposition, Fo is stable.
is reachable if, and only if, F G is observable, we obtain the following results.
270
The Innite Grammians Consider a continuous-time linear system

c
and is equal to:

2 Eo = x Q x
(34)
F H
which is stable, that is, all eigenvalues of F have negative real parts. In this case both Eqs. (12) as well as (21) are dened for t ; P :=
0
We summarize these results in the following proposition which is important in the theory of balanced representations and Hankel-norm model reduction. Lemma 4. Let P and Q denote the innite grammians of a linear stable system. (a) The minimal energy required to steer the state of the system from 0 to x is given by Eq. (33). (b) The maximal energy produced by observing the output of the system whose initial state is x is given by Eq. (34). (c) The states which are difcult to reach, that is, require large amounts of energy, are in the span of those eigenvectors of P which correspond to large eigenvalues. Furthermore, the states which are difcult to observe, that is, produce small observation energy, are in the span of those eigenvectors of Q which correspond to small eigenvalues. (d) The eigenvalues of the product of the reachability and of the observability grammians are input-output invariants called Hankel singular values of . Controllability in the Behavioral Framework A dynamical system in the classical framework is a mapping which transforms inputs u into outputs y. In many cases however, the distinction between inputs and outputs is not a priori clear. Consider for example the RLC (ResistorInductor Capacitor) network presented in the next section, and suppose that we are interested in the relationship between the current through the resistor RC and the voltage across the capacitor C. Is the voltage causing the current or vice versa? Other than the often encountered inherent difculty in distinguishing between inputs and outputs, it is desirable to have a framework in which the different representations of a given system (for example: input-output and input-state-output) are treated in a unied way. The need for a framework at a more abstract level than is provided by the inputoutput framework gave rise to the behavioral framework. For a tutorial account see Refs. (1) and (10). The variables considered are the external or manifest variables w and (possibly) a second set of variables, the socalled latent variables a. The manifest variables consist of u and y, without distinguishing between them. In the behavioral theory, a dynamical system is dened as a collection B of trajectories w. This set B , called the behavior of the system, is the primary object of study for system and control theoretic issues. In this section we will provide an overview of controllability in the behavioral framework. For further details and proofs, see Polderman and Willems (1). The trajectories w composing the behavior B are most often represented as solutions of appropriate equations, called behavioral equations; these are equations providing relationships between w and w, where is dened by Eq. (1). The most important type of such equations are the annihilating behavioral equations. An important special case of such be-
eF GG eF d ,
Q :=
0
eF H HeF d
(28)
P , Q are the innite reachability and innite observability grammians associated with c. These grammians satisfy the following linear matrix equations, called Lyapunov equations. Proposition 5. Given the stable, continuous-time system c as stated, the associated innite grammians P , Q satisfy the continuous-time Lyapunov equations F P + P F + GG = 0, F Q + QF + H H = 0 (29)
If the discrete-time system

d
F H
is stable, that is, all eigenvalues of F are inside the unit disk, the grammian Eqs. (13) as well as (22) are dened for t P :=
t >0
F t 1 GG (F )t 1 ,
Q :=
t >0
(F )t 1 H HF t 1
(30)
Notice that P can be written as P GG* FP F*; moreover Q H*H F*Q F. These are the so-called discrete Lyapunov or Stein equations: Proposition 6. Given the stable, discrete-time system d as stated, the associated innite grammians P , Q satisfy the discrete-time Lyapunov equations F P F + GG = P , F QF + H H = Q (31)
Recall Eqs. (18), (19), and (24), valid for both discrete- and continuous-time systems. From the denition of the grammians follows that: P (t2 ) P (t1 ), Q (t2 ) Q (t1 ), t2 t1 (32)
irrespective of whether we are dealing with discrete- or continuous-time systems. Hence from Eq. (19) it follows that the minimal energy Er, required for the transfer of state 0 to x, is obtained as the alloted time T tends to innity. Assuming reachability, this minimal energy is: Er2 = x P 1 x (33)
Similarly, the largest observation energy Eo, produced by the state x is also obtained for an innite observation interval,
271
havioral equations are: state variable (SV) equations; SV equations, in addition to w, make use of the latent variables a x, which can be assigned the property of state, and are called state variables. Another is the inputoutput equation representation, which thus appears as a special case of a more general system representation. For linear, time-invariant systems, annihilating behavioral equations representing have the following form. Let [s] denote the ring of polynomials in the indeterminate s with coefcients in , and n1n2[s] denote the n1 n2 polynomial matrices. The resulting equation has the form: R( )w = M ( )a, R R pq [s], M R pr [s] It relates (35)
w (t ) =
w 1 (t ) for t < 0 w 2 (t ) for t > t
In terms of annihilating behavioral equation representations such as Eq. (35), the corresponding system is controllable if, and only if, the rank of the (constant) matrix rank [R() M ()] = constant, C (36)
From this follows the condition for controllability of the SV and input/output models. There holds, respectively: rank [I F G] = constant = n, rank [N () D()] = constant = p, C
w B a
to its shifts or derivatives. This equation can be written explicitly in terms of the coefcient matrices of R and M. Let
Notice that the rst condition stated here is the same as condition 5 of theorem 2. Thus the behavioral denition of controllability provides a generalization of the classic concept. Furthermore the second condition provides a way of dening controllability without the denition of state. Examples 1. The previous issues are illustrated by means of two examples. First, consider an RLC circuit composed of the parallel connection of two branches: the rst branch is composed of an inductor L in series with a resistor RL; the second is composed of a capacitor C in series with a resistor RC. The driving force u is a voltage source applied to the two branches. Let the state variables x1, x2 be the current through the inductor, the voltage across the capacitor respectively. The state equations are
R ( s ) :=
i= 0
R i s i , R i R p q , M ( s ) :=
i= 0
M i si , M i R p r
Equation (35) becomes:

1
Ri ( i w )(t ) =
Mi ( i a )(t )
i= 0
i= 0
Since the differential or difference operator [R() M()] annihilates all trajectories
w B a
Eq. (35) is referred to as an annihilating behavioral equation. The special case of SV Eq. (4) is described by w u, a x, R() G and M(s) I F; while that of input/output equations is described by
w=
u y
RL 1 d R L x = x + u 0 dt 1 L 1 L L F= 1 d 1 1 0 x = x + u RC C dt 2 RCC 2 RCC 1 L G= 1 RCC

Since this is a continuous-time system, reachability and controllability are equivalent notions. The reachability matrix is
a is nonexistent, and R() [N() D()], where D is a square, nonsingular polynomial matrix of size p. A further important aspect at which the behavioral formalism departs from, and generalizes, the classical formalism is that related to controllability; controllability becomes namely an attribute of the system (i.e., of a collection of trajectories) as opposed to an attribute of a system representation (i.e., of equations generating these trajectories). Roughly speaking, a system is controllable if its behavior has the property: whatever the past history (trajectory), it can always be steered to any desired future trajectory. More precisely, a dynamical system with behavior B is said to be controllable, if for any w1, w2 B , there exists a t 0 and a w B such that
1 L R2 = [G FG] = 1 RCC
RL 2 L 1 2 2 RCC
Thus, reachability of this system is equivalent with the nonsingularity of R 2, that is, det R2 = 1 RCCL RL 1 L RCC =0
272
It readily follows that the reachability Grammian is g2 1 ( 1 e 2 L T ) 2 L P (T ) = g1 g2 (1 e ( L + C )T ) L + C
Controllability: Continuous-Time In continuous time, we consider systems of differential equations of the following general form: d x(t ) = f (x(t ), u(t )) dt (37)
g1 g2 (1 e ( L + C )T ) L + C g2 2 ( 1 e 2 C T ) 2 C
where L = RL 1 1 , C = , g = , g 2 = C L RCC 1 L
where f : n m n is a vector function which species, for the current state variables x(t) n and the current values of the control variables u(t) m, the direction of instantaneous movement. For each xed vector u m, f ( , u) is thought of as a vector eld. Linear (time-invariant) systems are a particular case, namely those systems for which f is a linear map, f (x, u ) = Fx + Gu for some matrices F of size n n and some matrix G of size n m. An Example. A simplied model of a front-wheel drive automobile uses a four-dimensional state space. The coordinates of the state
Hence the innite reachability grammian is g2 g1 g2 1 L + C 2 L P = 2 g2 g1 g2 L + C 2 C and it can be veried that FP P F* GG* 0. Assume now that the variable observed y is the sum of the voltages across the capacitor and across the resistor RL: y = R L x 1 + x 2 H = [R L 1]
x1 x1 x x x = 2 = 2 x 3 x4
denote, respectively, the position of the center of the front axle (coordinates (x1, x2)), the orientation of the car (angle , measured counterclockwise from the positive x-axis), and the angle of the front wheels relative to the orientation of the car (, also counterclockwise); see Figure 1. As controls, we take two-dimensional vectors u col (u1, u2), whose coordinates are proportional to the steering wheel velocity (u1) and the engine speed (u2) at each instant. Thus, a control u2(t) 0 corresponds to a pure steering move, while one with u1(t) 0 models a pure driving move in which the steering wheel is xed in one position. In general, a control is a function u(t) which indicates, at each time t, the current steering velocity and engine speed. Using elementary trigonometry, the following equations are obtained (choosing units so that the distance between the
The observability matrix is 1 R L O2 = 1 det O2 = RL RC LC det R2 R2 L L RC C Thus reachability and observability are lost simultaneously. If this happens, then one can reach any given state x1 and x2 L/RL x1; while only the linear combination RLx1 x2 can be deduced, but not x1 or x2 individually. 2. Consider the system given in input/output form by the equation d d y= u, u(t ), y(t ) R dt dt We will show that this system is not controllable. This equation can be rewritten as d / dt v 0, where v y u. All trajectories composing the behavior of this system are constants. But a trajectory dened by v(t) c1, t T, and v(t) c2, t T, c1 c2, does not belong to the behavior, as its derivative is not zero. Hence, since the trajectories of this system are not concatenable, the conclusion follows. NONLINEAR FINITE-DIMENSIONAL SYSTEMS
x2
( x 1, x 2)
x1
We turn attention now to nonlinear systems. Both continuous- and discrete-time systems will be discussed.
Figure 1. Four-dimensional car model.
273
front and rear axles is unity): 0 cos ( + ) 0 sin( + ) d x = u1 + u2 0 sin dt 1 0
(38)
This is of the form of Eq. (37), where f : 4 2 4. Note that, in practice, the angle would be restricted to some maximal interval (0, 0). For simplicity of exposition, we do not impose this constraint. Similarly, the orientation angle only makes sense as a number modulo 2, that is, angles differing by 2 correspond to the same physical orientation. Nonlinear control theory is usually developed in far more generality than we do here. The more general formalism allows states to evolve in general differentiable manifolds, instead of insisting, as we do here, in Euclidean state spaces. Thus, for instance, a more natural state space than 4 would be, for this example, 2 1 (0, 0), that is, the angle is thought of as an element of the unit circle. Analogously, we assume here that the controls may attain arbitrary values in 2; of course, a more realistic model would also incorporate constraints on their magnitude. Technical Assumptions. We will assume, for simplicity of exposition, that the function f is real-analytic. This means that f (x, u) can be expressed, around each point (x, u) n m, as a locally convergent power series. Analyticity is a condition which is satised in models derived from physical principles. The assumption of analyticity allows stating results in an elegant necessary and sufcient, rather than merely sufcient, manner. A control (or input) function is by denition of Lebesgue-measurable essentially bounded function u( ) dened on some interval of the form [0, T] and taking values in m (the reader may substitute piecewise continuous function without much loss of generality). We let Inp be the set of such controls. Accessibility. Consider any state n. For each control u : [0, T] m, we consider the solution of the initial problem d / dt x(t) f (x(t), u(t)), x(0) . If this solution is well-dened on the interval t [0, T], we denote the nal state x(T) as (, u). [If the solution does not exist on the entire interval, we do not dene (, u)]. The reachable set from , denoted as R (), is by denition the set of states reachable from the origin, that is, the set of states of the form {( , u ) Rn |u Inp} Denition. The system Eq. (37) is accessible from the state if the reachable set R () contains an open subset of n (that is, it has a nonempty interior). For linear systems d / dt x Fx Gu, and the zero initial state 0, R (0 ) = im (G, FG, . . ., F n 1 G ) (39)
ample, consider the system with equation d / dt x u2 (having dimension n 1 and input space also of dimension m 1). Take any initial state (for instance, 0). Clearly, R () [, ) [because d / dt x(t) 0, no matter which control function is applied]. Thus the system is accessible from every state. Observe that, as illustrated by this example, accessibility does not mean that must be in the interior of the set R () (local controllability), even for an equilibrium state. The reason that the accessibility question is studied is that it is far easier to characterize than controllability. Accessibility Rank Condition. Given any two vector elds f and g, one can associate the new vector eld [ f , g] dened by the formula [ f , g](x ) := g [x] f (x ) f [x] g(x ) where, in general, the notation h[x] means the Jacobian of a vector eld h, evaluated at the point x. This is called the Lie bracket of f and g. The Lie bracket of f and g can be interpreted in terms of certain trajectories that arise from integrating f and g, as follows. We let eth denote the solution at time t (possibly negative) of the differential equation d / dt x h(x) with initial value x(0) . (When the differential equation is linear, i.e., d / dt x Fx and F is a matrix, eth is precisely the same as etF, where etF is the exponential of the matrix F. In general, for nonlinear differential equations, eth is merely a convenient notation for the ow associated to the vector eld h.) Then, for any two vector elds f and g, etg et f etg et f = et
2 [ f ,g]
+ o(t 2 )( )
(40)
as t 0, as a simple computation shows. Therefore, one may understand the Lie bracket of f and g as the innitesimal direction that results from following solutions of f and g in positive time, followed by f and g in negative time. Another way to state this fact is by introducing the curve (t ) := e
tg t f tg tf
Observe that (0) and that the values of (t) are all in the set of points S attainable by positive and negative time solutions of the differential equations corresponding to f and g; the above expansion implies that d / dt (0) [f , g](), that is, there is a curve in S whose tangent is the Lie bracket of the two vector elds. Thus, Lie brackets provide new directions of innitesimal movement in addition to those corresponding to f and g themselves (and their linear combinations). Given now a system by Eq. (37), we consider, for each possible control value u m, the following vector eld: f u : Rn Rn : x f (x, u ) The accessibility Lie algebra L associated to the system given by Eq. (37) is the linear span of the set of all vector elds that can be obtained, starting with the f us and taking all possible iterated Lie brackets of them. For instance, if u1, u2, u3, u4 are
(where im indicates the span of the columns of the matrix) is a linear subspace. so accessibility from 0 is equivalent to complete controllability, that is, R (0) n. However, for nonlinear systems, the accessibility property is weaker. For ex-
274
any four control values, the vector eld [[ f u 1 , [ f u 2 , f u 3 ]], [ f u 3 , f u 4 ]] is in L . The system Eq. (37) satises the accessibility rank condition at the state if the vector space L ( ) := {X ( ), X L } Rn has the maximal possible dimension, that is, n. There is a special case, of interest because it appears very often in applications, especially to mechanical systems. This is the class consisting of systems for which f (x, u) is afne in u. That is, the equations can be written as d x = g0 ( x ) + dt
m
ing (11)) steer and drive respectively: 0 cos ( + ) 0 sin( + ) g1 = , g2 = 0 sin 1 0 Computing some brackets, we get the two new vector elds [again, we borrow our terminology from (11)]:
sin( + ) cos ( + ) wriggle := [steer, drive] = cos 0 and
u i gi ( x )
i= 1
(41)
for some vector elds gis. It is easy to verify that, for such systems, L is the Lie algebra generated by taking all possible iterated Lie brackets starting from the gis. For example, consider a linear system d x = Fx + Gu dt Here g0(x) Fx is a linear vector eld and the gi(x)s are the constant vector elds dened by the m columns of the matrix G. It then follows that, for each state , L () is the span of the vector F together with the columns of G, FG, . . ., Fn1G. In particular, for 0, one has that L (0) is the same as the right-hand side of Eq. (39). Seen in that context, the following result, which is valid in general, is not surprising: Theorem. The system Eq. (37) is accessible from if and only if the accessibility rank condition holds at . There is a subclass of systems for which far stronger conclusions can be drawn. This subclass includes all purely kinematic mechanical models. It is the class of afne systems without drift, that is, systems afne in u (as in Eq. 41, but for which, in addition, g0 0). We say that a system is completely controllable if R () n for every n, that is to say, every state can be steered to every other state by means of an appropriate control action. Theorem. A system afne without drift is completely controllable if and only if the accessibility rank condition holds at every state. This characterization of controllability for systems without drift belongs more properly to classical Lie theory and differential geometry. As far as control theory is concerned, most interesting questions concern more general classes of systems as well as the design of explicit algorithms for controllability, sometimes imposing optimality constraints. The Car Example. In the notations for systems afne in controls, we have the vector elds g1 and g2, which we call (follow-
sin cos slide := [wriggle, drive] = 0 0

(The bracket [wriggle, steer] equals drive, so it is redundant, in so far as checking the accessibility rank condition is concerned.) It turns out that these four brackets are enough to satisfy the accessibility test. Indeed, one computes det(steer, drive, wriggle, slide) 1 so there is accessibility from every state. Moreover, since this system is afne without drift, it is completely controllable. (Of course, it is quite obvious from physical reasoning, for this example, that complete controllability holds.) Consider in particular the problem of accessibility starting from the special state 0 (corresponding to the problem of exiting from a parallel parked spot). For 0, wriggle is the vector (0, 1, 1, 0), a mix of sliding in the x2 direction and a rotation, and slide is the vector (0, 1, 0, 0) corresponding to sliding in the x2 direction. This means that one can in principle implement innitesimally both of these motions. The wriggling motion is, based on the characterization of Lie brackets mentioned earlier, the one that arises, in a limiting sense, from fast repetitions of the following sequence of four basic actions: steer drive reverse steer reverse drive ( ) This is, essentially, what one does in order to get out of a tight parking space. Observe that wriggle(0) equals the sum of slide and rotate [a pure rotation, col(0, 0, 1, 0)]. Interestingly enough, one could also approximate the pure sliding motion in the x2 direction: wriggle, drive, reverse wriggle, reverse drive, repeat corresponds to the last vector eld. Note that the term t2 in Eq. (40) explains why many iterations of basic motions (*) are required in order to obtain a displacement in the wriggling direction: the order of magnitude t2 of a displacement in time t is much smaller than t. Remark. If the right-hand side f in Eq. (37) is assumed merely to be innitely differentiable, instead of analytic, the
275
accessibility rank condition is still sufcient for accessibility, but it is not a necessary condition. Consider for instance the system on 2, with 2 also as control space, and having equations as follows:
where ei denotes the ith coordinate vector, and more generally for all u, i and each integer k 0 let
k (Ad0 Xu,i )(x ) :=
=0
k 1 f0 fu fu +
ei
k f0 (x )
1 0 d + u2 x = u1 dt 0 (x1 )
where is the function dened by (x ) = e 1/x
2
The accessibility Lie algebra is now dened in terms of iterated Lie brackets of these vector elds, and the accessibility rank condition is dened in terms of this, analogously to the continuous time case. The main fact is, then, as follows. Theorem. The system Eq. (36) is accessible from zero if and only if the accessibility rank condition holds at zero. As in the continuous-time case, for linear (discrete time) systems, the condition reduces to the usual reachability test. k Xu,i are in fact all of the type FkGu, for vectors The vectors Ad0 u m. Accessibility and Controllability of Linearized Systems It is easy to prove that, for both continuous and discrete time systems, if the linearization about an equilibrium point is controllable as a linear system, then the accessibility condition holds, and, in fact, the system is locally controllable, that is, is in the interior of R (); see for example, Ref. 6. For instance, each state near zero can be reached from zero, for the system
for x 0, and (x) 0 for x 0. This system is easily shown to be accessiblein fact, it is completely controllable (any state can be steered to any other state)but the accessibility rank condition does not hold. Controllability: Discrete-Time We next consider discrete time systems. These are described by difference equations analogous to those for Eq. (37): x(t + 1 ) = f (x(t ), u(t )) (42)
where f : n m n is a function. This function now plays the role of specifying the state at time t 1, provided that the state at time t was x(t) and the control vector u(t) was applied at that instant. We again suppose that the function f is real-analytic. Now the set of controls, denoted again Inp, is the set of all possible sequences u(0), . . ., u(T) consisting of vectors in m. An additional assumption which we make for the discrete time system of Eq. (42) is that it is invertible, meaning that the map f ( , u ) is a diffeomorphism for each xed u; in other words, this map is bijective and has a nonsingular differential at each point. Imposing invertibility simplies matters considerably, and is a natural condition for equations that arise from the sampling of continuous time systems, which is one of the main ways in which discrete time systems appear in practice. Accessibility is dened as in the continuous-time case, using the analogous denition of R (). We discuss only the special case 0 (the general case is a bit more complicated), assuming that this state is in equilibrium for the system, that is, f ( 0, 0 ) = 0 There is an analogue of the accessibility rank condition for discrete time systems, and this is discussed next. The notation f u is as stated earlier, and in particular f 0 is the map f ( , 0). Recall that in the discrete case one assumes 1 invertibility, so that the inverse maps f u are well-dened and again analytic. For each i 1, . . ., m and each u m let Xu,i (x ) :=
1 fu fu + e i (x )
d x = x2 1 x2 + sin x1 dt 1 d x = x1 ex 2 + u cos x1 dt 1
2 because, up to rst order around 0 one has x1 x2 sin x2 x1 x1 h1, x1e u cos x1 x1 u h1, where h1 and h2 are higher-order terms in states and controls, which means that the linearization at the equilibrium 0 is the linear system with matrices
F=
0 1
1 , 0
G=
0 1
which is controllable. This is only a sufcient condition. the system d / dt x u3, in dimension one, is clearly (even) completely controllable, but its linearization at 0 gives the noncontrollable linear system d / dt x 0. A necessary and sufcient condition does exist linking accessibility and linear controllability, but it is more subtle. It is illustrated next, for simplicity, only for discrete-time systems. In continuous time, an analogous result holds, but it is slightly more complicated to state and prove (12). Observe that accessibility from corresponds to the requirement that the union of the images of the composed maps f k ( , ) : (Rm )k Rn k 0 cover an open subset, where we are denoting f k (x, (u1 , . . ., uk )) := f ( f (. . . f ( f (x, u1 ), u2 ), . . ., uk 1 ), uk ) for every state x and sequence of controls u1, . . . uk. A simple argument, based on a standard result in analysis (Sards the-
=0
276
orem) gives that accessibility is equivalent to the following property: there exists some positive integer k and some sequence of controls u1, . . . uk so that the Jacobian of f k(, ) evaluated at that input sequence, f k ( , ) [u1 , . . . uk ], has rank n. Consequently, accessibility is equivalent to accessibility in time exactly k. This Jacobian condition can be restated as follows: Consider the linearization of the system Eq. (42) along the trajectory x1 = , x2 = f (x1 , u1 ), x3 = f (x2 , u2 ), . . . that is, the linear time-varying system
if the initial state is . It follows that the derivatives with respect to the tis of this output are also equal, for 1 and 2, and for every such piecewise constant control. One may prove by induction that
k t1 . . . tk
t 1 =t 2 = ... = 0
h j (t1 , t2 , . . ., tk , u1 , u2 , . . ., uk , ) = LX LX . . . LX h j ( )
1 2 k
x(t + 1 ) = Ft x(t ) + Gt u(t ) with Ft = f [x t , u t ], x Gt = f [x t , u t ] u
where Xl(x) g0(x) uilgi(x). This expression is a multilinear function of the uils, and a further derivation with respect to these control value coordinates shows that the generators in Eq. (43) must coincide at x1 and x2. It turns out that (for analytic vector elds, as considered in this exposition), separability by O is necessary as well as sufcient, because Eq. (44) can be expressed as a power series in terms of the generators of Eq. (43). Thus, observability is equivalent to separation by the functions in O . The observability rank condition at a state is the condition that the dimension of the span of
m i1
Then, accessibility is equivalent to the existence of some sequence of controls u1, . . . uk for which this linearization is controllable as a time-varying linear system. Observability: Continuous-Time We present a brief outline of a nonlinear observability test, for the special case of continuous-time systems afne in Eq. (41), with an output map h : n p added to the system description. Two states and are distinguishable by input/ output experiments if there is at least some input which, when applied to the system in initial state , gives a different output than when applied to the system in state . An observable system is one with the property that every pair of distinct states is distinguishable. Thus an observable system is one for which, at least in principle, it is possible to distinguish between internal states by means of input/output measurements alone. Consider the vector space spanned by the set of all functions of the type Lg i . . . Lg i h j (x )
1 k
{ Lg i . . . Lg i h j ( )|i1 , . . ., ik {0, . . ., m}, j = 1, . . ., p} Rn

1 k
be n. An application of the implicit function theorem shows that this is sufcient for the distinguishability of states near . Remarks The early 1970s saw the beginnings of the systematic study of controllability and observability questions for continuous time nonlinear systems. Building upon previous work (13,14) on partial differential equations, the papers (15), (16), and (17), among others, provided many of the basic accessibility and controllability results. In discrete time, one of the early papers was (18). For more details on accessibility at an expository level, see for instance Refs. 19, 20, 21, or 6 in continuous time, and 22 in discrete time. These references should also be consulted for justications of all statements given here without proof. For afne systems without drift, for which the accessibility rank condition completely characterizes controllability, Lie techniques can be used to provide efcient algorithms for constructing controls; see for instance Ref. 23. Similarly, there are useful controllability algorithms available for special classes of systems such as so-called at systems, and for systems exhibiting special symmetries which arise from mechanical constraints; see, e.g., Ref. 24, and references therein. A complete characterization of controllability, as opposed to the weaker accessibility property, has eluded solution, even though substantial progress has been made (see for instance Ref. 25 and the references there). One way to understand the difculty inherent in checking controllability is by formulating the problem in terms of computational complexity. In Ref. 26, it is shown that, for wide classes of systems, testing for controllability is an NP-hard problem (hence most likely impossible to ever be amenable to an efciently computable characterization); this contrasts with accessibility, which, for the same classes of systems, can be checked in polynomial time. A different type of linearization is also of interest in control theory. Instead of merely taking rst-order approxima-
(43)
over all possible sequences i1, . . ., ik, k 0, out of 0, . . ., m and all j 1, . . ., p, where Lg .g for any function and any vector eld g (f denotes the gradient of f ). This is called the observation space, which we denote as O , associated to the system. We say that two states x1 and x2 are separated by O if there exists some O such that (x1) (x2). One can prove that if two states are separated by O then they are distinguishable. A sketch of the argument is as follows. Assume that 1 is indistinguishable from 2 and consider a piecewise constant control which is equal to u1 on [0, t1), equal to u2 on [t1, t1 t2), . . ., and equal to uk on [t1 tk1, t1 tk). By indistinguishability, we know that the resulting output at time t t1 tk is equal for both. In general, we denote the jth coordinate of this output value by h j (t1 , t2 , . . ., tk , u1 , u2 , . . ., uk , ) (44)
277
tions, one searches for changes of state and input variables which render a nonlinear system linear. For example, take the nonlinear system x x2 u. This system becomes linear if we consider the new input variable v x2 u, since the equations become x v. Observe that this linearized system may be stabilized globally by means of the feedback v x. (In terms of the original variables, this means that we picked the feedback law u x2 x.) In general, nding linearizing transformations is not as obvious as in this example, of course; much theory has been devoted to the search for Liealgebraic conditions which guarantee that a system can be so transformed, or as one says, feedback linearized. This line of work started with Brockett in the late 1970s, and major results were obtained by Jakubczyk, Respondek, Hunt, Su, and Meyer, in the early 1980s. See Ref. 6 for an elementary introduction to the subject. For some related recent developments, see Ref. 27. LINEAR INFINITE-DIMENSIONAL SYSTEMS Distributed parameter systems are those where the state variable is spatially dependent. Typical examples are systems described by the heat equation, wave equation, beam equations, or delay-differential equations. Due to such a dependence on spatial variables, the state space generally becomes innite-dimensional. The term distributed parameter systems is thus often used synonymously for systems with innite-dimensional state space. In what follows, we shall also employ this convention, and discuss controllability/observability of innite-dimensional systems. A formal generalization of the nite-dimensional (continuous-time) denitions would lead to the following state equations: d x(t ) = Fx(t ) + Gu(t ) dt y(t ) = Hx(t ) + Ju(t ) (45) (46)
u(t) G0 + + + 1/s F0
v(t) ehs
v(t h) H0
y(t)
F1
Figure 2. Retarded delay-differential system.
example, F is dened only on a domain D (F) which is a dense subspace of the state space X. Another example often encountered in practice is provided by delay systems. Consider a retarded delay-differential system (see Fig. 2):
d v(t ) = F0 v(t ) + F1 v(t h ) + G0 u(t ) dt y(t ) = H0 v(t h ), v(t ) R n
(47)
This system is described by the functional differential equation
F0 d v(t ) = dt zt ( ) 0
F1 E h G0 v(t ) + u(t ) 0 zt ( ) =: Fx(t ) + Gu(t )
(48)
y(t ) = [0 H0 E h ]
x(t ) , zt ( )
zt ( ) (L2 [h, 0] )n
(49)
where the input values u(t), state x(t), and output values y(t) are elements of, for instance, Hilbert spaces. For example, if we consider the heat equation 2x x (t , ) = 2 (t , ) + G( )u(t ) t with state space L2[0, 1] with boundary condition x(t, 0) x(t, 1) 0, then F is set to be the differential operator 2 / 2 with domain D ( F ) := { x H 2 ( 0 , 1 ) : x ( 0 ) = x ( 1 ) = 0 } Here H2(0, 1), which is the natural state space for these equations, is the space of functions in L2[0, 1] whose second-order derivatives again belong to L2[0, 1]. The important point to notice is that the F operator is in general, as in this example, not dened on the whole space, and is discontinuous (unbounded). Thus, some care has to be taken when extending the nite-dimensional framework. We need to assume that F is the innitesimal generator of a strongly continuous semigroup S(t), which plays a role analogous to that played by eFt in the nite-dimensional case. Typically, as in the previous
where x(t) : [v(t) zt( )], and zt( ) is the state in the delay element ehs which is related to v(t) via zt() v(t ). Eh denotes the point evaluation operator Ehz( ) : z(h). The domain of F in Eq. (48) is [v z( )] R n (H2(h, 0))n : z(0) v. Here the input operator G is bounded but the output operator C [0 C0Eh] is not. (Point evaluation in L2 cannot be continuous.) We thus consider the abstract system Eqs. (45), (46) with x(t) in a Hilbert space X, with input/output variables u(t) and y(t) being m- and p-valued. The operators G : m X and H : X p are often assumed to be bounded. However, there are many systems that do not satisfy this property: G may not take values in X or H may be only densely dened. The system described by Eqs. (48), (49) is an example of the latter case. For brevity of exposition, however, we will not be very rigorous about this point in what follows. Also, since J plays no role in controllability/observability, we will assume J 0. Denote this system by (X, F, G, H). The solution of Eq. (45) can be written as
t
x(t ) = S(t )x0 +
S(t )Gu( ) d
0
We assume that u is square integrable. Such a solution is often called the mild solution. Controllability and Observability The technical problems that arise due to the innite-dimensionality include the following:
278
In general, canonical (controllable and observable) realizations are not unique Controllability need not guarantee stabilizability; similarly, observability need not imply existence of an observer The problems involved are mostly topological, and this is one of the reasons why there are several nonequivalent controllability/observability notions. We start by introducing controllability concepts. In analogy with the nite-dimensional case, one wants to say that a system is controllable (or reachable) if every state x X can be reached from the origin by suitable application of an input. For distributed parameter systems, however, this does not usually occur due to the innite-dimensionality of the state space. For each T 0, dene a map (T) : L2[0, T] X by
T
subspace of X and is indeed extendable to X as a continuous mapping. While H is often unbounded, (T) still happens to be continuous in many examples [for example, for delay systems such as Eqs. (48), (49)]. The system is said to be observable in bounded time T 0 if (T) is a one-to-one mapping, that is, if (T)x 0 (almost everywhere) implies x 0. It is observable if (T)x 0 for all T 0 occur only if x 0. (This concept is often called approximate observability, indicating its duality to approximate controllability.) To state the observability condition differently, dene the observability map : X L2 loc [0, ) : x HS (t )x, t0 (53)
2 where Lloc [0, ) is the space of locally square integrable functions. Then is observable if and only if is one-to-one. For nite-dimensional systems these observability notions imply the following consequences:
( T )u : =
0
S(T )Gu( )d
(50)
We say that a state x is controllable from zero in time T if x belongs to the range of (T). The system is simply said to be controllable from zero if x
T >0
Observability always implies observability in bounded time Initial states can be determined continuously from output data The latter is an important key to constructing observers and state estimators. For innite-dimensional systems, this property does not hold, and we dene a topological notion of observability as follows. The system is said to be topologically observable if is continuously invertible when its codomain is restricted to im . It is topologically observable in bounded time T 0 if the same holds of (T). (For the same reason as in the case of approximate observability, some authors adopt the terminology exact observability, again indicating its duality to exact controllability.) Topological observability requires that the initial state determination be well posed. The system given by Eqs. (48) and (49) is topologically observable. This property is also crucial in proving uniqueness of canonical realizations. That is, if we understand that a canonical realization is one that is approximately controllable and topologically observable, then there is essentially only one canonical realization for a given input/ output behavior (31). Duality Controllability and observability are dual concepts to each other. Reversing the time axis, we see that the controllable states are those in the image of the mapping :
T >0
range
(T )
The system is said to be exactly controllable (reachable) if X=

T >0
range
(T )
(51)
Unlike the nite-dimensional case, exact controllability does not occur very often. In fact, when the G operator is bounded (and takes values in X), it has nite rank, and hence it can be shown that (T) is a compact operator. Thus the righthand side of Eq. (51) cannot be equal to X as the union of images of compact operators. (Of course, this argument does not apply when G does not take values in X; extended study has been made on systems with boundary controls; see Refs. 2830.) We are thus interested in a less restrictive condition of approximate controllability (reachability). This requires that, in place of Eq. (51), X=
T >0
range
(T )
(52)
Here M denotes the closure of M in X. This means that any state x X can be approximated arbitrarily closely by controllable elements. When X is a nite-dimensional space, approximate controllability coincides with standard controllability. We now give denitions for several notions of observability. Fix T 0. Dene (T) : X L2[0, T] by (T ) : x HS(t )x, 0tT
L2 [0, T ] X : u
0
S(t )Gu(t ) dt
(54)
The adjoint of is easily computable (at least formally) as

: X L2 loc [0, ) : x G S (t )x,
t0
(55)
Because of the strong continuity of S(t), this mapping is well dened if H is a bounded operator. If H is unbounded, we generally require that this mapping be dened on a dense
Dene * : (X, F*, H*, G*) as the dual system of . Then the mapping of Eq. (55) is the observability map of *. Since a bounded linear f : X Y has dense image if and only if its adjoint satises ker f * 0, we can say that is approximately controllable if and only if * is observable. Similarly,
279
is observable if and only if * is approximately controllable. Also, is topologically observable if and only if the adjoint mapping of in Eq. (53) is a surjective mapping.
Related Concepts and Controllability Criteria When is exponentially stable in the sense that there exist M, 0 such that S(t ) Me t then it is possible to relate controllability/observability to Lyapunov equations. In this case it is possible to extend the domain of and restrict the codomain of to L2[0, ). Dene
complete with suitable feedback. Since feedback does not alter controllability, this latter condition preserves controllability. Similar spectral tests are possible for systems with spectral F operators. For example, consider the case where F is the self-adjoint operator dened by a spectral representation (32):
Fx =
n=1
rn j=1
(x, n j )n j
P := Q :=
=
0
S(t )GG S (t ) dt S (t )H HS(t ) dt
where n : n 1, 2, . . . are distinct real numbers listed in decreasing order, nj, j 1, 2, . . ., rn, n 1, 2, . . . is an orthonormal basis in X, and ( , ) denotes the inner product. Let G be given by
m
=
0
Gu :=
i= 1
gi u i ,
gi X
Then is approximately controllable if and only if P is positive denite, and observable if and only if Q is positive denite. P and Q are called controllability and observability Grammians. Actually, when is observable, Q is a unique self-adjoint solution to the Lyapunov equation Q x1 , Fx2 + Fx1 , Q x2 = Hx1 , Hx2 , x1 , x2 D ( F )
Then (F, G, H) (H is irrelevant here and hence not dened) is approximately controllable if and only if ( gm , n 1 ) ( g1 , n 1 ) . . = rn . . rank . . ( g1 , n r n ) ( gm , n r n ) Remarks Controllability/observability questions for distributed parameter systems were rst addressed by Fattorini (33). Since then numerous papers have appeared, and the literature is too vast to be listed or surveyed here. For developments up to 1978, consult Russels survey (34). The recent textbook by Curtain and Zwart (32) gives an extensive set of further references. For abstract operator settings of linear systems, see Refs. 35, 36, and 37. There is also a vast amount of literature concerning controllability of systems described by partial differential equations. For the developments along this line, consult Refs. 28, 29, and 30. We discuss here some limited references that deal with further related subjects. Controllability/observability for delay systems has received considerable attention: see the work by Datko, Delfour, Langenhop, Kamen, Manitius, Mitter, OConnor, Pandol, Salamon, Triggiani, Yamamoto, and others [references cited in (32)]. The spectral condition for approximate controllability of retarded delay systems cited previously is due to the work of Manitius and Triggiani (38,39). It is also extended to various situations including neutral systems; see, for example, Refs. 40, 41, and 42. Controllability and observability are also important in realization theory. It was shown by Baras, Brockett, and Fuhrmann (43) that an approximately controllable and observable realization need not be unique. One needs a stronger condition to guarantee uniqueness; the results cited in Ref. 31 are one example. As with nite-dimensional systems, controllability and observability are closely related to the question of coprimeness of factorizations of transfer functions. For example, a factorization is (left) coprime (in an appropriately dened sense) if a certain system associated with this factorization is approximately controllable (see, for example, Ref. 44). However, in
A similar statement holds for controllability. Null controllability refers to the property that every state can be steered back to the origin by application of a suitable input. Its dual concept is the nal state observability (or reconstructibility). Similar theory is possible for these properties. Closely related concepts are those of stabilizability and detectability. However, in the innite-dimensional context, controllability need not imply stabilizability. The same can be said of observability and detectability. With a nite-rank input term one may not have enough control freedom to stabilize possibly innitely many unstable poles. Stabilizability often requires that there be only nitely many unstable poles. One may also wish to say that a system is spectrally controllable if all its nite-dimensional modal subsystems are controllable (in the usual sense for nite-dimensional systems). Some care must be exercised to be clear about the meaning of modal subsystems. Even if there exists a decomposition to modal subsystems, some modes may not be nitedimensional. For systems where eigenfunction expansion is possible, one can say that the system is approximately controllable if its eigenfunctions are complete and it is spectrally controllable. Systems with self-adjoint F operators are easy examples. Then several rank conditions are possible in testing controllability of each subsystem. Another class of systems that satisfy this condition is that of delay differential systems. For example, the retarded system of Eqs. (48), (49) is approximately controllable if and only if 1. rank[I F0 F1eh G0] n for every and 2. rank[F1 G0] n The rst condition guarantees spectral controllability and the second guarantees that the system can be made eigenfunction
280
CONTROLLABILITY AND OBSERVABILITY 18. M. Fliess and D. Normand-Cyrot, A group-theoretic approach to discrete-time nonlinear controllability, Proc. IEEE Conf. Dec. Control, 1981. 19. R. Hermann and A. J. Krener, Nonlinear controllability and observability, IEEE Trans. Autom. Control, 22: 728740, 1977. 20. H. J. Sussmann, Lie brackets, real analyticity, and geometric control, in R. W. Brockett, R. S. Millman, and H. J. Sussmann, (eds.), Differential Geometric Control Theory, Boston: Birkhauser, 1983. 21. A. Isidori, Nonlinear Control Systems, 3rd ed., London: SpringerVerlag, 1995. 22. B. Jakubczyk and E. D. Sontag, Controllability of nonlinear discrete-time systems: a Lie-algebraic approach, Invited Expository Article, SIAM J. Control Opt., 28: 133, 1990. 23. I. Kolmanovsky and N. H. McClamroch, Developments in nonholonomic control problems, Control Syst. Mag., 15 (6): 2036, December, 1995. 24. M. Kawski and H. J. Sussmann, Noncommutative power series and formal Lie-algebraic techniques in nonlinear control theory, in U. Helmke, D. Praetzel-Wolters, and E. Zerz (eds.), Operators, Systems and Linear Algebra: Three Decades of Algebraic Systems Theory, Stuttgart: B. G. Teubner, 1997, pp. 111129. 25. H. J. Sussmann, A general theorem on local controllability, SIAM J. Control Opt., 25: 158194, 1987. 26. E. D. Sontag, Controllability is harder to decide than accessibility, SIAM J. Control Opt., 26: 11061118, 1988. 27. D. Tilbury, R. M. Murray, and S. S. Sastry, Trajectory generation for the n-trailer problem using Goursat normal form, IEEE Trans. Autom. Control, 40: 802819, 1995. 28. C. Bardos, G. Lebeau, and J. Rauch, Sharp sufcient conditions for the observation, control and stabilization of waves from the boundary, SIAM J. Control Optimiz., 30: 10241065, 1992. 29. V. Komornik, Exact Controllability and StabilizationThe Multiplier Method, New York: Wiley, 1994. 30. J. E. Lagnese and J.-L. Lions, Modelling Analysis and Control of Thin Plates, New York: Springer-Verlag, 1989. 31. Y. Yamamoto, Realization theory of innite dimensional linear systems, part 1, Math. Syst. Theory, 15: 5577, 1981. 32. R. F. Curtain and H. J. Zwart, An Introduction to Innite-Dimensional Linear Systems Theory, New York: Springer-Verlag, 1995. 33. H. O. Fattorini, Some remarks on complete controllability, SIAM J. Control, 4: 686694, 1966. 34. D. L. Russel, Controllability and stabilizability theory for linear partial differential equations: recent progress and open problems, SIAM Rev., 20: 639739, 1978. 35. R. F. Curtain and A. J. Prichard, Innite dimensional linear systems theory, Springer Lecture Notes in Control and Information Sciences, 8: 1978. 36. P. A. Fuhrmann, Linear Systems and Operators in Hilbert Space, New York: McGraw-Hill, 1981. 37. G. Weiss, Transfer functions of regular linear systems, part I: characterization of regularity, Trans. Amer. Math. Soc., 342: 827 854, 1994. 38. A. Manitius and R. Triggiani, Function space controllability of linear retarded systems: a derivation from abstract operator conditions, SIAM J. Control Optimiz., 16: 595645, 1978. 39. A. Manitius, Necessary and sufcient conditions of approximate controllability for general linear retarded systems, SIAM J. Control Optimiz., 19: 516532, 1981. 40. D. A. OConnor and T. J. Tarn, On the function space controllability of linear neutral systems, SIAM J. Control Optimiz., 21: 306 329, 1983. 41. D. Salamon, Control and Observation of Neutral Systems, Boston: Pitman, 1984.
contrast to the nite-dimensional context, there is a large variety of freedom in choosing the algebra over which the factorization is considered. Typical examples are the Callier-Desoer algebra, the space of H functions, and the algebra of distributions with compact support. Each different choice leads to a different theory of realization/stabilization/controller parameterization as is the case with the nite-dimensional theory, but with much wider freedom. Each has its advantage in different contexts, reecting the variety of distributed parameter systems. The theories can therefore hardly be expected to be complete. Another topic that is in close connection with controllability/observability is the existence and uniqueness of solutions to Riccati equations. This topic is also related with H control theory and under active research. The current trend is in spectral factorization in an abstract functional equation setting. To this end, various questions concerning the well-posedness of system equations and transfer functions arise and are currently under study.
BIBLIOGRAPHY
1. J. W. Polderman and J. C. Willems, Introduction to Mathematical System Theory: A Behavioral Approach, New York: SpringerVerlag, 1998. 2. R. E. Kalman, On the general theory of control systems, Proc. 1st Int. Cong. IFAC, Moscow 1960, London: Butterworths, 1960, pp. 481492. 3. R. E. Kalman, Y. C. Ho, and K. Narendra, Controllability of linear dynamical systems, Contributions to Differ. Equations, 1: 189213, 1963. 4. R. E. Kalman, Lectures on controllability and observability, Centro Internazionale Matematico Estrio, Bologna, 1968. 5. R. E. Kalman, P. L. Falb, and M. A. Arbib, Topics in Mathematical System Theory, New York: McGraw-Hill, 1969. 6. E. D. Sontag, Mathematical Control Theory: Deterministic Finite Dimensional Systems, 2nd ed., New York: Springer-Verlag, 1998. 7. W. J. Rugh, Linear System Theory, 2nd ed., Upper Saddle River, NJ: Prentice-Hall, 1996. 8. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 9. W. L. Brogan, Modern Control Theory, Upper Saddle River, NJ: Prentice-Hall, 1991. 10. J. C. Willems, Paradigms and puzzles in the theory of dynamical systems, IEEE Trans. Autom. Control, AC-36: 259294, 1991. 11. E. Nelson, Tensor Analysis, Princeton, NJ: Princeton University Press, 1967. 12. E. D. Sontag, Universal nonsingular controls, Syst. Control Lett., 19: 221224, 1992. ber Systeme von linearen partiellen differen13. W. L. Chow, U tialgleichungen erster ord-nung, Math. Ann., 117: 98105, 1939. 14. R. Hermann, On the accessibility problem in control theory, in Int. Symp. Differ. Equations and Mech., New York: Academic Press, 1973. des syste ` mes non line aires, SIAM J. 15. C. Lobry, Controllabilite Contr., 8: 573605, 1970. 16. H. J. Sussmann and V. Jurdjevic, Controllability of nonlinear systems, J. Differ. Equations 12: 95116, 1972. 17. A. Krener, A generalization of Chows theorem and the bang-bang theorem to nonlinear control systems, SIAM J. Control, 12: 43 52, 1974.
CONTROL SYSTEM DESIGN, CONTINUOUS-TIME 42. Y. Yamamoto, Reachability of a class of innite-dimensional linear systems: an external approach with applications to general neutral systems, SIAM J. Control Optimiz., 27: 217234, 1989. 43. J. S. Baras, R. W. Brockett, and P. A. Fuhrmann, State-space models for innite-dimensional systems, IEEE Trans. Autom. Control, AC-19: 693700, 1974. 44. Y. Yamamoto, Pseudo-rational input/output maps and their realizations: a fractional representation approach to innite-dimensional systems, SIAM J. Control Optimiz., 26: 14151430, 1988.
281
A. C. ANTOULAS
Rice University
E. D. SONTAG
Rutgers University
Y. YAMAMOTO
Kyoto University
CONTROL, LIGHTING. See LIGHTING CONTROL. CONTROL, NYQUIST CRITERION. See NYQUIST CRITERION, DIAGRAMS, AND STABILITY.
CONTROL OF EXCITATION. See EXCITATION CONTROL

IN POWER SYSTEMS.
CONTROL OF INDUSTRIAL SYSTEMS. See INDUSTRIAL CONTROL.
CONTROL OF JET ENGINES. See JET ENGINE CONTROL,

IMPLEMENTATIONS.
CONTROL OF PERIODIC SYSTEMS. See PERIODIC

CONTROL.
CONTROL, OSCILLATORY. See OPEN-LOOP OSCILLATORY CONTROL.
CONTROL, POWER SYSTEM. See POWER SYSTEM CONTROL;
POWER SYSTEM TRANSIENTS.
CONTROL, PROCESS. See PROCESS CONTROL. CONTROL, RELAY. See RELAY CONTROL. CONTROL, ROBUST. See ROBUST CONTROL; ROBUST CONTROL ANALYSIS.
CONTROL SYSTEM ANALYSIS. See INTERVAL ANALYSIS

FOR CIRCUITS;
SERVOMECHANISMS.
Abstract : Control System Design, Continuous-Time : Wiley Encyclopedia of Electrical and Electronics Engineering : Wiley InterScience
G G G G
My Profile Log In Athens Log In
HOME
G
ABOUT US
G
CONTACT US
G
HELP
Browse this title

G
Control System Design, Continuous-Time

Standard Article Katsuhiko Ogata1 1University of Minnesota, Minneapolis, MN Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1004 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (282K)
Article Titles AZ Topics
Search this title
Abstract
The sections in this article are System Compensation Root-Locus Method Frequency-Response Method Root-Locus Approach to the Design of Control Systems Frequency-Response Approach to the Design of Control Systems Comparison of Lead, Lag, Lag-Lead Compensation Multi-Degrees-of-Freedom Control Concluding Comments
About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
CONTROL SYSTEM DESIGN, CONTINUOUS-TIME
281
as performance indexes in terms of state variables, then modern control approaches must be used. The systems that may be designed by a conventional or classical approach are usually limited to single-inputsingleoutput, linear time-invariant systems. The designer seeks to satisfy all performance specications by means of educated trial-and-error repetition. After a system is designed, the designer checks to see if the designed system satises all the performance specications. If it does not, then he or she repeats the design process by adjusting parameter settings or by changing the system conguration until the given specications are met. Although the design is based on a trial-anderror procedure, the ingenuity and know-how of the designer will play an important role in a successful design. An experienced designer may be able to design an acceptable system without using many trials. The primary objective of this article is to present procedures for the design and compensation of single-inputsingleoutput linear time-invariant control systems. Compensation is the modication of the system dynamics to satisfy the given specications. The methods to the control system design and compensation used in this article are the root-locus method and frequency-response method. These methods are commonly called the classical or conventional methods of control systems design. Note that in designing control systems by the root-locus or frequency-response methods the nal result is not unique, because the best or optimal solution may not be precisely dened if the time-domain specications or frequency-domain specications are given.
SYSTEM COMPENSATION Setting the gain is the rst step in adjusting the system for satisfactory performance. In many practical cases, however, the adjustment of the gain alone may not provide sufcient alteration of the system behavior to meet the given specications. As is frequently the case, increasing the gain value will improve the steady-state behavior but will result in poor stability or even instability. It is then necessary to redesign the system by modifying the structure or by incorporating additional devices or components to alter the overall behavior so that the system will behave as desired. A device inserted into the system for the purpose of satisfying the specications is called a compensator. The compensator compensates for decit performance of the original system. In discussing compensators, we frequently use such terminologies as lead network, lag network, and lag-lead network. If a sinusoidal input ei is applied to the input of a network and the steady-state output eo (which is also sinusoidal) has a phase lead, then the network is called a lead network. (The amount of phase lead angle is a function of the input frequency.) If the steady-state output eo has a phase lag, then the network is called a lag network. In a lag-lead network, both phase lag and phase lead occur in the output but in different frequency regions; phase lag occurs in the low-frequency region and phase lead occurs in the high-frequency region. A compensator having a characteristic of a lead network, lag network, or lag-lead network is called a lead compensator, lag compensator, or lag-lead compensator. In this article we specically consider the design of lead compensators, lag compensators, and lag-lead compensators.

This article discusses a means of improving performance of existing control systems and of designing new control systems with satisfactory performance. The most common approach to improving the performance of single-inputsingle-output control systems is to insert a suitable compensator in the system. In this article we are concerned with the design of various types of compensators. Actual control systems are generally nonlinear. However, if they can be approximated by linear mathematical models, we may use one or more of the well-developed design methods. In a practical sense, the performance specications given to the particular system suggest which method to use. If the performance specications are given in terms of transient-response characteristics and/or frequency-domain performance measures, then we have no choice but to use a conventional or classical approach based on the root-locus and/or frequencyresponse methods. If the performance specications are given
282
In such design problems, we place a compensator in series with the unalterable plant transfer function G(s) to obtain desirable behavior. The main problem then involves the judicious choice of the pole(s) and zero(s) of the compensator Gc(s) to alter the root loci or frequency response so that the performance specications will be met. In the actual design of a control system, whether to use an electronic, pneumatic, or hydraulic compensator is a matter that must be decided partially based on the nature of the controlled plant. For example, if the controlled plant involves ammable uid, then we have to choose pneumatic components (both a compensator and an actuator) to avoid the possibility of sparks. If, however, no re hazard exists, then electronic compensators are most commonly used. In fact, we often transform nonelectrical signals into electrical signals because of the simplicity of transmission, increased accuracy, increased reliability, ease of compensation, and the like. Lead, Lag, and Lag-Lead Compensation Lead compensation essentially yields an appreciable improvement in transient response and a small change in steadystate accuracy. It may accentuate high-frequency noise effects. Lag compensation, on the other hand, yields an appreciable improvement in steady-state accuracy at the expense of increasing the transient response time. Lag compensation will suppress the effects of high-frequency noise signals. Laglead compensation combines the characteristics of both lead compensation and lag compensation. The use of a lead or lag compensator raises the order of the system by 1 (unless cancellation occurs between the zero of the compensator and a pole of the uncompensated open-loop transfer function). The use of a lag-lead compensator raises the order of the system by 2 [unless cancellation occurs between zero(s) of the laglead compensator and pole(s) of the uncompensated open-loop transfer function], which means that the system becomes more complex and it is more difcult to control the transient response behavior. The particular situation determines the type of compensation to be used. ROOT-LOCUS METHOD
R(s)
C(s) G(s)
H(s)
Figure 1. Control system.
rameter is usually the gain, but any other variable of the open-loop transfer function may be used. Unless otherwise stated, we shall assume that the gain of the open-loop transfer function is the parameter to be varied through all values, from zero to innity. Angle and Magnitude Conditions The basic idea behind the root-locus method is that the values of s that make the transfer function around the loop equal 1 must satisfy the characteristic equation of the system. Consider the system shown in Fig. 1. The closed-loop transfer function is G (s ) C (s ) = R (s ) 1 + G ( s )H ( s ) The characteristic equation for this closed-loop system is obtained by setting the denominator of the right-hand side of this last equation equal to zero. That is, 1 + G ( s )H ( s ) = 0 or G(s )H (s ) = 1 (1)
Here we assume that G(s)H(s) is a ratio of polynomials in s. Since G(s)H(s) is a complex quantity, Eq. (1) can be split into two equations by equating the angles and magnitudes of both sides, respectively, to obtain the following:
Angle condition:
The basic characteristic of the transient response of a closedloop system is closely related to the location of the closed-loop poles. If the system has a variable loop gain, then the location of the closed-loop poles depends on the value of the loop gain chosen. It is important, therefore, that the designer know how the closed-loop poles move in the s-plane as the loop gain is varied. From the design viewpoint, in some systems simple gain adjustment may move the closed-loop poles to desired locations. Then the design problem may become the selection of an appropriate gain value. If the gain adjustment alone does not yield a desired result, addition of a compensator to the system will become necessary. A simple method for nding the roots of the characteristic equation has been developed by W. R. Evans and used extensively in control engineering. This method, called the rootlocus method, is one in which the roots of the characteristic equation are plotted for all values of a system parameter. The roots corresponding to a particular value of this parameter can then be located on the resulting graph. Note that the pa-
G(s )H (s ) = 180 (2k + 1 )
( k = 0, 1, 2, . . . )
Magnitude condition: |G(s )H (s )| = 1

The values of s that fulll both the angle and magnitude conditions are the roots of the characteristic equation, or the closed-loop poles. A plot of the points of the complex plane satisfying the angle condition alone is the root locus. The roots of the characteristic equation (the closed-loop poles) corresponding to a given value of the gain can be determined from the magnitude condition. FREQUENCY-RESPONSE METHOD By the term frequency response, we mean the steady-state response of a system to a sinusoidal input. In frequencyresponse methods, we vary the frequency of the input signal over a certain range and study the resulting response.
283
The Nyquist stability criterion enables us to investigate both the absolute and relative stabilities of linear closed-loop systems from a knowledge of their open-loop frequencyresponse characteristics. An advantage of the frequencyresponse approach is that frequency-response tests are, in general, simple and can be made accurately by use of readily available sinusoidal signal generators and precise measurement equipment. Often the transfer functions of complicated components can be determined experimentally by frequencyresponse tests. In addition, the frequency-response approach has the advantage that a system may be designed so that the effects of undesirable noise are negligible and that such analysis and design can be extended to certain nonlinear control systems. Frequency-Response Approach to the Design of Control Systems It is important to note that in a control system design, transient-response performance is usually most important. In the frequency-response approach, we specify the transient-response performance in an indirect manner. That is, the transient-response performance is specied in terms of the phase margin, gain margin, and resonant peak magnitude (they give a rough estimate of the system damping); the gain crossover frequency, resonant frequency, and bandwidth (they give a rough estimate of the speed of transient response); and static error constants (they give the steady-state accuracy). Although the correlation between the transient response and frequency response is indirect, the frequency-domain specications can be conveniently met in the Bode diagram approach. After the open loop has been designed by the frequencyresponse method, the closed-loop poles and zeros can be determined. The transient-response characteristics must be checked to see whether the designed system satises the requirements in the time domain. If it does not, then the compensator must be modied and the analysis repeated until a satisfactory result is obtained. Design in the frequency domain is simple and straightforward. The frequency-response plot indicates clearly the manner in which the system should be modied, although the exact quantitative prediction of the transient-response characteristics cannot be made. The frequency-response approach can be applied to systems or components whose dynamic characteristics are given in the form of frequency-response data. Note that because of difculty in deriving the equations governing certain components, such as pneumatic and hydraulic components, the dynamic characteristics of such components are usually determined experimentally through frequency-response tests. The experimentally obtained frequency-response plots can be combined easily with other such plots when the Bode diagram approach is used. Note also that in dealing with high-frequency noise we nd that the frequency-response approach is more convenient than other approaches. A common approach to the design by use of the Bode diagram is that we rst adjust the open-loop gain so that the requirement on the steady-state accuracy is met. Then the magnitude and phase curves of the uncompensated open loop (with the open-loop gain just adjusted) is plotted. If the specications on the phase margin and gain margin are not satis-
ed, then a suitable compensator that will reshape the openloop transfer function is determined. Finally, if there are any other requirements to be met, we try to satisfy them, unless some of them are contradictory to the other. ROOT-LOCUS APPROACH TO THE DESIGN OF CONTROL SYSTEMS The root-locus approach to design is very powerful when the specications are given in terms of time-domain quantities, such as the damping ratio and undamped natural frequency of the desired dominant closed-loop poles, maximum overshoot, rise time, and settling time. Consider a design problem in which the original system either is unstable for all values of gain or is stable but has undesirable transient-response characteristics. In such a case, the reshaping of the root locus is necessary in the broad neighborhood of the j axis and the origin in order that the dominant closed-loop poles be at desired locations in the complex plane. This problem may be solved by inserting an appropriate lead compensator in cascade with the feedforward transfer function. If it is desired to improve steady-state performance (such as to reduce the error in following the ramp input), insertion of a lag compensator in the feedforward path will do the job. If it is desired to improve both the transient-response and steady-state performance, insertion of a lag-lead compensator will accomplish the job. In what follows we discuss the lead, lag, and lag-lead compensation techniques. Lead Compensation The procedure for designing a lead compensator for the system shown in Fig. 2 by the root-locus method may be stated as follows: 1. From the performance specications, determine the desired location for the dominant closed-loop poles. 2. By drawing the root-locus plot, ascertain whether or not the gain adjustment alone can yield the desired closedloop poles. If not, calculate the angle deciency . This angle must be contributed by the lead compensator if the new root locus is to pass through the desired locations for the dominant closed-loop poles. 3. Assume the lead compensator Gc(s) to be
1 s+ Ts + 1 T , (0 < < 1 ) = Kc Gc (s ) = Kc 1 Ts + 1 s+ T
(2)
where and T are determined from the angle deciency. Kc is determined from the requirement of the open-loop gain.
Gc(s)
G(s)
284
j j3 Closed-loop poles j2 j1 5 4 R(s) + 4 s(s + 2) C(s) 3 2 0 1 j1 j2 j3
Figure 3. (a) Control system; (b) root-locus plot.
(a)
(b)
4. If static error constants are not specied, determine the location of the pole and zero of the lead compensator so that the lead compensator will contribute the necessary angle . If no other requirements are imposed on the system, try to make the value of as large as possible. A larger value of generally results in a larger value of Kv, which is desirable. (If a particular static error constant is specied, it is generally simpler to use the frequency-response approach.) 5. Determine the open-loop gain of the compensated system from the magnitude condition. Once a compensator has been designed, check to see whether all performance specications have been met. If the compensated system does not meet the performance specications, then repeat the design procedure by adjusting the compensator pole and zero until all such specications are met. If a large static error constant is required, cascade a lag network or alter the lead compensator to a lag-lead compensator. Example 1. Consider the system shown in Fig. 3(a). The feedforward transfer function is G (s ) = 4 s (s + 2 )
to the desired location by simple gain adjustment. This is, however, not the case for the present system. Therefore, we shall insert a lead compensator in the feedforward path. A general procedure for determining the lead compensator is as follows: First, nd the sum of the angles at the desired location of one of the dominant closed-loop poles with the open-loop poles and zeros of the original system, and determine the necessary angle to be added so that the total sum of the angles is equal to 180 (2k 1). The lead compensator must contribute this angle . (If the angle is quite large, then two or more lead networks may be needed rather than a single one.) If the original system has the open-loop transfer function G(s), then the compensated system will have the open-loop transfer function
G c ( s )G ( s ) =
1 T Kc 1 s+ T s+
G (s )
where
1 s+ Ts + 1 T = Kc Gc (s ) = Kc , Ts + 1 s + 1 T
(0 < < 1 )
The root-locus plot for this system is shown in Fig. 3(b). The closed-loop poles are located at s = 1 j 3 The damping ratio of the closed-loop poles is 0.5. The undamped natural frequency of the closed-loop poles is 2 rad/s. The static velocity error constant is 2 s1. It is desired to modify the closed-loop poles so that an undamped natural frequency n 4 rad/s is obtained, without changing the value of the damping ratio, 0.5. In the present example, the desired locations of the closed-loop poles are s = 2 j 2 3 In some cases, after the root loci of the original system have been obtained, the dominant closed-loop poles may be moved
Notice that there are many possible values for T and that will yield the necessary angle contribution at the desired closed-loop poles. The next step is to determine the locations of the zero and pole of the lead compensator. There are many possibilities for the choice of such locations. (See the comments at the end of this example problem.) In what follows, we shall introduce a procedure to obtain the largest possible value for . (Note that a larger value of will produce a larger value of Kv. In most cases, the larger the Kv is, the better the system performance.) First, draw a horizontal line passing through point P, the desired location for one of the dominant closed-loop poles. This is shown as line PA in Fig. 4. Draw also a line connecting point P and the origin. Bisect the angle between the lines PA and PO, as shown in Fig. 4. Draw two lines PC and PD that make angles /2 with the bisector PB. The intersections of PC and PD with the negative real axis give the necessary location for the pole and zero of the lead net-
285
j P A
2 2
where K 4Kc. The root-locus plot for the compensated system is shown in Fig. 5. The gain K is evaluated from the magnitude condition as follows: Referring to the root-locus plot for the compensated system shown in Fig. 5, the gain K is evaluated from the magnitude condition as
1 T C
1 D T
K ( s + 2.9 ) s(s + 2 )(s + 5.4 ) or
s = 2 + j 2 3
=1
K = 18.7
Figure 4. Determination of the pole and zero of a lead network.
It follows that G c ( s )G ( s ) = 18.7 (s + 2.9 ) s(s + 2 )(s + 5.4 )
work. The compensator thus designed will make point P a point on the root locus of the compensated system. The openloop gain is determined by use of the magnitude condition. In the present system, the angle of G(s) at the desired closed-loop pole is
The constant Kc of the lead compensator is Kc = 18.7 = 4.68 4
4 s (s + 2 )
= 210
s = 2 + j 2 3
Hence, Kc 2.51. The lead compensator, therefore, has the transfer function Gc (s ) = 2.51 s + 2.9 0.345s + 1 = 4.68 0.185s + 1 s + 5.4
Thus, if we need to force the root locus to go through the desired closed-loop pole, the lead compensator must contribute 30 at this point. By following the foregoing design procedure, we determine the zero and pole of the lead compensator, as shown in Fig. 5, to be Zero at s = 2.9, or T= 1 = 0.345, 2.9 T = 1 = 0.185 5.4 Pole at s = 5.4
The static velocity error constant Kv is obtained from the expression
Kv = lim sGc (s )G(s )

s0 s0
s18.7 (s + 2.9 ) s(s + 2 )(s + 5.4 ) = 5.02 s 1 = lim

Note that the third closed-loop pole of the designed system is found by dividing the characteristic equation by the known factors as follows:
Thus 0.537. The open-loop transfer function of the compensated system becomes K ( s + 2.9 ) s + 2.9 4 = Gc (s )G(s ) = Kc s + 5.4 s ( s + 2 ) s(s + 2 )(s + 5.4 )
s(s + 2 )(s + 5.4 ) + 18.7 (s + 2.9 )
= (s + 2 + j 2 3 )(s + 2 j 2 3 )(s + 3.4 )
j j4
15 15 8 6 2 4 5.4 2.9
j2
The foregoing compensation method enables us to place the dominant closed-loop poles at the desired points in the complex plane. The third pole at s 3.4 is close to the added zero at s 2.9. Therefore, the effect of this pole on the transient response is relatively small. Since no restriction has been imposed on the nondominant pole and no specication has been given concerning the value of the static velocity error coefcient, we conclude that the present design is satisfactory.
2
0 j2 j4
Figure 5. Root-locus plot of the compensated system.
Comments. We may place the zero of the compensator at s 2 and pole at s 4 so that the angle contribution of the lead compensator is 30. (In this case the zero of the lead compensator will cancel a pole of the plant, resulting in the second-order system, rather than the third-order system as we designed.) It can be seen that the Kv value in this case is 4 s1. Other combinations can be selected that will yield 30
286
phase lead. (For different combinations of a zero and pole of the compensator that contribute 30, the value of will be different and the value of Kv will also be different.) Although a certain change in the value of Kv can be made by altering the pole-zero location of the lead compensator, if a large increase in the value of Kv is desired, then we must alter the lead compensator to a lag-lead compensator. Lag Compensation Consider the case where the system exhibits satisfactory transient-response characteristics but unsatisfactory steadystate characteristics. Compensation in this case essentially consists of increasing the open-loop gain without appreciably changing the transient-response characteristics. This means that the root locus in the neighborhood of the dominant closed-loop poles should not be changed appreciably, but the open-loop gain should be increased as much as needed. This can be accomplished if a lag compensator is put in cascade with the given feedforward transfer function. The procedure for designing lag compensator for the system shown in Fig. 2 by the root-locus method may be stated as follows: 1. Draw the root-locus plot for the uncompensated system whose open-loop transfer function is G(s). Based on the transient-response specications, locate the dominant closed-loop poles on the root locus. 2. Assume the transfer function of the lag compensator to be
error constant without appreciably altering the original root loci. (Note that the ratio of the value of gain required in the specications and the gain found in the uncompensated system is the required ratio between the distance of the zero from the origin and that of the pole from the origin.) 6. Draw a new root-locus plot for the compensated system. Locate the desired dominant closed-loop poles on the root locus. (If the angle contribution of the lag network is very smallthat is, a few degreesthen the original and new root loci are almost identical. Otherwise, there will be a slight discrepancy between them. Then locate, on the new root locus, the desired dominant closed-loop poles based on the transient-response specications.) c of the compensator from the magnitude 7. Adjust gain K condition so that the dominant closed-loop poles lie at the desired location. Example 2. Consider the system shown in Fig. 6(a). The root-locus plot for the system is shown in Fig. 6(b). The closedloop transfer function becomes
1.06 C (s ) = R (s ) s(s + 1 )(s + 2 ) + 1.06 =
1.06 (s + 0.3307 j 0.5864 )(s + 0.3307 + j 0.5864 )(s + 2.3386 )
The dominant closed-loop poles are s = 0.3307 j 0.5864
1 s+ Ts + 1 T = Kc Gc (s ) = Kc 1 Ts + 1 s+ T
( > 1 )
(3)
Then the open-loop transfer function of the compensated system becomes Gc(s)G(s). 3. Evaluate the particular static error constant specied in the problem. 4. Determine the amount of increase in the static error constant necessary to satisfy the specications. 5. Determine the pole and zero of the lag compensator that produce the necessary increase in the particular static
The damping ratio of the dominant closed-loop poles is 0.491. The undamped natural frequency of the dominant closed-loop poles is 0.673 rad/s. The static velocity error constant is 0.53 s1. It is desired to increase the static velocity error constant Kv to about 5 s1 without appreciably changing the location of the dominant closed-loop poles. To meet this specication, let us insert a lag compensator as given by Eq. (3) in cascade with the given feedforward transfer function. To increase the static velocity error constant by a factor of about 10, let us choose 10 and place the zero and pole of the lag compenj
j2
Closed-loop poles
j1
3 + 1.06 s(s + 1) (s + 2)
j1
j2
(a) (b)
Figure 6. (a) Control system; (b) root-locus plot.
287
sator at s 0.05 and s 0.005, respectively. The transfer function of the lag compensator becomes c Gc ( s ) = K s + 0.05 s + 0.005
Kc
s +0.05 s + 0.005
1.06 s(s + 1) (s + 2)
Kc = 0.966 (a)
The angle contribution of this lag network near a dominant closed-loop pole is about 4. Because this angle contribution is not very small, there is a small change in the new root locus near the desired dominant closed-loop poles. The open-loop transfer function of the compensated system then becomes
2 1.5 Uncompensated system 1

Imag axis
c G c ( s )G ( s ) = K
s + 0.05 1.06 s + 0.005 s(s + 1 )(s + 2 ) K (s + 0.05 ) = s(s + 0.005 )(s + 1 )(s + 2 )
0.5 0 0.5 1 1.5
Original closed-loop pole Compensated system
where c K = 1.06K The block diagram of the compensated system is shown in Fig. 7(a). The root-locus plot for the compensated system near the dominant closed-loop poles is shown in Fig. 7(b), together with the original root-locus plot. Figure 7(c) shows the rootlocus plot of the compensated system near the origin. If the damping ratio of the new dominant closed-loop poles is kept the same, then the poles are obtained from the new root-locus plot as follows: s1 = 0.31 + j 0.55, The open-loop gain K is s2 = 0.31 j 0.55
New closed-loop pole
2 3 2.5 2 1.5 1 0.5 Real axis (b)
0.5
K=
s(s + 0.005 )(s + 1 )(s + 2 ) s + 0.05
s = 0 . 31 + j 0 . 55
0.5 0.4 0.3 0.2

Imag axis
= 1.0235
c is determined as Then the lag compensator gain K c = K = 1.0235 = 0.9656 K 1.06 1.06 Thus the transfer function of the designed lag compensator is 20s + 1 s + 0.05 = 9.656 Gc (s ) = 0.9656 s + 0.005 200s + 1 Then the compensated system has the following open-loop transfer function:
0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.4 0.2 0 0.2 Real axis (c) 0.4 0.6
1.0235 (s + 0.05 ) G1 ( s ) = s(s + 0.005 )(s + 1 )(s + 2 ) 5.12 (20s + 1 ) = s(200s + 1 )(s + 1 )(0.5s + 1 )
The static velocity error constant Kv is Kv = lim sG1 (s ) = 5.12 s 1
s0
Figure 7. (a) Compensated system; (b) root-locus plots of the compensated system and the uncompensated system; (c) root-locus plot of compensated system near the origin.
In the compensated system, the static velocity error constant has increased to 5.12 s1, or 5.12/0.53 9.66 times the origi-
288
nal value. (The steady-state error with ramp inputs has decreased to about 10% of that of the original system.) We have essentially accomplished the design objective of increasing the static velocity error constant to about 5 s1. Note that, since the pole and zero of the lag compensator are placed close together and are located very near the origin, their effect on the shape of the original root loci has been small. Except for the presence of a small closed root locus near the origin, the root loci of the compensated and the uncompensated systems are very similar to each other. However, the value of the static velocity error constant of the compensated system is 9.66 times greater than that of the uncompensated system. The two other closed-loop poles for the compensated system are found as follows: s3 = 2.326, s4 = 0.0549
1.4 1.2
Outputs c1 and c2
Compensated system
1 0.8 0.6 0.4 0.2 0 Uncompensated system
10
15
20 t (s) (a)
25
30
35
40
The addition of the lag compensator increases the order of the system from 3 to 4, adding one additional closed-loop pole close to the zero of the lag compensator. (The added closedloop pole at s 0.0549 is close to the zero at s 0.05.) Such a pair of a zero and pole creates a long tail of small amplitude in the transient response, as we will see later in the unit-step response. Since the pole at s 2.326 is very far from the j axis compared with the dominant closed-loop poles, the effect of this pole on the transient response is also small. Therefore, we may consider the closed-loop poles at s 0.31 j0.55 to be the dominant closed-loop poles. The undamped natural frequency of the dominant closedloop poles of the compensated system is 0.631 rad/s. This value is about 6% less than the original value, 0.673 rad/s. This implies that the transient response of the compensated system is slower than that of the original system. The response will take a longer time to settle down. The maximum overshoot in the step response will increase in the compensated system. If such adverse effects can be tolerated, the lag compensation as discussed here presents a satisfactory solution to the given design problem. Figures 8(a) and 8(b) show the unit-step response curves and unit-ramp response curves, respectively, of the compensated and uncompensated systems. Lag-Lead Compensation Lead compensation basically speeds up the response and increases the stability of the system. Lag compensation improves the steady-state accuracy of the system but reduces the speed of the response. If improvements in both transient response and steadystate response are desired, then both a lead compensator and a lag compensator may be used simultaneously. Rather than introducing both a lead compensator and a lag compensator as separate elements, however, it is economical to use a single lag-lead compensator. Consider the system shown in Fig. 2. Assume that we use the following lag-lead compensator:
50 45 40
Outputs c1 and c2
35 30 25 20 15 10 5 0
0 5 10 15 20 25 t (s) (b) 30 35 40 45 50 Compensated system Uncompensated system
Figure 8. (a) Unit-step response curves for the compensated and uncompensated systems; (b) unit-ramp response curves for both systems.
where 1. The design procedure may be stated as follows: 1. From the given performance specications, determine the desired location for the dominant closed-loop poles. 2. If the static velocity error constant Kv is specied, determine the value of constant Kc from the following equation:
Kv = lim sGc (s )G(s )

s0 s0
Gc (s ) = Kc
(T1 s + 1 )(T2 s + 1 ) = Kc T1 s + 1 ( T2 s + 1 )
1 s+ T1 s+ T1
1 s+ T2 1 s+ T2
= lim sKc G(s )

3. To have the dominant closed-loop poles at the desired location, calculate the angle contribution needed from the phase lead portion of the lag-lead compensator.
(4)
289
4 s(s + 0.5)
The open-loop transfer function of the compensated system is
Gc (s )G(s ) = Kc
1 T1 s+ T1 s+
1 T2 1 s+ T2 s+
4 s ( s + 0.5 )
Since the requirement on the static velocity error constant Kv is 80 s1, we have 4. For the lag-lead compensator, we later choose T2 sufciently large so that Kv = lim sGc (s )G(s ) = lim Kc
s0 s0
4 = 8Kc = 80 0.5
1 T2 1 s1 + T2 s+
is approximately unity, where s s1 is one of the dominant closed-loop poles. Determine the values of T1 and from the magnitude and angle conditions:
Thus Kc = 10 Noting that
4 s ( s + 0.5 )
s = 2 . 50 + j 4 . 33
= 235
Kc
1 s1 + T1 s1 + T1
G ( s1 ) = 1 1 T1 = s1 + T1 s1 +
the time constant T1 and the value of are determined from
s+
1 T1 s+ T1
40 s ( s + 0.5 ) 1 T1 s+ T1 s+
1 T1 = s = 2 . 5 + j 4 . 33 s+ T1 s+ = 55
s = 2 . 5 + j 4 . 33
8 =1 4.77
5. Using the value of just determined, choose T2 so that
5 <
1 s1 + T2 +1 1 s1 + T2 1 s1 + T2 < 0 1 s1 + T2
Referring to Fig. 10, we can easily locate points A and B such that APB = 55 , PA PB = 4.77 8
j P
j5 j4
The value of T2, the largest time constant of the laglead compensator, should not be too large to be physically realized. Example 3. Consider the control system shown in Fig. 9. It is desired to make the damping ratio of the dominant closedloop poles equal to 0.5 and to increase the undamped natural frequency to 5 rad/s and the static velocity error constant to 80 s1. Design an appropriate compensator to meet all the design specications. Let us use a lag-lead compensator of the form given by Eq. (4). The desired locations for the dominant closed-loop poles are at s = 2.50 j 4.33
55
j3 j2 j1
B 10 9 8
A 7 6 5 4 3 2 1 0 1 j1 2
j2 j3 j4
Figure 10. Determination of the desired pole-zero location.
290
(Use a graphical approach or a trigonometric approach.) The result is AO = 2.38, or 1 = 0.420, T1 = 2.38 BO = 8.34
Outputs
1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 2 2.5 t (s) (a) 3 3.5 4 4.5 5 Compensated system Uncompensated system
= 8.34T1 = 3.503
The phase lead portion of the lag-lead network thus becomes s + 2.38 s + 8.34
10
For the phase lag portion, we may choose

4
T2 = 10 Then
Outputs
3.5 3 2.5 2 1.5 1 0.5 Compensated system
1 1 = 0.0285 = T2 3.503 10 Thus, the lag-lead compensator becomes s + 2.38 Gc (s ) = (10 ) s + 8.34 s + 0.1 s + 0.0285
Uncompensated system
0.5
1.5
2 t (s) (b)
2.5
3.5
The compensated system will have the open-loop transfer function 40 (s + 2.38 )(s + 0.1 ) (s + 8.34 )(s + 0.0285 )s(s + 0.5 )
Figure 11. (a) Unit-step response curves for the compensated and uncompensated systems; (b) unit-ramp response curves for both systems.
G c ( s )G ( s ) =
No cancellation occurs in this case, and the compensated system is of fourth order. Because the angle contribution of the phase lag portion of the lag-lead network is quite small, the dominant closed-loop poles are located very near the desired location. In fact, the dominant closed-loop poles are located at s 2.4539 j4.3099. The two other closed-loop poles are located at s = 0.1003, s = 3.8604
uncompensated systems are shown in Fig. 11(a). The unitramp response curves for both systems are depicted in Fig. 11(b).
FREQUENCY-RESPONSE APPROACH TO THE DESIGN OF CONTROL SYSTEMS Lead Compensation We shall rst examine the frequency characteristics of the lead compensator. Then we present a design technique for the lead compensator by use of the Bode diagram. Characteristics of Lead Compensators. Consider a lead compensator dened by Kc j T + 1 j T + 1 (0 < < 1 )
Since the closed-loop pole at s 0.1003 is very close to a zero at s 0.1, they almost cancel each other. Thus, the effect of this closed-loop pole is very small. The remaining closed-loop pole (s 3.8604) does not quite cancel the zero at s 2.4. The effect of this zero is to cause a larger overshoot in the step response than a similar system without such a zero. The unit-step response curves of the compensated and
291
Im
Gc(s)
G(s)
1 (1 ) 2 =0 0
m =
Re
1 (1 + ) 2
Figure 12. Polar plot of a lead compensator ( jT 1)/( jT 1), where 0 1.
Figure 12 shows the polar plot of this compensator with Kc 1. For a given value of , the angle between the positive real axis and the tangent line drawn from the origin to the semicircle gives the maximum phase lead angle, m. We shall call the frequency at the tangent point m. From Fig. 12 the phase angle at m is m, where
Lead Compensation Techniques Based on the Frequency-Response Approach. The primary function of the lead compensator is to reshape the frequency-response curve to provide sufcient phase-lead angle to offset the excessive phase lag associated with the components of the xed system. Consider the system shown in Fig. 14. Assume that the performance specications are given in terms of phase margin, gain margin, static velocity error constants, and so on. The procedure for designing a lead compensator by the frequency-response approach may be stated as follows: 1. Assume the following lead compensator:
1 1 sin m = 2 = 1+ 1+ 2
(5)
Dene Kc = K Then Gc ( s ) = K Ts + 1 Ts + 1
(0 < < 1 )
(7)
Equation (5) relates the maximum phase lead angle and the value of . Figure 13 shows the Bode diagram of a lead compensator when Kc 1 and 0.1. The corner frequencies for the lead compensator are 1/ T and 1/(T) 10/ T. By examining Fig. 13, we see that m is the geometric mean of the two corner frequencies, or log m = Hence, 1 m = aT (6) 1 1 1 + log log 2 T T
G c ( s )G ( s ) = K
As seen from Fig. 13, the lead compensator is basically a highpass lter. (The high frequencies are passed, but low frequencies are attenuated.)
Ts + 1 Ts + 1 G (s ) = KG(s ) Ts + 1 Ts + 1 Ts + 1 = G (s ) Ts + 1 1
where G1 (s ) = KG(s )
10 dB 0 10 20 90 0 0.1 T
m
1 T
10 T in rad/s
10 T
100 T
Figure 13. Bode diagram of a lead compensator ( jT 1)/ ( jT 1), where 0.1.
Determine gain K to satisfy the requirement on the given static error constant. 2. Using the gain K thus determined, draw a Bode diagram of G1( j), the gain-adjusted but uncompensated system. Evaluate the phase margin. 3. Determine the necessary phase lead angle to be added to the system. 4. Determine the attenuation factor by use of Eq. (5). Determine the frequency where the magnitude of the uncompensated system G1( j) is equal to 20 log (1/ ). Select this frequency as the new gain crossover frequency. This frequency corresponds to m 1/ (T), and the maximum phase shift m occurs at this frequency.
292
5. Determine the corner frequencies of the lead compensator as follows:
40 20
Zero of lead compensator: Pole of lead compensator:
1 T 1 = T
dB
0 20 40 0 90
6. Using the value of K determined in step 1 and that of determined in step 4, calculate constant Kc from Kc = K
7. Check the gain margin to be sure it is satisfactory. If not, repeat the design process by modifying the polezero location of the compensator until a satisfactory result is obtained. Example 4. Consider the system shown in Fig. 15. The open-loop transfer function is G (s ) = 4 s (s + 2 )
17 2 4 810 20 40 60 100
180 1
in rad/s
Figure 16. Bode diagram for G1( j) 10G( j) 40/[j( j 2)].
It is desired to design a compensator for the system so that the static velocity error constant Kv is 20 s1, the phase margin is at least 50, and the gain margin is at least 10 dB. We shall use a lead compensator of the form dened by Eq. (7). Dene G1 (s ) = KG(s ) = 4K s (s + 2 )
where K Kc. The rst step in the design is to adjust the gain K to meet the steady-state performance specication or to provide the required static velocity error constant. Since this constant is given as 20 s1, we obtain
Kv = lim sGc (s )G(s ) = lim s

s0 s0
Ts + 1 G (s ) Ts + 1 1
s4K = 2K = 20 = lim s 0 s (s + 2 )
or K = 10 With K 10, the compensated system will satisfy the steadystate requirement. We shall next plot the Bode diagram of G1 ( j ) = 20 40 = j ( j + 2 ) j (0.5 j + 1 )
Figure 16 shows the magnitude and phase angle curves of G1( j). From this plot, the phase and gain margins of the system are found to be 17 and dB, respectively. (A phase margin of 17 implies that the system is quite oscillatory. Thus, satisfying the specication on the steady state yields a poor transient-response performance.) The specication calls for a phase margin of at least 50. We thus nd the additional phase lead necessary to satisfy the relative stability requirement is 33. To achieve a phase margin of 50 without decreasing the value of K, the lead compensator must contribute the required phase angle. Noting that the addition of a lead compensator modies the magnitude curve in the Bode diagram, we realize that the gain crossover frequency will be shifted to the right. We must offset the increased phase lag of G1( j) due to this increase in the gain crossover frequency. Considering the shift of the gain crossover frequency, we may assume that m, the maximum phase lead required, is approximately 38. (This means that 5 has been added to compensate for the shift in the gain crossover frequency.) Since sin m = 1 1+
m 38 corresponds to 0.24. Once the attenuation factor has been determined on the basis of the required phase lead angle, the next step is to determine the corner frequencies 1/ T and 1/(T) of the lead compensator. To do so, we rst note that the maximum phase lead angle m occurs at the geometric mean of the two corner frequencies, or 1/(T). [See Eq. (6).] The amount of the modication in the magnitude curve at 1/(T) due to the inclusion of the term (Ts 1)/(Ts 1) is
1 1 + j 1 = = 1 1 + j
4 s(s + 2)
1 + j T 1 + j T
= 1 /( aT )
293
Note that
+
1 1 1 = 6.2 dB = = 0 . 49 0.24 and G1( j) 6.2 dB corresponds to 9 rad/s. We shall select this frequency to be the new gain crossover frequency c. Noting that this frequency corresponds to 1/(T), or c 1/(T), we obtain 1 = c = 4.41 T and 1 c = = 18.4 T The lead compensator thus determined is Gc (s ) = Kc s + 4.41 0.227s + 1 = Kc s + 18.4 0.054s + 1
41.7(s + 4.41) s + 18.4
4 s(s + 2)
Figure 18. Compensated system.
The magnitude curve and phase-angle curve for Gc( j)/10 are shown in Fig. 17. The compensated system has the following open-loop transfer function: Gc (s )G(s ) = 41.7 4 s + 4.41 s + 18.4 s(s + 2 )
where the value of Kc is determined as Kc = 10 K = = 41.7 0.24
Thus, the transfer function of the compensator becomes Gc (s ) = 41.7 Note that Gc ( s ) Gc ( s ) G1 ( s ) = 10G(s ) = Gc (s )G(s ) K 10
40
0.227s + 1 s + 4.41 = 10 s + 18.4 0.054s + 1
The solid curves in Fig. 17 show the magnitude curve and phase-angle curve for the compensated system. The lead compensator causes the gain crossover frequency to increase from 6.3 to 9 rad/s. The increase in this frequency means an increase in bandwidth. This implies an increase in the speed of response. The phase and gain margins are seen to be approximately 50 and dB, respectively. The compensated system shown in Fig. 18 therefore meets both the steady-state and the relative-stability requirements. Note that for type 1 systems, such as the system just considered, the value of the static velocity error constant Kv is merely the value of the frequency corresponding to the intersection of the extension of the initial 20 dB/decade slope line and the 0 dB line, as shown in Fig. 17. Figures 19 and 20 show, respectively, the unit-step and unit-ramp responses of both the compensated system and uncompensated system. Lag Compensation Characteristics of Lag Compensators. Consider the lag compensator given by Eq. (3). Figure 21 shows a polar plot of the lag compensator. Figure 22 shows a Bode diagram of the compensator, where Kc 1 and 10. The corner frequencies of the lag compensator are at 1/ T and 1/(T). As seen from Fig. 22, where the values of Kc and are set equal to 1 and 10, respectively, the magnitude of the lag com-
20 Gc 10 6.2 dB Kv GcG G1 = 10G 40
dB
20
1.4 Compensated system 1.2 1

Outputs
0.8 0.6 0.4 0.2
0 90 180 1 G1 = 10G
Gc 10 GcG
50 2 4 6 10 20
in rad/s
40 60 100
3 t (s)
Figure 17. Bode diagram for the compensated system.
Figure 19. Unit-step response curves of the compensated and uncompensated systems.
294
5 4.5 4 3.5 3
Outputs
30 dB 20 10
Compensated system
0 0 90 0.01 T
2.5 2 1.5 1 0.5 0 0 0.5 1 1.5 2 2.5 t (s) 3 3.5 4 4.5 5 Uncompensated system
0.1 T
in rad/s
1 T
10 T
Figure 22. Bode diagram of a lag compensator ( jT 1)/( jT 1), with 10.
Dene Kc = K Then
Figure 20. Unit-ramp response curves of the compensated and uncompensated systems.
pensator becomes 10 (or 20 dB) at low frequencies and unity (or 0 dB) at high frequencies. Thus, the lag compensator is essentially a low-pass lter. Lag Compensation Techniques Based on the Frequency-Response Approach. The primary function of a lag compensator is to provide attenuation in the high-frequency range to give a system sufcient phase margin. The phase lag characteristic is of no consequence in lag compensation. The procedure for designing lag compensators for the system shown in Fig. 14 by the frequency-response approach may be stated as follows: 1. Assume the following lag compensator:
Gc ( s ) = K
Ts + 1 Ts + 1
G c ( s )G ( s ) = K
Ts + 1 Ts + 1 G (s ) = KG(s ) Ts + 1 Ts + 1 Ts + 1 = G (s ) Ts + 1 1
where G1 (s ) = KG(s ) Determine gain K to satisfy the requirement on the given static error constant. 2. If the uncompensated system G1( j) KG( j) does not satisfy the specications on the phase and gain margins, then nd the frequency point where the phase angle of the open-loop transfer function is equal to 180 plus the required phase margin. The required phase margin is the specied phase margin plus 5 to 12. (The addition of 5 to 12 compensates for the phase lag of the lag compensator.) Choose this frequency as the new gain crossover frequency. 3. To prevent detrimental effects of phase lag due to the lag compensator, the pole and zero of the lag compensator must be located substantially lower than the new gain crossover frequency. Therefore, choose the corner frequency 1/ T (corresponding to the zero of the lag compensator) 1 octave to 1 decade below the new gain crossover frequency. (If the time constants of the lag compensator do not become too large, the corner frequency 1/ T may be chosen 1 decade below the new gain crossover frequency.) 4. Determine the attenuation necessary to bring the magnitude curve down to 0 dB at the new gain crossover frequency. Noting that this attenuation is 20 log ,
( > 1 )
Im
Kc 0
=
Kc
=0
Re
Figure 21. Polar plot of a lag compensator Kc( jT 1)/( jT 1).
295
determine the value of . Then the other corner frequency (corresponding to the pole of the lag compensator) is determined from 1/(T). 5. Using the value of K determined in step 1 and that of determined in step 4, calculate constant Kc from K Kc = Example 5. Consider the system shown in Fig. 23. The open-loop transfer function is given by 1 G (s ) = s(s + 1 )(0.5s + 1 ) It is desired to compensate the system so that the static velocity error constant Kv is 5 s1, the phase margin is at least 40, and the gain margin is at least 10 dB. We shall use a lag compensator of the form
40 20 0 20 40 0 90 180 270 0.004 0.01 0.02 0.04 Gc GcG 11 dB G1
dB
Gc
G1 40 GcG 0.1 0.2 0.4 0.6 1 2 4
in rad/s
Figure 24. Bode diagrams for the uncompensated system, the compensator, and the compensated system. (G1: uncompensated system, Gc: compensator, GcG: compensated system.)
Dene Kc = K Dene also G1 (s ) = KG(s ) =
( > 1 )
The magnitude curve and phase-angle curve of G1( j) are shown in Fig. 24. From this plot, the phase margin is found to be 20, which means that the system is unstable. Noting that the addition of a lag compensator modies the phase curve of the Bode diagram, we must allow 5 to 12 to the specied phase margin to compensate for the modication of the phase curve. Since the frequency corresponding to a phase margin of 40 is 0.7 rad/s, the new gain crossover frequency (of the compensated system) must be chosen near this value. To avoid overly large time constants for the lag compensator, we shall choose the corner frequency 1/ T (which corresponds to the zero of the lag compensator) to be 0.1 rad/s. Since this corner frequency is not too far below the new gain crossover frequency, the modication in the phase curve may not be small. Hence, we add about 12 to the given phase margin as an allowance to account for the lag angle introduced by the lag compensator. The required phase margin is now 52. The phase angle of the uncompensated openloop transfer function is 128 at about 0.5 rad/s. So we choose the new gain crossover frequency to be 0.5 rad/s. To bring the magnitude curve down to 0 dB at this new gain crossover frequency, the lag compensator must give the necessary attenuation, which in this case is 20 dB. Hence, 20 log or = 10 1 = 20
K s(s 1 )(0.5s + 1 )
The rst step in the design is to adjust the gain K to meet the required static velocity error constant. Thus,
Kv = lim sGc (s )G(s ) = lim s

s0 s0
Ts + 1 G (s ) = lim sG1 (s ) s0 Ts + 1 1
= lim
s0
sK =K=5 s(s + 1 )(0.5s + 1 )
With K 5, the compensated system satises the steadystate performance requirement. We shall next plot the Bode diagram of 5 G1 ( j ) = j ( j + 1 )(0.5 j + 1 )
1 s(s + 1) (0.5s + 1)
The other corner frequency 1(T), which corresponds to the pole of the lag compensator, is then determined as 1 = 0.01 rad/s T
296
Thus, the transfer function of the lag compensator is
20 18 16 14
Outputs
1 s+ 10s + 1 10 = Kc Gc (s ) = Kc (10 ) 1 100s + 1 s+ 100

Since the gain K was determined to be 5 and was determined to be 10, we have 5 K = = 0.5 Kc = 10 The open-loop transfer function of the compensated system is G c ( s )G ( s ) = 5 (10s + 1 ) s(100s + 1 )(s + 1 )(0.5s + 1 )
12 10 8 6 4 2 0 0 2 4 6 Uncompensated system Compensated system 8 10 t (s) 12 14 16 18 20
The magnitude and phase-angle curves of Gc( j)G( j) are also shown in Fig. 24. The phase margin of the compensated system is about 40, which is the required value. The gain margin is about 11 dB, which is quite acceptable. The static velocity error constant is 5 s1, as required. The compensated system, therefore, satises the requirements on both the steady state and the relative stability. Note that the new gain crossover frequency is decreased from approximately 2 to 0.5 rad/s. This means that the bandwidth of the system is reduced. Figures 25 and 26 show, respectively, the unit-step and unit-ramp responses of the compensated and uncompensated systems. (The uncompensated system is shown in Fig. 23.) Lag-Lead Compensation Lag-Lead Compensation Based on the Frequency-Response Approach. The design of a lag-lead compensator by the frequency-response approach is based on the combination of the design techniques discussed under lead compensation and lag compensation.
Figure 26. Unit-ramp response curves for the compensated and uncompensated systems.
Let us assume that the lag-lead compensator is of the following form:
Gc (s ) = Kc
1 s+ (T1 s + 1 )(T2 s + 1 ) T1 = Kc T1 s + 1 ( T2 s + 1 ) s+ T1
1 T2 1 s+ T2 s+
(8)
1.4 1.2 1
where 1. The phase lead portion of the lag-lead compensator (the portion involving T1) alters the frequency-response curve by adding phase lead angle and increasing the phase margin at the gain crossover frequency. The phase lag portion (the portion involving T2) provides attenuation near and above the gain crossover frequency and thereby allows an increase of gain at the low-frequency range to improve the steady-state performance. Figure 27 shows a Bode diagram of a lag-lead compensator when Kc 1, 10, and T2 10T1. Notice that the magnitude curve has the value 0 dB at both low-frequency and highfrequency regions.
Compensated system
10 0 dB 10 20 30 90 0 90 0.001 T1
Outputs
0.8 0.6 0.4 0.2 0
10
15
20 t (s)
25
30
35
40
0.01 T1
0.1 T1
1 T1
10 T1
100 T1
in rad/s
Figure 27. Bode diagram of a lag-lead compensator given by Eq. (8) with Kc 1, 10, and T2 10T1.
Figure 25. Unit-step response curves for the compensated and uncompensated systems.
297
60
40 20 dB 0 20 40 90 0 90 180 270 0.01 Gc 50 GcG G 1 2

32
G GcG Gc 16 dB
Once we choose the gain crossover frequency to be 1.5 rad/ s, we can determine the corner frequency of the phase lag portion of the lag-lead compensator. Let us choose the corner frequency 1/ T2 (which corresponds to the zero of the phase-lag portion of the compensator) to be 1 decade below the new gain crossover frequency, or at 0.15 rad/s. Recall that for the lead compensator the maximum phase lead angle m is given by Eq. (5), where in Eq. (5) is 1/ in the present case. By substituting 1/ in Eq. (5), we have
1 1 = sin m = 1 +1 1+ 1
Notice that 10 corresponds to m 54.9. Since we need a 50 phase margin, we may choose 10. (Note that we will be using several degrees less than the maximum angle, 54.9.) Thus, = 10 Then the corner frequency 1/ T2 (which corresponds to the pole of the phase lag portion of the compensator) becomes 0.015 rad/s. The transfer function of the phase lag portion of the lag-lead compensator then becomes s + 0.15 6.67s + 1 = 10 s + 0.015 66.7s + 1 The phase lead portion can be determined as follows: Since the new gain crossover frequency is 1.5 rad/s, from Fig. 28, G( j1.5) is found to be 13 dB. Hence, if the lag-lead compensator contributes 13 dB at 1.5 rad/s, then the new gain crossover frequency is as desired. From this requirement, it is possible to draw a straight line of slope 20 dB/ decade, passing through the point (13 dB, 1.5 rad/s). The intersections of this line and the 0 dB line and 20 dB line determine the corner frequencies. Thus, the corner frequencies for the lead portion are 0.7 rad/s and 7 rad/s. Thus, the transfer function of the lead portion of the lag-lead compensator becomes s + 0.7 1 = s+7 10 1.43s + 1 0.143s + 1
0.04 0.1 0.2 0.4

in rad/s
4 6 10
Figure 28. Bode diagrams for the uncompensated system, the compensator, and the compensated system. (G: uncompensated system, Gc: compensator, GcG: compensated system.)
We shall illustrate the details of the procedure for designing a lag-lead compensator by an example. Example 6. Consider the unity-feedback system whose open-loop transfer function is G (s ) = K s(s + 1 )(s + 2 )
It is desired that the static velocity error constant be 10 s1, the phase margin be 50, and the gain margin be 10 dB or more. Assume that we use the lag-lead compensator given by Eq. (8). The open-loop transfer function of the compensated system is Gc(s)G(s). Since the gain K of the plant is adjustable, let us assume that Kc 1. Then lims0 Gc(s) 1. From the requirement on the static velocity error constant, we obtain K K = = 10 Kv = lim sGc (s )G(s ) = lim sGc (s ) s0 s0 s(s + 1 )(s + 2 ) 2 Hence, K = 20 We shall next draw the Bode diagram of the uncompensated system with K 20, as shown in Fig. 28. The phase margin of the uncompensated system is found to be 32, which indicates that the incompensated system is unstable. The next step in the design of a lag-lead compensator is to choose a new gain crossover frequency. From the phase angle curve for G( j), we notice that G( j) 180 at 1.5 rad/s. It is convenient to choose the new gain crossover frequency to be 1.5 rad/s so that the phase-lead angle required at 1.5 rad/s is about 50, which is quite possible by use of a single lag-lead compensator.
Combining the transfer functions of the lag and lead portions of the compensator, we obtain the transfer function of the laglead compensator. Since we chose Kc 1, we have Gc ( s ) = s + 0.7 s+7 s + 0.15 s + 0.015 = 1.43s + 1 0.143s + 1 6.67s + 1 66.7s + 1
The magnitude and phase-angle curves of the lag-lead compensator just designed are shown in Fig. 28. The open-loop transfer function of the compensated system is
G c ( s )G ( s ) =
(s + 0.7 )(s + 0.15 )20 (s + 7 )(s + 0.015 )s(s + 1 )(s + 2 ) 10 (1.43s + 1 )(6.67s + 1 ) = s(0.143s + 1 )(66.7s + 1 )(s + 1 )(0.5s + 1 )
(9)
298
1.6 1.4 1.2 1

Output
3.
0.8 0.6 0.4 0.2 0 0 2 4 6 8 10 12 t (s) 14 16 18 20
4.
Figure 29. Unit-step response of the compensated system.
The magnitude and phase-angle curves of the system of Eq. (9) are also shown in Fig. 28. The phase margin of the compensated system is 50, the gain margin is 16 dB, and the static velocity error constant is 10 s1. All the requirements are therefore met, and the design has been completed. The unit-step response and unit-ramp response of the designed system are shown in Figs. 29 and 30, respectively. COMPARISON OF LEAD, LAG, LAG-LEAD COMPENSATION 1. Lead compensation achieves the desired result through the merits of its phase-lead contribution, whereas lag compensation accomplishes the result through the merits of its attenuation property at high frequencies. (In some design problems both lag compensation and lead compensation may satisfy the specications.) 2. Lead compensation is commonly used for improving stability margins. Lead compensation yields a higher gain crossover frequency than is possible with lag compensation. The higher gain crossover frequency means larger bandwidth. A large bandwidth means reduction in the settling time. The bandwidth of a system with lead compensation is always greater than that with lag compensation. Therefore, if a large bandwidth or fast response is desired, lead compensation should be employed. If, however, noise signals are present, then a large bandwidth may not be desirable, since it makes the system
5.
6.
more susceptible to noise signals because of increase in the high-frequency gain. Lead compensation requires an additional increase in gain to offset the attenuation inherent in the lead network. This means that lead compensation will require a larger gain than that required by lag compensation. A larger gain, in most cases, implies larger space, greater weight, and higher cost. Lag compensation reduces the system gain at higher frequencies without reducing the system gain at lower frequencies. Since the system bandwidth is reduced, the system has a slower speed to respond. Because of the reduced high-frequency gain, the total system gain can be increased, and thereby low-frequency gain can be increased and the steady-state accuracy can be improved. Also, any high-frequency noises involved in the system can be attenuated. If both fast responses and good static accuracy are desired, a lag-lead compensator may be employed. By use of the lag-lead compensator, the low-frequency gain can be increased (which means an improvement in steadystate accuracy), while at the same time the system bandwidth and stability margins can be increased. Although a large number of practical compensation tasks can be accomplished with lead, lag, or lag-lead compensators, for complicated systems, simple compensation by use of these compensators may not yield satisfactory results. Then different compensators having different pole-zero congurations must be employed.
MULTI-DEGREES-OF-FREEDOM CONTROL In the classical design approaches presented in this article, we design control systems such that the response to the reference input is satisfactory. If the control system is subjected to other inputs, such as disturbance input and noise input, it is not possible to design the system such that the responses to the disturbance input and noise input are also satisfactory, in addition to the primary requirement that the response to the reference input is satisfactory. This is because the systems we considered so far simply do not have the freedom to satisfy requirements on the responses to disturbances and noises. If we wish to design high-performance control systems in the presence of disturbances and sensor noises, we must change the conguration of the control system. This means that we must provide additional degrees of freedom to the control system to handle additional requirements.
20 18 16 14
Output
D(s)
12 10 8 6 4 2 0 0 2 4 6 8 10 12 14 16 18 20 t (s)
R(s) +
Gc(s)
U(s) + +
Y(s) Gp(s)
B(s) + N(s) +
Figure 30. Unit-ramp response of the compensated system.
Figure 31. One-degree-of-freedom control system.
299
D(s)
Gc2(s)
R(s) + Gc1(s) + U(s) + + Gp(s) Y(s)
D(s)
R(s)
B(s)
Gc1(s)
+ U(s) + +
Y(s) Gp(s)
B(s)
Gc2(s) B(s) + N(s) +
N(s)
Figure 32. Two-degrees-of-freedom control system.
Figure 33. Two-degrees-of-freedom control system.
In what follows we rst discuss the single-degree-of-freedom control systems and then discuss the two-degrees-offreedom control systems. Finally, we present an example of three-degrees-of-freedom control systems that can satisfy the requirements on the responses to the reference input, disturbance input, and noise input. Single-Degree-of-Freedom Control Consider the system shown in Fig. 31, where the system is subjected to the disturbance input D(s) and noise input N(s). Gc(s) is the transfer function of the controller and Gp(s) is the transfer function of the plant. We assume that Gp(s) is xed and unalterable. For this system, three closed-loop transfer functions Y(s)/ R(s) Gyr, Y(s)/ D(s) Gyd, and Y(s)/ N(s) Gyn may be derived. They are
Among the three closed-loop transfer functions Gyr, Gyn, and Gyd, if one of them is given, the remaining two are xed. This means that the system shown in Fig. 31 is a one-degree-offreedom system. Two-Degrees-of-Freedom Control Next consider the system shown in Fig. 32, where Gp(s) is the transfer function of the plant and is assumed to be xed and unalterable. For this system, closed-loop transfer functions Gyr, Gyn, and Gyd are given, respectively, by
Gyr = Gyd Gyn
Gyr Gyd Gyn
Gc Gp Y (s ) = = R (s ) 1 + Gc Gp Gp Y (s ) = = D (s ) 1 + Gc Gp Gc Gp Y (s ) = = N (s ) 1 + Gc Gp
Gc1 Gp Y (s ) = R (s ) 1 + (Gc1 + Gc2 )Gp Gp Y (s ) = = D (s ) 1 + (Gc1 + Gc2 )Gp Gc1 + Gc2 )Gp Y (s ) = = N (s ) 1 + (Gc1 + Gc2 )Gp
Hence, we have
Gyr = Gc1 Gyd Gyn = Gyd Gp Gp
[In deriving Y(s)/ R(s), we assumed D(s) 0 and N(s) 0. Similar comments apply to the derivations of Y(s)/ D(s) and Y(s)/ N(s).] The degrees of freedom of the control system refers to how many of these closed-loop transfer functions are independent. In the present case, we have
Gyr = Gyn =
Gp Gyd Gp Gyd Gp Gp
In this case, if Gyd is given, then Gyn is xed, but Gyr is not xed, because Gc1 is independent of Gyd. Thus, two closed-loop transfer functions among three closed-loop transfer functions Gyr, Gyd, and Gyn are independent. Hence, this system is a twodegrees-of-freedom control system. Similarly, the system shown in Fig. 33 is also a twodegrees-of-freedom control system, because for this system
Gc2 R(s) + Gc1 +
D(s) Y(s) G1 G2
Gc3
N(s) +
Figure 34. Three-degrees-of-freedom system.
300
CONVEX OPTIMIZATION
Gyr = Gyd Gyn

Hence,
Gc1 Gp Y (s ) Gc2 Gp = + R (s ) 1 + Gc1 Gp 1 + Gc1 Gp Gp Y (s ) = = D (s ) 1 + Gc1 Gp Gc1 Gp Y (s ) = = N (s ) 1 + Gc1 Gp
K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle River, NJ: Prentice Hall, 1997. K. Ogata, System Dynamics, 3rd ed., Upper Saddle River, NJ: Prentice Hall, 1998. K. Ogata, Solving Control Engineering Problems with MATLAB, Upper Saddle River, NJ: Prentice Hall, 1994. C. L. Phillips and R. D. Harbor, Feedback Control Systems, Upper Saddle River, NJ: Prentice Hall, 1988.
Gyr = Gc2 Gyd + Gyn = Gyd Gp Gp
Gp Gyd Gp
KATSUHIKO OGATA
University of Minnesota
CONTROL SYSTEM SYNTHESIS. See DELAY SYSTEMS;

SERVOMECHANISMS.
Clearly, if Gyd is given, then Gyn is xed, but Gyr is not xed because Gc2 is independent of Gyd. Three-Degrees-of-Freedom Control In the control system shown in Fig. 34, the transfer functions Gc1, Gc2, and Gc3 are controllers, and transfer functions G1 and G2 are plant transfer functions that are unalterable. It can be shown that this control system is a three-degrees-of-freedom system. If a system has this conguration, then it is possible to design three controllers by use of the root-locus method and/or frequency-response method (or other methods) such that the responses to all three inputs are acceptable. CONCLUDING COMMENTS This article has presented easy-to-understand procedures for designing lead compensators, lag compensators, and lag-lead compensators by use of the root-locus method or frequencyresponse method. The systems are limited to single-input single-output, linear time-invariant control systems. For such systems various design methods are available in addition to the root-locus method and frequency-response method. Interested readers are referred to specialized books on control systems, as listed in the Reading List. Toward the end of this article we included discussions on multi-degrees-of-freedom control systems for the informed specialist. Most of the materials presented in this article were taken, with permission, from Katsuhiko Ogata, Modern Control Engineering 3/e, 1997. Prentice Hall, Upper Saddle River, New Jersey.
Reading List H. W. Bode, Network Analysis and Feedback Design, New York: Van Nostrand Reinhold, 1945. R. C. Dorf, Modern Control Systems, 6th ed., Reading, MA: AddisonWesley, 1992. W. R. Evans, Graphical analysis of control systems, AIEE Trans. Part II, 67: 547551, 1948. W. R. Evans, Control system synthesis by root locus method, AIEE Trans. Part II, 69: 6669, 1950. W. R. Evans, The use of zeros and poles for frequency response or transient response, ASME Trans., 76: 11351144, 1954. G. F. Franklin, J. D. Powell, and A. Emami-Naeini, Feedback Control of Dynamic Systems, Reading, MA: Addison-Wesley, 1986. B. C. Kuo, Automatic Control Systems, 6th ed., Upper Saddle River, NJ: Prentice Hall, 1991.
CONVERSION, THERMIONIC. See THERMIONIC CONVERSION.
CONVERTERS. See ANALOG-TO-DIGITAL CONVERSION. CONVERTERS, AC-AC. See AC-AC POWER CONVERTERS. CONVERTERS, AC-DC. See AC-DC POWER CONVERTERS. CONVERTERS, DC-AC. See DC-AC POWER CONVERTERS. CONVERTER TO BOOST VOLTAGE. See SYNCHRONOUS CONVERTER TO BOOST BATTERY VOLTAGE.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Convex Optimization Standard Article Sachin S. Sapatnekar1 1University of Minnesota, Minneapolis, MN Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1008 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (264K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Definitions of Convexity Convex Optimization Related Techniques Engineering Problems as Convex Optimization Problems Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
300
CONVEX OPTIMIZATION
CONVEX OPTIMIZATION
An optimization problem can be stated in the so-called standard form as follows: minimize f (x ) : R R g : Rm Rn (NLP)
subject to g (x ) 0,
representing the minimization of a function f of n variables under constraints specied by inequalities determined by functions g [g1, g2, . . ., gm]T. The functions f and gi are, in general, nonlinear functions. Note that inequalities can be handled under this paradigm by multiplying each side by 1, and equalities by representing them as pairs of inequalities. The maximization of an objective function f (x) can be achieved by minimizing f (x). The set F xg(x) 0 that satises the constraints on the nonlinear optimization problem is known as the feasible set, or the feasible region. If F covers all of (a part of) Rn, then the optimization is said to be unconstrained (constrained). Note that the above standard-form formulation may not be directly applicable to real life design problems, where often, multiple conicting objectives must be optimized. In such a case, multicriterion optimization techniques and Pareto optimality can be used to identify noninferior solutions (1). In practice, however, techniques to map the problem to the form in Eq. (NLP) are often used. When the objective function is a convex function and the constraint set is a convex set (both terms will be formally dened later), the optimization problem is known as a convex programming problem. This problem has the remarkable property of unimodality, i.e., any local minimum of the problem is also a global minimum. Therefore, it does not require special methods to extract the solution out of local minima in a quest to nd the global minimum. While the convex programming problem and its unimodality property have been known for a long time, it is only recently that efcient algoJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
CONVEX OPTIMIZATION
301
x1 x2 x1 x2 Convex set Nonconvex set

Figure 3. A separating hyperplane (line) in two dimensions between convex sets A and B.
A B
Figure 1. Convex and nonconvex sets.
rithms for the solution of these problems have been proposed. The genesis of these algorithms can be traced back to the work of Karmarkar (2) that proposed a polynomial-time interior-point technique for linear programming, a special case of convex programming. Unlike the simplex method for linear programming, this technique was found to be naturally extensible from the problem of linear programming to general convex programming formulations. It was shown that this method belongs to the class of interior penalty methods proposed by Fiacco and McCormick (3) using barrier functions. The work of Renegar (4) showed that a special version of the method of centers for linear programming is polynomial. Gonzaga (5) showed similar results for the barrier function associated with a linear programming problem, with a proof of polynomial-time complexity. Nesterov and Nemiorvsky (6) introduced the concept of self-concordance, studying barrier methods in their context. Further improvements in the computational complexity using approximate solutions and rankone updates were shown in the work of Vaidya (7). The work of Ye (8) used a potential function to obtain the same complexity as Renegars work without following the central path too closely.
Elementary Convex Sets Ellipsoids. An ellipsoid E(x, B , r) Rn centered at point x Rn is given by the equation { y | ( y x ) T B ( y x ) r2 } If B is a scalar multiple of the unit matrix, then the ellipsoid is called a hypersphere. The axes of the ellipsoid are given by the eigenvectors, and their lengths are related to the corresponding eigenvalues of B . Hyperplanes. A hyperplane in n dimensions is given by the region c Tx = b, c Rn , bR
Half spaces. A half space in n dimensions is dened by the region that satises the inequality c Tx b, c Rn , bR
Polyhedra. A (convex) polyhedron is dened as an intersection of half spaces, and is given by the equation x b ), P = {x |Ax A Rm n , b Rm
DEFINITIONS OF CONVEXITY Convex Sets Denition. A set C Rn is said to be a convex set if, for every x1, x2 C and every real number , 0 1, the point x1 (1 )x2 C. This denition can be interpreted geometrically as stating that a set is convex if, given two points in the set, every point on the line segment joining the two points is also a member of the set. Examples of convex and nonconvex sets are shown in Fig. 1.
corresponding to a set of m inequalities aT i x bi, ai Rn. If the polyhedron has a nite volume, it is referred to as a polytope. An example of a polytope is shown in Fig. 2. Some Elementary Properties of Convex Sets Property. The intersection of convex sets is a convex set. The union of convex sets is not necessarily a convex set. Property (Separating hyperplane theorem). Given two nonintersecting convex sets A and B, there exists a separating hyperplane cTx b such that A lies entirely within the half space cTx b and B lies entirely within the half space cTx b. This is pictorially illustrated in Fig. 3. Property (Supporting hyperplane theorem). Given a convex set C and any point x on its boundary, there exists a supporting hyperplane cTx b such that C lies entirely within the half space cTx b. This is illustrated in Fig. 4.
x
Figure 2. An example convex polytope in two dimensions as an intersection of ve half spaces. Figure 4. A supporting hyperplane (line) in two dimensions at the boundary point of a convex set S.
302
CONVEX OPTIMIZATION
x1 x2
x3
x4
x5
c =1 c = 2
Figure 5. An example showing the convex hull of ve points.
c=3 x
Denition. The convex hull of m points, x1, x2, . . ., xm Rn, denoted co(x1, x2, . . ., xm), is dened as the set of points y Rn such that
y=
X
m i= 1
i x i ,
i 0 i,
X
m i= 1
i = 1
Figure 7. Level sets of f (x,y) x2 y2.
The convex hull is thus the smallest convex set that contains the m points. An example of the convex hull of ve points in the plane is shown by the shaded region in Fig. 5. If the set of points xi is of nite cardinality (i.e., m is nite), then the convex hull is a polytope. Hence, a polytope is also often described as the convex hull of its vertices. Convex Functions Denition. A function f dened on a convex set is said to be a convex function if, for every x1, x2 and every , 0 1, f [x 1 + (1 )x 2 ] f (x 1 ) + (1 ) f (x 2 ) f is said to be strictly convex if the above inequality is strict for 0 1. Geometrically, a function is convex if the line joining two points on its graph is always above the graph. Examples of convex and nonconvex functions on Rn are shown in Fig. 6. Some Elementary Properties of Convex Functions Property. A function f (x) is convex over the set S if and only if f (x ) f (x 0 ) + [ f (x 0 )]T (x x 0 ) x , x 0 S
Property. A function f (x) is convex over the convex set S if and only if 2 f (x 0 ) 0 x 0 S
i.e., its Hessian matrix is positive semidenite over S. For strict convexity, 2f (x0) must be positive denite. Property. If f (x) and g(x) are two convex functions on the convex set S, then the functions f g and max( f, g) are convex over S. Denition. The level set of a function f (x) is the set dened by f (x) c where c is a constant. An example of the level sets of f (x, y) x2 y2 is shown in Fig. 7. Observe that the level set for f (x, y) c1 is contained in the level set of f (x, y) c2 for c1 c2. Property. If f is a convex function in the space S , then the level set of f is a convex set in S . This is a very useful property, and many convex optimization algorithms depend on the fact that the constraints are dened by an intersection of level sets of convex functions. Denition. A function g dened on a convex set is said to be a concave function if the function f g is convex. The function g is strictly concave if g is strictly convex. For a fuller treatment of convexity properties, the reader is referred to Ref. 9.
where f corresponds to the gradient of f with respect to the vector x. Strict convexity corresponds to the case where the inequality is strict.
f ( x)
g( x)
x1
Figure 6. Convex and nonconvex functions.
x2
x1
x2
Convex function
Nonconvex function
CONVEX OPTIMIZATION
303
CONVEX OPTIMIZATION Convex Programming Denition. A convex programming problem is an optimization problem that can be stated as follows: minimize f (x ) (CP)
Barrier Methods. The barrier technique of Fiacco and McCormick (3) is a general technique to solve any constrained nonlinear optimization problem by solving a sequence of unconstrained nonlinear optimization problems. This method may be used for the specic case of minimizing a convex function over a convex set S described by an intersection of convex inequalities S = {x | gi (x ) 0, i = 1, 2, . . ., m} where each gi(x) is a convex function. The computation required of the method is dependent on the choice of the barrier function. In this connection, the logarithmic barrier function (abbreviated as the log barrier function) for the set of inequalities is dened as
such that x S where f is a convex function and S is a convex set.
Such a problem has the property that any local minimum of f over S is a global minimum. Comment. The problem of maximizing a convex function over a convex set does not have the above property. However, it can be shown (10) that the maximum value for such a problem lies on the boundary of the convex set. For a convex programming problem of the type (CP), we may state without loss of generality that the objective function is linear. To see this, note that the problem (CP) may equivalently be written as min t:f (x) t, g(x) 0 Linear programming is a special case of nonlinear optimization, and more specically, a special type of convex programming problem where the objective and constraints are all linear functions. The problem is stated as
(x ) =
( Pn
0
i = 1 log[gi (x )]
for x S otherwise
minimize c Tx subject to where x b, Ax

mn
Intuitively, the idea of the barrier is that any iterative gradient-based method that tries to minimize the barrier function will be forced to remain in the feasible region, due to the singularity at the boundary of the feasible region. It can be shown that (x) is a convex function over S and its value approaches innity as x approaches the boundary of S. Intuitively, it can be seen that (x) becomes smallest when x is, in some sense, farthest away from all of the boundaries of S. The value of x at which the function (x) is minimized is called the analytic center of S. Example. For a linear programming problem of the type described in Eq. (LP), with constraint inequalities described by aT i x bi, the barrier function in the feasible region is given by
x0 bR ,
m
(LP) cR ,
n
AR
xR
The feasible region for a linear program corresponds to a polyhedron in Rn. It can be shown that an optimal solution to a linear program must necessarily occur at one of the vertices of the constraining polyhedron. The most commonly used technique for solution of linear programs, the simplex method (11), is based on this principle and operates by a systematic search of the vertices of the polyhedron. The computational complexity of this method can show exponential behavior for pathological cases, but for most practical problems it has been observed to grow linearly with the number of variables and sublinearly with the number of constraints. Algorithms with polynomial-time worst-case complexity do exist; these include Karmarkars method (2) and the ShorKhachiyan ellipsoidal method (12). The computational times of the latter, however, are often seen to be impractical. In the remainder of this section, we will describe various methods used for convex programming and for mapping problems to convex programs. Path-Following Methods This class of methods proceeds iteratively by solving a sequence of unconstrained problems that lead to the solution of the original problem. In each iteration, a technique based on barrier methods is used to nd the optimal solution. If we denote the optimal solution in the kth iteration as x* k , then n the path x* , x * , . . . in R leads to the optimal solution, and 1 2 hence techniques of this class are known as path-following methods.
(x ) =
X
n i= 1
log(bi a T i x)
The value of bi aT i x represents the slack in the ith inequality, i.e., the distance between the point x and the corresponding constraint hyperplane. The log barrier function, therefore, is a measure of the product of the distances from a point x to each hyperplane, as shown in Fig. 8(a). The value of (x) is minimized when in1(bi aT i x) is maximized. Coarsely speaking, this occurs when the distance to each constraint hyperplane is sufciently large. As a cautionary note, we add that while the analytic center is a good estimate of the center in the case where all con-
Analytic center
p (a)
Analytic center (b)
Figure 8. (a) Physical meaning of the barrier function for the feasible region of a linear program; (b) the effect of redundant constraints on the analytic center.
304
CONVEX OPTIMIZATION
straints present an equal contribution to the boundary, the presence of redundant constraints can shift the analytic center. The effect on the analytic center of repeating the constraint p ve times is shown in Fig. 8(b). We will now consider the convex programming problem specied as minimize f (x ) i = 1, 2, . . ., m
such that g(x ) 0,
The Self-Concordance Property and Step Length. The value of above is an adjustable parameter that will affect the number of Newton iterations required to nd the optimum value of the analytic center. Depending on the value of chosen, the technique is classied as a short-step, medium-step, or long-step (possibly even with 1) method. For a short-step method, one Newton iteration is enough, while for longer steps, further Newton iterations may be necessary. Nesterov and Nemirovskii (6) introduced the following idea of the self-concordance condition: Denition. A convex function : S R is self-concordant with parameter a 0 (a-self-concordant) on S if is three times continuously differentiable on S and for all x S and h Rm, the following inequality holds: |D3 (x )[h, h , h ]| 2a 1/2{D2 (x )[h , h]}3/2 where Dk(x)[h1, h2, . . ., hk] denotes the value of the kth differential of taken at x along the collection of directions [h1, h2, . . ., hk]. By formulating logarithmic barrier functions that are selfconcordant, proofs of the polynomial-time complexity of various interior point methods have been shown. An analysis of the computational complexity in terms of the number of outer and inner (Newton) iterations is presented in Refs. 13 and 6. Other Interior-Point Methods Afne Scaling Methods. For a linear programming problem, the nonnegativity constraints x 0 are replaced by constraints of the type X1 (x xc) 1, representing an ellipsoid centered at xc. The linear program is then relaxed to the following form, whose feasible region is contained in that of the original linear program: x = b , X 1 (x x c ) < 1} min{c Tx : Ax Note that the linear inequalities in Eq. (LP) have been converted to equalities by the addition of slack variables. This form has the following closed-form solution: x ( ) = x c X PAX Xc c PAX Xc
where each gi(x) is a convex function. The traditional barrier method (3) used to solve this problem formulates the corresponding unconstrained optimization problem minimize B() = f (x ) + (x )
and solves this problem for a sequence of increasing (constant) values of the parameter . When is zero, the problem reduces to nding the center of the convex set constraining the problem. As increases, the twin objectives of minimizing f (x) and remaining in the interior of the convex set are balanced. As , the problem amounts to the minimization of the original objective function f . In solving this sequence of problems, the outer iteration consists in increasing the values of the parameter . The inner iteration is used to minimize the value of B() at that value of , using the result of the previous outer iteration as an initial guess. The inner iteration can be solved using Newtons method (10). For positive values of , it is easy to see that B() is a convex function. The notion of a central path for a linearly constrained optimization problem is shown in Fig. 9. Method of Centers. Given a scalar value t f (x*), the method of centers nds the analytic center of the set of inequalities f (x) t and gi(x) 0 by minimizing the function log[t f (x )] + (x )
where (x) is the log barrier function dened earlier. The optimal value x* associated with solving the optimization problem associated with nding the analytic center for this barrier function is found, and the value of t is updated to be a convex combination of t and f (x*) as t t + (1 ) f (x ), >0
Optimum
= 10
=0 (Analytic center)
= 100
where PAX I XAT(AX2AT)1AX. The updated value of x is used in the next iteration, and so on. The search direction XPAXXc is called the primal afne scaling direction and corresponds to the scaled projected gradient with respect to the objective function, with scaling matrix X. Depending on the value of , the method may be a short-step or a long-step (with 1) method, and convergence proofs under various conditions are derived. For details of the references, the reader is referred to Ref. 13. We consider a general convex programming problem of the type (note that the linear objective function form is used here) {min f ( y ) = b Ty : gi ( y ) 0}
Figure 9. Central path for a linearly constrained convex optimization problem.
The constraint set here is similarly replaced by the ellipsoidal constraint (y yc)T H (y yc) 2, where yc is the center of
CONVEX OPTIMIZATION
305
the current ellipsoid, y is the variable over which the minimization is being performed, and H is the Hessian of the logn barrier function i1 log[gi(y)]. The problem now reduces to {min b Ty : ( y y c )T H ( y y c ) 2 } which has a closed-form solution of the form H 1b y ( ) = y b T H 1b This is used as the center of the ellipsoid in the next iteration. The procedure continues iteratively until convergence. Potential-Based Methods. These methods formulate a potential function that provides a coarse estimate of how far the solution at the current iterate is from the optimal value. At each step of a potential reduction method, a direction and step size are prescribed; however, the potential may be minimized further by the use of a line search (large steps). This is in contrast with a path-following method that must maintain proximity to a prescribed central path. An example of a potential-based technique is one that utilizes a weighted sum of the gap between the value of the primal optimization problem and its dual, and of the log barrier function value as the potential. For a more detailed description of this and other potential-based methods, the reader is referred to Refs. 6 and 13. Localization Approaches Polytope Reduction. This method begins with a polytope P Rn that contains the optimal solution, xopt. The polytope P may, for example, be initially selected to be an ndimensional box described by the set {x |xmin x(i ) xmax } where xmin and xmax are the minimum and maximum values of each variable, respectively. In each iteration, the volume of the polytope is shrunk while keeping xopt within the polytope, until the polytope becomes sufciently small. The algorithm proceeds iteratively as follows. Step 1. A center xc deep in the interior of the polytope P is found. Step 2. The feasibility of the center xc is determined by verifying whether all of the constraints of the optimization problem are satised at xc. If the point xc is infeasible, it is possible to nd a separating hyperplane passing through xc that divides P into two parts, such that the feasible region lies entirely in the part satisfying the constraint c Tx where c [gp(x)] is the negative of the gradient of a violated constraint gp, and cTxc. The separating hyperplane above corresponds to the tangent plane to the violated constraint. If the point xc lies within the feasible region, then there exists a hyperplane cT x
T
that divides the polytope into two parts such that xopt is contained in one of them, with c [f (x)]T being the negative of the gradient of the objective function, and being dened as cT xc once again. This hyperplane is the supporting hyperplane for the set f (x) f (xc) and thus eliminates from the polytope a set of points whose value is larger than the value at the current center. In either case, a new constraint cT x is added to the current polytope to give a new polytope that has roughly half the original volume. Step 3. Go to step 1 and repeat the process until the polytope is sufciently small. Note that the center in step 1 is ideally the center of gravity of the current polytope, since a hyperplane passing through the center of gravity is guaranteed to reduce the volume of the polytope by a factor of 1 1/ e in each iteration. However, since nding the center of gravity is prohibitively expensive in terms of computation, the analytic center is an acceptable approximation. Example. The algorithm is illustrated by using it to solve the following problem in two dimensions: minimize such that f ( x1 , x2 ) ( x1 , x2 ) S
where S is a convex set and f is a convex function. The shaded region in Fig. 10(a) is the set S, and the dashed lines show the level curves of f . The point xopt is the solution to this problem. The expected solution region is rst bounded by a rectangle with center xc, as shown in Fig. 10(a). The feasibility of xc is then determined; in this case, it can be seen that xc is infeasible. Hence, the gradient of the constraint function is used to construct a hyperplane through xc such that the polytope is divided into two parts of roughly equal volume, one of which contains the solution xc. This is illustrated in Fig. 10(b), where the region enclosed in darkened lines corresponds to the updated polytope. The process is repeated on the new,
f decreasing
xc xopt xc xopt
(a)
(b)
xc xc xopt xopt
(c)
(d)
Figure 10. Polytope reduction approach.
306
CONVEX OPTIMIZATION
Ek
f (xk )
Some elementary examples of such functions are: 1. Clearly, any convex (concave) function is also quasiconvex (quasiconcave). 2. Any monotone function f : R R is quasilinear. 3. The linear fractional function f (x) (aTx b)/(cTx d) where a, c, x Rn, is quasilinear over the half space xcTx d 0. Other Characterizations of Quasiconvexity. A function f dened on a convex set is quasiconvex if, for every x1, x2 : 1. For every , 0 1, f ( x1 (1 )x2) max[f (x1), f (x2)]. 2. If f is differentiable, f (x1) f (x2) (x2 x1)T f (x1) 0. Property. If f ,g are quasiconvex over , then the functions f for 0 and max( f,g) are also quasiconvex over . The composed function g( f(x)) is quasiconvex provided g is monotone increasing. In general, the function f g is not quasiconvex over . As in the case of convex optimization, the gradient of a quasiconvex objective can be used to eliminate a half space from consideration. The work in Ref. 14 presents an adaptation of the ellipsoidal method to solve quasiconvex optimization problems. Semidenite Programming The problem of semidenite programming (15) is stated as follows:
xk
Ek + 1
Figure 11. The ellipsoidal method.
smaller polytope. Its center lies inside the feasible region, and hence the gradient of the objective function is used to generate a hyperplane that further shrinks the size of the polytope, as shown in Fig. 10(c). The result of another iteration is illustrated in Fig. 10(d). The process continues until the polytope has been shrunk sufciently. Ellipsoidal Method. The ellipsoidal method begins with a sufciently large ellipsoid that contains the solution to the convex optimization problem. In each iteration, the size of the ellipsoid is shrunk, while maintaining the invariant that the solution lies within the ellipsoid, until the ellipsoid becomes sufciently small. The process is illustrated in Fig. 11. The kth iteration consists of the following steps, starting from the fact that the center xk of the ellipsoid E k is known: Step 1. In case the center is not in the feasible region, the gradient of the violated constraint is evaluated; if it is feasible, the gradient of the objective function is found. In either case, we will refer to the computed gradient as h(xk). Step 2. A new ellipsoid containing the half ellipsoid given by Ek {x | h(x k ) x h(x k ) x k )
T T
minimize c Tx subject to F (x ) 0 where F (x ) R

mm
(SDP) , c, x R
n
is computed. This new ellipsoid, E k1, and its center xk1, are given by the following relations:
xk+1 = xk Ak + 1 =
where
1 A g n+1 k k
n2 2 g T A A g Ak n2 1 n+1 k k k k
h(x k ) k = g h(x k )T Ak h(x k ) Step 3. Repeat the iterations in steps 1 and 2 until the ellipsoid is sufciently small. RELATED TECHNIQUES Quasiconvex Optimization Denition. A function f : S R, where S is a convex set, is quasiconvex if every level set La xf (x) a is a convex set. A function g is quasiconcave if g is quasiconvex over S. A function is quasilinear if it is both quasiconvex and quasiconcave.
Here, F(x) F0 F1 x1 . . . Fn xn is an afne matrix function of x, and the constraint F(x) 0 represents the fact that this matrix function must be positive semidenite, i.e., zT F(x) z 0 for all z Rn. The constraint is referred to as a linear matrix inequality. The objective function is linear and hence convex, and the feasible region is convex, since if F(x) 0 and F(y) 0, then for all 0 1 it can be readily seen that F(x) (1 ) F(y) 0. A linear program is a simple case of a semidenite program. To see this, we can rewrite the constraint set A x b (note that the here is a componentwise inequality, and it is not related to positive semideniteness) as F(x) diag(A x b), i.e., F0 diag(b), Fj diag(aj), j 1, . . ., n, where A [a1 a2 . . . an] Rmn. Semidenite programs may also be used to represent nonlinear optimization problems. As an example, consider the problem
minimize
(c Tx )2 d Tx x+b 0 subject to Ax
where we assume that dT x 0 in the feasible region. Note that the constraints here represent, as in the case of a linear program, componentwise inequalities. The problem is rst re-
CONVEX OPTIMIZATION
307
written as minimizing an auxillary variable t subject to the original set of constraints and a new constraint (cTx )2 t d Tx The problem may be cast in the form of a semidenite program as
A simple illustration of this technique is in minimizing the outer surface area of an open box of xed volume (say, 4 units) and sides of length x1, x2, x3. The problem can be stated as minimize x1 x2 + 2x1 x3 + 2x1 x3 subject to x1 x2 x3 = 4 By setting u1 x1x2, u2 2x1x3, u3 2x1x3, and applying the condition listed above, the minimum value of the objective 2 2 2 function is 3(u1u2u3)1/3 3(4x1 x2x3) 12. It is easily veried that this corresponds to the values x1 2, x2 2, x3 1. We add a cautionary note that some, but not all, posynomial programming problems may be solved using this simple solution technique. For further details, the reader is referred to Ref. 18. Optimization Involving Logarithmic Concave Functions A function f is a logarithmic concave (log-concave) function if log f is a concave function. Log-convex functions are similarly dened. The maximization of a log-concave function over a convex set is therefore a unimodal problem, i.e., any local minimum is a global minimum. Log-concave functional forms are seen among some common probability distributions on Rn, for example: 1. The Gaussian or normal distribution f (x ) = where 0. 2. The exponential distribution
n Y i= 1
minimize t subject to
2 x + b) diag(Ax 6 0 4
0
0 t c Tx
3 0 7 c Tx 5 0
d Tx
The rst constraint row appears in a manner similar to the linear programming case. The second and third rows use Schur complements (16) to represent the nonlinear convex constraint above as the 2 2 matrix inequality t c Tx cT x 0 d Tx
The two tricks shown here, namely, the reformulation of linear inequations and the use of Schur complements, are often used to formulate optimization problems as semidenite programs. Geometric Programming Denition. A posynomial is a function h of a positive variable x Rm that has the form
1 (2 )n det
h(x ) =
X Y
n
e 0 . 5 (x x c )
1 (x x ) c
i, j ) x ( i
i= 1
where the exponents (i,j) R and the coefcients j 0, j R. For example, the function f (x,y,z) 7.4x 2.6y3.18z4.2 3 x2y1.4z5 is a posynomial in the variables x, y, and z. Roughly speaking, a posynomial is a function that is similar to a polynomial, except that the coefcients j must be positive, and an exponent (i,j) can be any real number, not necessarily a positive integer. A posynomial has the useful property (17) that it can be mapped onto a convex function through an elementary variable transformation, x(i) ez(i). Such functional forms are useful because in the case of an optimization problem where the objective function and the constraints are posynomial, the problem can easily be mapped onto a convex programming problem. For some geometric programming problems, simple techniques based on the use of the arithmeticgeometric inequality may be used to obtain simple closed-form solutions to the optimization problems (18). The arithmeticgeometric inequality states that if u1, u2, . . ., un 0, then their arithmetic mean is no smaller than their geometric mean, i.e., u1 + u2 + + un (u1 u2 . . . un )1 / n n with equality occurring if and only if u1 u2 . . . un.
!
(i ) e ((1 )x (1 )+ (2 )x (2 )++ (n )x (n ))
f (x ) =
The following properties are true of log-concave functions: 1. If f and g are log-concave, then their product fg is logconcave. 2. If f (x, y) is log-concave, then the integral S f (x, y) dx is log-concave provided S is a convex set. 3. If f (x) and g(x) are log-concave, then the convolution S f (x y) g(y) dy is log-concave if S is a convex set (this follows from properties 1 and 2).
ENGINEERING PROBLEMS AS CONVEX OPTIMIZATION PROBLEMS There has been an enormous amount of recent interest in applying convex optimization to engineering problems, particularly as the optimizers have grown more efcient. The reader is referred to Boyd and Vandenberghes lecture notes (19) for a treatment of this subject. In this section, we present a sampling of engineering problems that can be posed as convex programs to illustrate the power of the technique in practice.
308
CONVEX OPTIMIZATION
Design Centering While manufacturing any system, it is inevitable that process variations will cause design parameters, such as component values, to waver from their nominal values. As a result, after manufacture, the system may no longer meet some behavioral specications, such as requirements on the delay, gain, and bandwidth, that it has been designed to satisfy. The procedure of design centering attempts to select the nominal values of design parameters to ensure that the behavior of the system remains within specications with the greatest probability and thus maximize the manufacturing yield. The random variations in the values of the design parameters are modeled by a probability density function (x,xc) : Rn [0,1], with a mean corresponding to the nominal value of the design parameters. The yield of the manufacturing process, Y, as a function of the mean xc is given by Y (x c ) =
If the hks are entirely known, then the problem is a linear program. However, if they are not known exactly, given the uncertainty in the hks, then it may be possible to say that their values lie within the ellipsoid H = {h| h = h c + F p , p 1} where p (pTp)1/2, h Rk1, F R(k1)q. The robust version of the optimization problem above must ensure that the error is minimized over all possible values of h within the ellipsoid. To consider the worst-case tracking error, the optimization problem may be written as
minimize worst-case error = max subject to Ulow u(t ) Uhigh, |u(t + 1 ) u(t )| S,
h H t = 1 , 2 ,..., M
max | y(t ) yd (t )|
t = 1, 2, . . ., N t = 1, 2, . . ., N 1
Z
x F
x (x , x c ) dx
The value of p at which the worst-case error is maximized may be derived analytically (23), and the corresponding worst-case tracking error at that point is
t = 1 , 2 ,..., M
where F corresponds to the feasible region where all design constraints are satised. A common assumption made by geometrical design centering algorithms for integrated circuit applications is that F is a convex bounded body. Techniques for approximating this body by a polytope P have been presented in Ref. 20. When the probability density functions that represent variations in the design parameters are Gaussian in nature, the design centering problem can be posed as a convex programming problem. The design centering problem is formulated as (21) maximize Y (x c ) =
max [ F T Dt u + |h T Dt u yd (t )|]
The problem is therefore described as a specic form of convex programming problem, called a second-order cone programming (SOCP) problem as follows:
minimize subject to Ulow u(t ) Uhigh,

T T
t = 1, 2, . . ., N t = 1, 2, . . ., N 1
S u(t + 1 ) u(t ) S,
Z
x P
F Dt u + [h Dt u ydes (t )]
x (x , x c ) dx
F T Dt u [h T Dt u ydes (t )]
Optimizing Structural Dynamics Consider a linear elastic structure consisting of a stack of k linear elastic bars connecting a set of p nodes. The topology and lengths of the bars and their material are xed, and the appropriate cross-sectional widths of the bars are to be determined. The elastic stored energy of this system is given by f Td, where f is the vector of load forces and d is the vector of (small) node displacements. The relation between f and d is given by f A(x) d, where A(x), called the stiffness matrix, k is an afne sum of the variables xi, given by A(x) i1xi Ai with the matrices A being all symmetric positive semidenite. The optimization problem of minimizing the elastic stored energy (24) can then be stated as follows:
where P is the polytope approximation to the feasible region F . Since the integral of a log-concave function over a convex region is also a log-concave function (22), the yield function Y(x) is log-concave, and the above problem reduces to a problem of maximizing a log-concave function over a convex set. Hence, this can be transformed into a convex programming problem. Robust Optimal Control Consider a single-input single-output discrete-time linear dynamic system with a nite impulse response described by y(t ) = h0 u(t ) + h1 u(t 1 ) + + hk u(t k ) where u is the input sequence, y is the output sequence, and ht is the tth impulse response coefcient. Given a desired response yd(t), the problem is to nd an nite bounded input sequence u(t) for which the output y(t) most closely tracks the desired response, subject to constraints on the slew rate of the input signal. The problem may be stated as
minimize subject to
f Td
k
X
j=1
ljxj v
f = A(x )d x j ,min x j x j ,max for j = 1, 2, . . ., k
minimize error =
t = 1 , 2 ,..., M
max |y(t ) yd (t )| t = 1, 2, . . ., N t = 1, 2, . . ., N 1
subject to Ulow u(t ) Uhigh, |u(t + 1 ) u(t )| S,
Here v is the maximum volume, and zj the length of the jth bar. The last constraint places simple bounds on the values of the x variables (clearly, all of these variables must be positive, since they correspond to physical lengths). We can then
CONVEX OPTIMIZATION
309
rewrite the problem by eliminating the d variables by substitution, as follows:
minimize subject to
f T A(x ) 1 f
X
k j=1
ljxj v for j = 1, 2, . . ., k
x j ,min x j x j ,max
the dominant eigenvalue of a matrix G1C, where G and C are, respectively, matrices representing the conductances (corresponding to the resistances) and the capacitances referred to above. The entries in both G and C are afne functions of the xis. The dominant time constant can be calculated as the negative inverse of the largest zero of the polynomial det(sC G). It is also possible to calculate it using the following linear matrix inequality: T dom = min{T |TG C 0} The 0 here refers to the fact that the matrix must be positive denite. To ensure that Tdom Tmax for a specied value of Tmax, the linear matrix inequality TmaxG(x) C(x) 0 must be satised. This sets up the problem in the form of a semidenite program as follows (28):
Using Schur complements, this leads to the semidenite programming formulation in x and t given by
minimize t subject to t f
k X j=1
fT 0 A (x ) ljxj v for j = 1, 2, . . ., k
minimize
X
n i= 1
l i xi
x j ,min x j x j ,max
VLSI Transistor and Wire Sizing
subject to Tmax G(x ) C(x ) 0 x min x x max
Convex Optimization Formulation. Circuit delays in integrated circuits often have to be reduced to obtain faster response times. Given the circuit topology, the delay of a combinational circuit can be controlled by varying the sizes of transistors, giving rise to an optimization problem of nding the appropriate areadelay tradeoff. The formal statement of the problem is as follows: minimize area subject to delay Tspec (TS)
Largest Inscribed Ellipsoid in a Polytope Consider a polytope in Rn given by P xaiT x bi, i 1, 2, . . ., L into which the largest ellipsoid E , described as follows, is to be inscribed: y + d | y 1}, E = {By B = BT > 0
The circuit area is estimated as the sum of transistor sizes, i.e.,
The center of this ellipsoid is d, and its volume is proportional to det B. The objective here is to nd the entries in the matrix B and the vector d. To ensure that the ellipsoid is contained within the polytope, it must be ensured that for all y such that y 1, y + d ) bi aT i (By
T Therefore, it must be true that supy1 (aT i By ai d) bi, or in other words, B ai bi aT d . The optimization problem i may now be set up as
area =
X
n i= 1
xi
where xi is the size of the ith transistor and n is the number of transistors in the circuit. This is easily seen to be a posynomial function of the xis. The circuit delay is estimated using the Elmore delay estimate (25), which calculates the delay as a maximum of path delays. Each path delay is a sum of resistancecapacitance products. Each resistance term is of the form a / xi, and each capacitance term is of the type bi xi, with the constants a and bi being positive. As a result, the delays are posynomial functions of the xis, and the feasible region for the optimization problem is an intersection of constraints of the form (posynomial function in xi s ) Tspec Since the objective and constraints are both posynomial functions in the xis, the problem is equivalent to a convex programming problem. Various solutions to the problem have been proposed, for instance, in Refs. 26 and 27. Semidenite Programming Formulation. In the problem (TS) above, the circuit delay may alternatively be determined from
maximize
log det B a i bi a T Ba i d, i = 1, 2, . . ., L
subject to B = BT > 0
This is a convex optimization problem (6) in the variables B and d, with a total dimension of n(n 1)/2 variables corresponding to the entries in B and n variables corresponding to those in d. Beamforming Antenna arrays are often used to detect and process signals arriving from different directions. Each sensor is associated with a parameter called its weight, typically a complex number, and the values of these weights determine the beam pattern. For a planar array with N elements, the beam pattern is given by the expression
G( )
X
N i= 1
wi gi ( ) exp j
2 (x cos + yi sin ) i
310
CONVEX OPTIMIZATION
The problem of optimal beamforming is to choose the weights in such a way that the pattern level in given areas i are minimized, subject to upper bound constraints j for other angles and a maximum level on the weights. The optimization problem is formally written as (29)
BIBLIOGRAPHY
1. W. Stadler, Multicriteria Optimization in Engineering and in the Sciences, New York: Plenum, 1988. 2. N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica, 4: 373395, 1984. 3. A. V. Fiacco and G. P. McCormick, Nonlinear Programming, New York: Wiley, 1968. 4. J. Renegar, A polynomial time algorithm, based on Newtons method, for linear programming, Math. Programming, 40: 59 93, 1988. 5. C. C. Gonzaga, An algorithm for solving linear programming problems in O(nL) operations, in Progress in Mathematical Programming: Interior Point and Related Methods, N. Meggido (ed.), New York: Springer-Verlag, 1988, pp. 128.
minimize |G(i )| subject to |G(i )| Ui |wk | W G(0 ) = 1

This may be rewritten as
for j = 1, 2, . . ., M
for k = 1, 2, . . ., N
minimize t subject to |G(i )| t , |G(i )| U j |wk | W G(0 ) = 1

The last constraint here is a normalization constraint and can be handled by decreasing the number of variables by one. Recalling that each wj is complex, we may choose the vector x as x = [Re w1 , Im w1 , Re w2 , Im w2 , . . ., Re wN 1 ), Im wN 1 , f ] and rewrite the problem in the corresponding convex form: subject to Ai x + b i < c T i x + di, i = 1, 2, . . ., L + M + N
i = 1, 2, . . ., L for j = 1, 2, . . ., M for k = 1, 2, . . ., N
6. Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM Studies in Applied Mathematics, Philadelphia: Society for Industrial and Applied Mathematics, 1994. 7. P. M. Vaidya, An algorithm for linear programming which requires O(((m n)n2 (m n)1.5n)L) arithmetic operations, Math. Programming, 47: 175201, 1990. 8. Y. Ye, An O(n3L) potential reduction algorithm for linear programming, Math. Programming, 50: 239258, 1991. 9. R. T. Rockafellar, Convex Analysis, Princeton, NJ: Princeton University Press, 1970. 10. D. G. Luenberger, Linear and Nonlinear Programming, Reading, MA: Addison-Wesley, 1984. 11. P. E. Gill, W. Murray, and M. H. Wright, Numerical Linear Algebra and Optimization, Reading, MA: Addison-Wesley, 1991. 12. A. J. Schrijver, Theory of Linear and Integer Programming, New York: Wiley, 1986. 13. D. den Hertog, Interior Point Approach to Linear, Quadratic and Convex Programming, Boston: Kluwer Academic, 1994. 14. J. A. dos Santos Gromicho, Quasiconvex Optimization and Location Theory, Amsterdam: Thesis Publishers, 1995. 15. L. Vandenberghe and S. Boyd, Semidenite programming, SIAM Rev., 38 (1): 4995, 1996. 16. G. H. Golub and C. F. Van Loan, Matrix Computations, 2nd ed., Baltimore, MD: The Johns Hopkins University Press, 1989. 17. J. G. Ecker, Geometric programming: methods, computations and applications, SIAM Rev., 22 (3): 338362, 1980. 18. R. J. Dufn and E. L. Peterson, Geometric Programming: Theory and Application, New York: Wiley, 1967. 19. S. Boyd and L. Vandenberghe, Introduction to Convex Optimization with Engineering Applications, Lecture Notes, Electrical Engineering Department, Stanford University, 1995. Available from http://www-isl.stanford.edu/ boyd. 20. S. W. Director and G. D. Hachtel, The simplicial approximation approach to design centering, IEEE Trans. Circuits Syst., CAS24: 363372, 1977. 21. S. S. Sapatnekar, P. M. Vaidya, and S. M. Kang, Convexity-based algorithms for design centering, IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., 13: 15361549, 1994. 22. A. Prekopa, Logarithmic concave measures and other topics, in Stochastic Programming, M. Dempster (ed.), London: Academic Press, 1980, pp. 6382. 23. S. Boyd, C. Crusius, and A. Hansson, Control applications of nonlinear convex programming, Electrical Engineering Department, Stanford University, 1997; J. Process Control, in press. 24. A. Ben-Tal and M. P. Bendsoe, A new method for optimal truss topology design, SIAM J. Optimiz., 3: 322358, 1993.
Here, the amplitude of G(i) is represented as the norm of a vector Aix bi where x R2N1, Ai R2(2N1), bi R2. The two components of Ai are the real and imaginary parts of G(i). The vectors ci and di are dened as follows:
c i = {[0, 0, . . ., 0, 1] [0, 0, 0, . . ., 0] d i = {[0, 0, . . ., 0, 0] Ui L W
for i = 1, 2, . . ., L for i = L + 1, . . ., L + M + N for i = 1, 2, . . . L
for i = L + 1, . . ., M for i = L + M + 1, . . ., L + M + N
Note that the objective here is linear, and the constraints quadratic. CONCLUSION This overview has presented an outline of convex programming. The use of specialized techniques that exploit the convexity properties of the problem have led to rapid recent advances in efcient solution techniques for convex programs, which have been outlined here. The applications of convex optimization to real problems of engineering design have been illustrated. Convex optimization techniques are used widely in control, for example, in Youla-based design and in design by linear matrix inequalities (LMIs). For Youla-based design, the reader is referred to Refs. 30 and 31. A good sourcebook for design by LMIs is Ref. 32 and a useful practical design tool is the LMI Control Toolbox (33).
CONVOLUTION 25. S. S. Sapatnekar and S. M. Kang, Design Automation for TimingDriven Layout Synthesis, Boston: Kluwer Academic, 1993. 26. J. Fishburn and A. E. Dunlop, TILOS: A posynomial programming approach to transistor sizing, in Proc. IEEE Int. Conf. Comput.-Aided Des., 1985, pp. 326328. 27. S. S. Sapatnekar et al., An exact solution to the transistor sizing problem using convex optimization, IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., 12: 16211632, 1993. 28. L. Vandenberghe, S. Boyd, and A. El Gamal, Optimal wire and transistor sizing for circuits with non-tree topology, in Proc. IEEE Int. Conf. Comput.-Aided Des., 1997, pp. 252259. 29. H. Lebret, Optimal beamforming via interior point methods, J. VLSI Signal Process., 14 (1): 2941, 1996. 30. S. P. Boyd and C. H. Baratt, Linear Controller Design: Limits of Performance, Englewood Cliffs, NJ: Prentice-Hall, 1991. 31. M. A. Dahleh and I. J. Diaz-Bobillo, Control of Uncertain Systems: A Linear Programming Approach, Englewood Cliffs, NJ: PrenticeHall, 1995. 32. S. P. Boyd et al., Linear Matrix Inequalities in System and Control Theory, SIAM Studies in Applied Mathematics, Philadelphia, PA: SIAM, 1994. 33. P. Gahinet et al., LMI Control Toolbox, Natick, MA: The MathWorks, 1995.
311
SACHIN S. SAPATNEKAR
University of Minnesota
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Delay Systems Standard Article R. Datko1 and J. K. Hale2 1Georgetown University, Washington, DC 2Georgia Institute of Technology, Atlanta, GA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1009 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (173K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Properties of Delay Systems Analytic Properties of Delay Systems Extensions to Infinite Dimensional Phase Spaces Time Delays in Control Systems About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
DELAY SYSTEMS
139
DELAY SYSTEMS
In most applications of mathematics to engineering it is tacitly assumed that the systems under consideration are causal. That is, the future state of the system depends only on its present state. In reality most electrical systems, particularly control systems, are subject to transportation and/or processing delays. Usually these delays are ignored, either because they are considered small or because they complicate the mathematical model. Thus a dilemma arises.
140
DELAY SYSTEMS
When does the realistic modeling of a physical system require the introduction of time delays into the mathematical model? One purpose of this article is to introduce the reader to the fundamental properties of time delay differential equations and to compare these to the corresponding ones for ordinary differential equations. The other is to sum up the structure and fundamental properties of delay systems of the type most frequently encountered in electrical engineering. Ordinary differential equations (ODE) in which a part of the past history affects the present state are called delay differential equations or functional differential equations (FDE). Some examples are
0
x (t ) = ax(t ) + bx(t 1 ) +
x(t + ) d
0 1
(1) (2)
d (x(t ) dx(t 1 )) = ax(t ) + bx(t 1 ) + c dt
x(t + ) d
tems using only the nite-dimensional approach. The monograph by J. K. Hale and S. M. Verduyn-Lunel (5) develops both the coarse and ne structure of delay systems using the powerful tools of innite-dimensional analysis together with the more mundane methods of linear algebra and complex analysis. There are two main categories of FDE considered in the engineering literature, retarded functional differential equations (RFDE) and neutral functional differential equations (NFDE). A delay system, written as a rst order system, is a RFDE if the derivative contains no delay terms, Eq. (1) is a RFDE. If the derivative contains delay terms in a rst-order system, the equation is called a neutral functional differential equation. Eq. (2) for d 0 is a NFDE. In engineering practice, only a certain class of NFDE is considered; namely, D-stable NFDE. A D-stable NFDE is one in which the difference equation associated with the derivative is uniformly exponentially stable (u.e.s.) Eq. (2) is a D-stable NFDE if d 1 since the respective difference equation, y(t ) dy(t 1 ) = 0 (3)
Although special examples of delay differential equations were investigated as early as the eighteenth century by Euler and Lagrange, their systematic development did not occur until this century. The initial impetus was the study of certain mathematical models in mechanics and the physical sciences which incorporated time delays in their dynamics. One of the most interesting control models was constructed by N. Minorsky in 1942 (1) in which he incorporated the fact that the automatic steering mechanism of a ship was subject to a time delay between a course deviation and the turning angle of the rudder. Perhaps the two most important contributors to the initial mathematical development of delay systems were A. D. Mishkis (2) and N. N. Krasovskii (3). Myshkis gave the rst systematic treatment of the existence and uniqueness problems for delay systems, and Krasovskii not only extended the second method of Lyapunov for stability to delay systems but also showed that the correct mathematical setting for linear time invariant delay differential equations was an innitedimensional space and not the nite-dimensional space where the system was dened. This is a crucial observation because a delay differential equation may be treated both as a process in a nite-dimensional space and one in an innite-dimensional space. Some properties of delay systems do not depend on their innite-dimensional character. For other properties, this is prerequisite, and yet other properties, whose computational nature is nite-dimensional, can only be established by considering the innite-dimensional system. An example of the rst situation is given by the representation of solutions of linear time invariant equations. These are obtained, as in the case of ODE, by using a combination of linear algebra and complex analysis. An innite-dimensional property is the notion of a solution to the initial value problem. A property which is innite-dimensional in nature, but sometimes computationally nite-dimensional, is the stability behavior of the homogeneous time independent systems considered in this article. The stability of these systems is determined by the zeros of an entire analytic function, as in the ODE case. However, the justication for this is based on the innite-dimensional nature of these systems. This nite-innite-dimensional duality is of critical importance in studying delay systems. The monograph by R. Bellman and K. Cooke (4) develops many fundamental stability properties of delay sys-
is u.e.s. If d 1, there is an unbounded solution of Eq. (3), and so the equation is not stable. If d 1, then Eq. (3) is stable, but not u.e.s. The denition of D-stability is due to M. A. Cruz and J. K. Hale (6). They showed that the stability properties of linear time invariant (LTI) RFDE and LTI Dstable NFDE are determined by the exponential solutions, just as in the case of LTI ODE. D. Henry (7) proved that a LTI NFDE cannot be uniformly exponentially stable unless it is D-stable. Until the papers of Cruz, Hale and Henry, engineers routinely gave conditions for the stability of neutral systems without realizing that D-stability was an essential requirement. PROPERTIES OF DELAY SYSTEMS The study of delay systems is a natural extension of the theory of ODE, and we shall describe the basic properties of these systems by comparing them with the analogous properties of ODE. Let Rn be Euclidean n-space. Initial Conditions and Solutions The initial condition for the solution of an ODE is given in the nite dimensional phase space Rn. The initial condition for the solution of an FDE or delay equation is an innitedimension space, called a Banach space. The reason for this is that the initial value of an FDE is a vector-valued function on Rn dened over an interval [h, 0], h 0 representing the delay. The point values of the solution evolve in Rn, which also is called the phase space of the system. This is the dimension duality property of an FDE. The space of initial conditions is innite-dimensional, and the phase space is nitedimensional. The notion of the solution of an FDE is weaker than for an ODE. This is because not all initial functions result in differentiable solutions. However for LTI FDE the Laplace transform provides a convenient alternative denition. The formal Laplace transform of an LTI FDE does not explicitly contain the derivative of the FDE, only its initial value. This is also true for ODE. Thus, if the formal Laplace transform of an
DELAY SYSTEMS
141
FDE is actually the Laplace transform of a vector-valued function in Rn, we call this function a solution to the equation. It is shown in Ref. 7 that this is indeed the case for LTI FDE. In the case where the FDE is nonlinear or linear with time dependent coefcients, the denition of a solution is more complex. In the applications to control problems in electrical engineering, LTI FDE occur most frequently. Solutions of nonhomogeneous LTI FDE have a variation of parameters representation. This is given by the convolution of an n n matrix-valued function, S(t), and the forcing function. The matrix function, S(t), is the inverse Laplace transform of the matrix version of the homogeneous system where the initial value is the zero matrix when t 0 and the identity matrix at t 0. This matrix-valued function is the analogue of the fundamental matrix of an LTI ODE and as a consequence is called the fundamental matrix of the system. However, it is not a matrix exponential function as it would be in the ODE case. Time varying nonhomogeneous linear FDE also have their solutions represented by fundamental matrices. However their computation is more difcult. One numerical procedure used to compute these matrices is known as the method of steps (4). This method works for the systems given by equations (1) and (2) if c 0 because the time delays are discrete. The method consists of nding the fundamental matrix of the system over the interval [0, h], then using this information to compute the fundamental matrix over the interval [h, 2h], etc. However, in many control problems, the indeterminate time behavior of the system is desired, and the method of steps is unsuitable for this purpose. Stability In the applications of delay systems to electrical engineering, which is practically synonymous with control theory, linear systems are used almost exclusively, and of these LTI systems predominate. For LTI systems, the Lyapunov stability theory is supercially the same as in the ODE case. However, in place of symmetric matrices, one uses symmetric functionals called Liapunov-Krasovskii functionals. The mathematical structure of these functionals was described by Yu. M. Repin (8) and N. N. Krasovskii (3). Their practical application to stability is very limited for two reasons. They are difcult to construct, even for very simple systems, and once a functional has been constructed, it is not easy to determine positive or negative deniteness. Their use in stability theory is usually on an ad hoc basis. That is, one guesses the form of the functional, applies it to the system, and hopes it will have the required positivity or negativity when applied to differentiable trajectories of the system. There is also an offshoot of the Lyapunov theory, called the method of Razumikhin (9). This method uses Lyapunov functions in place of functionals to determine stability. Its application requires that the FDE satisfy certain side conditions which are not always met in practice. However, when applicable, the method of Razumikhin is preferable to the standard Lyapunov method. On the other hand, unlike the Lyapunov method, the Razumikhin method does not have converse theorems on stability. The application of Lyapunov functionals or Razumikhin functions to LTI FDE is a means to the end of locating the eigenvalues of the system. The eigenvalues of an LTI FDE are the zeros of an associated entire analytic function and
determine, as in the ODE case, the qualitative behavior of the system. For LTI RFDE or LTI D-stable NFDE, if these zeros lie in the open left half of the complex plane, the system is u.e.s. Determining this condition or its absence is much more difcult than for LTI ODE, since no simple criterion is available, such as the Routh-Hurwitz criterion. There is a bounded input-bounded output (BIBO) criteria for linear homogeneous FDE which applies to both time invariant and time dependent systems, provided the coefcients of the time dependent system are uniformly bounded on the real line. This is known as the Perron condition (10). This condition states that if the nonhomogeneous version of such a system with zero initial conditions has bounded solutions for all bounded forcing terms, the system is u.e.s. Control and Stabilization There are several versions of the Pontryagin Maximum Principle for delay systems, and the most popular method of solving optimal control problems for delay systems is Bellmans Method of Dynamic Programming (11,12). However, even for ODE systems, these theories are rarely used in applications to electrical engineering. Their main drawback is that they are nearly impossible to apply to practical, nonlinear delay systems. Moreover, most electrical engineering control problems are design problems, not optimal control problems. The two major exceptions to this statement are Linear Quadratic Regulator (LQR) problems and H-optimization problems. The solutions of these problems result in linear feedback controls which stabilize a system. However, neither method requires the Pontryagin Maximum Principle or Dynamic Programming. The main emphasis in engineering control is stabilization, particularly for linear systems. One popular method of stabilization for single input-single output systems with one time delay is the use of a Smith Predictor. This is a stabilization procedure which uses a proportionalintegralderivative (PID) feedback. This is a standard method and is discussed in ADAPTIVE CONTROL and CLASSICAL DESIGN METHODS FOR CONTINUOUS TIME SYSTEMS. In this article, we concentrate on the other two major methods of feedback stabilization for delay system, LQR- and H-optimization. The LQR method attempts to minimize an integral called the cost over the positive real axis. The integrand is quadratic and positive semidenite in the space variable and quadratic and positive denite in the control variable. If this optimization is possible for all initial values of the systems, the optimal control is a feedback control which stabilizes the system. H-optimization optimizes an LTI FDE control problem with an unknown disturbance. Here, the cost is positive and quadratic in the space and control variables as in the LQR problem but quadratic and negative denite in the disturbance variable. For a xed admissible control function u, one attempts to maximize the cost in terms of the disturbance, then to minimize the resulting functional with respect to u. This is called a min-max problem. The optimal solution, if it exists, leads to a feedback control which is u.e.s. in the so-called worst case disturbance. For ODE, the LQR problem is solved using routine numerical packages. The numerical treatment of H-optimization for ODE is much more difcult than LQR-optimization, but more robust with respect to uncertainties in the system dynamics.
142
DELAY SYSTEMS
Theoretically, both methods may be used for LTI FDE and the properties of the resulting feedback controls are known. However, the practical implementation of either method is a project for future investigation (13).
(), the In terms of the operator-valued complex matrix S formal Laplace transform of Eqs. (4) and (5) is
() (0 ) x (, ) = S () +S
0 r
D j (h j )
j=1
ANALYTIC PROPERTIES OF DELAY SYSTEMS We introduce some additional notation which will be used in the remainder of this article. Let R (, ), R [0, ), Z be the complex plane, Zn be complex n-space, I be the ndimensional identity matrix, cT be the transpose of an n-column vector c, c be the complex conjugate of an n-column vector, BT be the transpose of an n m matrix B, c cTc be the length of an n-vector c, and det(A) be the determinant of a square matrix A. The set of all continuous n-vector functions from a closed interval [h, 0] into Zn is denoted by C(h). If is in C(h), then sup(t):h t 0. If x(t) is an n-vector function dened on [h, ), then, for t 0, xt x(t ):h 0, and xt() x(t ), h 0. If x(t) is a vector or matrix function on Zn, then L (x(t ))() = x () =
0
h j j=1
()Bu (A j + D j )e (+ hj) ( ) d + S () (7)
For any in C(h), the inverse Laplace transform exists and is called a solution to the initial-value problem in Eqs. (4) and (5) (14). If C(h) has a derivative, then the inverse Laplace transform of Eq. (7) is
r
x(t , , u ) = S(t ) (0 )
j=1 r 0 h j
D j (h j )
+
j=1 t
)) d (8) S(t h j )(A j ( ) + D j (
+
0
S(t )Bu( ) dv
x(t )e t dt
where ())(t ) S(t ) = L 1 (S (9)
is the Laplace transform of x(t). The inverse Laplace transform of any vector or matrix valued function, provided it ex ())(t). ists, is denoted by L 1(S The delay systems most commonly encountered in electrical engineering are LTI systems with discrete time delays. The fundamental properties of these systems serve as a paradigm for most other systems one encounters. These systems which include RFDE and NFDE often have their dynamics described by
The function S(t) formally satises Eq. (4) with the initial matrix S(0) I, S() 0 for 0 and is referred to as the fundamental matrix solution of Eq. (4). If the matrices in Eq. (4) are time varying, there is a representation of the solution similar to Eq. (8), but it is not obtained by using the Laplace transform. The matrix S(t) is replaced by a matrix S(t, ), where S(, ) I, and S(t, ) 0 if t. The matrix function S(t, ) is formally a matrix solution of Eq. (4) for t (5). Stability
d x(t ) dt
D j x(t h j )
j=1 r
= A0 x(t ) +
j=1
A j x(t h j ) + Bu(t ),
t0
(4)
The difference equation associated with the NFDE in Eq. (8) is

r
x(t )
and initial values x(t ) = (t ), h t 0, C (h ) (5)
j=1
D j x(t h j ) = 0
(10)
The condition for Eq. (4) to be D-stable is that the equation

r
In Eq. (4), the matrices Dj, Aj, and A0 are n n-matrices with real entries; the matrix B is an n m-matrix with real entries and 0 hj h, 1 j r. The m-vector u(t) is called the control. A solution x(t, , u) of Eq. (4), Eq. (5) is a function which satises Eq. (5), is continuously differentiable for t 0, and has a right hand derivative at t 0 which satises Eq. (4). We can use the Laplace transform to obtain the existence of and a specic representation for a solution. Let
det I
j=1
D j e h j
=0
(11)
() = I S
!
Dje
h j
A0
j=1
Aje
h j
(6)
j=1
has all of its solutions in Re for some 0. If we seek solutions of Eq. (10) of the form et for some nonzero complex n-vector , then must be an eigenvalue of the matrix in Eq. (11) and must be a corresponding eigenvector. For this reason, we say that satisfying Eq. (11) are eigenvalues of Eq. (10). We remark that if Eq. (10) is D-stable at one collection of the delays hj, 1 j r, then it is D-stable for all other values of the delays (5). The stability behavior of the homogeneous version of (4)(5), (i.e., when u(t) 0) is completely determined by the
DELAY SYSTEMS
143
eigenvalues of the system. For the same reason as indicated for Eq. (10), these are the zeros of the entire analytic function
r
If we consider only those solutions x(t) that satisfy the relation x(t) x(t r), then dV (x(t )) (a |b| )(x(t )) 0 dt Thus, by Theorem 4.2, Chapter 5, in Ref. 5, the system is u.e.s. for all r 0. In this case, the Razumikhin method yielded in a much more straightforward way the same result as above using the LyapunovKrasovskii functional. There is yet another way to determine u.e.s. of LTI FDE. This is to treat the delay terms as parameters in a family of LTI FDE, which reduce to an ODE when all the delays are zero. If the ODE is u.e.s., one tries to estimate the size of the delays that the family can tolerate and yet remain u.e.s. This is possible since for LTI RFDE or LTI D-stable NFDE, the maximal exponential rate of expansion or contraction of a system depends continuously on the delay parameters (16). This condition is an easy consequence of the Hurwitz Theorem in Complex Analysis (17) since the maximal rate is determined by an eigenvalue for which the real part can be chosen to be continuous in the delay parameters. To illustrate this method, consider Eq. (15). When r 0, the system is u.e.s. We try to nd the smallest positive value r for which the system has a nontrivial periodic solution of period 2 / w. This value, if it exists, satises the equation iw + a + b (cos i sin ) = 0, = wr, w > 0
det I
j=1
D j e h j
A0
j=1
A j e h j
=0
(12)
If the system is D-stable, then it is u.e.s. if and only if all solutions of Eq. (12) satisfy Re 0 (5). As mentioned above, before the papers of Cruz and Hale (6) and Henry (7), engineers often assumed a system was u.e.s. if solutions of Eq. (12) satised Re 0. If the D-stability condition is not satised, showing that the solutions of Eq. (12) satisfy Re 0 may be insufcient to determine stability or instability as the following two examples show (15). Consider the scalar systems d [x(t ) x(t 1 )] = x(t ) dt (13)
d d2 [x(t ) 2x(t 1 ) + x(t 2 )] + 2 [x(t ) x(t 1 )] + x(t ) = 0 dt 2 dt (14)

The eigenvalues of both systems are solutions of the equation (1 e) 1 0 which has all its solutions in Re 0. Equation (13) has all its solutions tending to zero as t tends to innity, but it is not u.e.s. Equation (14) has solutions which tend to innity as t tends to innity. The stability or instability of LTI FDE can be determined by LyapunovKrasovskii functions and sometimes by Razumikhin functions. However as was mentioned above, these are difcult to nd for all but the simplest systems and even then are usually selected on an ad hoc basis. To illustrate this, consider the scalar system x (t ) = ax(t ) + bx(t r ), a > 0, a > |b| > 0, r > 0 (15)
Since a b, for xed w, this equation has no solution for real . Thus, we conclude that the system is u.e.s. for all values of r. There is an important class of nonlinear FDE whose stability is determined by frequency domain methods. A typical example of such a system is one whose dynamics are described by the equations x (t ) = A0 x(t ) + A1 x(t h ) + b f ( ), = cT x (17)
For a constant c to be determined, choose the Lyapunov Krasovskii functional 1 V () = ((0 ))2 + c 2
0 r
where b is an n-vector, and f is a scalar function satisfying the sector condition a1 2 f ( ) a2 2 , 0 < a1 < a2 < a (18)
(( )) d
2
(16) The following theorem holds (18) Theorem. Assume that the system y (t ) = A0 y(t ) + A1 x(t h ) is u.e.s. and let (19)
Along differentiable trajectories of Eq. (15) dV (xt ) = (a + c )(x(t )2 + bx(t r )x(t ) c(x(t r ))2 dt This functional will be negative on C(r) if we choose c a /2. Therefore, by Theorem 2.1, Chapter 5, in Ref. 5, the region of u.e.s. contains the set of coefcients a, b with a b. Notice that this simple choice for the LyapunovKrasovskii functional yielded the stability region which is completely independent of the size of the delay. Now, consider the Razumikhin function for Eq. (15) given by V(x(t)) x2 /2. Along differentiable trajectories of Eq. (15), dV (x(t )) = x(t )(ax(t ) + bx(t r )) dt
K (iw ) = cT (iwI A0 A1 eiwh ) 1 b If there exists a q 0 such that for all w in R, Re(1 + iwq )K (iw ) 1 0 a
(20)
(21)
then each solution of Eq. (17) tends to zero as t tends to innity. The above theorem is also true for D-stable systems of the types in Eqs. (4) and (5) where bf () replaces Bu. This theo-
144
DELAY SYSTEMS
rem is one of a class of theorems which are known collectively as Popov-type theorems. An interesting corollary to the above theorem is that condition [Eq. (21)] guarantees the existence of a Lyapunov functional for the system [Eq. (17)] which has a particular structure. Corollary. If Eq. (20) is satised, then there exists a Lyapunov functional on C(h) of the type V (, ) = Q(, ) +
0
given terminal point in a nite time. There is also the notion of -controllability, that is, control from a given point to an ball of another given point. This latter form of control is more realistic for FDE systems, but in practice, neither form is much used in engineering design. A simple example may indicate the reason. Consider the scalar system x = ax(t ) + bx(t h ) + u(t ) (26)
f (s ) ds,
>0
(22)
such that Q is bilinear on C(h), Q(, ) 0 if 0 and, along differentiable trajectories of Eq. (17), dV(xt, xt)/ dt 0. The proof the above corollary is a simple extension of the same result for ODE given on p. 169 of Ref. 10. The converse is also true; that is, if the corollary is satised, then so is the theorem. The Lyapunov approach is in general not feasible, whereas the frequency domain or Popov approach is easily checked, especially by modern computing packages. As was mentioned above, there is a necessary and sufcient condition for determining the u.e.s. of time varying FDE. This is the Perron condition or bounded-input, boundedoutput criterion. We give an analytic form of this condition for the system whose dynamic is described by the equation x (t ) = A0 (t )x(t ) + A1 (t )x(t h ) + f (t ) (23)
where h 0 and b 0. For C(h) and 0, suppose that one desires to nd a u(t) with u(t) 1 which drives the solution of Eq. (26) with initial value to the zero function in some nite time T. The Laplace transform of the resulting motion is given by
x (, , u ) =
1 a be h
(0 ) +
be ( + h )( ) d + u () (27)
but remark that the basic condition holds for any linear FDE whose coefcient matrices are uniformly bounded. We assume that the square matrices A0(t) and A1(t) have all their entries uniformly bounded on R. It is known (5) that the solutions of Eq. (23) with initial conditions zero in C(h) may be represented in the form
t
x(t , t0 , f ) =
S(t , ) f ( ) d
t0
(24)
where S(t, ) 0 if t, S(, ) I and, for t , d (S(t , )) = A0 S(t , ) + A1 S(t h, ) dt (25)
Theorem. A necessary and sufcient condition for the homogeneous version of Eq. (23) to be u.e.s. is that for all f which are uniformly bounded on R, the vector function in Eq. (24) satises an inequality of the form x(t, t0, f ) Mf, where Mf is nite and depends only on f (10). Although the Perron condition or BIBO condition may be theoretically difcult to verify, a modied form often is used in control engineering. The linear system is subjected to periodic forcing functions at a variety of frequencies and with uniformly bounded gains. If the outputs are uniformly bounded over long time periods, the system is considered u.e.s. Control and Stabilization Controllability for an FDE system is function space control; that is, one seeks to control a given initial point in C(h) to a
Since both xt(, u) and u(t) are identically zero after time T, (, , u) and u () in Eq. (27) must be entire the functions x analytic functions (19). This means that u () must be chosen so that the numerator in Eq. (27) is zero when the denominator is zero. But the zeros of a beh are innite in number and can at best be approximated. There are several versions of the Pontryagin Maximum Principle for FDE control problems, and the theoretical method used to solve control problems is the Method of Dynamic Programming. From an engineering point of view, these are only of academic interest. Comprehensive reference sources for this area of control are Refs. 12 and 20. For multiple-input-multiple-output LTI ODE, there are three basic methods of feedback stabilization. These are pole placement, linear quadratic regulator (LQR)-optimization, and H-optimization pole placement. The latter has the simplest numerical structure but has less eclat than the other two methods. Pole placement methods are possible for LTI FDE. In practice the best one can hope for are constant gain feedbacks which guarantee a given decay rate. However, LQR-optimization and H-optimization have in theory been completely extended from LTI ODE to LTI FDE. There are at least two ways to look at these extensions. One way relates to the specic delay structure and minimizes the use of Banach space theory. The other embeds LTI delay systems into a general class of LTI innite-dimensional systems known as Pritchard-Salamon systems (P-S systems) and makes extensive use of the theory of Sobelev spaces (21). Here, we conne ourselves to the rst approach. A typical LQR-optimization problem is the following. For a given positive denite matrix W and a given C(h), choose the control function u(t) to minimize the functional F (u ) =
0
[xT (t )Wx(t ) + uT (t )u(t )] dt
(28)
subject to the constraint that x(t) is the solution with initial value of the equation x (t ) = A0 x(t ) + A1 x(t h ) + Bu(t ) (29)
If the minimum for all in C(h) is nite, then there is a bounded linear mapping K from C(h) into Zn such the optimal
DELAY SYSTEMS
145
u is given by a feedback u(t) BTKxt. Moreover, if q(t) Kxt, then

T q (t ) = Wx(t ) AT 0 q (t ) A1 q (t + h )
in (0, ) (21). The optimal solution is a linear feedback control, K, which maps C(h) into Zn. The optimal u and d satisfy u(t ) = BT Kxt , d (t ) = 1 T L Kxt 2 (39)
(30)
The eigenvalues of the feedback system x (t ) = A0 x(t ) + A1 x(t h ) BBT Kxt are the solutions with Re 0 of the equation (31)
If q(t) Kxt, then

T q (t ) = Wx(t ) AT 0 q (t ) A1 q (t + h )
(40)
I A0 A1 e h det W
BBT =0 T h I + AT 0 + A1 e
(32)
If 0 (dT()d())d , then the system x (t ) = A0 x(t ) + A1 x(t h ) + BBT Kxt + Ld (t ) (41)
There are variants of the optimization problem in Eq. (28). One is to optimize Eq. (28) where W is only positive semidenite. This condition requires some additional assumptions on the system [Eq. (29)], which are technical but do not alter the basic structure of the solution. The fundamental assumption needed to solve the LQR-problem is that system [Eq. (29)] can be stabilizable. This appears to be putting the cart before the horse, and in some sense, it is. For example, an LQR problem for the ODE system x Ax Bu, where A is an n n matrix, is solvable if rank [B, AB, , An1B] n. There is no such simple condition for LTI FDE. For instance, the n-dimensional system [Eq. (29)] for B b an n-vector is stabilizable if, for Re 0, rank[I A0 A1 e h , b] = n (33)
has all solution converging to the zero vector as t tends to innity. The system [Eq. (35)] with feedbacks [Eq. (39)] is u.e.s. There are many variants of the above problem. For example, one could attempt to optimize the functional C ( u, d ) =
0
xT (t h )x(t h ) + uT (t )u(t )
1 T d (t ) d (t ) dt 2
The basic structure of the optimizable problem is the same. The critical problem is to nd efcient numerical methods. Since this problem has not been adequately solved for LQRoptimization, it will be much more difcult for H-optimization. Fine Structure The solutions of the homogeneous version of Eqs. (4) and (5) generate a C0-semigroup on C(h). The domain of the innitesimal generator of this semigroup are those points in C(h) which are continuously differential and satisfy the condition
An example of H-optimization is to nd the min-max of the following system. Consider the functional
F ( u, d ) =
0
xT (t )Wx(t ) + uT (t )u(t )
1 T d (t )d (t ) dt 2 (34)
subject to the constraint x (t ) = A0 x(t ) + A1 x(t h ) + Ld (t ) + Bu(t ) (35)
0) (
r j=1
h j ) A0 (0 ) D j (
A j (h j ) = 0
j=1
(42)
where A0, A1, B, and W are the matrices in Eqs. (28) and (29), and L in an n r-matrix. It is assumed that the systems x (t ) = A0 x(t ) + A1 x(t h ) + Bu(t ) and x (t ) = A0 x(t ) + A1 x(t h ) + Ld (t ) are stabilizable. The object is to nd, if possible, the min max F (u, d )
u d
(36)
(37)
(38)
and to show that it is nonegative for all initial values in C(h) of the solution of Eq. (35). In this problem, the constant plays a critical role. There exist for which there is either no optimum or for which the optimum is negative for at least one in C(h). However, there is a smallest 0 0 for which Eq. (38) has a nonegative solution for all in C(h) and all
If all Dj 0, Eq. (42) is an RFDE. If the solutions of Eq. (11) lie in Re 0 for some 0, then the system is a Dstable NFDE. For RFDE, the spectra of the semigroup at t 1 is the origin plus eigenvalues e where is an eigenvalue which is a solution of the characteristic equation [Eq. (12)] with all Dj 0 (5). For D-stable NFDE, the spectra of the semigroup at t 1 has the essential spectrum with moduli 1 plus eigenvalues e, where is an eigenvalue which is a solution of the characteristic equation [Eq. (11)] (5). An LTI RFDE has only a nite number of its eigenvalues in any right half plane, whereas a D-stable LTI NFDE has a vertical strip in Re 0 which contains an innite number of its eigenvalues. The solutions of LTI RFDE become more differentiable as time increases. They pick up one derivative for each interval of length h. For example, even if in C(h) is not continuously differentiable over [h, 0], the solution xt() will be ktimes differentiable for t kh. LTI NFDE retain the smoothness of their initial conditions. For these reasons, LTI RFDE have been compared to LTI parabolic partial differential equations (PDE), and LTI D-stable NFDE have been compared to LTI hyperbolic PDE. However, the ne structures of
146
DELAY SYSTEMS
LTI RFDE and LTI parabolic PDE are dissimilar. For example, parabolic PDE generate analytic semigroups, and RFDE generate C0-semigroups which are compact for t h but are not analytic. One basic reason for this difference is that the eigenvalues of LTI RFDE do not belong to a sector in the complex plane. On the other hand, some LTI hyperbolic PDE and LTI D-stable NFDE have identical mathematical structures. In particular, the equations for the dynamics of transmission lines described by the telegraph equation can be transformed to D-stable NFDE (5). An example which illustrates the similarity between some hyperbolic PDE and D-stable NFDE, are the PDE system wtt = wxx 2awt a2 w = 0, w ( 0, t ) = 0, 0 < x < 1, t > 0 (43) (44)
(i) There is a nonnegative continuous function a(r) with a(r) as r such that, for all in C(h), a(|(0 )| ) V () (48)
(ii) There is a nonnegative continuous function b(r) such that lim sup
h0
1 [V (xh )() V ()] b (|(0 )| ) h
(49)
Then the solution xt 0 is stable and every solution of Eq. (47) is bounded. If b(r) is positive denite every solution of Eq. (47) tends to zero as t tends to innity. Similar results exist concerning stability and instability for autonomous and nonautonomous FDE. If the function V is continuously differentiable, then the computation of the left side of relation [Eq. (49)] may be performed on solutions with smooth initial functions. The corresponding differential inequality gives estimates on these smooth solutions. Since the initial data of these smooth solutions are dense in the space C(h), one obtains estimates on all solutions. In this sense, there is no essential difference in the method than for ODE. A complete description for the Lyapunov method is given in Ref. 5, Chapter 5. The stability of linear homogeneous periodic RFDE and linear homogeneous periodic D-stable NFDE can be determined by examining their solutions after any integer multiple of their period which is larger than the delay. This results in a bounded linear mapping, U, from C(h) into itself. The eigenvalues of U, called the characteristic multipliers of the system, determine the stability behavior of the system. If all of the multipliers lie inside the unit circle in the complex plane, then the system is u.e.s. If some are outside the unit circle, the system is unstable. If the multipliers lie inside or on the unit circle, the geometric multiplicity of those on the unit circle determines stability or instability of the system. Feedback Stabilization LTI FDE of the type given in Eqs. (4) and (5) are particular examples of PritchardSalamon control systems. Their acronym is PS system (21). They are the largest class of innite-dimensional control systems to which the theory of nite-dimensional LTI control theory can be most easily extended. The most diverse class of PS systems are those described by LTI FDE whose spaces of initial conditions are Hilbert spaces and whose solutions evolve in their space of initial conditions. LQR- and H-stabilization are in theory completely developed for PS systems, but in a very abstract setting (21). In the case of FDE, this setting requires the use of the innitesimal generator of the associated semigroup, which is an unbounded linear operator. This operator is not as easy to manipulate as the Laplace transform of the solution of the system and is the main reason why LQR- and H-stabilization is theoretically possible but computationally difcult for these systems. To illustrate the difculty, consider the following LQR problem for the nite-dimensional system x (t ) = Ax(t ) + Bu(t ) (50)
wx (1, t ) = Kwt (1, t )
where a 0 and K 0 are constant, and the D-stable NFDE
1 K 2a a d x(t ) e x(t 2 ) = [x(t ) + e 2a x(t 2 )] dt 1+K 1+k (45)

These systems have the same spectrum (22). Small Solutions Linear homogeneous FDE may have small solutions; that is, nontrivial solutions which decay faster than any exponential function. A characterization of the set of small solutions for D-stable LTI NFDE and LTI RFDE is contained in Ref. 5. A remarkable property of any D-stable LTI NFDE and LTI RFDE is that there is a 0 such that any small solution is identically zero in C(h) after time . An example of a system with small solutions is x = y(t 1 ), y = x(t ) (46)
All solutions of Eq. (46) whose initial functions satisfy x(0) 0 and y(t) 0 for 1 t 0 are small solutions which vanish in C(1) for t 1 [(5), p. 74]. Linear periodic systems also have small solutions, but these are not necessarily zero after a nite time (5). An example is the system x (t ) = [see e.g. (5) p. 250]. Stability As mentioned above, N. N. Krasovskii extended the Second Method of Lyapunov to FDE. An example of this extension is the following theorem (5). Theorem. Let f be a continuously differentiable mapping from C(h) into Rn with f (0) 0 and consider the RFDE x (t ) = f (xt ) (47) 1 + sin 2 t x(t 1 ) 2
Let V be a continuous mapping from C(h) into R which satises the following two conditions:
DELAY SYSTEMS
147
where the function to be optimized is given by Eq. (28). It is known that the optimal solutions have controls u(t) BTKx(t), where K is a unique positive denite n n matrix with the property that, for any n-vector, x0 in Zn, the analytic vector-valued function
I A W
BBT I + AT
x0 Kx0
(51)
ation of solutions, continuous dependence on data, and parameters, etc. for these systems are similar to the corresponding ones for delay systems in Rn. The major exception to this statement occurs for properties which depend on the compactness of closed bounded sets in Rn or Zn. These systems are often encountered in models in population ecology, genetic repression, control theory, climatology, coupled oscillators, age dependent populations, etc. (29,30). TIME DELAYS IN CONTROL SYSTEMS Time delays are sometimes desired in the design of control systems. For example in self-tuning control, one encounters systems of the form y(t ) + a1 y(t 1 )a2 y(t 2 ) + ana y(t na ) = b1 w(t 1 ) + b2 u(t 2 ) + + bub u(t nb )
in Z2n has no poles in the right half complex plane. Since the poles of the matrix function in Eq. (51) are symmetric with respect to the imaginary axis, the matrix K is uniquely determined once the solutions of det I A W BBT I + AT =0 (52)
are known. This method of nding the feedback is known as spectral factorization. If the matrices A, B, and W in Eqs. (52) and (50) are replaced by linear operators , , and W , where is unbounded, a PS LQR problem will symbolically be represented by an expression of the form in Eq. (51), and a spectral factorization exists for such a system (21). However, how does one in practice carry it out? This is the crux of the computational difculties for LQR- and H-optimization in PS systems. On the other hand, the LQR-optimization described by Eqs. (28) and (29) has the eigenvalues of the feedback system given by the solutions in Re 0 of Eq. (32). The methods used to obtain this result were system specic (23); that is, they depended on the explicit structure of the delay system and not its abstract representation, and in this instance, yielded more information. The same is true of the frequency domain criterion [Eq. (21)] used in the Popov problem described by Eq. (17). This problem has a PS setting (24). However, in this setting, one has to unravel the simple Popov criterion. Another instance of this is the Perron condition. This condition exists for the evolution of the solutions of the system in Eq. (23) in a Banach space setting, but in practice, one examines the output of a system in Rn when the forcing function is an n-vector not the output of an innite-dimensional vector. EXTENSIONS TO INFINITE DIMENSIONAL PHASE SPACES Delays may appear in PDE as well as ODE. For example, ut duxx + u(x, t 1 )(1u (x, t )) = 2 (53)
where the u-terms are the controls. These are known as DARMA (deterministic autoregressive and moving average) systems (31). They have their own methodology which is described in ADAPTIVE CONTROL in this encyclopedia. Our interest here is in systems where unpredicted delays appear in the controls, particularly feedback stabilized controls. If the control system is nite-dimensional of the type x (t ) = Ax(t ) + Bu(t ) and is stabilized by a feedback of the form u(t ) = Rx(t ) (56) (55)
then a small time delay in the control will not destroy the stabilization. Of course, the word small depends on the particular system. However, if system [Eq. (55)] is innite-dimensional, it may be unable to tolerate any delays, particularly if it is an abstract representation of a boundary stabilized hyperbolic PDE. The simplest example of such a system is given by wtt = wxx , w ( 0, t ) = 0, 0 < x < 1, t > 0 wx (1, t ) = u(t ) (57) (58)
If, in Eq. (58), the control is feedback and given by u(t ) = wt (1, t ) (59)
then all of the solutions of the resulting feedback system are identically zero after time t 2. However, if u(t ) = wt (1, t h ), h>0 (60)
(25), which is a nonlinear diffusion equation with a time delay. Extensions of time delay systems to PDE and abstract Banach spaces may be found in Refs. 2629. Time independent versions of these systems are often of the form x (t ) = Ax(t ) + f (xt ) (54)
then the system is unstableso much so that the following result holds (32). Theorem. Given any 0 there exists hn 0 as n and n in Z, Ren such that the system [Eqs. (57) and (58)] with the feedback u(t ) = wt (1, t hn ) (61)
where A generates a C0-semigroup in a Banach space, X, and f is a continuous mapping from the Banach C :[h, 0] X is continuous. The proofs of existence, uniqueness, continu-
148
DELAY SYSTEMS
has solutions w(x, t ) = e n t sinh nt (62)
Systems of the type [Eqs. (57) and (58)] are often approximated by nite-dimensional oscillatory systems of the form x + Ax = bu (63)
where A is a positive denite n n matrix, with eigenvalues 2 2 n , and b is an n-vector, which is not an eigen0 1 vector of A. Suppose the system [Eq. (63)] is stabilized by a feedback
T u(t ) = cT (t ) 1 x(t ) + c2 x
(64)
This could be accomplished by pole placement, LQR-optimization, or H-optimization. Theorem. The maximum time delay which the system
T x (t ) + Ax(t ) = (cT (t h )) b 1 x(t h ) + c2 x
(65)
can tolerate and remain u.e.s. is in the interval 0<h< 2 n (66)
This result implies that large-dimensional Galerkin approximations to systems of the type in Eqs. (57) and (58) become unstable for small time delays. Thus, there is a tradeoff between the dimension of the approximation and the tolerance for delays. The above example illustrates one of the important differences between LTI evolutionary systems whose initial space is innite dimensional and those for which this space is nite dimensional. It is instructive to make some general remarks about why such a situation might occur. Suppose that is a parameter varying in a subset S of a Banach space, and T(t), t 0 is a C0-semigroup of linear transformations on a Banach space X which is continuous in ; that is, T(t)x is continuous in (, t, x). Let r be the radius of the spectrum of T(1). The asymptotic behavior of the semigroup is determined by the spectrum (T(1)) of T(1). If (T(1)) is inside the unit circle in the complex plane, then each orbit T(t), t 0 will approach zero exponentially and uniformly. If there is a point in (T(1)) outside the unit circle, then there is an unbounded orbit, and we have instability. A fundamental role in the study of stability and the preservation of stability under perturbations in the parameter is the behavior of the essential spectrum e(T(1)) of T(1). Let re r(e(T(1))) denote the radius of the essential spectrum of T(1). If it is known that re 1, then the stability or instability of 0 is determined by eigenvalues of (T(1)). Furthermore, if 0 is unstable, then it is due to only a nite number of eigenvalues; that is, the instability occurs in a nite dimensional subspace of X. If the latter situation occurs, then it is natural to expect that the stabilization of the system could be accomplished by using a nite dimensional control. However, if it is required that the stabilization be insensitive to small changes in the parame-
ter, then this may not be the case. If re 1, then the instability of the system is of such a nature that it cannot be controlled by a nite dimensional control. In general, eigenvalues of T(1) can be chosen to be continuous functions of . On the other hand, the function re may not be continuous in . The function re will be continuous in at a point 0 if it is known that T(1) T0(1) is compact. This condition is not necessary, but it is sufcient. If this difference is only bounded, then there is the possibility of a large shift in re if we vary . For example, if re0 1, and the perturbation is only bounded, it is possible to have re 1 for a sequence of j 0, and the semigroup Tj(t) will be unstable for each j. Let us interpret these remarks in terms of the examples that we have been discussing above. For nite dimensional problems, the semigroup is compact and thus the asymptotic behavior of orbits is determined by the eigenvalues. Since the semigroup for a LTI RFDE is compact for t h, the continuous spectrum always is the point zero, and so the asymptotic behavior is again determined by dominant eigenvalues (nite in number), and these are generally continuous in parameters. For D-stable NFDE, the essential spectrum lies in the unit circle for all values of the delay. Therefore, the introduction of small delays in the control function does not disturb the stabilization property of the feedback control. It can happen that the solutions of the difference equation associated with the difference operator D for an NFDE has all solutions approaching zero exponentially and uniformly for a particular value of the delays, and a small change in the delays leads to exponential instability. If this is the case, then the asymptotic behavior of solutions of NFDE subjected to small variations in the delays is not determined by the eigenvalues of the semigroup, but by the essential spectrum, which in turn is determined by the eigenvalues of the difference equations associated to the difference operator D. In the example [Eqs. (57) and (59)], the natural space of 1 initial data is HB (0, 1) L2(0, 1), where B represents the homogeneous boundary conditions w 0 at x 0, wx 0 at x 1. In this case, the boundary control wt is bounded but not compact. If this control is implemented with a delay, then the radius of the essential spectrum is increased considerably and, in fact, leads to instability. It is possible to consider a more physical version of Eqs. (5759), for which the boundary control problem is insensitive to small delays in the time at which it is implemented. Consider the equation wtt wxx cwxxt = 0, with the boundary conditions w ( 0, t ) = 0, wx (1, t ) + cwxt (1, t ) = kwt (1, t h ) (68) 0 < x < 1, t > 0 (67)
where h 0, c 0, k 0 are constants. In any space for which one can dene a C0-semigroup for Eqs. (67) and (68), the control function is compact. Furthermore, the radius of the essential spectrum is determined by the same problem with k 0 and is given by e(1/ c) 1. Therefore, stability is preserved with small perturbations in the delay. For further discussion of this topic, see Refs. 3237.
DEMODULATORS
149
BIBLIOGRAPHY
1. N. Minorsky, Self-excited oscillations in dynamical systems possessing retarded action, J. Appl. Mech., 9: 6571, 1942. 2. A. D. Mishkis, General theory of differential equations with a retarded argument, Amer. Math. Soc. Trans., No. 55, 1951. 3. N. N. Krasovskii, Stability of Motion, Moscow, 1959, Translation: Palo Alto: Stanford Univ. Press, 1963. 4. R. Bellman and K. Cooke, Differential Difference Equations, New York: Academic, 1963. 5. J. K. Hale and S. Verduyn-Lunel, Introduction to functional differential equations, Appl. Math. Sci., Vol. 99, New York: SpringerVerlag, 1993. 6. M. A. Cruz and J. K. Hale, Asymptotic behavior of neutral functional differential equations, Arch. Ration. Mech. Anal., 34: 331 353, 1969. 7. D. Henry, Linear autonomous neutral functional differential equations, J. Differ. Equ., 15: 106128, 1974. 8. Yu. M. Repin, Quadratic Lyapunov functionals for systems with delay (in Russian), Prikl. Mat. Mekh., 29: 564566, 1965. 9. B. S. Razumikhin, Applications of Lyapunovs method to problems in the stability of systems with a delay (in Russian), Autom. Telemekh., 21: 740749, 1960. 10. A. Halanay, Differential Equations, New York: Academic, 1966. 11. E. A. Andreeva, V. B. Kolmanovskii, and P. E. Shaikhet, Control of Systems with After Effect, Moscow: Nauka, 1992. 12. X. Li and J. Yong, Optimal Control Theory for Innite Dimensional Systems, Boston: Birkha user, 1995. 13. C. Foias, A. Tannenbaum, and G. Zames, Weighted sensitivity minimization for delay systems, IEEE Trans. Autom. Control, 31: 763766, 1986. 14. J. K. Hale and K. R. Meyer, A class of functional differential equations of neutral type, Mem. Amer. Math. Soc., 76: 1976. 15. R. Datko, An example of an unstable neutral differential equation, Int. J. Control, 38: 263267, 1983. 16. K. Cooke and J. Ferreira, Stability conditions for linear retarded functional differential equations, J. Math. Anal. Appl., 96: 480 504, 1983. 17. S. Saks and A. Zygmund, Analytic Functions, Warsaw: Polish Scientic Publishers, 1971. 18. V. M. Popov and A. Halanay, On the stability of nonlinear control systems with time-lag (in Russian), Autom. Telemekh., 23: 31 38, 1962. 19. G. Doetsch, Handbuch der Laplace-Transformation, Birkhau ser: Basel, 1956, Vol. 3. 20. R. Gabasov and F. M. Kirillova, The Maximum Principle in the Theory of Optimal Control (in Russian), Mrnsk: Minsk Science and Technology, 1974. 21. B. von Keulen, H-Control for Distributed Parameter Systems: A State-Space Approach, Birkhau ser: Basel, 1993. 22. R. Datko, Two questions concerning the boundary control of certain elastic systems, J. Differ. Equ., 92: 2744, 1991. 23. R. Datko, The point spectrum of some LQR problems. Appl. Math. Optim., 31: 8599, 1995. 24. J. Louis and D. Wexler, The Hilbert space regulator problem and operator Riccati equation under stabilization, Ann. Soci. Sci. Brux., 105: 137165, 1991. 25. K. Yoshida, The Hopf bifurcation and its stability for semilinear diffusion equation with time delay arising in ecology, Hiroshima Math. J., 12: 321348, 1982. 26. W. E. Fitzgibbon, Semilinear functional differential equations in Banach space, J. Differ. Equ., 29: 114, 1978.
27. G. F. Webb, Functional differential equations and nonlinear semigroups in Lp-spaces, J. Differ. Equ., 20: 7189, 1976. 28. R. Datko, Representation of solutions and stability of linear differential-difference equations in a Banach space, J. Differ. Equ., 29: 105166, 1978. 29. J. Wu, Theory and applications of partial functional differential equations, Appl. Math. Sci. 119: 50102, 1996. 30. G. F. Webb, Theory of Nonlinear Age-Dependent Populations, New York: Dekker, 1985. stro 31. J. F. A m and B. Wittenmark, Adaptive Control, Reading, MA: Addison-Wesley, 1989. 32. R. Datko, Two examples of ill-posedness with respect to time delays revisited, IEEE Trans. Autom. Control, 42: 511, 515, 1997. 33. J. K. Hale, Effects of delays on dynamics, in A. Granas and J. Frigon (eds.), Topological Methods in Differential Equations and Inclusions, Dordrecht, The Netherlands: Kluwer, 1995, pp. 191238. 34. K. B. Hannsgen, Y. Renardy, and R. L. Wheeler, Effectiveness and robustness with respect to time delays of boundary feedback stabilization in one-dimensional viscoelasticity, J. Control Optim., 26: 12001234, 1988. 35. W. Desch and R. L. Wheeler, Destabilization due to delay in one dimensional feedback, Int. Ser. Numer. Math., 91: 6183, 1989. 36. H. Logemann, R. Rebarber, and G. Weiss, Conditions for robustness and nonrobustness of the stability of feedback systems with respect to small delays in the feedback loop, J. Control Optim., 34: 572600, 1996. 37. J. K. Hale, Effects of delays on stability and control, CDSNS Tech. Rep., Atlanta: Georgia Institute of Technology, 1997.
R. DATKO
Georgetown University
J. K. HALE
Georgia Institute of Technology
DELTA-SIGMA MODULATORS. See SIGMA-DELTA MODULATION.
Abstract : Digital Control : Wiley Encyclopedia of Electrical and Electronics Engineering : Wiley InterScience
G G G G
HOME
G
ABOUT US
G
CONTACT US
G
HELP
Browse this title

G
Digital Control
Standard Article Yutaka Yamamoto1 1Kyoto University, Kyoto, Japan Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1010 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (241K)
Search this title
Abstract
The sections in this article are z-Transform Classical Design Methods and Their Limitations Sampling Theorem Modern Approach
H /H Control Problems Some Examples Bibliographical Notes
Keywords: A/D conversion; D/A conversion; z-Transform; shannon's sampling theorem; aliasing; lifting; frequency Response; H /H control
DIGITAL CONTROL
445
DIGITAL CONTROL
The revolutionary advances in computer technology today have made it possible to replace conventional controllers with digital computers. Digital control thus refers to the control scheme in which the controller is a digital device, generally a digital computer. This means that we can make use of a much more advanced control logic and versatility than those made possible with conventional analog controllers. On the other hand, we also need an interface that connects a computer with real plants. In particular, Measurement is made at discrete instants in time Data must be spatially discretized to allow digital data handling In other words, digital controllers can handle data that are discretized both in time and space. The former discretization is usually referred to as sampling and the latter quantization. These two features place digital control systems outside the scope of the usual linear, time-invariant control systems. (There is also the problem of saturation effect when controllers have a xed word length. But this problem is much less studied in the context of digital control.) To see the situation more concretely, consider the unityfeedback digital control system shown in Fig. 1. Here r is the reference signal, y the system output, and e the error signal. These are continuous-time signals. The error e(t) goes through the sampler (or an A/D converter) S . This sampler
r+
e S
ed C(z)
ud H C(z)
Figure 1. A unity-feedback digital control system consisting of a continuous-time plant P(s), discrete-time controller C(z), sampler S and a hold device H .
446
DIGITAL CONTROL
Rf +
Figure 2. Quantization converts the slanted straight line (thin) to the piecewise step zigzag function (thick).
reads out the values of e(t) at every time step h called the sampling period, and produces a discrete-time signal ed[k], k 0, 1, 2, . . .. In this process, a quantization error (due to round-off) as shown in Fig. 2 occurs. The sampling operator S acts on a continuous-time signal w(t), t 0 as S (w )[k] := w(kh ), k = 0, 1, 2, . . .
(The quantization effect is omitted here.) The discretized signal is then processed by a discrete-time controller C(z) and becomes a control input ud. This signal then goes through another interface H called a hold device or a D/A converter to become a continuous-time signal. A typical example is the zero-order hold, where H simply keeps the value of a discretetime signal w[k] as a constant until the next sampling time: (H (w[k] ))(t ) := w[k], for kh t < (k + 1 ) h
A typical sample-hold action [with C(z) identity] is shown in Fig. 3. A simple D/A converter can be constructed with operational ampliers, resistors, and switching devices as depicted in Fig. 4. Because this construction requires high precision in resistors, more elaborate circuitry is adopted in practice. There are other types of hold devices, for example, a rstorder hold for various reasons. In this article, however, we conne ourselves to the zero-order hold above.
; ; ; ;
E
u[t]
2R
4R
8R
Figure 4. A D/A converter is constructed with an operational amplier, switching, and resistors.
In the process above, a quantization error occurs in the A/D conversion. This is a round-off error that occurs when we convert analog values to digital data (often with a xed wordlength), as shown in Fig. 2. This introduces a nonlinearity into the system although other system components may be linear. A possible effect is that the closed-loop system may exhibit typical nonlinear behavior, such as limit cycles. Such phenomena are, however, much less studied compared to the effect arising from data sampling in time, and one usually assumes that sufcient spatial resolution is guaranteed so that the effect of quantization is negligible. The term digital control is thus used almost synonymously with sampled-data control (that is, the control scheme where measurement and control actions occur intermittently with a xed period) and quantization effects are ignored. Usually one considers single-rate sampled-data control systems where sampling and hold actions occur periodically with a xed period in a synchronized way. In practice, however, there are varied situations in which different sampling rates are employed at distributed control stations. Such a situation leads to multirate sampled-data control systems. However, for the sake of simplicity this article deas with single-rate systems. z-TRANSFORM We start with a fundamental description of systems and se quences. Let w[k]k 0 denote a sequence with values in some vector space X. Typically, X is the n-dimensional vector space n, but we will later encounter an example where X is not nite-dimensional. The z-transform of w w[k]k 0 is dened to be the formal sum (mathematically, this is called a formal power series): Z [w](z ) :=
k=0
w[k]z k
Figure 3. A simple sample-hold combination maps a continuoustime signal to a piecewise step function.
(z). The negative with indeterminate z. It is also denoted as w powers of z is in accord with the usual convention. Here z is just a formal variable, and z-transform at this stage simply gives a convenient way of coding sequences via the correspondence t } zt.
DIGITAL CONTROL
447
It can be readily veried that the z-transform Z [w u] of the discrete convolution

k
( w u )[k ] : =
j=0
w[k j ]u[ j ]
is given by the product Z [w]Z [u] of the z-transforms of the two sequences, i.e., Z [w u] = Z [w]Z [u] As a special case, the multiplication by z1 yields the time shift (delay): w[k]k 0 w[k 1]k1. Similarly, the multipli cation by z yields the time-advance operator: w[k]k 0 w[k 1]k0. The z-transformation plays the role of the Laplace transformation in the continuous-time case. As with Laplace transforms, it is useful to consider the substitution of a complex number to the variable z. For example, the geometric se quence kvk 0 has the z-transform
k=0
ues on or outside the unit circle z : z 1. This is equivalent to the transfer function being analytic in z : z 1, provided that there are no hidden poles of (zI A)1 cancelled by the numerator. Let us now give a sample-point description of the continuous-time plant P(s). Let (A, B, C) be its (minimal) realization. For brevity, we assume that the direct feedthrough term of P(s) is zero. This means x (t ) = Ax(t ) + Bu(t ) y(t ) = Cx(t ) (4)
The rst objective is to give a description of this plant at sampling instants t kh, k 0, 1, 2, . . .. By the zero-order hold, the input to the plant for kh t (k 1)h is the constant ud[k]. Suppose that the state of the plant at t kh is x[k] x(kh). Then by integrating Eq. (4) from kh to (k 1)h, we obtain
h
x[k + 1] = x((k + 1 )h ) = e Ah x[k] +

0 h
e A (h ) Bud [k] d
k z k v =
zv z
(1)
= e Ah x[k] +
0
e A ( ) Bd ud [k] =: Ad x[k] + Bd u[k] (5)
y[k] = y(kh ) = Cx[k] =: Cd x[k]

In other words, the behavior of P(s) at the sampling instants can be described by a time-invariant discrete-time system (Ad, Bd, Cd). This is the formula due to Kalman and Bertram (1). Its transfer function Pd (z ) := C(zI e Ah ) 1
h 0
We can consider this as a function with complex variable z. The sequence kv tends to zero if and only if 1; this is equivalent to its z-transform being analytic in z : z 1. For a xed sampling period h, the z-transform of a continuous-time signal x(t) is understood to be the z-transform of its sampled sequence: Z [x](z ) :=
k=0
x(kh )z k
eA Bd
There is a way to compute Z [x](z) from its Laplace transform (see Theorem below). Note also that the z-transform of an exponential function et is z /(z eh): Z [e t ] ( z ) = z z e h (2)
is called the pulse transfer function of P(s). Then the composite transfer function of the closed-loop system, when conned to sampled instants, is given by Pd(z)C(z)(I Pd(z)C(z))1. We then have the following result. Theorem 1. The behavior of the sampled-data system in Fig. 1 at sampled instants can be described by a time-invariant, discrete-time equation. To be more precise, let (A0, B0, C0, D0) and (A, B, C) be the minimal realizations of C(z) and P(s), respectively. Then the behavior of the closed-loop system at sampled instants can be represented by system matrices
Let us now give a system description. Suppose that a discrete-time system x[k + 1] = Ax[k] + Bu[k] y[k] = Cx[k] + Du[k] (3)
is given. Taking z-transforms of sequences x[k], u[k], y[k] and using the fact that the multiplication by z induces the time-advance operator, we see that = Ax + Bu zx = Cy + Du y Solving this, we have = C(zI A ) 1 x0 + [D + C(zI A ) 1 B]u y where x0 is the initial state at time 0. The second term D C(zI A)1B is the transfer function of this system. It is (asymptotically) stable if and only if zI A has no eigenval-
A0 BdC0
B0 B0Cd , , [0 Ad Bd D0Cd Bd DD0
Cd ]
Example 1. Let P(s) 1/ s2. This has a realization
0 dx = dt 0 y = [1
1 0 x+ u 0 1 0] x
Equation (5) is then computed easily as
x[k + 1] =
1 0
h h2 /2 x[k] + u d [k ] 1 h 0] x[k]
y[k] = [1
448
DIGITAL CONTROL
Much of the classical theory for sampled-data control is devoted to the computation of the pulse transfer function from a given continuous-time transfer function. Note that this precedes the advent of the modern state space theory, and elaborate formulas in the z and Laplace transform domains have been found. For example, the following theorem is well known (2,3): Theorem 2. Let P(s) be a rational function such that it is analytic for s : s R for some R 0 and sP(s) as s for some real . Then
x(t), its modied z-transform Z [x](z, m), 0 m 1 is dened as Z [x](z, m ) :=

k=0
x(kh + mh )z k
(9)
As in Theorem 2, the following result holds. Theorem 4. Assuming the same conditions as in Theorem 2, the following formulas for the modied z-transform holds:
1 Z [P](z ) = 2 j
P (s ) ds hs 1 1e z P (s ) Res = 1 ehs z 1 poles of P ( s )
Z [P](z, m ) =
(6)
P(s )emhs ds mhs z 1 C 1e P(s )emhs Res = 1 emhs z 1 poles of P ( s ) 1 2 j
where denotes a contour that travels from c j to c j [c: abscissa of convergence of P(s); the coordinate to the right of which the Laplace integral dening P(s) converges] and goes around a semcircle on the left half-plane that encircles all poles of P(s). Once the sample-point behavior is specied by the procedure above, it is easy to give a description of the intersample behavior of the output or the state. Suppose, for example, that the state of the plant P(s) takes values x[k] at t kh, k 0, 1, 2, . . .. By integrating Eq. (4) from t kh to t kh (0 h), we get x(kh + ) = e and y(kh + ) = Cx(kh + ) (8)
A
The modied z-transform has a close connection with lifted transfer functions in the modern approach (see the section entitled Modern Approach). CLASSICAL DESIGN METHODS AND THEIR LIMITATIONS We briey review the classical design methods and their limitations. The rst is a design method based on the continuoustime design. Continuous-Time Design A simple, straightforward method is to employ a continuoustime design, obtain a continuous-time controller, and then convert the controller to a discrete-time system via some kind of discretization. Let Cc(s) be a continuous-time controller. Typical discretization methods are the following: Use the Tustin (bilinear) transformation: C(z ) = Cc 2 z1 h z+1
x[k] +
0
A ( )
Bud [k] d
(7)
This shows that if x[k] and ud[k] tend to zero as k , then the intersampling behavior x(kh ), 0 h also tends to zero uniformly for as k . This is because the right-hand side of Eq. (7) is just the multiplication and convolution of known continuous functions with x[k] and ud[k] over a nite interval. Therefore, the stability of a sampled-data system can be determined solely by its sample-point behavior. We summarize this observation in the form of a theorem: Theorem 3. The closed-loop system in Fig. 1 is stable if the discrete-time closed-loop system consisting of C(z) and Pd(z) is stable. Therefore, to stabilize the plant P(s) in the sampleddata setting Fig. 1, it is necessary and sufcient that Pd(z) be stabilized by C(z). This result gives a foundation for the classical treatment of sampled-data systems. To design (or at least to stabilize) a sampled-data system, one can equivalently stabilize the pulse transfer function Pd(z) derived from P(s). This led to the classical design procedure based on pulse transfer functions. Equations (7) and (8) are closely related to the notion of the modied z-transform. For a continuous-time function
Employ the backward difference (z 1)/ hz for approximating the differential operator s. Approximate Cc(s) by using the sample/hold equivalent Eq. (5). Among these, the rst method is most commonly used. It is well known that the Tustin transformation preserves stability: if Cc(s) is a stable transfer function (in the sense of continuous-time systems), then the transformed function Cc(2(z 1)/ h(z 1)) gives a stable (in the sense of discretetime systems) discrete-time transfer function. Although this is a great advantage in signal processing, care must be exercised in control system design, because this property does not guarantee the closed-loop stability. In fact, as h becomes larger, there is even a case in which the closed-loop stability is violated (see the example in the section entitled H Design). This is because the original continuous-time design does not usually take account of the sampling period. To take care of this, one has to pay more attention to various robustness properties, such as gain and phase margins, and so
DIGITAL CONTROL
449
on. To discuss such properties, frequency domain considerations are highly desirable. However, the notion of frequency response is not readily available. To see the situation, let C(z) be a discrete-time transfer function. Suppose that a sinusoid ejt is applied after sampling. Then the actual input to C(z) is ekjhk 0 with ztransform z /(z ejh) given by Eq. (2). The steady-state response of C(z) against this input is then given by ekjhC(ejh)k 0. It appears that we can discuss the frequency domain properties via C(ejh). For example, one might attempt to employ phase lead/lag compensation based on this quantity. However, due to sampling, this frequency response does not fully represent the nature of continuous-time inputs. For example, not only ejt but also ej(2n / h)t, n 1, 2, . . . give exactly the same sampled values ekjhk 0. The response is then governed by the same C(ejh). This means that sampling does not have enough resolution to distinguish all these sinusoids, and the notion of phase, which depends on the response against sinusoids, is unclear in such a sampled-data control setup. Another way of seeing this is to note that ejh ejh and hence C(ej(2 / h)h) C(ejh). This means that beyond the frequency / h, the same gain characteristic repeats periodically and C(ejh) cannot be treated as the same frequency response concept as in the continuous-time case. This is related to the notion of aliasing, which we examine in more detail in the section Modern Approach. It may be still possible to execute an elaborate continuoustime design that also works well in the sampled-data setting by looking more closely into the nature of the Tustin transformation. However, in such a method, a systematic design method such as H design theory is difcult to apply. Furthermore, one needs a more concrete understanding of the phenomena above, and this is much better done in the modern approach treated in the subsequent sections. Discrete-Time Design Yet another classical approach is based on the pulse transfer function Pd(z). As far as stability is concerned, one can deal only with Pd(z). It was also recognized that sampled-data control can achieve performance that is not possible with linear, time-invariant, continuous-time controller. For example, the so-called deadbeat control achieves the property that the output (or state) settles exactly to zero after a nite time period. This is done by placing all poles of the closed-loop system to zero; the output, at least at sampled instants, then becomes zero after a nite timea performance not possible with continuous-time controllers. It should be, however, noted that such a classical treatment also shares the weakness of the classical transfer function approach. Namely, it did not take account of hidden polezero congurations. In particular, it was observed that merely settling the output might sometimes induce very large intersample ripples. It was Kalman and Bertram (1) who introduced the state space approach for sampled-data systems. As we have already seen, the sample-time inputoutput relation is described by
h
Let us consider the stabilization by state feedback. If we employ a sampled state feedback ud [k] = Kx[k] then by Theorem 3 the stability of the closed-loop system is determined by the spectrum of A d Bd K (10)
Thus this is a purely discrete-time pole-placement problem. Furthermore, if we can set the eigenvalues of Eq. (10) all to zero, then x[k] becomes zero after a nitely number of steps if there is no external input. Also with Eqs. (7) and (8), not only x[k] but also the intersampling trajectory will settle to zero. This clearly shows the advantage of the state space theory, which was introduced around that time. The problem is thus reduced to the pole allocation for the discrete-time system (Ad, Bd) and the feasibility of this is reduced to the problem of determining the controllability and stabilizability of this pair. Naturally, we may as well assume that the continuous-time plant (A, B, C) is stabilizable or controllable. Otherwise, it is not possible to stabilize the plant even with continuous-time controllers. For brevity, let us consider controllability. The following result is well known (4): Theorem 5. Let 1, . . ., n be eigenvalues of A. Suppose that for no pair i, j (i j), i j is an integer multiple of 2 / h. Then (Ad, Bd) is controllable if and only if (A, B) is controllable. The proof is an easy consequence of the fact that the eigenvalues of Ad eAh are e1, . . ., en. This is a variant of the spectral mapping theorem, but we skip the details here (4,5). By the discussions above, it may appear that sampled-data control systems can be safely designed via discrete-time design methods. Note that, at least for the deadbeat control via state space, we can also settle the intersample behavior identically to zero after a nite number of steps. However, this is valid only for regulation problems, and the issue of the intersample behavior for tracking (servo control) problems, where exogenous signals are present, is quite different. To see this, consider the example depicted in Fig. 5. Here the continuoustime plant is 1/(s2 1) whose natural frequency is 1 rad/s. On the other hand, the tracking signal is sin (1 2 /0.1)t where the sampling period h is 0.1 s. It so happens that, at sampling instants t kh, k 0, 1, 2, . . ., this signal is identical with sin t because (2 /0.1)kh 2k. Therefore, for the discrete-time controller the tracking signal is no different
sin(1 + 20 )t
r+
e S
ed
e2h1 z e2h
ud H s2
1 +1
x[k + 1] = Ad x[k] + Bd ud [k] = e Ah x[k] +
e A Bd ud [k]
Figure 5. A unity feedback system with tracking reference signal sin(1 20)t.
450
DIGITAL CONTROL
strict sense, if we take the intersample behavior into account. This is closely related to the issue of the notion of aliasing effects and Shannons sampling theorem. We briey review these in the next section.
0.5
SAMPLING THEOREM
0
0.5
1 43
43.2
43.4
43.6
43.8
44
Let f (t) be a given continuous-time signal on (, ). To make the sampling well dened, we assume that f is a contin uous function. The sampled sequence is f (kh)k . As it is, this is just a sequence dened on the set of integers. The question here is how we should represent this sequence in the continuous-time domain. Recall that the z-transform of f (kh)k is
k =
Time (sec)
Figure 6. The simulation of Figure 5 shows that the input sin(1 20)t does not yield a sinusoid at the same frequency, and large intersample ripples result.
f (kh )z k
from sin t. The simulation result is shown in Fig. 6. The plant output is shown by the solid curve while the reference input is shown by the dashed curve. The output tracks sin t rather than sin(1 2 /0.1)t, and there is a large amount of intersample ripples due to the difference between sin(1 2 /0.1)t and sin t. This example shows the following: There can be large intersample ripples for sampled-data systems. Such ripples are difcult to characterize via the discretetime framework as described above. The ripples do not appear to be stationary. The observations above indicate that the discrete-time model Eq. (5) is generally not appropriate for describing sampleddata systems when there are nontrivial intersample ripples. What is indicated here is that we need a framework that can give a description for the continuous-time behavior of a sampled-data system. Suppose that we wish to describe a frequency response. Let sin t be an input applied to the sampled-data system shown in Fig. 1. For linear, time-invariant, stable continuous-time systems, it is well known that a single sinusoid yields another sinusoid in the steady-state output, with exactly the same frequency, possibly with gain and phase shifts. To be more precise, let G(s) be the transfer function of such a system. It is well known that the steady-state output is G( j ) sin t That is, as t , the output asymptotically approaches G( j) sin t. Such a separation principle does not hold for sampled-data systems. In fact, the example shown in Fig. 6 shows a counterexample: the steady-state output against sin(1 2 /0.1)t is sin t sinusoid, but with a different frequency. One of the reasons for such a phenomenon is that sampleddata systems are no longer time-invariant systems in a very
We have extended the denition in a natural way to the negative ks. We also recall that the multiplication by z1 is the right shift operator. Since the kth signal value f (kh) is placed at t kh, this right shift corresponds to the right shift by time length h in the continuous-time domain. It is also well known that in the Laplace transform domain the right shift operator by h is represented by the multiplication by ehs. Therefore, it is natural to represent the Laplace transform of the sequence f (kh)k by
k =
f (kh )e khs
The inverse Laplace transform of this is the train of impulses (Delta functions) multiplied by f (kh) at the kth step:
k =
f (kh )(t kh )
(11)
Observe that this is formally a multiplication of f (t) with the train of impulses
k =
(t kh )
(12)
and thus it is called the impulse modulation of f (t). The question that concerns us here is the following: Sup pose that we are given a sampled sequence f (kh)k , or khs k f (kh)e just as well. How much can we recover the original signal f (t) out of this piece of data? If we impose no condition on f (t), then the solution is clearly nonunique. There is innite freedom in the intersampling periods while passing through f (kh), k . A typical solution is obtained by assuming that f (t) is band-limited; that is, its Fourier transform is zero outside a bounded interval. This is the content of the following Shannon sampling theorem. Theorem 6. Let f be a continuous function that is Fourier transformable. Suppose that its Fourier transform is identi-
DIGITAL CONTROL
451
cally zero outside the interval ( / h , / h ) for some 0. Then f (t ) =

n =
f (nh )
sin (t /h n ) (t /h n )
( 2n / h). Then only the spectrum f () in the to n f fundamental frequency range ( / h, / h) remains. Applying the inverse Fourier transform, we obtain
(13)
()] = F f (t ) = F [f = h F [ ] =
n =
()
n =
( 2n/h ) f
We now briey indicate the outline of a proof. As noted above, Eq. (11) is obtained by multiplying f (t) to the train of impulses [Eq. (12)]. Hence its Fourier transform is just the convolution of the respective Fourier transforms (6). For the Fourier transform of Eq. (12), the following Poisson summation formula is well known (6):
f (nh )(t nh )
n =
f (nh )sinc(t nh )
F
n =
(t nh )
2 h
n =
2 n h
where sinc t : hF []. This function is easily computed as (14) sinc t = sin t /h t /h
It follows that
F
n =
f (nh )(t nh )
=F = =
f (t )
n =
(t nh )
n =
1 f() 2 1 f() h
2 h
n =
t 2 n h
2 n h
1 = h = 1 h
t 2 n f h n = 2 n f h n =
In other words, we get innitely many copies of the shifted () as shown in Fig. 7. This is because a sinusoid image of f sin t behaves precisely the same as sin( 2m / h)t at sampled points t nh, n 0, 1, 2, . . .. Such higher frequency signals that arise from sampling are called alias components. It is clearly not possible to recover the original signal f (t) from such data contaminated by aliasing. In particular, there is in ( 2n / h). (The period s : general an overlapping of f 2 / h of these spectra is called the sampling frequency and its half / h the Nyquist frequency.) However, it is possible to recover f (t) if such an overlapping does not occur. Indeed, it will be clear from Fig. 7 that if the is zero outside the interval ( / h, / h), original spectrum f then there is no overlapping among those copies. The band is zero outside ( / h , / h ) limited hypothesis that f guarantees this. To eliminate all unnecessary alias components, multiply the function
This readily implies Eq. (13). This result also clarifes the meaning of Theorem 5. When there is a pair of eigenvalues that differ only by an integer multiple of 2 / h, the corresponding two modes cannot be distinguished because they yield the same eigenvalues when discretized. Some remarks are in order. Although Eq. (13) certainly gives a well-dened reconstruction formula, it is crucially based on the assumption that the original signal f (t) is band limited. This assumption, which appears quite innocent, is seldom satised in practice. In fact, if f is band limited, it must be an entire function; that is, it is analytic on the whole complex plane. We can hardly expect real signals to be analytic functions. Therefore, the assumption for Eq. (13) can be satised only in an approximate sense. The second drawback is that Eq. (13) is not causal. In other words, it makes use of future sampled values f (nh) to reconstruct the current value f (t). It is therefore not physically realizable. To remedy this, one should be content with approximation, and a large portion of digital signal processing is devoted to the various solutions of this problem. Theorem 6 also yields the following observations: By sampling, intersampling information is generally lost. In particular, sinusoids sin( 2n / h)t, n 0, 1, 2, . . . cannot be mutually distinguished. However, this is about the maximum uncertainty introduced by sampling. After sampling, all the components that arise in the output are combinations of all such alias components sin( 2n / h)tn . The last statement still needs to be claried. The basic idea is the following: When a sinusoid sin t is sampled, it is converted to a modulated train of impulses as shown in Eq. (12). In other words, innitely many alias components sin( 2n / h)t are excited by sampling. To avoid an undesirable effect arising from such aliased components, it is generally necessary to place an analog low-pass lter (usually called an anti-aliasing lter) in front of the sampler. Since this cannot cancel the alias components completely, how much such alias components affect the overall performance is a concern. Such a question has been studied in the literature (2,3,7). However, its general structure is better understood in
() :=
1 0
(|| /h ) (|| > /h )
(15)
f( )
f( 2 /h) /h
/h
Figure 7. The spectrum of f repeats periodically with period 2 / h.
452
DIGITAL CONTROL
the scope of the modern approach, which we describe in the subsequent sections. MODERN APPROACH We now turn our attention to the foundation of the modern treatment of sampled-data systems. From what we have presented up to this section, it is clear that the fundamental difculty in sampled-data control systems lies in the fact that they involve two different time sets: one is discrete (arising from the digital controller) and the other is continuous (arising from the continuous-time plant). This difculty has been successfully circumvented in the modern approach. A New Model with Intersample BehaviorLifting While it is possible to recover intersample behavior via the modied z-transform, it implicitly assumes sampling inputs in its formulation. It is therefore not adequate for describing correspondence from the exogenous continuous-time inputs to continuous-time outputs. A new solution was introduced in 19901991 (812). The new idea, currently called lifting, makes it possible to describe sampled-data systems via a time-invariant, discretetime model while maintaining the intersample behavior. The idea is very simple. Let f (t) be a given continuous-time signal. Sampling surely results in a loss of intersample information. Then, instead of sampling f (t), we will represent it as a sequence of functions. Namely, we set up the correspondence (Fig. 8): L : f { f [k]( )} k = 0, f [k]( ) = f (kh + ), 0 <h
t kh. As in Eq. (5), the state x[k 1] at time (k 1)h is given by

h
x[k + 1] = e Ah x[k] +
0
e A (h ) Bu[k]( ) d
(17)
The difference here is that the lifted input u[k]( ) need not be a constant on (kh, (k 1)h), and the right-hand side integral gives an operator L2 [0, h ) Rn : u( )
h 0
e A (h ) Bu( ) d
While state-transition is described in the discrete timing as above, the system keeps producing an output. If we consider lifting of x(t), it is easily seen to be described by x[k]( ) = e A x[k] +
0
e A ( ) Bu[k]( ) d
Then the lifted output y[k]( ) is given by y[k]( ) = Ce A x[k] +

0
Ce A ( ) Bu[k]( ) d
(18)
Observe that Eqs. (17) and (18) take the form x[k + 1] = A x[k] + Bu[k] y[k] = C x[k] + D u[k] and the operators A , B , C , D do not depend on k. In other words, it is possible to describe this continuous-time system with discrete-timing, once we adopt the lifting point of view. To be more precise, the operators A , B , C , D are dened as follows:
This idea makes it possible to view (time-invariant or even periodically time-varying) continuous-time systems as linear, time-invariant discrete-time systems. (Basically the same idea that converts periodically time-varying discrete-time systems to time-invariant systems were encountered and rediscovered many times in the literature. It appears to date back at least to Ref. 13. Such a discrete-time lifting is also frequently used in signal processing, especially in multirate signal processing, and is called blocking.) The basic idea is the following: Let x (t ) = Ax(t ) + Bu(t ) y(t ) = Cx(t ) (16)
A : B: C : D:
Rn Rn : x e Ah x
L2 [0, h ) Rn : u
h 0
e A (h ) Bu( ) d (19)
0
Rn L2 [0, h ) : x Ce A x
L2 [0, h ) L2 [0, h ) : u Ce A ( ) Bu( ) d
be a given continuous-time plant. Then we lift the input u(t) to obtain u[k]( ). We consider that this lifted input is applied at the timing t kh (h is a prespecied sampling rate), and observe how it affects the system. Let x[k] be the state at time
f(t)
Thus the continuous-time plant in Eq. (16) can be described by a time-invariant discrete-time model. Once this is done, it is entirely routine to connect this expression with a discrete-time controller, and hence sampled-data systems (for example, see Fig. 1) can be fully described by time-invariant discrete-time equations, this time without sacricing the intersampling information. We will also denote this overall equation abstractly as x[k + 1] = A x[k] + Bu[k] (20)
0 h 0 h 2h 3h 4h 0
0 h 1
0 h 2
0 h 3
y[k] = C x[k] + D u[k]

k
Figure 8. Lifting maps a continuous-time signal to a discrete-time signal with function components.
Since this is purely a discrete-time system except that the inputoutput spaces are innite-dimensional (L2 spaces), all the formal developments presented earlier carries over to the
DIGITAL CONTROL
453
present situation without any change. For example, the transfer function is dened as G ( z ) := D + C ( z I A ) 1 B When an input u(z) is applied to system in Eq. (20), its zeroinitial state output is given by G(z)u(z). Note also that operator A in Eq. (19) is a matrix, and hence A in Eq. (20) is also a matrix. This means that the stability of Eq. (20) can be tested by the poles of G(z). It is stable if G(z) is analytic for z : z 1. Steady State and Frequency Response The time-invariance of the lifted model Eq. (20) naturally yields a denition of stead-state and frequency responses. Let G(z) be a stable transfer function as dened above. Take a sinusoid ejt as an input. Its lifted image is {e j kh e j } k=0 According to Eq. (1), the z-transform of this sequence is ze j z e jh Since G(z) is stable, expand it in a neighborhood of z ejh: (z ) G ( z ) = G ( e j h ) + ( z e j h )G (z) that is also analytic in z 1 (by the stability with some G of G). It follows that G (z ) ze zG(e )e = z e jh z e jh
j jh j
Since ej(ns)t is lifted to be ejkhej(ns)k 0, such aliasing high-frequency signals are in the form ejkhv(). Thus the denition above takes all the aliasing components into account, and takes the largest magnitude of enlargement among them.
Frequency Response via Sequence Spaces The observation above clearly shows that the gain of the frequency response takes all aliasing effects into account. It is however unclear that aliasing exhausts all the freedom in v L2[0, h). In other words, is it true that if we consider the largest gain imposed by considering all aliased components, does this give the same gain as Eq. (23)? This is indeed true; its proof is based on the following lemma which guarantees that the family ejn / hn (n 2 ns) forms an orthonormal basis of L [0, h), and hence any v L2[0, h) can be expanded into a series of aliased signals ejn, n 0, 1, 2, . . .. Lemma 1. Fix any [0, s). Then every L2[0, h) can be expanded in terms of ejnn as ( ) = with an = 1 h
h 0 n =
an e j n
(24)
e j n ( ) d =
1 ( j n ) h
(25)
(z )e j +G
The second term on the right tends to zero as k by the , and hence the output approaches analyticity of G jh j jh zG(e )e /(z e ). Therefore, the lifted output y[k]( ) asymptotically approaches y[k]( ) = (e j h )k G(e j h )[e j ]( ) (21)
where denotes the Laplace transform of when extended to L2[0, ) as 0 outside [0, h). Furthermore, the L2 norm is given by
2
=h
n =
|an |2
(26)
as k . Unless ejh 1, the asymptotic response above is really not in steady state. However, its modulus G(ejh)ej remains invariant at each sampling time. The change at each step is a phase shift induced by the multiplication by ejh. This explains why the ripples in Fig. 6 look similar but not really the same in different sampling periods. This observation motivates the following denition: Denition 1 Let G(z) be the transfer function of the lifted system as above. The frequency response operator is the operator G (e
jh
Let us apply this result to the frequency response dened by Eq. (23). Expand v L2[0, h) as v() vej. Note jkh jn jnt that e e k0 is the lifted image of e , and hence j t ejkhv()k 0 is ve . By Eq. (21), the asymptotic response of G(z) against this input is given by
e j kh G(e j h )[v] = e j kh G(e j h ) =
v e j
j
(27) ]v
e
=
j kh
G (e
jh
)[e
Expand G(ejh)[ej] in terms of ejn to get G ( e j h )[e j ] =

n =
) : L [0, h ) L [0, h )
2 2
(22)
regarded as a function of [0, s) (s : 2 / h). Its gain at is dened to be G (e j h ) =

jh
gn e j n
sup
v L 2 [0 , h )
G ( e j h )v v
Substituting this into Eq. (27), we obtain (23) e j kh G(e j h )[v] = e j kh

The maximum G(e ) over [0, s) is the H norm of G(z).
= n =
gn e j n v
454
DIGITAL CONTROL
Since ej(ns)h ejh, this is the kth step response of

= n =
gn e j n t v
where t kh . Interchanging the order of summation, this is equal to

n = =
where R (I D *D ). The important point to be noted here is that all the operators appearing here are actually matrices. For example, by checking the domain and range 1 spaces, we easily see that B R B * is a linear operator from n into itself, i.e., a matrix. Therefore, in principle, one can solve the singular value Eq. (29) by nding a nontrivial solution for Eq. (30) (provided R is invertible) (17,18). H / H 2 CONTROL PROBLEMS A signicant consequence of the modern approach to sampleddata control is that various robust control problems such as H / H2 control problems are now completely solved. The problem was initiated by Chen and Francis (19) and later solved in Refs. 9, 10, and 2022 in more complete forms; see Ref. 5 for the pertinent historical accounts. To state the problem more precisely, let us introduce the notion of generalized plants. Suppose that a continuous-time plant is given in the following form:
gn ()v
e jn t
(28)
This means that the response against vejt is again expressible as an innite sum of all such aliased signals. It should be intuitively clear that the largest gain among them again gives the gain of the frequency response, when such signals are equipped with norm (nv2)1/2. This isometric correspondence is guaranteed by the Parseval identity Eq. (26). This is the viewpoint adopted in Refs. 14 and 15 to discuss the frequency response of sampled-data systems; see also Ref. 16. It is also closer to the classical treatment based on the impulse modulation (3,7). Gain Computation The gain function G(ejh) is given as the operator norm at each frequency, and its computation is primarily an innite-dimensional problem. However, for most of the practical purposes, it can be computed as the maximal singular value (17). Our problem is thus reduced to that of solving the singular value equation [ 2 I G G ( e j h )]w = 0 (29)
x c (t ) = Axc (t ) + B1 w(t ) + B2 u(t ) z (t ) = C1 xc (t ) + D11 w(t ) + D12 u(t ) y(t ) = C2 xc (t )

Here w is the exogenous input, u(t) is the control input, y(t) is the measured output, and z(t) is the controlled output. The controller is of the following form:
xd [k + 1] = Ad xd [k] + Bd S y[k] v[k] = Cd xd [k] + Dd S y[k] u[k]( ) = H ( )v[k]

where H() is a suitable hold function. This is shown in Fig. 9. The objective here is to design or characterize a controller that achieves a prescribed performance level > 0 in such a way that Tzw
This is still an innite-dimensional equation. However, since A , B , C , D are nite-rank operators, we can reduce this to a nite-dimensional rank condition. Note that, by lifting, a realization of G(z) can be written in the form x[k + 1] = A x[k] + Bw[k] y[k] = C x[k] + D w[k] Its adjoint can then be easily derived as p[k] = A p[k + 1] + C v[k] e[k] = B p[k + 1] + D v[k] Taking the z transforms of both sides, setting z ejh, and substituting v y and e 2w, we get
<
(31)
z A C1 C2 B1 B2 D11 D12 0 0
jh
x = A x + Bw p = e j h A p + C (C x + D w )
S H( )
( 2 D D )w = e j h B p + D C x
Solving these, we obtain
jh
I 0
1 BR B 1 A + C D R B 1 A + BR D C 1 C ( I + D R D )C
Ad Cd
Bd Dd
0 I
x =0 p
(30)
Figure 9. Generalized plant construction of a sampled-feedback system where z denotes the controlled output, y is the measured output, w is the exogenous input, and u is the control input.
DIGITAL CONTROL
455
where Tzw denotes the closed-loop transfer function from w to z. This is the H control problem for sampled-data systems. The H2 control problem is obtained by replacing the H norm above by the H2 norm. The difculty here is that both w and z are continuoustime variables, and hence their lifted variables are innitedimensional. A remarkable fact here is that the H problem (and H2 problem as well) [Eq. (31)] can be equivalently transformed to the H problem for a nite-dimensional discretetime system. While we skip the details here [see the references above and (5)], we remark that this norm-equivalent discrete-time system is entirely different from the one given in the section on Discrete-Time Design in that it fully takes intersampling behavior into account. The difference will be exhibited by the design examples in the next section. SOME EXAMPLES
Initial condition results ( = 0.1) 2 1.5 1 0.5 0 0.5 1 1.5 2
Amplitude
To see the power of the modern design methods, let us consider two design examples. We start with the H design. H Design Consider the unstable second-order plant P(s ) := C p (sI A p ) 1 B p = with weight matrices Q1 / 2 = 1 R1/2 = 0.01 E = 0.01 N = 0.01 1 s 2 0.1s + 1
10
15 Time (s)
20
25
30
Figure 11. Time responses for h 0.1 by sampled-data (solid) and continuous-time (dash) H designs do not show much difference.
and the antialiasing lter aa (s ) := F 1 hs + 1
depicted in Fig. 10. We here compare the following two design results: The direct sampled-data H design The continuous-time H design with Tustin transformation In the continuous-time design, the antialiasing lter is bypassed. On the other hand, it is inserted in the sampled-data design to make the total design well posed.
Figure 11 shows the impulse responses of the designed closed loop for the sampling period h 0.1. The solid curve represents the response for the sampled-data design and the dashed curve shows that for the continuous-time design with Tustin transformation. They do not present much difference at this stage. However, when we increase the sampling period (i.e., decrease the sampling rate) to h 0.55 (Fig. 12), the continuous-time design is already very close to the stability margin. In the conventional design, one may conclude that this sampling period is already too long, and the whole conguration is not feasible for sampled-data implementation. But quite contrary to such an intuition, the sampled-data H design can tolerate such a long sampling period. The crucial
Initial condition results ( = 0.55) 2 1.5 1 0.5 0 0.5 1 1.5
Q1/2 zc R1/2 N E wc
Faa
Cp
(sI Ap)1
Bp
Amplitude
10
15 Time (s)
20
25
30
Figure 10. Generalized plant for sampled-data and continuous-time H design.
Figure 12. Time responses for h 0.55 exhibit a clear difference between sampled-data (solid) and continuous-time (dash) H designs.
456
DIGITAL CONTROL
50 40 30 20 10
Frequencygain response
Gain (dB)
0 10 20 30 101
2
2 Time
100 Frequency (rad/s)
101
Figure 15. Time responses for sampled-data (solid) and discretetime (dash) design show a clear difference.
Figure 13. Frequency response plots for h 0.55 support the observation in Fig. 11.
Consider a simple second-order plant P(s) 1/(s2 2s 1). For h 0.2, we execute Sampled-data (continuous-time based) H2 design Discrete-time H2 design Figures 14 and 15 show the frequency and time responses of the closed-loop systems, respectively. In Fig. 14, the solid (thick) curve shows the response of the sampled-design, whereas the dotted (thin) curve shows the discrete-time frequency response when the designed controller K is connected with the discretized plant Gd (i.e., a purely discrete-time frequency response). At a rst glance, it appears that the discretized design performs better, but actually it performs poorer when we compute the real (continuous-time) frequency response of G connected with Kd. The dashed curve shows this frequency response; it is similar to the discrete-time frequency response in the low-frequency range but exhibits a very sharp peak at the Nyquist frequency ( / h 15.7 rad/s, i.e., 1/2h 2.5 Hz). In fact, the impulse responses in Fig. 15 exhibit a clear difference between them. The solid curve shows the sampleddata design, and the dashed curve the discrete-time one. The latter shows an oscillatory response. Also, both responses decay to zero very rapidly at sampled instants. The difference is that the latter exhibits very large ripples, with periods of approximately 0.4 s. This corresponds to 1/0.4 Hz, which is the same as (2)/0.4 / h rad/s, i.e., the Nyquist frequency. This is precisely captured in the modern (lifted) frequency response in Fig. 14. It is worth noting that when h is smaller, the response for the discrete-time design becomes even more oscillatory, and shows a very high peak in the frequency response. The details may be found in Ref. 23. BIBLIOGRAPHICAL NOTES For classical treatments of sampled-data control, it is instructive to consult Refs. 2, 3, and 7. The textbooks (24,25) cover both classical and modern aspects of digital control. For dis-
difference here is that the sampled-data design incorporates the sampling period in the design procedure, whereas the continuous-time design does not. This gap becomes even clearer when we compare two designs via their frequency responses (Fig. 13). Whereas the sampled-data design exhibits a rather mild curve, the continuous-time design shows a very sharp peak at around 1.5 rad/s. Observe also that this frequency agrees precisely with the period of oscillation in the impulse response (Fig. 13). H 2 Design In the case of continuous-time design, slower sampling rates yield problems. For the sample-point discretization, fast sampling rates can induce very wild responses.
5 0 5
Gain (dB)
10 15 20 25 30 35 0.01 0.1 1 Frequency (rad/s) 10
Figure 14. Frequency response plots show the difference between the sampled-data (solid), discrete-time (dashed) difference; the thin dotted curve shows the frequency response with intersample behavior ignored.
DIGITAL CONTROL
457
crete-time design methods and other related topics, the reader is also referred to the handbook (26). For Shannons sampling theorem, consult Ref. 27 for various extensions and some historical accounts. As noted in the main text, discrete-time lifting has been introduced and rediscovered by several authors (see, for example, Refs. 13, 28, and 29). Aside from the H design, the H2 control problem has also been studied extensively (3033). L1-norm problems are studied in Refs. 3436. In relation to the H control problem, various robust stability problems have been studied (see, for example, Refs. 15 and 37). The treatment of frequency response given here is based on Refs. 16 and 17. Some computational aspects are discussed in Ref. 38. For other approaches not dealt with here, see also Refs. 39 and 40.
19. 20.
21.
22.
23.
BIBLIOGRAPHY
1. R. E. Kalman and J. E. Bertram, A unied approach to the theory of sampling systems, J. Franklin Inst., 267: 405436, 1959. 2. E. I. Jury, Sampled-Data Control Systems, New York: Wiley, 1958. 3. J. R. Ragazzini and G. F. Franklin, Sampled-Data Control Systems, New York: McGraw-Hill, 1958. 4. R. E. Kalman, Y. C. Ho, and K. Narendra, Controllability of linear dynamical systems, Contrib. Differ. Equations, 1: 189213, 1963. 5. T. Chen and B. A. Francis, Optimal Sampled-Data Control Systems, New York: Springer, 1995. 6. A. H. Zemanian, Distribution Theory and Transform Analysis, New York: Dover, 1987. 7. J. T. Tou, Digital and Sampled-Data Control Systems, New York: McGraw-Hill, 1959. 8. B. Bamieh et al., A lifting technique for linear periodic systems with applications to sampled-data control systems, Syst. Control Lett., 17: 7988, 1991. 9. G. Tadmor, Optimal H sampled-data control in continuous time systems, Proc. ACC, 1991, pp. 16581663. 10. H. T. Toivonen, Sampled-data control of continuous-time systems with an H optimality criterion, Automatica, 28: 4554, 1992. 11. Y. Yamamoto, New approach to sampled-data systems: A function space method, Proc. 29th CDC, 1990, pp. 18821887. 12. Y. Yamamoto, A function space approach to sampled-data control systems and tracking problems, IEEE Trans. Autom. Control, AC39: 703712, 1994. 13. B. Friedland, Sampled-data control systems containing periodically time varying members, Proc. 1st IFAC Congr., 1961, pp. 361367. 14. M. Araki, Y. Ito, and T. Hagiwara, Frequency response of sampled-data systems, Automatica, 32: 483497, 1996. 15. G. Dullerud and K. Glover, Robust stabilization of sampled-data systems to structured LTI perturbations, IEEE Trans. Autom. Control, AC-38: 14971508, 1993. 16. Y. Yamamoto and M. Araki, Frequency responses for sampleddata systemstheir equivalence and relationships, Linear Algebra Its Appl., 205206: 13191339, 1994. 17. Y. Yamamoto and P. P. Khargonekar, Frequency response of sampled-data systems, IEEE Trans. Autom. Control, AC-41: 166 176, 1996. 18. Y. Yamamoto, On the state space and frequency domain charac-
24.
25. 26. 27. 28.
29.
30. 31. 32. 33.
34. 35. 36.
37.
38.
39.
40.
terization of H-norm of sampled-data systems, Syst. Control Lett., 21: 163172, 1993. T. Chen and B. A. Francis, On the L 2-induced norm of a sampleddata system, Syst. Control Lett., 15: 211219, 1990. B. Bamieh and J. B. Pearson, A general framework for linear periodic systems with applications to H sampled-data control, IEEE Trans. Autom. Control, AC-37: 418435, 1992. P. T. Kabamba and S. Hara, Worst case analysis and design of sampled data control systems, IEEE Trans. Auto. Control, AC38: 13371357, 1993. N. Sivashankar and P. P. Khargonekar, Characterization and computation of the L 2-induced norm of sampled-data systems, SIAM J. Control. Optim., 32: 11281150, 1994. S. Hara, Y. Yamamoto, and H. Fujioka, Modern and classical analysis/synthesis methods in sampled-data controla brief overview with numerical examples, Proc. 35th IEEE CDC, 1996, pp. 12511256. stro K. J. A m and B. Wittenmark, Computer Controlled Systems Theory and Design, Upper Saddle River, NJ: Prentice-Hall, 1996, 3rd ed. D. Williamson, Digital Control and Implementation, New York: Prentice-Hall, 1991. W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, 1996. A. I. Zayed, Advances in Shannons Sampling Theory, Boca Raton, FL: CRC Press, 1993. J. H. Davis, Stability conditions deried from spectral theory: Discrete systems with periodic feedback, SIAM J. Control, 10: 1 13, 1972. P. P. Khargonekar, K. Poolla, and A. Tannenbaum, Robust control of linear time-invariant plants using periodic compensation, IEEE Trans. Autom. Control, AC-30: 10881096, 1985. T. Chen and B. A. Francis, H 2-optimal sampled-data control, IEEE Trans. Autom. Control, AC-36: 387397, 1991. P. P. Khargonekar and N. Sivashankar, H 2 optimal control for sampled-data systems, Syst. Control Lett., 17: 425436, 1991. B. Bamieh and J. B. Pearson, The H 2 problem for sampled-data systems, Syst. Control Lett., 19: 112, 1992. S. Hara, H. Fujioka, and P. T. Kabamba, A hybrid state-space approach to sampled-data feedback control, Linear Algebra Its Appl., 205206: 675712, 1994. G. Dullerud and B. A. Francis, L 1 performance in sampled-data systems, IEEE Trans. Autom. Control, AC-37: 436446, 1992. N. Sivashankar and P. P. Khargonekar, Induced norms for sampled-data systems, Automatica, 28: 12671272, 1992. B. Bamieh, M. A. Dahleh, and J. B. Pearson, Minimization of the L-induced norm for sampled-data systems, IEEE Trans. Autom. Control, AC-38: 717732, 1993. N. Sivashankar and P. P. Khargonekar, Robust stability and performance analysis of sampled-data systems, IEEE Trans. Autom. Control, AC-38: 5869, 1993. S. Hara et al., Computational aspects of gain-frequency response for sampled-data systems, Proc. 34th IEEE CDC, 1995, pp. 1784 1789. A. Feuer and G. C. Goodwin, Generalized sample hold functionsfrequency domain analysis of robustness, sensitivity and intersample difculties, IEEE Trans. Autom. Control, AC-39: 10421047, 1994. J. S. Freudenberg, R. H. Middleton, and J. H. Braslavsky, Inherent design limitations for linear sampled-data feedback systems, Int. J. Control, 61: 13871421, 1995.
YUTAKA YAMAMOTO
Kyoto University
458
DIGITAL FILTERS
DIGITAL CONTROL SYSTEMS DESIGN. See DISCRETE TIME SYSTEMS DESIGN METHODS.
Abstract : Discrete Event Dynamical Systems : Wiley Encyclopedia of Electrical and Electronics Engineering : Wiley InterScience
G G G G
HOME
G
ABOUT US
G
CONTACT US
G
HELP
Browse this title

G
Discrete Event Dynamical Systems

Standard Article Xi-Ren Cao1 1The Hong Kong University of Science and Technology, Kowloon, Hong Kong Copyright 1999 by John Wiley and Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1011 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (223K)
Search this title
Abstract
The sections in this article are DEDS Models Perturbation Analysis Applications: On-Line Optimization
602
DISCRETE EVENT DYNAMICAL SYSTEMS

In recent decades, many modern, large-scale, human-made systems (e.g., exible manufacturing systems, computer and communication networks, air and highway trafc networks, and the military C3I /logistic systems) have been emerging. These systems are called discrete event systems because of the discrete nature of the events. Research indicates that these human-made systems possess many properties that are similar to those of natural physical systems. In particular, the evolution of these human-made systems demonstrates some dynamic features; exploring these dynamic properties may lead to new perspectives concerning the behavior of discrete event systems. The increasing need for analyzing, controlling, and optimizing discrete event systems has initiated a new research area, the dynamics of discrete event systems. To emphasize their dynamic nature, these systems are often referred to as discrete event dynamic systems (DEDS) in the literature (1). This article reviews the fundamental theories and applications of DEDS. Since the dynamic behavior is closely related to time, we shall not discuss untimed models such as the automata-based model (2); these models are mainly used to study the logical behavior of a discrete event system. In 1988, the report of the panel of the IEEE Control Systems Society noted, Discrete event dynamic systems exist in many technological applications, but there are no models of discrete event systems that are mathematically as concise or computationally as feasible as are differential equations for continuous variable dynamical systems. There is no agreement as to which is the best model, particularly for the purpose of control (3). This statement is still true today. However, after the hard work of many researchers in the recent years, there are some relatively mature theories and many successful application examples. Most frequently used models for analyzing DEDS are queueing systems and Petri nets. Queueing models are usually used to analyze the performance (in most cases, steady state performance) of DEDS, and Petri nets provide a graphical illustration of the evolution of the system behavior and are particularly useful in analyzing behaviors comprising concurrency, synchronization, and resource sharing (4). Other models for DEDS include the more general but less structural models such as Markov processes and generalized semiMarkov processes (GSMP), and the max-plus algebra that is particularly suitable for modeling DEDS with deterministic event lifetimes that exhibit a periodic behavior.
603
One main theory that employs a dynamic point of view to study system behavior is the perturbation analysis (PA). The objective of PA is to obtain performance sensitivities with respect to system parameters by analyzing a single sample path of a discrete event system (59). The sample path, which describes how the DEDS evolves, can be obtained by observing the operation of a real system or by simulation. The technique is in the same spirit of the linearization of nonlinear continuous variable dynamic systems (6). The sample path based approach of PA motivates the research of on-line performance optimization of DEDS. Recent study shows that PA of discrete parameters (parameters that jump among discrete values) is closely related to the Markov decision problem (MDP) in optimization. The PA-based online optimization technique has been successfully applied to a number of practical engineering problems. The following section briey reviews some basic DEDS models. The next section introduces PA in some details. The nal section 4 discusses the application of PA in on-line optimization and points out its relations with the Markov decision problem. DEDS MODELS Queueing Systems The simplest queueing system is the M / M /1 queue, where customers arrive at a single server according to a Poisson process with rate and the service time for each customer is exponentially distributed with mean 1/ , . The steady state probability of n, the number of customers in the queue, is p(n ) = n (1 ) = , n = 0, 1, . . .
server j with probability qi, j and leaves the network with probM ability qi,0. We have j0 qi, j 1, i 1, 2, , M. The service time of server i is exponentially distributed with mean si 1/ i, i 1, 2, , M. The system state is n (n1, n2, , nM), where ni is the number of customers in server i. Let i be the arrival rate of the customers to server i. Then
M
i = 0 , i +
j=1
j q j ,i
i = 1, 2, . . ., M
It is known that the steady state distribution is

M
p(n ) = p(n1 , n2 , . . ., nM ) =
k=1
p (nk )
with p(nk ) = (1 k )k k
n
k =
k k
k = 1, 2, . . ., M
From this, the average number of customers in the queue is n = 1
A load-independent closed Jackson (Gordon-Newell) network is similar to the open Jackson network described above, except that there are N customers circulating among servers M according to the routing probabilities qi, j, k1 qi,k 1, i 1, M 2, , M. We have k1 nk N. We consider a more general case: the service requirement of each customer is exponential with a mean 1; the service rates, however, depend on the number of customers in the server. Let i,ni be the service rate of server i when there are ni customers in the server, 0 i,ni , ni 1, 2, , N, i 1, 2, , M. We call this a load-dependent network. In a load-independent network, i,ni i for all ni, i 1, 2, , M. The state of such a network is n (n1, n2, , nM). We use ni, j (n1, , ni 1, , nj 1, , nM), ni 0, to denote a neighboring state of n. Let
and the average time that a customer stays in the queue is T= 1 and let
(nk ) =
1 if nk > 0 0 if nk = 0
The more general model is a queueing network that consists of a number of service stations. Customers in a network may belong to different classes, meaning that they may have different routing mechanisms and different service time distributions. A queueing network may belong to one of three types: open, closed, or mixed. In an open network, customers arrive at the network from outside and eventually leave the network; in a closed network, customers circulate among stations and no customer arrives or leaves the network; A mixed network is open for some classes of customers and closed for others. An open Jackson network consists of M single-server stations and N single-class customers. Each server has an buffer with an innite capacity and the service discipline is rstcome-rst-served. Customers arrive at server i according to a Poisson process with (external) rate 0,i, i 1, 2, , M. After receiving the service at server i, a customer enters
(n ) =
k=1
(nk )k,n
Then the ow balance equation for the steady state probability p(n) is
M M
(n ) p(n ) =
i= 1 j = 1
(n j )i,n
i +1
qi , j p (n j , i )
Let yi 0, i 1, 2, , M, be the visit ratio to server i, that is, a solution (within a multiplicative constant) to the equation
M
yi =
j=1
q j ,i y j
j = 1, 2, . . ., M
604
Let Ai(0) 1, i 1, 2 , M, and

k
b1
b2 Mach. 1
Mach. 2 Exit
Ai (k ) =
j=1
i , j
i = 1, 2, . . ., M
b3
and for every n 1, 2, , N and M 1, 2, , M, let

m
Gm ( n ) =
n 1 ++ n m = n i = 1
yn i Ai (ni )
Figure 1. A reentrant line.
Then we have
p (n ) =
1 GM ( N )
M i= 1
yi i Ai (ni )
This equation is often referred to as a product-form solution. For load-independent networks, i,ni i, i 1, 2, , M. The product-form solution becomes
m
Gm ( n ) =
n 1 ++ n m = n i = 1
xi i
and
p (n ) =
1 GM ( N )
M i= 1
xi i
(1)
where xi yi / i yisi, i 1, 2, , M. There are a number of numerical methods for calculating p(n) and the steady state performance, among them are the convolution algorithm and the mean value analysis (7); in addition, analytical expressions exist for the normalizing constant GM(N) (10). For more about queueing theory, see, for example, Refs. 11 and 12. One typical example of the queueing model is the resource sharing problem. Consider the case where M resources are shared by N users and each resource can be held by only one user at any time. Every time a user grasps resource i, it holds the resource for a random time with si, A user, after the completion of its usage of resource i, requests the hold of resource j with a probability qi, j. This problem can be modeled exactly as a closed queueing network with N customers and M servers. This model can be successfully used in analyzing the performance of packet switches, where the users are the head-of-line packets and the resources are the channels, and the performance of data-base systems, where the users are programs and the resources are data records. Petri Nets Many DEDS consist of components [e.g., central processing units (CPUs), disks, memories, and peripheral devices in computer systems; and machines, pallets, tools, and control units in manufacturing systems] that are shared by many users and exhibit concurrency. This feature makes Petri nets a suitable model for DEDS. In a graphical representation, the structure of a Petri net is dened by three sets: a set of places P p1, p2, , pn, a set of transitions T t1, t2, , tm, and a set of arcs. A place is represented by a circle and a transition by a bar. An arc is represented by an arrow from a transition to a place or
a place to a transition. A place is an input (output) to a transition if an arc exists from the place (transition) to the transition (place). The dynamic feature of a Petri net is represented by tokens, which are assigned to the places. Tokens move from place to place during the execution of a Petri net. Tokens are drawn as small dots inside the circle representing the places. The marking of a Petri net is a vector M (m1, m2, , mn), where mi is the number of tokens in place pi, i 1, 2, , n. A marking corresponds to a state of the system. A Petri net executes by ring transitions. A transition is enabled if its every input place contains at least one token. When a transition is enabled, it may re immediately or after a ring delay, which can be a random number. The ring delay is used to model the service times. When a transition res, one token is removed from each input place and one token added to each output place. Thus, the number of tokens in a place and in the system my change during the execution. In addition to the arcs described above, another type of arc, called the inhibitor arc, is often used to model the priority among services. An inhibitor arc is drawn from a place to a transition, with a small circle at its end. When an inhibitor arc is used, if there is at least one token in the place, the transition cannot re. As an example, we consider the reentrant line (13) shown in Fig. 1. The system consists of two machines and three buffers. Work pieces arrive at buffer b1 with rate and get service from machine 1 with rate 1; after the service in b1, the piece moves to buffer b2 and gets service at machine 2 with rate 2; after the service in b2, the piece reenters machine 1 at buffer b3 and receives service with rate 3. Machine 1 can serve one piece at a time, and pieces in b3 have a nonpreemptive higher priority than those in b1. The Petri net model for the system is shown in Fig. 2. In the gure, places bi, i 1, 2, 3, represent the buffers, and transitions pi, i 1, 2, 3, represent the service processes of the pieces in the three buffers. If there is a token in places mi, i 1, 2, then machine i is available; if there is a token in si, i 1, 3, then the work piece in buffer bi is under service. It is clear that machine 1 is shared by the pieces in both b1 and b3, and the inhibitor arc from b3 to p4 models the priority. (For more about Petri net theory and applications, see Refs. 4 and 1416). The state process of a queueing system or a Petri net can be modeled as a Markov process, or more generally, as a generalized semi-Markov process. In this sense, both Markov processes and GSMPs are more general than queueing systems and Petri nets; however, these general models do not enjoy the structural property that queueing systems and Petri nets possess. In fact, the GSMP model is a formal description of the evolution mechanism of a queueing system. Readers are referred to Refs. 5 and 8 for a discussion of GSMP.
605
in
b1
p1 1
b2
p2 2
We can prove that for all i 1
s1 p4 m2 m1 p5 s3 Exit p3
Figure 2. The Petri net model for the reentrant line.
K (2i ) = K (2i+ 1 ) =
0 3 2 1
1 0 1 2
M is said to be of order 2 periodic. Cohen et al. (17) proved that all matrices possess such periodicity. Therefore, Eq. (2) can be used to study the periodic behavior of a DEDS. For more discussions, see Ref. 18. PERTURBATION ANALYSIS Perturbation analysis of DEDS is a multidisciplinary research area developed since early 1980s, with the initial work of Ho et al. (19). PA provides the sensitivities of performance measures with respect to system parameters by analyzing a single sample path of a DEDS. This area is promising because of its practical usefulness. First, compared with the standard procedure, which uses the difference of performance measures with two slightly different values for every parameter, this technique saves a great amount of computation in simulation, because PA algorithms can provide the derivatives with respect to all the parameters by using only a single simulation run. In addition, PA estimates are more accurate than those obtained from nite differences because the latter may encounter numerical problems caused by dividing two small numbers. Second, PA can be applied to on-line performance optimization of real-world systems by observing a sample path of an existing system; for these systems, changing the values of their parameters may not be feasible. Cao (20) observed that the simple PA algorithms based on a single sample path, called innitesimal perturbation analysis (IPA), in fact yield sample derivatives of the performance; although these sample derivatives are unbiased or strongly consistent for many systems, this is not the case for many others. This insight has set up two fundamental research directions: to establish IPA theory, including the proof of convergence of IPA algorithms, and to develop new algorithms for systems where IPA does not work well. After the hard work of many researchers in more than one decade, the theory for IPA is relatively mature, and many results have been obtained for problems where IPA does not provide accurate estimates. Innitesimal Perturbation Analysis (2) Let be a parameter of a stochastic discrete event system; the underlying probability space is denoted as (, F , P ). Let (), , be a random vector that determines all the randomness of the system. For example, for a closed queueing network, may include all the uniformly distributed random variables on [0, 1) that determine the customers service times and their transition destinations (say, in a simulation). Thus, a sample path of a DEDS depends on and ; such a sample path is denoted as (, ). Let T0 0, T1, , Tl, be the sequence of the state transition instants. We consider a sample path of the system
b3
The Max-Plus Algebra With the max-plus algebra proposed in Ref. 17, many DEDS can be modeled as linear systems. In the max-plus algebra, we dene the operation multiplication on two real numbers a and b as a b a b and the addition as a b max a, b. It is easy to verify that these two operations indeed dene an algebra. We give a simple example to illustrate how the max-plus algebra can be used to analyze the periodic behavior of a DEDS. Consider a single queue with deterministic interarrival time a and service time s. Let ak and dk be the arrival and departure times of the kth customer, respectively, k 1, 2, , and a1 0, d0 0. Then for k 1, 2, we have ak + 1 = a + ak dk = max{ak + s, dk 1 + s} In the max-plus algebra, this can be written as a linear equation xk+ 1 = Axk where k = 1, 2, . . .
xk =
ak dk 1
A=
a d
H d
with H being a large positive number. Thus, xk + 1 = A k xk
where Ak A(k1) A, k 1, and A1 A. An interesting property of a matrix under the max-plus algebra is the periodic property. This can be illustrated by an example. Consider
M=
1 3
5 3 =4 2 1
1 =4K 2
606
in a nite period [0, TL). The performance measured on this sample path is denoted as L(, ). Let L ( ) = E [L (, )] and ( ) = lim L (, )
L
i,1 i,1 i,2 t1 t2
(3)
Figure 3. The perturbation in a busy period.
w. p.1.
(4) All the papers published in the area of IPA deal with these two basic issues. Roughly speaking, the interchangeability requires that the sample performance function L(, ) be continuous with respect to . General conditions can be found in Refs. 20 and 8. The algorithms for obtaining sample derivatives are called IPA algorithms. Given a nite-length sample path (, ), we rst, by applying IPA algorithms, ctitiously construct a sample path for the DEDS with a slightly changed parameter and the same random vector, ( , ), called a perturbed sample path. The derivative of the performance with respect to can be obtained by comparing these two sample paths, the original one and the perturbed one. The principles used in IPA to determine the perturbed path are very simple. We take closed queueing networks as an example. The basic idea is that a change in parameter (say, a mean service time) will induce changes of the service completion times, and a change of a customers service completion time will affect the other customers service completion times. IPA rules describe how these changes can be determined. Figure 3 illustrates a busy period of a server, say server i, in a queueing network. Let Fi(s, ) be its service time distribution. The service time of its kth customer is si,k = Fi 1 (i,k , ) = sup{s : F (s, ) i,k }, where i,k, k 1, 2, , are uniformly distributed random variables on [0, 1). With the same i,k, in the perturbed sys1 (i,k, ). Thus, the tem, the service time changes to F i service time increases by
i,k
where E denotes the expectation with respect to the probability measure P . We assume that both the mean and limit in Eq. (3) and Eq. (4) exist. Thus L(, ) is an unbiased estimate of L() and an strongly consistent estimate of (), respectively. The goal of perturbation analysis is to obtain the performance derivative with respect to by analyzing a single sample path (, ). That is, we want to derive a quantity based on a sample path (, ) and use it as an estimate of L()/ or ()/ . Given a single sample path, the realization of the random vector is xed. Therefore, we x and consider L(, ) as a function of . This function is called a sample performance function. Now, we consider the following question: given a sample path (, ), can we determine the sample path ( , ) with the same and / 1? If we can, then we can get the performance for the perturbed system, L( , ), and furthermore, the derivative of the sample performance function: ( + (, ) = lim L 0 L , ) L ( ), ) (5)
This is called a sample derivative. It seems reasonable to choose the sample derivative / L(, ) as an estimate for L()/ or ()/ . This estimate is called the innitesimal perturbation analysis (IPA) estimate. We require that the estimate be unbiased or strongly consistent; that is, either E or lim [ (, )] = L lim [L (, )] (7) [ (, )] = E [L (, )] L (6)
= Fi 1 (i,k , + = Fi 1 (i,k , )
) Fi 1 (i,k , )
i , k = Fi (s i , k , )
(8)
In Eq. (5), the same random variable is used for both L( , ) and L(, ); this corresponds to the simulation technique common random variable in estimating the difference between two random functions. This technique usually leads to small variances. Equations (6) and (7) are referred to as the interchangeability in the literature (20). From the above discussion, the two basic issues for IPA are 1. To develop a simple algorithm that determines the sample derivative of Eq. (5) by analyzing a single sample path of a discrete event system; and 2. To prove that the sample derivative is unbiased and/or strongly consistent, that is, the interchangeability of Eq. (6) and/or Eq. (7) holds.
Equation (8) is called the perturbation generation rule, and i,k is called the perturbation generated in the kth customers service time. If the service time is exponentially distributed with its mean changed from si to si si, then Eq. (8) becomes
i,k
s i s s i i,k
(9)
The delay of a servers service completion time is called the perturbation of the server, or the perturbation of the customer being served. In Fig. 3, the perturbation of the rst customer is i,1 (si,1 / si) si. Because of this perturbation, the service starting time of the next customer is delayed by the same amount. Furthermore, the service time of the second customer increases by i,2 (si,2/ si) si, and thus the perturbation of the second customer is i,1 i,2 (see Fig. 3). In general, the service completion time of the kth customer in a
607
Server 1
Server 2
t0
t1
t2
t3
t4
Figure 4. Propagation of a single perturbation.
busy period will be delayed by j1 i, j, with i, j being determined by Eq. (8) or Eq. (9). This can be summarized as follows: a perturbation of a customer will be propagated to the next customer in the same busy period; the perturbation of a customer equals the perturbation generated in its service period plus the perturbation propagated from the preceding customer. If a perturbation at the end of a busy period is smaller than the length of the idle period following the busy period, the perturbation will not affect (i.e., will not be propagated to) the next busy period, because the arrival time of the next busy period depends on another servers service completion time. A perturbation at one server may affect other servers through idle periods. To see how servers may affect each other, we study the evolution of a single perturbation. In Fig. 4, at t1, server 1 has a perturbation , and before t1, server 2 is idle. At t1, a customer arrives from server 1 to server 2. Because server 1s service completion time is delayed by , server 2s service starting time will also be delayed by ; and as a result, its service completion time will also be delayed by the same amount. We say the perturbation is propagated from server 1 to server 2 through an idle period (server 2 has the same perturbation as server 1 after t1). At t3, this perturbation is propagated back to server 1. In summary, if a perturbation is smaller than the lengths of idle periods (we say that the original and the perturbed paths are similar), then the evolution of this perturbation on the sample path can be determined by the following IPA perturbation propagation rules:
k
not be larger than the length of the idle period; otherwise the idle period in the original sample path will disappear in the perturbed one and the simple propagation rules illustrated by Fig. 4 no longer hold. It is easy to see that for any nitelength sample path (, ), we can always (with probability one) choose a that is small enough (the size depends on ) such that the perturbations of all customers in the nite sample path are smaller than the shortest length of all the idle periods in the sample path. This explains the word innitesimal in IPA. Therefore, we can always use IPA propagation rules to get the perturbed sample path and the sample derivative. The perturbed sample path is completely determined by the perturbations of the servers. Given a sample path of a single-class closed Jackson network of M servers with sm, m 1, 2, , M, being the mean service times, the perturbations of the perturbed system with si changed to si si (with i xed) can be determined by the following algorithm. IPA Algorithm for Closed Jackson Networks 0. Create a vector v (v1, v2, , vM); set its initial value v (0, 0, , 0) 1. At the kth, k 1, 2, , service completion time of server i, set vi : vi si,k 2. If on the sample path, a customer from server j terminates an idle period of server l, then set vl : vj. Note that for simplicity in the algorithm we add si,k, instead of (si / si) si,k, to the perturbation vector. Thus, the perturbation of server m, m 1, 2, , M, is (si / si) vm, with vm being determined by the algorithm. We shall see that the term si / si is eventually cancelled in Eq. (11). The sample derivative can be obtained from these perturbations. Let the sample performance measure be
(f) L =
1 L
TL 0
f [N (t )] dt
(10)
where N(t) is the state process and f is a function dened on the state space. The steady state performance measure is
(f) ( f ) = lim L L
1. A perturbation of a customer at a server will be propagated to the next customer at the server until it meets an idle period, and 2. If, after an idle period, a server receives a customer from another server, then after this idle period the former will have the same perturbation as the latter (the perturbation is propagated from the latter to the former). The perturbation generation rule describes how perturbations are generated because of a change in the value of a parameter; perturbation propagation rules describe how these perturbations evolve along a sample path after being generated. Combining these rules together, we can determine the perturbed path. To apply the propagation rules, the size of the perturbation at the end of each busy period should not be larger than the length of the idle period that follows, and the size of the perturbation of a customer that terminates an idle period should
w. p.1
(I) If f I 1 for all n, then we have L TL / L and (I) 1/ , where is the throughout of the system. The sample deriva(f) tive of L can be easily obtained by adding performance calculations to the basic IPA algorithm as follows.
0. Set v (0, 0, , 0), H 0, and H 0 1,2. Same as steps 1 and 2 in the basic IPA algorithm 3. At server ms service completion time, denoted as Tl, set H H f [N(Tl)](Tl Tl1) and H H f [N(Tl)] f [N(Tl)]vm. In the algorithm, H records the value of the integral in Eq. (10), and 1 s i ( H) L s i
608
(f) represents the difference of L between the original and the perturbed paths. At the end of the sample path, we have
(f) L (s i , i ) =
H L
and
(f) L (s i , i ) =
s i H s i L
Thus, we can calculate the sample elasticity as follows

(f) (s i , i ) L s i H = (f) s i H L (s i , )
SPA attempts to nd a suitable Z so that the average performance function L(, Z is smooth enough and the interchangeability of Eq. (12) holds, even if Eq. (6) does not. If this is the case and the derivative of the conditional mean L(, Z ) can be calculated, then / L(, Z ) can be used as an unbiased estimate of / L(). The method was rst proposed in Ref. (23); a recent book Ref. (5) contains a detailed discussion about it. The main issues associated with this method are that it may not be easy to calculate / E[L(, Z )] and that the computation effort required may be large. Finite Perturbation Analysis. The sample derivative does not contain any information about the jumps in the performance function. This is because as goes to zero, the event sequence of any perturbed sample path is the same as that of the original path (two paths are similar). Thus, with IPA, we do not study the possibility that because of a parameter change , two events may change their order. In nite perturbation analysis (FPA), a xed size of is assumed. For any xed , the event sequence in the perturbed path may be different from that in the original path. FPA develops some rules that determine the perturbation when the event order changes. The FPA algorithm is more complicated than IPA, and it is usually approximate since only order changes between adjacent events are taken into account (9). Sample Path Constructability Techniques. Given the nature of IPA, it cannot be applied to sensitivities with respect to changes of a xed size or changes in discrete parameters. Motivated by the principles of IPA, we ask the following question: Given a sample path of discrete event system under parameter , it is possible to construct a sample path of the same system under a different parameter ? This problem is formulated as the sample path constructability (7). Normally, such constructability requires that the sets of events and states of the sample path to be constructed (with parameter ) belong to the sets of events and states of the original sample path. For example, one may construct a sample path for an M / M /1/ K 1 queue (where K 1 denotes the buffer size) from a sample path of an M / M /1/ K queue. Ref. 24 shows that for some systems with additional computation such sample path construction can be done even if some states in the sample path to be constructed do not appear in the original sample path. Techniques in this class include augmented system analysis (7,25), extended perturbation analysis (26), and the standard clock approach (27). Structural Innitesimal Perturbation Analysis. Structural innitesimal perturbation analysis (SIPA) was developed to address the problem of estimating the performance sensitivity with respect to a class of parameters such as the transition probabilities in Markov chains. At each state transition, in addition to the simulation of the original sample path, an extra simulation is performed to obtain a quantity needed to get the performance sensitivity. It has been shown that the extra simulation requires bounded computational effort, and that in some cases the method can be efcient (28). It is interesting to note that this approach can be explained by using the concept of realization discussed in the next subsection.
(11)
It has been shown that Eq. (11) is strongly consistent (9,6), that is,
(f) (s i , i ) L s i s i ( f ) = L ( f ) (s s i i ( f ) s i , ) L
lim
w. p.1
Similar algorithms and convergence results have been obtained for open networks, networks with general service time distributions, and networks in which the service rates depend on the system states (6). Glasserman (8) studied various IPA algorithms and their unbiasedness and strong consistency in the GSMP framework. Extensions of IPA For a sample derivative (i.e., the IPA estimate) to be unbiased and strongly consistent, usually requires that the sample function be continuous. This request, however, is not always satised. A typical example illustrating the failure of IPA is the two-server multiclass queueing network discussed in Ref. (21); later, Heidelberger et al. (22) discussed in detail a few extensions of the example. In the past decade, many methods have been proposed to extend IPA to a wider class of problems. Each of these methods has some success on some problems at the cost of increasing analytical difculty and computational complexity. We shall review only briey the basic concepts of these methods. Smoothed Perturbation Analysis. The idea of smoothed perturbation analysis (SPA) is to average out the discontinuity over a set of sample paths before taking the derivative and expectation. To illustrate the idea, we rst write the expected value of L(, ) as L ( ) = E [L (, )] = E {E [L (, ) | Z ]} where Z represents some random events in (, F , P ). Let L(, Z ) E[L(, )Z ], then L ( ) = E [L (, Z )] and Eq. (6) becomes E [ (, Z )] = E [L (, Z )] L (12)
609
maud (29) studies the perforRare Perturbation Analysis. Bre mance sensitivity with respect to the rate of a point process and proposes the method of rare perturbation analysis (RPA). The basic idea is that the perturbed Poisson process with rate with 0 is the superposition of the original Poisson process with rate and an additional Poisson process with rate . Thus, in a nite interval, the difference between the perturbed path and the original one is rare. The performance derivative is then obtained by studying the effect of these rare but big (meaning nite) perturbations on the system performance. The case 0 is called the positive RPA. When 0, the perturbed Poisson process with rate can be constructed by thinning the original Poisson process with the thinning probability / . That is, some arrival points in the original process will be taken away. The performance derivative is then obtained by studying the effect of the removal of these rare arrival points. This is called the negative RPA. Others in this direction include Refs. 30 and 31. Estimation of Second Order Derivatives. The single path based approach can also be used to estimate the second order derivatives of the performance of a DEDS by calculating the conditional expectations. See Ref. 32 for GI/G/1 queues and Ref. 33 for Jackson networks. Others. In addition to the above direct extensions of IPA, it also motivated the study of a number of other topics, such as the Maclaurin series expansion of the performance of some queueing systems (35), the rational approximation approach for performance analysis (36), and the analysis of performance discontinuity (37). Finally, besides the PA method, there is another approach, called the likelihood ratio (LR) method (3840), that can be applied to obtain estimates of performance derivatives. The method is based on the importance sampling technique in simulation. Compared with IPA, the LR method may be applied to more systems but the variances of the LR estimates are usually larger than those of IPA. Perturbation Realization One important concept regarding the sensitivity of steady state performance of a DEDS is the perturbation realization. The main quantity related to this concept is called the realization factor. This concept may provide a uniform framework for IPA and non-IPA methods. The main idea is: The realization factor measures the nal effect of a single perturbation on the performance measure of a DEDS; the sensitivity of the performance measure with respect to a parameter can be decomposed into a sum of the nal effects of all the single perturbations induced by the parameter change. Perturbation Realization For Closed Jackson Networks. Suppose that at time t 0, the network state is n and server i obtains a small perturbation , which is the only perturbation generated on the sample path. This perturbation will be propagated through the sample path according to the IPA propagation rules and will affect system performance. The realiza-
tion factor of a perturbation of server i at t 0 with state n, denoted as c(f)(n, i), is dened as
c( f ) (n, i ) = lim E
L
1
0
TL
f [N (t )] dt
0
TL
f [N(t )] dt (13)
where TL and N(t) represents the quantities in the perturbed path. A perturbation is said to be realized if at some time Tl all the servers have the same perturbation ; it is said to be lost if at some time Tl no server has any perturbation. It was proved that in an irreducible closed network a perturbation will be either realized or lost with probability one. The probability that a perturbation is realized is called the realization probability. Suppose that a perturbation is realized or lost at TL*. L* depends on the sample path, that is, . If the perturbation is lost, then f [N(t)] f [N(t)], for all t TL*; if it is realized, then f [N(t)] f [N(t )] for all t TL*. Therefore, from the Markov property, Eq. (13) becomes
c( f ) (n , i ) = E
1
0
TL
f [N (t )] dt
0
TL
f [N(t )] dt (14)
where L* is a random number, which is nite with probability one. Realization factors can be uniquely determined by a set of linear equations (6). The steady state performance sensitivity can be obtained by s i ( f ) = p(n )c( f ) (n , i ) (I ) s i all n (15)
where I(n) 1 for all n and p(n) is the steady state probability of n. A close examination reveals that the IPA algorithm provides a simple way for estimating the quantity alln p(n)c(f)(n, i) on a single sample path. The theory has been extended to more general networks, including open networks, state-dependent networks, and networks with generally distributed service times (6). Perturbation Realization for Markov Processes. Consider an irreducible and aperiodic Markov chain X Xn; n 0 on a nite state space E 1, 2, , M with transition probabilM ity matrix P [pij]M i1j1. Let (1, 2, , M) be the vector representing its steady state probabilities, and f [f (1), f (2), , f (M)]T be the performance vector, where T represents transpose and f is a column vector. The performance measure is dened as its expected value with respect to :
M
= E ( f ) =
i= 1
i f (i ) = f .
(16)
Assume that P changes to P P Q, with 0 being a small real number and Qe 0, e (1, 1, , 1)T. The performance measure will change to . We want to estimate the derivative of in the direction of Q, dened as
610
/ Q lim0 / . It is well known that IPA does not work for this problem. In this system, a perturbation means that the system is perturbed from one state i to another state j. For example, consider the case where qki , qkj , and qkl 0 for all l i, j. Suppose that in the original sample path the system is in state k and jumps to state i, then in the perturbed path, it may jump to state j instead. Thus, we study two independent Markov chains X Xn; n 0 and X n ; n 0 with X0 i and X 0 j; both of them have the same transition matrix P. The realization factor is dened as (34):
di j = E
[ f (Xn ) f (Xn )]|X0 = i, X0 = j i, j = 1, 2, . . ., M (17)
n=0
Thus, dij represents the long term effect of a change from i to j on the system performance. Equation (17) is similar to Eq. (13). If P is irreducibile, then with probability one the two sample paths of X and X will merge together. That is, there is a random number L* such that X L * XL* for the rst time. Therefore, from the Markov property, Eq. (17) becomes
discrete event systems, analytical approach does not usually exist for parametric optimization problems. One has to resort to simulation or experimental approaches, where the derivative estimates obtained by perturbation analysis can play an important role. There are two major algorithms used in stochastic optimization: KieferWolfowitz (KW) and RobbinsMonro (RM). Both are essentially the hill-climbing type of algorithm. The KW algorithm employs the performance difference as an estimate of the gradient. With PA, we can obtain, based on a single sample path of a DEDS, the estimates of the gradients. Thus, the RM algorithm, which is known to be faster than the KW algorithm, can be used. Suppose that we want to minimize a performance measure (), where (1, 2, , M) is a vector of parameters. In the RM algorithm using PA derivative, the (n 1)th value of the parameter , n1, is determined by (see, e.g., Ref. 41) n + 1 = n n where n ( ) (22)
di j = E
L 1 n=0
[ f (Xn ) f (Xn )]|X0 = i, X0 = j i, j = 1, 2, . . ., M (18)
n ( ) =
n n n ( ), ( ), , ( ) 1 2 M
which is similar to Eq. (14). The matrix D [dij] is called a realization matrix, which satises the Lyapunov equation D PDPT = F where F ef T feT, and e (1, 1, , 1)T is a column vector all of whose components are ones. The performance derivative is = QDT T Q (19)
Since D is skew-symmetric, that is, DT D, we can write D egT geT, where g [g(1), g(2), , g(M)]T is called a potential vector. We have = Qg. Q (20)
g(i) can be estimated on a single sample path by gn(i) n El0[f (Xl)]X0 i. There are a few other methods for estimating g and D by using a single sample path. The potential g satises the Poisson equation (I P + e )g = f (21)
is the estimate of the gradient of the performance function () at n with each component being the PA estimate, and n, n 1, 2, , are the step sizes. It usually requires that 2 n1 n and n1 n . Many results have been obtained in this direction. For example, Ref. 41 studied the optimization of J() T() C() for a single server queues, where T() is the mean system time, C() a cost function, and the mean service time. It was proved that under some mild conditions, the RobbinsMonro type of algoritm (22) converges even if we update using the IPA gradient estimate at any random times (e.g., at every customer arrival time). Other works include Refs. 42, 43, 44, and 45. The optimization procedures using perturbation analysis have been applied to a number of real-world problems. Successful examples include the bandwidth allocation problem in communications, (46,47), and optimization of manufacturing systems (4852). For performance optimization over discrete parameters, for example, in problems of choosing the best transition matrix, we may use the approach of realization matrix and potentials discussed in the last section. It is interesting to note that in this context, PA is equivalent to the Markov decision process (MDP) approach. To see this, let be vector of the steady state probability for the Markov chain with transition matrix P. From the Poisson equation [Eq. (21)], it is easy to prove = Qg (23)
Thus, perturbation realization in a Markov process relates closely to Markov potential theory and Poisson equations. APPLICATIONS: ON-LINE OPTIMIZATION A direct application of perturbation analysis is in the area of stochastic optimization. Because of the complexity of most
The right-hand side of Eq. (23) is the same as that of Eq. (20) except is replaced by . In policy iteration of an MDP problem, we choose the P corresponding to the largest Qg (P P)g (component-wise) as the next policy. This corresponds to choosing the largest / Q in PA, because all the components of and are positive. Therefore, the policy iteration procedure in MDP in fact chooses the steepest direction
611
of the performance measure obtained by PA as the policy in the next iteration. Thus, in this setting, PA is simply a single sample-path-based implementation of MDP. Further research is needed in this direction. Another on-going research related to DEDS optimization is the ordinal optimization technique (53), whose main idea is that by softening the goal of optimization and by comparing different schemes ordinally instead of obtaining the exact performance values, we can dramatically reduce the demand in the accuracy of the estimates.
21. X. R. Cao, First-order perturbation analysis of a single multiclass nite source queue, Performance Evaluation, 7: 3141, 1987. 22. P. Heidelberger et al., Convergence properties of innitesimal perturbation analysis estimates, Management Science, 34: 1281 1302, 1988. 23. W. B. Gong and Y. C. Ho, Smoothed perturbation analysis of discrete event dynamic systems, IEEE Trans. Autom. Control, 32: 858866, 1987. 24. Y. Park and E. K. P. Chong, Distributed inversion in timed discrete event systems, Discrete Event Dynamic Systems: Theory and Applications, 5: 219241, 1995. 25. A. A. Gaivoronski, L. Y. Shi, and R. S. Sreenivas, Augmented innitesimal perturbation analysis: An alternate explanation, Discrete Event Dynamic Systems: Theory and Applications, 2: 121 138, 1992. 26. Y. C. Ho and S. Li, Extensions of perturbation analysis of discrete event dynamic systems, IEEE Trans. Autom. Control, 33: 427 438, 1988. 27. P. Vakili, Using a standard clock technique for efcient simulation, Oper. Res. Lett., 11: 445452, 1991. 28. L. Y. Dai and Y. C. Ho, Structural innitesimal perturbation analysis (SIPA) for derivative estimation of discrete event dynamic systems, IEEE Trans. Autom. Control, 40: 11541166, 1995. maud, Maximal coupling and rare perturbation sensitivity 29. P. Bre analysis, Queueing Systems: Theory and Applications, 10: 249 270, 1992. 30. F. Baccelli, M. Klein, and S. Zuyev, Perturbation analysis of functionals of random measures, Adv. in Appl. Prob., 27: 306325, 1995. 31. F. J. Vazquez-Abad and K. Davis, Efcient implementation of the phantom RPA method, with an application to a priority queueing system, Adv. in Appl. Prob., submitted, 1995. 32. M. A. Zazanis and R. Suri, Perturbation analysis of the GI/G/1 queue, Queueing Systems: Theory and Applications, 18: 199 248, 1994. 33. B. Gang, C. Cassandras, and M. A. Zazanis, First and second derivative estimators for closed Jackson-like queueing networks using perturbation analysis techniques, Discrete Event Dynamic Systems: Theory and Applications, 7: 2968, 1997. 34. X. R. Cao and H. F. Chen, Perturbation realization, potentials, and sensitivity analysis of Markov processes, IEEE Trans. Autom. Control, 42: 13821393, 1997. 35. W. B. Gong and S. Nananukul, Rational interpolation for rare event probabilities, Stochastic Networks: Stability and Rare Event, New York: Springer, 1996. 36. W. B. Gong and J. Q. Hu, On the MacLauring Series of GI/G/1 Queue, J. Appl. Prob., 29: 176184, 1991. 37. X. R. Cao, W. G. Gong, and Y. Wardi, Ill-conditioned performance functions of queueing systems, IEEE Trans. Autom. Control, 40: 10741079, 1995. 38. P. W. Glynn, Likelihood ratio gradient estimation: An overview, Proc. Winter Simulation Conf., 366375, 1987. 39. M. I. Reiman and A. Weiss, Sensitivity analysis via likelihood ratio, Oper. Res., 37: 830, 844, 1989. 40. R. Y. Rubinstein, Sensitivity analysis and performance extrapolation for computer simulation models, Oper. Res., 37: 7281, 1989. 41. E. K. P. Chong and P. J. Ramadge, Optimization of queues using innitesimal perturbation analysis-based algorithms with general update times, SIAM J. Control Optim., 31: 698732, 1993. 42. Y. C. Ho and X. R. Cao, Perturbation analysis and optimization of queueing networks, J. Optim. Theory Appl., 40: 559582, 1983.
BIBLIOGRAPHY
1. Y. C. Ho (ed.), Dynamics of Discrete Event Systems, Proc. IEEE, 77: 1232, 1989. 2. P. J. Ramadge and W. M. Wonham, Supervisory control of a class of discrete event processes, SIAM J. Control Optim., 25: 206 230, 1987. 3. W. H. Fleming, (chair), Future directions in control theoryA mathematical perspective, Report of the Panel on Future Directions in Control Theory, 1988. 4. R. David and H. Alla, Petri nets for modeling of dynamic systemsA survey, Automatica, 30: 175202, 1994. 5. M. C. Fu and J. Q. Hu, Conditional Monte Carlo: Gradient Estimation and Optimization Applications, Norwell, MA: Kluwer Academic Publishers, 1997. 6. X. R. Cao, Realization Probabilities: the Dynamic of Queueing Systems, New York: Springer-Verlag, 1994. 7. C. G. Cassandras, Discrete Event Systems: Modeling and Performance Analysis, Aksen Associates, Inc., 1993. 8. P. Glasserman, Gradient Estimation Via Perturbation Analysis, Norwell, MA: Kluwer Academic Publishers, 1991. 9. Y. C. Ho and X. R. Cao, Perturbation Analysis of Discrete-Event Dynamic Systems, Norwell, MA: Kluwer Academic Publishers, 1991. 10. P. G. Harrison, On normalizing constants in queueing networks, Oper. Res. 33: 464468, 1985. 11. L. Kleinrock, Queueing Systems, vols. I, II, New York: Wiley, 1975. 12. J. Walrand, An Introduction to Queueing Networks, Englewood Cliffs, NJ: Prentice Hall, 1988. 13. S. H. Lu and P. R. Kumar, Distributed scheduling based on due dates and buffer priorities, IEEE Trans. Autom. Control, AC-36: 14061416, 1991. 14. J. L. Peterson, Petri Net Theory and the Modeling of Systems, Englewood Cliffs, NJ: Prentice Hall, 1981. 15. C. Reutenauer, The Mathematics of Petri Nets, London: PrenticeHall, 1990. 16. M. C. Zhou and F. DiCesare, Petri Net Synthesis for Discrete Event Control of Manufacturing Systems, Norwell, MA: Kluwer Academic Publisher, 1993. 17. G. Cohen et al., A linear system theoretical view of discrete event processes and its use for performance evaluation in manufacturing, IEEE Trans. Autom. Control, AC-30: 210220, 1985. 18. F. Baccelli et al., Synchronization and Linearity, New York: Wiley, 1992. 19. Y. C. Ho, A. Eyler, and T. T. Chien, A gradient technique for general buffer storage design in a serial production line, Int. J. Production Res., 17: 557580, 1979. 20. X. R. Cao, Convergence of parameter sensitivity estimates in a stochastic experiment, IEEE Trans. Autom. Control, AC-30: 834 843, 1985.
612
DISCRETE EVENT SYSTEMS
43. C. G. Cassandras and S. G. Strickland, On-line sensitivity analysis of Markov chains, IEEE Trans. Autom. Control, 34: 7686, 1989. 44. R. Suri and Y. T. Leung, Single run optimization of discrete event simulationsan empirical study using the M/M/1 queue, IIE Trans., 21: 3549, 1989. 45. Q. Y. Tang and H. F. Chen, Convergence of perturbation analysis based optimization algorithm with xed number of customers period, Discrete Event Dynamic Systems: Theory and Applications, 4: 359375, 1994. 46. C. A. Brooks and P. Varaiya, Using perturbation analysis to solve the capacity and ow assignment problem for general and ATM networks, IEEE Globcom, 1994. 47. N. Xiao, F. F. Wu, and S. M. Lun, Dynamic bandwidth allocation using innitesimal perturbation analysis, IEEE Infocom, 383 389, 1994. 48. M. Caramanis and G. Liberopoulos, Perturbation analysis for the design of exible manufacturing system ow controllers, Oper. Res., 40: 11071125, 1992. 49. A. Haurie, P. LEcuyer, and C. van Delft, Convergence of stochastic approximation coupled with perturbation analysis in a class of manufacturing ow control models, Discrete Event Dynamic Systems: Theory and Applications, 4: 87111, 1994. 50. H. M. Yan and X. Y. Zhou, Finding optimal number of Kanbans in a manufacturing system via perturbation analysis, Lecture Notes in Control and Information Sciences, 199, Springer-Verlag, 572578, 1994. 51. H. M. Yan, G. Yin, and S. X. C. Lou, Using stochastic optimization to determine threshold values for the control of unreliable manufacturing systems, J. Optim. Theory Appl., 83: 511539, 1994. 52. N. Miyoshi and T. Hasegawa, On-line derivative estimation for the multiclass single-server priority queue using perturbation analysis, IEEE Trans. Autom. Control, 41: 300305, 1996. 53. Y. C. Ho, Soft Optimization for Hard Problems, Computerized lecture via private communication, 1996.
XI-REN CAO
The Hong Kong University of Science and Technology
Abstract : Discrete Time Systems Design Methods : Wiley Encyclopedia of Electrical and Electronics Engineering : Wiley InterScience
G G G G
HOME
G
ABOUT US
G
CONTACT US
G
HELP
Browse this title

G
Discrete Time Systems Design Methods

Standard Article Mohammed S. Santina1 and Allen R. Stubberud2 1Boeing Defense and Space Group, Downey, CA 2University of California, Irvine, Irvine, CA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1005 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (308K)
Search this title
Abstract
The sections in this article are Classical Control System Design Methods Eigenvalue Placement with State Feedback Quadratic Optimal Control Step-Invariant Discrete-Time Observer Design Tracking System Design Simulation
DISCRETE TIME SYSTEMS DESIGN METHODS
643
out the extensive calculations required for their design. These advances in implementation and design capability can be obtained at low cost because of the widespread availability of inexpensive and powerful digital computers and their related devices. The focus of discussion is the modern, state space-based design of linear discrete-time control systems with an appreciation for classical viewpoints and methods. To begin, we present an overview of the classical approach to discrete-time tracking system design. The key concepts, involving specication of transient and steady state response requirements, are also much a part of the modern approach. Two important classes of control systems are the tracking system and the regulator. A tracking system is one in which the plant outputs are controlled so that they become and remain nearly equal to a set of externally applied reference signals. In a regulator, the objective is to bring the systemtracking outputs near zero in an acceptable manner, often in the face of disturbances. Thus regulation is a special case of tracking, in which the externally applied reference signals are zero. In the classical approach, whether in the discrete-time domain or the continuous-time domain, the designer begins with a lower-order controller and raises the controller order as necessary to meet the feedback system performance requirements. The digital controller parameters are chosen to give feedback system pole locations that result in acceptable zeroinput (transient) response. At the same time, requirements are placed on the overall systems zero-state response components for representative discrete-time reference inputs, such as steps or ramps. In general, tracking control system design has two basic concerns: 1. Obtaining acceptable zero-input response, that due to initial conditions 2. Obtaining acceptable zero-state system response to reference inputs In addition, if the plant to be controlled is continuous-time and the controller is discrete-time, a third concern is: 3. Obtaining acceptable between-sample response of the continuous-time plant Using the superposition theorem, the zero-input response, and the individual zero-state response contributions of each input can be dealt with separately. The rst concern of tracking system design is met by selecting a controller that places all of the overall system poles at desired locations inside the unit circle on the complex plane. Having designed a feedback structure to achieve the desired character of zeroinput response, additional design freedom can then be used to obtain good tracking of reference inputs. The rst two concerns of discrete-time tracking system design are the subject of this chapter. The third concern, however, is beyond our scope but is covered thoroughly in Ref. 1. CLASSICAL CONTROL SYSTEM DESIGN METHODS The tools of classical linear discrete-time control system design, which parallel the tools for continuous-time systems, are

This article discusses fundamental concepts in discrete-time control system design. The rapid advancements in digital system technology have radically altered the boundaries of control system design options. Currently, it is routinely practicable to design very complicated digital controllers and to carry
644
+

Plant Gp(z)
R(z)
Compensator Gc(z)
Y(z)
then TE (z ) = 1 E (z ) = R (z ) 1 + KG(z )H (z )
Figure 1. Cascade compensation of a unity feedback system.
Assuming that all the closed-loop poles of the system are inside the unit circle on the z plane, the steady-state error to a power-of-time input is given by the nal value theorem: limit e(k ) = limit (1 z 1 )E (z ) = limit
k z1 z1
the z-transform, stability testing, root locus, and frequency response methods. The standard classical control system design problem is to determine the transfer function Gc(z) of a compensator that results in a feedback tracking system with prescribed performance requirements. The basic system conguration for this problem is shown in Fig. 1. There are many variations on this basic theme, including situations where the system structure is more involved, where there is a feedback transmittance H(z), and where there are disturbance inputs to be considered. Usually, these disturbances are undesirable inputs that the plant should not track. The character of a systems zero-input response is determined by its pole locations, so the rst concern of tracking system design is met by choosing a compensator Gc(z) that results in acceptable pole locations for the overall transfer function: T (z ) = G c ( z )G p ( z ) Y (z ) = R (z ) 1 + G c ( z )G p ( z )
( 1 z 1 )R ( z ) 1 + KG(z )H (z )
(2)
Similar to continuous-time systems, there are three reference inputs for which steady-state errors are commonly dened. They are the step (position), ramp (velocity), and parabolic (acceleration) inputs. The step input has the form r(kT ) = A u(kT ) or, in the z domain, R (z ) = The ramp input is given by r(kT ) = AkTu(kT ) or R (z ) = ATz (z 1 )2 Az z1
Root locus is an important design tool because, with it, the effects on closed-loop system pole location of varying design parameters are quickly and easily visualized. The second concern of tracking system design is obtaining acceptable closed-loop zero-state response to reference inputs. For the discrete-time system shown in Fig. 2, the open-loop transfer function may be expressed as
and the parabolic input is given by r(kT ) = or 1 A(kT )2 u(kT ) 2
KG(z )H (z ) =
K (z + 1 )(z + 2 ) . . . (z + l ) (z 1 )n (z + 1 )(z + 2 ) . . . (z + m ) KN (z ) = (z 1 )n D (z )
(1)
R (z ) =
T 2 Az (z + 1 ) 2 (z 1 )3
If n is nonnegative, the system is said to be type n. The error between the input and the output of the system is E ( z ) = R ( z ) Y ( z )H ( z ) but Y (z ) = KE (z )G(z )
E(z) K G(z) Y(z)
+ R(z)
Table 1 summarizes steady-state errors using Eqs. (1) and (2) for various system types for power-of-time inputs. There are two basic ways of approaching classical discretetime control. In the sampled data approach, discrete-time signals are represented by continuous-time impulse trains so that all signals in a plant and controller model are continuous-time signals. This was appealing in the early days of digital control when digital concepts were new and most designers had backgrounds that were solidly in continuous-time control. Currently, however, there is little to recommend this complexity. In the conventional approach, which is used here, discrete-time signals are represented as sequences of numbers. Root Locus Design Methods
H(z)
Figure 2. A discrete-time control system.
We now present an overview of classical discrete-time control system design using an example. Similar to continuous-time systems, a root locus plot consists of a polezero plot of the open-loop transfer function of a feedback system, upon which

Table 1. Steady-State Errors to Power-of-Time Inputs Steady-State Error to Step Input Az R(z) z1 A 0 1K 0 N(1) D(1) AT N(1) K D(1) 0 0 AT 2 N(1) K D(1) 0 Steady-State Error to Ramp Input ATz R(z) (z 1)2 Steady-State Error to Parabolic Input ATz R(z) (z 1)2
645
System Type
2 n
0 0
is superimposed the locus of the poles of the closed-loop transfer function as some parameter is varied. For the conguration shown in Fig. 2 where the constant gain K is the parameter of interest, the overall transfer function of this system is T (z ) = KG(z ) 1 + KG(z )H (z )
and the poles of the overall system are the roots of 1 + KG(z )H (z ) = 0 which depend on the parameter K. The rules for constructing the root locus of discrete-time systems are identical to the rules for plotting the root locus of continuous-time systems. The root locus plot, however, must be interpreted relative to the z plane. Consider the block diagram of the commercial broadcast videotape-positioning system shown in Fig. 3(a). The transfer
Digital controller E(z) + G c( z ) D/A with S/H Tape motion dynamics G(s) = 40e s + 40
(1/120)s
function G(s) relates the applied voltage to the drive motor armature and the tape speed at the recording and playback heads. The delay term accounts for the propagation of speed changes along the tape over the distance of physical separation of the tape drive mechanism and the recording and playback heads. The pole term in G(s) represents the dynamics of the motor and tape drive capstan. Tape position is sensed by a recorded signal on the tape itself. It is desired to design a digital controller that results in zero steady-state error to any step change in desired tape position. Also, the system should have a zero-input (or transient) response that decays to no more than 10% of any initial value within a 1/30 s interval, which is the video frame rate. The sampling period of the controller is chosen to be T 1/ 120 s in order to synchronize the tape motion control with the 1/60 s eld rate (each frame consists of two elds of the recorded video). As shown in Fig. 3(b), the diagram of Fig. 3(a) has been rearranged to emphasize the discrete-time input
Desired tape position R(z)
Positionvelocity relation 1 s
Tape position
Position sensor A/D 1
(a)
G p( z ) R(z) + E(z) D/A with S/H 1 s P(s) A/D P(z)
G c( z )
G p( z )
(b)
Figure 3. Videotape-positioning system. (a) Block diagram. (b) Relation between discrete-time signals.
646
R(z) and the discrete-time samples P(z) of the tape position. The open-loop transfer function of the system is
R(z) +
E(z)
G c( z ) = K
1 e (1/120 )s 40e (1/120 )s 1 G (z ) = Z s s + 40 s 1 1/40 (1 /120 )s (1 /120 )s 1/40 + 2+ = Z [1 e ]e s s s + 40 z/40 z/120 z/40 = ( 1 z 1 )z 1 + + z1 (z 1 )2 z 0.72 0.00133 (z + 0.75 ) = z (z 1 )(z 0.72 )
The position error signal, in terms of the compensators ztransfer function Gc(z), is given by
G p( z ) = 0.00133 (z + 0.75) z (z 1) (z 0.72)
P(z)
(a)
Im Unit circle K = 1000
E (z ) = R (z ) Y (z ) = 1 1 = R (z ) 1 + G c ( z )G p ( z )
G c ( z )G p ( z ) R (z ) 1 + G c ( z )G p ( z )
0.75
K K= 1000 K=100
95 0.00133 Re K 10
0.72
For a unit step input sequence we have E (z ) = 1 1 + G c ( z )G p ( z ) z z+1
Assuming that the feedback system is stable, we obtain limit e(k ) = limit (1 z 1 )E (z ) = limit
k z1 z1
Circle of radius 0.56 (b)
1 1 + G c ( z )G p ( z )
Provided that the compensator does not have a zero at z 1, the system type is 1 and, therefore according to Table 1, the steady-state error to a step input is zero. For the feedback system transient response to decay at least by a factor 1/10 within 1/30 s, the desired closed loop poles must be located such that a decay of at least this amount occurs every 1/120 s steps. This implies that the closed-loop poles must lie within a radius c of the origin on the z plane, where c 4 = 0.1, c = 0.56
Im
Unit circle
0.00133 0.75 0.72 Re
Similar to continuous-time systems, one usually begins with the simplest compensator consisting of only a gain K. The feedback system is stable for 0 K 95; but as shown in Fig. 4, this compensator is inadequate because there are always poles at distances from the origin greater than the required c 0.56 regardless of the value of K. As shown in Fig. 5(a), another compensator with z-transfer function K (z 0.72 ) Gc ( z ) = z which cancels the plant pole at z 0.72 is tried. The root locus plot for this system is shown in Fig. 5(b). For K 90, the design is close to meeting the requirements, but it is not quite good enough. However, if the compensator pole is moved from the origin to the left as shown in Fig. 6, the root locus is pulled to the left and the performance requirements are met. For the compensator with z-transfer function 150 (z 0.72 ) Gc ( z ) = z + 0.4 (3)
(c)
Figure 4. Constant-gain compensator. (a) Block diagram. (b) Root locus for positive K. (c) Root locus for negative gain.
the feedback system z-transfer function is T (z ) = 0.2 (z + 0.75 ) G c ( z )G p ( z ) = 3 1 + G c ( z )G p ( z ) z 0.6z2 0.2z + 0.15
As expected, the steady-state error to a step input is zero: limit

z1
z 3 0.6z 2 0.4z =0 z3 0.6z2 0.2z + 0.15
The steady-state error to a unit ramp input is
1 ( z 2 + 0.4z ) 1 120 = limit 3 z 1 z 0.6z2 0.2z + 0.15 30

G c( z ) = K(z 0.72) z G p( z ) = 0.00133 (z + 0.75) z (z 1) (z 0.72)
647
R(z) +
E(z)
P(z)
Im
Circle of radius 0.56 Unit circle a = 0.2
(a)
Circle of radius 0.56 K=1000
Im
a = 0.4
K = 1000 K K 90 370 Unit circle 0.00133 Re K 90
1 Re a = 0.6
a = 0.6
0.75 K
370
(b)
Figure 7. Root locus plot as a function of the compensator pole location.
Figure 5. Compensator with zero at z 0.72 and pole at z 0. (a) Block diagram. (b) Root locus for positive K.
For a compensator with a z-transfer function of the form we obtain Gc ( z ) = 150 (z 0.72 ) z+a
The feedback system has the z-transfer function: T (z ) = 0.2 (z + 0.75 ) G c ( z )G p ( z ) = 3 1 + G c ( z )G p ( z ) z z2 + 0.2z + 0.15 + a(z2 z )
A root locus plot in terms of positive a is shown in Fig. 7, from which it is seen that choices of a between 0.4 and 0.5 give a controller that meets the performance requirements. Classical discrete-time control system design is an iterative process just like its continuous-time counterpart. Increasingly complicated controllers are tried until both the steadystate error and transient performance requirements are met. Root locus is an important tool because it easily indicates qualitative closed-loop system pole locations as a function of a parameter. Once feasible controllers are selected, root locus plots are rened to show quantitative results. Frequency Domain Methods
R(z)
E(z)
G c( z ) = K(z 0.72) z + 0.4
G p( z ) = 0.00133 (z + 0.75) z ( z 1) ( z 0.72)
P(z)
(a)
Frequency response characterizations of systems have long been popular because of the ease and practicality of steadystate sinusoidal response methods. Furthermore, frequency response methods do not require explicit knowledge of system transfer function models. For the videotape-positioning system, the open loop ztransfer function, which includes the compensator given by Eq. (3), is G c ( z )G p ( z ) = (150 )(0.00133 )(z + 0.75 ) z (z + 0.4 )(z 1 )
Im Circle of radius 0.56 K = 1000 K = 150 0.75 0.4 K = 1000 Unit circle K = 150 0.00133 Re
Substituting z ejT, we obtain G c ( e j T )G p ( e j T ) =

j T (e j T
0.1995 (e j T + 0.75 ) + 0.4 )(e j T 1 )
(4)
(b)
Figure 6. Compensator with zero at z 0.72 and pole at z 0.4. (a) Block diagram. (b) Root locus for positive K.
which has the frequency response plots shown in Fig. 8. At the phase crossover frequency (114.2 rad/s) the gain margin is about 11.5 dB, and at the gain crossover frequency (30 rad/s) the phase margin is about 66.5s. For ease of generating frequency response plots and to gain greater insight into the design process, the frequency domain methods such as Nyquist, Bode, Nichols, and so on, for discrete-time systems can be developed using the w transform. This is because in the w plane the wealth of tools and
648
50
6. Finally, transform the controller Gc(w) to Gc(z) according to the mapping

Gain margin
Gain (dB)
w=
z1 z+1
50 1 10
100
101 Frequency (rad/s)
102
103
90 180 270 360 101 100 101 Frequency (rad/s) 102 103 Phase margin
Figure 8. Frequency response plots of the videotape-positioning system.
techniques that were developed for continuous-time systems are directly applicable to discrete-time systems as well. The w transform is given by w= z1 , z+1 z= w+1 1w
Control system design for discrete-time systems using Bode, Nyquist, or Nichols methods can be found in Refs. 2 and 3. Frequency response methods are most useful in developing models from experimental data, in verifying the performance of a system designed by other methods, and in dealing with those systems and situations in which rational transfer function models are not adequate. The extension of the classical single-input/single-output control system design methods to the design of complicated feedback structures involving many loops, each of which might include a compensator, is not easy. Put another way, modern control systems require the design of compensators having multiple inputs and multiple outputs. Design is iterative, and it can involve considerable trial and error. Therefore when there are many design variables, it is important to deal efciently with those design decisions that need not be iterative. The powerful methods of state space offer insights about what is possible and what is not. They also provide an excellent framework for general methods of approaching and accomplishing design objectives. EIGENVALUE PLACEMENT WITH STATE FEEDBACK Consider a linear, step-invariant nth-order system described by the state equations x (k + 1 ) = Ax (k ) + Bu (k ) where x(k) is an n vector, and u(k) is an r vector. When the state x(k) of this system is available and is used for feedback, the input vector u(k) is given the form u (k ) = Ex (k ) + (k )
which is a bilinear transformation between the w plane and the z plane. The general procedure for analyzing and designing discrete-time systems using the w transform is summarized as follows: 1. Replace each z in the open-loop transfer function G(z)H(z) with z= to obtain G(w)H(w). 2. Substitute w j into G(w)H(w) and generate frequency response plots in terms of the real frequency , such as Nyquist, Bode, Nichols, and so on. The w plane can be thought of as if it were the s plane. 3. Determine the stability margins, crossover frequencies, bandwidth, closed-loop frequency response, or any other desired frequency response characteristics. 4. If it is necessary, design a compensator Gc(w) to satisfy the frequency domain performance requirements. 5. Convert critical frequencies in the w plane to frequencies in the z domain according to w+1 1w
Phase (deg)
where (k) is a vector of external inputs as shown in Fig. 9, and E is a gain matrix. The state equation for the plant with feedback becomes x (k + 1 ) = (A + BE )x (k ) + B (k ) If the plant is completely controllable, the eigenvalues of the feedback system, those of A BE, can be placed at any desired locations selected by the designer by appropriately choosing the feedback gain matrix E. This is to say that the
Plant
(k)
+ +
u(k)
x (k + 1) = Ax(k) + Bu(k)
2 = tan 1 T
Figure 9. State feedback.
649
designer can freely choose the character of the overall systems transient performance. When the plant state vector is not available for feedback, as is usually the case, an observer can be designed to estimate the state vector. As we shall see later, the observer state estimate can be used for feedback in place of the state itself. Eigenvalue Placement for Single-Input Systems If a single-input plant is in controllable canonical form, nding the feedback gains for arbitrary eigenvalue placement is especially simple because, in that form, each element of the feedback gain vector determines one coefcient of the feedback systems characteristic equation. In general, however, the plant is not in controllable form. One way for calculating the state feedback gain for eigenvalue placement for plants that are not in controllable canonical form is to transform the plant to controllable form, calculate the state feedback gain for the transformed system, and then transform the state feedback gain of the transformed system back to the original system (see Ref. 1). There are a number of other methods for nding the state feedback gain vector of single-input plants. Two of these methods are summarized below. Additional ones can be found in Refs. 1 and 48. The state feedback gain vector is given by Ackermanns formula:
1 e = jn M c c (A )
Letting Ev i = i we obtain v i = (iI A ) 1 B i If i is not an eigenvalue of A, the inverse matrix exists. If the eigenvalues i are distinct, the eigenvectors are linearly independent. Choosing the i to give n linearly independent eigenvectors, we obtain E [v 1v 2 . . . v n ] = EV = = [ 1 2 n ]
and the desired feedback gain matrix is E = V 1 (6)
For the previous example, choosing 1 1 and 2 1, we obtain e = [1.5 0]
(5)
where j n is the transpose of the nth-unit coordinate vector

jn = [0
which is the same result obtained using Ackermanns formula. The results of the above development can be extended to situations where A BE is required to have repeated eigenvalues as discussed in Ref. 1. Eigenvalue Placement with Multiple Inputs If the plant for eigenvalue placement has multiple inputs and if it is completely controllable from one of the inputs, then that one input alone can be used for feedback. If the plant is not completely controllable from a single input, a single input can usually be distributed to the multiple ones in such a way that the plant is completely controllable from the single input. For example, for the system
...
1]
Mc is the controllability matrix of the system, and c(A) is the desired characteristic equation with the matrix A substituted for the variable z. For example, for the completely controllable system
x1 ( k + 1 ) 1 = x2 ( k + 1 ) 3
1 0
x1 ( k ) 1 + u (k ) x2 ( k ) 2
it is desired to place the feedback system eigenvalues at z 0, 0.5. Then,

c (z )
= z ( z + 0.5 ) = z 2 + 0.5z
and
c (A )
x1 ( k + 1 ) 0.5 0 1 x1 ( k ) 0.5 2 x2 (k ) x2 (k + 1 ) = 0 x3 ( k + 1 ) x3 ( k ) 1 1 0 1 0 u1 (k ) + 0 2 u2 (k ) 1 1
we let u1 (k ) = 3(k ) and u2 (k ) = (k ) Thus
= A 2 + 0.5A
Using Ackermanns formula, the state feedback gain vector is
e = [0 = [1.5
1] 0]
1 2
1 3
2 3
1 1 + 0.5 3 3
1 0
Yet another method for calculating E is as follows. If i is an eigenvalue of (A BE), then there exists an eigenvector vi such that (A + BE )v i = iv i
x1 ( k + 1 ) 0.5 x2 (k + 1 ) = 0 x3 ( k + 1 ) 1
0 0.5 1
1 3 x1 ( k ) 2 x2 (k ) + 2 (k ) x3 ( k ) 0 2
650
which is a controllable single input system. If the desired eigenvalues are located at z1 0.1, z2 0.15, and z3 0.1, Ackermanns formula gives e = [0.152 0.0223 0.2807]
and hence the feedback gain matrix for the multiple input system is
E=
0.4559 0.1520
0.0669 0.0223
0.8420 0.2807
Equation (6) can also be applied to multiple input systems. Continuing with the example, if for each eigenvector, we choose 1 3 and 2 1, then
will place the feedback system eigenvalues arbitrarily, provided that the matrix I EC1D is nonsingular. If it is singular, a small change in the feedback gain matrix E which corresponds to small changes in the desired feedback system eigenvalue locations eliminates the singularity. If the nth-order system has more than n outputs, only n of these can be linearly independent, so excess linearly dependent output equations can simply be ignored when recovering a systems state from its output. To improve a feedback systems reliability and its performance in the presence of noise, one may wish instead to combine linearly dependent outputs with other outputs rather than ignore them. When the nth-order plant does not have n linearly independent measurement outputs, it still might be possible to select a feedback matrix E in u (k ) = E y (k ) Du (k ) + (k ) = ECx (k ) + (k ) to place all of the feedback system eigenvalues, those of (A BEC), acceptably. Generally, however, output feedback alone does not allow arbitrary feedback system eigenvalue placement. Pole Placement with Feedback Compensation We now present another viewpoint for placing the feedback system poles using a transfer function approach. Although our discussion is limited to single-input and single-output plants, the results can be generalized to the case of plants with multiple inputs and multiple outputs. Similar to output feedback, pole placement with feedback compensation assumes that the measurement outputs of a plant, not the state vector, are available for feedback. For an nth-order, linear, step-invariant, discrete-time system described by the transfer function G(z), arbitrary pole placement of the feedback system can be accomplished with an mth-order feedback compensator as shown in Fig. 10. Let the numerator and denominator polynomials on G(z) be Np(z) and Dp(z), respectively. Also, let the numerator and denominator of the compensator transfer function H(z) be Nc(z) and Dc(z), respectively. Then, the overall transfer function of the system is T (z ) = Np (z )Dc (z ) P (z ) G (z ) = = 1 + G ( z )H ( z ) Dp (z )Dc (z ) + Np (z )Nc (z ) Q (z )
6.5871 v 1 = 4.5506 , 0.3652

and therefore
6.9304 v 2 = 4.8442 , 0.5744
5.5076 v 3 = 3.4772 0.3046
3 E= 1
3 1
3 0.4559 V 1 = 1 0.1520
0.0669 0.0223
0.8420 0.2807
which agrees with the previous results. Eigenvalue Placement with Output Feedback It is the measurement vector of a plant, not the state vector, that is available for feedback. We now consider what eigenvalue placement can be performed with output feedback alone. With enough linearly independent outputs, the plant state can be recovered from the outputs and inputs and the state feedback results applied. With a single plant input and only a few outputs, the designers options for placing feedback system eigenvalues could be (and often are) severely limited. Multiple plant inputs can also be used to advantage for eigenvalue placement with output feedback, but it still may not be possible to achieve an acceptable design. If the nth-order plant with state and output equations x (k + 1 ) = Ax (k ) + Bu (k ) y (k ) = Cx (k ) + Du (k ) has n linearly independent outputs, that is, if the output coupling matrix C has n linearly independent rows, then the plant state can be recovered from the plant inputs and the measurement outputs: x (k ) = C 1 {y (k ) Du (k )} The output feedback u (k ) = Ex (k ) + (k ) = EC 1 {y (k ) Du (k )} + (k ) or u (k ) = (I + EC 1D ) 1EC 1y (k ) + (I + EC 1D ) 1 (k )
Plant with feedback T(z) Plant P(z) + G(z) = N p( z ) D p( z ) Y(z)
Feedback compensator N c( z ) H(z) = D c( z )
Figure 10. Pole placement with feedback compensation.
651
which has closed-loop zeros in P(z) that are those of the plant, in Np(z), together with zeros that are the poles of the feedback compensator, in Dc(z). For a desired set of poles of T(z), given with an unknown multiplicative constant by the polynomial Q(z), we obtain Dp (z )Dc (z ) + Np (z )Nc (z ) = Q(z ) The desired polynomial Q(z) has the form Q ( z ) = 0 ( z n + m + n + m 1 z n + m 1 + + 1 z + 0 ) where the s are known coefcients but the 0 is unknown. In general, for a solution to exist there must be at least as many unknowns as equations: n + m + 1 2m + 2 or m n1 (8) (7)
Therefore, the compensator H (z ) = 0.325z 0.00265 z + 0.1185
will place the closed-loop poles where desired. As far as feedback system pole placement is concerned, a feedback compensator of order n 1 (where n is the order of the plant) can always be designed. It is possible, however, that a lower-order feedback controller may give acceptable feedback pole locations even though those locations are constrained and not completely arbitrary. This is the thrust of classical control system design, in which the increasingly higher-order controllers are tested until satisfactory results are obtained. For the plant given by Eq. (9), for example, a zeroth-order feedback controller of the form H (z ) = K gives overall closed-loop poles at z 0.1428 and z 0.5 for K 1/6, which might be an adequate pole placement design. QUADRATIC OPTIMAL CONTROL We have shown in the previous section that provided the plant is completely controllable, a feedback gain matrix E can always be determined so that all of the eigenvalues of the feedback system can be placed arbitrarily. It can be easily shown that for a single-input plant, the feedback gain vector is unique. For multiple-input plants, however, there are many feedback gain matrices that lead to the same set of feedback eigenvalues. The process of selecting an optimum feedback gain matrix from among the many possible gain matrices is the subject of this section. The approach of selecting the optimal gain matrix is termed optimal regulation, in which the plant feedback is chosen to minimize a scalar performance measure that weights the control input and the error from zero of the plant state at each step. Principle of Optimality The discrete-time, linear-quadratic, optimal control problem is to nd the inputs u(0), u(1), . . ., u(N 1) to the plant with linear state equations x (k + 1 ) A (k )x (k ) + B (k )u(k ) such that a scalar quadratic performance measure (or cost function)
where n is the order of the plant and m is the order of the compensator. Equation (8) states that the order of the feedback controller is at least one less than the order of the plant. If the plant transfer function has coprime numerator and denominator polynomials (that is, plant polezero cancellations have been made), then a solution is guaranteed to exist. For example, consider the second-order plant G (z ) = Np (z ) (z + 1 )(z + 0.5 ) = z (z 1 ) Dp ( z ) (9)
According to Eq. (8), a rst-order feedback compensator of the form H (z ) = 1 z + 2 Nc (z ) = z + 3 Dc ( z )
places the three closed-loop poles of the feedback system at any desired location in the z plane by appropriate choice of 1, 2, and 3. Let the desired poles of the plant with feedback be at z 0.1. Then, Q(z ) = 0 (z 0.1 )3 = 0 (z3 0.3z2 + 0.03z 0.001 ) (10)
In terms of the compensator coefcients, the characteristic equation of the feedback system is
Dp (z )Dc (z ) + Np (z )Nc (z ) = z (z 1 )(z + 3 ) + (z + 1 )(z + 0.5 )(1z + 2 ) = (1 + 1 )z3 + (1.51 + 2 + 3 1 )z2 + ( 0 . 5 1 + 1 . 5 2 3 )z + 0 . 5 2 (11)
Equating coefcients in Eqs. (10) and (11) and solving for the unknowns gives 0 = 1.325, 1 = 0.325, 2 = 0.00265, 3 = 0.1185
J = x (N )P (N )x (N ) +
N 1 k=0
[x (k )Q (k )x (k ) + u (k )R (k )u (k )]
is minimized. The matrix P(N), the matrices Q(0), Q(1), . . ., Q(N 1), and the matrices R(0), R(1), . . ., R(N 1) are each taken to be symmetric because each denes a quadratic form. Each is assumed to be positive semidenite, which means that the contribution to J by each of the individual terms is never negative. The solution to the linear-quadratic optimal control problem is obtained by applying the principle of optimality, a tech-
652
Table 2. Procedure for Backward-in-Time Calculation of Optimal Quadratic Regulator Gains For the plant x(k 1) A(k)x(k) B(k)u(k) with state feedback u(k) E(k)x(k) and performance measure J x(N)P(N)x(N)
N1 i0
[x (i)Q(i)x(i) u (i)R(i)u(i)]

begin with i 1 and the known P(N) 1. E(N i) [B(N i)P(N 1 i)B(N i) R(N i)]1 B(N i)P(N 1 i)A(N i) 2. P(N i) [A(N i) B(N i)E(N i)]P(N 1 i) [A(N i) B(N i)E(N i)] E(N i)R(N i)E(N i) Q(N i) 3. Increment i and repeat steps 1, 2, and 3 until E(0) and (if desired) P(0) have been calculated. The minimum performance measure is min J x(0)P(0)x(0) u(0), . . ., u(N 1)
nique developed by Richard Bellman in the 1950s in connection with his invention of dynamic programming. To apply the principle of optimality, one begins at the next-to-last step N 1 and nds the last input u(N 1) that minimizes the cost of control from step N 1 to step N, J(N 1, N), as a function of the beginning state for that step, x(N 1). Then the input u(N 2) is found that minimizes J(N 2, N) when u(N 1) is as previously determined. One proceeds in this manner nding one control vector at a time, from the last to the rst, as a function of the systems state. This results in a recursive calculation of the optimal feedback gains for the linear-quadratic regulator as given in Table 2. Beginning with known N, P(N), Q, and R, the last feedback gain matrix E(N 1) is calculated. Using E(N 1), the matrix P(N 1) is computed. Then all of the indices are stepped backward one step and, with P(N 1), the feedback gain matrix E(N 2) is calculated. Using E(N 2), we can calculate P(N 2). The cycle is continued until E(0) is found. A formidable amount of algebraic computation is required; the user should therefore have digital computer aid for all but the lowest-order problems. Closed-Form Solution for Optimal Gain When the matrices A, B, Q, and R are constants, it is possible to generate an analytical expression for the optimal gain from which the numerical value can be calculated for any point of time. A closed-form solution for the optimal gain E(N 1) is summarized in Table 3. Derivation of this procedure along with detailed numerical examples can be found in Ref. 1. Steady-State Regulation For a completely controllable, step-invariant plant and constant cost weighting matrices Q and R, the optimum feedback gains, from E(N) backward, are not changed if the nal step N is changed. This is to say that the sequence of gains start-
ing at N and proceeding backward is always the same independent of the value of N. The procedure summarized in Table 3 for calculating the optimal regulator gain can be easily adapted to the steady-state regulator gain by replacing steps 5, 6, and 7 with the following single step 5: 5. Form
1 1 1 E = [B W 11W 21 B + R ] B W 11W 21 A
which gives a steady-state gain matrix. STEP-INVARIANT DISCRETE-TIME OBSERVER DESIGN In 1964, David Luenberger of Stanford University put forth the idea of observers, systems that recursively estimate the state of other systems. It was soon realized that observers offer a powerful, unied framework for feedback control system design. Full-Order Observers When the plant state vector is not entirely available for feedback, as is usually the case, the state is estimated with an observer, and the estimated state can be used in place of the actual state for feedback (see Refs. 9, 10). For an nth-order step-invariant discrete-time plant, x (k + 1 ) = Ax (k ) + Bu (k ) y (k ) = Cx (k ) + Du (k ) (12)
another nth-order system, driven by the inputs and outputs of the plant, of the form (k + 1 ) = F (k ) + Gy (k ) + Hu (k ) (13)
653
Table 3. Procedure for Calculation of Optimal Regulator Gains For the nth-order plant x(k 1) Ax(k) Bu(k) with state feedback u(k) E(k)x(k) and performance measure J x(N)P(N)x(N)
N1 i0
or x (k ) (k ) = (A GC )k [x (0 ) (0 )] = F k [x (0 ) (0 )] then the system in Eq. (13) is a full-order state observer of the plant in Eq. (12), if the matrix G can be chosen so that all the eigenvalues of F A GC are inside the unit circle in the complex plane. The observer error, then, approaches zero with step regardless of the initial values of x(0) and (0). That is, the observer state (k) will approach the plant state x(k). The full-order observer relations are summarized in Table 4. If all n of the observer eigenvalues (eigenvalues of F) are selected to be zero, then the characteristic equation of F is n = 0 and since every matrix satises its own characteristic equation, then Fn = 0 At the nth step, the error between the plant state and the observer state is given by x (n ) (n ) = F n [x (0 ) (0 )] so that x (n ) = (n ) and the observer state equals the plant state. Such an observer is termed deadbeat. In subsequent steps, the observer state continues to equal the plant state. There are several methods for calculating the observer gain matrix g for single-output plants. Similar to the situation with state feedback, if a single-output plant is in observable canonical form, nding the elements of the observer gain vector g for arbitrary eigenvalue placement is simple, because each element of the observer gain vector determines one coefcient of the observer characteristic equation. Usually, however, the plant is not in observable canonical form. One way of designing an observer for a completely observable singleoutput plant that is not in observable form is to change the
[x (i)Qx(i) u (i)Ru(i)]

begin with i 1 and the known P(N) 1. Form the matrix H
A QA1BR1B A1BR1B
QA1 A1
2. Find the eigenvalues and the corresponding eigenvectors of H 3. Generate the matrix W from eigenvectors such that W1HW D
0 1
where is the diagonal matrix of eigenvalues outside the unit circle on the z plane 4. Partition W into four n n submatrices as W
W11 W21
W12 W22
5. Form G(i) i[P(N)W22 W12]1[W11 P(N)W21]i 6. Form P(N i) [W11 W12G(i)][W21 W22G(i)]1 7. Form E(N i) [BP(N 1 i)B R]1BP(N 1 i)A where P(N 1 i) [W11 W12G(i 1)][W21 W22G(i 1)]1
is termed a full-order state observer of the plant, provided that the error between the plant state and the observer state,
x (k + 1 ) (k + 1 ) = Ax (k ) + Bu (k ) F (k ) Gy (k ) Hu (k ) = Ax (k ) + Bu (k ) F (k ) GCx (k ) GDu (k ) Hu (k ) = (A GC )x (k ) F (k ) + (B GD H )u (k )
is governed by an autonomous equation. When F and G are chosen as F = A GC H = B GD so that the error satises x (k + 1 ) (k + 1 ) = (A GC )[x (k ) (k )] (14) (15)
Table 4. Full-Order State Observer Relations Plant Model x(k 1) Ax(k) Bu(k) y(k) Cx(k) Du(k) Observer
(k 1) F(k) Gy(k) Hu(k)

where F A GC H B GD Observer Error x(k 1) k 1) F[x(k) (k)] x(k) (k) F k[x(0) (0)]
654
plant to the observable form, design an observer in that form, and then convert back to the original system realization. Another method of observer design for single-output plants that does not require transforming the system to observable canonical form is to use Ackermanns formula. Provided that (A, c) is completely observable, the eigenvalues of F A gc can be placed arbitrarily by choice of g according to Ackermanns formula: g=
1 0 (A )M 0 j n
Reduced-Order State Observers If a completely observable nth-order plant has m linearly independent outputs, a reduced-order state observer, of order n m, having an output that observes the plant state can be constructed. For the plant described by Eq. (12), when an observers state (k + 1 ) = F (k ) + Gy (k ) + Hu (k ) estimates a linear combination Mx(k) of the plant state rather than the state itself, the error between the observer state and the plant state transformation is given by
(16)
provided that (A, c) is completely observable. In Eq. (16), 0(A) is the desired characteristic equation of the observer eigenvalues with the matrix A substituted for the variable z, M0 is the observability matrix
Mx (k + 1 ) (k + 1 ) = MAx (k ) + MBu (k ) F (k ) Gy (k ) Hu (k ) = (MA GC )x (k ) F (k ) + (MB GD H )u (k )

where M is m n. For the observer error system to be autonomous, we require FM = MA GC H = MB GD so that the error is governed by Mx (k + 1 ) (k + 1 ) = F [Mx (k ) (k )] For a completely observable plant, the observer gain matrix g can always be selected so that all the eigenvalues of F are inside the unit circle on the complex plane. Then the observer error Mx (k ) (k ) = F k [Mx (0 ) (0 )] will approach zero asymptotically with step and then (k ) Mx (k ) If the plant outputs, which also involve linear transformation of the plant state, are used in the formulation of a state observer, the dynamic order of the observer can be reduced. For the nth-order plant given by Eq. (12) with the m rows of C linearly independent, an observer of order n m with n outputs (17)
c ... c A ... 2 c A M0 = ... . . . ... c A n

and jn is the nth-unit coordinate vector
0 . . . 0 . . . 0 jn = . . . . . . . . . 1
Another popular form of a full-order observer which is viewed as an error feedback system can be obtained by expressing the observer Eqs. (13), (14), and (15) in the form (k + 1 ) = (A GC ) (k ) + Gy (k ) + (B GD )u (k ) = A (k ) + Bu (k ) + G [y (k ) w (k )] where w (k ) = C (k ) + Du (k ) Here, the observer consists of a model of the plant driven by the input u(k) and the error between the plant output y(k) and the plant output that is estimated by the model w(k). This form of a full-order observer is similar to the Kalman Bucy lter (see Ref. 1).
W (k ) =
observes
0 I D (k ) + y (k ) + u (k ) I 0 0
W (k )
C x (k ) = Nx (k ) M
Except in special cases, the rows of M and the rows of C are linearly independent. If they are not so, slightly different observer eigenvalues can be chosen to give linear independence. Therefore, the observer output w (k ) = N 1w (k ) observes x(k).
DISCRETE TIME SYSTEMS DESIGN METHODS D + y(k)
655
(k) = 0
+
u(k) +
+ B
x(k+1) + Unit delay x(k) C
+ A D + (k+1) + + C G
+ +
Unit delay A
(k)
Figure 11. Eigenvalue placement with full-order state observer feedback.
Eigenvalue Placement with Observer Feedback When observer feedback is used in place of plant state feedback, the eigenvalues of the feedback system are those the plant would have if the state feedback were used and those of the observer. This result is known as the separation theorem for observer feedback. For a completely controllable and completely observable plant, an observer of the form (k + 1 ) = F (k ) + Gy (k ) + Hu (k ) w (k ) = L (k ) + N [y(k ) Du (k )] with feedback to the plant given by u (k ) = Ew (k ) (20) (18) (19)
can be designed such that the overall feedback system eigenvalues are specied by the designer. The design procedure proceeds in two steps. First, the state feedback is designed to place the n-state feedback system eigenvalues at desired locations as if the state vector were accessible. Second, the state feedback is replaced by feedback of an observer estimate of the same linear transformations of the state. As an example of eigenvalue placement with observer feedback, Figure 11 shows eigenvalue placement with full order state observer. The eigenvalues of the overall system are those of the state feedback and those of the full-order observer. TRACKING SYSTEM DESIGN The second concern of tracking system design, that of obtaining acceptable zero-state system response to reference inputs, is now discussed. It is assumed that the rst concern of tracking system designnamely, satisfactory zero-input response by feedback system eigenvalues placementhas been achieved.
A tracking system in which the plant outputs are controlled so that they become and remain nearly equal to externally applied reference signals r(k) is shown in Fig. 12(a). The outputs y(k) are said to track or follow the reference inputs. As shown in Fig. 12(b), a linear, step-invariant controller of a multiple-input/multiple-output plant is described by two transfer function matrices: one relating the reference inputs to the plant inputs, and the other relating the output feedback vector to the plant inputs. The feedback compensator is used for shaping the plants zero-input response by placing the feedback system eigenvalues at desired locations as was discussed in the previous subsections. The input compensator, on the other hand, is designed to achieve good tracking of the reference inputs by the system outputs. The output of any linear system can always be decomposed into two parts: the zero-input component due to the initial conditions alone, and the zero-state component due to the input alone. That is, y (k ) = y zero input (k ) + y zero state (k ) Basically, there are three methods for tracking system design: 1. Ideal tracking system design 2. Response model design 3. Reference model design Ideal Tracking System Design In this rst method, ideal tracking is obtained if the measurement output equals the tracking input: y zero state (k ) = r (k )
656

Reference inputs r(k) Controller
Plant inputs u(k)
Plant
Tracking and feedback outputs y(k)
An input compensator or a reference input lter, as shown in Fig. 12(d), with transfer function matrix G(z), for which (z ) = G (z )R (z ) gives
(a)
Y (z ) = T (z )G (z )R (z ) Ideal tracking is achieved if
Controller r(k)
T (z )G (z ) = I
Input compensator
(k)
u(k) Plant y(k)
Feedback compensator
(b)
r(k)
Input (k) compensator G r( z )
u(k) Plant
y(k)
where I is the identity matrix with dimensions equal to the number of reference inputs and tracking outputs. This is to say that ideal tracking is obtained if the reference input lter is an inverse lter of the plant with feedback. Reference input lters do not change the eigenvalues of the plant with feedback which are assumed to have been previously placed with output or observer feedback. When a solution exists, ideal tracking system design achieves exact zero-state tracking of any reference input. Because it involves constructing inverse lters, the ideal tracking system design may require unstable or noncausal lters. An ideal tracking solution can also have other undesirable properties, such as unreasonably large gains, high oscillatory plant control inputs, and the necessity of canceling plant poles and zeros when the plant model is not known accurately. Response Model Design
Feedback compensator
When ideal tracking is not possible or desirable, the designer can elect to design response model tracking, for which T (z )G (z ) = (z )
(c)
Reference inputs Reference r(k) input filter G r( z )
Plant inputs r(k)
Plant with feedback T(z)
Tracking outputs y(k)
(d)
Figure 12. Controlling a multiple-input/multiple-output plant. (a) The output y(k) is to track the reference input r(k). (b) A tracking system using the reference inputs and plant outputs. (c) Representing a controller with a feedback compensator and an input compensator. (d) Feedback compensator combined with plant to produce a plantwith-feedback transfer function matrix T(z).
where the response model z-transfer function matrix (z) characterizes an acceptable relation between the tracking outputs and the reference inputs. Clearly, the price one pays for the added design freedom of a reference model can be degraded tracking performance. However, performance can be improved by increasing the order of the reference input lter. Response model design is a generalization of the classical design technique of imposing requirements for a controllers steady-state response to power-of-time inputs. The difculty with the response model design method is in selecting suitable model systems. For example, when two or more reference input signals are to be tracked simultaneously, the response model z-transfer functions to be selected include not only those relating plant tracking outputs and the reference inputs they are to track, but also those relating unwanted coupling between each tracking output and the other reference inputs. Reference Model Tracking System Design The awkwardness of the practical response model performance design arises because of the difculty in relating performance criteria to the z-transfer functions of response models. An alternative design method models the reference input signals r(k) instead of the system response. This method, termed the reference model tracking system design, allows
The tracking outputs y(k) have initial transient errors due to any nonzero plant initial conditions; after that they are equal to the reference inputs r(k), no matter what these inputs are. As shown in Fig. 12(c), if the plant with feedback has the z-transfer function matrix T(z) relating the tracking output to the plant inputs, then (z ) Y (z ) = T (z )
657
the designer to specify a class of representative reference inputs that are to be tracked exactly, rather than having to specify acceptable response models for all the possible inputs. In the reference model tracking system design, additional external input signals r(k) to the composite system are applied to the original plant inputs and to the observer state equations so that the feedback system is, instead of being described by Eqs. (18), (19), and (20), described by Eq. (12) and (k + 1 ) = F (k ) + Gy (k ) + Hu (k ) + Jr (k ) w (k ) = L (k ) + N [y (k ) Du (k )] with u (k ) = Ew (k ) + Pr (k ) (23) (21) (22)
Observer of the signal model
Fictitous autonomous reference signal model
r(k)
Plant with observer feedback
y(k)
Figure 13. Observing a reference signal model.
In the reference model tracking system design, the concept of an observer is used in a new way; it is the plant with feedback that is an observer of the ctitious reference input model system as shown in Fig. 13. When driven by r(k), the state of the composite system observes x (k ) M (k ) where M satises, according to Eq. (17), M A M = B (25)
Then, the overall composite system has the state equations
x (k + 1 ) A + BENC = (k + 1 ) GC + H ENC +
BEL F + H EL
x (k ) (k )
BP r (k ) H P +J
= A x (k ) + B r (k )
and the output equation becomes y (k ) = [C + ENC DEL ]x (k ) + DPr (k )
The plant tracking output y(k) observes y (k ) = C x (k ) + D r (k ) C M (k ) + D r (k ) and for y (k ) r (k ) it is necessary that
= C x (k ) + D r (k ) where H = H + GD Examining the composite state coupling matrix A above shows that the coupling of external inputs r(k) to the feedback system does not affect its eigenvalues. The input coupling matrix B has matrices P and J which are entirely arbitrary and thus can be selected by the designer. Our objective is to select P and J such that the system output y(k) tracks the reference input r(k). Consider the class of reference signals which are generated by the autonomous state variable model of the form (k + 1 ) = r (k ) = (k ) (k ) (24)
C M (k ) + D r (k ) = r (k )
(26)
Equations (25) and (26) constitute a set of linear algebraic equations where the elements of M, P, and J are unknowns. If, for an initial problem formulation, there is no solution to the equations, one can reduce the order of the reference signal model and/or raise the order of the observer used for plant feedback until an acceptable solution is obtained. The autonomous reference input model has no physical existence; the actual reference input r(k) likely deviates somewhat from the prediction of the model. The designer deals with representative reference inputs, such as constants and ramps, and, by designing for exact tracking of these, obtains acceptable tracking performance for other reference inputs. SIMULATION One of the most important control system design tools is simulationthat is, computer modeling of the plant and controller to verify the properties of a preliminary design and to test its performance under conditions (e.g., noise, disturbances, parameter variations, and nonlinearities) that might be difcult or cumbersome to study analytically. It is usually through simulation that difculties with between-sample plant response are covered and solved. When a continuous-time plant is simulated on a digital computer, its response is computed at closely spaced discrete times. It is plotted by joining the closely spaced calculated response values with straight line segments in approximation
The output of this reference input model system may consist of step, ramp, parabolic, exponential, sinusoidal, and other common sequences. For example, the model
1 (k + 1 ) 2 = 2 (k + 1 ) 1 r(k ) = [1 0]
1 0
1 (k ) 2 (k ) 1 (k ) 2 (k )
produces the sum of an arbitrary constant plus an arbitrary ramp: r ( k ) = 1 ( 0 ) + 1 ( 0 )k
658
DISPATCHING
of continuous curve. A digital computer simulation of discretetime control of a continuous-time system involves at least two sets of discrete-time calculations. One runs at a high rate for simulation of the continuous-time plant. The other runs at a lower rate (say once every 10 to 50 of the former calculations) to generate new control signals at each discrete control step. BIBLIOGRAPHY
1. M. S. Santina, A. R. Stubberud, and G. H. Hostetter, Digital Control System Design, 2nd ed., Philadelphia: Saunders, 1994. 2. J. J. DiStefano III, A. R. Stubberud, and I. J. Williams, Feedback and Control Systems (Schaums Outline), 2nd ed., New York: McGraw-Hill, 1990. 3. B. C. Kuo, Digital Control Systems, 2nd ed., Philadelphia: Saunders, 1992. 4. G. F. Franklin, J. D. Powell, and M. L. Workman, Digital Control of Dynamic Systems, 2nd ed., Reading, MA: Addison-Wesley, 1990. 5. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1980. 6. C. T. Chen, Linear System Theory and Design, Philadelphia: Saunders, 1984. stro 7. K. J. A m and B. Wittenmark, Computer Controlled Systems, Englewood Cliffs, NJ: Prentice-Hall, 1987. 8. K. Ogata, Discrete-Time Control Systems, Englewood Cliffs, NJ: Prentice-Hall, 1987. 9. B. Friedland, Control System Design, New York: McGraw-Hill, 1986. 10. W .M. Wonham, Linear Multivariable Control: A Geometric Approach, 3rd ed., New York, Springer-Verlag, 1985.
MOHAMMED S. SANTINA
Boeing Defense and Space Group
ALLEN R. STUBBERUD
University of California, Irvine
DISCRETE TIME TRANSFORMS. See Z TRANSFORMS.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Filtering and Estimation, Nonlinear Standard Article Tyrone E. Duncan1 1University of Kansas Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1013 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (209K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Nonlinear Filtering Problem Formulation and Main Results Some Recent Areas of Nonlinear Filtering About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
480
FILTERING AND ESTIMATION, NONLINEAR

To have a historical perspective of the advent of nonlinear ltering and estimation, initially the development of linear ltering and estimation is described. The rst studies of linear ltering or linear estimation for stochastic processes were made by Kolmogorov (1,2), Krein (3,4) and Wiener (5). The research of Kolmogorov and Krein and the research of Wiener were done independently. Kolmogorov, who was motivated by Wold (6), gave a solution to the prediction problem for discrete-time stochastic processes. Since Kolmogorov and Krein were not motivated for their work by any specic applications, the formulae for the optimum predictor did not play a special role. However, Wiener was motivated for his work during World War II by the analysis of anti-aircraft re-control problems from ships. He solved the continuous-time linear prediction problem and derived an explicit formula for the optimum predictor. He also solved the ltering problem of estimating a stochastic signal process that is corrupted by an additive noise process. In this latter case Wiener expressed the solution in terms of an integral equation, the WienerHopf equation (7). Wiener had obtained this equation in his work on potential theory a number of years earlier. This relation alludes to the probabilistic interpretation of potential theory using Brownian motion (8). Wieners book (5) contains a number of elementary, explicitly solvable examples. The sum of the signal process and the additive noise process is called the observation process. The prediction problem is to estimate the signal process at some future time usually
on an ongoing basis from the observation process. The ltering problem is to estimate the signal process at the same time. The smoothing problem has a couple of variants: (1) Given the observation process in a xed time interval, estimate the signal process at each element in the time interval, and (2) estimate the signal process at a time that is a xed lag behind the observation process. The approach of Kolmogorov, Krein, and Wiener to these problems assumed that the stochastic processes are (wide-sense) stationary and that the innite past of the observation process is available. Both of these assumptions are not physically reasonable, so there was a need to relax these assumptions. In the late 1950s, control and system theory were undergoing a signicant change from the frequency-domain approach to the state-space approach. Transfer function descriptions of linear systems were replaced by ordinary differential equation descriptions of linear systems. This state-space approach provided an impetus to reexamine the linear ltering problem. Using this approach the signal process is modeled as the solution of a linear differential equation with a Gaussian white noise input so the signal process is GaussMarkov. The differential of the observations process is a linear transformation of the signal process plus Gaussian white noise. This ltering model does not require the innite past of the observations. The signal and the observation processes can evolve from some xed time with a Gaussian random variable as the initial condition for the differential equation that describes the signal process. The processes are not required to be stationary; and, in fact, the coefcients of the differential equation for the signal process and the linear transformation for the signal in the observation equation can be time-varying. While it is not required that the ordinary differential equation for the signal process be stable, which is implicit in the description for stationary processes, it is necessary to be able to model the signal process as the solution of an ordinary differential equation with a white noise input. In general a stationary Gaussian process may not have such a model. With the success of these linear ltering results that were developed particularly by Kalman (9) for discrete-time processes and by Kalman-Bucy (10) for continuous-time processes, an interest developed in trying to solve a ltering problem where the signal is a solution to a nonlinear differential equation with a white noise input. It is natural to call such a problem a nonlinear ltering problem. The precise description of such a problem required the introduction of a signicant amount of the modern theory of stochastic processes. The major technique for describing the signal process is the theory of stochastic differential equations that was initiated (11). by K. Ito The Gaussian white noise processes that appear as inputs in the nonlinear differential equations require more sophisticated mathematical methods than do the inputs to linear differential equations. This occurs because the linear transformations of white noise have one natural interpretation but the nonlinear transformations of white noise have no single natural interpretation. Interestingly, it was Wiener (12) who rst constructed the basic sample path property of the integral of Gaussian white noise that is called the Wiener process or Brownian motion and which provided the basis for the interpretations of nonlinear transformations of white noise. Many important proper vy (13). The ties of Brownian motion were determined by P. Le
481
solution of a stochastic differential equation (a nonlinear differential equation with a white noise input) required the theory of stochastic integrals (14), which depends on some martingale theory (15) associated with Brownian motion. No one denition of stochastic integrals arises naturally from the Riemann sum approximations to the stochastic integral. This phenomenon occurs because Brownian motion does (14) is not have bounded variation. The denition of K. Ito the most satisfying probabilistically because it preserves the martingale property of Brownian motion and Wiener integrals (stochastic integrals with deterministic integrands). denition of stoHowever, the calculus associated with the Ito chastic integral is somewhat unusual. The FiskStratonovich denition of stochastic integral (16,17) preserves the usual calculus properties, but the family of integrable functions is signicantly smaller. An uncountable family of distinct denitions of stochastic integrals can be easily exhibited (18). This choice or ambiguity in the denition of a stochastic integral has played an important role in nonlinear ltering because initially some nonlinear ltering solutions were given without specifying the interpretation or the denition of the stochastic integrals. This ambiguity often arose by a formal passage to the limit from discrete time. In general, to compute conditional statistics of the state process given the observation process, it is necessary to compute the conditional density of the state process given the observation process. For linear ltering the signal and the observation processes are Gaussian, so the conditional density is determined by the conditional mean and the conditional covariance. The conditional covariance is not random, so it does not depend on the observation process. The conditional mean can be shown to satisfy a stochastic differential equation that models the signal process and that has the observations as the input. These two conditional statistics (i.e., function of the observations are called sufcient conditional statistics (19) because the conditional density can be recovered from them. For nonlinear ltering the solution does not simplify so easily. In general there is no nite family of sufcient conditional statistics for a nonlinear ltering problem. The conditional density can be shown to satisfy a nonlinear stochastic partial differential equation (20,21). This equation is especially difcult to solve because it is a stochastic partial differential equaton and it is nonlinear. Even approximations are difcult to obtain. The conditional density can be expressed using Bayes formula (22,23), so that it has the same form as the Bayes formula in elementary probability though it requires function space integrals. The numerator in the Bayes formula expression for the conditional density is called the unnormalized conditional density. This unnormalized conditional density satises a stochastic partial differential equation that is linear. It is called the DuncanMortensenZakai (DMZ) equation of nonlinear ltering (2426). In nonlinear ltering, the question of nite dimensional lters describes the problem of nding nite dimensional solutions to the DMZ equation or to nite families of conditional statistics. A basic approach to this question on the existence or the nonexistence of nite-dimensional lters is the estimation algebra (27,28,28a), which is a Lie algebra of differential operators that is generated by the differential operators in the DMZ equation. Some families of nonlinear ltering problems have been given that exhibit nite-dimensional lters (e.g., see Ref. 29).
Some methods from algebraic geometry have been used to give necessary and sufcient conditions on the coefcients of the stochastic equations for small state-space dimension, so that the nonlinear ltering problem has nite-dimensional lters. Since the results for the existence of nite-dimensional lters for nonlinear ltering problems are generally negative, many approximation methods have been developed for the numerical solution of the DMZ equation or the equations for some associated conditional statistics. In their study of Wiener space (the Banach space of continuous functions with the uniform topology and the Wiener measure), Cameron and Martin (30) showed that any square integrable functional on Wiener space could be represented as an innite series of products of Hermite polynomials (Wick polynomials). K. Ito (31) rened this expression by using an innite series of multiple Wiener integrals. The relation between these two representations is associated with including or excluding the diagonal in multiple integration. This relation carries over to Stratonovich integrals, and the explicit relation between these two stochastic integrals in this case is given by the Hu Meyer formula (32). For the solution of the linear ltering problem it was well known from the early work that the observation process in the optimal lter appears with a linear transformation of the estimate as a difference and that this difference is a process that is white with respect to the observation process. Since a family of square integrable zero mean random variables generates a vector space with an inner product that is the expectation of a product of two of the random variables, the (linear) ltering problem can be posed as a projection problem in a vector space (Hilbert space). The occurrence of a process that is white with respect to the observations is natural from a GramSchmidt orthogonalization procedure and projections. This process has been historically called the innovation process (6). For Wiener ltering, this innovations approach was introduced in the engineering literature by Bode and Shannon (6a). For linear ltering it is straightforward to verify that the observation process and the innovation process are equivalent (that is, there is a bijection between them) by showing that a linear operator is invertible. For nonlinear ltering there is still an innovation process. It is more subtle to verify that the observation process and the innovation process are equivalent (33). Thus the nonlinear ltering solution has a vector space interpretation via orthogonalization and projections as for the linear ltering solution. However, this is not surprising because in both cases there is a family of (square integrable) random variables and the conditional expectation is a projection operator. This occurrence of the innovation process can be obtained by an absolute continuity of measures (34). In information theory, the mutual information for a signal and a signal plus noise can be computed similarly (35). The expression for the conditional probability or the conditional density, given the past of the observations as a ratio of expectations, has a natural interpretation as a Bayes formula (22,23) that naturally generalizes the well-known Bayes formula of elementary probability. The stochastic partial differential equations for the conditional probability density or the unnormalized conditional probability density are obtained by the change of variables (36). formula of K. Ito
482
The fact that the sample paths of Brownian motion (the formal integral of Gaussian white noise) do not have bounded variation has important implications concerning robustness questions. Wong and Zakai (37) showed that if Brownian motion in a stochastic differential equation is replaced by a sequence of piecewise smooth processes that converge (uniformly) to Brownian motion, then the corresponding sequence of solutions of the ordinary differential equations obtained from the stochastic differential equation by replacing the Brownian motion by the piecewise smooth processes do not converge to the solution of the stochastic differential equation for many nonlinear stochastic differential equations. This result of Wong and Zakai has important implications for nonlinear ltering. If nonlinear lters are constructed from time discretizations of the processes and a formal passage to the limit is made, then it may not be clear about the interpretation of the resulting solution. This question is closely related to the choice of the denition of stochastic integrals and the unusual s denition. In the early calculus that is associated with K. Ito development of nonlinear ltering theory, solutions were given that did not address the question of the denition of stochastic integrals. Generally speaking, formal passages to the limit from time discretizations require the Stratonovich denition of stochastic integrals because these integrals satisfy the usual properties of calculus. One classical example of the use of nonlinear ltering in communication theory is the analysis of the phase-lock loop problem. This problem arises in the extraction of the signal in frequency modulation (FM) transmission. The process that is received by the demodulator is a sum of a frequency modulated sinusoid and white Gaussian noise. The phase-lock demodulator is a suboptimal nonlinear lter whose performance is often quite good and which is used extensively in FM radio demodulation circuits. If the state of an nth-order linear stochastic system is reconstructed from samples of the output by well-known numerical differentiation schemes, then even in the limit as the sampling becomes arbitrarily ne, well-known computations such as quadratic variation do not converge to the desired results (8). This phenomenon did not occur for linear stochastic differential equations in the approach in Ref. 37. NONLINEAR FILTERING PROBLEM FORMULATION AND MAIN RESULTS In this section a nonlinear ltering problem is formulated mathematically and many of the main results of nonlinear ltering are described. A basic nonlinear ltering problem is described by two stochastic processes: (X(t), t 0), which is called the signal or state process; and (Y(t), t 0), which is called the observation process. These two processes satisfy the following stochastic differential equations: dX (t ) = a(t , X (t )) dt + b (t , X (t )) dB(t ) (t ) dY (T ) = h(t , X (t ), Y (t )) dt + g(t , Y (t )) dB (1) (2)
spectively. The following assumptions are made on the coefcients of the stochastic differential equations (1) and (2) that describe the signal or state process and the observation process, respectively. The stochastic processes are dened on a xed probability space (, F , P) with a ltration (F t, t 0). Often the space can be realized as a family of continuous functions. The -algebras are assumed to be complete with respect to the probability measure P. A1. The drift vector a(t, x) in Eq. (1) is continuous in t and globally Lipschitz continuous in x. The vector a(t, x) is continuous in t and globally Lipschitz continuous in x. A2. The diffusion matrix b(t, x) in Eq. (1) is Ho lder continuous in t, globally Lipschitz continuous in x, and globally bounded. The symmetric matrix c bTb is strictly positive denite uniformly in (t, x). The terms
ci j (t , x ) xi
2 ci j (t , x ) xi x j
i, j {1, . . ., n}
are continuous in t, globally Lipschitz continuous in x, and globally bounded. A3. The drift vector h(t, x, y) and the diffusion matrix g(t, y) in Eq. (2) are continuous in x. The symmetric matrix f gTg is strictly positive denite uniformly in (t, x); that is, fx, x cx2, where c 0. The global Lipschitz conditions on the coefcients of the stochastic differential equations in the space variables x and y ensure the existence and the uniqueness of the strong (i.e., sample path) solutions of these equations. The additional smoothness properties are used to verify properties of the transition density of the Markov process (X(t), Y(t), t 0). Since the soluton of Eq. (1) is generated by x0 and (B(t), t 0), the process (X(t), t 0) is independent of the Brownian (t), t 0) and the process (Z(t), t 0) where motion (B (t ) dZ(t ) = g(t , Z(t ))dB (3)
A transition probability measure or a transition probability function for a Markov process is a function P(s, x; t, ) for s [0, t), x d, and Bd the Borel -algebra on d that satises the following: 1. P(s, x; t, ) is a probability measure on (d, Bd) for all s [0, t). 2. P(s, ; t, ) is Bd-measurable for all s [0, t) and B d . 3. If s [0, t), u t and Bd, then P(s, x; u, ) = P(t , y; u, )P(s, x; t , dy ) (4)
where t 0, X(0) x0, Y(0) 0, X(t) n, Y(t) m, a: n n, b: n L (n, n), h: n m (t), t 0) are m, g: m L (m, m), (B(t), t 0), and (B independent, standard Brownian motions in n and m, re-
With the notion of transition probability measure (function), a Markov process can be dened. Denition. Let P(s, x; t, ) be a transition probability measure and be a probability measure on (d, Bd). A probability
483
measure P on ((d)t, B(d) ) is called a Markov process with transition function P(s, x; t, ) and initial distribution if
t
where L= ai 1 + xi 2 ci j 2 xi x j (10)
P(X (0 ) ) = ( ), and for each s [0, t) and Bd
P(x(t ) | (X (u ), 0 u s )) = P(s, X (s ); t , )
(5)
In the ltering solution the formal adjoint of L, L*, appears; that is, L = 1 (a ) + xi i 2 2 (c ) xi x j i j (11)
The random variable X(t) is the evaluation of an element on (d)t at t . Usually the Markov process is identied as (X(t), t 0) and the Markov property (5) is described as P(X (t ) |X (u ), 0 u s ) = P(s, X (s ); t , ) (6)
If P(s, x; t, ) P(t s, x, ), then the Markov process is said to be (time) homogeneous. If (X(t), t 0) is a homogeneous Markov process, then there is a semigroup of operators (Pt, t 0) acting on the bounded Borel measurable functions (39), which is given by (Pt )(x ) = Ex [ (X (t ))] = (y )P(0, x; t , dy ) (7)
The operator L is often called backward operator, and L* is called the forward operator. The stochastic integrals that occur in the solution of Eq. (14). For a (1) are interpreted using the denition of K. Ito smooth integrand the integral is the limit of Riemann sums where the integrand is evaluated at the left endpoint assuming that the integrand is suitably measurable. This denition preserves the martingale property of Brownian motion. process or a semimartingale, Let (M(t), t 0) be an Ito that is dM (t ) = (t ) dt + (t ) dB(t ) or dM (t ) = dA(t ) + dN (t ) (13) (12)
Consider the restriction of (Pt, t 0) to the bounded, continuous functions that vanish at innity which is a Banach space in the uniform topology. If (X(t), t 0) is a Markov process that is the solution of the stochastic differential equation dX (t ) = a(X (t )) dt + b (X (t )) dB(t ) where a( ) and b( ) satisfy a global Lipschitz condition, then the semigroup (Pt, t 0) has an innitesimal generator that s formula (36); that is, is easily computed from Ito Pff L f = lim t t 0 t for f D (L ) = f : lim
t 0
where (A(t), t ) is a process of bounded variation and (N(t), t 0) is a martingale. The FiskStratonovich or Stratonovich integral (16,17) of a suitable stochastic process ((t), t 0) is denoted as
t
(s ) dM (s )
0
(14)
(8) This integral is dened from the limit of nite sums that are formed from partitions as in RiemannStieltjes integration where the function is evaluated at the midpoint of each in integral terval formed from the partition. Recall that the Ito is formed by evaluating the integrand at the left endpoint of each of the subintervals formed from the partition (40). For the linear ltering problem of GaussMarkov processes it is elementary to show that the conditional probability density is Gaussian so that only the conditional mean and the conditional covariance have to be determined. Furthermore, the estimate of the state, given the observations that minimizes the variance, is the conditional mean. Thus one approach to the nonlinear ltering problem is to obtain a stochastic equation for the conditional mean or for other conditional statistics. The difculty with this approach is that typically no nite family of equations for the conditional statistics is closed; that is, any nite family of equations depends on other conditional statistics. The conditional probability density of the state X(t), given the observations (Y(u), 0 u t), is the density for the conditional probability that represents all of the probabilistic information about X(t) from the observations (Y(u), 0 u t). A conditional statistic can be computed by integrating the conditional density with a suitable function of the state. To obtain a useful expression for the conditional probability measure, it is necessary to use a result for the absolute
Pt f f exists t
It is straightforward to verify that

d
L=
i= 1
ai
1 + xi 2
ci j
2 xi x j
(9)
where c bTb. An analogous result holds if the Markov process is not homogeneous, so that dX (t ) = a(t , X (t )) dt + b (t , X (t )) dB(t ) where the theory of two-parameter semigroups is used so that Ps,t f (x ) = EX (s )= x [ f (X (t ))] and dPs,t f = Lf dt
484
continuity of measures on the Borel -algebra of the space of continuous functions with the uniform topology. These results center around the absolute continuity for Wiener measure, the measure for Brownian motion. The rst systematic investigation of Wiener measure in this context was done by Cameron and Martin (30), who initiated a calculus for Wiener measure. Subsequently, much work was done on general Gaussian measures (e.g., see Ref. 41). For Wiener measure and some related measures, a more general probabilistic approach was given by Skorokhod (42,43) and Girsanov (44). The following result is a version of Girsanovs result. Theorem. Let (, F , P) be a probability space with a ltration (F t, t [0, T]). Let ((t), t [0, T]) be an n-valued process that is adapted to (F t, t [0, T]) and let (B(t), t [0, T]) be an n-valued standard Brownian motion. Assume that E [M ( T )] = 1 where
t
A result for the absolute continuity of measures is given that follows from the result of Girsanov (44). For convenience it is stated that X V, though it easily follows that XY VZ and in fact there is mutual absolute continuity. Theorem. Let V and X be the probability measures on (, F ) for the process (V(t), t [0, T]) and (X(t), t [0, T]), respectively, that are solutions of Eqs. (18) and (1). Then X is absolutely continuous with respect to V, denoted X V, and d X = (T ) dV where is given by Eq. (19). Corollary. Let (V(t), t [0, T]) satisfy (18) on (, F , V). Then (t ) = B(t ) B
s t
(15)
b 1 (s, B(s ))a(s, B(s )) ds
M (t ) = exp
0
(s ), dB(s )
1 2
is a Brownian motion on (, F , X). |(s )|2 ds (16) It can be shown that the linear growth of the coefcients ensures that there is absolute continuity, so that Eq. (15) is satised (45). The following result gives the conditional probability measure in function space (22,23). Proposition. For t 0 the conditional probability measure of X(t), given (Y(u), 0 u t), is given by
Then the process (Y(t), t [0, T]) given by

t
Y (t ) = B(t )
0
(s ) ds
(17)
, where is a standard Brownian motion for the probability P M(T) dP. dP Let Z be the probability measure on the Borel -algebra of m-valued continuous functions for the process (Z(t), t 0) that is the solution of Eq. (3). Let (V(t), t 0) be the process that is the solution of dV (t ) = b (t , V (t )) dB(t ) V ( 0 ) = x0 (18)
P( , t | x0 , Yu , 0 u t ) =
E Z [1 t t ] E Z [t t ]
(21)
for (X(t)), the -algebra generated by X(t). The absolute continuity of measures and the associated RadonNikodym derivatives are important objects even in elementary probability and statistics. In this latter context there is usually a nite family of random variables that have a joint density with respect to Lebesgue measure. The likelihood function in statistical tests is an example of a computation of a RadonNikodym derivative. The conditional probability measure Eq. (21) is specialized to the conditional probability distribution and a density is given for this function. The conditional density is shown to satisfy a stochastic partial differential equation (17, 20, 46). Theorem. Let (X(t), Y(t), t 0) be the processes that satisfy Eqs. (1) and (2). If A1A5 are satised, then (t )), dY (t ) g (t ) p(t ) d p(t ) = L p + f 1 (t , Y (t ))(g(t ) g (22) where p(t ) = p(X (t ), t | x0 , Y (u ), 0 u t ) g(t ) = g(t , X (t ), Y (t )) (23) (24)
Let XY be the measure on the Borel -algebra of nm-valued continuous functions for the process (X(t), Y(t), t 0) that satisfy Eqs. (1) and (2). It follows from Girsanovs Theorem above that XY V Z. The RadonNikodym derivative (t) (t)(t) E[dXY / d(VZ) F t] is
t
(t ) = exp 1 2
0 t
c 1 (s, X (s ))a(s, X (s )), dX (s )

1
(19)
c
0
(s, X (s ))a(s, X (s )), a(s, X (s )) ds
(t ) = exp 1 2
0 t 0
f 1 (s, Y (s ))g(s, X (s ), Y (s )), dY (s ) f 1 (s, Y (s ))g(s, X (s ), Y (s )), g(s, X (s ), Y (s )) ds (20)
To indicate the expectation with respect to one of the function space measures, E is subscripted by the measurefor example, EX.
(t ) = g
E X [ (t )g(t )] E X [ (t )]
(25)
485
Equation (22) is a nonlinear stochastic partial differential (t)p(t), and equation. The nonlinearity occurs from the terms g the partial differential operator L* is the forward differential operator for the Markov process (X(t), t 0). Often only some conditional statistics of the state given the observations are desired for the nonlinear ltering problem solution. However, such equations are usually coupled to an innite family of conditional statistics. The following theorem describes a result for conditional statistics (47,48). Theorem. Let (X(t), Y(t), t 0) satisfy Eqs. (1) and (2). Assume that a(t, x) and b(t, x) in Eq. (1) are continuous in t and globally Lipschitz continuous in x, h(t, x, y) is continuous in t and globally Lipschitz continuous in x and y, and g(t, y) is continuous in t and globally Lipschitz continuous in y and f gTg is strictly positive denite uniformly in (t, y). If C2(n, )such that
T
where pX is the transition density for the Markov process (X(t), t 0). Assume that A1A3 are satised. Then r satises the following linear stochastic partial differential equations: dr(X (t ), t | x0 , Y (u ), 0 u t ) = L r + f 1 (t , Y (t ))g(t , X (t ), Y (t )), dY (t ) r The normalization factor for r is q (t ) = E X [ (t )] so that p(x, t | x0 , Y (u ), 0 u t ) = r(t )q 1 (t ) (33) (32)
E | (X (t ))| dt <
2 0 T
(26) (27) (28)
It is elementary to obtain a stochastic equation for q1(t) s formula; that is, using Ito
E
0 T
| D (X (t )), X (t ) |2 dt <
dq 1 (t ) = q 2 (t ) dq (t ) + q 3 (t ) f 1 (t )E X [ (t )g(t )], E X [ (t )g(t )] dt (34)

where dq (t ) = f 1 (t ) (t )g(t ), dY (t ) (35)
E
0
| D2 (X (t ))X (t ), X (t ) |2 dt <
then the conditional expectation of (X(t)), given the observations (Y(u), u t), (t ) = E [ (X (t )) | x0 , Y (u ), 0 u t ] satises the stochastic equation
To apply algebro-geometric methods to the nonlinear ltering problem the following form of the DMZ equation is used
T drt = [L (1/2 ) gt , gt ]rt dt + rt gt dY (t )
d (t ) = L (X (t )) dt + f 1 (t , Y (t )) g(t , X (t ), Y (t )) (t , X (t ), Y (t )), dY (t ) g (t , X (t ), Y (t )) dt (X (t ))g (29)

where L is the backward differential operator for the Markov process (X(t), t 0) and is conditional expectation for example,
(36)
(X (t )) =
E X [ (X (t )) (t )] E X [ (t )]
(30)
The stochastic equation for the condition probability density is a nonlinear stochastic partial differential equation. The stochastic equation for a conditional statistic is typically coupled to an innite famiily of such equations. The conditional density is more useful because it represents all of the probabilistic information about the state given the observations, but it is a nonlinear equation. If the so-called unnormalized conditional density is used, then the stochastic partial differential equation is linear. This unnormalized conditional density was given by Duncan (24), Mortensen (25), and Zakai (26). The equation is usually called the DuncanMortensenZakai (DMZ) equation. Theorem. Let (X(t), Y(t), t 0) be the processes that are the solutions to Eqs. (1) and (2). Let r be given by r(x, t | x0 , Y (u ), 0 u t ) = E X [ (t ) | X (t ) = x] pX (0, x0 ; t , x ) (31)
Recall that the symbol in Eq. (36) indicates the Stratonovich integral. The reason that this form of the DMZ equation is sometimes more useful is that it satises the usual rules of calculus. Thus the Lie algebras can be computed in the same way that would be used for smooth vector elds. A more elementary nonlinear ltering problem than the one for diffusion processes that is important for applications is the case where the signal or state process is a nite-state Markov process (in continuous time). The nite-state space for the process signicantly reduces the mathematical difculties. Let S s1, . . ., sn be the state space for the nite state Markov process and p i(t) P(X(t) si) and p (t) 1 n T [p (t), . . ., p (t)] . It follows that d p (t ) = A p (t ) dt (37)
where A is the intensity matrix or the transpose of the generator of the Markov process (X(t), t 0). The dependence of p on the initial value X(0) has been suppressed for notational convenience. By analogy with the DMZ equations (31) and (36) in this case for the nite-state Markov process (49) it follows that the unnormalized conditional density (t) satises
s t
(t ) = (0 ) +
0
A(s ) ds +
0
B(s ) dY (s )
(38)
486
or
s t
(t ) = (0 ) +
0
(A (1/2 )B2 )(s ) ds +

0
B(s ) dY (s ) (39)
The system (41) and (42) is said to be observable under the following condition: If x1, x2 M and x1 x2, then there is an input (control) function u such that the outputs associated with x1 and x2 are not identical. By analogy to the structure theory of linear systems, there is a state-space isomorphism theorem; that is, given two systems of the form (41) and (42) on two analytic manifolds such that the coefcients are complete analytic vector elds, the systems are observable and dim L (x) is minimal, and the two systems realize the same inputoutput map, then there is an analytic map between the manifolds that preserves trajectories (50). Associated with the equivalence of systems of the form (41) and (42) there is a realization of such systems that is observable and dim L (x) n; that is, the Lie algebra of vector elds evaluated at each point in the manifold has maximal dimension. In general, the DMZ equation (36) can be viewed as the state equation of an innite-dimensional system for (t) with the input function from the observation Y and the output t() (27,51). An investigation of the existence or the nonexistence of a nite dimensional realization of the inputoutput map is the investigation of the existence of nite-dimensional lters. A nite-dimensional (recursive) lter for t() is a stochastic equation d(t ) = a((t )) dt + b ((t )) dY (s ) t () = ((t )) (43) (44)
where B diag(s1, . . ., sn) and (t) [1(t), , n(t)]T. These equations are a nite family of bilinear stochastic differential equations for the unnormalized conditional probabilities. The conditional expectation of the statistic :S , denoted t(), is t () =
n i i = 1 (si ) (t ) n i i = 1 (t )
(40)
A Lie algebra associated with the DMZ equation (36) plays a basic role in determining the existence or the nonexistence of nite-dimensional lters for conditional statistics of the signal (or state) process. To introduce Lie algebras, its denition is given. Denition. A Lie algebra V over a eld k is a vector space over k with a bilinear form []:V V V (the Lie bracket) that satises for v1, v2, v3 V the following: 1. [v1, v2] [v2, v1], 2. [v1, [v2, v3]] [v2, [v3, v1]] [v3, [v1, v2]] 0. A Lie subalgebra of a Lie algebra V is a linear subspace of V that is a Lie algebra. If I, a subalgebra of V, is an ideal of V, then the quotient algebra is V / I, a vector space with the induced Lie bracket. A Lie algebra homomorphism :V1 V2 of the Lie algebras V1 and V2 is a linear map that commutes with the bracket operations, ([u, v]) [(u), (v)]. The algebro-geometric methods for the nonlinear ltering problem arose from the system theory for nite-dimensional, nonlinear, afne, deterministic control systems. Consider a deterministic control system of the form dx = f (x(t )) + dt
m
ui (t )gi (x(t ))
i= 1
(41)
where x(t) M, a smooth d-dimensional manifold. A controllability property has local signicance in analogy to its global signicance for linear control systems. Denition. The controllability Lie algebra of Eq. (41) is the Lie algebra L generated by f , g1, . . ., gm.L (x) is the linear space of vectors in TxM, the tangent space of M at x, spanned by the vector elds of L at x. The dimension of L (x) has implication for the local reachable set starting at x M. Another basic notion in system theory is observability. This condition implies that different statesthat is, different points in M can be distinguished using an appropriate control. Denition. Consider the control system (41). Let h C(M, )give the observation as y(t ) = h(x(t )) (42)
where (t) n. The application of the Lie algebraic methods described above and the use of nonlinear system theory presupposed a nite-dimensional manifold. For the DMZ equation (36) the solution evolves in an innite-dimensional manifold. Thus it is necessary to be precise when translating these nite-dimensional algebro-geometric results to the DMZ equation. If this approach can be applied to the DMZ equation, then the questions of the existence or the nonexistence of nite-dimensional stochastic equations for conditional statistics and the equivalence of two nonlinear ltering problems can be resolved. Even for nite-state Markov processes it can be determined if some conditional statistics are the solutions of stochastic equations whose dimension is signicantly smaller than the number of states of the Markov process (52). For the DMZ equation (36) by analogy with the nite-dimensional inputoutput systems (41) and (42), the Lie algebra generated by the operators L* (1/2)h, h and h, acting on smooth (C) functions is called the estimation algebra associated with Eqs. (41) and (42) (53,54,54a). To identify equivalent ltering problems it is important to investigate transformations that induce isomorphic estimation algebras. A simple, important transformation is a change of scale of the unnormalized conditional probability density r( ). Let :n be a strictly positive, smooth function and let r (t) r(t). This transformation acts on the generators of the estimation algebra as L 1 1 h, h 2
g 1 ,
487
Thus the estimation algebras are formally isomorphic. Furthermore, a smooth homeomorphism of the state space induces an estimation algebra that is formally isomorphic to the initial estimation algebra. The above two operations on the estimation algebra have been called the estimation formal equivalence group (55). If for some distribution of X(0) a conditional statistic, t(), can be described by a minimal nite-dimensional (recursive) lter of the form (43) and (44), then the Lie algebra of this system should be a homomorphism of the estimation algebra for this ltering problem. This property has been called the homomorphism principle for the ltering problem (56). This homomorphism principle can be a guide in the investigation of the existence of nite-dimensional lters. A specic example of this homomorphism property occurs when the estimation algebra is one of the Weyl algebras. The Weyl algebra Wn is the algebra of polynomial differential operators over with operators x1, . . ., xn, / x1, . . ., / xn. The Lie bracket is the usual commutator for differential operators. This Lie algebra has a one-dimensional center and the quotient Wn / is simple; that is, it contains no nontrivial ideals. For the estimation algebra these two properties imply that if Wn is the estimation algebra for a ltering problem, then either the unnormalized conditional density can be computed by a nite-dimensional lter or no conditional statistic can be computed by a nite-dimensional lter of the form (43) and (44). More specically, for n 0 there are no nonconstant homomorphisms from Wn or Wn / to the Lie algebra of smooth vector elds on a smooth manifold (57). As an example of a Weyl algebra occurring as an estimation algebra, consider dX (t ) = dB(t ) (t ) dY (t ) = X 3 (t ) dt + dB (45) (46)
It can be veried that the Lie algebra of the linear ltering equations of Kalman and Bucy is isomorphic to the oscillator algebra. A well-known example of a ltering problem given by (29) has a nite-dimensional lter and it is closely reBenes lated to the linear ltering problem. Consider the scalar ltering problem dX (t ) = f (X (t )) dt + dB(t ) (t ) dY (t ) = X (t ) dt + dB where f satises the differential equation df + f 2 (x ) = ax2 + bx + c dx for some a, b, c . It is assumed that this Riccati equation has a global solution, so that either a 0 or a b 0 and c 0. The unnormalized conditional density can be computed to verify that there is a ten-dimensional sufcient conditional (29) showed that a two-dimensional statistic. However, Benes lter provides a sufcient conditional statistic. The estima for (49)(50) is generated by tion algebra L 1 1 2 f x2 , x 2 2 x x 2 and is four-dimensional and solvable. The estimation algebra for (47)(48) arises for the algebra L by letting f 0. To L let F(x) x f , with the estimation algebra L associate L (x) exp(F(x)) and r (t, x) (x)r(t, x). Then the DMZ equation for (47) and (48) is transformed by the gauge transformation as dr = 1 2 1 [(a + 1 )x2 + bx + c] r + xr dY 2 x2 2 (51) (49) (50)
It is straightforward to verify that the estimation algebra for this ltering problem is the Weyl algebra W1. The homomorphism principle that has been described can be veried in principle for this estimation algebra to show that there are no nontrivial conditional statistics that can be computed with nite-dimensional lters of the form (43) and (44) (58). It is natural to consider the linear ltering problem using the estimation algebra method. Consider the following scalar model: dX (t ) = dB(t ) (t ) dY (t ) = X (t ) dt + dB (47) (48)
This has the same form as the DMZ equation for (49) and (50). Thus the nonlinear ltering problem (49) and (50) is obtained from the linear ltering problem (47) and (48) by a gauge transformation of the conditional density. Various other examples of nite-dimensional lters are available (e.g., see Refs. 5961). Ocone (62) showed that for a scalar ltering problem with the observation equation of the form (50) the two examples (47)(48) and (49)(50) are the only ones that give a nitedimensional estimation algebra. This result is given in the following theorem. Theorem. Let n m 1 in (49)(50) and let g 1 in the observation equation (2). Then the dimension of the estimation algebra is nite only if: 1. h(x) ax and df + f 2 = ax2 + bx + c dx or 2. h(x) ax2 x, a 0 and df + f 2 (x ) = h2 (x ) + a(2ax + )2 + b + c(2ax + ) 1 dx
The estimation algebra is a four-dimensional Lie algebra with the basis 1 1 2 x2 , x, , 1 2 x2 2 x This algebra is called the oscillator algebra in physics (27,28). The oscillator algebra is the semidirect product of 1 and the Heisenberg algebra that is generated by 1 1 2 x2 , x, and 2 x2 2 x
488
or df + f 2 (x ) = h2 (x ) + ax2 + bx + c dx Some multidimensional results are discussed in the section entitled Some Recent Areas of Nonlinear Filtering. Another family of nonlinear ltering problems given by Liptser and Shiryayev (63,64) that can be solved by the Gaussian methods is the conditional linear models. Let (X(t), Y(t), t 0) satisfy
and
(t ) = B(t , Y )B(t , Y ) + A(t , Y )R (t ) + R (t )A(t , Y ) dR +

j
(t )G j (t , Y ) G j (t , Y )R (56)
(t )H (t , Y ) H (t , Y )R (t ) dt R +
j
(t ) + R (t )G j (t , Y ) ] dY j (t ) [G j (t , Y )R
dX (t ) = [A(t , Y )X (t ) + a(t , Y )] dt + B(t , Y ) dB(t )

d
+
j=1
[G j (t , Y )X (t ) + g j (t , Y )] dY j (t )
(52)
(0) R0. where m (0) m0 and R
(t ) dY (t ) = [H (t , Y )X (t ) + h(t , Y )] dt + dB
(53)
SOME RECENT AREAS OF NONLINEAR FILTERING A generalization of the ltering problem occurs when some or all of the processes take values in an innite-dimensional chet space. A basic space such as a Hilbert, Banach, or Fre question is the existence of a probability measure on one of these innite-dimensional spaces. For example, for the existence of a zero mean Gaussian measure in a Hilbert space it is necessary and sufcient that the covariance is nuclear (trace class). The usual DaniellKolmogorov construction of a probability measure from a projective family of measures on nite-dimensional subspaces (nite dimensional distributions) does not guarantee a measurable space with a nice topology. However, in some cases in innite-dimensional spaces it is possible to use a cylindrical noise (e.g., the covariance of the Gaussian process is the identify) and have it regularized by the system so that the stochastic integral in the variation of parameters formula is a nice process. To describe this approach consider a semilinear stochastic differential equation sX (t ) = AX (t ) dt + f (X (t )) dt + Q1/2 dW (t ) (57)
where X(0) is a Gaussian random variable, Y(0) 0. The random variable X(t), given Y (t) (Y(u), u t), is conditionally Gaussian. More precisely, it is assumed that (X(t), t 0) is an n-valued (F t)-adapted process, (Y(t), t 0) is an m-val (t), t 0) are ued (F t)-adapted process, (B(t), t 0) and (B n m independent standard - and -valued Brownian motions, respectively, in the ltered probability space (, F , (F t), P), and X(0) is N(m0, R0). The functions A, a, B, Gj, gj, H, and h are dened on C(, m) with values in a suitable Euclidean space, and they are progressively measurable. The 1 functions A2, a, B2, Gj2, H, and h are in Lloc () for each m 1 y C(, ) . For each T 0, E (T) 1 where
T
(T ) = exp
0 T
H (s, Y )X (t ) + h(s, Y ), dY (s ) (54) |H (s, Y )X (s ) + h(s, Y )| ds

2 0
1 2
Haussmann and Pardoux (65) proved the following result. Theorem. Consider the ltering problem (52) and (53). For each T 0 the conditional distribution of X(t), given Y (T) (Y(u), u T), is Gaussian. Furthermore, if it is assumed that a, gj, h, h gj, A, B, Gj2, H2, h Gj, gj H, and H(t, Y)X(t) h(t, Y) are in L2([0, T] ) for all T , then the following result for the conditional mean m (t) and the conditional covariance R(t) can be veried (65) Theorem. Consider the ltering problem (52) and (53). The (t) satconditional mean m (t) and the conditional covariance R isfy the following equations:
where X(0), X(t) H, a separable, innite-dimensional Hilbert space, and (W(t), t 0) is a standard cylindrical Wiener process. A standard cylindrical Wiener process means that if 1, 2 H H*, 1, 2 0, and 1, 1 2, 2 1 where , is the inner product in H, then (1, W(t), t 0) and (2, W(t), t 0) are independent standard Wiener processes. If A is the generator of an analytic semigroup (S(t), t 0) and S(r)Q1/2 is HilbertSchmidt for each r 0 and
t 0
|S(r )Q1/2 |2 L
2 (H )
dr <
(58)
dm (t ) = A(t , Y )m (t ) + a(t , Y ) (t )H (t , Y ) [H (t , Y )m R (t ) + h(t , Y )] +

j
where L2(H) is the norm for the HilbertSchmidt operators, then the process (Z(t), t 0) where
(55)
Z(t ) =
0
(t )H (t , Y ) dt G (t , Y )R
j j
S(t r )Q1/2 dW (r )
(59)
+
j
(t )H j (t , Y ) dY j (t )] [G j (t , Y )m (t ) + g j (t , Y ) + R
is an H-valued process that has a version with continuous sample paths. Thus, the solution of Eq. (57) with some suit-
489
able assumptions on f (66) can be given by the mild solution (67)

t
X (t ) = S(t )X (0 ) +
0 t
S(t r ) f (X (r )) dr (60)
1 /2
Assume that n 4, where X(t) n and Y(t) . If E is the nite-dimensional estimation algebra of maximal rank, then the drift term f must be a linear vector eld plus a gradient vector eld and E is a real vector space of dimension 2n 2. Another basic question is to nd necessary conditions for nite-dimensional estimation algebras. It was conjectured by Mitter (28) that the observation terms are polynomials of degree at most one. An important result related to this conjecture is the following result of Ocone (62,78) that describes polynomials in the estimation algebra. Theorem. Let E be a nite-dimensional estimation algebra. If is a function in E , then is a polynomial of degree at most two. The following result of Chen and Yau (79) veries the Mitter conjecture for a large family of estimation algebras. Theorem. If E is a nite-dimensional estimation algebra of maximal rank, then the polynomials in the drift of the observation equation (6) are degree-one polynomials. Two basic approaches to the problem of nite-dimensional lters for nonlinear ltering problems are the WeiNorman approach and the symmetry (or invariance) group approach (e.g., see Ref. 80). Wei and Norman (81) provided a global representation of a solution of a linear differential equation as a product of exponentials. The WeiNorman approach requires an extension of the WeiNorman results to semigroups. This has been done by introducing some function spaces or using some results for the solutions of partial differential equations (82,83). This result is important for the construction of nite-dimensional lters from nite dimensional estimation algebras (e.g., see Refs. 82 and 83). Recall the problem of the existence of nite-dimensional lters for a linear ltering problem with a non-Gaussian initial condition. The question of nite-dimensional lters for nonlinear ltering problems can be formulated in different ways. In one formulation the probability law of the initial condition is xed. It has been shown (82) that a necessary condition for a nite-dimensional lter is the existence of a nontrivial homomorphism from the estimation algebra into the Lie algebra of vector elds on a manifold. Another formulation of the nite-dimensional lter problem is the requirement that a lter exist for all Dirac measures of the initial condition. It has been shown (82) that if a nite-dimensional lter has a regularity property with respect to initial conditions and dynamics, then the estimation algebra is nite-dimensional. For linear ltering it is elementary to verify that minimizing a quadratic form which is the negative of the formal exponent in a likelihood function gives the solution of the linear ltering problem. By analogy, an approach to the nonlinear ltering problem based on minimizing a formal likelihood function in function space was introduced in the 1960s (84,85). This approach has been generalized and made rigorous by Fleming and Mitter (86) by relating a ltering problem to a stochastic control problem. This method uses a logarithmic transformation.
+
0
S(t r )Q
dW (r )
This semigroup approach can be used to model stochastic partial differential equations arising from elliptic operators and delay-time ordinary differential equations. Some problems of ergodic control and stochastic adaptive control are described in Refs. 66 and 68. For the stochastic partial differential equations it is natural to consider noise on the boundary of the domain or at discrete points in the domain. Furthermore, signal processes can be considered to be on the boundary or at discrete points in the domain. Some of the descriptions of these noise processes can be found in Refs. 66 and 68. For the nonlinear ltering problem it can be assumed that the signal process is innite-dimensional and that the observation process is nite-dimensional; or perhaps more interestingly it can be assumed that the signal process is nite-dimensional, occurring at distinct points of the domain or the boundary, and that the observation process is innite-dimensional in the domain. Another nonlinear ltering formulation occurs when the processes evolve on manifolds. This approach requires the theory of stochastic integration in manifolds (69). Many wellknown manifolds arise naturally in the modeling of physical systems such as spheres and positive denite matrices. To justify the conditional mean as the minimum variance estimate and to compare the estimate and the signal, it is useful to model the signal process as evolving in a linear space or a family of linear spaces. The observation process can evolve in a manifold and have the drift vector eld depend on the signal process, or the observation process can be the process in the base of a vector bundle; for example, the tangent bundle and the signal can evolve in the bers of the vector bundle (70,71). These formulations allow for some methods similar to ltering problems in linear spaces. An estimation problem in Lie groups is solved in Ref. 72. The DMZ equation for a nonlinear ltering problem in a manifold is given in Ref. 73. A description of the stochastic calculus on manifolds with applications is given in Ref. 74. The study of estimation algebras for nonlinear ltering problems has been a recent active area for nonlinear ltering. A number of questions naturally arise for estimation algebras. A fundamental question is the classication of nitedimensional estimation algebras. This classication would clearly provide some important insight into the nonlinear ltering problem. This classication has been done for nitedimensional algebras of maximal rank that correspond to state-space dimensions less than or equal to four (60,7577). The result is described in the following theorem. Theorem. Consider the ltering problem described by the following stochastic differential equations: dX (t ) = f (X (t )) dt + g(X (t )) dV (t ) X ( 0 ) = x0 dY (t ) = h(X (t )) dt + dW (t ) Y (0 ) = 0 (61) (62)
490
For an example of this method of logarithmic transformation consider the following linear parabolic partial differential equation:
1 tr a(x ) pxx + g(x, t ), px + V (x, t ) p 2 p(x, 0 ) = p0 (x ) pt =
(63)
It is assumed that there is a C2,1 solution. If this solution is positive, then S log p satises the nonlinear parabolic equation: St = 1 tr a(x )Sxx + H (x, t , Sx ) 2 (64) (65) (66)
S(x, 0 ) = log p0 (x ) = S0 (x ) H (x, t , Sx ) = g(x, t ), Sx 1 a(x ), Sx , Sx V (x, t ) 2
This type of transformation is well known. For example, it transforms the heat equation (g V 0) into Burgers equation. The nonlinear PDE (64) is the dynamic programming (HamiltonJacobiBellman) equation for a stochastic control problem. For example, let (X(t), t 0) satisfy dX (t ) = (g(X (t ), t ) + u(X (t ), t )) dt + (X (t )) dB(t ) X (0 ) = x and let the cost functional be
t
(67)
J (x, t , u ) = Ex
0
L(X (s ), t s, u(s )) ds + S0 (X (t ))
(68)
where L(x, t , u ) = 1 1 a (x )u, u V (x, t ) 2 (69)
With suitable assumptions on the family of admissible controls and conditions on the terms in the model it can be shown from the Verication Theorem (87) that Eq. (63) is the dynamic programming equation for this stochastic control problem. This approach can provide a rigorous basis for the formal maximization of a likelihood function in function space. See Ref. 87a. An approach to the robustness of the nonlinear lter (87b,87c) is to obtain a so-called pathwise solution to the DuncanMortensenZakai (DMZ) equation by expressing the solution as an (observation) path dependent semigroup. The innitesimal generator of this semigroup is the conjugation of the generator of the signal process by the observation path multiplied by the drift in the observation where the Stratonovich form of the DMZ equation is used. The fact that the observation path appears explicitly rather than its differential implies the robustness of the solution of the DMZ equation. It is important to obtain estimation methods that are applicable to both stochastic disturbances (noise) and deterministic disturbances. For Brownian motion, a Hilbert (or Sobolev space) of functions that are functions that are absolutely continuous and whose derivatives are square integrable having
probability zero often plays a more important role than the Banach space of continuous functions that has probability one. This Hilbert space (or Sobolev space) alludes to the fact that there are some natural relations between stochastic and deterministic disturbances. In recent years the study of risk sensitive control problems has occupied an important place in stochastic control. Risk sensitive control problems (i.e., control problems with an exponential cost) have been used with the maximum likelihood methods in (85,87d) to obtain robust nonlinear lters, that is, lters that are effective for square integrable disturbances as well as Gaussian white noise (87e). These ideas are related to the approach of H control as a robust control approach. The robust nonlinear lter can be naturally related to a robust nonlinear observer. For a number of problems there is a natural relation between estimation for deterministic and stochastic systems. For example, a weighted least squares algorithm can be used for the identication of parameters for both deterministic and stochastic systems. Since it is usually not feasible to solve explicitly the stochastic equation for the conditional mean or the conditional covariance for a nonlinear ltering problem it is important to obtain lower and upper bounds on the ltering error. These bounds enable an effective comparison of the suboptimal lters with the optimal lter. The bounds have typically been obtained as the noise approaches zero (87f). An important theoretical and practical problem in nonlinear ltering is the innite time stability or continuity of the lter with respect to the initial conditions and the parameters of the lter. The problem of stability of the optimal nonlinear lter with respect to initial conditions is investigated in (87g) for two different cases. Stability of the Riccati equation for linear ltering is used to obtain almost sure asymptotic stability for linear lters with possible non-Gaussian initial conditions. For signals that are ergodic diffusions it is shown that the optimal lter is asymptotically stable in the sense of weak convergence of measures for incorrect initial conditions. Another stability property that is important for the optimal lter is asymptotic stability with respect to the parameters of the lter. Another important question in nonlinear ltering is to develop numerical methods for the DMZ equation. One numerical approach to the solution of the DMZ equation is to consider that it is a stochastic partial differential equation of a special form and use numerical methods from PDE for the numerical discretization of the problem (e.g., nite-difference schemes). There has been some success with this approach (8890), but it is limited to small space dimension and also often to the small intervals of time. Another approach is to use the Wiener chaos expansion that is based on an orthogonal expansion of a square integrable functional on Wiener space (30,31,91). The solution, r, of the DMZ equation is expressed in the following expansion (92): r(t , x ) = 1 (t , x ) (y ) ! (70)
where are Wick polynomials (special products of Hermite polynomials) formed from Wiener integrals and are HermiteFourier coefcients in the orthogonal expansion. The separation of x and y in the expansion (70) implies a splitting
491
in the computations, one that is associated with the Markov process (X(t), t 0) and the other one that depends on the observations (Y(t), t 0). The family of functions () can be computed recursively from a family of equations of Kolmogorov type (92,93). The Wick polynomials () depend on the observations and there are numerical methods to compute them. For the numerical success, the Wick polynomials have to be computed in special ways. A direct approach to the evaluation of the Wick polynomials and thereby the expansion (70) is limited to short time intervals because the errors of truncation of the innite series increase rapidly, probabily exponentially. The numerical methods for nonlinear ltering are still under active development. It seems that all of the methods are limited to at most two or three space dimensions, and many methods have signicant restrictions on the time intervals that are allowable for computations. However, these methods have been demonstrated to perform signicantly better than more elementary methods such as the extended linear lter which is commonly used. Many stochastic problems require nonlinear ltering, so the ltering problems have to be addressed. It should be clear that the area of nonlinear ltering is still an active area for research. This research includes mathematical investigations using methods from probability and geometry and implementation investigations that use numerical schemes for solving stochastic problems. BIBLIOGRAPHY
1. A. N. Kolmogorov, Sur linterpolation et extrapolation des suites stationnaires, C. R. Acad. Sci., 208: 2043, 1939. 2. A. N. Kolmogorov, Stationary sequences in Hilbert space (in Russian), Bull. Math. Univ. Moscow, 2: 6, 1941. 3. M. G. Krein, On a generalization of some investigations of G. Szego , W. M. Smirnov, and A. N. Kolmogorov, Dokl. Adad. Nauk SSSR, 46: 9194, 1945. 4. M. G. Krein, On a problem of extrapolation of A. N. Kolmogorov, Dokl. Akad. Nauk SSSR, 46: 306309, 1945. 5. N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications, New York: Technology Press and Wiley, 1949. (Originally issued in February 1942, as a classied National Defense Research Council Report) 6. H. Wold, A Study in the Analysis of Stationary Time Series, 2nd ed., Uppsala, Sweden: Almqvist and Wiksell, 1938. 6a. H. W. Bode and C. E. Shannon, A simplied derivation of linear least squares smoothing and prediction theory, Proc. IRE, 38: 417425, 1950. 7. N. Wiener and E. Hopf, On a class of singular integral equations, Proc. Prussian Acad., Math.-Phys. Ser., 696, 1931. 8. J. L. Doob, Wieners work in probability theory, Bull. Am. Math. Soc., 72: 6972, 1966. 9. R. E. Kalman, A new approach to linear ltering and prediction problems, Trans. ASME, Ser. D, J. Basic Eng., 82: 3445, 1960. 10. R. E. Kalman and R. S. Bucy, New results in linear ltering and prediction theory, Trans. ASME, Ser. D, J. Basic Eng., 83: 95 107, 1961. , On stochastic differential equations, Mem. Am. Math. Soc., 11. K. Ito 4: 1951. 12. N. Wiener, Differential space, J. Math. Phys., 2: 131174, 1923. vy, Processus Stochastiques et Mouvement Brownien, 2nd ed., 13. P. Le Paris: Gauthier-Villars, 1964.
, Stochastic integral, Proc. Imp. Acad. Tokyo, 20: 519524, 14. K. Ito 1944. 15. P. A. Meyer, Probability and Potentials, Waltham, MA: Blaisdell, 1966. 16. D. L. Fisk, Quasi-martingales and stochastic integrals, Technical Report 1, Dept. of Mathematics, Michigan State University, 1963. 17. R. L. Stratonovich, Conditional Markov process theory, Theory Probability Appl. (USSR), 5: 156178, 1960. 18. E. J. McShane, Stochastic differential equations and models of random processes, In Proceedings of the 6th Symposium on Mathematical, Statistics, and Probability, Berkeley, CA: University of California Press, 1973. 19. S. S. Wilks, Mathematical Statistics, New York: Wiley, 1962. 20. H. Kushner, On the differential equations satised by conditional densities of Markov processes, with applications, SIAM J. Control, 2: 106119, 1964. 21. R. L. Stratonovich, Application of the theory of Markov processes for optimum ltration of signals, Radio Eng. Electron. Phys. (URRS), 1: 119, 1960. 22. R. S. Bucy, Nonlinear ltering theory, IEEE Trans. Autom. Control, AC-10: 198, 1965. 23. G. Kallianpur and C. Striebel, Estimation of stochastic systems: Arbitrary system process with additive white observation errors, Ann. Math. Statist., 39: 785801, 1969. 24. T. E. Duncan, Probability Densities for Diffusion Processes with Applications to Nonlinear Filtering Theory and Detection Theory, Ph.D. Dissertation, Stanford University, Stanford, CA, June 1967. 25. R. E. Mortensen, Optimal Control of Continuous-Time Stochastic Systems, Ph.D. dissertation, University of California, Berkeley, CA, 1966. 26. M. Zakai, On the optimal ltering of diffusion processes, Z. Wahrsch, Verw. Geb. 11: 230243, 1969. 27. R. Brockett, Remarks on nite dimensional nonlinear estimation, risque, 764: 4755, 1980. Aste 28. S. K. Mitter, On the analogy between mathematical problems on nonlinear ltering and quantum physics, Ric. Autom., 10 (2): 163216, 1980. risque, 28a. S. K. Mitter, Filtering theory and quantum elds, Aste 7576: 199205, 1980. , Exact nite dimensional lters for certain diffusions 29. V. E. Benes with nonlinear drift, Stochastics, 5: 6592, 1981. 30. R. H. Cameron and W. T. Martin, The orthogonal development of non-linear functionals in a series of FourierHermite functions, Ann. Math., 48: 385392, 1947. , Multiple Wiener integral, J. Math. Soc. Japan, 3: 157 31. K. Ito 169, 1951. grales multiples de Stratonov32. Y. Hu and P. A. Meyer, Sur les inte ich, Lecture Notes in Mathematics, No 1321, Berlin: SpringerVerlag, pp. 7281, 1988. 33. D. F. Allinger and S. K. Mitter, New results on the innovations problem for nonlinear ltering, Stochastics, 4: 339348, 1981. 34. T. E. Duncan, Evaluation of likelihood functions, Inform. Control 13: 6274, 1968. 35. T. E. Duncan, On the calculation of mutual information, SIAM J. Math., 19: 215220, 1970. , On a formula concerning stochastic differentials, Nagoya 36. K. Ito Math. J., 3: 5565, 1951. 37. E. Wong and M. Zakai, On the relation between ordinary and stochastic differential equations, Int. J. Eng. Sci., 3: 213229, 1965. 38. T. E. Duncan, P. Mandl, and B. Pasik-Duncan, Numerical differentiation and parameter estimation in higher order stochastic systems, IEEE Trans. Autom. Control, 41: 522532, 1996.
492
FILTERING AND ESTIMATION, NONLINEAR 61. S. S.-T. Yau, Finite dimensional lters with nonlinear drift. I: A lclass of lters including both KalmanBucy lters and Benes ters, J. Math. Systems, Estim. Control 4: 181203, 1994. 62. D. Ocone, Topics in Nonlinear Filtering Theory, Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA, 1980. 63. R. Sh. Liptser and A. N. Shiryayev, Statistics of Random Processes. I, New York: Springer-Verlag, 1977. 64. R. Sh. Liptser and A. N. Shiryayev, Statistics of Random Processes. II, New York: Springer-Verlag, 1978. 65. U. Haussmann and E. Pardoux, A conditionally almost linear ltering problem with nongaussian initial condition, Stochastics, 23: 241275, 1988. 66. T. E. Duncan, B. Maslowski, and B. Pasik-Duncan, Ergodic boundary/point control of stochastic semilinear systems, SIAM J. Control Optim., 36: 1998. 67. G. Da Prato and J. Zabczyk, Stochastic Equations in Innite Dimensions, New York: Cambridge University Press, 1992. 68. T. E. Duncan, B. Maslowski, and B. Pasik-Duncan, Adaptive boundary control of linear distributed parameter systems described by analytic semigroups, Appl. Math. Optim., 33: 107 138, 1996. 69. T. E. Duncan, Stochastic integrals in Riemann manifolds, J. Multivariate Anal., 6: 397413, 1976. 70. T. E. Duncan, Estimation for jump processes in the tangent bundle of a Riemann manifold, Appl. Math. Optim., 4: 265274, 1978. 71. T. E. Duncan, Some ltering results on Riemann manifolds, Inform. Control, 35: 182195, 1977. 72. T. E. Duncan, An estimation problem in compact Lie groups, Syst. Control Lett., 10: 257263, 1988. 73. T. E. Duncan, Stochastic ltering in manifolds, In Proceedings of the International Federation on Autonomic Control World Congress, Elmsford, NY: Pergamon Press, 1981. 74. T. E. Duncan, An introduction to stochastic calculus in manifolds with applications, In J. S. Baras and V. Mirelli (eds.), Recent Advances in Stochastic Calculus, New York: Springer-Verlag, pp. 105140, 1990. 75. J. Chen, S. S. Yau, and C. W. Leung, Finite dimensional lters with nonlinear drift. VIII: Classication of nite dimensional estimation algebras of maximal rank with state space dimension 4, preprint. 76. R. Dong, L. Tam, W. Wong, and S. Yau, Structure and classication theorem of nite-dimensional exact estimation algebras, SIAM J. Control Optim., 29: 866877, 1991. 77. L. Tam, W. Wong, and S. Yau, On a necessary and sufcient condition for nite dimensionality of estimation algebras, SIAM J. Control Optim., 28: 173185, 1990. 78. D. Ocone, Finite dimensional Lie algebras in nonlinear ltering, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identication and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 629636, 1981. 79. J. Chen and S. S. Yau, Finite dimensional lters with nonlinear drift. VII: Mitter conjecture and structure of , preprint. 80. J. Baras, Group invariance methods in nonlinear ltering of diffusion processes, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identication and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 565572, 1981. 81. J. Wei and E. Norman, On the global representation of the solutions of linear differential equations as a product of exponentials, Proc. Am. Math. Soc., 15: 327334, 1964. 82. M. Cohen de Lara, Finite-dimensional lters. Part I: The Wei Norman techniques, SIAM J. Control Optim., 35: 9801001, 1997. 83. M. Cohen de Lara, Finite-dimensional lters. Part II: Invariance group techniques, SIAM J. Control Optim., 35: 10021029, 1997.
39. A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, New York: Springer-Verlag, 1983. 40. I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed., New York: Springer-Verlag, 1991. 41. I. E. Segal, Tensor algebras over Hilbert spaces. I, Trans. Am. Math. Soc., 81: 106134, 1956. 42. A. V. Skorokhod, On the differentiability of measures which correspond to stochastic processes. I: Processes with independent increments, Theory Probability Appl., 2: 407432, 1957. 43. A. V. Skorokhod, Studies in the Theory of Random Processes, Reading, MA: Addison-Wesley, 1965. 44. I. V. Girsanov, On transforming a certain class of stochastic processes by absolutely continuous substitution of measures, Theory Probability Appl., 5: 285301, 1960. 45. T. E. Duncan and P. Varaiya, On the solutions of a stochastic control system. II, SIAM J. Control, 13: 10771092, 1975. 46. A. N. Shiryayev, On stochastic equations in the theory of conditional Markov processes, Theor. Probability Appl., 11: 179184, 1966. 47. M. Fujisaki, G. Kallianpur, and H. Kunita, Stochastic differential equations for non-linear ltering, Osaka J. Math., 9: 1940, 1972. 48. G. Kallianpur, Stochastic Filtering Theory, New York: SpringerVerlag, 1980. 49. W. M. Wonham, Some applications of stochastic differential equations to optimal nonlinear ltering, SIAM J. Control, 2: 347 369, 1965. 50. H. Sussmann, Existence and uniqueness of minimal realizations of nonlinear systems, Math. Syst. Theory, 10: 263284, 1977. 51. R. Brockett and J. M. C. Clark, The geometry of the conditional density equation, In O. Jacobs et al. (eds.), Analysis and Optimization of Stochastic Systems, New York: Academic Press, 1980, pp. 299310. 52. S. I. Marcus, Low dimensional lters for a class of nite state estimation problems with Poisson observations, Syst. Control Lett., 1: 237241, 1982. 53. M. Davis and S. Marcus, An introduction to nonlinear ltering, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identication and Applications, Dordrecht, The Netherlands: D. Riedel, 1981, pp. 5375. 54. S. I. Marcus, Algebraic and geometric methods in nonlinear ltering, SIAM J. Control, 22: 817844, 1984. 54a. S. K. Mitter, Geometric theory of nonlinear ltering, Outils et ` les Mathematiques pour lAutomatiques, 3: CNRS, 3760, Mode 1983. 55. R. W. Brockett, Classication and equivalence in estimation theory, In Proceedings of the 18th IEEE Conference on Decision and Control, Ft. Lauderdale, FL, 1979, pp. 172175. 56. R. Brockett, Nonlinear systems and nonlinear estimation theory, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identication and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 442477, 1981. 57. M. Hazewinkel and S. Marcus, Some results and speculations on the role of Lie algebras in ltering, In M. Hazewinkel and J. Willems (eds.), Stochastic Systems: The Mathematics of Filtering and Identication and Applications, Dordrecht, The Netherlands: D. Riedel, pp. 591604, 1981. 58. M. Hazewinkel, S. Marcus, and H. Sussmann, Nonexistence of exact nite dimensional lters for conditional statistics of the cubic sensor problem, Systems Control Lett., 3: 331340, 1983. 59. D. Ocone, J. S. Baras, and S. I. Marcus, Explicit lters for diffusions with certain nonlinear drifts, Stochastics, 8: 116, 1982. 60. W. Wong, Theorems on the structure of nite dimensional estimation algebras, Systems Control Lett., 9: 117124, 1987.
FILTERING THEORY 84. A. E. Bryson and M. Frazier, Smoothing for linear and nonlinear dynamic systems, in Proceedings of the Optimum System Synthesis Conference, Wright-Patterson Air Force Base, Ohio, September, 1962, AST-TDR-63-119. 85. R. E. Mortensen, Maximum likelihood recursive nonlinear ltering, J. Optim. Theory Appl., 2: 386394), 1968. 86. W. H. Fleming and S. K. Mitter, Optimal control and nonlinear ltering for nondegenerate diffusion processes, Stochastics, 8: 6377, 1982. 87. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975. 87a. S. K. Mitter, Lectures on nonlinear ltering and stochastic control, Lecture Notes in Math, 972: 170207, 1983. 87b. M. H. A. Davis, On a multiplicative functional transformation arising in nonlinear ltering, Z. Wahr. Verw. Geb., 54: 125139, 1980. 87c. M. H. A. Davis, A pathwise solution of the equations of nonlinear ltering, Theory Prob. Appl., 27: 167175, 1982. 87d. O. Hijab, Minimum energy estimation, Ph.D. dissertation, Univ. California, Berkeley, CA, 1980. 87e. W. H. Fleming and W. M. McEneaney, Risk sensitive and robust nonlinear ltering, Proc. 36th IEEE Conf. Decision Cont., San Diego, 10881093, 1997. 87f. B. Z. Bobrovsky and M. Zakai, Asymptotic bounds on the minimal error of nonlinear ltering, Stochastic Systems: the mathematics of ltering and identication and applications, M. Hazeainkel and J. Willems (eds.), Dordrecht: Reidel, 1981, 573582. 87g. D. Ocone and E. Pardoux, Asymptotic stability of the optimal lter with respect to the initial conditions, SIAM J. Con. Optim., 34: 226243, 1996. 88. A. Bensoussan, R. Glowinski, and R. Rascanu, Approximations of the Zakai equation by splitting up method, SIAM J. Control Optim., 28: 14201431, 1990. 89. R. J. Elliott and R. Glowinski, Approximations to solutions of the Zakai ltering equation, Stochastic Anal. Appl., 7: 145168, 1989. , Approximation of the Zakai equation for nonlinear lter90. K. Ito ing, SIAM J. Control Optim., 34: 620634, 1996. 91. H. P. McKean, Geometry of differential space, Ann. Prob., 1: 197 206, 1973. 92. S. Lototsky, R. Mikulevicius, and B. L. Rozovskii, Nonlinear ltering revisited: A spectral approach, SIAM J. Control Optim., 35: 435461, 1997. 93. A. Budhiraja and G. Kallianpur, Approximation to the solution of the Zakai equation using multiple Wiener and Stratonovich integral expansions, Stochastics and Stoch. Repts., 56: 271315, 1996. 94. W.-L. Chiou and S.-T. Yau, Finite-dimensional lters with nonlinear drift. II: Brocketts problem on classication of nite-dimensional estimation algebras, SIAM J. Control Optim., 32: 297 310, 1994.
493
TYRONE E. DUNCAN
University of Kansas
FILTERING, LINEAR. See CONVOLUTION.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Gain Scheduling Standard Article Jeff S. Shamma1 1The University of Texas at Austin, Austin, TX Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1014 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (167K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Motivation What Is Gain Scheduling? Linearization Basic Gain Scheduling Advanced Methods for Gain Scheduling About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
GAIN SCHEDULING
MOTIVATION A classical tradeoff in control design is model accuracy versus model simplicity. While sophisticated models better represent a physical systems behavior, the resulting analysis and control design are more involved. One manifestation of this tradeoff is the use of nonlinear versus linear models. Most physical systems exhibit nonlinear behavior. Some common examples are saturations, rate limiters, hysteresis, and backlash. Predominantly nonlinear behavior may be found in robotic manipulator dynamics, aircraft or missile ight dynamics, undersea vehicle dynamics, jet engine combustion dynamics, and satellite attitude dynamics. Although analysis and control design for nonlinear systems remains an active topic of research, analysis of linear systems is signicantly less complicated, and there is an abundance of control design methodologies, ranging from classical control to multivariable robust control. One compromise is to linearize the system behavior, that is, approximate the behavior of a nonlinear system near a particular operating condition by a linear system. This simplication allows one to draw upon analysis and design methods for linear systems. However, this simplication comes at the cost of certain limitations: The nonlinear system must be conned to operating near the specied operating condition of the linearization. The linearization analysis may give misleading or inconclusive results. The linearization may ignore important nonlinear phenomena which dominate the system behavior. Despite these limitations, linearization remains a widely used method for control system analysis and design. In many cases, conning a nonlinear system to operating near a specied operating condition is too restrictive. One example is ight control. An aircraft typically experiences several ight conditions, including take-off, cruising at various altitudes, specialized maneuvers, and landing. No single linearization can adequately describe the aircraft dynamics at all of these conditions. Another example is boiler-turbine control in power generation. Typical operating conditions include power ramp up, steady power delivery at various levels, and power ramp down. Again, no linearization can adequately describe the dynamics at all operating conditions.
sign several linear controllers, one for each operating condition. Each individual controller is expected to achieve good performance whenever the nonlinear plant is near the controllers associated operating condition. As the plant varies from one operating condition to another, the gains of the individual controllers are interpolated, or scheduled, to match the changes in operating conditions. The nal result is a nonlinear controller which is constructed out of several local linear controllers. The implementation of a gain scheduled controller is depicted in Fig. 1. An auxiliary variable, usually called the scheduling variable, is used to update the gains of the linear controller. The scheduling variable should be a good indication of the current operating condition of the plant, and hence should be correlated with the plant nonlinearities. The scheduling variable can be a combination of endogenous signals, such as a plant measurements, or exogenous parameters which reect environmental conditions. One example is missile autopilot design. Useful scheduling variables are the angle-of-attack and dynamic pressure, both of which characterize the aerodynamic ight coefcients of the missile. The angle-of-attack is the angle between the missile body and velocity vector and can be considered a state variable, and hence endogenous to the missile dynamics. The dynamic pressure, which is a function of missle velocity and atmospheric pressure, is indicative of the environmental conditions. Atmospheric pressure is clearly an exogenous signal. Since the dynamic pressure is also a function of missile velocity, it can be considered an endogenous variable. However, the velocity variations in a simplied model are decoupled from the attitude dynamics, and hence, dynamic pressure may be modeled as an exogenous signal. This sort of ambiguity, namely that an exogenous signal is really an endogenous signal in a more sophisticated model, is common. Gain scheduling has seen widespread industrial application. It is perhaps the most prevalent nonlinear method for aircraft ight control and missle autopilot design. Other applications include power systems, process control, and automotive control. Despite its widespread usage, traditional gain scheduling has been an ad hoc design approach accompanied by heuristic guidelines.
Scheduling variables
Environmental conditions Operating conditions Nonlinear plant
WHAT IS GAIN SCHEDULING? Gain scheduling is an approach to overcome the local limitations associated with linearizations. The idea is simple and intuitively appealing. Given a nonlinear plant with a wide range of operating conditions, one can select several representative operating conditions within the operating regime, perform several linearizations of the nonlinear dynamics, and de205
Reference commands
Scheduled controller
Regulated outputs Measured outputs
Figure 1. Gain scheduled control implementation.
206
GAIN SCHEDULING
Some immediate drawbacks of gain scheduling are the following: The design of several controllers at several linearization points can be a tedious task. Gain scheduled designs typically assume a xed operating condition, even though the operating condition is varying. Although the resulting controller is nonlinear, it is still based on linearizations of plant dynamics, and hence may neglect important nonlinear phenomena. Two heuristic rules-of-thumb which guide successful gain scheduled designs are the following: The scheduling variable should vary slowly. The scheduling variable should capture plant nonlinearities. It is easy to interpret these guidelines in terms of the aforementioned drawbacks. Since gain scheduled designs assume a constant operating condition, slow variations among operating conditions should be tolerable. Similarly, since gain scheduling relies on a family of linearizations, the changes within this family should be indicative of plant nonlinearities. This article provides an overview of the gain scheduling design procedure, discusses the theoretical foundations behind gain scheduling, as well as limitations of traditional gain scheduling, and presents emerging techniques for gain scheduling which address these limitations. LINEARIZATION Linearization of Functions We begin by recalling some concepts from multivariable calculus. Let f : R n R p denote a multivariable function which maps vectors in R n to vectors in R p. In terms of the individual components,
Let r(x) denote the residual error of approximation; that is, r(x ) = f (x ) f (x0 ) + D f (x0 )(x x0 ) Then, lim r (x ) =0 | ( x x 0 )| (5) (4)
xx o
where v denotes the Euclidean norm of v R n, |v| = (vT v )1/2 Example 1. Let f : R 2 R 2 be dened as f (x ) = x1 |x1 | + x2 x2 1 x2 (7) (6)
Then
D f (x ) =
2|x1 | 2x 1 x 2
1 x2 1
(8)
Approximating f (x) near
x0 =
1 2
leads to
f (x ) f (x0 ) + D f (x0 )(x x0 ) = 3 2 + 2 4 1 1 x1 1 x2 2 (9)
f 1 (x1 , . . ., xn ) . . f (x ) = . f p (x1 , . . ., xn )
(1)
Now, let f : R n R m R p denote a multivariable function which maps vectors in R n and R m together to vectors in R p. In terms of the individual components,
In case f is differentiable, Df (x) denotes the p n Jacobian matrix of partial derivatives; i.e.,
x1 . . D f (x ) = . fp x1
(x )
.. .
(x )
f1 (x ) xn . . . fp (x ) xn
f 1 (x1 , . . ., xn , u1 , . . ., um ) . . f (x, u ) = . f p (x1 , . . ., xn , u1 , . . ., um )
(10)
(2)
In case f is differentiable, D1f (x, u) denotes the p n Jacobian matrix of partial derivatives with respect to the rst variable,
For f continuously differentiable, i.e., if Df (x) has continuous elements, we may approximate f (x) by the truncated Taylors series f (x ) f (x0 ) + D f (x0 )(x x0 ) (3)
x1 . . D1 f (x, u ) = . fp (x, u ) x1
(x, u )
.. .
f1 (x, u ) xn . . . fp (x, u ) xn
(11)
GAIN SCHEDULING
207
and D2 f (x, u) denotes the p m Jacobian matrix of partial derivatives with respect to the second variable,
1 (x, u ) u1 . . D2 f (x, u ) = . fp (x, u ) u1
and f ( x0 ) = 0 (21)
.. .
f1 (x, u ) um . . . fp (x, u ) um
(12)
Substituting these formulas into (18) and neglecting the residual term, r(x), leads to the approximate dynamics . x = Ax (22) where A = D f ( x0 ) (23)
As before, if f is continuously differentiable, we may approximate f (x, u) by f (x, u ) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 ) + D2 (x0 , u0 )(u u0 ) (13) Let
r(x, u ) = f (x, u ) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 ) + D2 (x0 , u0 )(u u0 )

denote the approximation residual. Then as before,
(14)
Equation (22) is called the linearization of Eq. (16) about the equilibrium x0. Intuitively, whenever x x0 is small, then x in the linearization should be a good approximation of x x0, and hence the linear dynamics of Eq. (22) should be a good approximation of the nonlinear dynamics of Eq. (16). It is often possible to make more denite statements about nonlinear dynamics based on an analysis of the linearization, in particular regarding the stability of the nonlinear system. First, recall the following stability denitions. Denition 3. Let x0 be an equilibrium of Eq. (16).
xx 0 uu 0
lim
r(x, u ) |x x0 |2 + |u u0 |2
=0
(15)
x0 is stable if for any 0, there exists a 0 such that |x(0 ) x0 | < |x(t ) x0 | (24)
Equations (5) and (15) indicate that the approximations are accurate up to the rst order. Linearization of Autonomous Systems In the last section, we saw how to approximate the static behavior of a nonlinear function by using a truncated Taylors series. We now show how similar tools can be used to approximate the dynamic behavior of a nonlinear system. Consider an autonomous nonlinear system x = f (x ) Denition 2. The vector x0 is an equilibrium of (16) if f ( x0 ) = 0 (17) (16)
Otherwise, x0 is unstable. x0 is asymptotically stable if in addition to being stable, |x(0 ) x0 | < lim x(t ) = x0
t
(25)
In words, stability implies that the solution to Eq. (16) stays near x0 whenever it starts sufciently close to x0, whereas asymptotic stability implies that the solution also asymptotically approaches x0. Theorem 4. Let f in Eq. (16) be continuously differentiable. The equilibrium x0 is asymptotically stable if all of the eigenvalues of Df (x0) have negative real parts. It is unstable if Df (x0) has at least one eigenvalue with a positive real part. Since the eigenvalues of Df (x0) determine the stability of the linearization (22), Theorem 4 states that one can assess the stability of a nonlinear system based on its linearization. Example 5. The equations of motion for a pendulum of length are g d d2 sin( ) = 0 +c dt 2 dt (26)
The reasoning for this terminology is that the initial condition x(0) x0 leads to the solution x(t) x0 for all time. So if the solution starts at x0, it remains at x0, hence the term equilibrium. In case f is continuously differentiable, we may rewrite (16) as x = f (x0 ) + D f (x0 )(x x0 ) + r(x ) (18)
where r(x) denotes the residual error in the approximation f (x ) f (x0 ) + D f (x0 )(x x0 ) Since x0 is both xed and an equilibrium, d dx ( x x0 ) = dt dt (20) (19)
where is the pendulum angle measure positive clockwise with 0 being the upright position, c is a friction coefcient, and g is gravitational acceleration. In state space form, these equations becomes x2 d x1 (27) = g sin(x1 ) cx2 dt x2
. where x1 , and x2
208
GAIN SCHEDULING
Linearizing about the upright equilibrium x0 0 leads to
x =
0 g/
1 x c
(28)
with the constant input u(t) u0 leads to the constant solution x(t) x0. Proceeding as in the autonomous case, whenever f is continuously differentiable, we may rewrite (33) as x = f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 ) + D2 f (x0 , u0 )(u u0 ) + r(x, u )] (35)
The linearization has an eigenvalue with positive real part. Hence, the upright equilibrium of the pendulum is unstable, as expected. Linearizing about the hanging equilibrium
where r(x, u) denotes the residual error in the approximation f (x, u ) f (x0 , u0 ) + D1 f (x0 , u0 )(x x0 ) + D2 f (x0 , u0 )(u u0 ) (36)
x0 =
leads to
x =
0 g/
1 x c
(29)
Dropping this residual term and using that f (x0, u0) 0 leads to the approximate linear dynamics . x = Ax + Bu (37) where
which is asymptotically stable. Therefore, the hanging equilibrium of the nonlinear pendulum is asymptotically stable, as expected. The above example demonstrates that different equilibrium points of the same nonlinear system can have different stability conditions. In some cases, using the linearization to assess stability may be inconclusive. Example 6. Consider the scalar nonlinear systems x = x3 and x = x3 (31) (30)
A = D1 f (x0 , u0 ),
B = D 2 f ( x0 , u 0 )
(38)
As before, Eq. (37) is called the linearization of the nonlinear Eq. (33) about the equilibrium (x0, u0). The quantity x approximates x x0, whereas the quantity u equals u u0, exactly. Denition 8. The equilibrium (x0, u0) of Eq. (33) is stabilized by the state feedback u G(x) if u0 G(x0) x0 is a stable equilibrium of the closed loop dynamics x = f (x, G(x )) The following is a direct consequence of Theorem 4. Theorem 9. Let f in Eq. (33) be continuously differentiable, and let Eq. (37) be the linearization of Eq. (33) about the equilibrium (x0, u0). Suppose the static linear feedback, u = K x (40) (39)
It is easy to see that the equilibrium x0 0 is asymptotically stable for the former system, while the same equilibrium is unstable for the latter system. Both systems have the same linearization at the equilibrium x0 0, . x =0 (32) (note that x represents different quantities in the two linearizations). In this case, stability analysis of linearization is inconclusive; it does not indicate either the stability or instability of the equilibrium x0 0. Linearization of Systems with Controls It is also possible to use linearization methods to synthesize controllers for a nonlinear system. Consider the controlled system x = f (x, u ) (33)
stabilizes the linearization of Eq. (37). Then the equilibrium (x0, u0) of Eq. (33) is stabilized by the feedback u = u 0 K ( x x0 ) (41)
Theorem 9 states that we can construct stabilizing feedback for a nonlinear system by designing stabilizing feedback for its linearization. Example 10. Recall the simple pendulum example, but now with a control torque input, x2 d x1 (42) = g sin(x1 ) cx2 + u dt x2 Linearizing about the upright equilibrium leads to
Denition 7. The pair (x0, u0) is an equilibrium of Eq. (33) if f ( x0 , u 0 ) = 0 (34)
The reasoning behind the term equilibrium is similar to the autonomous case. The initial condition x(0) x0 along
. x =
0 g/
1 0 x + u c 1
(43)
GAIN SCHEDULING
209
The feedback u = (k1 k 2 )x (44)
Then, the linearization about the equilibrium (x0, u0),
. x = Ax + Bx y = Cx
where A = D1 f (x0 , u0 ), B = D2 f (x0 , u0 ), C = Dg(x0 )
(53)
stabilizes the linearization for any k1 g / and k2 c. From Theorem 9, the feedback u = u0 (k1 = (k1 k2 )(x x0 ) (45)
k 2 )x
(54)
stabilizes the upright equilibrium, where we used that x0 0 and u0 0. In some cases, analysis of the linearization does not aid in the construction of stabilizing feedback. Example 11. Consider the scalar nonlinear system x = x + xu (46)
approximates the input-output behavior of Eq. (33) with measurement of Eq. (49). Here, x approximates x x0, y approximates y y0, and u exactly represents u u0. Theorem 13. Let f in Eq. (33) and g in Eq. (49) be continuously differentiable, and let Eq. (53) be the linearization of Eq. (33) about the equilibrium (x0, u0). Suppose the linear feedback,
z = Az + By u = Cz
(55)
Linearizing about the equilibrium (x0, u0) (0, 0) leads to . x =x (47)
stabilizes the linearization of Eq. (53). Then the equilibrium (x0, u0) of Eq. (33) is stabilized by the output feedback
which is not stabilizable. However, the constant feedback u 2 leads to the closed loop equations x = x which is stable. Now suppose that the state is not available for feedback. Rather, the control is restricted to measurements y = g (x ) (49) (48)
z = Az + B(y g(x0 )) u = u0 + Cz
(56)
Example 14. Suppose we wish to control the simple pendulum under the output feedback y = ( 1 0 )x Linearizing about the upright equilibrium leads to (57)
By using a similar analysis, we can construct stabilizing output feedback for the nonlinear system based on stabilizing output feedback for the linearization. Denition 12. The equilibrium (x0, u0) of Eq. (33) is stabilized by the dynamic output feedback z = F ( z, y ) u = G (z ) if for some z0, (z0, g(x0)) is an equilibrium of Eq. (50) u0 G(z0) (x0, z0) is an asymptotically stable equilibrium of the closed loop dynamics x = f (x, G(z )) z = F (z, g(x )) (51) (50)
. x =
0 g/
1 0 def x + u = Ax + Bu c 1
def
(58)
y = ( 1 0 )x = Cx
The observer based controller z = (A BK HC )z + H y u = Kz stabilizes the linearization for appropriate gain matrices (59)
K = (k1
k2 ),
H=
h1 h2
(60)
Since x0, u0, y0 0, the same controller (with input y and output u) stabilizes the nonlinear pendulum. BASIC GAIN SCHEDULING Gain Scheduled Command Following We now outline the basic procedure of gain scheduling in the context of command following. The nonlinear plant of interest is Eq. (33). The objective is to make the measured output Eq. (49) approximately follow reference commands, r.
Let (x0, u0) be an equilibrium of Eq. (33), and dene y0 g(x0). In case g is continuously differentiable, we can approximate Eq. (49) as y y0 + Dg(x0 )(x x0 ) (52)
210
GAIN SCHEDULING
The primary motivation of gain scheduling is to address local limitations associated with a control design based on a single linearization. The main problem is that the performance and even stability of the closed loop system can deteriorate signicantly when the system is not operating in the vicinity of the equilibrium. Example 15. Consider the scalar system x = x|x| + u y=x
Example 17. Recall the controlled pendulum x2 d x1 = g sin(x1 ) cx2 + u dt x2 An equilibrium family over the set S [, ] is
(67)
xeq (s ) =
(61)
s , 0
g ueq (s ) = sin s
(68)
By linearizing about the equilibrium (x0, u0) (0, 0), we obtain the linear control u x r. For r 0, this control law stabilizes the equilibrium (0, 0). The resulting closed loop system is x = x|x| x (62)
The associated linearization family is 0 1 0 x + u x = g 1 cos (s ) c
(69)
Step 2: Fixed Operating Condition Designs. Let us select several operating conditions, {s1 , s2 , . . ., sN } S (70)
For x(0) 1, the solution asymptotically approaches 0. However, for x(0) 1, the solution diverges to innity. Step 1: Construction of Linearization Family. Gain scheduling attempts to overcome local limitations by considering a family of linearizations, rather than a single linearization. Denition 16. The functions (xeq( ), ueq( )) dene an equilibrium family for the nonlinear system Eq. (33) over the set S if f (xeq (s ), ueq (s )) = 0 for all s S. Associated with an equilibrium family are the output equilibrium values yeq (s ) = g(xeq (s ))
def
which characterize the variations within the operating envelope. At the ith equilibrium, si, we can linearize the plant dynamics about the equilibrium (xeq(si), ueq(si)) and design a stabilizing linear controller to achieve approximate command following using any suitable linear design methodology. The result is an indexed collection of controllers,
z = A i z + Bi y + Li r u = Ci z
(63)
(71)
where r denotes the reference command in local coordinates, r = r yeq (si ) (72)
(64)
The equilibrium family induces the following linearization family for Eq. (33) with measurement Eq. (49), . x = A ( s )x + B ( s )u y = C ( s )x where
This step constitutes the core of gain scheduling, and accordingly, accounts for the bulk of the effort in a gain scheduled control design. Designing xed operating point controllers is especially tedious in the case of several design operating conditions. Step 3: Scheduling. The remaining step is to piece together a global controller from the individual local controllers. As the scheduling variable varies in time, the control gains are updated to reect the current operating condition of the plant. The resulting overall controller is
(65)
A(s ) = D1 f (xeq (s ), ueq (s )), B(s ) = D2 f (xeq (s ), ueq (s )), C(s ) = Dg(xeq (s ))
The variable, s, which we will call the scheduling variable, parameterizes a family of equilibrium points and plant linearizations. Typically, s can be a combination of both endogenous and exogenous signals (recall discussion of missile autopilot earlier). Any xed s will be called an operating condition, and the set S denes the operation envelope, or range of operating conditions.
z = A(s )z + B(s )(y yeq (s )) + L(s )(r yeq (s )),

(66)
u = ueq (s ) + C(s )z
(73)
The matrices A(s), B(s), C(s), and L(s) are functions of the scheduling variable, as are the vectors yeq(s) and ueq(s). It is important to note that the scheduling variable, s, which was held constant during the design phase, is now time varying. These matrix and vector functions are used to update the control parameters according to the variations in the scheduling variable. There are different options in how to schedule the controller parameters in Eq. (73).
GAIN SCHEDULING
211
Switched Scheduling. The operating envelope is divided into disjoint regions, Ri, so that S R1 . . . RN (74)
which are globally stable, as opposed to the local stability of Example 15. Notice in this example that the linear control term (x xeq(s)) has no effect on the closed loop equations. This is because of the smooth scheduling implementation with s x as the scheduling variable. In this case, a desirable feedback loop was eliminated in the scheduling implementation. It is also possible to introduce undesirable feedback during implementation. Theoretical Foundations The gain scheduled controller is designed so that stability and performance are achieved whenever the plant is in the vicinity one of the design operating conditions. Since the plant actually varies throughout the entire operating regime, an important question is to what degree the local properties of the individual operating point designs carry over to the global system. The overall closed loop equations for a gain scheduled system are
and the controller matrices in (73) are scheduled according to A1 , . s R 1 ; . , . . ., ueq (s ) A(s ) = . A , . s R N N ueq (s1 ), s R1 ; . . = (75) . s R u (s ), N eq N Continuous Scheduling. Any interpolation algorithm is used to construct continuous matrices which interpolate the design conditions so that A(si ) = Ai , . . ., ueq (si ) = ueq (si ) Some important points to consider are the following: Gain scheduling is still based on linearizations, and hence can ignore important nonlinear phenomena. The xed operating point designs assume a constant scheduling variable which is actually time varying. Implementing the gain scheduled controller introduces feedback loops which are not present in the xed operating point designs. Example 18. Recall the system of Example 15. The equilibrium family xeq (s ) = s, ueq (s ) = s|s| (77) (76)
x = f (x, u ) z = A ( s )z + B ( s )y + L ( s )r u = C(s )z + ueq (s ) y = g(x ) yeq (s ) r = r yeq (s )

In general, the scheduling variable, s, can be written as s = (x, r ) (84)
(83)
leads to the linearization family . x = 2|s|x +u (78)
Because of the simplicity of this system, we are able to design controls for all s, rather than selected s. A suitable linear design for command following is u = 3|s|x + (r x ) (79)
for an appropriate function, . Clearly, the overall system is nonlinear and hence, requires nonlinear methods for analysis. An analysis of these equations (see Bibliography for sources) leads to the conclusion that the overall gain scheduled system will exhibit similar stability and performance as the local designs whenever (1) the scheduling variable, s, changes sufciently slowly, and (2) the plant dynamics are predominantly nonlinear in the scheduling variable. The following sections provide some insight into these restrictions. LPV Systems. It is convenient to consider slow variation restriction in the context of linear parameter varying (LPV) systems. LPV systems are dened to be linear systems whose dynamics depend on exogenous time varying parameters which are unknown a priori, but can be measured upon operation of the control system. An LPV system can be represented in state space form as x = A( )x + B( )u y = C( )x (85)
Implementing this design using smooth scheduling leads to the gain scheduled control u = ueq (s ) + u = ueq (s ) 3|s|(x xeq (s )) + ((r xeq (s )) (x xeq s )) For the scheduling variable s x, the control becomes u = x|x| + (r x ) This feedback leads to the closed loop dynamics x = x + r (82) (81) (80)
where is a time varying parameter. Typical assumptions on are magnitude bounds; for example, | | max (86)
212
GAIN SCHEDULING
and rate bounds; for example, | max | (87)
By subtracting equation (91) from equation (89), we obtain
LPV systems form the underlying basis of gain scheduling. It is convenient to associate the parameter with the scheduling variable and the LPV structure with the linearization family, although this is not always the case as will be seen. The following is a classical result from differential equations stated in an LPV context. Theorem 2. If the equilibrium xo 0 of Eq. (85) is asymptotically stable for all constant , then it is asymptotically stable max is sufciently small. for all time varying provided that The relevance of Theorem 19 to gain scheduling is as follows. A closed loop LPV system is such that good stability and performance is expected for xed values of the parameter/ scheduling variable. However, performance and even stability can deteriorate in the presence of parameter time variations. Theorem 19 provides a sufcient condition for the xed parameter properties to carry over to the varying parameter setting. Example 20. A classical example of instability from xed parameter stability is the time-varying oscillator,
d dt
s 0 = M (s ) + B(u ueq (s )) v v veq (s )
(92)
If veq(s) is differentiable,
d veq (s ) = Dveq (s )s dt = Dveq (s )M12 (s )(v veq (s )) + Dveq (s )B1 (u ueq (s )) (93)
where the matrices M12(s) and B1 are appropriate sub-matrices of M(s) and B. Combining these equations leads to the alternate form of Eq. (89),
d dt
s v veq (s )
= 0 0 M12 (s ) M22 (s ) Dveq (s )M12 (s ) s v veq (s )
(94)
(95)
B1 (u ueq (s )) + B2 Dveq (s )B1

which can be written as
x (t ) =
0 (1 + (t )/2 )
1 x(t ) 0.2
(88)
These equations can be viewed as a mass-spring-damper system with time-varying spring stiffness. For xed parameter values, (t) o, the equilibrium xo 0 is asymptotically stable. However, for the parameter trajectory (t) cos(2t), it becomes unstable. An intuitive explanation is that the stiffness variations are timed to pump energy into the oscillations. Quasi-LPV Representation. It is also convenient to consider the relationship between the scheduling variable and plant nonlinearities in an LPV setting. The relationship between LPV systems and gain scheduling is not limited to linearization families. Consider the following special nonlinear plant in which the scheduling variable is a subset of the state,
d dt
where
s s = Anew (s ) + Bnew (s )u v v
(96)
v (t ) = v(t ) veq (s(t )),
u (t ) = u(t ) ueq (s(t ))
(97)
d dt
s s = (s ) + M (s ) + Bu v v
(89)
The original equations now take a quasi-LPV form, where the parameter is actually an endogenous variable. Note that no linearization approximations were made to bring Eq. (89) to the form Eq. (96). This transformation shows that an underlying LPV structure exists, even without linearizations, in the extreme case that the plant dynamics are nonlinear only in the scheduling variable. Any additional nonlinearities not captured by the scheduling variable enter as high order perturbations in Eq. (96). This transformation then reveals the importance of the scheduling variable to capture the plant nonlinearities. Example 21. Consider the nonlinear system
These equations represent the extreme case where the nonlinearities are entirely captured in the scheduling variable, s. Let (xeq(s), ueq(s)) be an equilibrium family, with
s xeq (s ) = veq (s )
so that
x = (90)
x1 |x1 | + x2 x2 1 x2 + u
(98)
and let s x1 be the scheduling variable. These equations take the form of Eq. (89). The resulting equilibrium family is
0 = (s ) + M (s )
s + Bueq (s ) veq (s )
(91)
xeq (s ) =
s , s|s|
ueq (s ) = s3 |s|
(99)
GAIN SCHEDULING
213
Performing the transformations described above leads to the quasi-LPV form
In order to convexify the problem, consider the change in variables Q = P 1 , Yi = Ki P 1 (106)
d dt
s = x2 (s|s| )
0 0
1 s2 2|s|
s x2 (s|s| )
(100)
0 + (u (s3 |s| ) 1
Given Q 0 and Yi, one can solve for the original variables P and Ki. With these variables, condition Eq. (105) is equivalent to
T T Ai Q BYi + QAT i Yi B < 0
(107)
ADVANCED METHODS FOR GAIN SCHEDULING Convex Optimization for LPV Systems Because of the availability of numerically efcient methods for large scale problems, convex optimization is an emerging technique for gain scheduling design of for LPV and quasiLPV systems. As seen in the previous section, the development of methods for LPV systems is directly pertinent to gain scheduling, since LPV systems form the underlying structure of a gain scheduled design. The main idea in convex optimization methods for LPV systems is to combine stability and performance parameters with controller parameters in a single convex optimization objective. We will demonstrate these methods in the following simple context. Consider the open loop LPV system x = ( A1 + (1 )A2 )x + Bu where the parameter is constrained by 0 1 (102) (101)
Now the set of Q 0, Y1, and Y2 which satisfy Eq. (107) is convex. This allows one to employ efcient convex feasibility algorithms which either produce a feasible set of matrices or determine denitely that no solution exists. Some important points to consider are the following: The scheduling process is built into the construction of the state feedback; it is not necessary to perform several xed parameter designs. Stability for arbitrarily fast parameter variations is assured. Theorem 22 is only a sufcient condition for stability, and hence may be conservative. The method extends to more general control objectives, other than stabilizing state feedback, including disturbance rejection and command following output feedback rate constrained parameter variations Extended/Pseudo-Linearization The objective in extended and pseudo-linearization is to impose that the closed loop system has a linearization family which is invariant in some desired sense. Let us consider the special case of a tracking problem with full state feedback for the nonlinear system Eq. (33). The objective is for the rst state, x1, to approximately track reference commands, r. Let the equilibrium family (xeq(s), ueq(s)) be an such that (1 0 ... 0 )xeq (s ) = s (108)
We are interested in constructing stabilizing gain scheduled state feedback. Let us impose the feedback structure u = ( K1 + (1 )K2 )x (103)
which mimics the LPV variations of the system. The closed loop dynamics are then x = ( (A1 BK1 ) + (1 )(A2 BK2 ))x (104)
A sufcient condition which guarantees the stability of (104) is the following. Theorem 22. The equilibrium xo 0 of Eq. (104) is asymptotically stable if there exists a positive denite matrix, P PT 0, such that for both i 1 and i 2, P(Ai BKi ) + (Ai BKi )T P < 0 (105)
Now consider the nonlinear feedback u = G(x, r ) where ueq (s ) = G(xeq (s ), s ) Then, a linearization family for the closed loop system x = f (x, G(x, r )) is (111) (110) (109)
It is important to note that Theorem 22 only provides a sufcient condition for stability. The main idea is that the matrix P denes the Lyapunov function, V(x) xTPx, for the closed loop system Eq. (104). Our objective is to nd matrices K1, K2, and P which satisfy Eq. (105). It can be shown that the set of matrices which satisfy Eq. (105) is not convex. This lack of convexity signicantly complicates any direct search process.
x = (D1 f (xeq (s ), G(xeq (s ), s )) + D2 f (xeq (s ), G(xeq (s ), s ))D1 G(xeq (s ), s ))x + D2 f ((xeq (s ), G(xeq (s ), s ))D2 G(xeq (s ), s )r (112)
214
GAIN SCHEDULING H. K. Khalil and P. K. Kokotovic, On stability properties of nonlinear systems with slowly varying inputs, IEEE Trans. Autom. Control, 36: 229, 1991. M. Vidyasagar, Nonlinear Systems Analysis, Nonlinear Systems Analysis, Englewood Cliffs, NJ: Prentice-Hall, Inc., 1993. A. Bacciotti, Local Stabilizability of Nonlinear Control Systems, Singapore: World Scientic Publishing Co., 1992. H. K. Khalil, Nonlinear Systems, 2nd Ed., New York: Macmillan, 1996. Overview of Gain-Scheduling and Its Theoretical Foundations J. Wang and W. J. Rugh, On parameterized linear systems and linearization families for nonlinear systems, IEEE Trans. Circuits Syst., 34: 650657, 1987. J. S. Shamma and M. Athans, Analysis of nonlinear gain-scheduled control systems, IEEE Trans. Autom. Control, 35: 898907, 1990.
One invariance objective is for the closed loop linearization family to be constant; for example, . x = Adesx + Bdes r (113)
One can state appropriate conditions involving partial differential constraint equations under which there exists any G which achieves this objective. The intention is that improved closed loop performance is possible if the closed loop linearizations have some sort of invariance property. Such improved performance, if any, is difcult to quantify, but simulation studies indicate the potential benets. Example 23. Consider the nonlinear system
x 1 x 2
x2 x1 x2 + x2 1 +u
(114)
W. J. Rugh, Analytical framework for gain-scheduling, IEEE Control Syst. Magazine, 11: 7984, 1991. J. S. Shamma and M. Athans, Gain scheduling: Potential hazards and possible remedies, IEEE Control Syst. Magazine, 12: 101 107, 1992.
The equilibrium family suggested by x1 is
xeq (s ) =
s , 0
ueq (s ) = s2
(115)
LPV System Analysis and Control J. S. Shamma and M. Athans, Guaranteed Properties of Gain Scheduled Control of Linear Parameter Varying Plants, Automatica, 27: 559564, 1991. S. M. Shahruz and S. Behtash, Design of controllers for linear parameter-varying systems by the gain scheduling technique, J. Math. Anal. Appl., 168: 125217, 1992. A. Packard, Gain scheduling via linear fractional transformations, Syst. Control Lett., 22: 7992, 1994.
Suppose we want the closed loop linearization family to have dynamics matrices
0 Ades = 1
1 , 2
Bdes
0 = 1
(116)
An obvious selection for G(x, r) is G(x, r ) = x1 x2 x2 1 x 1 2x 2 + r (117)
P. Apkarian and P. Gahinet, A convex characterization of gain-scheduled H controllers, IEEE Trans. Autom. Control, 40: 853864, 1995. P. Apkarian and R. J. Adams, Advanced gain-scheduling techniques for uncertain systems, IEEE Trans. Control Syst. Technol., 60: 21 32, 1998. Extended and Pseudo-Linearization C. Reboulet and C. Champetier, A new method for linearizing nonlinear systems: the pseudolinearization, Int. J. Control, 40: 631 638, 1984. W. T. Baumann and W. J. Rugh, Feedback control of nonlinear systems by extended linearization, IEEE Trans. Autom. Control, 31: 4046, 1986. J. Wang and W. J. Rugh, Linearized model matching for single-input nonlinear systems, IEEE Trans. Autom. Control, 33: 793796, 1988. J. Huang and W. J. Rugh, On a nonlinear servomechanism problem, Automatica, 26: 963972, 1990. J. Huang and W. J. Rugh, Approximate noninteracting control with stability for nonlinear systems, IEEE Trans. Autom. Control, 36: 295304, 1991. D. A. Lawrence and W. J. Rugh, Input-output pseudolinearization for nonlinear systems, IEEE Trans. Autom. Control, 39: 22072218, 1994. I. Kaminer, A. M. Pascoal, P. P. Khargonekar, and E. Coleman, A velocity algorithm for the implementation of gain-scheduled controllers, Automatica, 31: 11851192, 1995. D. A. Lawrence and W. J. Rugh, Gain scheduling dynamic linear controllers for a nonlinear plant, Automatica, 31: 381390, 1995.
Then setting u G(x, r) leads to the linear closed loop dynamics
x =
0 1
1 0 x+ r 2 1
(118)
The above choice for G achieves what is called feedback linearization, since the resulting closed loop dynamics are linear. It is not necessary that feedback linearization is achieved. For example, consider the modied dynamics
x 1 x 2
x2 + x2 2u x1 x2 + x2 1 +u
(119)
The resulting equilibrium families and linearization families are the same. Furthermore, the same choice of G achieves the desired invariance objective. FOR FURTHER REFERENCE
Nonlinear Systems Analysis M. Kelemen, A stability property, IEEE Trans. Autom. Control, 31: 766768, 1986. D. A. Lawrence and W. J. Rugh, On a stability theorem for nonlinear systems with slowly varying inputs, IEEE Trans. Autom. Control, 35: 860864, 1990.
GALLIUM ARSENIDE FIELD EFFECT TRANSISTOR LOGIC CIRCUITS Applications of Gain Scheduling K. J. Astrom and B. Wittenmark, Adaptive Control, Chapter 9, Reading, MA: Addison-Wesley, 1989. R. A. Nichols, R. T. Reichert, and W. J. Rugh, Gain scheduling for H controllers: A ight control example, IEEE Trans. Control Syst. Technol., 1: 6979, 1993. J. S. Shamma and J. R. Cloutier, Gain-scheduled missile autopilot design using linear parameter varying methods, J. Guidance, Control, Dynamics, 16: 256263, 1993. J. Reeve and M. Sultan, Gain scheduling adaptive control strategies for HVDC systems to accommodate large disturbances, IEEE Trans. Power Syst., 9: 366372, 1994. T. Meressi and B. Paden, Gain scheduled H controllers for a 2 link exible manipulator, J. Guidance, Control, Dynamics, 17: 537 543, 1994. P. Apkarian, P. Gahinet, and J. M. Biannic, Self-scheduled H-innity control of a missile via LMIs. AIAA J. Guidance, Control, Dynamics, 18: 532538, 1995. L. H. Carter and J. S. Shamma, Gain scheduled bank-to-turn autopilot design using linear parameter varying transformations, AIAA J. Guidance, Control, Dynamics, 19: 10561063, 1996. D. J. Leith and W. E. Leithead, Appropriate realization of gain scheduled controllers with application to wind turbine regulation, Int. J. Control, 65: 223248, 1996. R. D. Smith, W. F. Weldon, and A. E. Traver, Aerodynamic loading and magnetic bearing controller robustness using a gain-scheduled Kalman lter, J. Eng. Gas Turbines Power, 118: 846842, 1996. Alternative Perspectives on Gain-Scheduling E. Gazi, W. D. Seider, and L. H. Ungar, Control of nonlinear processes using qualitative reasoning, Comput. Chem. Eng., 18: 189 193, 1994. K. Pawelzik, J. Kohlmorgen, and K. R. Muller, Annealed competition of experts for a segmentation and classication of switching dynamics, Neural Computat., 8: 340356, 1996. S. C. Kramer and R. C. Martin, Direct optimization of gain scheduled controllers via genetic algorithms, J. Guidance, Control, Dynamics, 19: 3846, 1996.
215
JEFF S. SHAMMA
The University of Texas at Austin
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G
Recommend to Your Librarian
Browse this title

G
Save title to My Profile
H Infinity Control Standard Article Kemin Zhou1

1Louisiana
G G
Email this page Print this page
State University, Baton Rouge, LA

H
Advanced Product Search
Copyright 1999 by John Wiley & Sons, Inc. All rights reserved.
H
Search All Content Acronym Finder
DOI: 10.1002/047134608X.W1015
H
Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (345K)
Abstract
The sections in this article are and Norms Performance
Weighted
Robust Stabilization Selection of Weighting Functions General Problem Formulation and Solutions
Filtering Control
Understanding
Loop Shaping
Synthesis Additional Applications About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
H INFINITY CONTROL
This article describes an optimal multivariable control system design technique for achieving robustness under external disturbances and model uncertainties. We show that many robust control problems can be formulated as H norm optimization problems, and we describe analytically their solutions. We also give guidelines to the choice of design parameters and insights to this optimal and robust control theory. One of the motivations for the original introduction of H methods by Zames (1) was to bring plant uncertainty, specied in the frequency domain, back to the center stage as it had been in classical control, in contrast to analytic methods such as linear quadratic Gaussian (LQG) control. The H norm was found to be appropriate for specifying both the level of plant uncertainty and the signal gain from disturbance inputs to error outputs in the controlled system. The standard H optimal control problem is concerned with the feedback system shown in Fig. 1(a) where w represents an external disturbance, y is the measurement available to the controller, u is the output from the controller, and z is an error signal that should be kept small. The transfer function matrix G represents not only the conventional plant to be controlled but also any weighting functions included to specify the desired performance, which will be discussed in more detail later. Suppose that G is partitioned consistent with the inputs w and u and outputs z and y as
The closed loop transfer function from w to z, denoted by T zw , is dened to be the linear fractional transformation (LFT) of G on K :
The H optimal control problem is then to design a stabilizing controller K , so as to minimize the H norm of T zw , which is dened in the next section and is denoted by T zw . The H norm gives the maximum energy gain, or sinusoidal gain of the system. This is in contrast to the H 2 norm T zw 2 , which for example gives the variance of the output given white noise disturbances. The important property of the H norm comes from the application of the small gain theorem, which states that if T zw , then the system in Fig. 1(b) will be stable for all stable with < 1/ . It is probably the case that this robust stability consequence was one of the main motivations for the development of H methods. The synthesis of controllers that achieve an H norm specication hence gives a well-dened practical and mathematical problem.
1
H INFINITY CONTROL
Fig. 1. Most control systems can be put in this unied framework where w represents an external disturbance, y is the measurement available to the controller, u is the output from the controller, and z is an error signal that it is desired to keep small.
H 2 and H Norms
We consider a q-input and p-output dynamical system with the matrix transfer function G(s). Let G(s) have the following stabilizable and detectable state space realization:
We shall denote this state-space realization by
Many control design problems can be regarded as nding a suitable controller so that the undesirable responses of the system are made small in some sense. There are obviously many ways to dene the smallness for a given control problem. Here we are mainly interested in one way of dening the smallness: the H norm. For comparison we shall also mention another more classical way of dening the smallness in terms of the H 2 norm. Let R H 2 denote the set of strictly proper and real rational stable transfer matrices. In terms of state-space realizations, R H 2 is simply the set of nite dimensional systems with D = 0 and stable A. The H 2 norm of a G(s) R H 2 is dened as
H INFINITY CONTROL
where trace(M ), M , and i (M ) denote, respectively, the trace, the complex conjugate transpose, and the ith singular value of the matrix M . This norm can also be computed in the time domain as
where g(t) is the inverse Laplace transform of G(s). Thus the H 2 norm of a system is a measure of the total energy of the system impulse response. It can be computed using state-space realization as
where Q and P are observability Gramian and controllability Gramian, which can be obtained from the following Lyapunov equations
Let R H denote the set of proper (but not necessarily strictly proper) and real rational stable transfer matrices. In terms of state-space realizations, R H includes all nite dimensional systems with stable A matrices. The H norm of a transfer matrix G(s) R H is dened as
where (M ) denotes the largest singular value of a matrix M . When G(s) is a single input and single output system, the H norm of the G(s) is simply the peak value on the Bode magnitude plot of the frequency response G(j). It can also be regarded as the largest possible amplication factor of the systems steady state response to sinusoidal excitations. For example, the steady state response of the system with respect to a sinusoidal input u(t) = U sin(0 t + ) is
and thus the maximum possible amplication factor is
which is precisely the H norm of the transfer function. As an example, consider a standard second-order system
Then max = n 1 22 and G = |G(jmax )| = 1/(2 1 2 ). If G(s) is the description of a structure vibration, then max would be the most dangerous exciting frequency. In the multiple input and multiple output (MIMO) case, the H norm of a transfer matrix G R H is the peak value on the largest singular value Bode plot of the frequency response G(j). Analogous to the scalar
H INFINITY CONTROL
case, the H norm of G(s) can also be regarded as the largest possible amplication factor of the systems steady state response to sinusoidal excitations in the following sense: Let the sinusoidal inputs be
Then the steady state response of the system can be written as
for some yi , i , i = 1, 2, . . ., p. Furthermore,
where is the Euclidean norm. The H norm of a stable transfer matrix can also be thought of as the maximum amplication factor of the input energy of the system at the output of the system. More precisely,
where denotes the time domain convolution and L 2 [0, ) denotes the space of all square integrable functions with the norm dened as f 2 : = . Thus it is important to make the H norm of all undesirable transfer functions small in a feedback control system. That is one of the motivations for the development of H control theory. This discussion shows that the H norm of a transfer function can, in principle, be obtained either graphically or experimentally. To get an estimate, set up a ne grid of frequency points, {1 , . . ., N }. Then an estimate for G is
H INFINITY CONTROL
This value is usually read directly from a singular value Bode plot. The H norm can also be computed directly using state-space representations. Let
Then
where
Hence, the H norm of a matrix transfer function can be calculated to the specied accuracy by a bisection search. Example. Consider a mass/spring/damper system as shown in Fig. 2. The dynamical system can be described by the following differential equations:
Suppose that G(s) is the transfer matrix from (F 1 , F 2 ) to (x1 , x2 ) and suppose k1 = 1, k2 = 4, b1 = 0.2, b2 = 0.1, m1 = 1, and m2 = 2 with appropriate units. Then the H 2 norm of this transfer matrix is G(s) 2 = 2.56, whereas the H norm of this transfer matrix is G(s) = 11.47, which is shown as the peak of the largest singular value Bode plot in Fig. 3. Since the peak is achieved at max = 0.8483, exciting the system using the
H INFINITY CONTROL
Fig. 2. A two mass/spring/damper system with the external forces F 1 and F 2 as inputs and the positions of the two masses as outputs.
Fig. 3. The singular value Bode plot of the two mass/spring/damper system. The H norm G singular value plot of G(j).
is the peak of the largest
following sinusoidal input
gives the steady state response of the system as
This shows that the system response will be amplied 11.47 times for an input signal at the frequency max , which could be undesirable if F 1 and F 2 are disturbance force and x1 and x2 are the positions to be kept steady. We will see later how to design an H feedback controller to suppress this kind of vibration. We note that the state-space computational method is usually much more accurate than the graphical method. Consider, for example, the standard second-order system again with n = 1 and = 0.01. By the
H INFINITY CONTROL
analytic formula or the state-space computational method, we get that the H norm is 50.0025. To estimate the H norm graphically, we set up a frequency grid to compute the frequency response of G over a suitable range of frequency. Take, for example, 200 points in the frequency range of [0.1, 10] uniformly on the log scale, then we get an estimate of the norm 33.0743. This shows clearly that the graphical method may lead to a wrong answer for a lightly damped system if the frequency grid is not sufciently dense. Indeed, we would get the H norm 43.5567, 48.1834, and 49.5608 from the graphical method if 400, 800, and 1600 frequency points are used, respectively.
Weighted H Performance
We now consider how to formulate some performance objectives into mathematically tractable problems. It is well known that the performance objectives of a feedback system can usually be specied in terms of requirements on the sensitivity functions and/or complementary sensitivity functions or in terms of some other closed-loop transfer functions. For instance, the performance criteria for a scalar system may be specied as requiring
with S(j) = 1/(1 + P(j)K (j)) where P is the plant and K is the controller. However, it is much more convenient to reect the system performance objectives by choosing appropriate weighting functions. For example, this performance objective can be written as
with
In order to use W e in control design, a rational transfer function W e (s) is usually used to approximate this frequency response. The advantage of using weighted performance specications is obvious in multivariable system design. First, some components of a vector signal are usually more important than others. Second, each component of the signal may not be measured in the same units; for example, some components of the output error signal may be measured in terms of length, and the others may be measured in terms of voltage. Weighting functions are essential to make these components comparable. Also, we might be primarily interested in rejecting errors in a certain frequency range (e.g., low frequencies); hence, some frequency-dependent weights must be chosen. Consider a standard feedback diagram in Fig. 4. The weighting functions in Fig. 4 are chosen to reect the design objectives and knowledge on the disturbances and sensor noise. For example, W d and W i may be chosen to reect the frequency contents of the disturbances d and di . The weighting matrix W n is used to model the frequency contents of the sensor noise, whereas W e may be used to reect the requirements on the shape of certain closed-loop transfer functions (e.g., the shape of the output sensitivity function). Similarly, W u may be used to reect some restrictions on the control or actuator signals, and the dashed precompensator W r is an
H INFINITY CONTROL
Fig. 4. Standard feedback conguration with disturbance weights and performance weights. W i , W d , and W n represent the frequency contents of the input disturbance, output disturbance, and the sensor noise. W e represents the disturbance rejection requirement, and W u puts the limit on the control effort. W r shapes the input signal.
optional element used to achieve deliberate command shaping or to represent a nonunity feedback system in equivalent unity feedback form. A typical control design may involve making the sensitivity function small over a suitable frequency range while keeping the control effort within a reasonable limit. This may be mathematically formulated as minimizing
subject to some restrictions on the control energy or control bandwidth:
Or more frequently, one may introduce a parameter and a mixed criterion
Note that can be absorbed into W u , so there is no loss of generality in assuming = 1. Finding a controller so that the H norm of a certain closed-loop transfer function, such as the preceding one, is minimized is the H control problem. Similar H 2 norm minimization problems can be formulated if the disturbance is modeled as white noise and the performance is measured in terms of output power.
H INFINITY CONTROL
Robust Stabilization
Another way that a weighted H norm minimization problem can arise naturally is when we consider robust stability and robust performance of a closed-loop system with model uncertainties. For example, consider a unity feedback system with a family of additively perturbed uncertain dynamical systems:
and assume that K stabilizes the nominal plant P. Then by the small gain theorem, the uncertain system is stable for all admissible with < 1/ if and only if
Therefore, a related synthesis problem is to nd a controller K so that this inequality holds. be a normalized coprime factorization, that is, 1N As another example, let P = M
Now consider a family of coprime factor perturbed uncertain systems:
A controller K stabilizing the nominal system P will robustly stabilize the family Fp if and only if
Similarly, many other robust stability problems can be formulated. It should be noted that the H 2 norm cannot be used in the robust stability test because it does not satisfy a key multiplicative property (i.e., G1 G2 2 G1 2 G2 2 in general).
Selection of Weighting Functions

As we mentioned previously, a very important step in the H control design process is to choose the appropriate weights. The appropriate choice of weights for a particular problem is not trivial. On many occasions, the weights are chosen purely as a design parameter without any physical bases, so these weights may be treated as tuning parameters that are chosen by the designer to achieve the best compromise between the conicting objectives. Hence, the selection of weighting functions for a specic design problem often involves ad hoc xing, iterations, and ne-tuning. It should be guided by the expected system inputs and the relative importance of the outputs. It is very hard to give a general formula for the weighting functions that will work in every case. Nevertheless, we shall try to give some guidelines in this section by looking at a typical single input single output (SISO) problem.
10
H INFINITY CONTROL
Fig. 5. The desired shapes of S and KS and their upper bounds.
Consider the SISO feedback system shown in Fig. 4. Then the tracking error is
where S = (I + PK ) 1 is the output sensitivity function and T = I S = PK (I + PK ) 1 is the output complementary sensitivity function. Note that tracking error is closely related to the low-frequency gain of S. In particular, we must keep |S| small over the range of frequencies, typically low frequencies where r and d are signicant. For example, if we need the steady state error with respect to a step input to be no greater than , then we need |S(0)| . Hence, the steady state tracking requirement can be fullled by constraining the low-frequency gain of S. From classical control theory, we know that the dynamical quality of the system time response can be quantied by rise time, settling time, and percent overshoot. Furthermore, the speed of the system response is inversely proportional to the closed-loop bandwidth and the overshoot of the system response increases with the resonant peak sensitivity dened as M s := S . Let b = min {:|S(j)| 1}. Then we can regard b as the closed-loop bandwidth because, beyond this frequency, the closed-loop system will not be able to track the reference and the disturbance will actually be amplied. Now suppose that we are given the time domain performance specications. Then we can determine the corresponding requirements in frequency domain in terms of the low-frequency gain, the bandwidth b , and the peak sensitivity M s . Hence, a good control design should result in a sensitivity function S satisfying all these requirements as shown in Fig. 5. These requirements can be approximately represented as
If a steeper transition between low frequency and high frequency is desired, the weight W e can be modied as follows:
for some integer k 1.
H INFINITY CONTROL
11
The selection of control weighting function W u follows similarly from the preceding discussion by considering the control signal equation
The magnitude of |KS| in the low-frequency range is essentially limited by the allowable cost of control effort and saturation limit of the actuators; hence, in general the maximum gain M u of KS can be fairly large, whereas the high-frequency gain is essentially limited by the controller bandwidth bc and the (sensor) noise frequencies. Ideally, we would like to roll off as fast as possible beyond the desired control bandwidth so that the high-frequency noises are attenuated as much as possible. Hence, a candidate weight W u would be
However, the standard H control design techniques cannot be applied directly to a problem with an improper control weighting function. Hence, we shall introduce a faraway pole to make W u proper:
for a small
> 0 as shown in Fig. 5. Similarly, if a faster rolloff is desired, we may choose
for some integer k 1. The weights for MIMO problems can be initially chosen as diagonal matrices with each diagonal term chosen in the preceding form.
General Problem Formulation and Solutions

All the disturbance rejection problems and robust stabilization problems discussed in the previous sections can be put in a unied framework of linear fractional transformation as shown in Fig. 1(a). For example,
12 and
H INFINITY CONTROL
For the following discussion, let us assume that state-space models of G and K are available and that their realizations are assumed to be stabilizable and detectable. We say that a controller is admissible if it internally stabilizes the system. Optimal H 2 Control: Find an admissible controller K (s) such that T zw Optimal H Control: Find an admissible controller K (s) such that T zw is minimized. is minimized.
It should be noted that the optimal H controllers as dened here are generally not unique for MIMO systems. Furthermore, nding an optimal H controller is often both numerically and theoretically complicated. This is certainly in contrast to the standard LQG or H 2 theory, in which the optimal controller is unique and can be obtained by solving two Riccati equations without iterations. Knowing the achievable optimal (minimum) H norm may be useful theoretically because it sets a limit on what we can achieve. In practice, however, it is often not necessary and sometimes even undesirable to design an optimal controller, and it is usually much cheaper to obtain controllers that are very close in the norm sense to the optimal ones, which will be called suboptimal controllers. A suboptimal controller may also have other nice properties (e.g., lower bandwidth) over the optimal ones. Suboptimal H Control: Given > 0, nd an admissible controller K (s), if there are any, such that T zw
< .
For these reasons mentioned, we shall focus our attention on the suboptimal H control. We shall assume that the realization of the transfer matrix G takes the following form:
which is compatible with the dimensions of z(t) Rp 1 , y(t) Rp 2 , w(t) Rm1 , and u(t) Rm2 and the state x(t) Rn . We make the following assumptions:
H INFINITY CONTROL
13
Assumption 1 is necessary for the existence of stabilizing controllers. Assumption 2 means that the penalty on z = C1 x + D12 u includes a nonsingular penalty on the control u, that the exogenous signal w includes both plant disturbance and sensor noise, and that the sensor noise weighting is nonsingular. Relaxation of assumption 2 leads to singular control problems. Assumptions 3 and 4 are made for a technical reason: together with assumption 1 they guarantee that the two algebraic Riccati equations in the corresponding LQG or H 2 problem have the desired solutions. Dropping assumptions 3 and 4 would make the solution very complicated. Dene
The following H 2 and H control results can be found in Refs. 2 and 3. Theorem 1. There exists a unique optimal controller
that minimizes T zw
where
X 2 0 and Y 2 0 are the solutions to
such that both Ax B2 R 1 1 BT 2 X 2 and Ay Y 2 CT 2 R 1 2 C2 are stable. Moreover,
Theorem 2. Suppose G satises assumptions 14. Then there exists an admissible controller K such that F (G, K ) < (i.e., T zw < ) if and only if (1) there exists an X 0 such that
and Ax + (B1 BT 1 / 2 B2 R 1 1 BT 2 )X is stable; (2) there exists a Y 0 such that
and Ay + Y (CT 1 C1 / 2 CT 2 R 1 2 C2 ) is stable; (3) (X Y ) < 2 where () denotes the spectral radius.
14
H INFINITY CONTROL Furthermore, if these conditions are satised, all internally stabilizing controllers K (s) satisfying F (G, < can be parameterized as
K )
for any Q R H such that Q
< where
The controller obtained by setting Q = 0
is called the central controller. Comparing the H central controller to the H 2 optimal controller, we can see that the H central controller will approach the optimal H 2 controller as . Example. Consider again the two mass/spring/damper system shown in Fig. 2. Assume that F1 is the control force, F 2 is the disturbance force, and the measurements of x1 and x2 are corrupted by measurement noise:
Our objective is to design a control law so that the effect of the disturbance force F 2 on the positions of the two masses x1 and x2 are reduced in a frequency range 0 2. The problem can be set up as shown in Fig. 6, where
H INFINITY CONTROL
15
Fig. 6. Rejecting the disturbance force F 2 of the two mass/spring/damper system by a feedback control of F 1 .
is the performance weight and W u is the control weight. In order to limit the control force, we shall choose
Now let u = F 1 ,
then the problem can be formulated in a LFT form with
where P1 and P2 denote the transfer matrices from F 1 and F 2 to
respectively. Let
that is, we want to reject only the effect of the disturbance force F 2 on the position x1 . Then the optimal H 2 performance is F (G, K 2 ) 2 = 2.6584, and the H performance with the optimal H 2 controller is F (G, K 2 ) = 2.6079, whereas the optimal H performance with an H controller is F (G, K ) = 1.6101. This means
16
H INFINITY CONTROL
Fig. 7. The largest singular value Bode plot of the closed-loop two mass/spring/damper system with an H 2 controller and an H controller.
that the effect of the disturbance force F 2 in the desired frequency range 0 2 will be effectively reduced with the H controller K by 5/1.6101 = 3.1054 times at x1 . On the other hand, let
that is, we want to reject only the effect of the disturbance force F 2 on the position x2 . Then the optimal H 2 performance is F (G, K 2 ) 2 = 0.1659, and the H performance with the optimal H 2 controller is F (G, K 2 ) = 0.5202, whereas the optimal H performance with an H controller is F (G, K ) = 0.5189. This means that the effect of the disturbance force F 2 in the desired frequency range 0 2 will be effectively reduced with the H controller K by 5/0.5189 = 9.6358 times at x2 . Finally, set
that is, we want to reject the effect of the disturbance force F 2 on both x1 and x2 . Then the optimal H 2 performance is F (G, K 2 ) 2 = 4.087, and the H performance with the optimal H 2 controller is F (G, K 2 ) = 6.0921, whereas the optimal H performance with an H controller is F (G, K ) = 4.3611. This means that the effect of the disturbance force F 2 in the desired frequency range 0 2 will be effectively reduced only with the H controller K by 5/4.3611 = 1.1465 times at both x1 and x2 . This result shows clearly that it is very hard to reject the disturbance effect on both positions at the same time. The largest singular value Bode plots of the closed-loop system are shown in Fig. 7. We note that the H controller typically gives a relatively at frequency response because it tries to minimize the peak of the frequency response. On the other hand, the H 2 controller typically produces a frequency response that rolls off fast in the high-frequency range but with a large peak in the low-frequency range.
H INFINITY CONTROL
17
H Filtering
In this section we shall illustrate how an H ltering problem can be converted to a special H control problem. Suppose that a dynamic system is described by the following equations:
The ltering problem is to nd an estimate z of z in some sense using the measurement of y. The restriction on the ltering problem is that the lter has to be causal so that it can be realized (i.e., z must be generated by a causal system acting on the measurements). We will further restrict our lter to be unbiased, that is, given T > 0, the estimate z (t) = 0 t [0, T ] if y(t) = 0, t [0, T ]. Now we can state our H ltering problem. H Filtering: Given a > 0, nd a causal lter F (s) R H if it exists such that
with z = F (s)y. This H ltering problem can also be formulated in an LFT framework because
Hence, the ltering problem can be regarded as a special H problem. However, comparing this ltering problem to the control problems, we can see that there is no internal stability requirement in the ltering problem. Hence, the solution to this ltering problem can be obtained from the H solution in the last section by setting B2 = 0 and dropping the internal stability requirement. Thus, a rational causal lter F (s) is given by
Understanding H Control
Most existing derivations and proofs of the H control results given in Theorem 2 are mathematically quite complex. Some algebraic derivations are simple but they provide no insight to the theory for control engineers. In this section, we shall present an intuitive but nonrigorous derivation of the H results by using only some basic system theoretic concept such as state feedback and state estimation. In fact, we shall construct intuitively the output feedback H central controller by combining an H state feedback and an observer.
18
H INFINITY CONTROL
A key fact we shall use is the so-called bounded real lemma, which states that for a system z = G(s)w with state space realization G(s) = C(sI A) 1 B H , G < , which is essentially equivalent to
if and only if there is an X = X 0 such that
and A + BB X / 2 is stable. Dually, there is a Y = Y 0 such that
and A + YC C/ 2 is stable. Note that the system has the following state space realization:
To keep the presentation simple, we shall make some additional assumptions: D 12 C1 = 0, B1 D 21 = 0, D 12 D12 = I, and D21 D 21 = I. We shall rst consider state feedback u = Fx. Then the closed-loop system becomes
By the bounded real lemma, T zw
< implies that there exists an X = X 0 such that
which is equivalent, by completing the square with respect to F , to
Intuition suggests that we can take
which gives
This is exactly the X Riccati equation under the preceding simplied conditions. Hence, we can take F = F and X = X .
H INFINITY CONTROL Next, suppose that there is an output feedback stabilizing controller such that T zw 0 because the closed-loop system is stable. Consequently, we have
19
< . Then x() =
Substituting x = Ax + B1 w + B2 u and z = C1 x + D12 u into the above integral and using the X equation, and nally completing the squares with respect to u and w, we get
where v = u + B 2 X x = u F x and r = w 2 B 1 X x. Substituting w into the system equations, we have the new system equations
Hence the original H control problem is equivalent to nding a controller so that T vr
< or
Obviously, this also suggests intuitively that the state feedback control can be u = F x and a worst state feedback disturbance would be w = 2 B 1 X x. Since full state is not available for feedback, we have to implement the control law using estimated state:
where x is the estimate of x. A standard observer can be constructed from the new system equations as
where L is the observer gain to be determined. Let e := x x Then
20
H INFINITY CONTROL
Since it is assumed that T vr exists a Y 0 such that
< , it follows from the dual version of the bounded real lemma that there
The above equation can be written as
Again, intuition suggests that we can take
which gives
It is easy to verify that
where Y is as given in Theorem 2. Since Y 0, we must have
Hence the controller is given by
which is exactly the H central controller given in Theorem 2 under the simplied conditions. We can see that the H central controller can be obtained by connecting a state feedback with a state estimate under the worst state feedback disturbance.
H Loop Shaping
Consider the family of uncertain systems Fp again. It is now clear that nding a controller K such that it robustly stabilizes the family Fp is a standard H norm minimization problem. Now suppose P has a
H INFINITY CONTROL stabilizable and detectable state-space realization given by
21
and let Y 0 be the solution to
,N ) given by Then the left coprime factorization (M
is a normalized left coprime factorization. Furthermore,
Dene
Then bP,K > 0 implies that K also stabilizes robustly the following family of uncertain systems: (1) (2) (3) =P+ P = (I + P P = (I +
a
have the same number of unstable poles and such that P and P a < bP,K . ) P such that P and P have the same number of unstable poles and m m < bP,K . 1 P such that P and P have the same number of unstable poles and f) f < bP,K .
These conclusions also hold when the roles of plant and controller are interchanged. The number bP,K can also be related to the classical gain and phase margins of a SISO system:
Hence, bP,K is a good measure of a systems robustness. Dene
22
H INFINITY CONTROL so that the system is stable. It follows from the
Obviously, bopt is the largest admissible size of perturbation standard H solution that
where Q is the solution to the following Lyapunov equation
Moreover, for any > 1/bopt (P), a controller achieving bP,K > 1/ is given by
where
This stabilization solution can be used to devise an H loop-sharing design method. The objective of this approach is to incorporate the simple performance/robustness trade-off obtained in the loop shaping with the guaranteed stability properties of H design methods. Recall that good performance controller design requires that
particularly in some low-frequency range where (PK ) denotes the smallest singular value. And good robustness requires that
particularly in some high-frequency range. The H loop-shaping design procedure is developed by McFarlane and Glover (4) and is stated in the next section.
Loop-Shaping Design Procedure.

(1) Loop Shaping: The singular values of the nominal plant are shaped, using a precompensator W 1 and/or a postcompensator W 2 , to give a desired open-loop shape. The nominal plant P and the shaping functions W 1 , W 2 are combined to form the shaped plant Ps , where Ps = W 2 PW 1 . We assume that W 1 and W 2 are such that Ps contains no hidden modes. (2) Robust Stabilization: (a) If bopt (P) 1 return to (1) and adjust W 1 and W 2 . (b) Select bopt (Ps ); then synthesize a stabilizing controller K , which satises bPs,K . (3) The nal feedback controller K is then constructed by combining the H controller K with the shaping functions W 1 and W 2 such that K = W 1 K W 2 .
H INFINITY CONTROL
23
A typical design works as follows: the designer inspects the open-loop singular values of the nominal plant, and shapes these by pre- and/or postcompensation until nominal performance (and possibly robust stability) specications are met. (Recall that the open-loop shape is related to closed-loop objectives.) A feedback controller K with associated stability margin (for the shaped plant) bopt (Ps ) is then synthesized. If bopt (Ps ) is small, then the specied loop shape is incompatible with robust stability requirements, and should be adjusted accordingly; then K is reevaluated. Note that in contrast to the classical loop-sharing approach, the loop shaping here is done without explicit regard for the nominal plant phase information. That is, closed-loop stability requirements are disregarded at this stage. Also in contrast with conventional H design, the robust stabilization is done without frequency weighting. In fact, the preceding robust stabilization objective can also be interpreted as the more standard H problem formulation of minimizing the H norm of the frequency weighted gain from disturbances on the plant input and output to the controller input and output as follows:
This shows that the H loop-shaping design is equivalent to a standard H design with the shaping functions as weighting functions.
Synthesis
As we discussed at the beginning of this article, F (G, K ) guarantees the robust stability of the uncertain system shown in Fig. 1(b) for any (s) R H with < 1/ . However, if a system is built from components, which are themselves uncertain, then, in general, the uncertainty in the system level is structured, and this robust stability guarantee may be overly conservative. Because the interconnection model G can always be (s) chosen so that (s) is block diagonal, and, by absorbing any weights, < 1. Thus we can assume that takes the form of
with i < 1 and j < 1. The robust stability analysis for systems with such structured uncertainty is not as simple but can be formally characterized by using the structured singular value, see Ref. 5. Dene Cnn as
24
H INFINITY CONTROL
Then for M Cnn , the structured singular value of M , (M ), is dened as
The itself is not easy to compute. But good bounds can be obtained efciently. Let
Then for any
and D D, D
D and
and the equality holds if 2S + F 3. This bound can be used frequency by frequency to determine the system robust stability and performance with structured uncertainties. For example, the system in Fig. 1(b) is well posed, internally stable for all stable (s) with (s0 ) , Re(s0 ) 0, and < 1/ if and only if
This result leads us to the following synthesis problem:
This synthesis problem is not yet fully solved in the general case. A reasonable approach is to obtain a solution to an upper bound:
by iteratively solving for K and D. This is the so-called D-K iteration. The stable and minimum phase scaling matrix D(s) is chosen such that D(s) (s) = (s)D(s). For a xed scaling transfer matrix D, minK DF (G, K )D 1 is a standard H optimization problem because
For a given stabilizing controller K , infD,D 1 H DF (G, K )D 1 and it can be solved pointwise in the frequency domain:
is a standard convex optimization problem,
Then a D(s) is found to approximate the magnitude frequency response D uniformly (usually by curve tting). D-K iterations proceed by performing this two-parameter minimization in sequential fashion: minimizing over
H INFINITY CONTROL
25
K with D(s) xed, minimizing pointwise over D with K xed, minimizing again over K , and then again over D, and so on. With either K or D xed, the global optimum in the other variable may be found using the and H solutions. Although the joint optimization of D and K is generally not convex and the global convergence is not guaranteed, many designs have shown that this approach works very well. In fact, this is probably the most effective design methodology available today for dealing with such complicated problems.
Additional Applications
There are many additional extensions and development in the H control theory. Here are some of them: H loop-shaping techniques using -gap metric, see Ref. 6. Robust control design in the gap metric, see Robust control and Ref. 6. Linear matrix inequality (LMI) approach to H control, see Convex optimization. Time-varying and nite horizon H control and game theoretical approach to H control, see Refs. 7 and 8. Operator theoretic approach to H control, see Ref. 9. Chain-scattering approach to H control, see Ref. 10. H control with pole placement, see Ref. 11. H controller reduction, see Refs. 12 and 13. Linear parameter varying H control, see Ref. 14. Sampled-Data H control, see Ref. 15. H control for innite dimensional systems, see Refs. 16 and 17. H control for nonlinear systems, see Ref. 18. Software and applications, see Ref. 19. A comprehensive treatment of H control theory can be found in Refs. 13 and 20.
BIBLIOGRAPHY
1. G. Zames, Feedback and optimal sensitivity: model reference transformations, multiplicative semi-norms, and approximate inverses, IEEE Trans. Autom. Control, 26: 301320, 1981. 2. K. Glover and J. Doyle, State-space formulae for all stabilizing controllers that satisfy an H norm bound and relations to risk sensitivity, Systems and Control Letters, 11: 167172, 1988. 3. J. C. Doyle et al., State-space solutions to standard H 2 and H control problems, IEEE Trans. Autom. Control, 34(8): 831847, 1989. 4. D.C. McFarlane and K. Glover, A loop shaping design procedure using H synthesis, IEEE Trans. Autom. Control, 37(6): 759769, 1992. 5. A. Packard and J. C. Doyle, The complex structured singular value, Automatica, 29: 71109, 1993. 6. G. Vinnicombe, Frequency domain uncertainty and the graph topology, IEEE Trans. Autom. Control, 38(9): 13711383, 1993. 7. M. Green and D. J. N. Limebeer, Linear Robust Control, Englewood Cliffs, NJ: Prentice-Hall, 1995. 8. T. Bas aer and P. Bernhard, H -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, Systems and Control: Foundations and Applications, Boston: Birkhauser, 1991. 9. B. A. Francis, A Course in H Control Theory, Lecture Notes in Control and Information Sciences, Vol. 88, London: Springer-Verlag, 1987. 1997. 10. H. Kimura, Chain-Scattering Approach to H -Control, Boston: Birkhauser,
26
H INFINITY CONTROL
11. M. Chilali and P. Gahinet, H design with pole placement constraints: An LMI approach, IEEE Trans. Autom. Control, 41(3): 358367, 1996. 12. P. J. Goddard and K. Glover, Performance preserving frequency weighted controller approximation: A coprime factorization approach, Proceedings of the 33nd Conference on Decision and Control, Orlando, FL, pp. 27202725, December 1994. 13. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 14. A. Packard, Gain scheduling via linear fractional transformations, Systems and Control Lett., 22: 7992, 1994. 15. T. Chen and B. A. Francis, Optimal Sampled-Data Control Systems, London: Springer-Verlag, 1995. 16. R. F. Curtain and H. J. Zwart, An Introduction to Innite-Dimensional Linear Systems Theory, New York: SpringerVerlag, 1995. 17. C. Foias, H. Ozbay, and A. Tannenbaum, Robust Control of Innite Dimensional Systems, Lecture Notes in Control and Information Sciences, Vol. 209, London: Springer-Verlag, 1996. 18. A. van der Schaft, L2 -Gain and Passivity Techniques in Nonlinear Control, Lecture Notes in Control and Information Sciences, Vol. 218, London: Springer-Verlag, 1996. 19. G. Balas et al., -Analysis and Synthesis Toolbox, Natick, MA: The MathWorks, Inc., 1995. 20. K. Zhou and J. C. Doyle, Essentials of Robust Control, Upper Saddle River, NJ: Prentice-Hall, 1998.
KEMIN ZHOU Louisiana State University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Input-Output Stability Standard Article Chaouki T. Abdallah1, Francesco Amato2, Marco Ariola2 1University of Tennessee, Knoxville, TN 2Universit degli Studi di Napoli Federico II, Naples, Italy Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1018 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (396K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Basic Concepts Stability
p
Interconnected Systems Related Stability Concepts And Applications Conclusions Appendix 1: Definitions Appendix 2: Sufficient Conditions For Exponential Stability About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
INPUT-OUTPUT STABILITY
The earliest mathematical studies of control systems focused solely on the input-output stability of systems as described in the works of Black (1), Bode (2), and Nyquist (3). In fact, most of the classical control work was limited to the input-output study of single-inputsingle-output linear and mostly time-invariant systems. Notions of input-output stability for nonlinear systems were later advanced by Sandberg (4), Zames, (5,6) Safonov (7), and others. The rst book dedicated completely to the subject was by Willems (8) in 1971, followed shortly by that of Desoer and Vidyasagar in 1975 (9). With the popularity of state-space methods, Lyapunovstability concepts became the preferred analysis and design tools for nonlinear systems until the 1980s when researchers became interested again in the input-output behavior of systems. The relationships between the input-output and Lyapunov-stability concepts were developed in Refs. 10 and 11 and culminated in the various versions of the Kalman-Yakubovich-Popov (KYP) lemma (12). The current studies in input-output systems are highly dynamic with the introduction of new concepts such as input-to-state stability (13,14,15), the interaction with geometric nonlinear control (16), applications to robust control (16,17,18), research in the adaptive control for linear (19) and nonlinear systems (17), the interest in various mechanical and electric systems (20), and the publication of various theoretical and applied books (18,20,21). It is now clear that the two points of view (state-space and input-output) are complementary and that many deep relationships between the various stability concepts are yet to be explored. In this article we concentrate our discussion on continuous-time systems and survey the classical as well as the more recent results in the input-output approach. The major results included in this article are taken mainly from Refs. 9,18,22,23,24, and the reader is referred to those books for most proofs. An excellent 1995 chapter on input-output stability from a distance-separation point of view appeared in Ref. 25. Related concepts for discrete-time systems were presented in Ref. 9 and recently revived by Byrnes and co-workers (26,27), but will not be discussed here. In addition, while we mention absolute stability as an important application area of input-output concepts, we refer the reader to the article on Absolute Stability in this encyclopedia for details. The article starts with a collection of basic denitions followed by the general results on the basic concepts of input-output stability and results for testing input-output stability and its relationship with Lyapunov stability. Next, we discuss the stability of interconnected systems and present the small-gain and passivity results. Related concepts such as absolute stability, dissipativity, and input-to-state and input-to-output stability are then reviewed, followed by our conclusions. Various technical denitions are presented in the appendices. We have attempted to include the main references on the subject of input-output stability, striving to be current and relevant rather than encyclopedic.
Basic Concepts
The general ideas of input-output stability involve the relative size of signals as they are processed by dynamical systems. We will thus begin by providing mathematical measures of the size of signals and of the effect that a particular system has on that size.
1
In order to introduce the mathematical notions of input-output stability, we need some preliminary denitions of signal spaces. (A detailed treatment of measure, signal spaces, and signal norms is beyond the scope of this article. The reader is referred, for example, to Ref. 28.) The Lm p set, with p [1, +), consists of all functions u: [0, +) m that are Lebesgue measurable [i.e., functions that are the limits (except for a set of measure zero) of a sequence of piecewise constant functions], such that
where v
denotes the q-norm of the vector v = (v1 v2 vm )T , dened as
Remark 1. For the nite-dimensional set m all q-norms are equivalent, in the sense that for all k, h [1, +] there exist positive constants c1 and c2 such that
for all x [0, ).
. This is why, when dening the Lp m set, we do not need to specify q, as u(t)
for any xed t
The Lm set consists of all the functions u: [0, +) m that are measurable and essentially bounded on [0, +) (where essentially bounded means that the function is bounded except on a set of measure zero). For a function belonging to Lm p , p [1, +), we introduce the norm
whereas for a function belonging to Lm we introduce the norm
Note that at the left-hand side of denitions (2) and (3), we should specify the dependence of the norm on q, but we omit it to avoid cumbersome notation; the choice of q will be clear from the context. For the Lm 2 set, the norm is usually dened by setting q = 2 in Eq. (2), obtaining
The Lm 2 set with the norm (4) is of particular interest, since in some contexts the norm (4) is proportional to the energy of the signal u, as is the case, for instance, for which u(t) is the voltage across a resistance R. Then the total energy delivered to the resistance would be found by integrating the instantaneous power u(t)2 /R. As usual, in the following we shall denote the L1 p set, p [1, +), by Lp .
Remark 2. In contrast with nite-dimensional spaces (see Remark 1), for the innite-dimensional set Lm (the set of all measurable functions u: [0, ) m ) the p-norms may be nonequivalent. For instance, u(t) = 1/(1+t) with 0 < < 1 belongs to L and to Lp with p < 1/, but does not belong to Lp for p 1/. u(t) = sin(t) belongs to L but does not belong to Lp for any p [1, +).
In this article, we are interested in studying the stability properties of a generic operator G that maps a signal space into another signal space ,
We shall dene the operator G in Eq. (5) as being Lp -stable if for any u() Lm p the output y() belongs to L p . To give a formal denition of Lp stability and to provide a broad denition of unstable systems, we rst need to dene precisely the domain and the image of the operator G, which in general are not limited to Lm p and Lr p , respectively (for example, we may be interested in considering unbounded inputs and/or outputs). For this reason, we need to introduce some extended Ln p spaces. First, let us dene the truncation uT (t) of a signal u(t) in the following way:
r
The extended set Lm pe , with p [1, +], consists of all the functions u: [0, +)
such that
Introducing this extended space, we can treat unbounded signals. For instance, u(t) = t does not belong to Lp for any p [1, +] but its truncation
belongs to Lp , with p [1, +], for every nite T . Therefore u(t) = t belongs to Lpe for every p [1, + ]. Finally we end the section with the denition of causality of an operator G. When we later specify an operator in terms of state-space differential equations, causality is an intrinsic property of the operator: the output y at time t depends on the initial conditions and on the values of the input u up to time t. On the other hand, when dealing with a generic operator as in Eq. (5), we need to enforce causality. An operator G: Lm pe Lr pe is said to be causal if the value of the output at time t depends only on the values of the input up to time t as dened next. Denition 1 (causality). An operator G: Lm pe Lr pe is said to be causal if (G(u))T = (G(uT ))T T 0, u Lm pe . We are now ready to present the rst input-output stability concept.
Lp Stability
Now we give the denition of Lp -stable systems that transform Lp input signals into Lp output signals. Denition 2. An operator G: Lm pe Lr pe is Lp -stable if
Finite-gain Lp -stable if there exist nite constants p and p such that
Finite-gain Lp -stable with zero bias if there exists a nite constant p such that
For nite-gain Lp -stable systems, the smallest scalar p for which there is a p such that relation (7) is satised (when such p exist) is called the gain of the system. Similarly, for nite-gain Lp -stable systems with zero bias, the smallest scalar p such that relation (8) is satised (when such p exist), is called the gain with zero bias of the system. Regarding nite-gain Lp stability for a causal operator G, we have the following result. Lemma 1. (Ref. 29) Let G: Lm pe Lr pe be a causal nite-gain Lp -stable operator with constants p and p . Then
Proof. Since G is causal (G(u))T = (G(uT ))T . Moreover if u Lm pe , then uT Lm p for all T 0; hence G(uT ) Lm p . Finally, for a generic function x, xT Lp x Lp . Therefore the following inequalities hold:
For a causal, nite-gain Lp -stable operator with zero bias, it can be shown in the same way that
Remark 3. Recalling the denition of the Lm set with the norm (3), L stability is in fact what is normally termed bounded-input, bounded-output (BIBO) or external stability. It is usually dened in reference to a system specied through a state-space representation, and guarantees that if the input to the system is essentially bounded, then its output is also essentially bounded. Example: This example illustrates an L -stable and a nite-gain L -stable operator. Let us consider the function g: dened by g(u) = uk with k (1, +). Correspondingly, dene the operator G: Le Le that assigns to every input u(t), t 0 the output g(u(t)). We will show that G is L stable. Indeed set u L = c < +; then we have
Therefore G is a BIBO operator. It is not a nite-gain L -stable operator; however, since we cannot nd xed scalars and such that ck c + holds for all c [0, +). By similar arguments as above, it is easy to check that G associated with the function g(u) = uk with k (0, 1) is a nite-gain L -stable operator.
Example: There are some special cases when the gain of an operator can be numerically or explicitly found; one of these cases is the L2 gain for linear systems. Let us consider a linear time-varying (LTV) system in the form (for the sake of simplicity, we assume no feedthrough term)
where A(t), B(t), and C(t) are piecewise continuous and bounded. = A(t)x is exponentially stable (see Appendix 1 for the denition of We assume that the unforced system exponential stability and Appendix 2 for a necessary and sufcient condition for exponential stability of LTV systems). Let G be the input-output operator mapping u to y; then the L2 gain of G (which is also called energy gain and induced-operator norm of G) is dened by
For a given > 0, we nd (see Ref. 30) that 2 if and only if there exists an differential equation
> 0 such that the Riccati
admits a positive denite solution P(t) for which a symmetric matrix-valued function P(t) is said to be positive denite if there exists a positive such that xT P(t)x x2 for all x n and t 0. Therefore by conducting a binary search over , the L2 gain of G can be computed up to the desired precision. The time-invariant case is even simpler, since we only have to deal with the algebraic version of Eq. (11). In (s) denote, as usual, the transfer matrix this case there is actually another way of computing the L2 norm; let G 1 (s); in fact G(s) = C(sI A) B. It is possible to show that the L2 gain (10) is related to the transfer matrix G it is given by
where max (A) := is the maximum singular value of the matrix A, and A denotes the conjugate (j) and is denoted by G . transpose of a matrix A. The norm (12) is known in literature as the H norm of G So far, we have considered operators whose domain is the whole space Lm p . The following example motivates a local version of Denition 1, concerning input signals that lie in a subset of Lm p . Example: Let us consider the function g: (1, 1) dened by g(u) = 1/(1u2 ). As in Example 1 dene the associated operator G: Le Le that assigns to every input u(t), t 0, the output g(u(t)). Since the function is dened only when the input signal is such that
the G is not an L -stable operator according to Denition 1. However, let |u| c < 1; then
and so
which implies that G is an L -stable operator in a new sense made clear in the next denition. The following denition is an extension of one originally presented in Ref. 31. Denition 3. An operator G: Lm pe Lr pe is small-signal Lp -stable if there exists a positive constant r such that Eq. (6) is satised for all u Lm p with supt u(t) < r. Similarly, G is a small-signal nite-gain Lp -stable [operator small-signal nite-gain Lp -stable operator with zero bias] if there exists a positive constant r such that inequality (7) [inequality (8)] is satised for all u Lm p with supt u(t) < r. Remark 4. In Denition 3, we do not need to specify a particular norm to evaluate supt u(t) , since we are dealing with a norm in m and norms are equivalent in nite-dimensional spaces (see Remark 1). From Eq. (3) it is clear that if u(t) is uniformly bounded then so is the signal norm u() L . This is not true in general as it is easy to construct examples in which the signal norm u() Lp , with p [1, +), is arbitrarily large even if u(t) is uniformly bounded.
Sufcient conditions for Lp stability. In the previous section we have provided the denition of Lp stability for a given input-output operator. An important question then arises: How do we check whether an input-output operator is Lp -stable or not? To answer this question we should not focus, as done so far, on generic input-output operators; in this section we assume that the operators under consideration are specied in terms of a state-space representation of a dynamical system in the form
where x
,u
,y
, and
with D := {x n : x < r}, Du := {u m : u < ru }, where r and ru are positive numbers. It is important to note that Eqs. (13a) dene an input-output operator for any given initial state x0 . In other words, system (13) denes an entire class of input-output operators, each obtained in relation to a particular x0 . We should always distinguish between the concept of a system and that of an operator. In many textbooks, when looking at the input-output behavior of a system, it is assumed that the initial condition is zero, and therefore the system and the corresponding operator are the same object. As opposed to our treatment, the discussion of Ref. 25 maintains the distinction between the input-output concepts and the state-space description. In the remainder of this article, we assume that a state-space description of the dynamical system is given unless otherwise specied. This will allow us to provide in this section a sufcient condition for Lp stability
and, at the same time, to establish the rst connection between Lyapunov (internal) stability and Lp stability. Assume that x = 0 is an equilibrium point for system (13a) with u = 0, that is,
We shall see that, if x = 0 is an exponentially stable equilibrium point (see Appendix A for the denition) and some other additional technical assumptions hold, the corresponding input-output operator is Lp -stable for any x0 as described in the following theorem. Theorem 1. (Corollary 6.1 of Ref. 23) Assume that (1) x = 0 is an exponentially stable equilibrium point for system (13a) under the input u = 0; (2) f is continuously differentiable and the Jacobian matrices f /x and f /u are bounded, uniformly in t (3) h is of Lipschitz form with respect to x and u, that is, there exist positive constants 1 , 2 such that
for all (t, x, u) [0, +) D Du . Then there exists a constant r0 > 0 such that for each x0 satisfying x0 < r0 the operator dened by system (13) with initial condition x0 , is a small-signal nite-gain Lp -stable operator for each p [1, +]. If all the assumptions hold globally, with D = n and Du = m , then for each x0 n the operator dened by system (13) with initial condition x0 is a nite-gain Lp -stable operator for each p [1, +]. Note that a linear system
always satises assumption ii and iii of Theorem 1 globally if A() and B() are continuously differentiable (actually this hypothesis can be relaxed to piecewise continuity) and uniformly bounded and C() and D() are uniformly bounded; moreover, the exponential stability of x = 0 of system (14a) with u = 0 is always global. Therefore we can state the following corollary of Theorem 1. Corollary 1. Consider the linear system
where A() and B() are continuously differentiable and uniformly bounded and C() and D() are uniformly bounded. Assume that the equilibrium point x = 0 under u = 0 of Eq. (15a) is exponentially stable; then for each x0 n the operator dened by system (15) with initial condition x0 is nite gain Lp -stable for each p [1, +]. Recall that the L2 gain of the operator associated with the linear system (14) for x0 = 0 can be computed according to the procedure detailed in Remark 4. Finally, a sufcient condition for exponential stability is given in Appendix 2.
Relations between Lyapunov stability and Lp stability. So far in this section we have shown the
following (Theorem 1):
This represents the rst connection between Lyapunov and Lp stabilities. The remainder of this section is devoted to nd the reverse connection between Lyapunov and Lp stabilities. It is, however, difcult to nd a general result in the spirit of Theorem 1. Following the guidelines of Ref. 29 we shall restrict ourselves to time-invariant systems and focus on attractivity rather than exponential stability. Roughly speaking, the next theorem will show the following result:
Theorem 2. Assume that (1) system (13) is time invariant, reachable, and uniformly observable and (2) the input-output operator dened by system (13) with initial condition x(0) = 0 is a small-signal L2 -stable operator. Then x = 0 is an attractive equilibrium point for system (13). Moreover, if system (13) is globally reachable and the input-output operator is L2 -stable, x = 0 is a globally attractive equilibrium point. For the denitions of reachability, uniform observability, and attractivity see Appendix 1. A LTI system in the form
is globally reachable if and only if it is reachable and is uniformly observable if and only if it is observable. Moreover small-signal L2 stability implies L2 stability, and attractivity implies exponential stability. Therefore we can derive the following corollary of Theorem 2. Corollary 2. Assume that (1) system (16) is reachable and observable and (2) The input-output operator dened by system (16) with initial condition x(0) = 0 is L2 -stable. Then x = 0 is an exponentially stable equilibrium point for system (16).
Interconnected Systems
One of the main applications of the formalism of input-output stability is the study of the stability of interconnected systems, without explicit knowledge of the internal dynamics of the composite subsystems. Let us consider the feedback interconnection of Fig. 1, where G1 : Lm pe Lr pe and G2 : Lr pe Lm pe . Input-output stability allows us to investigate how the signals propagate through this scheme. Before presenting the main
Fig. 1. Basic feedback system.
results, we need to introduce the concept of well-posedness of the feedback interconnection. Well-posedness guarantees that for each choice of u1 Lm pe and u2 Lr pe there exist unique solutions e1 ,y2 Lm pe and e2 ,y1 Lr pe that satisfy the loop equations
L pe and G2 : Lr pe Lm pe are causal nite-gain Lp -stable operators with constants p1 ,p1 and p2 ,p2 , respectively. Moreover, suppose that the feedback interconnection is well-posed. Then the feedback system of Fig. 1 is a nite-gain Lp -stable system if
r
Small-Gain Theorems. Theorem 3. Consider the feedback system of Fig. 1. Suppose that G1 : Lm pe
and
Proof. Consider inputs u1 Lm p and u2 Lr p . Since the closed-loop system is well-posed, there exist unique solutions e1 ,e2 ,y1 ,y2 . With respect to e1 and e2 we have
10
Since G1 and G2 are causal nite-gain Lp -stable operators, we nd that (see Lemma 1)
After some trivial manipulations, recalling that p1 p2 < 1 by assumption, Eqs. (18a) become
Since for a generic function x, xT
Lp
Lp ,
we have
Now, the right-hand sides of inequalities (20) are independent of T . Therefore it can be easily shown that e1 and e2 belong to Lm p and Lr p , respectively, and that
In a similar way it can be shown that y1 and y2 belong to Lr p and to Lm p , respectively, and that inequalities (17c) and (17d) hold. The work of Safonov (7) exploited the general input-output concepts in order to study the robustness of closed-loop systems. His results and variations were later exploited in H robust control analysis and design. The small-gain theorem is thus extremely useful in studying the robustness of a closed-loop system, when a nominal system is subject to a perturbation as shown in Fig. 2. The next example shows one of these applications.
11
Fig. 2. Feedback robustness loop.
Example: Let us consider the feedback scheme of Fig. 2 where G(s) and (s) are asymptotically stable transfer matrices. The transfer matrix G(s) represents the nominal system, whereas (s) is a model of the uncertainty. Let u1 and u2 belong to L2 . In Example 2 we have seen how to compute the L2 gains of G(s) and (s); let
The small-gain theorem tells us that if
then the closed-loop system is a nite-gain L2 -stable system. In other words, it gives us an estimate of how large the perturbation (s) can be, in terms of its H norm, preserving the closed-loop L2 stability. There are various versions of the small-gain theorem, a sample of which is the incremental small-gain theorem below, which needs a preliminary denition. Denition 4. An operator G: Lm pe Lr pe is said to be an incrementally nite-gain stable operator if (1) G(u) Lr p when u 0 and (2) there exists a constant such that
for all T > 0 and for all u, v Lm pe . Theorem 4. Consider the interconnected system of Fig. 1. Let both G1 and G2 be incrementally nite-gain stable operators with respective gains 1 and 2 . Then, the feedback interconnection is well-posed and incrementally nite-gain stable from u = [u1 u2 ] to y = [y1 y2 ] if
Passivity Theorems. One of the main related concepts to the input-output stability concepts discussed so far is the concept of passive systems. In a way, while Lp stability deals with the effect the system has on the size of signals, passivity results deal with the effect the system has on the energy of signals. We start with few denitions and follow with the main results for interconnected systems.
12
Denition 5. We say that a system G: Lm 2e Lr 2e is (1) Passive if there exists such that for all T and all u Lm 2e
(2) Strictly passive if there exists > 0 and
such that for all T and all u Lm 2e
Theorem 5. Consider the interconnected system of Fig. 1. Assume that the systems G1 and G2 satisfy
The closed-loop system is L2 nite-gain stable if
Note that this theorem does not require both systems G1 and G2 to be passive, as long as one of the two systems is passive enough. If on the other hand, one of the two systems is passive and the other is strictly passive, the previous theorem simplies to the following. Theorem 6. Consider the interconnected system of Fig. 1, and let u2 = 0. Then, the closed-loop system is nite-gain L2 stable if one of the following conditions holds: G1 is passive and G2 is strictly passive G1 is strictly passive and G2 is passive.
In the special case of afne in the control systems, the passivity of one system is equivalent to the L2 -stability of a related system. This is detailed in the following theorem: Theorem 7. (Proposition 3.2.12 of Ref. 18) Let
and
where u = 1 (v z) and y = 1 (v + z). Then, system (22) is passive system (23) has L2 gain 1. 2 2
13
Fig. 3. The Lure problem.
Related Stability Concepts And Applications

In this section, we review various concepts that are related to input-output stability and discuss some of their applications. Dissipativity. In 1972, Willems (10,11) introduced the notion of dissipativity in an attempt to further unify input-output and Lyapunov-stability concepts. The notion of dissipativity is a generalization of passivity and captures the concept that a system will dissipate energy if the sum of the energy it stores and the energy it dissipates to the environment is less than the total energy that the environment supplies to it. This is a manifestation of the second law of thermodynamics and is the case of most physical systems that transform some form of energy to another, but also lose some in the process. In the following, + = [0, ). Denition 6. The system (13) is dissipative with respect to the supply rate w(u, y): exists a storage function V: n + such that
m
r if and only if there
for all u, all T 0, and all x(0)
Note that passivity can actually be dened as a special case of dissipativity by letting w(u, y) = uT y (therefore, the system is square and m = r). We can also dene other types of passivity as follows: the system is an input-strictly-passive (ISP) system if it is dissipative with supply rate w(u, y) = uT y u u 2 , u > 0, and it is an output-strictly-passive (OSP) system if it is dissipative with supply rate w(u, y) = uT y y y 2 , y > 0. The system is a state-strictly-passive (SSP) system if w(u, y) = uT y x (x), x > 0 and (x) is a positive-denite function of x. Note that an OSP system is necessarily L2 stable (18,23). In addition, one can guarantee the L2 stability of a system by making sure it is dissipative with the particular supply rate w(u, y) = 1 2 u 2 y 2 2 for some positive , which then becomes an upper bound on the L2 gain of the system (18,23). The Linear Case and the KYP Lemma. One of the more important applications of the input-output approach is in the solution of the so-called Lure problem shown in Fig. 3. The details of this approach are detailed in another chapter of this encyclopedia and only a few comments are included here for completeness. The basic question asked by Lure is to nd conditions in the single-inputsingle-output case on the linear system G(s) such that when the nonlinear block (t, y) is static (i.e., a non-time-varying function of y only), the closed-loop system is stable (32). Popov provided graphical, frequency-domain criterion for the absolute stability problem when the nonlinear block (y) is time invariant (33,34). Yakubovich (35) and Kalman (36) introduced different versions of the so-called positive-real or Kalman-Yakubovich-Popov (KYP) lemma to relate Popovs criterion to the existence of a special Lyapunov function. This then provides another connection between input-output stability concepts and Lyapunov concepts. Anderson then (12,22) extended the KYP lemma to the multi-inputmulti-output case. The KYP lemma has found various applications in adaptive control (19) and
14
has recently been generalized to the case in which the linear block G(s) is replaced by a nonlinear but afne nonlinear system = f (x) + g(x)u, y = h(x) (16,18). In the linear case, passivity concepts may be related to the concept of positive-realness, already introduced in the study of electrical networks (22). In fact, consider a stable, square, LTI system with minimal state-space realization
where u, y
,x
, and let the transfer function be
Since the state-space realization is minimal, then (A, B) is controllable and (C, A) is observable. Recall that (A, B) is controllable if and only if rank ( ) = n and (C, A) is observable if and only if rank ( ) = n where = [B AB An 1 B] and T = [CT AT CT (An 1 )T CT ]. Denition 7. Let H(s) be a proper m m rational transfer matrix. Then H(s) is positive real (PR) if (1) No element of H(s) has a pole in Re[s] > 0 (2) Any pole of an element of H(s) on the j axis must be simple and the associated residue matrix is positive semidenite Hermitian, and (3) For all real for which j is not a pole of an element of H(s), Z(j) + ZT (j) is positive semidenite. (4) H(s) is said to be strictly positive real (SPR) if H(s ) is PR for some > 0. For variations on this denition, the reader should consult Ref. 37, where various PR concepts are discussed. Lemma 2. Let H(s) be an m m transfer matrix (25) where (1) A is stable (2) (A, B) is controllable and (C, A) is observable Then, the transfer function H(s) is SPR if and only if there exist a positive denite symmetric matrix P, matrices W and L, and a real > 0 such that
INPUT-OUTPUT STABILITY Note that if D = 0, that is, H (s) is strictly proper, then Eqs. (26a) simplify to the more familiar
15
Lemma 3. H(s) is PR if and only if it is dissipative with storage function V(x) = xT Px. Hill and Moylan (38) and others expanded the dissipativity notions in order to explain the KYP lemmas. The KYP lemmas have applications to adaptive control for linear systems (19) and a generalization to nonlinear systems (17). Connections between passivity and stability are provided in the next lemma. Lemma 4. (Lemma 10.6 in Ref. 23) Given the autonomous system
then the following holds true. (1) If Eq. (27) is passive with a positive-denite storage function V(x), then the origin of Eq. (27) with zero input is stable. (2) If the system is OSP, then it is a nite-gain L2 -stable system. (3) If the system is OSP with a positive-denite storage function V(x) and zero-state observable (see Appendix 1), then the origin of Eq. (27) with zero input is asymptotically stable. (4) If the system is SSP with a positive-denite storage function V(x), then the origin of Eq. (27) with zero input is asymptotically stable.
Passication via Feedback. In recent years, the input-output approach has gained new footing as a design tool for nonlinear control systems. One of the main applications of such an approach has been to use feedback in order to render a closed-loop system passive or strictly passive (or the passication of an open-loop system). The main motivation for such designs is of course that a passive system will tolerate large-magnitude uncertainties as long as the uncertainties are passive (see Theorem 5). References 16 and 17 contain a large number of results on the passication of nonlinear systems. Roughly speaking, all designs require that the open-loop system be of minimum phase and of a relative degree one in order for it to be made passive using static output feedback. Such concepts have been generalized to a large class of nonlinear systems. As mentioned previously, and following the early concepts (9), there has been much recent work on the discrete-time versions of the input-output stability concepts including the passication designs in Refs. 26 and 27. Input-to-State and Input-to-Output Stability. In a series of papers (13,14,15), Sontag and co-workers have advanced the notion of input-to-state stability to study the behavior of state-space systems when the input is bounded. Roughly, the input-to-state stability concepts guarantee that the state x(t) is bounded for any bounded input u(t), which may be an external disturbance or a tracking signal. This idea is in some ways a more restrictive version of the input-output concepts unless y = x and is more tightly coupled to the Lyapunov-stability concepts. In what follows, we deal with system (13), or with its autonomous version:
16
Denition 8. The system (13a) is said to be locally input-to-state stable (ISS) if there exists a class L function , a class function (see Appendix 1 for the denitions of such functions), and positive constants k1 and k2 such that for any initial state x(t0 ) with x(t0 ) < k1 and any input u(t) with supt t0 u(t) < k2 , the solution x(t) exists and
The system is said to be ISS stable if the preceding requirement holds globally (i.e., if D = any bounded input u(t) and any initial condition x(t0 ).
and Du =
) for
Theorem 5.4. (Lemmas 5.4 and 5.5 (Ref. 23 Let f (t, x, u) be continuously differentiable and of global Lipschitz form in (x, u) uniformly in t. Then, if the system (13a) has a globally exponentially stable equilibrium point at x = 0, it is ISS. In the case for which the system is autonomous, f (x, u) in Eq. (28a) is continuously differentiable, and the origin is an asymptotically stable equilibrium point of Eq. (28a), then Eq. (28a) is ISS. Denition 9. The system (13) is locally input-to-output stable if there exists a class L function , a class function , and positive constants k1 and k2 such that for any initial condition x(t0 ) such that x(t0 ) < k1 and any input u(t) such that supt t0 u(t) < k2 , and for any t t0 0, the following holds true. (1) The solution x(t) exists. (2)
The system (13) is said to be input-to-output stable (IOS) if D = initial state x(t0 ) and any bounded input u(t).
, Du =
, and Eq. (30) holds for any
Note that while this is similar to the Lp -stability concepts presented previously, it is actually more general as the function need not be linear and the function need not be a constant. Theorem 9. (Theorem 6.3 of Ref. 23) Let f (t, x, u) be piecewise continuous in t and of local Lipschitz form in (x, u), and let h be piecewise continuous in t and continuous in (x, u). Assume that the system (13) is ISS, and that there exists class functions 1 and 2 such that
Then the system (13) is locally IOS. If all assumptions hold globally and Eq. (13a) is ISS, then it is IOS.
Conclusions
In this article we have attempted to summarize various concepts of input-output stability for nonlinear dynamical systems, focusing on the continuous-time case. We have presented the basic input-output concepts but also
17
some extensions and their applications to stability robustness analysis and design and to the adaptive control of linear and nonlinear systems, as mentioned previously. It is now clear that the connections between Lyapunov stability and input-output stability are strong and may be exploited for further design. On the other hand, it is clear that the input-output approach remains a versatile tool. This approach allows us to be able to determine the stability of the closed-loop system although we have have little knowledge of the internal dynamics of the open-loop system and its uncertainties. This is clearly an advantage when dealing with uncertain systems as the dynamics of the systems and its uncertainties may be unknown. One of the limitations of the input-output approach, however, is that it remains limited to the study of stability while other objectives such as the optimization of some performance indices remain beyond the reach of these techniques. We end this article by mentioning that prior to its introduction to feedback systems and control, the input-output approach was part of operator theory and functional analysis. Finally, the input-output approach has been applied to various areas such as communications (39,40) and to the study of neural network stability (41).
Appendix 1: Denitions
Denition 1.1 (Lipschitz functions). A function f (x) is said to be of local Lipschitz form on a domain D if for each point x0 D there exist a neighborhood D0 of the point x0 and a constant L0 such that
n
If equation (32) holds for all x D with the same constant L, then f is said to be of Lipschitz form on D. If f is of Lipschitz form on n than it is said to be of global Lipschitz form. Denition 1.1 can be extended to the case of f (t, x) provided that the Lipschitz condition holds uniformly on t for a given time interval. Denition 1.2. (Function of class . A continuous function : [0, +) [0, +) is said to be of class Denition 1.3. (Function of class ). A function : [0, +) [0, +) is said to be of class if it is strictly increasing and (0) = 0. and (s) as s .
if it is of class
Denition 1.4. (Function of class L). A function : [0, +) [0, +) [0, +) is said to be of class L if for each xed t the mapping (s, t) is of class and for each xed s the mapping (s, t) is decreasing with respect to t and (s, t) 0 as t . Example: The function (s): [0, +) s/(s+1) is of class since (s) = 1/(s+1)2 > 0, but it is not of class since lims (s) = 1 < +. The function (s): [0, +) s is of class since (s) = 1 > 0 and lims (s) = +. The function (s, t): [0, +) [0, +) se t is of class L. Indeed it is strictly increasing in s, since
18
INPUT-OUTPUT STABILITY strictly decreasing in t, since
and (s, t) 0 as t . Denition 1.5 (State transition matrix). Given the LTV system
with x n and A(t) piecewise continuous, the state transition matrix is dened as the unique solution of the matrix differential equation
( c, c):
nn
, (t, t0 )
(t, t0 ),
Consider the zero-input system
where f (): D
is of local Lipschitz form on D.
Denition 1.6 (Attractivity). Consider the zero-input system (1.2) and denote by s(t, x0 ) the solution starting from x0 at time t = 0. Assume that x = 0 is an equilibrium point of system (1.2); then x = 0 is attractive if there exists a domain Da D, 0 Da such that
The equilibrium point x = 0 is globally attractive if Da =
Denition 1.7 (Stability and Asymptotic Stability). Consider the zero-input system (1.2) and denote by s(t, x0 ) the solution starting from x0 at time t = 0. Assume that x = 0 is an equilibrium point of system (1.2); then x = 0 is stable if, for each > 0, there exists a = ( ) such that
unstable if it is not stable. asymptotically stable if it is stable and can be chosen such that x = 0 is attractive on the domain D = {x n , x < }. globally asymptotically stable if it is stable and globally attractive. Now consider the zero-input system
where f (,): [0, +) D
, D = {x
, x < r}, and t0 0.
19
Denition 1.8 (Exponential stability). Consider the zero-input system (1.3) and assume that x = 0 is an equilibrium point. Then x = 0 is exponentially stable if there exist positive numbers K, , c such that
The equilibrium x = 0 is globally exponentially stable if the above condition is veried for any initial state. Denition 1.9 (Zero-State Observability). The system (27) is said to be zero-state observable from the output y if for all initial conditions, y(t) 0 x(t) 0. The system is zero-state detectable if for all initial conditions y(t) 0 limt x(t) = 0. Denition 1.10 (Reachability). Consider the system
and denote by s(t, x0 , u) the solution starting from x0 at time t = 0 under the input u. Then system (1.5) is said to be reachable if there exists a class function and a set D := {x n : x < r}, such that for all x D there exists a time t and an input u such that u L ( x ) and s(t, 0, u) = x. The system is said to be globally reachable if all the assumptions hold for all x n . Denition 1.11 (Uniform observability). Consider the system
and denote by s(t, x0 , u) the solution starting from x0 at time t = 0 under the input u. Then system (1.6) is said to be uniformly observable if there exists a class function such that for all x,
Appendix 2: Sufcient Conditions For Exponential Stability

Theorem 2.1 (Sufcient condition for exponential stability). Let x = 0 be an equilibrium point of system (1.3) and assume there exists a continuously differentiable Lyapunov function V (,): (t, x) [0, +) D satisfying
for some positive constants k1 , k2 , k3 , and . Then the equilibrium x = 0 is exponentially stable. Moreover, if all the assumptions hold globally, x = 0 is globally exponentially stable.
20
Theorem 2.2 (Exponential stability of linear systems). The equilibrium point x = 0 of the LTV system
is exponentially stable if and only if there exist positive constants and k such that
where
(t, t0 ) is the state transition matrix of system (2.2).
Note that the concept of exponential stability is equivalent, for linear systems, to that of uniform asymptotic stability (see Ref. 23 for the denition). This equivalence is no longer true for nonlinear systems, where the concept of exponential stability is stronger than that of uniform asymptotic stability. Finally, note that for LTI systems a necessary and sufcient condition for exponential stability is the Hurwitz character of the A matrix, that is, all its eigenvalues should have negative real part.
BIBLIOGRAPHY
1. H. S. Black Wave Translation System, US Patent No. 2,102,671, December 21, 1937. 2. H. W. Bode FeedbackThe History of an Idea, in R. Bellman and R. Kalaba (eds.), Selected Papers on Mathematical Trends in Control Theory, pp. 106123. New York: Dover, 1964. 3. H. Nyquist Regeneration theory. Bell System Tech. J., 11: 126147, 1932. 4. I. W. Sandberg On the l2 -boundedness of solutions of nonlinear functional equations, Bell Sys. Tech. J., 43: 15811599, 1964. 5. G. Zames On the input-output stability of nonlinear time-varying feedback systems: Part i, IEEE Trans. Autom. Control, 11: 228238, 1966. 6. G. Zames On the input-output stability of nonlinear time-varying feedback systems: Part ii, IEEE Trans. Autom. Control, 11: 465477, 1966. 7. M. Safonov Stability and Robustness of Multivariable Feedback Systems, Cambridge: MIT Press, 1980. 8. J. C. Willems The Analysis of Feedback Systems, Cambridge: MIT Press, 1971. 9. C. A. Desoer M. Vidyasagar Feedback Systems: Input Output Properties, New York: Academic Press, 1975. 10. J. C. Willems Dissipative dynamical systems: Part i: General theory, Arch. Ration. Mech. Anal., 45: 325351, 1972. 11. J. C. Willems Dissipative dynamical systems part ii: Linear systems with quadratic supply rate, Arch. Ration. Mech. Anal., 45: 352393, 1972. 12. B. D. O. Anderson The small-gain theorem, the passivity theorem and their equivalence, J. Franklin Institute, 293(2): 105115, 1972. 13. E. Sontag On the input-to-state stability property, Eur. J. Control, 1: 2436, 1995. 14. E. Sontag Y. Wang Notions of input to output stability, Syst. Control Lett., 38: 235248, 1999. 15. E. Sontag Y. Wang Lyapunov characterizations of input to output stability, SIAM J. Control Optimization, 39, 226249, 2000. 16. C. I. Byrnes A. Isidori J. C. Willems Passivity, feedback equivalence, and the global stabilization of minimum phase nonlinear systems, IEEE Trans. Autom. Control, 36: 12281240, 1991. 17. M. Krst c Kanellakopoulos P. V. Kokotiv c Nonlinear and Adaptive Control Design, New York: Wiley-Interscience, 1995. 18. A. J. van der Schaft L2 -Gain and Passivity Techniques in Nonlinear Control, vol. 218, Lecture Notes in Control and Information Sciences, Heidelberg: Springer-Verlag, 1996. 19. K. S. Narendra A. M. Annaswamy Stable Adaptive Systems, Englewood Cliffs, NJ: Prentice-Hall, 1985. 20. R. Ortega et al. Passivity-Based Control of Euler-Lagrange Systems: Mechanical, Electrical and Electromechanical Applications, London: Springer-Verlag, 1998.
21
21. R. Lozano O. Egeland B. Maschke Dissipative Systems Analysis and Control: Theory and Applications, London: Springer, 2000. 22. B. D. O. Anderson S. Vongpanitlerd Network Analysis and Synthesis: A Modern Systems Theory Approach, Englewood Ciffs, NJ: Prentice Hall, 1973. 23. H. K. Khalil Nonlinear Systems, 2d ed., Upper-Saddle River, NJ: Prentice Hall, 1996. 24. S. Sastry Nonlinear Systems: Analysis, Stability, and Control, New York: Springer, 1999. 25. A. R. Teel T. T. Georgiou L. Praly E. Sontag Input-output stability, in W. S. Levine (ed.), The Control Handbook, pp. 895908, Boca Raton FL: CRC Press, 1995. 26. C. I. Byrnes W. Lin Losslessness, feedback equivalence, and the global stabilization of discrete-time nonlinear systems, IEEE Trans. Autom. Control, 39(1): 8398, 1994. 27. W. Lin C. I. Byrnes Passivity and absolute stabilization of a class of discrete-time nonlinear systems, Automatica, 31(2): 263267, 1995. 28. H. L. Royden Real Analysis, London: Macmillan (Collier-MacMillan Limited), 2d. ed., 1968. 29. M. Vidyasagar Nonlinear System Analysis, 2d ed., Upper-Saddle River, NJ: Prentice Hall, 1993. 30. G. Tadmor Input/output norms in general linear systems, Int. J. Control, 51(4): 911921, April 1990. 31. M. Vidyasagar A. Vannelli New relationships between input-output and Lyapunov stability, IEEE Trans. Autom. Control, 27(2): 481483, 1982. 32. A. I. Lure Some Nonlinear Problems in the Theory of Automatic Control, London: H.M. Stationery Ofce, 1957. 33. V. M. Popov Absolute stability of nonlinear systems of automatic control, Autom. Remote Control, 22: 857875, 1962. 34. V. M. Popov Hyperstability of Control Systems, New York: Springer, 1973. 35. V. A. Yakubovich Solution of certain matrix inequalities in the stability theory of nonlinear control systems, Engl. Translation Sov. Math. Dokl., 3: 620623, 1962. 36. R. E. Kalman Lyapunov functions for the problem of Lure in automatic control, Proc. Nat. Acad. Sci., 49: 201205, 1963. 37. J. T. Wen Time domain and frequency domain conditions for strict positive realness, IEEE Trans. Autom. Control, 33: 988992, 1988. 38. D. J. Hill P. Moylan Dissipative dynamical systems: Basic input-output and state properties, J. Franklin Inst., 5: 327357, 1980. 39. I. W. Sandberg On the properties of systems that distort signals, Bell Syst. Tech. J., 42: 20332047, 1963. 40. I. W. Sandberg Notes on PQ theorems, IEEE Trans. Circuits Syst. I: Fundam Theory Appl., 41: 303307, 1994. 41. A. Mayer-Base F. Ohl H. Scheich Asymptotic hyperstability for the Hopeld model of neural networks, in Proc. IEEE Joint Int. Conf. Neural Networks, pp. 24362441, Orlando, FL, 1994.
CHAOUKI T. ABDALLAH University of Tennessee FRANCESCO AMATO MARCO ARIOLA ` degli Studi di Napoli Federico II Universita
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Intelligent Control Standard Article Panos J. Antsaklis1 1University of Notre Dame, Notre Dame, IN Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1019 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (115K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Foundations of Intelligent Control Intelligent Learning Control Intelligent Control for High-Autonomy Systems Hybrid Systems Concluding Remarks About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
INTELLIGENT CONTROL AUTONOMOUS SYSTEMS, CONTROL, ROBOTS, VEHICLES INTELLIGENT SYSTEMS, ROBOTS, VEHICLES, MANUFACTURING
Intelligent control describes the discipline for which control methods are developed that attempt to emulate important characteristics of human intelligence. These characteristics include adaptation and learning, planning under large uncertainty, and coping with large amounts of data. The main difculty in specifying exactly what is meant by the term intelligent control stems from the fact that there is no agreed-upon denition of human intelligence and intelligent behavior, and the centuries-old debate of what constitutes intelligence is still continuing, nowadays among educators, psychologists, computer scientists, and engineers. Apparently the term intelligent control was coined in the 1970s by K. S. Fu. There are a number of areas related to the area of intelligent control. Intelligent control is interdisciplinary as it combines and extends theories and methods from areas such as control, computer science, and operations research. It uses theories from mathematics and seeks inspiration and ideas from biological systems. Intelligent control methodologies are being applied to robotics and automation, communications, manufacturing, and trafc control, to mention but a few application areas. Neural networks, fuzzy control, genetic algorithms, planning and expert systems, intelligent agents, and hybrid systems are all areas where related work is taking place. The areas of computer science and in particular articial intelligence provide knowledge representation ideas, architectures, methodologies, and tools such as semantic networks, frames, reasoning techniques, and computer languages. Concepts and algorithms developed in the areas of adaptive control and machine learning help intelligent controllers to adapt and learn. Advances in sensors, actuators, computation technology, and communication networks help provide the intelligent control hardware necessary for implementation and inuence the type of architecture used. For example, low-cost processing and communication devices together with low-cost sensors are making it possible to have distributed, networked, embedded control systems in many application areas. In the following, fundamental ideas of intelligent control are emphasized rather than particular methodologies; note that several related areas are described at length elsewhere in this encyclopedia. Fundamental ideas and characteristics of intelligent systems are introduced in the section on foundations of intelligent control, and a historical perspective is given in the section on intelligent learning control in which the role of machine learning is discussed. The quest for machines that exhibit higher autonomy has been the driving force in the development of control systems over the centuries, and this is discussed in the section
on intelligent control for high-autonomy systems. Hybrid systems that contain both continuous and digital components are also briey discussed, as they are central in intelligent control. FOUNDATIONS OF INTELLIGENT CONTROL The term intelligent control has come to mean, particularly to those outside the control area, some form of control using methodologies such as intelligent agents, genetic algorithms, or fuzzy and/or neural networks. Intelligent control, however, does not restrict itself only to those methodologies. The fact is that there are problems of control today that cannot be formulated and studied in the conventional differential or difference equation mathematical framework using conventional (or traditional) control methodologies; these methodologies were developed in the past decades to control dynamical systems. To address these problems in a systematic way, a number of methods have been developed in recent years that are collectively known as intelligent control methodologies. It is worth remembering at this point that intelligent control uses conventional control methods to solve lower level control problems and that conventional control is included in the area of intelligent control. In summary, intelligent control attempts to build upon and enhance the conventional control methodologies to solve new challenging control problems. Conventional and Intelligent Control The word control in intelligent control has a different, more general meaning than the word control in conventional control. First, the processes of interest are more general and may be described, for example, by discrete event system models, differential or difference equation models, or both. This has led to the development of theories for hybrid control systems, which study the control of continuousstate dynamic processes by discrete-state controllers. In addition to the more general processes considered in intelligent control, the control objectives can also be more general. For example, replace part A in satellite can be the general task for the controller of a space robot arm; this is then decomposed into a number of subtasks, several of which may include, for instance, follow a particular trajectory, which may be a problem that can be solved by conventional control methodologies. To attain such control goals for complex systems over a period of time, the controller has to cope with signicant uncertainty that xed feedback robust controllers or adaptive controllers cannot deal with. Since the goals are to be attained under large uncertainty, fault diagnosis, and control reconguration, adaptation and learning are important considerations in intelligent controllers. It is also clear that task planning is an important area in intelligent control design. So the control problem in intelligent control is an enhanced version of the problem in conventional control. It is much more ambitious and general. The area of intelligent control is interdisciplinary, and it attempts to combine and extend theories and methods from areas such as control, computer science, and operations research to attain demanding control goals in complex systems.
Intelligent Systems, Robots, Vehicles, Manufacturing
Note that the theories and methodologies from the areas of operations research and computer science cannot, in general, be used directly to solve control problems, as they were developed to address different needs. The techniques from computer science and operations research are primarily analysis tools developed for non-real-time systems, while in control, synthesis techniques to design realtime feedback control laws for dynamic systems are mainly of interest. Recent work on real-time systems in computer science has moved the research along lines that are of great interest to intelligent control. In view of this discussion, it should be clear that intelligent control research, which is mainly driven by applications, has a very important and challenging theoretical component. Signicant theoretical strides must be made to address the open questions. The problems are nontrivial, but the payoff is very high indeed. As it was mentioned above, the word control is intelligent control has a more general meaning than in conventional control; in fact it is closer to the way the term control is used in everyday language. Because intelligent control addresses more general control problems that also include the problems addressed by conventional control, it is rather difcult to come up with meaningful benchmark examples. Intelligent control can address control problems that cannot be formulated in the language of conventional control. To illustrate, in a rolling steel mill, for example, conventional controllers may include the speed (revolutions per minute) regulators of the steel rollers. In the intelligent control framework one may include, in addition, fault diagnosis, alarm systems, and perhaps the problem of deciding on the set points of the regulators, which are based on the sequence of orders processed, economic decisions, maintenance schedules, availability of machines, etc. All these factors have to be considered as they play a role in controlling the whole production process, which is really the overall goal. Another difference between intelligent and conventional control is the separation between controller and the system to be controlled. In conventional control the system to be controlled, called the plant, typically is separate and distinct from the controller. The controller is designed by the control designer, while the plant is, in general, given and cannot be changed. In intelligent control problems, which are most often complex and challenging, there may not be a clear separation of the plant and the controller; the control laws may be embedded in and part of the system to be controlled. This opens new opportunities and challenges as it may be possible to affect the design of processes in a more systematic way. Areas relevant to intelligent control, in addition to conventional control, include hybrid systems, planning and knowledge-based systems, intelligent agents, machine learning, search algorithms, fault diagnosis and control reconguration, and real-time and embedded systems, predicate logic, automata, Petri nets, neural networks, and fuzzy logic genetic algorithms. In addition, in order to control complex systems, one has to deal effectively with the computational complexity issue; this has been in the periphery of the interests of the researchers in conventional control, but it is clear that computational complexity is a central issue whenever one attempts to control complex systems.
Intelligence and Intelligent Control It is appropriate at this point to comment briey on the meaning of the word intelligent in intelligent control. The precise denition of intelligence has been eluding mankind for thousands of years. More recently, this issue has been addressed by disciplines such as psychology, philosophy, biology, and of course articial intelligence (AI). AI is dened to be the study of mental faculties through the use of computational models. No consensus has yet emerged of what constitutes intelligence. The controversy surrounding the widely used IQ tests also points to the fact that we are far away from having understood these issues. A term that has been introduced more recently is that of computational intelligence to distinguish machine intelligence from human intelligence. In this article we introduce and discuss several characterizations of intelligent systems that appear to be useful when attempting to address complex control problems. Intelligent controllers can be seen as machines which emulate human mental faculties such as adaptation and learning, planning under large uncertainty, and coping with large amounts of data in order to control complex processes effectively. This is the justication for the use of the term intelligent in intelligent control, since these mental faculties are considered to be important attributes of human intelligence. An alternative term, which is discussed later in this article, is autonomous (intelligent) control; it emphasizes the fact that an intelligent controller typically aims to attain higher degrees of autonomy in accomplishing and even setting control goals, rather than stressing the (intelligent) methodology that achieves those goals. Dening Intelligent Control Systems Intelligent systems can be characterized in a number of ways and along a number of dimensions. There are certain attributes of intelligent systems that are of particular interest in the control of systems; see Ref. 1. We begin with a general characterization of intelligent systems: An intelligent system has the ability to act appropriately in an uncertain environment, where an appropriate action is that which increases the probability of success, and success is the achievement of behavioral subgoals that support the systems ultimate goal. In order for a manmade intelligent system to act appropriately, it may emulate functions of living creatures and ultimately human mental facilities. An intelligent system can be characterized along a number of dimensions. There are degrees or levels of intelligence that can be measured along the various dimensions of intelligence. At a minimum, intelligence requires the ability to sense the environment, to make decisions, and to control action. Higher levels of intelligence may include the ability to recognize objects and events, to represent knowledge in a world model, and to reason about and plan for the future. In advanced forms, intelligence provides the capacity to perceive and understand, to choose wisely, and to act successfully under a large variety of circumstances so as to survive and prosper in a complex and often hostile environment. Intelligence can be observed to grow and evolve, both through growth in computational power and through accumulation of knowledge of how to sense, decide, and act
in a complex and changing world. This characterization of an intelligent system is rather general. Accordingly, a great number of systems can be considered intelligent. In fact, according to this denition even a thermostat may be considered to be an intelligent system, although of a low level of intelligence. It is common, however, to call a system intelligent when in fact it has a rather high level of intelligence. There exist a number of alternative but related denitions of intelligent systems that emphasize systems with high degrees of intelligence. For example, the following denition emphasizes the fact that the system in question processes information, and it focuses on manmade systems and intelligent machines: Machine intelligence is the process of analyzing, organizing, and converting data into knowledge; where (machine) knowledge is dened to be the structured information acquired and applied to remove ignorance or uncertainty about a specic task pertaining to the intelligent machine. This denition relates to the principle of increasing precision with decreasing intelligence of Saridis. Next, an intelligent system can be characterized by its ability to assign subgoals and control actions dynamically in an internal or autonomous fashion: Many adaptive or learning control systems can be thought of as designing a control law to meet well-dened control objectives. This activity represents the systems attempt to organize or order its knowledge of its own dynamical behavior so as to meet a control objective. The organization of knowledge can be seen as one important attribute of intelligence. If this organization is done autonomously by the system, then intelligence becomes a property of the system, rather than of the systems designer. This implies that systems that autonomously (self)-organize controllers with respect to an internally realized organizational principle are intelligent control systems. A procedural characterization of intelligent systems is given as follows: Intelligence is a property of the system that emerges when the procedures of focusing attention, combinatorial search, and generalization are applied to the input information in order to produce the output. One can easily deduce that once a string of the procedures given previously is dened, the other levels of resolution of the structure of intelligence grow as a result of the recursion. Having only one-level structure leads to a rudimentary intelligence that is implicit in the thermostat or to a variablestructure sliding mode controller. Control and Intelligent Systems The concepts of intelligence and control are closely related and the term intelligent control has a unique and distinguishable meaning. An intelligent system must dene and use goals. Control is then required to move the system to these goals and to dene such goals. Consequently, any intelligent system will be a control system. Conversely, intelligence is necessary to provide desirable functioning of systems under changing conditions, and it is necessary to achieve a high degree of autonomous behavior in a control system. Since control is an essential part of any intelligent system, the term intelligent control systems is sometimes used in engineering literature instead of intelligent systems
or intelligent machines. The term intelligent control system simply stresses the control aspect of the intelligent system. Characteristics or Dimensions of Intelligent Systems There are several essential properties present in different degrees in intelligent systems. One can perceive them as intelligent system characteristics or dimensions along which different degrees or levels of intelligence can be measured. In the following we discuss three such characteristics that appear to be rather fundamental in intelligent control systems. Adaptation and Learning. The ability to adapt to changing conditions is necessary in an intelligent system. Although adaptation does not necessarily require the ability to learn, for systems to be able to adapt to a wide variety of unexpected changes, learning is essential. So the ability to learn is an important characteristic of (highly) intelligent systems. Autonomy and Intelligence. Autonomy in setting and achieving goals is an important characteristic of intelligent control systems. When a system has the ability to act appropriately in an uncertain environment for extended periods of time without external intervention it is considered to be highly autonomous. There are degrees of autonomy; an adaptive control system can be considered as a system of higher autonomy than a control system with xed controllers, as it can cope with greater uncertainty than a xed feedback controller. Although for low autonomy no intelligence (or low intelligence) is necessary, for high degrees of autonomy, intelligence in the system (or high degrees of intelligence) is essential. Structures and Hierarchies. In order to cope with complexity, an intelligent system must have an appropriate functional architecture or structure for efcient analysis and evaluation of control strategies. This structure should provide a mechanism to build levels of abstraction (resolution, granularity) or at least some form of partial ordering so to reduce complexity. An approach to study intelligent machines involving entropy (of Saridis) emphasizes such efcient computational structures. Hierarchies (that may be approximate, localized, or combined in heterarchies) that are able to adapt may serve as primary vehicles for such structures to cope with complexity. The term hierarchies refers to functional hierarchies or hierarchies of range and resolution along spatial or temporal dimensions, and it does not necessarily imply hierarchical hardware. Some of these structures may be hardwired in part. To cope with changing circumstances the ability to learn is essential so these structures can adapt to signicant, unanticipated changes. In view of the preceding points, a working characterization of intelligent systems [or of (highly) intelligent (control) systems or machines] that captures the essential characteristics present in any such system follows: An intelligent system must be highly adaptable to signicant unanticipated changes, and so learning is essential. It must ex-
INTELLIGENT LEARNING CONTROL The term intelligent control was coined in the 1970s. Earlier used terms included learning control and selforganizing control. A brief description of some of the early developments in the area that is known today as intelligent control is now given. As discussed previously, learning is an important dimension or attribute of intelligent control. Highly autonomous behavior is a very desirable characteristic of advanced control systems, so they perform well under changing conditions in the plant and the environment (even in the control goals), without external intervention (note that intelligent autonomous control is discussed at length later). This requires the ability to adapt to changes affecting, in a signicant manner, the operating region of the system. Adaptive behavior of this type typically is not offered by conventional control systems. Additional decisionmaking abilities should be added to meet the increased control requirements. The controllers capacity to learn from past experience is an integral part of such highly autonomous controllers. The goal of introducing learning methods in control is to broaden the region of operability of conventional control systems. Therefore the ability to learn is one of the fundamental attributes of autonomous intelligent behavior; see Refs. 1 and 2. The ability of manmade systems to learn from experience and, based on that experience, improve their performance is the focus of machine learning. Learning can be seen as the process whereby a system can alter its actions to perform a task more effectively due to increases in knowledge related to the task. The actions that a system may take depend on the nature of the system. For example, a control system may change the type of controller used, or vary the parameters of the controller, after learning that the current controller does not perform satisfactorily within a changing environment. Similarly, a robot may need to change its visual representation of the surroundings after learning of new obstacles in the environment. The type of action taken by the machine is dependent upon the nature of the system and the type of learning system implemented. The ability to learn entails such issues as knowledge acquisition, knowledge representation, and some level of inference capability. Learning, considered fundamental to intelligent behavior, and in particular the computer modeling of learning processes have been the subject of research in the eld of machine learning since the 1960s; see Refs. 19 and 14. Learning and Adaptive Control The problem of learning in automatic control systems has been studied in the past, especially in the late 1960s, and it has been the topic of numerous papers and books; see, for example, Refs. 1519. References 15, 17, and 19 provide surveys on the early learning techniques. All of these approaches involve a process of classication in which all or part of the prior information required is unknown or incompletely known. The elements or patterns that are presented to the control system are collected into groups that correspond to different pattern classes or regions; see Ref. 19.
Figure 1. Intelligent autonomous controller functional architecture. The three levels of a hierarchical intelligent control architecture are the execution level, the coordination level, and the management or organization level.
hibit a high degree of autonomy in dealing with changes. It must be able to deal with signicant complexity, and this leads to certain types of functional architectures such as hierarchies or heterarchies. Some Examples Man-made systems that solve complex problems and incorporate some of the essential characteristics of intelligent control systems given previously do exist today. Here are some examples from Ref. 1: A hierarchically intelligent control system was designed and built at the Rensselaer Polytechnic Institute laboratories to do truss construction remotely in deep space for the NASA space station Freedom. This intelligent control system had a functional hierarchy that consisted of three levels: the lowest was the execution level, the highest was the organization level, and the middle was the coordination level (see Fig. 1 and the section on intelligent autonomous control later in this article); see Ref. 1. Intelligent control systems using the real-time control system (RCS) implementations from the National Institute for Standards and Technology (NIST): robot vision-based object pursuit; robot deburring; include an automated manufacturing research facility; robot machine loading or unloading for a milling workstation; multiple autonomous undersea vehicles; NASA space station telerobotics and the Mars vehicle; army eld material handling robot; DARPA submarine automation; coal mine automation; and army unmanned land vehicles. Other examples of existing intelligent control systems include mobile robots based on sulsumption and other architectures. See Refs. 16. For additional information and insight into the foundations of intelligent control, the interested reader may refer to Refs. 112.
Thus learning was viewed as the estimation or successive approximation of the unknown quantities of a function; see Ref. 15. The approaches developed for such learning problems can be separated into two categories: deterministic and stochastic. Where can learning be used in the control of systems? As it was already mentioned, learning plays an essential role in the autonomous control of systems. There are many areas in control in which learning can be used to advantage and these needs can be briey classied as follows: (1) Learning about the plant; that is, learning how to incorporate changes and then how to derive new plant models. (2) Learning about the environment; this can be done using methods ranging from passive observation to active experimentation. (3) Learning about the controller; for example, learning how to adjust certain controller parameters to enhance performance. (4) Learning new design goals and constraints. What is the relation between adaptive control and learning control? Learning is achieved, in a certain sense, when an adaptive control algorithm is used to adapt the controller parameters so that, for example, stability is maintained. In this case the system learns and the knowledge acquired is the new values for the parameters. Note, however, that if later the same changes occur again and the system is described by exactly the same parameters identied earlier, the adaptive control algorithm still needs to recalculate the controller and perhaps the plant parameters since nothing was kept in memory. So, in that sense the system has not learned. It has certainly learned what to do when certain types of changes take place. In particular, it has been told exactly what to do, that is, it was given the adaptive algorithm, and this is knowledge by rote learning. The knowledge represented by the new values of the controller and the plant parameters and the circumstances under which these values are appropriate are not retained. So a useful rule of thumb is that for a controller to be a learning controller, memory is required so that past knowledge is stored in such a way that it can be used to benet when a similar situation arises. With respect to terminology it is perhaps benecial at this point to bring in a bit of history. In the 1960s, adaptive control and learning received a lot of attention in the control literature. It was not always clear, however, what those terms meant. The comment by Y. Tsypkin, in Ref. 18 describes quite clearly the atmosphere of the period: It is difcult to nd more fashionable and attractive terms in the modern theory of automatic control than the terms of adaptation and learning. At the same time, it is not simple to nd any other concepts which are less complex and more vague. Adaptation, learning, self-organizing systems, and control were competing terms for similar research areas. The term pattern recognition was appearing together with adaptive, learning, and self-organizing systems in the control literature of that era. It is obvious that there was no agreement as to the meaning of these terms and their relation. Pattern recognition is today a research discipline in its own right, which develops and uses an array of methods ranging from conventional algorithms to articial intelligence methods. The term self-organizing system is not being used as much today in the control literature. Adaptive control has gained renewed popularity in the last decades and mainly emphasizes studies in the convergence of adap-
tive algorithms and in the stability of adaptive systems; the systems considered are primarily systems described by differential (or difference) equations where the coefcients are (partially) unknown. In an attempt to enhance the applicability of adaptive control methods, learning control has been reintroduced in the control literature; see, for example, Refs. 1012 and 20 for learning methods in control with emphasis on neural networks. INTELLIGENT CONTROL FOR HIGH-AUTONOMY SYSTEMS From a control systems point of view the use of intelligent control methods is a natural next step in the quest for building systems with higher degrees of autonomy. These ideas are discussed in the following. In the design of controllers for complex dynamical systems there are needs today that cannot be successfully addressed with the existing conventional control theory. They mainly pertain to the area of uncertainty. Heuristic methods may be needed to tune the parameters of an adaptive control law. New control laws to perform novelcontrol functions to meet new objectives should be designed while the system is in operation. Learning from past experience and planning control actions may be necessary. Failure detection and identication are needed. Such functions have been performed in the past by human operators. To increase the speed of response, to relieve operators from mundane tasks, to protect them from hazards, a high degree of autonomy is desired. To achieve this, highlevel decision-making techniques for reasoning under uncertainty and taking actions must be utilized. These techniques, if used by humans, may be attributed to intelligent behavior. Hence, one way to achieve a high degree of autonomy is to utilize high-level decision-making techniques, intelligent methods, in the autonomous controller. Autonomy is the objective, and intelligent controllers are one way to achieve it. Evolution of Control Systems and the Quest for Higher Autonomy The rst feedback device on record was the water clock invented by Ktesibios, a Greek living in Alexandria, Egypt around the 3rd century BC This was certainly a successful device, as water clocks of similar design were still being made in Baghdad when the Mongols captured that city in 1258 AD The rst mathematical model to describe plant behavior for control purposes is attributed to J. C. Maxwell, of the Maxwell equations fame. In 1868 Maxwell used differential equations to explain instability problems encountered with James Watts yball governor; the governor was introduced in 1769 to regulate the speed of steam engine vehicles. When Maxwell used mathematical modeling and methods to explain instability problems encountered with Watts yball governor, it demonstrated the importance and usefulness of mathematical models and methods in understanding complex phenomena and signaled the beginning of mathematical system and control theory. It also signaled the end of the era of intuitive invention. Control theory made signicant strides in the past 120 years, with
the use of frequency domain methods and Laplace transforms in the 1930s and 1940s and the development of optimal control methods and state-space analysis in the 1950s and 1960s. Optimal control in the 1950s and 1960s, followed by progress in stochastic, robust, adaptive, nonlinear hybrid, and networked control methods in the 1960s to today, has made it possible to control signicantly more complex dynamical systems than the original yball governor more accurately. Conventional control systems are designed today using mathematical models of physical systems. A mathematical model that captures the dynamical behavior of interest is chosen and then control design techniques are applied, aided by computer-aided design (CAD) packages, to design the mathematical model of an appropriate controller. The controller is then realized via hardware or software and it is used to control the physical system. The procedure may take several iterations. The mathematical model of the system must be simple enough so that it can be analyzed with available mathematical techniques and accurate enough to describe the important aspects of the relevant dynamical behavior. It approximates the behavior of a plant in the neighborhood of an operating point. The control method and the underlying mathematical theory were developed to meet the ever-increasing control needs of our technology. The need to achieve the demanding control specications for increasingly complex dynamical systems has been addressed by using more complex mathematical models such as nonlinear and stochastic ones, and by developing more sophisticated design algorithms for, say, optimal control. The use of highly complex mathematical models, however, can seriously inhibit our ability to develop control algorithms. Fortunately, simpler plant models, for example, linear models, can be used in the control design; this is possible because of the feedback used in control that can tolerate signicant model uncertainties. When the xed feedback controllers are not adequate, then adaptive controllers are used. Controllers can then be designed to meet the specications around an operating point where the linear model is valid, and then via a scheduler a controller emerges that can accomplish the control objectives over the whole operating range. This is, for example, the method typically used for aircraft ight control and it is a method used to design xed controllers for certain classes of nonlinear systems. Adaptive control in conventional control theory has a specic and rather narrow meaning. In particular, it typically refers to adapting to variations in the constant coefcients in the equations describing the linear plant; these new coefcient values are identied and then used, directly or indirectly, to reassign the values of the constant coefcients in the equations describing the linear controller. Adaptive controllers provide for wider operating ranges than xed controllers and so conventional adaptive control systems can be considered to have higher degrees of autonomy than control systems employing xed feedback controllers. Intelligent Control for High-Autonomy Systems There are cases in which we need to increase the operating range of the system signicantly. We must be able
to deal effectively with signicant uncertainties in models of increasingly complex dynamical systems in addition to increase the validity range of our control methods. We need to cope with signicant unmodeled and unanticipated changes in the plant, in the environment, and in the control objectives. This will involve the use of intelligent decisionmaking processes to generate control actions so that a certain performance level is maintained even though there are drastic changes in the operating conditions. I have found useful to keep in mind an example that helps set goals for the future and also teaches humility, as it shows how difcult, demanding, and complex autonomous systems can be. Currently, if there is a problem on the space shuttle, the problem is addressed by the large number of engineers working in Houston Control, the ground station. When the problem is solved the specic detailed instructions about how to deal with the problem are sent to the shuttle. Imagine the time when we will need the tools and expertise of all Houston Control engineers aboard the space shuttle or another space vehicle for extended space travel. What needs to be achieved to accomplish this goal is certainly highly challenging! In view of the above it is quite clear that in the control of systems there are requirements today that cannot be successfully addressed with the existing conventional control theory. It should be pointed out that several functions proposed in later sections, to be part of the high-autonomy control system, have been performed in the past by separate systems; examples include fault trees in chemical process control for failure diagnosis and hazard analysis, and control system design via expert systems. An Intelligent Control Architecture for High-Autonomy Systems To illustrate the concepts and ideas involved and to provide a more concrete framework to discuss the issues, a hierarchical functional architecture of an intelligent controller that is used to attain high degrees of autonomy in future space vehicles is briey outlined as an example; full details can be found in Ref. 21. This hierarchical architecture has three levels, the execution level, the coordination level, and the management or organization level. The architecture exhibits certain characteristics, which have been shown in the literature to be necessary and desirable in autonomous systems. Based on this architecture we identify the important fundamental issues and concepts that are needed for an autonomous control theory. Architecture Overview: Structure and Characteristics. The overall functional architecture for an autonomous controller is given by the architectural schematic of the Fig. 1. This is a functional architecture rather than a hardware processing one; therefore it does not specify the arrangement and duties of the hardware used to implement the functions described. Note that the processing architecture also depends on the characteristics of the current processing technology; centralized or distributed processing may be chosen for function implementation depending on available computer technology.
The architecture in Fig. 1 has three levels. At the lowest level, the execution level, there is the interface to the vehicle and its environment (the process in the gure) via the sensors and actuators. At the highest level, the management or organization level, there is the interface to the pilot and crew, ground station, or onboard systems. The middle level, called the coordination level, provides the link between the execution level and the management level. Note that we follow the somewhat standard viewpoint that there are three major levels in the hierarchy. It must be stressed that the system may have more or fewer than three levels, which, however, can be conceptually combined into three levels. Some characteristics of the system that dictate the actual number of levels are the extent to which the operator can intervene in the systems operations, the degree of autonomy or level of intelligence in the various subsystems, and the hierarchical characteristics of the plant. Note that the three levels shown in Fig. 1 are applicable to most architectures of intelligent autonomous controllers, by grouping together sublevels of the architecture if necessary. The lowest, execution level, involves conventional control algorithms, while the highest, management and organization level, involves only higher-level, intelligent, decision-making methods. The coordination level provides the interface between the actions of the other two levels and it uses a combination of conventional and intelligent decision-making methods. The sensors and actuators are implemented mainly with hardware. Software and perhaps hardware are used to implement the execution level. Mainly software is used for both the coordination and management levels. There are multiple copies of the control functions at each level, more at the lower and fewer at the higher levels. Note that the autonomous controller is only one of the autonomous systems on the space vehicle. It is responsible for all the functions related to the control of the physical system and allows for continuous on-line development of the autonomous controller and provides for various phases of mission operations. The tier structure of the architecture allows us to build on existing advanced control theory. Development progresses, creating each time higher-level adaptation and a new system that can be operated and tested independently. The autonomous controller performs many of the functions currently performed by the pilot, crew, or ground station. The pilot and crew are thus relieved from mundane tasks and some of the ground-station functions are brought aboard the vehicle. In this way the degree of autonomy of the vehicle is increased. Functional Operation. In Fig. 1, commands are issued by higher levels to lower levels and response data ows from lower lever upwards. However we allow the ow of data within a level and so the functional architecture is not a pure hierarchy but rather an heterarchy. Parameters of subsystems can be altered by systems one level above them in the hierarchy. There is a delegation and distribution of tasks from higher to lower levels and a layered distribution of decision-making authority. At each level, some preprocessing occurs before information is sent to higher levels. If requested, data can be passed from the lowest subsystem to the highest, for example, for display. All sub-
systems provide status and health information to higher levels. Human intervention is allowed even at the control implementation supervisor level, with the commands, however, passed down from the upper levels of the hierarchy. Here is a simple illustrative example to clarify the overall operation of the autonomous controller. Suppose that the pilot desires to repair a satellite. After dialogue with the management level via the interface, the task is rened to repair satellite using robot A. This is a decision made using the capability-assessing, performance-monitoring, and planning functions of the management level. The management level decides if the repair is possible under the current performance level of the system and in view of nearterm other planned functions. Using its planning capabilities, it then sends a sequence of subtasks to the coordination level sufcient to achieve the repair. This sequence could be to order robot A to go to satellite at coordinates (x, y, z); open repair hatch; repair. The coordination level, using its planner, divides, say, the rst subtask, go to satellite at coordinates (x, y, z), into smaller subtasks: go from start to (x1 , y1 , z1 ), then maneuver around obstacle, move to (x2 , y2 , z2 ), . . . , arrive at the repair site and wait. The other subtasks are divided in a similar manner. This information is passed to a control implementation supervisor at the coordination level, which recognizes the task and uses stored control laws to accomplish the objective. The subtask go from start to (x1 , y1 , z1 ) can, for example, be implemented using stored control algorithms to rst proceed forward 10 m, to the right 15 , etc. These control algorithms are executed in the controller at the execution level utilizing sensor information; the control actions are implemented via the actuators. Characteristics of Hierarchical Intelligent Controllers for High-Autonomy Systems Based on the architecture previously described, important fundamental concepts and characteristics that are needed for an autonomous intelligent control theory are now identied. The fundamental issues that must be addressed for a quantitative theory of autonomous intelligent control are discussed. There is a successive delegation of duties from the higher to lower levels; consequently the number of distinct tasks increases as we go down the hierarchy. Higher levels are concerned with slower aspects of the systems behavior and with its larger portions, or broader aspects. There is then a smaller contextual horizon at lower levels, that is, the control decisions are made by considering less information. Also notice that higher levels are concerned with longer time horizons than lower levels. Due to the fact that there is the need for high-level decision making abilities at the higher levels in the hierarchy, the proposition has been put forth that there is increasing intelligence as one moves from the lower to the higher levels. This is reected in the use of fewer conventional numeric-algorithmic methods at higher levels as well as the use of more symbolic decisionmaking methods. This is the principle of increasing intelligence with decreasing precision of Saridis; see also Ref. 8 and the references therein. The decreasing precision is reected by a decrease in time scale density, decrease in
bandwidth or system rate, and a decrease in the decision (control action) rate. (These properties have been studied for a class of hierarchical systems in Ref. 22.) All these characteristics lead to a decrease in granularity of models used, or equivalently, to an increase in model abstractness. Model granularity also depends on the dexterity of the autonomous controller. It is important at this point to discuss briey the dexterity of the controller. The execution level of a highly dexterous controller is very sophisticated and it can accomplish complex control tasks. The coordination level can issue commands such as move 15 cm to the right and grip standard, xed dimension cylinder in a dexterous controller, or it can completely dictate each mode of each joint (in a manipulator) move joint 1 15 then move joint 5 3 , etc., in a less dexterous one. The simplicity and level of abstractness of commands in an autonomous controller depend on its dexterity. The more sophisticated the execution level is the simpler are the commands that the control implementation supervisor needs to issue. Notice that a very dexterous robot arm may itself have a number of autonomous functions. If two such dexterous arms were used to complete a task that required the coordination of their actions, then the arms would be considered to be two dexterous actuators and a new supervisory autonomous controller would be placed on top for the supervision and coordination task. In general, this can happen recursively, adding more intelligent autonomous controllers as the lower-level tasks, accomplished by autonomous systems, need to be supervised. There is an ongoing evolution of the intelligent functions of an autonomous controller. It is interesting to observe the following. Although there are characteristics that separate intelligent from nonintelligent systems, as intelligent systems evolve, the distinction becomes less clear. Systems that were originally considered to be intelligent evolve to gain more characteristics of what is considered to be nonintelligent, numeric-algorithmic systems. An example is a route planner. Although there are AI route-planning systems now, as problems like route planning become better understood, more conventional numeric-algorithmic solutions are developed. The AI methods that are used in intelligent systems help us to understand complex problems so we can organize and synthesize new approaches to problem solving, in addition to being problem-solving techniques themselves. AI techniques can be viewed as research vehicles for solving very complex problems. As the problem solution develops, purely algorithmic approaches, which have desirable implementation characteristics, substitute AI techniques and play a greater role in the solution of the problem. It is for this reason that we concentrate on achieving autonomy and not on whether the underlying system can be considered intelligent. Models for Intelligent Controllers. In highly autonomous control systems, the plant is normally so complex that it is either impossible or inappropriate to describe it with conventional mathematical system models such as differential or difference equations. Even though it might be possible to describe some system accurately with highly complex nonlinear differential equations, it may be inappropriate
if this description makes subsequent analysis too difcult or too computationally complex to be useful. The complexity of the plant model needed in design depends on both the complexity of the physical system and how demanding the design specications are. There is a tradeoff between model complexity and our ability to perform an analysis of the system via the model. However, if the control performance specications are not too demanding, a more abstract, higher-level, model can be utilized, which will make subsequent analysis simpler. This model intentionally ignores some of the system characteristics, specically those that need not be considered in attempting to meet the particular performance specications; see also the discussion on hybrid systems later in this article. For example, a simple temperature controller could ignore almost all dynamics of the house or the ofce and consider only a temperature threshold model of the system to switch the furnace off or on. Discrete event system (DES) models using nite automata, Petri nets, queuing network models, Markov chains, etc., are quite useful for modeling the higher-level decision-making processes in the intelligent autonomous controller. The choice of whether to use such models will, of course, depend on what properties of the autonomous system need to be studied. The quantitative, systematic techniques for modeling, analysis, and design of control systems are of central and utmost practical importance in conventional control theory. Similar techniques for intelligent autonomous controllers do not exist. This is mainly due to the hybrid structure (nonuniform, nonhomogeneous nature) of the dynamical systems under consideration; they include both continuous-state and discrete-state systems. Modeling techniques for intelligent autonomous systems must be able to support a macroscopic view of the dynamical system; hence it is necessary to represent both numeric and symbolic information. The nonuniform components of the intelligent controller all take part in the generation of the low-level control inputs to the dynamical system; therefore they all must be considered in a complete analysis. Research could begin by using different models for different components of the intelligent autonomous controller, since much can be attained by using the best available models for the various components of the architecture and joining them via some appropriate interconnecting structure. For instance, systems that are modeled with a logical discrete event system (DES) model at the higher levels and a difference or differential equation at the lower level should be examined; see the discussion on hybrid systems later in this article. In any case, good understanding of hierarchical models is necessary for the analysis and synthesis of intelligent autonomous controllers.
Research Directions. One can roughly categorize research in the area of intelligent autonomous control into two areas: conventional control theoretic research, which addresses the control functions at the execution and coordination levels, and the modeling, analysis, and design of higher-level decision-making systems found in the management and coordination levels.
It is important to note that in order to obtain a high degree of autonomy it is necessary to adapt or learn. Neural networks offer methodologies to perform learning functions in the intelligent autonomous controller. In general, there are potential applications of neural networks at all levels of hierarchical intelligent controllers that provide higher degrees of autonomy to systems. Neural networks are useful at the lowest execution levelwhere the conventional control algorithms are implemented via hardware and softwarethrough the coordination level, to the highest management level, at which decisions are being made based on possibly uncertain and/or incomplete information. One may point out that at the execution levelconventional control levelneural network properties such as the ability for function approximation and the potential for parallel implementation appear to be very important. In contrast, at higher levels abilities such as pattern classication and the ability to store information in, say, an associative memory appear to be of signicant interest. Machine learning is of course important at all levels. We stress that in control systems with high degrees of autonomy we seek to widen the operating range of the system signicantly so that nontrivial failures and environmental changes can occur and performance will still be maintained. All of the conventional control techniques are useful in the development of autonomous controllers and they are relevant to the study of autonomous control. It is the case, however, that certain techniques are more suitable for interfacing to the autonomous controller and for compensating for signicant system failures. For instance, the area of restructurable or recongurable control systems studies techniques to recongure controllers when signicant failures occur. Conventional modeling, analysis, and design methods should be used whenever applicable for the components of the intelligent autonomous control system as well as fuzzy controllers. For instance, they should be used at the execution level of many autonomous controllers. The symbolicnumeric interface is a very important issue; consequently it should be included in any analysis. There is a need for systematically generating less detailed, more abstract models from differential or difference equation models to be used in higher levels of the autonomous controller; see the later discussion on hybrid systems. Tools for the implementation of this information extraction also need to be developed. In this way conventional analysis can be used in conjunction with the developed analysis methods to obtain an overall quantitative, systematic analysis paradigm for intelligent autonomous control systems. In short, we propose to use hybrid modeling, analysis, and design techniques for nonuniform systems. This approach is not unlike the approaches used in the study of any complex phenomena by the scientic and engineering communities.
HYBRID SYSTEMS Hybrid control systems contain two distinct types of systems, systems with continous dynamics and systems with discrete dynamics, that interact with each other. Their study is central in designing intelligent control systems
with a high degree of autonomy and it is essential in designing discrete event supervisory controllers for continuous systems; see Refs. 126. Hybrid control systems typically arise when continuous processes interact with, or are supervised by, sequential machines. Examples of hybrid control systems are common in practice and are found in such applications as exible manufacturing, chemical process control, electric power distribution, and computer communication networks. A simple example of a hybrid control system is the heating and cooling system of a typical home. The furnace and air conditioner, along with the heat-ow characteristics of the home, form a continuous-time system that is to be controlled. The thermostat is a simple discrete-event system that basically handles the symbols {too hot, too cold} and {normal}. The temperature of the room is translated into these representations in the thermostat and the thermostats response is translated back to electrical currents that control the furnace, air conditioner, blower, etc. Since the continuous and discrete dynamics coexist and interact with each other it is important to develop models that accurately describe the dynamic behavior of such hybrid systems. In this way it is possible to develop control strategies that fully take into consideration the relation and interaction of the continuous and discrete parts of the system. In the past, models for the continuous- and discrete-event subsystems were developed separately; the control law was then derived in a rather empirical fashion, except in special cases such as the case of digital controllers for linear time-invariant systems. The study of hybrid systems provides the backbone for the formulation and implementation of learning control policies. In such policies, the control acquires knowledge (discrete data) to improve the behavior of the system as it evolves in time. Hybrid systems have become a distinctive area of study due to opportunities to improve on traditional control and estimation technologies by providing computationally effective methodologies for the implementation of digital programs that design or modify the control law in response to sensordetected events, or as a result of adaptation and learning. The interested reader should consult Refs. 25, 26 and especially the references therein. Certain important issues in hybrid systems are now briey discussed using a paradigm of a continuous system supervised by a DES controller from Refs. 23 and 24. The hybrid control system of interest here consists of a continuous-state system to be controlled, also called the plant, and a discrete-state controller connected to the plant via an interface; see Fig. 2. The plant contains all continuous-state subsystems of the hybrid control system, such as any conventional continuous-state controllers that may have been developed and a clock if time and synchronous operations are to be modeled. The controller is an event-driven, asynchronous DES, described by a nite state automaton or an ordinary Petri net. The hybrid control system also contains an interface that provides the means for communication between the continuous-state plant and the DES controller. The interface receives information from the plant in the form of a measurement of a continuous variable z(t), such as the continuous state, and issues a sequence of symbols {z (i)}
10
CONCLUDING REMARKS There may be the temptation to classify the area of intelligent autonomous systems as simply a collection of methods and ideas already addressed elsewhere, the need only being some kind of intelligent assembly and integration of known techniques. This is not true. The theory of control systems is not covered by, say, the area of applied mathematics, because control has different needs and therefore asks different questions. The problems of interest in intelligent systems require development of novel concepts, approaches, and methods in computer science, operations research control systems, to mention but a few. The marriage of all these elds can only be benecial to all. Computer science and operation research methods are increasingly used in control problems, while control system concepts such as feedback and methods are providing the base for new theories and methods in those areas. Intelligent control for high degree of autonomy systems is a vibrant research and applications area where developments are followed by interdisciplinary research and advances in computational networking, sensing, and artworking technologies.
Figure 2. Hybrid supervisory control architecture. The interface receives continuous measurements z(t) and issues a sequence of symbols {z (i)} which the DES controller processes to issue a sequence of control symbols {r (i)}. These are translated by the interface to (piecewise) continuous input commands r(t).
to the DES controller. It also receives a sequence of control symbols {r (i)} from the controller and issues (piecewise) continuous input commands r(t) to the plant. The interface plays a key role in determining the dynamics and the behavior of the hybrid control system. Understanding how the interface affects the properties of the hybrid system is one of the fundamental issues in the theory of hybrid control systems. The interface can be chosen to be simply a partitioning of the state space; see Ref. 23. If memory is necessary to derive an effective control, it is included in the DES controller and not in the interface. Also the piecewise continuous command signal issued by the interface is simply a staircase signal, not unlike the output of a zero-order hold in a digital control system. Including an appropriate continuous system at (the input of) the plant, signals such as ramps and sinusoids can be generated if desired. So the simple interface is used without loss of generality. It allows analysis of the hybrid control system with development of properties such as controllability, stability and determinism, in addition to control design methodologies; see Refs. 23 and 24. In general, the design of the interface depends not only on the plant to be controlled, but also on the control policies available, as well as on the control goals. Depending on the control goals, one may or may not need, for example, detailed state information; this corresponds to small or large regions in the partition of the measured signal space (or greater of lower granularity). This is, of course, not surprising as it is rather well known that to stabilize a system, for example, requires less detailed information about the systems dynamic behavior than to do, say, tracking. The fewer the distinct regions in the partitioned signal space, the simpler (fewer states) the resulting DES plant model and the simpler the DES controller design. Since the systems to be controlled via hybrid controllers are typically complex, it is important to make every effort to use only the necessary information to attain the control goals. This leads to simpler interfaces that issue only the necessary number of distinct symbols and to simpler DES plant models and controllers. The question of systematically determining the minimum amount of information needed from the plant in order to achieve specic control goals via a number of specialized control policies is an important question.
BIBLIOGRAPHY
1. P. J. Antsaklis, Dening intelligent control, Report of the IEEE Control Systems Society Task Force on Intelligent Control (P. J. Antsaklis, Chair; J. Albus, K. Baheti, J. D. Birdwell, M. Lemmon, M. Mataric, A. Meystel, K. Narendra, K. Passino, H. Rauch, G. Saridis, H. Stephanou, P. Werbos, Members), IEEE Control Syst. Mag., 14 (3): 45, 5866, 1994. 2. P. J. Antsaklis, K. M. Passino (eds.), An Introduction to Intelligent and Autonomous Control, Norwell, MA: Kluwer, 1993. 3. A. Meystel, Autonomous Mobile Robots, River Edge, NJ: World Scientic, 1991. 4. K. P. Valavanis, G. N. Saridis, Intelligent Robotic System Theory: Design and Applications, Norwell, MA: Kluwer, 1992. 5. J. S. Albus and A. M. Meystel, Intelligent Systems:Architecture, Design, Control, Wiley-Interscience, 2001. 6. R. A. Brooks Cambrian Intelligence: The Early History of the New AI, The MIT Press, 1999. 7. J. S. Albus, Outline for a theory of intelligence, IEEE Trans. Syst. Man Cybern., 21 (3): 432509, 1991. 8. G. N. Saridis, K. P. Valavanis, Analytical design of intelligent machines, Automatica, 24 (2): 123133, 1988. 9. J. S. Sinha and A. M. Meystel, Engineering of Mind: An Introduction to the Science of Intelligent Systems, WileyInterscience, 2001. 10. D. A. White, D. A. Sofge (eds.), Handbook of Intelligent Control Neural, Fuzzy, and Adaptive Approaches, New York: Van Nostrand Reinhold, 1992. 11. M. M. Gupta, N. K. Sinha (eds.), Intelligent Control: Theory and Practice, Piscataway, NJ: IEEE Press, 1994. 12. N. K. Sinha andM. M. Gupta (Eds), Soft Computing & Intelligent Systems: Theory and Applications, Academic Press, 2000. 13. R. S. Michalski, J. G. Carbonell, T. M. Mitchell, Machine LearningAn Articial Intelligence Approach, Palo Alto, CA: Tioga, 1983.See also R. S. Michalski andG. Tecuci (eds.), Ma-
Intelligent Systems, Robots, Vehicles, Manufacturing chine LearningA Multistrategy Approach, San Francisco: Morgan-Kaufmann, 1994, Vol. IV. T. M. Mitchell, Machine Learning, WCB/McGraw-Hill, 1997. K. S. Fu, Learning control systemsReview and outlook, IEEE Trans. Autom. Control, 15: 210221, 1970. J. M. Mendel, K. S. Fu, Adaptive, Learning and Pattern Recognition Systems, New York: Academic, 1970. J. Sklansky, Learning systems for automatic control, IEEE Trans. Autom. Control, 11: 619, 1966. Y. A. Z. Tsypkin, Adaptation and Learning in Automatic Systems, New York: Academic, 1971. Y. A. Z. Tsypkin, Self-learningWhat is it? IEEE Trans. Autom. Control, 13: 608612, 1968. J. A. Farrell, M. M. Polycarpou, Adaptive Approximation Based Control: Unifying Neural, Fuzzy and Traditional Adaptive Approximation Approaches, Wiley-Interscience, 2006. P. J. Antsaklis, K. M. Passino, Introduction to Intelligent Control Systems with High Degree of Autonomy, inP. J. Antsaklis andK. M. Passino (eds.), An Introduction to Intelligent and Autonomous Control, Norwell, MA: Kluwer, 1993. K. M. Passino, P. J. Antsaklis, Event rates and aggregation in hierarchical discrete event systems, J. Discrete Event Dynamic Syst. Theory. Appl., 1 (3): 271287, 1992. P. J. Antsaklis, J. A. Stiver, M. D. Lemmon, Hybrid System Modeling and Autonomous Control Systems, inR. L. Grossman et al. (eds.), Hybrid Systems, New York: Springer-Verlag, 1993, 366392. J. A. Stiver, P. J. Antsaklis, M. D. Lemmon, A logical DES approach to the design of hybrid control systems, Math. Comput. Model., 23 (11/12): 1996. P. J. Antsaklis, A Brief Introduction to the Theory and Applications of Hybrid Systems, Proceedings of the IEEE, Special Issue on Hybrid Systems: Theory and Applications, P. J. Antsaklis (Ed.), Vol. 88, No. 7,pp. 879887, July 2000. P. J. Antsaklis and X. D. Koutsoukos, Hybrid Systems: Review and Recent Progress, in Software-Enabled Control: Information Technologies for Dynamical Systems, T. Samad andG. Balas (Eds.), IEEE Press,pp 273298, 2003.
11
14. 15. 16. 17. 18. 19. 20.
21.
22.
23.
24.
25.
26.
PANOS J. ANTSAKLIS
University of Notre Dame Notre Dame, IN, 46556
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Jet Engine Control, Implementations Standard Article H. Austin Spang III1 and Harold Brown2 1GE Research and Development Center, Schenectady, NY 2Gray Fox Technologies, Cincinnati, OH Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1017 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (273K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Basic Engine Types Simplified Engine Theory Control Requirements Sensors And Actuators Engine Controls Concluding Remarks About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
14
JET ENGINE CONTROL, IMPLEMENTATIONS
tion and deceleration schedules to provide transient limit protection. More advanced controls schedule variable engine geometry and augmentor fuel, provide fan and booster stall protection, control variable parasitic engine ows, improve integrated engine-airframe performance, and provide engine health monitoring and diagnostics. It should be noted that only recently have electronic computers been used to implement engine controls. This is primarily due to the inherent need for safe operation and the harsh temperature and vibration environment in which the computer operates. Many engines in use today are controlled by a hydromechanical controller commonly referred to as an HMU. These are ingenious mechanical computers, which implement the desired control strategy in terms of cams and mechanical integrators. Of necessity, the implemented control strategies must be fairly simple. A drawing of a typical HMU is shown in Fig. 1. More detailed discussions of their operation can be found in (4). The changeover to electronic controllers began in the 1980s as rugged integrated circuits became available and as the need for improved performance led to increased functionality and tighter control. Pratt and Whitney calls its controller a Digital Engine Control (DEC), while General Electric calls it a Full Authority Digital Electronic Control (FADEC). These are highly customized computers, whose complexity depends mainly on the number of sensor inputs and actuator outputs. Such electronic controllers result in higher engine operating efciencies, by allowing tighter engine control through the use of higher loop gains and improved strategies to reduce transient overshoot or undershoot. It also allows implementation of control algorithms, which would be difcult to implement mechanically.
BASIC ENGINE TYPES

In one form or another, jet engines power all but the smallest airplanes and helicopters. They produce propulsive thrust from the thermal energy of jet fuel. Sir Frank Whittle (1) is credited with developing the rst jet engine during the 1930s. It was similar to the turbosupercharger that had been developed in the 1920s, which also used a single-stage centrifugal compressor, a combustor, and a single-stage turbine. Apparently unaware of Whittles work, a jet engine was also patented in Germany by Hans von Ohain and Max Hahn (2,3) in 1936. The subsequent German development led to the JUMO 109 engine, which had many design features of more modern engines, such as a multistage axial compressor, turbine blade cooling, and a variable-area exhaust nozzle. Unfortunately, it was limited by available materials to an operating life of about 10 hours. Feedback control has been as essential part of a jet engine from the beginning. Engines are most effective when they can be operated at or near their mechanical or aerothermal limitations, such as rotor speeds, turbine temperatures, internal pressures, and so on. Controlling at but not exceeding a limit is a very important aspect of engine control, which must, therefore, provide both regulation and limit management. Minimum control requirements include a main fuel control for setting and holding steady-state thrust, with fuel accelera-
Three basic types of jet engines are in current use: 1. Turbojets 2. Turbofan engines 3. Turboprop/turboshaft engines The turbojet was the earliest form of jet engine, and is the simplest of the three. Its major components include a compressor, combustor, turbine (which drives the compressor), and exhaust nozzle. It produces a relatively high specic thrust, dened as thrust per kilogram of airow. It is the best type of engine for high subsonic and supersonic ight speeds. The turbofan uses a turbojet for its core and adds a fan in front of the core compressor and a second power turbine behind the core turbine, to drive the fan, as shown in Fig. 2. The ow capacity of the fan is designed to be substantially larger than the compressor, so that the excess air can be bypassed around the core and exhausted through a separate nozzle. The bypass approach reduces engine specic thrust, but increases propulsion efciency, thereby reducing fuel consumption and is the engine of choice for subsonic commercial airplanes. The turboprop or turboshaft engine includes the turbojet core and power turbine, but has no fan. Its power turbine can drive an external propeller or helicopter rotor through a gear
Evacuated bellows To engine nozzles Min press and shut off valve Burner press Fail-safe orifice Filter Atmos Burner-press.limiting-valve adj. Fine filter Ratio lever Droop cam followers Fixed orifice Throttle valve Min. flow adj.
Balancing nozzle Flapper valve
Fixed orifice Burner-press. limiting valve
Water drain Throttle valve position adj. Press ratio reg.
Burnerpress sensor position adj.
Coarse filter Fuel inlet from pump Filter relief valve Press-reg valve adj. Press-reg servo sensor Flapper valve Fuel-temp compensator
Thrustreversing actuator Shut off lever
Atmos Reverse signal Idle trim Droop adj. Min. ratio adj. Droop cam Dec. inc Burnerpressure output Mult. lever
Thrust reverser inlet signal
Forward signal Max trim Speed- Speedset lever set cam Speed cam follower Droop lever Max ratio adj. Sequencing valve cam 90 stop Speed servo pos. adj. Speedsensing servo Burner press. Atmospheric press. Overboard drain leakage Comp inlet press. Increasing R.P.M. Sequencing valve Accel. limiting cam
Atmos
Reverse thrust stop

Speed servo stop
Shut off stop
Fail-safe orifice Fixed orifice Press-reg valve Bypass to pump interstage
15
Feedback lever stop Water drain
Power lever stop
Feed-back lever
Forward thrust stop
Shut off lever stop Detent roller
Speed-servo rate adj.
Power lever
Pump discharge press.
Throttle-valve metered press.
Speed-sensing governor
Servo press.
Bypass to pump interstage
Return to engine pump inlet (boost press.)
Overboard drain
Regulated press. N2 drive
Figure 1. Hydromechanical controllerHamilton Standard JFC25 (4).
16
Figure 2. Pratt & Whitney PW4084 turbofan engine.
reduction unit. The rotor or propeller further increases total engine air ow, decreases specic thrust, and increases propulsion efciency. The turboshaft is the best type of powerplant for helicopters and small, lower speed aircraft. Variations of the above basic engine types can include: Dual rotor core engines containing two compressors and two turbines Turbofans with a booster compressor between the fan and core compressor for supercharging the core Mixed-ow turbofans which mix the bypass and core discharge ows and exhaust both through a single nozzle Turbojet augmentors, turbofan fan burners, and mixed ow augmentors for increasing engine thrust for better takeoff, transonic acceleration, or combat capabilities SIMPLIFIED ENGINE THEORY The most common form of jet engine is the high bypass ratio turbofan. It will be described in this article. More detailed discussion on engine design and operation can be found in (5,6). A turbofan engine bypasses a substantial fraction of the inlet air around the hot section or core of the engine, in order to achieve a high propulsive efciency. A simplied diagram of such an engine is shown in Fig. 3.
Engine inlet HPC inlet
The numbers in Fig. 3 refer to standardized (5,7) engine station locations: 0 1 2 25 3 4 45 5 8 9 Freestream ambient air conditions Inlet entry Fan entry High-pressure compressor entry High-pressure compressor exit Burner exit/high-pressure turbine entry High-pressure turbine exit/low-pressure turbine entry Turbine exit Nozzle throat Exhaust nozzle exit
Double-digit numbers 1218 are used for bypass ow stations from fan tip entry (station 12) through the bypass duct to the bypass nozzle (station 18). Reciprocating automobile engines operate on a four-stroke Otto cycle. Their internal combustion process achieves extremely high pressures through constant volume combustion, which results in a high power per kilogram of air ow.
HPC discharge LPT discharge Afterburner inlet Exhaust Exhaust nozzle nozzle inlet discharge Exhaust nozzle throat
Combustor discharge
2.5
Figure 3. Engine station locations.
17
Conversely, the jet engine operates on a continuous ow Brayton cycle, which ideally involves isentropic compression and constant pressure combustion. It operates at a substantially lower maximum temperature and pressure than the Otto cycle. Figure 4 shows pressure-volume and temperatureentropy (TS) diagrams for the ideal Brayton cycle. It is the basis of the turbojet engine and the core of the turbofan. It contains the following thermodynamic processes: 02 23 34 45 59 Isentropic compression in the engine inlet Isentropic compression in the engine fan, booster, and compressor Constant pressure heat release in the combustor Isentropic expansion in the high-pressure and lowpressure turbines Isentropic expansion to atmospheric pressure in the exhaust nozzle
Compression System The turbofan compression system consists of a fan, a booster, and one or more compressors. Operating characteristics of each are a function of their inlet total temperature and pressure, rotational speed, and discharge pressure. It is convenient to normalize ow and speed with respect to inlet temperatures and pressures. The following corrected parameters can be used to represent operating characteristics independent of the actual inlet temperature and pressure levels: Corrected ow (Wa / a) Corrected speed (N / a) Pressure ratio (Pb /Pa) Adiabatic efciency (c)
An isentropic process means that there is no temperature raise and the process is reversible. Hence entropy is constant. The actual cycle involves near isentropic compression and expansion processes, a pressure loss in the combustor, and an incomplete expansion to near-atmospheric pressure in the exhaust system. Total temperatures and pressures, which include the effect of the air velocity, are used for all internal engine conditions. Alternative options, which include the turbojet, turboshaft, and turbo-augmented cycles, will not be discussed, but can be determined from similar techniques. The following sections describe each of the turbofan engine processes in more detail. Inlet Compression Air ow is supplied to the engine by the inlet which compresses the inlet air. Assuming that the ambient air pressure, P0, and temperature, T0, is not moving and the inlet is moving at the ight mach number, M, the total pressure and temperature at the inlet is: P2 = r P0 1 + T2 = T0 1 + 1 M2 2 1 M2 2
1 /k
where W is the ow rate, is the inlet temperature divided by the ambient standard sea level temperature, is the inlet pressure divided by the ambient standard sea level pressure, N is the compressor rotational speed, and c is the compressor adiabatic efciency. The subscripts a and b refer to inlet and discharge conditions, respectively, for the fan, booster or compressor. Operating characteristics of a typical compressor are shown in terms of the above corrected parameters in the compressor map in Fig. 5. Given measured corrected speed and pressure ratio, one can determine corrected ow and efciency (not shown in Fig. 5). Exit temperature and required work can then be obtained from:
Tb = Ta 1 +
(Pb /Pa )k 1 c
(3)
(4)
HP = Wa c p (Tb Ta )
(1) (2)
where P is pressure, T is temperature, M is ight Mach number, is the specic heat ratio of air (constant pressure specic heat/constant volume specic heat), k is the ratio ( 1)/ , and r is the ram recovery (actual total pressure/ideal total pressure) which is approximately 1 for subsonic ight.
where HP is the work required to drive the compressor and cp is specic heat of air at constant pressure. This work is expended heating the air from Ta to Tb. Compressor characteristics must be obtained from extensive testing of individual stages, the full compressor, and sometimes the entire engine. Stable compressor operation is limited to the region below the compressor stall line shown in Fig. 5. Two modes of instability can occur: surge, which is a longitudinal ow oscillation over the length of the compressor and turbine, and stall, which is the lack of pressure rise between the compressor blades. Often stall occurs at low rotor speeds and surge at high rotor speeds. Both surge and stall generate violent axial oscillations of the internal air column, which can cause sub-
Combustion 3
Pressure
4 3
Pressure
Combustion Turbine
4 Turbine 5 Nozzle 9
Compressor 2 Inlet 0
Compressor 2 Inlet 0
5 Nozzle 9
Volume
Entropy
Figure 4. Pressure-volume and temperature-entropy (TS) diagrams for turbojet.
18

High
Surge line @ Sea level @ 35,000 ft @ 50,000 ft Surge zone
Pressure ratio
Acceleration line P t3 P t2 Deceleration line
N/ T2 High
Low Low
Steady-state operating line @ Sea level @ 35,000 ft @ 50,000 ft Wa T2
Zone of poor compressor efficiency
Corrected shaft speed
N/ T2 Low
High
Figure 5. Engine compressor map.
T2
stantial damage to both the compressor and the engine. Fan stalls can be caused by operation with too small a fan duct nozzle area, booster stalls by a throttle reduction to a lower engine rotational speed, and compressor stalls by a rapid throttle increase. The engine and control system must be designed to avoid surge/stall, with sufcient design margin to offset the effects of altitude, increased tip clearances, component deterioration, and engine/airow operation at high angles of attack. Combustion System Fuel is burned in the combustor at a slight pressure drop, and the resulting products of combustion are expanded in the turbines. The required fuel ow can be obtained from: Wf = W4 c p (T4 T3 ) Q f f cpf T4 (5)
Turbine Expansion The turbine expansion system provides the power to drive the compression system. The high-pressure turbine drives the compressor through the high-pressure shaft, and the lowpressure turbine drives the fan and booster through the lowpressure shaft. The operating characteristics of each turbine are dened in terms of the following corrected parameters: Corrected ow function (WgaTa /Pa) Corrected speed (Na / Ta) Temperature ratio [(Ta Tb)/ Ta] Adiabatic efciency (t)
2000
T4 T3; Combustion total temperature rise (F)
1800 1600 1400
Effect of varying T3
where Wf is the fuel ow rate, Qf is the higher heating value of the fuel, and f is the combustion efciency. The subscript 4 refers to high-pressure turbine inlet conditions. Depending on the combustor pressure, the combustor will operate only in certain regions, as shown in Fig. 6. The high-temperature blowout region is referred to as rich blowout and the low temperature region is referred to as lean blowout. The total gas ow downstream of the combustor is the sum of the air ow and the fuel ow. The specic heat of the gas mixture can be obtained from the equation: c pg = c pa + f cpf 1+ f (6)
1000 800 600 400 200
Bl
ow
ou
1200
e tr
gio
T3
Operating area
Other mixture properties, such as enthalpy and entropy but not can be obtained by a similar process. A signicant amount of air is required to cool the combustor liner, but it is returned to the gas stream prior to the turbine.
Blowout region 0 0 10 20 30 40 50 60 70 Combustion total pressure inches (HG ABS.) 80
Figure 6. Combustor blowout limits.
19
The subscripts a and b refer to inlet and discharge conditions, respectively, for the high-pressure and low-pressure turbines. Operating characteristics of a typical turbine are illustrated, in terms of the above corrected parameters, in the turbine map shown in Fig. 7. Given measured corrected speed and temperature ratio, one can determine corrected ow and efciency. Note that the turbine inlet ow is choked at a constant value over a large part of the operating range. The temperature ratio across the turbine can be obtained from the required work of the compressor (for the high-pressure turbine) or the work of the fan and booster (for the lowpressure turbine): Ta Tb Ta Bypass Duct A fraction of the fan discharge air is bypassed around the engine core, and exhausted through either a separate bypass nozzle, or mixed with the core stream and exhausted through the core nozzle. In either case, the bypassed air improves the propulsive efciency of the engine, and makes it the preferred approach for the worlds large commercial eet. The bypass ratio is dened as: Bypass ratio = Total fan inlet air ow Core inlet air ow (8) = HP Wga c pg Ta (7)
nozzle can be used for exhaust ow, which is either subsonic or sonic, and a converging-divering (C-D) nozzle would be required for supersonic ow. The throat or minimum area of the nozzle will regulate the amount of ow that can be exhausted through the nozzle. The required throat area for the core stream can be obtained from:
A8 =
Wg8 T8 8 N8 P8
(9)
where 8 is the sonic ow function and N8 is the ratio of ow to sonic ow: 8 = g/R (10)
3
N8 = M8
1.2 2 1 + .2M 8
(11)
The throat Mach number must be 1 if the nozzle total-tostatic pressure ratio is greater than the critical value of about 1.8. If the pressure ratio is less than critical, the throat Mach number will be less than 1, and can be obtained from the relationship: M= 2 [(PT /PO )k 1] 1 (12)
It represents a major turbofan engine design parameter. The bypass duct operates with a slight pressure drop of about 5 percent of the fan discharge pressure and no loss in total temperature. Exhaust System The core and bypass streams are expanded through core and bypass nozzles to the pressure of ambient air. A converging
The exit area will effect the thrust output of the engine. Similar equations can be used for the bypass or mixed ow streams. Augmentor Military engines generally have an augmentor, either behind the low-pressure turbine or in the bypass duct. Augmentors are sometimes referred to as afterburners. They are used to increase engine thrust for selected segments of the ight, such as takeoff, climb, acceleration to supersonic speed, or combat. Augmentation is a relatively inefcient approach for generating thrust. This penalty can be minimized by maintaining the engine at its maximum nonaugmented setting, thereby minimizing the thrust increment provided by the augmentor. Augmentation requires a variable exhaust nozzle. The reason can be seen from Eq. (9). For a xed A8, an increase in T8 must be offset by an increase in P8. Since combustion is essentially a constant-pressure process, the increase in P8 results in an increase in turbine pressure, P5 and, hence, an increase in P3, moving the engine operating line closer to stall. From Eq. (7), the increase in P5 also produces less work from the turbine, which will reduce core rotor speed. The control system will increase main fuel ow, to keep rotor speed constant, resulting in increased temperature of the turbine. Thus A8 must be opened to maintain a constant P8, to avoid compressor stall and overtemperature of the turbine. Engine Trade-Offs Engine operating characteristics are set predominantly by four interacting design variables: (1) bypass ratio, (2) turbine inlet temperature, (3) overall pressure ratio, and (4) fan pressure ratio. The best design choice will be dependent on the
.28 .26
Nozzles choke
5.0
4.0 .24 .85 .22 .20 T4 T5 T4 .18 .16 .14 .12 110 .10 60 70 80 90 100 N/ T4 110 120 P4/P5 = 2.0 130 140
T = .84
122 120 W P4 T4
3.0
= 115
Figure 7. Turbine map.
20
intended application or mission, the level of technology available, the degree of subsequent growth capability required, and expected competition from other engines. Bypass ratio will have the most dominant effect on engine performance. High bypass ratios of 4 to 8 are used for most large commercial engines. Increased bypass ratio will improve (decrease) specic fuel consumption (SFC) at cruise, and improve specic thrust (thrust per kilogram/second of air ow) at takeoff. Ultra-high bypass ratios of 10 to 20 have been considered for improved cruise performance, but would require an unducted fan (8) or reduction gears between the fan and low-pressure turbine. Lower bypass ratios of 1 to 3 provide improved thrust for ight Mach numbers of 1 to 2, and are used for military ghters and bombers. A pure turbojet has a zero bypass ratio, and would be used for supersonic transport. High turbine inlet temperature leads to improved specic thrust and a lighter engine, but requires more expensive turbine materials and a more complex turbine cooling system, which reduces cruise performance. The proper balance will depend on the relative importance of specic thrust, which sets engine size, and weight and cruise performance, which sets fuel requirements. Military applications will tend to demand higher temperatures, to achieve a lighter engine weight, while commercial applications will place a stronger emphasis on cruise performance. Overall pressure ratio, which is the compressor discharge pressure divided by fan inlet pressure, will affect both takeoff and cruise performance. Optimum pressure ratio tends to increase will increased turbine inlet temperature, but decreases as ight Mach number increases. Pressure ratios of 40 : 1 and 50 : 1 could be effective with modern temperatures at takeoff, but should be limited to the 10 : 1 to 15 : 1 range at supersonic speeds. Extremely high-pressure ratios would require the use of high alloy steel or titanium at the rear of the compressor, and cooled-cooling air for use in the high-pressure turbine. High fan pressure ratio improves takeoff performance, but increases exhaust noise. The achievable pressure ratio will be dependent on the work available from the low-pressure turbine. Both will be dependent on turbine inlet temperature and overall pressure ratio. Fan pressure ratios of 2 to 3 can be achieved on low bypass military engines, but would be limited to the 1.6 to 2.0 regime for high bypass commercial engines. Mixed-ow turbofans would also require the use of fan pressure ratios that produce duct pressure levels roughly equal to the turbine discharge pressure, in order for mixing to occur. CONTROL REQUIREMENTS The overall function of an engine controller is to provide thrust in response to throttle position. It must achieve the requested thrust with the lowest specic fuel consumption. It must also insure that the following limits are not exceeded: Maximum fan speed Maximum compressor speed Maximum turbine temperature Fan stall Compressor stall Maximum compressor discharge pressure
Minimum compressor discharge pressure Lean burner blowout Rich burner blowout For an aircraft turbine engine, it is necessary to achieve maximum thrust with minimum engine weight. This means that all components operate at mechanical or thermal limits for at least one of the engines critical operating conditions. At other operating conditions, operation at only one or more of the above limits may be required. Figure 8 shows, various typical limits as function of mach number and altitude. The control must directly or indirectly control each limiting parameter and limit engine thrust, so that no limits are exceeded. Engine operation at maximum power will, consequently, require operation at one or more of the engine operating limits. Part power engine operation should occur below all limits and at the lowest specic fuel consumption for the thrust requested. SENSORS AND ACTUATORS The pilot controls aircraft speed by setting the throttle position to a thrust which will permit operation at the desired speed. One would like to run the engine directly to the required thrust, by modulating engine fuel ow until the required thrust has been achieved. Similarly, one would like to operate at the fan and compressor stall limits and the turbine inlet temperature limits. However, none of these parameters can be measured directly in ight. Thrust can only be measured in a test cell, where one can establish a stationary reference. Likewise, stall margins are established in test rigs, by actually stalling the fan or compressor. Thus one must select practical measurements that are related to the ideal measurements. These practical measurements must also be relatively immune to engine-to-engine variation due to manufacturing tolerances and engine deterioration. Generally, Monte Carlo
Various structural and overspeed limitations
Constant T2 limit T4 and T7 limits separate hard augmentor light-off
Primary and augentor A ignition and blow out l t i Low t pressure u limit d e
Minimum burner pressure at low power
Maximum turbine inlet temperature
High pressure limit
Blade loading and burner case limits Mach number Figure 8. Engine limits (19).
21
simulations are performed to establish the appropriate selection. Various thrust setting parameters can be used to set thrust indirectly. GE uses fan rotor speed, Pratt and Whitney uses core engine pressure ratio (EPR), and Rolls Royce uses an integrated engine pressure ratio (IEPR, which is a owweighted average of the core and bypass duct pressure ratios). Commonly selected basic sensors are: N1 N2 EPR W f / P3 T45 Fan speed Core speed Engine pressure ratio P5 /P2 Fuel ow divided by compressor exit pressure Low pressure turbine inlet temperature
Three of these are required for the direct determination of the critical overspeed and overtemperature limits. Because of the need to shield the thermocouples, the T45 measurement has a slow response time, and is not particularly good for control purposes. Either EPR or Wf / P3 is used to control the engine. Wf / P3 is very commonly used and is a natural control parameter. To see why, note that the ow through the turbine is: c c W4 = P4 = P3 T4 T4 (13)
since there is very little pressure drop across the burner. Therefore Wf P3 = Wf W4 c T4 (14)
Since the fuel air ratio, Wf /W4, is directly related to ame temperature and, hence, turbine temperature T4, so is Wf / P3. Thus Wf / P3 provides good control of T4, good reaction to stall, and good reaction to burner blowout. An added advantage is that Wf / P3 is relatively easy to sense. Commercial engines will generally have variable compressor stator vanes, a fuel control valve, and a variable booster bleed valve (if there is a booster). The variable stators are for improving low-speed stall margin and high-speed performance in the core compressor. The booster bleed valve is used to bleed booster air into the bypass duct, to improve booster stall margin at part power and during throttle reductions. Both are generally controlled open-loop. The only closed-loop control in a commercial engine is the main fuel control. Military engines will generally add an afterburner for increasing thrust during takeoff, transonic acceleration, and combat. The afterburner will require both an augmentor fuel control and a variable exhaust nozzle throat area (A8), to permit a wider range of augmented thrust and to manage interactions with the main fuel control. ENGINE CONTROLS Proportional and Integral Control The simplest and most common type of control is based on a proportional control. With minor variations, it is the basis of
all hydromechanical controls, and has been used for some of the earliest electronic controls as well. More recent engines with electronic controls use proportional plus integral control, to minimize errors. The control can be subdivided into a steady-state control and a control for transient operation. The steady-state control, which maintains engine operation along its steady-state operating line, will be discussed rst. The transient controls necessary to accelerate and decelerate the engine while meeting stall, ameout, and temperature limitations, will then be added. A block diagram of the single-input, single-output proportional plus integral control, which maintains the engine at a desired operating point, is shown in Fig. 9. A fan speed demand schedule establishes the desired fan speed as a function of inlet temperature and throttle angle. Fan speed error is determined from the difference between the demand speed and the actual fan speed. A delta Wf / P3 signal is obtained proportional to the fan speed error. This signal is multiplied by the P3 sensor signal and the resulting change in fuel ow is used to drive the fuel metering valve. The gain is scheduled as a function of core speed, to meet the control requirements over the entire ight envelop. In an electronic control, the desired fuel ow is computed directly, rather than as a ratio, with P3. The control will provide the control response for small increases or decreases in throttle angle. Controllers based on EPR or IEPR substitute pressure ratio demand schedules and error signals for the corresponding fan speed parameters. With proportional control there is always an error or droop between the desired speed and the actual speed. As the load on the engine increases, this error will increase. The loop gain is normally made high, to minimize this droop. The use of integral control eliminates this problem. The throttle schedule is designed to provide thrust as a linear function of the throttle angle. It also maintains the engine at the most efcient operating point possible. Engine thrust can be shown to be a nonlinear function of engine speed. The throttle schedule inverts this nonlinear relationship, to provide the desired speed as a function of throttle angle. Transient operation is illustrated in Fig. 10. It shows Wf / P3 as a function of core speed for a variety of engine operating modes during engine accelerations and decelerations. An acceleration will occur when the throttle is advanced to a demand speed higher than the current speed. A min-select strategy will then lead to the following series of processes: The positive speed error will result in a proportional Wf / P3 error, which will open the fuel valve, admitting more fuel to the engine and result in an acceleration from point 1 along the speed governor line to point 2, where it will intersect the maximum fuel ratio line. Min-select strategy will switch operation to the maximum ratio line, and it will continue to accelerate to point 3, where it will intersect the compressor surge limit line. Min-select strategy will switch operation again, this time to the surge limit line, where it will continue to accelerate at a slower pace to point 4. At point 4, the min-select strategy will switch operation back to the maximum fuel ratio line, and the engine will continue to accelerate to point 5.
22
JET ENGINE CONTROL, IMPLEMENTATIONS Scheduled fuel Throttle T2 Operating schedule Desired fan speed Gain P3 Engine
Figure 9. Block diagram of proportional controller.
Fan speed
At this point, operation will be switched back to the speed governor line, leading to the steady-state operating point at the demanded speed at point 6. This nal process will reduce Wf / P3, until the engine stabilizes at point 6. An engine deceleration will occur when the throttle is retarded to a speed demand lower than the current speed. A max-select strategy during the deceleration will then lead to the following: The negative speed error will lead to a proportional Wf / P3 error, which closes the fuel valve, reduce fuel ow, and cause a deceleration along the speed governor line to point 7. Max-select strategy will switch operation at point 7 to the minimum fuel ratio line to point 8. Max-select strategy will switch operation again at point 8 to the speed governor line, which will lead back to steady-state operation at point 1. Fuel ow will be increased until the engine stabilizes at the demanded speed at point 1. Note that each of the above lines in Fig. 10 are generally functions of inlet pressure (P2) and possibly temperature (T2) as well. To make the engine accelerate or decelerate rapidly
and remain as efcient as possible, the control is designed to keep the engine close to or on each limit. The above processes will occur during large increases or decreases in throttle/speed demand. During small throttle increases, the minimum select process will transition engine operation from the 19 speed governor line directly to the 910 speed governor line. This will lead to stable operation at point 10. A small deceleration command at point 6 will result in a similar transition at point 11 to stable operation at point 12. Other limits can be handled in a similar fashion. The deceleration limit provides sufcient fuel to maintain combustion. Except for high ows, it can be independent of speed. An idle speed controller is provided, to maintain minimum engine speed. It is a proportional controller similar to Fig. 9, with no additional schedule and the reference speed is the desired idle speed. Overtemperature and overspeed are also handled as proportional controls, with the maximum temperature or speed as the reference. The overtemperature controller senses T45. A block diagram of the complete control is shown in Fig. 11. Ndot Control A variant of proportional control uses the derivative of rotor speed (9,10) (Ndot or rotor acceleration), rather than rotor speed, to control engine acceleration and deceleration. Direct
Surge area 6 Max. ratio line Rich blowout 5 (8-9) area (5-6) Speed (78) (2-3) 3D cam governor 3D cam 3D (4-5) cam Speed (6-7) (9-10) governor Governor 3D cam droop lines Speed 3 (infinite number) governor 2 Steady state lines (3-4) 4 12 Speed governor Speed reset (11-12) (10-11) 11 Min. flow stop line 10 (1-2) Min. ratio line Speed 1 governor 0 Idle N Percent rotor speed Nmax
14.7 psia [101.36 kPa] 6 psia [41.37 kPa] 2 psia [13.79 [kPa] 7 8
2 psia [13.79 kPa]
14.7 psia 9 [101.36 kPa]
Wf /Ps3
100%
Figure 10. Characteristic schedules.
controller

Fan speed T2 Core speed T2 Core speed T2 Core speed T2 Core speed T2 Acceleration controller Min Max Min Max and min fuel limits Max Engine Fan speed controller PS3
23
Deceleration controller
Idle controller Operating schedule controller
Figure 11. Block diagram of complete controller.
control of acceleration, rather than speed, allows tighter control of engine acceleration, thereby improving transient response and reducing mechanical stress. While rotor acceleration cannot be easily measured directly, a second-order lter applied to speed can be used to give a good approximation. The algorithm shown in Fig. 12 replaces that shown in Fig. 9. The previous acceleration schedule is replaced with one that directly species allowable engine acceleration. A lead lag compensator is necessary to improve transient response. The Ndot control will result in a more consistent acceleration between a cold engine that has not been run for at least 30 minutes, and a warm engine that is being reaccelerated in a shorter time. Fan Speed Control Throttle position is converted to a specic fan speed requirement as a function of engine inlet temperature, T2, and inlet pressure, P2. Fan speed is held to the desired speed by replacing the proportional control with an isochronous integral control. The bandwidth of the loop will determine the speed of response and, hence, how tightly the fan speed is maintained. The core speed oats, within limits, at a speed necessary to provide the sufcient power to maintain fan speed. The agility of a helicopter (11) depends on maintaining rotor speed and, hence, power turbine speed during maneuvers such as a waveoff from an autorotational descent. The load on the rotor is changed by the pilot, changing either the col-
lective or cyclic settings. System responsiveness is determined by the dynamics of the power turbine isochronous governor. Unfortunately, the helicopter rotor rst torsional resonance mode limits the bandwidth of the control. Notch lters centered at the rotor resonance are used to allow high crossover and, therefore, a more responsive system. A further renement of fan speed control is called Power Management Control. This control integrates aircraft and engine requirements, to compute the necessary thrust levels to maintain uniform aircraft speed. The controller uses information on the specic aircraft conguration, inlet pressure, temperature, Mach number, and engine bleeds, to calculate the desired thrust and a reference fan speed or pressure ratio. In order to compensate for engine-to-engine quality variation and deterioration, the pilot then moves the throttle to match the reference fan speed or pressure ratio. Aircraft speed is maintained, during variations in inlet pressure and temperature, by closed-loop adjustment of these reference values. Multivariable Control As indicated previously, commercial engines are generally controlled by a single, closed-loop main fuel control. The additional actuators have minimal interactions between each of the open-loop controls and with the main fuel control. Consequently, single-input, single-output design techniques are adequate.
Fan speed controller Wf /P3 demand Integrator T2 Ndot schedule Compensator Min select
P3
Engine
2nd order rate filter
Compressor speed
Figure 12. N-dot controller.
24
This is, however, not the case for military engines, which generally have an afterburner augmentor, to increase engine thrust for selected segments of the ight, such as takeoff, climb, acceleration to supersonic speed, or combat. The augmentor fuel ow will determine augmentor temperature, T6, which affects engine pressure ratio and turbine temperature. However, the primary combustor fuel control also controls these variables. Thus the augmentor fuel control loop and the combustor fuel control strongly interact. Early design approaches used the concept of spectral separation, which made the combustor fuel control loop an order of magnitude faster than the augmentor control loop. More recent designs have used a multivariable approach, to achieve two control loops of approximately the same response time. Another application of multivariable control is on a variable cycle engine (VCE), which has a variable area bypass injector, or VABI. The VABI allows the bypass ratio to be changed during ight. When closed, the engine behaves more like a turbojet, providing more thrust during supersonic ight. Opening the VABI makes the engine more like a turbofan, improving specic fuel consumption during cruise. Much of the published literature on multivariable engine control is focused on the regulator problem of maintaining the engine near the desired operating trajectory. It is based on linear models valid for small signal analysis, and avoids the important problem of limit protection. The early 19741982 applications are summarized by Zeller, Lehtinen, and Merrill (11). The earliest work was that of McMorran and MacFarlane (12,13), which was tested on a two-spool afterburning engine. They used fuel ow to control compressor speed and A8 to control fan speed due to the engine bypass duct. The International Forum on Alternatives for Linear Multivariable Control (14), held in 1977, is of particular interest, because it showed the use of several multivariable techniques using a linear model of the F100 engine as a theme problem. Four authors developed control strategies based on three different multivariable techniques: multivariate transfer functions (15,16), Inverse Nyquist Array (17), and Characteristic Root Locus (18). Each strategy used three or four of the measured variables, fan speed, N1, compressor speed, N2, compressor exit speed, P3, exhaust pressure, P7, and turbine inlet temperature, FTIT and controlled fuel ow, Wf , exhaust nozzle areas, A8, and compressor guide vanes, CIVV, or fan guide vanes, RCVV. In all cases, decoupled control of each of the measured variables was achieved. One of the most complete early multivariable designs extensively tested was that by De Hoff and Hall (1922) for the F100 engine, using extended linear quadratic regulator techniques. This work went beyond the previous studies, with ability to handle large power excursions without exceeding engine or actuator limits and to operate over the entire engine operating envelope. A block diagram of the control system is shown in Fig. 13. The feedback law itself is an optimal regulator structure, with integral trims for steady-state accuracy and engine limit tracking. Linear controllers were designed at ve operating points: sea level static, subsonic and supersonic. Dominant gain elements were determined by assessing the closed-loop eigenvalue sensitivity to each gain element. Over 50 percent of the controller gains were eliminated in the nal implementation, with little or no effect on system performance. Important gain elements were tted with univariate functions of fan inlet pressure and temperature and
core speed. Unique to this control design is the transition trajectory generator, whose purpose is to smooth rapid throttle changes by providing a piecewise linear transition from the current engine state to the requested state. The rate of transition depended on whether the engine is at low, medium, or high power. Takeoff thrust for the F100 is dened as the thrust obtained at maximum allowable turbine temperature. At altitude conditions, the minimum burner pressure denes engine idle. Thus various physical limits must be held exactly at various ight conditions, as shown in Fig. 10. In theory, an optimal controller could have been designed for each limit at appropriate operating conditions. This would have required an exponentially large number of gain matrices to cover all combinations. De Hoff and Hall (19) used an ad hoc approach to solve this problem, by designing single-loop spectrally separated integral trims for each input, corresponding to each desired set point and number of unsaturated controls. The control was switched whenever an actuator saturated or an engine limit was reached. A very similar multivariable design was developed for a GE23 variable cycle engine (23,24). A much larger number of linear design points were needed, due to the greater changes in engine conguration. Instead of using integral trim to handle engine limits, a model of the engine was incorporated into the transition logic. This allowed the generated trajectories always to satisfy engine limits. The problem with this approach is that it is an open-loop feedforward approach. How well the engine limits are held depends on the accuracy of the model and how well it can handle engine-to-engine variations. One possible solution to handling engine limits is demonstrated by Adibhatla (25,26) for a short takeoff and vertical landing airplane (STOVL) using an F110 engine. This is a full-range multivariable design, which was pilot evaluated in the NASA Ames xed-base and vertical-motion simulators. The primary objective of this engine control is to manage thrust, through the aft nozzle during cruise, and through ejectors at the aircraft wing roots and ventral ports on the underside of the aircraft during transition and hover. An estimate of thrust is determined based on fan speed, fan operating line, fuel ow, and the three thrust port areas. During cruise a 2 2 controller regulates fan speed and aft thrust, using fuel ow and aft nozzle area. During transition and hover, the controller is expanded to a 4 4 controller, in order to additionally regulate the ejector and ventral thrust using the ejector and ventral area actuators. To prevent engine stall and overtemperature, and to ensure sufcient pressure for customer bleed, three single-input, single-output regulators were designed as limit regulators. Hanus technique (27) for integrator antiwindup is used to transition between these ve regulators and to handle actuator limits. In this technique, an observer-based structure is used, such that the normal controller is modied to assume the form: x c = Ac xc + Bc (ys p y ) + L(sat(u ) u ) u = Cc xc + Dc (ys p y ) (15)
where Ac Bc Cc Dc the state space description of the ve controller dynamics with the desired set point ysp and the engine output y, sat(u) is the actual bounded engine input and L is the anti-windup gain. When a given regulator is controlling
25
Nominal input + + CP Integral control CI Gain schedules Engine limit protection Actuators Engine
Reference point schedules
Transition control
+ LQR State
Limit flags CP Sensors and FITT estimator
Figure 13. F100 multivariable controller (19).
the engine actuators, its elements of sat(u) u are zero. The gain L forces the remaining sat(u) u elements to zero, thereby tracking the ones currently applied to the engine. This ensures a smooth transition from one regulator to another, and automatically handles the problem of engine limits. Recently, Kapoor and Teel (28) have extended Hanus technique by replacing L(sat(u) u) with an equivalent dynamic system. This allows more design freedom and guarantees stability. An example is shown where their dynamic scheme gives response very close to the unconstrained case, while using a static L is unstable. An excellent summary of multivariable control development in the United Kingdom from 1985 to 1995 is given by Dadd, Sutton, and Greig (29). A variety of control laws have been tested on either the RB199 or Sprey Mk202 engines. Each was distinguished primarily by the combination of sensor used, since there was very limited exibility in terms of available actuators. Again the Hanus technique (27) was used to handle limit protection and controller selection. Of specic interest is a fast response test on the Sprey Mk202 engine, using a 3 3 controller for most of the operating regime, and a 2 2 for low power settings where the high pressure inlet vanes are fully closed. The input output parameters chosen for the 3 3 controller were: Inputs Fuel ow Inlet guide vanes A8 nozzle areas Outputs Fan speed Compressor speed Bypass duct Mach number
duction in time to go from 50 percent to 100 percent thrust. It was found that the specic fuel consumption was not affected signicantly, while the thermal cycle range of the turbines was reduced. A second set of tests was performed, with increased integral gains to shift the closed-loop poles. This improved the robustness and disturbance rejection of the control. CONCLUDING REMARKS This article has focused on the basic theory of engine operation and on control designs currently in commercial and military use. Single-input, single-output controls are used on commercial engines where the emphasis in on economical operation. Multivariable controls provide enhanced performance for military engines. In both cases, the management of limits is an essential part of the control. Due to limited space, it has not been possible to discuss the many ongoing advanced research areas in engine controls. Of immediate interest to the authors is the work (30,31) using a real-time model of the engine as an observer to provide estimates of unmeasurable variables, such as thrust and stall margins. Of much longer-term potential is the work on active control of stall (3234) and combustion (3537). BIBLIOGRAPHY
1. G. G. Smith, Gas Turbines and Jet Propulsion, New York: Philosophical Library, 1955. 2. J. V. Casamassa and R. D. Bent, Jet Aircraft Power Systems, 3rd ed., New York: McGraw-Hill, 1965. 3. E. W. Constant II, The Origins of the Turbojet Revolution, Baltimore, MD: John Hopkins University Press, 1980. 4. I. E. Treager, Aircraft Gas Turbine Engine Technology, 2nd ed., New York: McGraw-Hill, 1979. 5. G. C. Oates, Aerothermodynamics of Gas Turbine and Rocket Propulsion, Revised and Enlarged, Washington, DC: American Institute of Aeronautics and Astronautics, 1988.
The 2 2 controller used fuel ow and nozzle area to control fan speed and bypass duct Mach number. Both the 2 2 and the 3 3 controllers were run simultaneously, with authority switched between them as the guide vanes either approached or moved off their limit. During sea level static tests, it was found that this fast response multivariable controller produced a 60 percent re-
26
JET TRANSPORT MAINTENANCE 28. N. Kapoor and A. Teel, A Dynamic Windup Compensation Scheme Applied to a Turbofan Engine, Proc. 36th IEEE Conf. on Decision and Control, San Diego, CA, 46894694, 1997. 29. G. J. Dadd, A. E. Sutton, and A. W. M. Greig, Multivariable control of military engines, Proc. Advanced Aero-Engine Concepts and Controls, Defense Research Agency, Farnborough, England, June 1996. (AGARD Presentation, Seattle WA, Sept. 1995.) 30. W. J. Dwyer, Adaptive Model-Based Control Applied to a Turbofan Aircraft Engine, M.S. Thesis, Department of Mechanical Eng., MIT, Cambridge, MA, June 1990. 31. S. Adibhatla and T. Lewis, Model-Based Intelligent Digital Engine Control, AIAA 97-3192, 33rd AIAA Joint Propulsion Conf. & Exhibit, Seattle, WA, July, 1997. 32. A. H. Epstein, J. E. Ffowcs-Williams, and E. M. Greitzer, Active suppression of aerodynamic instabilities in turbomachines, J. Propulsion and Power, 5 (2): 204211, 1989. 33. I. J. Day, Active Suppression of Rotating Stall and Surge in Axial Compressors, International Gas Turbine and Aeroengine Congress and Exposition, ASME Paper 91-GT-87, June, 1991. 34. K. M. Eveker et al., Integrated control of rotating stall and surge in aeroengines, Proc. Sensing, Actuation, and Control in Aeropropulsion Conf., Orlando, FL, Apr. 1995. (SPIE Proceedings, Vol. 2494, 1995, pp. 2135.) 35. E. J. Gutmark et al., Active control in combustion systems with vortices, Proc. 4th IEEE Conf. On Control Applications, Albany, NY, Sept. 1995, pp. 679684. 36. D. St. John and S. Samuelson, Optimal, active control of oxides of nitrogen (Nox) emissions from a natural gas-red burner using a simple genetic algorithm., Proc. 4th IEEE Conf. on Control Applications, Albany, NY, Sept. 1995, pp. 673678. 37. E. J. Gutmark et al., Compact waste incinerator based on vortex combustion, Combustion Science and Tech., 121: 333349, 1996.
6. H. Cohen, G. F. C. Rogers, and H. I. H. Saravanamuttoo, Gas Turbine Theory, 4th ed., Essex U.K.: Longman, 1996. 7. Aerospace Recommended Practise, ARP 755A, Society of Automotive Engineers, 1974. 8. K. Mordoff, General Electric ies rst unducted fan demonstrator, Aviation Week & Space Technology, 125: 31, Aug. 25, 1986. 9. D. A. Parsons, N-dot schedules dynamic compensation system for gas turbinesInputs sum of speed and rate of change of speed of gas generator to schedule to output desired acceleration as function of generator speed, U.S. Patent 5029441, 1991. 10. J. Zeller, B. Lehtinen, and W. Merrill, The Role of Modern Control Theory in the Design of Controls for Aircraft Turbine Engines, NASA Tech., Memo. 82815, Lewis Research Center, Cleveland, OH, 1982. (Also in AIAA Twentieth Aerospace Sciences Conf., Orlando, FL, Jan. 1114, 1982.) 11. D. R. Gilmore, Jr., Two decades of T700 control system growth, Proc. of the 4th CASI Symposium on Propulsion, May 5, 1992, pp. 1525. 12. P. D. McMorran, Design of gas-turbine controller using inverse Nyquist method, Proc. of IEE , 117, 1970. 13. A. G. J. MacFarland et al., Application of multivariable control techniques to aircraft gas turbines, Conf. on Multivariable Control Systems Design and Application, Sept. 13, 1971. 14. M. K. Sain, J. L. Peczkowski, and J. L. Melsa, eds. Alternatives for Linear Multivariable Control with Turbofan Engine Theme Problem, Nat. Eng. Consortium, Chicago, 1978. 15. L. G. Hofmann, G. L. Teper, and R. F. Whitbeck, Application of Frequency Domain Multivariable Control Synthesis Techniques to an Illustrative Problem in Jet Engine Control, 5170, in Reference 14. 16. J. L. Peczkowski and M. K. Sain, Linear multivariable synthesis with transfer functions, pp. 7187, in Reference 14. 17. H. A. Spang, III, Insight into the application of the inverse Nyquist array method, pp. 138155, in Reference 14. 18. B. Kouvaritakis and J. M. Edmunds, The characteristic frequency and characteristic gain design method for multivariable feedback systems, pp. 229246, in Reference 14. 19. R. L. DeHoff et al., F100 Multivariable Control Synthesis Program, Vols. I and II, AFAPL-TR-77-35, June 1977. 20. C. J. Skira, R. L. DeHoff, and W. E. Hall, Jr., Design, Evaluation and Test of an Electronic, Multivariable Control for the F100 Turbofan Engine, AGARD-PP-274, Advanced Control System for Aircraft Power, Cologne, Germany, Oct. 1979. 21. R. L. DeHoff and W. E. Hall, Jr., Multivariable quadratic synthesis of an advanced turbofan engine controller, J. of Guidance and Control, I: 136142, 1978. 22. R. L. DeHoff and W. E. Hall, Jr., Optimal control of turbine engines, J. of Dynamic Systems, Measurements and Controls, 101: 117126, 1979. 23. S. M. Rock and R. L. DeHoff, Variable Cycle Engine Multivariable Control Synthesis, AFAPL-TR-79-2043, 1979. 24. R. L. DeHoff and W. E. Hall, Jr., Multivariable control design principles applied to a variable cycle turbofan engine, Proc. of the Asilomer Conf. on Circuits, Systems, and Computers, Pacic Grove, CA, Nov. 1978. 25. S. Adibhatla, Propulsion Control Law Design for the NASA STOVL Controls Technology Program, AIAA, 93-4842, AIAA Int. Powered Lift Conf., Santa Clara, CA, 1993. 26. S. Adibhatla et al., STOVL Controls Technology, Vol. I, Integrated Flight/Propulson Control Design, NASA Contractor Report 195361, 1994. 27. R. Hanus, M. Kinnaert, and J. L. Henrotte, Conditioning technique, A general anti-windup and bumpless transfer method, Automatica, 23: Nov. 1987.
H. AUSTIN SPANG, III

GE Research and Development Center
HAROLD BROWN
Gray Fox Technologies
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Kalman Filters and Observers Standard Article Mohinder S. Grewal1 1California State University, Fullerton, Fullerton, CA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1020 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (1787K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are White Noise Linear Estimation The Linear Optimal Estimator In Discrete Time (Kalman Filter) The Continuous-Time Optimal Estimator (Kalman-Bucy Filter) Nonlinear Estimation The Matrix Riccati Differential Equation Controllers, Observers, And The Separation Principle Implementation Methods About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
KALMAN FILTERS AND OBSERVERS FILTERING THEORY GAUSSIAN PROCESSES NONLINEAR FILTERS
Theoretically, a Kalman lter is an estimator for what is called the linear quadratic Gaussian (LQG) problem, which is the problem of estimating the instantaneous state of a linear dynamic system perturbed by Gaussian white noise, by using measurements linearly related to the state, but corrupted by Gaussian white noise. The resulting estimator is statistically optimal with respect to any quadratic function of estimation error. R. E. Kalman introduced the lter in 1960 (1). Practically, the Kalman lter is certainly one of the greater discoveries in the history of statistical estimation theory, and one of the greatest discoveries in the twentieth century. It has enabled humankind to do many things that could not have been done without it, and it has become as indispensable as silicon in the makeup of many electronic systems. Its most immediate applications have been for the control of complex dynamic systems such as continuous manufacturing processes, aircraft, ships, or spacecraft. In order to control a dynamic system, one must rst know what the system is doing. For these applications, it is not always possible or desirable to measure every variable that one wants to control. The Kalman lter provides a means for inferring the missing information from indirect (and noisy) measurements. In such situations, the Kalman lter is used to estimate the complete state vector from partial state measurements and is called an observer. The Kalman lter is also used for predicting the outcome of dynamic systems that people are not likely to control, such as the ow of rivers during ood conditions, the trajectories of celestial bodies, or the prices of traded commodities. From a practical standpoint, this article will present the following perspectives: 1. Kalman ltering is an algorithm made from mathematical models. The Kalman lter makes it easier to solve a problem, but it does not solve the problem all by itself. As with any algorithm, it is important to understand its use and function before it can be applied effectively. The purpose of this article is to ensure sufcient familiarity with the use of the Kalman lter that it can be applied correctly and efciently. 2. The Kalman lter is a recursive algorithm. It has been called ideally suited to digital computer implementation, in part because it uses a nite representation of the estimation problemby a nite number of variables (2). It does, however, assume that these variables are real numbers with innite precision. Some of the problems encountered in its use arise from the distinction between nite dimension and Figure 1. Foundational concepts in Kalman ltering.
nite information, and the distinction between nite and manageable problem sizes. These are all issues on the practical side of Kalman ltering that must be considered along with the theory. 3. It is a complete statistical characterization of an estimation problem. It is much more than an estimator, because it propagates the entire probability distribution of the variables it is tasked to estimate. This is a complete characterization of the current state of knowledge of the dynamic system, including the inuence of all past measurements. These probability distributions are also useful for statistical analysis and predictive design of sensor systems. 4. In a limited context, the Kalman lter is a learning process. It uses a model of the estimation problem that distinguishes between phenomena (what we are able to observe), noumena (what is really going on), and the state of knowledge about the noumena that we can deduce from the phenomena. That state of knowledge is represented by probability distributions. To the extent that those probability distributions represent knowledge of the real world, and the cumulative processing of knowledge is learning, this is a learning process. It is a fairly simple one, but quite effective in many applications. Figure 1 depicts the essential subjects forming the foundations for Kalman ltering theory. Although this shows Kalman ltering as the apex of a pyramid, it is but part of the foundations of another disciplinemodern control theoryand a proper subset of statistical decision theory (3). Applications of Kalman ltering encompass many elds. As a tool, the algorithm is used almost exclusively for estimation and performance analysis of estimators and as observers for control of a dynamical system. Except for a few fundamental physical constants, there is hardly anything in the universe that is truly constant. The orbital parameters of the asteroid Ceres are not constant, and even the xed stars and continents are moving. Nearly all physical systems are dynamic to some degree. If we want very precise estimates of their characteristics over time, then we must take their dynamics into consideration. Table 1 gives examples of common estimation problems. We do not always know the dynamics very precisely. Given this state of partial ignorance, the best we can do is express ignorance more preciselyusing probabilities.
Nonlinear Filters
The Kalman lter allows us to estimate the state of such systems with certain types of random behavior by using such statistical information. A few examples of such systems are listed in Table 1. The third column of Table 1 lists some sensor types that we might use to estimate the state of the corresponding dynamic systems. The objective of design analysis is to determine how best to use these sensor types for a given set of design criteria. These criteria are typically related to estimation accuracy and system cost. Because the Kalman lter uses a complete description of the probability distribution of its estimation errors in determining the optimal ltering gains, this probability distribution may be used in assessing its performance as a function of the design parameters of an estimation system, such as: The types of sensors to be used The locations and orientations of the various sensor types with respect to the system to be estimated The allowable noise characteristics of the sensors The preltering methods for smoothing sensor noise The data sampling rates for the various sensor types The level of model simplication to reduce implementation requirements This analytical capability of the Kalman lter enables system designers to assign error budgets to subsystems of an estimation system and to trade off the budget allocations to optimize cost or other measures of performance while achieving a required level of estimation accuracy. Furthermore, it acts like an observer by which all the states not measured by the sensors can be constructed for use in the control system applications. WHITE NOISE It is common engineering practice to model uncertainty in terms of Gaussian probability distributions and dynamic uncertainty in terms of linear dynamic systems disturbed by uncorrelated (white noise) processeseven though empirical analysis may indicate that the probability distributions are not truly Gaussian, the random processes are not truly white, or the relationships are not truly linear. Although this approach may discard useful information, we continue the practice for the following reasons: 1. Approximation Probability distributions may not be precisely Gaussian, but it is close enough. Nonlinear systems are often smooth enough that local linearization is adequate. Even though the icker noise observed in electronic systems cannot be modeled precisely using only white noise, it can often be done closely enough for practical purposes. 2. Simplicity These models have few parameters to be estimated. Gaussian distributions are characterized by their means and variances, and white noise processes are characterized by their variances.
3. Consistency Linearity preserves Gaussianity. That is, Gaussian probability distributions remain Gaussian under linear transformations of the variates. 4. Tractability These models allow us to derive estimators minimizing expected squared errors. 5. Good Performance The resulting estimators have performed well for many important applications, despite apparent discrepancies between models and reality. 6. Adaptability These estimators can often be extended to estimate parameters of the model or to track slow random variations in parameters. 7. Extendability The variances used for calculating feedback gains can also be used for comparing performance to modeled performance, detecting anomalous behavior, and rejecting anomalous sensor data. Vector-valued random processes x(t) and y(t) are called uncorrelated if their cross-covariance matrix is identically zero for all times t1 and t2 :
where E is the expected value operator and T is the transpose of the vector. The random process x(t) is called uncorrelated if
where (t) is the Dirac delta function (actually, a generalized function), dened by
Similarly, a random sequence xk in discrete time is called uncorrelated if
where
() is the Kronecker delta function, dened by
Q1 (t) and Q2 (k) are the intensity matrices of the white noise process and sequence. If Q1 (t) and Q2 (t) are constant, the processes and sequences are stationary. If the probability distribution of a white noise process at each instant of time is Gaussian, then it is completely dened by its rst two moments, mean and variance. If Ex(t) = 0, the process is called zero mean. A white noise process or sequence is an example of an uncorrelated process or sequence. Generally, a white noise process has no time structure. In other words, knowledge of the white process value at one instant of time provides no knowledge of what its value will be (or was) at any other time point.
Nonlinear Filters
LINEAR ESTIMATION Linear estimation addresses the problem of estimating the state of a linear stochastic system by using measurements or sensor outputs that are linear functions of the state. We suppose that the stochastic systems can be represented by the types of plant and measurement models (for continuous and discrete time) shown as equations in Table 2, with dimensions of the vector and matrix quantities. The measurement and plant noise vk and wk , respectively, are assumed to be zero-mean Gaussian processes, and the initial value xo is a Gaussian random variable with known mean x0 and known covariance matrix Po . Although the noise sequences wk and vk are assumed to be uncorrelated, this restriction can be removed, modifying the estimator equations accordingly. The objective of statistical optimization is to nd an estimate of the n state vector xk represented by x k , a linear function of the measurements zi , . . . , zk , which minimizes the weighted mean-squared error
or its discrete-time equivalent. This implementation automatically minimizes the expected risk associated with any quadratic loss function of the estimation error. The statistical performance of the estimator can be predicted a priori (i.e., before it is actually used) by solving the nonlinear differential (or difference) equations used in computing the optimal feedback gains of the estimator. These are called Riccati equations, named in 1763 by Jean le Rond DAlembert (17171783) for Count Jacopo Francesco Riccati (16761754), who had studied a secondorder scalar differential equation, although not the form that we have here (13, 14). Kalman gives credit to Richard S. Bucy for the discovery that the Riccati differential equation serves the same role as the Wiener-Hopf integral equation in dening optimal gains. The Riccati equation also arises naturally in the problem of separation of variables in ordinary differential equations and in the transformation of two-point boundary value problems to initial value problems (15). The behavior of their solutions can be shown analytically in trivial cases. These equations also provide a means for verifying the proper performance of the actual estimator when it is running. THE LINEAR OPTIMAL ESTIMATOR IN DISCRETE TIME (KALMAN FILTER) Suppose that a measurement has been made at time tk and that the information that it provides is to be applied in updating the estimate of the state x of a stochastic system at time tk . It is assumed that the measurement is linearly related to the state by an equation of the form zk = Hxk + vk , where H is the measurement sensitivity matrix and vk is the measurement noise. The optimal linear estimate is equivalent to the general (nonlinear) optimal estimator if the random variables x and z are jointly Gaussian. Therefore, it sufces to seek an updated estimate x k (+) (observation zk is included in the estimate) that is a linear function of the a priori estimate and the measurement z:
where M is any symmetric nonnegative denite weighting matrix. We will now derive the mathematical form of an optimal linear estimator for the states of linear stochastic systems given in Table 2. This is called the linear quadratic Gaussian estimation problem. The dynamic systems are linear, the performance cost functions are quadratic, and the random processes are Gaussian. Let us consider similar types of estimators for the LQG problem: Filters use observations up to the time that the state of the dynamic system is to be estimated:
Predictors estimate the state of the dynamic system beyond the time of the observations:
This is a relatively minor distinction, and the differences between the respective estimators are correspondingly slight. A straightforward and simple approach using orthogonality principles is used in the derivation of estimators. These estimators will have minimum variance and the unbiased and consistent. Interested readers may refer to Refs. 212. The Kalman lter can be characterized as an algorithm for computing the conditional mean and covariance of the probability distribution of the state of a linear stochastic system with uncorrelated Gaussian process and measurement noise. The conditional mean is the unique unbiased estimate. It is propagated in feedback form by a system of linear differential equations or by the corresponding discrete-time equations. The conditional covariance is propagated by a nonlinear differential equation
where x k () is the a priori estimate (observation zk is not included in the estimate) of xk and x k (+) is the a posteriori value of the estimate. k are as yet unknown. The weighting matrices K1 k and K k such that the new esWe seek those values of K1 k and K timate x k (+) will satisfy the orthogonality principle. This orthogonality condition can be written in the form
Equations Table 2 and for x k (+) from Eq. (9) into Eq. (10), then we will observe from Eqs. 1 and 2 that the data z1 , . . . , zk do not involve the noise term wk . Therefore, because the random sequences wk and vk are uncorrelated, it follows that Ewk zT i = 0 for 1 i k.
Nonlinear Filters
Using this result, we can obtain the following relation:
Substitute for xk , x k (+) and zk from Eqs. <xref target="W1020-mdis-0001 W1020-mdis-0009"/>, and <xref target="W1020-mdis-0020"/>, respectively. Then
It can be shown that E(wk1 zT i ) = 0 because wk is a white noise sequence. But because zk = Hk xk + vk , Eq. (12) can be rewritten as
However, by the system structure We also know that Eq. (11) holds at the previous step, that is
Substituting for K1 k , zk , and xk (),
and Let because vk is a white noise sequence uncorrelated with wk white noise i = 1, . . . k 1. Then Eq. (13) can be reduced to the form
be the error covariance matrix before update. Then
and therefore
Let Equation (16) can be satised for any given xk if
be the error covariance matrix after update. Substituting Eq. (17) into Eq. (9), we obtain
Clearly, this choice of K1 k causes Eq. (9) to satisfy a portion k is chosen such of the condition given by Eq. (10) where K that Eq. (11) is satised. Let
Subtract xk from both sides to obtain
Vectors xk (+) and xk () are the estimation errors after and before updates, respectively. From Eq. (11)
Substituting Eq. (31) into Eq. (29),
and also [subtract Eq. (11) from Eq. (21)]
Nonlinear Filters
k from Eq. (28), Substituting for K
Therefore,
Let
Subtract xk from both sides
Postmultiply by xT k () both sides and take the expected value:
arrows show the successive values assumed by the variables, with the annotation (in parentheses) on the arrows indicating which input variables dene the indicated transitions. Note that each variable assumes two distinct values at each discrete time: its a priori value corresponding to the value before the information in the measurement is used and the a posteriori value corresponding to the value after the information is used. At the beginning of the design phase of a measurement and estimation system, when neither real nor simulated data are available, just the covariance calculations can be used to obtain preliminary indications of lter performance. Covariance calculations consist of solving the estimator equations with steps 13, repeatedly. It is important to notice that the covariance calculations are independent of data zk . Covariance calculations will involve the plant noise matrix Q, measurement noise matrix R, state transition matrix , measurement matrix H, and initial covariance matrix Po . Step 4 of the Kalman lter implementation [computation of x k (+)] can be implemented only for state vector propagation where simulator or real data sets are available. In the design tradeoffs, the covariance matrix update (steps 1 and 3) should be checked for symmetry and positive deniteness. Failure to attain either condition is a sign that something is wrong. One possibility is that it is an illconditioned problem. In order to overcome ill-conditioning, another equivalent expression for Pk (+) is called the Joseph form, as shown Eq. (32)
Summary of Equations for the Discrete-Time Kalman Estimator The equations derived in the previous section are summarized in Table 3. In this formulation of the lter equations, G has been combined with the plant covariance by multiplying Gk1 and GT k1 , for example,
The relation of the lter to the system is illustrated in the block diagram of Fig. 2. The computational procedure for the discrete-time Kalman estimator follows: 1. Compute Pk () with Pk1 (+), k1 , Qk1 given to initialize the procedure. k with Pk () computed from step 1 and 2. Compute K Hk , Rk given to initialize the procedure. k computed from step 2 and 3. Compute Pk (+) with K Pk () from step 1. k from step 4. Compute x k (+) with computed values of K 3 and with given initial estimate x0 and data set zk . Figure 3 shows a typical time sequence of values assumed by the ith component of the estimated state vector (plotted with solid circles) and its corresponding variance of estimation uncertainty (plotted with open circles). The
Note that the right-hand side of this equation is the summation of two symmetric matrices. The rst of these is positive denite and the second is nonnegative denite, thereby making Pk (+) a positive denite matrix. Other techniques are described in the implementation methods to alleviate the ill-conditioning. k and Pk (+) are not that useful (817). Other forms for K It can be shown that state vector update, Kalman gain, and error covariance equations represent an asymptotically stable system; therefore, the estimate of state x k becomes independent of the initial estimate x o , Po as k is increased. It is also obvious that the Kalman gain and error covariance equations are independent of the observation. These equations are used for covariance analysis purposes. THE CONTINUOUS-TIME OPTIMAL ESTIMATOR (KALMAN-BUCY FILTER) Analogous to the discrete-time case, the continuous-time random process x(t) and the observation z(t) are given by
Nonlinear Filters
Figure 2. Block diagram of discrete-time Kalman lter.
This section provides a formal derivation of the continuous-time Kalman estimator. A rigorous derivation can be achieved by using the orthogonality principle as in the discrete-time case. In view of the main objective (to obtain efcient and practical estimators), less emphasis is placed on continuous-time estimators, except in academia. Let t be the time interval [tk tk1 ]. The following relationships are used:
where 0( t2 ) consists of terms with powers of t greater than and equal to two. Discrete measurement noise covariance in terms of continuous covariance is given by
Figure 3. Sequence of values of lter variable in discrete time.
Discrete process noise covariance in terms of continuous covariance is given by
Equations <xref target="W1020-mdis-0034"/> and <xref target="W1020-mdis-0037"/> can be combined. By substituting these relations, we can get the result where F(t), G(t), H(t), Q(t), and R(t) are n n, n n, l n, n n, and l l matrices, respectively. The covariance matrices Q and R are positive denite.
It is desired to nd the estimate of n state vector x(t) represented by x (t), which is a linear function of the measurements z(t)o t T which minimizes the scalar equation
The Kalman gain of Eq. (28) becomes, in the limit,
where M is a symmetric positive denite matrix. The initial estimate and covariance matrix are x 0 and P0 .
Nonlinear Filters
Substituting Eq. (50) into Eq. (51) and taking the limit as t 0, we obtain the desired result
with P(t0 ) as the initial condition. This is called the matrix Riccati differential equation. Methods for solving it will be discussed in the next section. The differential equation can be rewritten by using the identity
to transform Eq. (52) to the form
In similar fashion, the state vector update equation can be derived from Eqs. 30 and 35 by taking the limit as t 0 to obtain the differential equation for the estimate:
lters and predictors can be found in References 2, 4, and 79. Suppose that a continuous or discrete stochastic system can be represented by nonliner plant and measurement models as shown in Table 4. Although afne (i.e., linear and additive) transformations of Gaussian random variables have Gaussian distributions, the same is not always true in the nonlinear case. Consequently, it is not necessary that w and v be gaussian. They may be included as arguments of the nonlinear functions f and h, respectively. However, the initial value x0 may be assumed to be a gaussian random variate with known mean and known n n covariance matrix P0 . The objective is to estimate xk or x(t) to satisfy a specied performance criterion (as given previously). Applying linearization techniques (comparison with Taylor series expansion and discarding the 2nd and higher order terms) to get simple approximate solutions to nonlinear estimation problems requires that f and h are twice continuously differentiable (7, 20). Linearization About a Nominal Trajectory A trajectory is a particular solution of a stochastic system, with a particular instantiation of the random variates involved. The trajectory is a vector-valued sequence {xk |k = 0, 1, 2, 3, . . . } for discrete-time systems, and a vector-valued function x(t), 0 t, for continuous-time systems. The term nominal in this case refers to that trajectory obtained when the random variates assume their expected values. For example, the sequence {xN k } obtained as a solution of the equation
with initial condition x (0). Equations <xref target="W1020-mdis-0051 W1020-mdis-0054"/>, and <xref target="W1020-mdis-0055"/> dene the continuous-time Kalman estimator, which is also called the Kalman-Bucy lter (119). The Wiener lter is dened for stationary systems in continuous time, and the Kalman lter is dened for either stationary or nonstationary systems in either discrete time or continuous time, but nite state dimension. To demonstrate the connections on problems satisfying both sets of constraints, like the continuous-time Kalman Bucy estimator Eqs. <xref target="W1020-mdis-0051"/>, <xref target="W1020-mdis-0054"/>, and <xref target="W1020mdis-0055"/>, letting F, G, H be constants; the noises be stationary (Q and R constants), and the lter reach steady state (P constant). That is, as t , then P (xt) 0. The Riccati differential Eq. (54) becomes the algebraic Riccati equation for continuous time systems. Taking the Laplace of Eq. (55) leads in a transfer function with constant gain K and represents the steady state Kalman-Bucy lter, which is identical to the Wiener lter (3). NONLINEAR ESTIMATION Linear estimators for discrete and continuous systems have been derived. The combination of functional linearity, quadratic performance criteria, and Gaussian statistics is essential to this development. The resulting optimal estimators are simple in form and powerful in effect. Many dynamic systems and sensors are not absolutely linear, but they are not far from it. Following the considerable success enjoyed by linear estimation methods on liner problems, extensions of these methods were applied to such nonlinear problems. This section investigates the model extensions and approximation methods used for applying the methodology of Kalman ltering to these slightly nonlinear problems. More formal derivations of these nonlinear
with zero process noise and with the mean xN 0 as the initial condition would be a nominal trajectory for a discrete-time system. The word perturbation has been used by astronomers to describe a minor change in the trajectory of a planet (or any free-falling body) due to secondary forcessuch as those produced by other gravitational bodies. Astronomers had learned long ago that the actual trajectory can be accurately modeled as the sum of the solution of the two-body problem (which is available in closed form) and a linear dynamic model for the perturbations due to the secondary forces. This technique also works well for many other nonlinear problems, including the problem at hand. In this case, the perturbations are due to the presence of random process noise and errors in the assumed initial conditions. If the function f in the previous example is continuous, then the state vector xk at any instant on the trajectory will vary smoothly with small perturbations of the state vector xk1 at the previous instant. These perturbations are the result of off-nominal (i.e., off-mean) values of the random variates involved. These random variates include the initial value of the state vector x0 , the process noise wk , and (in the case of the estimated trajectory) the measurement noise vk . If f is continuously differentiable, then the inuence of the perturbations on the trajectory can be represented by a Taylor series expansion about the nominal trajectory. The likely magnitudes of the perturbations are determined by
Nonlinear Filters
the variances of the variates involved. If these perturbations are sufciently small relative to the higher-order coefcients of the expansion, then we can obtain a good approximation by ignoring terms beyond some order. (However, we must usually evaluate the magnitudes of the higherorder coefcients before making such an assumption.) Let the symbol denote perturbations from the nominal
Let h be sufciently differentiable, then the measurement can be represented by a Taylor series:
or
so that the Taylor series expansion of f(x, k 1) with respect to x at x = xN k1 is If the higher-order terms in this expansion can be ignored, then we can represent the perturbation in zk as
where the rst-order variational term is
or
which is an l n constant matrix. In the continuous case, the corresponding nonlinear differential equations for plant and observation are If the higher-order terms in x can be neglected, then
where the rst-order approximation coefcients are given by
with the dimensions of the vector quantities the same as in the discrete case. Similar to the case of the discrete system, the linearized differential equations can be derived as
a n n constant matrix.
Equations <xref target="W1020-mdis-0075"/> and <xref target="W1020-mdis-0077"/> represent linearized continuous model equations. The variables x (t) and z (t) are the
Nonlinear Filters
perturbations about the nominal values as in the discrete case (3). Linearization About the Estimated Trajectory The problem with linearization about the nominal trajectory is that the deviation of the actual trajectory from the nominal trajectory tends to increase with time. As the deviation increases, the signicance of the higher-order terms in the Taylor series expansion of the trajectory also increases. A simple but effective remedy for the deviation problem is to replace the nominal trajectory with the estimated trajectorythat is, to evaluate the Taylor series expansion about the estimated trajectory. If the problem is observable (as evidenced by the covariance of estimation uncertainty getting smaller until it reaches a steady state), then the deviations between the estimated trajectory (along which the expansion is made) and the actual trajectory will remain sufciently small that the linearization assumption is valid (7, 20). The principal drawback to this approach is that it tends to increase the real-time computational burden. Even for linearization about a nominal trathough , H, and K jectory may have been precomputed ofine, they must be computed in real time as functions of the estimate for linearization about the estimated trajectory. The only modication required is to replace xN k1 by x k1 and xN k by x k in the evaluations of partial derivatives. Now the matrices of partial derivatives become
Figure 4. Estimator linearized about a nominal state.
The models and implementation equations of the extended Kalman lter are summarized in Table 6. The last three equations in this table are the same as those for the standard Kalman lter, but the other equations are noticeably different from those of the linearized Kalman lter in Table 5. It has been said that modeling is the hardest part of Kalman ltering. This is especially true when there are nonlinearities in the physical equation that must be linearized. Developing a good Kalman lter model is partly art and partly science. As a general rule, we look for models that are simple enough to be implemented but, at the same time, still represent the physical situation with a reasonable degree of accuracy (3). THE MATRIX RICCATI DIFFERENTIAL EQUATION In order to implement a Kalman lter, the Riccati equation must be solved. This section presents a brief discussion of solution methods for the Riccati differential equation for the Kalman-Bucy lter. A more thorough treatment of the Riccati equation can be found in Ref. 21. Transformation to a Linear Equation
and
The matrices have the same general form as for linearization about a nominal trajectory, except for the evaluations of the partial derivatives:
and
The Riccati differential equation was rst studied in the eighteenth century as a nonlinear scalar differential equation, and a method was derived for transforming it to a linear matrix differential equation. That same method works when the dependent variable of the original Riccati differential equation is a matrix. That solution method is derived here for the matrix Riccati differential equation of the Kalman-Bucy lter. An analogous solution method for the discrete-time matrix Riccati equation of the Kalman lter is derived in the next section. A matrix product of the sort AB1 is called a matrix fraction, and a representation of a matrix N in the form
Linearized and Extended Kalman Filters. The block diagram of Fig. 4 shows the data ow of the estimator linearized about a nominal trajectory of the state dynamics. Note that the operations within the dashed box have no inputs. These are the computations for the nominal trajectory. Because they have no inputs from the rest of the estimator, they can be precomputed ofine. The models and implementation equations for the linearized discrete Kalman lter are summarized in Table 5. Note that the last three equations in this table are identical to those of the standard Kalman lter.
will be called a fraction decomposition of N. The matrix A is the numerator of the fraction, and the matrix B is its denominator. It is necessary that the matrix denominator be nonsingular. The Riccati differential equation is nonlinear. However, a fraction decomposition of the covariance matrix results in a linear differential equation for the numerator and denominator matrices. The numerator and denominator matrices will be functions of time, such that the product
10
Nonlinear Filters
A(t)B1 (t) satises the matrix Riccati differential equation and its boundary conditions. By taking the derivative of the matrix fraction A(t)B1 (t) with respect to t and using the fact that
Solution of the Algebraic Riccati Equation We have seen in the previous subsections the difculty of obtaining a solution of the general Riccati differential equation in closed form (i.e., as a formula in the parameters of the mode), even for the simplest (scalar) problem. There is no general formula for solving higher-order polynomial equations (i.e., beyond quartic).This is at the limit of complexity for nding closed-form solutions to algebraic Riccati equations by purely algebraic means. Beyond this relatively low level of complexity, it is necessary to employ numerical solution methods. Numbers do not always provide us as much insight into the characteristics of the solution as formulas do, but they are all we can get for most problems of practical signicance. The MacFarlane-Potter-Fath Eigenstructure Method MacFarlane, Potter, and Fath discovered (independently) that the solution P() of the continuous-time form of the steady state matrix Riccati differential equation can be expressed in the form (22)
we can arrive at the following decomposition of the matrix Riccati differential equation:
where the matrices A and B are n n and the 2n-vectors eik are characteristic vectors of the continuous-time system Hamiltonian matrix
This can be formalized in somewhat greater generality as a lemma. Combining Eqs. 88 and 89 gives Lemma 1 If A and B are n n matrices such that B is nonsingular and
The last equation is a linear rst-order matrix differential equation. The dependent variable is a 2n n matrix, where n is the dimension of the underlying state variable. The Hamiltonian matrix is given by
for a n n matrix D, then P = AB1 satises the steady state matrix Riccati differential equation
the initial values of A(t) and B(t) must also be constrained by the initial value of P(t). This is easily satised by taking A(t0 ) = P(t0 ) and B(t0 ) = I, the identity matrix. In the time-invariant case, the Hamiltonian matrix is also time-invariant. As a consequence, the solution for the numerator A and denominator B of the matrix fraction can be represented in matrix form as the product
Proof Equation (95) can be written as two equations,
If we multiply both of these on the right by B1 and the last of these on the left by AB1 , we obtain the equivalent equations
where e
is a 2n 2n matrix.
Nonlinear Filters
11
or taking the differences of the left-hand sides and substituting P for AB1
is a solution at time tk+1 , where
which was to be proved. In the case that A and B are formed in this way from n characteristic vectors of c , the matrix D will be a diagonal matrix of the corresponding characteristic values. Therefore, to obtain the steady state solution of the matrix Riccati differential equation by this method, it sufces to nd n characteristic vectors of c such that the corresponding B-matrix is nonsingular. As will be shown in the next section, the same trick works for the discrete-time matrix Riccati equation. The Matrix Riccati Equation in Discrete Time The representation of the covariance matrix as a matrix fraction is also sufcient to transform the nonlinear discrete-time Riccati equation for the estimation uncertainty into a linear form. The discrete-time problem differs from the continuous-time problem in two important aspects: 1. The numerator and denominator matrices will be propagated by a 2n 2n transition matrix, and not by differential equations. The approach is otherwise similar to that for the continuous-time Riccati equation, but the resulting 2n 2n state transition matrix for the recursive updates of the numerator and denominator matrices is a bit more complicated than the coefcient matrix for the linear form of the continuous-time matrix Riccati equation. 2. There are two distinct values of the discrete-time covariance matrix at any discrete time-stepthe a priori value and the a posteriori value. The a priori value is of interest in computing Kalman gains, and the a posteriori value is of interest in the analysis of estimation uncertainty. The linear equations for matrix fraction propagation of the a priori covariance matrix are derived later. The method is then applied to obtain a closed-form solution for the scalar time-invariant Riccati equation in discrete time and to obtain a method for exponential speedup of convergence to the asymptotic solution. Lemma 2 If the state transition matrices gular and
k
Proof The following annotated sequence of equalities starts with the product Ak+1 B1 k+1 as dened and proves that it equals Pk+1 :
where the Hemes inversion formula is given in Ref. 3. This completes the proof. This lemma is used later to derive a closed-form solution for the steady state Riccati equation in the scalar timeinvariant case. The MacFarlane-Potter-Fath Eigenstructure Method The method presented for the steady state solution of the time-invariant matrix Riccati differential equation (i.e., in continuous time) also applies to the Riccati equation in discrete time (22). As before, it is formalized as a lemma. Lemma 3 If A and B are n n matrices such that B is nonsingular and
are nonsin-
is a nonsingular matrix solution of the discrete-time Riccati equation at time tk , then
12
Nonlinear Filters
for a n n nonsingular matrix D, then P = AB1 satises the steady state discrete-time matrix Riccati equation
linear rst-order differential equations. We deal with only continuous and linear differential equation models. Nonlinear models can be linearized as shown previously:
Proof If Pk = AB1 , then it was shown in Lemma 2 that 1 where Pk+1 = AB
where x(t) represents the n-dimensional state vector and u is r-dimensional deterministic control input. F(t) and G(t) are known time-varying matrices. The objective of control is to keep x(t) close to zero without excessive control effort. This objective is to be achieved by minimizing the quadratic cost function.
where
Consequently,
The solution is provided by the optimal state feedback control
0 (t) is an r n time-varying control gain matrix. where K 0 (t) is given by The value of K
That is, AB1 is a steady state solution, which was to be proved. In practice, A and B are formed from n characteristic vectors of d . The matrix D will be a diagonal matrix of the corresponding nonzero characteristic values. The algebraic Riccati equation can be solved by using MATLAB. The algebraic matrix Riccati equation for the continuous case uses Schurs method which is slower, but more robust when the system is not able to be diagonalized (3, 23). The matrix Riccati equation for the continuous case can be converted to vector nonlinear coupled differential equations. Fourth-order Runge-Kutta (self-starting), fourthorder Adams method (non-self-starting), and AdamsMoultons predictor corrector can be used to solve these equations (24). CONTROLLERS, OBSERVERS, AND THE SEPARATION PRINCIPLE Optimal control theory was developed under the inuence of such great researchers as Pontryagin, Bellman, Kalman, and Bucy (2530). Kalman introduced a number of state variable concepts. Among these were controllability, observability, optimal linear quadratic regulator (LQR), state feedback, and optimal state estimation (Kalman ltering). In LQR problems, the dynamics of the system to be controlled are represented by the state-space model, a set of
where the n n matrix P0 (t) is the solution of the Riccati matrix differential equation
subject to boundary conditions at the terminal time T, P0 (T) = M0 . The same solving techniques can be applied as developed previously. This Riccati equation has boundary conditions given as compared to initial condition in the case of estimation problems. For time-invariant cases, x(t) 0 as t . For this reason, a terminal weighing matrix M0 need not be included in Eq. (110) and F(t) = F, C(t) = C, Q0 (t) = Q0 , R0 (t) = R0 . The optimal control correction is
and P0 is a constant symmetric position denite n n matrix, which is the solution of the algebraic matrix Riccati equation
The same solving technique can be applied as developed previously. In implementation, the state variables, which are generally unavailable for direct measurement, would be substituted by their estimates (see Fig. 5) by an observer or
Nonlinear Filters
13
Figure 5. Block diagram of separation principle.
Kalman lter. The remarkable property of the LQG control problem is that the optimal control u(t) is generated from the estimated state x (t), generated by the Kalman lter by means of the relationship
0 is precisely the one determined where the gain matrix K in the solution of the deterministic LQR [see Eq. (116)] 0 (t) x(t) under the assumption problem. That is, u(t) = K that the complete state vector is measured exactly. The LQG problem solution separates into the solution of a linear quadratic deterministic problem and the solution of a linear Gaussian estimation problem. The key theorem that shows this property is often called the separation theorem. The importance of the separation principle is that the LQG regulator design procedures can be accomplished in two separate stages: (1) the Kalman lter design and (2) the control feedback design. This means that all results derived separately for the deterministic optimal control problem and the optimal estimation problems are still valid (20, 26). IMPLEMENTATION METHODS We have discussed what Kalman lters are and how they are supposed to behave. Their theoretical performance has been shown to be characterized by the covariance matrix of estimation uncertainty, which is computed as the solution of a matrix Riccati differential and difference equation. A relationship between optimal deterministic control and optimal estimation problems has been described via the separation principle. Soon after the Kalman lter was rst implemented on computers, it was discovered that the observed meansquare estimation errors were often much larger than the values predicted by the covariance matrix, even with simulated data. The variances of the lter estimation errors were observed to diverge from their theoretical values, and the solutions obtained for the Riccati equations were observed to have negative variances. Riccati equations should have positive or zero variances. Current work on the Kalman lter primarily focuses on development of robust and numerically stable implementation methods. Numerical stability refers to robustness
against roundoff errors. Numerically stable implementation methods are called square root ltering because they use factors of the covariance matrix of estimation uncertainty or its inverse, called the information matrix. Numerical solution of the Riccati equation tends to be more robust against roundoff errors if Cholesky factors of a symmetrical nonnegative denite matrix P is a matrix C such that CCT = P. Cholesky decomposition algorithms solve for C that is either upper triangular or lower triangular. Another method is modied Cholesky decomposition. Algorithms solve for diagonal factors and either a lower triangular factor L or an upper triangular factor U such that P = UDu UT = LDL LT where DL and Du are diagonal factors with nonnegative diagonal elements. Another implementation method uses square root information lters that use a symmetric product factorization of the information matrix P1 . Alternative Kalman lter implementations use these factors of the covariance matrix (or its inverse) in three types of lter operations: (1) temporal updates, (2) observation updates, and (3) combined updates (temporal and observation). The basic algorithm methods used in these alternative Kalman lter implementations fall into four general categories. The rst three of these categories are concerned with decomposing matrices into triangular factors and maintaining the triangular form of the factors through all the Kalman ltering operation. The fourth category includes standard matrix operations (multiplication, inversion, etc.) that have been specialized for triangular matrices. These implementation methods have succeeded where the conventional Kalman lter implementations have failed (3, 31). Even though uses are being explored in virtually every discipline, research is particularly intense on successful implementation of Kalman ltering to global positioning systems (GPS), inertial navigation systems (INS), and guidance and navigation. GPS is a satellite-based system that has demonstrated unprecedented levels of positioning accuracy, leading to its extensive use in both military and civil arenas. The central problem for GPS receivers is the precise estimation of position, velocity, and time based on noisy observations of satellite signals. This provides an ideal setting for the use of Kalman ltering. GPS technology is used in automobile, aircraft, missiles, ships, agriculture, and surveying. In 1995 the United States began development of the Wide Area Augmentation System (WAAS) under the auspices of the Federal Aviation Administration (FAA) and the Department of Transportation (DOT), to provide precision approach capability for aircraft. Without WAAS, ionospheric disturbances, satellite clock drift, and satellite orbit errors cause too much error in the GPS signal for aircraft to perform a precision landing approach. Additionally, signal integrity information as broadcast by the satellites is insufcient for the demanding needs of public safety in aviation. WAAS provides additional integrity messages to aircraft to meet these needs. WAAS includes a core of approximately twenty-ve wide area ground reference stations (WRS) positioned throughout the United States which have precisely surveyed coordinates. These stations compare the GPS signal measure-
14
Nonlinear Filters 6. R. S. Bucy, Optimal ltering for correlated noise, J. Math Anal. Appl., 20: 18, 1967. 7. M. S. Grewal et al., Application of Kalman Filtering to GPS, INS, and Navigation, Anaheim, CA: KFCA,Jan. 2007. 8. A. H. Jazwinski, Stochastic Processes and Filtering Theory, New York: Academic, 1970. 9. P. S. Maybeck, Stochastic Models, Estimation, and Control, New York: Academic, 1979, Vol. 1. 10. P. S. Maybeck, Stochastic Models, Estimation, and Control, New York: Academic, 1982, Vol. 2. 11. J. M. Mendel, Kalman ltering and other digital estimation techniques, IEEE Individual Learning Package, 1987. 12. H. W. Sorenson (ed.), Kalman Filtering: Theory and Application, New York: IEEE Press, 1985. 13. M. A. Poubelle et al., A miscellany of results on an equation of Count J. F. Riccati, IEEE Trans. Autom. Control, AC-31: 651654, 1986. 14. J. F. Riccati, Animadversationnes in aequationes differentiales secundi gradus, Acta Eruditorum Quae Lipside Publicantur Suppl., 8: 6673, 1724. 15. L. Dieci, Numerical integration of the differential Riccati equation and some related issues, SIAM J. Numer. Anal., 29: 781815, 1992. 16. T. Kailath, R. A. Geesey, An innovations approach to least squares estimation. Part IV: Recursive estimation given lumped covariance functions, IEEE Trans. Autom. Control AC16: 720726, 1971. 17. T. Kailath, An innovations approach to least squares estimation. Part I: Linear ltering in additive white noise, IEEE Trans. Autom. Control, AC-13: 646655, 1968. 18. R. E. Kalman, New methods in Wiener ltering, in Proc. First Symposium on Engineering Applications of Random Function Theory and Probability, New York: Wiley, 1963, Chapter 9. 19. R. E. Kalman, R. S. Bucy, New results in linear ltering and prediction theory. ASME J. Basic Eng., Ser. D, 83: 95108, 1961. 20. M. Athans et al. (guest eds.), Special issue on linear-quadraticgaussian problem, IEEE Trans. Autom. Control, AC-16: 529552, 1971. 21. S. Bittanti, A. J. Laub, J. C. Willems (eds.), The Riccati Equation, New York: Springer-Verlag, 1991. 22. A. G. J. MacFarlane, An eigenvector solution of the optimal linear regulator, J. Electron. Control, 14: 643654, 1963. 23. R. G. Brown, P. Y. C. Hwang, Introduction to Random Signals and Applied Kalman Filtering, 3rd ed., New York: Wiley, 1997. 24. M. L. James, G. M. Smith, J. C. Wolford, Applied Numerical Methods for Digital Computation, 4th ed. New York: HarperCollins College Publishers, 1993. 25. L. S. Pontrjagin, On the zeros of some elementary transcendental functions, Izv. Akad. Nauk SSSR, Ser. Mat., 6: 115134, 1942; Eng. Transl.: Amer. Math. Soc. Transl., 2: 95110, 1955. 26. R. E. Bellman, The theory of dynamic programming, Proc. Natl. Acad. Sci. USA, 38: 716719, 1954. 27. R. E. Bellman, Introduction to the Mathematical Theory of Control Processes, New York: Academic, 1967, Vol. 1. 28. R. E. Kalman, Contribution to the theory of optimal control, Bol. Soc. Matem. Mex., pp. 102119, 1960. 29. R. E. Kalman, On the general theory of control systems, Proc. 1st IFAC Cong., Moscow, 1960, Vol. 1, pp. 481492, 1961. 30. R. S. Bucy, P. D. Joseph, Filtering for Stochastic Processes, with Applications to Guidance, New York: Wiley, 1968.
ments with the measurements that should be obtained at the known coordinates. The WRS send their ndings to a WAAS master station (WMS) using a land-based communications network, and theWMS calculates correction algorithms and assesses the integrity of the system. The WMS then sends correction messages via a ground uplink system (GUC) to geostationary (GEO) WAAS satellites covering the United States. The satellites in turn broadcast the corrections on a per-GPS satellite basis at the same L, 1575.42 MHz frequency as GPS. WAAS-enabled GPS receivers receive the corrections and use them to derive corrected GPS signals which enable highly accurate positioning. On July 10, 2003, the WAAS system was activated for general aviation, covering 95% of the United States and portions of Alaska. In September 2003, improvements enabled WAAS-enabled aircraft to approach runways to within 250 feet altitude before requiring visual control. Currently, there are two Inmarsat III GEO satellites serving the WAAS area, the Pacic Ocean Region (POR) satellite, and the East Atlantic Ocean Region (AOR-W) satellite. In March 2005, two additional WAAS GEO satellites were launched (PanAmSat Galaxy XV and Telesat (Anik F1R), and are planned to be operational in 2006. These satellites plus the two existing satellites will improve coverage of North America and all but the northwest part of Alaska. The four GEO satellites will be positioned at 54 , 107 , and 133 West longitude, and at 178 East longitude. In 2006, WAAS is projected to be available over 99% of the time, and its coverage will include the full continental United States and most of Alaska. Although primarily intended for aviation applications, WAAS will be useful for improving the accuracy of any WAAS-enabled GPS receiver. Such receivers are already available in low-cost handheld versions for consumer use. Positioning accuracy using WAAS is currently quoted at less than 2 meters of lateral error and less than 3 meters of vertical error, which meets the aviation Category I precision approach requirement of 16 meters lateral error and 4 meters vertical error. Kalman lters are an integral part of the WAAS system (32). Kalman lters are used in bioengineering, trafc systems, photogrammetry, and myriad process controls. The Kalman lter is observer, parameter identier in modeling, predictor, lter, and smoother in a wide variety of applications. It has become integral to twenty-rst century technology.
BIBLIOGRAPHY
1. R. E. Kalman, A new approach to linear ltering and prediction problems, ASME J. Basic Eng., 82: 3445, 1960. 2. A. Gelb et al., Applied Optimal Estimation, Cambridge, MA: MIT Press, 1974. 3. M. S. Grewal, A. P. Andrews, Kalman Filtering Theory & Practice Using MATLAB (Second Edition), New York, NY: Wiley & Sons, 2001 (8th printing). 4. B. D. O. Anderson, J. B. Moore, Optimal Filtering, Englewood Cliffs, NJ: Prentice Hall, 1979. 5. A. E. Bryson, Jr., Y. C. Ho, Applied Optimal Control, Waltham, MA: Blaisdell, 1969.
Nonlinear Filters 31. G. J. Bierman, Factorization Methods for Discrete Sequential Estimation, New York: Academic, 1977. 32. M. S. Grewal, L. R. Weill, and A. P. Andrews, Global Positioning Systems, Inertial Navigation, and Integration (Second Edition), New York, NY: Wiley & Sons, 2007.
15
MOHINDER S. GREWAL
California State University, Fullerton Fullerton, CA, 92634
16
Nonlinear Filters
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Large-Scale and Decentralized Systems Standard Article Dragoslav D. iljak1 and Aleksandar I. Zeevi1 1Santa Clara University, Santa Clara, CA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1021 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (267K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Decentralized Control: An Example Interconnected Plants and Controllers Decentralized Feedback Structures Stabilization Adaptive Control Overlapping Decentralized Control Optimization About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
LARGE-SCALE AND DECENTRALIZED SYSTEMS
209

The size of a system is a subjective notion, and so is the notion of large-scale systems. In the following we will take a pragmatic view, and consider a system large whenever it is conceptually or computationally attractive to decompose it into interconnected subsystems. Typically the subsystems are of small size and can be solved efciently. The individual solutions can then be combined in some way to obtain a solution for the overall system. If the system is decomposed along the boundaries of the physical subsystems, the subsequent analysis may produce important information about the interplay between subsystems behavior and the nature of interconnections. It may, however, be computationally inefcient. To reduce the computational effort, one should develop universal decomposition
210
techniques, which need not be constrained by physical boundaries of subsystems, but may lead to efcient solutions of large problems utilizing modern computer architectures. With that in mind, our rst objective will be to describe several decomposition schemes that can either be used as preconditioners for decentralized control design, or can serve to speed up computations involving the control of large-scale systems (usually through the use of parallel processing). To efciently decompose a large-scale system, it is generally convenient to represent it in the form of a graph. Depending on whether or not the graph is weighted, we can roughly distinguish between two broad classes of decomposition algorithms. In problems where weights are not assigned to edges of the graph, decomposition schemes typically exploit topological properties such as structure and sparsity to obtain an appropriate partitioning (18). Whenever possible, it is also useful to incorporate any existing information regarding the physical attributes of the system (such as hierarchical structures or repetitive blocks that are built into the design). Decomposition algorithms of this type are commonly applied for solving large sparse systems of linear equations, and a number of them have been successfully utilized in parallelizing control-related computational problems. A conceptually different class of algorithms arises when the system is viewed as a weighted graph. In this case the partitioning strategy changes signicantly, since we can now utilize edge weights to identify weakly coupled subsystems and establish hierarchical relationships between them. Decomposition schemes based on this approach have found numerous applications not only in parallel computing, but also in decentralized control design. In the following sections particular attention will be devoted to the epsilon decomposition algorithm (911), due to its efciency and simplicity; in this context, we will also examine the concept of overlapping, and its application to the control of large-scale systems. Our second objective in this paper will be to provide the motivation and describe the basic ideas and techniques for decentralized control of dynamic systems. The accumulated experience in controlling large complex systems suggests three basic reasons for using decentralized control structures: dimensionality, information structure constraints, and uncertainty (6). By decomposing a system of large dimension into subsystems, a designer can devise decentralized strategies for solving control problems that would be either impractical or impossible to solve using a single centralized controller. Furthermore, in a large complex system where databases are developed around the plant with distributed sources of data, a need for fast control actions in response to local inputs and perturbations dictates use of distributed (that is, decentralized) measurement and control structures. A restriction on what and where the information is delivered in a large system is a common structural constraint in building controllers and estimators. A good example is the standard automatic generation control in power systems, where the decentralized schemes are used to reduce the cost of communication that would be demanded by a centralized control strategy spread over distant geographical areas. In modeling and control of large systems, it has been long recognized that models of subsystems can be obtained with increasing levels of accuracy and versatility. The essential uncertainty resides in the interconnections between the subsystems, since these interconnections are often poorly known in
x1
y1
x2
y2
x3
y3
Figure 1. Bipartite graph B.
both deterministic and stochastic terms. One of the reasons for using decentralized control strategies is their inherent robustness to a wide variety of structured and unstructured perturbations in the subsystems and their interconnections. Furthermore, the strategies can be made reliable with respect to both interconnection and controller failures involving individual subsystems. Epsilon Decompositions A natural way to introduce epsilon decompositions is to consider the system of linear algebraic equations Ax = b (1)
and its solution by the Jacobi iterative method (12). In this context, the epsilon decomposition algorithm will be used to permute matrix A into a form that ensures rapid convergence to the solution. The algorithm itself is remarkably simplegiven a matrix A (aij) and a value of parameter 0, all elements satisfying aij are set to zero. The resulting sparsied matrix is then permuted into a block-diagonal form, and all the variables in the same block are considered to be strongly coupled. After such a permutation, the matrix A can be represented as A = AD + AC (2)
where AD is block-diagonal and all elements of AC are less than or equal to one in magnitude. The following example illustrates how such a permutation can be identied, and subsequently utilized to iteratively solve Eq. (1). Example 1. Consider the matrix 1 0.05 A = 0.01 1 0.1 0.1
2 0.1 2
(3)
and the corresponding bipartite graph B in Fig. 1, in which vertices yi and xj are connected if and only if aij 0. If we remove all edges that correspond to elements aij 0.1, we obtain the subgraph B shown in Fig. 2. It is easily seen that
x1 y1
x2
y2
x3
y3
Figure 2. Subgraph B.
211
x1
y1
x3
y3
it is easily veried that no permutation can produce weakly coupled diagonal blocks. On the other hand, if repeated verti in Fig. 5, ces are allowed, we obtain the expanded bigraph B which now has two disconnected components. This transformation corresponds to a rectangular permutation matrix V, which is uniquely dened by the ordering of x and y vertices in the graph. Specically, VA = AV where (8)
x2
y2
Figure 3. Components of B.
the vertices and edges of B can now be regrouped into two disconnected components, as indicated in Fig. 3. The permutation dened by the vector p (1 3 2) now produces a matrix which satises Eq. (2), with 0.1 and 1 2 0 0 0 0.5 AD = 0 .1 0 AC = - 0 (4) - - - - -0 - - - - -1 - - -- - - -2 ---- , 0 0 1 0.1 1 0
= A
---------
---------
0 0
0 0
(9)
and
The impact of weak coupling is easily seen when Eq. (1) is solved using the Jacobi iterative method
1 xk + 1 = xk A D (Axk b )
(k = 0, 1, . . .)
(5)
1 0 V = 0 0
0 1 0 1
0 0 1 0
(10)
Namely, if the original matrix A of Eq. (3) is partitioned so that 1 0.05 0 AD = 0-.01 0(6) - - - - - -1 -----0 0 2
now has an It is easily veried that the expanded matrix A epsilon decomposition in the sense of Eq. (2).
Scaling. Another issue that arises in the practical application of epsilon decompositions is that elements of the matrix A can widely vary in size. In such cases it may not be possible to nd a meaningful value for , and row scaling needs to be utilized to obtain a more uniform distribution of element values. This process is demonstrated by the following example. Example 3. Let matrix A be 10 A = 0.1 4.5 dened as 5 0.3 0.2 0.02 1 100
1 we obtain I A D A2 1, and the process diverges; on the other hand, the partitioning in Eq. (4) obtained by epsilon decomposition results in rapid convergence.
Overlapping Epsilon Decompositions. Given a matrix A and a particular choice of parameter , there is no guarantee that A can be permuted into the form shown in Eq. (2). The obvious remedy in such cases is to repeat the decomposition with a larger value of ; alternatively, we can use the concept of overlapping. The following example illustrates the basic ideas behind overlapping epsilon decomposition. Example 2. Consider the following matrix A=
---------
(11)
In this case 0.3 is obviously not a feasible choice, since the entire second row would be eliminated. However, if each row is scaled by the element with the maximal absolute value, we obtain A = AD + AC (12)
(7)
where all entries larger than in magnitude are denoted by . The corresponding bipartite graph B is given in Fig. 4, and
x1 y1
x1
y1
x2(1)
y2(1)
x2
y2
x3
y3
x3
y3
x2(2)
y2(2)
Figure 4. Subgraph B.
. Figure 5. Expanded subgraph B
212
where 0.1 and
1 0.5 0 AD = 1- - - - 0 , -0 -. -5 -----0 0 1
---------
---------
0 0 0.3 AC = - -0 - - - - - -0 - - - - - -1 - - 0.45 0.1 0
(13)
Note that in this case A can be represented only as A AD AC (without ), but scaling does identify block-diagonal dominance, since
1 A D AC =
Figure 7. The undirected graph.
AD AC
(14) Nested Dissection Nested dissection is a typical representative of BBD decomposition methods. It is relatively simple algorithm, in which the matrix is assumed to be structurally symmetric and can therefore be represented by an undirected graph. The following example illustrates the decomposition procedure. Example 4. Let us consider the structurally symmetric matrix
This feature is frequently used to enhance the convergence of the iterative process in Eq. (5).
Structural Decompositions By their very nature, epsilon decompositions are ideally suited for iterative solutions of Eq. (1). In contrast, there is an entire class of decompositions that are aimed at solving Eq. (1) directly; in this approach, no matrix entries are discarded, and the decompositions are designed to achieve certain desirable structures. A structure that is of particular interest in the analysis and simulation of large-scale systems is the bordered block-diagonal (BBD) form shown in Fig. 6. The appeal of this structure lies in its inherent potential for parallel computation, a feature that has been widely exploited in different areas of engineering. Numerous algorithms have been developed for permuting a matrix into the BBD form, based on diverse concepts ranging from node clustering (2) and diakoptics (1,13) to various forms of graph dissection (3,4,7). Despite their obvious differences, all these methods have a common goal in identifying a minimal border that induces a block-diagonal structure in the rest of the matrix. The border size is an important issue in the solution of the system in Eq. (1), since the computation time depends heavily on the number of nonzero elements in the border. In this section, we will briey describe three methods for obtaining BBD structures. We begin our analysis with the classic nested dissection algorithm of George and Liu (3).
0 1 2B B B 3 B0 B0 4B B 5B B B 6 @
7 0
1 2 0 0
3 4 0 0 0 0 0 0
5 0 0 0 0
6 7 0 0 0 0 0 0 0
1 C C C C C C C C C C A
(15)
and the corresponding undirected graph in Fig. 7. The basic idea of nested dissection is to arrange the vertices of this graph into a rooted level structure, such as the one in Fig. 8. To achieve optimal results, this structure should have as many levels as possible, which is assured by choosing an appropriate vertex as the root (14,15). Once a rooted level structure is formed, the nested dissection algorithm identies the middle level (in this example, vertex 2) and removes it from the graph, so that two disconnected components are left. The middle level is then placed in the border, as illustrated in Eq. (16). In general, this proce-
5 1 2 3 7
Figure 6. The BBD matrix structure.
Level 0 6 Level 1 Level 2 4 Level 3 Level 4
Figure 8. A rooted level structure.
213
dure can be recursively repeated on each of the remaining blocks:
The Balanced Bordered Block-Diagonal Decompositions. The balanced BBD decomposition algorithm is recursive and consists of two basic steps. Step 1. Select a maximal allowable block size Nmax. Given this choice, move as many vertices as necessary to the border so that each block has size Nmax. A typical situation after this step is shown in Fig. 9. Step 2. The border is obviously too large after the rst step; consequently, in step 2 we reconnect border vertices one by one. In this process, the next vertex to be reconnected is always the one that results in the smallest increase in block sizes (such an algorithm is called greedy). The process continues as long as there are at least two blocks left (in other words, it terminates when we establish that the next reconnection will result in a single block). Once two diagonal blocks and an initial border have been obtained, steps 1 and 2 are repeated on each block (which makes the algorithm nested); the local borders are then added to the initial border. This procedure continues until we obtain a desired number of diagonal blocks of approximately equal size (hence the term balanced). Advantages of this algorithm are its execution speed and numerical simplicity, which result from the fact that only the sizes of blocks are considered, not their contents. In addition, since all diagonal blocks are balanced in size, the workload can be evenly distributed across different processors; this feature is critical for an efcient parallel solution of Eq. (1). To illustrate the effectiveness of this decomposition, in Figs. 10 and 11 we show how it is applied to a highly irregular matrix that arises in the modeling of the US electric power network. DECENTRALIZED CONTROL: AN EXAMPLE The underlying idea of decentralized control is decomposition. A dynamic system is considered as an interconnection of subsystems, which have independent inputs and outputs. A satisfactory performance of the overall system is achieved by controlling each individual subsystem using local feedback, whereby local inputs are connected to local outputs (or states).
----------------------
0 1 5 0 0 0 0 1B 0 0 0 C B C B C 6 B 0 0 0 C B ---------------------- C 3B 0 0 0 C B C B 4 B0 0 0 C C B C 7 @ 0 0 0 0A
-------------------
6 3
7 2
(16)
-----------------------0 0
The nested dissection algorithm was found to be very successful for matrices with a regular structure (such as those arising in the numerical solution of partial differential equations using nite elements). However, this decomposition is much less effective for systems such as electric circuits, where the matrix structure is typically highly irregular (4). Several different solutions where proposed to alleviate this problem, dating back to the work of Sangiovanni-Vincentelli et al. (2). We now describe two recent decomposition algorithms, which have been successful over a wide range of matrix structures. Decompositions Using Eigenvectors of Graphs. The idea of eigenvector decompositions was introduced by Pothen et al. (5), and is based on the Laplacian matrix of a graph. This matrix is dened as QDA (17)
where A is the adjacency matrix of the graph, and D is a diagonal matrix whose entries represent vertex degrees. It can be shown that matrix Q is always positive semidenite, with at least one zero eigenvalue; the smallest positive eigenvalue of Q is denoted 2, and the corresponding eigenvector is denoted by X2. The decomposition procedure can now be summarized as follows: 1. Compute eigenvector X2, and determine its median component xl. 2. Partition the vertices of the graph in the following way: for any vertex i, if xi xl, set i A; otherwise, i B. In this way, the vertices will be partitioned into two approximately equal sets, A and B. 3. All the edges connecting sets A and B constitute an edge separator, H. The objective now is to nd a minimal vertex cover for H (that is, the minimal number of vertices that need to be removed so that all edges in set H are removed). This vertex cover constitutes the separator, which appears in the border of the BBD structure. 4. Repeat steps 13 on the remaining components after the separator is removed. Decompositions based on eigenvectors of graphs were found to be effective and applicable to a wide range of matrix structures. However, for large matrices computing the second eigenvector can be difcult, if not impossible. This consideration has motivated the development of the balanced BBD decomposition (11,16), which is described next.
Figure 9. Situation after step 1.
214
0
m u1 u2
1000
1
k
l
2000
3000
Figure 12. Inverted pendulums.
4000
5000 0 1000 2000 3000 4000 5000
Figure 10. A model for the US electric power network (5300 5300).
-----------------
1, When the state and input vectors are chosen as x (1, T T 2, 2) and u (u1, u2) , the interconnected system is represented by the state equations 0 1 0 0 0 0 2 g 1 ka2 ka 0 0 0 2 2 2 ml ml l ml S: x = - - - - - - - - - - - - - - - - - - - - - - - - - - - - - x + - - - - - - - - - - - u 0 0 0 1 0 0 1 ka2 ka2 g 0 0 0 ml 2 ml 2 l ml 2
-----------------
To introduce the decentralized control problem, let us consider the two inverted pendulums interconnected by a spring as shown in Fig. 12. The control objective is to keep the pendulums upright by using the inputs u1 and u2. After linearization around the equilibrium state 1 2 0, the equations of motion are
(19)
In choosing the feedback control laws to achieve the objective, the information structure constraint is essential: Each 1)T input u1 and u2 can depend only on local states x1 (1, 2)T of each individual pendulum. In other words, and x2 (2, u1 = u1 (x1 ), u 2 = u 2 ( x2 ) (20)
1 = mgl 1 ka2 (1 2 ) + u1 ml 2 2 = mgl 2 ka2 (2 1 ) + u2 ml 2
(18)
Since the system S is linear, a reasonable choice is a linear control law u1 = kT 1 x1 , u2 = kT 2 x2 (21)
where feedback gain vectors k1 = (k11, k12 )T ,

1000
k2 = (k21 , k22 )T
(22)
need to be selected to stabilize S, that is, to keep the pendulums in the upright position. When the two pendulums are considered as two subsystems
2000
S1 : S2 :
x 1 = x 2 =
0 0
1 0 x + u 0 1 1 1 0 x + u 0 2 2
(23)
3000
the overall system breaks up into two interconnected subsystems as

4000
S:
x 1 = x 2 =
0 0
1 0 0 x + u +e 0 1 1 1 0 0 x + u +e 0 2 2
0 0 x +e 0 1 0 0 x +e 0 1
0 x 0 2 0 x 0 2 (24)
5000 0 1000 2000 3000 4000 5000
Figure 11. The matrix after a balanced BBD decomposition.
where g / l, 1/ ml2, a2k / ml2, and e (a / a)2.
215
By choosing the decentralized control of Eq. (21), we effectively intend to stabilize the interconnected system S by stabilizing the two subsystems in Eq. (23). This turns out to be a robust control strategy, since it can produce an overall closed-loop system that can remain stable despite the essential uncertainty about the height a of the spring. This uncertainty is represented by the normalized interconnection parameter e, which can take any value between 0 and 1. When an interconnected system is stabilized in this way, it is said to be connectively stable (6). By using the control law of Eq. (21) in Eq. (23) we obtain the closed-loop subsystems as
trices have proper dimensions, and N 1, 2, . . ., N. At present we are interested in disjoint decompositions, that is,
T T T x = ( xT 1 , x2 , . . ., xN ) T T T u = (uT 1 , u2 , . . ., uN )
(30)
y=
T T T ( yT 1 , y2 , . . ., yN )
where x(t) n, u(t) m, and y(t) l are the state, input, and output of the overall system S, such that
: S 1 : S 2
0 x 1 = k11 0 x 2 = k21
1 x k12 1 1 x k22 2
Rn = Rn 1 Rn 2 Rn N Rm = Rm 1 Rm 2 Rm N Rl = Rl 1 Rl 2 Rl N
x = AD x + BD u + AC x + BC u y = CD x + CC x where
(31)
(25)
A compact description of the interconnected system S is S: (32)
and the overall closed-loop system becomes
: S
0 x 1 = k11 x 2 = 0 k21
1 0 x +e k12 1 1 0 x +e k22 2
0 0 x +e 0 1 0 0 x +e 0 1
0 x 0 2 0 x 0 2 (26)
AD = diag{A1 , A2 , . . ., AN } BD = diag{B1 , B2 , . . ., BN } CD = diag{C1 , C2 , . . ., CN }

and the coupling block matrices are AC = (Aij ), BC = (Bij ), CC = (Cij ) (34)
(33)
It is interesting that the system in this example belongs to the class of interconnected systems that can always be stabilized by decentralized feedback. A decentralized control law can be chosen to connectively stabilize the closed-loop system even if the spring is shifting up and down the length of the S pendulums in an unpredictable way. The class of decentrally stabilizable systems is described next.
The collection of N decoupled subsystems is described by SD : x = A D x + BD u y = CD x (35)
INTERCONNECTED PLANTS AND CONTROLLERS To describe representations of plants, which are required in the design of decentralized controllers, let us consider a linear system S: x = Ax + Bu y = Cx as an interconnection (27)
which is obtained from Eq. (32) by setting the coupling matrices to zero. Important special classes of interconnected systems are input (BC 0) and output (CC 0) decentralized systems, where inputs and outputs are not shared among the subsystems. Inputoutput decentralized systems are described as S: x = AD x + BD u + AC x y = CD x (36)
S:
x i = A i xi + B i u i +
j N
(Aij x j + Bij u j ) iN (28)
where both BC and CC are zero. This structural feature helps to a great extent when decentralized controllers and estimators are designed for large plants. A static decentralized state feedback u = KD x is characterized by a block-diagonal gain matrix (37)
yi = Ci xi +
j N
Cij x j ,
of N subsystems Si : x i = A i xi + B i u i yi = Ci xi , iN (29)
KD = diag{K1 , K2 , . . ., KN }
(38)
which implies that each subsystem Si has its individual control law ui = Ki xi , iN (39)
where xi(t) ni, ui(t) mi, yi(t) li are the state, input, and output of the subsystem Si at a xed time t , all ma-
216
with a constant gain matrix Ki. The control law u of Eq. (37), which is equivalent to the totality of subsystem control laws of Eq. (39), obeys the decentralized information structure constraint requiring that each subsystem Si is controlled using its locally available state xi. The resulting closed-loop system is described as : S x = (AD BD KD )x + AC x (40)
yi and inputs ui, but are collectively responsible for the stabilization of S. The closed-loop system is
S&CD :
x A BKDC = z GD C
BHD FD
x z
(46)
The basic result of Ref. 17 states that the closed-loop system S&CD is stabilizable by decentralized controllers Ci if and only if the set of decentralized xed modes
When dynamic output feedback is used under decentralized constraints, then controllers of the following type are considered: Ci : z i = Fi zi + Gi yi ui = Hi zi Ki yi , iN (41)
=
KD
(A BKDC ) =
K1 ,..., KN
A
i N
KC B i i i
(47)
which can be rewritten in a compact form as a single decentralized controller dened as CD : z = FD z + GD y u = HD z KD y where (42)
lies in the open left half plane. This result includes the interconnected system of Eq. (44) in an obvious way. A simple characterization of decentralized xed modes was provided in Ref. 18. For any subset I i1, . . ., iP of the index set N , let I C j1, . . ., jNP denote the complement of I in N , and dene
I = [B ,B , . . ., B ], B i i i
1 2 P
c C I
z=
T T T (zT 1 , z2 , . . ., zN ) ,
y=
T T T ( yT 1 , y2 , . . ., yN )
u=
T T T (uT 1 , u2 , . . ., uN )
(43)
C j1 Cj 2 = . . . j C N P
(48)
are the state z r, input y l, and output u m of the controller CD. By combining the system S and the decentralized dynamic controller CD, we get the composite closed-loop system as
Then a complex number is a decentralized xed mode of S if and only if
rank
A I c C
I
I B 0
<n
(49)
S&CD :
x AD BD KDCD + AC = z GDCD
BD HD FD
x z
(44)
DECENTRALIZED FEEDBACK STRUCTURES Before a search for stabilizing decentralized feedback begins, it is necessary to determine whether such a feedback exists for a given plant. It is well known that if there are no restrictions on the information structure of the linear system S of Eq. (27), it can be stabilized if and only if the uncontrollable or unobservable modes of the system are stable. However, this is not the case when information constraints are present and the plant has unstable decentralized xed modes. Fixed Modes Let us consider the system S of Eq. (27) in the form
for some I N . Thus, the appearance of a xed mode can be attributed to a special polezero cancellation, which cannot be removed by constant decentralized feedback. However, under relatively mild conditions, such xed modes can be eliminated by time-varying decentralized feedback. Structurally Fixed Modes Graph theory serves as a suitable environment for both conceptual and numerical analysis of large-scale systems (6), because it allows the designer to take advantage of the special structural features of a large system before attempting a costly quantitative analysis. In particular, the theory is suitable for handling the lack of exact knowledge of system parameters by considering the existence rather than the true value of a connection between any two variables in the system (19). A graph-theoretic characterization of controllability was introduced by Lin (20). He established that in an uncontrollable pair (A, B), loss of controllability is either due to an insufcient number of nonzero parameters (indicating a lack of sufcient linkage among system variables), or due to a perfect matching of system parameters. In the former case, the pair (A, B) is structurally uncontrollable in the sense that all pairs having the same structure as (A, B) are uncontrollable. By describing the structure of (A, B) using a directed graph, structural controllability can be checked via efcient graphtheoretic algorithms (6). In this context, a pair (A, B) is structurally controllable if and only if:
S:
x = Ax +
i N
u B i i (45) iN
x, yi = C i
where only the inputs and outputs are partitioned as in Eq. (28), but the state (and thus the matrix A) is considered as a whole. Either the subsystems are ignored for technical reasons, or there are no natural or useful decompositions of the system into interconnected subsystems. In this case, the controllers Ci described in Eq. (41) still use local measurements
217
1. The system graph is input-reachable, that is, each state variable can be reached along a directed path by at least one input, and 2. The system graph has no dilation, that is, there exists no subset of state variables whose number exceeds the total number of all state and input variables directly affecting these variables. The concept of structurally xed modes represents a generalization of the idea of structural controllability (21). Let D (V , E ) be a directed graph associated with the system S of Eq. (27), where V U X Y is a set of vertices corresponding to inputs, states, and outputs of S, and E is a set of directed edges corresponding to nonzero elements of the system matrices A, B, and C. To every nonzero aij there corresponds an edge from vertex xj to vertex xi, to every nonzero bij an edge from uj to xi, and to every nonzero cij one from xj to yi. Given a feedback pattern K, which adds a feedback edge from yj to ui for every kij 1, one obtains a digraph DK (V , E E K) that completely describes the structure of both the system S and the feedback constraint specied by K, a special case of which is the decentralized constraint. In this case, permissible controllers have the structure
1. All state vertices of DK are covered by vertex disjoint cycles, and 2. No strong component of DK contains only state vertices, where a strong component is a maximal subgraph whose vertices are reachable from each other. How this graph-theoretic criterion can be used to choose a minimum number of feedback links that avoid structurally xed modes is explained in Refs. 6 and 22.
STABILIZATION It has been common practice to use decentralized control in the stabilization of large interconnected systems. Each subsystem is stabilized independently using local feedback, and stability of the overall system is established using the MatrosovBellman concept of vector Liapunov functions. While a vector Liapunov function may offer more exibility and computational simplicity than a scalar one, it remains inherently conservative. The use of vector functions, however, has been justied by the presence of uncertainty in the interconnections, which can cause a breakup of the system during operation along the subsystem boundaries. The method of vector Liapunov functions is a natural tool for making decentrally controlled systems robustly stable to interconnection failures, that is, connectively stable (6). Vector Liapunov Functions Let us assume that the plant is governed by linear time-invariant equations
N
CK :
z i = Fi zi +
j J i
gij y j kij yi
j J i
ui = hT i zi
(50)
where J i j : kij 1. If we choose a gain matrix K to conform with the feedback structure K, then the set
K
=
K
(A BKC )
(51)
S:
can be conveniently specied as the set of xed modes with respect to the decentralized feedback structure constraint dened by K. It is fairly easy to show (6) that the system S can be stabilized by the constrained controller CK if and only if K is contained in the open left half plane. To characterize K as in Eq. (49), let us consider the index sets I M 1, 2, . . ., M and replace I C by J iI C J i, where I C now refers to the complement of I in M . In order to formulate graph-theoretic conditions for the existence of structurally xed modes, let us recall that two systems are said to be structurally equivalent if they have the same graph. Then, a system S is said to have structurally xed modes with respect to a given feedback structure K if every system structurally equivalent to S has xed modes with respect to K. Structurally xed is a generic concept: Having structurally xed modes is a property of a class of systems sharing the same graph. Most importantly, if a system has no structurally xed modes, then it either has no xed modes, or the xed modes can be removed by arbitrarily small perturbations of system parameters. This means that if a system has no structurally xed modes for a given K, then (generically) it can be stabilized by a controller with a gain K and structure K, decentralized structure being a special case. It was shown in Ref. 21 that a system S has no structurally xed modes with respect to a feedback pattern K if and only if:
x i = A i xi + B i u i +
j=1
eij Aij x j ,
iN
(52)
which are an obvious derivative of Eq. (28) save for the insertion of the interconnection parameters eij [0, 1]. The parameters are coefcients of the N N interconnection matrix E (eij), which are used to model the uncertain strength of interconnections. To stabilize S we use the decentralized control laws of Eq. (39) to stabilize each individual closed-loop system : S i x i = (Ai Bi Ki )xi , iN (53)
This is presumably a relatively easy task, because the subsystems have low dimensions. Stability of each decoupled closedloop system follows from a subsystem Liapunov function vi : ni . These individual functions are then stacked up to form a vector Liapunov function v(x) [v1(x1), v2(x2), . . ., vN(xN)]T. Finally, the vector Liapunov function is used to form a scalar Liapunov function V : n of the form V (x ) = d T v (54)
where d (d1, d2, . . ., dN)T is a positive vector (di 0, i N ). The Liapunov function V(x) is utilized to establish stabil-
218
ity of the overall closed-loop system
for the system S of Eq. (27). By applying the epsilon decomposition to M and regrouping the variables, we get S as
: S
x + x i = A i i
eij Aij x j ,
j=1
iN
(55)
N
S:
xi = A i xi + B i u i +
j=1 j =i
Aij x j + Bij u j
iN (61)
i Ai BiKi. where A , Taking the total time derivative of V(x) with respect to S after lengthy but straightforward computations (6), we get (x ) d T W z V
S
(56)
By applying the decentralized feedback of Eq. (39), we obtain the closed-loop system as
where z (x1, x2, . . ., xN)T, and W (wij) is the aggregate matrix dened as
: S
x + x i = A i i
N j=1 j =i
x , A ij j
iN
(62)
wij =
i eii ii , i = j eij ij , i= j
(57)
ij Aij BijKj. In this case, the aggregate matrix where A W (wij) is dened as
The positive numbers i depend on the choice of vis, and the nonnegative numbers ij are bounds on interconnection matrices Aij. The binary numbers eij are elements of the N N fundamental interconnection matrix E (eij), which dene the nominal structure of S,
wij =
i ,
i= j
ij , i = j
(63)
eij =
1 if S j acts on Si 0 if S j does not act on Si
(58)
We recall (6) that if the threshold is sufciently small, then W is an M matrix: The smaller the absolute values of wijs, the easier it is for W to satisfy the M-matrix conditions. We note, however, that the smaller is, the smaller the number of subsystems will be, implying a smaller reduction in the dimensionality of the stability problem via decomposition. Decentrally Stabilizable Systems Vector Liapunov functions provide only sufcient conditions for stability of interconnected systems, and one may search in vain for stabilizing control. For this reason, there are a number of results aimed at identifying classes of interconnected systems which can always be stabilized by decentralized feedback. The most popular (but also the most restrictive) conditions for decentral stabilizability are the matching conditions (23), Im Bi Im Aij (64)
It is a well-known result (6,19) that: is connectively stabilized by The interconnected system S decentralized control ui Kixi, i N , if the aggregate matrix W is an M matrix. A matrix W (wij) with nonpositive off-diagonal elements is an M matrix if and only if there exists a positive vector d such that the vector c=W d
T
(59)
follows by the is a positive vector as well, and stability of S (x)S standard Liapunov theorem involving V(x) 0 and V 0. The connective property of stability, which requires stability to hold for all E E, is concluded from the fact that W(E) W(E) element by element, and W is an M matrix whenever W is. Epsilon Decompositions Epsilon decompositions are ideal preconditioners for the stabilization of large-scale systems using decentralized control and vector Liapunov functions (6,9). To see this, assume that C 0 and form the augmented matrix
These conditions simply imply that the interconnections may be made to enter the subsystem through the input matrices Bi, that is, we have
N
S:
x i = A i xi + B i
ui +
j=1
eij Dij x j
iN
(65)
which means that Aij BiDij for some matrices Dij. Nonmatching conditions for decentral stabilizability of single-input subsystems have been considered in Ref. 24. The system S is described as S: x i = A i xi + b i u i +
j N
eij Aij x j ,
iN
(66)
M=
A 0
B 0
(60)
where, without loss of generality, the subsystem pairs (Ai, bi) are assumed to be in the controllable canonical form. For each
219
interconnection matrix Aij, dene an integer
mij =
max{q p : aij pq = 0}, Aij = 0 n, Aij = 0
(67)
Thus, mij is the distance between the main diagonal and a line parallel to the main diagonal that borders all nonzero elements of Aij. Given an index set I N , let J denote any permutation of I . Then, the following holds: The system S of Eq. (66) is stabilizable by decentralized state feedback ui kT i xi, i N , if
(mij 1 ) < 0
i I j J
(68)
for all I and all permutations J . In the special case of matching conditions, mij nj ni. The condition in Eq. (68) was obtained via vector Liapunov functions, and therefore guarantees decentral stabilizability even when the elements of the interconnection matrices Aij are bounded nonlinear time-varying functions of the state x. There are a large number of results that characterize wider classes of decentrally stabilizable systems (e.g., Refs. 2427), most of them surveyed in Ref. 6. Some of the results broadened the scope of the given framework to include multivariable systems, time delays in the interactions, time-varying models, and stochastic perturbations. Especially interesting are schemes for decentralized output control, which utilize dynamic output controllers or decentralized estimators for reconstruction of the state of the overall system from the subsystem state estimates. Robustness. There are many useful and interesting properties of systems stabilized by decentralized feedback. First, stability of decentrally controlled systems can tolerate nonlinearities in the interconnections; the nonlinear interconnections need not be known, since only their size must be limited. is shown to be stable, it autoOnce the closed-loop system S matically follows (6) that any nonlinear time-varying version : S N x + h (t , x ), x i = A i i i iN (69)
ized control have been obtained by Chen et al. (32,33), Zhang et al. (34), and Hassan et al. (35). Since the introduction of multiple control system concepts for reliable stabilization (36), there have been numerous papers dealing with controller or sensor/actuator failures. Solutions to this problem in the algebraic setting were given by zgu Vidyasagar and Viswanadham (37), O ler (38), Tan et al. (39), and Gu ndes and Kabuli (40); relying on adaptation, by Cho et al. (41); using H , by Veillette et al. (42), Medanic (43), and Park and Bien (44); and in the context of linear quadratic control by Veillette (45). An application of the multiple control system concept to reliable control of steam generators was offered by Wu and Lin (46). There are a number of interesting recent results concerning xed-gain designs of decentralized control for interconnected systems. In Ref. 47 a stabilization scheme was proposed in the parameter space of linear systems, which was formulated in the standard framework of convex programming. This opened up the possibility of using a variety of concepts and algorithms available in the linear matrix inequalities approach (48) for robust decentralized designs. A promising result is the decentralized quadratic stabilization of interconnected systems (49), which can be reduced to an H control problem (see also Refs. 50 and 51). There is a large body of literature on decentralized control design using frequency-domain methods, which is based on the work of Rosenbrock (52). The initial results were obtained in Ref. 53 and further developed in Refs. 5456 using the Mmatrix theory and block-diagonal dominance. Parametrization of decentralized stabilizing controllers by the fractional representation approach was considered in Refs. 5760. Finally, we should mention a number of books (6165) where a wide variety of useful methods and techniques can be found for the design of stabilizing decentralized control.
ADAPTIVE CONTROL Decentrally stabilizable systems with suitable interconnection structures can (always) be stabilized by using local states or outputs, often employing high-gain feedback. How high the gains should be depends on how strong the interconnections are. More often than not, it is difcult, or impossible, to predict and limit the size of the coupling among the subsystems, implying that xed-gain controllers may be incapable in stabilizing the system. In such cases, one has to use adaptive controllers, which can adjust the gains to values needed for overall stability (66). Let us consider a single-input, single-output version of S,
is connectively stable, provided the conical constraints of S

N
hi (t , x )
j=1
eij ij x j ,
iN
(70)
on the interconnection functions hi : n ni hold for all (t, x) n. This robustness result is useful in practice because, typically, either interconnections are poorly known, or they change during the operation of the controlled system. Obviously, the result includes the case when the interconnection parameters eij are considered as nonlinear time-varying functions eij : n [0, 1]. Connective decentralized stabilization was considered by Tan and Ikeda (28) and, when applied to robotic manipula and Vukobratovic (29), Mills and Goldenberg tors, by Stokic (30), and Mills (31). New results in robust design of decentral-
S:
x i = Ai xi + bi ui + Pi vi yi = c T i xi w i = Qi xi , iN (71)
where xi ni, ui , and yi are the state, input, and output of the subsystem Si, and vi mi and wi li are the interconnection inputs and outputs of Si from and to other subsystems Sj, j N , which are related as vi = hi (t , w ), iN (72)
220
The crucial restriction on the functions hi : l mi is that they are bounded:

N
hi (t , w )
j=1
ij w j
(73)
where ij are nonnegative, but unknown, numbers. The matrices Ai and vectors bi and ci dening the subsystem Si are not specied, except for the fact that pairs (Ai, bi) are controllable and that pairs (Ai, ci) are observable. The uncertainty about the triples (Ai, bi, ci) compounds the essential uncertainty about the overall system S caused by our ignorance of the interconnections. The control objective is to force each state xi(t) of Si to track the state xmi(t) of the corresponding reference model
Mi :
x mi = Ami xmi + bmi ri ymi = cT mi xmi , iN
(74)
where ri is the external (reference) input. To achieve this objective, we assume that there exist subsets I N and J N I such that
Pi = bi pT i , Qi = q i cT i ,
iI i J
(75)
for some constant vectors pi mi and qi li, which are matching conditions requiring that either incoming disturbances enter through the control channel or outgoing disturbances pass through the measurement channel. These conditions, as shown in Ref. 67, make the system stabilizable by high-gain decentralized feedback, and are crucial for the adaptation scheme to work. The basic requirement of the adaptive regulator is to drive the state of the overall system S to zero. For this purpose, the local control laws are chosen as ui = iT xi , iN (76)
where i ni is the time-varying adaptation gain vector. To arrive at suitable adaptation laws i(t), we choose Amis as (satisfactorily) stable constant matrices and set i = Ri (kT i x i )x i , iN (77)
where Ri and ki are appropriate constant matrices and i(t0) is nite. Finally, the closed-loop system is
: S
x i = (Ami + bi iT )xi + bi pT i hi i = Ri (kT i x i )x i , iN
(78)
where i ni is the ith parameter adaptation error dened as i i * i , and * i is a model-matching constant parameter vector. We denote the solutions of the closed-loop system T T S by (x, )(t; t0, x0, 0), where x (xT 1 , x2 , . . ., xN) and T T T (T 1 , 2 , . . ., N) . Relying on the decentral stabilizability con iljak (66) established the followdition of Eq. (75), Gavel and S ing basic result: are globally bounded, The solutions (x, )(t; t0, x0, 0) of S and x(t; t0, x0, 0) 0 as t .
The adaptive stabilizability conditions, which are based on structural restriction in Eq. (75), allow the controller gains to rise to whatever level is necessary to ensure that stability of the subsystem overrides the perturbations caused by interconnection uctuations, so long as they are nite. The boundedness part of the above result ensures the boundedness of the adaptation gains and therefore the realizability of the decentralized adaptive control scheme. The above basic result has many extensions. In the statetracking problem, the state x(t) of the plant S follows the state xm(t) of the reference model M despite the change in the size and shape of interconnections. Furthermore, the steadystate tracking error e(t) x(t) xm(t) can be made as small as desired. When the subsystem models are known, which is often the case in practice, the adaptive decentralized scheme extends to multiinput, multioutput subsystems with added simplicity in implementation; the scheme requires only one adaptation parameter per subsystem. Under relatively mild conditions the scheme can accommodate output feedback controllers as well. Since the initial work on adaptive decentralized control by Hmamed and Radouane (68), a wide variety of schemes have been developed. Unmodeled dynamics with fast and slow modes in the subsystems was considered by Ioannou and Ko (69). The use of inputoutput models was initiated by kotovic Wiemer and Unbehauen (70) for a discrete-time problem of decentralized adaptive control. A state-space approach to the same problem was proposed by Reed and Ioannou (71) as well as Kamoun et al. (72) and Yang and Papavassilopoulos (73). Motivated by models of mechanical systems, Shi and Singh (74,75) considered higher-order interconnections with polynomial bounds. Poor transient behavior of the standard adaptive schemes, which is caused by insufcient knowledge of subsystem parameters, can be improved by exchanging the output signals between individual subsystems and assuming weak coupling (76). A major drawback of the early adaptive decentralized schemes was the relative-degree restriction on the subsystems. This restriction was rst removed by Ortega and Herrera (77) by using the concept of higher-order tuning. The same result was later achieved by Wen (78) by applying the integrator backstepping procedure. The original scheme of iljak (66) was extended by Lyon (79) to the case Gavel and S when the relative order of each subsystem does not exceed two. Due to the high-gain nature of the adaptive scheme proposed in Refs. 66, 80, a variable-structure decentralized control (81,82) became a logical candidate for improving the design. Sasaki et al. (83) developed a variable-structure version of their decentralized adaptive scheme for control of distributed systems. Inputoutput models with relay-type controllers were used by Brusin and Ugrinovskaya (84); see also Refs. 85 and 86. In Refs. 87 and 88, coordinate transformations have been utilized to broaden the class of systems that can be stabilized by decentralized adaptive control. Indirect adaptive schemes have been proposed by Wen (89) and Spooner and Passino (90). Finally, a partially decentralized adaptive control has been developed in Ref. 91. OVERLAPPING DECENTRALIZED CONTROL In a wide variety of natural and industrial systems, subsystems share common parts (6). In these cases, for either con-
221
ceptual or computational reasons it is advantageous to use overlapping decentralized control. We will use the overlapping epsilon decomposition to provide a justication for such a practice. Let us consider a linear constant system S: x = Ax + Bu (79)
-------
where x n, u m, and the block matrices A13 0 A11 A12 B11 0- - - 0 A = A21 A22 A23 , B= ------------0 B32 A31 A32 A33
(80)
are decomposed along the dashed lines dening the two overlapping subsystems. By using a linear transformation x = Vx where V is the n n matrix I1 0 V = 0 0 (81)
of It is crucial to note from Eq. (87) that the expansion S Eq. (86) has an epsilon decomposition, while the original system S of Eq. (80) does not. Decentralized stabilization schemes can now take advantage of the weak coupling among the two subsystems, and use local feedback to stabilize the by independent stabilization of the two subsysexpansion S tems. It is easy to show that stability of the expanded system implies stability of the original system S. S The circle of ideas and methods surrounding the overlapping decompositions and control structures have been organized into a general mathematical framework known as the inclusion principle (6). In a wide variety of problems arising in the theory and practice of large-scale systems, the principle produced interesting conceptual insights and useful solution procedures (9298).
------
---------
OPTIMIZATION Despite considerable efforts and a large number of new results in the theory of large complex systems, the fundamental problem of optimal decentralized control has remained unsolved for over two decades. The simple reason has been the fact that the decentralized information structure constraints have not been successfully incorporated into any of the standard optimization frameworks. Neither Pontryagins maximum principle nor Bellmans dynamic programming can handle the lack of complete state observation, a difculty recognized by Fleming (99) as far back as the late sixties. For this reason, there have been a large number of results relying on pragmatic suboptimality concepts, which have capitalized on effective solutions of robustness issues in the suboptimality framework (6). The standard practice has been to optimize each decoupled subsystem using linear quadratic (LQ) control laws. Then, suboptimality of the interconnected closed-loop system, which is driven by the union of the locally optimal LQ control laws, is determined with respect to the sum of the quadratic costs chosen for the subsystems. Under relatively mild conditions, suboptimality implies stability. Furthermore, the degree of suboptimality, which is computed with respect to the globally optimal union of decoupled subsystems, can serve as a measure of robustness with respect to a wide spectrum of uncertainties residing in both the subsystems and their interactions. It has been shown (100) how the classical measures of gain and phase margins, as well as the gain reduction tolerance, can be incorporated in the decentralized LQ control of large interconnected systems. Recently, a solution to the optimal decentralized stochastic control problem has been offered in the classical optimization framework of Lagrange (101) relying on the constrained optimization approach proposed in Ref. 102. The principal idea is to redene the information structure constrains as differential equations, and attach Lagrange multipliers to each constraint to obtain sufcient as well as necessary conditions for optimality of decentralized control laws. The multipliers are functions of time and state, and, because of the role they play in the optimization process, they are termed the Lagrange Liapunov multipliers. The sufcient conditions are formulated in terms of the HamiltonJacobi-like equations and are proved by using the method of global optimization (103). In contrast, the necessary conditions of optimality, which are de-
0 I2 I2 0
0 0 0 I3
(82)
and the identity matrices are compatible with the blocks of A, we get the expansion of S as : S x x =A + Bu (83)
where x n ,n n1 2n2 n3, and n1, n2, n3 are dimensions of the square matrices A11, A22 and A33. The system matrices are
= VAU + M, A and
= VB + N B
(84)
0 0 0 I1 1 U = 0 1 I I 0 2 2 2 2 0 0 0 I3 1 1 A 2 A12 0 2 12 0 1 A 1 2 22 2 A22 M= 1 1 0 2 A22 2 A22 1 1 0 2 A32 2 A32 N=0
0 0 0 0
(85)
of Eq. (83) has the form Then, the expansion S 0 A13 0 A11 A12 B11 A 0 0 0 A 21 A22 : ----------- - - - - - - - - - - - - - - - - - -23 - x S x = - - - + u (86) A21 0 0 0 A22 A23 0 B32 A31 0 A32 A33 where the dashed lines delineate two disjoint subsystems, so as that we can write the expansion S
: S
-------
-------
B A 0 A 11 - - - - - - - 12 -- x + - - 11 x = ---------- u 0 B A A 22 21 22
------------
------------
(87)
222
LARGE-SCALE AND DECENTRALIZED SYSTEMS 3. A. George and J. W. H. Liu, Computer Solution of Large Sparse Positive Denite Systems, Englewood Cliffs, NJ: Prentice-Hall, 1981. 4. I. S. Duff, A. M. Erisman, and J. K. Reid, Direct Methods for Sparse Matrices, Oxford: Clarendon Press, 1986. 5. A. Pothen, H. D. Simon, and K. P. Liou, Partitioning sparse matrices with eigenvectors of graphs, SIAM J. Matrix Anal. Appl., 11: 430452, 1990. iljak, Decentralized Control of Complex Systems, Boston: 6. D. D. S Academic Press, 1991. 7. A. George, J. Gilbert, and J. W. H. Liu (eds.), Graph Theory and Sparse Matrix Computation, New York: Springer-Verlag, 1993. iljak, Balanced decompositions of sparse evic and D. D. S 8. A. I. Zec systems for multilevel parallel processing, IEEE Trans. Circuits Syst., 41: 220233, 1994. iljak, Nested decompositions and clus9. M. E. Sezer and D. D. S tering of complex systems, Automatica, 22: 321331, 1986. iljak, Nested epsilon decompositions of 10. M. E. Sezer and D. D. S linear systems: Weakly coupled and overlapping blocks, SIAM J. Matrix Anal. Appl., 12: 521533, 1991. iljak, A block-parallel Newton method evic and D. D. S 11. A. I. Zec via overlapping epsilon decompositions, SIAM J. Matrix Anal. Appl., 15: 824844, 1994. 12. L. A. Hageman and D. M. Young, Applied Iterative Methods, New York: Academic Press, 1981. 13. H. H. Happ, Piecewise Methods and Applications to Power Systems, New York: Wiley, 1980. 14. N. E. Gibbs, W. G. Poole, and P. K. Stockmeyer, An algorithm for reducing the bandwidth and prole of a sparse matrix, SIAM J. Numer. Anal., 13: 236250, 1976. 15. J. G. Lewis, Implementation of the GibbsPooleStockmeyer and GibbsKing algorithms, ACM Trans. Math. Software, 8: 180189, 1982. iljak and A. I. Zec evic , A nested decomposition algorithm 16. D. D. S for parallel computations of very large sparse systems, Math. Prob. Eng., 1: 4157, 1995. 17. S. H. Wang and E. J. Davison, On the stabilization of decentralized control systems, IEEE Trans. Autom. Control, 18: 473 478, 1973. 18. B. D. O. Anderson and D. J. Clements, Algebraic characterization of xed modes in decentralized control, Automatica, 17: 703712, 1981. iljak, Large-Scale Dynamic Systems: Stability and Struc19. D. D. S ture, New York: North-Holland, 1978. 20. C. T. Lin, Structural controllability, IEEE Trans. Autom. Control, 19: 201208, 1974. iljak, Structurally xed modes, Sys. Con21. M. E. Sezer and D. D. S trol Lett., 1: 6064, 1981. iljak, Decentralized control, in W. S. 22. M. E. Sezer and D. D. S Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, 1996, pp. 779793. 23. G. Leitmann, One approach to the control of uncertain systems, ASME J. Dynam. Syst. Meas. and Control, 115: 373380, 1993. iljak, On decentrally stabilizable large24. M. Ikeda and D. D. S scale systems, Automatica, 16: 331334, 1980. iljak, On decentralized stabilization and 25. M. E. Sezer and D. D. S structure of linear large-scale systems, Automatica, 17: 641 644, 1981. 26. Z. C. Shi and W. B. Gao, Stabilization by decentralized control for large-scale interconnected systems, Large Scale Syst., 10: 147155, 1986. 27. G. H. Yang and S. Y. Zhang, Decentralized robust control for interconnected systems with time-varying uncertainties, Automatica, 32: 16031608, 1996.
rived very much in the spirit of the Flemings result, require the solution of a two-point boundary value problem with a minimum condition of the Pontryagin type. Let us present the main idea of this new development with a minimum of technical elements. A system S, which is composed of N interconnected subsystems, is described by a stochastic differential equation of Itos type,
S:
dxi = +
Aii (t )xi + Bi (t )ui (t , x )

n
(88) dt + Ci (t , x ) dwi , iN
Aij (t )x j
j =i
With the system S we associate a quadratic cost
J=
1 E 2
N i= 1
t1 t0
T [x T i Qi (t )xi + ui Ri (t )ui ] dt
(89)
The design objective is to determine a control u(t, x) that drives the system S optimally with respect to cost J and, at the same time, satises the essential information restriction that the control law ui at each individual subsystem can utilize only the locally available state xi. More precisely, we have the following problem: Determine a control u(t, x) that minimizes the functional J with respect to the system S under the decentralized information structure constraints ui = ui (t , xi ), iN (90)
By reformulating the constraints in Eq. (90) as differential equations ui (t , x ) = 0, x i iN (91)
where the vector xi is the state vector x without the state xi of the ith subsystem, we can append Eq. (91) to the equations of motion in Eq. (88) and use the standard Lagrange optimization framework in the manner suggested by Pontryagin in his maximum principle and by Bellman in his version of the HamiltonJacobi equation. When the additional assumption of Gaussian state evolution is added, the optimality conditions obtained in the new approach provide a feedback structure for decentralized control, which involves Riccati-type equations in the same way as in the classical regulator theory of Kalman (104). This fact is expected to play a major role in applications of optimal decentralized control to complex interconnected systems. BIBLIOGRAPHY
1. H. H. Happ, Diakoptics and Networks, New York: Academic Press, 1971. 2. A. Sangiovanni-Vincentelli, L. K. Chen, and L. O. Chua, An efcient heuristic cluster algorithm for tearing large scale networks, IEEE Trans. Circuits Syst., 24: 709717, 1977.
LARGE-SCALE AND DECENTRALIZED SYSTEMS 28. X. L. Tan and M. Ikeda, Decentralized stabilization for expanding construction of large-scale systems, IEEE Trans. Autom. Control, 35: 644651, 1990. and M. Vukobratovic , Practical stabilization of robotic 29. D. Stokic systems by decentralized control, Automatica, 20: 353358, 1984. 30. J. K. Mills and A. A. Goldenberg, Global connective stability of a class of robotic manipulators, Automatica, 24: 835839, 1988. 31. J. K. Mills, Stability of robotic manipulators during transition to and from compliant motion, Automatica, 26: 861874, 1990. 32. Y. H. Chen and M. C. Han, Decentralized control design for interconnected uncertain systems, in C. T. Leondes (ed.), Control and Dynamic Systems, Boston: Academic Press, 1993, pp. 219266. 33. W. J. Wang and Y. H. Chen, Decentralized robust control design with insufcient number of controllers, Int. J. Control, 65: 1015 1030, 1996. 34. S. Y. Zhang, K. Mizukami, and H. S. Wu, Decentralized robust control of uncertain large-scale interconnected nonlinear dynamical systems, J. Optim. Theory Appl., 91: 235256, 1996. 35. M. F. Hassan et al., Expert robust decentralized controller for uncertain large-scale systems, IEE Proc. Control Theory Appl., 143: 519529, 1996. iljak, Reliable control using multiple control systems, Int. 36. D. D. S J. Control, 31: 303329, 1980. 37. M. Vidyasagar and N. Viswanadham, Reliable stabilization using a multicontroller conguration, Automatica, 21: 599602, 1985. zgu 38. A. B. O ler, Decentralized control: A stable proper fractional approach, IEEE Trans. Autom. Control, 35: 11091117, 1990. iljak, and M. Ikeda, Reliable stabilization via 39. X. L. Tan, D. D. S factorization methods, IEEE Trans. Autom. Control, 37: 1786 1791, 1992. 40. A. N. Gu ndes and M. G. Kabuli, Reliable stabilization with integral action in decentralized control systems, Proc. 34th IEEE CDC, New Orleans, 1995, pp. 26472652. 41. Y. J. Cho, Z. Bien, and B. K. Kim, Reliable control via an adaptive redundant controller, Int. J. Control, 50: 385398, 1989. , and W. R. Perkins, Design of reli42. R. J. Veillette, J. V. Medanic able control systems, IEEE Trans. Autom. Control, 37: 290 304, 1992. , Design of reliable controllers using redundant 43. J. V. Medanic control elements, Proc. ACC, San Francisco, 1993, pp. 3130 3134. 44. S. H. Park and Z. Bien, Design of reliable control systems for uncertain linear systems, Proc. Asian Control Conf., Tokyo, 1994, pp. 101104. 45. R. J. Veillette, Reliable linear-quadratic state-feedback, Automatica, 31: 137143, 1995. 46. W. Wu and C. Lin, Optimal reliable control system design for steam generators in pressurized water reactors, Nucl. Technol., 106: 216224, 1994. 47. J. C. Geromel, J. Bernussou, and P. L. D. Peres, Decentralized control through parameter space optimization, Automatica, 30: 15651578, 1994. 48. S. Boyd et al., Linear Matrix Inequalities in System and Control Theory, Philadelphia: SIAM, 1994. 49. G. K. Zhai and M. Ikeda, Decentralized H control of large-scale systems, Proc. 32nd IEEE CDC, San Antonio, 1993, pp. 1652 1653. 50. R. A. Date and J. H. Chow, A parametrization approach to optimal H2 and H decentralized control problems, Automatica, 29: 457463, 1992.
223
51. M. Ikeda, Decentralized H controller design for large-scale systems, Proc. 35th IEEE CDC, Kobe, Japan, 1996, pp. 16. 52. H. H. Rosenbrock, Computer-Aided Control System Design, New York: Academic Press, 1974. 53. M. Araki and O. I. Nwokah, Bounds for closed-loop transfer functions of multivariable systems, IEEE Trans. Autom. Control, 20: 666670, 1975. 54. W. H. Bennett and J. S. Baras, Decomposition and decentralized control system design: A review of frequency domain methods, Proc. 24th IEEE CDC, Ft. Lauderdale, 1985, pp. 18281835. iljak, and T. Matsumoto, Decentralized control 55. Y. Ohta, D. D. S using quasi-block diagonal dominance of transfer function matrices, IEEE Trans. Autom. Control, 31: 420430, 1986. 56. J. H. Seo, A frequency domain method for decentralized stabilization based on quasi-block diagonal dominance, Int. J. Control, 64: 4159, 1996. 57. A. N. Gu ndes and C. A. Desoer, Algebraic Theory of Linear Feedback Systems with Full and Decentralized Compensators, Heidelberg: Springer-Verlag, 1990. zgu 58. A. B. O ler, Linear Multichannel Control: A System Matrix Approach, UK: Prentice-Hall Int., 1994. 59. V. Manousiouthakis, On the parametrization of all decentralized stabilizing controllers, Sys. Control Lett., 21: 397403, 1993. 60. D. D. Sourlas and V. Manousiouthakis, Best achievable decentralized performance, IEEE Trans. Autom. Control, 40: 1858 1871, 1995. 61. M. Jamshidi, Large-Scale Systems: Modeling, Control, and Fuzzy Logic, Upper Saddle River, NJ: Prentice-Hall, 1997. 62. J. Lunze, Feedback Control of Large-Scale Systems, Englewood Cliffs, NJ: Prentice-Hall, 1992. 63. H. Tamura and T. Yoshikawa, Large-Scale Systems Control and Decision Making, New York: Marcel Dekker, 1990. 64. C. T. Leondes (ed.), Control and Dynamic Systems, Orlando, FL: Academic Press, 1986, vol. 22, 1985; vols. 23, 24. 65. M. S. Mahmoud, M. F. Hassan, and M. G. Darwish, Large-Scale Control Systems, New York: Marcel Dekker, 1985. iljak, Decentralized adaptive control: 66. D. T. Gavel and D. D. S Structural conditions for stability, IEEE Trans. Autom. Control, 34: 413426, 1989. iljak, Robust decentralized 67. O. Huseyin, M. E. Sezer, and D. D. S control using output feedback, IEE Proc., D-129: 310315, 1982. 68. A. Hmamed and L. Radouane, Decentralized nonlinear adaptive feedback stabilization of large-scale interconnected systems, IEE Proc., D-130: 5762, 1983. , Decentralized adaptive control 69. P. Ioannou and P. V. Kokotovic of interconnected systems with reduced-order models, Automatica, 21: 401412, 1985. 70. P. Wiemer and H. Unbehauen, On the stability of decentralized adaptive control systems, Int. J. Adaptive Control Signal Process., 4: 116, 1990. 71. J. S. Reed and P. Ioannou, Discrete-time decentralized adaptive control, Automatica, 24: 419421, 1988. 72. M. Kamoun, A. Titli, and M. B. Kamoun, Robust decentralized model reference adaptive control of discrete-time interconnected systems, Int. J. Control, 37: 18411856, 1992. 73. W. Y. Yang and G. P. Papavassilopoulos, Decentralized adaptive control in a game situation for discrete-time, linear, time-invariant systems, Proc. ACC, Baltimore, 1994, pp. 22943399. 74. L. Shi and S. K. Singh, Decentralized adaptive controller design for large-scale systems with higher order interconnections, IEEE Trans. Autom. Control, 37: 11061118, 1992. 75. S. K. Singh and L. Shi, Robust adaptive controllers for interconnected mechanical systems: Inuence of types of interconnec-
224
LASER APPLICATIONS IN MEDICINE LQG control, Proc. 13th IFAC World Congr., San Francisco, 1996, L: pp. 712. iljak, Decentralized , M. Stanojevic , and D. D. S 98. S. S. Stankovic suboptimal LQ control of a platoon of vehicles, Proc. 8th IFAC Symp. Transportation Sys., Chania, Greece, 1997, pp. 8186. 99. W. Fleming, Optimal control of partially observable diffusions, SIAM J. Control, 6: 194214, 1968. iljak, Sensitivity of large-scale control 100. M. E. Sezer and D. D. S systems, J. Franklin Inst., 312: 170197, 1981. iljak, Optimal decentralized control 101. S. V. Savastyuk and D. D. S for stochastic dynamic systems, in R. P. Agarwal (ed.), Recent Trends in Optimization Theory and Applications, Singapore: World Scientic, 1995, pp. 337352. 102. M. M. Khrustalev and S. V. Savastyuk, Conditions for optimality of stochastic systems of diffusion type in problems with constraints on the controlobservation process, Sov. Math. Dokl., 41: 256260, 1990. 103. V. F. Krotov, A technique of global bounds in optimal control theory, Control Cybernet., 17: 115144, 1988. 104. R. E. Kalman, Contributions to the theory of optimal control, Bul. Soc. Mat. Mex., 5: 102119, 1960.
ILJAK DRAGOSLAV D. S EVIC ALEKSANDAR I. ZEC
tions on time-invariant and time-varying systems, ASME J. Dynam. Syst. Meas. Control, 116: 456473, 1994. 76. A. Datta, Performance improvement in decentralized adaptive control: A modied model reference scheme, IEEE Trans. Autom. Control, 38: 17171722, 1993. 77. R. Ortega and A. Herrera, A solution to decentralized adaptive stabilization problem, Syst. Control Lett., 13: 229306, 1993. 78. C. Wen, Decentralized adaptive regulation, IEEE Trans. Autom. Control, 39: 21632166, 1994. 79. J. Lyon, Note on decentralized adaptive controller design, IEEE Trans. Autom. Control, 40: 8991, 1995. 80. R. Wei, M. E. Sezer, and O. Ocali, Robust adaptive stabilization of a class of systems under structured nonlinear perturbations with application to interconnected systems, Int. J. Control, 63: 183194, 1996. 81. H. Khurana, S. I. Ahson, and S. S. Lamba, On stabilization of large-scale control systems using variable structure system theory, IEEE Trans. Autom. Control, 31: 176178, 1988. 82. G. P. Matthews and R. A. DeCarlo, Decentralized tracking for a class of interconnected nonlinear systems using variable structure control, Automatica, 24: 187193, 1988. 83. M. Sasaki et al., Finite-dimensional decentralized VS-MRAC of non-linear interconnected distributed parameter systems, Int. J. Syst. Sci., 24: 11651188, 1993. 84. V. A. Brusin and E. Yu, Ugrinovskaya, Decentralized adaptive control with a reference model, Avtom. i Telemekhan., 10: 29 36, 1992. 85. A. C. Wu and L. C. Fu, New decentralized MRAC algorithms for large-scale uncertain systems, Proc. ACC, Baltimore, 1994, pp. 33893393. 86. G. Feng and Y. A. Jiang, Variable structure based decentralized adaptive control, Proc. ACC, Baltimore, 1994, pp. 34093410. 87. S. Jain and F. Khorrami, Decentralized adaptive control of a class of large-scale interconnected nonlinear systems, IEEE Trans. Autom. Control, 42: 136154, 1997. 88. S. Jaoin and F. Khorrami, Decentralized adaptive output feedback design for large-scale nonlinear systems, IEEE Trans. Autom. Control, 42: 729735, 1997. 89. C. Wen, Indirect robust totally decentralized adaptive control of continuous-time interconnected systems, IEEE Trans. Autom. Control, 40: 11221126, 1995. 90. J. T. Spooner and K. M. Passino, Adaptive control of a class of decentralized nonlinear systems, IEEE Trans. Autom. Control, 41: 280284, 1996. 91. M. T. Ho and A. Datta, Partially decentralized adaptive control of interconnected systems with improved performance, Int. J. Control, 64: 529561, 1996. iljak, Overlapping decompositions, expan92. M. Ikeda and D. D. S sions, and contractions of dynamic systems, Large Scale Syst., 1: 2938, 1980. iljak, and D. E. White, Decentralized control 93. M. Ikeda, D. D. S with overlapping information sets, J. Optim. Theory Appl., 34: 279310, 1981. iljak, Overlapping decentralized control 94. M. Ikeda and D. D. S with input, state, and output inclusion, Control Theory Adv. Technol., 2: 155172, 1986. zgu .O 95. A. Iftar and U ner, Contractible controller design and optimal control with state and input inclusion, Automatica, 26: 593 597, 1990. 96. A. Iftar, Decentralized estimation and control with overlapping input, state, and output decomposition, Automatica, 29: 511 516, 1993. iljak, Stochastic inclu , X. B. Chen, and D. D. S 97. S. S. Stankovic sion principle applied to decentralized overlapping suboptimal
Santa Clara University
LARGE SCALE INTEGRATION. See VLSI CIRCUIT

LAYOUT.
LARGE-SCALE SYSTEMS. See LARGE-SCALE AND DECENTRALIZED SYSTEMS.
LARGE SUPERCONDUCTING DETECTOR SOLENOIDS. See HIGH-ENERGY PHYSICS PARTICLE DETECTOR

MAGNETS.
LASER ABLATION. See LASER APPLICATIONS IN MEDICINE.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Linear Dynamical Systems, Approximation Standard Article A. C. Antoulas1 1Rice University, Houston, TX Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1049 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (397K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The SchmidtMirsky Theorem and the Aak Generalization Construction of Approximants Examples Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
LINEAR DYNAMICAL SYSTEMS, APPROXIMATION
403
Two basic and desirable features of an approximation theory are the existence of: a constructive methodology for obtaining reduced-complexity models an appropriate quantization of the approximation error, in other words, the estimation of some measure of the error between the given high-complexity model and the derived reduced-complexity model(s) The importance of the second item cannot be overemphasized: In approximating a system, one wishes to have some idea of what has been eliminated. Often an additional desirable feature consists in looking for reduced-complexity models within a specied class, e.g. the class of stable systems For instance, in case the nominal model is stable, if the intended use of a reduced-complexity model is in open-loop, it is imperative that the latter also be stable. We will deal with linear, time-invariant, discrete- and continuous-time, nite-dimensional systems described by convolution sums or integrals. In addition, we will only consider the approximation of stable systems. Most of the approximation approximationfail to methods availablefor example, Pade satisfy the requirements listed above. We will discuss the theory of Hankel-norm approximation and the related approximation by balanced truncation. The size of systemsincluding error systemsis measured in terms of appropriately dened 2-norms, and complexity is measured in terms of the (least) number of state variables (i.e., rst-order differential or difference equations) needed to describe the system. This approach to model reduction has an interesting history. For operators in nite-dimensional spaces (matrices), the problem of optimal approximation by operators of lower rank is solved by the SchmidtMirsky theorem (see Ref. 1). Although this is not a convex problem and consequently conventional optimization methods do not apply, it can be solved explicitly by using an ad hoc tool: the singular value decomposition (SVD) of the operator. The solution involves the truncation of small singular values. In the same vein, a linear dynamical system can be represented by means of a structured (Hankel) linear operator in appropriate innite dimensional spaces. The AdamjanArov Krein (AAK) theory (see Refs. 2 and 3) generalizes the SchmidtMirsky result to dynamical systems, that is to structured operators in innite-dimensional spaces. This is also known as optimal approximation in the Hankel norm. The original setting of the AAK theory was functional analytic. Subsequent developments due to Glover (4) resulted in a simplied linear algebraic framework. This setting made the theory quite transparent by providing explicit formulae for the quantities involved. The Hankel norm approximation problem can be addressed and solved both for discrete- and continuous-time systems. However, the following is a fact: The discrete-time case is closer to that of nite-dimensional operators and to the SchmidtMirsky result. Therefore, the intuitive understanding of the results in this case is more straightforward. In the continuous-time case, on the other hand, while the interpreta-

Approximation is an important methodology in science and engineering. In this essay we will review a theory of approximation of linear dynamical systems that has a number of desirable features. Main ingredients of this theory are: the 2norms used to measure the quantities involved, in particular the Hankel norm, the innity norm, and a set of invariants called the Hankel singular values. The main tool for the construction of approximants is the all-pass dilation (unitary extension) of the original dynamical system.
404
tion of the results is less intuitive than for the discrete-time case, the corresponding formulas turn out to be simpler (see Remark 3). Because of this dichotomy we will rst state the results in the discrete-time case, trying to make connections and draw parallels with the SchmidtMirsky theorem (see section entitled The SchmidtMirsky Theorem and the AAK Generalization). The numerous formulas for constructing approximants, however, will be given for the continuous-time case (see section entitled Construction of Approximants). The Hankel-norm approximation, as well as model reduction by balanced truncation, inherits an important property from the SchmidtMirsky result: Unitary operators form the building blocks of this method and cannot be approximated; a multiple singular value indicates that the original operator contains a unitary part. This unitary part has to either be included in the approximant as a whole or discarded as a whole; it must not be truncated. The same holds for the extension to Hankel-norm approximation. The fundamental building blocks are all-pass systems, and a multiple Hankel singular value indicates the existence of an all-pass subsystem. In the approximation, this subsystem will be either eliminated or included as a whole; it must not be truncated. Consequently, it turns out that the basic operation involved in constructing approximants is all-pass dilation (unitary extension)that is, the extension of the number of states of the original system so that the aggregate becomes all-pass (unitary) [see Main Theorem (2.3)]. The article contains three main sections. The rst, entitled The SchmidtMirsky Theorem and the AAK Generalization, reviews 2-norms and induced 2-norms in nite dimensions and states the SchmidtMirsky result. The second half of this section presents the appropriate generalization of these concepts for linear discrete-time systems. An operator which is intrinsically attached to a linear system is the convolution operator S . Therefore, the problem of optimal approximation in the 2-induced norm of this operator arises naturally. For reasons explained in the section entitled Approximation of in the 2-Induced Norm of the Convolution Operator, however, it is currently not possible to solve this problem, except in some special cases [see Remark 4(b)]. A second operator, the Hankel operator H , is obtained by restricting the domain and the range of the convolution operator. The 2-induced norm of H is the Hankel norm of . It turns out that the optimal approximation problem is solvable in the Hankel norm of . This is the AAK result, stated in the section entitled The AAK Theorem. The main result of optimal and suboptimal approximation in the Hankel norm is presented next. The fundamental construction involved is the of . all-pass dilation (unitary extension) The most natural norm for the approximation problem is the 2-induced norm of the convolution operator, which is equal to the innity norm of the associated rational transfer function (i.e., the maximum of the amplitude Bode plot). However, only a different problem can be solved, namely the approximation in the 2-induced norm of the Hankel operator (which is different from the convolution operator). Although the problem solved is not the same as the original one, the singular values of the Hankel operator turn out to be important invariants. Among other things, they provide a priori computable bounds both upper and lowerfor the innity norm of the error systems.
The rst part of the section entitled Construction of Approximants presents the fundamentals of continuous-time linear systems. The convolution and the Hankel operators are introduced together with the grammians and the computation of the singular values of the Hankel operator H . These are followed by Lyapunov equations, inertia results, and statespace characterizations of all-pass systemsall important ingredients of the theory. Subsequently, various formulas for optimal and suboptimal approximants are given. These formulae are presented for continuous-time systems since (as mentioned earlier) they are simpler than their discrete-time counterparts. First comes an inputoutput approach applicable to single-input single-output systems (see section entitled InputOutput Construction Method for Scalar Systems); it involves the solution of a polynomial equation which is straightforward to set up and solve. The remaining formulas are all state-space-based. The following cases are treated (in increasing complexity): square systems, suboptimal case; square systems, optimal case; general systems, suboptimal case. These are followed by the important section entitled Error Bounds of Optimal and Suboptimal Approximants; these bounds concern the 2-induced norms of both the Hankel and the convolution operators of error systems. The section entitled Balanced Realizations and Balanced Model Reduction, presents the closely related approximation method by balanced truncation. Again error bounds are available, although balanced approximants satisfy no optimality properties. This article concludes with three examples and a brief overview of some selected recent developments, including an overview of the approximation problem for unstable systems. Besides the original sources, namely Refs. 24, parts of the material presented below can also be found in books (5,6) and in lecture notes (7). For the mathematical background needed we refer to Ref. 8. THE SCHMIDTMIRSKY THEOREM AND THE AAK GENERALIZATION In this section we will rst state the SchmidtMirsky theorem concerned with the approximation of nite-dimensional unstructured operators in the 2-norm, which is the operator norm induced by the vector Euclidean norm. Then, discretetime linear dynamical systems are introduced together with the associated convolution operator; this is a structured operator dened on the space of square summable sequences. It is argued in the section entitled Approximation of in the 2Induced Norm of the Convolution Operator that due to the fact that the convolution operator has a continuous spectrum, it is not clear how the SchmidtMirsky result might be generalized in this case. However, roughly speaking by restricting the domain and the range of the convolution operator, the Hankel operator is obtained. It turns out that the Schmidt Mirsky result can be generalized in this case; this is the famous AAK theorem followed by the section entitled The Main Result. 2-Norms and Induced 2-Norms in Finite Dimensions The Euclidean or 2-norm of x n is dened as x
2
:=
2 x2 1 + + xn
405
Given the linear map A : n m, the norm induced by the Euclidean norm in the domain and range of A is the 2-induced norm: x It readily follows that Ax 2 x A Ax 2 max (A A ) = x 2 x x 2 where ( )* denotes complex conjugation and transposition. By choosing x to be the eigenvector corresponding to the largest eigenvalue of A*A, the above upper bound is attained, and hence the 2-induced norm of A is equal to the square root of the largest eigenvalue of A*A: A
2-ind
values the Frobenius norm of A dened by Eq. (3), is A

F
2 2 1 + + n
2-ind
:= sup
x =0
Ax 2 x 2
(1)
The following central problem can now be addressed. PROBLEM 1. OPTIMAL LOW-RANK APPROXIMATION. Given the nite matrix A, nd a matrix X of the same size but lower rank such tht the 2-induced norm of the error E : A X is minimized. The solution of this problem is provided by the Schmidt Mirsky theorem (see, e.g., Ref. 1, page 208): Theorem 1. SchmidtMirsky. Given is the matrix A of rank n. For all matrices X of the same size and rank at most k n, there holds: AX
2-ind
max (A A )
(2)
k + 1 (A )
(5)
The m n matrix A can also be considered as an element of the (mn)-dimensional space mn. The Euclidean norm of A in this space is called the Frobenius norm: A
F
The lower bound k1(A) of the error is attained by X*, which is obtained by truncating the dyadic decomposition of A, to the leading k terms:
k
:=
trace (A A ) =
i, j
A2 i, j
(3)
X :=
i= 1
i ui v i
(6)
The Frobenius norm is not an induced norm. It satises A2-ind AF. The SVD and the SchmidtMirsky Theorem Consider a rectangular matrix A nm; let the eigenvalue decomposition of the symmetric matrices A*A and AA* be A A = V SV V , AA = USU U where U, V are (square) orthogonal matrices of size n, m, respectively (i.e., UU* In, VV* Im). Furthermore SV, SU are diagonal, and assuming that n m we have
2 2 SV = diag(1 , . . ., n , 0, . . ., 0 ), 2 2 , . . ., n ), SU = diag(1
Remark 1. (a) The importance of this theorem lies in the fact that it establishes a relationship between the rank k of the approximant, and the (k 1)st largest singular value of A. (b) The minimizer X* given above is not unique, since each member of the family of approximants
k
X (1 , . . ., k ) :=
i= 1
(i i )ui v i,
0 i k+ 1, i = 1, . . ., k (7)
i i + 1 0
Given the orthogonal matrices U, V and the nonnegative real numbers 1, . . ., n, we have A = USV where S is a matrix of size n m with i on the diagonal and zeros elsewhere. This is the singular value decomposition (SVD) of A; i, i 1, . . ., n, are the singular values of A, while the columns ui, vi of U, V are the left, right singular vectors of A, respectively. As a consequence, the dyadic decomposition of A follows:
n
attains the lower bound, namely k1(A). (c) The problem of minimizing the 2-induced norm of A X over all matrices X of rank at most k, is a nonconvex optimization problem, since the rank of the sum (or of a linear combination) of two rank k matrices is, in general, not k. Therefore, there is little hope of solving it using conventional optimization methods. (d) If the error in the above approximation is measured in terms of the Frobenius norm, the lower bound in Eq. (5) is 2 2 replaced by k 1 n . In this case, X* dened by Eq. (6) is the unique optimal approximant of A, which has rank k. 2-Norms and Induced 2-Norms in Innite Dimensions The 2-norm of the innite sequence x : is x
2
:=
+ x(1 )2 + x(0 )2 + x(1 )2 +
A=
i= 1
i ui v i
(4)
This is a decomposition in terms of rank one matrices; the rank of the sum of any k terms is k. In terms of the singular
The space of all sequences over which have nite 2-norm is denoted by 2( ;)2 is known as the Lebesgue space of squaresummable sequences. Similarly, for x dened over the negative or positive integers , , the corresponding spaces of sequences having nite 2-norm are denoted by 2(), 2().
406
The 2-norm of the matrix sequence X : pm is dened as X

2
:=
S has (block) Toeplitz and lower-triangular structure. The rational p m matrix function
+ X (1 )
2 F
+ X (0 )
2 F
+ X (1 )
2 F
The space of all p m sequences having nite 2-norm is depm noted by 2 (). Given the linear map A : X Y, where X, Y are subspaces of 2(), the norm induced by the 2-norm in the domain and range is the 2-induced norm, dened by Eq. (1); again Eq. (2) holds. Linear, Discrete-Time Systems A linear, time-invariant, nite-dimensional, discrete-time system is dened as follows: : x(t + 1 ) = Ax(t ) + Bu(t ) , y(t ) = Cx(t ) + Du(t ) tZ
H ( z ) :=
t =0
h (t )zt = C(zI A ) 1 B + D =
pi j (z ) qi j (z )
1 i p, 1 j m
is called the transfer function of . The system is stable if the eigenvalues of A are inside the unit disk in the complex plane, that is, i(A) 1; this condition is equivalent to the impulse pm response being an 2 matrix sequence: h 2 (). If is not stable, its impulse response does not have a nite 2-norm. However, if A has no eigenvalues on the unit circle, the impulse response can be reinterpreted so that it does have a nite 2-norm. This is done next. Let T be a basis change in the state space such that the matrices A, B, C can be partitioned as A= A+ 0 0 A , B= B+ B , C (C+ C )
x(t) n is the value of the state at time t, u(t) m and y(t) p are the values of the input and output at time t, respectively; and A, B, C, and D are constant maps. For simplicity, we will use the notation := A C B D R(n + p ) (n + m ) (8)
where the eigenvalues of A and A are inside and outside of the unit disk, respectively. In contrast to Eq. (9), the 2-impulse response denoted by h2 of is dened as follows: h where
2
:= h 2 + + h 2
(12)
The dimension (or order) of the system is n: dim n. We will denote the unit step function by I ( I(t) 1 for t 0, and zero otherwise) and the Kronecker delta by ((0) 1, and zero otherwise). The impulse response h of is h (t ) = CAt 1 BI(t ) + D(t ) (9)
h2+ (t ) := C+ At + B+ I(t ) + (t )D h2 (t ) := C At B I(t )

where as before is the unit step function and is the Kronecker symbol. Accordingly, we will write 2 . Notice that the algebraic expression for he transfer function remains the same in both cases:
To we associate the convolution operator S which maps inputs u into outputs y:
S : u y,
where y(t ) = (h u )(t )

t
H 2 (z ) = H2+ (z ) + H2 (z ) = C+ (zI A+ ) 1 B+ + D + C (zI A ) 1 B = H (z )

What is modied is the region of convergence of H from
:=
=
h (t )u( ), t Z (10)
This convolution sum can also be written in matrix notation:
0 . 1 0. . . C B .. B B C B y(2 )C B B B C B B B y(1 )C B C =B B C y (0 ) C B B B C B B y (1 ) C B B @ A @
. . .
h (0 ) h (1 ) h (2 ) h (3 ) . . .
h (0 ) h (1 ) h (2 ) . . .
S
h (0 ) h (1 ) . . .
h (0 ) . . .
..
|max (A )| < |z| to |max (A+ )| < |z| < |min (A )| This is equivalent to trading the lack of stability (poles outside the unit disk) for the lack of causality (h2 is nonzero for negative time). Thus a system with poles both inside and outside of the unit disk (but not on the unit circle) will be interpreted as a possibly antistable 2 system 2, by dening the impulse response to be nonzero for negative time. Consequently h2 2(). The matrix representation of the corresponding convolution operator is
0 . 1 . . C B B C u(2 )C B B C B u(1 )C B C B u (0 ) C B C B C u (1 ) C B @ A
. . .
(11)
0. .. B B B B B B 2 =B B B B @
. . . h2 + (0 ) h2 + (1 ) h2 + (2 ) h2 + (3 ) . . .
. . . h2 (1 ) h2 + (0 ) h2 + (1 ) h2 + (2 ) . . .
. . . h2 (2 ) h2 (1 ) h2 + (0 ) h2 + (1 ) . . .
. . . h2 (3 ) h2 (2 ) h2 (1 ) h2 + (0 ) . . .
.. .
(13)
407
Notice that S 2 has block Toeplitz structure, but is no longer lower triangular. All discrete-time systems with no poles on the unit circle will be interpreted as 2 systems 2. For simplicity of notation, however, they will still be denoted by . They are composed of two subsystems: (a) the stable and causal part and (b) the stable but anti-causal part : . For square 2 systemsthat is, 2 systems having the same number of inputs and outputs p m the class of all-pass 2 systems is dened as follows: For all pairs u, y satisfying Eq. (10), there holds y
2
to explore the possibility of optimal approximation of in the 2-induced norm of the convolution operator S . In this regard the following result holds. Proposition 1. Let : infz1 min(H(z)), while * : * supz1 max(H(z)). Every in the interval [*, *] is a singular value of the operator S . We conclude that since the singular values of S form a continuum, it is not a priori clear how the SchmidtMirsky result might be generalized to the convolution operator of . Remark 2. Despite the above conclusion, the approach discussed below yields as byproduct the solution of this problem in the special case of the one-step model reduction; see Remark 4(b). Approximation of in the 2-Induced Norm of the Hankel Operator The next attempt to address this approximation problem is by dening a different operator attached to the system . Recall that m 2 (I ) denotes the space of square-summable sequences of vectors, dened on the interval I , with entries in m. Given the stable and causal system , the following operator is dened by restricting the domain and the range of the convolution operator, Eq. (10):
= u
(14)
for some xed positive constant . This is equivalent to S S = 2 I H (z 1 )H (z ) = 2 Im , |z| = 1
This last condition says that the transfer function (scaled by ) is a unitary matrix on the unit circle. Approximation of in the 2-Induced Norm of the Convolution Operator Let be an 2 system. The convolution operator S can be considered as a map: S :
m 2 (
Z)
p ( 2
Z)
H :
The 2-induced norm of is dened as the 2-induced norm of S : := S = sup
u =0
(Z)
(Z+) where y+ (t ) =
1 =
u y+ ,
h (t )u ( ), t0 (16)
2-ind
2-ind
S u 2 u 2
Due to the equivalence between the time and frequency domains (the Fourier transform is an isometric isomorphism), this norm can also be dened in the frequency domain. In particular,
2-ind
H is called the Hankel operator of . Its matrix representation in the canonical bases is
= sup maxH (z ) =: H
| z |= 1
y (0 ) y (1 ) y (2 ) . . .
h (1 ) h (2 ) h (3 ) . . .
h (2 ) h (3 ) h (4 ) . . .
H
h (3 ) h (4 ) h (5 ) . . .
.. .
u(1 ) u(2 ) u(3 ) . . . (17)
This latter quantity is known as the norm of H. If the system is single-input single-output, it is the supremum of the amplitude Bode plot. In the case where is stable, H is analytic outside the unit disk, and the following holds:
2-ind
= sup maxH (z ) =: H
| z | 1
Thus the Hankel operator of maps past inputs into future outputs. It has a number of properties given next. The rst one for single-input single-output systems is due to Kronecker. Proposition 2. Given the system dened by Eq. (8), the rank of H is at most n. The rank is exactly n if, and only if, the system is reachable and observable. Furthermore, if is stable, H has a nite set of nonzero singular values. In order to compute the singular values of the Hankel operator, we dene the reachability matrix R and the observability matrix O : R (A, B ) = [B AB A2 B ], O (C, A ) = [R (A , C )]
This is known as the h norm of H. For simplicity, we will use the notation
2-ind
:= S
2-ind
= H
(15)
It will be clear from the context whether the subscript stands for the norm of the h norm. If is all-passthat is, Eq. (14) is satisedthen 2-ind H . Our aim is the generalization of the SchmidtMirsky result for an appropriately dened operator. It is most natural
408
both of these matrices have rank at most n. The reachability and observability grammians are
P := RR =
t 0
At BB (A )t ,
Q := O O =
t 0
(A )t CCAt (18)
An important property of the Hankel singular values of is that twice their sum provides an upper bound for the 2-induced norm of (see also the section entitled Error Bounds of Optimal and Suboptimal Approximants). Lemma 1. Given the stable system with Hankel singular values i, i 1, . . ., n (multiplicities included), the following holds true: 2-ind 2(1 n). The AAK Theorem. Consider the stable systems and of dimension n and k, respectively. By Proposition 2, H has rank n and H has rank k. Therefore, the SchmidtMirsky theorem implies that H H
2-ind
The quantities P and Q are n n symmetric, positive semidenite matrices. They are positive denite if and only if is reachable and observable. The grammians are (the unique) solutions of the linear matrix equations: AP A + BB = P , A Q A + C C = Q (19)
which are called discrete-time Lyapunov or Stein equations. The (nonzero) singular values i of H are the square roots of the (nonzero) eigenvalues i of H * H . The key to this computation is the fact that the Hankel operator can be factored in the product of the observability and the reachability matrices: H = O (C, A )R (A, B ) For ui m 2 (), ui 0, there holds
k + 1 (H )
(23)
The question which arises is to nd the inmum of the above norm, given the fact that the approximant is structured (block Hankel matrix): infH H 2-ind. A remarkable result due to Adamjan, Arov, and Krein, code-named AAK result, asserts that this lower bound is indeed attained for some or dimension k. The original sources for this result are Refs. 2 and 3.
pm Theorem 2. AAK Theorem. Given the 2 () sequence of matrices h (h(t))t0, such that the associated Hankel matrix pm H has nite rank n, there exists an 2 () sequence of matrices h (h (t))t0, such that the associated Hankel matrix * * H has rank k and in addition *
H H ui = i2 ui R O ORui = i2 ui RR O O Rui = i2 Rui

P Q
Thus, ui is an eigenfunction of H * H corresponding to the nonzero eigenvalue i2 iff R ui 0 is an eigenfunction of the product of the grammians P Q : i2 (H ) = i (H H ) = i (PQ ) (20)
H H
2-ind
= k + 1 (H )
(24)
If p m 1, the optimal approximant is unique. The result says that every stable and causal system can be optimally approximated by a stable and causal system * of lower dimensions; the optimality is with respect to the 2induced norm of the associated Hankel operator (see Fig. 1). The Main Result. In this section we will present the main result. As it turns out, one can consider both suboptimal and optimal approximants within the same framework. Actually, as shown in the sections entitled State-Space Construction for Square Systems: Suboptimal Case and State-Space Construction for Square Systems: Optimal Case, the formulas for suboptimal approximants are simpler than their optimal counterparts. PROBLEM 2. Given a stable system , we seek approximants * satisfying k + 1 ( )
H
Proposition 3. The nonzero singular values of the Hankel operator H associated with the stable system are the square roots of the eigenvalues of the product of the grammians P Q . Denition 1. The Hankel singular values of the stable system as in Eq. (10), denoted by
1 ( ) > > q ( ) with multiplicity

q
ri , i = 1, . . ., q,
i= 1
ri = n
(21)
are the singular values of H dened by Eq. (16). The Hankel norm of is the largest Hankel singular value:
H
< k ( )
:= 1 ( )
( ) + e
Figure 1. Construction of approximants.
The Hankel operator of a not necessarily stable 2 system is dened as the Hankel operator of its stable and causal part : H : H . Thus, the Hankel norm of a system having poles both inside and outside the unit circle is dened to be the Hankel norm of its causal and stable part. In general,
2-ind
(22)
409
This is a generalization of Problem 1, as well as the problem solved by the AAK theorem. The concept introduced in the next denition is the key to its solution. : Denition 2. Let e be the parallel connection of and e : . If e is an all-pass system with norm , is called an -all-pass dilation of . As a consequence of the inertia result of the section entitled The Grammians, Lyapunov Equations, and an Inertia Result, the all-pass dilation system has the following crucial property. be an -all-pass dilation of , where Main Lemma 1. Let has exactly k poles inside satises Eq. (25). It follows that the unit disk, that is, dim k. We also restate the analog of the SchmidtMirsky result [Eq. (23)], applied to dynamical systems: Proposition 4. Given the stable system , let have at most k poles inside the unit disk. Then
H
stable approximant of in the 2-induced norm of the convolution operator (i.e., the norm). (b) We are given a stable system and seek to compute an approximant in the same class (i.e., stable). In order to achieve this, the construction given above takes us outside has this class of systems, since the all-pass dilation system poles both inside and outside the unit circle. In terms of matrices, we start with a system whose convolution operator S is a (block) lower triangular Toeplitz matrix. We then com , which is no longer lower pute a (block) Toeplitz matrix S is unitary. It triangular, such that the difference S S , which is then follows that the lower left-hand portion of S , has rank r and approximates the Hanthe Hankel matrix H kel matrix H , so that the 2-norm of the error satises Eq. (25). (c) The suboptimal and optimal approximants can be constructed using explicit formulae. For continuous-time systems, see the section entitled Construction Formulas for Hankel-Norm Approximants.
CONSTRUCTION OF APPROXIMANTS The purpose of this section is to present, and to a certain extent derive, formulas for suboptimal and optimal approximants in the Hankel norm. Because of Theorem 3, all we need is the ability to construct all-pass dilations of a given system. To this goal, the rst subsection is dedicated to the presentation of important aspects of the theory of linear, continuoustime systems; these facts are used in the second subsection. The closely related approach to system approximation by balanced truncation is briey discussed in the section entitled Balanced Realizations and Balanced Model Reduction. Linear, Continuous-Time Systems L 2 Linear Systems. For continuous-time functions, let L n ( I ) : = { f : I R n , I R} Frequent choices of I : I , I or I . The 2-norm of a function f is f := f (t ) 2 2 dt
1 /2
k + 1 ( )
This means that the 2-induced norm of the Hankel operator of the difference between and is no less than the (k 1)st singular value of the Hankel operator of . Finally, recall that if a system has both stable and unstable poles, its Hankel norm is that of its stable part. We are now ready for the main result which is valid for both discrete- and continuoustime systems. be an -all-pass dilation of the linear, staTheorem 3. Let ble, discrete- or continuous-time system , where k + 1 ( ) < k ( ) (25)
has exactly k stable poles and consequently It follows that k + 1 ( ) In case k1() , k + 1 ( ) =
H
<
(26)
t I
f L n (I )
The corresponding L 2 space of square-integrable functions is

n L2 (I ) := { f L n (I ), f 2
Proof. The result is a consequence of the following sequence of equalities and inequalities: k + 1 ( ) +
H
< }
The 2-norm of the matrix function F : I pm, is dened as F := F (t )

2 F 1 /2
=
2
The rst inequality on the left side is a consequence of Main Lemma 1, the equality follows by denition, the second inequality follows from Eq. (22), and the last equality holds by is -all-pass. construction, since Remark 3. (a) For 1(), the above theorem yields the solution of the Nehari problem, namely to nd the best anti-
dt
t I
where the subscript F denotes the Frobenius norm. The space of all p m matrix functions having nite 2-norm is pm denoted by L 2 (I ). Let A : X Y, where X and Y are subq spaces of L 2(), for some q. The 2-induced norm of A is dened as in Eq. (1), and Eq. (2) holds true.
410
We will consider linear, nite-dimensional, time-invariant, continuous-time systems described by the following set of differential and algebraic equations:
dx(t ) = Ax(t ) + Bu(t ) , dt y(t ) = Cx(t ) + Du(t )
tR
From now on, all continuous-time systems with no poles on the imaginary axis will be interpreted as L 2 systems; to keep the notation simple, they will be denoted by instead of 2. The stable and causal subsystem will be denoted by , and the stable but anti-causal one will be denoted by : . The Convolution Operator. The convolution operator associated to is dened as follows:
where u(t) m, x(t) n, and y(t) p are the values of the input, state, and output at time t, respectively. The system will be abbreviated as A = C B D R
(n + p ) (n + m )
S : u y, where y(t ) = (h u )(t ) :=

(27)
h (t )u( ) d , tR (30)
In analogy to the discrete-time case, we dene the continuous-time unit step function I(t) ( I(t) 1 for t 0, and zero otherwise) and the delta distribution . The impulse response of this system is h (t ) = CeAt BI(t ) + (t )D while the transfer function is H (s ) = C(sI A ) 1 B + D Unless A has all its eigenvalues in the left half-plane (LHP), the impulse response h is not square-integrablethat is, pm does not belong to the space L 2 (). In this case we shall say that the system is not an L 2 system. If, however, A has no eigenvalues on the j axis, it can be interpreted as an L 2 system by appropriate redenition of the impulse response. As in the discrete-time case, let there be a state transformation such that A= A+ 0 0 A , B= B+ B , C = (C+ C ) (28)
Corresponding to Proposition 1 we have the following: Proposition 5. Let * : inf min(H( j)), while * : sup max(H( j)). Every in the interval [ , *] is a singu* lar value of the operator S . Since the singular values of S form a continuum, the same conclusion as in the discrete-time case follows (see also Remark 2). Again, the 2-induced norm of turns out to be the innity norm of the transfer function H:
2-ind
:= S
2-ind
= H
(31)
If is all-pass [Eq. (14)], then 2-ind H . The Grammians, Lyapunov Equations, and an Inertia Result. For stable systems (i.e., R e(i(A)) 0), the following quantities are dened: P :=
0
eAt BB eA t dt ,
Q :=
0
eA t CCeAt dt
(32)
where the eigenvalues of A and A are in the LHP and right half-plane (RHP), respectively. The L 2-impulse response is dened as follows: h where
2
:= h 2 + + h 2
(29)
They are called the reachability and observability grammians of , respectively. By denition, P and Q are positive semidenite. It can be shown that reachability and observability of are equivalent to the positive deniteness of each one of these grammians. As in the discrete-time case, the grammians are the (unique) solutions of the linear matrix equations AP + P A + BB = 0, A Q + Q A + C C = 0 (33)
h2+ (t ) := C+ e
A+ t A t
B+ I(t ) + (t )D B I(t )
h2 (t ) := C e
As in the discrete-time case,
H 2 (s ) = H2+ (s ) + H2 (s ) = C+ (sI A+ ) 1 B+ + D + C (sI A ) 1 B = H (s )

This is equivalent to trading the lack of stability (poles in the RHP) to the lack of causality (h2 is nonzero for negative time). What is modied is the region of convergence from Re(max (A )) < Re(s ) to Re(max (A+ )) < Re(s ) < R (min (A ))
which are known as the continuous-time Lyapunov equations [cf. Eq. (19)]. Such equations have a remarkable property known as inertia result: There is a relationship between the number of eigenvalues of A and (say) P in the LHP, RHP, and the imaginary axis. More precisely, the inertia of A nn is in (A ) := {(A ), (A ), (A )} (34)
where (A), (A), and (A) are the number of eigenvalues of A in the LHP, on the imaginary axis and in the RHP, respectively. Proposition 6. Let A and X X* satisfy the Lyapunov equation: AX XA* R, where R 0. If the pair (A, R) is reach-
411
able, then the inertia of A is equal to the inertia of X: in(A) in(X). Main Lemma 1 is based on this inertia result. Remark 3. One of the reasons why continuous-time formulas are simpler than their discrete-time counterparts is that the Lyapunov equations [Eqs. (34)] are linear in A, while the discrete-time Lyapunov equations [Eqs. (19)] are quadratic in A. All-Pass L 2 Systems. All-pass systems are a subclass of L 2 systems. As indicated in Theorem 3, all-pass systems play an important role in Hankel norm approximation. The following characterization is central for the construction of suboptimal and optimal approximants (see sections entitled State-Space Construction for Square Systems: Suboptimal Case and State-Space Construction for Square Systems: Optimal Case). The proof is by direct computation; for details see Ref. 7. Proposition 7. Given is as in Eq. (27) with p m. The following statements are equivalent. is -all-pass. For all inputoutput pairs (u, y) satisfying y h u, there holds: y2 u2. 2 H* (j)H( j) Im There exists Q Q * nn, such that the following equations are satised:
The Hankel singular values of H and their multiplicities will be denoted as in Eq. (21). Finally,
H
+ H
2-ind
Lemma 1 can be strengthened in the continuous-time case (see also the section entitled Error Bounds of Optimal and Suboptimal Approximants). Lemma 2. Let the distinct Hankel singular values of the stable system be i, i 1, . . ., q; following holds: 2-ind 2(1 q). A Transformation Between Continuous- and Discrete-Time Systems. As mentioned in the introductory paragraphs of this article, the formulas for the construction of approximants will be given for the continuous-time case because they are generally simpler than their discrete-time counterparts. Thus, in order to apply the formulas to discrete-time systems, the system will have to be transformed to continuous-time rst, and at the end the continuous-time optimal or suboptimal approximants obtained will have to be transformed back to discretetime approximants. One transformation between continuous- and discrete-time systems is given by the bilinear transformation z (1 s)/(1 s) of the complex plane onto itself. The resulting relationship between the transfer function Hc(s) of a continuoustime system and that of the corresponding discrete-time system Hd(z) is Hc (s ) = Hd The state space maps
c
1+s 1s
A Q + Q A + C C = 0 Q B + C D = 0 D D = 2 Im
The solutions P and Q of the Lyapunov equations AP + P A + BB = 0, A Q + Q A + CC = 0
(35)
:=
A C
B , D
:=
F H
G J
are related as given in Table 1. Furthermore, the proposition that follows states that the Hankel and innity norms remain unchanged by this transformation. Proposition 8. Given the stable continuous-time system c with grammians P c and Q c, let d with grammians P d and Q d, be the discrete-time system obtained by means of the transformation given above. It follows that P c P d and Q c Q d. Furthermore, this bilinear transformation also preserves the innity norms (i.e., the 2-induced norms of the associated convolution operators): c d.
satisfy P Q 2In, and in addition we have D*D 2Im. The Continuous-Time Hankel Operator. In analogy to the discrete-time case, we dene the operator H of the stable system which maps past inputs into future outputs:
m H : L2 (R ) L2p (R+ ) 0
u y+ , where y+ (t ) =
h (t )u ( ) d , t0 (36)
Table 1. Transformation Formulas Continuous-Time 1s 1s Discrete-Time F (I A)(I A)1 G 2(I A)1B H 2C(I A)1 J D C(I A)1B
H is called the Hankel operator of . Unlike in the discretetime case, however (see Eq. (17)), H has no matrix representation. It turns out that just as in Eq. (20), it can be shown that the nonzero singular values of the continuous-time Hankel operator are the eigenvalues of the product of the two grammians: i2 (H ) = i (H H ) = i (PQ ) (37)
A, B, C, D
A (F I)1(F I) B 2(F I)1G C 2H(F I)1 D J H(F I)1G
z1 z1
F, G, H, J
412
Construction Formulas for Hankel-Norm Approximants We are ready to give some of the formulas for the construction of suboptimal and optimal Hankel-norm approximants. As mentioned earlier, all formulas describe the construction of all-pass dilation systems [see Eq. (3)]. We will concentrate on the following cases (in increasing degree of complexity): An inputoutput construction applicable to scalar systems. Both optimal and suboptimal approximants are treated. The advantage of this approach is that the equations can be set up in a straightforward manner using the numerator and denominator polynomials of the transfer function of the given system (see the section entitled InputOutput Construction Method for Scalar Systems). A state-space-based construction method for suboptimal approximants (see the section entitled State-Space Construction for Square Systems: Suboptimal Case) and for optimal approximants (see the section entitled StateSpace Construction for Square Systems: Optimal Case) of square systems. A state-space-based parameterization of all suboptimal approximants for general (i.e., not necessarily square) systems (see the section entitled General Case: Parameterization of All Suboptimal Approximants). The optimality of the approximants is with respect to the Hankel norm (2-induced norm of the Hankel operator). The section entitled Error Bounds of Optimal and Suboptimal Approximants gives an account of error bounds for the innity norm of approximants (2-induced norm of the convolution operator). The section entitled Balanced Realizations and Balanced Model Reduction discusses model reduction by balanced truncation which uses the same ingredients as the Hankel norm model reduction theory. The approximants have a number of interesting properties, including the existence of error bounds for the innity norm of the error. However, no optimality holds. InputOutput Construction Method for Scalar Systems. Given the polynomials a i0 aisi, b i0 bisi, and c i0 cisi satisfying c(s) a(s)b(s), the coefcients of the product c are a linear combination of those of b: c = T(a )b where c : (c c1 . . . c1 c0)* 1, b : (b . . . b0)* 1, and (a) is a Toeplitz matrix with rst column (a . . . a0 0 . . . 0)* 1, and rst row (a 0 . . . 0) 1(1). We will also dene the sign matrix
The basic construction given in Theorem 3 hinges on the con of . Let struction of an -all-pass dilation H (s ) = p (s ) , q (s ) H (s ) = p (s ) q (s )
H be -all-pass. We require that the difference H H e Therefore, the problem is as follows: Given and the polynomials p and q such that deg(p) deg(q) : n, nd polynomi and q of degree at most n such that als p
p q q p = pq qp = q q q q qq
(38)
This polynomial equation can be rewritten as a matrix equation involving the quantities dened above: T( p )q T (q ) p = T(q )q = T ( q )K q Collecting terms we have ( T ( p ) T ( q )K , T(q )) q p =0 (39)
The solution of this set of linear equations provides the coef . Furthermore, this cients of the -all pass dilation system system can be solved for both the suboptimal i and the optimal i cases. We will illustrate the features of this approach by means of a simple example. For an alternative approach along similar lines, see Ref. 9. Example. Let be a second-order system, that is, n 2. If , that we normalize the coefcient of the highest power of q 2 1, we obtain the following system of equations: is, q
0 p2 q 2 p1 + q 1 p0 q 0 0
0 0 p2 + q 2 p1 q 1 p0 + q 0 W( )
q2 q1 q0 0 0
0 q2 q1 q0 0
0 0 q2 q1 q0
q 1 q 0 p 2 p 1 p 0 p2 + q 2 p1 q 1 p0 + q 0 0 0
K = diag(
...
, 1, 1, 1 )
of appropriate size. Given a polynomial a with real coefcients, the polynomial c* is dened as c(s)* : c(s). This means that c = K c
This can be solved for all which are not roots of the equation det W() 0. The latter is a polynomial equation of second degree; there are thus two values of , 1, and 2, for which the determinant of W is zero. It can be shown that the roots of this determinant are the eigenvalues of the Hankel operator H ; since in the single-input single-output case H is selfadjoint (symmetric), the absolute values of 1 and 2, are the singular values of H . Thus both suboptimal and optimal approximants can be computed this way. (See also the rst example of the section entitled Examples.) State-Space Construction for Square Systems: Suboptimal Case. In this section we will discuss the construction of ap-
413
proximants in the case where m p. Actually this is no loss of generality because if one has to apply the algorithm to a nonsquare system, additional rows or columns of zeros can be added so as to make the system square. For details we refer to Ref. 4. Consider the system as in Eq. (27), with R e(i(A)) 0 and m p. For simplicity it is assumed that the D-matrix of is zero. Compute the grammians P and Q by solving the Lyapunov equations [Eqs. (34)]. We are looking for = A C B D
Solving these two equations for the unknown quantities, we obtain
= A + C (CP DB ) A = Q B + C D B = (CP DB ) C
(45)
denoted by e, is such that the parallel connection of and -all-pass. First, note that e has the following state-space representation:
A
e
= C
A C
B B D
has exwhere * : (*)1. There remains to show is that A actly k stable eigenvalues. From the Lyapunov equation for 1P . By construction [see Eq. Q e it also follows that Q (43)], has k positive and n k negative eigenvalues; the (i.e., in (Q ) k, 0, n k). Furthersame holds true for Q , A is observable, because otherwise A has more, the pair C eigenvalues on the j axis, which is a contradiction to the original assumption that A is stable. Then by Proposition 6, , which completes the is equal to that of A the inertia of Q proof. has dimension n and the dilated Corollary 1. The system has system e has dimension 2n. The stable subsystem dimension k. State-Space Construction for Square Systems: Optimal Case. The construction of the previous section can be extended to include the optimal case. This section presents the formulae which rst appeared in Section 6 of Ref. 4. In this case we need to construct the all-pass dilation e for equal to the lower bound in Eq. (25), namely k1(). Thus, k1 is an eigenvalue of the product of the grammians P Q of multiplicity, say r. There exists a basis change in the state space such that P = Ir k+ 1 0 0 P2 , Q= Ir k+ 1 0 0 Q2 (46)
(40)
According to the last characterization of Proposition 7, e is all-pass iff the corresponding grammians and D matrices satisfy Pe Qe =
2
I2n ,
= D D
Im
(41)
/ is a unitary matrix of size m. The key This implies that D to the construction of P e and Q e, is the unitary dilation of P and Q . Consider the simple case where the grammians are scalar P p, Q q, and 1. In this case we are looking for lling in the ? so that p ? ? ? q ? ? ? = 1 0 0 1
assuming that pq 1. It readily follows that one solution is
p 1 pq
1 pq q (1 pq )
q 1
1 p 1 pq
1 0
0 1
(42)
The balancing transformation Eq. (60) discussed in the section entitled Balanced Realizations and Balanced Model Reduction accomplishes this goal. Clearly, only the pair P 2 and Q 2 needs to be dilated. As in can be obtained the suboptimal case, explicit formulae for by using Eq. (35) of Proposition 7. Partition A, B, and C conformally with P and Q : A= A11 A21 A12 A22 , B= B1 B2 , C = (C1 C2 )
In the general (nonscalar) case, this suggests dening the quantity: :=

2
In PQ
(43)
Assuming that is not equal to any of the eigenvalues of the product P Q (i.e., to any of the singular values of the Hankel operator H ), is invertible. Keeping in mind that P and Q do not commute, the choice of P e and Q e corresponding to Eq. (42) is Pe = P
rm where A11 rr, B1, C* , and Ir denotes the r r 1 identity matrix. The (1, 1) block of the Lyapunov equations * C* yields B1B1 1 C1; this implies the existence of a unitary matrix U of size m, such that
B1U = C1 ,
UU = Im
Qe =
Q In
In
1
(44)
Using Eqs. (45) we construct an all-pass dilation of the subsystem (A22, B21, C12); solving the corresponding equations we obtain
Once the matrices P e and Q e for the dilated system e have been constructed, the next step is to construct the matrices . According to Eq. (35) of , B of the dilation system , and C A Proposition 6, the Lyapunov equation Ae *Q e Q eAe Ce * Ce 0 and the equation Q eBe C* e De 0 have to be satised.
= U D k+1 = Q B + C D B 2 2 2 = (C P DB )( C 2 2 2 = A + CC A 22 2
1 2)
(47)
414
2 (nr)(nr) where 2 : k . Hence unlike the subop1 P 2Q 2 is not arbitrary. In the single-input single-outtimal case, D is completely determined by the above relationput case, D ship (is either 1 or 1) and hence there is a unique optimal approximant. This is not true for systems with more than one input and/or more than one output. Furthermore, in the onestep reduction case (i.e., k1 n) the all-pass dilation system has no antistable part; that is, it is the optimal Hankel norm approximant.
Error Bounds of Optimal and Suboptimal Approximants. Given is the stable m m system , having Hankel singular values be the i and multiplicity ri, i 1, . . ., q [see Eq. (21)]. Let -all-pass dilation of , where q (the smallest singular is n q, value). Following Corollary 2, the dimension of ; that is, it is stable. Thus by the which implies that same corollary, for a one-step reduction, the all-pass dilation system has no unstable poles; for simplicity we will use the notation
q
Corollary 2. (a) In contrast to the suboptimal case, in the has dimension r, and the dilated system e optimal case, has dimension 2n r. Furthermore, the stable subsystem has dimension n r. (b) In the case where k1 n, since , is stable and all-pass with norm n. General Case: Parameterization of All Suboptimal Approximants. Given the system as in Eq. (27), let satisfy Eq. (25). The following rational matrix (s) has size (p m) (p m): (s ) := I p+ m C (sI A ) 1 Q 1C J where (48)
:= (q )
Thus q is all-pass of magnitude q, and consequently we have i (

q)
= i ( ),
i = 1, . . ., n rq
(52)
We now approximate q by q1 through a one-step optimal Hankel-norm reduction. By successive application of the same procedure, we thus obtain a sequence of all-pass systems i, i 1, . . ., q, and a system 0 consisting of the matrix D0, such that the transfer function of is decomposed as follows: H (s ) = D0 + H1 (s ) + + Hq (s ) (53)
C := A := Q := J :=
C 0 A 0 Q In Ip 0
0 B 0 A In P 0 Im
R( p + m ) 2n R2 n 2 n R2 n 2 n , R( p + m ) ( p + m )
where Hk(s) is the transfer function of the stable all-pass sysq tem k having dimension 2n ik ri, k 1, . . ., q; the dik k mension of the partial sums i1 Hi(s) is equal to i1 ri, k 1, 2, . . ., q. Thus Eq. (53) is the equivalent of the dyadic decomposition Eq. (4), for . From the above decomposition we can derive the following upper bound for the H norm of . Assuming that H() D 0, we obtain H ( s ) D0
H1 (s )
1
+ + Hq (s )
q
Putting these expressions together we obtain
Evaluating the above expression at innity yields D0

2
1 + + q
(s ) =
11 21
12 22
I p + C(sI A ) 1 1 P C := B (sI + A ) 1 C
C(sI A ) 1 1 B Im B (sI + A ) 1 Q 1 B (49)
Thus, combining this inequality with H(s) D02 1 q, yields the bound given in Lemma 2: H (s )
2 (1 + + q )
(54)
By construction, is J-unitary on the j axis; that is, *(j)J( j) J. Dene

1( ) 2( )
This bound can be sharpened by computing an appropriate D0 mm as in Eq. (53): H ( s ) D0
:=
(s )
(s ) Im
11 (s ) (s ) + 21 (s ) (s ) +
12 (s ) 22 (s )
1 + + q
(55)
(50)
The proof of the following result can be found in Chap. 24 of Ref. 10. Recall Theorem 3. is an -all-pass dilation if and only if Theorem 4. H H =
1(
2(
) 1
(51)
Finally, we state a result of Ref. 4, on the Hankel singular values of the stable part e of the all-pass dilation e. It will be assumed for simplicity that each Hankel singular value , where has multiplicity one: ri 1. Assume that e is an optimal Hankel approximant of of dimension k; the following holds: 1(e) 2k1(e) k1(), ) k2(), . . ., nk(e) nk1( ) 2k2(e) 1( n(). Using these inequalities we obtain an error bound for the H norm of the error of with a degree k optimal approx . First, note that imant :=
where (s) is a p m anti-stable contraction (all poles in the right half-plane and (s) 1).
D0
1 ( ) + + n k 1 ( )
415
D0 k1() . Combining all This implies previous inequalities we obtain the bounds:
the input is zero (u 0), is equal to x o Q xo (59)
k + 1
+ D0
k + 1 ( ) + 1 ( )
+ + n k 1 ( ) k + 1 ( ) + + n ( )
(56)
The left-hand-side inequality in Eq. (25), and the above nally yield upper and lower bounds on the H norm of the error: k + 1 +
2 (k+ 1 + + n )
(57)
The rst upper bound in Eq. (56) above is the tightest; but both a D term and the singular values of the antistable part of the all-pass dilation are needed. The second upper bound in Eq. (56) is the second-tightest; it only requires knowledge of the optimal D term. Finally the upper bound in Eq. (57) is the least tight among these three. It is, however, the most useful, since it can be determined a priori, that is, before the computation of the approximants, once the singular values of the original system are known. Remark 4. (a) In a similar way a bound for the innity norm of suboptimal approximants can be obtained. Given , let be an all-pass dilation satisfying Eq. (25). Then, similarly to Eq. (56), the following holds: r + 1 +
2 ( + r + 1 + + n )
We conclude that the states which are difcult to reachthat is, those which require a large amount of energy to reach are in the span of the eigenvectors of the reachability grammian P corresponding to small eigenvalues. Moreover, the states which are difcult to observethat is, those which yield small amounts of observation energyare those which lie in the span of the eigenvectors of the observability grammian Q corresponding to small eigenvalues as well. This observation suggests that one way to obtain reduced order models is by eliminating those states which are difcult to reach and/or difcult to observe. However, states which are difcult to reach may not be difcult to observe, and vice versa; as simple examples show, these properties are basisdependent. This suggests the need for a basis in which states which are difcult to reach are simultaneously difcult to observe, and vice versa. From these considerations, the question arises: Given a continuous- or discrete-time stable system , does there exist a basis of the state space in which states which are difcult to reach are also difcult to observe? The answer to this question is afrmative. The transformation which achieves this goal is called a balancing transformation. Under an equivalence transformation T (basis change TAT1, in the state space: x : Tx, det T 0) we have A CT1, while the grammians are transformed TB, and C B as follows: = TP T, P = T Q T 1 P Q = T (PQ )T 1 Q
is all(b) For a one-step Hankel-norm reduction, pass; that is,

H
= q
is an optimal approximant which shows that in this case not only in the Hankel norm but in the innity norm as well. Thus for this special case, and despite Propositions 1 and 5, the Hankel-norm approximation theory yields a solution for the optimal approximation problem in the 2-norm of the convolution operator S ; recall that this is the problem we initially wished to solve. Balanced Realizations and Balanced Model Reduction A model reduction method which is closely related to the Hankel-norm approximation method is approximation by balanced truncation. This involves a particular realization of a linear system given by Eq. (27), called balanced realization (the D matrix is irrelevant in this case). We start by explaining the rational behind this particular state-space realization. The Concept of Balancing. Consider a stable system with positive denite reachability and observability grammians P and Q . It can be shown that the minimal energy required for the transfer of the state of the system from 0 to some nal state xr is
1 x xr rP
The problem is to nd T, det T 0, such that the transformed and Q are equal. This will ensure that the grammians P states which are difcult to reach are precisely those which are difcult to observe. Denition 3. The stable system is balanced iff P Q . is principal-axis balanced iff P = Q = S := diag(1 , . . ., n ) The existence of a balancing transformation is guaranteed. Lemma 3. Balancing Transformation. Given the stable system and the corresponding grammians P and Q , let the matrices R, U, and S be dened by P : R*R and RQ R* : US2U*. A (principal axis) balancing transformation is given by T := S1/2U R (60)
(58)
Similarly, the largest observation energy produced by observing the output, when the initial state of the system is xo and
where (as before) R* : (R*).1 To verify that T is a balancing transformation, it follows by direct calculation that TP T* S and T*Q T1 S. We also note that if the Hankel singular values are distinct (i.e., are deterhave multiplicity one), balancing transformations T mined from T given above, up to multiplication by a sign matrix L that is, a diagonal matrix with 1 on the diagonal: LT. T
416
Model Reduction. Let be balanced with grammians equal to S, and partition:
A= B=
The systems
A11 A21 B1 B2 ,
A12 A22
S=
S1 0
0 S2
, (61)
C = (C1 C2 )
:=
Aii Ci
Bi
i = 1, 2
are called reduced-order systems obtained form by balanced truncation. These have certain guaranteed properties. However, these properties are different from discrete- and continuous-time systems. Hence we state two theorems. For a proof see, for example, Ref. 6. Theorem 5. Balanced Truncation: Continuous-Time Systems. Given the stable (no poles in the closed right halfplane) continuous-time system , the reduced-order systems i, i 1, 2, obtained by balanced truncation have the following properties: 1. i, i 1. 2, satisfy the Lyapunov equations AiiSi * 0, and Ai* *Ci 0. FurtherSiAi* i BiBi i Si SiAii Ci more, Aii, i 1, 2, have no eigenvalues in the open right half-plane. 2. If S1 and S2 have no eigenvalues in common, both 1 and 2 have no poles on the imaginary axis and are, in addition, reachable, observable, and balanced. 3. Let the distinct singular values of be i, with multiplicities mi, i 1, . . ., q. Let 1 have singular values i, i 1, . . ., k, with the corresponding multiplicities mi, i 1, . . ., k, k q. The H norm of the difference between the full-order system and the reduced-order system 1 is upper-bounded by twice the sum of the neglected Hankel singular values:
1
Remark 5. (a) The last part of the above theorems says that if the neglected singular values are small, then the Bode plots of and 1 are guaranteed to be close in the H norm. The difference between part 3 for continuous- and discrete-time systems above is that the multiplicities of the neglected singular values do not enter in the upper bound for continuoustime systems. (b) Proposition 8 implies that the bilinear transformation between discrete- and continuous-time systems preserves balancing (see also A Simple Discrete-Time Example below). (c) Let hank and bal be the reduced-order systems obtained by one step Hankel-norm approximation, and one step balanced truncation, respectively. It can be shown that hank bal is all-pass with norm q; it readily follows that
bal
2 q
and
bal H
2 q
A consequence of the above inequalities is that the error for reduction by balanced truncation can be upper-bounded by means of the singular values of as given in Theorem 5. Furthermore, this bound is valid both for the H norm and for the Hankel norm. (d) Every linear, time-invariant, continuous-time and stable system , can be expressed in balanced canonical form. In the generic case (distinct singular values) this canonical form is given in terms of 2n positive numbers, namely the singular values i 0, and some bi 0, as well as n signs si 1, i 1, . . ., n. The quantities i : sii are called signed singular values of ; they satisfy H(0) 2(1 n). For details on balanced canonical forms we refer to the work of Oberfor example, Ref. 11. EXAMPLES In this section we will illustrate the results presented above by means of three examples. The rst deals with a simple second-order continuous-time system. The purpose is to compute the limit of suboptimal Hankel-norm approximants as tends to one of the singular values. The second example discusses the approximation of a discrete-time system [third-order nite impulse response (FIR) system] by balanced truncation and Hankel-norm approximation. The section concludes with the approximation of the four classic analog lters (Butterworth, Chebyshev 1, Chebyshev 2, and Elliptic) by balanced truncation and Hankel-norm approximation. A Simple Continuous-Time Example Consider the system given by Eq. (27), where n 2, m p 1, and
2 (k+ 1 + + q )
(62)
If the smallest singular value is truncatedthat is, S2 qIrq equality holds. Theorem 6. Balanced Truncation: Discrete-Time Systems. Given the stable, discrete-time system (no poles in the complement of the open unit disk) , the reduced-order systems i obtained by balanced truncation have the following properties: 1. i, i 1, 2, have no poles in the closed unit disk; these systems are, in general, not balanced. 2. If min(S1) max(S2), 1 is, in addition, reachable and observable. 3. The h norm of the difference between full- and reduced-order models is upper-bounded by twice the sum of the neglected Hankel singular values, multiplicities included:
1
1 1 2 1 1 + 2 A= 1 1 1 + 2 2 2 C = (1 1 ), D=0
B=
1 , 1
2 trace (S2 )
where 1 2. This system is in balanced canonical form; this means that the grammians are P Q diag(1, 2) : S; this canonical form is a special case of the forms discussed in Ref. 11.
417
We wish to compute the suboptimal Hankel-norm approximants for 1 2. Then we will compute the limit of this family for 2 and 1, and we will show that the system obtained is indeed the optimal approximant. From Eq. 2 2 (43), 2I2 S2 diag(2 1 , 2 2 ); the inertia of is 1, 0, 1; furthermore, from Eq. (45) we obtain
Approximation by Balanced Truncation. A balanced realization of this system is given by
F := H
G = J
= A
1 2 1 ( + 1 ) 2 (1 + 2 )( + 1 ) 1 2 1 + 1 , 1 + 2 ,
1 (1 + 2 )( + 2 ) 2 2 2 ( + 2 )
= 5 1 /4 ,
0 0 0 0 0 0 3 5+5 = , 10
0 0 =
, 3 55 10
= B = C = D
where the reachability and observability grammians are equal and diagonal:
is equal to the inertia of , A has one Since the inertia of A stable and one unstable poles (this can be checked directly by is negative). As 2 we noticing that the determinant of A obtain
P = Q = S = diag(1 , 2 , 3 ), 51 3 = 2 = 1, 2
1 =
5+1 2
The second- and rst-order balanced truncated systems are
= A = C
2 1 21 (1 + 2 ) 0 1 1 + 2 1 2 2
2 1 22 (1 + 2 ) 0 , = D 2
= B
2 1 0
d ,2
= =
,
d ,1
F2 H2 F1 H1
G2 J2 G1 J1
= =
0 0
0 0 0
0 0
This system is not reachable but observable (i.e., there is a pole-zero cancellation in the transfer function). A state-space representation of the reachable and observable subsystem is = A 2 1 , 21 (1 + 2 ) = , B 2 1 C= 1 , 1 + 2 = D 2
Notice that d,2 is balanced, but has singular values which are different from 1 and 2. d,1 is also balanced since G1 H2, but its grammians are not equal to 1. Let c denote the continuous-time system obtained from d by means of the bilinear transformation described in the section entitled A Transformation Between Continuous- and Discrete-Time Systems
. If we choose it to Equations (45) depend on the choice of D be , the limit still exists and gives a realization of the optimal system which is equivalent to A, B, C, D given above. Finally, if 1, after a pole-zero cancellation, we obtain the following reachable and observable approximant: = A 1 2 , 21 (1 + 2 ) = , B 1 2 = C 1 , 1 + 2 = D 1
1 2 2
c
2 1 2 2
2 2 2 1 + 2 2
A C
B = D
2 2 2 +
+ 2 2
where 2( ). Notice that c is balanced. We now compute rst and second reduced-order systems c,1 and c,2 by truncating c:
This is the best antistable approximant of that is, the Nehari solution [see Remark 3(a)]. A Simple Discrete-Time Example In this section we will consider a third-order discrete-time FIR system described by the transfer function: H (z ) = z2 + 1 z3 (63)
c, 2
A2 = C2 = A1 C1
B2 D2 B1 D1
1 2 2 = = 2 + 1 2 2 +
2 1 2 + 2
+ 2 2
c, 1
Let d,2 and d,1 be the discrete-time systems obtaining by transforming c,2 and c,1 back to discrete-time:
We will rst consider approximation by balanced truncation. The issue here is to examine balanced truncation rst in discrete-time and then in continuous-time, using the bilinear transformation of the section entitled A Transformation Between Continuous- and Discrete-Time Systems, and compare the results. Subsequently, Hankel-norm approximation will be investigated.
d ,2 = d ,1 =
F 2 H2 F 1 H1
G 2 J 2 G 1 J 1
2 ( + )3 /2 2 2 ( 2 + )2 /(1 + 2 )
3 /2 ( + )3 /2 2
418
The conclusion is that d,2 and d,1 are balanced and different from d,2 and d,1. It is interesting to notice that the singular value of d,1 is 2 (1 1)/ 5, while that of d,1 is 1; 2 satises 2 2 1. Furthermore, the singular values of d,2 are 52 /4, 52 /4 which satisfy the following interlacing inequalities: 3 < 5 2 /4 < 2 < 5 2 /4 < 1
Again using the transformation of the section entitled A Transformation Between Continuous- and Discrete-Time Systems we obtain the following discrete-time optimal approximant: Hd ,2 (z ) = Hc,2 z1 z+1 = z z 2 3
The numerical values of the quantities above are 1 1.618034, 3 0.618034, 0.66874, 1.08204, 0.413304, 2.47598, and .361241. Hankel-Norm Approximation. The Hankel operator of the system described by Eq. (63) is
Notice that the optimal approximant is not an FIR system. It has poles at 3. Furthermore, the error Hd (z ) Hd ,2 (z ) = 3 1 3 z 2 z 3 (z 2 3 )
01 B 0 B B B = B1 B B @0
0 1 0 0 . . .
1 0 0 0
0 0 0 0
1 C C C C C C C A .
..
is all-pass with magnitude equal to 3 on the unit circle [as predicted by Corollary 2(b)]. The corresponding optimal Hankel matrix of rank 2 is
The SVD of the 3 3 principal submatrix of H is
1 0 1
0 1 0
1 0 0
0r 1 B B B 05 =B B B @r
3 5 1 0 0
0 1 0
r 1 3 C 5 C C 0 C r C C 1 A
5
01 B 0 B B B 3 B =B0 B B 2 B @3
0 3 0 2 3 0 . . .
3 0 2 3 0 3 3
0 2 3 0 3 3 0
2 3 0 3 3 0 4 3
1 C C C C C C C C C A .
..
is In this particular case the 3 3 principal submatrix of H also an optimal approximant of the corresponding submatrix of H .
0 2 0
0 0 3
0r 1 B 5 B B B 0 B B @ r 3
0 1 0
r 1 3 C 5 C C 0 C r C C 1 A
5
(64)
1 0 3
0 3 0
3 0 2 3
0r 1 B B B 05 =B B B @r
3 5 0 3 0
0 1 0
r 1 3 C 5 C C 0 C r C C 1 A
5
where i, i 1, 2, 3, are as given earlier. It is tempting to conjecture that the optimal second-order approximant is obtained by setting 3 0 in Eq. (64). The problem with this procedure is that the resulting approximant does not have Hankel structure. To compute the optimal approximant, the system is rst transformed to a continuous-time system using the transformation of the section entitled A Transformation Between Continuous- and Discrete-Time Systems; we obtain the transfer function Hc (s ) = Hd 1+s 1s = 2 ( s3 s2 + s 1 ) (s + 1 )3
2 1 + 3 0 0
0 0 0
0r 1 B 5 B B B 0 B B @ r 3
0 1 0
r 1 3 C 5C C 0 C r C C 1 A
5
(65)
Notice that the above decomposition can be obtained from Eq. (64) by making use of the freedom mentioned in Eq. (7). Finally it is readily checked that the Hankel matrix consisting of 1 as (1, 1) entry and 0 everywhere else is the optimal approximant of H of rank one. The dyadic decomposition [Eq. (53)] of H is
where Hd is the transfer function dened in Eq. (63). Applying the theory discussed in the section entitled State-Space Construction for Square Systems: Optimal Case, we obtain the following second-order continuous-time optimal approximant: Hc,2 (s ) = ( s2 1 ) 2 ( 1 3 )s + 2 1 s + ( 1 3 )
H (z ) = H1 (z ) + H2 (z ) + H3 (z ) = 1 + 3 1 + 3 z2 z 3 (z 2 3 )
1 1 3 z 2 + 2 z z (z 2 3 )
Notice that each Hi is i-all-pass, and the degree of H1 is one, that of H1 H2 is two, and nally that of all three summands is three.
419
A Higher-Order Example In our last example, we will approximate four well-known types of analog lters by means of balanced truncation and Hankel norm approximation. These are: 1. 2. 3. 4. B Butterworth C1 Chebyshev 1: C2 Chebyshev 2: E Elliptic:
1% ripple in the pass band (PB) 1% ripple in the stop band (SB) 0.1% ripple both in the PB and in the SB
In each case we will consider 20th-order low-pass lters, with pass-band gain equal to 1, and cut-off frequency normalized to 1. Figure 2 shows the Hankel singular values of the fullorder models. It follows from these plots that in order to obtain roughly comparable approximation errors, B and C2 will
HSV Butterworth N = 20 1 0.8 0.6 0.4 0.2 0 0 5 10 15 20 1 0.8 0.6 0.4 0.2 0
HSV Chebyshev 1 N = 20
have to be approximated by systems of lower order than C1 and E; we thus choose to approximate B and C2 by 8thorder models, and we choose to approximate C1 and E by 10th-order models. It is interesting to observe that for the Chebyshev-2 lter the difference of the singular values, 13 20, is of the order 107. Thus C2 has an (approximate) 8thorder all-pass subsystem of magnitude 0.05. Consequently, C2 cannot be approximated with systems of order 13 through 19. Similarly, the Chebyshev-1 lter has an all-pass subsystem of order 3; consequently approximations of order 1, 2, and 3 are not possible. The subscript bal stands for approximation by balanced truncation, the subscript hank stands for optimal Hankelnorm approximation, FOM stands for full-order model, and ROM stands for reduced-order model. Figure 3 gives the amplitude Bode plots of the error systems and tabulates their H norms and upper bounds. We observe that the 10th-order Hankel-norm approximants of C1 and E are not very good in the SB; one way for improving them is to increase the approximation order; another is to compute weighted approximants (see, e.g., Ref. 12). In Table 2, more detailed bounds for the approximants will be given in the case where the approximant contains an optimal D-term Bound 1 is the rst expression on the right-hand side of Eq. (56), and Bound 2 is the second expression on the righthand side of the same expression:
Bound 11 := 9 ( ) + 1 ( )
0 5 10 15 20
+ + 11 ( ) 9 ( ) + + 20 ( ) =: Bound 12 Bound 21 := 11 ( ) + 1 ( ) + + 9 ( ) 11 ( ) + + 20 ( ) =: Bound 22

Finally, Figure 4 shows the amplitude Bode plots of the ROMs obtained by Hankel and balanced reductions.
HSV Chebyshev 2 N = 20 1 0.8 0.6 0.4 0.2 0 0 5 10 15 20 1 0.8 0.6 0.4 0.2 0 0
HSV Elliptic N = 20
10
15
20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Butterworth 9.9998e-01 9.9952e-01 9.9376e-01 9.5558e-01 8.2114e-01 5.6772e-01 2.9670e-01 1.1874e-01 3.8393e-02 1.0402e-02 2.3907e-03 4.6611e-04 7.6629e-05 1.0507e-05 1.1813e-06 1.0622e-07 7.3520e-09 3.6804e-10 1.1868e-11 1.8521e-13
Chebyshev 1 9.5000e-01 9.5000e-01 9.5000e-01 9.4996e-01 9.4962e-01 9.4710e-01 9.3362e-01 8.8268e-01 7.5538e-01 5.5133e-01 3.3740e-01 1.8209e-01 9.8132e-02 6.3256e-02 5.2605e-02 5.0362e-02 5.0036e-02 5.0003e-02 5.0000e-02 5.0000e-02
Chebyshev 2 8.9759e-01 7.3975e-01 5.1523e-01 3.0731e-01 1.6837e-01 9.5205e-02 6.3885e-02 5.3342e-02 5.0643e-02 5.0103e-02 5.0014e-02 5.0002e-02 5.0000e-02 5.0000e-02 5.0000e-02 5.0000e-02 5.0000e-02 5.0000e-02 5.0000e-02 5.0000e-02
Elliptic 9.8803e-01 9.8105e-01 9.6566e-01 9.3602e-01 8.8469e-01 8.0582e-01 6.9997e-01 5.7677e-01 4.5166e-01 3.3873e-01 2.4576e-01 1.7413e-01 1.2135e-01 8.3613e-02 5.7195e-02 3.9014e-02 2.6729e-02 1.8650e-02 1.3613e-02 1.0869e-02
CONCLUSION The computational algorithms that emerged from balancing and Hankel-norm model reduction have found their way to software packages. The Matlab toolboxes, robust control toolbox (13) and -toolbox (14), contain m-les which address the approximation problems discussed above. Two such m-les are sysbal and hankmr; the rst is used for balancing and balanced truncation, while the second is used for optimal Hankel-norm approximation (including the computation of the anti-stable part of the all-pass dilation). We will conclude with a brief discussion of some articles which were written since Glovers seminal paper (4), namely, Refs. 9, 12, 1520. Hankel-norm approximation or balanced truncation can be applied to stable systems. The approximants are guaranteed to be close to the original system, within the bounds given in the section entitled Error Bounds of Optimal and Suboptimal Approximants; the important aspect of this theory is that these bounds can be computed a priori that is, before computing the approximant. The bounds are in tems of the H norm which is a well-dened measure of distance between
Figure 2. Analog lter approximation: Hankel singular values (curves and numerical values).
420

Butterworth: FOMROM Chebyshev 1:: FOMROM 10 0 10 20 30 40 101 100 101 50 101 100 101
10 0 10 20 30 40 50
Chebyshev 2: FOMROM 10 0 10 20 30 40 50 101 100 101 10 0 10 20 30 40 50
Elliptic: FOMROM
101
100
101
Norm of the Error and Bounds
9
Butterworth Chebyshev 2 0.0383 0.0506
hank 0.0388 0.1008 hank 0.4113 0.2700
bal 0.0779 0.0999
2( 9 + . . . + 20) 0.1035 1.2015
11
Chebyshev 1 Elliptic 0.3374 0.2457
bal 2( 11 + . . . + 20) 0.6508 0.3595 1.9768 1.5818
Figure 3. Analog lter approximation: Bode plots of the error systems for model reduction by optimal Hankel-norm approximation (continuous curves), balanced truncation (dashdot curves), and the upper bound [Eq. (57)] (dashdash curves). The table compares the peak values of these Bode plots with the lower and upper bounds predicted by the theory.
stable systems and has a direct relevance in feedback control theory. When the system is unstable, however, Hankel-norm approximation may not be the right thing to do, since it would not predict closeness in any meaningful way. To bypass this difculty Ref. 15 proposes approximating unstable systems (in the balanced and Hankel norm sense) using the (normalized) coprime factors, derived from the transfer function. Another way to go about reducing unstable systems is by adopting the gap metric for measuring the distance between two systems; the gap is a natural measure of distance in the context of feedback control. For details on this approach we refer to the work by Georgiou and Smith (16), and references therein.
The second paper (9) authored by Fuhrmann uses the authors polynomial models for a detailed analysis of the Hankel operator with given (scalar) rational symbol (i.e., single-input single-output systems). Furhmanns computations follow a different approach than the one used in Ref. 4. The Schmidt pairs (singular vectors) are explicitly computed and hence the optimal Hankel approximants are obtained. This analysis yields new insights into the problem; we refer to Theorem 8.1 on page 184 of Ref. 9, which shows that the decomposition of the transfer function in the basis provided by the Schmidt pairs of the Hankel operator provides the balanced realization of the associated state space representation. Another advantage is that it suggests the correct treatment of the polynomial equation [Eq. (38)], which yields a set of n linear equa-
Table 2. Various Upper Bounds for the Norm of the Error Analog Filter Butterworth Chebyshev 2 Optimal D 0.0384 0.1010 Optimal D Chebyshev 1 Elliptic 0.2988 0.2441
9
0.0384 0.0506
hank 0.0389 0.1008 hank 0.4113 0.2700
Bound 11 0.0390 0.5987 Bound 21 0.5030 0.3492
Bound 12 0.0517 0.6008 Bound 22 0.9839 0.7909
11
0.3374 0.2458

Butterworth: ROM (Hankel and Balanced) 0 10 20 30 40 102 101 100 101 0 10 20 30 40 102 101 100 101 Chebyshev 1: ROM (Hankel and Balanced)
421
problem of approximating a nite-dimensional Hankel matrix remains largely unknown. Some partial results are provided in Ref. 20 which relate to results in Ref. 9. BIBLIOGRAPHY
1. G. W. Stewart and J. Sun, Matrix Perturbation Theory, New York: Academic Press, 1990. 2. V. M. Adamjan, D. Z. Arov, and M. G. Krein, Analytic properties of Schmidt pairs for a Hankel operator and the generalized SchurTakagi problem, Math. USSR Sbornik, 15: 3173, 1971. 3. V. M. Adamjan, D. Z. Arov, and M. G. Krein, Innite block Hankel matrices and related extension problems, Amer. Math. Soc. Trans., 111: 133156, 1978. 4. K. Glover, All optimal Hankel-norm approximations of linear multivariable systems and their L -error bounds, Int. J. Control, 39: 11151193, 1984. 5. M. Green and D. J. N. Limebeer, Linear Robust Control, Upper Saddle River, NJ: Prentice-Hall, 1995.
Chebyshev 2: ROM (Hankel and Balanced) 0 10 20 30 40 102 101 100 101 0 10 20 30 40
Elliptic: ROM (Hankel and Balanced)
102
101
100
101
6. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 7. A. C. Antoulas, Lectures on optimal approximation of linear systems, Draft, Dept. Elec. Comp. Eng., Rice Univ., Houston, TX, 1998. 8. P. A. Fuhrmann, Linear Systems and Operators in Hilbert Space, New York: McGraw-Hill, 1981.
Figure 4. Analog lter approximation: Bode plots of the reducedorder models obtained by balanced truncation (dashdot curves) and Hankel-norm approximation (continuous curves).
tions, instead of 2n 1 as in Eq. (39). Further developments along the same lines are given in Ref. 21. Weighted Hankel-norm approximation was introduced in Ref. 17. Reference 12 presents a new frequency-weighted balanced truncation method. It comes with explicit L norm error bounds. Furthermore, the weighted Hankel-norm problem with anti-stable weighting is solved. These results are applied to L -norm model reduction by means of Hankel-norm approximation. Reference 18 discusses a rational interpolation approach to Hankel-norm approximation. The next article (19) addresses the issue of balanced truncation for second-order form linear systems. These are systems which arise in applications and which are modeled by position and velocity vectors. Thus in simplifying, if one decides to eliminate the variable qi, the derivative q i should be eliminated as well, and vice versa. Toward this goal, new sets of invariants are introduced and new grammians as well. However, no bounds on the H norm of the error are provided. The question whether in bounds in Eqs. (57) and (62) are tight arises. It can be shown that this property holds for systems having zeros which interlace the poles; an example of such systems are RC circuits [this result follows from Eq. (57) and Remark 5(d), after noticing that the Hankel operator in this case is positive denite]. For an elementary exposition of this result we refer to Ref. 22; for a more abstract account, see Ref. 23. The problems of approximating linear operators in the 2induced norm, which are (a) nite-dimensional, unstructured and (b) innite-dimensional structured (Hankel), have been solved. The solutions of these two problems exhibit striking similarities. These similarities suggest the search of a unifying framework for the approximation of linear operators in the 2-induced norm. For details see Ref. 24. In particular the
9. P. A. Fuhrmann, A polynomial approach to Hankel norm and balanced approximations, Linear Alg. Appl., 146: 133220, 1991. 10. J. A. Ball, I. Gohberg, and L. Rodman, Interpolation of Rational Matrix Functions, Basel: Birkha user Verlag, 1990. 11. R. Ober, Balanced parameterization of classes of linear systems, SIAM J. Control Optim., 29: 12511287, 1991. 12. K. Zhou, Frequency-weighted L norm and optimal Hankel norm model reduction, IEEE Trans. Autom. Control, 40: 16871699, 1995. 13. R. Y. Chiang and M. G. Safonov, Robust Control Toolbox, Natick, MA: The MathWorks, 1992. 14. G. J. Balas et al., -analysis and Synthesis Toolbox, Natick, MA: The MathWorks, 1993. 15. D. G. Meyer, A fractional approach to model reduction, Proc. Amer. Control Conf., Atlanta, 1988, pp. 10411047. 16. T. T. Georgiou and M. C. Smith, Approximation in the gap metric: Upper and lower bounds, IEEE Trans. Autom. Control, 38: 946951, 1993. 17. G. A. Latham and B. D. O. Anderson, Frequency-weighted optimal Hankel normal approximation of stable transfer functions, Syst. Control Lett., 5: 229236, 1985. 18. T. Auba and Y. Funabashi, Interpolation approach to Hankel norm model reduction for rational multi-input multi-output systems, IEEE Trans. Circuits Syst. I; Fundam. Theory Appl., 43: 987995, 1996. 19. D. G. Meyer and S. Srinivasan, Balancing and model reduction for second-order form linear systems, IEEE Trans. Autom. Control, 41: 16321644, 1996. 20. A. C. Antoulas, On the approximation of Hankel Matrices, in U. Helmke, D. Pra tzel-Wolters, and E. Zerz, (eds.), Operators, Systems and Linear Algebra, Stuttgart: Teubner Verlag, 1997, pp. 1723. 21. P. A. Fuhrmann and R. Ober, A functional approcah to LQG balancing, Int. J. Control, 57: 627741, 1993. 22. B. Srinivasan and P. Myszkorowski, Model reduction of systems zeros interlacing the poles, Syst. Control Lett., 30: 1924, 1997.
422
LINEAR NETWORK ELEMENTS
23. R. Ober, On Stieltjes functions and Hankel operators, Syst. Control Lett., 27: 275278, 1996. 24. A. C. Antoulas, Approximation of linear operators in the 2-norm, Linear Algebra Appl., special issue on Challenges in Matrix Theory, 1998.
A. C. ANTOULAS
Rice University
LINEAR ELECTRIC COMPONENTS. See LINEAR NETWORK ELEMENTS.
LINEAR FREQUENCY MODULATION. See CHIRP

MODULATION.
LINEARLY SEPARABLE LOGIC. See THRESHOLD LOGIC.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Model Reference Adaptive Control Standard Article Anuradha M. Annaswamy1 1Massachusetts Institute of Technology, Cambridge, MA, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1022 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (132K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Identification Model Reference Model Model Reference Adaptive Control Parameter Identification Persistent Excitation About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
MODEL REFERENCE ADAPTIVE CONTROL
441

The aim of control is to keep the relevant outputs of a given dynamic process within prescribed limits. Denoting the process to be controlled as a plant and denoting its input and output as u and y, respectively, the aim of control is to keep the error (e1 y yd) between the plant output and a desired output yd within prescribed values. If yd is a constant, the control problem is referred to as regulation and if yd is a function of time, the problem is referred to as tracking. In the former case, the value of yd around which the system is to be
regulated is also referred to as a set point or operating point. The goal of control, in both cases, is to ensure that the output error e1 is as small as possible, in the presence of disturbances and modeling errors, for all time, and that the controlled system is stable. Feedback control is one procedure by which regulation and tracking can be accomplished in a number of dynamic processes. When the differential equations describing the behavior of the plant are linear and known a priori, powerful analytical techniques in both time domain and frequency domain have been developed. When the characteristics of the plant are unknown, both regulation and tracking can be viewed as adaptive control problems. The eld of adaptive control in general, and model reference adaptive control in particular, has focused on problems where the uncertainties in the system are parametric. Such parametric uncertainties occur due to a variety of reasons on practical applications. Typically system dynamics, which are invariably nonlinear, are linearized to derive the requisite linear controller. The resulting linear model and its parameters are therefore dependent on and vary with the operating condition. Parameters also may vary due to aging, disturbances, or changes in the loading conditions. Parameters may be unknown due to approximations made in the modeling process. In all these cases, a controller that provides a uniformly satisfactory performance in the presence of the parametric uncertainties and variations is called for. The adaptive approach to this problem is to design a controller with varying parameters, which are adjusted in such a way that they adapt to and accommodate the uncertainties and variations in the plant to be controlled. By providing such a time-varying solution, the exact nature of which is determined by the nature and magnitude of the parametric uncertainty, the closed-loop adaptive system seeks to enable a better performance. The results that have accrued in the eld of adaptive control over the past three decades have provided a framework within which such time-varying, adaptive controllers can be designed so as to yield stability and robustness in various control tasks. Model reference adaptive control refers to a particular class of adaptive systems. In this class, adaptive controllers are designed by using a reference model to describe the desired characteristics of the plant to be controlled. The use of such reference models facilitates the analysis of the adaptive system and provides a stability framework. Two philosophically different approaches, indirect control and direct control, exist for synthesizing model reference adaptive controllers. In the indirect approach, the unknown plant parameters are estimated using a model of the plant before a control input is chosen. In the direct approach, an appropriate controller structure is selected and its parameters are directly adjusted so that the output error is minimized. For the sake of mathematical tractability, the desired output yd needs to be characterized in a suitable form, which is generally accomplished by the use of a reference model. Thus in a model reference problem formulation, the indirect approach employs both an identication model and a reference model while the direct approach uses a reference model only. We describe these models in further detail below. IDENTIFICATION MODEL Mathematical modeling is an indispensable part of all sciences, whether physical, biological, or social. One often seeks
442
to characterize the cause-and-effect relations in an observed phenomenon using a model and tune the model parameters so that the behavior of the model approximates the observed behavior for all cases of interest. One form of quantitative models is mathematical and in the cases of dynamic systems, these take the form of differential or difference equations. Alternatively, a general mathematical model which represents the inputoutput behavior of a given process can also be used for identication. The model obtained in the latter case is often referred to as an identication model, since those from the rst approach are either not available or too complex for control purposes. Often, especially for linear problems, frequency-domain methods are used to identify the system parameters. When measurement noise is present, the identication methods include statistical criteria so as to determine the model that best ts the observed data. Systems identication, which is based on such approaches, is a well-developed area of systems theory (1).
while in the latter the performance index implicitly includes the reference model as Ii =
0
(y p A m y p ) T Qi ( y p Am y p ) + uT Ru dt ,
Qi > 0
In both cases, it can be shown that quadratic optimization theory can be used to determine the control input. In the former case, the optimal input has the form u(t ) = Km ym (t ) + K p x p (t ) and in the latter case we have u(t ) = K p x p (t ) The structure of the controller can be used in an adaptive situation when the parameters of the plant are unknown, though the control parameters have to be estimated to compensate for parametric uncertainties. Reference Model with Inputs
REFERENCE MODEL The use of a reference model for controls can be traced to aircraft systems. Often, the situation therein is such that the controls designer is sufciently familiar with the plant to be controlled and its desired properties; thus by choosing the structure and parameters of a reference model suitably, its outputs can be used as the desired plant response. While in principle such a model can be linear or nonlinear, considerations of analytical tractability have made linear reference models more common in practice. Explicit and Implicit Model-Following Two methods that have been studied extensively in this context include explicit and implicit model-following methods (2), both of which include the use of a reference model described by the homogeneous differential equation y m = A m ym (1) In Eq. (1), the output of the reference model was specied as the output of a homogeneous differential equation. A more general formulation of a reference model includes external inputs and is of the form x m = Am xm + Bm r, ym = Cm xm (3)
where the constant matrix Am mm is chosen so that the desired dynamics in terms of transient behavior, decoupling of modes, bandwidth, and handling qualities is captured. Suppose that the plant to be controlled is described adequately by an nth-order differential equation with m(m n) outputs as x p = Apxp + Bpu y p = Cp x p
where Am is a stable n n matrix with constant elements, Bm and Cm are constant matrices with appropriate dimensions, and r is an arbitrary continuous uniformly bounded input. The goal of the control input u into the plant in Eq. (2) so that the output yp(t) tracks the output ym(t) as closely as possible. In this case, the reference input r along with the model in Eq. (3) with the parameters Am, Bm, Cm determines the output of the reference model. The introduction of the reference inputs signicantly increases the class of desired trajectories that can be represented by a reference model. For a perfect model following to occur, the differential equations governing yp and ym as well as the initial conditions yp(t) and ym(t) have to be identical. This imposes restrictive conditions on the matrices Ap, Bp, Am, and Bm, in terms of their canonical forms. It has been shown by Berger that the requisite control input in this case is of the form u(t ) = K p x p (t ) + Km xm (t ) + Kr r(t ) In an adaptive situation, it is more reasonable to have the objective of asymptotic model-following where yp(t) is desired to follow ym(t) as t . The problem in this case is to determine the conditions under which this can be achieved amidst parametric uncertainties. MODEL REFERENCE ADAPTIVE CONTROL The model reference adaptive control (MRAC) problem can be qualitatively stated as the following: Given a plant P with an inputoutput pair u( ), yp( ), along with a stable reference model M whose inputoutput pair is given by r( ), ym( ),
(2)
The reference model in Eq. (1) is chosen so that the output yp follows ym as closely as possible. The explicit and implicit model-following methods are based on different performance indices of the model-following error yp ym. In explicit modelfollowing, the performance index is of the form Ie =
0
[(y p ym )T Qe (y p ym ) + uT Ru] dt
443
where r is a bounded piecewise continuous function, determine the control input u(t) for all t t0 so that
t
M r + C( ) e
lim |y p (t ) ym (t )| = 0
Much of what is well known in MRAC concerns the case when the plant and model are linear and time-invariant though there have been a number of advances in the early 1990s in adaptive control of nonlinear systems (3,4) by making use of results in feedback linearization (5). It becomes evident from the statement of the problem that considerable prior information regarding the plant P is needed to have a well-posed problem. Such information is critical while determining the structure of the reference model and the controller. For instance, the controller must be such that it makes use of all measurable signals in the system, is differentiator-free, and results in a bounded control input. For the plant output to follow the model output, the class of models M has to be constrained in some sense. Obviously, M depends on the prior information regarding the class P of plants. For example, if the reference input r is a pulse train and the model M has a unity transfer function, it is clear that the output of the plant cannot follow ym asymptotically with a bounded input u and a differentiator-free controller. To determine M for linear time-invariant plants, results related to model-following in linear systems theory (6,7), LQG methods (8), and pole-placement can be utilized. Once the classes of plants P and M are determined, the structure of the controller that generates u can be found. When the parameters of the plant are known, the requisite controller has a linear structure. However, in order to compensate for the parametric uncertainty in P, the model reference adaptive controller has a nonlinear structure where the nonlinearity arises due to the fact that the controller parameters are adjusted on-line as a function of the system variables that are measured. To better illustrate the nature of the nonlinearity in MRAC, we dene the two parts of an MRAC, the algebraic and the analytic. In what follows, we focus our attention only on the case when the plant P and the model M are linear and time-invariant. Algebraic Part and Analytic Part We parameterize the controller C by a vector : m, where C is linear and time-invariant if is a constant. By using model reference approaches, one can determine the controller structure and a parameter * such that if is equal to * in C , the closed-loop system determined by the plant together with the model has an output which asymptotically follows ym. Such a design process marks the rst step of an MRAC design and is referred to as the algebraic part. The aim of adaptation is to generate the control input u such that limtyp(t) 0 when the plant parameters are unknown. Since u(t) is determined by the manner in which the parameter (t) is adjusted in the controller, the problem can be equivalently stated in terms of (t). The second part of a MRAC design, referred to as the analytic part, consists of determining the rule by which (t) is to be adjusted at each instant of time so that the closed-loop remains stable and the output error e(t), dened as e(t) yp(t) ym(t), tends to zero as t . The adjustment rule for (t) is referred to as the adaptive law.
Figure 1. The MRAC problem.
The MRAC Problem With the above denitions, the MRAC problem can be stated below (Fig. 1). Suppose the inputoutput pair of a linear timeinvariant plant P with unknown parameters is u( ), yp( ). 1. Determine the class M of stable LTI reference models such that if the inputoutput pair of the model is given by r( ), ym( ), a uniformly bounded input u to the plant P, generated by a differentiator-free controller, exists which assures that
t
lim |y p (t ) ym (t )| = 0
(4)
2. Determine a differentiator-free controller C() parameterized by a vector (t) m, which generates u, such that for a constant value *, the transfer function of the plant together with the controller is equal to the transfer function of M. 3. Determine a rule for adjusting (t) such that the closedloop system is stable and Eq. (4) is satised. When a disturbance is present, such an asymptotic output tracking may not be possible if very little information is available about the disturbance. In such a case, the goal of MRAC is to minimize the error between yp and ym as much as possible. Since stability is vital to the satisfactory operation of any dynamic system and since in general adaptive systems are nonlinear, one of the major difculties in designing adaptive systems is ensuring their stability properties. This often serves as a guideline while solving the MRAC problem stated above. Error Model Approach The solution of the MRAC problem is often signicantly facilitated by an error model approach. This approach consists of studying the relationship between two kinds of errors commonly present in any adaptive system: (1) the tracking error e between the plant output and the model output and (2) the between the estimated adaptive parameter parameter error and its desired value. If the evolution of the error e is determined by the differential equation (t ), t ) e (t ) = f 1 (e(t ), (5)
then the MRAC problem can be formulated as the determination of the adaptive law (t ) = f 2 (e(t ), t ) (6)
444
in such a way as to ensure closed-loop stability and asymptotic tracking. Focusing attention directly on the error rather than on the actual response of the plant or the reference model enables the designer to concentrate on the essential features of the problem and determine the adaptive law by inspection. Such an approach has facilitated the design of many adaptive systems both in the disturbance-free case as well as when disturbances and modeling errors are present. Solution to the MRAC Problem By the year 1980, several solutions to the MRAC problem when the plant to be controlled is linear and time-invariant were proposed. One such solution is summarized below. Suppose the plant P to be controlled is described by the transfer function Wp ( s ) = k p Z p (s ) R p (s )
The structure of the model transfer function guarantees that a constant vector * exists such that when (t) *, the plant together with the controller has the same transfer function as that of the model. The structure of the controller guarantees that the underlying error model is of the form e1 = kp T ] Wm (s )[ km (8)
where kp as well as the coefcients of the monic polynomials Zp(s) and Rp(s) are unknown. The degree of Rp(s), n, and the degree of Zp(s), m n 1, are assumed to be known. Zp(s) is assumed to be a Hurwitz polynomial. The sign of kp is assumed to be known. (This assumption was relaxed in (9) by replacing sign(kp) in the adaptive law by a nonlinear gain. In this article, we assume that sign(kp) is known for ease of exposition). The reference model M is chosen to have a transfer function Zm (s ) Wm (s ) = km Rm ( s ) where Rm(s) and Zm(s) are monic Hurwitz polynomials with degree n and n m. The structure of the controller and the adaptive laws for adjusting the control parameters are given separately for the cases when the relative degree n* n m is unity and when it is greater than or equal to two. In both cases, the objective is to solve problems 13 stated above and accomplish the tracking stated in Eq. (4). Case 1: n* 1. In this case, the model transfer function is chosen to be strictly positive real (SPR) (10). This can be accomplished since the relative degree is one and Wm(s) has asymptotically stable poles and zeros, by interlacing the zeros with the poles. The control input u is chosen as
*. The structure of the adaptive law in Eq. where (7) and the fact that Wm(s) is SPR enables one to select an appropriate Lyapunov function of all of the states of the adaptive system. As a result, the closed-loop system remains bounded and e1(t) tends to zero as t . Case 2: n* 2. When the relative degree is greater than unity, even though the same error model structure as in Eq. (8) can be derived, it is not possible to choose an SPR model transfer function. This requires additional processing of the error signal and the sensitivity function in order to construct the necessary adaptive law. In particular, an augmented error 1 is constructed as
1
= e1 + e2
e2 = T Wm (s ) Wm (s ) T Dening the ltered sensitivity function as where = Wm (s ) one can show that the underlying error model, when kp is known, is simplied from Eq. (8) to
1
T. =
(9)
As a result, an adaptive law of the form . = 1 + T (10)
u = T (t )(t ) 1 = 2 = 1 + u 2 + y p
T T T = [r, 1 , y p , 2 ] T T T = [k , 1 , 0 , 2 ]
where (n1)(n1) is asymptotically stable, with det(sI ) Zm(s), (, ) is controllable, n is the control parameter to be adjusted appropriately so that (4) is achieved, and is a sensitivity function which essentially estimates the state of the system on-line. The requisite adaptive law, assuming that km 0, is given by = sign(k p ) e1 , >0 (7)
can be chosen. Such an adaptive law guarantees that the closed-loop system remains bounded and that 1(t) 0 asymptotically. The normalization in Eq. (10) is needed to establish the global boundedness of signals. Recently, other adaptive control structures and adaptive laws have been proposed (11) that do not employ such normalization, which has the potential to lead to better transient performance. The most important distinction between the approach in (10) and that in (9) is that the former prescribes an explicit Lyapunov function for the adaptive system and hence provides bounds on the tracking errors and parameter errors that can be estimated a priori. Results in model reference adaptive control have been extended in several different directions, including robustness properties in the presence of disturbances and unmodeled dynamics, time-varying parameters and, most notably, adaptive control of nonlinear dynamic systems (3,4,10,12,13). Extensions to multivariable adaptive control and stable adaptive control in the presence of very few assumptions on the plant have also been proposed (10). Improvement of the transient response of the adaptive system by using multiple models and switching and tuning has also been proposed (14). Adaptive control techniques including self-tuning regulators, and auto-
MODULATION ANALYSIS FORMULA
445
matic tuning, as well as practical aspects of control implementation and applications, can be found in Ref. (15). PARAMETER IDENTIFICATION The discussions earlier pertain to the global stability of the adaptive system and conditions under which the output error 1 will converge to zero. However, if one desires to match the closed-loop transfer function with that of the model, then the parameter estimate (t) must approach * asymptotically. In other words, parameter identication has to be carried out. This also will ensure that the adaptive system will have an improved transient response as initial conditions and reference input change and better robustness with respect to different kinds of perturbations. In the context of Case 1 above, the problem of parameter identication can be stated as follows: The output error 1 and satisfy the differential equations the parameter error
shown to be persistently exciting in 2n if the input has n distinct frequencies and the system is controllable. BIBLIOGRAPHY
1. L. Ljung and T. So derstrom, Theory and Practice of Recursive Identication, Cambridge, MA: M.I.T. Press, 1985. 2. H. Erzberger, Analysis and design of model following control systems by state space techniques, In Proc. JACC, 1968. 3. I. Kanellakopoulos, P. V. Kokotovic, and A. S. Morse, Systematic design of adaptive controllers for feedback linearizable systems, IEEE Trans. Autom. Control, 34: 12411253, 1991. 4. D. Seto, A. M. Annaswamy, and J. Baillieul, Adaptive control of nonlinear systems with a triangular structure, IEEE Trans. Autom. Control, 39: 14111428, 1994. 5. A. Isidori, Nonlinear Control Systems, 2nd ed., New York: Springer-Verlag, 1989. 6. A. S. Morse, Structure and design of linear model following systems, IEEE Trans. Autom. Control, 18: 346354, 1973. 7. S. H. Wang and C. A. Desoer, The exact model matching of linear multivariable systems, IEEE Trans. Autom. Control, 17: 347 349, 1972. 8. G. Stein and M. Athans, The LQG/LTR procedure for multivariable feedback control design, IEEE Trans. Autom. Control, 32: 105114, 1987. 9. D. R. Mudgett and A. S. Morse, Adaptive stabilization of linear systems with unknown high frequency gains, IEEE Trans. Autom. Control, 30: 549554, June 1985. 10. K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems, Englewood Cliffs, NJ: Prentice-Hall, 1989. , I. Kanellakopoulos, and P. V. Kokotovic , Nonlinear de11. M. Krstic sign of adaptive controllers for linear systems, IEEE Trans. Autom. Control, 39: 738752, 1994. 12. S. Sastry and M. Bodson, Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1989. 13. P. Ioannou and J. Sun, Stable and Robust Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1995.
14. K. S. Narendra and J. Balakrishnan, Adaptive control using switching and tuning, In Proc. Eighth Yale Workshop on Appl. Adaptive Syst. Theory, New Haven, CT: Yale University, 1994. 15. K. J. Astrom and B. Wittenmark, Adaptive Control, 2nd Ed., Reading, MA: Addison-Wesley, 1995.
, e = Ae + bT . = e1
e1 = h T e
(11)
where hT(sI A)1b is SPR. Find the conditions on under will converge to zero. On the other hand, in which e1 and Case 2 the parameter error evolves as . = u uT where u= 1 + T The problem once again is the determination of conditions on (t) 0 as t . These conditions are labeled u under which as persistent excitation of the corresponding signal and are discussed further below.
PERSISTENT EXCITATION A function u : m is said to be persistently exciting in m if it satises the inequality
t + T0 t
(12)
ANURADHA M. ANNASWAMY
Massachusetts Institute of Technology
u( )u ( ) d I
T
t t0
(13)
for some constants t0, T0, and . Several statements equivalent to (13) can be given, one of which is that, for every unit vector w in m, we obtain 1 T0
t + T0 t
MODELS, FUZZY. See FUZZY MODEL FUNDAMENTALS. MODELS OF ELECTRICAL MACHINES. See ELECTRIC
MACHINE ANALYSIS AND SIMULATION.
|uT ( )w| d
t t0
(14)
MODULAR INSTRUMENTATION. See CAMAC. MODULARIZATION. See SUBROUTINES. MODULATION. See DIGITAL AMPLITUDE MODULATION; INFORMATION THEORY OF MODULATION CODES AND WAVEFORMS.
It can be shown that for m 2n, if satises Eq. (13), then (t) 0 in Eq. (12), and if satises Eq. (13), the errors limt in Eq. (11) converge to zero asymptotically, which ensures that parameter identication will take place. Typically, a vector signal generated using n distinct frequencies can be shown to be persistently exciting in n. The state of a 2nth-order asymptotically stable system can be
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Multivariable Systems Standard Article David Castaon1 1Boston University, Boston, MA, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1023 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (174K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Multivariable Linear Systems About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
MULTIVARIABLE SYSTEMS
51
MULTIVARIABLE LINEAR SYSTEMS Introduction With the development of miniaturized, cheap, sensor, and actuator technology, many of todays control problems must coordinate the actions of multiple actuators, based on multiple output signals from diverse sensors. Such systems, illustrated in Fig. 1, are referred to as multi-input, multi-output (MIMO) systems. An important class of MIMO systems is linear MIMO systems, whose relationship between input and output signals is represented by linear transformations. This article introduces the basic concepts for analyzing linear time-varying and time-invariant MIMO systems for continuous time input and output signals. Extensions of the concepts in this article to discrete time signals are straightforward and are found in the references at the end. The chapter discusses input-output and state-space models of linear MIMO systems and introduces the concepts of controllability and observability for state-space models. It also discusses modal analysis for state-space models of time invariant MIMO systems, MIMO poles and zeros, and singular-value analysis for characterizing the frequency response of linear, time-invariant MIMO systems. Input-Output Models Input-output models capture the essential relationships between inputs to a system and the outputs of that system. Instead of focusing on the internal representation and operation of a system, input-output models represent these internal effects implicitly within a transformation from inputs to outputs, as illustrated in Fig. 1. In this section, we review results on input-output models of linear MIMO systems in continuous time. Consider a system with input u and output y, where the relationship between input and output is denoted by the map
u1
y1
...
um
System
...
yp
Figure 1. Multi-input, multi-output system.
52
y H(u). A system is linear if it satises the following properties: 1. Homogeneity. For any scalar multiple a, the response to a scaled input is equal to the scaled output response. That is H (au ) = aH (u ) 2. Superposition. Given two inputs u1, u2, the response to the combined input u1 u2 is the sum of the individual output responses. That is H (u1 + u2 ) = H (u1 ) + H (u2 ) For MIMO continuous time systems, the inputs consist of vector-valued signals u(t) Rm and the outputs consist of vector-valued signals y(t) Rp, both of which are functions of the continuous time parameter t. Assume that such a system is well behaved, in that small changes in the input signals over a nite time result in small changes in the output signal over the same interval. The general form of the input-output description of such systems is given by y(t ) =

so that
t
y(t ) =
H (t , )u( ) d
(5)
Another important property of the example is that the impulse response depends only on the difference t . This property is known as time invariance. Denition. The input-output system Eq. (1) is time-invariant if H (t , ) = H (t , 0 ) H (t ) (6)
for all t and . The last equality is a slight abuse of notation introduced by convenience. If Eq. (1) is time-invariant, then y(t) is the convolution of H(t) and u(t): y(t ) =
+
H (t )u( ) d =
H ( )u(t ) d
(7)
Such a system is called a linear time-invariant (LTI) system. An LTI system is causal if and only if H(t) 0 for all t 0. State-Space Models
H (t , )u( ) d
(1)
In the previous integral equation, H(t, ) is the p m impulse response or weighting pattern, that is, if the components of u(t) are impulses of the form u j (t ) = (t t0 ), then yi (t ) = hi j (t t0 ), i = 1, . . ., p where hij(t, ) is the ijth element of H(t, ). Example. Consider a two-input, two-output system described by the following impulse response: (3) uk (t ) = 0, k = j (2)
In contrast with input-output models, state-space models provide an explicit representation of the internal operations of a system, leading to the transformation between input time functions and output time functions. The state of a system at a given time provides a complete summary of the effects of past inputs to the system, which is sufcient to uniquely characterize system output responses to future inputs. This article focuses on linear MIMO systems where the state takes values in a nite-dimensional space of dimension n. Such models are analyzed with concepts from linear algebra and matrix differential equations. The general form of a state-space model for a linear system with m inputs u(t) Rm and p outputs y(t) is given by the matrix differential equation d x(t ) = A(t )x(t ) + B(t )u(t ) dt y(t ) = C(t )x(t ) + D(t )u(t ) (8) (9)
H (t , ) = (t ) + 0.5 e (t ) u 1 (t ) (t ) + e 2 (t ) u 1 (t ) 0 e 3 (t ) u 1 (t )
where u1(t) is the unit step function, which is 0 for t 0 and 1 otherwise. The output of the system is given by y1 (t ) = y2 (t )

with initial condition specied at time t0 as x(t0) x0. In Eqs. (89), x(t) Rn is the system state at time t, A(t) is the n n system matrix, B(t) is the n m input matrix, C(t) is the p n output matrix, and D(t) is the p m feedthrough matrix. Example. Consider the state-space model specied by the following matrices: 1 0 0 1 0 A = 0 2 0 ; B = 1 0 0 0 3 0 1
H (t , )
u1 ( ) d u1 ( )
In the previous example, note that the impulse response H(t,) is zero for t. Thus, the output y(t) depends only on inputs up to time t. Systems with this special property are known as causal. Denition. The input-output system Eq. (1) is causal if H (t , ) = 0 for t < (4)
C=
0.5 0
0 1
0 ; 1
D=
1 1
0 0
This state-space model is equivalent to the input-output model in the previous subsection, as seen later.
53
An important property of state-space models is that they generate causal input-output maps, because past inputs affect only future outputs through the value of the current state. To determine the implied relationship between inputs and outputs in Eqs. (89), we need expressions for the solution of the vector differential equation in Eq. (8). First consider solutions of the unforced (homogeneous) equation d x(t ) = A(t )x(t ) dt (10)
If the initial time at t0 and it is assumed that the system starts at rest, the output y(t) becomes
t
y(t ) =
C(t )
A (t , )B ( )u ( )
d + D(t )u(t )
(14)
which is a causal input-output model for the system. Thus, every state-space model leads to a corresponding input-output model for the system. The question of whether input-output models have a nite-dimensional state-space representation is addressed in the next section. Example (continued). Because of the special diagonal, time-invariant form of the A matrix, it is straightforward to compute the state transition matrix as
starting from arbitrary initial conditions x(t0) at time t0. Under the assumption that the matrix A(t) is piecewise continuous, there exists a unique continuous solution of Eq. (10) given by x(t ) =
A (t , t0 )x(t0 )
(11)
for some n n matrix continuous matrix A(t, t0). The matrix A(t, ), known as the state transition matrix, has the following properties: 1. A(t, t) I, where I is the n n identity matrix. 2. If A(t) A is time-invariant, then
A (t , )
(t ) e 0 ( t , ) = A 0
0 e
2 (t )
0 0 e 3 (t )
Because C, B do not depend on time, the product CA(t, )B is written as C

A (t , )B
= e A (t ) Substituting Eq. (14) yields
0.5e ( t ) e 2 (t )
0 e 3 (t )
where the matrix exponential is dened by the series

t
At
i= 0
(At )i i!
y(t ) = =
0.5e ( t ) e 2 (t )
1 0 u( ) d + 1 e 3 (t )
0 u(t ) 0
3. A(t, t0) is the unique continuous solution of the n n matrix differential equation d X (t ) = A(t )X (t ) dt with initial condition X(t0) I. 4. For every t, , t0, the following compositional property holds:
A (t , ) A ( , t0 )
(t ) + 0.5e (t ) (t ) + e 2 (t )
0 u( ) d e 3 (t )
which is equivalent to the input-output model in the previous subsection. State-space models have important qualitative properties which are useful for control design. Of particular interest are the properties of stability, controllability, and observability, discussed in greater length in the articles by Sontag, Freudenberg and Vidyasagar. Some of the relevant concepts are summarized here. Stability. Given an n-dimensional state-space model with matrices [A(t), B(t), C(t), D(t)], there are two types of stability properties of interest, internal stability and input-output stability. Internal stability is the stability of trajectories of the homogeneous system Eq. (10) and thus involves only the matrix A(t). Different types of internal stability are possible. Asymptotic stability corresponds to all solutions to Eq. (10) converging to 0 as t , and exponential stability corresponds to all solutions converging to zero exponentially fast, that is, x(t ) Ke at (15)
A (t , t0 )
5.
A (t , )
= A(t )
A (t , ).
Given the state transition matrix A(t, ), the general solution of Eq. (8) is written as
t
x(t ) =
A (t , t0 )x(t0 )
+
t0
A (t , )B ( )u ( )
d ,
t t0 (12)
It is straightforward to use the properties of A(t, ) to verify that Eq. (11) satises Eq. (8). Using Eq. (9), the input-output relationship of a state-space model is written
y(t ) = C(t ) (t , t0 )x(t0 )

t
+
t0
C(t )
A (t , )B ( )u ( )
d + D(t )u(t )
(13)
for some K, a 0, where the vector norm is the standard Euclidean vector magnitude. Exponential stability is equivalent to
A (t , )
Me a (t ) , t
(16)
54
where M is the matrix norm corresponding to the square root of the largest eigenvalue of the matrix MTM. Input-output stability refers to the full input-output map in Eq. (13). A MIMO system is said to be bounded-input/ bounded-output (BIBO) stable if bounded inputs lead to bounded outputs, that is, if u(t) K1 for all t t0 implies y(t) K2 0 for all t t0. For state-space models, if B(t), C(t), D(t) are bounded, then exponential stability guarantees BIBO stability. However, the converse is not true, as shown by the example below:
Conversely, assume that the system is controllable but that the matrix WC(t0, t1) is not invertible. Then, there must exist a nonzero vector z Rn such that zTWC(t0, t1)z 0, that is,
t1 t0
zT
A (t0 , )B ( )B
( )
T A (t0 , )z
d = 0
d x(t ) = x(t ) + u(t ) dt y(t ) = u(t )

In this example, the state x does not affect the output y. Thus, the system is internally unstable although it is BIBO stable. Controllability and Observability. The concepts of controllability and observability of state-space models characterize the degree to which inputs and outputs determine the internal state trajectory of a state-space model. This section presents an overview of these concepts for linear, state-space models. For a more detailed exposition of these concepts see CONTROLLABILITY AND OBSERVABILITY. Consider a linear system with a state-space model whose matrices are A(t), B(t), C(t), D(t), which are assumed to be continuous functions of time. The system is said to be controllable on the interval [t0, t1] if, given any initial state x0 at t t0 and any desired nal state x1 t t1, it is possible to specify a continuous input u(t), t0 t t1 so that if x(t0) x0, then x(t1) x1. The system is observable on the interval [t0, t1] if, given knowledge of u(t), t0 t t1 and y(t), t0 t t1, the initial state x(t0) (and thus the entire state trajectory x(t), t [t0, t1]) is uniquely determined. Conditions for verifying controllability and observability are determined from the explicit representation of the trajectories of state-space models in Eqs. (11) and (13). Using Eq. (11), controllability is equivalent to nding a control u(t), t [t0, t1], which solves
Since the integrand is nonnegative, it follows that zTA(t0, t) B(t) 0 for all t [t0, t1]. Controllability implies that a control exists which yields x(t1) 0 for x(t0) z. From Eq. (17), this requires that
z=
t1 t0
A (t0 , )B ( )u ( )
Because of the choice of z, it follows that
zT z =
t1 t0
[z T
A (t0 , )B ( )]u ( )
d = 0
implying that z 0. The controllability Gramian has many properties. It is symmetric, positive-semidenite for all t1 t0, and satises the following matrix differential equation:
d W (t , t1 ) = A(t )WC (t , t1 ) + WC (t , t1 )AT (t ) B(t )BT (t ) (19) dt C WC (t1 , t1 ) = 0

Direct integration of this matrix equation is preferred to the integral expression for numerically computing the controllability Gramian. In the special case that the matrices A, B do not depend on time, there is a simple algebraic test for controllability. Dene the matrix Mc (A, B ) = (B AB A2 B An 1 B )
x(t1 )
A (t1 , t0 )x(t0 )
t1 t0
A (t1 , )B ( )u ( )
(17)
for any pair of states x(t1), x(t0). Dene the controllability Gramian as the n n matrix
WC (t0 , t1 ) =
t1 t0
where n is the dimension of the state. The state-space model is controllable if and only if the rank of Mc(A, B) is n. Observability is characterized similarly. Using Eq. (13), it is sufcient to consider the case where u(t) 0 for t [t0, t1], so that the output response is given by y(t ) = C(t ) (t , t0 )x(t0 ), t > t0 (20)
A (t0 , )B ( )B
( )
T A (t0 , )
(18)
The system is controllable on [t0, t1] if and only if the matrix WC(t0, t1) is invertible. To establish this, if the inverse exists, 1 then the control u(t) BT(t)T A(t0, t)WC (t0, t1)[x(t0) A(t0, t1)x(t1)] is continuous and, when substituted in Eq. (17) yields
t1 t0 A (t1 , )B ( )u ( ) t1 t0
Dene the observability Gramian as the n n matrix
W0 (t0 , t1 ) =
t1 t0
( , t0 )CT ( )C( ) ( , t0 )x d
d
T
The observability Gramian is again symmetric, positivesemidenite, and satises the matrix differential equation
A (t1 , )B ( )B
( )
T A (t0 , )
1 WC (t0 , t1 ) x(t0 )
A (t0 , t1 )x(t1 ) 1 A (t1 , t0 )WC (t0 , t1 )WC (t0 , t1 )[x(t0 ) A (t1 , t0 )x(t0 )
d W (t , t1 ) = AT (t )W0 (t , t1 ) W0 (t , t1 )A(t ) C(t )T C(t ) (21) dt 0 W0 (t1 , t1 ) = 0

A (t0 , t1 )x(t1 )]
= x(t1 )
The system is observable on [t0, t1] if and only if W0(t0, t1) is invertible. If the system is observable, then, in the absence
55
of external inputs, the initial condition is given by

1 x(t0 ) = W0 (t0 , t1 ) t1 t0 T
( , t0 )CT ( )y( ) d
In the special case where the matrices A, C are independent of time, observability is determined from the matrix M o ( A, C ) = M c ( A T , C T ) T The state-space model is observable if and only if the matrix Mo(A, C) has rank n. State-Space Realization of Input-Output Models An important question in MIMO systems is determining when a causal input-output model of the form
t
additional extraneous states can be added which do not affect the input-output behavior of the system. It is important to identify realizations with minimal numbers of states. If a realization with state dimension n exists and no other realization exists with dimension less than n, then realizations with state dimension n are known as minimal realizations. The following result provides an answer to the realization problem: Theorem (3). There exists a state space realization of dimension n for the weighting pattern G(t, ) if and only if there exists a p n matrix matrix function H(t) and an n m matrix function F(t), both continuous for all t, such that G(t , ) = H (t )F ( ) for all t, . The proof of this theorem is straightforward. The invertibility properties of the state transitional matrix guarantee that, for a state-space model, its input-output relationship is factored as C(t ) (t , )B( ) = C(t ) (t , 0 ) (0, )B( ) so that H(t) C(t)(t, 0), F(t) (0, t)B(t). Conversely, given H(t), F(t), the state-space model x (t ) = F (t )u(t ) y(t ) = H (t )x(t ) (24)
y(t ) =
t0
G(t , )u( ) d + D(t )u(t )
(22)
[where G(t, ) does not contain generalized functions such as impulses] is represented by a nite-dimensional state-space model of the form
d x(t ) = A(t )x(t ) + B(t )u(t ) dt y(t ) = C(t )x(t ) + D(t )u(t )
(23)
where x(t) Rn, u(t) Rm, y(t) Rp and A(t), B(t), C(t), D(t) are continuous matrices. The converse of the question is straightforward. Statespace models correspond to causal input-output models of the form
t
y(t ) =
t0
C(t )
A (t , )B ( )u ( )
d + D(t )u(t ),
t t0
is a realization for H(t)F(), because the state transitional function (t, ) is the identity. Next, consider the problem of determining whether a statespace realization is a minimal realization. The answer is tied to the concepts of controllability and observability of statespace models discussed in greater depth in CONTROLLABILITY AND OBSERVABILITY. Theorem (3). Suppose the linear system Eq. (22) is a realization of the weighting pattern G(t,). Then, Eq. (22) is a minimal realization if and only if for some t0 and tf t0, Eq. (22) is controllable and observable on the interval [t0, tf]. Another important question is determining the existence of a time-invariant realization for a weighting pattern. The answer is provided in the following result. Theorem (3). A weighting pattern G(t,) is realizable by a time-invariant linear system Eq. (23) if and only if it is realizable, continuously differentiable in both t and , and G(t,) G(t , 0) for all t and . Under these conditions, there exists a time-invariant minimal realization of G(t,). Linear Time-Invariant MIMO Systems The analysis of linear MIMO systems in the time domain is greatly simplied in the time-invariant case by using results from linear algebra. Furthermore, the time invariance property allows applying transform techniques to represent the behavior of the system in the frequency domain, as illustrated in the next subsection. This subsection discusses the analysis of causal LTI MIMO systems described by state-space models
In the problem of realizability, it is straightforward to identify the correspondence between D(t) in Eqs. (22) and (23). Thus, the focus is on identifying matrices A(t), B(t), C(t) in Eq. (23) from G(t,) in Eq. (22). Two variations of this problem are of interest. In the rst variation, the function G(t,) is continuous and known for all values of t, . This information corresponds closely to that provided by a state-space model, because the term C(t)A(t,)B() is dened for all t and and is continuous by the assumptions for a state-space model. In the second variation, the function G(t,) is known only for values t , corresponding to causal observations of impulse responses of the system. In this variation, the realization problem is more complex and requires additional smoothness assumptions on G(t,). In this overview, we focus on the rst variation where G(t,) is known and continuous for all t, . The interested reader should consult (1,2,3) for further details on realization from impulse responses. Denition. A state-space model Eq. (23) is a realization of a weighting pattern G(t,) if, for all t, , G(t,) C(t)A(t,)B(). There are many possible realizations corresponding to a specic weighting pattern G(t,). Any change of basis in the state space results in an equivalent realization. In addition,
56
in the time domain. Relevant concepts from linear algebra are introduced as needed. Consider a MIMO LTI system described in state space form as
d x(t ) = Ax(t ) + Bu(t ) dt y(t ) = Cx(t ) + Du(t )

n m
(25)
chapter, assume that right and left eigenvectors are scaled so that the inner product of eigenvectors for the same eigenvalue is 1, that is, uiHvi 1 for all i 1, . . ., n. A useful property of eigenvectors is that the right and left eigenvectors corresponding to different eigenvalues are mutually orthogonal. Let ui and vi denote, respectively, the left and right eigenvectors corresponding to eigenvalue i. Then, uH i vj = 0 whenever i j. Now consider a matrix M Cnn with n distinct eigenvalues i, i 1, . . ., n, and, with corresponding left and right eigenvectors ui, vi, i 1, . . ., n. It can be shown that the n eigenvectors vi form a basis for the space Cn. In this case, the matrix M is represented in terms of these eigenvalues and eigenvectors in a dyadic expansion as
n
where the state x(t) R , the input u(t) R , the output y(t) Rp, and the matrices A, B, C, D do not depend on time. The algebraic structures of the matrices A, B, C, D completely determine the qualitative behavior of the system, as is evident after some concepts from linear algebra are reviewed. Eigenvalues and Eigenvectors. Let Cn denote the space of ndimensional, complex-valued vectors. Consider a vector v Cn of dimension n, expressed in terms of its real and imaginary parts as
a1 b1 v1 . . . . = + j v= . . . . . = a + jb vn an bn
where ai, bi, i 1, . . ., n are real-valued. Denote by v the Hermitian of the vector v, dened as the complex conjugate of the transpose of v:
H
M=
i= 1
i vi uH i
(26)
This expansion is useful for computing powers of M, because

n
Mk =
i= 1
H k i vi ui
resulting from the orthogonality property of left and right eigenvectors corresponding to different eigenvalues. System Modes. Consider the LTI state-space model in Eq. (25). In the absence of inputs u(t), the response of the system is determined by the initial condition x(0) and the system matrix A, as x(t ) = eAt x(0 )
vH = aT jbT Given two vectors u, v C , the inner product of u and v is dened as the complex-valued scalar product u, v uHv. The Euclidean norm of a vector v Cn is given by
n
v v=
H i= 1
(a2 i
b2 i)
The norm 2 is used to determine the size of vectors v Cn and corresponds to the standard notion of vector length. Denote by Cnn the space of (n n)-dimensional, complexvalued matrices. Let M Cnn be a square matrix. An eigenvalue of M is a complex number which is a root of the characteristic polynomial of M: det(I M ) = 0 Associated with each distinct eigenvalue are nonzero left and right eigenvectors u, v Cn, which satisfy the linear equations
Assume that the matrix A has distinct eigenvalues i, i 1, . . ., n, with corresponding left and right eigenvectors ui, vi, i 1, . . ., n. A treatment of the general case with repeated eigenvalues is found in (12). Using Eq. (26) in the expansion of eAt yields
eAt = (At )0 + At + A2
n
t2 + 2
n n
=
i= 1 n
(i t )0 vi uH i +
i= 1
(i t )vi uH i +
i= 1
(it )2 vi uH i + 2
=
i= 1
e i t v i u H i
Thus, the unforced system response is given as
Mv = v and uH M = uH
In the special case where the matrix M is real-valued (M Rnn), if is an eigenvalue of M with a nonzero imaginary part, then its complex conjugate is also an eigenvalue of M. Because the previous equations are linear any multiple of an eigenvector is also an eigenvector. Thus, eigenvectors can be scaled to have any nonzero magnitude. In the rest of this x(t ) =
n i= 1
e i t v i [u H i x ( 0 )]
This is interpreted as follows: The initial condition x(0) is decomposed into its contributions along n different system modes using the left eigenvectors. The ith system mode is dened as eitvi and has its own characteristic exponent i. When the initial condition x0 corresponds to a right eigenvector vi, the state response x(t) eitvi is focused along the same direction vi.
57
The system modes are also used to understand the output response of the system in the presence of input signals u(t). Substituting the dyadic expansion of eAt into the output response Eq. (13) yields
t
y(t ) = CeAt x(0 ) +

0 n
C(t )eA (t ) Bu( ) d + Du(t )
= +
e i t [(Cvi )(uH i x ( 0 )]
t
i= 1 n
(Cvi )(uH i B)
i= 1
e i (t ) u( ) d + Du(t )
MIMO Transfer Function Matrix. One of the powerful tools of classical single-input, single-output (SISO) control theory is frequency-domain analysis using transform methods for LTI systems. SISO systems are often characterized by their transfer functions relating input signals to output signals in the frequency domain with Laplace transforms. Transform techniques are also applied to LTI MIMO systems to obtain generalizations of system transfer functions to MIMO systems, as follows. Consider an LTI MIMO system, characterized by the impulse response matrix-valued function H(t), which describes the input-output behavior of the system as y(t ) =

The term uiHB indicates how the control action affects the ith mode. Similarly, the term Cvi shows how much the ith mode affects the system output y(t). Thus, modal analysis of LTI systems decomposes the performance of MIMO systems into a superposition of n independent modes which are exited by the input signals and initial condition. Based on Eq. (27), one can derive intuitive conditions for controllability and observability of LTI systems using the system modes. In particular, note that the ith mode is uncontrollable if uiHB 0, because the input has no effect on the ith mode trajectory. Thus, controllability requires that uiHB 0 for all modes i 1, . . ., n. Similarly, the ith mode does not affect the output if Cvi 0. In this case, an initial condition of vi yields an identical output to an initial condition of 0 and thus is unobservable. Observability requires that Cvi 0 for all modes i 1, . . ., n. Example. Consider the state-space model specied by the following matrices:
H (t )u( ) d
(27)
where H(t) includes generalized functions, such as the unit impulse (t). Let X(s) denote the bilateral Laplace transform of the function x(t): X (s ) =
+
x(t )e st dt
(28)
For the MIMO LTI system Eq. (27), application of Laplace transforms on both sides yields Y (s ) = H (s )U (s ) (29)
1 A= 0 0 1 B = 1 0 0.5 C= 0 D= 1 1
0 0 1 0 1 0 0
1 2 0
2 1 3
where Y(s) is the p-dimensional, two-sided Laplace transform of the output y(t), U(s) is the m-dimensional, two-sided Laplace transform of the input u(t), and H (s) is the p m twosided Laplace transform of the impulse response H(t), called the system transfer function matrix. In coordinates, this relationship is given by
m
Yi (s ) =
k=1
Hik (s )Uk (s ),
k = 1, . . ., p
0 1
where H ik(s) is the Laplace transform of Hik(t). For causal LTI systems described in state-space form, as in Eq. (25), the transfer function matrix is obtained directly from the state-space representation. Assume that the system is at rest with no initial conditions. Taking bilateral Laplace transforms of both sides in Eq. (25) yields
The eigenvalues of A are 1 1, 2 2, 3 3. A set of left and right eigenvectors is given by
sX (s ) = AX (s ) + BU (s ) and Y (s ) = CX (s ) + DU (s )
Solving these equations simultaneously, Y (s ) = [C(sI A ) 1 B + D]U (s ) which yields the transfer function matrix H (s) C(sI A)1B D. Note that, although there can be different statespace models for a given LTI system, the transfer function matrix H (s) is unique. There are some special properties of system transfer function matrices of MIMO LTI systems. First, the variable s en-
1 1 0 u1 = 1 , v1 = 0 ; u2 = 1 , 1.5 0 1 1 0 0.5 v1 = 1 ; u3 = 0 , v1 = 1 0 1 1
Using these modes, it is straightforward to verify that Cvi 0 and uiHB 0 for i 1, . . ., 3, which establishes that the system is controllable and observable.
58
ters into the expression in the inverse (sI A)1. If A is an (n n) matrix, this means that the entries of H (s) are rational functions, ratios of polynomials, with denominator degree no greater than n. Furthermore, the numerator degree is no greater than n either and is strictly less than n for all entries unless D 0. Transfer function matrices with entries as rational functions with numerator degree less than or equal to denominator degree are known as proper. If the numerator degree is strictly less than the denominator degree for each entry, the transfer function matrix is known as strictly proper. Multivariable Poles and Zeros. For SISO LTI systems, the poles and zeros of the system are determined from the transfer function, consisting of a ratio of a numerator polynomial and a denominator polynomial. The roots of the numerator polynomial determine the zero frequencies of the system, frequencies which, if present at the input, are blocked by the system and are thus not present at the output. Similarly, the roots of the denominator polynomial determine the poles that are frequencies appearing at the output in response to initial conditions with no external input. Although a rich theory exists for generalizing the SISO transfer function decomposition to transfer function matrices of MIMO systems using polynomial matrices and matrix fraction descriptions (see e.g. [12]), the simplest denition of MIMO poles and zeros is given in terms of state-space models. Consider the LTI state-space model of Eq. (25). The poles of the system are the complex frequencies that appear in the output in response to initial conditions. Based on the discussion of the previous subsections, these frequencies are the eigenvalues of the matrix A. This is also seen directly from the transfer function matrix H (s) C(sI A)1B D. Using the expression for inverting a matrix, it is clear that the denominator of all of the entries in the transfer function matrix is given by det(sI A). Thus, the poles correspond to roots of the equation det(sI A) 0, which are the eigenvalues of A. In contrast with multivariable poles, there are several ways in which zeros have been dened for LTI MIMO systems. First consider a system with equal number of inputs and outputs (m p), and assume that the state-space model in Eq. (25) is minimal (thus controllable and observable). Multivariable transmission zeros are dened as complex frequencies where, given a particular nonzero combination of input directions at that frequency and initial conditions, there is no output generated by the system. The formal denition is given here: Denition. The system Eq. (25) has a transmission zero at the complex frequency sk if there exist complex vectors uk Cm, xk Cn, one of which is nonzero, such that the system Eq. (25) with initial condition x(0) xk, and input u(t) ukeskt, t 0 has the property that y(t) 0 for all t 0. The initial condition xk must be chosen carefully to ensure that the state trajectory does not contain modes other than those of the input eskt, because those modes are observable (the minimality assumption) and lead to nonzero outputs. Thus, x(t) xkeskt is a solution for the trajectory of the system. Substituting this solution in the system Eq. (25) with input
u(t) ukeskt gives the following equations after dividing by the term eskt: sk xk = Axk + Buk 0 = Cxk + Dxk Rearranging these equations as a set of linear equations in the unknowns xk, uk yields
sk I A C
B D
xk uk
0 0
This is in the form of a generalized eigenvalue problem. Indeed, under the assumption of a minimal realization, the MIMO transmission zeros are obtained as the roots of the following equation:
det s
I 0
0 A 0 C
B D
=0
For a given transmission zero sk, the generalized eigenvector associated with that transmission zero provides the initial condition and input directions xk, uk which yield zero output at that input frequency. Example. Consider the MIMO LTI system described by state-space matrices 1 0 0 A = 0 2 0 0 0 3 1 0 B = 1 0 0 1
C= D=
0.5 0 1 1 0 0
0 1
0 1
The transfer function matrix for this two-input, two-output system is given by
H (s ) =
s+ 1.5 s+1 s+3 s+2
0
1 s+3
Because A is diagonal, the poles of the system are easily determined as 1, 2, 3. Solving the generalized eigenvalue problem, one obtains two transmission zeros, at frequencies 1.5 and 2. In particular, the input
u(t ) =
with initial condition
1 e 1 . 5 t 4.5
2 x(0 ) = 2 3
59
yields output y(t) 0. Similarly, the input
u(t ) =
with initial condition
0 2t e 1
0 x(0 ) = 1 1
also yields y(t) 0 for t 0. Note the presence of both a zero and a pole at 2, without a pole-zero cancellation. Note also that the zero at 2 is not a zero of any individual transfer function entry in the transfer function matrix. Now consider the general case where the number of inputs m is different from the number of outputs p and the state space model Eq. (25) is still controllable and observable. If the number of inputs is less than the number of outputs, the appropriate generalization is the concept of a right transmission zero, as dened here: Denition. The system Eq. (25) has a right transmission zero at the complex frequency sK if there exist complex vectors uk Cm, xk Cn, both of which are not identically zero, such that sk xk = Axk + Buk 0 = Cxk + Dxk In essence, a right transmission zero is a complex frequency where, for an appropriate input direction and initial condition, the output is identically zero. When the number of inputs m is greater than the number of outputs p, there are additional complications, because there can exist nonzero proper (m 1) transfer functions U(s) such that H (s)U(s) 0 for all s! Instead of declaring every complex frequency a transmission zero, one denes the concept of a left transmission zero, as follows: Denition. The system Eq. (25) has a left transmission zero at the complex frequency sk if there exist complex vectors k Cm, k Cn, both of which are not identically zero, such that
T T T sk k = k A + k C T B + BT 0 = k kD
uncontrollable. Then, any uncontrollable mode of the system k with left eigenvector uk is a left transmission zero with direction k 0, k uk. Any unobservable mode of the system with right eigenvector vk is a right transmission zero with directions xk vk, uk 0. Thus, the presence of unobservable and uncontrollable modes gives rise to transmission zeros in the same directions as the modes of the system, leading to pole-zero cancellations. Singular Values and MIMO Frequency Response Consider an LTI MIMO system, specied by its transfer function matrix H (s). Assume that the system is bounded-input, bounded-output stable, with no initial conditions. The transfer function H (s) can be interpreted as the complex gain of the linear system in response to bounded inputs of the form est. That is, if the input is dened as u(t) ves0t for t 0 for some complex number s0 with nonpositive real part and some direction vector v Rm, the output y(t) is given by y(t ) = H (s0 )ves 0 t The frequency response of the system is the set of transfer functions H ( j) for all frequencies R. Thus, the frequency response of the system denes the outputs corresponding to sinusoidal inputs of the form e jt. In single-input, single-output (SISO) systems, the transfer function is a scalar. Thus, the frequency response is characterized by the complex-valued function H ( j), which is represented by a magnitude and phase. In contrast, the frequency response of MIMO systems is a complex, matrix-valued function of the frequency, which has a range of gains, depending on the direction a of the sinusoidal input. To understand how to represent this effect, it is useful to review some concepts of gains for complex-valued matrices. Complex Matrices and Gains. At a specic frequency, the transfer function matrix H ( j) is a complex-valued matrix of dimension p m. Denote by Cpm the space of complex-valued matrices of dimension p m. Any matrix M Cpm, is decomposed into its real and imaginary parts, as M = A + jB where A, B Rpm. In a manner similar to a vector, the Hermitian of a matrix is dened as the complex conjugate of its transpose, that is, M H = AT jBT Given a matrix M Cpm, the spectral norm of the matrix, denoted as M2, is the maximum amplication of any input unit vector, dened as M
2
that is, a left transmission zero is a right transmission zero of the state-space model
d x(t ) = AT x(t ) + CT u(t ) dt y(t ) = BT x(t ) + DT u(t )
(30)
= max Mv
v 2 2
For square systems, any frequency that is a left transmission zero is also a right transmission zero. As a nal note on this topic, consider a state-space realization that is not minimal, so that it is either unobservable or
A complex-valued square matrix is called Hermitian if MH M. A nonsingular, complex-valued matrix is called unitary if M1 MH, which implies MMH MHM I. Hermitian matrices have the property that all of their eigenvalues are
60
real-valued. This can be readily derived by noting that, for an eigenvalue with right eigenvector u, (uH Mu )H = (uH u )H = H uH u = uH M H u = uH Mu = uH u which establishes that is equal to its complex conjugate and thus is a real number. Hermitian matrices also have the property that repeated eigenvalues have a full complement of eigenvectors, and thus Hermitian matrices are represented as diagonal matrices with an appropriate change of basis. The eigenvalues of unitary matrices have unit magnitude, as is readily seen from
a nearly diagonal transformation. Dene the matrix of singular values as
1 0 . = . . 0
0 2 . .. . . . 0 0 ( p k ) k
0 0 . . . k
0k (m k )
0 ( p k ) (m k )
Note that k min(m, p); if k m or k p, some of the zero blocks in the above matrix are removed. The singular value decomposition states the following: Theorem. Given a matrix M Cpm, there exist a p p unitary matrix U and m m unitary matrix V such that M = U VH = U H MV The column vectors of U are called the left singular vectors of M, and the column vectors of V are called the right singular vectors of M. Because U and V are unitary, they correspond to an orthogonal change of basis in Cp and Cm, respectively. It is easy to show that the left singular vectors of M are the normalized right eigenvectors of the p p matrix MMH and that the right singular vectors of M are the normalized right eigenvectors of the m m matrix MHM. Reliable and efcient numerical techniques for computing singular-value decompositions are available in commercial software packages. The singular-value decomposition allows us to estimate the gain of the matrix M when acting on an input u of unit Euclidean norm, as follows. Let y Mu denote the output of M with input u. Using the singular-value decomposition yields y = U V Hu = U v where v is also a unit norm vector because V is unitary. Then, y
2
(uH M H )(Mu ) = (H uH )(u ) = ||2 uH u = u H ( M 1 M )u = u H u

Thus, unitary matrices acting on vectors preserve the Euclidean norm. Let M Cpp be unitary and u Cp be an arbitrary vector. Then Mu
2
(Mu )H Mu =
u H ( M H M )u =
uH u
Now consider an arbitrary matrix M Cpm. The square matrices MHM and MMH are Hermitian and thus have realvalued eigenvalues. They also have the additional property that the eigenvalues are nonnegative, and the nonzero eigenvalues of MMH are equal to the nonzero eigenvalues of MHM. Let denote an eigenvalue of MMH with eigenvector v. Then, vH (MM H )v = v
2 2
= (M H v )H M H v = M H v
2 2
which shows that 0. If is a nonzero eigenvalue of MMH with eigenvector u, then M H (MM H )u = M H u = (M H u ) = (M H M )M H u which establishes that is also an eigenvalue of MHM with eigenvector MHu. Note that MHu must be nonzero if is nonzero. For a general matrix M Cpm with rank k, the singular values of M are the k square roots of the nonzero eigenvalues of MHM or MMH. Let i(M), i 1, . . ., k denote the k singular values and i(MHM) denote the corresponding k nonzero eigenvalues of MHM. Then, i (M ) = i (M H M ) = i (MM H ), i = 1, . . ., k
= UHy
= U HU v
This establishes that M

2
1 (M )
If u is the rst column of V, the unitary property of V gives
Because the k nonzero eigenvalues of MHM are real and positive, the singular values are also real and positive. Assume that the singular values are ordered in descending order, that is, 1 (M ) 2 (M ) . . . k (M ) > 0 The singular-value decomposition of a matrix M Cpm states that there are convenient changes of bases in Cp and Cm, so that the linear transformation M can be visualized as
1 0 VHu = . . . 0
which shows that M
2
1 (M )
which establishes that the spectral norm of M is equal to the maximum singular value. It is also possible to establish a lower bound on the gain of the matrix M under the condition that the number of outputs
61
p is greater than or equal to the number of inputs m. Let m min(p, m). When the rank of M is less than m, dene the singular values k1 . . . m 0. Then, a similar argument as previous establishes that, for unit norm vectors m, Mu
2
in radians/second on a semilog scale. Figure 2 illustrates the MIMO frequency response plot for the transfer function matrix
H (s ) =
m 0
s+ 1.5 s+1 s+3 s+2
0
1 s+3
If the rank of M is m, the lower bound is strictly positive. When the number of outputs is less than the number of inputs (p m), M must have a nontrivial null space, and thus the lower bound is always 0. Singular-Value Representation of MIMO Frequency Response. Now consider the MIMO frequency response of a bounded-input, bounded-output stable system with transfer function matrix H (s). Assume that the number of inputs m is less than or equal to the number of outputs p. When the input vector is a complex exponential of the form u(t) aejt, the output vector is given by y(t ) = H ( j )u(t ) = H ( j )ae j t = be j t for some complex vector b H ( j)a. A useful characterization of the MIMO frequency response is provided in terms of bounds on the gain of complex matrix H ( j) as a function of frequency. For each frequency , the singular-value decomposition of H ( j) is obtained as H ( j ) = U ( j ) ()V H ( j ) with nonnegative singular values 1(), . . ., m(). Assume that a has unit Euclidean norm. Then, for each frequency, the maximum and minimum singular values 1(), m() are available, and 1 H ( j )a
2
The information contained in such a Bode plot provides direction-independent information concerning the magnitude of the frequency response at specic frequencies. In particular, at frequencies where the minimum singular value is large, all of the singular values of the system are large, and thus the system has a large gain in all directions. In regions where the maximum singular value is small, all of the singular values are small, and the system has a small gain in all directions. At other frequencies, it is necessary to use the direction of the input a to determine the magnitude of the frequency response. The maximum and minimum singular values provide bounds on the range of gains which are possible. The unitary matrix V( j) is a change of basis transformation on the input space. Thus, VH( j)a is an m-dimensional complex vector which is a decomposition of a into components along the right singular vectors which form a new basis in the input space. The magnitude and phase of the frequency response for each singular direction are readily evaluated from the matrices () and U( j).
BIBLIOGRAPHY
1. L. M. Silverman, Representation of linear dynamical systems, IEEE Trans. Autom. Control, 16: 554567, 1971. 2. E. W. Kamen, New results in realization theory for linear timevarying analytic systems, IEEE Trans. Autom. Control, 24: 866 877, 1979. 3. W. J. Rugh, Linear System Theory, Upper Saddle River, NJ: Prentice-Hall, 1996. 4. G. Strang, Linear Algebra and Its Applications, San Diego: Harcourt Brace Jovanovich, 1988. 5. J. M. Maciejowski, Multivariable Feedback Design, Wokingham, UK: Addison-Wesley, 1989. 6. A. G. J. MacFarlane and N. Karcanias, Poles and zeros of linear multivariable systems: A survey of the algebraic, geometric and complex variable theory, Int. J. Control, 24: 3374, 1976. 7. H. H. Rosenbrock, State-Space and Multivariable Theory, New York: Wiley, 1973. 8. M. K. Sain and C. B. Schrader, The role of zeros in the performance of multi-input, multi-output feedback systems, IEEE Trans. Educ., 33: 244257, 1990. 9. C. B. Schrader and M. K. Sain, Research in systems zeros: A survey, Int. J. Control, 50 (4): 14071433, 1989. 10. E. W. Kamen, Poles and zeros of linear time-varying systems, Linear Algebra and its Applications, 98: 263289, 1988. 11. M. G. Safonov, A. J. Laub, and G. L. Hartmann, Feedback properties of multivariable systems: The role and use of the return difference matrix, IEEE Trans. Autom. Control, AC-26: 4765, 1981. 12. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1980.
ON DAVID CASTAN
A simple way of visualizing the gain of the transfer function matrix is to plot the maximum and minimum singular values, expressed in decibels, in a Bode plot against frequency
10 0 10
Gain (dB)
20 30 40 50 60 70 101 100 101 102 Frequency (rad/s) 103
Figure 2. Singular-value MIMO frequency response.
Boston University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Nonlinear Control Systems, Analytical Methods Standard Article Derek Atherton1,2 and Sarah Spurgeon3,4 1School of Engineering 2University of Sussex, Brighton, England 3Department of Engineering 4University of Leicester, Brighton, England Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1024 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (1141K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Phase Plane Method Absolute Stability Criteria Describing Function Method Limit Cycles in Relay Systems Sliding Mode Control Methods Other Describing Functions Relay Autotuning Multivariable Systems Sliding Mode Control
file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20E...0ENGINEERING/04.%20automatic%20control/W1024.htm (1 of 2)18.06.2008 22:20:43
NONLINEAR CONTROL SYSTEMS, ANALYTICAL METHODS
A large number of methods exist for the analysis and design of linear control systems. Unlike a linear system, a nonlinear system does not satisfy the superposition property, which means not only that it may be difcult to deduce how the system will respond to a specic input if its response is known to a different input, but also nonlinear systems exhibit unique behavior due to the effects of the nonlinearity. In many systems it is possible to separate the static nonlinear effects from the dynamic so that a nonlinear system can often be accurately modeled as a combination of static nonlinear elements and linear dynamic elements. Here the concentration is primarily on analytical methods for nonlinear systems which are associated with those aspects of linear theory usually referred to as classical control. This means basically that for systems other than second order, frequency domain, rather than state space, models and formulations are used. An exception to this is the material on variable structure systems. The state space theme, and some design methods presented within that framework are given in the following article. A block diagram of a simple nonlinear feedback system which will receive signicant attention in this article is shown in Fig. 1. Although it only contains one nonlinear element, its presence can change the whole behavioral possibilities of the feedback loop compared with the linear situation, and its form is adequate for discussing many of the analysis and design techniques presented in this article. Since all practical systems contain some form of nonlinearity, it is important that basic concepts relating to the effects of nonlinearity are well understood. When this is the case it will allow the designer to assess qualitatively, if not quantitatively, the possible effects of nonlinear operation at various points within a feedback system and to take them into account in the design. This may allow the analysis and design to be done using one or more linearized models. A full nonlinear simulation may then be used to check that the design works satisfactorily when the nonlinear effects are included. This approach works satisfactorily in many instances, particularly if gain scheduling is used to counteract the effects of changes produced by any nonlinearity; however, this approach cannot be used for all situations. Many nonlinear effects which take place in control systems may be modeled approximately using static nonlinearities. These include saturation in ampliers, dead zones in valves, friction, and backlash in gears. Depending on the approach to be used in the analysis or design when these nonlinearities exist, it may be necessary to approx-
Figure 1. A simple nonlinear feedback system.
imate their characteristics by either simple continuous mathematical functions, such as polynomials, or linear segmented approximations. To apply some methods which we will discuss, it may even be necessary to use coarser approximations to a nonlinearity simply to say that it is conned within a sector. These types of nonlinearities are often referred to as inherent nonlinearities, since for a satisfactory design they will exist due to the devices used, although for analysis we may wish to neglect them. It is also true that good designs will always be nonlinear, since trying to ensure linear operation of a system will involve the selection of oversized components such as pumps, motors, and heaters. Nonlinearity may be introduced intentionally into control systems to compensate for existing nonlinear effects, or to implement a design strategy which is either preferable technically or more economical. The simple feedback system of Fig. 1, provided that the nonlinearity or transfer functions are suitably chosen, may exhibit a variety of behaviors which are unique to nonlinear systems. First, the performance of the system, even for a specic type of input, will depend upon the amplitude of the input. The response, for example, to a small step input may be quite different from that of a large step input. If the autonomous systemthat is, the system with no input is released from several initial states, then the resulting behavior may be appreciably different for each state. For example, instead of reaching a stationary equilibrium, the system may move from some initial conditions into a limit cycle, a continuous oscillation which can be reached from a subset of initial conditions. This behavior is distinct from an oscillation in an idealized linear system since the magnitude of this latter oscillation is dependent upon the initial energy input to the system. A limit cycle is a periodic motion, but its waveform may be signicantly different from the sinusoid of an oscillation. The autonomous nonlinear system may also have a chaotic motion, a motion which is repeatable from given initial conditions but which exhibits no easily describable mathematical form, is not periodic, and exhibits a spectrum of frequency components. If a sinusoidal input is applied to the system, then the output may be of the same frequency but will also contain harmonics or other components related to the input frequency. This output too, for certain frequencies and amplitudes of the input, may not be unique but has an amplitude dependent upon the past history of the input or the initial conditions of the system. The sinusoidal input may also cause the system to oscillate at a related frequency so that the largest frequency component in the output is not the same as that of the input. Also if, for example, the autonomous system has a limit cycle, then the addition of a sinusoidal input will cause the limit cycle frequency to change and possibly cause synchronization of the limit cycle frequency with the input frequency or one of its harmonics. In many instances the phenomena just mentioned are undesirable in a control system, so that one needs techniques to ensure that they do not occur. Control systems must be designed to meet specic performance objectives, and to do this one is required to design a control law which is implemented based on measurements or estimation of the system states or, by simple functions of the system variables, typically the error signal. Many systems can be
Nonlinear Control Systems, Analytical Methods
made to operate satisfactorily with the addition of a simple controller in the error channel, which is shown by the transfer function Gc (s) in Fig. 1. Typical performance criteria, which the system may be required to meet, are that it is stable, has zero steady-state error and a good response to a step input, suitably rejects disturbances, and is robust to parameter variations. Although one reason for using feedback control is to reduce sensitivity to parameter changes, specic design techniques can be used to ensure that the system is more robust to any parameter changes. If the process to be controlled is strongly nonlinear, then a nonlinear controller will have to be used if it is required to have essentially the same step response performance for different input step amplitudes. Some control systemsfor example, simple temperature control systemsmay work in a limit cycle mode, so that in these instances the designer is required to ensure that the frequency and amplitude variations of the controlled temperature are within the required specications. In the next section, we examine in some detail various approaches which can be used for investigating the analysis and design of nonlinear systems. The rst topic discussed is the phase plane method, which can normally only be used to investigate second-order systems. It is a useful technique, since it can be used when more than one nonlinearity exists in the system and since many control problems, such as position control systems, can be modeled approximately by second-order dynamics. As mentioned previously, one specication for the design may be that the system must be stable. For linear systems, assessment of stability is a simple problem, but this is not the case for a nonlinear system, even when it is as simple as that shown in Fig. 1. Several absolute stability criteria exist for checking whether the system of Fig. 1 will be stable, and these are discussed in more detail later. The criteria presented are easy to use; and the circle criterion in particular, being basically an extension of the Nyquist criterion, is easy to implement and follow. A disadvantage, however, is that all these criteria only produce sufcient conditions so that if the condition is violated the system may still be stable. To try to obtain an estimate of the possibility of this being the situation, the describing function method has been used by engineers for many years. The difculty with the describing function approach, which approximates the nonlinearity by its gain to a sinusoidal input, is that the results are not exact. It does, however, enable the designer to obtain more insight into the situation, and, of course, the ideas can often be further checked by simulation. The describing function approach can also be helpful for system design in terms of shaping the frequency response of the system to produce a more stable situation or for indicating possible nonlinear effects which can be added in the controller to counteract the nonlinear effects in the plant. Describing functions for other than a single sinusoid can be obtained, and these allow some of the more complex aspects of the behavior of nonlinear systems to be investigated. These include, for example, synchronization and subharmonic generation as well as estimating more accurately the frequency content of any limit cycle. Relay-type characteristics are often introduced in control system to provide economic designs or to produce variable structure
systems. First, a method for the determination of limit cycles in relay systems is presented. This is an interesting approach, since it allows the exact evaluation of a limit cycle and also an exact determination of whether it is stable or not. The method in this sense is unique, since exact limit cycle data for systems with any order dynamics containing a relay can be obtained. As with the design of linear control systems, the issue of robustness to unmodeled dynamics and parameter uncertainty is also pertinent in the nonlinear control area. One such robust technique is the so-called variable structure or sliding mode approach. Variable structure control systems (VSCS) are characterized by a set of feedback control laws and an associated decision rule or switching function. This decision rule has as its input some measure of the current system behavior and produces as an output the particular feedback control law which should be used at that instant in time. The resulting variable structure system (VSS) may be regarded as a combination of subsystems, where each subsystem has a xed control law which is valid for specied regions of system behavior. One of the advantages of introducing this additional complexity into the system is the ability to combine useful properties of each of the subsystems into a very exible closed-loop control strategy. Indeed, it will be seen that a VSS may be designed to possess new properties which are not present in any of the composite structures. Utilization of these natural ideas began in the late 1950s in the Soviet Union and formed the foundations for signicant contributions to the area of robust nonlinear control. Of particular interest in the area of VSS is the so-called sliding mode behavior, where the control is designed to drive and then constrain the system state and lie within a neighborhood of the switching function. There are two signicant advantages with this approach to controller design. First, the dynamic behavior of the system may be tailored by the particular choice of switching function. Second, the closed-loop response becomes totally insensitive to changes in certain plant parameters and will completely reject a particular class of external disturbances. This invariance property clearly renders sliding mode control a strong candidate for robust control. In addition, the ability to specify performance directly makes sliding mode control attractive from the design perspective. This is seen from the wide exposure of sliding mode control to many applications areas including robotics, aerospace, and automotive industries. The sliding mode design approach involves two stages. The rst consists of the design of an appropriate switching function to ensure that the system behavior during sliding motion satises the required design specications. This is termed the existence problem. In the simplest case, this will be seen to amount to the design of a linear full-state feedback controller for a particular subsystem. The second design stage is concerned with the selection of a control law which will make the switching function attractive to the system state. This is termed the reachability problem. It is important to note that this control law is not necessarily discontinuous in nature.
THE PHASE PLANE METHOD The phase plane method was the rst approach used by control engineers for studying the effects of nonlinearity in feedback control systems. The technique can generally only be used for systems represented by second-order differential equations. It had previously been used in nonlinear mechanics and for the study of nonlinear oscillations. Smooth mathematical functions were assumed for the nonlinearities so that the second-order equation could be represented by two nonlinear rst-order equations of the form
Equilibrium, or singular points, occur when
Figure 2. Limit cycle solution of Van der Pol equation with = 1.0.
and the slope of any solution curve, or trajectory, in the x1 x2 state plane is
where is a positive constant. The phase plane form of this equation can be written as
The slope of a trajectory in the phase plane is A second-order nonlinear differential equation representing a control system with smooth nonlinearity can typically be written as
and if this is rearranged as two rst-order equations, choosing the phase variables as the state variablesthat is, x1 = x, x2 = xit can be written as
which is only singular (that is, at an equilibrium point), when the right-hand side of Eq. (4) is 0/0, that is, x1 = x2 = 0. The form of the singular point, which is obtained from linearization of the equation at the origin, depends upon , being an unstable focus for < 2 and an unstable node for > 2. All phase plane trajectories have a slope of r when they intersect the curve
which is a special case of Eq. (3). A variety of procedures have been proposed for sketching state (phase) plane trajectories for Eqs. 3 and 5. A complete plot showing trajectory motions throughout the entire state (phase) plane from different initial conditions is known as a state (phase) portrait. Knowledge of these original methods, despite the immense improvements in computation since they were rst proposed, can be particularly helpful for obtaining an appreciation of the system behavior. When simulation studies are undertaken, phase plane graphs are easily obtained and they are often more helpful for understanding the system behavior than displays of the variables x1 and x2 against time. Many investigations using the phase plane technique were concerned with the possibility of limit cycles in the nonlinear differential equations. When a limit cycle exists, this results in a closed trajectory in the phase plane; typical of such investigations was the work of Van der Pol. He considered the equation
One way of sketching phase plane behavior is to draw a set of curves for various selected values of r in Eq. (5) and marking the trajectory slope r on the curves, a procedure known as the method of isoclines. Figure 2 shows a simulation result from a small initial condition leading to the stable limit cycle solution for = 1.0. Many nonlinear effects in control systems, such as saturation and friction, are best approximated by linear segmented characteristics rather than continuous mathematical functions. This is an advantage for study using the phase plane approach, since it results in a phase plane divided up into different regions but with a linear differential equation describing the motion in each region. To illustrate the approach, consider a basic relay position-control system with nonlinear velocity feedback having the block diagram shown in Fig. 3. First, let us assume that the hysteresis in the relay is negligible (i.e., = 0) and that h is large so that the velocity-feedback signal will not saturate. Denoting the system position output by x1 and its derivative x1 by x2 , we note that the relay output of 1 or 0 is equal to x1/K and that the relay input is equal to x1 x2 = 1. Taking the dead zone of the relay
Figure 3. Block diagram of relay positioncontrol system.
to be equal to 1, the motion of the system is described by
Thus in the phase plane, which has x1 as abscissa and x2 as ordinate, the dashed lines x1 + x2 = 1 in Fig. 4 divide the plane into the three regions, for each of which the motion is described by one of the above three simple linear secondorder differential equations. The solution of
in terms of the phase-plane coordinates x1 and x2 is
Figure 4. Initial condition response.
where x10 and x20 are the initial values of x1 and x2 . Since Eq. (6) describes a parabola, which for the special case of K = 0 has the solution x2 = x20 , it is easy to calculate the systems response from any initial condition (x10 , x20 ) in the phase plane. Figure 4 shows the response from (4.6, 0) with = 1 and K = 1.25. The initial parabola meets the rst switching boundary at A; the ensuing motion is horizontalthat is, at constant velocityuntil the second switching boundary is reached at B. The resulting parabola meets the same switching boundary again at C, at which point motion from either side of the switching line through C will be directed toward C, so that the resulting motion is a sliding motion. Responses from any other initial conditions are obviously easy to nd, but, from the one response shown, several aspects of the systems behavior are readily apparent. In particular, the system is seen to be stable since all responses will move inward, possibly with several overshoots and undershoots, and will nally slide down a switching boundary to 1. Thus a steady-state error of unit magnitude will result from any motion. When the velocity-feedback signal saturatesthat is, when |x2 | > hthe input signal to the relay is x1 h. The switching boundaries change to those shown in Fig. 5, but the equations describing the motion between the boundaries remain unaltered. Therefore for a large step input the response will become more oscillatory when the velocity saturates. When the hysteresis is nite then the switch-
Figure 5. Changed switching boundaries due to saturation.
ing lines for positive (negative) x2 move to the right(left) at their intersection with the x1 axis. If h is large it is then easily shown that a limit cycle, as shown in Fig. 6 for = 1 and = 0.5, will occur. Trajectories both inside and outside the limit cycle have their motion directed toward it. Similarly, it is straightforward to draw phase-plane trajectories for a nite hysteresis and smaller values of h.
Figure 6. Response terminating in a limit cycle for = 1 and = 0.5.
Figure 7. Graphical illustration of the Popov criterion.
ABSOLUTE STABILITY CRITERIA A very important question in control is to be able to ascertain the stability of a feedback system. The problem for linear systems was examined over a century ago in Cambridge, England, by Routh, who published his famous work on the stability of motion in 1877. As a result of this work and further contributions, most notably by Nyquist, several approaches are now available for determining the stability of a feedback loop such as Fig. 1 when the nonlinearity n(x) is replaced by a linear gain K. The methods provide necessary and sufcient conditions for stability. The rst work on the stability of nonlinear systems by Lyapunov was published in 1892, and since that time there have been many attempts to determine necessary and sufcient conditions for the stability of the autonomous feedback systemthat is, r = 0of Fig. 1. Lyapunov formulated an approach for determining sufcient conditions, but the difculty of his method is that it requires determination of a function of the system states which then must be shown to satisfy certain properties. There is no general approach for nding a suitable function; when one is found, it does not guarantee that a better function does not exist which will prove stability in a larger domain in the state space. The problem has therefore been researched by many people with the objective of obtaining conditions for stability which are relatively easy to use. Several frequency-domain results (1) giving sufcient, but not necessary, conditions for stability have been determined which use limited information about the nonlinearity, n(x), typically its sector bounds or the sector bounds of its slope. The nonlinearity n(x) has sector bounds (k1 , k2 ); that is, it is conned between the straight lines k1 x and k2 x if k1 x2 < xn(x) < k2 x2 for all x. Similarly, it has slope bounds (k 1 , k 2 ) if k 1 x2 < xn (x) < k 2 x2 , where n (x) = dn(x)/dx. The Popov criterion (2) states that a sufcient condition for the autonomous system of Fig. 1 to be stable if G(s) is stable and G() > k1 is that a real number q > 0 can be found such that for all we obtain
Figure 8. Illustration of the circle criterion.
Fig. 7, where for the system to be stable a line of slope q1 can be drawn through the point k1 so that the Popov locus G*(j) lies to the right of the line. The Popov locus is given by
where the nonlinearity n(x) lies in the sector (0, k). The theorem has the simple graphical interpretation shown in
The circle criterion (3) is more conservative than the Popov criterion but can be used when both the nonlinearity is time varying and there is a bounded input, r, to the system of Fig. 1. Satisfaction of the circle criterion guarantees that the autonomous system is absolutely stable and the system with bounded input has a bounded output. The criterion uses the Nyquist locus, G(j), and for stability of the system of Fig. 1 with n(x) in the sector (k1 , k2 ) it is required that G(j) for all real has the following properties. If the circle C has its diameter from 1/k1 to 1/k2 on the negative real axis of the Nyquist diagram, then (1) if k1 k2 < 0, G(j) should be entirely within C, (2) if k1 k2 > 0, G(j) should lie entirely outside and not encircle C, and (3) if k1 = 0 or k2 = 0, G(j) lies entirely to the right of 1/k2 or to the left of 1/k1 . The situation for stability in case (2) is shown in Fig. 8. Two simple transformations are also useful for investigating the absolute stability of the autonomous system of Fig. 1. Feeding forward around the nonlinearity and backward around the dynamics G(s), through a constant gain , whose effects cancel out, changes the nonlinearity sector to (k1 , k2 ) and the linear transfer function
to G(s)/[1 + G(s)]. Alternatively feeding backward around n(x) and forward around G(s) changes the nonlinearity sector to (k1 /(1 k1 ), (k2 /(1 k2 )) and changes the linear transfer function to + G(s). This is needed in order to apply the Popov criterion to the general nite sectorthat is, n(x) in the sector (k1 , k2 ). Prior to the derivation of these frequency-domain results, Aizermann had put forward a conjecture that the autonomous system of Fig. 1 would be stable for a nonlinearity sector bounded by (k1 , k2 ) if for k1 k2 > 0 the Nyquist locus G(j) of a stable transfer function did not touch or encircle the line between 1/k1 and 1/k2 , which is of course the diameter of the circle of Fig. 8. Several counterexamples have been put forward to show that the conjecture is incorrect; however, it can be shown that if the conjecture is satised, the system may possess a limit cycle but its output cannot go unbounded (4). For a monotonic nonlinearity with slope bounds (k 1 , k 2 ) and k 1 k 2 > 0, an off-axis circle criterion exists (5). This states that the autonomous system of Fig. 1 with a nonlinearity satisfying the aforementioned conditions will be absolutely stable if the Nyquist locus of a stable transfer function does not encircle a circle centered off the real axis and which intercepts it at (1/k 2 , 1/k 2 ).
for clarity, called the sinusoidal DF (SDF). The Sinusoidal Describing Function We assume that if in Fig. 1 we have x(t) = a cos , where = t and n(x) is a symmetrical odd nonlinearity, then the output y(t) will be given by the Fourier series.
where
and
The fundamental output from the nonlinearity is a1 cos + b1 sin , so that the DF is given by
DESCRIBING FUNCTION METHOD The describing function (DF) method was developed simultaneously in several countries during the 1940s. Engineers found that control systems which were being used in many applicationsfor example, gun pointing and antenna controlcould exhibit limit cycles under certain conditions rather than move to a static equilibrium. They realized that this instability was due to nonlinearities, such as backlash in the gears of the control system, and they wished to obtain a design method which could ensure that the resulting systems were free from limit cycle operation. They observed that when limit cycles occurred the observed waveforms at the system output were often approximately sinusoidal, and this indicated to them a possible analytical approach. Initial investigations therefore focused on the autonomous feedback system with a single nonlinear element shown in Fig. 1 containing a static nonlinearity n(x) and linear dynamics given by the transfer function G(s) = Gc (s)G1 (s). It was recognized that if a limit cycle existed in the autonomous system with the output c(t) approximately sinusoidal, then the input x(t) to the nonlinearity could be assumed sinusoidal, the corresponding fundamental output of the nonlinearity could be calculated, and conditions for this sinusoidal self-oscillation could be found, if the higher harmonics generated at the nonlinearity output were neglected. This is the concept of harmonic balance, in this case balancing the rst harmonic only, which had previously been used by physicists to investigate such aspects as the generation of oscillations in electronic circuits. The DF of a nonlinearity was therefore dened as its gain to a sinusoidthat is, the ratio of the fundamental of the output to the amplitude of the sinusoidal input. Since describing functions can be used for other than a single sinusoidal input to a nonlinearity, as discussed in the latter part of this article; this DF is often,
which may be written
where
Alternatively, in polar coordinates,
where
It is further easily shown that if n(x) is single valued, then b1 = 0. Although Eqs. 7 and 8 are an obvious approach to the evaluation of the fundamental output of a nonlinearity, they are somewhat indirect, in that one must rst determine the output waveform y( ) from the known nonlinear characteristic and sinusoidal input waveform. This is avoided if the substitution = cos1 (x/a) is made, in which case, after some simple manipulations, it can be shown that
The function p(x) is the amplitude probability density function of the input sinusoidal signal and is given by
An additional advantage of using Eqs. 9 and q0 is that they easily yield proofs of some interesting properties of the DF for symmetrical odd nonlinearities. These include the following:
1. For a double-valued nonlinearity the quadrature component Nq (a) is proportional to the area of the nonlinearity loop, that is, Nq (a) = (1/a2 ) (area of nonlinearity loop) 2. For two single-valued nonlinearities n (x) and n (x), with n (x) < n (x) for all 0 < x < b, we obtain N (a) < N (a) for input amplitudes less than b. 3. For the sector bounded single-valued nonlinearity that is, k1 x < n(x) < k2 x for all 0 < x < bwe have k1 < N(a) < k2 for input amplitudes less than b. This is the sector property of the DF, and it also applies for a double-valued nonlinearity if N(a) is replaced by M(a). When the nonlinearity is single-valued, it also follows directly from the properties of Fourier series that the DF, N(a), may also be dened as follows: 1. The variable gain, K, having the same sinusoidal input as the nonlinearity, which minimizes the meansquared value of the error between the output from the nonlinearity and that from the variable gain. 2. The covariance of the input sinusoid and the nonlinearity output divided by the variance of the input. Evaluation of the Describing Function Tables of DFs for a variety of nonlinear characteristics can be found in many books (6, 7). However, to illustrate the evaluation of the DF of a nonlinearity a few simple examples are considered below. Cubic Nonlinearity. For this nonlinearity n(x) = x3 and using Eq. (16), one has
Figure 9. Saturation characteristic and output waveform.
ideal saturation characteristic shown in Fig. 9 the nonlinearity output waveform y( ) is as shown in the same gure. Because of the symmetry of the nonlinearity the fundamental of the output can be evaluated from the integral over a quarter period so that
which for a > gives
with = sin1 /a. Evaluation of the integrals gives
which on substituting for gives the result
Since for a < the characteristic is linear, giving N(a) = m, the DF for ideal saturation is mNS (/a), where
giving N(a) = 3a2 /4. Alternatively from Eq. (23) we have
Alternatively one can integrate Eq. (9) by parts to give

a
a1 = (4/a) The integral n = xn p(x) dx is known as the nth moment of the probability density function; and for the sinusoidal distribution with p(x) = (1/)(a2 x2 )1/2 , n has the value
0
n (x)(a2 x2 )1/2 dx
if
n(0) = 0
so that using the substitution x = a sin , this yields
as before. Therefore a1 = (4/a) 34 a4 = 3a3 /4, as before. Saturation Nonlinearity. The DF can also be found by taking the nonlinearity input as a sin , in which case for the Relay with Dead Zone and Hysteresis. The characteristic of a relay with dead zone and hysteresis is shown in Fig. 10 together with the corresponding input, assumed equal to a cos , and the corresponding output waveforms. Using
Eqs. 7 and 8 over the interval /2 to /2 and assuming that the input amplitude a is greater than + gives
valued or whether G(j) is available from a transfer function G(s) or as measured frequency response data. Typically the functions G(j) and N(a) are plotted separately on Bode, Nyquist, or Nichols diagrams. Alternatively, stability criteria such as the HurwitzRouth or root locus plots may be used for the characteristic equation
where = cos1 [( )/a] and = cos1 [( + )/a,] and although here it should be remembered that the equation is appropriate only for s j. Figure 12 illustrates the procedure on a Nyquist diagram, where the G(j) and C(a) = 1/N(a) loci are plotted and shown intersecting for a = a0 and = 0 . The DF method therefore indicates that the system has a limit cycle with the input sinusoid to the nonlinearity, x, equal to a0 sin(0 t + ), where depends on the initial conditions. When the G(j) and C(a) loci do not intersect, the DF method predicts that no limit cycle will exist if the Nyquist stability criterion is satised for G(j) with respect to any point on the C(a) locus. Obviously, if the nonlinearity has unit gain for small inputs, the point (1, j0) will lie on C(a) and it may then be used as the critical point, analogous to the situation for a linear system. When the analysis indicates that the system is stable, its relative stability may be indicated by evaluating its gain and phase margin. These can be found for every amplitude a on the C(a) locus, so it is usually appropriate to use the minimum values. In some cases a nonlinear block also includes dynamics so that its response is both amplitude and frequency dependent and its DF will be N(a, ). A limit cycle will then exist if
Thus
For the alternative approach, one must rst obtain the inphase and quadrature nonlinearities which are shown in Fig. 11. Using Eqs. 23 and 24, one obtains
The DFs for other relay characteristics can easily be found from this result. For no hysteresis, = 0; for no dead zone, = 0; and for an ideal relay, = = 0. It is easily shown that the DF of two nonlinearities in parallel is equal to the sum of their individual DFs, a result which is very useful for determining DFs, particularly of linear segmented characteristics with multiple break points. Several procedures (6) are available for obtaining approximations for the DF of a given nonlinearity either by numerical integration or by evaluation of the DF of an approximating nonlinear characteristic dened, for example, by a quantized characteristic, linear segmented characteristic, or Fourier series. Stability and Limit Cycles To study the possibility of limit cycles in the autonomous closed loop system of Fig. 1, the nonlinearity n(x) is replaced by its DF N(a). Thus, the open-loop gain to a sinusoid is N(a)G(j) and a limit cycle will exist if
where G(j) = GC (j)G1 (j). This condition means that the rst harmonic is balanced around the closed loop. Since G(j) is a complex function of and N(a) may be a complex function of a, a solution to Eq. (13) will yield both the frequency and amplitude a of a possible limit cycle. Various approaches can be used to examine Eq. (13) with the choice affected to some extent by the problem for example, whether the nonlinearity is single- or double-
To check for possible solutions of this equation, a family of C(a, ) loci, usually as functions of a for xed values of , may be drawn on the Nyquist diagram. A further point of interest when a solution to Eq. (40) exists is whether the predicted limit cycle is stable. This is obviously important if the control system is designed to have a limit cycle operation, as in the case of an onoff temperature control system, but it may also be important in other systems. If, for example, an unstable limit cycle condition is reached, the signal amplitudes will not become bounded but may continue to grow. The stability of a limit cycle, provided that only one solution is predicted, can be assessed by applying the Nyquist stability criterion to points on the C(a) locus at both sides of the solution point. If the stability criterion indicates instability (stability) for a point on C(a) with a < a0 and indicates stability (instability) for a point on C(a) with a > a0 , then the limit cycle is stable (unstable). The situation is more complicated when multiple limit cycle solutions exist and the above criterion is a necessary but not sufcient result for the stability of the limit cycle (8). The stability of the limit cycle can then normally be ascertained by examining the roots of the characteristic equation
Figure 10. Relay and input/output waveforms.
Figure 11. In-phase and quadrature characteristics for the relay of Fig. 10.
evaluated from
where n (x) and p(x) are as previously dened. It can also be shown that Ni (a) is related to N(a) by the equation
Figure 12. Illustration of limit cycle evaluation.
Thus, for example, for an ideal relay, putting = = 0 in Eq. (12) gives N(a) = 4h/a, and substituting this value in Eq. (14) yields Ni (a) = 2h/a. As an example of using the DF to investigate the possibility of a limit cycle, consider Fig. 1 with n (x) = x (x3 /6), Gc (s) = 1 and G2 (s) = K (1 s)/s(s + 1). For this nonlinearity, N(a) = 1 (a2 /8), so that the C(a) locus starts at 1 on the Nyquist diagram and, as a increases, moves along the then, for a greater negative real axis to for a = 2 than this value, the C(a) locus returns along the positive real axis from to the origin as a becomes large. An os-
where Ni (a) is known as the incremental describing function (IDF). For a single-valued nonlinearity, Ni (a) can be
10
cillation will only start to build up, assuming zero initial conditions, if the feedback loop with G(s) alone is unstable since N(a) 1 for a 0. This requires the characteristic equation
to have a root with a positive real part; that is, K > 1. G(j) has 180 phase shift when = 1 when its gain is K. Thus the DF solution for the amplitude of the limit cycle is given by
which results in
giving
As K is increased, because of the shape of the nonlinearity, the limit cycle becomes more distorted. For example, if K = 2.4, the DF solution gives = 1 and a = 2.10, whereas if four harmonics are balanced, which requires a computer program, the limit cycle frequency is 0.719 and the amplitudes of the fundamental, third, fth, and seventh harmonics at the input to the nonlinearity are 2.515, 0.467, 0.161, and 0.065, respectively. As the DF approach is a method for evaluating limit cycles, it is sometimes suggested that it cannot be used to guarantee stability of a feedback system, since instability may be exhibited by a signal in the system becoming unbounded, not oscillatory. It is, however, known for the autonomous feedback system of Fig. 1 that if the symmetric odd, single-valued nonlinearity n(x) is sector-bounded such that k1 x < n(x) < k2 x for x > 0 and n(x) tends to k3 x for large x, where k1 < k3 < k2 , then the nonlinear system is either stable or possesses a limit cycle, provided that the linear system with gain K replacing N is stable for k1 < K < k2 . Thus for this situation, which is often true in practice, the nonexistence of a limit cycle guarantees stability. Accuracy of the Describing Function Since the DF method is an approximate analytical approach, it is desirable to have some idea of its accuracy. Unfortunate consequences may result if a system is predicted to be stable, and in practice this turns out not to be the case. Although many attempts have been made to nd solutions for this problem, those that have been obtained either are difcult to apply or produce results which are often as conservative as the absolute stability criteria discussed earlier. Since, as has already been shown, the C(a) locus of a sector bounded, single-valued nonlinearity is the diameter of the circle in the circle criterion, errors in the DF method are related to its inability to predict a phase shift which the fundamental may experience in passing through the nonlinearity, rather than an incorrect magnitude of the gain. When the input to a single-valued nonlinearity is a sinusoid together with some of its harmonics, it is easy
to show that the fundamental output is not necessarily in phase with the fundamental input; that is, the fundamental gain has a phase shift. The actual phase shift which occurs varies with the harmonic content of the input signal in a complex manner, since the phase shift depends on the amplitudes and phases of the individual harmonic input components. From an engineering viewpoint, one can therefore obtain a good idea of the accuracy of a DF result by estimating the distortion, d, in the waveform at the input to the nonlinearity. This is relatively straightforward when a limit-cycle solution is obtained since the sinusoidal signal corresponding to the DF solution can be taken as the nonlinearity input and the harmonic content of the signal fed back to the nonlinearity input calculated. Experience indicates that the percentage accuracy of the DF method in predicting the fundamental amplitude and frequency of the limit cycle is usually better than the percentage distortion in the fedback signal. It is also important to note that the amplitude predicted by the DF is an approximation to the fundamental of the limit cycle, not its peak amplitude. It is possible to estimate the limit cycle more accurately by balancing additional harmonics, as mentioned earlier. Although algebraically this is difcult apart from loops with a nonlinearity having a simple mathematical descriptionfor example, a cubicit can be done computationally. The procedure involves solving sets of nonlinear algebraic equations, but good starting guesses can usually be obtained for the magnitudes and phases of the other harmonic components from the waveform fedback to the nonlinearity, assuming that its input is the DF solution. Further Aspects Before concluding this section on the DF method, it is important to mention two other facets of its application. In introducing the DF, it was indicated that the existence of a limit cycle is usually undesirable; thus if the DF indicates such behavior, the system must be compensated to remove the limit cycle. If the parameters of n(x) and G1 (s), with Gc (s) = 1, in Fig. 1 are such that a limit cycle is indicated, because the loci G1 (j) and C(a) intersect, then a compensator Gc (s) can be added with a transfer function such that the loci Gc (j)G1 (j) and C(a) do not intersect. Shaping frequency responses to achieve a specic form is a familiar approach in linear control theory, so this approach can be easily applied. Other approaches such as adding additional feedback paths to compensate for the effect of the nonlinearity may also be possible. This procedure has the advantage, as can the approach of designing a nonlinear controller, of producing an approximately linear system. A feature of nonlinear systems, as mentioned earlier, is that they possess unique forms of behavior. One such interesting feature is the jump resonance which can occur when a nonlinear feedback system, such as Fig. 1, has a sinusoidal input. Equations can be set up using the DF approach for the feedback loop to balance the harmonic at the frequency of the input sinusoid. Two nonlinear algebraic equations are obtained; and for some situations they can have three, rather than one, solutions for a small
11
range of input frequencies. The DF can also be used to show that only two of the solutions will be stable, which means that the approximately sinusoidal output from the feedback loop may have two possible values, within this frequency range, which, if it exists, is found near to the resonant frequency of the linearized system. When the input frequency is changed so that the solution of the equations moves from the two-stable-solution to the one-solution (or vice versa) region, a discontinuous change, or jump, in the magnitude of the output may occur.
Figure 13. Relay output waveform.
where UG (n) = gn cos n and VG (n) = gn sin n Eq. (15) can be written as
LIMIT CYCLES IN RELAY SYSTEMS In this section an exact method for the evalution of limit cycles and their stability is discussed which makes use of the fact that the output from a relay is not continuously affected by its input (6, 9). The input only controls the switching instants of the relay and has no further effect on the output until it causes another switching. Therefore to investigate limit cycles in relay systems the analysis starts by assuming a typical relay output waveform, y(t), which for a symmetrical odd limit cycle in a loop having a relay with dead zone and hysteresis takes the form shown in Fig. 13, where T and t are unknown and the initial switching is at time t1 . Then to nd a possible limit cycle in the autonomous system of Fig. 1 the steady-state response of G(s) to this waveform has to be determined. Several slightly different approaches are possible, but here we follow that used by Tsypkin, primarily because for a relay with no dead zone it allows a simple comparison with the DF method. y(t) is expanded in a Fourier series which gives y(t ) = 2h
Similarly, c(t) can be shown to be given by
To satisfy the above-mentioned switching conditions at times t1 and t1 + t, assuming t1 to be zero without loss of generality, and bearing in mind that x(t1 ) should be positive and x(t1 + t)negative, we require that
where RP and IP denote the real and imaginary parts, respectively. The IP expressions give two nonlinear algebraic equations which, if they have solutions, yield the unknown parameters t and T of possible limit cycles. Using these solution values, the corresponding relay input waveform x(t) can be found, from which the RP conditions can be checked, as can the continuity conditions
+[1 cos(n t )]sin[n(t t1 )]} The output c(t) is then given by to conrm that the relay input signal does not produce switchings other than those assumed. Since closed-form expressions exist for the A loci of simple transfer functions, analytical solutions can be obtained for the exact frequencies, 1/T, of limit cycles for some specic systems, especially those in which the relay has no dead zone. Then t = and the above two nonlinear algebraic equations are identical since only one unknown, T, remains. When the nonlinear algebraic equations are solved computationally the closed-form expressions for the A loci may be used, or their value may be determined by taking a nite number of terms in the series of Eqs. 16 and 17 (6, 9). Another interesting feature of this method is that it is also possible to determine whether a solution to the nonlinear algebraic equations corresponds to a stable or an unstable limit cycle (10). The analysis has assumed a symmetrical odd limit cycle but can be extended to situations where this is not the case. More nonlinear algebraic equations have then to be solved to obtain any possible limit cycle solutions. It is also possible to extend the approach to
n=1(2)
1 {sin(n t )cos[n(t t1 )] n
where gn = |G(jn)| and n = G(jn). Using the A loci, dened by
12
Figure 14. Graphs giving exact and approximate solutions for limit cycle.
Figure 15. Stable limit cycle solution.
nd multi-pulse type limit cycles which may exist in relay control of resonant plant transfer functions. Two Examples As a simple application of the above method, consider a relay with no dead zonethat is, = 0so that one has the single relationship
which yields the frequency of the limit cycle. If G(s) = K/s(1+ s ), then the above expression gives the equation
where = for the limit cycle frequency . This compares with the DF solution for the same problem, which yields the equation
Figure 16. Unstable limit cycle solution.
It is also interesting that, since the line with RP < 0 and IP = /4h corresponds to C(a), the negative reciprocal of the DF, the exact and approximate DF solutions can be compared graphically. This is done in Fig. 14, which shows the G(j) and AG (0, ) loci for K = = 1 and the C(a) locus for h/ = 3. The exact limit-cycle frequency is 1.365 rad/s, and the approximate solution using the DF method is 1.352 rad/s. The accuracy of the DF result may be used to conrm the lter hypothesis, since it can be shown that as is increased, thus making G(s) a better low-pass lter, the error in the DF solution for the frequency of the limit cycle decreases. Consider as a second example a feedback system having a relay with output 1 and dead zone 1, along with a transfer function G(s) = 5/s(s2 + 3s + 1). Use of the DF method indicates that the system has two limit cycles, both of frequency 1.000 rads/s, with the larger amplitude one stable and the smaller amplitude one unstable. Two nonlinear algebraic equations need to be solved using the Tsypkin method to nd the frequency and pulse width of any limit cycles. Software with graphical facilities is available to do this and the two limit cycles shown in Figs. 15 and Fig. 16 are found. The larger amplitude limit cycle of 15 are found. The larger amplitude limit cycle of 15 is shown
by the method to be stable with frequency 0.988 rads/s and pulse width 1.967 s, and the smaller amplitude one of Fig. 16 with frequency 0.736 rads/s and pulse width 0.716 s is unstable. It should also be noted that the larger amplitude limit cycle is much closer to a sinusoid so that its frequency is nearer the DF solution of 1.000 rads/s. SLIDING MODE CONTROL METHODS For ease of exposition, consider the uncertain linear time invariant system with m inputs given by
where A Rnn and B Rnm with 1 m. Without loss of generality it is assumed that the inputs are independent. The nonlinear function f: R Rn Rm Rn is assumed to be unknown but bounded by some known functions and represents the parameter uncertainties, nonlinearities, or disturbance signals which are present in the system. Let S be the hyperplane dened by
where S Rmn is of full rank. This will dene the switching function. It should be noted that the choice of S need
13
not be restricted to a hyperplane and more general, nonlinear, possibly time-varying switching functions may be chosen. If there exists a nite time ts such that the solution to Eq. (61) satises
Having dened the switching function, it is necessary to establish sufcient conditions which guarantee that an ideal sliding motion will take place. These will amount to ensuring that in a certain domain enclosing the sliding surface, the trajectories of s(t) must be directed toward it. The associated so-called reachability condition is perhaps most succinctly expressed as
then a sliding motion is taking place for all t ts . This section will rst consider how to design the switching function so that the sliding motion is stable. The problem of designing variable structure control laws so that in nite time the system states are forced on to and subsequently remain on the hyperplane S is considered next. The total insensitivity to a particular class of uncertainty is then demonstrated. The section will conclude with a straightforward example to illustrate the mathematical concepts. For sliding mode design it is necessary that the system assumes an appropriate canonical form. This so-called regular form is obtained using the orthogonal matrix T1 Rnn whereby
This choice is readily justied by considering the function
which is positive denite. Its time derivative along any trajectory is
It follows that if Eq. (70) holds, then V tends to zero and therefore s tends to zero. This guarantees that a sliding mode is attained. The control signal must thus be dened to satisfy Eq. (70). Subject to this constraint, there are obviously a great many possible control congurations. A common structure is given by
where B2 Rmm and is full rank. The transformation matrix exists as B is full rank and can be readily found via QR decomposition. By using the coordinate transformation x T1 x then the nominal linear system can be written as
where u1 (t) is a linear state feedback law and un (t) is a discontinuous or switched component of the form
where x1 R , x2 R . Effectively, the system has been decomposed into two connected subsystems, only one of which is directly affected by the system input. If the switching function matrix from Eq. (62) is partitioned compatibly in this coordinate system, then
m
n m
The extensive interest in sliding mode control is primarily due to its robustness properties. When sliding, a system is completely insensitive to any uncertainty which is implicit in the channels of the input distribution matrix; such uncertainty is termed matched uncertainty. The reason for this invariance property is easily demonstrated by a consideration of the uncertain state space system
where S1 Rm(nm) and S2 Rmm . During ideal sliding, the motion is given by
where f is an unknown but bounded forcing function. In the sliding mode
Assuming S2 is chosen by design to be nonsingular, it follows that
and thus
where M = S1 2 S1 . It further follows that in the sliding mode, m of the states can be expressed in terms of the remaining (n m) and thus a reduction in order occurs. The reduced order motion is determined from substituting for x2 (t) from Eq. (21) in Eq. (20) as
Substituting from Eq. (75) into Eq. (77),
where ueq is not the applied control signalwhich will be of the form of Eq. (73)but does denote the equivalent linear control that would be required to maintain the sliding mode. This may be expressed as
The hyperplane design problem can therefore be considered to be one of choosing a state feedback matrix M to prescribe the required performance to the reduced order subsystem dened by (A11 , A12 ). It can be shown that controllability of the nominal (A, B) pair is sufcient to guarantee controllability of the (A11 , A12 ) pair.
Substituting this expression for the equivalent control in Eq. (75) yields
14
The dynamics in the sliding mode are thus completely invariant to the signal f. The system behavior will be determined entirely by Eq. (69) when sliding despite the presence of such matched uncertainty. Any unmatched uncertainty will affect the dynamics of Eq. (69), but such unmatched effects can be minimized by designing M such that the subsystem of Eq. (69) is maximally robust. In order to illustrate the theoretical concepts of VSCS, consider the double integrator system
The system can be expressed in the state-space form
where x1 = y and x2 = y. This system is already in an appropriate regular form for sliding mode design. Consider application of a negative feedback law
The phase portraits when k = 0.5 and k = 1.5 are shown in Fig. 17. Neither control law yields an acceptable transient when employed as the sole controller; an oscillation is seen to exist in both cases. Consider instead the VSCS dened by
An asymptotically stable motion is seen to result as shown in Fig. 18. By introducing a rule for switching between two control structures, which independently do not provide stability, a stable closed-loop system is formed. Such heuristic arguments can be used to motivate the advantages of a variable structure control approach. However, for design purposes a more logical algorithmic approach is required.<gureAnchor gures="W1024-g-0017 W1024-g0018"/> Consider now the switching function dened by
Figure 17. (a) u = 1.5x1 ; (b) u = 0.5x1 . Double integrator control.
lum system. The design parameters m and k are both set equal to unity. Figure 19 shows the resulting phase plane plot. The system enters a sliding mode and the normalized pendulum is forced to behave as the free rst-order system
This is seen to provide a rst-order motion in the sliding mode where the pole dening the transient response is determined by the selection of m. The control signal is dened by solving for u from the relationship
during this phase of motion. The dynamics in the sliding mode have been wholly specied by the choice of switching function despite the presence of a matched uncertainty contribution. OTHER DESCRIBING FUNCTIONS In a previous section the discussion on describing functions was primarily restricted to the sinusoidal describing function, SDF, since this is used extensively in looking at the effects of nonlinearity in practical systems. Many control systems, however, are subject to inputs or disturbances which cannot be dened deterministically but only as random signals with a given frequency spectrum and amplitude probability density function. The most common amplitude probability density function, p(x), considered is the Gaussian density function which, when it has a zero
This clearly ensures that the reachability condition [Eq. (23)] is satised. Essentially, the switching function is differentiated and the resulting state derivatives are replaced with the original system dynamics. Equation (27) thus yields an expression for the control signal in terms of the states and the value of the switching function. The resulting controller will nd the switching function at least locally attractive. For simulation, the double integrator model is subject to a disturbance signal a1 sin (x1 (t)) which acts in the range of the input distribution matrix. In this way a controller designed for a nominal double integrator model is implemented upon a normalized pendu-
15
Figure 18. Double integrator with variable structure control system.
ate p(x) is used. When the input x is a sinusoidal or Gaussian signal, however, it can also be shown that the error signal between the nonlinearity and linear gain outputs that is, n(x) Keq xis uncorrelated with the input x (6). Typically, when dealing with Gaussian inputs to a simple nonlinear feedback system, the mean-squared values of the signals at various points in the loop can be calculated approximately using the RDF for the nonlinearity. In many feedback systems it may be necessary to take account of bias, as well as other signals, due to constant input or disturbance signals or because of asymmetry in nonlinear characteristics. In this case the nonlinearity, again using the minimum mean-squared error denition, may be modeled by two DFs, one for the bias, , and one for the other input (6, 7). When the other input is considered as a sinusoid of amplitude a, then the two DFs for the singlevalued nonlinearity n(x) are given by
and
the former being the DF for the sinusoid and the latter for the bias. Here p(x) = 1/(a2 x2 )1/2 for the sinusoidal signal. Use of this DF allows, amongst other possibilities, for the determination of limit cycles with bias. For example, if in Fig. 1 the input r(t) has a constant value R, then balancing the bias and fundamental of the limit cycle gives the two equations
Figure 19. Normalized pendulum with sliding mode control.
mean, is given by
A general way to dene a DF, as mentioned earlier, is that it is that value of gain, Keq , which, when fed with the same input as the nonlinearity n(x), will give a minimum value for the mean-squared error between the nonlinearity output and the gain output. It is then relatively easy to show that
when the input signal x has a Gaussian distribution. When this result was rst found, the gain was called the equivalent gain, hence the notation Keq , not the random describing function (RDF) by which name it is now usually known. The result for x with any zero-mean amplitude probability density function p(x) is
a formula which can be used for a single-valued nonlinearity with sinusoidal or Gaussian inputs when the appropri-
The equations can be solved to nd the bias and sinusoidal amplitude a of the limit cycle at the input to the nonlinearity. The above approach can be used in principle to obtain a DF representation for a nonlinearity whose input consists of any number of uncorrelated signals, but for practical reasons the approach is difcult to justify for more than two or possibly three components. A difculty in applying such multiple input describing functions is understanding the errors which are caused by neglecting not only higher harmonics of the input signals but also cross-modulation products which may be produced at frequencies lower than those in the nonlinearity input signal. Reasonably successful use, however, of the DF approach for two related frequency sinusoidal inputs to a nonlinearity has been achieved to give results of some value in control system design. This requires consideration of two inputs such as a cos t and b cos(3t + ) so that the describing functions for the two signals become functions of the three parameters a, b and , not just a and b (6). Analytically, results can only be obtained for simple nonlinearities such as a cubic; but by using computational methods, other characteristics can be considered (11). This procedure has been used to investigate subharmonic oscillations and synchronization phenomena when the feedback loop of Fig. 1 has an input r(t) which is sinusoidal, and it
16
Obviously, the relay onoff levels control the limit cycle amplitude; and if these are varied, some information may be found about any nonlinearity in the plant (14, 15). In such cases it may be possible to make the system behavior more linear by incorporating appropriate nonlinearity in the PID elements of the controller. MULTIVARIABLE SYSTEMS So far in this presentation, only simple nonlinear feedback systems such as Fig. 1 have been considered, apart from when discussing the phase plane. In principle there is no difculty in extending use of both the DF and Tsypkin approaches to feedback loops with more than one nonlinear element, although for the latter approach the nonlinearities must be of the relay type. The problem with using the sinusoidal describing function approach is that the assumption of a sinusoidal input to all the nonlinearities must be reasonable for the situation under investigation. For some congurations of nonlinearities and linear dynamic elements in a feedback loop, this will not be true. Using the Tsypkin approach, more nonlinear algebraic equations are formulated and their possible solutions must be investigated (6, 16). Several investigators have produced results for studies on the multivariable, typically two-inputtwo-output version of Fig. 1. Here the nonlinearity consists of four individual, or in many cases only two on the diagonal, nonlinear elements. Using describing function analysis for this conguration can often be justied since good ltering may exist for all theor in the case of two inputtwo output, two feedback loops. Software written to investigate such systems, using both the DF and Tsypkin methods, has been described in the literature (1719). Absolute stability results, similar to those given earlier but which result in more complicated graphical procedures, have been extended to the two-inputtwo-output multivariable system (20). Like the situation for the single-inputsingle-output system, however, the results they produce are often very conservative and may not be of value for practical problems. SLIDING MODE CONTROL The detailed discussion here comments further on uncertain systems. It is obviously desirable to consider other nominal nonlinear system descriptions in order to broaden the applicability of the method. Some work in this area has considered problems relating to a particular application areafor example, roboticsand has developed sliding mode controllers for such specic classes of nonlinear system (21). Other work has tried to consider more general nonlinear system descriptions. Some of the conditions placed upon the particular system representation can be quite restrictive. One example of this is the class of feedback linearizable systems. It is possible to augment the traditional linearizing feedback with a sliding mode component which will provide robustness to some uncertainty in the sliding mode. However, the conditions which must be satised to feedback linearize the system initially are quite restrictive and so limit the applicability of the methodol-
Figure 20. Kc and wc from the Nyquist plot.
has also been used for the more accurate determination of limit cycle waveforms by balancing both the fundamental and another harmonic, usually the third. RELAY AUTOTUNING A relatively recent application of the DF approach in control system design is its use in the relay autotuning method for setting PID controller parameters. The procedure is very useful in those cases where it is difcult to obtain a good mathematical model for the process or retuning has to be done on site by an operator. The basic concept employed is that knowledge of the critical frequency, c , and gain, Kc , of a process illustrated on a plant frequency response shown in Fig. 20 can often provide sufcient information for setting the parameters of a PID controller. The approach was suggested many years ago by Ziegler and Nichols (12), where Kc and c were found by placing the PID controller in the P mode and adjusting the gain until oscillation took place. There were difculties doing this in practice, however, one problem being that the oscillation amplitude is only limited by saturation in the controller, actuator or plant. More recently therefore, Astrom and Hagglund (13) recommended estimating c and Kc from results obtained by replacing the controller with an ideal relay, a feature, as illustrated in Fig. 21, which can easily be built into a modern microprocessor controller. When this is done the amplitude of the limit cycle, a, at the relay input and its frequency, c , are measured. Then according to DF theory, c = 0 and Kc = 4h/a. Strictly speaking, a should be the amplitude of the fundamental frequency component, and the results are only exact when the limit cycle is sinusoidal. In many cases, however, these formulae provide reasonable estimates for c and Kc which may then be used in an appropriate algorithm (many of which have been put forward recently) for setting the PID parameters. If the form of the plant transfer function is known but not its parameters, then it may be possible, certainly for low-order transfer functions, to make use of the known Tsypkin solution for the limit cycle to estimate the plant parameters. When the plant has several unknown parameters, more than one autotuning test may have to be done using different values of hysteresis in the relay or possibly with bias signals introduced into the loop.
17
Figure 21. Block diagram for relay autotuning.
ogy. Perhaps the most widely applicable method to date has resulted from the development of sliding mode control schemes for differential inputoutput system representations (22). These yield dynamic controllers which act as a natural lter on any discontinuous elements of the control signal and are applicable to a fairly broad class of nonlinear systems. This is thus a considerable breakthrough in the development of robust controllers for nonlinear system descriptions. The previous exposition relates to state-feedback based sliding mode control schemes (23). For practical application, controllers based upon measured output information are required (24). There are two ways to approach this problem. A dynamical system, or observer, may be used to estimate unmeasurable system states. Because of the inherent robustness of sliding mode controllers, some signicant work has considered the development of an associated sliding mode observer (25, 26). The robustness properties have been shown to transfer. However, there are restrictions on the (A, B, C) triple used for observer design. In particular, the invariant zeros of (A, B, C) are shown to play a crucial role in determining the zero dynamics in the sliding mode. It thus follows that these invariant zeros must be stable. Despite this restriction, the closed-loop robustness properties of such a sliding mode controllerobserver scheme are excellent. The sliding observer is also nding a promising area of application relating to fault detection and isolation. The second approach to output feedback based sliding mode control is to develop design methodologies which produce output dependent control strategies (27). This restricts the class of systems which may be considered as the switching surface must be output-dependent, and thus S must lie in the range of the output distribution matrix. Again the sliding mode dynamics will be dependent upon the location of the system transmission zeros. The development of design methodologies and associated case studies relating to output feedback-based sliding mode control strategies and sliding mode controllerobserver strategies require further development. The development of sliding mode controllers based upon output measurements for nonlinear systems is very much an open research problem.
3. I. W. Sandberg, A frequency domain condition for the stability of feedback systems containing a single time-varying nonlinear element, Bell Syst. Tech. J., 43 (4): 16011608, 1964. 4. D. P. Atherton, D. H. Owens, Boundedness properties of nonlinear multivariable feedback systems, Electron. Lett., 15 (18): 559ff, 1979. 5. Y. S. Cho, K. S. Narendra, An off-axis circle criterion for the stability of feedback systems with a monotonic linearity, IEEE Trans. Autom. Control, 13 (4): 413416, 1968. 6. D. P. Atherton, Nonlinear Control Engineering: Describing Function Analysis and Design, London: Van NostrandReinhold, 1975. 7. A. Gelb, W. E. Vander Velde, Multiple-Input Describing Functions and Nonlinear System Design, New York: McGraw-Hill, 1996. 8. S. K. Choudhury, D. P. Atherton, Limit cycles in high-order nonlinear systems, Proc. Inst. Electr. Eng., 121: 717724, 1974. 9. P. A. Cook, Nonlinear Dynamical Systems, London: PrenticeHall International, 1986. 10. R. Balasubramanian, Stability of limit cycles in feedback systems containing a relay, IEE Proc., Part D, 1: 2429, 1981. 11. J. C. West, J. L. Douce, R. K. Livesley, The dual input describing function and its use in the analysis of nonlinear feedback systems, Proc. Inst. Electr. Eng., Part B, 103: 463474, 1956. 12. J. G. Ziegler, N. B. Nichols, Optimum settings for automatic controllers, Trans. ASME, 64: 759768, 1942. 13. K. J. Astrom, T. Hagglund, Automatic tuning of simple regulators with specications on phase and amplitude margins, Automatica, 20 (5): 645651, 1984. 14. J. H. Taylor, K. J. Astrom, A Nonlinear PID Autotuning Algorithm, Seattle, WA: ACC, 1986, pp. 16. 15. D. P. Atherton, M. Benouarets, O. Nanka-Bruce, Design of nonlinear PID controllers for nonlinear plants, Proc. IFAC World Cong. 93, Sydney, Australia, 1993, Vol. 3, pp. 355358. 16. D. P. Atherton, Conditions for periodicity in control systems containing several relays, 3rd IFAC Cong., London, 1966, Paper 28E. 17. J. H. Taylor, Applications of a general limit cycle analysis method for multivariable systems, inR. V. Ramnath, J. K. Hedrick, andH. M. Paynter (eds.), Nonlinear System Analysis and Synthesis, New York: ASME, 1980, Vol.2. 18. D. P. Atherton et al., Suns, the Sussex University Nonlinear Control Systems Software, 3rd IFAC/IFAD Symp. Comput. Aided Des. Control Eng. Syst., Copenhagen, 1985, pp. 173178. 19. O. P. McNamara, D. P. Atherton, Limit cycle prediction in free structured nonlinear systems, IFAC Cong., Munich, 1987, Vol. 8, pp. 2328. 20. D. P. Atherton, Stability of Nonlinear Systems, New York: Wiley, Research Studies Press, 1981.
BIBLIOGRAPHY
1. K. S. Narendra, J. H. Taylor, Frequency Domain Criteria for Absolute Stability, New York: Academic Press, 1973. 2. V. M. Popov, Absolute stability of nonlinear control systems of automatic control, Autom. Remote Control, 22: 857858, 1962.
18
21. J. J. E. Slotine, W. Li, Applied Nonlinear Control, London: Prentice-Hall International, 1991. 22. H. Sira Ramirez, On the dynamical sliding mode control of nonlinear systems, Int. J. Control, 57: 10391061, 1993. 23. V. I. Utkin, Sliding Modes in Control Optimisation, Berlin: Springer-Verlag, 1992. 24. C. Edwards, S. Spurgeon, Sliding Mode Control: Theory and Applications, London: Taylor & Francis, 1997. 25. J. J. E. Slotine, J. K. Hedrick, E. A. Misawa, On sliding observers for nonlinear systems, J. Dyn. Syst. Meas. Control, 109: 245252, 1987. 26. C. Edwards, S. K. Spurgeon, On the development of discontinuous observers, Int. J. Control, 59: 12111229, 1994. 27. S. Hui, S. H. Zak, Robust output feedback stabilisation of uncertain systems with bounded controllers, Int. J. Robust Nonlinear Control, 3: 115132, 1993.
DEREK ATHERTON SARAH SPURGEON

School of Engineering University of Sussex Brighton, England, BN1 9QT Department of Engineering University of Leicester Brighton, England, BN1 9QT
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Nonlinear Control Systems, Design Methods Standard Article Antonio Loria1 and Henk Nijmeijer2 1University of California at Santa Barbara 2University of Twente, Enschede, The Netherlands Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1025 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (268K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Regulation Problem Linear Time-Invariant Systems Feedback Linearization Passivity-Based Control Control Under Model and Parameter Uncertainties Third-Order Feedback Linearizable Systems Conclusion Acknowledgments Keywords: nonlinear control; feedback linearization; EulerLagrange systems; passivity; observers; robust control About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
NONLINEAR CONTROL SYSTEMS, DESIGN METHODS
519

A basic problem in control theory may be described as follows: Given a plant, design a control mechanism in such a way that the plant together with the controller meets certain design specications. The above regulation problem arises in numerous situations; for instance, the temperature in a house is regulated by a thermostat to keep the temperature in the house constant, notwithstanding changing external effects such as outdoor temperature, wind, open doors, and so on. Other regulation devices in everyday life are easy to nd: washing machines, modern automobiles, and so on. Probably the rst mathematical study on regulation ever published was written by J. C. Maxwell (18311870). His paper On governors published in the Proceedings of the Royal Society of London in 1868 treats the problem of tuning centrifugal governors to achieve fast regulation towards a constant speed, thereby avoiding oscillatory motions (hunting) of a steam engine. Clearly, in the past century a lot of theoretical and practical work has been carried out on the regulation problem, and it is certainly beyond the scope of this article to present a historical review of the subject; readers further interested in the subject are referred to Ref. 1. However, one particular type of controllers, the so-called proportional integral differential (PID) controller, originally proposed by N. Minorsky in 1922, deserves separate mentioning. In a PID controller the control signal is built up as a weighted sum of three terms; a proportional term (proportional to the error between the actual and desired value of the to-be-controlled plants output) drives the plants output to the reference. An integral term (of the error) compensates for the steady-state error caused by uncertainties in the plants model, and a differential term (proportional to the time derivative of the plants output) speeds up the convergence towards the desired reference. The PID controller has had and still has many applications in technical systems. More recent methods in nonlinear control theory that should be mentioned are feedback linearization, passivitybased control, and Lyapunov control.
The feedback linearization approach applies to a small class of systems for which it is possible to use a nonlinear control law which, given an appropriate coordinate change, cancels all nonlinearities in the system. The rationale behind this approach is that the resulting closed-loop system is linear, and thus linear control theory is then applicable. A drawback is that this technique may fail if one does not know the plants model accurately; this uncertainty can lead to instability or, in the best case, to a steady-state error. An alternative approach is the so-called passivity-based control. This technique applies to a certain class of systems which are dissipative with respect to a storage function (2). Passive systems constitute a particular case of dissipative systems for which the storage function happens to be an energy function. Hence, the rationale behind the passivity-based approach is physical: Roughly speaking, a passive system is a system from which one cannot pull out more energy than is fed in. A very simple example of a passive system is a conventional RLC network, which dissipates part of the supplied electrical energy in the form of heat. A fundamental property of passive systems is that the interconnection of two passive systems is passive. With this motivation, in passivity-based control one aims at designing passive controllers, so that the closed-loop system have some desired energy properties. In many cases, seeking for passive controllers results in compensating instead of canceling the nonlinearities of the system, which can give considerably more robust results than using feedback linearization. Modern control theory leans on the so-called Lyapunov stability theory which was launched by the Russian mathematician A. M. Lyapunov in his celebrated article (3). The Lyapunov theory consists of a set of mathematical tools, including comparison equations, which help us to analyze the asymptotic behavior of the solutions of a (possibly nonlinear and time-varying) differential equation. The advantage of this theory is that it allows us to know the asymptotic behavior of the solutions without solving the differential equation. The price paid for this is that one must nd a suitable Lyapunov function for the system in question, which satises some desired properties. More precisely, the Lyapunov function is positive denite while its time derivative is negative denite. This is in general not an easy task. Thus, the Lyapunov control approach consists of proposing a positive denite Lyapunov function candidate and designing the control input in such a way that its time derivative becomes negative denite. Then, some conclusions about the stability of the system can be drawn. Although in general, it is very difcult to nd a suitable Lyapunov function, often a good start is to use the total energy function (if available) of the system in question. This may motivate us to think that the passivity-based approach and Lyapunov control are very related since, for a physical system, the storage function is the total energy of the system. Nevertheless, it must be pointed out that the passivity-based approach is based upon the inputoutput properties of the system; that is, the system is viewed as an operator which transforms an input into some output, regardless of the internal state of the system. The Lyapunov control approach is based upon the asymptotic behavior of the systems state. Both methods are complementary to one another. We consider in more detail passivity-based and feedback linearization schemes, and the reader may consult Ref. 4 and
520
5 for introductory texts on Lyapunov theory. We consider from a mathematical perspective the regulation problem of an important class of nonlinear passive systems: the so-called EulerLagrange (EL) systems. This class of systems includes various mechanical systems such as the robot manipulators. Our motivation to illustrate these control techniques by addressing the regulation problem for EL systems is multiple: Not only are EL systems passive (hence passive controllers do a good job for this class), but also the stability theory of Lya punov was inspired upon the previous work of J. H. Poincare (18541912) on second-order nonlinear systems and, in particular, mechanical systems. Moreover, even though the Lagrangian formulation is most popular in mechanics, it must be remarked that it applies to a wide class of physical systems which are modeled using variational principles. Thus, on one hand, the EL class is fairly wide; hence the available results are applicable to many physical (industrial) systems. On the other hand, the theory is simple enough to be treated in a few pages and to give the reader a general appreciation of the avor of nonlinear systems control design in this context. We consider the regulation problem which consists of designing a feedback that enforces asymptotic tracking of the output to be controlled (e.g., the endpoint of a rigid manipulator) towards a given desired trajectory. Provided that an exact model of the plant is given and assuming that full state (that is, joint position and velocity) measurements are available, a tracking controller can easily be designed, like for instance the computed torque controller (6). See, for example, Refs. 7, 8, and 18 for a literature review. Unfortunately, in many cases, neither an exact model is available, nor are joint velocities measured. The latter problem can be resolved by introducing an observer, a dynamic system which uses only position information (or more general, the measurable output) to reconstruct the velocity signal (or more general, the unmeasured part of the state). Then, the controller is implemented by replacing the velocity by its estimate. It is interesting to note that even though it is well known that the separation principle (see section entitled Linear Time-Invariant Systems) does not apply for nonlinear systems (specically, an observer that asymptotically reconstructs the state of a nonlinear system does not guarantee that a given stabilizing state feedback law will remain stable when using the estimated state instead of the true one), the rationale behind this approach is precisely that the estimate will converge to the true signal and, in some particular cases of EL systems, this in turn entails stability of the closed loop. To cope with a nonexact plant model, one may proceed in different ways, depending on to what extent the model uncertainties appear. In essence, discussion in this article will be limited to the occurrence of some unknown (linearly depending) parameters, such as an unknown mass of the endtool, or there will be limited discussion of the model structure. In the rst case a parameter adapting or a PID controller can be used whereas in the last case we return to a (high-gain) PD controller design. Some of the above-mentioned results for EL plants are discussed in some detail in the following sections and can be found in the literature (8,9). See also Ref. 7 for a comprehensive tutorial on adaptive control of robot manipulators. One may wonder whether results as announced above for fully actuated systems also extend to other classes of nonlinear systems. Apart from the case where the system is as-
sumed to be linear (see Refs. 10), the observer and observer controller problems turn out to be difcult in general. At the moment, very few general results are at hand (11); as an illustration in the last section we discuss one specic mathematical example of a single-input single-output nonlinear system. xx is the Euclidean norm Notation. In this article, x n of x . The largest and smallest eigenvalues of a square symmetric matrix K are kM and km, respectively. The extended n n space of square integrable signals is L 2 e u , T T 0 : 0 u(t)2 dt . We denote B x n : x a ball of radius , centered at the origin in n. THE REGULATION PROBLEM We dene the general regulation problem for NL systems as follows: Tracking Control of NL Systems. Consider the system x = f (t , x, u ) y = h (x ) (1) (2)
where x n is the state, u k is the control input, y m is the output to be controlled, and functions f and h are continuous in their arguments and t 0. Given a desired reference output trajectory yd(t) assume that there exists a (unique) desired state xd(t) and input ud(t) trajectories which generate yd(t), that is, x d = f (t , xd , ud ) yd = h ( xd ) (3) (4)
Then the tracking control problem consists in nding, if possible, a smooth control law u such that for any bounded initial conditions (t0, x0), x0 x(t0),
t
lim y (t )
, tlim (y(t ) yd (t )) = 0
(5)
Remark 1. It is important to note that we have assumed that there exists xd(t) and ud(t) which are solutions of Eq. (3). This point is crucial since in practical applications one should pay attention to the denition of the desired output trajectory, which may not be realizable due to the structure of the system. In this article we do not address the so-called trajectory generation problem (see, for instance, Ref. 12) or the inversion problem considered in Ref. 13, but we simply assume that the triple (xd, ud, yd) is given. Note that in the problem formulation above we have not imposed any specic structure for the control law; however, one can distinguish (at least) the following variants of the tracking control problem depending on the measurements which are available: 1. (State feedback). Assume that the full state is available for measurement; then nd, if possible, a control law u of the form (a) (Static state feedback) u (t, x, xd), or (b) (Dynamic state feedback) u (t, x, xd, xc), x c ( t, x, xd, xc) with xc being the dynamic compensator state, say xc l.

2. (Output feedback). Assume that only an output z k(x) is measurable; then nd, if possible, a control law u of the form (a) (Static output feedback) u (t, z, z, xd),or (b) (Dynamic output feedback) u (t, z, xd, xc), x c (t, z, xd, xc) with, again, xc l being the dynamic compensator state.
521
as if they were the true ones, that is, u = K x (12)
follows from Eq. (9). To nd an answer to this queswhere x tion we write the overall controlled system plus observer:
x A BK = 0 x
BK A LC
x x
(13)
LINEAR TIME-INVARIANT SYSTEMS Before presenting some results on state and output feedback control of nonlinear systems we briey revisit some facts about linear systems theory. It is well known that the controllerobserver design problem for linear systems is, completely solved (10), but the static output feedback problem is only partially understood. Consider the linear time-invariant (LTI) forced system x = Ax + Bu y = Cx (6)
and calculate the roots of its characteristic polynomialthat is, the roots of p(s ) = det sI A + BK 0 BK sI A + LC
or, equivalently, the roots of p(s ) = det(sI A + BK ) det(sI A + LC ) That is, the characteristic polynomial of the overall system is the product of the characteristic polynomials of the observer [Eq. (11)] and the controlled system [Eq. (8)]. Thus one can design the controller and observer separately without caring if the true states are available or if the observer will be used in open or closed loop. This nice property is called the separation principle and, unfortunately, in general it is exclusive to linear systems. For this reason, we are obliged to explore new techniques to achieve output feedback control for nonlinear systems. FEEDBACK LINEARIZATION
where y m, x n, u k, and A, B, and C are matrices of suitable dimensions. Assume that we wish to achieve the tracking control for system (6), which in the linear case boils down to solving the tracking problem for xd 0, with corresponding yd 0 and ud 0. In order to do so we use the linear state-feedback controller u = Kx where K is chosen in a way that the closed-loop system x = (A BK )x (8) (7)
Consider a single input nonlinear system x = f ( x ) + g ( x )u (14)
is exponentially stable; that is, the matrix (A BK) must be Hurwitz. Indeed a necessary and sufcient condition for the matrix K to exist is the stabilizability of the system [Eq. (6)]. Now, as we have mentioned we are also interested in the output feedback control problem. To date, even for linear systems the tracking control problem using static output feedback is not yet completely solved, and we therefore look at the observer-design problem. For system [Eq. (6)] a linear observer is given by x = Ax + Bu + L(y y ) y = Cx (9) (10)
where x n, u , and f and g are smooth vector elds on n; that is, their derivatives exist and are continuous up to innite order. In this section we describe the feedback linearization approach for system [Eq. (14)] around an equilibrium point x0 n; that is, f (x0 0. The local feedback linearization problem for the system [Eq. (14)] is to ndif possiblea static state feedback law u = (x ) + (x )v, (x0 ) = 0 (15)
and a smooth, local coordinate transformation z = S(x ), S ( x0 ) = 0 R n (16)
where L is an output-injection matrix. Note that the estima dynamics has the form tion error x xx x = (A LC )x (11)
such that the closed-loop system [Eqs. (14)(15)] in the z coordinates is a controllable system and without loss of generality may be assumed to be in the Brunovski form:
It is clear that one can nd an L such that Eq. (11) be asymptotically stablethus x 0 as t if the pair (A, C) is detectable (10). At this point we have successfully designed an asymptotically stabilizing feedback and a asymptotically converging observer. The natural question which arises now is whether it is possible to use the state estimates in the feedback Eq. (7)
0 z1 . . d .= . dt . . 0 zn
1 .. . .. .
z1 0 . . . . . + . v . 0 1 . . 1 0 zn
(17)
522
The local linearization problem is solvable only under quite restrictive conditions on f and g. In order to see this we will derive a set of sufcient conditions for the solvability of the linearization problem. Assuming there exists a feedback (15) and a coordinate transformation (16) that linearize the closedloop system, we note the following. Let zi Si(x), i 1, . . ., n; then we have, using the rst equation of Eq. (17)
2. For any two vector elds X1, X2 in the set adfi g, i 0, . . ., n 2 we have
[X1 , X2 ](x ) =
n2 1=0
i (x )ad if g(x )
for certain functions 0, . . ., n2. Note that the theorem above gives necessary and sufcient conditions for local feedback linearizability. For global results, further conditions on f and g are required. In the following example of a exible joint pendulum the local solution extends to the full state space and thus become global. Example 1. Consider the model of a robot link with a exible joint (15,16)
z 1 =
S1 (x ) S1 (x ) (x ) x = (x ) f (x ) x x S1 (x ) + ( x ) g ( x )u = z 2 = S 2 ( x ) x
(18)
Dening Lf S(x) S(x)/ x f (x) as the directional or Lie derivative of function S(x) in the direction of f , we obtain from Eq. (18) that S2 (x ) = L f S1 (x ) 0 = Lg S1 (x ) (19) (20)
Iq 1 + mgl sin q1 + k (q1 q2 ) = 0 Jq 2 k (q1 q2 ) = u
(27)
In an analogous way we derive, using the ith equation of Eq. (17) Si+ 1 (x ) = L f Si (x ), 0 = Lg Si (x ), i = 1, . . . , n 1 i = 1, . . . , n 1 (21) (22)
If we introduce the Lie bracket of two vector elds g1 and g2 on n denoted by [g1, g2] as g g [g 1 , g 2 ] ( x ) = 2 ( x )g 1 ( x ) 1 ( x )g 2 ( x ) x x (23)
where q1 is the angle of the link, q2 is the angle of the motor shaft, and u is the torque applied to the motor shaft. The exibility is modeled via a torsional spring with constant k, m is the mass of the link, and l is the distance from the motor shaft to the center of mass of the link. J and I are the momenta of inertia of motor and link, respectively. With x (q1, q 1, q2, q 2) we obtain a system of the form Eq. (14) with
then it follows that the function S1(x) should satisfy S1(x0) 0 and the n 1 conditions Lg S1 (x ) = L[ f ,g] S1 (x ) = L[ f ,[ f ,g]] S1 (x ) = L[ f ,... ,[ f ,g]... ] S1 (x ) = 0 (24) In other words, if the local feedback linearization problem is solvable, then there exists a nontrivial function S1(x) that satises the n 1 partial differential equations [Eq. (24)]. The functions and in the feedback [Eq. (15)] are given as follows. Since z n = f ( x )S n ( x ) + L g S n ( x )u = v we obtain from Eq. (15) (x ) = (Lg Sn (x )) 1 L f Sn (x ), (x ) = (Lg Sn (x )) 1 (26) (25)
mgl sin x k (x x ) 1 1 3 I f (x ) = x4 k ( x1 x3 ) J 0 0 g (x ) = 0 1/J
x2
(28)
One may verify that conditions (1) and (2) for feedback linearization are fullled about the equilibrium x0 0. In order to nd the linearizing coordinate change [Eq. (16)] and feedback [Eq. (15)], we rst solve Eq. (24). Using Eq. (28), this set of partial differential equations reads as S1 (x ) S1 (x ) S1 (x ) = = =0 x2 x3 x4 having as a (nonunique) nontrivial solution z 1 = S 1 ( x ) = x1 which via Eq. (21) implies z 2 = S 2 ( x ) = L f S 1 ( x ) = x2 z 3 = S3 (x ) = L f S2 (x ) = z 4 = S4 (x ) = L f S3 (x ) = mgl k sin x1 (x1 x3 ) I I mgl k x cos x1 (x2 x4 ) I 2 I (30) (31) (32) (29)
In general one may not expect that a nontrivial function S1(x) exists that fullls Eq. (24). Writing the iterated Lie 1 brackets of the vector elds f and g as adfkg [f , adk f g], k 0 1, 2, . . ., with adf g g, one can derive the following necessary and sufcient conditions for the existence of S1(x) (see, for example, Refs. 13 and 14). Theorem 1. Consider the system [Eq. (14)] about an equilibrium point x0. The local feedback linearization problem is solvable about x0 if and only if 1. The vector elds adfigi 0, . . ., n 1 are linearly independent.
523
Finally using Eq. (26) we nd the linearizing feedback Eq. (15) as
(x ) = (Lg S4 (x )) 1 =
IJ k (x ) = (Lg S4 (x )) 1 L f S4 (x ) = IJ mgl mgl k sin x1 x2 cos x1 + 2 + k I I I k k mgl k + (x1 x3 ) + + cos x1 I I J J
In order to better understand the passivity concept we should think of a system like a black box which transforms some input into some output. More precisely, we say that a system with input u and output y denes a passive operator u y if the following energy balance equation is veried:
T T
H (T ) H (0 ) +
0
i u(t ) 2 dt +
o y(t ) 2 dt
0
dissipated
stored energy
=
0
u(t )y(t )dt

supplied
(35)
As a matter of fact, the above derivations are globally dened, and in addition Eqs. (29)(32) have a physical interpretation. In a similar way, the feedback linearization problem may be stated and solved for multivariable nonlinear systems but it is beyond the scope of this article to go further into this topic; interested readers are referred to Refs. 13 and 14. However, let us mention a simple example: the computed torque controller for rigid joint robot manipulators (EL systems), whose dynamical model is D ( q p )q p + C (q p , q p )q p + g (q p ) = u (33)
where qp n is the vector of link positions (generalized position coordinates), D(qp) D(qp) 0 is the inertia matrix, C(qp, q p) is the Coriolis and centrifugal forces matrix, g(qp) is the gravitational vector, and u n is the vector of control torques. The tracking control problem for system [Eq. (33)] is to make the link position qp follow a desired trajectory yd(t) qpd(t). The computed torque controller is a feedback linearization approach which, since it was proposed in Ref. 6, has become very popular. The control law is given by u = D ( q p )v + C ( q p , q p )q p + g (q p ) (34)
where H(T) is the total energy of the system at time instant T. Expressed in words, the energy balance equation (35) establishes that one cannot pull more energy out of a passive system than the energy which was fed in. To illustrate this simple idea, consider an ordinary RLC (with all elements con i is the current nected in series) network: In this case, y running through the resistor, while u v is the input voltage. T Hence, if we look at Eq. (35), the term 0 iu(t)2 dt corresponds to the (electrical) potential energy stored in the capaciT tor, while the term 0 oy(t)2 dt corresponds to the (electrical) potential energy dissipated in the resistor (considering R o). The energy stored in the inductance corresponds to the (magnetic) kinetic energy which has been considered in the terms H(T) H(0). The stored energy in the capacitor plus the term H(T) is called available storage and since H(0) 0, it satises
T T T
H (T ) +
0
i v(t )
dt <
0
v(t )i(t ) dt
0
o i(t )
dt
where v is a new input to be dened. It is easy to see that by substituting Eq. (34) in Eq. (33) we obtain the linear closed-loop dynamics q p v; then in order to solve the tracking problem for Eq. (33) we choose v Kp(qp qpd) Kd(q p q pd) q pd and we obtain that the closed loop is globally exponentially stable for any positive denite matrices of Kp and Kd. Note that in this simple example, the feedback linearizationwhich is globaldoes not require any change of coordinates like Eq. (17). On the other hand, one may verify that system [Eq. (33)] does fulll multivariable conditions similar to those given in Theorem 1 that guarantee the feedback linearizability. Except from several more recent modications of the computed torque controller (see Ref. 17 and references therein) the computed torque controller [Eq. (34)] requires full state feedback in order to obtain a linear closed-loop system; and in fact, output feedback will never lead to linear dynamics in closed loop. It is therefore attractive to investigate alternative tracking strategies. PASSIVITY-BASED CONTROL The Passivity Concept As we mentioned in the introduction, the physical interpretation of the passivity concept is related to the systems energy.
that is, we can recover less energy than what was fed to the circuit. Formally, the denition of passivity we will use is the following (4): Denition 1. Let T 0 be any. A system with input u n n L2 e and output y L 2e denes a passive operator : u y if there exists a such that
T
u(t )T y(t ) dt
0
(36)
The operator is output strictly passive (OSP) if moreover, there exists o 0 such that
T T
u(t )T y(t ) dt o
0 0
y(t )
dt +
(37)
Finally, is said to be input strictly passive (ISP) if there exists i 0 such that
T 0 T
u(t )T y(t ) dt i
u(t )
0
dt +
(38)
It should be noted that mainly every physical system has some passivity property; this has motivated researchers to use passivity-based control, that is, to exploit the passivity
524
properties of the plant in order to achieve the control task by preserving the passivity in closed loop. The literature on passive systems is very diverse. We will illustrate this technique on a class of passive systems, the EulerLagrange systems. It is worth remarking that the robot manipulators belong to this class. The Lagrangian Formulation EulerLagrange (EL) systems can be characterized by the EL parameters {T ( q, q , V (q ), F (q )} (39)
where the entries of the matrix C(q, q ) are called the Coriolis and centrifugal forces; the kjth entry is Ck j (q, q )
ci jk (q )q i
i
(46)
ci jk (q )
,1 2
Dik (q ) D jk (q ) Di j (q ) + q j qi qk
(47)
With these denitions of matrices D(q) and C(q, q ) the following properties hold: P1. The matrix D(q) is positive denite, and the matrix (q) 2C(q, q N(q, q ) D ) is skew symmetric, that is, N N. Moreover, there exist some positive constants dm and dM such that dm I < D (q ) < dM I (48)
where q n are the generalized coordinates and n corresponds to the number of degrees of freedom of the system. We focus our attention on fully actuated EL systemsthat is, systems for which there is a control input available for each generalized coordinate. Moreover, we assume that the kinetic energy function is of the form 1 T D ( q )q T ( q, q ) = q 2 (40)
P2. The matrix C(x, y) is bounded in x. Moreover, it is easy to see from Eq. (46) that C(x, y) is linear in y, then for all z n C(x, y )z = C(x, z )y C(x, y ) kc y , kc > 0 (49) (50)
where the inertia matrix D(q) satises D(q) D(q) 0. Next, V(q) represents the potential energy which is assumed to be bounded from below; that is, there exists a c such that V(q) c for all q n, and F (q ) q Rq with R R 0 is the Rayleighs dissipation function. EL systems are dened by the EL equations d dt L ( q, q ) q L ( q, q ) =Q q (41)
Furthermore we will focus our attention on those systems for which the following additional property on the potential energy holds: P3. There exists some positive constants kg and kv such that kg sup
where L (q, q T(q, q ) ) V(q) is the Lagrangian function. We assume that the external forces, Q n, are composed only of potential forces (derived from a time-invariant potential V(q)) u n and dissipative forces F (q )/ q , hence
q n
R R
2V ( q ) , q2 V (q ) , q
q R n q R n
(51) (52)
kv sup
Q=u
F (q ) q
q n
(42) It is well known (7) that EL systems have some nice energy dissipation properties: Proposition 1. (Passivity). An EL system denes a passive operator from the inputs u to the actuated generalized velocities q , with storage function, the total energy function. Moreover, it is output strictly passive if there is a suitable dissipation that is, if q (F (q )/ q ) oq 2 for some o 0. Below, we present other properties of EL systems which are related to the stability in the sense of Lyapunov. For the sake of clarity, we distinguish two classes of EL systems, fully damped and underdamped systems. The proposition below establishes conditions for internal stability of fully damped EL systems. After Joseph L. La Grange, the equilibria of a mechanical system correspond to the minima of the potential energy function (see Ref. 19 for a denition). Inspired by this well-known fact, we can further establish the following: Proposition 2. (GAS with full damping). The equilibria of a fully damped free EL system (i.e., with u 0) are (q, q )
At this point, we nd it convenient to partition the vector q as q col[qp qc] where we call qp the undamped coordinates and call qc the damped ones. With this notation we can distinguish two classes of systems: An EL system with parameters [Eq. (39)] is said to be a fully damped EL system if ( 0) q T F (q ) q q
2
(43)
An EL system is underdamped if q T F (q ) q c q
2
(44)
It is also well known (18) that the Lagrangian equations [Eq. (41)] can be written in the equivalent form [note that Eq. (45) is exactly the same as Eq. (33) with Raleigh dissipation zero] D ( q )q + C ( q, q )q + g (q ) + F (q ) =u q (45)
525
(q, 0), where q is the solution of V (q ) =0 q (53)
ties of Lagrangian systems described in previous sections, it becomes natural to consider El controllers (22) with generalized coordinates qc nc and EL parameters Tc(qc, q c), Vc(qc, qp), F c(q c). That is, the controller is a Lagrangian system with dynamics
The equilibrium is unique and stable if it is a global and unique minimum of the potential energy function V(q) and V is proper (4). Furthermore, this equilibrium is globally asymptotically (GAS) stable if the map dened by the Rayleigh dissipation function is input strictly passive. As far as we know, the rst article which establishes sufcient conditions for asymptotic stability of underdamped Lagrangian systems is more than 35 years old (20). In the proposition below we show that global asymptotic stability of a unique equilibrium point can still be ensured even when energy is not dissipated in all directions, provided that the inertia matrix D(q) has a certain block diagonal structure and the dissipation is suitably propagated. Proposition 3. (GAS with partial damping). The equilibrium (q , q) (0, q) of a free (u 0) underdamped EL system is GAS if the potential energy function is proper and has a global and unique minimum at q q, and if
Tc (qc , q c ) Vc (qc , q p ) Fc (q c ) c ( q c )q D c ( q c )q c + D c + + =0 qc qc q c (54)

Note that the potential energy of the controller depends on the measurable output qp, and therefore qp enters into the controller via the term Vc(qc, qp)/ qc. On the other hand, the feedback interconnection between plant and controller is established by u= Vc (qc , q p ) qp (55)
then the closed-loop system is Lagrangian and its behavior is characterized by EL parameters T(q, q ), V(q), F (q ), where
T ( q, q )
1. D(q ) 2. q T
D p (q p ) 0
2
0 , where Dc (qc ) Rn c n c . Dc ( q c ) for some > 0. V (q ) = 0 has only isolated qc
, Tp (q p , q p ) + Tc (qc , q c ), V (q ) , Vp (q p ) + Vc (qc , q p ), p ) + Fc (q c ) F (q ) , F p (q
F (q ) q c q
3. For each qc , the function zeros in q p .
Condition (2) establishes that enough damping is present in the coordinates qc while the other two conditions help to guarantee that the energy dissipation suitably propagates from the damped coordinates to the undamped ones. Hence, one can think of an underdamped EL system as the interconnection of two EL systems. As a matter of fact the feedback interconnection of two EL systems yields an EL system. Output-Feedback Set-Point Control In this section we illustrate the passivity-based control approach by addressing the position-feedback set-point control problem of EL systems. The results we will present are based on the so-called energy shaping plus damping injection methodology. Launched in the seminal paper (21), this methodology aims at shaping the potential energy of the plant via a passive controller in such a way that the new energy function has a global and unique minimum at the desired equilibrium. It is worth remarking that this methodology was originally proposed in the context of robot control; however, it has been proved useful in the solution of other control problems as it will be clear from this section. Also, it shall be noticed that the passivity property of robot manipulators was rst pointed out in Ref. 7. Motivated by the energy shaping plus damping injection technique of Takegaki and Arimoto, as well as by the proper-
The resulting feedback system is the feedback interconnection of the operator p : up qp, dened by the dynamic equation [Eq. (33)] and the operator c : qp up, dened by Eqs. (54) and (55). Note that the EL closed-loop system is damped only through the controller coordinates qc. From the results presented in section entitled The Lagrangian Formulation we see that to attain the GAS objective, V(q) must have a global and unique minimum at the desired equilibrium, q qd, and F (q ) must satisfy Eq. (44). These conditions are summarized in the proposition below whose proof follows trivially from Proposition 3. Proposition 4. (Output feedback stabilization) (22). Consider an EL plant (33) where u m, m n, with EL parameters Tp(qp, q p), Vp(qp), F p(q p). An EL controller (54), (55) with EL parameters Tc(qc, q c), Vc(qc, qp), F c(q c), where q T c Fc (q c ) q c q c
2
for some 0, solves the global output feedback stabilization problem if 1. (Dissipation propagation) For each trajectory such that qc const and Vc(qc, qp)/ qc 0, we have that qp const. 2. (Energy shaping) V(q)/ q 0 admits a constant solution q such that qpd [Inp 0]q, and q q is a global and unique minimum of V(q), and V is proper. For instance, this is the case if 2V(q)/ q2 In 0, 0 q n.
526
A simple example of EL controllers is the dirty derivatives lter, widely used in practical applications: q c = A(qc + Bq p ) = (qc + Bq p ) (56) (57)
where A, B are diagonal positive denite matrices. With an obvious abuse of notation this system lies in the EL class and has the EL parameters:
stability. As far as we know, the rst nonlinear PID controller is due to Ref. 24 [even though Kelly (24) presented his result as an adaptive controller, in the sequel it will become clear why we use the PID qualier] which was inspired upon the results of Tomei (25). In order to motivate the nonlinear PID of Ref. 24, let us rst treat in more detail the PD-like adaptive control law of Tomei p + u = K p q p Kd q together with the update law ( q p ) (60)
Tc (qc , q c ) = 0, F c ( q c ) = Vc (qc , q p ) =
1 T 1 1 q B A q c 2 c
1 (qc + Bq p )T B 1 (qc + Bq p ) 2
= ( q p )T q p +
2q p 1+2 q p
2
(61)
The controller above injects the necessary damping to achieve asymptotic stability and its action has the following nice passivity interpretation. First, we recall that the EL plant Eq. (41) denes a passive operator u q p. On the other hand, the controller Eq. (54) denes a passive operator q p Vc(qp, qc)/ qp. These properties follow, of course, from the passivity of EL systems established in Proposition 1. It sufces then to choose u as in Eq. (55) to render the closed-loop passive.
CONTROL UNDER MODEL AND PARAMETER UNCERTAINTIES In all the results presented above, we assumed that we had accurate knowledge about the systems model and its parameters; however, this rarely happens to be the case in practical applications. It is of interest then to use techniques such as robust and/or PID control. PID Control PID control was originally formulated by Nicholas Minorsky in 1922; since then it has become one of the most applied control techniques in practical applications. In the western literature, the rst theoretical stability proof of a PID in closed loop with a robot manipulator is due to Ref. 23. We reformulate below the original contribution of the authors. Proposition 5. Consider the dynamic model [Eq. (33)] in closed loop with the PID control law u = KP q p KD q p + = KI q p (0 ) = 0 Rn (58) (59)
where is a suitably dened positive constant. Tomei (25) proved that under this adaptive control law, the position error is globally asymptotically convergent. The normalization used in Eq. (61), probably rst introduced by Koditschek (26), helps in guaranteeing the globality, in the un-normalized case being only semiglobally convergent. Note that the result of Tomei is based on a PD plus gravity cancellation; since the gravity vector is not well known, an adaptive update law must be used. Let us consider that instead of cancelling the gravity term, we compensate it at the desired position, then we will be aiming at estimating the con . More precisely, consider the control law stant vector (qpd) (24) p + u = KP q p KD q together with the update law 1 q p = = (q pd )T q p + 1+ q p (63) (q pd ) (62)
where 0 is a small constant. Kelly (24) proved that this adaptive controller in closed loop with Eq. (33) results in a globally convergent system. As a matter of fact, since the regressor vector (qpd) is constant, the update law [Eq. (63)], together with the control input [Eq. (62)], can be implemented as a nonlinear PID controller by integrating out the velocities vector from Eq. (63): = 1
t
(q pd )T q p +
q p (0 ) dt + 1+ q p
Then, if KP kgI and KI is sufciently small, the closed loop is asymptotically stable. The proposition above establishes only local asymptotic stability. By looking at the proof (see Ref. 23) of the above result we see that what hampers the global asymptotic stability is the quadratic terms in q p contained in the Coriolis matrix. This motivates us to wonder about the potential of a linear controller designed for a nonlinear plant. As a matter of fact, with some smart modications one can design nonlinear PIDs which guarantee global asymptotic
Note that the choice KP KP KI, with KI 1/ (qpd) (qpd), yields the controller implementation u = KP q p KD q p + = KI q p , 1+ q p (0 ) = 0 Rn (64) (65)
Following Ref. 24, one can prove global asymptotic stability of the closed-loop system [Eqs. (33), (64), (65)]. An alternative trick to achieve GAS is the scheme of Arimoto (27), who proposed the following nonlinear PID:
527
Proposition 6. Consider the dynamic model [Eq. (33)] in closed loop with the PID control law u = KP q p KD q p + = KI sat(q p ), (0 ) = 0 R
n
(66) (67)
Then, if KP kgI and if KI is sufciently small, the closed loop is asymptotically stable. It is clear from the proof (see Ref. 27) that the use of a saturation function in the integrator helps to render the system globally asymptotically stable, just as the normalization did in Tomeis and Kellys schemes. In the sequel we assume that velocities are not available for measurement. In general it is a difcult problem to design adaptive output feedback controllers for nonlinear systems and achieve global convergence. The result we present below is inspired by the work of Kelly and the key observation that when compensating with the (unknown) gravity vector evaluated at the desired position, one can simply integrate the velocities out of the update law. Proposition 7. Consider the dynamic model of the EL plant Eq. (33) in closed loop with the PI2D control law u = KP q p KD + = KI (q p ), q c = A(qc + Bq p ) = qc + Bq p (0 ) = 0 Rn (68) (69) (70) (71)
also qp besides q p, at the output. This can be accomplished with a storage function that includes cross terms. Very recently, Ref. 27 showed, by using a saturation function sat( ), that the nonlinear PID (66) can be regarded as the feedback interconnection of two passive operators 1 : z sat(q p) q p) q p and 2 : sat(q p z; hence the closed loop system is also passive. The same can be proven for the normalized scheme of Kelly [Eqs. (64) and (65)]. As a matter of fact it can be proven that 1 is OSP (17). Instrumental in proving OSP for 1 is the use of either a normalization or a saturation function; unfortunately in the case of the PI2D controller, these tricks do not lead us to OSP actually, and the output strict passiability property we can establish is only local. That is, the property holds only for inputs that restrict the operators output to remain within a compact subset (29). Nonetheless, this compact subset can be arbitrarily enlarged with high gains, and this explains the semiglobalinstead of globalnature of this result. Robust Control We have assumed so far that even though the plants model is known, some uncertainties over the parameters exist. Nevertheless, in some applications it may happen that we have only a very rough idea of the plants model. For instance, some lightweight robot manipulators with direct-drive motors present highly nonlinear and coupled dynamics for which a model is not known. It is good to know that, at least for a certain class of EL plants, still a high-gain PD control can be used, leading to some robustness satisfactory results. In particular, for the EL plant model we have D ( q p )q p + C (q p , q p )q p + g (q p ) + F (q p) + T = u (74)
Let KP, KI, KD, A diagai, and B diagbi be positive denite diagonal matrices with
4d M B> I dm KP > (4kg + 1 )I
(72) (73)
where F(q p) is the vector of frictional torques which satises F(q p) kf1 kf2 q p for all q p and T is the vector of load disturbances which is bounded as T kt; we have the following result (30): Proposition 8. Consider the EL plant [Eq. (74)], let e qp qpd be the position error, and let e an estimate of it. Consider the control law u = Kd e Kp e ) e = w + Ld (e e w = L p (e e ) (75) (76) (77)
col[q where kg is dened by Eq. (49), and dene x p, q p, , g(qpd)]. Then, given any (possibly arbitrarily large) initial condition x(0), there exist controller gains that ensure limt x(t) 0.
In Ref. 28, precise bounds for the controller gains are given, depending on bounds on the plants parameters. Note that the PI2D controller is linear and, as in the case of a conventional PID scheme, it only establishes semiglobal asymptotic stability. The technical limitation is the high nonlinearities in the Coriolis matrix. See Ref. 28 for details. In the sequel we give an interpretation of the nonlinear PID controllers above, as well as the PI2D scheme, from the passivity point of view, or more precisely passiabilitythat is, the possibility of rendering a system passive via feedback. From Proposition 1 we know that the plants total energy function T(qp, q p) V(qp) qualies as a storage function for the supply rate w(u, q ) uq p. From this property, output strict passiability of the map u q p follows taking u KDq p u1, with u1 an input which shapes the potential energy. Other applications, including the present study of PI controllers, require a passiability property of a map including
Then for any set of bounded initial conditions (t0, x(t0)) we can always nd sufciently large gains Kp, Kd, Lp, and Ld such that the trivial solution of the closed-loop system: x(t) col[e, e , e 0 is uniformly ultimately bounded. That is, for , e every bounded initial conditions (t0; x(t0)) there exist a nite constant 0 and a time instant t1(, x(t0)) such that x(t ) , t t0 + t1
Moreover, in the limit case, when Lp, Ld the origin is asymptotically stable. A particular case of the result above is presented in Ref. 31 where velocity measurements are used. Like in Ref. 31 the
528
uniform ultimate boundedness result of Proposition 8 is of local nature because the controller gains depend on the initial conditions x(t0); nevertheless, it is important to remark that the Proposition states that for any set of nite initial conditions there always exist control gains . . .; that is, the result is semiglobal. However, even without the knowledge of bounds over the plants parameters, the closed loop system can be made uniformly ultimately bounded by selecting the control gains sufciently large. Hence there is no need to quantify these bounds a priori. It is also important to remark that the linear control scheme [Eq. (75)(77)] allows quick response in an online implementation, due to its simplicity. Since this control scheme completely ignores the system dynamics, however, the conditions of Proposition 8 (see [30] for details) may require that Kd and Ld be large to obtain an acceptable tracking performance. Such high gain implementations are not always desirable in practical applications. For this reason it may be protable to add model-based compensation terms to the control input, when available. See, for instance, Ref. 17 and references therein. THIRD-ORDER FEEDBACK LINEARIZABLE SYSTEMS So far we considered n coupled second-order (fully actuated EL) systems to illustrate different control design methods for nonlinear systems. Even though the class of EL plants includes a large number of physical systems it is interesting to investigate output feedback control of higher-order systems. As a matter of fact, the topic of output feedback control of nonlinear systems is one of the most studied in the literature and very few particular results exist guaranteeing global asymptotic stability; see, for example, a recent study of the semiglobal problem (11). In this section we illustrate this problem by addressing the partial state feedback control of a complex system, the socalled Ro ssler system (see, for instance, Ref. 32) x 1 = ( x2 + x3 ) x 2 = x1 + ax2 x 3 = x3 ( x1 b ) + c + u (78) (79) (80)
in closed loop with Eq. (78) is GAS. Now, let us suppose that only x2 and x3 are measurable. In this particular case, a nonlinear observer for x1 can be easily designed using the Lyapunov control approach (4). Motivated by the control law Eq. 1): (81), consider the control law u u(x2, x3, x u = x3 (x 1 b ) c k1 x 1 k 2 x2 k 3 x3 1 is the estimate of x1. Consider the function where x V (x, x 1 ) = 1 2 2 x + x2 2 2 + x3 + x 1 + x2 ( x3 x1 ) 2 1 (83) (82)
1 is the estimation error and 0 is sufwhere x 1 x1 x ciently small to ensure that V(x) is positive denite and radially unbounded. Then, let V(x) be a Lyapunov function candidate for system Eq. (78) in closed loop with the control law Eq. (82). We proceed now with the design of a reduced ob 1 f (x2, x3, x 1). First, evaluating the time derivative server x 1) along the trajectories of the closed loop Eqs. (78), of V(x, x (82) we get after some bounding
2 2 (x, x V 1 ) 1 x2 1 1 + x 1 x 1 2 x2 2 x3 + (x3 + x2 )(k1 + x3 )x
where 1, 2, and 3 are positive constants. Hence, by setting x 1 = (x3 + x2 )(k1 + x3 ) the Lyapunov function becomes negative semidenite. A simple analysis using the KrasovskiiLaSalles invariance principle shows that the closed-loop system is GAS. Note, moreover, that the observer can be implemented as x 1 = (x3 + x2 )(k1 + x3 ) (x2 + x3 ) without measurement of x1. In the case when more variables are unmeasurable, one may think that a similar procedure leads to the design of an observer for the unavailable variables. Unfortunately, this seems not the case when only x2 or x3 are considered available for measurement. Moreover, the lack of a physical interpretation for the Ro ssler system makes this task more difcult. The lesson one can take from this illustrative example is that observer-based schemes become complicated even if the system itself is feedback-linearizable. The lack of (physical) passivity properties hampers the use of passivity-based control. CONCLUSION We have briey illustrated different control design methods for nonlinear systems. We derived necessary and sufcient conditions to solve the local feedback linearization problem and illustrated this approach on the exible joints robots case. We focused our attention into a special class of second-order systems, the EL systems. However, the Lagrangian formulation applies to all elds in which variational principles can be used to model the plant in question; hence this class includes a wide number of physical systems such as robot manipulators. We saw that the EL class has some nice energy properties which can be exploited by using passivity-based control. The goal of this methodology is to design a controller and a dy-
where a, b, and c are positive constants. It can be seen from simulations that the trajectories of this system have a chaotic behavior, for instance if a 0.2, b 5, and c 0.2. A behavior is chaotic if it has a sensitive dependence on initial conditions. By this, we mean that the difference between two solutions of a differential equation with a slight difference in the initial conditions grows exponentially (32). The motivation to consider the Ro ssler system is to investigate to what extent the techniques used for second-order systems can be successfully used for third-order feedback linearizable systems. Note for the Ro ssler system that if the whole state is supposed to be measured, then it is easy to see that there exist positive gains k1, k2, and k3 such that the feedback linearizing control law u = x3 (x1 b ) c k1 x1 k2 x2 k3 x3 (81)
NONLINEAR DYNAMIC PHENOMENA IN CIRCUITS
529
namic extension to the plant which renders the closed-loop system passive. This approach appeared very useful in solving the set-point control problem. ACKNOWLEDGMENTS This work was partially carried out while the rst author was with the Department of Applied Mathematics of the University of Twente. His work has been partially sponsored by the project European Capital and Human Mobility, under grant no. ERB CHRX CT 93-0380. BIBLIOGRAPHY
1. IEEE Control Systems Magazine, Special issue on History of Control, June 1996. 2. J. C. Willems, Dissipative dynamical systems. Part I: General theory, Arch. Rat. Mech. Anal., 45 (5): 321351, 1972. ` me de la stabilite de mouvement, Ann. 3. A. M. Lyapunov, Proble Fac. Sci. de Toulouse, 9: 203474, 1907. (Translation from the original article published in Commun. Soc. Math., Kharkov, 1893, reprinted in Ann. Math. Studies, 17: Princeton 1949). See also Stability of Motion, New York: Academic Press: 1996. 4. M. Vidyasagar, Nonlinear Systems Analysis, Englewood Cliffs, NJ: Prentice-Hall, 1993. 5. H. Khalil, Nonlinear Systems, New York: Macmillan, 1992. 6. A. K. Bejczy, Robot arm dynamics and control, Technical Report TM 33-669, California Institute of Technology, Jet Propulsion Laboratory, 1974. 7. R. Ortega and M. Spong, Adaptive motion control of rigid robots: A tutorial, Automatica, 25-6: 877888, 1989. 8. J. T. Wen and S. D. Bayard, New class of control laws for robot manipulators, Parts I and II, Int. J. Control, 47-5: 12881310, 1988. 9. H. Berghuis and H. Nijmeijer, Global regulation of robots using only position measurements, Syst. Contr. Lett., 21: 289293, 1993. 10. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1980. 11. A. Teel and L. Praly, Tools for semiglobal stabilization by partial state and output feedback, SIAM J. Control Opt., 33 (5): 1443 1488, 1995. 12. P. Rouchon et al., Flatness, motion planning and trailer systems. In Proceedings of the 32nd IEEE Conference on Decision Control (San Antonio, TX), 1993, pp. 27002705. 13. H. Nijmeijer and A. J. van der Schaft, Nonlinear Dynamical Control Systems, New York: Springer-Verlag, 1990. 14. A. Isidori, Nonlinear Control Systems. Berlin: Springer-Verlag, 3rd ed., 1995. 15. R. Marino and S. Nicosia, On the feedback control of industrial robots with elastic joints: a singular perturbation approach, Tech di Roma Tor nical Report 84.01, Dip. Ing. Elettronica, Universita Vergata, June 1984. 16. R. Marino and S. Nicosia, On the feedback control of industrial robots with elastic joints: a singular perturbation approach. In First IFAC Symposium on Robot Control, (Barcelona, Espana), 1985, pp. 1116. 17. A. Loria, On output feedback control of Euler-Lagrange systems, ` gne, PhD thesis, University of Technology of Compiegne, Compie France, October 1996. Available on http://www-ccec.ece.ucsb.edu/ people/aloria. 18. M. Spong and M. Vidyasagar, Robot Dynamics and Control, New York: Wiley, 1989.
19. E. Marsden and A. J. Tromba, Vector Calculus. New York: W. H. Freeman, 1988. 20. G. K. Pozharitskii, On asymptotic stability of equilibria and stationary motions of mechanical systems with partial dissipation, Prikl. Mat. i Mekh., 25: 657667, 1961. English translation in J. Appl. Math. Mech., 25: 1962. 21. M. Takegaki and S. Arimoto, A new feedback method for dynamic control of manipulators, ASME J. Dyn. Syst. Meas. Control, 103: 119125, 1981. 22. R. Ortega et al., On passivity-based output feedback global stabilization of EulerLagrange systems, Int. J. Robust and Nonlinear Control, special issue on Control of Nonlinear Mechanical Systems, 5 (4): 313325, 1995 (H. Nijmeijer and A. J. van der Schaft, eds.). 23. S. Arimoto and F. Miyazaki, Stability and robustness of PD feedback control with gravity compensation for robot manipulator. In F. W. Paul and D. Yacef-Toumi (eds.), Robotics: Theory and Applications DSC, Vol. 3, 1986, pp. 6772. 24. R. Kelly, Comments on: Adaptive PD control of robot manipulators, IEEE Trans. Robotics Automat., 9 (1): 117119, 1993. 25. P. Tomei, Adaptive PD control for robot manipulators, IEEE Trans. Robotics Automat., 7 (4): 565570, 1991. 26. D. E. Koditschek, Application of a new Lyapunov function to global adaptive attitude tracking. In Proceedings of the 27th IEEE Conference on Decision Control (Austin, TX), 1988. 27. S. Arimoto, A class of quasi-natural potentials and hyper-stable PID servo-loops for nonlinear robotic systems, Trans. Soc. Instrument Control Eng., 30 (9): 10051012, 1994. 28. R. Ortega, A. Loria, and R. Kelly, A semiglobally stable output feedback PI2D regulator for robot manipulators, IEEE Trans. Automat. Control, 40 (8): 14321436, 1995. 29. H. Pota and P. Moylan, Stability of locally dissipative interconnected systems, IEEE Trans. Automat. Control, AC-38 (2): 308 312, 1993. 30. H. Berghuis and H. Nijmeijer, Robust control of robots via linear estimated feedback, IEEE Trans. Automat. Control, 39 (10): 21592162, 1994. 31. Z. Qu and J. F. Dorsey, Robust tracking control of robots by a linear feedback law, IEEE Trans. Automat. Control, 36: 1081 1084, 1991. 32. T. Kapitaniak, Controlling Chaos, New York: Academic Press, 1996.
ANTONIO LORIA
University of California at Santa Barbara
HENK NIJMEIJER
University of Twente
NONLINEAR DISTORTION MEASUREMENT. See INTERMODULATION MEASUREMENT.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Nyquist Criterion, Diagrams, and Stability Standard Article Halit Eren1 and Bert Wei Juet Wong1 1Curtin University of Technology, Bentley, Australia Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1026 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (222K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Encirclements and Enclosures Stability Conditions The Principal Argument Simplified Nyquist Criteria Nyquist Diagrams Effects of Time Delays Nyquist Stability Criterion for Digital Systems The Inverse Nyquist and Nyquist Plot for Multivariable Systems About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
NYQUIST CRITERION, DIAGRAMS, AND STABILITY

Table 1. List of Manufacturers
AAI Corporation P.O. Box 126 Hunt Valley, MD 21030-0126 Tel: 800-655-2616 Fax: 410-628-8616 Cape Software, Inc. 333-T N. Sam Houston Parkway, E. Houston, TX 77060-2403 Tel: 281-448-5177 Fax: 281-448-2607 Design Technology Corporation 5 Suburban Park Drive Billerica (Boston), MA 01821 Tel: 508-663-7000 Fax: 508-663-6841 MicroMath Research Ltd. P.O. Box 71550 Salt Lake City, UT 84171-0550 Tel: 801-483-2949 Fax: 801-483-3025 Mitech Corporation 43-T Nagog Park Acton, MA 01720 Tel: 978-263-7999 Fax: 978-263-8081
19
Murphy Software Company 1000 Town Center, Suite 1950 Southeld, MI 48075 Tel: 248-351-0900 Fax: 248-351-0906 ProModel Corporation 1875-T S. State Street, Suite 3400 Orem, UT 84097-8075 Tel: 801-223-4600 Fax: 801-226-6046 Scientic and Engineering Software 4301-T Westbank Drive Austin, TX 78746-6546 Tel: 512-328-5544 Fax: 512-327-6646 Wolfram Research, Inc. 100-T Trade Center Drive, Suite 600 Champaign, IL 61820-4853 Tel: 217-398-0700 Fax: 217-398-0747

H. Nyquist (18891976), born in Sweden, is known for his contributions to telephone transmission problems in the 1920s. He is also well known for his contributions in the stability of feedback systems. In 1927 H. S. Black invented the negative feedback amplier. Part of the output signal was returned to the ampliers input to reduce the total system gain. This resulted in a atter frequency response, a wider bandwidth, and a decrease in the nonlinear distortion, since improperly designed ampliers were unstable, producing undesired results. In the late 1920s and early 1930s, H. Nyquist and H. W. Bode, a colleague at the Bell Telephone Laboratories, developed a mathematical analysis for feedback ampliers. Later developments evolved into sinusoidal frequency analysis and design techniques applied to feedback control systems. Although, at that time, the stability criteria for vibrating mechanical systems already existed and had been applied to feedback systems, the idea of algebraic problems on complex roots of polynomials were just arising. The Nyquist criterion offered geometrical solutions and was much easier to apply to ampliers. Later, the Nyquist criterion was used to provide vital information on stability essential in the analysis and design of
general control systems. It is a graphical method that relates the stability of a closed-loop system to the open-loop frequency response and the locations of poles and zeros. The Nyquist diagram method was found to be very useful in the design of linear feedback systems of all types. An important application in World War II was the feedback control of direction of guns that employed electromechanical feedbackcontrolled servomechanisms. Before computers became widespread, Nyquist diagrams were largely obtained by calculations and hand-drawn graphics. But, today many companies offer a diverse range of computer software for simulation, analysis and design of control problems. Some of these manufacturers are listed in Table 1. Popular software such as MATLAB, MATHCAD, and SIMULINK include control system tools. In design and stability analysis, the Nyquist method exhibits distinct features. It provides the same information on the stability of a control system, as does the RouthHurwitz criterion. In addition, the Nyquist method indicates the relative stability or instability of the system. Properties of the frequency-domain plots of the loop transfer function G(s)H(s) provide an alternative approach to root locus, and give information on the frequency-domain characteristics of the closedloop system. One other major feature of the Nyquist method is that it relates a closed-loop system to the transient response and steady-state errors. ENCIRCLEMENTS AND ENCLOSURES Before embarking into the Nyquist stability criterion, the concepts of encirclement and enclosures need to be established. These concepts are essential in the interpretation of the Nyquist plots and stability analysis.
20
j s-plane s1 C A B 1
Figure 1. Encirclements and enclosures of points by contours.
By taking the absolute value of both sides of Eq. (1) yields |c(t )| =
0
r(t ) g( ) d
(2)
s1
or |c(t )|
0
|r(t )|| g( )| d
(3)
Let the input r(t) be bounded by a nite positive number R such that |r(t )| R Thus, Eq. (2) becomes |c(t )| R
0
In a complex plane, a point is said to be encircled by a closed path if it lies inside the path. If the closed path has a direction assigned for being in a clockwise or counterclockwise direction, then the region inside the path is considered to be encircled in that prescribed direction. A point or a region is said to be enclosed by a path if it is encircled in the counterclockwise direction. Alternatively, the point or region is enclosed if it lies on the left of the path when the path is traversed in any prescribed direction. The second denition is more useful in situations where only some portion of the closed path is drawn. As an example, consider the two cases illustrated in Fig. 1. In accordance with the foregoing denitions, the contour 1 encircles the point A twice. The point B is encircled only once, whereas points C and D are not encircled at all by 1. As the contour 1 traverses in the counterclockwise direction, the points A and B lie on the left side; therefore, they are both enclosed, and the points C and D are not enclosed. The contour 2 encircles only the point D in the clockwise direction. Interestingly enough, the contour 2 does not enclose the point D, but it encloses other points A, B, and C. A path can encircle a point N number of times, and the magnitude of N can be determined by drawing a phasor from the point to an arbitrary point s1 along the path , as illustrated for point A in the contour 1. The point s1 is traversed along until it returns to its starting position. The net number of revolutions traversed by the phasor is N and the total angle traced is 2N radians. STABILITY CONDITIONS The stability of linear time-invariant systems depends upon the roots of the characteristic equation on the s-plane. In contrast to root locus plots, the Nyquist criterion does not give the exact locations of the roots of the characteristic equation but indicates the locations of the roots with respect to the left or the right half of the s-plane. In many control systems, the relations of the bounded-inputs to the bounded-outputs (BIBO) dene the stability. The denition states that if bounded inputs yield to bounded outputs, then the system is considered stable. The BIBO relationship can also be related to the roots of the characteristic equation as shown next. Lets take a linear time-invariant system, as illustrated in Fig. 2, with r(t) as the input, c(t) as the output, and g(t) as the impulse response. The convolution integral relating the function r(t), c(t), and g(t) is given by c(t ) =
0
(4)
| g( )| d
(5)
If c(t) is also to be bounded by a positive nite number C, with |c(t )| C < then
(6)
R
0
| g( )| d C <
(7)
Dividing Eq. (7) through by R and letting C / R equal to Q, a positive nite number, results in
0
| g( )| d Q <
(8)
For Eq. (8) to hold, the integral of the absolute value of g(t) must be nite. The Laplace transform may be used to show the relationship between the roots of the characteristic equation and Eq. (8). For g(t) L[ g(t )] =
0
g(t )e st dt = G(s )
(9)
Where s is a complex number having real and imaginary components on the s-plane as s j. Taking the absolute value on both sides of the equation yields |G(s )| =
0
g(t )e st dt
0
|g(t ) e st | dt
(10)
But since |e st | = |e t | (11)
r(t)
g(t)
c(t)
r(t )g( ) d
(1)
Figure 2. Block diagram representation of a linear time invariant system. The relation between input and output is given by the impulse response transfer function g(t). The g(t) may be a simple gain or a complex function involving derivatives and integrals.
21
substituting Eq. (11) into Eq. (10) gives

0
Table 2. Calculation of Points for the Illustration of Mapping for Contours in Fig. 3
| g(t ) e t | dt
(12)
1 F (s) 2 F (s) 3 F (s) 4 F (s)
Note that the imaginary part, j, of the complex variable s does not bear any importance in the proof leading to the BIBO stability. All that is needed is the mathematical relation between real parts of the poles in the s-plane, that is whether they lie in the right half or the left half of the complex plane. If one or more roots of the characteristic equation lies in the right half of the s-plane, or on the j-axis if is greater than or equal to zero, then |e t | R = 1 Substitution of Eq. (13) into Eq. (12) yields
0
s1j 4 7 j 5 5 s5j 5 j4 s 3 j 4 3 j 25 25 s 7 3j 4 7 j 3 3
s1j 7 4 j 5 5 s5j 5 j4 s 3 j 3 4 j 25 25 s 7 3j 7 4 j 3 3
s 1 j 7 4 j 13 13 s3j 3 j4 s 5 j 5 4 j 41 41 s 7 3j 21 12 j 65 65
s 1 j 7 4 j 13 13 s3j 3 j4 s 5 j 5 4 j 41 41 s 7 3j 21 12 j 65 65
(13) trarily chosen continuous closed path s in the s-plane, then the values of s in s maps a new closed continuous path F on the complex F(s)-plane. Some examples of mapping are presented in Table 2 and illustrated in Fig. 3. In Fig. 3, it can be seen that for every closed contour on the s-plane there is a corresponding contour on the F(s)-plane. If the contour s traverses in a selected direction say counterclockwise, Fig. 3 shows that the direction of traverse of F can either be in the same or opposite direction depending on the number of poles and zeros of function F(s) located on contour s. Lets express F(s) in the following familiar form as seen in the control theory
Z
R| g(t )| dt =
0
| g(t )| dt
(14)
Note that Eq. (14) does not satisfy the BIBO relation because the equation is not bounded as in Eq. (8). Hence, to satisfy the BIBO stability, the roots of the characteristic equation or the poles of G(s) must lie on the left side of the j-axis. A system is classied to be marginally stable if the rst-order poles, poles of conjugate pairs, lie on the j-axis. However, multiple-order poles or repeating conjugate pairs of poles represent an unstable system. In addition, a system is classied as unstable if more than one pole exists at the origin. Another denition that is worth mentioning and that helps in the understanding of the Nyquist criterion is the steadystate error. Steady-state error is the difference between the input and the output as t for a given test input. The steady state errors are generally described for three main types of test inputs: the step, the ramp, and the parabolic. Often, control systems are subjected to these inputs to test their ability to give the required outputs. Usually, these test inputs are in the form of electrical signals that have dened waveforms. For example, the parabolic input has a constant second derivative, which represents acceleration with respect to accelerating targets. In general, the output of any system can be represented by the sum of the natural response and the forced response. In relation to the Nyquist stability criterion, a steady state error can be calculated from the closed-loop transfer function of the system M(s) or the loop transfer function G(s)H(s). THE PRINCIPAL ARGUMENT The Nyquist criterion is based on a theorem using the theory of complex variables, which leads to the principal argument. The principal argument may be presented in a number of ways. Here, two approaches will be presented. CASE 1. In this case, a rigorous mathematical approach may be employed by using theories of contours and mappings in complex planes. Assume that F(s) is a function of s and singlevalued, that is, for each point in the s-plane there exist a corresponding point, including innity, in the F(s)-plane, and the function consists of a nite number of poles and a nite number of zeros in the s-plane. Now, suppose that there is an arbi-
(s + zi ) F (s ) =
i= 1 P
(15)
(s + pk )
k=1
By using the Lucas formula, F(s)/ F(s) can be written as
F (s ) = F (s )
Z i= 1
P 1 1 (s + z i ) k = 1 (s + pk )
(16)
where F(s) is the rst derivative of F(s) with respect to s. To illustrate this important point, lets take an example F (s ) = Calculate F (s ) = s2 2s + 1 (s + 2 )2 (s + 3 )2 (18) s+1 (s + 2 )(s + 3 ) (17)
The ratio of F(s)/ F(s) can be found to be s2 2s + 1 F (s ) = F (s ) (s + 1 ))(s + 2 )(s + 3 ) Writing the partial fractions (19)
s2 2s + 1 A B C F (s ) = = + + F (s ) (s + 1 )(s + 2 )(s + 3 ) (s + 1 ) (s + 2 ) (s + 3 ) (20)
22

Im s-plane
3.00 2.00 1.00 0.
s4
the f (s)-plane. The theorem shows that if a complex function f (s) is analytical (differentiable) bounded in a region by a simple closed curve , then f (s ) ds = f (s ) ds = 0 (22)
s3
s1
s2
Re 1.00 2.00 3.00 7.00 F(s) = s + 4 s-4 5.00 3.00 1.00 1.00 3.00 5.00 7.00
However, consider a complex function f (s) 1/(s a) with a pole at a on the s-plane. Lets draw a unit circle centered at s a on the s-plane described by (t) a ej where the angle is 0 2k, and k is the number of encirclements of the point a by the unit circle. Then the integral becomes f (s ) ds = 1 ds = ln(s + a )| (s + a ) (23)
Im 8.00 F(s)-plane 6.00 4.00 F2
The right-hand side of this equation can be evaluated by substituting values of s(t) a ej yielding ln(s + a )| = ln(e j a + a )| = j | Substituting the values of as 0 2k gives (24)
2.00
F3 F1
F4 Re
f (s ) ds =
1 ds = 2 kj (s + a )
(25)
0.
2.00 4.00
6.00 8.00 7.00 5.00 3.001.00 1.00 3.00 5.00 7.00 9.00
On the contrast to Eq. (22) where f (s) is analytic in the splane, for f (s)) having a singular point in the contour in the s-plane, the resulting closed contour is no longer zero but equal to multiples of 2j. This indicates that the contour on the f (s)-plane containing all the values of s on the s-plane goes through k number of revolutions around the origin. The number of revolutions around the origin depends on the number of times the point a is encircled in the s-plane. The encirclement k now can be expressed as k= 1 2 j f (s ) ds = 1 2 j 1 ds (s + a ) (26)
Figure 3. Mapping of s-plane contours to F(s)-plane. If the addition of number of poles and zeros of F(s) in the s-plane is other than zero, the contour on the F(s)-plane encircles the origin. This is clearly illustrated by the corresponding contours of 2 and 3.
it can be proven that A 1, B 1, and C 1, thus giving 1 1 1 F (s ) = F (s ) (s + 1 ) (s + 2 ) (s + 3 ) (21)
If only one encirclement has taken place in the s-plane, k 1, the result of this integration equals 1, indicating one revolution around the origin. Previously, a unit circle was considered, but the theory can be generalized for any closed contour around the pole a. We know that a complex function expressed as a ratio, as in Eq. (15), can be expressed as Eq. (16). Now, lets write Eq. (16) as
and verifying the general Eq. (16). Equations (16) and (21) indicate that the zeros of F(s) appear as the denominators of the new equation with positive signs. The poles of F(s) still appear in the denominators, but they have negative signs. After having stated this important point, we can turn our attention to Cauchys theorem. Although Cauchys theorem may be of great mathematical interest, the intention of this article is not to discuss the intricacies involved in the theorem but rather to use it to illustrate relevant points in order to establish a solid understanding of Nyquist stability criteria. Cauchys theorem is based on the complex integrals, mapping of points and closed contours between the s-plane and
f (s ) =
F (s ) = F (s )
Z i= 1
P 1 1 (s + z i ) k = 1 (s + pk )
(27)
where f (s) can be viewed as the residues of F(s). Now, substitute this Eq. (27) into Eq. (25)
Z
f (s ) ds =
i= 1
1 ds (s + zi )
P k=1
1 ds (s + pk )
(28)
Making use of the similarities between Eq. (23) and Eq. (28) will yield the following conclusions. There will be Z number of revolutions around the origin, one for each zi for k 1.
23
Also, there will be P number of revolutions around the origin as a result of pole pk, but this time they will be in the opposite direction, which is indicated by the negative sign of Eq. (28). From Eq. (26), the following may be written 1 2 j f (s ) ds = Z P (29)
R(s) +
E(s)
G(s)
C(s)
H(s)
This equation indicates that the number of times the origin is encircled depends on the difference between the zeros and poles, which are located on the contour of the s-plane. There are many other approaches to arrive at similar conclusions, and further information may be found in the bibliography at the end of this article. This illustrative approach claries for readers many points, which otherwise might have been difcult to understand without intensive mathematical knowledge. As a result of these explanations, the principle argument can now be stated as follows: Let F(s) be a singled-valued function with a nite number of poles in the s-plane, and let s be chosen such that it does not pass through the poles or zeros of F(s). Thus the corresponding F locus mapped in the F(s)-plane will encircle the origin given by the formula N = ZP where N number of encirclements of the origin made by the path F, Z number of zeros of F(s) encircled by the path s, P number of poles of F(s) encircled by the path s. The values of N can be positive, zero, or negative depending upon the number of zeros and the number of poles of F(s) encircled by s. 1. N 0 (or Z P). The path s encircles more zeros than poles of F(s) in either the clockwise or counterclockwise direction, and F will encircle the origin of the F(s)plane N times in the same direction as that of s. 2. N 0 (or Z P). The path s encircles an equal number of poles and zeros, or no poles and zeros of F(s) in s, and F will not encircle the origin in the F(s)-plane. 3. N 0 (or Z P). This is similar to the case N 0 but F encircles the origin of the F(s)-plane in the opposite direction as that of s. In this analysis, for convenience, the origin of the F(s)-plane is selected to be the critical point from which the value of N is determined. However, it is possible to designate other points in the complex plane as the critical point depending on the requirement of the application. In the case of the Nyquist criterion, the critical point is the 1 on the real axis of the F(s)-plane. CASE 2. Another way of presenting the principal argument may be to begin with the derivation of the relationship between the open-loop and closed-loop poles or zeros viewed from the characteristic equation. Lets take a closed-loop control system with single input and single output as shown in (30)
Figure 4. Block diagram of a closed-loop system. A closed-loop system has a forward path transfer function G(s) and a feedback path transfer function H(s). The relation between the input and output can be expressed in terms of these two terms in the form of a system transfer function to be used in analysis and design of systems.
Fig. 4. Writing the transfer functions in Laplace transforms as G (s ) = and H (s ) = then G ( s )H ( s ) = NG (s )NH (s ) D G ( s )D H ( s ) (33) NH (s ) DH ( s ) (32) NG (s ) DG ( s ) (31)
Hence, the characteristic equation 1 + G ( s )H ( s ) = DG (s )DH (s ) + NG (s )NH (s ) D G ( s )D H ( s ) (34)
and the closed-loop transfer function M (s ) = NG (s )DH (s ) G (s ) = 1 + G ( s )H ( s ) DG (s )DH (s ) + NG (s )NH (s ) (35)
From Eqs. (31) to (34), it can be observed that the poles of 1 G(s)H(s) are the same as the poles of G(s)H(s). But more importantly, the zeros of the function 1 G(s)H(s) are the same as the poles of M(s) of the closed-loop system. Although simple, this observation bears particular importance in the understanding of the Nyquist criterion. Assume that (s) equals 1 G(s)H(s) and has poles and zeros in the s-plane, as shown in Fig. 5. As any point s1 of contour s is substituted in (s), it maps to a point on contour . For the purposes of illustration, assume that s encloses a pole and two zeros. Also, two poles and a zero lie outside or are unbounded by the contour. As the point s1 moves around the contour in a chosen clockwise direction, each of the pole and zero vectors connected to that point trace angles. Take the general equation (s ) = (s + z1 )(s + z2 ) (s + zZ ) , m, n {1, 2, . . .} (s + p1 )(s + p2 ) (s + pP ) (36)
This is equivalent to (s ) = | (s )|\ (s ) (37)
24
j s-plane s1 p2 z3
As a matter of interest, similar conclusions can be drawn from the Eqs. (27)(29), taking (s) analogous to F(s)
p2 p 1 p1
z 1
z1
1 2 j
f (s ) ds =
1 2 j
F (s ) ds 1 = F (s ) 2 j
d[ln F (s )] = Z P (40)
z 3
p3
p 3
z2
z2
and writing this equation as
1 2 j
d[ln F (s )] =
1 2 j
d[ln |F (s )|] + j arg[ln F (s )] = Z P (41)
Figure 5. Angles traced by poles and zeros. As the point s1 traverses around the closed contour, each pole and zero trace 360. The angles traced by the poles and zeros outside the contour go to a minimum and then to a maximum and then back to the original value tracing the same path in the opposite direction. The net angle traced becomes zero.
where | ( s )| = |s + z1 ||s + z2 | |s + zZ | |s + p1 ||s + p2 | |s + pP | (38)
and s z1 s zz, s p1 s pP are length of vectors and the angles are
(s ) = \ (s + z 1 ) + + \ (s + z Z ) \ (s + p1 ) \ (s + pP )
(39)
From Fig. 5, we can deduce that as the point s1 traverses around the contour once, the poles and zeros encircled by the contour s go through a complete rotation, each tracing an angle of 2 radians. On the other hand, the poles or zeros that lie outside of s undergo a net angular change of 0 radians. Because of the positive and negative signs of the Eq. (39), the net number of rotation is equal to the difference between the number of zeros and poles lying inside contour s.
For a closed contour, the rst part of the integration, d[ln F(s)] will be zero, but the second term is the 2 times the net encirclement of the origin. As discussed earlier, the poles of characteristics equation (s) 1 G(s)H(s) are also the poles of G(s)H(s). Because the denominator DG(s)DH(s) of Eq. (34) is much simpler than the numerator DG(s)DH(s) NG(s)NH(s), the poles of the equation can be determined relatively easily. Also, the zeros enclosed by s are the zeros of (s), and they are the unknown poles of the closed-loop system. Therefore, P equals the number of enclosed open-loop poles, and Z equals the number of enclosed closed-loop poles. Thus, Z N P indicates that the number of closed-loop poles inside the contour equals the number of open-loop poles of G(s)H(s) inside the contour, hence the number of clockwise rotations of the mapping about the origin. If the contour in the s-plane includes the entire right half plane (RHP), as illustrated in Fig. 6, the number of closedloop poles enclosed by the contour determines the stability of the system. Because it is possible to count the number of open-loop poles P (usually by inspection) inside the bounding contour in the RHP, the number of enclosures of the origin N indicates the existence of the closed-loop poles on the RHP. The poles and zeros of G(s)H(s) are usually known and if the mapping function is taken to be (s) equals G(s)H(s) instead of 1 G(s)H(s), the resulting contour is the same except that it is translated one unit to the left. Thus the number of rotations about the point 1 in the G(s)H(s)-plane may be
j s = j LHP + j s-plane RHP s = j LHP
j s-plane RHP
R 0
Figure 6. The Nyquist path. The Nyquist path covers the entire right half of the splane, avoiding poles and zeros located on the imaginary axis, as shown in (a). The existence of closed-loop poles on the RHP indicates unstable conditions. In (b), poles and zeros on the imaginary axis are included in the Nyquist path.
j = Poles or zeros R = Infinite radius
(a)
(b)
25
Table 3. The Possible Outcomes of Number of Poles and Zeros on the s-Plane (s)-Plane Locus Direction of s-Plane Locus Clockwise Counterclockwise Clockwise Counterclockwise Clockwise Counterclockwise Encircles of the Origin N N N N 0 0 Direction of Encirclement Clockwise Counterclockwise Counterclockwise Clockwise None None
With the new critical point at (1, j0), it will be necessary to dene two sets of N, Z, and P as follows: N0 number of encirclement around the origin made by F(s). Z0 number of zeros F(s) encircled by the Nyquist path in the right half of the s-plane. P0 number of poles F(s) encircled by the Nyquist path in the right half of the s-plane. N1 number of encirclement around the point (1, j0) made by F(s). Z1 number of zeros 1 F(s) encircled by the Nyquist path in the right half of the s-plane. P1 number of poles 1 F(s) encircled by the Nyquist path in the right half of the s-plane. When the closed-loop system has only a single feedback system having the loop transfer function of G(s)H(s), then F(s) G(s)H(s). Now it becomes clear that P0 = P 1 (42)
NZP N0 N0 N0
counted instead of the origin. Hence the Nyquist criterion may be stated as follows: If a contour s encircles the entire right half plane, the number of closed-loop poles Z in the RHP of the s-plane can be determined by the number of open-loop poles P in the RHP and the number of revolutions N of the resulting contour around the point 1 in the G(s)H(s)-plane. This mapping is called the Nyquist diagram or the Nyquist plot of G(s)H(s). A summary of all the possible outcomes of the principle argument is given in Table 3. The method discussed is also called the frequency response technique. Around contour s in the RHP the mapping of the points on the j-axis through G(s)H(s) is the same as using the substitution s equals j, hence forming the frequency response G( j)H( j). Thus the frequency response over the positive j-axis from 0 to are used to determine the Nyquist plot. That is, instead of tracing the entire RHP, it is sufcient to use just a part of the contour s. The Nyquist criteria could have easily been built upon the tracing of the left half plane (LHP); however, the solution is a relative one. In Fig. 6(a,b) it can be seen that the contour s encircles the entire right half of the s-plane in the counterclockwise sense. The reason for this is because in mathematics counterclockwise is traditionally dened to be positive. Observe on Fig. 6(a) that small semicircles are drawn along the j-axis because the Nyquist path must not go through any of the poles or zeros of (s). If any poles or zeros fall on the j-axis, then the path s should detour around these points. Only the poles or zeros that lie in the RHP of the s-plane need to be encircled by the Nyquist path. From the principal argument the stability of a closed-loop system can be determined, after the Nyquist path is specied, by plotting the function (s) 1 F(s) where F(s) equals to G(s)H(s) and the s variable is chosen along the Nyquist path. The behavior of the (s) plot, or the new path , is referred to as the Nyquist plot of (s), with respect to the critical point, the origin. Because the function F(s) is usually known and is much simpler to construct, the Nyquist plot of F(s) arrives at the same conclusion about the stability of a closed-loop system. This is simply done by shifting the critical point from the origin to the point (1, j0) on F(s)-plane. This is because the origin of the [1 F(s)]-plane corresponds to (1, j0) of the F(s)-plane.
because F(s) and 1 F(s) always have the same poles. The result is similar to the one derived earlier in the discussion. The other stability requirements are that for the closed-loop stability Z 1 = 0 and for the open-loop stability P0 = 0 (44) (43)
Z1 must be zero because of the zeros of 1 G(s)H(s) and the poles of the closed-loop transfer function M(s) as discussed earlier; any poles that lie in the left-hand plane causes system instability. For the case of the open-loop stability, the P0 is the number of poles of F(s) encircled by the Nyquist path in the right half of the s-plane and must be zero for stability conditions. The discussions presented so far may be summarized as follows: 1. For a given feedback control system, the closed-loop transfer function is given by Eq. (35), and the denominator function represent the closed-loop transfer function as given by Eq. (34), which is equal to (s). The Nyquist path is dened in accordance with the pole and zero properties of F(s) on the j-axis. 2. The Nyquist plot of F(s) is constructed in the [G(s)H(s)]plane. 3. The value of N0 is determined by observing the behavior of the Nyquist plot of F(s) with respect to the origin. Similarly the value N1 is determined with respect to the point (1, j0). 4. After determining N0 and N1, the value of P0, if not already known, can be obtained from N0 = Z0 P0 (45)
26
if Z0 is known. With P0 determined, P1 is also known via Eq. (42), and Z1 can then be calculated with N 1 = Z 1 P 1 and from Eq. (43) Eq. (46) simplies to N 1 = P 1 (47) (46)
Also, two quantities 1 and 2 will be dened to represent the total angle traversed by the Nyquist plots of F(s) with respect to the point (1, j0), corresponding to the points s1 and s1, respectively. Thus the two new quantities may be written as 1 = 2 N 1,1 = 2 (Z 1 P 1 ) and 2 = 2 N 1,2 = 2 (Z 1 P P 1 ) (51) (50)
Now the Nyquist criterion may also be stated in the following manner. For a closed-loop system to be stable, the Nyquist plot of F(s) must encircle the critical point (1, j0) as many times as the number of poles of F(s) that lie in the right half of the s-plane. The encirclements, if any, must be in the clockwise direction when the Nyquist path is dened in the counterclockwise sense.
SIMPLIFIED NYQUIST CRITERIA The Nyquist criterion discussed previously requires the construction of the Nyquist plot corresponding to the Nyquist path in the s-plane. Complication arises when the F(s)-plane poles or zeros that lie on the j-axis, as in Fig. 6(a), indicate small indentations around these points. As pointed out in Kuo (1), Yeung and Lai came up with a simplied version of the Nyquist criterion for closed-loop systems that requires only the Nyquist plot corresponding to the positive j-axis of the s-plane. In the development of this simplied criterion, two paths as shown in Fig. 6(a,b) are considered. The rst path s1 encircles the entire right half of the s-plane excluding all the poles and zeros that lie on the j-axis. The second path s2 encircles the excluded poles and zeros that may exist. Now new quantities may be dened as follows: Z1 number of zeros of 1 F(s) in the right half of the s-plane. P1 number of poles of 1 F(s) in the right half of the s-plane, and is equal to P0, which are poles of F(s) in the right half of the s-plane. P number of poles F(s) or 1 F(s) that lie on the jaxis including the origin. N1,1 number of times the point (1, j0) of the F(s)-plane is encircled by the Nyquist plot of F(s) corresponding to s1. N1,2 number of times the point (1, j0) of the F(s)-plane is encircled by the Nyquist plot of F(s) corresponding to s2. According to the Nyquist criterion, N 1,1 = Z 1 P 1 and N 1,2 = Z 1 P P 1 (49) (48)
To analyze the Nyquist path, it is best to consider the path having three major sections. The rst section is the portion from s equals to j to j along the semicircle having an innite radius, the second portion is the path along the jaxis excluding the small indentations, and the nal sections include the small indentations. Because the Nyquist plot is symmetrical at j 0, the angles traversed are identical for positive and negative values of . Hence 1 = 211 + 12 + 13 and 2 = 211 12 + 13 where (53) (52)
11 angle traversed by the Nyquist plot of F(s) with respect to (1, j0) corresponding to the positive or negative side of the j-axis, excluding the small indentations and the factor of two emmerging in Eq. (52) and Eq. (53). 12 angle traversed by the Nyquist plot of F(s) with respect to (1, j0) corresponding to the small indentations along the j-axis s1. Also, with the direction of the small indentations of s2 different from that of its counterpart, the negative sign emerges in Eq. (53). 13 angle traversed by the Nyquist plot of F(s) with respect to (1, j0) corresponding to the semicircle with innite radius on the Nyquist path.
Generally, for a physical realizable transfer function, the number of poles cannot exceed the number of zeros of F(s). Therefore, the Nyquist plot of F(s) corresponding to the innite semicircle must be a point on the real axis or a trajectory around the origin of the F(s)-plane. The angle 13 traversed by the phasor from the point at (1, j0) to the Nyquist plot along the semicircle is always zero. Combining Eq. (52) and Eq. (53) yields 1 + 2 = 411 since 13 is zero, 1 + 2 = 2 (2Z 1 P 2P 1 ) hence 11 = (Z 1 0.5P P 1 ) (56) (55) (54)
where Eq. (49) includes the number of poles of F(s) or 1 F(s) that lie on the j-axis.
27
Equation (56) means that the net angle traversed by the phasor from the (1, j0) point to the F(s) Nyquist plot corresponding to the positive j-axis of the s-plane excluding any of the small indentations, that is The number of zeros of 1 F(s) in the right half of the splane minus the sum of half the poles on the j-axis and the number of poles of F(s) in the right half of the s-plane multiplied by radians. This means that the Nyquist plot can be constructed corresponding to s 0 to s j portion of the Nyquist path. For an unstable closed-loop system, the number of roots of the characteristic equation that fall in the right half of the splane can be determined via Eq. (55). As mentioned earlier, a closed-loop system is stable only if Z1 is equal to zero. Hence, 11 = (0.5P + P 1 ) (57)
SOLUTION. To obtain the Nyquist plot, rearrange this equation, substitute s j, and assign the nominal value, K 1, G( j )H ( j ) = (1 + j )(1 + 0.05 j ) j (1 + 5 j )(1 + 2.5 j ) (59)
Find magnitudes and angles in terms of variable as
1 + 2 1 + (0.05)2 |G( j )H ( j )| = 1 + (5)2 1 + (2.5)2

and
(60)
G( j )H ( j ) = tan 1 + tan 1 0.05 90 tan 1 5 tan 1 2.5
(61)
This indicates that for closed-loop system stability the phase traversed by the Nyquist plot of F(s) where s varies from zero to j with respect to (1, j0) point cannot be positive because P and P1 cannot be negative.
Now Nyquist contour may be applied by substituting values of from zero to on the positive part of the imaginary axis on the s-plane. By avoiding the pole located on the origin and substituting small positive values on the positive and negative side of zero, it is possible to observe the following features:
NYQUIST DIAGRAMS The Nyquist analysis is based on the assumption that the control systems are linear; hence, the dynamic performances are described by a set of linear differential equations. Because of the nature of feedback control systems, the degrees of numerator of the loop transfer function F(s) G(s)H(s) is always less than or equal to the degree of the denominator. All the Nyquist diagrams presented here are based on these two assumptions. As explained previously, when plotting Nyquist diagrams, it is sufcient to assign values for the complex variable s on the j-axis avoiding possible poles and zeros on the imaginary axis. The frequency response of G(s)H(s) can be determined by substituting s j and by nding the imaginary and complex components of G( j)H( j). Alternatively, G( j)H( j) can be written in polar form, and magnitudes and angles are determined for plotting on a polar graph paper. These techniques will be illustrated in the following examples. Example 1. Plot the Nyquist diagram of a closed-loop control system as in Fig. 4 with a loop transfer function G ( s )H ( s ) = K (s + 1 )(s + 20 ) 250s(s + 0.2 )(s + 0.4 ) (58)
0+ 0
|G( j )H ( j )| |G( j )H ( j )| 0 |G( j )H ( j )| 0 |G( j )H ( j )|
and and and and
G( j )H ( j ) 90 G( j )H ( j ) 90 G( j )H ( j ) 90 G( j )H ( j ) 90
These features indicate that for a clockwise rotation in the s-plane covering the entire right half plane, the graph starts from the innity on the imaginary axis (in either the fourth or the third quadrant) and approaches zero again from the 90 for 0 . Similarly, the graph starts from 0 at an angle 90 and approaches to innity with the same angle 90 for 0. By substituting intermediate values for , the results in Table 4 may be obtained. We can see that the plot goes numerically above 180 between 0.4 rad/s and 0.5 rad/s. It also falls back to be numerically less than 180 after 2.0 rad/s. This means that it crosses the real axis on the negative side of the s-plane twice. At this point, a polar plot graph paper may be used to sketch the curve outlined in Table 4. Or a second table, which shows the real and imaginary components of G( j)H( j), may be made from Table 4 by using the relation [Rej R cos Rj sin ] as in Table 5. An alternative approach to calculate the real and imaginary components of G( j)H( j) is introduced in the next example. The Nyquist plot of Table 5 is shown in Fig. 7. It is worth noting that Table 5 could also have been drawn by rearranging Eq. (59) as real and imaginary components. This is a long procedure, but using it allows us to calculate the exact values
Table 4. Calculation of Magnitudes and Angles of G( j)H( j) rad/s G( j)H( j) G( j)H( j) 0 90 0.1 124 9.0 0.4 175 0.79 0.5 181.6 0.55 1.0 188.9 0.1 2.0 182.8 0.021 4.0 174.2 0.005 90 0
28
Table 5. Calculation of Real and Imaginary Components of G( j)H( j) rad/s G( j)H( j) G( j)H( j) 0 0 0.1 5.03 7.46 0.4 0.787 0.069 0.5 0.55 0.018 1.0 0.099 0.015 2.0 0.021 0.0001 4.0 0.005 0.0005 0 0
of the gain and phase margins if an approximate estimation from the plot is not permissible. By using Eqs. (60) and (61), it can be shown that G( j)H( j) 0.51 and G( j)H( j) 180 for 0.481 rad/s. Remember that value is for the nominal K where K 1. For other values of K, Eq. (60) should have been written as
transfer function G( j)H( j) and assign the nominal value (K 1) G( j ) = (1 + j 2) j ( j 1 ) (64)
1 + 2 1 + (0.05)2 |G( j )H ( j )| = K 1 + (5)2 1 + (2.5)2
(62)
Find magnitudes and angles in terms of variable as 1 + 4 2 G( j ) = 1 + 2 and G( j ) = tan 1 2 90 tan 1 /(1 )
(65)
Therefore, the curve will pass to the left-hand side of the 1 point on the real axis if G( j)H( j) 1. Hence K 0.51 1 gives the value K 1.9 in which the plot will encircle the 1 j0 point in the clockwise direction, thus leading to instability. This indicates that there is a zero of the characteristic equation on the RHP of the s-plane or a RHP pole of the closed loop transfer function, hence the instability. From the preceding analysis, it is known that the plot crosses the real axis again at a frequency slightly greater than 2.0 rad/s. By substituting a value 2.57 rad/s, from Eqs. (60) and (61), it can be shown that G( j)H( j) 0.011 and G( j)H( j) 180. As explained previously the corresponding value of K 91 obtained from K 0.011 1. For K 91 and above, the system becomes stable again. Example 2. Plot the Nyquist diagram of unity feedback control system which has a forward gain transfer function G (s ) = ( 1 + 2s ) s (s 1 ) (63)
(66)
Lets consider the nominal value of K. Observing the extreme values for in the clockwise direction and starting from 0 gives
0+ |G( j )H ( j )| and G( j )H ( j ) 270 or 90 |G( j )H ( j )| 0 and G( j )H ( j ) +270 or 90 |G( j )H ( j )| 0 and G( j )H ( j ) +90 0 |G( j )H ( j )| and G( j )H ( j ) 90
It is important to highlight the angle equation tan1 /(1) because the negative sign in the denominator indicates what quadrant the angle is for varying . These features indicate that for a clockwise rotation of a contour in the s-plane covering the entire right half plane, the graph starts from the innity on the imaginary axis from the rst or second quadrant and approaches zero from the 90 for 0 . Similarly, the graph starts from 0 at an angle 90 and approaches innity with the same angle 90 for 0. Table 6 may be obtained by substituting values for , but this time for only 0 . The Nyquist plot is given in Fig. 8. The open-loop transfer function has one pole on the RHP, and therefore P 1. In order for this system to be stable, N must be equal to 1, that is, one counterclockwise encirclement of the 1 j0 point. As shown in Fig. 8, the rotation of the curve is in a counterclockwise direction, and it encircles the origin once; hence, the system is stable. From Table 6, we can see that the graph crosses the real axis between 0.6 rad/s and 1.0 rad/s. By guessing and by using the trial-and-error method, this crossover frequency may be determined as 0.7 rad/s. At this frequency, the magnitude is 2.0. As in the case of Example 1, the critical value of the gain can be found from K 2.0 1 to be K 0.5. An alternative mathematical approach can be employed to nd the real and imaginary components of the loop transfer function as G( j ) = (1 + j 2) 32 + j ( 23 ) (1 + j 2) = 2 = (67) j ( j 1 ) j 2 + 4
SOLUTION. In this example, because G(s) has a pole on the RHP of the s-plane, it is open-loop unstable. As before, in order to obtain the Nyquist plot, substitute s j in the loop
Im
=0
G(s)H(s)-plane
R 1
Re
= 0+
Figure 7. The Nyquist plot of Example 1. As the Nyquist path on the s-plane traverses in the clockwise direction, the corresponding path of G(s)H(s) traverses in the same direction on the G(s)H(s)-plane. Because the Nyquist plot does not encircle the critical 1 j0 point, this system is stable.
29
Table 6. Calculation of Magnitudes and Angles of G( j)H( j) rad/s G( j)H( j) G( j)H( j) 0 90 0.1 107.0 10.15 0.4 150.5 2.97 0.6 171.16 2.23 1.0 198.4 1.58 4.0 248.8 0.49 10.0 261.4 0.02 270 0
The graph crosses the real axis when the imaginary part is equal to zero, that is ( 23) 0 or 1/ 2 rad/s. The intersection can be calculated by substituting the value of in the real component of G( j)H( j) as 3 (1/2 ) 32 = 2 = 2 4 + (1/2 ) + (1/4 ) (68)
Phase margin is the angle, in degrees, by which the locus must be rotated in order that gain crossover point passes through 1 j0. The gain margin is measured in decibels and expressed in phase-crossover frequency as GM = 20 log10 = 1 dB |G( j c )H ( j c )| (69)
There are many examples of Nyquist plots in control engineering books (15). Figure 9 illustrates typical Nyquist plots of some of the selected control systems. Stability Margins In the preceding examples, we have demonstrated that the Nyquist plots of the loop transfer function, G( j)H( j), depends on the values of K. This is illustrated in Fig. 10. As the K is increased or decreased, as the case may be, at a certain value the locus passes through 1 j0 point. At this point, the system exhibits sustained oscillations. As K increases further, the system becomes unstable. Generally, oscillations increase as the locus of G( j)H( j) gets closer to the 1 j0 point. The closeness of the locus to the critical point is measured by the stability margins usually expressed in the form of phase and gain margins, as illustrated in Fig. 10. These margins indicate relative stability and hence help the design of control systems to achieve desired responses. The gain and phase margins may be dened as follows. Gain margin is the amount of gain that can be allowed to increase before the closed loop system becomes unstable.
When the loop transfer function G( j)H( j) passes through the 1 j0 point, the gain margin is 0 dB. The negative or positive value of gain margin depends on the number of poles and zeros of G( j)H( j) on the RHP. If the stability is evaluated when the locus crosses the real axis on the right of the 1 j0 point (Example 1), the gain margin is positive. If the stability is evaluated on the left of the 1 j0 point (Example 2), then the gain margin is negative. The phase margin can be determined by calculating the gain-crossover frequency when the gain of the G( j)H( j) 1 and by evaluating the phase angle of the system at that frequency. That is PM = G( j )H ( j ) 180 (70)
As in the case of gain margin, the sign of phase margin is relative to stability condition and the shape of the locus. In Example 1, a negative value for the PM indicates unstable condition whereas in Example 2, the negative value implies stability. Effects of Adding Poles and Zeros Control systems are often designed by introducing additional poles and zeros to the system. Extra poles and zeros in the system change the shape of the Nyquist diagrams and alter phase and gain margins. The inuences of additional poles and zeros on the Nyquist locus can be evaluated by comparing the loci of different systems given in Fig. 9. Some observations may be made as follows. The mathematical difference between parts (a) and (b) in Fig. 9 is the additional pole. In this case, the Nyquist locus is shifted by 90 as , occupying quadrants 3 and 4 instead of quadrant 4 only. Adding an extra pole introduces further 90, and the locus occupies three quadrants. In this case, the risk of instability exists because the possibility of encirclement of 1 j0 is introduced. In Fig. 9, the effect of adding a pole at s 0 can be seen by observing parts (a) and (d) or (b) and (e). In both cases, Nyquist loci are rotated by 90 for all frequencies. Adding a nite pole increases the risk of instability.
Im
=0
G(s)H(s)-plane
Re
= 0+
Figure 8. The Nyquist plot of Example 2. The open-loop transfer function of this control system has a pole on the RHP; hence, the system is open-loop unstable. However, the Nyquist plot encircles the critical 1 j0 point once in the counterclockwise direction, indicating that there are no closed-loop poles on the RHP. Therefore, the system is stable.
30

Nyquist diagrams 0.8 0.6
K G(s)H(s) = (s1 + 1)
0.6 0.4 0.2

Imaginary axis
Nyquist diagrams
Nyquist diagrams 8
0.4
Imaginary axis
G(s)H(s) = K (s1 + 1)(s2 + 1)

Imaginary axis
6 4 2 0 2 4 6 G(s)H(s) = K (s 1+1)(s2+1)(s3+1)
0.2 0 0.2 0.4 0.6
0 0.2 0.4 0.6 0.8 1
0.5
0 Real axis (a)
0.5
0.8 1
0.5
0 Real axis (b)
0.5
8 4
2 4 Real axis (c)
10
Nyquist diagrams 2 1.5 1

Imaginary axis
Nyquist diagrams 2 1.5 G(s)H(s) = K (s 1+1)(s2+1)
G(s)H(s) =
0.5 0 0.5 1 1.5 2 0 0.5 2 1.5 1 0.5 Real axis (d)

Nyquist diagrams 10 8 6 G(s)H(s) = K (s z +1) (s 1+1)(s2+1)
Imaginary axis
K (s 1+1)
1 0.5 0 0.5 1 1.5
1.5
2 2
1.5 1 0.5 0 0.5 Real axis (e)

Im
1.5
G(s)(H)s = KesT(1 esL) where L > T
Imaginary axis
4 2 0 2 4 6 8 10 2 1.5 1 0.5 0 0.5 Real axis (f) 1 1.5 2

(g) 1 Re
Figure 9. The Nyquist plots of selected control systems.
31
Im
Im G(s)H(s)-plane Unit circle
G(s)H(s)-plane
Re
1 PM GM
Re
=0
Increasing K
Figure 10. Phase and gain margins. The closeness of the locus to the critical 1 j0 point is measured by the margins. Good gain and phase margins are obtained for K1. As K increases, both gain and phase margins become zero (for K2) indicating critical stability. Further increase in K leads to unstable conditions. Note the changes in the gain and phase margins for varying K.
Increasing
Figure 11. The effect of time delays. Pure time delays do not introduce any extra poles and zeros into the system. However, the magnitude is equal to unity for all frequencies, the phase ( T) affects the stability. For large values of time delay T, the system may be unstable.
The effect of adding a zero into the system can be seen in parts (c) and (f) in Fig. 9. In this case, the loop transfer function increases the phase of G(s)H(s) by 90 as . This result conrms the general knowledge that addition of a derivative control or a zero makes the system more stable.
tion of a single loop, single input and single output of a system may be written as M (z ) = G (z ) C (z ) = R (z ) 1 + GH (z ) (73)
EFFECTS OF TIME DELAYS The Nyquist criterion can be utilized to evaluate the effects of time delays on the relative stability of feedback control systems. With the pure time delays, the loop transfer function may be written as G(s )H (s ) = e sT G1 (s )H1 (s ) (71)
where T is the time delay. The term esT does not introduce any additional poles or zeros within the contour. However, it adds a phase shift to the frequency response without altering the magnitude of the curve. This is because
where z is the z-transform dened as z esT. The stability of the system can be studied by investigating the zeros of the characteristic equation 1 GH(z) 0. For the system to be stable, all the roots of the characteristic equation must be inside the unit circle in the z-plane. As in the continuous-time systems, the investigation of the Nyquist plot of GH(z) with respect to critical point, 1 j0, indicates the system stability. The general theory presented for continuous time control systems is applicable to discrete time systems with minor modications. Here, an example will be given to illustrate the use of Nyquist in discrete time control systems.
Im z G(z)H(z)-plane
|G( j )H ( j )| = |e j T ||G1 ( j )H1 ( j )| = | cos () j sin()||G1 ( j )H1 ( j )|
(72)
0.0231 1
The term containing the time delay is cos() j sin() 1, but the phase is tan1(sin T /cos T) T. This shows that the phase grows increasingly negative in proportion to the frequency. A plot of the effect of time delay is given in Fig. 11. Because of the addition of the phase shift, the stability of the system is affected for large values of T.
= 3.14
Re z
=0
NYQUIST STABILITY CRITERION FOR DIGITAL SYSTEMS The Nyquist stability criterion can equally be applied to linear continuous-data discrete time systems to graphically determine the stability. Generally, the closed-loop transfer func-
Figure 12. Nyquist plot of Example 3. The Nyquist path on the zplane must have small indention at z 1 on the unit circle. The Nyquist plot of path of GH(z) in the GH(z)-plane intersects the negative real axis at 0.231 when 3.14 rad/s. For stability, the value of K must be less than 4.33.
32
Im G(s)H(s)-plane
THE INVERSE NYQUIST AND NYQUIST PLOT FOR MULTIVARIABLE SYSTEMS Inverse Nyquist is simply the reciprocal of the complex quantity in the Nyquist plot. They nd applications particularly in multiple loop and multivariable systems where graphical analysis may be preferred. The Nyquist stability criterion applied to inverse plots can be stated as a closed loop system stable, if the encirclement of the critical 1 j0 point by the 1/ G(s)H(s) is in the counterclockwise direction for a clockwise Nyquist path in the splane. As in the case of a normal Nyquist, the number of encirclements must equal the number of poles of 1/ G(s)H(s) that lie in the right half of the s-plane. Inverse Nyquist plots is particularly useful in the analysis of multi-inputmulti-output control systems. In the multivariable feedback control systems, the relations between inputs and outputs may be expressed in matrix form as C(s ) = [I + KG(s )H(s )] 1G(s )KR(s )
x
Re
1.5 G(s) = s3 + 2s2 +2s+1 1 4s+1 H(s) = I
(75)
Figure 13. Examples of Nyquist plots of multivariable systems. The Nyquist plot for multivariable systems carries similar information as in the single-inputsingle-output systems. The number and the direction of encirclements of the critical 1 j0 point conveys the message about the stability. But rigorous mathematical analysis is necessary because matrices are involved.
where G(s), H(s) and K are n n matrices. Similar to single-inputsingle-output systems, the output is exponentially stable iff det[I KG(s)H(s)]1 has no poles in the right half of the s-plane. The Nyquist diagrams can be obtained by appropriately considering the K values as K diagk1, k2, . . ., kn and the gij(s) elements of G(s). A typical example of a Nyquist diagram of a multivariable control systems is shown in Fig. 13. This example is also given by Westphal (5); interested readers could refer to that book for further details. BIBLIOGRAPHY
Example 3. Show the Nyquist plot of a discrete time system with transfer function of GH (z ) = 0.632z (z 1 )(z 0.368z )
(74)
for a sampling period of T 1 s. SOLUTION. The loop transfer function GH(z) does not have any poles outside the unit circle, but it has one pole on the unit circle. As in the case of s-plane zeros on the imaginary axis, the Nyquist path on the z-plane must have small indention at z 1 on the unit circle. The Nyquist path, shown in Fig. 12, intersects the negative real axis of the GH(z)-plane at 0.231 when the value of 3.14 rad/s. The critical 1 j0 point may be encircled if 0.231K 1, that is K 4.33.
1. B. J. Kuo, Automatic Control Systems, 6th ed., Englewood Cliffs, NJ: Prentice-Hall, 1991. 2. E. Kreyszing, Advanced Engineering Mathematics, 7th ed., New York: John Wiley, 1993. 3. W. S. Levine, The Control Handbook, Boca Raton, FL: CRC Press, 1996. 4. K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle River, NJ: Prentice-Hall, 1997. 5. L. C. Westphal, Sourcebook of Control Systems Engineering, Cambridge, UK: Chapman & Hall, 1995.
HALIT EREN BERT WEI JUET WONG

Curtin University of Technology
NYQUIST STABILITY. See NYQUIST CRITERION, DIAGRAMS, AND STABILITY.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Open-Loop Oscillatory Control Standard Article B. Lehman1, S. Weibel2, J. Baillieul3 1Northeastern University, Boston, MA, 2Northeastern University, Boston, MA, 3Boston University, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1028 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (298K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Problems in Open-Loop Oscillatory Control Stabilization by Oscillatory Controls: Methods and Solutions Performance Improvements and Transient Behavior Resonance Phenomena in Periodically Forced Systems Acknowledgments Appendix 1. Classical Averaging Theory Appendix 2. Floquet Theory About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
150
OPEN-LOOP OSCILLATORY CONTROL

Conventional control systems rely on feedback, feedforward, or a combination of the two. In a feedback control system, the controlled variable is usually compared with a reference variable, and the difference between the two, the error, is used to regulate the system. In a feedforward control system, an appropriate additive control signal is introduced to compensate for disturbances. While feedback and feedforward rely on different principles, both methods require measurements. In a feedback control system, the controlled variable is measured. Likewise, in a feedforward control system the measurement of disturbances is used in the implementation. However, measurements of states or disturbances are often costly, difcult, or even impossible to obtain. In these cases, feedback and feedforward are not feasible means of control. Consider the particle accelerator originally described in (1) and later discussed in detail in (2). The control objective is to focus a beam of particles along the accelerator. In cyclic accelerators with azimuth symmetrical magnetic elds, the plant, a beam of particles, is described by d 2x + 2 (1 n ) x = 0 d 2 d 2z + 2 nz = 0 d 2
(1)
151
where x and z are state coordinates representing small betatron oscillations of the beam, is the betatron wave number, is an independent variable (azimuth), and n is the eld index of refraction. For proper operation, the eld index of refraction should satisfy 0 n 1. However, for this range of n, the beam focusing is often unacceptable. Feedback is usually not possible due to the difculty of measuring x and z. Feedforward also has similar measurement difculties. In such cases, a natural question is whether control is possible for such unmeasureable systems. For many systems, one alternative is open-loop oscillatory control, sometimes referred to as vibrational control (not to be confused with vibration control where the idea is to reduce vibrations). Open-loop oscillatory control is a fairly recently developed control methodology that does not require measurements of states or disturbances. Instead, zero mean periodic excitation is used to modify the plant behavior in such a way that control is achieved as the result of the systems natural response to the excitation. For example, oscillations in the cyclic accelerator can be introduced by appropriately focusing and defocusing sectors of the magnetic lens. This causes a suppression of betatron oscillations and thereby makes the focus beam more acceptable. An early heuristic description of this phenomena was given by Livingston (1), but it was not until 1980 that the heuristically controlled azimuth accelerator was explained in the context of open-loop oscillatory control in (2). Denition 1. Open-loop Oscillatory Control. The utilization of periodic (or almost periodic) control laws, without the use of measurements in order to induce a desired dynamic response in a system is referred to as open-loop oscillatory control or vibrational control. The simplicity of open-loop oscillatory control synthesis is offset by the difculty added in introducing explicit time dependence in the state system models. In order to simplify the analysis, open-loop oscillatory control algorithms may restrict the control action so the controlled system admits a small parameter. One way to obtain a small parameter is to introduce periodic excitation whose frequency is an order of magnitude larger than the highest system natural frequency. The small parameter will then arise as the result of a rescaling of time. For such systems, the time-varying open-loop controlled system can be approximated by the behavior of a time-invarient averaged equation, to which the usual analytical techniques for time-invariant systems may be applied. This result forms the basis of classical averaging theory in applied mathematics and dynamical systems. Within the context of forced mechanical systems and averaging, energy methods and a quantity called the averaged potential provide the most direct method of analysis. In the absence of distinct system time or length scales, the local stability of an equilibrium or periodic orbit can be studied by the analysis of the linearized systems rst return map, or monodromy matrix, obtained through Floquet theory. One of the most compelling examples of open-loop oscillatory control is the stabilization of the simple pendulums inverted equilibrium by high frequency vertical oscillation of the pendulums suspension point. This discovery is usually attributed to Bogoliubov (3,4) and Kapitsa (5), although earlier references to similar phenomena exist (6). More recent accounts of this stabilization may be found in (7,8,9,10),
where the averaged potential is used to study global structure and stability of the periodically forced system. The main result is that for a given forcing amplitude, there is a unique critical forcing frequency at which the inverted equilibrium experiences a pitchfork bifurcation. For forcing frequencies higher than the critical frequency, the inverted equilibrium is stable. Reference 10 considers a generalization of the classic problem, where the periodic forcing is directed along an incline with respect to the horizontal. In this case, the pendulum tends to stabilize in congurations aligned with the direction of the forcing, and in fact this phenomenon holds in the general n-link case as well (11). Open-loop oscillatory control has been applied to many systems, and new applications continue to emerge. In (12) the technique was applied to exothermic chemical reactions in a continuous stirred tank reactor (CSTR). In this work, it was shown that by modulating the input and exit chemical feed rates of the CSTR, it is possible to operate in stabilized averaged conversion rates that would otherwise be unstable unless expensive feedback is applied. Although, on average, the same amount of input chemical has been used, the stable operating regimes of the CSTR change substantially with the use of an oscillatory control input. In similar work, the results of (12) are analytically extended to include chemical reactions in a CSTR with delayed recycle stream (13,14). Historically, all of this work on oscillatory open-loop control was prompted by the work of periodic operation of chemical reactors using the sometimes heuristic techniques of push-pull, periodic optimization, and asynchronous quenching (12,15). Experimental applications of open-loop oscillatory control have also included laser illuminated thermochemical systems (16), stabilization of plasma (17), and car parking algorithms (18). In (19), sufcient conditions are given for a periodic process to minimize periodic paths. This approach generalized the result in (20) that showed that periodic paths improve aircraft fuel economy. Other analytic applications of open-loop oscillatory control include rotating chains (21,22), n-link pendula (11), axial compressors (23), and population models (24,25). In the work by Lehman et al. (13,25,26) the technique of oscillatory open-loop control is developed for systems with time-delays. Bentsman and Hong (27,28) have extended the technique to parabolic partial differential equations (PDEs). The application of open-loop control to delay systems and PDEs shows interesting potential since these types of innite dimensional systems are often difcult to control when using feedback. Likewise, there has been success in combining the benets of open-loop oscillations with conventional feedback in order to robustly stabilize systems with zeros in the openright half plane and systems with decentralized xed zeros (2932). As with all other control algorithms, the important issues of designing open-loop oscillatory control include stability, transient response, and accuracy of the controlled system. Certainly, the most important issue is stability. Many classical results on stability of operating points for systems with oscillatory inputs depend on eigenvalues of the averaged system lying in the left half plane, or equivalently the eigenvalues of the monodromy matrix lying within the unit disk. However, there has been growing interest in the stabilization of systems to which such classical results do not apply. These include the mechanical systems studied in (7,8,10,11,33),
152
where eigenvalues locations are typically symmetric with respect to the imaginary axis. Coron (34) has shown the existence of a time-varying feedback stabilizer for systems whose averaged versions have eigenvalues on the imaginary axis. Additional interest in this design derives from the observation that it provides a method of smooth feedback stabilization for systems which Brockett (35) had previously shown were not stabilizable by smooth, time-invariant feedback. Stability of a system is concerned with the asymptotic behavior of the system. Often it is important to study trajectories of systems as steady-state behavior is being approached. Analysis of such trajectories when there is an oscillatory control input is a difcult task. The oscillatory control is usually designed to be high frequency. As a result, the controlled system is composed of a fast zero average oscillatory trajectory superimposed on a slow trajectory. Therefore, the designer must attempt to control the slow part of the trajectory and ignore (or lter out) the high frequency component. One disadvantage of open-loop oscillatory control is its accuracy. It is well known that driving a nonlinear system with a periodic signal generally excites an array of resonances, and under appropriate conditions chaos in the homoclinic tangles of unstable resonances [See (36) for a complete exposition on this topic]. While subharmonic resonances and chaos tend to be suppressed at high forcing frequencies, 1 : 1 resonances (primary resonances, or periodic orbits), whose averages correspond to xed points of an averaged representation of the dynamics, persist. If a stable 1 : 1 resonance has no association with a xed point of the time-varying system (i.e., it arises through a bifurcation), it is called a hovering motion. These high frequency responses limit the utility of open-loop oscillatory control when control accuracy is important. PROBLEMS IN OPEN-LOOP OSCILLATORY CONTROL Classes of Systems This section considers systems of ordinary differential equations, with inputs, of the form x = f (x, u ) (2)
equal to 0, it is hoped that the periodic forcing can impose a stabilizing effect on the system. In this case, Eq. (2) becomes dx = f (x, 0 + (t )) dt (3)
Denition 2. Vibrationally Stabilizable. An equilibrium point xs of Eq. (2) is said to be vibrationally stabilizable if for any 0 there exists a PAZ vector (t) such that Eq. (3) has an asymptotically stable periodic solution, x*(t), characterized by x xs ; x = 1 T
T 0
x (t ) dt
It is often preferable that Eq. (3) has a xed equilibrium point, xs. However, this is not usually the case since the right hand side of Eq. (3) is time varying and periodic. Therefore, the technique of vibrational stabilization is to determine vibrations (t) such that the (possibly unstable) equilibrium point xs bifurcates into a stable periodic solution whose average is close to xs. The engineering aspects of the problem consist of: 1. Finding conditions for the existence of stabilizing periodic inputs 2. Determining which oscillatory inputs, u( ), are physically realizable and 3. Determining the shape (waveform type, amplitude, phase) of the oscillations to be inserted which will ensure the desired response At this time, it may be useful to explain why it is necessary to use time-varying control inputs as opposed to simply using classical time-invariant open-loop control techniques. Suppose that there is a single-input single-output linear timeinvariant (LTI) system with proper transfer function Y(s)/ U(s) n(s)/ d(s), where Y and U are the Laplace transform of the output and the input, respectively, and n and d are polynomials in s. If all the roots of d(s) 0 have negative real parts, then open-loop control can be used to arbitrarily place system poles simply by letting U(s) d(s)/ p(s), where p(s) 0 has the desired system pole location and the degree of p is greater or equal to the degree of d. At times, this polezero cancellation open-loop control strategy might give desired system performance, especially if there is no need for feedback (perhaps because there are no disturbances). Unfortunately, though, perfect polezero cancellation is not possible. This may not be worrisome if all the roots are in the left-half plane, but when there exists at least one root of d(s) 0 with positive real part, LTI open-loop control cannot stabilize a system. On the other hand, when u(t) is an oscillatory open-loop control input, stabilization is often possible, even when there is a pole in the right-half plane. Indeed, oscillatory open-loop controls have also shown a remarkable robustness to disturbances in many experimental applications (12,37). This is a quality that is absent in LTI open-loop control. Remark 1. This subsection has attempted to state the problem of stabilization in its broadest terms. There are classes of
where x n and u m. The function f : n m n will always be assumed sufciently continuous so that solutions to Eq. (2) exist. Models of this form describe most of the systems appearing in the recent engineering literature on open-loop oscillatory control, as discussed in detail in (37). Stabilization Here we introduce the problem of vibrational stabilization, as found in (24,38). Suppose that (39) has an unstable equilibrium point, xs when u constant 0, and the goal is to determine a control input u(t) that stabilizes this operating point. In addition, suppose this stabilization is to be performed without any state or disturbance measurements. For the stabilization problem being considered, the methods of oscillatory open-loop control are as follows. Introduce into Eq. (2) oscillatory inputs according to the law u(t) 0 (t) where 0 is a constant vector and (t) is a periodic average zero (PAZ) vector, that is, (t) (t T) with T 0 (t)dt 0. Even though the average value of u(t) remains
153
systems, however, for which discussion of stabilization and stability is problematic. Such systems include conservative systems, or more specically, Hamiltonian systems. Hamiltonian systems include dissipation-free mechanical systems, and include many electrical and optical systems as well. The primary defect of Hamiltonian systems as far as control theory is concerned is that the strongest stability these systems can possess is neutral stability; that is, eigenvalues/poles on the imaginary axis. For this reason, standard concepts from control theory seldom yield strong stability results. Progress has recently been made in developing techniques for the stability analysis of these systems. The new techniques make use of the system energy, and in the case of periodically forced systems the averaged potential, to assess the stability of equilibriums. A technique for the equilibrium and stability analysis of a large class of periodically forced Hamiltonian systems is presented later in this article. Transient Behavior and Performance Once a system is determined to be stable, the next issue in evaluating its performance is to determine how quickly the solutions decay to their steady state. This nite time transient behavior is sometimes crucial to system performance. For LTI systems, there are several methods that can be used to obtain estimates for the transient behavior of the output. For example, an estimated output trajectory is obtained from information on the location of the dominant system eigenvalues. Even for nonlinear time-invariant systems, it is common to examine the eigenvalues of a Jacobian linearization in order to examine the rates of decay of solutions. For systems subject to oscillatory open-loop control, the analysis techniques are not so straightforward. As previously mentioned, the control inputs cause Eq. (3) to be time-varying, and analysis of time-varying systems remains an open area of research. However, since it has been assumed that the control inputs have a special structure, that is, periodic and high-frequency, it will be possible to apply the method of averaging to nd approximations of the system transient behavior. Essentially the problem of controlling the transient behavior of time-varying system Eq. (2) is to 1. Determine control inputs, (t) in Eq. (3) so that the solutions to Eq. (3) can be approximated by the solutions of a simpler equation 2. Control the transient behavior of the approximate equation Sometimes this simpler equation turns out to be purely timeinvariant and in the form of dy = P( y ), dt w = q( y) (4)
where, once again, w approximates x. The oscillatory openloop control results in a superposition of fast oscillatory trajectories on slow trajectories. The slow dynamics are represented by y, and h can be a fast periodic function. In either of the above two cases, it is hoped to nd oscillatory control input u(t) such that the transient performance of w meets desired objectives. Since the state equation of the approximations are time-invariant, the analysis becomes simpler. In fact, even though Eq. (5) is time-varying, it is only the output equation which explicitly depends on t. Therefore, many of the well established tools can be applied directly to the state equation. In particular, when P(y) Ay B, then the eigenvalues of matrix A help determine the qualitative features of transient behavior. Steering and Path Planning for Kinematically Nonholonomic Systems An application of open-loop oscillatory control which lies largely outside the boundaries of this chapter is the use of periodic functions in path generation for so-called kinematically nonholonomic systems. Such systems include wheeled vehicles such as the unicycle, autonomous wheeled robots, and cars with or without trailers. More generally, kinematically nonholonomic systems are systems which possess nonintegrable constraints, and typically the state equations do not include dynamic effects, such as torques, accelerations, and forces. Since this class of problems does not involve the articles central themes of stabilization and improvement of transient performance, only a brief description is given. Consider the special case of Eq. (2) in which
m
f (x, u ) =
i= 1
u i gi ( x )
A large body of literature has been published on the use of oscillatory inputs designed to force such systems along prescribed paths. The reader is referred to (37) and its references for details of these types of problems. The types of problems encountered in this application include the following: 1. The prescribed endpoint steering problem requires that given any pair of points x0, x1 n, a vector of piecewise analytic control inputs u( ) (u1( ), . . ., um( )) is to be determined to steer from some state x0 at time t 0 to x1 at time t T 0. 2. The trajectory approximation steering problem requires that given any sufciently regular curve : [0, T] n, a sequence [uj( )] of control input vectors is found such that the corresponding sequence of trajectories converges (uniformly) to . Several authors have suggested constructive methods for periodic controllers in this context, and further details may be found in (37,3944). STABILIZATION BY OSCILLATORY CONTROLS: METHODS AND SOLUTIONS Applications of Classical Averaging Theory The goal of the open-loop oscillatory control is usually to stabilize an unstable equilibrium xs of Eq. (2). This is performed
where w and y are both vectors in n, and w approximates x, the solution to Eq. (3). Often, though, a time-varying output equation is used and the approximate equation becomes dy = P( y ), dt w = h(t , y ) (5)
154

Equilibrium and stability analysis Open-loop oscillatory input Unstable plant (autonomous) Periodic, timevarying plant Transform and average Stable average plant
Connection between systems valid if small parameter sufficiently small. Stable periodic time-varying plant
Figure 1. Flow graph of typical open loop oscillatory control design procedure.
Controlled system
by selecting the (t) in Eq. (3) to be periodic zero average signals, such as sinusoidal inputs or zero average square waves. The frequency of the input is selected to be large, or equivalently, as Fig. 1 shows, the period is small. The periodic system, in the form of Eq. (3) can then be transformed into the form of Eq. (1.1) in Appendix 1, where turns out to be proportional to the period. At this point, the transformed system can be averaged. If the averaged system has a uniformly asymptotically stable equilibrium point, then this implies that there will be a uniformly asymptotically stable periodic orbit of the transformed time-varying system in the vicinity of the equilibrium point. The nal criteria for vibrational stabilization is that the periodic orbit satisfying Eq. (3) remain in the vicinity of xs (even though a transformation is used prior to averaging). This is the reason for introducing the denition of x*, which is the average value of the periodic solution of Eq. (3). What follows is a step-by-step procedure for the analysis of open-loop oscillatory control laws by the classical method of averaging. A brief introduction to the topic of averaging and references to more comprehensive accountings may be found in Appendix A.1. Many summaries of this procedure detailed in this section can also be found in the literature (e.g. see Refs. 13,24,25,37). The following discussion is based on (37). Assume that f in Eq. (3) has a special structure so that Eq. (3) can be rewritten as dx = f 1 (x(t )) + f 2 (x(t ), (t )) dt (6)
form Eq. (1.1) in Appendix 1. To make this desired transformation, consider the so called generating equation given as dx = f 2 (x(t ), u (t )) dt Suppose that this generating equation has a period T general solution h(t, c), for some u ( ) and t t0, where h : n n n and c is uniquely dened for every initial condition x(t0) n. Introduce into Eq. (7) the Lyapunov substitution x(t) h(t, z(t)) to obtain h(t , z (t )) dz = dt z
1
f 1 (h(t , z (t ))
(8)
If time is rescaled by letting t, 1/ , then using the standard abuse of notation of letting znew() zold( / ), Eq. (8) becomes dz = d h( , z ( )) z
1
f 1 (h( , z ( ))
(9)
Equation (9) is a periodic differential equation in standard form with normalized period T 2 and averaging can be applied. The averaged equation (autonomous) corresponding to Eq. (9) is given as
dy = P(y( )); d
P (c ) =
1 2
2 0
h( , c ) c
f 1 (h( , c )) d (10)
where f 1(x(t)) f 1(0, x(t)) and the function f 2(x(t), (t)) is linear with respect to its second argument. Additionally, assume that (t) is periodic of period T (0 T 1) and of the form (t) u (t), where 2 / T, and u ( ) is some xed period2 function. Since the primary interest is high frequency forcing, the usual implication is that the amplitude of (t) is large. It is possible, however, that u ( ) has small amplitude, making the amplitude of (t) small also. Then Eq. (6) can be rewritten as dx = f 1 (x(t )) + f 2 (x(t ), u (t )) dt (7)
It is now possible to convert the averaged equation back to fast time to obtain dy = P( y(t )) dt (11)
In order to proceed with the stability analysis, Eq. (7) will be transformed to an ordinary differential equation in standard
By the theory of averaging, there exists an 0 0 such that for 0 0, the hyperbolic stability properties of Eqs. (9) and (10) are the same. This also implies that for sufciently large, the hyperbolic stability properties of Eqs. (8) and (11) are also the same. Specically, if ys is an asymptotically stable equilibrium point of Eq. (11) (it will also be an asymptotically stable equilibrium point of Eq. (10)), this implies that, for sufciently large, there exists a unique T-periodic solution,
155
*(t) satisfying Eq. (8), in the vicinity of ys that is asymptotically stable also. Furthermore, T is known to be equal to 2 / . Since the transform x(t) h(t, z(t)) is a homeomorphism, there will exist an asymptotically stable T-periodic solution to Eq. (7) given by x*(t) h(t, *(t)). Equation (2) is said to be T vibrationally stabilized provided that x* 1/ T 0 x*(t)dt remains in the vicinity of xs.
Example 1. Oscillatory stabilization of scalar differential equations. Consider the scalar linear differential equation x(n ) + (a1 + u1 (t )) x(n 1 ) + . . . + (an + un (t )) x = 0 (12)
Example 2. Oscillatory stabilization of a second-order LTI system in state space. This example is a slight modication of the problem discussed by (46). Consider the second-order system
x =
0.6 0.8
1.3 0 + 1.6 0
1 u(t ) x 0
(15)
where u is the scalar control. It is easy to verify that when u 0 the equilibrium point xs 0 is unstable (check the eigenvalues of the system). Suppose u(t) cos(t). Then
In (45), the problem of stabilizing Eq. (12) is studied using zero average periodic control inputs in the form ui (t ) = ki sin(t + i ) i = 1, 2, . . ., n (13)
0.6 dx = dt 0.8
1.3 0 x+ 1.6 0
cos (t ) x 0
(16)
where ki are constants. Furthermore, the results determined in (45) show that the impact of the control u1 for stabilization is nonexistent. Hence, assume that k1 0. This system can easily be rewritten in state space form of m q Aq i1 ui(t)Biq. However, due to the results determined in (45) there is no need for this. For sufciently large the hyperbolic stability properties of xs 0 in Eq. (12) are the same as the hyperbolic stability properties of the equilibrium point ys 0 of the corresponding differential equation with constant coefcients given by y ( n ) + ( a 1 + 1 )y ( n 1 ) + . . . + ( a n + n )y = 0 where i = k2 ki cos (2 i ) 2 i = 1, 2, . . ., n (14)
which is in the form of Eq. (7). The generating equation is therefore
x =
0 0
cos (t ) x 0
which has solution x2 c2 and x1 c1 sin(t)c2. Now introduce the substitutions x2 z2 and x1 z1 sin(t)z2 into Eq. (16) and convert time to t with 1/ to obtain
dz = d
1 0
sin( ) 1
0.6 0.8
1.3 1.6
1 0
sin( ) z ( ) (17) 1
The impact of the above result is that it presents a calculation formula for the system. Without knowledge of any periodic transformations or mathematical analysis, it is possible to select the gain and phase of each oscillatory control to stabilize the zero equilibrium of Eq. (12) based on the stability properties of Eq. (14), for sufciently large . Since all the coefcients in Eq. (14) are known, the analysis becomes simple. Some important comments on Eq. (14) need to be made. First, notice that since 1 0, this implies that the coefcient of the n 1th derivative in Eq. (14) cannot be changed. This coefcient is equal to the negative of the sum of all system eigenvalues ( trace[A]). Hence, for vibrational stabilization to take place, it must be that a1 0. Reference 45 shows this to be a necessary and sufcient condition for scalar differenm tial equations. (In fact, for all systems q Aq i1 ui(t)Biq with ui(t) zero average, the trace[A] must always be less than zero for vibrational stabilization to be possible.) This trace condition is never satised for linearized versions of the mechanical systems treated in the following section, indicating one direction in which the theory has been considerably extended in recent years. Next, note that 2 is always positive, and therefore, the coefcient of the n 2th derivative in Eq. (14) can only be increased. The quantitites i, i 3 can be made either positive or negative; however, they depend on k2. Therefore, oscillatory control must enter through the a2 coefcient or else all i will be zero and vibrational stabilization will not take place.
which is now in a form that averaging can take place. Taking the average of Eq. (17) and converting back to regular time t leads to the equation corresponding to Eq. (11) of
0.6 dy = dt 0.8
1.3 0.4 2 y(t ) 1.6
(18)
The eigenvalues of Eq. (18) have negative real part when 2.5. The equilibrium point at zero remains unchanged. Therefore, for sufciently large (equivalently sufciently small 0) and for 2.5 the equilibrium xs 0 of Eq. (15) is vibrationally stabilized. Example 3. Oscillatory stabilization of a simple pendulum: Classical Averaging. Consider a simple pendulum consisting of a massless but rigid link of length to which a tip of mass m and inertia I is attached, and let denote the counterclockwise rotation about the vertical hanging conguration. Suppose the hinge point of the pendulum is forced to oscillate vertically, where the elevation of the hinge above some reference height at time t is given by R(t). An illustration of such a system is given in Fig. 2. Accounting for Rayleigh damping and gravitational forces, the pendulum dynamics can be b written sin + mg sin = 0 + b +m R I (19)
(t) sin t, where Suppose R(t) sin t. Then R () . Writing Eq. (19) as a system of rst order equa-
156
the averaged equation, then for sufciently large forcing frequencies there exists an asymptotically stable periodic orbit near the inverted equilibrium. A simple linearization of the averaged equation reveals that the stability condition for the inverted equilibrium is given by 22 2Ig / m. Remark 2. Note that in the absence of dissipative forces that the linearized averaged system will possess eigenvalues either of the form 1,2 where , or of the form 1,2 i, where i . Hence the system is never asymptotically stable in the absence of damping, and stability results in this case are weak. The lack of asymptotic stability is a characteristic of Hamiltonian, and more generally, conservative systems. The averaging technique of the next subsection is more suited to such systems and yields stronger stability results. Remark 3. Simple nonquantitative experiments demonstrating the stabilization described in this example are not difcult to build and work remarkably well. Such an experiment is shown in Fig. 3. In this experiment, the rotary motion of a dc motor is rectied to periodic linear motion by the mechanism shown in the left frame of Fig. 3. Note that by virtue of the construction, the forcing amplitude is xed and the forcing frequency can vary. It is observed that when current is applied to the motor, the inverted equilibrium is unstable until the forcing frequency reaches a critical frequency at which the inverted equilibrium experiences a bifurcation which renders it stable, as depicted in the right frame of Fig. 3. The inverted equilibrium is then stable for all higher frequencies. Remark 4. To this point, the main goal has been to use averaging as a means of studying the local stability properties of periodically excited systems. Under certain conditions, however, the averaged system gives far more information about the global structure of the periodically excited system. As essentially a perturbation technique, averaging theorems as found in (36,47,48) give no clues as to how large the small parameter can be perturbed off zero before the averaged dynamics fail to describe the forced dynamics. For sufciently large, a variety of undesirable nonlinear effects arise, such as subharmonic resonance and stochasticity, which are not captured in any way by the simple averaging of the nonautonomous dynamics. Because of the inherent difculty of the analysis, theory for the prediction of nonlinear effects in this range has been slow to emerge. A later section briey illustrates some of the features of periodically excited systems exhibit when is allowed to vary. Averaging for Mechanical Systems Recently, interest has emerged in using high frequency oscillatory forcing to control the dynamics of mechanical systems. The typical applications setting is a controlled Lagrangian system where only some of the degrees of freedom are directly controlled: d L L =u dt q 1 q1 d L L =0 dt q 2 q2 (20) (21)
R(t)
Figure 2. A simple pendulum whose hinge point undergoes vertical motion.
, it is clear that the rst order tions where x1 and x2 system can be written in the form of Eq. (7). Following the steps detailed in the previous section, the generating equation is found to be
x 1 = 0, x 2 =
which has the solution
m sin t sin x1 I
x1 = c1 = h1 (t , c ), x2 = m cos t sin c1 + c2 = h2 (t , c ) I
Introducing the transformation
x1 = z 1 , x2 = m cos t sin z1 + z2 I
letting t, and letting 1/ , Eq. (9) specializes to
z 1 = z 2 = +
m cos sin z1 + z2 I m I
2
cos2 cos z1 sin z1
mg sin z1 I
m m b b z cos cos z1 + cos sin z1 z2 I 2 I2 I
Therefore the averaged equations, given by Eq. (11), are
y 1 = y2 , y 2 = 1 2 m I
2
cos y1 sin y1
mg b sin y1 y2 I I
Notice that the averaging preserves the upper equilibrium, T and xs 1/ T 0 h(t, ys)dt. Therefore, by the previous discussion, if the inverted equilibrium is asymptotically stable for
157
where it is assumed dim q1 m and dim q2 n, and u is an m-vector of controls. (Systems of this form have been called super-articulated in the literature, and the reader is referred to (49) for details and references.) Within this class of models, it is further assumed that there is enough control authority to always be able to completely specify any trajectory q1( ) over an interval of interest. When this is the case, q1( ), q 1( ), and q 1( ) are viewed collectively as generalized inputs, and are used to control the dynamics of the conguration variables q2( ). The starting point may thus be taken to be a (generalized) control system [see Fliess (50) for an introduction to generalized control systems] prescribed by a Lagrangian L ( q, q ; x, v ) = If 1 T q M ( q , x )q + v T A ( q , x )q Va (q; x, v ) 2 (22)
L (q1 , q 1 ; q2 , q 2 ) = 1 (q T T 1,q 2) 2
M11 T M12
M12 M22
q 1 V (q1 , q2 ) q 2
is the Lagrangian associated with Eqs. (20) and (21), then with the identications q1 x, q 1 v, q2 q, M22 M , M12 A , and V a(q;x,v) V (x,qV ) VTM11V, the connection between the Lagrangian dynamics prescribed by Eq. (22) and Eq. (21) is clear. To simplify averaging, perform the usual Legendre transform H pq L , where p L / q , and write the resulting Hamiltonian in terms of the variables q, p; x, v H (q, p; x, v ) = 1 ( p A T v )T M 1 ( p M T v ) + Va 2 (23)
This quantity is not a proper Hamiltonian since in general H / t 0. It is remarkable that if the (generalized) input functions x( ), and v( ) x ( ) are restricted to be periodic and the simple average of H over one period is computed, (i) the resulting quantity H will itself be a proper Hamiltonian, and (ii) in many cases the dynamics associated with H will closely approximate the dynamics of the nonautonomous system prescribed by Eq. (23). Recall that the simple average is the time average over one period of H (q, p; x(t), v(t)) where q and p are viewed as variables which do not depend on the time t. The averaged Hamiltonian Eq. (23) can be written
H (q , p ) 1 T 1 1 p M p vT AM 1 p + vT AM 1 (M 1 ) 1 M 1 A T v 2 2 1 T 1 + v AM 1 A v vT AM 1 (M 1 ) 1 M 1 A T v + V 2 2 1 1 1 T = (M p M A v )T (M 1 ) 1 (M 1 p M 1 A T v ) 2 = averaged kinetic energy 1 1 + vT AM 1 A T v vT AM 1 (M 1 ) 1 M 1 A T v + V 2 2

Figure 3. A simple experiment to demonstrate the stabilization of the inverted equilibrium of the vertically forced pendulum. The picture on the left (a) shows the mechanism which recties the rotary motion of the dc motor into periodic linear motion. The picture on the right (b) shows the pendulum stabilized in the inverted equilibrium.
averaged potential (24)

The averaged potential given in Eq. (24) is an energy-like function of the generalized coordinates q which is abbrevi-
158
ated V A(q). A complete understanding of the relationship between the dynamics of nonautonomous Hamiltonian systems of Eq. (23) and the appropriate counterparts for averaged Hamiltonian S of Eq. (24) does not presently exist. There are very broad classes of such systems, however, for which it is possible to prove the validity of the following: Averaging Principle for Periodically Forced Hamiltonian Systems. The dynamics associated with Eq. (23) under periodic forcing (x(t), v(t)) are locally determined in neighborhoods of critical points of the averaged potential V A(q) as follows: If q* is a strict local minimum of V A( ), then provided the frequency of the periodic forcing (x( ), v( )) is sufciently high, the system will execute motions conned to a neighborhood of q*. If (q, p) (q*, 0) is a hyperbolic xed point of the corresponding averaged system (i.e., the Hamiltonian system determined by Eq. (24)), then there is a corresponding periodic orbit of the forced system such that the asymptotic stability properties of the xed point (q*, 0) of the averaged system coincide with the asymptotic stability properties of the periodic orbit for the forced system. This type of averaging for the analysis of periodically forced mechanical systems has been treated in (7) in the case in which M and A in Eq. (22) do not depend explicitly on the variable x. A detailed stability analysis based on Floquet theory appears in (8), but this is restricted to the case in which local minima of the averaged potential correspond to rest point of the nonautonomous dynamics. In the more general case, the motion of systems dened by Eq. (23) are organized around local minima of the averaged potential which are not rest points of Eq. (23). The theory is less well developed for this case, but (33) and (10) analyzes the class of single input systems and presents detailed results on the correspondence between averaged system phase portraits and the correspond maps of the nonautonomous system dened by ing Poincare Eq. (23). Example 4. Oscillatory stabilization of a simple pendulum: Averaged potential. This example illustrates the use of the averaged potential in analyzing the dynamics of a pendulum whose hinge point is forced to undergo oscillatory linear motion which is not necessarily vertical as in the last example. Suppose (x, y) gives the coordinates of the horizontal and vertical displacement of the hinge point of a pendulum attached to a sliding block which is controlled to execute the oscillatory motion
g
q0
Figure 4. Periodically forced pendulum where the forcing is directed along a line of angle with respect to the horizontal.
This equation may be derived from a Lagrangian of the form 2 + m cos ( )v(t ) , v ) = 1 I + mg cos L (, 2 where v(t) cos t. The averaged potential for this system is VA ( ) = (m )2 cos2 ( ) mg cos 4I
When /2, the hinge of the pendulum undergoes the vertical oscillation described in the previous example. The averaged potential has two or four critical points in the interval [0, 2) depending on whether or not 22 is less than or larger than 2Ig / m. Clearly the equilibrium is a strict local minimum of the averaged potential if and only if 22 2Ig / m. According to the theory of averaging presented in (7) and (8), the pendulum will execute stable motions conned to a neighborhood of this equilibrium for sufciently large values of . This also recovers the result obtained in the previous example. Remark 5. This example illustrates nonclassical behavior in the case /2. For this case there will be, for sufciently large values of , strict local minima of the averaged potential which are not equilibrium points of the nonautonomous Eq. (25). Nevertheless, the pendulum will still execute motions conned to neighborhoods of such local minima. For details on this type of emergent behavior, see (33) and (10). Remark 6. The strategy behind the very simple (open loop) control designs associated with the averaged potential (and more generally with systems having oscillatory control inputs) is to produce robustly stable emergent behavior which is related to the critical point structure of the averaged potential. The design method for control laws in this category involves designing the averaged potential functions themselves by means of appropriately chosen inputs. The guiding theory for this approach remains very much under development. Floquet Theory Another body of theory used in the study of the stability of equilibriums and periodic orbits of systems controlled by open-loop oscillatory inputs is Floquet theory. As described in Appendix 2, the central idea behind the theory is that the local stability of an equilibrium or periodic orbit may be deter-
x(t ) cos = sin t y(t ) sin

where prescribes the direction of the oscillatory motion, and is the frequency. This system is illustrated in Fig. 4. If, as in the last example, the pendulum has total mass m and inertia I about its hinge point, the motion under this oscillatory forcing is described by a second order differential equation m 2 cos ( ) sin t + mg sin = 0 I (25)
159
mined from the eigenvalues of the monodromy matrix M. The monodromy matrix represents the growth or decay of solutions of the linearized system, where the linearization is about that equilibrium or periodic orbit. In general, computing the monodromy matrix is not straightforward. The calculation is relatively easy, however, if the linearization of the system state equations is piecewise constant in t. For example, suppose that the linearized time-varying system is x = A(t )x where A(t) A1 on 0 t t, and A(t) A2 on t t T, such that A1A2 A2A1. Then the monodromy matrix M can be obtained by computing the state transition matrix on the interval [0, T]; that is
0.500
0.000 0.000
10.000
(T, 0 ) = =e
(t , 0 ) (T, t )
t 0 A 1 dt
T A dt 2 t
Figure 5. Regions of stability (darkened) and instability (light) for the vertically forced simple pendulum. In this gure, m g I 1.
2 2 1/ I [mg m2] 0 and 2 1/ I [mg where 1 m2] 0. Stability of the xed point may be determined from the eigenvalues 1, 2 of (0, 1/ ), which are the roots of the characteristic polynomial
=M
While in mechanical problems the assumption of piecewise constant forcing is somewhat nonphysical, such approximations are often useful when the forcing frequency is sufciently large. Piecewise constant forcing is often not a problem in the analysis electrical and electronic systems, where such forcing is common. Example 5. Oscillatory stabilization of a simple pendulum: Floquet theory. In the previous examples, it was shown that the inverted pendulum may be stabilized with high frequency vertical oscillation by averaging the time-varying equations and studying the stability of the inverted equilibrium with the averaged system. In this example, stabilization is inferred by linearizing the pendulum dynamics about the inverted equilibrium and studying the eigenvalues of the monodromy matrix. To facilitate the calculation, assume that the pendulum is forced by square wave forcing, that is, in Eq. (19) (t) 2u(t) where u(t) u(t 1/ ) is a square wave R which switches periodically between 1 and 1. Also, assume that the damping term b 0. This assumption simplies the following calculations and more importantly allows us to show that pendulum stabilization does not require dissipation. Linearizing Eq. (19) about the inverted equilibrium gives rise to the linear system 0 1 1 1 = 1 2 2 [mg m 2 u(t )] 0 I Because the input u(t) is piecewise constant, the state transition matrix (t, 0) over one period of u(t) may be computed as follows:
2 1
1 2
sin
sinh
+ 2 cos
cosh
+1 = 0
and the usual stability condition is that both eigenvalues must lie within the unit disk in the complex plane. Given the form of the characteristic polynomial, an equivalent condition for stability is trace (0, 1/ ) 2, which in this case may be written 2 cos
1
cosh
2 1
1 2
sin
sinh
<2
The boundary between regions of stability and instability may be approximated by expanding the trigonometric functions in Taylor series around zero and solving a truncated inequality for in terms of . Note beforehand that in doing so, it has been implicitly assumed that 1 /2 0 and 2 /2 which implies is large. For the present example, the stability regions displayed in Fig. 5 have been obtained by numerically calculating 1 and 2 over the indicated ranges of and . Remark 7. Note that in the absence of damping, (0, 1/ ) is an area-preserving map. This fact implies that 1 and 2 are constrained to either lie on the real axis such that 2 1/ 1, or lie on the unit disk in the complex plane. In this case, stability results are fairly weak (lack of asymptotic stability), but are typical of Hamiltonian, and more generally, conservative systems. In the presence of dissipation, the eigenvalues may occur as conjugate pairs inside the unit disk, implying asymptotic stability. Remark 8. Note that results obtained through Floquet theory are strictly local results. In the case where the controlled system is nonlinear, proofs of stability and instability are strictly for the equilibrium or periodic orbit, and no information is given about the asymptotic behavior of solutions nearby. This remark is not surprising for neutrally stable equilibria, but it is in fact sometimes true for unstable equilibria as well. As an example, the reader is referred to the parametrically excited pendulum example later in this article,
(0, 1/) (0, 1/2) (1/2, 1) 1 1 sin 1 sinh 2 cosh 2 cos 1 2 2 2 2 1 2 = 1 1 2 2 1 sin sinh cos cosh 2 2 2 2 2 1 2 2 1 1 cosh + sinh sin cos 2 2 2 2 1 = 1 2 1 cosh + 2 cos sinh 2 1 sin 2 2 2 2 1 1 1 2 sinh + cos sin 1 cosh 2 2 2 2 2 2 1 1 2 1 1 1 cosh sinh sin cos 2 2 2 2 2 =
160
where in Fig. 8(b) it is seen that while the origin is unstable, orbits which pass arbitrarily close to the origin are in fact bounded by KAM (for Kolmogorov, Arnold, and Maser) tori and subharmonic resonance bands. Understanding when proofs of instability obtained by Floquet theory imply unbounded solutions for such systems is a current topic of research.
It is seen that the net effect was to increase both the 2(1 n) and the 2n terms, which as a result, improved system performance by suppressing the betatron oscillations in both the x and z directions (this simplied model did not take into account the interaction of oscillations along x and z). Hence, performance has been improved via the introduction of oscillatory open-loop control, provided that is sufciently large. This example demonstrates that the benets of introducing oscillations can, at times, help system performance even when stabilization is not an issue. Analysis Method For Transient Behavior From the theory presented above, it is possible to approximate the transient behavior of Eqs. (7) and (9), using the techniques found in Refs. (38) and (25). Assume that Eq. (11) has an asymptotically stable equilibrium point, zs, in the vicinity of the equilibrium point of Eq. (2), xs, with u constant 0. By the method of averaging, for sufciently large , the solutions of Eqs. (8) and (11) remain arbitrarily close to each other provided they have the same initial conditions. That is, y(t) z(t) for all time. The solution to Eq. (7) is given by x(t) h(t, z(t)), where z(t) is the solution to Eq. (8). This transformation is a homeomorphism. Therefore, the quantities in Eq. (5) can be selected so that h(t, c) is the solution to the generating equation and P is as dened in Eq. (11). Now dene w h(t, y(t)) where y(t) is the solution to y (t) P(y(t)) in Eq. (11). For sufciently large , x(t ) h(t , y(t )) = w(t ) It is possible to analyze the transient behavior of Eq. (7) in two different manners: 1. Analyze the transient behavior of y(t) and then examine the behavior of x(t) through the direct relation x(t) w(t). This technique predicts many of the fast oscillatory parts of the trajectory x. 2. Analyze a moving average of x(t), given by x(t) where x(t ) H (z (t )); H (c ) 1 T
T
PERFORMANCE IMPROVEMENTS AND TRANSIENT BEHAVIOR Sometimes stability in a system is not in question, but certain performance specications are the design constraints. This section begins by revisiting the particle accelerator. Then a method to analyze the transient behavior of a system is given and followed by an example. Example 6. Suppression of betatron oscillations in cyclic accelerators. Consider the cyclic accelerator subject to oscillations of focusing and defocusing sectors of the magnetic lens given in Eq. (1). The work of (1) describes an experimental method of betatron oscillations by alternating gradient focusing. To make the focusing improve, it is desirable that both the 2(1 n) and the 2n terms increase simultaneously, with 0 n 1. Clearly, though, if n is viewed as the control variable, then it is not possible to both increase and decrease n and 1 n simultaneously. Instead, the technique of focusing and defocusing a lens is introduced, which, according to (2) is modelled
d2x + 2 [1 (n + K ( , ))] x = 0 d 2 d2z + 2 [n + K ( , ) ] z = 0 d 2
(26)
where K is a PAZ function with frequency ; for example, K(, ) () sin . Suppose that the oscillatory control input is given by K(, ) sin(), that is, () , where is a constant. Dene x1 x, x2 x , x3 z and x4 z . Then the state space representation of Eq. (26) becomes
dx1 d dx2 d dx3 d dx4 d
= x2 = 2 (1 n ) x1 + 2 = x4 = 2 nx3 2 sin( ) x3 sin( ) x1 (27)
h(, c ) d
0
(29)
which is precisely the form of Eq. (7). Using the previously discussed techniques, the equation corresponding to Eq. (11) becomes
and T is the period of h(t, ). This is done by approximating x(t) H(y(t)) and once again, analyzing the transient behavior of y (making this technique more typical of Eq. (4)). Since the fast dynamics in h are averaged out, this technique introduces some error. On the other hand, since H does not explicitly depend on t, the analysis becomes simpler. In either of the two methods, controlling y(t) in Eq. (11) governs how to control x(t). Example 7. Transient behavior of the vertically oscillating pendulum. Referring back to Example 3 and using the same notation, it is possible to now approximate the trajectories of
d y + 2 ( 1 n + 0 . 5 2 )y = 0 d 2 d 2 + 2 (n + 0.5 2 ) = 0 d 2
(28)
161
, the pendulums angular veloc, the angular position, and ity. The averaged equations of the system are
y 1 = y2 y 2 = 1 2 m I
2
where
H (y )
cos y1 sin y1 mg b sin y1 y2 I I
1 2 = y2
2 0
m cos s sin y1 + y2 ds I
The transformations utilized are
x1 = z 1 x2 = m cos t sin z1 + z2 I
Therefore, it is possible to obtain estimates on the transient behavior of the vertically oscillating pendulum using one of the two following methods: Method 1. The estimate on is given by y1. The esti is given by mate on m cos t sin y1 + y2 I
Note that the approximation obtained by Method 2 is merely the averaged system. The approximation obtained by Method 1, as well as the transient solution obtained from the original equation of motion, are compared to the averaged solution in Fig. 6. It is clear from the gure that the averaged system accurately captures the averaged behavior of the system as it stabilizes. The phase plot in the lower left of the gure shows some discrepancies between the trajectory obtained by Method 1 with the trajectory of the original system, but in the lower right gure it is seen that the Method 1 approxima is actually quite close to the velocity of the original tion of system. As Fig. 6 shows, Method 1 is more accurate than Method 2. However, Method 1 utilizes a time-varying output equation, making it a more complicated technique.
Method 2. The estimate on is given by y1. The esti is given by mate on H (y )
RESONANCE PHENOMENA IN PERIODICALLY FORCED SYSTEMS Resonances arise as the natural result of subjecting any system to periodic excitation. The resonances produced vary
1.5 1
Theta
1.5 Averaged system Method 1

Theta
1 0.5 0 0.5 1
Averaged system Original system
0.5 0 0.5 1
2.6
2.8
3 Theta
3.2
3.4
2.6
2.8
3 Theta
3.2
3.4
1.5 Method 1 Original system
1.5 1
Theta
1 Method 1 Original system
0.5 0 0.5 1
Theta
0.5
0.5
2.6
2.8
3 Theta
3.2
3.4
10 t
15
20
25
Figure 6. A comparison of the behaviors of the averaged/Method 2 system, the Method 1 system, and the original system. In these plots, m g I 1, b 0.5, 0.2, and 10.
162
which, if 1/ 2 and / 2, is recognizable as a nonlinear version of Mathieus equation

l(t) = + cos t k m F0cos t g q m
q + ( + sin t )q = 0 Note also that by letting 2, Eq. (30) is merely the equation of the motion of the vertically forced pendulum discussed in Examples 3, 4, and 5. In this example, however, it is assumed that 1. The natural tool for visualizing the global dynamics of single-degree-of-freedom periodically forced system is the Poin map, where Poincare sections are taken at the end of care every forcing period. Features of the phase portrait typically map. For example, have equivalent features in the Poincare xed points and periodic orbits in the phase portrait are asso map. The Poincare ciated with periodic points of the Poincare map also preserves hyperbolic structures of the phase portrait; for example, normally hyperbolic periodic points of the map reect the normally hyperbolic structure of the Poincare corresponding equilibriums and periodic orbits of the phase maps clearly show the bands of stochasportrait. Poincare ticity, indicating the presence of chaotic behavior and subharmonic resonances, which are periodic orbits periodic of some rational multiple of the forcing period. The results of simple simulations of the PEP are shown in Figs. 8(af). Using averaging techniques described previously, it can be shown that the averaged system is merely the unperturbed ( 0) system, the phase portrait of which is shown in Fig. 8(a). In general, in implementing an openloop oscillatory control law, an objective is to choose the forc map closely resembles ing parameters so that the Poincare the averaged phase portrait. In the present examples, there are three ways to systematically adjust and : (i) x and let vary, (ii) x and let vary, or (iii) let () or () and adjust the independent variable. Physically, it is often most reasonable to x the forcing amplitude and control the frequency, hence for the current example attention is restricted to this case. The ve other plots in Fig. 8 reect the changes which take map when is xed at 0.5 and inplace in the Poincare creases from 1 to 9. Passing from the upper right frame [(b), 1)] to the frames in the middle row [(c, d), 3, 5] to the lower frames [(e, f), 7, 9], the general trend is clearly that subharmonic resonances and separatrix splitting is suppressed as increases. In Fig. 8(b), the origin is unstable, and this instability would be predicted by a Floquet analysis of the origin. In Figs. 8(c)(f), the origin is stable, as indicated by the presence of KAM tori. What varies with the excitation is the minimum radius at which subharmonic resonance bands exist. Progressing from Figs. 8(b) to (f), observe that the inner resonance bands are being pushed out towards the separatrix as frequency increases. As a consequence, the region in which regular quasiperiodic ow dominates increases. In addition to pushing subharmonic resonance bands out, increasing frequency also has the effect of reducing the area of lobes formed by the intersection of stable and unstable manifolds of the two periodic points. The signicance of this observation is that as frequency increases, the set of initial conditions which are transported out of the region between the separatrices of the averaged system decreases. This is an important observation, because the averaged phase portrait
Figure 7. The periodically forced springmass system (left) and the parametrically excited pendulum (right).
based on the type of system being excited; for example the simple resonance produced by exciting the spring-mass system illustrated in the left frame of Fig. 7 is different from the parametric resonance exhibited by the parametrically excited pendulum illustrated on the right. The difference in the resonant behavior exhibited by the two systems is that the forced mass-spring system has a resonance at a single frequency, where the parametrically excited pendulum has resonances at subharmonics of the natural frequency, similar to those of Mathieus equation. As a nonlinear system, the parametrically excited pendulum also has resonances which form as the result of the breaking and tangling separatrix solutions, as map. visualized with the period-1 return map or Poincare In designing open-loop oscillatory controls for the type of stabilization described in this article, a primary objective should be to choose the forcing frequencies such that undesirable resonances are avoided. The main obstacle to prevent the control theorist from ensuring this using the methods presented in this article is that (i) averaging techniques do not generally capture phenomena of periods different than the period over which the system is averaged, and (ii) Floquet theory gives only local stability information for a single equilibrium or periodic orbit. Experience and the application of more powerful tools in the analysis of such systems has shown that these effects are often avoided by choosing the forcing frequency to be sufciently large (10). This observation is very consistent with the sufciently small conditions imposed by averaging theory, although the arguments used by averaging theory are for purely analytical reasons. The literature concerning the global dynamics of nonlinear systems excited by periodic inputs is vast, fairly technical, and tends to lie completely outside the boundaries of conventional control theory. In this section, only broad concepts and observations are presented in the form of an example, and the reader is referred to such texts as (36,51,52) for extensive overviews of the eld. Example 8. Qualitative features of parametrically excited pendulum dynamics. To illustrate the kind of responses which might arise in subjecting a nonlinear system to periodic excitation, consider the parametrically excited pendulum (PEP) with no damping, which is often described by the differential equation q + (1 + sin t ) sin q = 0 (30)
Remark 9. Note that after a rescaling of time t, Eq. (30) takes the form q + 1 + 2 sin sin q = 0 2
163
2 1.5 1 0.5
q
2 1.5 1 0.5
q
0 0.5 1 1.5 2 3 2 1 0 q 1 2 3
0 0.5 1 1.5 2 4 3 2 1 0 q 1 2 3 4
(a) Averaged phase portrait 2 1.5 1 0.5

q q
(b) Gamma = 0.5, omega = 1 2 1.5 1 0.5 0 0.5 1 1.5
0 0.5 1 1.5 2 3 2 1 0 q 1 2 3
0 q
(c) Gamma = 0.5, omega = 3 2 1.5 1 0.5

q q
(d) Gamma = 0.5, omega = 5 2 1.5 1 0.5 0 0.5 1 1.5
0 0.5 1 1.5 2 3 2 1 0 q 1 2 3
0 q
(e) Gamma = 0.5, omega = 7
(f) Gamma = 0.5, omega = 9
maps showing separatrix splitting and resonance bands for the parametriFigure 8. Poincare cally excited pendulum. The phase portrait for the unperturbed simple pendulum is shown in maps of the forced system to the right and underneath. the upper left frame, with Poincare maps are indicated in each plot. Note that the Poincare Forcing parameters used in the Poincare maps of the forced system more closely resemble the averaged phase portrait as the forcing frequency becomes large.
provides no information whatsoever on the existence of a set of initial conditions which lies inside the separatrices but are transported out. In (f), it is unlikely that any subharmonic resonances exist, and the stable and unstable manifolds of the periodic points have closed to form a barrier to transport as indicated by the averaged phase portrait in (a).
Remark 10. The example gives somewhat anecdotal evidence that choosing a sufciently high forcing frequency tends to suppress the negative features of periodic excitation. This has also been found to be the case in the cart and pendulum problem described in Examples 3, 4, and 5 [see (10)], the vertically forced rotating chain (33), and in an entire class of
164
periodically forced single-degree-of-freedom systems (33). Current work revolves around extending this understanding to other classes of single-degree-of-freedom systems and multi-degree-of-freedom systems. Remark 11. It has also been observed that dissipation tends to have benecial effects beyond guaranteeing the asymptotic stability of certain equilibriums. Dissipation generally has the effect of breaking phase space separatrices and imposing hyperbolicity on systems with elliptic structures. As a result, elliptic structures, such as KAM tori and resonance bands, are destroyed and initial conditions which, in the absence of dissipation, belong to a KAM torus or resonance limit on a xed point or periodic orbit. In addition, with sufcient dissipation, intersecting stable and unstable manifolds completely separate, giving rise to distinct basins of attraction. As with frequency, the extent to which dissipation helps eliminate undesirable nonlinear effects is largely dependent on its magnitude. In (10), it was seen for the cart and pendulum problem of Example 4 that there exists a minimum damping coefcient such that for all values less than this minimum value, manifold intersections and resonances persist. Recent work has suggested that the same issue arises in multi-degree-offreedom systems. Remark 12. Unfortunately, there is no known generally applicable rule-of-thumb for deciding what constitutes a sufciently large forcing frequency. Experiments, simulation, and preliminary analysis suggest that for many systems, a rule of thumb might be for the forcing frequency to be an order of magnitude larger than the largest natural frequency of the controlled system. This rule of thumb presents a problem for many-degree-of-freedom systems like rotating chains or innite dimensional systems like strings and beams, where natural frequencies tend to be very large. In addition, there exist counterexamples where the averaged phase portrait and map resemble each other for small forcing frequenPoincare cies, but possess completely different features at high frequencies. These topics represent the current focus of much of the research in this eld.
ing theory addresses the relationship between the original, time-varying Eq. (1.1) and the autonomous averaged system given by y = f0 ( y) where f 0 : n n is dened f 0 (y ) 1 T
T
(1.2)
f (s, y ) ds
0
and where y is treated as a constant in the integration. For sufciently small , solutions of Eq. (1.2) provide good approximations to solutions of Eq. (1.1). Since there are many mathematical tools that can be used to analyze and control the time-invariant system Eq. (1.2), the problem of determining the behavior of time-varying periodic system Eq. (1.1) has been greatly simplied. Specically, if it is assumed that x(t0) y(t0) then for sufciently small , the following statements hold: On any nite time interval, assuming the same initial conditions at time t0, the solutions to Eqs. (1.1) and (1.2) remain close to each other. As becomes smaller, then the approximation becomes better and tends to zero in the limit. If the solution to Eq. (1.2) approaches a uniformly asymptotically stable equilibrium point, then, under additional mild assumptions, the solutions to Eqs. (1.2) and (1.1) remain close to each other on innite time intervals. As becomes smaller, then the approximation becomes better and tends to zero in the limit. If Eq. (1.2) has a uniformly asymptotically stable equilibrium point, then Eq. (1.1) has an asymptotically stable periodic solution in the vicinity of this equilibrium point. For a detailed discussion on the theoretical framework of averaging, the reader is referred to (36,47,53,54). APPENDIX 2. FLOQUET THEORY
ACKNOWLEDGMENTS Brad Lehman gratefully acknowledges the support of the National Science Foundation through an NSF President Faculty Fellowship, grant CMS 9596268. John Baillieul would like to express gratitude for support from the United States Air Force Ofce of Scientic Research under grant F49620-96-10059. APPENDIX 1. CLASSICAL AVERAGING THEORY The classical method of averaging was originally developed for periodic systems of the form x = f (t , x ) (1.1)
Floquet theory is concerned with local stability for systems of the form x = A(t ) x (2.1)
where x n, A(t) : nn nn, and A(t T) A(t). Such systems arise as linearizations of Eq. (2) around an equilibrium or periodic orbit. As described in many standard texts on differential equations, the fundamental result upon which Floquet theory is built is that the fundamental matrix solution of Eq. (2.1) can be written as the product of a periodic matrix P(t) and a constant exponential growth or decay; that is, (t ) = P(t )e Bt where (t) is a fundamental matrix associated with Eq. (2.1), P(t) is a n n matrix periodic in t of period T, and eBt is the matrix exponential of a constant n n matrix B. By the periodicity of A(t), if (t) is a fundamental matrix, so is (t T)
where x n, f : n n, f (t T, ) f (t, ) and 0 1. For simplicity, assume that f has continuous second partial derivatives in its second argument. Classical averag-
165
and (t T) is linearly dependent on (t). Then there exists a constant n n matrix M such that (t + T ) = (t )M
11. S. Weibel, J. Baillieul, and B. Lehman, Equilibria and stability of an n-pendulum forced by rapid oscillations. In Proc. 36th IEEE CDC, pp. 11471152, San Diego, 1997. 12. A. Cinar, J. Deng, S. Meerkov, and X. Shu, Vibrational control of an exothermic reaction in a CSTR: theory and experiments. AIChE J., 33: 353365, 1987. 13. B. Lehman, Vibrational control of time delay systems. In J. Wiener and J. K. Hale, (eds.), Ordinary and Delay Equations, Pitman Research Notes in Mathematical Series 272. Harlow, Essex, England: Longman Scientic & Technical, 1992. 14. B. Lehman, I. Widjaya, and K. Shujaee, Vibrational control of chemical reactions in a CSTR with delayed recycle, J. Math. Anal. & Appl., 193: 2859, 1995. 15. J. E. Bailey, Periodic operation of chemical reactors: A review, Chem. Eng. Com., (1): 111124, 1973. 16. J. Fakhfakh and J. Bentsman, Experiment with vibrational control of a laser illuminated thermochemical system, ASME J. Dyn. Syst., Meas. Control, 110: 353360, 1990. 17. S. M. Osovets, Dynamic methods of retention and stabilization of plasma, Soviet Phys. - Usp., 17 (2): September 1974. 18. D. Tilbury, R. Murray, and S. Sastry, Trajectory generation for the n-trailer problem using goursat normal form, IEEE Trans. Autom. Control, AC40: 802819, 1995. 19. J. L. Speyer, A second variational theory for optimal periodic processes, IEEE Trans. Autom. Control, AC-29-2: 138148, 1984. 20. J. L. Speyer, Nonoptimality of steady-state cruise for aircraft. AIAA J., 14 (11): 16041610, 1976. 21. S. Weibel and J. Baillieul, Averaging and energy methods for robust open-loop control of mechanical systems. To appear in Essays on Mathematical Robotics. Berlin: Springer-Verlag, 1998. 22. S. Weibel and J. Baillieul, Oscillatory control of bifurcations in rotating chains. In Proc. 1997 ACC, Albuquerque, 27132717, 1997. 23. J. Baillieul, S. Dahlgren, and B. Lehman, Nonlinear control designs for systems with bifurcations and applications to stabilization and control of compressors. In Proc. 34th CDC, New Orleans, 30623067, 1995. 24. R. Bellman, J. Bentsman, and S. M. Meerkov, Vibrational control of nonlinear systems: Vibrational stabilizability, IEEE Trans. Autom. Control, AC-31: 710716, 1986. 25. B. Lehman, J. Bentsman, S. V. Lunel, and E. I. Verriest, Vibrational control of nonlinear time lag systems with bounded delay: averaging theory, stabilizability, and transient behavior, IEEE Trans. Autom. Control, AC-39: 898912, 1994. 26. B. Lehman and J. Bentsman, Vibrational control of linear time lag systems with bounded delay, IEEE Trans. Autom. Control, AC-37: 15761582, 1992. 27. J. Bentsman and K.-S. Hong, Vibrational stabilization of nonlinear parabolic systems with neumann boundary conditions, IEEE Trans. Autom. Control, AC-36: 501507, 1991. 28. J. Bentsman and K.-S. Hong, Transient behavior analysis of vibrationally controlled nonlinear parabolic systems with neumann boundry conditions, IEEE Trans. Autom. Control, AC-38: 1603 1607, 1993. 29. S. Lee, S. Meerkov, and T. Runolfsson, Vibrational feedback control: zero placement capabilities, IEEE Trans. Autom. Control, AC-32: 604611, 1987. 30. K. Shujaee and B. Lehman, Vibrational feedback control of time delay systems. In Proc. 34th IEEE CDC, 936941, 1995. 31. K. Shujaee and B. Lehman, Vibrational feedback control of time delay systems, IEEE Trans. Autom. Control, AC-42: 15291545, 1997.
where M eBT and B is a constant n n matrix. Without loss of generality, let t 0 and (0) I where I is the n n identity matrix. Then (T ) = IM = M and therefore M represents the rate of growth or decay of the solution. Stable solutions decay or at least remain bounded. Hence, the condition for an equilibrium to be stable is that all the eigenvalues i of M satisfy i 1. M itself is called the monodromy matrix, and eigenvalues i of M are called Floquet multipliers. Complex numbers i such that i eiT are called Floquet exponents. Floquet theory is a very classical method, and there is a vast literature describing the basic theory and applications. Some of the classic texts on the topic include Hale (47), Magnus and Winkler (55), Stoker (56), and Yakubovich and Starzhinskii (57). Floquet theory is most famous for its application in the study of Hills equation and Mathieus equation, which, because of their second order structure, bear special relevance in the stability of periodically forced mechanical systems. Hills equation is the topic of (55), and various classic applications to (electrical) engineering are given in (56). References (47) and (57) give comprehensive summaries of the fundamental theory as well as examples.
BIBLIOGRAPHY
1. M. S. Livingston, High Energy Accelerators, New York: WileyInterscience, 1954. 2. S. M. Meerkov, Principle of vibrational control: theory and applications, IEEE Trans. Autom. Control, AC-25: 755762, 1980. 3. N. N. Bogoliubov, Perturbation theory in nonlinear mechanics, Sb. Stroit. Mekh. Akad. Nauk Ukr. SSR, 14: 934, 1950. 4. N. N. Bogoliubov and Y. A. Mitropolsky, Asymptotic Methods in the Theory of Nonlinear Oscillators, International Monographs on Advanced Mathematics and Physics. New York: Gordon and Breach Science Pub., 1961. 5. P. L. Kapitsa, Dynamic stability of a pendulum with a vibrating point of suspension, Zh. Ehksp. Teor. Fiz., 21 (5): 588598, 1951. 6. A. Stephenson, On induced stability, Phil. Mag., (17): 765766, 1909. 7. J. Baillieul, Stable average motions of mechanical systems subject to periodic forcing. In M. J. Enos (ed.), Dynamics and Control of Mechanical Systems: The Falling Cat and Related Problems: Fields Institute Communications. 123, Providence, R.I.: American Mathematical Society, 1993. 8. J. Baillieul, Energy methods for the stability of bilinear systems with oscillatory inputs, Int. J. Robust Nonl. Cont., 285301, July 1995. Special Issue on the Control of Nonlinear Mechanical Systems. 9. S. Weibel, J. Baillieul, and T. J. Kaper, Small-amplitude periodic motions of rapidly forced mechanical systems. In Proc. 34th IEEE CDC, New Orleans, 533539, 1995. 10. S. Weibel, T. J. Kaper, and J. Baillieul, Global dynamics of a rapidly forced cart and pendulum, Nonl. Dyn., 13: 131170, 1997.
166
OPERATIONAL AMPLIFIERS 53. P. Lochak and C. Meunier, Multiphase Averaging for Classical Systems with Applications to Adiabatic Theorems. In Applied Mathematical Sciences. vol. 72, New York: Springer-Verlag, 1988. 54. J. A. Sanders and F. Verhulst, Averaging Methods in Nonlinear Dynamical Systems, Applied Mathematical Sciences. vol. 59, Berlin: Springer-Verlag, 1985. 55. W. Magnus and S. Winkler, Hills Equation. Tracts of Mathematics. vol. 20, New York: Interscience Publishers, 1966. 56. J. J. Stoker, Nonlinear Vibrations in Mechanical and Electrical Systems. New York: Interscience Publishers, 1950. 57. Y. A. Yakubovich and V. M. Starzhinskii, Linear Differential Equations with Periodic Coefcients, vols. 1, 2, Jerusalem: Keter Publishing House Jerusalem, Ltd., 1975.
32. L. Trave, A. M. Tarras, and A. Titli, An application of vibrational control to cancel unstable decentralized xed modes, IEEE Trans. Autom. Control, AC30: 9599, 1985. 33. S. Weibel, Applications of Qualitative Methods in the Nonlinear Control of Superarticulated Mechanical Systems, PhD thesis, Boston University, 1997. 34. J. M. Coron, Global asymptotic stabilization for controllable systems without drift, Math. Control, Signals, Syst., 5: 295312, 1992. 35. R. W. Brockett, Asymptotic stability and feedback stabilization. In R. W. Brockett, R. S. Millman, and H. J. Sussmann, eds., Differential Geometric Control Theory. Basel: Birkhau ser, 1983. 36. J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Vol. 42, Applied Mathematical Sciences. Berlin: Springer-Verlag, 1983. 37. J. Baillieul and B. Lehman, Open-loop control using oscillatory inputs. In W. S. Levine, ed., The Control Handbook, Boca Raton, FL: CRC Press and IEEE Press, 1996, pp. 967980. 38. R. Bellman, J. Bentsman, and S. M. Meerkov, Vibrational control of nonlinear systems: Vibrational controllability and transient behavior, IEEE Trans. Autom. Control, AC-31: 717724, 1986. 39. J. Baillieul, Geometric methods for nonlinear optimal control problems, JOTA, 25 (4): 519548, 1978. 40. J. Baillieul, Multilinear optimal control. In Proc. Conf. Geometry Control Eng.: NASA-Ames, Brookline, MA: Math. Sci. Press, Summer 1976. 41. G. W. Haynes and H. Hermes, Nonlinear controllability via lie theory, SIAM J. Control, 8 (4): 450460, 1970. 42. N. E. Leonard and P. S. Krishnaprasad, Motion control of driftfree, left invariant systems on lie groups, IEEE Trans. Autom. Control, 40 (9): 1995. 43. H. J. Sussman and W. Liu, Lie bracket extension and averaging: The single bracket case. In Z. Li and J. F. Canny (eds.), Nonholonomic Motion Planning. 109147. Boston: Kluwer Academic Publishers, 1993. 44. H. J. Sussman and W. Liu, Limits of highly oscillatory controls and approximation of general paths by admissible trajectories. In Proc. 30th IEEE CDC, December 1991. 45. S. M. Meerkov and M. Tsitkin, The effectiveness of the method of vibrational control for dynamic systems described by a differential equation of order n, Automation and Remote Control (translation of Avtomatika i Telemekhanika), 525529, 1975. 46. R. Bellman, J. Bentsman, and S. M. Meerkov, Stability of fast periodic systems, IEEE Trans. Autom. Control, AC-30: 289291, 1985. 47. J. K. Hale, Ordinary Differential Equations. Texts and Monographs in Pure and Applied Mathematics, Malabar, FL: Robert E. Krieger Publishing, 1969. 48. J. A. Sanders, On the fundamental theory of averaging, SIAM J. Math. Anal., 14: 110, 1983. 49. D. Seto and J. Baillieul, Control problems in superarticulated mechanical systems, IEEE Trans. Autom. Control, AC-39-12: 24422453, 1994. 50. M. Fliess, Generalized controller canonical forms for linear and nonlinear dynamics, IEEE Trans. Autom. Control, AC-35-9: 994 1001, 1990. 51. A. J. Lichtenberg and M. A. Lieberman, Regular and Chaotic Dynamics. Volume 38 of Applied Mathematical Sciences. Berlin: Springer Verlag, 1992. 52. S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos. In Texts in Applied Mathematics. vol. 2, Berlin: Springer Verlag, 1990.
B. LEHMAN S. WEIBEL
Northeastern University
J. BAILLIEUL
Boston University
OPEN REGION PROBLEMS. See INTEGRAL EQUATIONS. OPERATING SYSTEMS, NETWORK. See NETWORK OPERATING SYSTEMS.
OPERATING SYSTEMS, UNIX. See UNIX.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Optimal Control Standard Article Kazufumi Ito1 and Karl Kunisch2 1North Carolina State University, Raleigh, NC 2Karl-Franzens-Universitt Graz, Graz, Austria Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1029 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (291K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Descriptive Example and Basic Concepts Existence and Necessary Optimality Linear Quadratic Regulator Problem Dynamic Programming Principle and HamiltonJacobiBellman Equation Linear Quadratic Regulator Theory and Riccati Equations Numerical Methods Acknowledgments About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
364
OPTIMAL CONTROL
with initial conditions y ( 0 ) = y0 and d y (0 ) = v0 dt
OPTIMAL CONTROL
Optimal control theory is concerned with the development of techniques that allow one to control physical phenomena described by dynamical systems in such a manner that a predescribed performance criterion is minimized. The principal components of an optimal control problem are the mathematical model in the form of a differential equation, a description of how the control enters into this system, and a criterion describing the cost. The start of optimal control theory, as a mathematical discipline, dates back to the mid 1940s. The increasing interest in and use of methods provided by optimal control theory is linked to the rise of the importance of mathematical models in many diverse areas of scienceincluding chemistry, medicine, biology, management, and nanceand to ever increasing computing power, which allows the realization of optimal control strategies for practical systems of increasing difculty and complexity. While optimal control theory has its roots in the classical calculus of variations, its specic nature has necessitated the development of new techniques. In contrast with general optimization problems, whose constraints are typically described by algebraic equations, the constraints in optimal control problems are given by dynamical systems. The dynamic programming principle, the Pontryagin maximum principle, the HamiltonJacobiBellman equation, the Riccati equation arising in the linear quadratic regulator problem, and (more recently) the theory of viscosity solutions are some of the milestones in the analysis of optimal control theory. Analyzing an optimal control problem for a concrete system requires knowledge of the systems-theoretic properties of the control problem and its linearization (controllability, stabilizability, etc.). Its solution, in turn, may give signicant additional insight. In some cases, a suboptimal solution that stabilizes the physical system under consideration may be the main purpose of formulating an optimal control problem, while an exact solution is of secondary importance. In the rst section we explain some of the concepts in optimal control theory by means of a classical example. The following sections describe some of the most relevant techniques in the mathematical theory of optimal control. Many monographs, emphasizing either theoretical or control engineering aspects, are devoted to optimal control theory. Some of these texts are listed in the bibliography and reading list.
Here y(t) is the angular displacement, m is the mass, and g is the gravitational acceleration. Further, u(t) represents the applied force, which will be chosen from a specied class of functions in such a way that the system described by Eq. (1) behaves in a desired way. We refer to y and u as the state and control variables. Due to the appearance of the sine function, Eq. (1) constitutes a nonlinear control system. It will be convenient to express Eq. (1) as a rst-order system. For this purpose, we dene x(t) col(x1(t), x2(t)), where x1(t) y(t) and x2(t) (d / dt)y(t). Then we obtain the rst-order form of Eq. (1), which is of dimension n 2: d dt x1 (t ) x2 (t ) = x2 (t ) g sin x1 (t ) + u(t ) (2)
with initial condition x(0) x0 (y0, v0) R2, where we assume m 1. In general, a control system is written in the form d x(t ) = f (t , x(t ), u(t )), dt x ( 0 ) = x0 (3)
with state vector x(t) Rn, control input u(t) Rm, and f : R1 Rn Rm Rn. If f is independent of time t [f f (x, u)], then the system is said to be autonomous. Next we formulate a sample control system associated to Eq. (1). For that purpose, note that the stationary solutions to the uncontrolled system, which are characterized by f (x, u) 0 for u 0, are given by (0, 0) and (, 0). Our objective is to regulate the state x(t) R2 to the stationary state (, 0). Thus a control u must be determined that steers the system described by Eq. (1) from the initial state x0 to the vertical position (y ) or into its neighborhood (inverted-pendulum problem). This objective can be formulated as an optimal control problem: minimize the cost functional
J (x, u ) =
0
tf
[|x1 (t ) |2 + |x2 (t )|2 + |u(t )|2 ] dt
(4)
+ [|x1 (t f ) |2 + |x2 (t f )|2 ]

subject to Eq. (2), over u L2(0, tf; R1), the space of squareintegrable functions on (0, tf). The nonnegative constants and are the weights for the control cost and target constraint at the terminal time tf 0, respectively. The integrand x1(t) 2 x2(t)2 describes the desired performance of the trajectory [the square of distance of the current state (x1(t), x2(t)) to the target (, 0)]. The choice of the cost functional J contains a certain freedom. Practical considerations frequently suggest the use of quadratic functionals. A general form of optimal control problems is given by min
0 tf
DESCRIPTIVE EXAMPLE AND BASIC CONCEPTS Control Problem We consider the controlled motion of a pendulum described by m d2 y(t ) + mg sin y(t ) = u(t ), dt 2 t>0 (1)
f 0 (t , x(t ), u(t )) dt + g(t f , x(t f ))
(5)
subject to Eq. (3), over u L2(0, tf; Rm) with u(t) U a.e. in (0, tf), where U is a closed convex set in Rm describing con-
OPTIMAL CONTROL
365
straints that must be observed by the class of admissible controls. In the terminology of the calculus of variations, Eq. (5) is called a Bolza problem. The special cases with f 0 0 and with g 0 are referred to as the Lagrange and the Mayer problem, respectively. If tf 0 is nite, then Eq. (5) is called a nite-time-horizon problem. In case g 0 and tf , we refer to Eq. (5) as an innite-time-horizon problem. The signicance of the latter is related to the stabilization of Eq. (3). If Eq. (5) admits a solution u*, we refer to it as the optimal control, and the associated state x* x(u*) is the optimal trajectory. Under certain conditions, the optimal control can be expressed as a function of x*, that is, u*(t) K(t, x*(t)) for an appropriate choice of K. In this case u* is said to be given in feedback or closed-loop form. If the nal time tf itself is a free variable and f 0 1, then Eq. (5) becomes the time optimal control problem. For certain analytical and numerical considerations, the treatment of the fully nonlinear problem (5) can be infeasible or lengthy. In these cases, a linearization of the nonlinear dynamics around nominal solutions will be utilized. Linearization We discuss the linearization of the control system (3) around stationary solutions. Henceforth x stands for a stationary solution of f (x, u) 0, where u is a nominal constant control. Let A Rnn and B Rnm denote the Jacobians of f at (x, u); that is, A = f x (x, u) and B = f u (x, u ) (6)
at x (0, 0) and A= 0 g 1 0 and B = 0 1 (6b)
at x (, 0). The linearized control system for the inverted pendulum is given by d x(t ) = dt 0 g 1 x(t ) + 0 0 u(t ) 1
where x1(t) is the relative angle from . Stability In this and the following subsection we restrict our attention to linear control systems of the form (7). One of the main objectives of optimal control is to nd controls in state feedback form u(t) Kx(t) with K Rmn such that the closed-loop system d x(t ) = Ax(t ) BKx(t ) = (A BK )x(t ) dt (8)
is asymptotically stable, in the sense that x(t)Rn 0 as t for all x0 Rn. Recall that a system of the form d x(t ) = Ax(t ) dt is asymptotically stable if and only if all eigenvalues of the matrix A satisfy Re 0. For example, for the matrices in Eqs. (6a) and (6b) we have
Dening z(t) x(t) x and v(t) u(t) u and assuming that f is twice continuously differentiable, Eq. (3) can be expressed as d z (t ) = Az (t ) + Bv(t ) + r(z (t ), v(t )) dt where the residual dynamics r satisfy |r(z (t ), v(t ))|R n const. [|z (t )|2 + |v(t )|2 ] This implies that the residual dynamics r are dominated by the linear part Az(t) Bv(t) if (z(t), v(t)) is sufciently small. We obtain the linearization of the control system (3) around (x, u): d x(t ) = Ax(t ) + Bu(t ), dt x ( 0 ) = x0 x (7)
det(I A ) = 2 + g = 0
= { gi} (marginal stability)
and det (I A ) = 2 g = 0 = { g} (instability)
respectively. In particular, this implies that the uncontrolled inverted-pendulum problem is unstable in the sense of Liapunov stability theory. For the closed-loop feedback system (8) associated with the inverted pendulum with feedback matrix chosen in the form K = (0 ) R1 2
we nd that the eigenvalues of A BK are given by ( 2 4g) and hence the closed-loop system is asymptotically stable for appropriately chosen 0. Moreover, if we apply the feedback u(t) Kz(t) with z(t) x(t) (, 0) to the original system (2), then the closed system is locally asymptotically stable by Liapunov stability theory. Linear Quadratic Regulator Problem In order to construct the optimal stabilizing feedback law u(t) Kx(t) for the linear system (7), we consider the innite-time-horizon linear quadratic regulator (LQR) problem
where now x(t) and u(t) represent the translated coordinates x(t) x and u(t) u, respectively. We refer to Eq. (7) as a linear control system. For the example of the pendulum we nd A= 0 g 1 0 and B = 0 1
(6a)
min
0
[xt (t )Qx(t ) + ut (t )Ru(t )] dt
(9)
366
OPTIMAL CONTROL
subject to Eq. (7), where Q, G Rnn are symmetric nonnegative denite matrices and R Rmm is symmetric and positive denite. The optimal solution u*( ) to Eq. (9) is given in feedback form by u (t ) = R 1 Bt Px (t ) where the optimal trajectory x*( ) satises d x (t ) = (A BR 1 Bt P )x (t ), dt x ( 0 ) = x0
the initial and terminal times. Also we shall not systematically discuss the bangbang principle, which states that, for certain control systems with controls constrained to lie in a convex compact set, the optimal controls are achieved in the extremal points of the admissible control set. Existence of Optimal Controls The problem of the existence of admissible controltrajectory pairs (x, u) and of solutions to the optimal control problem (12)(15) has stimulated a signicant amount of research. Here we can only give the avor of some of the relevant aspects required to guarantee existence of optimal controls. Let us assume in this subsection that tf is xed. Then the optimal control problem can be stated in the form of a nonlinear mathematical programming problem for (x, u) H1(0, tf, Rn) L2(0, tf, Rm): min J (x, u ) (16)
(10)
and the symmetric nonnegative denite matrix P Rnn satises the matrix Riccati equation At P + PA PBR 1 Bt P + Q = 0 (11)
In the section titled Linear Quadratic Regulator Problem we shall return to a detailed discussion of this equation and its signicance in optimal control theory. EXISTENCE AND NECESSARY OPTIMALITY In this section we consider the optimal control problems of Lagrange type min J (x, u ) =
0 tf
subject to the equality constraints
d x(t ) f (t , x, u ) E (x, u ) = dt =0 (t f , x(t f ))

and u K = {u(t ) U a.e. in (0, t f )}
(17)
f 0 (t , x(t ), u(t )) dt
(12)
(18)
subject to the dynamical system d x(t ) = f (t , x(t ), u(t )) dt control constraints u(t ) U (a closed set in Rm ) (14) (13) with K a closed convex subset of L2(0, tf; Rm) and E considered as a mapping from H1(0, tf; Rn) L2(0, tf; Rn) L2(0, tf; Rn) Rp. Question (a) above is equivalent to the existence of feasible points satisfying the equality and control constraints (17) and (18). The existence of optimal controls can be argued as follows: Under an appropriate assumption the state function x x( , u) L2(0, tf; Rn) can be dened as the unique solution to Eq. (13) with initial condition x(0) x0, so that the control problem (12)(15) can be written as min J (x(u ), u ) over u K with (t f , x(u )(t f )) = 0 (19)
and initial and target constraints x ( 0 ) = x0

2 m
and (t f , x(t f )) = 0
n m n 0
(15)
over u L (0, tf; R ), where f : R R R R , f : R Rn Rm R, and : R Rn Rp are C1 functions. In contrast with Eq. (5), we generalize the control problem in that we restrict the trajectory to reach a target described by the manifold (tf, x(tf)) 0. If, for example, tf 0 is free, f 0(t, x, u) 1, and (t, x) x, then the objective is to bring the system to rest in minimum time. Typical forms of the control constraint set U are given by U u Rm : u and U u Rm : ui 0, 1 i m. In order to obtain a rst insight into the problem of (12) (15), one needs to address the questions of (a) the existence of admissible candidates (x, u) satisfying Eqs. (13)(15), (b) the existence and uniqueness of solutions to the optimal control problem (12)(15), and (c) necessary optimality conditions. In the remainder of this section we shall present some of the ideas that were developed to answer these questions. For detailed information we refer to the bibliography and reading list and to additional references given in the listed works. In spite of their importance, in practice we shall not consider problems with constraints on the trajectories except at
Suppose that the admissible control set K is compact, that is, every bounded sequence in K has a strongly convergent subsequence in K, and the solution map u L2(0, tf; Rm) (x(u), x(u)(tf)) L2(0, tf, Rn) Rn is strongly continuous. Moreover, assume that the functional J is lower semicontinuous, that is, J (lim xn , lim un ) lim inf J (xn , un ) for all strongly convergent sequences (xn, un) in L2(0, tf, Rn) L2(0, tf, Rm). Then the control problem (12)(15) has a solution. In fact, let inf J(x(u), u) over u K with (tf, x(u)(tf)) 0, and let un be a minimizing sequence, that is, J(x(u1), u1) J(x(u2), u2) , with limn J(x(un), un) and the constraints in Eq. (19) are satised. Due to the compactness assumption for K, there exists a subsequence unk of un such that unk u* for some u* K. The continuity assumption for the control to solution mapping implies that
OPTIMAL CONTROL
367
(tf, x(u*)(tf)) 0, and from the semicontinuity of J it follows that

J (x(u ), u ) = J (lim x(un k ), lim un k ) lim inf J (x(un k ), un k ) =
This implies that J(x(u*), u*) and u* is a solution to Eqs. (12)(15). Alternatively to the compactness assumption for K, we may assume that either limu J(x(u), u) , or that K is bounded. We then also require that the solution map u L2(0, tf; Rm) (x(u), x(u)(tf)) L2(0, tf; Rn) Rn be continuous when L2(0, tf; Rm) is endowed with the weak topology and that the functional J is weakly sequentially lower semicontinuous. Then, using arguments similar to the ones above, the existence of a solution to Eqs. (12)(15) again follows. Pontryagin Maximum Principle An important step toward practical realization of optimal control problems is the derivation of systems of equations that must be satised by the optimal controls and optimal trajectories. The maximum principle provides such a set of equations. It gives a set of necessary optimality conditions for the optimal control problem (12)(15). We shall require the Hamiltonian associated with Eqs. (12)(15) given by = 0 f 0 (t , x, u ) + f (t , x, u ) H (t , x, u, ) (0, ) R Rn. where Theorem 1. Assume that f 0, f , are sufciently smooth, and suppose that (x*, u*) minimizes the cost functional in Eq. (12) (t) (0, (t)) subject to Eqs. (13)(15). Then there exists (t) never vanishes on [0, tf], and Rn1 with 0 0 such that 1. Maximum condition: t )) H (t , x (t ), u, ( t )) H (t , x (t ), u (t ), ( for all u U 2. Adjoint equation: d (t )) (t ) = Hx (t , x (t ), u (t ), dt 3. Transversality: t f )), (t f )) T f (H (t f , x (t f ), u (t f ), ( where Ttf is the tangent space to the manifold described by (t, x) 0 at (tf, x*(tf)). *) that satises the concluAn admissible triple (x*, u*, sions of Theorem 1 is called an extremal element. The function x* is called the extremal trajectory, and u* is called the extremal control. Remarks 1. The maximum principle provides a necessary condition for optimality. It is simple to nd examples illustrating (20)
the fact that it is in general not a sufcient optimality *) can be an extremal elecondition, that is, (x*, u*, ment without (x*, u*) being a solution to the control problem in (12)(15). 2. We refer to the literature for the proof of the maximum principle. A proof is sketched in the next subsection. We also mention that the following fact plays an essential role. Let s [0, tf], and consider the problem (12)(15) with initial time 0 replaced by s and initial condition x(s) x*(s). Then the optimal statecontrol pair restricted to [s, tf] is optimal for the control problem starting at s with x(s) x*(s). 3. Suppose tf is xed, that is, 0(t) t tf and that the target constraint is described by p additional conditions i(x) 0, i 1, . . ., p. Then the transversality condition can be expressed as
t f )), (t f )) (H (t f , x (t f ), u (t f ), ( = 0 (1, 0, . . ., 0 ) + (0, x (x (t f ))

for some (0, ) R Rp. Here we set x(x*(tf)) p i1 i grad i(x*(tf)). 4. If one can ascertain that 0 0 (normality), then without loss of generality we can set 0 1, and conditions 23 of Theorem 1 can be equivalently expressed as d (t ) = Hx (t , x , u , ) dt (t f ) = x (x (t f )) (21)
If tf is xed and no other target constraints are given, then normality holds. In fact, from the adjoint equation and the transversality condition we have d 0 (t ) = 0 f x (t , x (t ), u ) (t ) f x (t , x (t ), u (t )) dt with (tf) 0. If 0 were 0, then (t) 0 on [0, tf], which gives a contradiction. 5. The maximum principle is based on rst-order information of the Hamilton H. Additional assumptions involving, for example, convexity conditions or second-order information are required to ascertain that a pair (x*, u*) satisfying conditions 13 of Theorem 1 is, in fact, a solution to the problems in Eqs. (16)(18). Some sufcient optimality conditions are discussed in the next two subsections. Example. We conclude this subsection with an example. Let us denote by x1, x2, x3 the rates of production, reinvestment, and consumption of a production process. Dynamical constraints are given by d x (t ) = x2 (t ), dt 1 x1 ( 0 ) = c > 0
and it is assumed that x1 x2 x3 and xi 0 for i 1, 2, 3. The control function is related to the state variables and is chosen as u(t) x2(t)/ x1(t). The objective consists in maximizing the total amount of consumption given by
T
=
0
x3 (t ) dt
368
OPTIMAL CONTROL
on the xed operating period [0, T] with T 1. Setting x x1, this problem can be formulated as a Lagrange problem:
T
the problem (22), it follows that the optimal control is given by
min
J=
0
[u(t ) 1]x(t ) dt
(22)
u (t ) =
1 on [0, T 1] 0 on (T 1, T ]
subject to d x(t ) = u(t )x(t ), dt x(0 ) = c and u(t ) U = [0, 1]
Lagrange Multiplier Rule Here we present a necessary optimality condition based on the Lagrange multiplier rule and establish the relationship to the maximum principle. As in the section titled Existence of Optimal Controls, it is assumed that tf is xed. We recall the denition of E in Eq. (19) and dene the Lagrange functional L : H1(0, tf, Rn) L2(0, tf, Rm) L2(0, tf, Rn) Rp R given by L(x, u, , ) = J (x, u ) + ((, ), E (x, u ))L 2 (0,t
f
To apply the maximum principle, note rst that x(t) 0 on [0, T] for all admissible control u. The Hamiltonian H is given by H = 0 (u 1 )x + ux the adjoint equation is d = Hx = 0 (u 1 ) u dt and the transversality condition implies that (T) 0. Since (0, (t)) 0 on [0, T], it follows that 0 0. Thus normality of the extremals holds, and we set 0 1. The maximum condition implies that [1 u (t )]x (t ) + (t )x (t )u (t ) (1 u )x (t ) + (t )x (t )u for all u [0, 1] Since necessarily x*(t) 0, the sign of (t) 1 determines u*(t), that is,
,R n ) R p
1 Further dene HL (0, tf, Rn) as the set of functions in H1(0, tf, n R ) that vanish at t 0. We have the Lagrange multiplier rule:
Theorem 2. Assume that (x(u*), u*) minimizes the cost functional in Eq. (16) subject to Eqs. (17) and (18) and that the regular point condition
1 0 int{E (x(u ), u )(h, v u ) : h HL (0, t f , Rn ) and v K } (23)
holds. Then there exists a Lagrange multiplier (, ) L2(0, tf, Rn) Rp such that
Lx (h ) = Jx (x , u )h + ((, ), Ex (x , u ))h = 0
1 for all h HL ( 0, t f , R n )
(24)
(Lu , u u ) 0
for all u K
u (t ) =
1 if (t ) 1 > 0 0 if (t ) 1 0
where the partial derivatives Lx and Lu are evaluated at (x*, u*, , ). Let us establish the relationship between the Lagrange multiplier (, ) and the adjoint variable of the maximum principle. From the rst line in Eq. (24) one deduces
tf 0 0 [ fx (t , x , u ) f x (t , x , u )]h +
The adjoint equation is therefore given by d = (1 )u 1, dt (T ) = 0
We can now derive the explicit expression for the extremal elements. Since is continuous, there exists a 0 such that (t) 1 on [, T]. On [, T] we have u*(t) 0. It follows that (t) T t on [, T], and hence reaches 1 at T 1. Since (d / dt)() 1 and (d / dt)() 0, there exists an such that (d / dt) 0 on [, ]. This implies that (t) 1 and thus u*(t) 1 on [, ], and consequently (t ) = e (t ) and x (t ) = et on [, ]
d h dt dt + x (x (t f ))h(t f ) = 0
1 (0, tf; Rn). An integration-by-parts argument imfor all h HL plies that
tf 0
t ft
0 [ fx (s, x , u ) f x (s, x , u )] ds
+ x (x (t f )) +
and thus (t ) =
t tf
d h(t ) dt = 0 dt
for some 0. Now we can argue that is necessarily 0 and that on [0, ] u (t ) = 1,
x (t ) = ce ,
t
and (t ) = e
(t )
0 [ f x (s, x , u ) + f x (s, x , u )] ds x (x (t f ))
We have thus derived the form of the only extremal on [0, T]. Since one can easily argue the existence of a solution to
a.e. in (0, tf). If f and f 0 are sufciently regular, then H1(0, tf, Rn) and (, ) satisfy Eq. (21).
OPTIMAL CONTROL
369
For certain applications, a Hilbert space framework may be too restrictive. For example, f (u) sin u2 is well dened but not differentiable on L2(0, tf; R). In such cases, it can be more appropriate to dene the Lagrangian L on W1,(0, tf; Rn) L(0, tf; Rn) L(0, tf; Rn) Rp. Let us briey turn to sufcient optimality conditions of second order. To simplify the presentation, we consider the case of minimizing J(x, u) subject to the dynamical system (13) and initial and target constraints [Eq. (15)], but without constraints on the controls. If (x*, u*) satisfy the maximum principle and f 0, f are sufciently regular, then Huu (t ) 0 for t [0, t f ]
pose we introduce additional scalar components for the dynamical system and for the target constraint by d x = 0 and p+ 1 (x ) = g(x ) txn+ 1 dt n+ 1 where x (x1, . . ., xn) as before, and the augmented cost functional is f0 = f 0 (t , x, u ) + xn+ 1 We nd
tf 0
where H(t) H(t, x*(t), u*(t), (t)). A basic assumption for second-order sufcient optimality is given by the Legendre Clebsch condition Huu (t ) < 0 for t [0, t f ] (25)
f0 (t , x(t ), u(t )) dt =
0
tf
f 0 (t , x(t ), u(t )) dt + t f xn+ 1 f 0 (t , x(t ), u(t )) dt + g(x(t f ))
=
0
tf
This condition, however, is not sufcient for u* to be a local minimizer for the control problem in Eqs. (12)(15). Sufcient conditions involve positivity of the Hessian of the Lagrange functional L at (x*, u*, , ) with (, ) as in Theorem 2. This condition, in turn, is implied by the existence of a symmetric solution Q to the following matrix Riccati equation:
For the augmented system, the initial conditions are x(0) x0, while xn1(0) is free. The maximum principle can be generalized to allow for trajectories that are constrained to lie on an initial manifold M0 Rn1. In this case a transversality condition at t 0 must be added to part 3 of Theorem 1: 0 )), (0 )) T0 (H (0, x (0 ), u (0 ), ( where T0 is the tangent space to M0 at (0, x*(0)). For the Bolza problem the initial manifold is characterized by t 0 and x x0 0, and thus the transversality condition at t 0 implies n1(0) 0. The adjoint condition 2 of Theorem 1 turns out to be
= Q f x (t ) f x (t )T Q + Hxx (t ) Q T [Q f u (t ) Hxu (t )]Huu (t ) 1 [ f u (t )Q Hxu (t )] Q(t ) = (x(t )) on ker ( x ( t )) xx x f f f
(26)
where f (t) f (t, x*(t), u*(t)). We have the following result: Theorem 3. Let (x*, u*) denote a pair satisfying Eqs. (13) and (15), and assume the existence of a Lagrange multiplier (, ) in the sense of Theorem 2 with U Rm. If, further, f and f 0 are sufciently regular, Eq. (25) holds, and Eq. (26) admits a symmetric C1-solution, then u* is a local solution of Eq. (12). Moreover, there exist c 0 and 0 such that J (x, u ) J (x , u ) + c|(x, u ) (x , u

d (t )) (t ) = Hx (t , x (t ), u (t ), dt d (t ) = 0 , dt n+ 1 n + 1 (0 ) = 0
where the Hamiltonian H is dened by Eq. (20), and the transversality condition 3 of Theorem 1 is given by (H (t f ), (t f )) + 0 (0, gx (x(t f ))) Tt
f
)| 2 L 2 ( 0 ,t f ; R n + m )
for all (x, u) satisfying Eq. (13), Eq. (15), and (x, u) (x*, u*)L(0,tf ;Rnm) . The fact that perturbations (x, u) are only allowed in L so as to obtain an L2 bound on variations of the cost functional is referred to as the two-norm discrepancy. Bolza Problem Here we discuss the maximum principle for a Bolza type problem, where the cost functional of Eq. (12) is replaced by min J (x, u ) =
0 tf
If we assume that tf xed and that no target constraints at tf are present, then normality holds and conditions 23 of Theorem 1 can be expressed as d ) (t ) = Hx (x u , dt (t f ) = gx (x (t f )) (27)
For a restricted class of Bolza problems, the maximum principle provides a sufcient optimality condition. We have the following result: Theorem 4. Consider the Bolza problem min
0 tf
[ f 0 (t , x(t )) + h0 (t , u(t ))] dt + g(x(t f ))
f 0 (t , x(t ), u(t )) dt + g(x(t f ))
subject to d x(t ) = Ax(t ) + h(t , u(t )), dt x(t ) = x0
with g : Rn R. Augmenting the system (13), the Bolza problem can be expressed as a Lagrange problem. For this pur-
370
OPTIMAL CONTROL
and the control constraint u(t) U, where tf is xed and g and f 0 are C1 functions that are convex in x. If (0 1, (t), x*(t), u*(t)) is extremal, then u* is optimal. Proof. From the maximum condition h (t , u (t )) + (t )h(t , u (t )) h (t , u ) + (t )h(t , u )
0 0
subject to the linear control system d x(t ) = Ax(t ) + Bu(t ) + f (t ), dt x ( 0 ) = x0 (30)
for all u U where (t) satises d 0 (t ) = f x (t , x (t )) (t )A, dt (t f ) = gx (x (t f ))
(28)
where f (t) L2(0, tf; Rm) represents a disturbance or an external force, Q, G Rnn are symmetric and nonnegative matrices, and R Rmm is symmetric and positive denite. This problem is referred to as the nite-time-horizon linear quadratic regulator problem. The Hamiltonian H is given by H = 1 (xt Qx + ut Ru ) + (Ax + Bu + f (t )) 2 where we have used the fact that 0 1 established in the last subsection. From Eq. (27) we obtain the form of the adjoint equation for (t): d (t ) = (t )A + xt (t )Q, dt (t f ) = xt (t f )G
For all admissible pairs (x, u) satisfying d x(t ) = Ax + h(t , u ), dt we have x ( 0 ) = x0
d d d 0 (x ) = x + x = ( f x A )x + (Ax + h ) dt dt dt 0 (, x )x + h(, u ) = fx
Combined with Eq. (28), this implies d d 0 0 (x ) f x (x ) f x ( , x )x h 0 ( , u ) ( , x )x h 0 ( , u ) dt dt Integration of this inequality on [0, tf] yields
and the maximum condition implies that u (t ) = R 1 Bt t (t ) Thus the maximum principle reduces to a two-point boundary value problem. If we dene p , then the optimal triple (x, u*, p) is characterized by u*(t) R1Btp(t) and
(t f )x (t f )
tf 0
0 [ fx (t , x )x + h0 (t , u )] dt tf 0 0 [ fx (t , x )x
d x(t ) = Ax(t ) BR 1 Bt p(t ) + f (t ), x ( 0 ) = x0 dt d p(t ) = At p(t ) Qx(t ), p(t f ) = Gx(t f ) dt
(31)
(t f )x(t f )
+ h (t , u )] dt
0
In the section titled Linear Quadratic Regulator Theory and Riccati Equations we discuss the solution to Eq. (31) in terms of matrix Riccati equations. There we shall also consider the innite-time-horizon problem with tf . Time Optimal Control
By Eq. (27), the last inequality implies
gx (x (t f ))[x(t f ) x (t f )] +
tf 0 tf 0
0 fx (t , x )(x
x ) dt
[h0 (t , u ) h0 (t , u )] dt
Note that (x) (x*) x(x*)(x x*) for all x, x* for any convex and C1 function . Since g, f 0 are convex, we have
A time optimal control problem consists of choosing a control in such a way that a dynamical system reaches a target manifold in minimal time. Without control constraints, such problems may not have a solution. In the presence of control constraints, the optimal control will typically be of bangbang type. The following class of examples illustrates this behavior. We consider the time optimal control problem min tf =
tf 0
g(x(t f )) +
tf 0
[ f 0 (t , x ) + h0 (t , u )] dt
tf 0
1 dt
(32)
g(x (t f )) +
[ f 0 (t , x ) + h0 (t , u )] dt
subject to the linear control system d x(t ) = Ax(t ) + Bu(t ) dt x ( 0 ) = x0 (33)
which implies that u* is optimal. LINEAR QUADRATIC REGULATOR PROBLEM We consider the special optimal control problem
and control as well as target constraints ui [1, 1] for 1 i m, and x(t f ) = 0
min 1 = 2
J ( x0 , u )
tf 0
[x (t )Qx(t ) + u (t )Ru(t )] dt + x (t f )Gx(t f )

t t t
(29)
Assume that (A, B) is controllable, that is, for every x0 Rn and every target x1 at tf there exists a control u L2(0, tf, Rm) that steers the system (33) from x0 to x1. We recall that,
OPTIMAL CONTROL
371
for the linear autonomous control system (33), controllability is equivalent to the requirement that the Kalman rank (B, AB, . . ., ABn1) n. A sufcient condition for the existence of an optimal control to Eq. (32) for arbitrary initial conditions x0 in Rn is that (A, B) is controllable and that A is strongly stable (the real parts of all eigenvalues of A are strictly negative). The Hamiltonian for Eq. (32) is H = 0 1 + (Ax + Bu ) and the adjoint equation is given by d = A dt The transversality condition implies that H(tf) 0 and hence (t) eA(tft) for some R1n. As a consequence of the maximum condition, we nd (t )Bu (t ) (t )Bu and hence u i (t ) = sign gi (t ), for 1 i m where g(t) : (t)B eA(tft)B. We claim that g(t) is nontrivial. In fact, if g(t) 0 for some t [0, tf], then, since (A, B) is controllable, 0 and (t) 0. We have H (t f ) = 0 + (t f )[Ax(t f ) + Bu(t f )] = 0 = 0 and thus (0, (t)) is trivial if g(t) 0. This gives a contradiction to Theorem 1. In the remainder of this subsection we consider a special linear control system (rocket sled problem) and provide its solution. Let y(t), the displacement of a sled with mass 1 on a friction-free surface be controlled by an applied force u(t) with constraint u(t) 1. By Newtons second law of motion, (d2 / dt2)y(t) u(t). If we dene x1(t) y(t) and x2(t) (d / dt)y(t), then the state x(t) col(x1(t), x2(t)) satises Eq. (33) with A= 0 0 1 0 and B = 0 1 for all u [1, 1]n
that u* 1. Then the equations (d / dt)x1(t) x2(t), (d / dt)x2(t) 1 have solutions of the form x2 (t ) = t + c1 and x1 (t ) = (t + c1 )2 + c2 2
2 Thus, the orbit is on the manifold x1 x2 /2 c2 oriented upwards. Similarly, for u* 1 the orbit is on the manifold 2 2 oriented downwards. Since the optimal conx1 x2 /2 c trols have at most one switch and the orbits must terminate at (0, 0), it follows that the optimal control u* is given in feedback form by
u (t ) = U (x1 (t ), x2 (t )) =
1 if (x1 , x2 ) is above S 1 if (x1 , x2 ) is below S
2 /2 (x1 where S is the switching curve consisting of x1 x2 2 0) and x1 x2 /2 (x1 0). The feedback law is in general robust, since it possesses the self-correcting property.
DYNAMIC PROGRAMMING PRINCIPLE AND HAMILTONJACOBIBELLMAN EQUATION In this section we discuss Bellmans dynamic programming principle (Refs. 1,2) for optimal control problems. Derivation of the HamiltonJacobiBellman Consider the Bolza problem min subject to d x(t ) = f (t , x(t ), u(t )), dt x(s ) = y, u(t ) U (35) J (s, y; u ) =
s tf
f 0 (t , x(t ), u(t )) dt + g(x(t f ))
(34)
where U is a closed convex set in Rm. Under appropriate conditions on f , Eq. (35) has a unique solution x x(t; (s, y)), and moreover x C(s, tf; Rn) depends continuously on (s, y) [0, tf] Rn and u L1(s, tf; Rm). As discussed in the preceding section, sufcient conditions on f 0 and g, which guarantee the existence of an optimal pair (x*, u*) for each (s, y) [0, tf] Rn, are well known. We dene the minimum-value function V(s, y) by V (s, y ) = min J (s, y; u )
uK
We observe that the system (A, B) is a single-input controllable system that is marginally stable. This implies existence of an optimal control u*. From the above discussion it follows that u* must satisfy u (t ) = sign 2 (t ) The adjoint equation implies that (1(t), 2(t)) is given by 1 (t ) = 1 and 2 (t ) = 1 (T t ) + 2
Then V satises the optimality principle:
min
s
f 0 (t , x(t ), u(t )) dt + V ( , x( )) : u U on [s, ] = V (s, y ) (36)
In fact, the cost functional J is additive in the rst variable: for [s, tf], J (s, y; u ) =
s
for some nonzero (1, 2). Hence the optimal control assumes at most the values 1 and 1 (bangbang control) and it has at most one switch between these two values. Assume
f 0 (t , x(t ), u(t )) dt + J ( , x( ); u )
(37)
372
OPTIMAL CONTROL
and thus
s
for all u(s) u U, and f 0 (t , x(t ), u(t )) dt + V ( , x( )) J (s, y; u ) Vt (s, y ) + f (s, y, u (s ))Vx (s, y ) + f 0 (s, y, u (s )) = 0 which is Eq. (38). Moreover, we have the following dynamical programming principle. Theorem 5. (Verication Theorem). Let V be a solution of the HJB equation (38) such that V C1((0, tf) Rn) and V(tf, x) g(x). Then we have 1. V(s, x) J(s, x; u) for any admissible control u. 2. If u* (t, x) U is the unique solution to
for all u K. Thus,
V (s, y ) min
s
f 0 (t , x(t ), u(t )) dt + V ( , x( )) : u Uad on [s, ]
V (s, y )
which implies Eq. (36). Suppose that V is continuously differentiable. Then V satises the so-called HamiltonJacobiBellman (HJB) equation: Vt (s, y ) + min[ f (s, y, u )Vx (s, y ) + f 0 (s, y, u )] = 0
u U
f (t , x, u )Vx (t , x ) + f 0 (t , x, u ) = min[ f (t , x, u )Vx (t , x ) + f 0 (t , x, u )]

u U
(40)
(38)
and the equation d x(t ) = f (t , x(t ), (t , x(t ))) dt x (s ) = y
We derive Eq. (38) using the optimality principle (36). Let K be of the form u
u =
u on (s, ) u on ( , t f )
has a solution x*(t) C1(s, tf; Rn) for each (s, y) [0, tf] Rn, then the feedback solution u*(t) (t, x*(t)) is optimal, that is, V(s, y) J(s, y; u*). Proof. Note that for u K d V (t , x(t )) = Vt (t , x(t )) + f (t , x(t ), u(t ))Vx (t , x(t )) dt and for any pair (x, u) Rn U, Vt + f (t , x, u )Vx (t , x ) + f 0 (t , x, u ) 0 and thus d V (t , x(t )) f 0 (t , x(t ), u(t )) dt Hence we have V (s, y )
s tf
where u(t) U on (s, ) and u minimizes J(, x(); u) over the interval [, tf]. From Eq. (37) we have J (s, x; u ) =
s
f 0 (t , x(t ), u(t )) dt + V ( , x( ))
If we set u(t) u*(t) on [s, ], then, from Eq. (36) u (t) u*(t) on [, tf] minimizes J(, x*(); ) on [, tf] and V (s, y ) =
s
f 0 (t , x (t ), u (t )) dt + V ( , x ( ))
(39)
Since V is assumed to be C1, we have d d V (t , x(t )) = Vt + Vx x(t ) = Vt + f (t , x(t ), u(t )) Vx dt dt or equivalently, V ( , x( )) = V (s, y ) +
s
f 0 (t , x(t ), u(t )) dt + g(x(t f )) = J (s, y; u )
for all admissible controls u. Similarly, if u*(t) U attains the minimum in Eq. (40) with x x*(t), then (Vt + fVx ) dt
), the equation above Eq. (39) Now, since V(s, y) J(s, x; u implies
s
d V (t , x (t )) = Vt (t , x (t )) + f (t , x (t ), u (t ))Vx (t , x (t )) dt = f 0 (t , x (t ), u (t ))
and thus
[Vt (t , x(t ), u(t )) + f (t , x(t ), u(t ))Vx (t , x(t )) + f 0 (t , x(t ), u(t ))] dt 0
V (s, y ) =
s
tf
f 0 (t , x (t ), u (t )) dt + g(x (t f )) = J (s, x; u )
where, from Eq. (39), the equality holds if u u* on [s, ]. Thus,
Relation to the Maximum Principle In this section we discuss the relation between the dynamic programming and the maximum principle. Dene the Hamil by tonian H (t , x, u, p ) = f 0 (t , x, u ) ( f (t , x, u ), p ) n H R (41)
s +
lim
1 s
[Vt + f (t , x(t ), u(t )) Vx + f 0 (t , x(t ), u(t ))] dt
= Vt (s, y ) + f (t , y, u(s ))Vx (s, y ) + f 0 (s, y, u(s )) 0
OPTIMAL CONTROL
373
(t, x, u, p) attains the maximum over u We assume that H (t, x, p) and that is locally U at the unique point u Lipschitz. Let us dene (t , x, u, p ) H (t , x, p ) = max H
u U
subject to Eq. (35), where inf x(t) is the exit time from an open set in Rn. It can be shown that, if we dene the value function V(y) infuU J(y, u), then it satises the HJB equation min[ f 0 (x, u ) + f (x, u ) Vx ] = 0,
u U
V (x ) = g(x ) on x (47)
Then Eq. (38) can be written as Vt H (t , x, Vx ) = 0 (42)
Assume that V C1,2((0, tf) Rn). Then, dening p(t) Vx(t, x*(t)), we obtain d x (t , x (t ), u (t ), p(t )), p(t ) = H dt p(t f ) = gx (x (t f )) (43)
For the specic case f (x, u) u, f 0 1, g 0, U [1, 1], and (1, 1), with x, u R, the HJB equation (47) becomes |Vx (x )| + 1 = 0 with V (1 ) = 0 (48)
and u*(t) (t, x*(t), p(t)) is an optimal control. In fact, by Eq. (42)
d Vx d Vx (t , x (t )) = (t , x (t )) + Vxx (t , x (t )) x (t ) dt t dt = Hx (t , x, Vx (t , x (t ))) + Vxx (t , x (t ))H p (t , x, Vx (t , x (t ))) + Vxx (t , x (t )) f (t , x (t ), u (t )) x (t , x (t ), u (t ), Vx (t , x (t ))) =H

where Vxx 2V / xi xj Rnn. Here we have used the fact that
It can be proved that Eq. (48) has no C1 solution, but there are innitely many Lipschitz continuous solutions that satisfy it a.e. in (1, 1). The viscosity method is developed as a mathematical concept that admits non-C1 solutions and selects the solution corresponding to the optimal control problem to the HJB equation. We return now to the general problem presented in the rst subsection of this section. Denition 1. A function v C((0, tf] Rn) is called a viscosity solution to the HJB equation vt H(t, x, vx) 0 provided that for all C1(), if v attains a (local) maximum at (t0, x0), then t H (t , x, x ) 0 at (t0 , x0 )
H p (t , x, p ) = f (t , x, u ) and
0 Hx (t , x, p ) = f x (t , x, u )t p f x (t , x, u )t
and if v attains a (local) minimum at (t0, x0), then t H (t , x, x ) 0 at (t0 , x0 )
(t, x, u, p) over U. We (t, x, p) U maximizes H where u observe that Eq. (43) represents the adjoint equation of the maximum principle with adjoint variable given by Vx( , x*). Next let us set U Vx. Then U satises Ut (t , x ) + f (t , x, (t , x, U (t , x ))) Ux (t , x ) Hx (t , x, U (t , x )) = 0 (44)
It is clear that a C1 solution to Eq. (42) is a viscosity solution, and if v is a viscosity solution of Eq. (42) and Lipschitz continuous, then vt H(t, x, vx) 0 a.e. in (0, tf) Rn. The viscosity solution concept is derived from the vanishing viscosity method illustrated by the following theorem. Theorem 6. Let V(t, x) C1,2((0, tf) Rn) be a solution to the viscous equation Vt H (t , x, V ) + V = 0, V (t f , x ) = g(x ) (49)
Hence, setting u(t) (t, x(t), p(t)), the necessary optimality conditions
d x(t ) = f (t , x, u(t )) dt d p(t ) = Hx (t , x, p(t )) dt d V (t ) = f 0 (t , x, u(t )) dt
If V(t, x) V(t, x) uniformly on compact sets as 0, then V(t, x) is a viscosity solution to Eq. (42).
(45)
Proof. We need to show that t H (t , x, x ) 0 at (t0 , x0 )
are the characteristic equations of the rst order partial differential equations (PDEs) (42) and (44). Viscosity Solution Method In this section we discuss the viscosity solution method for the HJB equation. For motivation, we rst consider the exit time problem min J (y, u ) =
0
for all C1((0, tf) Rn) such that V attains a local maximum at (t0, x0). Choose a function C1((0, tf) Rn) such that 0 1 for (t, x) (t0, x0) and (t0, x0) 1. Then (t0, x0) is a strict local maximum of V . Dene V , and note that, since V V uniformly on compact sets, there exists a sequence (t, x) such that (t, x) (t0, x0) as 0 and attains a local maximum at (t, x). The necessary optimality condition yields
t
f 0 (x(t ), u(t )) dt + g(x( ))
(46)
= 0,
= 0,
xx
at (t , , x )
374
OPTIMAL CONTROL
It follows that t H (t , x, x ) + t H (t , x, x ) + at (t , x )
Here h* denotes the conjugate function of h, which is dened by h (v ) = sup{vu h(u )}

u U
Since is an arbitrary function with the specied properties, t H(t, x, x) 0 at (t0, x0). For example, let V(x) be the solution to |Vx (x )| + 1 + Vxx = 0, Then the solution V is given by V (x ) = 1 |x| + (e 1/ e|x|/ ) and we have lim V(x) 1 x as 0. Moreover, V(x) 1 x is a viscosity solution to Vx(x) 1 0. We can also check that any other Lipschitz continuous solution is not a viscosity solution. It can be proved, in a general context, that the viscosity solution is unique (Refs. 3,4). As we saw for the exit time problem, the value function V is not necessarily differentiable. But it always superdifferentiable. Here we call a function superdifferentiable at y0 if there exists p Rn such that lim sup
yy 0
(50)
V (1 ) = V (1 ) = 0
We assume that h* is Gateaux-differentiable with locally h* Lipschitz Gateaux derivative h* p . Then we have u p (v), U attains the maximum of (v, u) h(u) over u where u U. In this case, the HJB equation is written as Vt + a(x ) Vx h (b (x )tVx ) + l (x ) = 0 (51)
with V(tf, x) g(x). As a specic case, we may consider the linear quadratic control problem where
f (t , x, u ) = A(t )x + B(t )u + f (t ),
t t f 0 (t , x, u ) = 1 2 [x Q (t )xu R(t )u]
and U Rm. Then we have

1 t Vt + [A(t )x + f (t )]Vx 1 | R 1 (t )Bt (t )Vx |2 R + 2 x Q (t )x = 0 2 (52)
(y ) (y0 ) ( p, y y0 ) 0 |y y0 |
Suppose that g(x) xtGx. Then V(t, x) xtP(t)x xv(t) is a solution to Eq. (52), where P(t) Rnn satises the differential Riccati equation
and we denote the set of p such that the above inequality holds by D(y0). Based on the notion of viscosity solution, one can express the dynamic programming and the maximum principle without assuming that V is C1 as follows (see, e.g., Ref. 2). Theorem 7. The value function V(s, y) is continuous on (0, tf) Rn, and locally Lipschitz continuous in y for every s [0, tf]. Moreover, V is a viscosity solution of the Hamilton JacobiBellman equation, and every optimal control u* to the problem Eqs. (34)(35) is given by the feedback law u (t ) = (t , x (t ), (t ))

dP (t ) + At (t )P(t ) + P(t )A(t ) dt P(t )B(t )R 1 (t )Bt (t )P(t ) + Q(t ) = 0

with P(tf) G, and the feedforward v(t) satises
(53)
d v(t ) = [A BR 1 Bt P(t )]t v(t ) = P(t ) f (t ), dt
v(t f ) = 0 (54)
for some
(t )
D+ x V (t , x (t ))
In the control-constrained case with U u Rm : u 1 and h(u) u2, we nd
for every t [0, T], where x*( ) is the optimal trajectory of Eq. (35) corresponding to u*. Applications Here we consider the case f (t , x(t ), u(t )) = a(x ) + b (x )u and f 0 (t , x, u ) = l (x ) + h(u ) and
h ( p ) =
1 | p|2 2
if | p| < 1
1 2
| p|
if | p| 1
h p ( p) =
if | p| < 1
p/| p| if | p| 1
where it is assumed that h : U R is convex, is lower semicontinuous, and satises h(u ) |u|2 We nd for some > 0
so that h* C1(Rm) W2,(Rm). LINEAR QUADRATIC REGULATOR THEORY AND RICCATI EQUATIONS In this section we rst revisit the nite-horizon LQR problem in Eqs. (29)(30) and show that the optimal control u*(t) can be expressed in the feedback form u (t ) = R 1 Bt [P(t )x(t ) + v(t )] (55)
min{ f 0 (t , x, u ) + p f (t , x, u )}
u U
= a(x ) p + l (x ) max{u b (x )t p h(u )}

u U
= a(x ) p h (b (x )t p )
OPTIMAL CONTROL
375
where the symmetric matrix P(t) and the feedforward v(t) satisfy Eqs. (53) and (54). The matrix K(t) R1BtP(t) describing the control action as a function of the state is referred to as the feedback gain matrix. The solution in the form of Eq. (55) can be derived from the dynamical programming principle in the preceding subsection, but here we prefer to give an independent derivation based on the two-point boundary value problem (31). Since this equation is afne, we can assume that p(t ) = P(t )x(t ) + v(t ) (56)
We turn to the innite-time-horizon problem: min J ( x0 , u ) = 1 2

0
[xt (t )Qx(t ) + u(t )Ru(t )] dt
(60)
subject to the linear control system d x(t ) = Ax(t ) + Bu(t ), dt x ( 0 ) = x0
This problem need not admit a solution. For example, consider the system d x(t ) = dt 1 0 1 x(t ) + 1 1 u(t ) 0
More precisely, let (x(t), p(t)) denote a solution to Eq. (31). Then for each t [0, tf], the mapping from x(t) Rn to p(t) Rn dened by forward integration of the rst equation in Eq. (31) with initial condition x(t) on [t, tf] and subsequent backward integration of the second equation of Eq. (31) with terminal condition p(tf) Gx(tf) on [t, tf] is afne. Substituting Eq. (56) into the second equation of Eq. (31), we obtain d d d P(t )x(t ) + P(t ) x(t ) + v(t ) = Qx(t ) At [P(t )x(t ) + v(t )] dt dt dt and from the rst equation in Eq. (31) we derive
and let Q I and R be arbitrary. Then there exists no admissible control u L2(0, ; Rm) such that J(x0, u) is nite, unless x2(0) 0. Under the assumption that for each x0 Rn there exists at least one admissible control such that J(x0, u) is nite (nitecost condition), it can be shown that the optimal control is given in feedback form by u (t ) = R 1 Bt P x (t ) where the nonnegative symmetric matrix P is dened in the following theorem. A sufcient condition for the nite-cost condition is that the pair (A, B) is stabilizable, that is, there exists a matrix K Rmn such that A BK is asymptotically stable. In this case, the closed-loop system with feedback control u(t) Kx(t) is exponentially stable, and we have J(x0, Kx(t)) Mx02 for some M 0 independent of x0 Rn. The following result is referred to as LQR theory. It relies on the notions of detectability and observability. The pair (A, Q1/2) is called detectable if there exists a matrix G such that A GQ1/2 is asymptotically stable. Further (A, Q1/2) is called observable if, for some 0, the kernel of the mapping x Q1/2eAx is trivial. Observability of (A, Q1/2) is equivalent to controllability of (At, Q1/2). Theorem 8 (LQR) 1. Assume that for each x0 Rn there exists at least one admissible control such that J(x0, u) is nite. For any tf 0, let Ptf ( ) denote the solution to the Riccati equation Eq. (53) with G 0. Then Ptf (0) converges monotonically to a nonnegative symmetric matrix P as tf , and P satises the algebraic Riccati equation At P + P A P BR 1 Bt P + Q = 0 The control u (t ) = R 1 Bt P x (t ) is the unique solution to the LQR problem (60), and J ( x0 , u ) = 1 xt P x = 2 0 0
u L 2 (0 , ; R m )
d P(t )t + AP(t ) + P(t )A P(t )BR 1 Bt P(t ) + Q x(t ) dt d + v(t ) + [A BR 1 Bt P(t )]t v(t ) + P(t ) f (t ) = 0 dt
This equation holds if Eqs. (53) and (54) are satised. By standard results from the theory of ordinary differential equations, there exists a unique symmetric and nonnegative solution Ptf (t) C1(0, tf; Rnn) with Ptf (t) G to the Riccati equation (53). If x*(t) is a solution to d x (t ) = [A BR 1 Bt Pt (t )]x (t ) BR 1 Bt v(t ) + f (t ) (57) f dt where v(t) is a solution to Eq. (54), and if we set p(t) P(t)x*(t) v(t), then the pair (x*(t), p(t)) is a solution to Eq. (31). Thus, the triple (x*(t), u*(t), pt(t)) satises the maximum principle. From Theorem 3 or from Eq. (58) below, it follows that the feedback solution (55) is optimal. The formula (56) is called the Riccati transformation. It transforms the TPBV problem (31) into a system of initial value problems backwards in time. Moreover, the feedback solution given by Eq. (55) is unique. This follows from the fact that for arbitrary u L2(0, tf; Rm), multiplying Eq. (57) from the left by [Ptf (t)x*(t)]t using Eqs. (53) and (54), and integrating the resulting equation on [0, tf], we have
(61)
J ( x0 , u ) = J ( x0 , u ) +
1 2
tf 0
|u(t ) + R 1 Bt [Pt x(t ) + v(t )]|2 R dt

f
(58)
2 where yR ytRy and
J ( x0 , u ) =
1 2 +
t xt 0 Pt (0 )x0 + 2v (0 ) x0
f
min
J ( x0 , u )
tf 0
(59) [vt (t )BR 1 Bt v(t ) + 2vt (t ) f (t )] dt

Conversely, if there exists a nonnegative symmetric solution P to Eq. (61), then for all x0 Rn there exists an
376
OPTIMAL CONTROL
admissible control u such that J(x0, u) is nite and P P. 2. Suppose that (A, Q1/2) is detectable and that Eq. (61) admits a solution. Then the closed-loop matrix A BR1BtP is asymptotically stable, and P is the unique nonnegative symmetric solution to Eq. (61). If, moreover, (A, Q1/2) is observable, then P is positive denite. Proof. For part 1 note that, due to Eq. (59), we have for f tf t
t xt 0 Pt (0 )x0 x0 Pt ( 0 )x 0
f f
where f L2(0, ; Rn). Assume that (A, B) is stabilizable and that (A, Q1/2) is detectable. Then there exists a unique optimal solution u* given by u (t ) = R 1 Bt [P x (t ) + v(t )] where P is the unique solution to Eq. (61) and v L2(0, ; Rn) satises d v(t ) + (A BR 1 Bt P )v(t ) + P f (t ) = 0, dt v ( ) = 0
f. The assumption on the exisThus, Ptf (0) Ptf (0) for tf t tence of admissible controls shows that eitPtf (0)ei and (ei ej)tPtf (0)(ei ej) are monotonically nondecreasing and bounded with respect to tf. Here ei denotes the ith unit vector in Rn. Dening P limtf Ptf (0), it follows that P is symmetric, is nonnegative, and moreover satises the steady-state equation Eq. (61). It can then be argued that the feedback control u*(t) R1BtPx*(t) is the unique optimal solution to the LQR problem (60). To prove the last assertion of part 1, suppose that P is a nonnegative symmetric solution to Eq. (61). Let x(t) be the solution to (d / dt)x(t) (A BR1BtP)x(t) with x(0) x0, and let u(t) R1BtPx(t). Then
Proof. From Theorem 8 it follows that A BR1BtP is asymptotically stable and thus v L2(0, ; Rn) and u*(t) L2(0, ; Rm). Since (A, Q1/2) is detectable, there exists a matrix G Rnn such that A GQ is asymptotically stable. For arbitrary admissible controls u L2(0, ; Rn) x(t ) = e(A GQ )t x0 +
t 0
e(A GQ )(t s ) [GQx(t ) + Bu(t ) + f (t )] dt
From the Fubini inequality

0
|x(t )|2 dt M (|x0 |2 + |u|2 + | f |2 ) L 2 (0 , ; R m ) L 2 (0 , ; R n )
d t [x (t )Px(t )] = 2xt (t )P(A BR 1 Bt P )x(t ) dt = xt (t )(Q + PBR 1 Bt P )x(t )

Integration of this equation on [0, tf] implies
tf 0
for some M 0. Thus limt x(t) exists and is zero. Taking the limit tf in Eq. (58), we obtain J ( x0 , u ) = J ( x0 , u ) + 1 2
0
|u(t ) + R 1 Bt [P x(t ) + v(t )]|2 R dt
which proves the theorem. Assume that (A, B) is stabilizable. Then there exists a solution P to Eq. (61), for which, however, A BR1BtP is not necessarily asymptotically stable. The following theorem shows that there exists a maximal solution P to the Riccati equation (61) and gives a sufcient condition such that A BR1BtP is asymptotically stable. Theorem 10. Assume that (A, B) is stabilizable. For 0 let P be the nonnegative symmetric solution P to the Riccati equation At P + PA PBR 1 Bt P + Q + I = 0 Then P converges monotonically to a nonnegative symmetric matrix P as 0. The matrix P is a solution to Eq. (61), and P P for all nonnegative symmetric solutions P to Eq. (61). Moreover, if we assume that the Hamiltonian matrix H= A Q BR 1 Bt At (62)
(xt Qx + ut Ru ) dt + x(t f )t Px(t f ) = xt 0 Px0
t t t and thus J(x0, u) x0 Px0 and x0 Px0 x0 Px0 for every n x0 R . To verify part 2, note that
(A BR 1 Bt P )t P + P (A BR 1 Bt P ) + Q + P BR 1 Bt P = 0
It can be shown that (A BR1BtP, (Q PBR1BtP)1/2) is detectable (observable) if (A, Q1/2) is detectable (observable). Hence, it follows from the Liapunov criterion (5) that A BR1BtP is asymptotically stable and moreover that P is positive denite if (A, Q1/2) is observable. For the proof of uniqueness we refer to the literature (see, e.g., Ref. 5). In the following theorem we consider the LQR theory with external forcing. Theorem 9. Consider the innite-time-horizon problem min subject to d x(t ) = Ax(t ) + Bu(t ) + f (t ), dt x ( 0 ) = x0 1 2
0
[x (t )Qx(t ) + u (t )Ru(t )] dt
t t
has no eigenvalues on the imaginary axis, then A BR1BP is asymptotically stable. For the stability-constrained LQR problem of minimizing Eq. (60) subject to
d x(t ) = Ax(t ) + Bu(t ) + f (t ), dt x(0 ) = x0 and
|x(t )|2 dt <
OPTIMAL CONTROL
377
the unique optimal control u* is given by u (t ) = R 1 [Bt P + x(t ) + v(t )] where v(t) L2(0, , Rn) satises d v(t ) + (A BR 1 Bt P+ )v(t ) + P+ f (t ) = 0, dt v ( ) = 0
1 Assume that U11 is nonsingular, and dene P U21U 11 . Since PU11 U21, we have
(A WP )U11 = U11 S11
and
(Q At P )U11 = PU11 S11
Thus P(A WP)U11 (Q AtP)U11, and moreover A WP 1 U11S11U 11 . Conversely, if P satises AtP PA PWP Q 0. Then H I P 0 I = A WP Q At P W At
Due to the importance of nding the stabilizing feedback gain, solving Eq. (61) is of considerable practical importance. We therefore close this section by describing the PotterLaub method. We also refer to Refs. 6, 7 for iterative methods based on the NewtonKleimann and Chandrasekhar algorithms. The PotterLaub method uses the Schur decomposition of the Hamiltonian matrix (62) and is stated in the following theorem. Theorem 11 1. Let Q, W be symmetric n n matrices. Solutions P to the algebraic Riccati equation AtP PA PWP Q 0 coincide with the set of matrices of the form P VU1, where the n n matrices U [u1, . . ., un], V [v1, . . ., vn] are composed of upper and lower halves of n real Schur vectors of the matrix H= A Q W At
and I P Thus H I P 0 I = I P 0 I A WP 0 W At + PW 0 I A WP 0 W At + PW = A WP P(A WP ) W At
The proof of assertions 24 can be found in Ref. 6. In summary, the stabilizing solution P corresponds to the stable eigen subspace of H, and the eigenvalues of the resulting closed-loop system coincide with those of S11. NUMERICAL METHODS In this section we discuss numerical methods for the nonlinear regulator problem
T
and U is nonsingular. 2. There exist at most n eigenvalues of H that have negative real part. 3. Suppose [u1, . . ., un] are real Schur vectors of H corresponding to eigenvalues 1, . . ., n, and i j for 1 i, j n. Then the corresponding matrix P UV1 is symmetric. 4. Assume that Q, W are nonnegative denite and (A, Q1/2) is detectable. Then the solution P is symmetric and nonnegative denite if and only if Re k 0, 1 k n. Proof. We prove part 1. Let S be a real Schur form of H, that is, HU US with UtU I and S= Thus U11 H U21 where U11 U21 is made up of n Schur vectors of H corresponding to the S11 block. We observe that AU11 WU21 = U11 S11 and QU11 At U21 = U21 S11 = U11 U21 S11 S11 0 S12 S22
min subject to
J ( x0 , u ) =
[l (x(t )) + h(u(t ))] dt + g(x(T ))

0
(63)
d x(t ) = f (x(t ), u(t )), dt We assume that
x ( 0 ) = x0 ,
u(t ) U
(64)
( f (x, u ) f (y, u ), x y ) |x y|2
for all u U
(65)
and moreover either that U Rm is bounded or that h(u) c1u2 and ( f (x, u ), x ) |x|2 + c2 |u|2 (66)
for constants , c1, c2 0, independent of x, y Rn and u U. Also we assume that for each (x, p) Rn Rn the mapping u h(u ) + ( p, f (x, u )) admits a unique minimizer over U, denoted by (x, p). Finally, we assume that l, h, g, and f are sufciently smooth with l and g bounded from below.
378
OPTIMAL CONTROL
Discrete-Time Approximation We consider the discretized problem

N
min
J N (uN ) =
k=1
[l ( x k ) + h ( u k )]
t + g ( xN )
(67)
Proof. First, we show that x N and p N are uniformly Lipschitzian in N. It will be convenient to drop the tilde in the notation for these sequences. In the case that U is bounded, we proceed by taking the inner product of Eq. (68) with xk and employing Eq. (65):
1 (|xk |2 2
|xk 1 |2 )
t [|xk |2 + | f (0, uk )| |xk |]

1 k 2 k 2 t [( + 1 2 )|x | + 2 | f (0, u )| ]
subject to xk xk 1 = f (xk , uk ) and uk U t 1kN (68)
In case U is unbounded, Eq. (66) implies that

1 (|xk |2 2
where N t T, which realizes the implicit Euler scheme for time integration of Eq. (64) and rst-order integration of the cost functional (63). Note that if t 1, then the mapping (x) x t f (x, u) is dissipative, that is, (F(x1, u) F(x2, N u), x1 x2) (1 t )x1 x22. Thus for u ukk 1 in U k N there exists a unique x x k1, satisfying the constraint Eq. (68) and depending continuously on u. Moreover, if t 1, then there exists an optimal pair (xk, uk) to the problem Eqs. (67), (68). The necessary optimality condition for that problem is given by
|xk 1 |2 )
N
t (|xk |2 + c2 |uk |2 )
and, by assumption on h, k1 uk2 t is bounded uniformly in N. In either case, by the discrete-time Gronwalls inequality we obtain that xk M1 for some M1 0 uniformly in k and N. The condition (65) implies that ( f x (x, u ) p, p ) | p|2 and taking the inner product of Eq. (69) with pk, we obtain
1 ( | p k |2 2
xk xk 1 = f ( xk , u k ) t pk + 1 pk = f x ( x k , u k )t p k + l x ( x k ) t u k = ( xk , p k ) U
| p k + 1 |2 )
t [| pk |2 + |lx (xk )| | pk |]
1 k 2 k 2 t [( + 1 2 )| p | + 2 |lx (x )| ]
(69)
Thus pk M2 for some M2 uniformly in k and N. Due to the Lipschitz continuity of , we nd that uk bounded uniformly in k and N, and from Eq. (69), pk pk + 1 xk xk 1 , are bounded uniformly t t Using Lipschitz continuity of a second time, we nd that (uk uk1)/ t is uniformly bounded as well. By the compactness of Lipschitz continuous sequences in L2(0, T), there ex such that (xN ists a subsequence N , uN, pN) converges to (x*, 2 n m n u*, p*) in L (0, T; R R R ) and pointwise a.e. in (0, T). From Eq. (69),
t
for 1 k N, with x0 x0 and pN1 gx(xN). It is noted that Eq. (69) is a sparse system of nonlinear equations for col(col(x1, . . ., xN), col(p1, . . ., pN)) RnN RnN. We have the following result: Theorem 12. Assume that is Lipschitz continuous, that t 1, and that uNN 1 is a sequence of solutions to Eqs. (67), (68) with associated primal and adjoint states (xN, pN)N N denote the step func1 such that Eq. (69) holds. Let u tion dened by u N N(t) uk on (tk1, tk), 1 k N, and let x N and p be the piecewise linear functions dened by x N (t ) = xk 1 + and p N (t ) = pk + pk + 1 pk (t tk 1 ) t xk xk 1 (t tk 1 ) t
xN (t ) = x0 +
f (x N (t ), uN (t )) dt
0
N is the piecewise constant sequence dened by xk, where x 1 k N. By Lebesgues dominated convergence theorem, we nd that x* coincides with the solution x(t; u*) to Eq. (64) associated with u*. For v L2(0, T; Rm) let vN be the piecewise t constant approximation of v, dened by vk (1/ N) tk1 v(t) k dt, 1 k N. Then J N (uN ) J N (vN ) for all v
Then the sequence (x N, u N) in H1(0, T; Rn) L2(0, T; Rm) N, p 1 n H (0, T; R ) has a convergent subsequence as t 0, and for every cluster point (x, u, p), u K is an optimal control of Eqs. (63), (64), and (x, u, p) satises the necessary optimality condition
d x(t ) = f (x(t ), u(t )), x ( 0 ) = x0 dt d p(t ) = f x (x(t ), u(t ))t p(t ) + lx (x(t )), dt u(t ) = (x(t ), p(t )) U
and x(t, vN) x(t, v) as N , and thus, by the Lebesgue dominated convergence theorem, J(x0, u*) J(v; x0) for all admissible control, that is, (x*, u*) is an optimal pair. It is not difcult to argue that the triple (x*, u*, p*) satises the necessary optimality [Eq. (70)]. Construction of Feedback Synthesis The numerical realization of feedback synthesis for problems that are not of the LQR type has not received much research attention up to now. In this subsection we propose a method
p(T ) = gx (x(T )) (70)
OPTIMAL CONTROL
379
for the construction of the feedback synthesis K to the problem (63)(64), which is still under investigation. As we discussed in the Section titled Dynamic Programming Principle and HamiltonJacobyBellman Equation, the optimal feedback law is given by K(t, x(t)) KT(t, x(t)) (x(t), Vx(t, x(t)), where V(t, x) is the solution to HJB Eq. (38) and we stress the dependence of K on T. Let us assume that f (x, u) f (x) Bu, and h /2u2. Then 1 KT (t , x(t )) = Bt Vx (t , x(t )) (71)
best interpolation W Wad based on the stationary equation f (x) Vx h*(BtVx) l(x) 0 by
min
k
|[ f (xk ) Wx (xk ) h (Bt Wx (xk )) + l (xk )]+ |2 +

x0
|Wx (x0 ) px 0 (0 )|2
(74)
over Wad, where x max(0, x). Then we set K(x) (x, Wx). ACKNOWLEDGMENTS K. Itos research was supported in part by AFSOR under contracts F-49620-95-1-0447 and F-49620-95-1-0447. K. Kunischs research was supported in part by the Fonds zur Fo rderung der wissenschaftlichen Forschung, SFB Optimization and Control. BIBLIOGRAPHY
1. R. Bellman, Dynamic Programming, Princeton, NJ: Princeton Univ. Press, 1957. 2. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Control, New York: Springer-Verlag, 1975. 3. M. G. Crandall and P. L. Lions, Viscosity solutions of Hamilton Jacobi equations, Trans. Amer. Math. Soc., 277: 142, 1983. 4. W. Fleming and M. Soner, Controlled Markov Processes and Viscosity Solutions, Berlin: Springer-Verlag, 1993. 5. W. M. Wonham, Linear Multivariable Control: A Geometric Approach, Berlin: Springer-Verlag, 1974. 6. D. L. Russell, Mathematics of Finite-Dimensional Control Systems, Theory and Design, New York: Marcel Dekker, 1979. 7. H. T. Banks and K. Ito, A numerical algorithm for optimal feedback gains in high dimensional linear quadratic regulator problems, SIAM J. Control Optim., 29: 499515, 1991. Reading List V. M. Alekseev, V. M. Tikhomirov, and S. V. Fomin, Optimal Control, New York: Plenum, 1987. M. Athans and P. Falb, Optimal Control, New York: McGraw-Hill, 1963. L. D. Berkovitz, Optimal Control Theory, New York: Springer-Verlag, 1974. A. E. Bryson and Y. Ho, Applied Optimal Control, New York: Wiley, 1975. M. R. Hestenes, Calculus of Variations and Optimal Control, New York: Wiley, 1963. H. Hermes and J. L. LaSalle, Functional Analysis and Time Optimal Control, New York: Wiley, 1969. H. Knobloch, Higher Order Necessary Conditions in Optimal Control Theory, Berlin: Springer-Verlag, 1981. H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New York: Wiley-Interscience, 1972. B. Lee and L. Marcus, Foundations of Optimal Control Theory, New York: Wiley, 1967. L. W. Neustadt, Optimization: A Theory of Necessary Conditions, Princeton, NJ: Princeton Univ. Press, 1976.
Since the problem under consideration is autonomous, we can write KT(t, x(t)) KTt(0, x(t)). Using the relationship between the HJB equation and the Pontryagin maximum principle, as described in the section titled Dynamic Programming Principle and HamiltonJacobyBellman Equation, we construct a suboptimal feedback law. It is based on the fact that if we set p(t) Vx(t, x(t)), then the pair (x(t), p(t)) satises the TPBV problem (70). Thus, if we dene the function x0 Rn px0(0) Rn, where (x, p)x0 is the solution to Eq. (70) with x(0) x0, then KT (0, x0 ) = (x0 , px 0 (0 )), x0 Rn (72)
The dependence of the feedback gain KT on T is impractical, and we replace it by a stationary feedback law v(t) K(x(t)), which is reasonable if T is sufciently large. Based on these observations, a suboptimal feedback law can be constructed by carrying out the following steps: 1. Choose T 0 sufciently large, as well as a grid Rn, and calculate the solutions (x, p)x0 to the TPBV problem for all initial conditions determined by x0 . Thus we obtain the values of K at x0 . 2. Use an interpolation method based on K at the grid points of to construct a suboptimal feedback synthesis K. The interpolation in step 2 above can be based on appropriate Greens functions, for example,
(x ) = K
G(x, x j ) j
j=1
with
G(xi , x j ) j = K (xi ),
1iM (73)
The Greens function interpolation (73) has the following variational principle: Consider min
Rn
| K (x )|2 dx
subject to K (xi ) = i ,
1iM
Then the optimal solution K is given by Eq. (73), where Greens function G satises the biharmonic equation 2G(x) (x). For instance, G(x, y) x y (biharmonic Greens function) in R3. In our numerical testings we found that G(x, y) x y, 1.5 4, works very well. Alternatively, we may employ the following optimization method. We select a class Wad C1(Rn) of parametrized solutions to the HJB equation and collocation points xk in Rn. Then we determine the
KAZUFUMI ITO
North Carolina State University
KARL KUNISCH
Karl-Franzens-Universita t Graz
380
OPTOELECTRONICS IN VLSI TECHNOLOGY
OPTIMAL FILTERING. See WIENER FILTERS. OPTIMIZATION, CIRCUIT. See MINIMIZATION. OPTIMIZATION, CONVEX. See CONVEX OPTIMIZATION. OPTIMIZATRION, FUZZY. See MINIMIZATION. OPTIMIZATION IN SCHEDULING. See SCHEDULING. OPTIMIZATION, LIFE CYCLE. See LIFE CYCLE OPTIMIZATION.
OPTIMIZATION OF LOGIC CIRCUITS. See LOGIC

SYNTHESIS.
OPTIMIZATION PROBLEMS/TECHNIQUES. See

TRAVELING SALESPERSON PROBLEMS.
OPTION CONTRACTS. See STOCK MARKETS. OPTODES, OPTRODES. See CHEMICAL SENSORS. OPTOELECTRONIC PHENOMENA. See PHOTOCONDUCTIVITY.
OPTOELECTRONICS. See LIGHT EMITTING DIODES, DEVICES.
OPTOELECTRONICS, INTEGRATED. See INTEGRATED

OPTOELECTRONICS.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Periodic Control Standard Article S. Bittanti1 and P. Colaneri1 1Politecnico di Milano, Milano, Italy Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1030 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (219K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Basics in Periodic Systems Analysis Time-Invariant Representations Periodic Systems in Frequency Domain Monodromy Matrix and Stability Periodic Control Periodic Optimal Control Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
PERIODIC CONTROL
59
PERIODIC CONTROL
The fact that a periodic operation may be advantageous has been well known to humankind since time immemorial. All farmers know that it is not advisable to grow the same product repeatedly in the same eld because the yield can be improved by rotating crops. So, cycling is good. More recently, similar concepts have been applied to industrial problems. Traditionally, almost every continuous industrial process was set and kept, in the presence of disturbances, at a suitable steady state. However, there are circumstances under which a periodic time-varying action proves to be better. This observation germinated in the eld of chemical engineering where it was seen that the performance of a number of catalytic reactors improved by cycling; see the pioneering contributions in Refs. 13. Unfortunately, as pointed out in Ref. 4, periodic control was still considered too advanced in the industrial control scenario, in that the steady-state operation is the norm and unsteady process behaviour is taboo. Its use was therefore conned to advanced (aerospace or classied) applications, such as those treated in Refs. 5 and 6. Today, however, the new possibilities offered by current control technology, together with the theoretical developments of the eld, have opened the way for using periodic controllers in place of the traditional stationary ones. In fact, the term periodic control takes a wider signicance in the contemporary literature. In addition to the control problems that arise when operating a plant periodically, periodic control also includes all situations where either the controller or the plant is a proper periodic system. One of the reasons behind such an extension is the possible improvement of the performances, in terms of stability and robustness, of plants described by time-invariant models, when using a periodic controller (see Ref. 7). The diffusion of digital apparatuses in control has also contributed to the increasing importance of periodic control because computer-controlled systems are often based on sample-and-hold devices for output measurements and input updating. In multivariable control, it may also be necessary, for technological or economical reasons, to adopt different sampling and/or hold intervals for the various actuators or transducers. For example, certain variables may exhibit a much slower dynamic than others so that different sampling interJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
60
PERIODIC CONTROL
vals must be adopted. In such situations, we usually resort to an internal clock for setting a synchronization mode; the digital equipment complementing the plant performs the selection of the output sampling instants and the control updating instants out of the clock time points. It turns out that these sampling selection mechanisms are described by periodic models in discrete time, with period equal to the least common factor of the ratios between the sampling and updating intervals over the basic clock period. The overall control system obtained in this way is known as a multirate sampleddata system (8,9). Finally, resorting to control laws that are subject to periodic time variations is natural to govern phenomena that are intrinsically periodic. An important eld where we encounter such dynamics is helicopter modeling and control, as witnessed by the fact that a full chapter of the classical reference book in the eld (10) is devoted to periodic systems. The main interest in this framework is rotor dynamics modeling. Indeed, consider the case of level forward ight, when the velocity vector of the ying machine is constant and parallel to its body. Those ight conditions are achieved by imposing a periodic pattern on the main control variables of the helicopter (i.e., the pitch angles of each blade). Consequently, the aerodynamic loads present a cyclic pattern, with period determined by the rotor revolution period, and any model of the rotor dynamics is periodic (see Refs. 1113). The interest for periodic systems goes far beyond these situations. Periodicity arises in the study of nonlinear time-invariant systems dealing with closed orbit operations. Classical examples of such situations are relayoperated control plants, hysteretic oscillators, and processes subject to seasonal-load effects. For the study of system behavior against small perturbations, a linearized approximated model is often used. And, although the original system is time-invariant, the linearization procedure generates periodicity in the approximated linear model (see Refs. 1416). In this article, the most important techniques of periodic control will be outlined, avoiding, however, overly technical details. The article is organized as follows. The rst part deals with the analysis of periodic systems. Initially, it is shown how state-space periodic models arise from multirate sampling or linearization around closed orbits. The periodic input/output representation is also introduced as an alternative to state-space modelization. Then, the possibility of analyzing a periodic system via time-invariant models is investigated and a number of techniques are introduced. Further, the frequency-response concept for periodic systems is outlined. The fundamental concept of stability comes next. It calls for the denition of the monodromy matrix and involves the theory of Floquet and Lyapunov. In passing, the notion of cyclostationary stochastic process is touched on and briey discussed. The second part is devoted to periodic control, and discusses three main problems: (1) choice of the control signal in order to force a periodic regime with better performance than any possible steady state operation, (2) periodic control of time-invariant plants, and (3) periodic control of periodic systems. The literature on the subject is so vast that it is impossible to cover all aspects of theoretical and application interest. The interested reader will nd a rather detailed list of references
in the bibliography, including survey papers (1722) and early reference books (23,24).
BASICS IN PERIODIC SYSTEMS ANALYSIS A basic classication of linear periodic systems depends on the nature of the time variable t. We focus our attention herein on continuous-time or discrete-time periodic systems. In the former case t is a real variable, whereas in the latter t is an integer. State-Sampled Representation Nowadays, the most widely used mathematical modelization of dynamical systems hinges on the concept of state-variable. The state variables are latent variables that establish a bridge between the input variables u(t) and the output variables y(t). They are collected in a vector denoted by x(t), and the basic state-space model is the set of difference equations x(t + 1 ) = A(t )x(t ) + B(t )u(t ) y(t ) = C(t )x(t ) + D(t )u(t ) in discrete-time, or the set of differential equations x (t ) = A(t )x(t ) + B(t )u(t ) y(t ) = C(t )x(t ) + D(t )u(t ) in continuous time. Matrices A( ), B( ), C( ), and D( ) are real matrices, of appropriate dimensions, that depend periodically on t: A(t + T ) = A(t ), C(t + T ) = C(t ), B(t + T ) = B(t ) D(t + T ) = D(t )
The smallest T for which these periodicity conditions are met is called the system period. These state-space models may be generalized and extended in various ways, among which are the class of descriptor models (25). Periodicity Induced by Linearization. As mentioned earlier, a linear periodic system can be used to describe the small perturbation behavior along a periodic regime. For example, consider the continuous-time nonlinear system (t ) = f ( (t ), v(t )) (t ) = h( (t ), v(t ))
( ), and let v ( ), ( ) be an associated periodic regime of period T. This means that ( ) is a periodic solution of period T associated with the periodic input v ( ) and that ( ) is the corresponding periodic output. The linearized equations result in a linear continuous-time system with
u(t ) = v(t ) v (t ), (t ), x(t ) = (t ) y(t ) = (t ) (t )
PERIODIC CONTROL
61
and matrices
A(t ) = C(t ) =
f ( , v ) h( , v )
,v= v =
, ,
B(t ) = D(t ) =
f ( , v ) v h( , v ) v
The overall behavior of the obtained system is ruled by the discrete-time output variables y i(k), i 1, 2 and the discretetime input variables u j(k), j 1, 2 dened as
,v= v =
y i (k ) = y(ky i ), u j (t ) = u j (k ),
i = 1, 2 t [ku j , ku j + u j ), j = 1, 2
,v= v =
,v= v =
These matrices are obviously periodic of period T. Mutatis mutandis, the same reasoning applies in discrete-time as well. The linearization rationale is illustrated in Fig. 1. Periodicity Induced by Multirate Sampling. Multirate schemes arise in digital control and digital signal processing whenever it is necessary to sample the outputs and/or update the inputs with different rates. To explain in simple terms how multirate sampled-data mechanisms generate periodicity, consider a system with two inputs and two outputs described by the time-invariant differential equations
For the modelization, however, it is advisable to introduce the fast-rate signals y (k ) = y(k ), u (k ) = u (k )
In contrast, y (k) and u (k) can be considered as the slow-rate samples. The sampling selector, namely the device operating the passage from the fast sampled-output to the slow sampled-output, is described by the linear equation y ( k ) = N ( k )y (k ) where
x (t ) = Ax(t ) + B
u1 (t ) u2 (t )
u (t ) y1 (t ) = Cx(t ) + D 1 y2 (t ) u2 (t )
The two outputs yi( ), i 1, 2, are sampled with sampling intervals yi, i 1, 2, whereas the two inputs uj( ), j 1, 2, are updated at the end of intervals of length uj, j 1, 2, and kept constant in between. The sampling and updating instants are denoted by tyi(k), i 1, 2 and tuj(k), j 1, 2, k integer, respectively. Typically, these instants are taken as multiples of the basic clock period . Moreover, for simplicity, assume that t y i ( k ) = k y i , t u j ( k ) = k u j with
N (k ) =
n1 (k ) 0
0 n2 (k )
ni (k ) =
1 0
if k is a multiple of y i / otherwise
Note that matrix N( ) is periodic with the period given by the integer Ty dened as the least common multiple of y1 / and y2 / . As for the hold device, introduce the holding selector matrix
S (k ) =
Time-invariant nonlinear system v
s1 ( k ) 0
0 s2 ( k )
= f( ,v) = h ( ,v)
with
s j (k ) =
0 if k is a multiple of u j / 1 otherwise
u=v~ v
Periodic regime ~, ~ ~ u x,y
~ y =
Then, the analog input signal u( ) is given by u(t ) = u (k ), t [k , k + )
(k) is obtained from the slow where the fast updated signal u one u (k) according to the holding selector mechanism
u A(. ), B(. ), C(. ), D( .) Periodic linear system
Figure 1. Linearization around a periodic orbitthe dynamics of the nonlinear system in the orbit vicinity is governed by a linear periodic system.
v(k + 1 ) = S(k )v(k ) + (I S(k ))u (k ) u (k ) = S(k )v(k ) + (I S(k ))u (k ) Matrix S(k) is periodic of period Tu given by the least common multiple of u1 / and u2 / . This situation is schematically illustrated in Fig. 2, where u1 3, u2 4, y1 2, y2 5, so that Tu 12 and Ty 10.
62
PERIODIC CONTROL
~ u
S (.) Holding selector (period 12)
^ u ZOH Zero-order holder
A,B,C,D
^ y
N (.)
~ y
System
Sampler ( )
Sampling selector (period 10)
u1
y1
u2 y2
~ u i (slow updating) ^ u i (fast updating)
~ y i (slow sampling) ^ y i (fast sampling)
Figure 2. A multirate sampled-data system with two inputs and two outputs. The symbol denotes the clock period. The rst output signal y1 is sampled at rate y1 2 and the second y2 at rate y2 5. Hence, the sampling selector is a periodic discrete-time system with period Ty 10. Moreover, the rst input signal u1 is updated at rate u1 3 and the second u2 at rate u2 4. The holding selector is a periodic discrete-time system with period Tu 12. The period of the global system is therefore T 60.
The overall multirate sampled-data system is a discretetime periodic system with state
where
( k ) = e A (k ) = B
A
x (k ) =
x (k ) v (k )
e
0
d BS(k )
and equations
e A d B(I S(k )) I S (k )
S (k )
( k )x ( k )u x (k + 1 ) = A (t ) + B (k ) ( k )x ( k )u y (k ) = C (k ) + D (k )
(k ) = N (k )[C DS(k )] C (k ) = N (k )D(I S(k )) D
PERIODIC CONTROL
63
The so-obtained system is periodic with period T given by the least common multiple of Tu and Ty (T 60 in the example of Fig. 2). Lagrange Formula. The free motion of the periodic system, i.e., the solution of the homogeneous equation
suming that input u( ) over the interval [, t) is known]. This is not true, in general, in discrete-time since the transition matrix is singular when A(i) is singular for some i. Input-Output Representation Another mathematical representation of periodic systems is based on a direct time-domain relationship between the input and the output variables without using any intermediate latent variables. In discrete time, this leads to a model of the form y(t ) = F1 (t )y(t 1 ) + F2 (t )y(t 2 ) + + Fr (t )y(t r ) + G1 (t )u(t 1 ) + G2 (t )u(t 2 ) + + Gs (t )u(t s )
x (t ) = A(t )x(t )
in continuous time
x(t + 1 ) = A(t )x(t ) in discrete time

starting from state x() at time is obtained as x(t ) =
A (t , )x( )
where the transition matrix A(t, ) is given by

A (t , ) =
t=
A(t 1 )A(t 2 ) . . . A( ) t >
in discrete time and by the solution of the differential matrix equation t

A (t , )
= A(t )
A (t , ),
( , ) = I
in continuous time. Therefore, the state solution with a generic initial state x() and input function u( ) is
t
x(t ) =
A (t , )x( )
+
j= +1
A (t ,
j )B ( j 1 )u ( j 1 )
in discrete time and

t
where Fi( ) and Gj( ) are periodic real matrices. Such a representation is frequently used whenever a cyclic model must be estimated from data, as happens in model identication, data analysis, and signal processing. For the passage from a statespace periodic system to an input-output periodic model and vice versa, see, for example, Refs. 26 and 27. Input-output periodic models can also be introduced in continuous time by means of differential equations with time-varying coefcients. Note that state-space or input-output periodic models are used in the stochastic modeling of cyclostationary processes, a type of stochastic nonstationary processes with periodic characteristics. In such a context, the input u( ) is typically a remote signal described as a white noise. Then, the input-output models are known as PARMA models, where PARMA means periodic auto-regressive moving average. In the following sections, the main attention will be focused on state-space models.
x(t ) =
A (t , )x( ) +
A (t , )B ( )u ( )
TIME-INVARIANT REPRESENTATIONS As seen before, periodicity is often the result of ad hoc operations over time-invariant systems. On the other hand, in periodic systems analysis and control, a major point is to address the backward problem of nding a way to transform a periodic system into a time-invariant one. In such a way, we can resort to the results already available in the time-invariant realm. Sample and Hold The simplest way to achieve stationarity is to resort to a sample-and-hold procedure. Indeed, with reference to a continuous or a discrete-time periodic system, suppose that the input is kept constant over a period, starting from an initial time point . That is, u(t ) = u (k ), t [kT + , kT + T + )
in continuous time. These expressions are known as Lagrange formulas (also called variations-of-constants formulas). We can easily see that the periodicity of the system entails the biperiodicity of matrix A(t, ), namely that
A (t
+ T, + T ) =
A (t , )
The transition matrix over one period

A ( )
A (
+ T, )
plays a major role in the analysis of periodic systems and is known as monodromy matrix at time . Reversibility. In continuous time, there is no analytic expression for the transition matrix. However, its determinant can be worked out from the so-called Jacobi formula. In other words,
t
det[
A (t , )]
= exp
trace[A( )] d
Therefore, for any choice of t and , the transition matrix is invertible. This means that the system is reversible, in that the state x() can be uniquely recovered from x(t), t [as-
Then the evolution of the system state sampled at kT [i.e., x(k) x(kT )] is governed by a time-invariant equation in discrete time. Precisely, x ( k + 1 ) =
A (T
+ , )x (k ) + ( )u (k )
64
PERIODIC CONTROL
where
The time-lifted reformulation can then be introduced:
( ) =
T +
T + 1 i=
A (T A (T
+ , )B( ) d + , i + 1 )B ( i )
in continuous time in discrete time
x (k + 1 ) = F x (k ) + G u (k ) y (k ) = H x (k ) + E u (k ) Note that if u( ) is kept constant over the period, then the state equation of this lifted reformulation boils down to the sample-and-hold state equation. In this reformulation, only the output and input vectors were enlarged, whereas the dimension of the state-space was preserved. In the cyclic reformulation, (31,32), every systems signal, say v(t) of dimension q, is transformed into an enlarged signal v(t) of dimension qT. This transformation takes place according to the following rule (given over one period starting from a given initial instant ):
Generalized Sample and Hold. It is possible to generalize the sample-and-hold representation by equipping the holding mechanism with a time-varying periodic modulating function H( ) acting over the period as follows: u(t ) = H (t )u (k ), t [kT + , kT + T + )
In this way, the evolution of the sampled state x (k) is still governed by the previous equations provided that B(t) is replaced by B(t)H(t). Such a generalized sample-and-hold representation allows for a further degree of freedom in the design of periodic controllers. Indeed, function H( ) is a free parameter to be chosen by the designer; see Ref. 28. Lifted and Cyclic Reformulations Despite its interest, the sample-and-hold representation is by no means an equivalent reformulation of the original periodic system because the input function is constrained into the class of piecewise constant signals. Truly equivalent reformulations can be pursued in a number of ways, depending on the transformations allowed, in frequency or in time, on the input, state, and output signals. For ease of explanation, it is advisable to focus on discrete time, where the most important reformulations are timelifted, cyclic, and frequency-lifted representations. The time-lifted reformulation goes back to early papers (29, 30). The underlying rationale is to sample the system state with a sampling interval coincident with the system period T and to organize the input and output signals in packed segments of subsequent intervals of length T, so as to form input and output vectors of enlarged dimensions. That is, let be a sampling tag and introduce the packed input and packed output segments as follows: u (k ) = [u(kT + ) u(kT + + 1 ) . . . u(kT + + T 1 ) ] y (k ) = [y(kT + ) y(kT + + 1 ) . . . y(kT + + T 1 ) ] The vectors u ( ) are known as lifted input and lifted ( ) and y output signals. The introduction of the lifting concept enables us to determine x(k 1) x(kT T ) from x(k) x(kT ) and then to work out y (k) from x(k). More precisely, dene F Rnn, G RnmT, H RpTn, and E RpTmT as
v( ) . . v ( ) = . ,
v( v ( + 1 ) =
+ 1 ) , . . ., . . .
. . v ( + T 2 ) = , . v( + T 2 )
. . v ( + T 1 ) = . v( + T 1 )
where is any element, typically set to zero. Obviously, the previous pattern repeats periodically for the other periods. This signal transformation is used for the input, output, and state of the system. Then, we can relate the cyclic input to the cyclic state by means of a time-invariant state-equation and the cyclic output to the cyclic state via a time-invariant transformation. In this way, we obtain an nT-dimensional time-invariant system with mT inputs and pT outputs. Finally, the frequency-lifted reformulation is based on the following considerations. For a discrete-time (vector) signal v(t), let V(z) be its z-transform. Now, one can associate with V(z) the frequency augmented vector Vf (z) as follows:
F = G = [
A ( ) A ( + T, + 1 )B ( ) A ( + T, + 2 )B ( + 1 ) . . .
B( + T 1 )] H = [C( ) E = {(E )ij }, 0

A (
+ 1, ) C( + 1 ) . . .
A (
+ T 1, ) C( + T 1 ) ] i< j i= j
i, j = 1, 2, . . ., T
V (z ) V (z) V ( z 2 ) V f (z ) = . . . T 1 V ( z )
where e2j / T. By applying this procedure to the z-transforms of the input and output signals of the periodic system, it is possible to establish an input-output correspondence described by a matrix transfer function; see Ref. 33. Such a
(E )ij =
D( + i 1 ) C( + i 1 )
A ( + i 1, + j )B ( + j 1 ) i > j
PERIODIC CONTROL
65
transfer function is referred to as the frequency-lifted representation. The three reformulations are input-output equivalents of each other. Indeed, for any pair of them it is possible to work out a one-to-one correspondence between the input-output signals. For the correspondence between the cyclic and the time-lifted reformulations, see Ref. 22. Lifting and Cycling in Continuous Time In continuous time, the frequency-lifted reformulation can be appropriately worked out as well leading to innite-dimensional time-invariant systems. For example, the time-lifted reformulation appears as in discrete time, but now G, H, and E are linear operators on/from Hilbert spaces. On this topic, the interested reader is referred to Refs. 34 and 35.
Consider now the Fourier series for the periodic matrix coefcients. That is, A(t ) =
kZ
Ak e jk
and similarly for B(t), C(t), and D(t), and plug the expansions of the signals x(t), u(t), x (t) and the matrices A(t), B(t), C(t), D(t) into the system equations. By equating all terms at the same frequency, we obtain an innite-dimensional matrix equation of the following kind: sX = (A N )X + BU Y = CX + DU where X , U , and Y , are doubly innite vectors found with the harmonics of x, u and y respectively, organized in the following fashion: X
T T T T T = [. . ., xT 2 , x 1 , x 0 , x 1 , x 2 , . . .]
PERIODIC SYSTEMS IN FREQUENCY DOMAIN The frequency domain representation is a fundamental tool in the analysis and control of time-invariant linear systems. It is related to the well-known property that, for this class of systems, sinusoidal inputs result into sinusoidal outputs at the same frequency and different amplitude and phase. A similar tool can be worked out for periodic systems by making reference to their response to the so-called exponentially modulated periodic (EMP) signals. Herein, we limit our attention to continuous-time systems. Then, given any complex number s, a (complex) signal u(t) is said to be EMP of period T and modulation s if u(t ) =
kZ
and similarly for U and Y . A , B , C , and D are doubly innite Toeplitz matrices formed with the harmonics of A( ), B( ), C( ), and D( ), respectively, as
uk e s k t
. A =
..
. . . A0 A1 A2 A3 A4 . . .
. . . A 1 A0 A1 A2 A3 . . .
. . . A 2 A 1 A0 A1 A2 . . .
. . . A 3 A 2 A 1 A0 A1 . . .
. . . A 4 A 3 A 2 A 1 A0 . . .
.. .
where sk = s + jk The quantity T 2 / is the named period of the EMP signal. The class of EMP signals is a generalization of the class of T-periodic signals. As a matter of fact, an EMP signal with s 0 is just an ordinary time-periodic signal. Indeed, as it is easy to verify, an EMP signal is such that u(t + T ) = u(t ), = esT
and similarly for B , C , and D . As for matrix N , it is the block diagonal matrix N = blkdiag{ jk I}, kZ
Then, we can dene the harmonic transfer function as the operator (s ) = C [sI (A N )] 1B + D G Such an operator provides a most useful connection between the input harmonics and the output harmonics (organized in the innite vectors U and Y , respectively). In particular, if we take s 0 (so considering the truly periodic regimes), the appropriate input/output operator is G (0 ) = C [N A ] 1B + D If u( ) is a sinusoid, this expression enables us to compute the amplitudes and phases of the harmonics constituting the output signal y( ) in a periodic regime. In general, the input/output operator representation of a periodic system may be somewhat impractical, given that it is innite-dimensional. From an engineering viewpoint, anyway, this model can be satisfactorily replaced by a nitedimensional approximation obtained by truncation of the Fourier series of the system matrices, which in turn implies
In much the same way as a time-invariant system subject to a (complex) exponential input admits an exponential regime, a periodic system of period T subject to an EMP input of the same period admits an EMP regime. In such a regime, all signals of interest can be expanded as EMP signals as follows:
x(t ) =
kZ
xk e s k t s k xk e s k t
kZ
x (t ) = y(t ) =
kZ
yk e s k t
66
PERIODIC CONTROL
that matrices A , B , C , D , and N also are truncated and have therefore nite dimensions. Analyzing the frequency domain behavior of a continuoustime periodic system in terms of the Fourier expansion of its coefcients is a long-standing idea in the eld; see Ref. 36 for a classical reference and a more recent paper (37). Interestingly enough, it can be shown that the discretetime version of this rationale leads to a nite-dimensional time-invariant system whose transfer function coincides with that of the frequency-lifted reformulation. MONODROMY MATRIX AND STABILITY The monodromy matrix A() relates the value of the state in free motion at a given time-point to the value after one period T. Precisely, if u( ) 0 over the considered interval of time, x( + T ) =
A ( )x( )
region (i.e., the left half plane in continuous time and the unit disk in discrete time) and nevertheless the system is unstable. Notable exceptions in continuous time are slowing-varying matrices or high-frequency perturbed matrices, see Refs. 38 and 39, respectively. A celebrated stability condition can be formulated in terms of the so-called Lyapunov equation. There are two possible formulations of such an equation, known as ltering Lyapunov equations and control Lyapunov equation, reecting the fact that Lyapunov equations may arise in the analysis of both ltering and control problems. In continuous time, the ltering Lyapunov equation takes the form (t ) = P(t )A(t ) + A(t )P(t ) + Q(t ) P and the control Lyapunov equation is (t ) = A(t ) P(t ) + P(t )A(t ) + Q(t ) P where Q( ) is a periodic [Q(t T) Q(t), t] and positive denite [xQ(t)x 0, t, x 0] matrix. It turns out that the continuous-time periodic system is stable if and only if the Lyapunov equation (in any of the two forms above) admits a (unique) periodic positive-denite solution P( ). An analogous result holds in discrete time, by making reference to P(t + 1 ) = A(t )P(t )A(t ) + Q(t ) P(t ) = A(t ) P(t + 1 )A(t ) + Q(t ) as ltering and control Lyapunov equations, respectively. As before, Q( ) is periodic and positive denite. The Lyapunov stability theorem can be expressed in a more general form by referring to positive semidenite matrices Q( ) provided that further technical assumptions on the pair (A( ), Q( )) are met with; see Ref. 40 for more details on the theoretical aspects and Ref. 41 for the numerical issues. It is useful to point out that the above Lyapunov stability condition can also be stated in a variety of different forms. In particular, it is worth mentioning that one can resort to the Lyapunov inequality, i.e., in continuous time
Therefore, the sampled state x(k) x( kT) is governed in the free motion by the time-invariant discrete-time equation x ( k + 1 ) =
A ( )x (k )
This is why the eigenvalues of A() play a major role in the modal analysis of periodic systems. In the literature, such eigenvalues are referred to as the characteristic multipliers of A( ). Note that, although the monodromy matrix may depend upon , the characteristic multipliers are constant (21). Moreover, in continuous time, all characteristic multipliers are different from zero as can be easily seen from the Jacobi formula. Conversely, a discrete-time system may exhibit null characteristic multipliers. This happens when at least one among matrices A(i), i 0, 1, . . ., T 1 is singular, so that the system is nonreversible. Obviously the family of periodic systems includes that of time-invariant ones, in which case the monodromy matrix takes the expression
A ( )
e AT AT
Therefore, denoting by an eigenvalue of A, the characteristic multipliers of a time-invariant system seen as a periodic system of period T are given by eT and T in continuous time and discrete time, respectively. In general, the monodromy matrix is the basic tool in the stability analysis of periodic systems. Indeed, any free motion goes to zero asymptotically if and only if all characteristic multipliers have modulus lower than one. Hence, a periodic system (in continuous or discrete time) is stable if and only if its characteristic multipliers belong to the open unit disk. To be more precise, this stability concept is usually referred to as asymptotic stability. However, there is no need for this article to introduce all possible notions of stability, so the attribute asymptotic is omitted for the sake of conciseness. Notice that there is no direct relation between the eigenvalues of A(t) and the system stability. In particular, it may well happen that all eigenvalues of A(t) belong to the stable
(t ) > A(t )P(t ) + P(t )A(t ) P
(ltering)
(t ) > A(t ) P(t ) + P(t )A(t ) (control) P

and in discrete time P(t + 1 ) > A(t )P(t )A(t ) (ltering)
P(t ) > A(t ) P(t + 1 )A(t ) (control) Here it is meant that, given two square matrices M and N, M N is equivalent to saying that M N is positive denite. Then, an equivalent stability condition is that the system is stable if and only if the Lyapunov inequality admits a periodic positive denite solution. The advantage of expressing the condition in this form is that no auxiliary matrix Q( ) is required.
PERIODIC CONTROL
67
Cyclostationary Processes In the stochastic realm, the so-called cyclostationary processes are well suited to deal with pulsatile random phenomena and are the subject of intense investigation in signal processing; see Ref. 42. Specically, a stochastic process with periodic mean and covariance function (t, ) satisfying the biperiodicity condition (t T, T) (t, ) is said to be a cyclostationary process. In particular, its variance (t, t) is T-periodic. The periodic Lyapunov equation serves as a fundamental tool in the analysis of these processes. Assume that the initial state x(0) of the system is a random variable with zero mean and covariance matrix P0, and assume also that the input of the system is a white-noise process, independent of x(0), with zero mean and unitary intensity. Then (21), under the stability assumption, the state of the periodic system asymptotically converges to a zero mean cyclostationary process with variance (t, t), which can be computed via the periodic ltering Lyapunov equation by letting Q(t) B(t)B(t) and P(0) P0. It turns out that
t
tion, as can easily be seen in the simple case T 2, A(0) 0, A(1) 1, for which the equation S(t 1)A(t)S(t)1 constant does not admit any solution. In the reversible case, such a representation always exists, can be obtained by solving A T A() and the and matrix A transformation S( ) is given by t (t , ) 1 S(t ) = A Again, it can be seen that such S( ) is periodic of period T and satises the linear difference equation (t )A(t ) 1 S(t + 1 ) = AS with initial condition S() I. Whenever a Floquet representation exists, the eigenvalues are named characteristic exponents. In continuous time, of A the correspondence between a characteristic multiplier z and a characteristic exponent s is z esT, whereas in discrete time, such a correspondence is z sT. Main references for the Floquet theory and stability issues are Refs. 24, 36, 44, and 45. It should be emphasized that Floquet theory does not consider systems driven by external inputs. This nontrivial extension is touched upon in the sequel. PERIODIC CONTROL The early developments of periodic control were concentrated on the problem of forcing a periodic regime in order to improve the performance of an industrial plant (periodic optimization). At present, the term periodic control has taken a wider sense, so as to include the design of control systems where the controller and/or the plant are described by periodic models. Periodic Optimization In the 1970s, it was observed that there is evidence that periodic operation [of catalytic reactors] can produce more reaction products or more valuable distribution of products, [and that] the production of wastes can perhaps be suppressed by cycling (4). Ever since, the same idea has been further elaborated in other application elds, such as aeronautics (5,46,47), solar energy control (48), and social and economic sciences (49). This list of papers is largely incomplete, but in the bibliography the interested reader can nd many more useful references. In continuous time, the basic periodic optimization problem can be stated as follows. Consider the system x (t ) = f (x(t ), u(t )) y(t ) = h(x(t )) subject to the periodicity constant
lim { (t , t ) P(t )} = 0
Floquet Theory One of the long-standing issues in periodic systems is whether it is possible to nd a state-coordinate transformation leading to a periodic system with constant dynamic matrix. In this way, the eigenvalues of such a dynamic matrix would determine the modes of the system. With reference to linear differential equations, this issue was considered by various mathematicians of the nineteenth century. Among them, a prominent role was played by the French scientist Gaston Floquet (18471920) who worked out a theory to solve linear homogeneous periodic systems, which is now named after him (43). This theory can be outlined in a simple form as follows. If S( ) is a T-periodic invertible state-space transformation, (t) x (t) S(t)x(t), then, in the new coordinates, the dynamic A is given by
(t ) = A
(t )S(t ) 1 S(t )A(t )S(t ) 1 + S S(t + 1 )A(t )S(t )

1
The Floquet problem is then to nd S(t) (if any) in order to (t) A . obtain a constant dynamic matrix A In continuous time, it can be shown that such a transformation S( ) does exist, and the Floquet problem can be T can be obtained by solving eA A(), solved. Indeed, A where is any given time point. The appropriate transformation S( ) is simply given by S(t ) = e A (t )
A ( , t )
Such a matrix is indeed periodic of period T and satises the linear differential equation (t ) = AS (t ) S(t )A(t ) S with initial condition S() I. The discrete-time case is rather involved. Indeed, certain nonreversible systems do not admit any Floquet representa-
x (T ) = x (0 ) and to further constraints of integral or pathwise type. The performance index to be maximized is J= 1 T
T
g(x(t ), u(t )) dt
0
68
PERIODIC CONTROL
If we limit the desired operation to steady-state conditions, then an algebraic optimization problem arises, which can be tackled with mathematical programming techniques. Indeed, letting u(t) const u and x(t) const x, the problem becomes that of maximizing J g(x, u) under the constraint f (x, u) 0. When passing from steady-state to periodic operations, an important preliminary question is whether the optimal steady-state regime can be improved by cycling or not. Denoting by u0 the optimal input at the steady state, consider the perturbed signal u(t ) = u 0 + u where u(t) is a periodic perturbation. A problem for which there exists a (nonzero) periodic perturbation with a better performance is said to be proper. The issue of proper periodicity was originally dealt with in Refs. 50 and 51, by means of calculus of variation concepts. The underlying rationale is to express u( ) in its Fourier expansion: u =
is that of maximizing the output power J= 1 T

T
y(t ) y(t ) dt
0
under the periodicity state constraint x (0 ) = x (T ) and the input power constraint 1 T
T
u(t ) u(t ) dt 1
0
then the optimal input function is given by a sinusoidal signal of suitable frequency. In the single-input single-output case, denoting by G(s) the system transfer function, the optimal frequency is that associated with the peak value of the Bode diagram |G( j | |G( j )|, (In modern jargon, is the value of the frequency associated with the H norm of the system.) In particular, the problem is proper if 0. Otherwise, the optimal steady-state operation cannot be improved by cycling. Periodic Control of Time-Invariant Systems The application of periodic controllers to time-invariant linear plants has been treated in an extensive literature. Again, the basic concern is to solve problems otherwise unsolvable with time-invariant controllers or to improve the achievable control performances. A typical line of reasoning adopted in this context can be explained by referring to the classical output stabilization problem, namely the problem of nding an algebraic feedback control law based on the measurements of the output signal in order to stabilize the overall control system. If the system is x (t ) = Ax(t ) + Bu(t ) y(t ) = Cx(t ) with the control law u(t ) = Fy(t ) the problem is to nd a matrix F (if any) such that the closedloop system x (t ) = (A + BFC )x(t ) is stable. Although a number of necessary and sufcient conditions concerning the existence of a stabilizing matrix F have been provided in the literature, no effective algorithms are available for its determination, as discussed in Ref. 61. Moreover, it may be difcult or impossible to stabilize three linear time-invariant plants (62). Periodic sampled control seems to offer a practical way to tackle this problem. Indeed, consider the time-varying control law based on the sampled measurements of y( ) u(t ) = F (t )y(kT ), t [kT, kT + T )
Uk e
k =
jk t
2 = T
By means of variational techniques, it is possible to work out a quadratic expression for the second variation of the performance index 2J =
k = Uk (k )Uk
where U* k is the conjugate transpose of Uk. Matrix () is a complex square matrix dened on the basis of the system equations linearized around the optimal steady-state regime and on the basis of the second derivatives of the so-called Hamiltonian function associated with the optimal control problem, again evaluated at the optimal steady-state regime. Notice that () turns out to be a Hermitian matrix, namely it coincides with its conjugate transpose [*() ()]. Thanks to the preceding expression of 2J, it is possible to work out a proper periodicity condition in the frequency domain, known as -test. Basically, this test says that the optimal control problem is proper if, for some 0, () is partially positive [i.e., there exists a vector x 0 such that x*()x 0]. The partial positivity of () is also a necessary condition for proper periodicity if we consider the weak variations u( ). In the single-input single-output case, the test can be given a graphical interpretation in the form of a circle criterion. Variational tools have been used in periodic control by many authors (see Refs. 5257). Moreover, along a similar line, it is worth mentioning the area of vibrational control (see Ref. 58), dealing with the problem of forcing a time-invariant system to undertake a periodic movement in order to achieve a better stabilization property or a better performance specication. In general, if we leave the area of weak variations, periodic optimization problems do not admit closed-form solutions. There is, however, a notable exception, as pointed out in Refs. 59 and 60. With reference to a linear system, if the problem
PERIODIC CONTROL
69
The modulating function F( ) and the sampling period T have to be selected in order to stabilize the closed-loop system, now governed by the equation x(kT + T ) = Ac x(kT ) where
T
Periodic Control of Periodic Systems A typical way to control a plant described by a linear periodic model is to impose u(t ) = K (t )x(t ) + S(t )v(t ) where K( ) is a periodic feedback gain [K(t T) K(t), t], S ( ) is a periodic feedforward gain [S(t T) S(t), t], and v(t) is a new exogenous signal. The associated closed-loop system is then x (t ) = (A(t ) + B(t )K (t ))x(t ) + B(t )S(t )v(t ) in continuous time and x(t + 1 ) = (A(t ) + B(t )K (t ))x(t ) + B(t )S(t )v(t ) in discrete time. In particular, the closed-loop dynamic matrix is the periodic matrix A(t) B(t)K(t). The main problems considered in the literature follow: 1. Stabilization. Find a periodic feedback gain in such a way that the closed-loop system is stable [any K( ) meeting such a requirement is named stabilizing gain]. 2. Pole Assignment. Find a periodic feedback gain so as to position the closed-loop characteristic multipliers in given locations in the complex plane. 3. Optimal Control. Set v( ) 0 and nd a periodic feedback gain so as to minimize the quadratic performance index
Ac = e AT +
0
e A (T ) BF ( )C d
The crucial point is the selection of matrix F( ) for a given period T. A possibility, originally proposed in Ref. 28, is to consider an F( ) given by the following expression
F (t ) = B eA
(T t ) 0
e A (T ) BB e A
(T )
with matrix Z still to be specied. Note that this formula is valid provided that the matrix inversion can be performed (this is indeed the case under the so-called reachability condition). In this way, the closed-loop matrix Ac takes the form Ac = e AT + ZC Then, provided that some weak condition on the pair (A, C) is met, period T and matrix Z can be selected so as to stabilize Ac, (or, even, to assign its eigenvalues). The generalized sample-and-hold philosophy outlined previously in the simple problem of stabilization has been pursued in many other contexts, ranging from the problem of simultaneous stabilization of a nite number of plants (28) to that of xed poles removal in decentralized control (63), from the issue of pole and/or zero-assignment (6469), to that of gain margin or robustness improvement (7,70), from adaptive control (71) to model matching (28), and so on. When using generalized sample-data control, however, the intersample behavior can present some critical aspects, as pointed out in several papers, such as Refs. 72 and 73. Indeed, the action of the generalized sample-and-hold function is a sort of amplitude modulation, which, in the frequency domain, may lead to additional high-frequency components centered on multiples of the sampling frequency. Consequently, there are nonnegligible high-frequency components both in the output and control signals. To smooth out these ripples, remedies have been studied, see Refs. 9 and 74. An obvious possibility is to continuously monitor the output signal and to adopt the feedback control strategy u(t) F(t)y(t), with a periodic gain F(t), in place of the sampled strategy before seen. This point of view is adopted in Ref. in 75, where a poleassignment problem in discrete time is considered. In the control of time-invariant systems, linear-quadratic optimal control theory represents a cornerstone achievement of the second half of the twentieth century. We can wonder whether, by enlarging the family of controllers from the timeinvariant class to the class of periodic controllers, the achievable performance can be improved. To this question, the reply may be negative, even in the presence of bounded disturbances, as argued in Ref. 76.
J=
0 k=0
[x(t ) Q(t )x(t ) + u(t ) R(t )u(t )] dt
x ( k ) Q ( k ) x ( k ) + u ( k ) R ( k )u ( k )
4. Invariantization. Find a feedback control law such that the closed-loop system is time-invariant up to a periodic state-space coordinate change. 5. Exact Model Matching. Let y(t) C(t)x(t) be a system output variable. Find a feedback control law such that the closed loop input-output relation [from v( ) to y( )] matches the input-output behavior of a given periodic system. 6. Tracking and Regulation. Find a periodic controller in order to guarantee closed-loop stability and robust zeroing of the tracking errors for a given class of reference signals. We now briey elaborate on these problems by reviewing the main results available in the literature. As a paradigm problem in control, the stabilization issue is the starting point of further performance requirement problems. A general parametrization of all periodic stabilizing gains can be worked out by means of a suitable matrix inequality. Specically, by making reference to discrete time, the ltering Lyapunov inequality seen in the section devoted to the monodromy matrix and stability enables us to conclude that the closed-loop system associated with a periodic gain K( ) is stable if and only if there exists a positive denite
70
PERIODIC CONTROL
periodic matrix Q( ) satisfying the inequality: Q(t + 1 ) > (A(t ) + B(t )K (t ))Q(t )(A(t ) + B(t )K (t )) , t Then, it is possible to show that a periodic gain is stabilizing if and only if it can be written in the form: K (t ) = W (t ) Q(t ) 1 where W( ) and Q( ) 0 are periodic matrices (of dimensions m n and n n, respectively), solving the matrix inequality Q(t + 1 ) > A(t )Q(t )A(t ) + B(t )W (t ) A(t ) + A(t )W (t )B(t ) + B(t )W (t ) Q(t ) 1W (t )B(t ) , t that can be equivalently given in a linear matrix inequality (LMI) form. The pole assignment problem (by state feedback) is somehow strictly related to the invariantization problem. Both problems have been considered in an early paper (77), where continuous-time systems are treated, and subsequently in Refs. 7880. The basic idea is to render the system algebraically equivalent to a time-invariant one by means of a rst periodic state feedback (invariantization) and then to resort to the pole assignment theory for time-invariant systems in order to locate the characteristic multipliers. Thus, the control scheme comprises two feedback loops, the inner for invariantization and the outer for pole placement. Analogous considerations can be applied in the discretetime case (81), with some care for the possible nonreversibility of the system. The model matching and the tracking problems are dealt with in Refs. 82 and 83, respectively. Finally, the optimal control approach to periodic control deserves an extensive presentation and therefore is treated in the next section. PERIODIC OPTIMAL CONTROL As for the vast area of optimal control, attention focuses herein on two main design methodologies, namely (i) linear quadratic control and (ii) receding horizon control. Both will be presented by making reference to continuous time. Linear Quadratic Periodic Control For a continuous-time periodic system, the classical nite horizon optimal control problem is that of minimizing the quadratic performance index over the time interval (t, tf): J (t , tf , xt ) = x(tf ) Pt x(tf ) +
f
(t) and D (t) are to be tuned by the designer. In this where C way, the performance index can also be written in the (perhaps more popular) form
J (t , tf , xt ) = x(tf ) Pt x(tf ) +
f
tf t
{x( ) Q( )x( )
+ 2u( ) S( )x( ) + u( ) R( )u( )} d

()C (), S() D (), and R() ()C where Q() C ()D (). We will assume for simplicity that the problem is D nonsingular [i.e., R() 0, ]. This problem is known as the linear quadratic (LQ) optimal control problem. To solve it, the auxiliary matrix equation (t ) P(t ) + P(t )A (t ) P(t )B(t )R(t ) 1B(t ) P(t ) + Q (t ) (t ) = A P where (t ) = A(t ) B(t )R(t ) 1 S(t ), A (t ) = Q(t ) S(t ) R(t ) 1 S(t ) Q
is introduced. This is the well-known differential Riccati equation, in one of its many equivalent forms. More precisely, because the coefcients are periodic, the equation is referred to as the periodic differential Riccati equation. Let ( , tf) be the backward solution of the periodic Riccati equation with terminal condition (tf, tf) Ptf . Assuming that the state x( ) can be measured, the solution to the minimization problem can be easily written in terms of ( , tf) as follows u( ) = where
o ( , tf o ( , tf
)x( )
) = R( ) 1 [B( )
( , tf ) + S( )]
Moreover, the value of the performance index associated with the optimal solution is J o (t , tf , xt ) = xt (t , tf )xt The passage from the nite horizon to the innite horizon problem (tf ) can be performed provided that (t, tf) remains bounded for each tf t and converges as tf : In other words, if there exists P(t) such that
tf
lim
(t , tf ) = P(t ), t
tf t
z ( ) z ( ) d
where xt is the system initial state at time t, Ptf 0 is the matrix weighting the nal state x(tf), and z( ) is a performance evaluation variable. Considering that the second term of J(t, tf, xt) is the energy of z( ), the denition of such a variable reects a main design specication. A common choice is to select z(t) as a linear combination of x(t) and u(t), such that (t )x(t ) + D (t )u(t ) z (t ) = C
Under suitable assumptions concerning the matrices [A( ), B( ), Q( ), S( )], the limit matrix P( ) exists and is the unique positive semidenite and T-periodic solution of the periodic differential Riccati equation. The optimal control action is given by u( ) = Ko ( )x( ) where Ko() is the periodic matrix obtained from o(, tf) by letting tf . Finally, the optimal innite horizon performance index takes on the value
tf
lim J o (t , tf , xt ) = xt P(t )xt
PERIODIC CONTROL
71
If the state is not accessible, we must rely instead on the measurable output y(t ) = C(t )x(t ) + D(t )u(t ) A rst task of the controller is then to infer the actual value of the state x(t) from the past observation of y( ) and u( ) up to time t. This leads to the problem of nding an estimate (t) of x(t) as the output of a linear system (lter) fed by the x available measurements. The design of such a lter can be carried out in a variety of ways, among which it is worth mentioning the celebrated Kalman lter, the implementation of which requires the solution of another matrix Riccati equa (t) is available, the contion with periodic coefcients. When x trol action is typically obtained as u( ) = Ko ( )x ( ) Thus, the control scheme of the controller takes the form of a cascade of two blocks, as can be seen in Fig. 3. Periodic optimal ltering and control problems for periodic systems have been intensively investigated (see Refs. 8493). For numerical issues see, for example, Ref. 94. Receding Horizon Periodic Control The innite horizon optimal control law can be implemented provided that the periodic solution of the matrix Riccati equation is available. Finding such a solution may be computationally demanding so that the development of simpler control design tools has been considered. Among them, an interesting approach is provided by the so-called receding horizon control strategy, which has its roots in optimal control theory and remarkable connections with the eld of adaptive and predictive control (see Refs. 95 and 96). Among the many research streams considered in such a context, the periodic stabilization of time-invariant systems is dealt with in Ref. 97 under the heading of intervalwise receding horizon control; see also Ref. 96. The approach was then extended to periodic
systems in Ref. 98. The problem can be stated as follows. Consider the optimal control problem with S( ) 0 and R( ) I, and write the performance index over the interval (tf T, tf) as
J = x(tf ) Pt x(tf ) +
f
tf tf T
{x( ) Q( )x( ) + u( ) u( )} d
Assume that the solution ( , tf) of the Riccati equation with terminal condition (tf, tf) Ptf is such that
tf
= Pt
f
(t f T, tf ) 0
This condition is usually referred to as cyclomonotonicity condition. Now, consider the periodic extension Pe( ) of ( , tf) Pe (t + kT ) = (t , tf ), t (tf T, tf ], integer k
Then, under mild assumptions on [A( ), B( ), Q( )], it turns out that the receding horizon control law u( ) = B( ) Pe ( )x( ) is stabilizing. Although such a control law is suboptimal, it has the advantage of requiring the integration of the Riccati equation over a nite interval (precisely over an interval of length T, which must be selected by the designer in the case of time-invariant plants and coincides with the system periodor a multiplein the periodic case). However, for feedback stability, it is fundamental to check if the cyclomonotonicity condition is met. If not, we are led to consider a different selection of matrix Ptf . Some general guidelines for the choice of Ptf can be found in the literature. The simplest way is to 1 choose Ptf indenitely large, ideally such that P tf 0. Indeed, as is well known in optimal control theory, such condition guarantees that the solution of the differential Riccati equation enjoys the required monotonicity property. CONCLUSION Optimal control ideas have been used in a variety of contexts, and have been adequately shaped for the needs of the specic problem dealt with. In particular, ad hoc control design techniques have been developed for the rejection or attenuation of periodic disturbances, a problem of major importance in the emerging eld of active control of vibrations and noise; see Refs. 99 and 100. BIBLIOGRAPHY
1. F. J. Horn and R .C. Lin, Periodic processes: A variational approach, I and EC Proc. Des. Development, 6: 2130, 1967. 2. J. E. Bailey and F. J. Horn, Improvement of the performances of a xed-bed catalytic reactor by relaxed steady-state operation, AIChE J., 17: 550553, 1971. 3. J. E. Bailey, Periodic operation of chemical reactor: A review, Chem. Eng. Commun., 1: 111124, 1973. 4. J. E. Bailey, The importance of relaxed control in the theory of periodic optimization, in A. Marzollo, (ed.), Periodic Optimization, pp. 3561, New York: Springer-Verlag, 1972.
Optimal periodic gain (Control Riccati eq.)
A(.), B( .), C( .), D(.)
Plant
^ x
Periodic Kalman filter (Filtering Riccati eq.)
Overall periodic controller

Figure 3. Periodic optimal control based on the measurement of the output signal. The controller is constituted of two blocks. The rst one (Kalman lter) elaborates the external signals of the plant (u of the unmeasureable state. The secand y) to provide an estimate x ond block consists in an algebraic gain providing the command u from the estimated state x .
72
PERIODIC CONTROL 5. J. L. Speyer, On the fuel optimality of cruise, AIAA J. Aircraft, 10: 763764, 1973. 6. G. Sachs and T. Christopoulou, Reducing fuel consumption of subsonic aircraft by optimal cycling cruise, AIAA J. Aircraft, 24: 616622, 1987. 7. P. P. Khargonekar, K. Poolla, and A. Tannembaum, Robust control of linear time-invariant plants using periodic compensation, IEEE Trans. Autom. Control, AC-30: 10881096, 1985. 8. M. A. Dahleh, P. G. Vulgaris, and L. S. Valavani, Optimal and robust controllers for periodic and multirate systems, IEEE Trans. Autom. Control, 37: 9099, 1992. 9. M. Araki, Recent developments in digital control theory, 12th IFAC World Cong., Sidney, vol. 9, 251260, 1993. 29. E. I. Jury and F. J. Mullin, The analysis of sampled-data control systems with a periodic time-varying sampling rate, IRE Trans. Autom. Control, 24: 1521,1959. 30. R. A. Mayer and C. S. Burrus, Design and implementation of multirate digital lters, IEEE Trans. Acoust. Speech Signal Process., 1: 5358, 1976. 31. B. Park and E. I. Verriest, Canonical forms for discrete-linear periodically time-varying systems and a control application, 28th IEEE Conf. Decision Control, Tampa, pp. 12201225,1989. 32. D. S. Flamm, A new shift-invariant representation for periodic systems, Syst. Control Lett., 17: 914, 1991. 33. C. Zhang, J. Zhang, and K. Furuta, Performance analysis of periodically time-varying controllers, 13th IFAC World Cong., San Francisco, pp. 207212, 1996. 34. P. Colaneri, Hamiltonian matrices for lifted systems and periodic Riccati equations in H2 / H analysis and control, 30th IEEE Conf. Decision and Control, Brighton, UK, pp. 19141919, 1991. 35. B. A. Bamieh and J. B. Pearson, A general framework for linear periodic systems with application to H sampled data control, IEEE Trans. Autom. Control, 37: 418435, 1992. 36. H. DAngelo, Linear Time-varying Systems: Analysis and Synthesis, Boston: Allyn and Bacon, 1970. 37. N. M. Wereley and S. R. Hall, Frequency response of linear time periodic systems, 29th IEEE Conf. Decision Control, Honolulu, 1990. 38. C. A. Desoer, Slowing varying systems x A(t)x, IEEE Trans. Autom. Control, 14: 780781, 1969. 39. R. Bellman, J. Bentsman, and S. M. Meerkov, Stability of fast periodic systems, IEEE Trans. Autom. Control, 30: 289291, 1985. 40. P. Bolzern and P. Colaneri, The periodic Lyapunov equation, SIAM J. Matrix Anal. Application, 9: 499512, 1988. 41. A. Varga, Periodic Lyapunov equations: Some application and new algorithms, Int. J. Control, 67: 6987, 1997. 42. W. A. Gardner (ed.), Cyclostationarity in Communications and Signal Processing, New York: IEEE Press, 1994. 43. G. Floquet, Sur les equations differentielles lineaires a coefcients periodiques, Annales de lEcole Normale Superieur, 12: 4789, 1883. 44. A. Halanay, Differential EquationsStability, Oscillations, Time-lags, New York: Academic Press, 1986. 45. P. Van Dooren and J. Sreedhar, When is a periodic discretetime system equivalent to a time-invariant one?, Linear Algebra and its Applications, 212/213: 131151, 1994. 46. E. G. Gilbert, Periodic control of vehicle cruise: Improved fuel economy by relaxed steady-state, quasi steady-state and quasi relaxed steady-state control, Automatica, 12: 159166, 1976. 47. J. Speyer, Non-optimality of steady-state cruise for aircraft, AIAA J., 14: 16041610, 1976. 48. P. Dorato and H. K. Knudsen, Periodic optimization with applications to solar energy control, Automatica, 15: 673679, 1979. 49. G. Feichtinger and A. Novak, Optimal pulsing in advertising diffusion models, Optim. Control Appl. Meth., 15: 267276, 1994. 50. G. Guardabassi, Optimal steady-state versus periodic operation: A circle criterion, Ricerche di Automatica, 2: 240252, 1971. 51. S. Bittanti, G. Fronza, and G. Guardabassi, Periodic control: A frequency domain approach, IEEE Trans. Autom. Control, 18: 3338, 1973. 52. S. Bittanti, A. Locatelli, and C. Maffezzoni, Second variation methods in periodic optimization, J. Opt. Theory Appl., 14: 31 49, 1974. 53. E. G. Gilbert, Optimal periodic control: A general theory of necessary conditions, SIAM J. Control Optimization, 15: 717746, 1977.
10. W. Johnson, Helicopter Theory, Princeton, NJ: Princeton University Press, 1981. 11. D. P. Schrage and D. A. Peters, Effect of structural coupling of parameters on the ap-lag forced response of rotor blades in forward ight using Floquet theory, Vertica, 3 (2): 177185, 1979. 12. R. M. McKillip, Periodic control of individual blade control helicopter rotor, Vertica, 9 (2): 199225, 1985. 13. D. Teves, G. Niesl, A. Blaas, and S. Jacklin, The role of active control in future rotorcraft, 21st Eur. Rotorcraft Forum, Saint Petersburg, Russia, III.10.117, 1995. 14. W. Wiesel, Perturbation theory in the vicinity of a stable periodic orbit, Celestial Mechanics, 23: 231242, 1981. 15. J. Hauser and C. C. Chunt, Conserse Lyaupov functions for exponentially stable periodic orbits, Systems and Control Letters, 23: 2734, 1994. 16. C. C. Chung and J. Hauser, Nonlinear H control around periodic orbits, Syst. Control Lett., 30: 127137, 1997. 17. G. O. Guardabassi, A. Locatelli, and S. Rinaldi, Status of periodic optimization of dynamical systems, J. Opt. Theory Appl., 14 120, 1974. 18. E. Noldus, A survey of optimal periodic control of continuous systems, Journal A, 16: 1116, 1975. 19. G. Guardabassi, The optimal periodic control problem, Journal A, 17: 7583, 1976. 20. S. Bittanti and G. O. Guardabassi, Optimal periodic control and periodic system analysis: An overview, 25th IEEE Conf. Decision Control, pp. 14171423, 1986. 21. S. Bittanti, Deterministic and stochastic periodic systems, in S. Bittanti (ed.), Time Series and Linear Systems, Lecture Notes in Control and Information Sciences, vol. 86, pp. 141182, Berlin: Springer-Verlag, 1986. 22. S. Bittanti and P. Colaneri, Analysis of discrete-time linear periodic systems, in C. T. Leondes (ed.), Control and Dynamic Systems, vol. 78, San Diego: Academic Press, 1996. 23. A. Marzollo, Periodic Optimization, vols. 1, 2, New York: Springer Verlag, 1972. 24. V. A. Yakubovich and V. M. Starzhinskii, Linear Differential Equations with Periodic Coefcients, New York: Wiley, 1975. 25. T. Kaczorek, Two-Dimensional Linear Systems, New York: Springer Verlag, 1995. 26. S. Bittanti, P. Bolzern, and G. O. Guardabassi, Some critical issues on the state representation of time-varying ARMA models, IFAC Symp. Identication System Parameter Estimation, York, pp.15791583, 1985. 27. P. Colaneri and S. Longhi, The minimal realization problem for discrete-time periodic systems, Automatica, 31: 779783, 1995. 28. P. T. Kabamba, Control of linear systems using generalized sampled-data hold functions, IEEE Trans. Autom. Control, 32: 772783, 1987.
PERIODIC CONTROL 54. D. S. Bernstein and E. G. Gilbert, Optimal periodic control: The -test revisted, IEEE Trans. Autom. Control, 25: 673684, 1980. 55. J. Speyer and R. T. Evans, A second variational theory of optimal periodic processes, IEEE Trans. Autom. Control, 29: 138 148, 1984. 56. Q. Wang and J. Speyer, Optimal periodic control: A general theory of necessary conditions, SIAM J. Control and Optimization, 15: 717746, 1990. 57. Q. Wang, J. Speyer, and L. D. Dewell, Regulators for optimal periodic processes, IEEE Trans. Autom. Control, 40: 1777 1778, 1995. 58. S. Lee, S. Meerkov, and T. Runolfsson, Vibrational feedback control: Zero placement capabilities, IEEE Trans. Autom. Control, 32: 604611, 1987. 59. S. Bittanti, G. Fronza, and G. Guardabassi, Periodic optimization of linear systems under control power constraints, Automatica, 9 (1): 269271, 1973. 60. S. Bittanti and G. Guardabassi, Optimal cyclostationary control: A parameter-optimization frequency domain approach, VIII IFAC World Congr., 6: 191196, 1981. 61. V. L. Syrmos, C. T. Abdolloh, D. Doreto, and K. Grigoriedis, Static output feedbackA survey, Automatica, 33: 125137, 1997. 62. V. Blondel and M. Gevers, Simultaneous stabilization of three linear systems is rationally undecidible, Math. Control, Signals Syst., 6: 135145, 1994. 63. B. D. O. Anderson and J. Moore, Time-varying feedback laws for decentralized control, IEEE Trans. Autom. Control, 26: 1133 1139, 1981. 64. A. B. Chammas and C. T. Leondes, On the design of linear timeinvariant systems by periodic output feedback, Int. J. Control, 27: 885903, 1978. 65. B. A. Francis and T. T. Georgiou, Stability theory for linear time-invariant plants with periodic digital controllers, IEEE Trans. Autom. Control, 33: 820832, 1988. 66. H. M. Al-Rahmani and G. F. Franklin, Linear periodic systems: Eigenvalues assignment using discrete periodic feedback, IEEE Trans. Autom. Control, 34: 99103, 1989. 67. A. B. Chammas and C. T. Leondes, Pole assignment by piecewise constant output feedback, Int. J. Control, 44: 1661 1673, 1986. 68. J. L. Willems, V. Kucera, and P. Brunovski, On the assignment of invariant factors by time-varying feedback strategies, Syst. Control Lett., 5: 7580, 1984. 69. T. Kaczorek, Pole placement for linear discrete-time systems by periodic output feedbacks, Syst. Control Lett., 6: 267269, 1985. 70. A. W. Olbrot, Robust stabilization of uncertain systems by periodic feedback, Int. J. Control, 45 (3): 747758, 1987. 71. R. Ortega and G. Kreisselmeier, Discrete-time model reference adaptive control for continuous time systems using generalized sample data hold functions, IEEE Trans. Autom. Control, 35: 334338, 1990. 72. K. L. Moore, S. P. Bhattacharyya, and M. Dahleh, Capabilities and limitations of multirate control schemes, Automatica, 29: 941951, 1993. 73. A. Feuer and C. A. Goodwin, Generalized sampled-data functions: Frequency domain analysis of robustness, sensitivity, and intersample difculties, IEEE Trans. Autom. Control, 39: 1042 1047, 1994. 74. A. Feuer, C. A. Goodwin, and M. Salgado, Potential benets of hybrid control for linear time-invariant plants, Amer. Control Conf., Albuquerque, 1997.
73
75. D. Aeyels and J. Willems, Pole assignment for linear time-invariant systems by periodic memoryless output feedback, Automatica, 28: 11591168, 1992. 76. H. Chapellat and M. Dahleh, Analysis of time-varying control strategies for optimal disturbance rejection and robustness, IEEE Trans. Autom. Control, 37: 17341745, 1992. 77. P. Brunovski, A classicaton of linear controllable systems, Kybernetika, 3: 173187, 1970. 78. M. Kono, Pole-placement problem for discrete-time linear periodic systems, Int. J. Control, 50 (1): 361371, 1989. 79. V. Hernandez and A. Urbano, Pole-placement problem for discrete-time linear periodic systems, Int. J. Control, 50: 361 371, 1989. 80. P. Colaneri, Output stabilization via pole-placement of discretetime linear periodic systems, IEEE Trans. Autom. Control, 36: 739742, 1991. 81. O. M. Grasselli and S. Longhi, Pole placement for nonreachable periodic discrete-time systems, Math. Control, Signals Syst., 4: 437453, 1991. 82. P. Colaneri and V. Kucera, The model matching problem for discrete-time periodic systems, IEEE Trans. Autom. Control, 42: 14721476, 1997. 83. O. M. Grasselli and S. Longhi, Robust tracking and regulation of linear periodic discrete-time systems, Int. J. Control, 54: 613 633, 1991. 84. J. R. Canabal, Periodic geometry of the Riccati equation, Stochastics, 1: 432435, 1974. 85. E. C. Bekir and R.S. Bucy, Periodic equilibria for matrix Riccati equations, Stochastics, 2: 1104, 1976. 86. H. Kano and T. Nishimura, Periodic solutions of matrix Riccati equations with detectability and stabilizability, Int. J. Control, 29: 471481, 1979. 87. S. Bittanti, P. Colaneri, and G. Guardabassi, Periodic solutions of periodic Riccati equations, IEEE Trans. Autom. Control, 29: 665667, 1984. 88. V. Hernandez and L. Jodar, Boundary problems for periodic Riccati equations, IEEE Trans. Autom. Control, 30: 11311133, 1985. 89. M. A. Shayman, On the phase portrait of the matrix Riccati equation arising from the periodic control problem, SIAM J. Control Optim., 23: 717751, 1985. 90. S. Bittanti, P. Colaneri, and G. Guardabassi, Analysis of the periodic Riccati equation via canonical decomposition, SIAM J. Control Optim., 24: 11381149, 1986. 91. C. E. de Souza, Riccati differential equation in optimal ltering of periodic non-stabilizing systems, Int. J. Control, 46: 1235 1250, 1987. 92. S. Bittanti, P. Colaneri, and G. De Nicolao, The difference Riccati equation for the periodic prediction problem, IEEE Trans. Autom. Control, 33: 706712, 1988. 93. S. Bittanti, A. J. Laub, and J. C. Willems (eds.), The Riccati Equation, Berlin: Springer Verlag, 1991. 94. J. J. Hench and A. J. Laub, Numerical solution of the discretetime periodic Riccati equation, IEEE Trans. Autom. Control, 39: 11971210, 1994. 95. D. W. Clarke, C. Mohtadi, and P. C. Tuffs, Generalized predictive control: part 1The basic algorithm; part 2Extensions and interpolators, Automatica, 23: 137160, 1987. 96. R. R. Bitmead, M. R. Gevers, I. R. Petersen, and R. J. Kaye, Monotonicity and stabilizability properties of solutions of the Riccati difference equation: Propositions, lemmas, theorems, fallacious conjectures and counterexamples, Syst. Control Lett., 5: 309315, 1986.
74
PERIODIC NONLINEAR CIRCUITS
97. W. H. Kwon and A. E. Pearson, Linear systems with two-point boundary Lyapunov and Riccati equations, IEEE Trans. Autom. Control, 27: 436441, 1982. 98. G. De Nicolao, Cyclomonotonicity, Riccati equations and periodic receding horizon control, Automatica, 30: 13751388, 1994. 99. S. Bittanti and M. Lovera, A discrete-time periodic model for helicopter rotor dynamics, 10th IFAC Symp. Syst. Identication, 2: 577582, 1994. 100. P. Arcara, S. Bittanti, and M. Lovera, Periodic control of helicopter rotors for attenuation of vibrations in forward ight, IEEE Trans. Control Syst. Technol., 1998 (in press).
S. BITTANTI P. COLANERI
Politecnico di Milano
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

PID Control Standard Article Donald A. Pierre1 1Montana State University, Bozeman, MT Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1033 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (182K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Characteristics Of Controlled Processes The Essence of the Pid Terms Alternative Pid Forms Two-Degree-Of-Freedom Pid Practical Derivative Action Velocity or Incremental Pid Form Proportional Band Anti-Reset-Windup Bumpless Transfer Digital Pid Control Pid Tuning
Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
446
PID CONTROL
PID CONTROL
PID control strategies are by far the most widely employed of all strategies used in the automatic feedback control of industrial systems. The acronym PID stands for proportional, integral, derivative. Although the central concept of PID control can be gleaned from an examination of how these three terms are blended to form a control signal, the intelligent application of PID control in any given case requires an understanding of linear and nonlinear properties of the system that is being controlled, and practical PID controllers incorporate features such as bumpless transfer and anti reset windup. The proportional term of the PID controller forms a part of the control signal based on a proportional gain times a system error. The derivative term of the PID controller forms a part of the control signal based on a gain times the rate of change of a system error. Similarly, the integral term of the PID controller forms a part of the control signal based on a gain times the integral of a system error. This integral term basically forces the error from a nonzero value (a value that would exist without integral action present) to a zero value the integral term in effect resets the error to zero. Because of this phenomenon, the integral term of the PID controller is sometimes called the reset term. This terminology is especially prevalent in older literature; Eckman, for example, uses reset consistently in place of integral (1). Much of this article is devoted to practical issues of PID controllers. Several alternative forms of PID algorithms are examined, and their relative merits are discussed. Rules for tuning the parameters of the controller are considered, the literature base devoted to this topic is exceedingly large. Tuning rules are concerned with ways of assigning controller parameters (proportional gain, integral gain, and derivative gain) to achieve good performance based on various process model assumptions. That there are a variety of tuning rules should come as no surprise; generally there are tradeoffs to be made in alternative performance criteria and different tuning rules weight some performance criteria more than others. Tuning rules tend to emphasize system performance in the neighborhood of an operating point, and therefore generally are based on linearized models of the process being controlled. Tuning rules are but one aspect of PID controller design. Of equal importance is how the system performs when large input changes occur. For most systems, the control signal input to the system is limited by maximum and minimum bounds; when the control signal is saturated at one of these bounds for any length of time, the error between the desired system output and the actual system output can be quite large. Within the PID controller, this error signal is being integrated, and unless some mechanism is employed to control the integration process (either by selectively turning it off and on or by selectively forcing the integrator output to track an-
other signal) the output by the PID integrator can literally wind up to exceedingly large values, requiring large periods of time to unwind and leading to unacceptably long transient response. There are a variety of ways to counter integrator wind up, all of which are classied under the heading of antireset-windup strategies; some of the more common ones are described in this article. Prior to the 1960s, most PID controllers were implemented with analog techniques. With the steady advance in digital controller technology since that time, the majority of industrial PID controllers are now implemented digitally to take advantage of exibility of programming and decreased sensitivity to environmental changes. A major section of this article therefore is devoted to digital implementation issues. PID controllers are the workhorses of industrial process control, and they provide excellent results under a variety of conditions. There are circumstances, however, where other controllers (or at least enhanced PID controllers) offer superior results. Systems that are highly oscillatory often are difcult to control with PID alone, as are systems that exhibit long pure time delays. Systems having three dominant modes of comparable time constants generally can be controlled better by using a third-order controller, perhaps one that augments the desirable properties of a basic PID controller. This article is arranged in sections: (1) characteristics of controlled processes; (2) the essence of the PID terms; (3) alternative PID forms; (4) practical derivative action; (5) velocity or incremental PID form; (6) proportional band; (7) antireset-windup; (8) bumpless transfer; (9) digital PID control (with subsections on analogous implementation of digital PID, incremental digital PID, recursive digital PID, signal property considerations, other digital PID issues, and an emulation method of digital PID design); and (10) PID tuning.
CHARACTERISTICS OF CONTROLLED PROCESSES Before a controller for a process is specied, the process to be controlled should be characterized, at least in some broad sense. There are many different types of processes that are controlled automatically. Examples range from uid levels in tanks to readwrite heads of computer disk storage devices. The control inputs to a process are supplied through one or more actuators. For example a motor driven valve can be an actuator for uid owing into a tank. Process outputs that are being controlled (e.g., the uid level in a tank) are measured using appropriate sensors. The basic control of one output variable by the use of one control input variable is called single-input single-output (SISO) control. For more complex systems, multi-input multi-output (MIMO) control may be required. PID control was developed initially as an SISO control strategy, but it has been extended in various ways to MIMO control. Consider a process that responds to an increase in the control signal by having an increase in the controlled output. The output response generally is delayed from that of the controlled input because of time lags caused by system dynamics. Both dynamic and static characteristics of the process are of interest. Static (steady-state) characteristics of the process often can be measured in the following way: rst, the control input is set at a specic value; next, after the output of the process has settled to a steady-state value, this steady-state
PID CONTROL
447
value is tabulated next to the associated input value; and the preceding two steps are then repeated over the entire range of inputs that are realizable to create a steady-state or static inputoutput plot. Although static characteristics generally are nonlinear, they often exhibit regions of operation over which linear approximations apply, and it is possible to include nonlinear gain in the control to broaden the effective linear range of operation. The preceding approach must be modied in those cases where a constant control input results, after transients have settled, in a constant rate of change in the controlled output. In that case, the process contains a pure integration term, and the static characteristic of interest is a plot of the rate of change of the controlled output as a function of the control input. Disturbance signals can also inuence the behavior of a process. In the static measurements described in the preceding paragraph, it is assumed that disturbance signals have had negligible inuence. The controller for an SISO system is supplied with realtime information concerning the relationship of the controlled output of the system to some desired controlled output. The desired controlled output generally is called the reference input to the system. It is preferable to have the controller supplied with real-time values of both the reference input r(t) and the controlled output c(t); in some cases, however, only the error signal e(t), e(t) r(t) c(t), is available to the controller. The control problem is classied as a regulator problem if the reference input remains constant for long periods of time, and the controller strives to maintain the controlled output at a constant value in spite of disturbance inputs. The control problem is classied as a tracking problem if the controller is required to make the controlled output track a time-varying reference input such that the error between the two is maintained near zero, even in the presence of disturbance inputs. PID controllers are used in both regulator and tracking control applications. As with other types of controllers, PID controllers also are expected to reduce the sensitivity of the controlled system to changes in parameters in the process being controlled. THE ESSENCE OF THE PID TERMS Figure 1 shows the PID parts of a PID-controlled process (the PID parts alone do not constitute a practical PID controller, as will be evident later). The essence of how the PID parts work is as follows. The Proportional Term First consider the proportional term kpe(t) with proportional gain kp being a positive constant, and assume that only the
k pe ( t ) + ki tedt + + kdde/dt u(t) +
proportional term is active (temporarily assume ki kd 0 and disturbance input d(t) 0). In this case, the PID controller reduces to a proportional controller (P controller). With the system initially in equilibrium, suppose that the reference input r(t) increases abruptly from zero to a constant steadystate level rss. The controlled output is then less than the reference input causing an increase in the error e(t). The corresponding positive control signal kpe(t) results in an increase in the controlled output (the increase is not instantaneous and depends on the static and dynamic characteristics of the process). Assuming that the closed-loop system is stable, the error signal e(t) approaches a steady-state level ess such that ess = rss Ks kp ess (1)
where the static process gain Ks generally is dependent on the value of kpess. Note that Eq. (1) can be rearranged to obtain ess as follows: ess = rss /(1 + Ks kp ) (2)
Assuming Ks 0, it would appear that ess could be made arbitrarily small by making kp arbitrarily large; however, for any given system, an upper limit ku on kp invariably exists such that the closed loop system is unstable for kp ku, where the ultimate value ku depends on the system dynamics. The Integral Term The steady-state error described above generally can be reduced to zero by the inclusion of the integral term. With ki 0, kp 0, and kd 0, the controller of Fig. 1 is a PI controller. In this case, consider what would happen to the integrator output if it started at 0 at t 0, and the error signal e was constant at e ess 0; the integrator output would then equal kiesst, but this increases indenitely as t increases! The logical conclusion is that ess must be zerothe output of the integrator is part of the control signal u(t), and as u(t) increases, so also does the controlled output c(t) to the point where rss css ess 0 thereby inhibiting further increases in the output of the integrator. The Derivative Term In Fig. 1, if ki 0 with kp 0 and kd 0, the controller is a PD controller. If all three gain terms are nonzero, the controller is a three-term PID controller. The kd term plays the role of an anticipatory or predictive element. That is, even if the e(t1) is zero, if e (t1) de(t1)/ dt is large, the implication is that the error for t t1 is going to increase. By including kde in the control, the controlled output is forced to increase sooner than it would otherwise, with the goal of reducing the anticipated future error. This action is useful in compensating for dynamic lags that invariably are present in the process being controlled. However, if signicant measurement noise is present in the measured output signal, the kd term can have a detrimental effect on performance (the derivative of rapidly changing high-frequency noise can be extremely large even if the magnitude of the noise is small). Because of this, and because the output of any real circuit is band-limited, practical implementation of the kd term requires the use of a frequencydependent differentiator, as is described later in this article.
P r(t) + D e(t) I
d(t) + Process c(t)
Figure 1. A simplied version of PID control, with one degree of control freedom and without anti-reset-windup protection.
448
PID CONTROL
Performance Objectives The values of kp, ki, and kd should be selected to provide good system response under design conditions, and to reduce harmful effects of process changes that lead to off-design conditions. System response is often described by step response characteristics such as: (1) the 1090% rise time resulting from a step change in the reference input; (2) the percent overshoot in response to a step input; (3) the peak time associated with the peak overshoot in response to a step input; and (4) the time required for the system output to settle to within 2% of its nal value in response to a step input. Of interest are response characteristics caused by process disturbances (represented by d(t) in Fig. 1) in addition to responses activated by reference inputs r(t). Note that for constant disturbance values, with d(t) dss, the reason given in the previous Subsection on the integral term for ess being 0 still holds, and the controlled output c(t) approaches the steady-state reference input rss independent of the dss value, assuming of course that practical upper and lower bounds on the output of the integrator are not exceeded. If the reference input, in addition to the error signal is available to the controller, a modied reference input may be benecial. For example, if a controlled output exhibits too much overshoot in response to a step change in a reference input, a rate-limited version of the reference input can be generated and inserted in place of the actual reference input. Conversely, if a controlled output reacts too slowly in response to a change in a reference input, a rate-enhanced version of the reference input can be generated and inserted in place of the actual reference input. In some applications, future values of a reference input are available to the controller; in machine-tool control, for example, a desired contour to be traversed is known in advance. This look-ahead or preview information can be used to reduce the error between the reference input and the controlled output. ALTERNATIVE PID FORMS The control action of Fig. 1 is expressed as
t
rivative time. Either Eq. (4) or Eq. (5) can be arranged as a ratio of polynomials in s. The two roots of the numerator polynomial [the zeros of Gc(s)] can be assigned to arbitrary real or complex conjugate values by proper assignment of the PID gains. Another commonly used form of the basic PID is the series or interacting form:
Gc ( s ) = K
1+
1 sTi
1 + sTd
(6)
Note that the zeros of Eq. (6) are restricted to be real, and therefore the Gc(s) of Eq. (6) is less general than those of Eqs. (4) or (5). However, this series form can be augmented in a simple way to counter windup (to be described in the section on anti-reset-windup). Also, from a historical perspective, the series form was more readily implemented in early pneumatic and hydraulic equipment, some of which is still in use. Because there are several alternative forms of PID, manufacturers of PID controllers do not always use the same terminology. It is important therefore to determine the terminology and the particular PID controller forms adopted by suppliers of PID controller equipment under consideration. For consistency in this article, however, the parallel form of Eq. (4) is emphasized. TWO-DEGREE-OF-FREEDOM PID Figure 2 is a block diagram of a linear system model of a PIDcontrolled system. Anti-reset-windup and bumpless-transfer features are not included in the linear model. In Fig. 2, G(s) is the Laplace transform transfer function associated with a particular operating point of the process. Note that the integral action operates on the error signal e(t) r(t) c(t) as previously described, but that now different proportional and derivative gains apply to the reference input r(t) and the controlled output c(t). In terms of Laplace transforms of the associated signals, U (s ) = kp R(s ) kpC(s ) + s kd R(s ) kdC(s ) + ki E (s ) s (7)
u(t ) = kp e(t ) + ki
e() d + kd
de(t ) dt
(3)
where kp is the proportional gain, ki is the integral gain, and kd is the derivative gain. In terms of Laplace transforms, the Laplace transform transfer function associated with Eq. (3) is Gc ( s ) (s ) k = kp + i + kd s ,U E (s ) s (4)
Whereas the feedback system of Fig. 1 has one degree of control freedom, that of Fig. 2 has two degrees of control freedom: (1) the kp, ki, and kd gains can be selected to achieve desired goals regarding closed-loop stability, disturbance rejection, gains can be and sensitivity reduction; and (2) the kp and kd selected to achieve performance goals associated with the re-
The above form of the PID terms is called the parallel form. An alternative form to that of Eq. (4) is called the standard or noninteracting form and is characterized by
R(s)
PD + E(s) ki s + + PD U(s)
D(s) + + G(s) + + N(s) C(s)
Gc ( s ) = K 1 +
1 + Td s Ti s
(5)
with the obvious relationships kp K, ki K / Ti, and kd KTd. Although these two forms are equivalent, the standard form of Eq. (5) uses different terminology: the Ti constant is called the integral time; and the Td constant is called the de-
Figure 2. Linear system features of two-degree-of-freedom PID control.
PID CONTROL
449
sponse of the system to the reference input. If G(s) in Fig. 2 were a second-order system, with c(t) and c (t) viewed as state variables, the PID control form in Fig. 2 would be exactly the control form that would be expected from a modern state-variable viewpoint. For higher-order systems, the c(t) and c (t) states are often the most signicant ones, so that PID control in many cases can be tuned to give excellent results. Clearly, however, there are cases where higher-order transfer functions should be used in place of the PD and PD terms of Fig. 2.
ep
k ps 1 + ps + + + u(t)
ki k ps 2 (1 + ds2)
1 s
ed
Integrator as part of actuator
PRACTICAL DERIVATIVE ACTION

Figure 3. Linear aspects of a velocity form PID controller.
The derivative term of the PID controller is often described as simply kds. However, this term has an output response of A cos(t) to the sinusoidal input A sin(t). For large values of , corresponding to high-frequency noise, the response A cos(t) can be overly large even if the input amplitude A is reasonable. Thus, not only is pure derivative action not achievable, it generally is undesirable and cannot be tolerated in practical systems. The kds term invariably is replaced by a ltered version, as follows: Gd ( s ) = kd s 1 + s
within the dashed lines of Fig. 3. Note that time constants p and d are included to keep the high-frequency gain of the controller bounded. At low frequencies, the output of the top block in Fig. 3 is approximately kpe p(t), where ep (t )
p ,k r(t ) c(t ) k p
(12)
(8)
and the low-frequency output of the lower block in Fig. 3 is approximately kde d(t), where ed (t )
d ,k r(t ) c(t ) k d
where the time-constant is chosen such that measurement noise [N(s) in Fig. 2] does not have a signicant impact on the control action U(s). The transfer function from N(s) to U(s) is GNU (s ) (s ) GPID (s ) = ,U N (s ) 1 + G ( s )G ( s )
PID
(13)
(9)
where GPID (s ) = kp + kd s ki + s 1 + s
(10)
In Eq. (9), at high frequencies where both 1 and G( j) 0, we have GNU ( j ) GPID ( j ) kp + k ki + d j
(11)
The kd / term in Eq. (11) replaces a corresponding term jkd that would have resulted from the use of the pure derivative form. The PID transfer function of Eq. (10) has four adjustable parameters and is therefore a four-term compensator.
VELOCITY OR INCREMENTAL PID FORM In some applications, the integral part of the PID controller is an attribute of the actuator that drives the process. In that case, the velocity form of PID is appropriate, as displayed
Equations (12) and (13) correspond to the two-degree-of-freedom control loop of Fig. 2. A potential problem with the velocity PID form is that the second-derivative of an error signal must be approximated; for the same reasons described in the preceding section, this process leads to difculties in the presence of signicant measurement noise. Also, with the integrator external to the controller, it is essential that ki not be zero in Fig. 3 so that no attempt is made to cancel the 1/ s of the integrator by the s inherent in a velocity form of the PD controller. This means that the velocity PID of Fig. 3 has the integral term as an essential term of the controller. However, if the control signal stro u(t) is available as a measured signal, A m and Ha gglund (2) show how to use it to obtain a valid PD form from the velocity PID form. The velocity form of the PID controller does have some advantages: (1) if the integrator in the actuator is implemented in a way that limits the output of the integrator at the saturation bound (when either the upper or lower saturation bound is intercepted), then no integrator windup can occur, and no special anti-reset-windup strategy need be implemented; (2) if the control is transferred from automatic to manual or visa versa, the signal supplied to the actuator is of an incremental control form, and the resulting transfer in control generally will not cause a large swing in the process outputthe transfer of control will be bumpless; and (3) when the velocity PID is implemented digitally (in which case it is called an incremental PID algorithm), the PID values being accumulated (corresponding to those at the summation point in Fig. 3) are incremental in nature, and often can be accommodated by a
450
PID CONTROL
digital word length that is shorter than that required by a corresponding parallel PID algorithm. PROPORTIONAL BAND The control signal associated with any practical actuator has a realistic lower bound umin and an upper bound umax. For any u(t) umax the effective control is umax, and for any u(t) umin the effective control is umin. When either of the preceding conditions exist, the system is said to be in saturation. The term proportional band refers to conditions on the error signal such that the effective control signal is not saturated. Special Case of Proportional Band First, consider the case where only proportional control (P control) is applied [i.e., Fig. 1 with ki kd 0, and d(t) 0]; umin / kp to emax in this case, there will be a range from emin umax / kp in which the control signal will not be saturated, and the corresponding proportional band PB is PB = umax umin kp (14)
lersany controller that has a lag term may exhibit windup, and ways of controlling the windup should be considered to enhance performance. Many anti-reset-windup methods have been developed. They are based on one of two approaches: (1) conditional integration, in which integration is interrupted and/or modied when control signal saturation is likely; or (2) integrator tracking, whereby the output of the integrator is forced to track a signal usually with the objective of reducing the control signal magnitude to the saturation boundary. In either case, the anti-windup mechanism must avoid getting lockedup, such that the integrator does not perform as required under normal conditions. A Conditional Integration Example Conditional integration schemes tend to be heuristic and application dependent. A given approach may work reasonably well in one application, only to fail in another. Because of their use of logic, conditional integration schemes are readily incorporated in digital implementations of PID. One approach is to condition the integration on both kpe(t) and kde (t): if either one of these values exceeds preassigned bounds, the integration process is suspended and the output of the integrator is reset to a desirable level (perhaps zero); on the other hand, if both values are less then their respective bounds, the integration process is uninhibited; and in either case, the control signal u(t) can be formed on the basis of the three PID terms. Anti-reset-windup for Two-degree-of-freedom Control A basic tracking implementation of anti-reset windup is depicted in Fig. 4. The output from the PD and PD block is q (t ) = kp r(t ) kp c(t ) + kd r (t ) kd c (t ) (16)
Alternatively, we say in this case that the error is in the proportional band if emin e(t) emax. General Case of Proportional Band When all three terms are present in the PID controller, but d(t) 0 in Fig. 1, the control signal u(t) will be out of saturation only if the instantaneous e(t) r(t) c(t) satises
umin ki
e() d kd e (t ) < r(t ) c(t ), and kp r(t ) c(t ) < umax ki

t
e() d kd e (t ) kp
(15)
In practice, both derivative terms in the above expression would be ltered versions of the derivatives [for example, kdc (t) would be replaced by the output of the Gd(s) lter of Eq. (8) with the input to the lter being c(t)]. The saturation block in Fig. 4 is characterized by
Thus, the proportional band varies in a complicated way, depending both on the integral and the derivative of the error. ANTI-RESET-WINDUP The control signal of a given system can be in saturation for long periods of time for many reasons. For example, if a large step increase in the reference input occurs at t 0, the error signal for some time after t 0 also will be large, and its direct effect on the control signal will be to force it into saturation until the system output is able to match to some degree the reference input. Although the effective control signal is in saturation, the integrator in the controller accumulates the area under the large e(t) curve, unless restricted to do otherwise. If the integrator output is not restricted in some way, it takes a relatively long time for the integrator output to reduce to a normal level; it does not even start to decrease until after the controlled output c(t) has overshot the desired reference input value [causing e(t) to turn negative and allowing the output of the integrator to start to decrease]. Thus it is essential that the integrator output be managed in some effective way whenever the control signal is in saturation. It is of interest to note that windup is not limited to PID control-
w, umin < w < umax u= umax, w umax u ,wu min min
(17)
r c + +
PD & PD' q 1 s + p + w u
e=rc
ki
kt
Figure 4. A PID controller with anti-reset-windup provided by a tracking loop. The nonlinear block has a gain of 1 for w in the range umin w umax.
PID CONTROL
451
e=rc
PD
q + +
BUMPLESS TRANSFER When a feedback loop is switched from an automatic mode of control to a manual mode of operation or vice versa, it is often important to avoid large instantaneous jumps in the control action. Certain forms of PID implementation lend themselves in a natural way to smooth transitions from one mode of control to another. Velocity (incremental) PID implementations supply incremental values to be accumulated by an integrating actuator, so large transitions in effective control are not incurred when the source of the incremental changes is switched. For a general parallel implementation of PID control, tracking can be used to obtain bumpless transfer in addition to anti-reset-windup. Figure 6 is a diagram of such a system. The relationship between w and u in this diagram is that given by Eq. (17). When the switch is in the manual mode, as shown, the human operator supplies a value em to drive the system: if the nonlinear block is not saturated, the control u increases (or decreases) at a rate given by kmem; at the same time, the signal uPID tracks um because of the upper tracking loop, with tracking gain kt; and when the nonlinear block is saturated, the lower tracking loop insures that the output of the lower integrator will be maintained at either umax or umin, as appropriate. When the switch to automatic control is made, the uPID value starts with the most recent tracked value of um plus any additional component supplied by the PD terms operating on error signals involving r(t) and c(t). Similarly, when the switch is in the PID position, the manual mode output tracks the control u(t) supplied by the PID unit because of the presence of the lower tracking loop, also with tracking gain kt. Considerations of bumpless transfer also apply when parameters in the controller are changed abruptly. Such changes are especially easy to make in digital implementations of PID control. Real-time changes in the controller parameters can be motivated by monitored changes in the dynamics of the system being controlled. When these changes are done automatically, the resulting control system is classied as an adaptive control system. The particular PID parameter that is most readily conditioned for bumpless transfer is
1 1 + s T i'
Figure 5. A series-form PID controller with anti reset windup. The nonlinear block has the same characteristics as that in Fig. 4, and the indicated positive feedback results in the linearized transfer function from q to u being (1 sTi)/(sTi).
From the above equation, when w is in the range umin w umax, u w and the tracking error v in Fig. 4 is zero, resulting in normal operation of the PID algorithm. When w umax, however, with the tracking gain kt 0, the additional input to the integrator is kt(umax w) ktv 0, causing p(t) to tend to decrease until w reduces to umax. The larger the tracking gain kt, the faster the tracking process. In high noise environments, the tracking gain should be reduced to a reasonable level to avoid over zealous tracking. When w umin, the tracking error v becomes positive, and the integration process forces w to tend to increase towards umin. Thus, whenever the control is in saturation, the output of the integrator is automatically adjusted to a level that renders w near the saturation boundary, and any change in r(t) or c(t) that tends to move w away from saturation results in normal PID control. Anti-reset-windup for the Series Form of PID In the special case of the series PID form, anti-reset-windup can be implemented in a particularly simple way, as shown in Fig. 5. The saturation block in Fig. 5 is characterized by Eq. (17). Note that the feedback in the loop in Fig. 5 is positive. When the forward path is not saturated, u(t) w(t) and linear feedback theory can be used to show under these conditions that
U (s ) = 1 +
1 sTi
Q (s )
(18)
r
The PD block in Fig. 5 provides (approximately) Q(s ) = K (1 + sTd )E (s ) (19)
c (Closed-loop feedback) e=rc ki + +
PD & PD' + +
1 s
uPID
The combination of Eqs. (18) and (19) implements the series PID form given in Eq. (6). When the forward path in Fig. 5 is saturated at u umax, the contribution that u makes to w tends to umax. Similarly, when u is saturated at u umin, the contribution that u makes to w tends to umin. Thus the integration process is effectively bounded and windup is avoided. An alternative and somewhat more desirable anti-resetwindup strategy for the series PID can be obtained by moving the saturation block from the forward path in Fig. 5 to the feedback path, but keeping the saturation block to the right of the rst-order lag block. This enables the full effect of the PD action to be supplied to the process while yet limiting the integral action.
kt + um
(Manual) em
km
+ +
1 s
kt +
Figure 6. A general analog PID controller featuring anti-resetwindup and bumpless transfer.
452
PID CONTROL
ki. If the error signal e(t) is integrated rst and then multiplied by ki, any abrupt change in ki will result in a corresponding abrupt change in u(t). On the other hand, if the weighted error kie(t) is integrated in real time, the effect of an abrupt change in ki will be smoothed by the integration process. In terms of a block diagram representation, the ki block should precede the 1/ s block for bumpless transfer with respect to changes in ki. DIGITAL PID CONTROL Digital PID control often is implemented by having the system signals of interest [r(t), c(t), and e(t)] sampled periodically with sample period T; the sampled input signals are supplied to a microprocessor through an analog-to-digital converter (ADC), and the microprocessor is programmed to supply the control signal u(t) to the system using a digital-to-analog converter (DAC). Usually u(t) appears to the input of the process being controlled as a sampled-and-held signal with sample period T. Within the microprocessor, the linear aspects of PID control are based on the following calculations (using a parallel form):
k
propriate rst backward differences, with a gain of Kd kd / T. Similarly, the integration is approximated by a summation of samples, and the summation gain Ki kiT is used in place of the ki block in Fig. 6. The upper tracking loop in the gure can be implemented digitally, but note that the z-transform model of the 1/ s for the integrator is z /(z 1). If the output of the kt block is delayed by one sample period for convenience in the calculations for the tracking loop, the tracking loop will have the following characteristic equation: 1 + [kt /(z 1 )] = 0 (23)
which places a pole of the tracking loop at z 1 kt. For stability of the tracking loop, this pole must have a magnitude less than one; and a value of kt between 0.5 and 1 is recommended. Incremental Digital PID Equation (20b) can be used to show that u(k ) u(k 1 ) = Kp [e(k ) e(k 1 )] + Ki e(k ) + Kd [e(k ) 2e(k 1 ) + e(k 2 )] (24)
u(kT ) = Kp e(kT ) + Ki
m= 0
e(mT ) + Kd {[e(kT ) e[(k 1 )T ]} (20a)
To simplify notation, we use u(k) to denote u(kT), etc., thereby replacing the above equation by
k
u(k ) = Kp e(k ) + Ki
m= 0
e(m ) + Kd [e(k ) e(k 1 )]
(20b)
This falls naturally into the incremental PID form. If the actuator of the control loop contains the integrator, then during each sample period, the digital microprocessor simply has to perform the calculations given on the right-hand side of Eq. (24), normalize the result by dividing by T, and send the resulting approximate derivative value through the DAC to the integrating actuator. Of course a slight modication of the above is required if two-degree-of-freedom control is employed; namely, the value V, V = (Kp Kp )r(k ) + (Kd Kd )[r(k ) r(k 1 )] (25)
The proportional term of course is Kpe(k); the integral term k is the sum Ki m0 e(m); and the derivative term is now the rst backward difference term Kd[e(k) e(k 1)]. Using z transforms, the transfer function from U(z) to E(z) is given by GPID (z )
would need to be added to the right-hand side of Eq. (24). Recursive Digital PID
z z1 + Kd U (z )/E (z ) = Kp + Ki z1 z
(21)
Equation (24) can be rearranged in the recursive form u(k ) = u(k 1 ) + (Kp + Ki + Kd )e(k ) (Kp + 2Kd )e(k 1 ) + Kd e(k 2 ) (26a)
which can be placed over a common denominator to obtain the form GPID (z ) = b0 z2 + b1 z + b2 z (z 1 ) (22)
Also, using the z-transform relationship of Eq. (22), it can be shown that the associated difference equation is u (k ) = u (k 1 ) + b0 e (k ) + b1 e (k 1 ) + b2 e (k 2 ) (26b)
where b0, b1, and b2 are related to Kp, Ki, and Kd in a straightforward way (to be described). Essentially all digital PID z-transform transfer functions can be based on Eq. (22), which has two poles, one at z 0 and one at z 1. The two zeros of Eq. (22) can be either two real zeros or a pair of complex conjugate zeros located to achieve performance objectives. Analogous Implementation of Digital PID The general continuous PID implementation of Fig. 6 can be converted to digital form by using the following approximations. Assume that the lower portion of the gure is left unchanged, but that uPID is supplied from the output of a DAC and hold circuit. The derivative action is approximated by ap-
By comparing coefcients in Eqs. (26a) and (26b), it follows that b0 Kp Ki Kd, b1 (Kp 2Kd), and b2 Kd. The recursive form of Eq. (26b) is an excellent way to implement digital PID. It lends itself naturally to anti-resetwindup: each time u(k) is calculated, if it is outside the saturation bounds, it is reassigned the value of the nearest saturation bound. For real-time implementation, we use the following notation: u current value of u; u1 most recent past value of u; e current value of e; e1 most recent past value of e; and so on. The following sequence of events constitutes a ow diagram for computer code to implement the algorithm:
PID CONTROL
453
1. 2. 3. 4. 5. 6. 7. 8. 9.
Initialize e1 e2 u1 0. Compute temp u1 b1e1 b2e2. Wait for the sample period. At the sample period, obtain e r c using appropriate A/D converters. Compute u temp b0e. If u umax, assign u umax, or if u umin, assign u umin. Output u to the DAC. Assign in proper order e2 e1, e1 e, and u1 u. Return to step (2).
Note that the above steps are arranged to minimize the number of calculations required between the instant that e is obtained from the ADC and the instant that u is sent to the DAC. This process minimizes the computational time delay that is introduced into the control loop; computational time delay can have a detrimental effect on system performance. Also, for two-degree-of-freedom implementations, the terms from Eq. (25) would need to be included appropriately in the code. Signal Property Considerations Both control signals and measured process signals are constrained in a variety of ways. Common examples of constrained control signals are (1) signals that are continuously adjustable between saturation levels; (2) control signals that can be assigned only a few valuesas in the case of a relay with dead zone, with possible output values umax, 0, or umax; and (3) control signals that are pulse-width modulated. Even in case (1) above, when the control is supplied by the output of a DAC, the nite word length of the DAC may have a noticeable quantization effect for small steady-state error signals. In case (2) above, when the control is supplied by an electromechanical relay (once a common practice, but now largely displaced by power electronics), a correct balance of P and D terms is needed to avoid excessive switching of the relay; in all-electronic implementations of such systems, rapid switching actually may be included intentionally to achieve a sliding mode of control, and nonlinear control strategies involving nonlinear functions of e and e are appropriate. Case (3) above is especially prevalent in electronic drives for dc motors. This is because a power amplier dissipates the least energy if it is either full on or full off. To reduce heat buildup in the amplier, the control is implemented as follows: if the desired control action from t kT to t (k 1)T is uo where umax uo umax, the effectively equivalent pulse-width control supplied is
smaller than the dominant mechanical time constant of the motor and its load, the above control provides smooth operation of the motor, and a conventional linear model of the motor-and-drive generally can be used in the design of the control system. Measured process signals supplied to a digital controller are quantized, sometimes as a result of ADC conversions, but in other cases as a result of the types of measurement sensors employed. For example, an optical rotary position sensor provides a nite number of angular positions for one complete revolution of the sensor disk. In this case, when the magnitude of e(t) is very small, the sensor error can switch abruptly from 0 to or , where is the effective quantization level of the sensor. To avoid very-low-level oscillations of the control under such conditions, it may be necessary to zero the input to the PID integrator when e(t) is small, and to effect a small dead-zone notch in the control action. Other Digital PID Issues Other issues in digital controller design are sampling period selection, antialiasing, and process modeling. Ideally, the sampling period T should be selected to be an order of magnitude smaller than the dominant time constants of the system being controlled. However, extremely small values of T may lead to round-off error and computational problems on niteword-length computers. The choice of sampling period also is affected by the amount of process and measurement noise. The folding frequency associated with T is 0.5/ T Hz. When a signal that has a frequency component above the folding frequency is sampled, the sampled signal exhibits an associated aliased low-frequency component. Thus, for digital implementation, generally signals are ltered with an analog antialiasing lter prior to sampling. Often adequate for the purpose is a rst-order low-pass lter with a corner frequency near 0.5/ T Hz. When a digital controller is designed using z-transform methods, it is helpful to have a representative z-transform transfer-function model of the process being controlledfor example, a model corresponding to a sample-and-hold circuit followed by the process transfer function G(s) of Fig. 2. In the case of a conventional zero-order hold circuit, the required z-transform transfer function of the sample-and-hold in series with the process is Ghp (z ) = (1 z 1 )Z{G(s )/s} or Ghp (z ) = (1 z 1 )Z
t
(29a)
u(t ) =
umaxsign(uo ), kT t < kT + T 0, kT + T t < (k + 1 )T
g( ) d
0
(29b)
(27)
The characteristic equation of the feedback loop is then where T Tuo / umax, and
1, u o > 0 sign(uo ) = 0, uo = 0 1, u < 0 o
1 + GPID (z )Ghp (z ) = 0
(30)
(28)
When the sample period T is larger than the major electrical time constant of the motor drive circuit, but is substantially
with closed-loop poles of the system being roots of the characteristic equation. The PID gains can be adjusted to obtain desirable pole locations inside the unit circle of the z plane, and a variety of other digital controller design methods can be employed.
454
PID CONTROL
Emulation Method of Digital PID Design The process of selecting of PID gains to achieve desirable goals is called tuning. Because most PID tuning methods have been developed for analog PID controllers, a natural question is can we design an analog PID using established tuning rules and then translate the analog PID gains into meaningful digital PID gains? Of course, the answer is generally yes. However, there are several factors that should be considered in doing this. In the rst place, the sample-and-hold operation that is inserted in the control loop has the effect of introducing a time delay, of approximately T /2 s into the loop. Also, computational delay, denoted by T s where 0 1, is introduced. These delays need to be taken into account when converting from analog PID gains to digital PID gains. Consider the following analog GPID(s): GPID (s ) = kp + kd s ki + s 1 + s (31)
In a general emulation approach developed by Pierre and Pierre (3), GPID(s) is replaced by digital controller transfer function Gc(z): Gc (z ) = Ga (z )GPID 2 z1 T z+1 (32)
where the replacement of s by (2/ T)(z 1)/(z 1) in GPID is the well known Tustin approximation, and where Ga ( z ) [0.5 (1 + a ) + (1 a ) ] , [0.5(3 a) + (1 a)]zz a (33)
in which a is an additional tuning parameter (typically 0.3 a 0.4). It is readily shown that the dc gain of Ga(z) is Ga(1) 1 and that the zero in Eq. (33) is to the right of the pole thereby providing phase lead. Ga(z) compensates to some degree for sample-and-hold delay and for computational delay T in the control loop. As a specic example of Ga(z), the case where a 0.2 and 0 in Eq. (33) gives Ga ( z ) = 1.6z 0.4 z + 0.2 (34)
of the process being controlled. There are many representative sources that can be consulted for details on a wide variety of alternative tuning rules (2,47). Common tuning rules are described in this section. Most of these approaches give excellent results if the process being controlled is well damped and is dominantly second order. Systems that are essentially rst order can often be tuned well with PI control only. Systems with large time delay can be controlled with PID control in combination with a Smith predictor (8). From a frequency response viewpoint, the PID integral term supplies 90 of phase lag at low frequencies, and the practical derivative term supplies somewhat less than 90 of phase lead at some intermediate to high frequencies. Thus, if the system requires more than 90 of phase-lead compensation over some frequency range to achieve good performance, PID control alone will not be adequate. For lightly damped oscillatory open-loop systems, one method that is tempting (but should be approached with caution) is to place the zeros of the PID controller at the pole locations of the process, to cancel the oscillatory poles. This approach should be avoided if the PID controller employed has one degree of freedom only: although the response mode of the canceled poles will not be excited by reference inputs, disturbance inputs will excite the oscillatory mode in the controlled output, and the PID controller with its zero gain at the oscillatory frequency will not supply damping. For twodegree-of-freedom systems, the PID gains within the loop can be assigned to add damping to the oscillatory mode, whereas the independent PD factors associated with the reference input can be adjusted to provide blocking zeros. Sensitivity of oscillatory pole-zero cancellations with respect to parameter changes also is of concern (9). ZieglerNichols Tuning A widely applied tuning rule, the ultimate gain rule, was developed in 1942 by Ziegler and Nichols (10). This rule is based on experimentation with the process to be controlled. First, with ki and kd set to zero in Eq. (3), the proportional gain kp is increased until the system starts to oscillate; the value of kp that starts the system oscillating is denoted as ku and is called the ultimate gain. The period of the oscillation, Tu, also is recorded. As an alternative procedure, rather than determining ku and Tu by forcing the actual system into instability, if the frequency response G( j) is available, it can be used to obtain both ku and Tu analytically, as follows. First, the frequency u at which the angle of G( ju) 180 is determined, and the corresponding value of G( ju) Au is obtained. The value of ku satises the characteristic equation 1 + k u G ( j u ) = 0 (36)
It is important to have a in Eq. (33) bounded away from z 1; poles of the controller near z 1 often are ringing poles, generating oscillations of period 2T in u(t) that are not readily observed in c(t). From Eqs. (31) and (32), along with assigning T /2, it follows that Gc ( z ) = Ga ( z ) k p + ki T (z + 1 ) kd (z 1 ) + 2 (z 1 ) Tz (35)
A controller based on the Gc(z) of Eq. (35) can be implemented using a variety of digital control programming methods; practical implementations insure that integrator windup is avoided. PID TUNING Numerous studies have been made to develop assignment rules to specify PID parameters on the basis of characteristics
and therefore ku 1/ Au and Tu 2 / u. An ultimate gain tuning rule is then: k p = 0.6k u ki = 1.2ku /Tu and kd = 3ku Tu /40 (37c) (37a) (37b)
PID CONTROL
455
This ultimate-gain rule is but a rule-of-thumb; although it was developed with a nominal 20% step response overshoot goal, it is easy to nd cases where it results in overshoots in excess of 50%. It should not be surprising that the rule does not apply well to all cases because process characteristics vary widely. Modied ZieglerNichols Methods Many variations of the ultimate gain rule have been developed. One popular one credited to Harriott in (4) is as follows. First, with ki and kd set to zero in Eq. (3), kp is adjusted until the step-response of the closed loop exhibits a decay ratio of 0.25 (the percent overshoot of the second peak in the step response is one-fourth of the percent overshoot of the rst peak in the step response). Let kc denote the critical value of kp corresponding to this 0.25 decay ratio. Also, let Tc denote the difference between the time of occurrence of the second peak and that of the rst peak. Next, assign ki and kd on the basis that ki = 1.5kc/Tc and kd = kc Tc /6 (38b) (38a)
of s3 1s2 2s 3, where the roots of this polynomial are assigned to achieve desirable damping and natural frequency. Explicit relations for kp, ki, and kd can be developed in terms of 1, 2, 3, b1, b2, a1, and a2. Tuning methods also can be based on frequency response characteristics and the sensitivity function S( j )
, 1 + G ( j1)G( j)
c
(43)
Generally the values of kp, ki, and kd should be such that max |S( j )| < 1.5
(44)
Values smaller than 1.5 in Eq. (44) usually correspond to well damped systems. Tuning methods based on optimization techniques can be applied directly to a particular system if a valid G(s) repre stro sentation of the controlled process is available. A m and Ha gglund (2) do a systematic evaluation of many cases to generate tuning diagrams that can be used to obtain desirable PID gains for a variety of G(s) forms. CONCLUSION In his classical paper on governors, Maxwell (11) in 1868 clearly expressed the difference in effects produced by proportional and integral control. However, there is no one person that can be credited with having invented PID control. Many of the early developments, those involving anti-reset-windup for example, were company proprietary and therefore were not available in the open literature. With the advent of computer control in the 1960s, many of the traditional PID techniques were reassessed and translated into the digital control algorithms. In this article, the basics of PID control have been described. Operational features of both linear PID terms and nonlinear characteristics have been examined. Digital PID algorithms have been described and have a special signicance in the modern era of digital control. Today, PID controllers can be purchased from many manu stro facturers [see A m and Ha gglund (2) for a recent listing of companies and features available]. In addition to being used for SISO control, PID controllers are used in a variety of other applications. In cascade control, one control variable is generated on the basis of several measured variables. In selector control one actuator can be driven from a selected PID unit, and then automatically switched to a different PID unit based on a max/min selector to control some other process signal if it starts to deviate from a desired range. In ratio control two outputs are controlled with one of the outputs required to be a xed percentage of the other output. In split-range control and multiactuator control there may be fewer measured signals than actuators, and the amount of control from each actuator has to be automatically balanced to achieve the desired overall process performance goals. Feedforward control of disturbances, when they can be measured, often leads to vastly improved disturbance rejection. Automatic gain scheduling of PID parameters can be based on measured process conditions. And interactions of coupled control loops can be reduced by properly designed coupled PID controllers. In many cases, as-
And nally, while conducting additional closed-loop step-response tests, adjust all three gains by the same percentage until desirable overshoot conditions are achieved. More Advanced Tuning Methods When the transfer function G(s) of Fig. 2 is known and is a reasonably good model of the process being controlled, an array of tuning methods are available, depending on the form of G(s). For example, variations of the ZieglerNichols methods are based on cases where G(s) assumes forms such as G (s ) = or G (s ) = K0 e sT0 s (40) K0 e sT0 0 s + 1 (39)
A pole-placement approach has been developed (2) for those cases where G(s) is of the form G (s ) = b1 s + b2 s2 + a 1 s + a 2 (41)
The characteristic equation for the closed loop is 1 + G c ( s )G ( s ) = 0 (42)
When the PID transfer function of Eq. (4) is substituted for Gc(s), and G(s) of Eq. (41) is substituted into Eq. (42) also, the resulting equation reduces to a third-order polynomial having coefcients that are functions of kp, ki, kd, b1, b2, a1, and a2. A desirable placement of the poles of the system can be achieved by equating coefcients of the above polynomial to coefcients
456
PIECEWISE-LINEAR TECHNIQUES
pects of PID control are blended with other control concepts, leading to higher-order controllers. PID developments in the future will be, and already have been to some extent, coupled with fuzzy control, neural-network control, and adaptive control. BIBLIOGRAPHY
1. D. P. Eckman, Principles of Industrial Process Control, 11th ed., New York: Wiley, 1961. stro 2. K. J. A m and T. Ha gglund, PID Controllers: Theory, Design, and Tuning, 2nd ed., Research Triangle Park, NC: Instrument Society of America, 1995. 3. D. A. Pierre and J. W. Pierre, Digital controller design alternative emulation approaches, ISA Trans. J. Instr. Soc. Amer., 34: 219228, 1995. 4. B. G. Liptak, (ed.), Instrument Engineers Handbook, Vol. II, Process Control, New York: Chilton, 1970. 5. R. Isermann, Digital Control Systems, Vol. 1, 2nd ed., Berlin: Springer-Verlag, 1989. 6. C. L. Smith, Digital Computer Process Control, Scranton, PA: International Textbook, 1972. stro 7. C. C. Hang, K. J. A m, and W. K. Ho, Renements of the ZieglerNichols tuning formula, IEE Proc., D138 (2): 111118, 1991. 8. T. Ha gglund, A predictive PI controller for processes with long dead times, IEEE Control Syst. Mag., 12 (1): 5760, 1992. 9. D. A. Pierre, Root locus near isolated pole-zero dipoles and a counterintuitive case of digital control compensation, IEEE Trans. Auto Control, AC-29: 439441, 1984. 10. J. G. Ziegler and N. B. Nichols, Optimum settings for automatic controllers, Trans. ASME, 64 (8): 759768, 1942. 11. J. C. Maxwell, On governors, Proc. R. Soc. Lond., 16: 270283, 1868. Also published in R. Bellman and R. Kalaba (eds.), Mathematical Trends in Control Theory. New York: Dover Publications, 1964, pp. 317.
DONALD A. PIERRE
Montana State University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Position Control Standard Article Gang Tao1, Avinash Taware1, Minyan Shi1 1University of Virginia, Charlottesville, VA Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1016 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (651K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Adaptive Position Control of Manipulators Adaptive Control of Teleoperation Systems Summary Keywords: adaptive control; robustness; stability; PD control; inverse dynamics; path interpolation; transparency; teleoperation; masterslave control; tracking About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
POSITION CONTROL
Position control has many applications, such as control of the elevator angle of a ghter, control of the antenna angle in a satellite tracking system, and control of robot manipulators. Often, the essence of position control is that the tracking error between the desired system output and the actual system output is used to generate a suitable control input to drive the tracking error to zero. In other words, tracking control is an important part of motion control, as it solves the problem of determining the control inputs necessary for a system to track a desired trajectory and provides a way to achieve accurate performance. In this article, we present adaptive position control of robot manipulators and teleoperation systems. Robot manipulators are composed of links connected by joints. The joints may be electrically, hydraulically, or pneumatically actuated. The number of joints determines the number of degrees of freedom (DOF) of the manipulator. Position control of a robot manipulator involves control of the positions of the joints. Once given a set of desired trajectories for all the joints, the controller is designed to track these trajectories so that the end effector of the manipulator sweeps the desired positions in the workspace. The primary method of sensing the positions is with position encoders located on the joints, either on the shaft of the motor that actuates the joint or on the joint itself. At times, direct sensing of the end-effector position with the help of a camera is used to improve the accuracy of the manipulator in tracking a desired trajectory. A teleoperation system involves two distant yet coupled robots: a local master robot and a remote slave robot. In teleoperation, the human operator controls the master robot. Motion commands are measured on the master robot and transmitted to the slave robot, which executes these commands and is expected to track the motion of the master robot. In addition, the contact force information sensed by the slave robot is reected to the master robot for force perception. Thus, the master acts as an position input device that generates a desired trajectory. The goal of position or tracking control is to design the necessary control input that makes the slave track the motion of the master. This control problem is an example of masterslave control. The article is organized as follows: In the following section, we present robust adaptive control schemes of robot manipulators. First, we present dynamic models of robot manipulators with time-varying parameters or unmodeled dynamics. Second, we present the controller structure and adaptive law for the time-varying parameter case and show the signal boundedness and the tracking performance of the robot system. Third, we present and analyze stable adaptive control schemes for robot manipulators with unmodeled dynamics. Some common topics of position control relevant to robot manipulators such as PD control, inverse dynamics, and path or trajectory interpolation are discussed in the fourth subsection. In the third section, we present adaptive control of teleoperation systems. Adaptive control schemes for teleoperation systems with unknown jumping parameters and with parametrizable and unparametrizable smooth time-varying parameters are presented. We also present some control issues relevant to teleoperation systems with communication time delays.
POSITION CONTROL
Adaptive Position Control of Manipulators

To make robot manipulators capable of handling large loads in the presence of uncertainty on the mass properties of the load or its exact position in the end effector, robust adaptive control designs for robot manipulators have been developed. In Slotine and Li (1,2) an adaptive control scheme has been proposed for the motion control of robot manipulators, which guarantees global stability and asymptotic zero tracking error between the actual joint trajectory and the desired one and needs only the measurements of the joint position and velocity. This consists of a proportional derivative (PD) feedback part and a full dynamics feedforward compensation part, with the unknown manipulator and payload parameters being estimated online. The algorithm is computationally simple, because of an effective exploitation of the particular structure of manipulator dynamics. Various modied versions of this scheme have been shown to be applicable to robot systems with unmodeled dynamics [Reed and Ioannou (3)], and joint exibility [Spong (4)]. Recently, there has been considerable research interest in neural network control of robots, and satisfactory results have been obtained in solving some of the special issues associated with the problems of robot control. In Lewis, Jagannathan, and Yeildirek (5), neural network controllers are designed for robot manipulators in a variety of applications, including position control, force control, parallel-link mechanisms, and digital neural network control. These model-free controllers offer a powerful and robust alternative to adaptive control. In Ge et al. (6), a comprehensive study of robot dynamics, structured network models for robots, and systematic approaches for neural-network-based adaptive controller design for rigid robots, exible joint robots, and robots in constraint motion are presented. In this article, we will present a robust adaptive control scheme, based on the scheme developed by Slotine and Li (1,2) with a modied controller structure and a modied adaptive law [Tao (7)], which ensures the signal boundedness in the presence of time-variations in the manipulator parameters and a mean tracking error of the order of the parameter variations, which are not required to be small. We will also show similar results for a class of unmodeled dynamics. The allowance for the existence of possible large parameter variations and unmodeled dynamics yields signicant potentials for applications of the proposed robust adaptive manipulator controller. Manipulator Models and Parametrization. In this subsection, we rst present the mathematical models of robot manipulators with time-varying parameters or unmodeled dynamics, and their parametrized forms, and then use a two-link planar manipulator to illustrate the manipulator modeling and parametrization. Manipulator Models. To derive the dynamic equations of a n-link robot manipulator (see Fig. 1, which shows an illustrative four-link manipulator) whose parameters may explicitly depend on time, we use the EulerLagrange equations [Spong and Vidyasagar (8); Ortega and Spong (9)] of a mechanical system:
where q = (q1 ,. . ., qn )T is a set of position variables of n (n > 0) joints of the robot manipulator, u = (u1 ,. . ., un )T is the applied joint torque, and L is the Lagrangian dened as L = K P, the difference between the kinetic energy K and the potential energy P, in the form
POSITION CONTROL
Fig. 1.
Robot manipulator.
with D (q, t) Rnn being the symmetric and positive denite manipulator inertia matrix. For Eqs. (1), (2) to represent the manipulator dynamics with time-varying parameters, the mass and the moment of inertia of each link of the manipulator should not explicitly depend on q . Letting dij be the ijth element of D (q, t) and (q, t) = P (q, t)/q, and substituting Eq. (2) in Eq. (1), we obtain the manipulator dynamic equation:
where the kjth element of C(q, q , t Rnn is ckj = n i = 1 (dkj /qi + dki /qj dij /qk ) q i. A key feature of the manipulator model (3) is that the inertia matrix D(q, t) and the potential energy P(q, t) are explicitly time-dependent, which takes into account the effect of changes in environment of the robot system or changes of the manipulator dynamics with time. Moreover, an important property of the manipulator model (3) is
where M (q, q , t) = dD(q, t)/dt D(q, t)/t, whose ijth element is (dij /q)T q . When D(q, t) = D(q) does not )]x = 0, which is well known explicitly depend on t, that is, D(q, t)/t = 0, Eq. (4) becomes xT [dD(q)/dt 2C(q, q in the robotics literature. A manipulator with unmodeled dynamics may be modeled as
POSITION CONTROL
where H i , i = 1, 2, 3, with appropriate dimensions, are linear operators with rational transfer matrices, q), g2 (q), g3 (u) are certain vector functions of representing the unmodeled part of the robot dynamics, and g1 ( q , q, u. The functions D(q), C(q, q ), (q) have been dened above, and, for simplicity, they are assumed not explicitly time-dependent in the unmodeled dynamics problem. The manipulator model (5) is generalized from some practical robot systems [Reed and Ioannou (3); Ortega and Spong (9)]. Control Objective and Parametrization. Our control objective is, for a given reference signal qd (t), to generate the applied torque u for the manipulator (3) or (5) with unknown parameters so that all signals in the robot system are bounded and the joint position q tracks qd as closely as possible. To achieve such an objective we rst use the transformation technique developed in Slotine and Li (1,2) to parametrize the manipulator model (3) or (5). Let be any n n constant matrix whose eigenvalues have positive real parts; dene
Clearly, it follows from Eq. (6) that
and s, v, v depend only on q, qd , q , q d, q d and not on the joint acceleration vector q (t). Using Eq. (7), we express the manipulator model (3) as
where Y (q, qd , q , q d, q d , t) is an n r matrix of known functions for some r > 0, and (t) Rr contains , q d, q d , t) is bounded for bounded parameters, which may be time-varying. In Eq. (8), the regressor Y (q, qd , q , q d, q d q, qd , q Let xt be the truncated x at time t. Denote by the Euclidean vector norm or the induced matrix norm, and by ( 1 , 2 ) the L (L1 , L2 ) vector norm or the induced operator norm [Desoer and Vidyasagar (10)], as the case may be. We make the following assumptions about the manipulator model (8): (t) for some constants 0 > 0, > 0; (1) (t) 0 , (2) D(q, t)/t f (q) for some constant > 0 and known f (q) bounded for bounded q. Similarly, for the manipulator model (5), we obtain
where Rr is a constant vector, and Y (q, qd , q , q d, q d ) is not explicitly time-dependent. We make the following assumptions about the manipulator model (9):
POSITION CONTROL
Fig. 2.
Two-link planar manipulator.
(1) (g1 ( q))t f 1 ( q(t)), (g2 (q))t f 2 (q(t)) for some known f 1 ( q), f 2 (q) that are bounded for bounded q , q, and H 1 1 , H 2 2 for some constants 1 > 0, 2 > 0; (2) (g3 (u))t ut , and H 3 i i , where H 3 i is the ith row of H 3 , i = 1,. . ., n. We also make an assumption on the desired joint position vector qd (t): d (t), q d (t) are bounded. (1) qd (t), q Assumption (A1) requires only the boundedness of the manipulator parameters and their derivatives, not the smallness of the time variations of the parameters. Smallness of the parameter variations is usually an assumption for the design of adaptive control schemes for time-varying plants, but it is not needed here because of the special structure of the robot manipulator dynamics. Assumption (A2) requires that D (q, t)/t satisfy a certain relative boundedness condition. Assumption (A3) requires that the L gains of H 1 , H 2 be nite and q), g2 (q) satisfy certain relative boundedness conditions. Assumption (A4) is similar to (A3), but i 0, i = g1 ( 1,. . ., n, are to be specied for the robust stability of the adaptive robot system. We note that the bounds , 1 , 2 are not needed for the adaptive controller design. An Illustrative Example. In this sub-subsection, we consider a two-link planar manipulator [Spong and Vidyasagar (8)], shown in Fig. 2 as an illustrative example for the robot system modeling and parametrization. The manipulator conguration may be described as follows: there are two revolute joints with joint angles q1 , q2 , and two links with masses M 1 , M 2 , lengths l1 , l2 , distances lc1 , lc2 from the joints to the mass centers, and rotational inertias I1 , I2 . The inertia matrix D(q, t) has four elements: d11 = M 1 lc1 2 + M 2 (l1 2 lc2 2 + 2l1 lc2 cos q2 )+ I1 +l2 , d12 = d21 = M 2 (lc2 2 + l1 lc2 cos q2 ) + I2 , d22 = M 2 lc2 2 + I2 ; and the potential energy is P(q, t) = , t) (M 1 lc1 + M 2 l1 ) g sin q1 + M 2 lc2 g sin (q1 + q2 ), where g is the gravitational acceleration. The matrix C(q, q 2 , c12 = ( q1 + q 2 )h, c21 = q 1 h, c22 = 0, where h = M 2 l1 lc2 sin q2 . in Eq. (3) has four elements: c11 = h q
POSITION CONTROL
The manipulator model without parameter variations and unmodeled dynamics is D(q) s + C(q, q )s = u , q d, q d ), where Y (q, qd , q
When = (t) is time-varying, the manipulator model is Eq. (8) with Y (q, qd , q , q d, q d , t) and
i (t)| i , we obtain the bound in (A1) as Assuming that | and the bounds , f (q) in (A2) as 2 2 = (1 + 2 + 43 2 + 64 2 + 5 2 + 46 2 )1/2 , f (q) = 1. For the unmodeled dynamics problem, the manipulator model is Eq. (9). The bounds 1 , 2 , i in (A3) and (A4) depend on the nature of the unmodeled dynamics. Solution to the Parameter Variation Problem. In this subsection, we rst present an adaptive control scheme for robot manipulators modeled by Eq. (8) and then analyze the stability and tracking properties of the proposed adaptive controller. If the inertia D(q, t) = D(q) and the potential energy P(q, t) = P(q) are not explicitly time-dependent, that is, D(q, t)/t = 0, and (t) = is constant in Eq. (8), then the adaptive control scheme proposed by Slotine and Li (1,2),
guarantees that the closed-loop system is globally stable and convergent in the sense that q(t), q (t), (t) are bounded, and limt e(t) = 0, as the positive denite function
has the property
(t) = sT (t)K D s(t) 0 [also see Spong et al. (11) for further analysis].
POSITION CONTROL
Fig. 3.
The switching modication.
When D(q, t), P(q, t) are both explicitly time-dependent, we have obtained the manipulator model as Eq. (8) in which (t) is time-varying and the term D(q, t)/t q appears. If the parameters in D(q, t), P(q, t) were , q d, q d , t) ( t) known, then (t) and D(q, t)/t q could be calculated so that the control law u(t) = Y (q, qd , q + [D(q, t)/t] q K D s could be implemented, which guarantees global stability and asymptotic tracking. For unknown D(q, t), P(q, t), next we present an adaptive control scheme that is robust with respect to the time variation of (t) and [D(q, t)/t] q . , q d, q d , t), s, K D , dened before, we propose the following feedback controller structure With Y (q, qd , q for the manipulator (8):
and the following update law for (t):
where (t) as shown in Fig. 3 is the switching signal [Ioannou and Tsakalis (12)] using a priori knowledge of the upper bound M on supt0 (t) :
This adaptive control scheme has the following stability and tracking properties.
POSITION CONTROL
Theorem 1. All closed-loop signals are bounded, and the tracking error e(t) = q(t) qd (t) satises
for some constants 0 > 0, 0 > 0, and any t2 > t1 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of (t) = 0, D(q, t)/t = 0. parameter time variations, that is, when Proof:. Consider the positive denite function
From Eqs. (4, 7, 8), (16, 17, 18) and from (A1), (A2), we have that
Since , , 0 are constants and (t) dened in Eq. (19) satises
it follows from the second inequality of (2.22) that (t) 0 for (t), s(t) outside a certain bounded set. Therefore s(t) and (t) are bounded, which, in view of Eqs. (7), (16), implies that q(t), q (t), u(t) are also bounded. Using the fact that (t)
T
(t)(t) 0 and the rst inequality of Eq. (22), we obtain
POSITION CONTROL for some constant k1 > 0. Since V (t) is bounded, from Eq. (24) we have
for some constants 1 > 0, 1 > 0 and any t2 > t1 0. To show that Eq. (25) implies Eq. (20), let us consider the relation s(t) = e (t) + e(t), where is a stable matrix [see Eq. (7)], and denote by H the linear operator or the impulse-response matrix from s(t) to e(t) as the case may be, that is, e(t) = H [s](t) = 0 t H (t )s() d. It follows that
Since the operator H is exponentially stable and e(t) is bounded, both H 2 and t1 t2 H (t t1 )e(t1 ) 2 dt are nite for any t2 > t1 0. Hence from Eqs. (25) and (26) we prove Eq. (20). (t) = 0 and D(q, t)/t = 0, that is, = = 0, it follows from Eq. (20) that e(t) L2 . This, together When with the boundedness of e (t) = s(t) e(t), proves limt e(t) = 0. To implement the controller (16), we need the knowledge of f (q) to generate the bounding signal m(t) in Eq. (17). A more sophisticated choice of f (q) admits a wider class of D(q, t)/t, but may make the implementation of m(t) more complicated. We also note that the above design does not need the knowledge of the bounds , . For a chosen f (q), different choices of k0 in generating m(t) may have different effects on the tracking performance, while increasing k0 may reduce the effect of in the mean error (20). For the signal boundedness and the mean tracking error (20), parameter variations characterized by and are not required to be small. This is an important feature of the robot system. With q , q available for measurement, the manipulator mode (3) is equivalent to the rst-order model (8), for which the adaptive controller allows large parameter variations to exist. Solution to the Unmodeled Dynamics Problem. Consider the manipulator (9) with unmodeled q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)](t) were available for measurement and were dynamics. If the terms H 1 [g1 ( , q d, q d ) H 1 [g1 ( q)](t) H 2 [g2 (q)](t) H 3 [g3 (u)](t) K D s(t) could known, then the control law u(t) = Y (q, qd , q be implemented so that d/dt[sT (t)D(q)s(t)] = 2sT (t)K D s(t), showing the boundedness of s(t) and exponentially fast tracking. However, to ensure the boundedness of u(t), one needs H 3 = maxi = 1,...,n i < 1. q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)(t), and are unknown, To solve the adaptive control problem in which H 1 [g1 ( , q d, q d ), K D , s(t) = (s1 (t),. . ., sn (t))T , (t), dened before, we propose the following feedback with Y (q, qd , q
10
POSITION CONTROL
controller structure for the manipulator (9):
and the following update law for (t):
The stability and tracking properties of this adaptive control scheme are: Theorem 2. All closed-loop signals are bounded for any i [0, i ] i = 1,. . .,n, and the tracking error e(t) satises
for some constants 0 > 0, 0 > 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of the unmodeled dynamics, that is, when H 1 = 0, H 2 = 0, H 3 = 0.
POSITION CONTROL Proof:. Considering V (t) dened in (15), from Eqs. (9), (27), (32) and (A3), (A4) we obtain
11
From Eq. (30), we see that
Hence it follows from Eqs. (34), (35) that, for 0 i i , i = 1,. . .,n, we have (t) 0 whenever (t) and s(t) are outside a certain bounded set, that is, s(t), (t) are bounded, and so are q(t), q (t). From Eqs. (27, 28, 29, 30), (A3), (A4), and the boundedness of s(t), q(t), q (t), (t), we have
for some constant k2 > 0, which, together with Eq. (31), implies that u(t) is bounded. Using Eqs. (34), (35) and the fact that (t)
T
(t)(t) 0, we obtain
For = maxi = 1,...,n i , Eq. (37) implies that
for some constants 1 > 0, 1 > 0, and any t2 > t1 0, which implies Eq. (33).
12
POSITION CONTROL When H 1 = 0, H 2 = 0, H 3 = 0, the expression (34) for (t) becomes
Since sT (t)3 (t) 0, Eq. (39) shows that s(t) L2 . Hence, from Eq. (7), it follows that e(t) L2 and e (t) is bounded. Therefore we have limt e(t) = 0. We have thus proved the signal boundedness of the closed-loop system in the presence of H 1 [g1 (q)](t), H 2 [g2 (q)](t), H 3 [g3 (q)](t). The gains of the linear operators H 1 , H 2 are assumed to be nite but not small. The gain of H 3 is required to be small to ensure the boundedness of u(t). The modifying term (t)(t) in Eq. (32) can be replaced by 0 (t), 0 > 0. The signal boundedness follows, but Eq. (33) is changed to
This scheme cannot guarantee asymptotic tracking in the absence of the unmodeled dynamics, though the scheme does not need the knowledge of the upper bound on . The use of the bounding signals mi (t) dened in Eq. (28) is the key to ensuring signal boundedness in the presence of the unmodeled dynamics satisfying (A3), (A4). To generate these signals, the knowledge of the stability margin of the unmodeled dynamics is not needed. Alternative bounding signals may be used under i i other assumptions for the unmodeled dynamics. For example, if 3 i , where 3 is the ith row of H 3 (s)(s variable), i = 1,. . ., n, for some known constant a0 > 0, and (s+a0 ) 1 [g3 (u)])t + a0 ) (this s is the Laplace 1 (s+a0 ) [u])t , then we can choose m3 (t) = (s+a0 ) 1 [u])t and set 0 < i a0 . Another choice of m3 (t) i i is the bounding signal [ /(s+0 )][ u ](t) when g3 (u) u and 3 1 i , where 3 is the ith row of H 3 (s 0 )s with the Laplace variable s, for some known constant 0 > 0. For this m3 (t), the condition on i is < 0 and i > 0 for i = 1,. . .,n. We note that a similar L1 -norm condition can be established for ( 1 ,. . . , m ) T H 1 and H 2 to design a robust adaptive controller. q)](t), H 2 [g2 (q)](t), H 3 [g3 (u)](t) are present, Eq. (27) with (t) = becomes a When is known and H 1 [g1 ( robust nonadaptive controller, which results in e(t) converging exponentially to a residual set whose size is of the order of 1 2 /k10 2 + 2 2 /k20 2 + . Next, we present the robust adaptive control design assuming that the upper bounds on the gains of the unmodeled dynamics H 1 , H 2 are known: ), g2 (q) are the same as in (A3), and H j i (1) (A3a)(g1 q the ith row of H j , j = 1, 2, i = 1,. . ., n.
:
ji for some known constants ji > 0,where H j i is
We propose to choose j (t) = (j1 (t),. . .,jn (t))T , j = 1,2,3, in Eq. (27) as
POSITION CONTROL
13
This scheme guarantees that all signals in the closed-loop system are bounded for 0 i 3i , i = 1,. . ., n, and the tracking error e(t) satises
for some constants 0 > 0, 0 > 0, and any t2 > t1 0. Moreover, e(t) L2 and limt e(t) = 0 in the absence of the unmodeled dynamics H 1 , H 2 , and H 3 . We see from Eq. (43) that the mean tracking error explicitly depends only on the design parameter , not on the bounds ji dened in (A3a). A smaller may result in a smaller mean tracking error (2.43). Hence, with the knowledge of the unmodeled dynamics bounds, improvements of the tracking performance may be achieved by using the control signals dened by Eq. (41). Another interesting result is the adaptive controller with a so-called variable structure [Utkin (13)]: letting ji 0 in (2.41), j = 1,2,3, i = 1,. . .,n, we obtain
It can be shown that for 0 i 3i , i = 1,. . .,n, all closed-loop signals are bounded and the tracking error asymptotically converges to e(t) = 0 with possible chatterings. For a variable structure controller, (t) = 0 can be used in the update law (32). As a nal remark, we note that the proposed designs can be combined to solve the problem in which both the parameter variation and unmodeled dynamics are present. Proportional Derivative Control, Inverse Dynamics, and Path Interpolation. In this subsection, we discuss some of the general concepts related to the position control of robot manipulators. Proportional Derivative Control. We rst derive a PD control law for each joint of a manipulator based on a single-input single-output (SISO) model. Coupling effects among the joints are regarded as disturbances. Permanent-magnet dc motors along with gear reduction are commonly used in practice to actuate the joints of the manipulator. For such dc-motor-actuated robotic manipulator, a simplied version of the dynamics of the kth joint can be given as in Spong and Vidyasagar (8),
where J eff k = J mk + rk 2 dkk (q) is the effective joint inertia of the kth actuator (motor plus gear, J mk ) and the manipulator link [dkk (q) is the kth diagonal element of D(q) in Eq. (3)], Beff k = Bmk + (K bk K mk /Rk ) is the effective damping of the kth actuator (motor plus gear, Bmk ) with K bk the back emf constant, K mk the torque constant, and Rk the armature resistance; mk is the kth motor (rotor) angular position; V ak is the armature voltage of the kth motor; rk is the kth gear ratio; k = K mk /Rk ; and k is the actuator dynamics (3) specied for the kth joint and is treated as a disturbance to simplify the problem, since in that case, we maintain the
14
POSITION CONTROL
linearity of (2.45). The last can be given as
where cijk = (dkj /qi + dki /qj dij /qk ) and k (q) = P(q)/qk with P(q) the potential energy. The setpoint tracking problem is dened as the problem of tracking a constant step reference command d = [d1 ,. . .,dn ] for n joints. This type of control is adequate for applications not involving very fast motion, especially in robots with large gear reduction between the actuators and the links. A PD compensator for each of the n joints can now be used to achieve setpoint tracking:
where K pk , K dk are the proportional and the derivative gains, respectively. The characteristic polynomial of the closed-loop system is
indicating that the closed-loop system will be stable for all positive values of K pk and K dk and bounded disturbances. The tracking error is given by
For a step reference input dk (s) = dk /s and a constant disturbance k (s) = k /s, the steady-state error is ekss = rk k / k K pk [see Spong and Vidyasagar (8)]. Thus, the steady-state error due to a constant disturbance is smaller for larger gear reduction and can be made arbitrarily small by making the position gain K pk large. By using integral control as well (PID), we can achieve zero steady-state error while keeping gains small and rejecting step disturbances. However, the PD or PID compensators perform poorly for position control when there are large uncertainties in system parameters, or when varying disturbances and unmodeled dynamics are present, as is common in applications. In such situations, the adaptive control designs presented in the preceding two subsections perform much better than the PD or PID controller. In the PD compensator presented above, the coupling effects among the joints were regarded as disturbances. In reality, the dynamic equations of a robot manipulator form a complex, nonlinear, and multivariable system. Such a dc-motor-driven n-joint actuator may be represented in a matrix equation as
where D(q) is the time-invariant n n inertia matrix, C(q, q ) and (q) are the time-invariant versions of C(q, q , t) and (q, t) in Eq. (3) respectively, is a diagonal matrix with elements J mk /rk 2 , the input joint torque has components uk = (K mk /rk Rk ) V ak , and B has elements k = Bmk + K bk Lmk /Rk (with Lmk the inductance), for k = 1, . . ., n. An independent joint PD control scheme can be written for the system (50) as in Spong and
POSITION CONTROL Vidyasagar (8):
15
where = qd q is the error between the desired and the actual joint displacements, and P , D are diagonal matrices of positive proportional and derivative gains, respectively. In the absence of gravity [(q) = 0], the PD control law (51) achieves asymptotic tracking of the desired joint positions. In presence of gravity, Eq. (51) alone cannot guarantee asymptotic tracking and has to be modied as
to cancel the steady-state error due to the effect of the gravitational terms [see Spong and Vidyasagar (8)]. For detailed analysis and performance study of PD controller for robot manipulators, please refer to Spong and Vidyasagar (8). Inverse Dynamics. Using inverse dynamics, a more complex nonlinear control technique can be implemented for trajectory tracking of rigid manipulators [Spong and Vidyasagar (8)]. Consider the system given by Eq. (50) in a more simplied form,
where = D + , h = C q + q + . The idea of inverse dynamics is to seek a nonlinear feedback control law u = (q, q ) which when substituted into Eq. (53) results in a linear closed-loop system. Since the inertia matrix M is invertible, the control law
reduces the system q = V a with V a as the new input to the system, the armature voltages to be applied to to the actuator motors. Thus, we have a double integrator system with n uncoupled double integrators. The nonlinear control law (54) is called the inverse dynamics control and achieves a new linear and decoupled system, making it possible to design V ak to control a simple linear second-order system and can be designed as
where 0 , 1 are diagonal matrices of position and velocity gains, respectively, and is the reference. The gains could be chosen to get a joint response that is equal to the response of a critically damped linear second-order system with desired natural frequencies for each of the desired speeds of the responses of the joints. The inverse dynamics can be viewed as an input transformation that transforms the problem from one of choosing torque input commands, which is difcult, to one of choosing acceleration input commands, which is easy. There are many crucial issues of implementation and robustness that must be addressed to implement Eq. (54), and the reader is referred to Spong and Vidyasagar (8). Path Interpolation. The simplest type of robot motion is point-to-point motion. In this approach the robot is commanded to go from an initial conguration to a nal conguration without regard to the intermediate path followed by the end effector. To understand the concept of conguration, it is helpful to review some terminology used in Spong and Vidyasagar (8). Suppose a robot has n + 1 links numbered from 0 to n starting
16
POSITION CONTROL
Fig. 4.
Path interpolation: via points to plan motion around obstacles.
the base of the robot, which is taken as link 0. The joints are numbered 1 to n, and the ith joint is the point in space where links i 1 and i are connected. The ith joint variable is denoted by qi . A coordinate frame is attached rigidly to each link. We attach an inertial frame to the base and call it frame 0. Frames 1 to n are chosen such that frame i is rigidly attached to link i. Now the conguration is given by the transformation matrix that transforms the coordinates of a point from frame j to frame i and is denoted by T i j . For example, for a seven-link robot manipulator, the initial and nal congurations that are of interest in point-to-point motion are the transformation matrices that transform the coordinates of frame 6 to frame 0; let them be denoted by T 0 6 init and T 0 6 nal. This type of motion is suitable for materials transfer jobs where the workspace is clear of obstacles. Given the desired initial and nal positions and orientation of the end effector, the inverse kinematic solution is evaluated to nd the required initial and nal joint variables. Suppose, di j denotes the position of frame j with respect to frame i, and Ri j denotes the orientation of frame j relative to frame i. For the manipulator with seven links, the motion of the rst three, joints is calculated by computing the joint variables q1 , q2 , and q3 corresponding to d0 3 init and d0 3 nal. The motion of the nal three joint variables is found by computing a set of Euler angles corresponding to R3 6 init and R3 6 nal [Spong and Vidyasagar (8)]. For some purposes, such as obstacle avoidance, the path of the end effector can be further constrained by the addition of via points intermediate to the initial and the nal congurations as shown in Fig. 4. Different techniques of generating smooth trajectories in joint space, given the initial and nal joint variables, are presented in Spong and Vidyasagar (8).
Adaptive Control of Teleoperation Systems

A teleoperation system involves two distant yet coupled robots: a local master robot and a remote slave robot. An ideal teleoperation is the one in which the impedance felt by the human operator is matched to the impedance of the slave environment [Lawrence (14)]. The term transparency is used to describe such an ideal teleoperation. For teleoperation systems with known and time-invariant dynamics, a transparency control scheme is proposed in Lawrence (14), with a modication to handle communication time delays. For teleoperation systems with unknown time-invariant dynamics, an adaptive control scheme based on Slotine and Li (1,2) algorithm is presented in Hashtrudi-Zaad and Salcudean (15). Stability and signal boundedness of a similar adaptive control system are investigated in Lee and Chung (16). Despite recent progresses in teleoperation, transparency issues for teleoperation systems with unknown time-varying parameters, such as jumping and smoothly but rapidly changing parameters, including control designs and transparency characterizations, remain open.
POSITION CONTROL
17
Fig. 5.
Structure of a teleoperation system.
In this article, we present new transparency concepts suitable for adaptive control of teleoperation systems with time-varying parameters [Shi et al. (17)]. Adaptive control schemes for teleoperation systems with jumping or rapidly time-varying parameters are developed [Shi et al. (17)]. The developed adaptive control schemes lead to stable and transparent teleoperations in the presence of unknown constant or jumping or fast-varying parameters. The teleoperation systems to be controlled are assumed to have no communication time delay. In the rst subsection, we present the new concepts of weak transparency, asymptotic weak transparency, and approximate weak transparency, and formulate the transparency control problem for four types of teleoperation systems with no communication time delay [Shi et al. (17)]. In the next subsection, we present adaptive control schemes for teleoperation systems with unknown jumping parameters and with parametrizable and unparametrizable smoothly time-varying parameters [Shi et al. (17)]. In the last subsection we present some control issues relevant to teleoperation systems with communication time delays. Teleoperation Systems. In this section, we present the general structure of a teleoperation system and its dynamic description, introduce several new concepts for transparency of teleoperation systems, and state the adaptive control objective with which the new transparency concepts are to be veried. Dynamics of a Teleoperation System. A teleoperation system consists of ve subsystems: the human operator, the master robot, the communication channels, the slave robot, and the slave environment, as shown in Fig. 5. The term teleoperator refers to the master and slave manipulators connected by the communication channels. Bilateral teleoperation involves velocity and force information transfer between the master and the slave. Communication time delays commonly exist in teleoperation systems due to the large distance and restrictive data transfer. These delays are assumed to be absent in the following analysis, for connement to fundamentals and for simplicity of analysis. In Fig. 5, vh is the velocity of the human operators hand, vm is the velocity of the master end effector, vs is the velocity of slave end effector during contact, F h is the force applied by the human operator to the master robot, F e is the force exerted by the slave robot on its environment, and F s is the coordinating torque. In the absence of communication time delay, vsd (t) = vm (t) and F md = F s (t). In the presence of communication time delay T , vsd (t) = vm (t T ) and F md = F s (t T ). In the following analysis, no communication time delay is assumed. For analysis, a network representation of a teleoperation system is useful and Fig. 6 shows one commonly used in the literature, in which the human operator and the slave environment are represented by one-port networks, and the teleoperator by a two-port network. The blocks Zh , Zm , Zs , and Ze represent respectively the dynamics of a human operator, a master robot, a slave robot, and the slave environment; signals m and s denote control torques for master and slave robots; signals vh and ve refer to velocities of the human operators hand and the slave environment. Note that vh equals vm , the velocity of master end effector, and ve equals vs , the velocity of slave end effector during contact. The signals F h , F h , F e represent respectively the force generated by the human operator, the force applied by the human operator to the master robot, and the force exerted by the slave robot on its environment. As in Hashtrudi-Zaad and Salcudean (15), Lee and Chung (16), and Raju et al. (18), we consider the dynamics of the master and slave robots as
18
POSITION CONTROL
Fig. 6.
A two-port network for a teleoperation system.
where M , B, and K are inertia, damping, and stiffness parameters; the signal x is the position of end effector; and the signal denotes the control torque with subscript m for the master and s for the slave. From Eq. (56), we see that Zm (s) = M m s + Bm + K m /s. Let Cm and Cs denote the master and slave feedback control, and Ci , i = 1, . . ., 4, represent the data communication control for signals vm , F e , F h , and vs , respectively. Then the torques m and s have the following descriptions:
where the minus sign indicates the feedback signals. We also assume the human operator and the slave environment are passive, and as in Raju et al. (18), we use a generalized massdampingspring model to describe the human operator and the slave environment,
where M , B, and K are the inertia, damping, and stiffness parameters with subscript h for the human operator and e for the slave environment. Substituting Eq. (61) into Eq. (57), we get the slave system
where M = M s + M e , B = Bs + Be , and K = K s + K e . Four types of teleoperation systems are usually met in applications: teleoperation systems (i) with known time-invariant dynamics, (ii) with unknown constant environment, (iii) with jumping environment parameters, and (iv) with smooth time-varying environment parameters. The transparency of adaptive teleoperation control
POSITION CONTROL
19
systems is of main interest in this article, for which we will introduce new concepts suitable for adaptive control when the system parameters are unknown, for different cases of parameter uncertainties. We rst consider teleoperation systems with no communication time delay. The stability analysis is easy and simple when communication delay is not involved. A closed-loop transfer function can be obtained for the bilateral system, and the traditional tools such as the root locus technique and the RouthHurwitz stability criterion can be used for stability analysis. It is reasonable to assume no communication delay to develop the basic adaptive control techniques along with the stability analysis. In the last subsection below, we present some of the control issues relevant to teleoperation with communication time delays. Transparency of a Teleoperation System. The impedance transmitted to or felt by human operator, Zt (see Fig. 6), is dened by F h = Zt vh , in the frequency domain. Denition 1. [Lawrence (14)]. A teleoperation system is transparent if
This means that in a transparent teleoperation, the human operator feels as if he were manipulating the slave environment directly. Note that when the slave robot is in contact with its environment, its velocity, vs and the environment force F e are related by the impedance Ze as F e = Zs vs in the frequency domain. Since vh = vm , if the slave exactly reproduces the motion of the master (i.e., vs = vm , and Zt = Ze , then F h = F e , that is, the master accurately feels the slave contact force. That is, for a transparent teleoperation, the velocity tracking from the slave to the master leads to force tracking from the master to the slave. Denition 2. A teleoperation system is weakly transparent if
The property (64) is called weak transparency because it only needs Zt = Ze for some specic operation frequencies at which vs = vm . Denition 3. A teleoperation system is asymptotic weakly transparent if
This weak transparency is ensured in adaptive teleoperation control systems with parametric uncertainties. Denition 4. A teleoperation system is approximate weakly transparent if
for some constant c1 > 0, c2 > 0, > 0 and some design parameter > 0. In this case, it is expected that the design parameter > 0 in the control system can be chosen to be large so that the tracking error vs (t) vm (t) can be made small.
20
POSITION CONTROL
Denition 5. A teleoperation system is approximate weakly transparent in the mean if
for some constant 1 > 0 and 2 > 0, and any t2 > t1 0. This weak transparency is ensured in adaptive teleoperation systems with both parametric and structural uncertainties. In this case, it is expected that 1 = 0 for an adaptive control system, for some design parameter > 0 that can be made small. Control Objective. The control objective is to develop controllers Cm , Cs , and Ci , i = 1,. . ., 4, that ensure (i) closed-loop signal boundedness, (ii) limt [vs (t) vm (t)] = 0 [vs (t) tracks vm (t) as closely as possible in the sense (66) or (67)], and (iii) F h = F e for the slave environment with constant parameters, or with jumping parameters, or with smoothly time-varying parameters, in the presence of parameter uncertainties. This is a masterslave control problem.
Control Designs. In this subsection, we will rst review two existing control designs: one for teleoperation systems with known constant parameters, and one for teleoperation systems with unknown constant parameters. We will then present new adaptive control schemes for time-varying teleoperation systems with jumping parameters or with smoothly time-varying parameters. The teleoperation systems in consideration are assumed to have no communication time delay. For the new proposed control schemes, we will analyze the system performance in terms of stability and transparency. The teleoperation system is said to be stable if the state variables of the system are bounded at any time. Design for System with Known Constant Parameters. As in Lawrence (14), the forces and velocities of the teleoperator two-port as shown in Figure 6 are related by a hybrid matrix H :
where H ij , i, j = 1, 2, representing the inputoutput relation of the teleoperator two-port, are H 11 = (Zm + Cm ) D(Zs + Cs C3 C4 ) + C4 , H 12 = (Zm + Cm ) D(I C3 C2 ) C2 , H 21 = D(Zs + Cs C3 C4 ), H 22 = (I C3 C2 ) with D = (C1 + C3 Zm + C3 Cm ) 1 . Solving for F h , vh from Eq. (68), we get
To achieve Zt = Ze , it is needed that H 11 = H 22 = 0, H 12 = 1, H 21 = 1. Therefore, it can be derived that C1 = Zs + Cs , C2 = 1, C3 = 1, C4 = (Zm + Cm ) with Cm , Cs stable. This control scheme achieves transparency for teleoperation systems with known dynamics in the absence of communication time delay. Sufcient conditions for stability of the teleoperation system are derived in Lawrence (14). A tradeoff between stability and transparency is necessary [Lawrence (14)]. With the modied control schemes, good transparency is achieved at lower frequency. However, Zt = Ze cannot be ensured for all frequencies. In the next sub-subsections, we will consider the adaptive control problems when the slave systems parameters are unknown, to show that new transparency concepts dened in Denitions 3.3, 3.4, 3.5 are
POSITION CONTROL
21
useful for adaptive teleoperation systems. To design an adaptive control law, we assume zero communication time delay and m [Lee and Chung (16)]. m , 022 (1) The master position signal xm is bounded with bounded derivatives x
Design for System with Unknown Constant Parameters. In this section, the slave environment is assumed to have unknown constant parameters. Our control objective, as specied in the sub-subsection Transparency of a Teleoperation System above, is to achieve (i) closed-loop signal boundedness; (ii) limt [vs (t) vm (t)] = 0, and (iii) F h = F e . The last two properties imply asymptotic weak transparency of the teleoperation system. A control scheme based on Slotine and Lis (1) design is rst applied to achieve signal boundedness and velocity tracking. Force matching is then designed by using the relationship of (3.13) with H 11 = 0, H 12 = 1. Adaptive Control Design. The slave system as dened in Eq. (62) is
where M = M s + M e , B = Bs + Be , K = K s + K e , and M s , Be , K e > 0 are unknown constants. m ( t) Let > 0 be a design parameter, and dene the signals e(t) = xs (t) xm (t), v(t) = x v. As in Slotine and Li (1), the control law is chosen as
e(t), s(t) = x s ( t)
where Y ( v, v, xs ) = [ v v xs ] is a vector of known signals, = [M B K ]T is a vector of unknown parameters, the estimate of , and K D > 0 is a design gain parameter. Choose the adaptive law as
is
where
R33 is positive denite. Consider the Lyapunov function V =

2
(Ms2 +
), where
. Then it follows that = (K D + B)s 0, which implies that all the closed-loop signals are bounded, and that e, e converge to zero asymptotically as the time t goes to [Slotine and Li (1)]. Transparency and Stability. With velocity tracking from the slave to the master, the force tracking from the master to the slave will lead to a weak transparent teleoperation. Because F h is related to F e as F h = H 11 ve + H 12 (F e ), the condition for F h = F e is
Recall H 11 = (Zm + Cm )D(Zs + Cs C3 C4 ) + C4 , H 12 = (Zm + Cm )D(I C3 C2 ) C2 , and D = (C1 + C3 Zm + C3 Cm ) 1 . The following design [Lee and Chung (16)] will satisfy the condition (71):
Thus far asymptotic velocity tracking and force tracking are ensured, which lead to asymptotic weak transparency for a teleoperation system with unknown constant parameters. The developed teleoperation system is stable because the master, the slave, and their controllers are passive, and the human operator and the slave environment are also passive by assumption.
22
POSITION CONTROL
Design for System with Unknown Jumping Parameters. Parameter variations in this case are characterized by piecewise constant behavior. An example is the slave robot intermittently contacting with different environments. Assume that parameters B and K in the slave system (69) are unknown and piecewise constant, as modeled by
where Bi and K i , i = 1,. . ., l, are constants, which last for certain durations of time, and f i (t), i = 1,. . ., l, are functions indicating which values of Bi and K i are taken by B(t) and K (t) at a given time. The indicator functions, f i (t), are known functions that reveal the durations and the time instants of the parameter discontinuities, and are dened as
The function f i (t) = 1 indicates the value Bi or K i is taken by the system parameter B(t) or K (t). In addition, f i (t) indicates that only one value can be active at any time, that is, f i (t)f j (t) = 0 for i = j. With (t) = [M B(t) (t) = (t), so we cannot use (t) in Eq. (70) to ensure (t) 0 for any t K (t)]T , it follows that (t) = (t) > 0. In this sub-subsection, we rst present an adaptive control scheme for the slave system (69) with unknown jumping parameters described by (73, 74, 75). Adaptive Control Design. We propose the following controller structure:
where v(t), s(t), Y ( v , v, xs ), and K D are dened in the preceding sub-subsection, and =[ T estimate of = [M B(t) K (t)] , with (t) = 1 f 1 (t) + 2 f 2 (t) + . . . + l f l (t) and (t) = 1 f 1 (t) + . . . + l f l (t). Substituting (3.21) into (3.14) reveals
]T is the 2 f 2 ( t) +
where the estimate errors function
M,
Bi , and
K i , i = 1,. . ., l. Let us choose the positive
POSITION CONTROL where m , b i, k i > 0, i = 1,. . ., l. We choose the adaptive laws for
i, i
23
and K i as
With this adaptive design, the derivative of Eq. (78),
(t), becomes
For stability analysis, we need the following lemma. Lemma 1. [Tao (19)]. If f (t) L , f (t) L2 , then limt f (t) = 0. The fact that (t) = (K D + B)s2 0 implies s L2 L , L , i L , i L , i = 1, 2,. . ., l. Since s L , as from assumption (A1) we have xm , x m s=e + e, we conclude that e, e L2 L . Hence xs , x L . From Eqs. (76) and (77) it follows that s , s L ; then e L . Applying Lemma 3.1, we conclude that the position tracking error e(t) and velocity tracking error e (t) = vs (t) vm (t) go to zero as t goes to . In summary, we have proven the following theorem. Theorem 3. The adaptive controller (76) with the adaptive laws (79, 80, 81), applied to the system (69) with jumping parameters (73, 74, 75), guarantees that all closed-loop signals are bounded and the tracking errors e(t) and e (t) go to zero as t goes to . Transparency and Stability. With velocity tracking, controllers that ensure force tracking will also lead to asymptotic weak transparency of the teleoperation system. For such transparency, the force control (72) is also a choice for a teleoperation system with jumping parameters. Because the parameter jumping is bounded, the resulting jumping in acceleration and velocity is bounded as well. This will not change the passivity of the slave system, because its elements are still passive. Hence the system stability is guaranteed with respect to passivity. Design for Smooth Time-Varying Parameters. Parameter variations in those systems are characterized by continuous bounded functions with bounded derivatives. The slave system is represented by
where m(t) > 0, B(t) > 0, K (t) > 0 represent the time-varying mass, damping, and spring parameters. This model follows from the EulerLagrange equation [Spong and Vidyasagar (8)] with kinetic energy K = x s M ( t) x s . Transparent teleoperation designs for known and unknown time-varying parameters are considered in this section. To achieve weak transparency the key is velocity tracking and force tracking between slave and master robots.
24
POSITION CONTROL
Control Design for Known Time-Varying Parameters. A control scheme that ensures asymptotically weak transparency is proposed rst for the teleoperation system with known time-varying slave system. This scheme is then extended to the time-varying slave system with bounded parameter disturbances. Design I for Known Time-Varying Parameters.Design I for Known Time-Varying Parameters slave system (83) with known time-varying parameters, we propose the control scheme For the
where (t) = [M (t) B(t) K (t)]T , Y ( v , v, xs ) = [ v , v, xs ], v = x m (t) e(t), s = x s (t) v, and e(t) = xs (t) xm (t), as in the sub-subsection Design for System with Unknown Constant Parameters above, and K D > 0 is a design gain to be specied later. Substituting the controller (84) into the slave system (83) reveals
Dene the positive function
The time derivative
(t) of V (t) is
To ensure
(t) 0, we choose K D to be such that
The result that (t) 0 implies that s L2 L . Since s = e + e, we conclude that e and e L2 L . Hence s L . From Eqs. (84) and (85) we have s , s L , and therefore e L . Applying Lemma 1, we conclude xs , x that the tracking errors e(t) and e (t) = vs (t) vm (t) go to zero as t goes to . In summary, we have the following results: Theorem 4. All signals in the closed-loop system with the time-varying model (83) and the controller (84) (t) go to zero as t goes to . where K D satises (3.32) are bounded, and the tracking errors e(t) and e Design II for Time-Varying Parameters with Unknown Disturbances.Design II for Time-Varying Parameters with Unknown Disturbances In this case, the system parameters (t) and M (t) satisfy the assumptions: (1) The time-varying parameter vector (t) satises
POSITION CONTROL for some known parameter 0 (t) R3 and some unknown but bounded disturbance (t) < 1 for some constant 1 > 0. (2) The time-varying parameter (t) satises
( t)
25
R3 such that
for some known function for some constant 2 > 0.
0 (t),
and some unknown but bounded disturbance
M ( t)
such that |
M ( t) |
< 2
We propose the controller structure as
where K D > 0 is the design gain. We choose the positive function V (t) as dened in Eq. (86). Choose K D > + 2 + k0 , for some design parameter k0 > 0. Then M (t)s2 , we have
(B + k0 )s2 + 1 2 /4K D + 2 2 /4K D , which implies that
V (t) is bounded. Since V =
which implies
where 1 > 0 is a constant. We then have
and
where k1 > 0, k2 > 0, and 1 > 0 are constants, k0 is a design parameter that can be chosen to be large, and so (t) + e(t) where > 0 is a constant, we have is K D > 0. Since s(t) = e
26
POSITION CONTROL
where c1 > 0, c2 > 0, d1 > 0, and d2 > 0 are some constants, and 1 = be chosen to large so that the errors in Eqs. (93) and (94) are small. In summary, we have the following results.
is a design parameter that can
Theorem 5. All signals in the time-varying system (83) with parameter disturbances (A2), (A3) and controller (90) are bounded, and the tracking errors e(t), e (t) satisfy Eqs. (93) and (94), respectively. Moreover, e(t) L2 , 2 (t) = 0 in the absence of parameter disturbances, that is, when = 0, e (t) L , and limt e(t) = 0, limt e M = 0. Adaptive Control for Unknown Time-Varying Parameters. Transparent teleoperations are designed for two types of slave systems: those with unknown smooth time-varying (parametrizable) parameters, and those with unknown and disturbed time-varying (unparametrizable) parameters. An adaptive control scheme is proposed for the rst type of system to achieve asymptotic weak transparency. With modication, this scheme ensures approximate weak transparency in the mean for the second type of system. Design I for Parametrizable Parameter Variations.Design I for Parametrizable Parameter Variations present an adaptive control design for systems satisfying the following assumptions: (1) The unknown time-varying parameter vector (t) satises We
for some known function Y 0 (t) R3 r and some unknown but constant parameter 0 Rr , for some r 1 [under this assumption, Y ( v , v, xs )(t) = Y ( v , v, xs )Y 0 (t)0 , so that Slotine and Lis (1) design in the sub-subsection Design for System with Unknown Jumping Parameters above can be applied]. (2) The time-varying term (t)( x s + v) can be expressed as
r for some known function Z(xs , x s , xm , x m , t) R1r and some unknown but constant parameter 0 R , for some r 1.
We propose the adaptive controller structure
where 0 , 0 are the estimates of 0 and 0. Including the controller (97) in the slave system (83) leads to
POSITION CONTROL Dene the parameter errors = 0 , = 0 , and choose the positive function
27
where
> 0 and
> 0 are constant matrices of the appropriate dimensions. To ensure that

0
0, we choose the adaptive laws for
and
as
With this choice of 0 and 0 , we have = K D s2 0, which implies that s L2 L and 0 , 0 L . Since s L . From Eq. (97) it follows that s , s L ; therefore, s=e + e, we conclude that e, e L2 L . Hence xs , x (t) = vs (t) vm (t) go to zero as t e L . Applying Lemma 3.1, we conclude that the tracking errors e(t) and e goes to . In summary, we have the following results. Theorem 6. The adaptive controller (97) with the adaptation law (100) and (101) applied to the time-varying system (83) guarantees that all closed-loop signals are bounded and the tracking error e(t) and e (t) go to zero as t goes to . Design II for Unparametrizable Parameter Variations.Design II for Unparametrizable Parameter Variations We assume the unparametric parameters having a parametric part and bounded disturbance part. They satisfy the modied assumptions: (1) The parameter (t) satises
where Y 0 (t) and 0 are the same as that dened in (A4), such that 0 < M 1 for some constant M 1 0, and (t) < 1 for some constants 1 > 0. (2) The term (t)( x s + v) satises
where Z(xs , x s , xm , x m , t) and 0 are the same as that dened in (A5) such that | 0 | < M 2 for some constant (t)| < Y 1 (t)2 for some constant 2 > 0 and some known function Y 1 (t). Remark: One choice M 2 > 0, and | s (t) + v(t). of Y 1 (t) is Y 1 (t) = x
28
POSITION CONTROL We propose the controller structure as
and the adaptive law for 0 and 0 as
where , are switching signals dened as
for some constants 0 > 0 and 0 > 0. The Lyapunov candidate function is same as dened in Eq. (99). Using the facts that
T 0 0 0, that Sun (20)], and that 0 T
0 and
T and 0 T 0 go unbounded if 0 (t) and 0 (t) go unbounded [see Ioannou and 0 and 2 are nite [see assumptions (A4 ) and (A5 )], we have that V (t) is bounded, and
Since V (t) is bounded, from Eq. (109) we have
for some constants 0 , 0 > 0, and any t2 > t1 0. Because of the relation s(t) = e (t) + constant, we can obtain
e(t) and
> 0 is
POSITION CONTROL and
29
where 1 , 1 , 2 , and 2 are some positive constants [see Tao (7)]. In this case, the design parameter K D > 0 can be chosen to be large so that the mean errors are small in Eqs. (111) and (112). In summary, we have the following results. Theorem 7. All signals in the closed-loop system (83) with unparametric time-varying parameters, and adaptive control law (104) and adaptive law (105, 106, 107, 108, 109, 110), are bounded, and the tracking errors (t) L2 and limt e(t) = 0 limt e (t) e(t), e (t) satisfy Eqs. (111) and (112), respectively. Moreover, e(t) L2 , e = 0 in the absence of parameter disturbances, that is, when = 0, = 0. Transparency and Stability. For teleoperation systems with known time-varying parameters or parametric time-varying parameters, the adaptive control schemes (84) and (97) ensure velocity tracking from the slave to the master. Therefore the force tracking design as in Eq. (72) will lead to asymptotic weak transparency (Denition 3). For time-varying systems with bounded disturbances, an arbitrary small tracking error can be obtained by increasing the design gain K D . By using the force tracking design in Eq. (72), approximate weak transparency (Denition 4) or approximate weak transparency in the mean (Denition 5) is achieved. Stability of the resulting releoperation system is ensured by the boundedness of all the closed-loop signals. Teleoperation with Communication Time Delay. Communication time delay in a bilateral teleoperation system reduces system stability and performance. Delay on the order of a tenth of a second were shown to destabilize the teleoperator. The stability problem becomes difcult when a communication time delay T is present, because a time delay introduce a factor e sT into the system and hence makes the system innitedimensional. In bilateral teleoperation, the force reection from the slave, introduced for providing the feeling of the remote task, has effects on the masters motion, which generates disturbances on the desired motion. The communication delay may worsen the situation as well. With a time delay T , the conventional communication law results in tracking of both position and force in the steady state. However, with the delay, the system is not passive, and will probably never reach a steady state. A preliminary result on a modied control scheme that provides improved tracking performance for a noncontact task in the presence of time delay and for arbitrary master trajectories has been developed in Shi et al. (21). In the previous research, the passivity formalism and network expression were used to investigate the stability of a bilateral teleoperation system. In these methods, the human operator input is assumed to be bounded, and the human operator and the environment are assumed to be passive. In the presence of communication time delay, and with the passivity assumptions about the operator and the environment, passivity of the system depends on passivity of the communication block. Two approaches can be used to produce a passive communication block. The rst was developed by Anderson and Spong (22) using scattering transformation theory. This solution uses the transmission line equations as a basis for deriving a passive communication control law. By applying scattering theory, it is shown how conventional approaches lead to innite gain of a scattering operator at nite frequencies, and how by implementing a set of time delay equations this instability can be overcome. The resulting system is then passive for all time delays. The proposed control law maintains steady-state force and velocity tracking. The second approach to produce a passive communication block is developed by Niemeyer and Slotine (23). This approach uses an energy formulation to construct a teleoperation system that imitates physical systems and obeys an energy conservation law. A wave variable is utilized to characterize time delay systems and leads to a new conguration for force-reecting teleoperation.
30
POSITION CONTROL
Since the dynamic control of the remote slave by an operator is severely restricted by the time delays in the transmission, it is important to provide consistent dynamic performance locally at the remote site in the face of uncertainties and varying operating conditions. With the development of high-speed and high-capacity computer networks, it is possible to deliver teleoperation over a public computer network. The problem of varying communication time delays arises in such a teleoperation system. Adaptivity of a teleoperation system to uncertain time delay in also desirable. The related stability, tracking, and transparency problems of a bilateral teleoperation system under uncertain environment but now with communication time delay are important issues to be addressed. Adaptive control solutions proposed in the subsection Control Designs need to be modied to provide adaptation mechanisms for adjusting the controller parameters to achieve desired system performance, despite system uncertainties due to the unknown slave environment, and now in the presence of communication time delays. The passivity-based solution to the bilateral teleoperator time delay problem developed by Anderson and Spong (22) is based on the result in circuit theory that a circuit consisting of passive elements only is passive and therefore stable. However, if some elements in a circuit representing a teleoperation system are not passive, one cannot use passive network theory to conclude the stability of the teleoperation system. On the other hand, if the transfer function of a teleoperation system is positive real, then the system is passive. In Shi et al. (24), a notion of positive realness has been used to investigate the passivity of the teleoperation system proposed by Anderson and Spong (22). Shi et al. (24) have also proposed a modied control scheme that use the master accelaration information (with delayed operation, which can be obtained from the velocity information) for slave control and ensures that in the absence of slave environment torque the slave position tracks that of the master asymptotically, that is, achieves improved tracking performance for the teleoperation system.
Summary
Position control for robot manipulators and teleoperation systems involves many dimensions of control theory, such as controller design, robustness analysis, and adaptive designs, along with many practical applications. Robust adaptive control schemes have been presented to handle situations in which the robot system has bounded parameter variations or/and unmodeled dynamics of bounded gains. The distinct feature of the manipulator dynamics were used to dene bounding signals in the controller structure, whose parameters are updated from a robust adaptive law to ensure signal boundedness and tracking errors of the order of parameter variations and unmodeled dynamics, which may not be small. Some common topics relevant to position control of robot manipulators, such as PD control, inverse dynamics control, and path or trajectory interpolation, were also discussed. Adaptive motion control of teleoperation systems was addressed. Several new concepts of transparency were dened for teleoperation control systems with unknown parameters. These new transparency concepts are useful for developing adaptive control schemes for control of a teleoperation system with unknown constant parameters, or with unknown jumping parameters, or with unknown smooth but large time-varying parameters. Weak transparency properties have been established for such adaptive teleoperation control systems. Some important control issues for teleoperation systems with communication time delay were also discussed.
BIBLIOGRAPHY
1. J. Slotine W. Li On the adaptive control of robot manipulators, Int. J. Robotics Res., 6 (3): 4959, 1987. 2. J. Slotine W. Li Adaptive manipulator control: A case study, IEEE Trans. Automat. Control, AC-33: 9951003, 1988. 3. J. S. Reed P. A. Ioannou Instability analysis and robust adaptive control of robotic manipulators, IEEE Trans. Robotics Automation, 5: 381386, 1989.
POSITION CONTROL
31
4. M. W. Spong Adaptive control of exible joint manipulators, Systems Control Lett., 13: 1521, 1989. 5. F. L. Lewis S. Jagannathan A. Yeildirek Neural Network Control of Robot Manipulators and Nonlinear Systems, London: Taylor & Francis, 1999. 6. S. S. Ge T. H. Lee C. J. Harris Adaptive Neural Network Control of Robotic Manipulators, River Edge, NJ: World Scientic, 1998. 7. G. Tao On robust adaptive control of robot manipulators, Automatica, 28 (4): 803807, 1992. 8. M. W. Spong M. Vidyasagar Robot Dynamics and Control, New York: Wiley, 1989. 9. R. Ortega M. Spong Adaptive motion control of rigid robots: A tutorial, Automatica, 25 (6): 877888, 1989. 10. C. A. Desoer M. Vidyasagar Feedback Systems: InputOutput Properties, New York: Academic Press, 1975. 11. M. W. Spong R. Ortega R. Kelly Comments on Adaptive manipulator control: A case study, IEEE Trans. Automat. Control, AC-35: 761762, 1990. 12. P. A. Ioannou K. Tsakalis A robust direct adaptive controller, IEEE Trans. Automat. Control, AC-31: 10331043, 1986. 13. V. Utkin Variable structure systems with sliding modes, IEEE Trans. Automat. Control, AC-22: 212222, 1977. 14. D. A. Lawrence Stability and transparency in bilateral teleoperation, IEEE Trans. Robotics Automation, 9 (5): 624637, 1993. 15. K. Hashtrudi-Zaad S. E. Salcudean Adaptive transparent impedance reecting teleoperation, Proc. IEEE Int. Conf. on Robotics and Automation, Minneapolis, 1996, pp. 13691374. 16. H. K. Lee M. J. Chung Adaptive controller of a masterslave system for transparent teleoperation, J. Robotic Systems, 15 (8): 465475, 1998. 17. M.-Y. Shi et al. Adaptive control of teleoperation systems, Int. J. Robotics Res. Preliminary version, Proc. 38th IEEE Conf. on Decision and Control, Phoenix, AZ, 1999, pp. 791796. 18. G. J. Raju G. C. Verghese T. B. Sheridan Design issues in 2-port network models of bilateral remote manipulation, Proc. 1989 IEEE Int. Conf. on Robotics and Control, 1989, pp. 13161321. 19. G. Tao A simple alternative to the Barbalat lemma, IEEE Trans. Automat. Control, 42: 698, 1997. 20. P. A. Ioannou J. Sun Robust Adaptive Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 21. M.-Y. Shi et al. Control issues in teleoperations with communication time delays, Proc. 32nd Ann. Conf. on Information Sciences and Systems, Princeton, NJ, 1998. 22. R. J. Anderson M. W. Spong Bilateral control of teleoperator with time delay, IEEE Trans. Automat. Control, 34: 494501, 1989. 23. G. Niemeyer J. E. Slotine Stable adaptive teleoperation, IEEE J. Ocean Eng. 16: 152162, 1991. 24. M.-Y. Shi G. Tao J. H. Downs Positive realness and tracking of teleoperation systems, Proc. 37th IEEE Conf. on Decision and Control, Tampa, FL, 1998, pp. 25272532.
GANG TAO AVINASH TAWARE MINYAN SHI University of Virginia
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Process Control Standard Article Michael A. Gevelber1 1Boston University, Boston, MA, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1034 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (223K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Process Control Analysis and Design Issues Application Examples: Electronic Materials Processing About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
PROCESS CONTROL
Process control was initially developed from the practices of process industries such as paper, steel, and chemical manufacturing. Beginning in the early part of the century, it became more widely practiced starting in the 1940s. Applications now include processes in microelectronics manufacturing, such as bulk crystal growth, chemical vapor deposition, and etching. Process control is important because it provides a means for improving process yield and reducing variation, and also enables new processing capabilities to manufacture unique engineered materials and structures. Process control provides an approach to developing an automatic control system for complex, nonlinear, distributed parameter systems involving mechanical, transport, electrical, material, and chemical attributes. The term process control is used broadly to cover a variety of functions including supervisory control, failure detection, and the lower level control that determines the required inputs to achieve a desired objective. It is applicable to large plants incorporating many subprocesses and for single processes. This article primarily focuses on the issues associated with developing the lower level of control for a single process. A major goal of process control is to automatically determine the process input settings to achieve the desired output condition (referred to as the command signal) while minimizing the variation of the output from the desired level (e.g., error). Variations are caused by external inputs called disturbances and variation of the plant, such as aging. Performance is evaluated in terms of dynamic aspects (how fast the system needs to respond) and magnitude of the error. A powerful conceptual approach used in control system analysis is to evaluate the output error as a function of command and disturbance signals, which provides a useful basis for designing compensators to meet performance specications. Taguchis concept of the process variability loss function provides an important basis for evaluating the value of process control (1). Given target quality objectives, deviation from that target should be characterized by some loss function. Thus, for a quadratic loss function, the narrower the distribution, the greater value the process provides. One way manufacturers deal with process variations is to divide (or grade) production output into various quality classes, a practice common throughout semiconductor manufacturing. This approach, however, actually adds cost because sorting provides no additional value. In addition, there is the loss associated with the value that could have been attained if the production had met a narrower distribution. In this context, process control is valuable if used to reduce variation, but the cost of the solution must be balanced against the return. There are two major control options: feedforward and feedback. Feedback utilizes an error signal formed by comparing the actual output to the desired value, which is then used to determine a corrective adjustment to the plants inputs. Feedback corrects for unknown variations, but it is reactive, acting only after an error has occurred. Feedforward control utilizes process knowledge to determine the input values required to achieve the desired output without process measurements. Such process knowledge can be informal, such as that of the equipment operator, or expressed formally in a mathematical model. Feedforward is anticipatory for known conditions, but does not compensate for unknown factors, such as disturbances or plant variations. Feedforward control is typically used in cases where a cost-effective sensor is not available and a robust model is available.
1
PROCESS CONTROL
Such an approach has been used to control wafer temperature distribution in tube furnaces where it is not practical to measure the wafers temperature distribution. Feedback systems are implemented either utilizing measurements in real-time or after a process is completed. Real time implies that the output (or related process states) are measured as the process is running and the inputs are adjusted on the y to compensate for errors. If the process variations or commands are slow relative to processing time (i.e., the batch time), then real-time control is not necessarily needed. In this case, run-to-run control is used where the results are measured after the process is completed. If the disturbances have long-term behavior relative to process batch time, run-to-run control works well assuming that there is only one independent disturbance. In cases where variations are fast relative to process timescales, closed-loop control provides signicant performance advantages. For example, in spot welding, the thermal dynamics of the process vary signicantly throughout the welding cycle, so that development of a dynamic measurement controller provides signicant performance improvements (2). Similarly, real-time control also works well to eliminate time-varying disturbances. It also is advantageous if one wants to build in process exibility (i.e., the ability to change process output levels), even for slower processes, when process modeling is not accurate enough to meet the performance objectives. It is not always possible to directly measure the variable related to the process objective. An alternative strategy is to identify and control an intermediate or secondary process variable related to the primary variable. In crystal growth, for example, dislocation density is an important material objective for optoelectronic devices which is not feasible to measure in realtime. Even if it were measurable, it might not be desirable because, once a dislocation has been introduced in the crystal matrix, it cannot be removed. However, by controlling the crystals thermal gradients, dislocations are prevented from forming in the rst place. A related strategy is to use cascade control structures wherein disturbance paths are identied and local loops eliminate their effect before they propagate to the primary control objectives. Other approaches to process control include minimizing input variations [the objective of statistical process control (SPC)], thereby eliminating external disturbances, or selecting the operating regime so that the output is insensitive to the disturbance (the later approach is Taguchis design of experiments technique). The disadvantage of SPC is that it requires continued effort to track down all disturbances, whereas feedback control operates automatically. The disadvantage of the design of experiments is that it requires an extensive set of experiments on the production equipment, which could signicantly affect production. Taguchis technique, however, utilizes existing equipment without additional capital or development costs. In some cases, it is benecial to combine these techniques. For example, design of experiments is used to determine the desired operating point when number of input parameters are to be set, and local feedback is used to eliminate disturbances and plant variation. Feedback control, however, can be implemented only if there is a manipulable input that compensates for the specic disturbance. Developing a process control system requires addressing a number of issues beyond the design of the control algorithm (i.e., the equations that map measurement information to the input levels). Important issues include specication of the relevant process objectives (both technical and economic), identifying the constraints posed by the equipment and process physics, determining which control structure best achieves the desired performance objectives, and considering opportunities for improving the process capabilities by changing the system design. Answering these questions typically requires a good understanding of the relevant process physics in addition to the relevant control principles. Modeling is useful in answering these questions, but should be performed from a control perspective. Such a perspective seeks to understand the system by identifying its fundamental dynamics and the limitations they pose, its disturbances, objectives, and available manipulable inputs. One major control structure issue stems from the fact that more than one input typically affects the desired output(s). Thus, it is important to understand which inputs can achieve the best performance. One way to evaluate the performance of alternatives is utilizing frequency-domain techniques to quantify gain and bandwidth for alternative inputs. Additionally, one should evaluate if some outputs are more difcult to control
PROCESS CONTROL
independently. For example, in a welding process, it is difcult to independently control the width of the heat affected zone and the size of the weld nugget. These problems are related to the inherent coupling of a process, and it may not be physically possible to achieve all the objectives. Thus, an important objective of the analysis is to determine such inherent conicts. A control perspective is also useful in selection of the operating regime, i.e., the specication of the nominal operating conditions for all inputs and process settings or trajectories. Important factors include how fast the process proceeds, the signicance of different disturbances, and which aspects of the process physics dominate. The operating regime is selected from a basic understanding of the dominant process physics and/or from formal optimization techniques, such as design of experiments. Consideration should also be given to redesign of the process and/or alternative control structures (i.e., combinations of feedback, feedforward, or cascade) to improve performance. Important examples of the benet of integrating system and control design include eliminating known disturbances, adding new actuators to achieve independent control of an additional output, and sizing actuators to meet the performance specication. Development of the control system requires identication of the fundamental dynamics of the process. Important issues to address are factors that fundamentally limit control performance or that need to be compensated for through system or control design. Some of these features might suggest changes in the operating regime selected to avoid the problem, such as if there are poorly damped modes. Other features, such as nonlinearities, need to be identied and considered in designing the control algorithm.
Process Control Analysis and Design Issues

This section provides a more formal statement of the analysis and design options for control. While this analysis is presented in terms of linear systems, the analytical framework of control theory provides guidance and a formal basis for developing an appropriate process control system. Control Structures and Performance Analysis. Feedforward and feedback control is represented by the block diagrams in Fig. 1. Each block represents the mapping of an input u to an output y. The plant (e.g., process) is represented by g and the controller by k. Lower case letters represent single-input, single-output (SISO) systems whereas upper case letters represent multiple-input, multiple-output (MIMO) systems. Thus g is a scalar operator mapping u to y (y = gu) whereas G is a matrix operator mapping the vector of inputs to the vector of outputs (y = G u). The dynamics of linear constant coefcient systems is analyzed in the Laplace domain with complex transform variable s in terms of transfer functions such as g(s) and k(s). (The functional dependence on s will not be explicitly indicated henceforth.) Time-varying inputs for r, n, and d are readily analyzed in terms of a family of inputs (impulse, step, and ramp) and in the frequency domain (see the following section). The input-output relationships for a SISO system are as follows (3,4):
where do is an unknown disturbance that acts on the plant and is reected at the output, di is a disturbance that affects a process input, and n is a noise signal that corrupts the measurement of the output. (These
PROCESS CONTROL
Fig. 1. (a) Feedforward and (b) feedback control structures.
relationships are generalized for MIMO systems by matrix algebra.) The output disturbance do indicates that there is an effect on the output but the functional form is not known, whereas an input disturbance di represents variations of the manipulated input or additional external inputs whose functional input/output mapping is known. The achievable control performance is analyzed in terms of the error where e = r y. The performance of the two structures is expressed as
Equation (4) reveals that feedback control reduces process variation, that is |e| can be made small. In contrast, process variation for feedforward control is reduced only by eliminating disturbances and plant variation (see Eqs. (3) and (5)). Analysis of the steady-state performance to step inputs provides insight into feedback control performance. This can be done by utilizing the nal value theorem, which for step inputs corresponds to evaluating the magnitude of the corresponding closed-loop transfer functions [T = gk/(1 + gk), S = 1/(1 + gk)] at steady state (i.e., s = 0.). In general, making |gk| 1 achieves good performance for following commands and rejecting disturbances because this makes |S| 1. Performance with respect to insensitivity to noise, however, is poor because |T | 1. Note that several dynamic factors limit the maximum gain that can be used, thereby limiting achievable performance (see the following section). One can consider the feedback controller k as an amplier because u = kem (where em is the measured error) and u is a power-level signal. Viewing k as an amplier is consistent with requiring |gk| 1 to achieve good performance. A simple version of such a controller is a proportional gain (i.e., P-type controller). More complex forms with varying gains for different frequency ranges can achieve better performance. In cases where
PROCESS CONTROL
u is not an electrical signal, the controller can consist of two parts: the control algorithm and the actuator that manipulates the physical input u. The achievable performance relative to variation of the process and/or model error, g, is given by
where g = m + g and m is the nominal process or model. Thus, feedforward is sensitive to the relative model error, but feedback control compensates for such error or process variation, provided S is made small. The different approaches to reducing process variation are analyzed by linearizing the output about the nominal operating point (W o , yo ) and expressing the output variation as
where the nonlinear mapping of the material inputs w and manipulable inputs u is given by y = f (w, u). Classical SPC and Taguchi are considered feedforward approaches wherein SPC seeks to identify the assignable causes of the variation and to eliminate them, thus making |w| small. Taguchi seeks to identify the operating regime (i.e., values of wo , uo ) whereby the output is insensitive to the variations, that is |f /w| 1. The merit of this method is that it requires no additional capital expense. However, because it is based on experimentally mapping the input/output space, it might be quite expensive to run the experiments on the production line. The required condition for implementing a feedback solution is that the plant gain g = f /u = 0. Thus, in some cases there are disturbances or operating conditions that cannot be corrected for by feedback. Closed-Loop Frequency and Dynamic Analysis. Frequency analysis yields important insight into designing an appropriate process control system and into factors that pose important performance limitations. It is also used to analyze the performance of more complex inputs and nonlinear systems linearized about a nominal operating point. The output response of a linear system g(s) to a sinusoidal input u(t) = sin t is given by
where |g(j)| is the magnitude of the complex transfer function evaluated at frequency and is a phase shift. The frequency characteristics of a system can be visualized by plotting the magnitude of g(s) as a function of frequency (s = j) on a log-log plot, known as a Bode plot (Fig. 2). An important characteristic is the range of frequencies with uniform amplication called the bandwidth BW . The Laplace domain provides a basis for relating the characteristics of the Bode plot to the time-domain characteristics of the process. For example, the open-loop bandwidth approximates the open-loop dynamic response as BW dominant 1 dominant where dominant is the dominant pole of the system, and dominant is the dominant time constant of the system (3). Frequency analysis is useful for analyzing the error magnitude and dynamics for closed-loop systems and for compensator design. Important design insight is obtained by comparing the magnitude plots of |gk|, sensitivity S(s), and closed-loop T (s) transfer functions (Fig. 2). The closed-loop bandwidth of T (CLBW ) is bounded by the frequency that |gk| 1. Since the closed-loop bandwidth characterizes the range of frequencies
PROCESS CONTROL
Fig. 2. Bode magnitude plot of open- and closed-loop transfer functions.
for good command following, disturbance rejection, insensitivity to plant variations, and speed of response, it is used as a primary design variable. Thus K (s) is selected to achieve the desired crossover frequency for gk. The desired performance of the system is expressed in terms of the range of frequencies where r, d, and n have signicant energy (i.e., large magnitude). Noise and stability pose important performance limitations because S and T cannot be made small in the same frequency range. Thus, good performance cannot be achieved if the power spectrums for desired command following and disturbances overlap the noise spectrum. In addition, the Nyquist stability criterion poses a conservative magnitude bound on |gk| relative to the uncertainty of the open-loop process dynamics as |T | < |g/m|. Thus, CLBW is limited by model uncertainty, and an accurate process model is needed through the desired closed-loop bandwidth. (Similar logic can also be used for determining the model accuracy required to achieve performance objectives for a model-based feedforward control approach.) The frequency-domain analysis also provides a context for selecting appropriate actuators. Each process typically has a variety of inputs that might be used as the manipulable variable for real-time control. From a general nonlinear representation of the process, y = f (u1 , u2 , . . .) where ui is the ith input of l different inputs, one can linearize the system about the nominal operating point yo , uo i . Because at least p inputs are needed to independently control p outputs, the rst design decision is which p inputs achieve the best performance. For single-input, single-output (SISO) systems, the question reduces to Which of the actuators achieve the best performance? The comparison is made in terms of the open-loop bandwidth, reecting how fast that input affects output, and the open-loop gain, which indicates how large an impact input has on output. Inputs with the greatest bandwidth and gain typically result in the best performance. However it is not necessarily true that all high-bandwidth systems have large gain, thus possibly requiring a tradeoff analysis. High-bandwidth systems require less gain to meet the desired closed-loop bandwidth than slower systems, thus typically they have a larger robustness margin. In addition, they may have less model error over the same frequency range because the dominant pole is at a higher frequency. Similarly, high-gain systems require less controller gain to achieve a desired error performance level, thereby improving the robustness bounds. Alternative actuators are compared by plotting the Bode plots for each input. Because the inputs are of different units, the transfer functions should be scaled so they can be compared on a consistent basis. For example, normalizing by the nominal value of each input expresses each input as a fractional variation of the nominal input. In selecting actuators, one should also evaluate limitations posed by dynamic issues such as nonminimum phase characteristics including time delays, open-loop instabilities, and right-half-plane (RHP) zeros (4). This factors fundamentally limit achievable closed-loop performance. For example, a RHP zero causes defective transient responses in that the initial response is in the direction opposite to the nal steady-state. In practice, the closed-loop bandwidth is limited to the bandwidth of the RHP zero.
PROCESS CONTROL
Fig. 3. MIMO system block diagram.
MIMO Systems. Additional issues arise for MIMO systems because of cross-coupling between inputs and outputs. This coupling causes more than one output when a single input is varied (Fig. 3). MIMO systems are represented by y = G u, where G is a p l matrix mapping l inputs to p outputs. The frequency analysis is extended to MIMO systems by utilizing the spectral norms and principal gains of the transfer matrix which are calculated by a singular value decomposition. For a p p matrix, there are p singular values, i , which are positive scalars (3,4). Thus, for |u| = 1, (G)|y| (G) where , are the maximum and minimum singular values of G(s). (G)|y| (G)(G)|y| (G) There are a variety of control design algorithms for MIMO systems that compensate for cross-coupling. However, there are some systems, where the coupling makes it difcult to independently control all outputs. 1. Mathematically, a large This problem is characterized a large condition numbers for G, that is, 2 (G) = / condition number indicates that the matrix is close to loosing rank, that there are output directions which entail large control efforts. Such large inputs might saturate the actuators, which could result in loss of stability or inability to achieve the desired output value. On a practical basis, trying to design a system to operate under these conditions requires actuators which, for the most part are not used signicantly, except to reach sensitive directions. Another analysis method that provide similar insight is the relative gain array (RGA) (3). An important solution for poorly conditioned systems is to seek alternative system designs that improve the ability to control the process. Determining the reason for the poor conditioning is helpful and is obtained from the singular vectors of the process or by decomposing the transfer matrix into column vectors (15). Poor conditioning results from several actuators with similar effects on the outputs or an input that does not have the same magnitude of impact as the others. Alternative designs can be proposed once the nature of the limitation is determined. Because both i and 2 depend on the input/output scaling, all variables should be scaled consistently. Use of dimensionless variables compensate for the different units used for each input and output. Different normalization methods are used such as scaling about the nominal operating point, or dening perturbations that reect relevant engineering scales, such as tolerances, saturation limits, or error limits. For example, chemical engineers normalize outputs by transmitter spans and inputs by appropriate valve gains (3). Other Control Structures. Two important variations of control structure use alternative measurements of the primary process output. In processes where it is not practical or possible to directly measure the process output, a secondary variable related to the primary objective is used for feedback. For example, in bulk crystal growth, dislocation density is important because it affects the electro-optical characteristics of the material, but it cannot be measured in real time. Because dislocations are related to temperature gradients, one can instead control the temperature gradients to prevent the dislocation from being introduced into the
PROCESS CONTROL
Fig. 4. Cascade control structure.
crystal matrix. In this example, there is an added benet because a process state upstream of the output is controlled by feedback. Because feedback control is reactive, it only takes control action after an error has occurred. In the case of dislocations, however, once a dislocation has been introduced into the crystal matrix, it cannot be eliminated. Thus by controlling the upstream variable (e.g., the temperature gradients), one prevents the defect from occurring. A related option is to utilize a cascade control structure (Fig. 4) (3). If a disturbance or a known process variation can be identied and measured, it is possible to close the loop around this variable to ensure that the variation does not propagate downstream to affect the primary objective. The benet of this practice is that it results in better overall performance for a level of desired robustness because it signicantly reduces the action that the primary control loop needs to perform. Control Algorithms. There are a variety of methods for designing the control algorithm, and the reader is referred to the other related articles for specic details. For linear systems, important design methods include classical design methods such as PID for SISO systems, and more advanced techniques such as optimal control, robust control, H innity, and model predictive (3) which can be used for both SISO and MIMO systems. These methods require a dynamic process model which can be either derived from rst principles or experimentally using system identication techniques. Alternatively, adaptive control techniques do not require a system model. For systems that vary in time or with a changing parameter, gain scheduling is used to interpolate between control designs developed for different conditions. Where there are signicant distributions of disturbances or noise, stochastic control design can be used. Some systems have characteristics that require special approaches. These include nonlinearities and distributed parameter systems (which entail spatial variations). Other control approaches provide important advantages such as neural networks for nonlinear systems, fuzzy logic that responds to different conditions, and non-real-time techniques based on statistics and designed experiments (4,5).
Application Examples: Electronic Materials Processing Furnaces. Control of temperature is critical in many processes because it determines process uniformity, yield, material characteristics, and production rate. Two types of thermal systems used in the electronic fabrication industry that illustrate interesting control issues are tube furnaces to heat many wafers at the same time and rapid thermal processing (RTP) systems for single wafers. Control of Tube Furnaces. Tube furnaces applications include dopant diffusion and oxidation, which are highly sensitive to temperature variations across the wafer. In addition, temperature differences also induce thermal stress that damage the wafer. Because wafers are loaded into a quartz boat and heated from the edge, the rate at which the wafers are brought up to the processing temperature is limited by the radial conduction of heat and the thermal dynamics of the furnace and load. Typically, independently actuated, multiply segmented heaters are used to compensate for spatial thermal variations. These variations occur
PROCESS CONTROL
because of different loading (i.e., the number of wafers processed each in run), the spatial characteristics of heat transfer modes (such as natural convection effects that differ signicantly for vertical and horizontal furnaces), and end effects. Important process objectives include the rate of heat-up/cool down (which affect production rate without providing value), maintaining thermal uniformity, and achieving the process setpoint temperature. Because of sensor limitations, it is difcult to directly measure the wafer temperature. Thermocouples used in the furnace include spike thermocouples, located near the heater elements, and prole thermocouples, located near the wafer edges. Thus, tube-furnaces are controlled by secondary measurements. The process is a coupled MIMO problem because each furnace segment affects its neighbors. Traditionally, decoupled proportional-integral-differential (PID) loops control each zone, which does not take into account the cross talk of neighboring elements. This coupling limits how tightly each loop is tuned, because unmodeled dynamics cause it to become unstable or introduce a disturbance into the neighboring region. Several model-based control schemes have been developed to overcome the measurement limitations. These schemes are essentially a hybrid of feedback and feedforward, where the model is infers the wafer thermal distribution based on the process model and measurement data and designs a MIMO controller. Because many of the primary heat transfer coefcients dependent on operating temperature, wafer loading, geometry, and material properties that are difcult to measure directly and/or highly variable, the model has a number of coefcients that must be empirically determined with instrumented dummy wafers. A MIMO control approach enables more aggressive control action because the otherwise ignored interactions between zones is now taken into accout. Development and application of such a process has been undertaken by a team combining the manufacturer (Motorola), the furnace vendor (Silicon Valley Group), and the control company (Voyan) (5). Implementation has reduced process cycle time by 18% and the fractional variation in standard deviation (dened as the standard deviation normalized by the set-point temperature) was reduced from 1.67 to 0.77% at high temperatures (950 C). Rapid Thermal Processing. Rapid thermal processing (RTP) technology enables fast processing for a variety of semiconductor manufacturing applications including diffusion, oxidation, chemical vapor deposition, and nitridation. Tungsten-halogen lamps heat the entire wafer surface, thereby minimizing processing times and achieving novel structures by avoiding the slow thermal relaxation that occurs with slow ramp rates. RTPs commercial prospects are in low-volume production of application-specic integrated circuits (ASIC), shortening the development time for new products, and possibly competing with large conventional fabricating processes because of the reduced processing time. Typical operating characteristics include ramp rates on the order of 50 C/s from ambient up to 1100 C, constant temperature for 15 minutes, followed by a rapid cool down rate. Temperature uniformity across the wafer is a critical performance objective. For example, the sensitivity of coating thickness variations to temperature variation for oxidation or CVD deposition of lms can be calculated from the deposition rate relation Rdep = k exp (E/kT ) because these processes are typically thermally activated. Normalized sensitivities for the growth of polysilicon layers are 2.5%/ C of variation. Real-time control is critical in several aspects of RTP. Although the ramp rate is determined by the lamp power ux, achieving uniform thermal distribution in both steady-state and during the ramps is difcult because of heat transfer variations across the wafer. The different view factors, wafer xturing, and the variations in radiation and convective heat transfer over the range of temperatures and pressures prevent use of a single open-loop thermal actuator design to achieve uniform temperature for all conditions. Even a single closed-loop controller will not work. Process nonlinearity is characterized by gain values and time constants that vary by an order of magnitude over the operating regime (7). System design is important in achieving good control performance for RTP. Independently driven thermal actuators are desirable to compensate for heat transfer variations across the wafer. In a joint TIStanford design, three independently controllable concentric rings of actuators were used (6). The maximum input power was 2 kW, 12 kW, and 24 kW (starting with the inner ring), consistent with the increased loss toward the outer portion of the wafer. Even though reectors were used, there is some actuator overlap because each portion of the wafer sees more than one bank of lamps. Thus, the MIMO aspects of the problem must be
10
PROCESS CONTROL
Fig. 5. RTP comparison of open-loop (dashed line) and closed-loop (solid line) operation for two 24 wafer lots (7).
considered at both the system design and control design levels. At the system design level, it is appropriate to evaluate the condition number of the system, that is, 2 for the DC transfer matrix. In the rst design, 2 110, indicating that there are some output directions that would result in large actions in one or more actuators (1 = 12.2, 2 = 1.1, 3 = 0.11). This might not be a signicant problem because the maximum singular value corresponds to a condition of uniform temperature, which is the desired command signal. It does indicate that the lamps could be wired in two independent banks. However, it is difcult to compensate for a disturbance corresponding to the worst condition. To solve this problem, a bafe was designed to change the heat uxes of the actuators to the wafer, thereby reducing the condition number to 23. Note that a low condition number does not necessarily mean that the actuators are decoupled. Making accurate measurements is a signicant challenge for RTP control. Thermocouples are not desirable because they require contact with the wafer and entail some amount of lag. Pyrometers are noncontact, but are sensitive to variations in surface emissivity related to temperature and lm characteristics. Acoustic sensors, based on the temperature sensitivity of the wafers elasticity, are being developed that avoid these problems. One solution for optical pyrometer drift is to use run-to-run control to estimate the sensor error (7). Figure 5 compares open-loop RTP control with closed-loop. The open-loop varies widely, indicating that conventional run-to-run control does not perform well because there are no long-term trends. The closed-loop control, in contrast, has a ramp component, indicative of a systematic sensor error. The run-to-run measurement to detect this drift would be lm thickness, because it is strongly temperature-dependent. A number of different design approaches have been used to develop the control algorithm (6). Utilization of independent PID loops does not result in good performance because of the cross talk between inputs. The large range of operating temperatures requires a design technique to handle the related nonlinearities. The control design must also deal with the MIMO nature of the process and the signicant modeling uncertainties. One approach is to incorporate both a feedforward and feedback approach implemented with internal model control (IMC). The process model is obtained from rst principles or by system identication. To compensate for nonlinear variations, gain scheduling is implemented by parameterizing the model coefcients as functions
PROCESS CONTROL
11
of temperature. A linear quadratic Gaussian (LQG) design was also developed on the basis of linear models obtained from system identication. LQG easily accommodates the MIMO nature of the design problem and can also be augmented to include integrated error states ensuring zero steady-state error. Results achieved are control of ramp temperature distribution to 5 C and 0.5 C during hold. Etching. Etching is a process that transfers patterns to a wafer by removing material not covered by a mask. Researchers at the University of Michigan have applied real-time control to reactive ion etching (RIE), which utilizes plasma ions to assist the etching process (8,9). A bias voltage accelerates the ions toward the wafer surface, enhancing etching normal to the surface. The process chemistry is quite complex including both gas phase and surface reactions. Etching process objectives include selectivity, the ability to etch only the desired layer and not other material; uniformity of etch rate over the wafer; anisotropy, the ability to etch the layer primarily in the vertical direction to prevent overhangs under the mask; and the etch depth. The process can be divided into two coupled parts: plasma generation and etching. Inputs to the process include power, gas ow rates, and pressure set point where the pressure is regulated by varying a throttle valve upstream of a vacuum pump. Fluorine concentration [F] and bias voltage are outputs of the plasma process and inputs to the etch process. The conventional control practice is to set the power input and ow rate levels and use closed loop control to maintain chamber pressure. These variables are far away from the physics that determine the etch characteristics, suggesting that regulating intermediate plasma variables might yield better performance. A cascaded control structure is used with an inner loop around the plasma generator and the outer loop using the etch rate to determine the set points of the inner loop. If no real-time etch rate measurements are available, run-to-run control is used to adjust the set-point values of the intermediate variables. The function of the closed-loop control of plasma variables is to reject plasma generation process disturbances. Such disturbances include variations in the RF power and matching network, variations in the performance of the mass ow controllers, aging of the chamber, due to polymers deposited on the inside surfaces, and loading variations that occur when the amount of surface area varies (because of the number of wafers or mask variation). Experiments indicate that etching has important dynamic characteristics that could also benet from closed loop control. At the end of the etch, less material is left so that the etch rate increases. In some cases, this results in etching the sidewalls, resulting in undercutting. Thus, whereas the conventional practice is to utilize simple end-point detection by monitoring when new species are introduced into the gas phase, additional benets can be achieved by etch rate control. There can also be initial transients due to contaminant variations, which change total etch time. Having dynamic control capability enables compensation for both of these factors. Because the plasma generator is nonlinear, either local linear controllers can be developed to operate at a set point or a strategy to compensate for the nonlinearities can be applied. Important design limitations are imposed by both sensor and actuator considerations. Because of the poor signal-to-noise ratio of the uorine concentration sensor, the bandwidth is limited to less than 1 radian/second. Some of the actuators, such as the throttle valve, are highly nonlinear, so that the regulation range is restricted for use in linear local controllers. To minimize steady-state error, free integrator dynamics were added to the controller. Both PID and LQG/LTR methods are used. The resulting performance of real-time control for maintaining plasma states is shown in Fig. 6, which indicates superior rejection of load variability and a more uniform etch rate throughout the process (8). It is also desirable to control etch uniformity across the wafer surface, requiring a multivariable control approach. If real-time measurement of etch rate and uniformity output is not practical, a postprocess measurement run-to-run controller is used, built from a response surface mapping of the local plasma states to the etch outputs (9). Development of the feedback control algorithm requires a more general approach because the MIMO controller entails signicant change in the operating regime. One approach is to consider that the process dynamics are linear and are coupled to a nonlinear static map of the inputs to output magnitudes (e.g., a Hammerstein model 9a). One can design a linear dynamic controller that operates in conjunction with a static controller to compensate for the nonlinearity.
12
PROCESS CONTROL
Fig. 6. RIE etch rate responses for different loading conditions: (a) open-loop, (b) closed-loop (8).
Experimental evaluation reveals that the pure run-to-run approach has results nearly comparable to the cascade approach for uniformity. This suggests that there may be a lack of good control authority in the selected operating regime and/or that the disturbances are more related to the etch process and, therefore, are not compensated for by the plasma real-time controller. Chemical Vapor Deposition. Chemical vapor deposition (CVD) enables the manufacture of coatings with a variety of properties used in semiconductor production. Different coating structures and compositions
PROCESS CONTROL
13
are achieved by proper choice of precursors and deposition conditions which to date are primarily determined empirically. There are numerous CVD variations in terms of different precursors, energy sources, and reactor design. Important processing objectives include composition, structure, thickness, and uniformity of the deposited coating. Process examples used to provide insight into the application of control include control of metallorganic CVD (MOCVD) and deposition of TiN. MOCVD uses liquid precursors in a bubbler to bring a gas phase of the metallorganic molecules into the reactor. Both III-V and II-VI compounds have been grown for applications, such as light emitting diodes, laser structures, and photodiodes. Such structures need precise tailoring of the spatial composition of the coating, which must be repeatable run-to-run. Two approaches employed to control coating composition include (1) control of the actuators and (2) feedback control from a measurement of composition. The rst approach will work if the system is operating in a regime where mass transport limits growth but fails if chemical reactions interfere. Gaffney et al. (10) focus on improving the performance of the input actuators, relying on a feedforward control approach to meet the composition objectives. This approach works if there are no process disturbances (other than inlet material variations) and there are no signicant dynamics to the process relative to the desired deposition structures. Utilization of measurements related to the state of the bubbler enable direct feedback control of this actuator with a simple control law. Evaluation of the performance is established by measuring the optoelectronic characteristics of superlattices grown with ve alternating layers of 3000 GaInAs with 200 layers of InP. Warnicks et al. (11) approach is to measure the composition and thickness of the coating in real time with an ellipsometer. Because of the range of compositions desired to grow Alx Ga1 x , the input-output mapping is nonlinear. The dopant rate dynamics are approximated as a simple integration of feed material inputs with a time delay to represent the transport delay, without detailed representation of process states and/or coupling. A Hammerstein model is used because the linear dynamics are assumed to be invariant to the nonlinearities. This could be valid for a specic operating point, for example, if the bulk pressure and temperature of the process are held constant but there are variations in composition. A nonlinear mapping is proposed that decouples the two outputs (composition and thickness) and linearizes the input-output mapping. Thus, linear control techniques are used for the two resulting decoupled linear systems. Free integrators are incorporated to insure asymptotic performance of the closed-loop system, in spite of the signicant parametric modeling error that might exist. The bound on transient errors is established by the achievable closed-loop bandwidth. The most signicant limitation is that the accuracy of the measurements is not clearly established, thereby introducing a possible DC error in all control efforts. The transient performance characteristics of the closed-loop system are also limited by the dead time. Because the processes are operated in the mass transport regime, the time delay also corresponds to the rate of growth, and limits how sharp a transition is made. In contrast, the feedforward approach of actuator control does not have such a limitation. Major problems with each of these approaches suggest that a combination of methods could achieve better performance. The feedforward aspect would overcome the limitations posed by the time delay, whereas feedback would compensate for parameter uncertainties and unmodeled process physics. The combined control system could result in tighter control of spatial composition, thereby enabling faster growth rates with acceptable error. To generalize control of CVD, one should consider the control structure design question. Gevelber et al. (12,13) evaluated alternative actuator performance capabilities for CVD, where, for many applications, control of grain size in addition to uniformity is critical. A mechanistic model of nucleation and growth suggests that independent control of temperature and deposition rate are required to control grain size. To determine which of the available actuators should be utilized, both the steady-state gain (Fig. 7) and dynamic characteristics (Fig. 8) of each actuator are evaluated in terms of controlling deposition rate (12,13). This analysis is conducted as a function of temperature because it strongly parameterizes changes in the dominant physics. Several input variables are ruled out because they have poor features like sudden reduction of actuator effectiveness
14
PROCESS CONTROL
Fig. 7. Steady-state normalized gains of Rdep for various inputs for TiN CVD (13).
(partial pressure of TiCl4 ). Analysis of the dynamic features are obtained from an analysis of the linearized system. One of the actuators has right-half-plane zero dynamics (pressure) that would limit the achievable control performance. In the case presented, a clear choice is given solely in terms of gain because the open-loop dynamics are similar. Note, however, that the dynamics vary signicantly with temperature. Selection of the appropriate operating regime must account for the variety of conicting performance objectives, such as maximizing growth rate while minimizing nonuniformities, while considering the variations in the dominant process physics. Modeling analysis provides useful insight into answering these questions and should be directed to obtaining information about the dominant phenomena scale. Thus, for this example, because the dominant time delay scales with the reactor volume and ow rate Q, as T D /Q, one can adjust the operating point (here the total ow rate Q), to minimize the fundamental limitation posed by the time delay (12). In hot-wall batch reactors, the reactant concentration varies throughout the reactor, resulting in varying growth rate from point to point. A ratio indicating the sensitivity is developed to determine which operating regime enables one to meet uniformity requirements while maximizing growth rates. Crystal Growth. Growth of bulk crystals by the Czochralski process is one of the rst steps in wafer fabrication. The process begins by bringing a seed crystal to thermal equilibrium with the molten semiconductor, and pulling the crystal from the melt. Heat transfer is a major phenomenon that determines growth characteristics. Important material properties that determine the crystal quality for different applications include dislocation density and dopant distribution. Si crystals are currently grown dislocation-free for commercial use with diameters of 200 mm. Some crystals, however, are more difcult to grow. For example, GaAs has a lower thermal conductivity than Si and a lower critically resolved shear stress (the stress at which a dislocation is
PROCESS CONTROL
15
Fig. 8. (a) Bode and (b) 2% step response of Rdep for various inputs for TiN CVD, T = 1300K (12).
formed). Thus temperature gradients are higher, resulting in greater thermal stress and higher dislocation densities. Dislocations are formed at the melt-crystal interface and also formed and multiplied in the crystal. The diameter is determined at the interface between the crystal and melt, whereas the dopant distribution is determined by the melt ow and the interface. Because the nature of this coupling, it cannot be guaranteed that any arbitrary set of material objectives can be achieved. For example, the interfacial shape that minimizes dislocation formation might not be optimal for other defects and dopant distribution, suggesting that a tradeoff
16
PROCESS CONTROL
analysis is required. To the extent that the detailed physics can be modeled, the analysis can be formalized. However, because much of the important physics is not analytically tractable, important insight is obtained from understanding the physics and experimental investigations. Feedback systems were developed in the 1960s for automatic diameter control. Several different diameter measuring schemes are used including optical and weight measurement. Although there are a number of actuators for the process including heater power, pulling rates, rotation rates of the crystal and crucible position, and possibly an external magnetic eld applied to minimize melt turbulence, typically only heater power and pulling rate are used for diameter control. However, because the other material objectives are coupled to the interface, they could adversely be affected by the action of the diameter servo loop. Examination of the process indicates that several disturbances act on the growth interface (14). Because most pullers are run with a xed charge, the reduction in melt level changes the heat ux and, therefore, the growth dynamics. In addition, as the crystal grows longer, it experiences a changing thermal environment. Thus, a cascaded control structure with an inner melt temperature loop can eliminate the melt disturbance. Direct compensation for the changing thermal environment is more difcult unless one considers designing an independent thermal actuator. The benet of such an actuator is even more signicant if it is desired to control the diameter and also the interface shape. Gevelbers analysis of the MIMO nature of this problem reveals that the system is poorly conditioned without such an actuator (15). Modeling the dominant process physics reveals a right-half-plane zero associated with the weight measurement (14). The modeling provides a parametric expression of the zeros location relative to the process poles which is used to determine when there are signicant performance limitations. The parametric description of the pole location is also used to understand when the system is underdamped, resulting in growth variations. This explains, for example, why it is more difcult to grow GaAs crystals and is used to help select the operating regime. Modeling also suggests important design consequences. For example, although a bottom heater does not achieve better control authority over the interface shape, it does result in a signicantly faster system that may yield performance advantages. Conventional control loops on pullers have been implemented with PID logic. Rivera and Seider have proposed a model predictive controller (MPC) and control structure to help integrate the different process objectives (16). A cascaded control structure accounts for the different timescales. One loop is the bulk controller that manipulates the heater power inputs. The objectives of this loop include pulling velocity, thermal stresses, and the melt dominant distribution. The other loop controls the radius by manipulating the pulling velocity. MPC coupled with models of the process achieves control solutions which directly take into account constraints to meet the other objectives.
BIBLIOGRAPHY
1. M. S. Phadke, Quality Engineering Using Robust Design, Englewood Cliffs, NJ: Prentice-Hall, 1989. 2. K. Haefner et al., Real time adaptive spot welding control. In D. E. Hardt (ed.), Control Methods for Manufacturing Processes. New York: The Dynamic Systems and Controls Division, ASME, November 27December 2, 1988, pp. 5162. 3. W. L. Luyben, Process Modeling, Simulation and Control for Chemical Engineers, 2nd ed., New York: McGraw-Hill, 1990. 4. J. M. Maciejowski, Multivariable Feedback Design, Reading, MA: Addison-Wesley, 1989. 5. C. Porter et al., Improving furnaces with model-based temperature control. Solid State Technology, 39 (11): 119131, 1996. 6. T. Kailath et al., Control for advanced semiconductor device manufacturing: A case history. In W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, 1996. 7. C. Schaper, T. Kailath, Thermal model validation for rapid thermal chemical vapor deposition of polysilicon. J. Electrochem. Soc., 143 (1): 374381, 1996.
PROCESS CONTROL
17
8. O. D. Patterson, P. P. Khargonekar, Reduction of loading effect in reactive ion etching using real time closed loop control, J. Electrochem. Soc., 144 (8): 28662871, 1997. 9. M. Hankinson et al., Combined Real-Time and Run-to-Run Control of Etch Depth and Spatial Uniformity in Plasma Etching, J. Electrochem. Soc., 144 (7): 24732479, 1997. (a) L. Ljung,, System Identication, Englewood Cliffs, NJ: Prentice-Hall, 1987. 10. M. S. Gaffney et al., Control of III-V epitaxy in a metalorganic chemical vapor deposition process: impact of source ow control on composition and thickness, J. Crystal Growth, 167 (12): 816, 1996. 11. S. C. Warnick, M. A. Dahleh, Feedback control of MOCVD growth of submicron compound semiconductor lms. IEEE Trans. Control Syst. Technol., 6: 6271, 1998. 12. M. Gevelber, M. T. Quinones, M. L. Bufano, Towards closed-loop control of CVD coating microstructure, J. Mater. Sci. Eng. A, 209: 377383, 1996. 13. M. A. Gevelber et al., Modelling TiN Deposition for Control of CVD. In T. M. Bessmann et al. (eds.), 96-5: Int. Conf. CVD XIII, 189th Meet. Electrochem. Soc., Los Angeles, May 1996, pp. 157162. 14. M. A. Gevelber, G. Stephanopoulos, Dynamics and control of the Czochralski process, II. Objectives and control structure design, J. Crystal Growth, 91: 199217, 1988. 15. M. A. Gevelber, G. Stephanopoulos, Control and system design for the Czochralski crystal growth process, J. Dynamic Syst., Meas., Control, 115 (1): 115121, 1993. 16. R. IrizarryRivera, W. D. Seider, Model-Predictive Control of the Czochralski Crystallization Process: Part I Conduction-Dominated Melt, J. Crystal Growth, 178: 593611, 1997.
MICHAEL A. GEVELBER Boston University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Robust Control Standard Article Michael G. Safonov1 1University of Southern California, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1047 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (363K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Origins of Robust Control Theory Sources of Uncertainty Stability and Robustness Multivariable Robustness Further Reading About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
592
ROBUST CONTROL
ROBUST CONTROL
The role of feedback in automatic control systems is to exploit evolving real-time measurements to obtain more precise control over system behavior. A robust control system is one that performs within specied tolerances despite uncertain variations in the controlled plant within given bounds. Robust control theory is that branch of mathematical system theory concerned with the design and analysis of robust feedback control systems. The block diagram in Fig. 1 represents a typical feedback control system consisting of an uncertain plant and a controller. The plant might be an aircraft or missile, a robotic lunar surface explorer, a chemical process, an automobile engine, or a nuclear power plant. It might be something as small as the magnetic read-head positioning system in a computer disk drive or as large as the global economy. It could be almost any complex dynamic system. The controller, on the other hand, is typically a computer or a microprocessor, though it could be simply the thermostat in a home heating control system or the mechanical linkage in the eighteenthcentury yball governor invented by James Watt for controlling steam engines. From a control theorists perspective the uncertain plant and the controller are simply mathematical relationships, for example, differential equations. Robust control theory is focused on the quantitative analysis of the consequences of plant uncertainty. The need for robust control arises when a control system design must be based on an inexact mathematical model of the true physical plant.
ORIGINS OF ROBUST CONTROL THEORY Since Watts invention of the yball governor in 1788, designers of feedback control systems have implicitly sought robust designs. Robustness considerations are implicit in the classical feedback design methods based on root locus and frequency responses. But prior to 1960, the issue of robustness had not been generally recognized by mathematical control theorists, and the term robustness itself did not appear in the literature of mathematical control theory before 1975. Apparently the rst clear mathematical formulation of a robust feedback control problem was produced by I. Horowitz (6) in the early 1960s. Horowitz studied simple linear time-invariant feedback control systems with a single feedback loop for plant models with several uncertain parameters. He correctly recognized that plant uncertainty is a dominant factor in determining what can and cannot be achieved with feedback control, at least for simple single-loop feedback systems, and
d Command + r Error e K(s) Control u G(s) Plant + + Output y
Controller
Figure 1. Robust control concerns designing feedback controllers so that the output y(t) will precisely track command signals r even when the plant is uncertain.
he showed how to design uncertainty-tolerant controllers using classical root-locus and frequency-response methods. Unfortunately, the signicance of Horowitzs contribution was not immediately noticed by control researchers. A gap between mathematical control theory and control engineering practice emerged in the 1960s and early 1970s, as the mathematical theory of feedback control increasingly decoupled from engineering practice. Time-tested root-locus and frequency-response techniques were regarded by researchers as ad hoc and simplistic. They sought to pose feedback design as a mathematical optimization. But, explicit mathematical representations of robustness issues were not incorporated in their mathematical representations of the problems of feedback control. In those early days, mathematical control research generally presumed that available mathematical models were sufciently accurate. The profound signicance of this omission was not immediately evident. In the 1960s, mathematical proofs of optimality were widely regarded as sufcient evidence that the emergent, but as yet untried, modern mathematical theories would produce superior results. But in the early 1970s, researchers began to be jolted as early attempts to apply mathematical optimization theories to the design of complex multivariable feedback controllers for military aircraft and submarines resulted in some surprising failures. In one unpublished design study carried out at Systems Control Inc. under the supervision of D. Kleinman with MITs Michael Athans as consultant, a linear quadratic Gaussian (LQG) controller for a Trident submarine caused the vessel to unexpectedly surface in nonlinear simulations involving moderately rough seas. In another example, a 1977 journal paper describing the disappointing results of an LQG control design study for the F-8C Crusader aircraft concluded euphemistically with the observation that The study has pinpointed certain theoretical weaknesses . . . as well as the need for using common sense pragmatic techniques to modify the design based on pure theory. A lack of attention to robustness was quickly identied as the dominant factor in these failures. In 1976, the modern eld of robust control theory was born. The term robustness was introduced into the control theory lexicon in papers by E. J. Davison and by M. Safonov and M. Athans. Mathematical control theorists in Michael Athans MIT laboratory began to refocus their attention on methods for analyzing and optimizing the robustness of feedback control systems. Researchers sought to salvage as much as possible of the mathematical theory developed in the 1960s. They worked to link linear optimal control theory to classical root-locus/pole-placement methods. In 1978 researcher Gunter Stein summarized the mood of the time, explaining that his goal was to make modern control theoretical methods work. Researchers were beginning to reexamine classical frequency-response methods and to take a closer look at Horowitzs work on uncertainty-tolerant single-loop feedback design. They were seeking multivariable generalizations of Horowitzs ideas that would mesh well with the sophisticated mathematics of modern optimal control theory. Though attempts at direct generalizations of Horowitzs ideas to more complex multivariable feedback systems with several feedback control loops proved unfruitful, mathematical control researchers quickly discovered that multivariable robustness analysis methods could be developed based on the so-called small-gain theorem of nonlinear stability theory.
ROBUST CONTROL
593
Along with the Lyapunov, ZamesSandberg and other nonlinear stability theories, this now became the core of the emergent mathematical theory of robust control. Prior to 1976, the Lyapunov and ZamesSandberg theories were generally regarded as esoteric tools for nonlinear stability analysis. Despite some suggestive early remarks by stability theory pioneers G. Zames and V. Popov, by 1975, the robustness implications of nonlinear stability theories had yet to be developed. Today, many still regard the mid 1976 change of focus of mathematical control theory from optimal control to robustness as a revolutionary paradigm shift, though, unlike many scientic revolutions, the shift occurred with remarkably little controversy. In 1977 Safonov showed that the general problem of robust stability analysis is equivalent to computing a topological separation of graphs of feedback operators. Lyapunov, conic sector, positivity small-gain and other nonlinear stability theories emerge as special cases. Other researchers, such as J. C. Doyle, G. Stein and N. R. Sandell exploited H. H. Rosenbrocks multivariable version of the Nyquist stability criterion to develop a simplied, and hence more widely accessible, linear explanation of multivariable robustness criteria, singular-value (-plots), and structured singular-value (-plots) frequency-response plots. By 1981, the concept of singular-value, loop shaping and the closely related concept of mixed sensitivity became central to understanding robust control. For a multivariable control system with loop transfer function matrix L(s), design specications were posed in terms of the sensitivity matrix S(s) [I L(s)]1 and the complementary sensitivity matrix T(s) L(s)[I L(s)]1. From these denitions, it follows that there is the fundamental constraint S (s ) + T (s ) = I (1)
variable, feedback control systems. Though each of these methods has the exibility to optimize robustness in a very general setting, their most common use is in conjunction with weighted, mixed-sensitivity, performance criteria where they are used to nd a controller K(s) for a plant G(s) so as to minimize the size of the following transfer function matrix:
W1 (s )S(s ) W2 (s )K (s )S(s ) W3 (s )T (s )
Here W1, W2, W3 are frequency-dependent weights specied by the designer, L(s) G(s)K(s) is the loop transfer function, K(s) is the controller, and G(s) is the plant to be controlled. SOURCES OF UNCERTAINTY Uncertainty is always present to some degree in mathematical models of physical systems. It arises from ignorance of physical details, from approximations, or from unpredictable external effects, such as noise or weather changes. Models derived from idealized physical laws and theoretical principles are uncertain. This includes, for instance, Newtons force law, the second law of thermodynamics, Kirchhoff s current law, Boyles ideal gas law, Maxwells electromagnetic equations, and the law of supply and demand from economic theory. Though some laws are very good predictors of behavior, all such laws are idealizations with fundamental limits of accuracy. Moreover, applications of such laws and principles usually entail assumptions that further degrade the models accuracy of prediction. Approximations produce modeling uncertainty. Linearized models are uncertain, and neglected nonlinear distortion becomes uncertainty. Engineers designing stability augmentation systems for aircraft, robot manipulator arms, or similar mechanical systems employ a rigid-body approximation neglecting exible bending modes, thereby introducing uncertainty into the mathematical models of these systems. Deliberate truncation of modes to simplify a model is called model reduction. Uncertainty arises even when all exible modes are included in models, because the numerical values in the equations for all modes are seldom known with precision. Mathematical models derived from experimental data are also uncertain. Knowledge of the past is uncertain and predictions of the future are always uncertain. The approximate curve-tting processes implicit in most algorithms for system identication mean that experimentally identied models usually can not exactly reproduce even the past data. And even when they do, forecasts of future behavior derived from such models are still never guaranteed to be accurate. STABILITY AND ROBUSTNESS It turns out that the effects of even very small model uncertainties are amplied many times by a feedback system. Thus, it is vital that control engineers quantify the tolerance of their designs to such uncertainties. Because modeling uncertainty is usually a dominant factor in determining how precisely it is possible to control a plants behavior, an engineer wishing to optimize control performance must quantify
whence there is a fundamental tradeoff between sensitivity S(s) and complementary sensitivity T(s). In the mixed-sensitivity framework, the concept of multiplicative uncertainty M became central. Given a true plant G(s) and a nominal plant model G0, a multiplicative uncertainty M is dened by G ( s ) = [I +
M (s )]G0(s )
When knowledge of plant uncertainty is limited to bounds on the size of the multiplicative uncertainty matrix M, good control performance results when the size of S(s) is smaller than unity over a specied control bandwidth, say B, with s j. The singular values of a matrix are a particularly useful measure of its size. To this day in the majority practical robust control designs, performance is specied in terms of S and T. Singular values of S( j) are constrained to be small compared to unity over that desired control bandwidth ( B). And, singular values of T( j) are constrained to be smaller than unity at higher frequencies (beyond B) at which multiplicative uncertainties are expected to be large because of the effects of small unmodeled time delays, phase lags, and other sorts of parasitic dynamics. Optimal H2, H, and -synthesis are very exible robust control theories introduced in the 1980s. They are currently the principal methods employed for designing robust, multi-
594
ROBUST CONTROL
the intrinsic tradeoff between uncertainty tolerance and control precision, that is the intrinsic tradeoff between robustness and performance. Examples of extreme sensitivity to small effects abound in nature. For example, a ball balanced on the top of a hill is unstable. Left undisturbed, it would, in principle, remain forever at rest on the mountain peak. Yet, a disturbance smaller than the apping of a buttery wing causes the ball to plummet. Such extreme sensitivity to small disturbances is called instability. Unstable systems are never robust. On the other hand, a ball at rest in the cusp of a valley is stable and is not dislodged even by a major earthquake. Further, so long as gravity remains an attractive force, then balls at rest in valleys remain stably at rest. Not only is the ball stable, but it remains stable for any large nonnegative variation in the gravitational constant. A ball in a valley is said to possess the property of stability robustness. But even the property of stability itself can be extremely sensitive to small effects. If one had a knob which could reduce the Earths gravitational attraction until gravitational attraction became negative, then balls in valleys would no longer have stability robustness. As the pointer on the knob approached zero, the property of stability itself would be extremely sensitive to the position of the knob. With the knob set a little bit above zero, a ball in a valley would remain stably at rest. But with the pointer on the knob ever so slightly below zero, attraction would become repulsion, and stability robustness would no longer hold. The ball would be launched into space in response to the slightest change in the knobs setting. The property of stability for a ball in a valley is thus not robust to negative variations in the gravitational constant. Of course, although there is little prospect that gravity will reverse itself, there are many situations encountered by engineers in which stability is not robust to variations in physical parameters. Robustness in Classical Single-Loop Control Although the term robustness was not generally used in control theory until the 1970s, robustness has always concerned feedback control system designers. For example, consider the yball governor system invented by James Watt in 1788 to regulate the rotational velocity of a steam engine. A sketch of Watts invention is shown in Fig. 2. The yball governor device is coupled to the main shaft of a steam engine via a pulley. It maintains a set value for the rotational velocity of the shaft by opening the steam valve when velocity decreases and closing it when it increases. A simplied linear model for a steam engine feedback system incorporating a yball governor is given by the following differential equations:
Figure 2. An early example of a feedback control system is the Watt yball governor. The device enhances the robustness of steam plants against engine load variation and steam pressure changes.
corresponding, respectively, to the position of the fulcrum in the mechanical linkage between the yball governor and the steam valve and the physical properties of the steam valve. The equilibrium solution of these equations (obtained by setting all the d / dt and d2 / dt2 terms to zero) is y(t) K /(1 K)r where K KgKs. Hence, if K is much larger than one, then y(t) is very nearly equal to its desired value r, assuming that equilibrium is achieved. A more detailed analysis of the differential equations that model the yball governor shows that they have a time-varying solution of the form y(t ) = K r + c1 et + c2 e t sin(t + ) 1+K
d2 d y(t ) + 2 y(t ) + y(t ) = Ks u(t ) dt 2 dt d Flyball governor: u(t ) + u(t ) = Kg (r(t ) y(t )) dt Steam engine:
Here y(t) denotes the angular velocity of the steam engines output shaft at time t, u(t) indicates that the steam valve position at time t, and r(t) denotes an externally specied desired shaft velocity. The constants Kg and Ks are constants
where 1 K1/3, 1 0.5K1/3, 0.5 3 K1/3, and the three numbers c1, c2, are determined by the initial conditions (d / dt)y(0), y(0), and u(0). When 1 K 8, the timevarying terms c1et and c2et sin(t ) both decay in magnitude to zero as time t increases toward innity, so that equilibrium is approached asymptotically. However, if K 8, then 0, and the et term becomes increasingly large, predicting that the steam engines velocity y(t) alternately increases and decreases with ever growing amplitude. For example, if the steam valve has gain constant Ks 1, then the mathematics predicts that the yball governor system is unstable for governor gain Kg 8. A governor gain of Kg 7 would be ne. But, if the true value of the engine gain Ks were uncertain and known to within only 33%, then one might have Ks 1.33 in which case the model predicts that the yball governor system is unstable for governor gain Kg 8/1.33 6. Whence, a value of governor gain Kg offers insufcient robustness to accommodate a 33% variation in steam engine gain about a nominal value of Ks 1.
Steam
ROBUST CONTROL
595
MULTIVARIABLE ROBUSTNESS In the 1960s and 1970s as increasingly complex control systems with multiple feedback loops were deployed, engineers typically evaluated the robustness of these systems against variations one-loop-at-at-a-time. That is, they examined the effects of varying each feedback while holding all the other gains xed at their nominal values. This proved unwise, because a system may tolerate large, even innite, variations in individual feedback gains, yet may be destabilized by even very small simultaneous variations in several feedback gains. The term multivariable robustness refers to the tolerance of feedback systems to such simultaneous variations. As early as 1971, H. H. Rosenbrock and M. Athans urged greater attention to the issue of simultaneous variations in several feedback gains. And Rosenbrock even developed crude methods for designing robust controllers for such systems, provided that the loops were approximately decoupled, a condition that Rosenbrock called diagonal dominance. But, for many, the telling example that made the case for multivariable robustness theory was the following very simple example produced by J. C. Doyle in 1978. Multivariable Robustness Consider a two-loop multivariable feedback system with nominal loop transfer function
e u 1 s2 + 100 s 100 10(s + 1) 10(s + 1) s 100
1 + 1 1 + 2
Multiplicative uncertainties
Plant G0 (s)
Unstable 1 Stable Unstable
Stable 1 1
1 Unstable
G0 ( s ) =
s 100 1 s2 + 100 10 (s + 1 )
10 (s + 1 ) s 100
Figure 3. Doyles 1978 two-loop feedback example shows that oneloop-at-a-time stability robustness analysis is unwise. In this case, the feedback system remains stable for large variations in either of the two gain perturbations 1 and 2 when the other is xed to be zero, but is unstable for small simultaneous variations.
If one closes either one of the two feedback loops around this system, the open-loop transfer function in the other loop is simply given by g (s ) = 1 s
G ( s ) = G0 ( s ) +
A (s )
whence the system remains stable if the gain in the other loop assumes any nonnegative value. Moreover, this implies that gain margin is innite and the phase margin is 90 in each loop, when evaluated one-loop-at-a-time. The system seems to be extraordinarily robust when examined one-loop-at-a-time. However, this is not the case when gains in both loops are varied simultaneously, as the stability diagram in Fig. 3 indicates. Uncertainty An important aspect of robust control theory is quantifying the difference between two transfer function matrices, say G(s) and G0(s). Typically, G0(s) is a nominal mathematical model, and G(s) is the corresponding true system and the difference is called uncertainty. The uncertainty may be viewed as additive or multiplicative, or it may be regarded in terms of more exotic linear fractional transformation (LFT) or normalized coprime matrix fraction description (MFD) transformations. Useful quantitative measures of size are singular values, H norms, and the gap metric. Additive and Multiplicative Uncertainty. An additive uncertainty is a matrix, say A(s), which produces the true system G(s) when added the nominal model G0, that is
A multiplicative uncertainty is a matrix M(s) such that the true and nominal are obtained by multiplying the model G0(s) by [I M(s)]. Because matrix multiplication is not generally commutative, the right multiplicative uncertainty [say Mr(s)] and left multiplicative uncertainty [say Ml(s)] are generally different, that is, G ( s ) = G 0 ( s )[I +
Mr (s )]
= [I +
Ml (s )]G (s )
does not imply that Mr(s) equals Ml(s). Feedback control systems with additive and multiplicative uncertainties are depicted in Fig. 4. LFT Uncertainty. LFT is an acronym standing for linear fractional transformation. In the case of LFT uncertainty, the true plant is presumed to be related to a perturbational matrix via a relationship of the form G(s ) = P22 + P21 (I P22 ) 1 P12. where (2)
P (s ) =
P11 P21
P12 P22
(3)
is a given transfer function matrix. The LFT transformation Eq. (2) is also sometimes called the Redheffer star product
596
ROBUST CONTROL
True plant G(s) A + + +
where [N0(s), D0(s)] and [N(s), D(s)] are stable normalized coprime left MFDs of G0(s) and G(s), respectively. The gap metric has attracted much interest from theorists because it has a rich mathematical structure with many subtle symmetries and geometric interpretations. Singular Values and H Norms. Given a true plant G(s) and a nominal plant G0(s), it is useful to have a quantitative measure of the size of the difference between the two. In robust control theory, singular values and H norms provide the desired measures of size. An n m matrix, say A, is a mapping of m-vectors x into n-vectors Ax. Likewise, a transfer function matrix, say A(s), is mapping of signal vectors x(s) into signal vectors A(s)x(s). Singular values and H-norms indicate the size or, more precisely, the gain of such mappings. Every complex n m matrix A with rank r has a singularvalue decomposition (SVD) A = U V where U and V are square unitary matrices
K(s)
G 0( s )
True plant G(s) M + K(s) G 0( s ) + +
Figure 4. Inaccuracies in a plant model G(s) can be represented as additive uncertainty A or multiplicative uncertainty M.
and denoted G (s )
=
P
0 0
MFD Uncertainty and the Gap Metric. Every real transfer function matrix G(s) can be decomposed into a ratio G ( s ) = N ( s )D 1 ( s ) where N(s) and D(s) are both stable transfer function matrices. Such a decomposition is called a stable right matrix fraction description (MFD) of G(s). Stable MFDs are not unique. For example, another stable MFD is given by
is an r r diagonal matrix and 0 1 0 0 0 2 =. .. . . .
0 0 . . . r
(s ) N N (s ) = Q (s ) (s ) D (s ) D
(4)
with 1 2 . . . r 0. The positive numbers 1, 2, . . ., r are called the singular values of A. The orthogonal columns u1, . . ., un. v1, . . ., vm of U and V are called the singular vectors. The SVD of A is sometimes also written in terms of the singular vectors as
r
A=
i= 1
i ui v i
where Q(s) is any stable invertible transfer function matrix. A stable right MFD N(s), D(s) of G(s) is said to be coprime if every other stable MFD of G(s) can be written in the form of Eq. (4) for some stable Q(s). A right MFD is said to be normalized if the matrix
The largest singular value of A is also a measure of the vector gain of the matrix A, that is 1 (A ) = max
x
U (s ) =
N (s ) D (s )
Ax x
where x denotes the Euclidean norm x=

2 2 x2 1 + x2 + + xn
has the all-pass property U(s)U(s) I. MFDs play a role in some more exotic quantitative measures of uncertainty used in robust control theory. An example is the gap metric (G, G0) which, roughly speaking, is the sine of the angle between the graphs of G(s) and G0(s). Formally, the gap is computed via solution of a certain H optimization problem, namely
The H norm of a stable transfer function matrix, say P(s), is dened in terms of the largest singular value of its frequency response P
, sup 1 [P( j )]
(G(s ), G0(s )) =
Q (s ) stable
min
N (s ) N0 (s ) Q (s ) D (s ) D0 ( s )
For an unstable transfer function P(s), the H-norm by denition is innite, that is, whenever P(s) has unstable poles P .
ROBUST CONTROL
597
The following are other useful properties of singular values: 1. 2. 3. 4. 5. 6.
1(A B) 1(A) 1(B). 1(AB) 1(A)1(B). 1(A) 1/ n(A1) for A an n n invertible matrix. r Trace(A*A) i1 i2(A). 1(Aij) 1(A) for any submatrix Aij of A.
max{1 (A ), 1 (B )} 1 A B 2 max{1 (A ), 1 (B )} (5)
According to an interpretation of the small-gain theorem known as the performance robustness theorem, robustness of stability for all n uncertainties i 1 (i 1, . . ., n) is equivalent to performance robustness in the absence of the ctitious uncertainty n. The canonical robust control problem is described mathematically as follows. Given a multi-input, multi-output (MIMO) plant transfer function matrix P(s), nd a stabilizing controller F(s) so that the closed-loop transfer function matrix remains stable for all diagonal uncertainty matrices 0 0 0 1 0 0 0 2 = . . .. . . . . .
Canonical Robust Control Problem The canonical robust control problem concerns systems of the general form depicted in Fig. 5. The nth ctitious uncertainty is not an actual uncertain gain. This ctitious uncertainty of size n 1 is wrapped as a purely imaginary loop from a disturbance input d to regulated error output signal e to represent the performance specication that the gain of the transfer function matrix from d to e should be less than one, that is, 1 [Ted ( j )] < 1 for all
Fictitious uncertainty n Uncertainty 1 0 0 n - 1 u1
with 1 and, perhaps, subject to an additional constraint D where D is a specied subset of the set of diagonal matrices with H-norm less than one. Closely related to the canonical robust control problem are the concept of the multivariable stability margin and the structured singular value. Consider the system in Fig. 5. Denote by Ty1u1(s) the closed-loop transfer function from u1 to y1 with the uncertainty removed. Loosely, Ty1u1(s) is the closed-loop transfer function that the diagonal uncertainty sees as it looks back into the rest of the system. Given a stable transfer function matrix Ty1u1(s), the multivariable stability margin Km(Ty1u1) associated with Ty1u1(s), by denition, is the size of the smallest destabilizing D. The structured singular value [Ty1u1(s)], by denition, is the reciprocal of the multivariable stability margin. More formally, for any stable transfer function Ty1u1( j),
1 [Ty 1 u 1 ( j )]
, Km [Ty u , inf 1 ( D
1 1
( j )] ) | det[I Ty 1 u 1 ( j ) ] = 0
0 y1
Sigma Plots and Mu Plots A well-known stability result called the small-gain theorem says that, if the loop gain of a feedback system is less than one, then it is stable. For linear time-invariant systems, the H norm of the system transfer function is the relevant measure of gain. Hence, for the canonical robust control problem, a sufcient condition for closed-loop stability is that Ty1u1 1. Now, the uncertainty has, by hypothesis, 1. Hence, by the small-gain theorem, a sufcient condition closed-loop stability is that Ty1u1 1 or, equivalently, 1 [Ty 1 u 1 ( j )] < 1 for all
Plant P(s)
P11 P21
P12 P22
y2 Controller
u2
K(s)
Figure 5. The canonical robust control problem represents a wide spectrum of uncertain control design problems. It accommodates robust performance requirements via a ctitious uncertainty n.
The problem of choosing the feedback K(s) so that the foregoing H inequality holds is known as the H control problem. To gain a sense of the tolerance of uncertainty at each frequency, control engineers plot singular value Bode plots, such as in Fig. 6. The number 1/ 1[Ty1u1( j)] is interpreted as a lower bound on the size Km( j) of the smallest destabilizing uncertainty ( j). Hence, singular values provide a convenient upper bound on the structured singular value, namely, [Ty 1 u 1 ( j )] 1 [Ty 1 u 1 ( j )] (6)
598
ROBUST CONTROL
matrix
10
4
103
Peak value: 976.93 = 18.89 = 1.08 =
Matlab function: sigma.m (complex full-matrix ) ssv.m with perron option (complex ) ssv.m with muopt (real )
d1 0 D= . . . 0
0 d2 0
0 0 .. . 0
0 0 . . . dn
102
that (T) 1(DTD1), whence, (T )
101
, D diagonal inf 1 (DTD 1 )
100
Usually, the upper bound is a fairly tight bound on the true value of . Computation of via LMIs. In practice, the foregoing optimization over D is usually reduced to an equivalent optimization problem involving a linear matrix inequality (LMI), namely
10-1
100 Hz
101
Figure 6. Singular value Bode plots are widely used to evaluate robustness, but other less-conservative upper-bounds on (Ty1u1) are better when the uncertainty is structured. Shown here are Bode plots of several upper-bounds computed using the Matlab Robust Control Toolbox functions for a particular Ty1u1.
max
X
subject to X T X T 0
The greatest value of for which this optimization is feasible yields (T) , and the optimal diagonal scaling D X1/2. Solutions to LMI optimization problems are computed via standard algorithms. An LMI-based D, G-scaling technique extends the LMI concept to further tighten the upper bound on when some of the uncertainties i are known to be real constants. For example, the Matlab Robust Control Toolbox function muopt.m computes this way. Kharitonov Theorem A system is stable if and only if all its poles have negative real parts. The characteristic polynomial associated with a transfer function Ty1u1(s) is any polynomial whose roots are its poles. One very simple but useful test for robustness allows evaluating the stability of a system whose characteristic polynomial has several uncertain real coefcients. Consider the polynomial p(s) ansn an1sn1 a1 a0. Further, suppose that each of the coefcients ai (i 1, . . ., n) is a real number which varies in the range ai ai ai. The Kharitonov theorem (6) gives necessary and sufcient conditions for all of the roots of the polynomial p(s) to be robustly stable (i.e., have negative real parts) for all values of the coefcients in the specied ranges. The Kharitonov theorem says that robust stability is assured if, and only if, four specic polynomials are stable. The four Kharitonov polynomials are obtained by setting all of the even and all of the odd coefcients to their minimal values and maximal values, respectively. Specically, the four Kharitonov polynomials are
The bound is conservative in the sense that generally the inequality is strict. In the special case in which the uncertainty is unstructured, however, then [Ty1u1( j)] 1]Ty1u1( j)]. An uncertain matrix is said to be unstructured if it is a full (not a diagonal) matrix about which nothing else is known (so the set D is the set of all complex matrices). From the denition of , it follows that a necessary and sufcient condition for the linear time-invariant in Fig. 5 to be stable is given by 1 [ ( j )][Ty 1 u 1( j )] < 1 for all If one denes 1 and as the gains of and Ty1u1, respectively, then this constitutes a sort of nonconservative version of the loop-gain-less-than-one small gain. It is known informally as the small theorem. This interpretation of as a nonconservative replacement for the singular value in applications of the small-gain theorem involving diagonally structured s is in fact the reason why is called the structured singular value. Conservativeness, Scaling, and Multipliers. As noted previously, the singular value 1(T) of a matrix T is generally only a conservative upper bound on the desired quantity (T). And in general this bound can be arbitrarily poor. For example, if
T=
0 0
1000 0
p1 (s ) =
i odd
a i si +
i even
a i si a i si
i even
then one may readily compute from the denitions that 1(T) 1000 and (T) 0. So, the ratio of 1 / can be very large. It can even be innite. Diagonal scaling is one technique for reducing the conservativeness of the bound on provided by singular values. In particular, it can be shown that, for any diagonal scaling
p2 (s ) =
i odd
a i si + ai si +
i odd i even
p3 (s ) = p4 (s ) =
i odd
a i si a i si
i even
ai si +
ROBUST CONTROL
599
For example, if p(s) is a degree three polynomial with each coefcient ai ranging between 3 and 5, then
Further suppose, for simplicity, that the following conditions hold: D11 = 0 D22 = 0 D 12 D12 = I D21 D 21 = I D 12C1 = 0 B 1 D 21 = 0 (11)
p 1 ( s ) = 3s 3 + 3s 2 + 3s + 3 p 2 ( s ) = 3s + 5s + 3s + 5
3 2
p 3 ( s ) = 5s 3 + 3s 2 + 5s + 3 p 4 ( s ) = 5s 3 + 5s 2 + 5s + 5
Robust Controller Synthesis. From the small theorem, it follows that the canonical robust control problems are solved if the controller K(s) stabilizes P(s) and if the resultant closedloop system Ty1u1(s) satises the condition [Ty1u1( j)] 1 for all . Formally, this leads to the -synthesis optimization opt
The foregoing conditions are not very restrictive. Indeed, by a suitable choice variables, these simplifying conditions can be made to hold in nearly all cases [see (21)]. The formula for an H controller K(s) that solves the standard H control problem is most commonly expressed in terms of stabilizing solutions P and Q of the following two algebraic Riccati equations:
0 = PA + A P P(B2 B2 B1 B1 )P + C1C1 and 0 = AQ + QA Q(C2C2 C1C1 )P + B1 B1

or, equivalently, in terms of the stable eigenspaces of two Hamiltonian matrices
, K (s )min sup [Ty u stabilizing
1 1
( j )]
(7)
The previous -synthesis problem Eq. (7) constitutes a formal mathematical formulation of the canonical robust control problem. The canonical robust problem is solved if and only if opt 1. H Synthesis Computing , a nonconvex optimization, is generally difcult. The solution to the -synthesis problem Eq. (7) is generally even harder. Therefore, in practice, engineers often choose to minimize a conservative singular-value upper bound on ; see Eq. (6). This leads to the problem opt or, equivalently, opt
HP
, ,
A C1C1 A B1 B1
B2 B2 B1 B1 A C2C2 C1C1 A
(12)
HQ
(13)
, K (s )min sup 1 [Ty u stabilizing , K (s )min Ty u stabilizing
1 1
( j )]
Provided that there exists a controller K(s) that solves the standard H problem, one such K(s) is given in terms of solutions to P, Q of these two Riccati equations as
(8)
u2 = F (I QP ) 1 x
T C1 )x + B2 u2 + H (y2 C2 x ) x = (A + QC1
1 1
(9)
where F B2 P and H QC2 . H Existence Conditions. The foregoing formula gives a solution to the standard H control problem Eq. (10), provided that a solution exists. In general, no solution may exist, so it is important to have precise and numerically testable mathematical existence conditions. The following are the necessary and sufcient conditions for the existence of a solution: 1. Rank(D12) dim(u2) and rank(D21) dim(y2). 2. 1((D12 )D11) 1 and 1(D11(D2 1)) 1 where (D) denotes an orthogonal matrix whose column span is null(D). 3. Neither of the Hamiltonians Eqs. (1213) has any purely imaginary eigenvalues; that is, Imag[i(HP)] 0 i and Imag[i(HQ)] 0 i. (This is necessary and sufcient for the two Riccati equations to have stabilizing solutions that is, solutions such that A (B1B1 B2B2 )P and A Q(C1 C1 C2 C2) have only eigenvalues with negative real parts.) 4. Both A B2F and A HC2 are stable (i.e., both have all eigenvalues with strictly negative real parts). (Note: In theory, this condition holds if and only if the Riccati solutions P and Q are both positive-semidenite matrices, but in practice semideniteness cannot be checked reliably with a nite precision computation.)
This is the H optimal control problem. It is closely related to the standard H control problem which is to nd, if it exists, a controller K(s) for the system in Fig. 5 such that Ty 1 u 1
<1
(10)
Optimal H controllers possess the property that their closedloop singular-value frequency-response is completely at, that is opt = Ty 1 u 1
= 1 [Ty 1 u 1 ( j )] for all
This atness of optimal H controllers is called the all-pass property, analogous to the at response of all-pass lters that arise in circuit theory and signal processing. H Control Riccati Equations. Suppose that the plant P(s) in Fig. 5 is given in state-space form as
x A y1 = C1 y2 C2
B1 D11 D21
B2 x D12 u1 D22 u2
600
ROBUST CONTROL
5. All of the eigenvalues of PQ have moduli less than one: |i (PQ )| < 1 for all i.
Mixed Sensitivity and Loop Shaping There is one H robust control problem of considerable practical signicance, the weighted mixed-sensitivity problem in which
Implicit in condition 4 above is the requirement that the plant be both stabilizable and detectable, that is, the pair (A, B2) must be stabilizable and the pair (C2, A) must be detectable. If this is not so, then condition 4 cannot hold.
Ty 1 u 1
W1 (s )S(s ) = W2 (s )K (s )S(s ) W3 (s )T (s )
-Iteration. The solution to the H optimal control problem Eq. (9) is computed via successive solutions to the standard H problem Eq. (10) via a technique called -iteration. The basic idea is to scale the plant matrix P(s) in Eq. (3) by multiplying its rst row by a nonnegative scalar and then to test the H existence conditions for various values of . The greatest value of , for which all ve of the H existence conditions hold, yields the minimal value of the H cost opt as opt 1/ . The H optimal controller is the solution to the corresponding standard H problem. -Synthesis
The problem of -synthesis involves computing a controller K(s) that solves optimization Eq. (7). No such algorithm exists, but the combination of H optimal control together with diagonal scaling is employed to produce good suboptimal solutions. The algorithm is called the DK-iteration, and it proceeds as follows: Input: The plant P(s) shown in Fig. 5. Initialize: Set D(s) I, K(s) 0, Ty1u1(s) P11(s). Step 1. Replace the plant by the diagonally scaled plant
Figure 7 shows how a conventional control system is augmented with weights W1, W2, W3 to produce an augmented plant P(s) such that the closed-loop Ty1u1 has the indicated form, namely,
W1 W1 G 0 W2 P (s ) = 0 W3 G G I
Let us suppose (as is often the case in applications of the theory) that the control-signal weight W2 is absent, so that Ty1u1 reduces to
Ty 1 u 1 =
W1 (s )S(s ) W3 (s )T (s )
The all-pass property of H optimal controllers ensures that opt = Ty 1 u 1
= 1 [Ty1 u1 ( j )] for all .
P (s )
D 0
0 I
P11 P21
P12 P22
D 0
0 I
(14)
From this, combined with the singular-value property Eq. (5) and the fundamental constraint S(s) T(s) I, it follows that, to within a factor of 2, opt max{1 [W1 ( j )S( j )], 1 [W3 ( j )T ( j )]}. Whence, it follows from the properties of singular values that, at frequencies where 1[S( j)] 1, i [L( j )] n [L( j )] W1 ( j ) 1 1 [S( j )] opt
Step 2. Compute the solution K(s) to the H optimization problem Eq. (9) for the diagonally scaled plant Eq. (14) using the -iterative technique, and set Ty 1 u 1 = P11 + P12 K (I P11 K ) 2 P21 Step 3. At each frequency , compute the diagonal scaling D( j) that minimizes 1(DTy1u1D1). Step 4. Using a curve-tting method, nd a polynomial transfer function D(s) that approximates the frequency response D( j) computed in Step 3. Step 5. Go to Step 1. There are no theoretical guarantees that this algorithm converges to a joint minimum in K(s) and D(s). In fact, it may not even nd a local minimum. But, at least it improves the controller, which is enough for it to be useful in some engineering applications. Renements on this algorithm have recently appeared in which D, G-iteration handles real uncertainties with less conservatism. Also, a technique for bypassing the curve tting in Step 4 is available. The optimal D(s) is a prespecied order computed via an LMI-related technique [see (22)].
and, at other frequencies where 1(T( j)) 1, i [L( j )] 1 [L( j )] 1 [T ( j )] opt W3 ( j )
where L(s) G(s)K(s) is the loop transfer function matrix of the control system in Fig. 7. The situation is depicted in Fig. 8. The key feature to notice is that, inside the control loop bandwidth, the singular values i[L( j)] of the loop transfer function are bounded below by W1( j)/ opt 1 and, outside the control bandwidth, the singular values are bounded above by opt / W3( j) 1. All of the loop transfer function singular values cross over in the intermediate frequency range, as depicted in Fig. 7. The implication is that the shapes of the Bode magnitude plots of W1( j) and 1/ W3( j) specify with considerable precision the actual shapes of the optimal singular-value plots of
ROBUST CONTROL
601
Augmented plant P (s)
W 1( s )
y 1a
W 2( s )
y 1b
y1
+ u1
u G(s)
y W 3( s ) y 1c
u2
y2 Controller K(s)
Figure 7. In mixed-sensitivity H robust control design, the plant model is augmented with weighting lters W1(s), W2(s) and W3(s) that determine the shape of the resulting closed-loop sensitivity and complementary sensitivity Bode plots.
the optimal loop-transfer function L( j) G( j)K( j). Thus, the weights W1(s) and W3(s) are high-precision knobs for shaping the loop-transfer functions singular values. The H theory automatically ensures that the controller K(s) is stabilizing. Because of this, the weighted mixed-sensitivity H loop shaping is a highly popular and useful method for multiloop control design. FURTHER READING History. For pre-1900 control history, including an account of the invention of the Watt yball governor, see Ref. 1. For an accurate account of the early developments in robust control theory, see Ref. 3 and the monograph Ref. 2. Reprints of key journal papers are provided in the anthology (5). Reference 4 is an extensive bibliography on robust control.
Textbooks. Reference 6 describes the early 1960s work of Horowitz on uncertainty tolerant design for single-loop feedback control systems. References 7 and 8 provide a good discussion of the most useful techniques of robust control theory along with numerous design examples. More theoretical treatments are in the texts (9), (11), and (12). Adaptive robust control theory is treated in Ref. 10. A more specialized focus on robustness analysis methods related to the Kharitonov theory is in (13). Software. Software for robustness analysis, H control design, -synthesis, and related topics is provided by the Matlab toolboxes (14), (15), and (16). The users guides accompanying these software products all contain extensive tutorial texts covering the theory, its use, and numerous design examples. Advanced Topics. The robustness implications of the gap metric is examined in Ref. 19. Recently LMI-based methods and have become increasingly important in robust control theory, leading to signicant reductions in conservatism and extensions to difcult multi-objective problems, simultaneous stabilization, and gain scheduling for slowly varying plants; see Refs. 17 and 18. A key issue in applying robust control theory is the question of how evolving experimental data are incorporated to identify uncertainty sizes and to adaptively enhance robustness. The latter is the focus of unfalsied control theory; see Ref. 20.
dB
1(L) 1(T)
Performance bound W1 0 1
1 W 3 Robustness bound
1(S) n(L)
BIBLIOGRAPHY
1. O. Mayr, The Origins of Feedback Control, Cambridge, MA: MIT Press, 1970. 2. M. G. Safonov, Stability and Robustness of Multivariable Feedback Systems, Cambridge, MA: MIT Press, 1980. Based on the authors PhD Thesis, Robustness and stability aspects of stochastic multivariable feedback system design, MIT, 1977.
Figure 8. The mixed-sensitivity weights W1(s) and W3(s) provide engineers with the means to directly specify the desired shape of the compensated loop transfer function L(s). The Bode plot of L(s) is sandwiched between the plot of W1(s) and the plot of 1/ W3(s). Loop-shaping is achieved by manipulating the weights W1(s) and W3(s).
602
ROBUST CONTROL ANALYSIS
3. M. G. Safonov and M. K. H. Fan, Editorial, Special issue on multivariable stability margin, Int. J. Robust Nonlinear Control, 7: 97103, 1997. 4. P. Dorato, R. Tempo, and G. Muscato, Bibliography on robust control, Automatica, 29: 201213, 1993. 5. P. Dorato, ed., Robust Control, New York: IEEE Press, 1987. 6. I. Horowitz, Synthesis of Feedback Systems, New York: Academic Press, 1963. 7. J. M. Maciejowski, Multivariable Feedback Design, Reading, MA: AddisonWesley, 1989. 8. S. Skogestad and I. Postlethwaite, Multivariable Feedback Control, New York: Wiley 1996. 9. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Englewood Cliffs, NJ: Prentice-Hall, 1996. 10. P. Ioannou and J. Sun, Stable and Robust Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1994. 11. M. Green and D. J. N. Limebeer, Linear Robust Control, Englewood Cliffs, NJ: Prentice-Hall, 1995. 12. T. Basar and P. Bernhard, H-Optimal Control and Related Minimax Design Problems A Dynamical Game Approach, Boston, MA: Birkhau ser, 1995. 13. B. R. Barmish, New Tools for Robustness in Linear Systems, New York: Macmillan, 1994. 14. R. Y. Chiang and M. G. Safonov, Robust Control Toolbox, Mathworks, Natick, MA, 1988 (Ver. 2.0, 1992). 15. G. J. Balas et al., -Analysis and Synthesis Toolbox (-Tools), Mathworks, Natick, MA, 1991. 16. P. Gahinet et al., LMI Control Toolbox, Mathworks, Natick, MA, 1995. 17. K. C. Goh, M. G. Safonov, and J. H. Ly, Robust synthesis via bilinear matrix inequalities, Int. J. Robust Nonlinear Control, 6: 10791095, 1996. 18. S. Boyd et al., Linear Matrix Inequalities in Systems and Control Theory, Philadelphia, PA: SIAM, 1994. 19. T. T. Georgiou and M. C. Smith, Optimal robustness in the gap metric, IEEE Trans. Autom. Control, AC-35: 673686, 1990. 20. M. G. Safonov and T. C. Tsao, The unfalsied control concept and learning, IEEE Trans. Autom. Control, 42: 843847, 1997. 21. M. G. Safonov, D. J. N. Limebeer, and R. Y. Chiang, Simplifying the H theory via loop-shifting, matrix-pencil and descriptor concepts, Int. J. Control, 50 (6): 24672488, 1989. 22. M. G. Safonov and R. Y. Chiang, Real/complex Km-synthesis without curve tting, in C. Leondes, (ed.), Control and Dynamic Systems, New York: Academic Press, 1993. Vol. 56 (Part 2), pp. 303324.
Robust control can be generally dened as the control, by means of xed compensators, of uncertain plants (i.e., of systems with uncertain dynamics and unknown disturbance signals). Robustness in a control system refers to its ability to cope with uncertainties in a satisfactory manner, maintaining its stability and performance as desired. Uncertainty in signals and systems is inevitable, reecting both the complexity of the physical world and the limitation in human understanding. Uncertain signals typically arise as a result of the randomness and unpredictability of environmental effects and are of an unmeasurable and unpredictable nature. Uncertain system dynamics, on the other hand, can be attributed to changes in the actual system and to modeling errors, be they accidental or deliberate. Generally, uncertain dynamics may come from the following sources: 1. Imperfect or incomplete knowledge of physical processes. This represents the information unattainable because of ones limited knowledge or inadequate measurements. It can be particularly acute for complex systems and processes (e.g., those found in biomedical engineering). 2. Parameter variations. Every physical system will undergo a change in parameter values under different operating conditions. Aging itself can be a factor. 3. Neglected high-frequency dynamics, time delays, nonlinearities, and the like. It may occur as a result of a sheer lack of knowledge, or the difculty to model these characteristics. It may also occur because of the desire for models of low complexity. While robustness is a concept of universal signicance, robustness analysis for control systems is the study of whether a system, however designed, can meet specied stability and performance goals in the presence of uncertainty within a prescribed range. Uncertain plants can be modeled in various ways. In particular, models can be stochastic or purely deterministic. In robust control, uncertain systems are typically modeled deterministically, as bounded sets of system models. A property is then said to hold robustly if it holds for every model in the set. The simplest case is that of unstructured uncertainty: the model set consists of all systems in a certain neighborhood (e.g., all transfer functions lying within a certain distance of a distinguished nominal system). Such a description is particularly appropriate to account for unmodeled dynamics. One rather typical example is the modeling of exible structures. It is well known that, in general, a exible structure cannot be accurately represented by a nite-dimensional system. For control design purposes, however, we desire, and most often are compelled to nd, an approximate nite-dimensional model with a relatively low order. In doing so, a common practice is to include in the nominal model a small number of dominant modes in the low-frequency range and to treat the high-frequency modes as modeling uncertainty. Evidently, this description is also appropriate for modeling errors resulting from model reduction, or from any frequency response truncation. Moreover, it can be used to cover, albeit in a conservative way, parameter variations. The latter amounts to drawing up a frequency response envelope to describe the range of parameter variation in frequency domain.
MICHAEL G. SAFONOV
University of Southern California

Robustness is a property inherently sought after in engineering systems. The concept is directly linked to such issues as design viability and system reliability. In broad terms, robustness can be regarded as the capability to withstand unknown, unexpected, and often hostile conditions that can adversely affect a systems behavior. A system must be sufciently robust in order to function properly under undesirable circumstances, conducting its task as designed. As engineering systems are becoming more and more complex and are required to operate in increasingly more uncertain environments, robustness has become increasingly more crucial.
603
Finally, in robustness analysis, it is common to introduce a ctitious uncertainty to represent a performance objective. A more accurate account of parametric uncertainty calls for model sets within which individual models are uniquely characterized by the value(s) of one or more parameter(s) (e.g., transfer function coefcients). Typically each such parameter takes values in a known range. Such models account for the fact that parameters in physical devices are bound to vary with time, environment, and operating conditions. Ackermanns car steering problem (1) offers a good illustrative example. In his study of a four-wheel steering vehicle, he found that the vehicle mass and the adhesion between tires and road surface are signicant uncertain parameters. This is easily understandable. The vehicle mass certainly varies with load, and in a more subtle sense, it varies with fuel consumption. The adhesion changes with road condition, wearing of tires, and weather condition. We can think of other uncertain parameters by considering the human-vehicle system as a whole or by considering a whole batch of vehicles as a family of systems. In these scenarios, differences among individual drivers and vehicles can all constitute signicant uncertain factors. Yet a more striking example is an aircraft, whose aerodynamic coefcients vary in large magnitudes due to changes in altitude, maneuvering, and weather. There is an inherent trade-off between delity and simplicity in modeling uncertain systems. In a sense, unstructured and parametric uncertainties may be considered the two extremes. While an unstructured perturbation furnishes a simple characterization and is useful for simplifying robustness analysis, it contains little information and may often be too conservative. On the other hand, uncertain parameters often yield a more natural and accurate model, but such elaborate descriptions tend to complicate analysis. The process of robustness analysis, therefore, calls for a judicious balance between uncertainty description and complexity of analysis. In its full generality, however, the description of an uncertain system should take into account both parametric variations and unmodeled dynamics. Uncertainty descriptions of this kind are called structured. To be sure, robustness is not entirely a new concept in control system analysis and design. In retrospect, the need for a control system to tolerate unmodeled dynamics and parameter variations is precisely what motivated feedback control, and it has been a primary goal in control system design since its birth. This is well recognized in classical control design and, at least implicitly, is embedded in classical loop shaping methods. Concepts such as gain and phase margins may well be regarded as elementary robustness measures. However, it was not until the late 1970s that the term began to appear routinely in the control literature, when the need for robustness was reexamined and was gaining increasing recognition. Robust control as a research direction soon thrived and became a dening theme. After two decades of intense activity, it has evolved into a broad research area rich in theory and potential applications. The progress has been rapid and vast, leading to the development of a variety of key concepts and techniques, among which notably are the H / theory, the Kharitonov/polynomial approach, and analyses based on state-space formulations and the Lyapunov theory. The structured singular value (2), also known as , was introduced in the early 1980s as a very general framework for studying structured uncertainty in linear time-invariant
models. It denes an exact measure of robust stability and performance in the frequency domain. The method is a natural progression of earlier work on using singular values to extend the concept of stability margin to multivariable systems, an idea that was heavily inuenced by operator theoretic results such as the Small Gain Theorem. The main impetus for the development of the structured singular value theory, evidently, has been the recognition that unstructured uncertainty is too crude a model, often leading to excessive conservatism. From a computational point of view, the success of hinges critically upon whether and how it can be computed both accurately and efciently. Unfortunately, this is known to be difcult. Recent studies have shown that computation of generally amounts to solving a so-called NP-hard decision problem, which, in the language of computing theory, is one that suffers from an exponential growth in its computational complexity. Although this by no means implies that every problem is computationally difcult, it nevertheless points to the unfortunate conclusion that the computation of in general poses an insurmountable difculty in the worst case. In retrospect, it thus comes as no surprise that the major progress in computing has been made by some forms of approximation, specically, readily computable bounds. While no denitive conclusion has been drawn concerning the gap between and such bounds, it is reassuring that the bounds are often reasonably tight and that they have other interpretations of engineering signicance. The Kharitonov theory is another robustness analysis approach, developed in parallel with , which deals almost entirely with robust stability issues under parametric uncertainty; only in rare cases is unstructured uncertainty also taken into consideration. The research in this area has a natural heritage from such a classical stability test as the RouthHurwitz criterion and was mainly triggered by a landmark result published by V. L. Kharitonov (3) in 1978, later referred to as Kharitonovs theorem. Kharitonov considered the question of stability of parameterized families of polynomials. Here polynomials are thought of as characteristic polynomials of systems described by rational transfer functions and thus are stable if their zeros all lie in the open left-half of the complex plane (continuous time) or in the open unit disk (discrete time). In the continuous-time case, Kharitonov showed that, for an uncertain interval polynomial whose coefcients each vary independently in a given interval, stability of the entire family can be assessed by testing merely four simply constructed extreme polynomials. From an aesthetic point of view, Kharitonovs theorem possesses remarkable elegance, reducing an otherwise seemingly impossible task to a simple problem. From an engineering perspective, however, the theorem is likely to nd only limited utility because very rarely would an uncertain system yield a family of characteristic polynomials of the form required in the problem description. Thus, Kharitonovs work triggered a urry of activities in the search for more realistic results, and soon came various generalizations. Two notable features stand out from the robust stability conditions available in this category. First, they are stated either in terms of a nite number of polynomials or as graphical tests requiring a frequency sweep. Second, the main success to date pertains to uncertainty descriptions in which polynomial coefcients depend linearly on uncertain parameters.
604
Robust stability analysis in state-space formulation often comes under the names of stability radius, interval matrix, and stability bound problems. The key issue here is to determine the largest uncertainty size under which stability is preserved. Unlike in the aforementioned two approaches, parametric uncertainty in state-space representation is dened in terms of perturbations to system matrices, which can be either unstructured or structured, but most often are only allowed to be real. In a natural way, the robust stability problem translates to one of how the perturbations may affect the eigenvalues of the system matrix, the solution of which can draw upon rich theories from linear algebra and Lyapunov analysis. Thus, unsurprisingly, most of the robustness conditions have this avor. Alternatively, the problem can also be recast as one of analysis or one of polynomial stability. In the former case, we need to compute with respect to solely real uncertainties, for which the computation schemes are known to be conservative. In the latter case, the coefcients in the resultant polynomial will depend on the unknown parameters in a multilinear or multinomial fashion. The Kharitonov approach cannot provide a satisfactory answer for such polynomial families either. The problem thus appears to be a very difcult one and is known to have been solved only for a number of isolated cases. Most notably, recent progress has made it possible to compute the stability radius in an efcient manner when the matrix perturbation is unstructured. For structured uncertainty, unfortunately, the problem is completely open; recent studies showed too that it is in general NP-hard. Accordingly, the majority of results in the latter class are sufcient conditions for robust stability. In conclusion, robustness is a key concept, vital for engineering design in general and for control system design in particular. Robust control has matured into a eld rich in theory and potential applications. By focusing on these three selected areas, this chapter is limited only to robustness analysis of linear time-invariant systems, that is, control of linear time-invariant plants by linear time-invariant controllers, which itself is condensed from a vast collection of results and techniques. Nevertheless, we should note that the concept and theory of robust control goes far beyond the boundary of linear time-invariant systems and, in fact, has been quickly branching to the domains of nonlinear control, adaptive control, and the like. As a whole, it has been, and will continue to be, a driving force behind the evolution of control theory.
Thus each plant in the model set corresponds to the selection of one element in each of the uncertain blocks. The nominal plant corresponds to the choice 0 for all elementary uncertain blocks. Note that the assumption that all the uncertainty balls have unity radius is made at no cost: any size information can be embedded into known blocks (e.g., connected in cascade with the uncertain blocks). It turns out that, for uncertain block diagrams of this type, the transfer function (or transfer function matrix) between any two nodes (or tuples of nodes) in the block diagram is given by a linear fractional transformation (LFT). Similarly, when such an uncertain plant is connected with a feedback compensator, the transfer function between any two nodes will also be an LFT. LFTs have attractive mathematical properties that can be used to advantage at the modeling, analysis, and synthesis stages. More on LFTs can be found, e.g., in Ref. 7. Robust Stability It should be intuitively clear that block diagrams of the type just described can always be redrawn in the form of an M loop as depicted in Fig. 1 (external input and outputs have been left out). Here M corresponds to the nominal system, which is comprised of closed-loop transfer functions as elements and has an input channel and an output channel for each elementary uncertain block, and is a block-diagonal matrix whose diagonal blocks are the elementary uncertain blocks. For its generality, the M loop paradigm has found wide acceptance in robust control (see, for example, Refs. 2 and 48). Throughout most of this article, we will assume that the nominal system, or equivalently M, is linear and time-invariant, as are all instances of the uncertainty blocks, equivalently, of . We will also assume that M and all instances of are in H . In this case, an immediate payoff of the LFT uncertainty description and the ensuing representation of the system via the M loop is the following strong form of the Small Gain Theorem, a necessary and sufcient condition for well-posedness and stability of the M loop, in the case where consists of a single uncertainty block, ranging over the unit ball in H . Small Gain Theorem. Let M H . Then the M loop is well-posed and BIBO stable for all H with 1 if and only if M 1. As alluded to earlier, the H norm of a causal, stable, continuous-time transfer function matrix M is dened as M
THE STRUCTURED SINGULAR VALUE Uncertain Systems Throughout this article, we consider model sets P with the following property: P can be represented by a block diagram with some of the blocks being fully known systems, and others being elementary uncertain blocks. The latter are elementary sets, namely, unit balls in simple vector spaces. For example, some uncertainty blocks might be the real interval [1, 1] and others might be the unit ball of H , the set of transfer function matrices (linear time-invariant systems) that are causal and bounded-input bounded-output (BIBO) stable; the size or H -norm of H is dened to be the supremum of its largest singular value over the imaginary axis (continuous time) or unit disk (discrete time).
= sup (M ( j))
where denotes the largest singular value.
Figure 1. M loop.
605
As an example consider a model set of the output multiplicative uncertainty type. Specically, let P = (I + w )P0 where P0 is the transfer matrix of a linear, time-invariant nominal model, w is a scalar weighting transfer function, and ranges over the unit ball in H . The weight w is introduced to account for the fact that the amount of uncertainty is usually frequency-dependent; in particular, system dynamics are often poorly known at high frequencies. Suppose that a feedback controller K has been tentatively selected to stabilize the nominal system P0. (We use the negative feedback convention, i.e., the loop transfer function is KP0.) Isolating from the nominal closed-loop system, we obtain an M loop with M = wKP0 (I + KP0 )
1
namics, consider now the model set (1 w1c)Pa with Pa explicitly given as Pa (s ) = 1 sa
where a can take any value in the interval [0.9,1.1]. We may write a 1 w2r with w2 0.1 and r 1, and Pa can be represented as a loop with P0(s) 1/(s 1) in the forward path and w2r in the feedback path (again using the negative feedback convention). Let K be a feedback controller (still with the negative feedback convention) that stabilizes P0. By extracting the uncertainty blocks c and r, we obtain an M loop with
M= =
w2 P0 (I + P0 K ) 1 w1 P0 (I + P0 K ) 1
r
w2 KP0 (I + P0 K ) 1 w1 KP0 (I + P0 K ) 1
Since K stabilizes P0, M is stable. As a note of interest, we expect a keen connection between the Small Gain Theorem and the classical Nyquist criterion. Indeed, this can be best observed by examining single-input/ single-output systems. In such case P and K are scalar, and thus so is M. Since both M and are stable, Nyquists criterion implies that the M loop is stable whenever the Nyquist plot of M does not encircle the critical point 1 j0. Clearly, this will be the case for every satisfying ( jw) 1 for all w if and only if M( jw) 1 holds at all frequencies (including ). The Small Gain Theorem suggests that one way to obtain a robustly stable system, or more generally to obtain a robust design, is to make sure that the H norm of a certain system transfer function is small enough. This has triggered an entire eld of research known as H design, which is discussed elsewhere in this encyclopedia. The focus of the present article is the case when is block-diagonal (i.e., when the uncertainty model consists of several blocks or, in other words, when the uncertainty is structured). Typically, two types of uncertainty blocks are considered in the literature: (i) the set of real, constant scalar multiples of the identity, with the scalar having magnitude no larger than one, and (ii) the set of causal and BIBO stable (H ) transfer function matrices, with H -norm no larger than one. The latter corresponds to unmodeled dynamics. The former, on the other hand, is used to represent parametric uncertainty, particularly when a same uncertain parameter affects more than one coefcients in a transfer function. For example, concurrent variation as a function of temperature (e.g., dilation) of multiple quantities in a mechanical system can result in such a block. This description is more general than the simpler scalar nonrepeated blocks. Examples with structured uncertainty arise with plants modeled as being affected by uncertainty at more than one physical location, e.g., P = ( I + w1
1 )P0 (I
0
c
where r is a real number ranging over [1,1] and c is a scalar transfer function ranging over the unit ball in H . Clearly, the condition M 1 in the Small Gain Theorem remains sufcient for robust stability when is restricted to be block-diagonal. However, it is in general no longer necessary. A renement of the Small Gain Theorem for the structured uncertainty case was proposed by Doyle and Safonov in the early 1980s (2,4). We adopt here the framework introduced by Doyle, that of the structured singular value also known as . The Small Theorem states that, if the uncertainty is restricted to be block-diagonal, then the correct renement is essentially (see Ref. 9 for a precise statement) to replace M with M, where for a continuous-time transfer function matrix M, M
= sup (M ( j))
e
and ( ) denotes the structured singular value of its matrix argument with respect to the block-structure under consideration. The set e is the extended real line; if no parametric uncertainty is present, however, ( ) is continuous and e can be replaced by . Similarly, for a discrete-time system, M
= sup (M (e j ))
[0 , 2 )
But what specically is ? This is discussed next. The Structured Singular Value Let us denote by the set of values taken by ( j) [or (ej)] as ranges over the set of block diagonal transfer function matrices of interest, with the unit ball restriction lifted, namely let = {diag(
r
+ w2
2)
where both input and output uncertainty are accounted for. Another instance arises in the context of the robust performance problem, discussed in a later section. For an example including both parametric uncertainty and unmodeled dy-
):
r,
c,
C}
606
with
r c C r r := {diag(1 Ik , . . ., m I ) : ir R} r km
1 r
tightened to the following (12):
max R (MQ ) (M )
C}
QQ
:=
c c {diag(1 Ik , . . ., m I ) c km r +mc mr +1 C 1 , . . ., C mC )
ic
DD + , GG
inf
{ 0 : MDM H + GM H MG 2 D < 0}
(2)
:= {diag(
c i
k m + m + i k m + m + i r c r c }
The rst and third block types, often referred to as repeated real and full complex blocks, correspond to values of parametric and dynamic uncertainty, respectively. The second block type, known as repeated complex, often arises in analyzing multidimensional (10) and time-delay systems (11), and is also used sometimes when an LFT state-space representation of transfer functions is sought (5). It is worth noting that while as just dened is usually adequate for representing uncertainties frequently encountered, it can be extended further to accommodate more general situations. For example, full real blocks (i.e., unknown real matrices) may be added whenever desired. The structured singular value (M) of a matrix M with respect to the block structure is dened to be 0 if there is no such that det(I M) 0, and (M ) = min{ ( ) : det(I M ) = 0}
Here the superscript H indicates complex conjugate transpose, Q is the subset of consisting of matrices whose complex blocks are unitary, D is the subset of D consisting of Hermitian positive denite matrices, G is the subset of D consisting of skew-Hermitian matrices (i.e., GH G) with zero blocks in correspondence with repeated real blocks in , and the sign indicates that the matrix expression is constrained to be negative denite. The lower bound in condition (2) is always equal to (M). The upper bound is never greater than that in condition (1) (it reduces to it when G 0 is imposed) and, as was the case for condition (1), can be computed by solving an LMI problem. See, for example, Section 8.12 in Ref. 8. For the class of problems where the matrix M has rank one, Young (13) showed that the right-hand side in inequalities [Eq. (2)] is equal to (M). In that case, Chen et al. (14) obtained an explicit formula for (M). Let M baH, where a and b are column vectors. Let also a and b be partitioned into subvectors ai and bi compatibly with . For i 1, . . ., mr mc, let i aiHbi. Moreover, dene
mr +mc
otherwise. It can be checked that for structures simply consisting of one full complex block as in the Small Gain Theorem, (M(j) becomes the largest singular value of M(j), and M is thus equal to M. Given a matrix M and a block structure , computing (M) is generally not an easy task. Indeed, this computation is known to be NP-hard, even when is simplied to a structure containing only full complex blocks. Thus estimates of (M), e.g., upper and lower bounds on (M), are often used instead. These, as well as other properties of , are discussed next. Let U be the set of unitary matrices in and D be the set of nonsingular matrices that commute with every . The latter consist of block-diagonal matrices with scalar multiples of the identity in correspondence with full complex blocks (C), and with arbitrary blocks in correspondence with those constrained to be scalar multiples of the identity (r, c). Then the following result holds: max R (MU ) (M ) inf (DMD 1 )
U U DD
=
i= m r + 1
|i | +
mr +mc +mC
ai
i= m r + m c + 1
bi
Then,
(M ) = inf
x
mr i= 1
|Re(i ) + x Im (i )| +
!
2
1+x
(3)
Furthermore, if we assume with no loss of generality that for some l mr, Im(i) 0 for 1 i l, and Im(i) 0 for i l, and that Re(l ) Re(1 ) Im (1 ) Im (l )
then the inmum is achieved at one of the following points:
(1)
xk = x0 = xk =
Re(k ) , Im (k ) 2
k = 1, . . ., l |Im (i )|
l i= 1
Here R is the largest absolute value of a real eigenvalue of its matrix argument. Inequalities [see Eq. (1)] are of special interest in the case of purely complex uncertainty structures. In that case, (i) the lower bound is equal to (M) and R can be replaced by the spectral radius , and (ii) the upper bound is equal to (M) whenever mC 2mc is no greater than 3, and extensive numerical experimentation suggests that it is never (or at least seldom) much larger. Moreover, the upper bound can be computed efciently by solving a convex optimization problem, in fact, a linear matrix inequality (LMI) problem. LMIs dene a special class of convex optimization problems and are discussed elsewhere in this encyclopedia. For uncertainty structures where real (scalar multiple of the identity) blocks are present, inequalities [Eq. (1)] can be
; ;
l i= 1
|Im (i )|
2 l i = k + 1 |Im (i )| 2 l i = k + 1 |Im (i )|
k i= 1
|Im (i )|
k i= 1
|Im (i )|
Finally, the inmum cannot be achieved at x0 unless x0 (, x1], and for k 1, . . ., l, it cannot be achieved at xk unless xk [xk, xk1]. The rank-one case just alluded to is one of the rare instances for which one can obtain an explicit expression for (M). This expression not only simplies the computation of the upper bound in condition (2) but also was found useful in studying robust stability of uncertain polynomials. Indeed, as
607
1 2 M
1 2
blocks p(s) such that p 1. It thus follows that robust performance holds if and only if
e
sup (M ( j)) < 1
Figure 2. Robust performance setup.
will be discovered shortly, an important class of stability problems for uncertain polynomials can be formulated in terms of a rank-one problem. Consequently, the result furnishes a unifying tool for the stability problems and a link between analysis and the Kharitonov approach to robustness analysis. Robust Performance A key reason for the popularity of the framework is that it encompasses not only the robust stability problem but also the following robust performance problem: determine whether, for all plants in the given model set, the energy (integral of the square of the magnitude) in a specied error output signal remains below a specied threshold whenever the disturbance inputs energy is less than a specied value. Consider the block diagram of Fig. 2 where, as compared to Fig. 1, external (disturbance) input and (error) output are made explicit. Given a block diagram such as the one of Fig. 2, the input-output transfer function in the continuous-time case is given by the linear-fractional transformation F (M (s ), (s )) = M11 (s ) + M12 (s ) (s )(I M22 (s ) (s )) 1M21 (s ) where Mij(s) is the transfer function from input j to output i of M(s), i,j 1,2, when the feedback connection through (s) is removed. [Thus M22(s) is the transfer function matrix formerly denoted M(s).] The issue at hand is to determine, under the assumption that M(s) H , whether robust performance holds, that is, whether it is the case that, for all (s) in our unit uncertainty ball, F (M (s ), (s ))
where now denotes the structured singular value corresponding to the augmented block structure diag(kpkp, ) (i.e., the block-structure corresponding to the actual uncertainty, augmented with a full complex block). For an example of a typical robust performance problem, consider an uncertain plant described by the multiplicative uncertainty model set P = (I + w1 )P0 with a xed feedback controller K. It is desired to determine whether w2S 1 for all possible in a possibly structured unit uncertainty ball, where S is the sensitivity function (i.e., using the negative feedback convention, S (I PK)1). Here w1 and w2 are stable transfer functions introduced for frequency-weighting purposes. For simplicity, w1 and w2 are assumed to be scalars. Using the transformation just outlined, we obtain
M=
w2 (I + P0 K ) 1 w1 (I + KP0 ) 1 K
w2 (I + P0 K ) 1 P0 w1 (I + KP0 ) 1 KP0
(5)
In the single-input/single-output case, M has rank one. In the present case mr mc 0, so the right-hand side of Eq. (3) is simply and the right-hand side of the expression dening reduces to its second term. Thus
(M ( j )) = |w2 ( j )(1 + P0 ( j )K ( j )) 1 | + |w1 ( j )(1 + K ( j )P0 ( j )) 1 K ( j )P0 ( j )|

and the condition for robust performance can be expressed as |w2 ( j )| + |w1 ( j )K ( j )P0 ( j )| < |1 + P0 ( j )K ( j )| Re Extensions The structured singular value may be further generalized in many directions, depending on uncertainty descriptions and characterizations. Some of these generalizations are summarized next. Nondiagonal uncertainty structure. The uncertainty structure need not be diagonal. It can contain unknown, independently bounded blocks in every entry. Doyle (2) and Kouvaritakis and Latchman (15) showed that the analysis may be converted into one based on the standard , but this may lead to a substantial increase in computational effort. Chen et al. (16,17) proposed a computational scheme that renders the computation growth insignicant. Uncertainty with phase information. Tits et al. (18) adopted a notion of with phase, in which not only are uncertainties known to be bounded by given quantities, but also their phases are known to vary in given ranges. The formulation gives a more detailed uncertainty de-
<1
(4)
This is readily handled by noting that, in view of the Small Gain Theorem, for any xed (s) such that the system is stable, condition (4) is equivalent to the stability of the augmented system depicted in Fig. 3 for all ctitious uncertainty
1 2 M
1 2
p
Figure 3. Fictitious uncertainty for robust performance.
608
scription, and it requires extensions of the concept of phase and of . 1-norm bounded uncertainty. Khammash and Pearson (19,20) studied structured uncertainties bounded in 1 norm, which is another active research area in robust control, concerning peak-to-peak system response. They showed that robust stability can be assessed by computing the spectral radius of a positive matrix constructed from the impulse response of the nominal system. Time-varying uncertainty. Shamma (21) and Megretsky (22) examined the robust stability problem with respect to structured time-varying uncertainties. They showed that if the uncertainty is allowed to vary arbitrarily over time, robust stability holds if and only if for some D D , DM(s)D 1. It is readily checked that the lefthand side (known as scaled H -norm), is similar to the right-hand side in condition (1), except that here the same D must be used at all frequencies. Subsequently, Poolla and Tikku (23) showed that, if the time variation of the uncertainty is arbitrarily slow, then robust stability holds if and only if the right-hand side in condition (1) is less than 1 at all frequencies. Finally, while may be custom made and seems to be an all-encompassing paradigm when extended appropriately, it cannot be applied to models in which the uncertainty block is allowed to be unstable. An effective robustness measure for the latter situation is furnished by the gap metric, a concept discussed elsewhere in this encyclopedia.
where q is an unknown vector that may or may not represent physical parameters. When q varies over a bounded set Q m, a family of polynomials are generated: P = { p(s, q ) : q Q} (7)
The problem of concern is to determine if the polynomial family P is robustly Hurwitz stable, by which we mean that every member in P is Hurwitz stable. We shall assume that the coefcients ak(q) are continuous functions of q. Furthemore, we assume that an(q) 0 for all q Q (i.e., all polynomials in P have the same degree). For control system analysis, it is typical to restrict the polynomial family P to the following classes, arranged by order of increased complexity. 1. P a: the coefcients ak(q) are afne functions of q. For example, p(s, q ) = s2 + (q1 + 2q2 + 3 )s + (4q1 + 5q2 + 6 ) 2. P m: the coefcients ak(q) are multiafne functions of q. For example, p(s, q ) = s3 + (2q1q2 + 2q1 q3 + q3 + 1 )s2 + ( 4 q 2 q 3 + 5 )s + ( q 1 q 2 q 3 + 1 ) 3. P p: the coefcients ak(q) are multivariate polynomials in q. For example,
2 2 p(s, q ) = s3 + (2q2 1 q 2 + 2 q 1 q 3 + q 1 q 3 + 1 )s
THE KHARITONOV APPROACH The Kharitonov approach, named after Russian mathematician V. L. Kharitonov whose celebrated 1978 theorem is often considered to be the cornerstone of the eld, is largely concerned with the issue of determining zero locations for a family of polynomials whose coefcients vary in a bounded set. Hence, by nature, it can be best presented in a framework different from that of the M loop, or , namely, directly as a polynomial stability problem. This issue, however, is connected to the M loop paradigm in an intimate fashion. To see this, simply consider a model set comprising proper real rational functions whose coefcients take values in certain bounded intervals. To determine robust stability of such a plant together with any compensator will then amount to checking whether the set of all resultant closed-loop characteristic polynomials have zeros in the stability region. For continuous-time systems, our main focus, the stability region of interest, is the open left half of the complex plane. Other regions of interest include the open unit disk, a shifted left half plane, and a sector; these regions can be imposed to study stability of discrete-time systems or to enforce pole placement constraints. A polynomial is generally said to be Hurwitz stable, or is referred to as a Hurwitz polynomial, if its zeros lie in the open left half plane. A general description for a set of polynomials of interest is
n
2 2 + ( 4 q 2 q 3 + 5 )s + ( q 2 1 q2 q3 + 1 )
It should be rather evident that P a P m P p and hence that the complexity in analysis increases in that same order. At present, the only available methods for tackling P m and P p are largely ad hoc, via either local optimization or graphical approaches, and they are either conservative or computationally formidable. In particular, when Q is an ball [i.e., a hyperrectangle (box) parallel to the coordinate axes], the problem of testing the stability of P m is known to be NP-hard. The class P a, as the sole tractable case, merits a particularly thorough study. A polynomial family P in this class consists of all polynomials of the form
m
p(s, q ) = p(s, q0 ) +
k=0
(qk q0 k ) pk (s )
(8)
Here q0 belongs to Q and may be regarded as the nominal value of uncertain parameter vector q, and the pk(s)s are xed polynomials. Evidently, one can assume with no loss of generality that p(s, q0) is Hurwitz stable, which is necessary for P to be robustly Hurwitz stable as q varies over Q. Let p [1, ], and let p be the standard p Ho lder norm dened on the Euclidean space m. That is,
p(s, q ) =
k=0
a k ( q )s ,
k
q
(6)
m p
!1/ p
, 1 p< p=
|qi | i= 1 max |qi |,

1im
609
Then a common description for Q adopts the notion of unit p balls centered at q0, dened as Q = {q : q q 0
p
1}
(9)
When the coefcient ak(q) depends on qk alone [i.e., when pk(s) is a constant multiple of sk], the p ball description gives rise to a class of most studied polynomial families. Such families can be expressed as
n
Because p(s, q0) is stable and by assumption an(q) never vanishes on Q, we can conclude from the continuity properties of the zeros of polynomials that if some member in P is not Hurwitz stable, then some polynomial in P must have a zero on the imaginary axis. Thus the entire family P is Hurwitz stable if and only if 0 / p( j , Q ),
P =
k=0
k qk s : q q
k
(10)
for some (possibly different) q0 and some xed scalars k. In particular, for p , 1, 2, such P is referred to as an interval polynomial, a diamond polynomial, and a spherical polynomial, respectively. Arguably, the entire success of the Kharitonov approach may be best summarized as a triumph over the polynomial family in Eq. (8) with the p norm characterization in Eq. (9), but not beyond, of which the most shining example is Kharitonovs celebrated theorem. Interval Polynomials An interval polynomial can be equivalently expressed as
n
that is, the value set never contains the origin. This is a rather straightforward fact known as early as in 1929 and is generally referred to in the literature as the zero exclusion principle. Note, in particular, that for a polynomial p(s, q) in the family described by Eq. (11), for any q Q, the real and imaginary parts of p( j, q) are respectively given by Re p( j , q ) = q0 q2 2 + q4 4 and Im p( j , q ) = j (q1 q3 2 + q5 4 ) Thus the real part (resp. imaginary part) of p( j, q) depends only on the parameters with even (resp. odd) subscript. For an interval polynomial, therefore, it becomes clear that Re K1 ( j ) = Re K2 ( j ) Re p( j , q ) Re K3 ( j ) = Re K4 ( j )
P =
k=0
q k s k : q k q k qk
(11)
Im K1 ( j ) = Im K4 ( j ) Im p( j , q ) Im K3 ( j ) = Im K2 ( j ) Because the four Kharitonov polynomials are themselves in P , it follows that, at each , the value set is a rectangle in the complex plane with vertices Ki( j), as depicted in Fig. 4. As increases, the rectangle evolves in a continuous fashion. Its dimensions vary, but its edges remain parallel to the axes, and the relative positions of the Ki( j)s do not change. Now, a polynomial p(s) of degree n with positive coefcients is Hurwitz stable if and only if the phase of p( j) monotonically increases from 0 to n /2 as goes from 0 to . This is best seen by noting that p(s) can always be factorized as p(s) an(s si) in terms of its zeros si, and by considering the phase of each factor separately. When this is understood, Kharitonovs Theorem becomes self-evident. Indeed, if the Kis are Hurwitz stable, then the phase of each Ki( j) increases monotonically from 0 to n /2 and the entire Kharitonov rectangle rotates around the origin by a total angle of n /2 without ever touching it. Stability of P then follows from the zero exclusion principle. Kharitonovs Theorem opened an era epitomized by the search for so-called vertex results in robust stability analysis, as manifested by the construction of the four Kharitonov polynomials at the vertices of the hyperrectangle Q. This theme
where the qks and the qks are xed. Kharitonovs original treatment of the interval polynomial problem is of an algebraic nature. He constructed four extremal members of P ,
K1 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K2 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K3 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K4 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
later dubbed Kharitonov polynomials. In a remarkable fashion, Kharitonov showed that the stability of the entire interval polynomial family can be ascertained by testing merely these four. Kharitonovs Theorem. The interval polynomial is Hurwitz stable if and only if K1(s), K2(s), K3(s), and K4(s) are all Hurwitz stable. Subsequent development showed that for polynomials of degree 5, 4, and 3, the test can be further simplied, requiring checking only 3, 2, and 1 of the four Kharitonov polynomials, respectively. Dasgupta (24) gave a geometrical interpretation to Kharitonovs Theorem in frequency domain, which sheds light on why such a puzzling result would hold. The interpretation makes use of the concept of value set. The idea is to examine the image of the polynomial family when s lies on the boundary of the stability region, namely p( j , Q ) = { p( j , q ) : q Q}
Im K2(j) K1(j) K3(j) K4(j) Re

Figure 4. Kharitonov rectangle.
610
persisted in much of the subsequent work. Although Kharitonovs Theorem is without question a milestone in control theory, and perhaps a measured intellectual triumph in general, we should note that the interval polynomial family is nevertheless a very special instance of an afne uncertainty class, and hence the theorem has a rather limited scope in application. The quest thus continues. Edge Theorem In light of Kharitonovs Theorem, it is tempting to contemplate the possibility that a similar vertex result would hold for the general afne polynomial family with Q a hyperrectangle. This, unfortunately, turns out to be false; a counterexample can be readily constructed to demonstrate the opposite. What can be said about a polynomial family P in class P a? Bartlett et al. (25) provided an answer. In what is now known as the Edge Theorem, they took the bounding set Q to be a convex polytope in m. Let qi, i 1, . . ., l, be the vertices of Q. Then, it is well known that Q can be represented as a convex hull of the vertices. That is,
l l
the polytope because some of these segments are not edges. In some cases, however, it is not an easy task to check which such segments are edges and which are not. The Edge Theorem generalizes Kharitonovs Theorem in two important aspects. First, it is applicable to general afne polynomial families, with Q an arbitrary polytope. Secondly, the stability region may be any open, simply connected set (27,28), although only stated here in terms of the open left half plane. In contrast, Kharitonovs Theorem addresses only Hurwitz stability. On the other hand, as the number of parameters increases, the computation effort required in the Edge Theorem can be enormous. For example, when Q is a box in m, with 2m vertices, the number of edges is m2m1, a very large number even for moderate values of m. Finally, to a lesser extent, the Edge Theorem is not applicable to situations where the boundary of Q is curved, of which the spherical polynomial family is an example. Graphical Tests The complexity of the Edge Theorem provides a direct motivation to search for computationally more tractable stability criteria, and graphical conditions become a natural avenue to explore. Not only are graphical criteria time-honored tools in classical stability analysis, but the zero exclusion principle, with all its simplicity and transparency, also prompts a deeper investigation of such tools. We can generally feel that an important asset of the zero exclusion principle is its generality, both in terms of uncertain polynomial families and in terms of stability regions. Barmish (29) studied the issue systematically; earlier conditions had appeared sporadically on stability of perturbed polynomials in isolated cases. Barmishs approach stems from a geometrical argument: a convex polygon in the complex plane does not intersect the origin as long as it can be separated from it by a straight line or, equivalently, as long as the vertices can be separated, as a whole, from the origin by such a line. This observation led to his construction of a testing function, which is to be evaluated along the boundary of the stability region. After this is accomplished, we can determine stability by plotting the testing function. Barmishs test is certainly one step forward compared to a pure brute-force computation; however, it remains somewhat ad hoc and is computationally overwhelming. Because it requires evaluations at all vertices, it does not clear the hurdle of exponential growth in complexity. On a lesser scale, Tsypkin and Polyak (30) obtained a graphical test for a simpler problem. They examined the polynomial family in Eq. (10). Let p [1, ] be given. Furthermore, for r [1, ] such that (1/ p) (1/ r) 1, dene
r Xr () = (0 + (2 2 )r + (4 4 )r + )1/r r Yr () = (1 + (3 2 )r + (5 4 )r + )1/r 0 2 0 4 R() = 0 q0 0 2 q2 + 4 q4 + 0 2 0 4 I () = 1 q0 1 3 q3 + 5 q5 +
Q = conv{q1 , . . ., ql } =
i= 1
i q i :
i= 0
i = 1, i 0
Because ak(q) is an afne function of q, it follows that P = conv{ p(s, q1 ), . . ., p(s, q l )} This implies that the value set p( j, Q) is a polygon, generated by the vertex polynomials p(s, qi). It should be rather clear that the interval polynomial family is generated by a polytopea hyperrectangleand so is the diamond polynomial family. Edge Theorem. The afne polynomial family (polytope of polynomials) P a is Hurwitz stable if and only if for each edge point q of Q, p(s, q) is Hurwitz stable. Let qi and qj be two vertices of Q connected by an edge. When varies from 0 to 1, the polynomial pij (s, ) = p(s, q i ) + (1 ) p(s, q j ) denes a line segment (of polynomials) connecting p(s, qi) and p(s, qj). The theorem shows thatquoting directly from Ref. 25 it sufces to check the edges. Because an edge polynomial involves only one parameter, its stability can be readily tested, by resorting to either a graphical test based upon a root locus, or the numerical solution of a generalized eigenvalue problem (see, for example, Chapter 4 in Ref. 26). The heuristics behind the Edge Theorem are simple. Because for large values of , the value set does not contain the origin (this is easy to see), the polynomial family will be Hurwitz stable if and only if there is no frequency at which the origin belongs to the boundary of the (polygonal) value set. It should be intuitively clear that every point on the boundary of the value set must be the image of a point on an edge of the polytope of polynomials. Thus, if for some the origin does belong to this boundary, the corresponding edge polynomial must be unstable. Note that it is wasteful to check the entire set of all polynomial segments joining two vertices of
Note that R() Re(p( j, q0)) and I() Im(p( j, q0)). Tsypkin-Polyak Criterion. The polynomial family P of Eq. (10) is Hurwitz stable if and only if p(s, q0) is Hurwitz stable,

0 0 qn 1, q0 1, and
611
|R()| Xr ()
|I ()| Yr ()
> 1,
( 0, )
(12)
This condition was independently obtained by Hinrichsen and Pritchard (31). Up to this point, we could assert that the stability problem for afne polynomial families remains largely unresolved. However, as yet another observation, we nd that at each , the zero exclusion condition denes two linear constraints in terms of perturbed coefcients, imposed on the real and imaginary parts of p( j, q), respectively. These constraints, together with a convex bounding set Q, dene in turn a convex feasibility condition; when the parameters vary independently, it reduces further to a linear program. This is a simple but conceptually appealing observation. It led to a reformulation via linear programming, due to Saridereli and Kern (32) and to Tesi and Vicino (33), which can be solved readily for each and then plotted graphically. Qiu and Davison (34) went further to demonstrate that for very general bounding sets it sufces to solve an optimization problem with one variable only, and the problem can be solved explicitly in special cases. Finally, Chen et al. (14,35) recognized that the problem can be reformulated as a special rank-one problem for each , and showed that stability can be ascertained by evaluating an explicit formula. These results led to the nal resolution of the afne polynomial family.
with respect to the block structure diag(1, . . ., m): i . In light of the formula for rank-one , an explicit condition can then be stated. Such a result clearly applies to general stability regions, and it furnishes a frequency sweeping condition for robust stability. Note that we may interpret this result alternatively based upon the zero exclusion principle. Indeed, under the condition that Q is the unit ball centered at q0, all the polynomials in P will have zeros in a specied region if and only if the zeros of p(s, q0) are in that region and (M(s)) 1 for all s on the boundary of the region. This follows because, according to the zero exclusion principle, it is both necessary and sufcient that
m
min
q q0
p(s, q0 ) +
k=0
(qk q0 k ) pk (s ) = 0 > 1
and the Kharitonov Approach

Indeed, there is an inherent linkage between analysis and Kharitonov approach, whenever the latter applies, in that both approaches yield necessary and sufcient conditions for problems of the same nature. However, for a rather long time a clear link seemed elusive. The main cause, it seems, lay in how to reconcile an optimization-based formulation such as , and explicit results from the Kharitonov approach. Can one, for example, derive Kharitonovs theorem from , or vice versa? The explicit formula of rank-one given earlier lends an answer. Specically, for a general afne polynomial family P a [(Eq. (8)] with Q the unit ball, robust stability can be checked by computing , with a rank-one matrix M(s) constructed as 1 pm (s ) . p1 (s ) M (s ) = . . p(s, q0 ) p(s, q0 ) 1 To see this, observe that p(s, q) in Eq. (8) is the characteristic polynomial of the M loop of Fig. 1 with diag(q1 0 0 q1 , . . ., qm qm ). Indeed,
m
in order for the polynomial family p(s, q) in Eq. (8) to have no zero on or exterior to the boundary of the region, for all possible q Q. More generally, it is possible to extend the denition of by means of more general norms and to use this extended to study the robust stability of an afne family P with a more general bounding set Q. Such a generalization also leads to a similar expression when the M matrix in question has rank one (14,35). In particular, when the stability region is restricted to the open left half plane, and Q is the unit p ball centered at q0 with ak(q) kqk, the expression for the generalized rank-one , denoted as p( ) for purpose of distinction, is found to be Yr ()/|I ()| if R() = 0 if I () = 0 Xr ()/|R()| p (M ( j )) = () Y () X r r p p p p 1/ p (Xr ()|I ()| + Yr ()|R()| ) otherwise which leads to a similar condition for robust Hurwitz stability. This condition is slightly more general than, but essentially replicates, the graphical criterion by Tsypkin and Polyak. Note that for p , the polynomial family becomes an interval polynomial, and the stability condition reduces to checking whether p(s, q0) is Hurwitz stable, q0 1, and min X1 () Y1 () , |R()| |I ()| < 1, ( 0, )
A little thought reveals that the latter is equivalent to determining whether one of the four conditions Re(K1( j)) 0, Re(K3( j)) 0, Im(K1( j)) 0, and Im(K3( j)) 0 holds. Clearly, this is further equivalent to the requirement that the rectangular value set in Fig. 4 never contains the origin. Extensions There is an immense body of literature devoted to polynomial stability problems. Various extensions to Kharitonovs Theorem have been obtained. They generally fall into the categories of vertex results and frequency-sweeping conditions, consisting of delicate studies and intricate technical details. We summarize some of the highlights next. A recent and comprehensive account can be found in the books by Barmish (26) and by Bhattacharyya et al. (36).
det(I
M (s )) = 1 +
k=0
pk (s ) p(s, q ) (qk q0 k) = 0 p(s, q ) p(s, q0 )
Thus, stability of p(s, q) for all q Q is equivalent to stability of the M loop for all diagonal matrices with real (parametric) entries lying in [1, 1]. The condition for this is that M 1 where the structured singular value is computed
612
Vertex and Edge Results. Much of the work in this direction continues the thread in Kharitonovs Theorem, focusing on simple uncertainty descriptions and leading to stability tests based on vertex and/or edge polynomials. Some notable examples follow. Complex Interval Polynomials. The polynomial family has complex coefcients whose real and imaginary parts are allowed to vary independently in given intervals. Eight vertex polynomials need to be tested to ascertain stability. Diamond Polynomials. This polynomial family is described in Eq. (10), with p 1. Eight vertex polynomials are required as well. Stabilization of Interval Plants via First-Order Compensators. The numerator and denominator of the plant transfer function are interval polynomials, and it is to be stabilized by a rst-order compensator in closed loop. It sufces to stabilize 16 vertex plants, constructed based upon the vertex numerator and denominator polynomials. Generalized Kharitonov Theorem. It concerns linear combination of interval polynomials and requires checking certain polynomial segments in addition to vertices. Other stability conditions based on vertex polynomials are also available. As the complexity of uncertainty structure increases slightly, they usually require testing more (e.g., 32 or 64) vertex polynomials. A clear insight concerning uncertainty structure and the required number of vertices, however, remains unavailable. Performance Issues. The entire Kharitonov theory is largely successful for determining stability of uncertain polynomials. However, a number of results are also available regarding properties of transfer functions, which have implications toward performance issues. Some examples follow. H Norm of Interval Transfer Functions. When the numerator and denominator of a transfer function are both interval polynomials, the H norm of the transfer function can be computed over 16 vertex transfer functions, provided that the four Kharitonov polynomials associated with the denominator are stable. Peak Magnitudes of Closed-Loop Transfer Functions. The peak H norm of closed-loop transfer functions can be computed over the edges of the plant family, when it is an interval plant. Nyquist and Bode Envelopes. The Nyquist and Bode plots of open or closed-loop transfer functions associated with an interval plant lie in envelopes determined by plots generated by vertex and edge plants. Other Extensions. Additional extensions may be found in the following categories. Schur Stability. The Kharitonov theory has been extended with varying degrees of success to other stability regions, such as the unit circle. These results are useful for studying stability of discrete-time systems and for addressing other performance issues.
Unstructured Uncertainty. Unmodeled dynamics may be included along with parametric uncertainties. They may be accommodated either in the rank-one formula or by small gain-type conditions involving vertex and edge plants. Nonlinear Systems. In a system consisting of an interval plant and a static, sector bounded nonlinear component, stability conditions similar to the Popov and circle criteria have been obtained, which also require the testing of vertex and edge plants. Multilinear Uncertainty Structure. The entire success in the Kharitonov approach relies on the key assumption that polynomial coefcients depend linearly on uncertain parameters, and the utility of all the results in this area is thus measured by how much the uncertainty can deviate from this description. Little success has been achieved in this endeavor. A fundamental barrier, as implicated by the zero exclusion principle, is that the stability problem is one of optimization subject to nonlinear, nonconvex constraints. STATE-SPACE APPROACH Dynamical systems are often described by state-space equations. Accordingly, it is common to model system uncertainty as perturbations to system matrices. An uncertain continuous-time system in this spirit may be described by x (t ) = (A + B C )x(t ) (13)
Here A, B, and C are known matrices of appropriate dimensions. The matrix A is assumed to be stable. The system uncertainty is represented by a set of allowed values for the real matrix , which may be unstructured or structured. Typical perturbation classes considered in the literature are as follows, arranged in increasing order of generality. In all cases, 0 is given. Unstructured Perturbation. The set consist of all real matrices with spectral norm less than a given number:
U
= { real : ( ) }
Element-by-Element Perturbation. Each element in varies in a given interval. Let rij 0 be given. The set is dened as r11 11 r1m 1m . . . . . real : = E = . . . . , rn1 n1 rnm nm = max{|ij | : rij > 0} i, j Linear Combination. The allowable set of perturbations is described by
k L
real:
=
i= 1
i Ei ,
= max |i |
i
where the Eis are given.
613
Evidently, an uncertain discrete-time system can be described in exactly the same manner. The problem of interest is to determine the size of the perturbation matrix, measured by a norm of choice, so that the system remains stable. This issue naturally translates into one concerning how the eigenvalues of a stable matrix A would vary when it is perturbed by . More specically, would the eigenvalues cross the stability boundary? And if they do, what is the minimal such that at least one of the eigenvalues leaves the stability region? These questions may be addressed by examining the characteristic polynomial (s, ) = det (sI A B C )
There essentially exists no result for the structured stability radius other than those already known for . For the unstructured stability radius, much of the early work was devoted to derivation of bounds. One representative example is r(A, B, C ) 1 C(sI A ) 1 B (14)
This, of course, is a rather straightforward consequence of the Small Gain Theorem. Recently, however, Qiu et al. (37) obtained the following exact, readily computable formula. Real Stability Radius. Let G(s) C(sI A)1B, and 2( ) be the second largest singular value. Then, Re[G(s )] Im[G(s )] r(A, B, C ) 1 = sup inf 2 1 Im[G(s )] Re[G(s )] s D (0 , 1] The signicance of this result lies in that for any s , the function 2( ) is unimodal in over (0, 1), and hence its inmum can be computed effectively. Furthermore, when is the open left half plane or the open unit disk, that is, when Hurwitz or Schur stability is of concern, Sreedhar et al. (38) developed a fast-converging algorithm for the maximization with respect to s. Consequently, from a computational standpoint, the unstructured stability radius problem can be considered largely resolved. Interval Matrices An interval matrix is a family of real matrices in which all elements are known only within certain closed intervals. In precise terms, the interval matrix AI [A, A] is the set of matrices dened by AI = {A : a ij aij aij } that is, each aij of A is conned elementwise to lie within an interval determined by aij and aij, the corresponding elements of A and A, respectively. An interval matrix AI is said to be stable if every A AI is stable. Evidently, interval matrix and set E share the same uncertainty description. Interval matrices are direct matrix analogues of interval polynomials, and hence there has been a lingering temptation for extension of Kharitonovs Theorem to the former. Unfortunately, neither vertex nor edge results exist for interval matrices. In fact, more recent studies showed that in order to determine stability of an interval matrix, we must solve an NP-hard decision problem. This in a way explains why only sufcient stability conditions are available. One approach of attack is to analyze eigenvalue distribution. Heinen (39) and Argoun (40) examined the problem on the basis of Gershgorins Theorem, and their developments culminated in a subsequent work of Chen (41), leading to a number of simple, albeit conservative, stability conditions. As a representative example of these results, consider an interval matrix AI such that aii 0. Let W be constructed as 0 i= j max{|a ij |, |aij |} W = [wij ], wij = i= j |aii |
or equivalently, the characteristic equation det(I C(sI A ) 1 B ) = 0 Thus, it becomes clear at the outset that the problem may be tackled in principle by using a polynomial approach. Similarly, it can also be analyzed as a problem. The latter can be easily seen with respect to E and L, by rearranging the elements of these sets into diagonal matrices and by dening the M matrix appropriately. For U, we may simply adopt a full real block structure and dene accordingly. It should be pointed out, nevertheless, that both and the polynomial approach will lead to complications in the present context. On the one hand, the computation of with respect to a real is generally very difcult, and approximation by its upper bound can be very conservative. On the other hand, the characteristic polynomial (s, ) will generally exhibit a multilinear or multinomial dependence of its coefcients on , for which the Kharitonov theory is ill-equipped; indeed, it is not difcult to see that the coefcients of (s, ) are multilinear in ij if E, and are multinomial functions of k if L. In summary, both approaches are ineffective and conservative. By far this uncertainty description poses the most difcult challenge in robust stability analysis, and the state-space approach is the least developed. Results are scarce, and only in rare cases are they nonconservative. Stability Radius A notion frequently encountered in studying the state-space uncertainty description is that of stability radius. This notion is closely related to , but it is less developed. Let be a stability region of concern, and be its boundary. Furthermore, denote by (A) the spectrum of A. Then for any norm of interest, the stability radius associated with the triple (A, B, C) is dened by r(A, B, C ) = inf{ : , (A + B C ) D = }
In other words, it denes the minimal perturbation size leading to instability, or the distance of A to the set of unstable matrices. By denition, it thus follows directly that the matrix family A BC: , has all eigenvalues in whenever r(A, B, C) . Moreover, in view of the preceding discussion, we may regard the stability radius as the reciprocal of the maximum of a certain , with respect to an appropriate block structure and a matrix M. For further distinction, the stability radius is said to be unstructured if is unstructured and structured otherwise.
614
Then a sufcient condition for Hurwitz stability of AI is found by Chen (41) to be (W ) < 1 A useful feature of this result, and more generally of conditions obtained by using Gershgorins Theorem, is that it lends a ready characterization via the so-called M-matrices. The latter aspect makes it possible to unify a number of sufcient stability conditions in different forms. Alternatively, Yedavalli and others studied interval matrices from a Lyapunov analysis standpoint. This is collectively inspected next. Lyapunov Analysis The Lyapunov theory, as anticipated, is widely employed in robust stability analysis pertaining to state-space formulation, yielding various results concerning stability radius and interval matrices. One common thread in this approach is to nd a single quadratic Lyapunov function applicable to the entire family of the perturbed matrices; the technique is often referred to in the literature as quadratic stability. Another lies in the simplicity of the stability conditions. Let us begin with the unstructured uncertainty set U. By constructing the usual Lyapunov function V (x ) = 1 x T Px 2 we nd that the entire family of matrices A BC: U is Hurwitz stable if there exists a positive denite matrix P 0 such that PA + AT P + 2 BBT + CTC < 0 (15)
Furthermore, Zhou and Khargonekar (42) observed that the uncertainty description E can be regarded as a special case of L, for which they provided the stronger Hurwitz stability condition
1 2
(|PEi +
i= 1
EiT P| )
< 1/
(18)
where, again, P 0 is the unique solution to Eq. (17). Subsequent developments led to further extensions for problems with even more detailed uncertainty descriptions. For example, the ks may be allowed to vary in asymmetric intervals. Moreover, because rather obviously any interval matrix can be represented alternatively in the form of A : E, these conditions can be applied to determine the Hurwitz stability of an interval matrix as well. Yet another issue clearly of interest is whether it is possible to derive vertex versions of these sufcient conditions. Boyd and Yang (44) examined stability problems for matrix polytopes. Specically, they postulated the uncertainty description A = conv{A1 , . . ., Ak } A sufcient condition for A to be Hurwitz stable can be easily found to be the existence of a P 0 such that PAi + AT i P < 0, i = 1, . . ., k (19)
Similarly, for the uncertainty set L, a vertex condition can be obtained as
1 2
k i (PEi i= 1
EiT P )
< 1/
(20)
This, of course, does not come as a surprise. According to the well-known Bounded Real Lemma (42), it is equivalent to C(sI A ) 1 B
< 1/
The latter condition clearly coincides with condition (14). More results are available for the special case when B C I. For the structured uncertainty set E, Yedavalli (43) gave the sufcient condition |P|R + RT |P| 2 < 1/ (16)
for all combinations of the i in 1, 1. It should be rather evident that this condition improves upon inequality (18). Both conditions (19) and (20) may be regarded as vertex results in the matrix perturbation case, and both can be posed and solved as LMI problems. CONCLUSION Summary For the past two decades, modeling uncertainty and robustness has resurfaced as a dominating theme in control theory and application and is now held unanimously by theoreticians and practitioners as the most important concern in control system design. For both its intrinsic appeal and practical signicance, robust control as a whole attracted considerable interest and underwent a period of immense development, bringing control theory to a new height. Many important issues have been addressed. Many remain unresolved. The ultimate puzzle, it now appears, lies in the fundamental conict between problem complexity and computational tractability. Of the three main research areas surveyed in this article, the structured singular value provides the most general formulation for uncertainty modeling and is the most systematically developed tool in robustness analysis. The major issues
for the Hurwitz stability of the matrix family A : E. Here P 0 is the unique solution to the Lyapunov equation PA + AT P = 2I (17)
P denotes the modulus matrix of P (i.e., each entry of P is the absolute value of the corresponding entry of P) and R is given by
r11 . R= . . rn 1
. . .
r1 n . . . rnn
615
in analysis are clearly generality of uncertainty description, conservatism of analysis, and ease of computation. The main success achieved with this approach, unquestionably, lies in the progress in computing . While it cannot be computed exactly in general, various computational schemes have been developed for computing it approximately, and commercial software programs are now available. This paves the way for its application to a series of engineering design problems, ranging from disk drive control to ight control. Successful applications to other potential areas, including robot manipulators, exible structures, magnetic bearings, and chemical processes, have also been reported in laboratory experiments. The Kharitonov approach, unlike analysis, was more restrictive in scope in its early phase of development. However, it has undergone a bottom-up growth pattern as the uncertainty descriptions become progressively more general and sophisticated. Overall, the Kharitonov and state-space methods may be broadly classied as a parametric approach toward robustness analysis, originating from interval polynomials and culminating at state-space uncertainty descriptions. The main appeal of this approach, it appears, lies in its quest for analytical solutions, more appealing than mere computationbased tools. The main success in the entire parametric approach, which remains the state-of-the-art today, is the resolution of the afne uncertain polynomial family case, for which necessary and sufcient stability conditions are available, in terms of both edge tests and graphical conditions. On the other hand, the multilinear/multinomial polynomial family and the state-space uncertainty description are the weakest link, for which only sufcient stability conditions are available with unknown conservatism, and more systematic, efcient, computation-based approximate tests are called for. At present, only a few applications of the Kharitonov theory are reported in the literature, including Ackermanns car steering problem and an automotive engine control problem investigated by Abate et al. (45) (see also Chapter 3 in Ref. 26). It should be rather evident that the fundamental bottleneck in all robustness analysis methods, be it analysis or Kharitonov approach, lies in computational complexity, and the ultimate challenge is in the conquest over the curse of dimensionality. No matter whether this can be achieved or not, we should be consciously aware that the dilemma is the natural cause of problem generality and hence complexity and results from the search of optimal solutions. In engineering system design, we should therefore reconcile and seek a judicious trade-off between these conicting requirements. To Probe Further In light of the difculties encountered in robustness analysis with respect to structured and/or parametric uncertainties, a number of researchers recently examined complexity issues from a computational standpoint, drawing upon concepts and techniques from computing science and operation research. The main discoveries are in the following areas. with Real Uncertainties. Braatz et al. (46) showed that the computation of real is NP-hard. with Complex Uncertainties. Toker and Ozbay (47) proved that the computation of complex is also NPhard.
with Real and Complex Uncertainties. Braatz et al. (46) and Toker and Ozbay (47) both showed that the computation of is NP-hard. The Bounds. Toker (48) and Fu (49) showed that the problem of nding an accurate bound for is NP-hard. Interval Matrix. Coxson and DeMarco (50) showed that stability of interval matrices amounts to an NP-hard problem. These results indicate that a worst-case instance exists in each class of the problems, for which it is rather unlikely that computational complexity can be bounded via a polynomial function of the problem dimension. It thus comes as no surprise that the problems are difcult, and indeed are intractable in general. From a technical standpoint, the computational difculty in question may be best seen as an outcome of nonlinear, nonconvex optimization problems. Although only explored systematically in recent years, complexity issues have been under contemplation for a long time and have led to alternative, computationally tractable approximations and formulations. One notable remedy is to resort to formulations based upon LMIs, and problems in this class include those that can be described via integral quadratic constraints (IQC). Both LMIs and IQCs offer in essence an energy-based perspective toward system analysis, and they draw heavily upon concepts in classical passivity and dissipativity theory, leading to readily computable, albeit only sufcient, robust stability conditions. For a comprehensive treatment of control-relevant LMI and convex programming problems, see Ref. 51, or the relevant chapter in this encyclopedia. Megretsky and Rantzer (52) provided a detailed account of the IQC technique. The computational complexity results just discussed are strongly linked to the worst-case nature of the robustness problems; that is, the requirement of robustness must be met for all possible instances. Is so stringent a requirement truly necessary? This question prompted a reexamination of robustness issues, and it led to a recent venture departing almost entirely from the worst-case formulation. A number of researchers argued that worst-case scenarios hardly occur in practice, that a worst-case analysis is not only overly demanding but also too pessimistic, and that, after all, worstcase analysis problems are often intractable. The argument thus motivated the description of uncertainty via probabilistic measures, and accordingly probabilistic approaches to robustness analysis. In this new thinking, the deterministic uncertainty description is discarded altogether and is replaced by a probability description characterizing the likelihood that the uncertainty may lie in a bounded set. The robustness condition then amounts to determining the probability under which the system may become unstable. Recent studies (53 56) show that a variety of problems, which are NP-hard in the deterministic setting, become readily solvable computationally when formulated probabilistically. The area, however, is entirely open and is not without obstacles of its own.
BIBLIOGRAPHY
1. J. Ackermann, Robust Control Systems with Uncertain Physical Parameters, London: Springer-Verlag, 1996.
616
ROBUST CONTROL ANALYSIS 27. M. Fu and B. R. Barmish, Polytopes of polynomials with zeros in a prescribed set, IEEE Trans. Autom. Control, 34 (5): 544546, 1989. 28. A. L. Tits, Comments on polytopes of polynomials with zeros in a prescribed set, IEEE Trans. Autom. Control, 35 (11): 1276 1277, 1990. 29. B. R. Barmish, A generalization of Kharitonovs four polynomial concept for robust stability problems with linearly dependent coefcient perturbations, IEEE Trans. Autom. Control, 34 (2): 157 165, 1989. 30. Ya. Z. Tsypkin and B. T. Polyak, Frequency domain criteria for lp-robust stability of continuous linear systems, IEEE Trans. Autom. Control, 36 (12): 14641469, 1991. 31. D. Hinrichsen and A. J. Pritchard, An application of state space methods to obtain explicit formulae for robustness measures of polynomials, in M. Milanese et al. (ed.), Robustness in Identication and Control, Boston, MA: Birkhauser, 1989. 32. M. K. Saridereli and F. J. Kern, The stability of polynomials under correlated coefcient perturbations, Proc. 26th IEEE Conf. Decision Control, 1987, pp. 16181621. 33. A. Tesi and A. Vicino, Robustness analysis for linear dynamical systems with linearly correlated parametric uncertainties, IEEE Trans. Autom. Control, 35 (2): 186191, 1990. 34. L. Qiu and E. J. Davison, A simple procedure for the exact stability robustness computation of polynomials with afne coefcient perturbations, Syst. Control Lett., 13: 413420, 1989. 35. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular value and stability of uncertain polynomials, Part 2: A missing link, Syst. Control Lett., 23 (2): 97109, 1994. 36. S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The Parametric Approach, Upper Saddle River, NJ: PrenticeHall, 1995. 37. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31 (6): 879890, 1995. 38. J. Sreedhar, P. Van Dooren, and A. L. Tits, A fast algorithm to compute the real structured stability radius, in R. Jeltsch and M. Mansour (eds.), Stability Theory: Proc. Centenary Conf. Int. Series Numerical Math. (ISNM), Basel: Birkha user, 1996, vol. 232, pp. 219230. 39. J. A. Heinen, Sufcient conditions for stability of interval matrices, Int. J. Control, 39 (6): 13231328, 1984. 40. M. B. Argoun, On sufcient conditions for the stability of interval matrices, Int. J. Control, 44 (5): 12451250, 1986. 41. J. Chen, Sufcient conditions on stability of interval matrices: Connections and new results, IEEE Trans. Autom. Control, 37 (4): 541544, 1992. 42. K. Zhou and P. P. Khargonekar, Stability robustness bounds for linear state-space models with structured uncertainty, IEEE Trans. Autom. Control, 32 (7): 621623, 1987. 43. R. K. Yedavalli, Improved measures of stability robustness for linear state space models, IEEE Trans. Autom. Control, 30 (6): 577579, 1985. 44. S. Boyd and Q. Yang, Structured and simultaneously Lyapunov functions for system stability problems, Int. J. Control, 49 (6): 22152240, 1989. 45. M. Abate et al., Application of some new tools to robust stability analysis of spark ignition engines: A case study, Proc. Amer. Control Conf., 1992, pp. 932936. 46. R. P. Braatz et al., Computational complexity of calculation, IEEE Trans. Autom. Control, 39 (5): 10001002, 1994. 47. O. Toker and H. Ozbay, On the complexity of purely complex computation and related problems in multidimensional systems, IEEE Trans. Autom. Control., 43 (3): 409414, 1998.
2. J. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., 129-D (6): 242250, 1982. 3. V. L. Kharitonov, Asymptotic stability of an equilibrium position of a family of systems of linear differential equations, Differentialnye Uraveniya, 14 (11): 14831485, 1978. 4. M. G. Safonov, Stability margins of diagonally perturbed multivariable feedback systems, IEE Proc., 129-D: 251256, 1982. 5. A. Packard and J. C. Doyle, The complex structured singular value, Automatica, 29 (1): 71109, 1993. 6. W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, 1996. 7. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 8. K. Zhou and J. C. Doyle, Essentials of Robust Control, Upper Saddle River, NJ: Prentice-Hall, 1998. 9. A. L. Tits and V. Balakrishnan, Small- theorems with frequency-dependent uncertainty bounds, Math. Control. Signals, Syst., in press, 1998. 10. J. M. Krause, Structured singular value analysis of multidimensional system stability, IEEE Trans. Autom. Control, 34 (6): 638 639, 1989. 11. J. Chen and H. A. Latchman, Frequency sweeping tests for stability independent of delay, IEEE Trans. Autom. Control, 40 (9): 16401645, 1995. 12. M. K. H. Fan, A. L. Tits, and J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodelled dynamics, IEEE Trans. Autom. Control, 36 (1): 2538, 1991. 13. P. M. Young, The rank one mixed problem and Kharitonovtype analysis, Automatica, 30: 18991911, 1994. 14. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular value and stability of uncertain polynomials, Part 1: The generalized , Syst. Control Lett., 23 (1): 5365, 1994. 15. B. Kouvaritakis and H. Latchman, Necessary and sufcient stability criterion for systems with structured uncertainties: The major principle direction alignment principle, Int. J. Control, 42 (3): 575598, 1985. 16. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular values with non-diagonal structures, Part 1: Characterizations, IEEE Trans. Autom. Control, 41 (10): 15071511, 1996. 17. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular values with non-diagonal structures, Part 2: Computations, IEEE Trans. Autom. Control, 41 (10): 15111516, 1996. 18. A. L. Tits, V. Balakrishnan, and L. Lee, Robustness under bounded uncertainty with phase information, IEEE TAC, in press. 19. M. Khammash and J. B. Pearson, Robust disturbance rejection in l1-optimal control systems, Syst. Control Lett., 14: 93101, 1990. 20. M. Khammash and J. B. Pearson, Performance robustness of discrete-time systems with structured uncertainty, IEEE Trans. Autom. Control, 36 (4): 398412, 1991. 21. J. Shamma, Robust stability with time-varying structured uncertainty. IEEE Trans. Autom. Control, 39 (4): 714724, 1994. 22. A. Megretski, Necessary and sufcient conditions of stability: A multiloop generalization of the circle criterion, IEEE Trans. Autom. Control, 38 (5): 753756, 1993. 23. K. Poolla and A. Tikku, Robust performance against time-varying structured perturbations, IEEE Trans. Autom. Control, 40 (9): 15891602, 1995. 24. S. Dasgupta, Kharitonovs theorem revisited, Syst. Control Lett., 11: 381384, 1988. 25. A. C. Bartlett, C. V. Hollot, and L. Huang, Root locations of an entire polytope of polynomials: It sufces to check the edges, Math. Control, Signals, Syst., 1: 6171, 1988. 26. B. R. Barmish, New Tools for Robustness of Linear Systems, New York: Macmillan, 1994.
ROUNDOFF ERRORS 48. O. Toker, On the conservatism of upper bound tests for structured singular value analysis, Proc. 35th Conf. Decision Control, Kobe, Japan: 1996, pp. 12951300. 49. M. Fu, The real is hardly approximable, IEEE Trans. Autom. Control, 42 (9): 12861288, 1997. 50. G. E. Coxson and C. L. DeMarco, The computation complexity of approximating the minimal perturbation scaling to achieve instability in an interval matrix, Math. Control, Signals Syst., 7 (4): 279292, 1994. 51. S. Boyd et al., Linear Matrix Inequalities in System and Control Theory, Studies in Applied Mathematics, Philadelphia: SIAM, 1994, vol. 15. 52. A. Megretski and A. Rantzer, System analysis via integral quadratic constraints, IEEE Trans. Autom. Control, 42 (6): 819 830, 1997. 53. R. F. Stengel and L. R. Ray, Stochastic robustness of linear timeinvariant systems, IEEE Trans. Autom. Control, 36 (1): 8287, 1992. 54. B. R. Barmish and C. M. Lagoa, The uniform distribution: A rigorous justication for its use in robustness analysis, Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34183423. 55. R. Tempo, E. W. Bai, and F. Dabbene, Probabilistic robustness analysis: Explicit bounds for the minimum number of samples, Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34243428. 56. X. Chen and K. Zhou, On the probabilistic characterization of model uncertainty and robustness, Proc. 36th IEEE Conf. Decision Control, 1997, pp. 38163821.
617
JIE CHEN
University of California
L. TITS ANDRE
ROBUSTNESS ANALYSIS. See ROBUST CONTROL

ANALYSIS.
ROBUST SIGNAL PROCESSING. See NONLINEAR

SYSTEMS.
ROLLBACK RECOVERY. See PROGRAM DIAGNOSTICS. ROLLING METALS. See METALS INDUSTRY. ROTATION SPEED. See TACHOMETERS.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Robust Control Analysis Standard Article Jie Chen2 and Andr L. Tits3 2University of California, Riverside, CA 3University of Maryland, College Park, MD Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1035 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (247K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Structured Singular Value The Kharitonov Approach State-Space Approach Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
602
Robust control can be generally dened as the control, by means of xed compensators, of uncertain plants (i.e., of systems with uncertain dynamics and unknown disturbance signals). Robustness in a control system refers to its ability to cope with uncertainties in a satisfactory manner, maintaining its stability and performance as desired. Uncertainty in signals and systems is inevitable, reecting both the complexity of the physical world and the limitation in human understanding. Uncertain signals typically arise as a result of the randomness and unpredictability of environmental effects and are of an unmeasurable and unpredictable nature. Uncertain system dynamics, on the other hand, can be attributed to changes in the actual system and to modeling errors, be they accidental or deliberate. Generally, uncertain dynamics may come from the following sources: 1. Imperfect or incomplete knowledge of physical processes. This represents the information unattainable because of ones limited knowledge or inadequate measurements. It can be particularly acute for complex systems and processes (e.g., those found in biomedical engineering). 2. Parameter variations. Every physical system will undergo a change in parameter values under different operating conditions. Aging itself can be a factor. 3. Neglected high-frequency dynamics, time delays, nonlinearities, and the like. It may occur as a result of a sheer lack of knowledge, or the difculty to model these characteristics. It may also occur because of the desire for models of low complexity. While robustness is a concept of universal signicance, robustness analysis for control systems is the study of whether a system, however designed, can meet specied stability and performance goals in the presence of uncertainty within a prescribed range. Uncertain plants can be modeled in various ways. In particular, models can be stochastic or purely deterministic. In robust control, uncertain systems are typically modeled deterministically, as bounded sets of system models. A property is then said to hold robustly if it holds for every model in the set. The simplest case is that of unstructured uncertainty: the model set consists of all systems in a certain neighborhood (e.g., all transfer functions lying within a certain distance of a distinguished nominal system). Such a description is particularly appropriate to account for unmodeled dynamics. One rather typical example is the modeling of exible structures. It is well known that, in general, a exible structure cannot be accurately represented by a nite-dimensional system. For control design purposes, however, we desire, and most often are compelled to nd, an approximate nite-dimensional model with a relatively low order. In doing so, a common practice is to include in the nominal model a small number of dominant modes in the low-frequency range and to treat the high-frequency modes as modeling uncertainty. Evidently, this description is also appropriate for modeling errors resulting from model reduction, or from any frequency response truncation. Moreover, it can be used to cover, albeit in a conservative way, parameter variations. The latter amounts to drawing up a frequency response envelope to describe the range of parameter variation in frequency domain.

Robustness is a property inherently sought after in engineering systems. The concept is directly linked to such issues as design viability and system reliability. In broad terms, robustness can be regarded as the capability to withstand unknown, unexpected, and often hostile conditions that can adversely affect a systems behavior. A system must be sufciently robust in order to function properly under undesirable circumstances, conducting its task as designed. As engineering systems are becoming more and more complex and are required to operate in increasingly more uncertain environments, robustness has become increasingly more crucial.
603
Finally, in robustness analysis, it is common to introduce a ctitious uncertainty to represent a performance objective. A more accurate account of parametric uncertainty calls for model sets within which individual models are uniquely characterized by the value(s) of one or more parameter(s) (e.g., transfer function coefcients). Typically each such parameter takes values in a known range. Such models account for the fact that parameters in physical devices are bound to vary with time, environment, and operating conditions. Ackermanns car steering problem (1) offers a good illustrative example. In his study of a four-wheel steering vehicle, he found that the vehicle mass and the adhesion between tires and road surface are signicant uncertain parameters. This is easily understandable. The vehicle mass certainly varies with load, and in a more subtle sense, it varies with fuel consumption. The adhesion changes with road condition, wearing of tires, and weather condition. We can think of other uncertain parameters by considering the human-vehicle system as a whole or by considering a whole batch of vehicles as a family of systems. In these scenarios, differences among individual drivers and vehicles can all constitute signicant uncertain factors. Yet a more striking example is an aircraft, whose aerodynamic coefcients vary in large magnitudes due to changes in altitude, maneuvering, and weather. There is an inherent trade-off between delity and simplicity in modeling uncertain systems. In a sense, unstructured and parametric uncertainties may be considered the two extremes. While an unstructured perturbation furnishes a simple characterization and is useful for simplifying robustness analysis, it contains little information and may often be too conservative. On the other hand, uncertain parameters often yield a more natural and accurate model, but such elaborate descriptions tend to complicate analysis. The process of robustness analysis, therefore, calls for a judicious balance between uncertainty description and complexity of analysis. In its full generality, however, the description of an uncertain system should take into account both parametric variations and unmodeled dynamics. Uncertainty descriptions of this kind are called structured. To be sure, robustness is not entirely a new concept in control system analysis and design. In retrospect, the need for a control system to tolerate unmodeled dynamics and parameter variations is precisely what motivated feedback control, and it has been a primary goal in control system design since its birth. This is well recognized in classical control design and, at least implicitly, is embedded in classical loop shaping methods. Concepts such as gain and phase margins may well be regarded as elementary robustness measures. However, it was not until the late 1970s that the term began to appear routinely in the control literature, when the need for robustness was reexamined and was gaining increasing recognition. Robust control as a research direction soon thrived and became a dening theme. After two decades of intense activity, it has evolved into a broad research area rich in theory and potential applications. The progress has been rapid and vast, leading to the development of a variety of key concepts and techniques, among which notably are the H / theory, the Kharitonov/polynomial approach, and analyses based on state-space formulations and the Lyapunov theory. The structured singular value (2), also known as , was introduced in the early 1980s as a very general framework for studying structured uncertainty in linear time-invariant
models. It denes an exact measure of robust stability and performance in the frequency domain. The method is a natural progression of earlier work on using singular values to extend the concept of stability margin to multivariable systems, an idea that was heavily inuenced by operator theoretic results such as the Small Gain Theorem. The main impetus for the development of the structured singular value theory, evidently, has been the recognition that unstructured uncertainty is too crude a model, often leading to excessive conservatism. From a computational point of view, the success of hinges critically upon whether and how it can be computed both accurately and efciently. Unfortunately, this is known to be difcult. Recent studies have shown that computation of generally amounts to solving a so-called NP-hard decision problem, which, in the language of computing theory, is one that suffers from an exponential growth in its computational complexity. Although this by no means implies that every problem is computationally difcult, it nevertheless points to the unfortunate conclusion that the computation of in general poses an insurmountable difculty in the worst case. In retrospect, it thus comes as no surprise that the major progress in computing has been made by some forms of approximation, specically, readily computable bounds. While no denitive conclusion has been drawn concerning the gap between and such bounds, it is reassuring that the bounds are often reasonably tight and that they have other interpretations of engineering signicance. The Kharitonov theory is another robustness analysis approach, developed in parallel with , which deals almost entirely with robust stability issues under parametric uncertainty; only in rare cases is unstructured uncertainty also taken into consideration. The research in this area has a natural heritage from such a classical stability test as the RouthHurwitz criterion and was mainly triggered by a landmark result published by V. L. Kharitonov (3) in 1978, later referred to as Kharitonovs theorem. Kharitonov considered the question of stability of parameterized families of polynomials. Here polynomials are thought of as characteristic polynomials of systems described by rational transfer functions and thus are stable if their zeros all lie in the open left-half of the complex plane (continuous time) or in the open unit disk (discrete time). In the continuous-time case, Kharitonov showed that, for an uncertain interval polynomial whose coefcients each vary independently in a given interval, stability of the entire family can be assessed by testing merely four simply constructed extreme polynomials. From an aesthetic point of view, Kharitonovs theorem possesses remarkable elegance, reducing an otherwise seemingly impossible task to a simple problem. From an engineering perspective, however, the theorem is likely to nd only limited utility because very rarely would an uncertain system yield a family of characteristic polynomials of the form required in the problem description. Thus, Kharitonovs work triggered a urry of activities in the search for more realistic results, and soon came various generalizations. Two notable features stand out from the robust stability conditions available in this category. First, they are stated either in terms of a nite number of polynomials or as graphical tests requiring a frequency sweep. Second, the main success to date pertains to uncertainty descriptions in which polynomial coefcients depend linearly on uncertain parameters.
604
Robust stability analysis in state-space formulation often comes under the names of stability radius, interval matrix, and stability bound problems. The key issue here is to determine the largest uncertainty size under which stability is preserved. Unlike in the aforementioned two approaches, parametric uncertainty in state-space representation is dened in terms of perturbations to system matrices, which can be either unstructured or structured, but most often are only allowed to be real. In a natural way, the robust stability problem translates to one of how the perturbations may affect the eigenvalues of the system matrix, the solution of which can draw upon rich theories from linear algebra and Lyapunov analysis. Thus, unsurprisingly, most of the robustness conditions have this avor. Alternatively, the problem can also be recast as one of analysis or one of polynomial stability. In the former case, we need to compute with respect to solely real uncertainties, for which the computation schemes are known to be conservative. In the latter case, the coefcients in the resultant polynomial will depend on the unknown parameters in a multilinear or multinomial fashion. The Kharitonov approach cannot provide a satisfactory answer for such polynomial families either. The problem thus appears to be a very difcult one and is known to have been solved only for a number of isolated cases. Most notably, recent progress has made it possible to compute the stability radius in an efcient manner when the matrix perturbation is unstructured. For structured uncertainty, unfortunately, the problem is completely open; recent studies showed too that it is in general NP-hard. Accordingly, the majority of results in the latter class are sufcient conditions for robust stability. In conclusion, robustness is a key concept, vital for engineering design in general and for control system design in particular. Robust control has matured into a eld rich in theory and potential applications. By focusing on these three selected areas, this chapter is limited only to robustness analysis of linear time-invariant systems, that is, control of linear time-invariant plants by linear time-invariant controllers, which itself is condensed from a vast collection of results and techniques. Nevertheless, we should note that the concept and theory of robust control goes far beyond the boundary of linear time-invariant systems and, in fact, has been quickly branching to the domains of nonlinear control, adaptive control, and the like. As a whole, it has been, and will continue to be, a driving force behind the evolution of control theory.
Thus each plant in the model set corresponds to the selection of one element in each of the uncertain blocks. The nominal plant corresponds to the choice 0 for all elementary uncertain blocks. Note that the assumption that all the uncertainty balls have unity radius is made at no cost: any size information can be embedded into known blocks (e.g., connected in cascade with the uncertain blocks). It turns out that, for uncertain block diagrams of this type, the transfer function (or transfer function matrix) between any two nodes (or tuples of nodes) in the block diagram is given by a linear fractional transformation (LFT). Similarly, when such an uncertain plant is connected with a feedback compensator, the transfer function between any two nodes will also be an LFT. LFTs have attractive mathematical properties that can be used to advantage at the modeling, analysis, and synthesis stages. More on LFTs can be found, e.g., in Ref. 7. Robust Stability It should be intuitively clear that block diagrams of the type just described can always be redrawn in the form of an M loop as depicted in Fig. 1 (external input and outputs have been left out). Here M corresponds to the nominal system, which is comprised of closed-loop transfer functions as elements and has an input channel and an output channel for each elementary uncertain block, and is a block-diagonal matrix whose diagonal blocks are the elementary uncertain blocks. For its generality, the M loop paradigm has found wide acceptance in robust control (see, for example, Refs. 2 and 48). Throughout most of this article, we will assume that the nominal system, or equivalently M, is linear and time-invariant, as are all instances of the uncertainty blocks, equivalently, of . We will also assume that M and all instances of are in H . In this case, an immediate payoff of the LFT uncertainty description and the ensuing representation of the system via the M loop is the following strong form of the Small Gain Theorem, a necessary and sufcient condition for well-posedness and stability of the M loop, in the case where consists of a single uncertainty block, ranging over the unit ball in H . Small Gain Theorem. Let M H . Then the M loop is well-posed and BIBO stable for all H with 1 if and only if M 1. As alluded to earlier, the H norm of a causal, stable, continuous-time transfer function matrix M is dened as M
THE STRUCTURED SINGULAR VALUE Uncertain Systems Throughout this article, we consider model sets P with the following property: P can be represented by a block diagram with some of the blocks being fully known systems, and others being elementary uncertain blocks. The latter are elementary sets, namely, unit balls in simple vector spaces. For example, some uncertainty blocks might be the real interval [1, 1] and others might be the unit ball of H , the set of transfer function matrices (linear time-invariant systems) that are causal and bounded-input bounded-output (BIBO) stable; the size or H -norm of H is dened to be the supremum of its largest singular value over the imaginary axis (continuous time) or unit disk (discrete time).
= sup (M ( j))
where denotes the largest singular value.
Figure 1. M loop.
605
As an example consider a model set of the output multiplicative uncertainty type. Specically, let P = (I + w )P0 where P0 is the transfer matrix of a linear, time-invariant nominal model, w is a scalar weighting transfer function, and ranges over the unit ball in H . The weight w is introduced to account for the fact that the amount of uncertainty is usually frequency-dependent; in particular, system dynamics are often poorly known at high frequencies. Suppose that a feedback controller K has been tentatively selected to stabilize the nominal system P0. (We use the negative feedback convention, i.e., the loop transfer function is KP0.) Isolating from the nominal closed-loop system, we obtain an M loop with M = wKP0 (I + KP0 )
1
namics, consider now the model set (1 w1c)Pa with Pa explicitly given as Pa (s ) = 1 sa
where a can take any value in the interval [0.9,1.1]. We may write a 1 w2r with w2 0.1 and r 1, and Pa can be represented as a loop with P0(s) 1/(s 1) in the forward path and w2r in the feedback path (again using the negative feedback convention). Let K be a feedback controller (still with the negative feedback convention) that stabilizes P0. By extracting the uncertainty blocks c and r, we obtain an M loop with
M= =
w2 P0 (I + P0 K ) 1 w1 P0 (I + P0 K ) 1
r
w2 KP0 (I + P0 K ) 1 w1 KP0 (I + P0 K ) 1
Since K stabilizes P0, M is stable. As a note of interest, we expect a keen connection between the Small Gain Theorem and the classical Nyquist criterion. Indeed, this can be best observed by examining single-input/ single-output systems. In such case P and K are scalar, and thus so is M. Since both M and are stable, Nyquists criterion implies that the M loop is stable whenever the Nyquist plot of M does not encircle the critical point 1 j0. Clearly, this will be the case for every satisfying ( jw) 1 for all w if and only if M( jw) 1 holds at all frequencies (including ). The Small Gain Theorem suggests that one way to obtain a robustly stable system, or more generally to obtain a robust design, is to make sure that the H norm of a certain system transfer function is small enough. This has triggered an entire eld of research known as H design, which is discussed elsewhere in this encyclopedia. The focus of the present article is the case when is block-diagonal (i.e., when the uncertainty model consists of several blocks or, in other words, when the uncertainty is structured). Typically, two types of uncertainty blocks are considered in the literature: (i) the set of real, constant scalar multiples of the identity, with the scalar having magnitude no larger than one, and (ii) the set of causal and BIBO stable (H ) transfer function matrices, with H -norm no larger than one. The latter corresponds to unmodeled dynamics. The former, on the other hand, is used to represent parametric uncertainty, particularly when a same uncertain parameter affects more than one coefcients in a transfer function. For example, concurrent variation as a function of temperature (e.g., dilation) of multiple quantities in a mechanical system can result in such a block. This description is more general than the simpler scalar nonrepeated blocks. Examples with structured uncertainty arise with plants modeled as being affected by uncertainty at more than one physical location, e.g., P = ( I + w1
1 )P0 (I
0
c
where r is a real number ranging over [1,1] and c is a scalar transfer function ranging over the unit ball in H . Clearly, the condition M 1 in the Small Gain Theorem remains sufcient for robust stability when is restricted to be block-diagonal. However, it is in general no longer necessary. A renement of the Small Gain Theorem for the structured uncertainty case was proposed by Doyle and Safonov in the early 1980s (2,4). We adopt here the framework introduced by Doyle, that of the structured singular value also known as . The Small Theorem states that, if the uncertainty is restricted to be block-diagonal, then the correct renement is essentially (see Ref. 9 for a precise statement) to replace M with M, where for a continuous-time transfer function matrix M, M
= sup (M ( j))
e
and ( ) denotes the structured singular value of its matrix argument with respect to the block-structure under consideration. The set e is the extended real line; if no parametric uncertainty is present, however, ( ) is continuous and e can be replaced by . Similarly, for a discrete-time system, M
= sup (M (e j ))
[0 , 2 )
But what specically is ? This is discussed next. The Structured Singular Value Let us denote by the set of values taken by ( j) [or (ej)] as ranges over the set of block diagonal transfer function matrices of interest, with the unit ball restriction lifted, namely let = {diag(
r
+ w2
2)
where both input and output uncertainty are accounted for. Another instance arises in the context of the robust performance problem, discussed in a later section. For an example including both parametric uncertainty and unmodeled dy-
):
r,
c,
C}
606
with
r c C r r := {diag(1 Ik , . . ., m I ) : ir R} r km
1 r
tightened to the following (12):
max R (MQ ) (M )
C}
QQ
:=
c c {diag(1 Ik , . . ., m I ) c km r +mc mr +1 C 1 , . . ., C mC )
ic
DD + , GG
inf
{ 0 : MDM H + GM H MG 2 D < 0}
(2)
:= {diag(
c i
k m + m + i k m + m + i r c r c }
The rst and third block types, often referred to as repeated real and full complex blocks, correspond to values of parametric and dynamic uncertainty, respectively. The second block type, known as repeated complex, often arises in analyzing multidimensional (10) and time-delay systems (11), and is also used sometimes when an LFT state-space representation of transfer functions is sought (5). It is worth noting that while as just dened is usually adequate for representing uncertainties frequently encountered, it can be extended further to accommodate more general situations. For example, full real blocks (i.e., unknown real matrices) may be added whenever desired. The structured singular value (M) of a matrix M with respect to the block structure is dened to be 0 if there is no such that det(I M) 0, and (M ) = min{ ( ) : det(I M ) = 0}
Here the superscript H indicates complex conjugate transpose, Q is the subset of consisting of matrices whose complex blocks are unitary, D is the subset of D consisting of Hermitian positive denite matrices, G is the subset of D consisting of skew-Hermitian matrices (i.e., GH G) with zero blocks in correspondence with repeated real blocks in , and the sign indicates that the matrix expression is constrained to be negative denite. The lower bound in condition (2) is always equal to (M). The upper bound is never greater than that in condition (1) (it reduces to it when G 0 is imposed) and, as was the case for condition (1), can be computed by solving an LMI problem. See, for example, Section 8.12 in Ref. 8. For the class of problems where the matrix M has rank one, Young (13) showed that the right-hand side in inequalities [Eq. (2)] is equal to (M). In that case, Chen et al. (14) obtained an explicit formula for (M). Let M baH, where a and b are column vectors. Let also a and b be partitioned into subvectors ai and bi compatibly with . For i 1, . . ., mr mc, let i aiHbi. Moreover, dene
mr +mc
otherwise. It can be checked that for structures simply consisting of one full complex block as in the Small Gain Theorem, (M(j) becomes the largest singular value of M(j), and M is thus equal to M. Given a matrix M and a block structure , computing (M) is generally not an easy task. Indeed, this computation is known to be NP-hard, even when is simplied to a structure containing only full complex blocks. Thus estimates of (M), e.g., upper and lower bounds on (M), are often used instead. These, as well as other properties of , are discussed next. Let U be the set of unitary matrices in and D be the set of nonsingular matrices that commute with every . The latter consist of block-diagonal matrices with scalar multiples of the identity in correspondence with full complex blocks (C), and with arbitrary blocks in correspondence with those constrained to be scalar multiples of the identity (r, c). Then the following result holds: max R (MU ) (M ) inf (DMD 1 )
U U DD
=
i= m r + 1
|i | +
mr +mc +mC
ai
i= m r + m c + 1
bi
Then,
(M ) = inf
x
mr i= 1
|Re(i ) + x Im (i )| +
!
2
1+x
(3)
Furthermore, if we assume with no loss of generality that for some l mr, Im(i) 0 for 1 i l, and Im(i) 0 for i l, and that Re(l ) Re(1 ) Im (1 ) Im (l )
then the inmum is achieved at one of the following points:
(1)
xk = x0 = xk =
Re(k ) , Im (k ) 2
k = 1, . . ., l |Im (i )|
l i= 1
Here R is the largest absolute value of a real eigenvalue of its matrix argument. Inequalities [see Eq. (1)] are of special interest in the case of purely complex uncertainty structures. In that case, (i) the lower bound is equal to (M) and R can be replaced by the spectral radius , and (ii) the upper bound is equal to (M) whenever mC 2mc is no greater than 3, and extensive numerical experimentation suggests that it is never (or at least seldom) much larger. Moreover, the upper bound can be computed efciently by solving a convex optimization problem, in fact, a linear matrix inequality (LMI) problem. LMIs dene a special class of convex optimization problems and are discussed elsewhere in this encyclopedia. For uncertainty structures where real (scalar multiple of the identity) blocks are present, inequalities [Eq. (1)] can be
; ;
l i= 1
|Im (i )|
2 l i = k + 1 |Im (i )| 2 l i = k + 1 |Im (i )|
k i= 1
|Im (i )|
k i= 1
|Im (i )|
Finally, the inmum cannot be achieved at x0 unless x0 (, x1], and for k 1, . . ., l, it cannot be achieved at xk unless xk [xk, xk1]. The rank-one case just alluded to is one of the rare instances for which one can obtain an explicit expression for (M). This expression not only simplies the computation of the upper bound in condition (2) but also was found useful in studying robust stability of uncertain polynomials. Indeed, as
607
1 2 M
1 2
blocks p(s) such that p 1. It thus follows that robust performance holds if and only if
e
sup (M ( j)) < 1
Figure 2. Robust performance setup.
will be discovered shortly, an important class of stability problems for uncertain polynomials can be formulated in terms of a rank-one problem. Consequently, the result furnishes a unifying tool for the stability problems and a link between analysis and the Kharitonov approach to robustness analysis. Robust Performance A key reason for the popularity of the framework is that it encompasses not only the robust stability problem but also the following robust performance problem: determine whether, for all plants in the given model set, the energy (integral of the square of the magnitude) in a specied error output signal remains below a specied threshold whenever the disturbance inputs energy is less than a specied value. Consider the block diagram of Fig. 2 where, as compared to Fig. 1, external (disturbance) input and (error) output are made explicit. Given a block diagram such as the one of Fig. 2, the input-output transfer function in the continuous-time case is given by the linear-fractional transformation F (M (s ), (s )) = M11 (s ) + M12 (s ) (s )(I M22 (s ) (s )) 1M21 (s ) where Mij(s) is the transfer function from input j to output i of M(s), i,j 1,2, when the feedback connection through (s) is removed. [Thus M22(s) is the transfer function matrix formerly denoted M(s).] The issue at hand is to determine, under the assumption that M(s) H , whether robust performance holds, that is, whether it is the case that, for all (s) in our unit uncertainty ball, F (M (s ), (s ))
where now denotes the structured singular value corresponding to the augmented block structure diag(kpkp, ) (i.e., the block-structure corresponding to the actual uncertainty, augmented with a full complex block). For an example of a typical robust performance problem, consider an uncertain plant described by the multiplicative uncertainty model set P = (I + w1 )P0 with a xed feedback controller K. It is desired to determine whether w2S 1 for all possible in a possibly structured unit uncertainty ball, where S is the sensitivity function (i.e., using the negative feedback convention, S (I PK)1). Here w1 and w2 are stable transfer functions introduced for frequency-weighting purposes. For simplicity, w1 and w2 are assumed to be scalars. Using the transformation just outlined, we obtain
M=
w2 (I + P0 K ) 1 w1 (I + KP0 ) 1 K
w2 (I + P0 K ) 1 P0 w1 (I + KP0 ) 1 KP0
(5)
In the single-input/single-output case, M has rank one. In the present case mr mc 0, so the right-hand side of Eq. (3) is simply and the right-hand side of the expression dening reduces to its second term. Thus
(M ( j )) = |w2 ( j )(1 + P0 ( j )K ( j )) 1 | + |w1 ( j )(1 + K ( j )P0 ( j )) 1 K ( j )P0 ( j )|

and the condition for robust performance can be expressed as |w2 ( j )| + |w1 ( j )K ( j )P0 ( j )| < |1 + P0 ( j )K ( j )| Re Extensions The structured singular value may be further generalized in many directions, depending on uncertainty descriptions and characterizations. Some of these generalizations are summarized next. Nondiagonal uncertainty structure. The uncertainty structure need not be diagonal. It can contain unknown, independently bounded blocks in every entry. Doyle (2) and Kouvaritakis and Latchman (15) showed that the analysis may be converted into one based on the standard , but this may lead to a substantial increase in computational effort. Chen et al. (16,17) proposed a computational scheme that renders the computation growth insignicant. Uncertainty with phase information. Tits et al. (18) adopted a notion of with phase, in which not only are uncertainties known to be bounded by given quantities, but also their phases are known to vary in given ranges. The formulation gives a more detailed uncertainty de-
<1
(4)
This is readily handled by noting that, in view of the Small Gain Theorem, for any xed (s) such that the system is stable, condition (4) is equivalent to the stability of the augmented system depicted in Fig. 3 for all ctitious uncertainty
1 2 M
1 2
p
Figure 3. Fictitious uncertainty for robust performance.
608
scription, and it requires extensions of the concept of phase and of . 1-norm bounded uncertainty. Khammash and Pearson (19,20) studied structured uncertainties bounded in 1 norm, which is another active research area in robust control, concerning peak-to-peak system response. They showed that robust stability can be assessed by computing the spectral radius of a positive matrix constructed from the impulse response of the nominal system. Time-varying uncertainty. Shamma (21) and Megretsky (22) examined the robust stability problem with respect to structured time-varying uncertainties. They showed that if the uncertainty is allowed to vary arbitrarily over time, robust stability holds if and only if for some D D , DM(s)D 1. It is readily checked that the lefthand side (known as scaled H -norm), is similar to the right-hand side in condition (1), except that here the same D must be used at all frequencies. Subsequently, Poolla and Tikku (23) showed that, if the time variation of the uncertainty is arbitrarily slow, then robust stability holds if and only if the right-hand side in condition (1) is less than 1 at all frequencies. Finally, while may be custom made and seems to be an all-encompassing paradigm when extended appropriately, it cannot be applied to models in which the uncertainty block is allowed to be unstable. An effective robustness measure for the latter situation is furnished by the gap metric, a concept discussed elsewhere in this encyclopedia.
where q is an unknown vector that may or may not represent physical parameters. When q varies over a bounded set Q m, a family of polynomials are generated: P = { p(s, q ) : q Q} (7)
The problem of concern is to determine if the polynomial family P is robustly Hurwitz stable, by which we mean that every member in P is Hurwitz stable. We shall assume that the coefcients ak(q) are continuous functions of q. Furthemore, we assume that an(q) 0 for all q Q (i.e., all polynomials in P have the same degree). For control system analysis, it is typical to restrict the polynomial family P to the following classes, arranged by order of increased complexity. 1. P a: the coefcients ak(q) are afne functions of q. For example, p(s, q ) = s2 + (q1 + 2q2 + 3 )s + (4q1 + 5q2 + 6 ) 2. P m: the coefcients ak(q) are multiafne functions of q. For example, p(s, q ) = s3 + (2q1q2 + 2q1 q3 + q3 + 1 )s2 + ( 4 q 2 q 3 + 5 )s + ( q 1 q 2 q 3 + 1 ) 3. P p: the coefcients ak(q) are multivariate polynomials in q. For example,
2 2 p(s, q ) = s3 + (2q2 1 q 2 + 2 q 1 q 3 + q 1 q 3 + 1 )s
THE KHARITONOV APPROACH The Kharitonov approach, named after Russian mathematician V. L. Kharitonov whose celebrated 1978 theorem is often considered to be the cornerstone of the eld, is largely concerned with the issue of determining zero locations for a family of polynomials whose coefcients vary in a bounded set. Hence, by nature, it can be best presented in a framework different from that of the M loop, or , namely, directly as a polynomial stability problem. This issue, however, is connected to the M loop paradigm in an intimate fashion. To see this, simply consider a model set comprising proper real rational functions whose coefcients take values in certain bounded intervals. To determine robust stability of such a plant together with any compensator will then amount to checking whether the set of all resultant closed-loop characteristic polynomials have zeros in the stability region. For continuous-time systems, our main focus, the stability region of interest, is the open left half of the complex plane. Other regions of interest include the open unit disk, a shifted left half plane, and a sector; these regions can be imposed to study stability of discrete-time systems or to enforce pole placement constraints. A polynomial is generally said to be Hurwitz stable, or is referred to as a Hurwitz polynomial, if its zeros lie in the open left half plane. A general description for a set of polynomials of interest is
n
2 2 + ( 4 q 2 q 3 + 5 )s + ( q 2 1 q2 q3 + 1 )
It should be rather evident that P a P m P p and hence that the complexity in analysis increases in that same order. At present, the only available methods for tackling P m and P p are largely ad hoc, via either local optimization or graphical approaches, and they are either conservative or computationally formidable. In particular, when Q is an ball [i.e., a hyperrectangle (box) parallel to the coordinate axes], the problem of testing the stability of P m is known to be NP-hard. The class P a, as the sole tractable case, merits a particularly thorough study. A polynomial family P in this class consists of all polynomials of the form
m
p(s, q ) = p(s, q0 ) +
k=0
(qk q0 k ) pk (s )
(8)
Here q0 belongs to Q and may be regarded as the nominal value of uncertain parameter vector q, and the pk(s)s are xed polynomials. Evidently, one can assume with no loss of generality that p(s, q0) is Hurwitz stable, which is necessary for P to be robustly Hurwitz stable as q varies over Q. Let p [1, ], and let p be the standard p Ho lder norm dened on the Euclidean space m. That is,
p(s, q ) =
k=0
a k ( q )s ,
k
q
(6)
m p
!1/ p
, 1 p< p=
|qi | i= 1 max |qi |,

1im
609
Then a common description for Q adopts the notion of unit p balls centered at q0, dened as Q = {q : q q 0
p
1}
(9)
When the coefcient ak(q) depends on qk alone [i.e., when pk(s) is a constant multiple of sk], the p ball description gives rise to a class of most studied polynomial families. Such families can be expressed as
n
Because p(s, q0) is stable and by assumption an(q) never vanishes on Q, we can conclude from the continuity properties of the zeros of polynomials that if some member in P is not Hurwitz stable, then some polynomial in P must have a zero on the imaginary axis. Thus the entire family P is Hurwitz stable if and only if 0 / p( j , Q ),
P =
k=0
k qk s : q q
k
(10)
for some (possibly different) q0 and some xed scalars k. In particular, for p , 1, 2, such P is referred to as an interval polynomial, a diamond polynomial, and a spherical polynomial, respectively. Arguably, the entire success of the Kharitonov approach may be best summarized as a triumph over the polynomial family in Eq. (8) with the p norm characterization in Eq. (9), but not beyond, of which the most shining example is Kharitonovs celebrated theorem. Interval Polynomials An interval polynomial can be equivalently expressed as
n
that is, the value set never contains the origin. This is a rather straightforward fact known as early as in 1929 and is generally referred to in the literature as the zero exclusion principle. Note, in particular, that for a polynomial p(s, q) in the family described by Eq. (11), for any q Q, the real and imaginary parts of p( j, q) are respectively given by Re p( j , q ) = q0 q2 2 + q4 4 and Im p( j , q ) = j (q1 q3 2 + q5 4 ) Thus the real part (resp. imaginary part) of p( j, q) depends only on the parameters with even (resp. odd) subscript. For an interval polynomial, therefore, it becomes clear that Re K1 ( j ) = Re K2 ( j ) Re p( j , q ) Re K3 ( j ) = Re K4 ( j )
P =
k=0
q k s k : q k q k qk
(11)
Im K1 ( j ) = Im K4 ( j ) Im p( j , q ) Im K3 ( j ) = Im K2 ( j ) Because the four Kharitonov polynomials are themselves in P , it follows that, at each , the value set is a rectangle in the complex plane with vertices Ki( j), as depicted in Fig. 4. As increases, the rectangle evolves in a continuous fashion. Its dimensions vary, but its edges remain parallel to the axes, and the relative positions of the Ki( j)s do not change. Now, a polynomial p(s) of degree n with positive coefcients is Hurwitz stable if and only if the phase of p( j) monotonically increases from 0 to n /2 as goes from 0 to . This is best seen by noting that p(s) can always be factorized as p(s) an(s si) in terms of its zeros si, and by considering the phase of each factor separately. When this is understood, Kharitonovs Theorem becomes self-evident. Indeed, if the Kis are Hurwitz stable, then the phase of each Ki( j) increases monotonically from 0 to n /2 and the entire Kharitonov rectangle rotates around the origin by a total angle of n /2 without ever touching it. Stability of P then follows from the zero exclusion principle. Kharitonovs Theorem opened an era epitomized by the search for so-called vertex results in robust stability analysis, as manifested by the construction of the four Kharitonov polynomials at the vertices of the hyperrectangle Q. This theme
where the qks and the qks are xed. Kharitonovs original treatment of the interval polynomial problem is of an algebraic nature. He constructed four extremal members of P ,
K1 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K2 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K3 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 + K4 (s ) = q0 + q1 s + q2 s2 + q3 s3 + q4 s4 + q5 s5 +
later dubbed Kharitonov polynomials. In a remarkable fashion, Kharitonov showed that the stability of the entire interval polynomial family can be ascertained by testing merely these four. Kharitonovs Theorem. The interval polynomial is Hurwitz stable if and only if K1(s), K2(s), K3(s), and K4(s) are all Hurwitz stable. Subsequent development showed that for polynomials of degree 5, 4, and 3, the test can be further simplied, requiring checking only 3, 2, and 1 of the four Kharitonov polynomials, respectively. Dasgupta (24) gave a geometrical interpretation to Kharitonovs Theorem in frequency domain, which sheds light on why such a puzzling result would hold. The interpretation makes use of the concept of value set. The idea is to examine the image of the polynomial family when s lies on the boundary of the stability region, namely p( j , Q ) = { p( j , q ) : q Q}
Im K2(j) K1(j) K3(j) K4(j) Re

Figure 4. Kharitonov rectangle.
610
persisted in much of the subsequent work. Although Kharitonovs Theorem is without question a milestone in control theory, and perhaps a measured intellectual triumph in general, we should note that the interval polynomial family is nevertheless a very special instance of an afne uncertainty class, and hence the theorem has a rather limited scope in application. The quest thus continues. Edge Theorem In light of Kharitonovs Theorem, it is tempting to contemplate the possibility that a similar vertex result would hold for the general afne polynomial family with Q a hyperrectangle. This, unfortunately, turns out to be false; a counterexample can be readily constructed to demonstrate the opposite. What can be said about a polynomial family P in class P a? Bartlett et al. (25) provided an answer. In what is now known as the Edge Theorem, they took the bounding set Q to be a convex polytope in m. Let qi, i 1, . . ., l, be the vertices of Q. Then, it is well known that Q can be represented as a convex hull of the vertices. That is,
l l
the polytope because some of these segments are not edges. In some cases, however, it is not an easy task to check which such segments are edges and which are not. The Edge Theorem generalizes Kharitonovs Theorem in two important aspects. First, it is applicable to general afne polynomial families, with Q an arbitrary polytope. Secondly, the stability region may be any open, simply connected set (27,28), although only stated here in terms of the open left half plane. In contrast, Kharitonovs Theorem addresses only Hurwitz stability. On the other hand, as the number of parameters increases, the computation effort required in the Edge Theorem can be enormous. For example, when Q is a box in m, with 2m vertices, the number of edges is m2m1, a very large number even for moderate values of m. Finally, to a lesser extent, the Edge Theorem is not applicable to situations where the boundary of Q is curved, of which the spherical polynomial family is an example. Graphical Tests The complexity of the Edge Theorem provides a direct motivation to search for computationally more tractable stability criteria, and graphical conditions become a natural avenue to explore. Not only are graphical criteria time-honored tools in classical stability analysis, but the zero exclusion principle, with all its simplicity and transparency, also prompts a deeper investigation of such tools. We can generally feel that an important asset of the zero exclusion principle is its generality, both in terms of uncertain polynomial families and in terms of stability regions. Barmish (29) studied the issue systematically; earlier conditions had appeared sporadically on stability of perturbed polynomials in isolated cases. Barmishs approach stems from a geometrical argument: a convex polygon in the complex plane does not intersect the origin as long as it can be separated from it by a straight line or, equivalently, as long as the vertices can be separated, as a whole, from the origin by such a line. This observation led to his construction of a testing function, which is to be evaluated along the boundary of the stability region. After this is accomplished, we can determine stability by plotting the testing function. Barmishs test is certainly one step forward compared to a pure brute-force computation; however, it remains somewhat ad hoc and is computationally overwhelming. Because it requires evaluations at all vertices, it does not clear the hurdle of exponential growth in complexity. On a lesser scale, Tsypkin and Polyak (30) obtained a graphical test for a simpler problem. They examined the polynomial family in Eq. (10). Let p [1, ] be given. Furthermore, for r [1, ] such that (1/ p) (1/ r) 1, dene
r Xr () = (0 + (2 2 )r + (4 4 )r + )1/r r Yr () = (1 + (3 2 )r + (5 4 )r + )1/r 0 2 0 4 R() = 0 q0 0 2 q2 + 4 q4 + 0 2 0 4 I () = 1 q0 1 3 q3 + 5 q5 +
Q = conv{q1 , . . ., ql } =
i= 1
i q i :
i= 0
i = 1, i 0
Because ak(q) is an afne function of q, it follows that P = conv{ p(s, q1 ), . . ., p(s, q l )} This implies that the value set p( j, Q) is a polygon, generated by the vertex polynomials p(s, qi). It should be rather clear that the interval polynomial family is generated by a polytopea hyperrectangleand so is the diamond polynomial family. Edge Theorem. The afne polynomial family (polytope of polynomials) P a is Hurwitz stable if and only if for each edge point q of Q, p(s, q) is Hurwitz stable. Let qi and qj be two vertices of Q connected by an edge. When varies from 0 to 1, the polynomial pij (s, ) = p(s, q i ) + (1 ) p(s, q j ) denes a line segment (of polynomials) connecting p(s, qi) and p(s, qj). The theorem shows thatquoting directly from Ref. 25 it sufces to check the edges. Because an edge polynomial involves only one parameter, its stability can be readily tested, by resorting to either a graphical test based upon a root locus, or the numerical solution of a generalized eigenvalue problem (see, for example, Chapter 4 in Ref. 26). The heuristics behind the Edge Theorem are simple. Because for large values of , the value set does not contain the origin (this is easy to see), the polynomial family will be Hurwitz stable if and only if there is no frequency at which the origin belongs to the boundary of the (polygonal) value set. It should be intuitively clear that every point on the boundary of the value set must be the image of a point on an edge of the polytope of polynomials. Thus, if for some the origin does belong to this boundary, the corresponding edge polynomial must be unstable. Note that it is wasteful to check the entire set of all polynomial segments joining two vertices of
Note that R() Re(p( j, q0)) and I() Im(p( j, q0)). Tsypkin-Polyak Criterion. The polynomial family P of Eq. (10) is Hurwitz stable if and only if p(s, q0) is Hurwitz stable,

0 0 qn 1, q0 1, and
611
|R()| Xr ()
|I ()| Yr ()
> 1,
( 0, )
(12)
This condition was independently obtained by Hinrichsen and Pritchard (31). Up to this point, we could assert that the stability problem for afne polynomial families remains largely unresolved. However, as yet another observation, we nd that at each , the zero exclusion condition denes two linear constraints in terms of perturbed coefcients, imposed on the real and imaginary parts of p( j, q), respectively. These constraints, together with a convex bounding set Q, dene in turn a convex feasibility condition; when the parameters vary independently, it reduces further to a linear program. This is a simple but conceptually appealing observation. It led to a reformulation via linear programming, due to Saridereli and Kern (32) and to Tesi and Vicino (33), which can be solved readily for each and then plotted graphically. Qiu and Davison (34) went further to demonstrate that for very general bounding sets it sufces to solve an optimization problem with one variable only, and the problem can be solved explicitly in special cases. Finally, Chen et al. (14,35) recognized that the problem can be reformulated as a special rank-one problem for each , and showed that stability can be ascertained by evaluating an explicit formula. These results led to the nal resolution of the afne polynomial family.
with respect to the block structure diag(1, . . ., m): i . In light of the formula for rank-one , an explicit condition can then be stated. Such a result clearly applies to general stability regions, and it furnishes a frequency sweeping condition for robust stability. Note that we may interpret this result alternatively based upon the zero exclusion principle. Indeed, under the condition that Q is the unit ball centered at q0, all the polynomials in P will have zeros in a specied region if and only if the zeros of p(s, q0) are in that region and (M(s)) 1 for all s on the boundary of the region. This follows because, according to the zero exclusion principle, it is both necessary and sufcient that
m
min
q q0
p(s, q0 ) +
k=0
(qk q0 k ) pk (s ) = 0 > 1
and the Kharitonov Approach

Indeed, there is an inherent linkage between analysis and Kharitonov approach, whenever the latter applies, in that both approaches yield necessary and sufcient conditions for problems of the same nature. However, for a rather long time a clear link seemed elusive. The main cause, it seems, lay in how to reconcile an optimization-based formulation such as , and explicit results from the Kharitonov approach. Can one, for example, derive Kharitonovs theorem from , or vice versa? The explicit formula of rank-one given earlier lends an answer. Specically, for a general afne polynomial family P a [(Eq. (8)] with Q the unit ball, robust stability can be checked by computing , with a rank-one matrix M(s) constructed as 1 pm (s ) . p1 (s ) M (s ) = . . p(s, q0 ) p(s, q0 ) 1 To see this, observe that p(s, q) in Eq. (8) is the characteristic polynomial of the M loop of Fig. 1 with diag(q1 0 0 q1 , . . ., qm qm ). Indeed,
m
in order for the polynomial family p(s, q) in Eq. (8) to have no zero on or exterior to the boundary of the region, for all possible q Q. More generally, it is possible to extend the denition of by means of more general norms and to use this extended to study the robust stability of an afne family P with a more general bounding set Q. Such a generalization also leads to a similar expression when the M matrix in question has rank one (14,35). In particular, when the stability region is restricted to the open left half plane, and Q is the unit p ball centered at q0 with ak(q) kqk, the expression for the generalized rank-one , denoted as p( ) for purpose of distinction, is found to be Yr ()/|I ()| if R() = 0 if I () = 0 Xr ()/|R()| p (M ( j )) = () Y () X r r p p p p 1/ p (Xr ()|I ()| + Yr ()|R()| ) otherwise which leads to a similar condition for robust Hurwitz stability. This condition is slightly more general than, but essentially replicates, the graphical criterion by Tsypkin and Polyak. Note that for p , the polynomial family becomes an interval polynomial, and the stability condition reduces to checking whether p(s, q0) is Hurwitz stable, q0 1, and min X1 () Y1 () , |R()| |I ()| < 1, ( 0, )
A little thought reveals that the latter is equivalent to determining whether one of the four conditions Re(K1( j)) 0, Re(K3( j)) 0, Im(K1( j)) 0, and Im(K3( j)) 0 holds. Clearly, this is further equivalent to the requirement that the rectangular value set in Fig. 4 never contains the origin. Extensions There is an immense body of literature devoted to polynomial stability problems. Various extensions to Kharitonovs Theorem have been obtained. They generally fall into the categories of vertex results and frequency-sweeping conditions, consisting of delicate studies and intricate technical details. We summarize some of the highlights next. A recent and comprehensive account can be found in the books by Barmish (26) and by Bhattacharyya et al. (36).
det(I
M (s )) = 1 +
k=0
pk (s ) p(s, q ) (qk q0 k) = 0 p(s, q ) p(s, q0 )
Thus, stability of p(s, q) for all q Q is equivalent to stability of the M loop for all diagonal matrices with real (parametric) entries lying in [1, 1]. The condition for this is that M 1 where the structured singular value is computed
612
Vertex and Edge Results. Much of the work in this direction continues the thread in Kharitonovs Theorem, focusing on simple uncertainty descriptions and leading to stability tests based on vertex and/or edge polynomials. Some notable examples follow. Complex Interval Polynomials. The polynomial family has complex coefcients whose real and imaginary parts are allowed to vary independently in given intervals. Eight vertex polynomials need to be tested to ascertain stability. Diamond Polynomials. This polynomial family is described in Eq. (10), with p 1. Eight vertex polynomials are required as well. Stabilization of Interval Plants via First-Order Compensators. The numerator and denominator of the plant transfer function are interval polynomials, and it is to be stabilized by a rst-order compensator in closed loop. It sufces to stabilize 16 vertex plants, constructed based upon the vertex numerator and denominator polynomials. Generalized Kharitonov Theorem. It concerns linear combination of interval polynomials and requires checking certain polynomial segments in addition to vertices. Other stability conditions based on vertex polynomials are also available. As the complexity of uncertainty structure increases slightly, they usually require testing more (e.g., 32 or 64) vertex polynomials. A clear insight concerning uncertainty structure and the required number of vertices, however, remains unavailable. Performance Issues. The entire Kharitonov theory is largely successful for determining stability of uncertain polynomials. However, a number of results are also available regarding properties of transfer functions, which have implications toward performance issues. Some examples follow. H Norm of Interval Transfer Functions. When the numerator and denominator of a transfer function are both interval polynomials, the H norm of the transfer function can be computed over 16 vertex transfer functions, provided that the four Kharitonov polynomials associated with the denominator are stable. Peak Magnitudes of Closed-Loop Transfer Functions. The peak H norm of closed-loop transfer functions can be computed over the edges of the plant family, when it is an interval plant. Nyquist and Bode Envelopes. The Nyquist and Bode plots of open or closed-loop transfer functions associated with an interval plant lie in envelopes determined by plots generated by vertex and edge plants. Other Extensions. Additional extensions may be found in the following categories. Schur Stability. The Kharitonov theory has been extended with varying degrees of success to other stability regions, such as the unit circle. These results are useful for studying stability of discrete-time systems and for addressing other performance issues.
Unstructured Uncertainty. Unmodeled dynamics may be included along with parametric uncertainties. They may be accommodated either in the rank-one formula or by small gain-type conditions involving vertex and edge plants. Nonlinear Systems. In a system consisting of an interval plant and a static, sector bounded nonlinear component, stability conditions similar to the Popov and circle criteria have been obtained, which also require the testing of vertex and edge plants. Multilinear Uncertainty Structure. The entire success in the Kharitonov approach relies on the key assumption that polynomial coefcients depend linearly on uncertain parameters, and the utility of all the results in this area is thus measured by how much the uncertainty can deviate from this description. Little success has been achieved in this endeavor. A fundamental barrier, as implicated by the zero exclusion principle, is that the stability problem is one of optimization subject to nonlinear, nonconvex constraints. STATE-SPACE APPROACH Dynamical systems are often described by state-space equations. Accordingly, it is common to model system uncertainty as perturbations to system matrices. An uncertain continuous-time system in this spirit may be described by x (t ) = (A + B C )x(t ) (13)
Here A, B, and C are known matrices of appropriate dimensions. The matrix A is assumed to be stable. The system uncertainty is represented by a set of allowed values for the real matrix , which may be unstructured or structured. Typical perturbation classes considered in the literature are as follows, arranged in increasing order of generality. In all cases, 0 is given. Unstructured Perturbation. The set consist of all real matrices with spectral norm less than a given number:
U
= { real : ( ) }
Element-by-Element Perturbation. Each element in varies in a given interval. Let rij 0 be given. The set is dened as r11 11 r1m 1m . . . . . real : = E = . . . . , rn1 n1 rnm nm = max{|ij | : rij > 0} i, j Linear Combination. The allowable set of perturbations is described by
k L
real:
=
i= 1
i Ei ,
= max |i |
i
where the Eis are given.
613
Evidently, an uncertain discrete-time system can be described in exactly the same manner. The problem of interest is to determine the size of the perturbation matrix, measured by a norm of choice, so that the system remains stable. This issue naturally translates into one concerning how the eigenvalues of a stable matrix A would vary when it is perturbed by . More specically, would the eigenvalues cross the stability boundary? And if they do, what is the minimal such that at least one of the eigenvalues leaves the stability region? These questions may be addressed by examining the characteristic polynomial (s, ) = det (sI A B C )
There essentially exists no result for the structured stability radius other than those already known for . For the unstructured stability radius, much of the early work was devoted to derivation of bounds. One representative example is r(A, B, C ) 1 C(sI A ) 1 B (14)
This, of course, is a rather straightforward consequence of the Small Gain Theorem. Recently, however, Qiu et al. (37) obtained the following exact, readily computable formula. Real Stability Radius. Let G(s) C(sI A)1B, and 2( ) be the second largest singular value. Then, Re[G(s )] Im[G(s )] r(A, B, C ) 1 = sup inf 2 1 Im[G(s )] Re[G(s )] s D (0 , 1] The signicance of this result lies in that for any s , the function 2( ) is unimodal in over (0, 1), and hence its inmum can be computed effectively. Furthermore, when is the open left half plane or the open unit disk, that is, when Hurwitz or Schur stability is of concern, Sreedhar et al. (38) developed a fast-converging algorithm for the maximization with respect to s. Consequently, from a computational standpoint, the unstructured stability radius problem can be considered largely resolved. Interval Matrices An interval matrix is a family of real matrices in which all elements are known only within certain closed intervals. In precise terms, the interval matrix AI [A, A] is the set of matrices dened by AI = {A : a ij aij aij } that is, each aij of A is conned elementwise to lie within an interval determined by aij and aij, the corresponding elements of A and A, respectively. An interval matrix AI is said to be stable if every A AI is stable. Evidently, interval matrix and set E share the same uncertainty description. Interval matrices are direct matrix analogues of interval polynomials, and hence there has been a lingering temptation for extension of Kharitonovs Theorem to the former. Unfortunately, neither vertex nor edge results exist for interval matrices. In fact, more recent studies showed that in order to determine stability of an interval matrix, we must solve an NP-hard decision problem. This in a way explains why only sufcient stability conditions are available. One approach of attack is to analyze eigenvalue distribution. Heinen (39) and Argoun (40) examined the problem on the basis of Gershgorins Theorem, and their developments culminated in a subsequent work of Chen (41), leading to a number of simple, albeit conservative, stability conditions. As a representative example of these results, consider an interval matrix AI such that aii 0. Let W be constructed as 0 i= j max{|a ij |, |aij |} W = [wij ], wij = i= j |aii |
or equivalently, the characteristic equation det(I C(sI A ) 1 B ) = 0 Thus, it becomes clear at the outset that the problem may be tackled in principle by using a polynomial approach. Similarly, it can also be analyzed as a problem. The latter can be easily seen with respect to E and L, by rearranging the elements of these sets into diagonal matrices and by dening the M matrix appropriately. For U, we may simply adopt a full real block structure and dene accordingly. It should be pointed out, nevertheless, that both and the polynomial approach will lead to complications in the present context. On the one hand, the computation of with respect to a real is generally very difcult, and approximation by its upper bound can be very conservative. On the other hand, the characteristic polynomial (s, ) will generally exhibit a multilinear or multinomial dependence of its coefcients on , for which the Kharitonov theory is ill-equipped; indeed, it is not difcult to see that the coefcients of (s, ) are multilinear in ij if E, and are multinomial functions of k if L. In summary, both approaches are ineffective and conservative. By far this uncertainty description poses the most difcult challenge in robust stability analysis, and the state-space approach is the least developed. Results are scarce, and only in rare cases are they nonconservative. Stability Radius A notion frequently encountered in studying the state-space uncertainty description is that of stability radius. This notion is closely related to , but it is less developed. Let be a stability region of concern, and be its boundary. Furthermore, denote by (A) the spectrum of A. Then for any norm of interest, the stability radius associated with the triple (A, B, C) is dened by r(A, B, C ) = inf{ : , (A + B C ) D = }
In other words, it denes the minimal perturbation size leading to instability, or the distance of A to the set of unstable matrices. By denition, it thus follows directly that the matrix family A BC: , has all eigenvalues in whenever r(A, B, C) . Moreover, in view of the preceding discussion, we may regard the stability radius as the reciprocal of the maximum of a certain , with respect to an appropriate block structure and a matrix M. For further distinction, the stability radius is said to be unstructured if is unstructured and structured otherwise.
614
Then a sufcient condition for Hurwitz stability of AI is found by Chen (41) to be (W ) < 1 A useful feature of this result, and more generally of conditions obtained by using Gershgorins Theorem, is that it lends a ready characterization via the so-called M-matrices. The latter aspect makes it possible to unify a number of sufcient stability conditions in different forms. Alternatively, Yedavalli and others studied interval matrices from a Lyapunov analysis standpoint. This is collectively inspected next. Lyapunov Analysis The Lyapunov theory, as anticipated, is widely employed in robust stability analysis pertaining to state-space formulation, yielding various results concerning stability radius and interval matrices. One common thread in this approach is to nd a single quadratic Lyapunov function applicable to the entire family of the perturbed matrices; the technique is often referred to in the literature as quadratic stability. Another lies in the simplicity of the stability conditions. Let us begin with the unstructured uncertainty set U. By constructing the usual Lyapunov function V (x ) = 1 x T Px 2 we nd that the entire family of matrices A BC: U is Hurwitz stable if there exists a positive denite matrix P 0 such that PA + AT P + 2 BBT + CTC < 0 (15)
Furthermore, Zhou and Khargonekar (42) observed that the uncertainty description E can be regarded as a special case of L, for which they provided the stronger Hurwitz stability condition
1 2
(|PEi +
i= 1
EiT P| )
< 1/
(18)
where, again, P 0 is the unique solution to Eq. (17). Subsequent developments led to further extensions for problems with even more detailed uncertainty descriptions. For example, the ks may be allowed to vary in asymmetric intervals. Moreover, because rather obviously any interval matrix can be represented alternatively in the form of A : E, these conditions can be applied to determine the Hurwitz stability of an interval matrix as well. Yet another issue clearly of interest is whether it is possible to derive vertex versions of these sufcient conditions. Boyd and Yang (44) examined stability problems for matrix polytopes. Specically, they postulated the uncertainty description A = conv{A1 , . . ., Ak } A sufcient condition for A to be Hurwitz stable can be easily found to be the existence of a P 0 such that PAi + AT i P < 0, i = 1, . . ., k (19)
Similarly, for the uncertainty set L, a vertex condition can be obtained as
1 2
k i (PEi i= 1
EiT P )
< 1/
(20)
This, of course, does not come as a surprise. According to the well-known Bounded Real Lemma (42), it is equivalent to C(sI A ) 1 B
< 1/
The latter condition clearly coincides with condition (14). More results are available for the special case when B C I. For the structured uncertainty set E, Yedavalli (43) gave the sufcient condition |P|R + RT |P| 2 < 1/ (16)
for all combinations of the i in 1, 1. It should be rather evident that this condition improves upon inequality (18). Both conditions (19) and (20) may be regarded as vertex results in the matrix perturbation case, and both can be posed and solved as LMI problems. CONCLUSION Summary For the past two decades, modeling uncertainty and robustness has resurfaced as a dominating theme in control theory and application and is now held unanimously by theoreticians and practitioners as the most important concern in control system design. For both its intrinsic appeal and practical signicance, robust control as a whole attracted considerable interest and underwent a period of immense development, bringing control theory to a new height. Many important issues have been addressed. Many remain unresolved. The ultimate puzzle, it now appears, lies in the fundamental conict between problem complexity and computational tractability. Of the three main research areas surveyed in this article, the structured singular value provides the most general formulation for uncertainty modeling and is the most systematically developed tool in robustness analysis. The major issues
for the Hurwitz stability of the matrix family A : E. Here P 0 is the unique solution to the Lyapunov equation PA + AT P = 2I (17)
P denotes the modulus matrix of P (i.e., each entry of P is the absolute value of the corresponding entry of P) and R is given by
r11 . R= . . rn 1
. . .
r1 n . . . rnn
615
in analysis are clearly generality of uncertainty description, conservatism of analysis, and ease of computation. The main success achieved with this approach, unquestionably, lies in the progress in computing . While it cannot be computed exactly in general, various computational schemes have been developed for computing it approximately, and commercial software programs are now available. This paves the way for its application to a series of engineering design problems, ranging from disk drive control to ight control. Successful applications to other potential areas, including robot manipulators, exible structures, magnetic bearings, and chemical processes, have also been reported in laboratory experiments. The Kharitonov approach, unlike analysis, was more restrictive in scope in its early phase of development. However, it has undergone a bottom-up growth pattern as the uncertainty descriptions become progressively more general and sophisticated. Overall, the Kharitonov and state-space methods may be broadly classied as a parametric approach toward robustness analysis, originating from interval polynomials and culminating at state-space uncertainty descriptions. The main appeal of this approach, it appears, lies in its quest for analytical solutions, more appealing than mere computationbased tools. The main success in the entire parametric approach, which remains the state-of-the-art today, is the resolution of the afne uncertain polynomial family case, for which necessary and sufcient stability conditions are available, in terms of both edge tests and graphical conditions. On the other hand, the multilinear/multinomial polynomial family and the state-space uncertainty description are the weakest link, for which only sufcient stability conditions are available with unknown conservatism, and more systematic, efcient, computation-based approximate tests are called for. At present, only a few applications of the Kharitonov theory are reported in the literature, including Ackermanns car steering problem and an automotive engine control problem investigated by Abate et al. (45) (see also Chapter 3 in Ref. 26). It should be rather evident that the fundamental bottleneck in all robustness analysis methods, be it analysis or Kharitonov approach, lies in computational complexity, and the ultimate challenge is in the conquest over the curse of dimensionality. No matter whether this can be achieved or not, we should be consciously aware that the dilemma is the natural cause of problem generality and hence complexity and results from the search of optimal solutions. In engineering system design, we should therefore reconcile and seek a judicious trade-off between these conicting requirements. To Probe Further In light of the difculties encountered in robustness analysis with respect to structured and/or parametric uncertainties, a number of researchers recently examined complexity issues from a computational standpoint, drawing upon concepts and techniques from computing science and operation research. The main discoveries are in the following areas. with Real Uncertainties. Braatz et al. (46) showed that the computation of real is NP-hard. with Complex Uncertainties. Toker and Ozbay (47) proved that the computation of complex is also NPhard.
with Real and Complex Uncertainties. Braatz et al. (46) and Toker and Ozbay (47) both showed that the computation of is NP-hard. The Bounds. Toker (48) and Fu (49) showed that the problem of nding an accurate bound for is NP-hard. Interval Matrix. Coxson and DeMarco (50) showed that stability of interval matrices amounts to an NP-hard problem. These results indicate that a worst-case instance exists in each class of the problems, for which it is rather unlikely that computational complexity can be bounded via a polynomial function of the problem dimension. It thus comes as no surprise that the problems are difcult, and indeed are intractable in general. From a technical standpoint, the computational difculty in question may be best seen as an outcome of nonlinear, nonconvex optimization problems. Although only explored systematically in recent years, complexity issues have been under contemplation for a long time and have led to alternative, computationally tractable approximations and formulations. One notable remedy is to resort to formulations based upon LMIs, and problems in this class include those that can be described via integral quadratic constraints (IQC). Both LMIs and IQCs offer in essence an energy-based perspective toward system analysis, and they draw heavily upon concepts in classical passivity and dissipativity theory, leading to readily computable, albeit only sufcient, robust stability conditions. For a comprehensive treatment of control-relevant LMI and convex programming problems, see Ref. 51, or the relevant chapter in this encyclopedia. Megretsky and Rantzer (52) provided a detailed account of the IQC technique. The computational complexity results just discussed are strongly linked to the worst-case nature of the robustness problems; that is, the requirement of robustness must be met for all possible instances. Is so stringent a requirement truly necessary? This question prompted a reexamination of robustness issues, and it led to a recent venture departing almost entirely from the worst-case formulation. A number of researchers argued that worst-case scenarios hardly occur in practice, that a worst-case analysis is not only overly demanding but also too pessimistic, and that, after all, worstcase analysis problems are often intractable. The argument thus motivated the description of uncertainty via probabilistic measures, and accordingly probabilistic approaches to robustness analysis. In this new thinking, the deterministic uncertainty description is discarded altogether and is replaced by a probability description characterizing the likelihood that the uncertainty may lie in a bounded set. The robustness condition then amounts to determining the probability under which the system may become unstable. Recent studies (53 56) show that a variety of problems, which are NP-hard in the deterministic setting, become readily solvable computationally when formulated probabilistically. The area, however, is entirely open and is not without obstacles of its own.
BIBLIOGRAPHY
1. J. Ackermann, Robust Control Systems with Uncertain Physical Parameters, London: Springer-Verlag, 1996.
616
ROBUST CONTROL ANALYSIS 27. M. Fu and B. R. Barmish, Polytopes of polynomials with zeros in a prescribed set, IEEE Trans. Autom. Control, 34 (5): 544546, 1989. 28. A. L. Tits, Comments on polytopes of polynomials with zeros in a prescribed set, IEEE Trans. Autom. Control, 35 (11): 1276 1277, 1990. 29. B. R. Barmish, A generalization of Kharitonovs four polynomial concept for robust stability problems with linearly dependent coefcient perturbations, IEEE Trans. Autom. Control, 34 (2): 157 165, 1989. 30. Ya. Z. Tsypkin and B. T. Polyak, Frequency domain criteria for lp-robust stability of continuous linear systems, IEEE Trans. Autom. Control, 36 (12): 14641469, 1991. 31. D. Hinrichsen and A. J. Pritchard, An application of state space methods to obtain explicit formulae for robustness measures of polynomials, in M. Milanese et al. (ed.), Robustness in Identication and Control, Boston, MA: Birkhauser, 1989. 32. M. K. Saridereli and F. J. Kern, The stability of polynomials under correlated coefcient perturbations, Proc. 26th IEEE Conf. Decision Control, 1987, pp. 16181621. 33. A. Tesi and A. Vicino, Robustness analysis for linear dynamical systems with linearly correlated parametric uncertainties, IEEE Trans. Autom. Control, 35 (2): 186191, 1990. 34. L. Qiu and E. J. Davison, A simple procedure for the exact stability robustness computation of polynomials with afne coefcient perturbations, Syst. Control Lett., 13: 413420, 1989. 35. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular value and stability of uncertain polynomials, Part 2: A missing link, Syst. Control Lett., 23 (2): 97109, 1994. 36. S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The Parametric Approach, Upper Saddle River, NJ: PrenticeHall, 1995. 37. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31 (6): 879890, 1995. 38. J. Sreedhar, P. Van Dooren, and A. L. Tits, A fast algorithm to compute the real structured stability radius, in R. Jeltsch and M. Mansour (eds.), Stability Theory: Proc. Centenary Conf. Int. Series Numerical Math. (ISNM), Basel: Birkha user, 1996, vol. 232, pp. 219230. 39. J. A. Heinen, Sufcient conditions for stability of interval matrices, Int. J. Control, 39 (6): 13231328, 1984. 40. M. B. Argoun, On sufcient conditions for the stability of interval matrices, Int. J. Control, 44 (5): 12451250, 1986. 41. J. Chen, Sufcient conditions on stability of interval matrices: Connections and new results, IEEE Trans. Autom. Control, 37 (4): 541544, 1992. 42. K. Zhou and P. P. Khargonekar, Stability robustness bounds for linear state-space models with structured uncertainty, IEEE Trans. Autom. Control, 32 (7): 621623, 1987. 43. R. K. Yedavalli, Improved measures of stability robustness for linear state space models, IEEE Trans. Autom. Control, 30 (6): 577579, 1985. 44. S. Boyd and Q. Yang, Structured and simultaneously Lyapunov functions for system stability problems, Int. J. Control, 49 (6): 22152240, 1989. 45. M. Abate et al., Application of some new tools to robust stability analysis of spark ignition engines: A case study, Proc. Amer. Control Conf., 1992, pp. 932936. 46. R. P. Braatz et al., Computational complexity of calculation, IEEE Trans. Autom. Control, 39 (5): 10001002, 1994. 47. O. Toker and H. Ozbay, On the complexity of purely complex computation and related problems in multidimensional systems, IEEE Trans. Autom. Control., 43 (3): 409414, 1998.
2. J. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., 129-D (6): 242250, 1982. 3. V. L. Kharitonov, Asymptotic stability of an equilibrium position of a family of systems of linear differential equations, Differentialnye Uraveniya, 14 (11): 14831485, 1978. 4. M. G. Safonov, Stability margins of diagonally perturbed multivariable feedback systems, IEE Proc., 129-D: 251256, 1982. 5. A. Packard and J. C. Doyle, The complex structured singular value, Automatica, 29 (1): 71109, 1993. 6. W. S. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, 1996. 7. K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control, Upper Saddle River, NJ: Prentice-Hall, 1996. 8. K. Zhou and J. C. Doyle, Essentials of Robust Control, Upper Saddle River, NJ: Prentice-Hall, 1998. 9. A. L. Tits and V. Balakrishnan, Small- theorems with frequency-dependent uncertainty bounds, Math. Control. Signals, Syst., in press, 1998. 10. J. M. Krause, Structured singular value analysis of multidimensional system stability, IEEE Trans. Autom. Control, 34 (6): 638 639, 1989. 11. J. Chen and H. A. Latchman, Frequency sweeping tests for stability independent of delay, IEEE Trans. Autom. Control, 40 (9): 16401645, 1995. 12. M. K. H. Fan, A. L. Tits, and J. C. Doyle, Robustness in the presence of mixed parametric uncertainty and unmodelled dynamics, IEEE Trans. Autom. Control, 36 (1): 2538, 1991. 13. P. M. Young, The rank one mixed problem and Kharitonovtype analysis, Automatica, 30: 18991911, 1994. 14. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular value and stability of uncertain polynomials, Part 1: The generalized , Syst. Control Lett., 23 (1): 5365, 1994. 15. B. Kouvaritakis and H. Latchman, Necessary and sufcient stability criterion for systems with structured uncertainties: The major principle direction alignment principle, Int. J. Control, 42 (3): 575598, 1985. 16. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular values with non-diagonal structures, Part 1: Characterizations, IEEE Trans. Autom. Control, 41 (10): 15071511, 1996. 17. J. Chen, M. K. H. Fan, and C. N. Nett, Structured singular values with non-diagonal structures, Part 2: Computations, IEEE Trans. Autom. Control, 41 (10): 15111516, 1996. 18. A. L. Tits, V. Balakrishnan, and L. Lee, Robustness under bounded uncertainty with phase information, IEEE TAC, in press. 19. M. Khammash and J. B. Pearson, Robust disturbance rejection in l1-optimal control systems, Syst. Control Lett., 14: 93101, 1990. 20. M. Khammash and J. B. Pearson, Performance robustness of discrete-time systems with structured uncertainty, IEEE Trans. Autom. Control, 36 (4): 398412, 1991. 21. J. Shamma, Robust stability with time-varying structured uncertainty. IEEE Trans. Autom. Control, 39 (4): 714724, 1994. 22. A. Megretski, Necessary and sufcient conditions of stability: A multiloop generalization of the circle criterion, IEEE Trans. Autom. Control, 38 (5): 753756, 1993. 23. K. Poolla and A. Tikku, Robust performance against time-varying structured perturbations, IEEE Trans. Autom. Control, 40 (9): 15891602, 1995. 24. S. Dasgupta, Kharitonovs theorem revisited, Syst. Control Lett., 11: 381384, 1988. 25. A. C. Bartlett, C. V. Hollot, and L. Huang, Root locations of an entire polytope of polynomials: It sufces to check the edges, Math. Control, Signals, Syst., 1: 6171, 1988. 26. B. R. Barmish, New Tools for Robustness of Linear Systems, New York: Macmillan, 1994.
ROUNDOFF ERRORS 48. O. Toker, On the conservatism of upper bound tests for structured singular value analysis, Proc. 35th Conf. Decision Control, Kobe, Japan: 1996, pp. 12951300. 49. M. Fu, The real is hardly approximable, IEEE Trans. Autom. Control, 42 (9): 12861288, 1997. 50. G. E. Coxson and C. L. DeMarco, The computation complexity of approximating the minimal perturbation scaling to achieve instability in an interval matrix, Math. Control, Signals Syst., 7 (4): 279292, 1994. 51. S. Boyd et al., Linear Matrix Inequalities in System and Control Theory, Studies in Applied Mathematics, Philadelphia: SIAM, 1994, vol. 15. 52. A. Megretski and A. Rantzer, System analysis via integral quadratic constraints, IEEE Trans. Autom. Control, 42 (6): 819 830, 1997. 53. R. F. Stengel and L. R. Ray, Stochastic robustness of linear timeinvariant systems, IEEE Trans. Autom. Control, 36 (1): 8287, 1992. 54. B. R. Barmish and C. M. Lagoa, The uniform distribution: A rigorous justication for its use in robustness analysis, Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34183423. 55. R. Tempo, E. W. Bai, and F. Dabbene, Probabilistic robustness analysis: Explicit bounds for the minimum number of samples, Proc. 35th IEEE Conf. Decision Control, 1996, pp. 34243428. 56. X. Chen and K. Zhou, On the probabilistic characterization of model uncertainty and robustness, Proc. 36th IEEE Conf. Decision Control, 1997, pp. 38163821.
617
JIE CHEN
University of California
L. TITS ANDRE
ROBUSTNESS ANALYSIS. See ROBUST CONTROL

ANALYSIS.
ROBUST SIGNAL PROCESSING. See NONLINEAR

SYSTEMS.
ROLLBACK RECOVERY. See PROGRAM DIAGNOSTICS. ROLLING METALS. See METALS INDUSTRY. ROTATION SPEED. See TACHOMETERS.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Self-Tuning Regulators Standard Article Bjrn Wittenmark1 1Lund University, Lund, Sweden Copyright 2007 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1037.pub2 Article Online Posting Date: August 17, 2007 Abstract | Full Text: HTML PDF (233K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Basic Idea Algorithms for Self-Tuning Control Some Theoretical Problems Practical Issues and Implementation About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
SELF-TUNING REGULATORS
A self-tuning regulator (STR) is a controller that automatically nds its parameters in the control law. Another name or synonym is self-adjusting controller. STR is a class of adaptive controllers used when the process to be controlled has constant but unknown parameters. However, STR can also be used in an adaptive context. BASIC IDEA Controller Design Procedure The design of a controller contains several steps: 1. Finding specications for the closed-loop system 2. Determination of a model for the process to be controlled 3. Decision on a design method 4. Calculation of the parameters in the controller In many cases, it is desirable to automate these steps. This is the idea behind adaptive and self-tuning regulators. The specications for the closed-loop system depend on such things as quality constraints on the controlled variable, available magnitude (power) of the control signal, and nonlinearities of the system to be controlled. This implies that the specications are determined by the process engineer at the start of the design procedure. The specications often lead to a natural choice of the design method. For instance, if the main specication is to keep the process output constant and if the disturbances are occasional large disturbances, then the design procedure can be a method that as quickly as possible eliminates the inuence of the disturbance. The choice of specications and design method is thus usually made by the designer of the control loop. In STRs, as well as in adaptive controllers, steps 2 and 4 above are automatically taken care of by the controller. The structure of a self-tuning controller is best described from the block diagram in Fig. 1. The self-tuning regulator consists of two closed loops. The rst loop is a conventional controller feedback-loop consisting of the process and the controller where the output of the process is measured and compared with the desired output (reference signal) of the closed-loop system. The mismatch between the reference and output signals is used to compute the control action that is sent to the process. The controller has parameters that determine its properties. These parameters are determined by the second loop in the STR, the updating loop. In Fig. 1, the updating loop has two main blocks. The rst block is an estimator, which determines a mathematical model of the process based on the measured inputs and outputs. The second block carries out the design of the controller. This block uses the process model and the specications to determine the controller parameters that then are sent to the controller. It is necessary that the controller feedback-loop be closed all the time to take care of the inuence of disturbances and changes in the reference signal. The updating
loop for the controller parameters can be switched off as soon as the estimated parameters have converged to their nal values, that is, when the controller has tuned or adjusted itself to the specications and the process. The result is a self-tuning regulator. However, if the process is changing over time it is necessary to update continuously the process model and the controller parameters. We then have an adaptive controller. This implies that an STR is an adaptive controller if the parameter updating is not switched off. The STRs are thus a special class of adaptive controllers. One of the rst descriptions of the idea of STRs is found in Kalman (1) where updating using parameter estimation and design is described. The term self-tuning regulator was om and Wittenmark (2) who gave the rst coined by Astr analysis of the steady-state properties of the STR based on minimum variance control. The stability of the closedloop system and the convergence properties were analyzed in Goodwin, Ramadge and Caines (3). More details of the properties of self-tuning and adaptive controllers om and can be found in Wellstread and Zarrop (4) and Astr Wittenmark (5). Classication of Self-Tuning Regulators The STR in Fig. 1 contains both a block for estimation and a block for design. An STR in this conguration is usually called an indirect self-tuning regulator. The reason is that the controller parameters are obtained indirectly by rst nding a process model. In many cases it is possible to make a reparameterization of the process and the controller such that the controller parameters can be estimated directly. This leads to a direct STR. Ways to do this reparameterization are discussed below. Applications of Self-Tuning Regulators The computations in an STR are quite straightforward, but contain nonlinear and logical operations. This implies that STRs are implemented using computers. The algorithm can be a block in a software package that is used for larger process control applications, or the STR can be implemented in dedicated hardware for a few control loops. Self-tuning control has, since the mid-1970s, been used for many applications, mainly in the process industry. Applications are found in areas of pulp and paper, chemical reactors, autopilots, and dialysis machines. Self-tuning regulators and adaptive controllers in general have found their main uses in three categories of applications: When the process has long time delays When feedforward can be used When the disturbances acting on the process have time-varying characteristics The main reason self-tuning or adaptive controllers have a great advantage in these cases is that for good control of these types of processes it is necessary to have models of the process and/or of the disturbances to be controlled. The estimator part of the self-tuning controller can make an estimate of the process and use that in the design.
Self-Tuning Regulators
Figure 1. Block diagram of a self-tuning regulator.
Linear STRs are not appropriate to use when the process is very nonlinear. The updating mechanism will then not be sufciently fast. In such cases the nonlinearities should be built into the process model and the controller. ALGORITHMS FOR SELF-TUNING CONTROL This section describes in more detail how STRs are constructed. It also gives the main properties of STRs. To describe the algorithms we need to specify the process model, the specications, the controller, the estimator, and the design method. We will use discrete-time models for the process and the controller since most implementations of STRs are done using computers. It is, however, also possible to derive continuous-time STRs. Process Model The process is described as a sampled-data linear system. The process is also assumed to have a single input and a single output. The model is given as a difference equation
the system can be written as
or
where H(q1 ) is called the pulse-transfer function. Specications The specications give the desired performance of the closed-loop system. The specications can be given in many different ways depending on the purpose of the closed-loop system. It is common to distinguish between the servo and regulator cases. In the servo case, we give the desired performance in the form of the time or frequency response when the reference value is changed or when an occasional large disturbance has inuenced the system. The typical specications are bandwidth or response time. Further, things as overshoot or damping can be specied. One way to give the specications is in the form of a reference model Hm dening the desired output ym
where y(k) is the output signal at sampling instant k and u(k) is the control signal. Disturbances will be introduced below. It is assumed that the time is scaled such that the sampling period is one time unit. The parameter d is the time delay of the system. Equation (1) is a general description of a linear sampled-data system. To get a more compact way of describing the system, we introduce the backwardshift operator q1 . The backward-shift operator is dened in the following way
That is, operating on a time sequence it shifts the time argument one step backwards. Using the backward-shift operator and the polynomials
where uc is the reference signal. Normally, dm = d, but may also be longer. In the regulator case, we study the closed-loop performance when disturbances essentially are acting on the system while the reference signal is constant. The disturbance is then usually modeled as a stochastic process, in general, ltered white noise. Typical performance indices are to minimize the variance of the output signal around a desired reference value or to minimize a combination of output and input variations. Controller The controller is dened as
The controller has a feedback part dened by the polynomials R*(q1 ) and S*(q1 ) and a feedforward part dened by R*(q1 ) and T*(q1 ). Using Eq. (4) on the process of Eq. (2) gives the closed-loop system
Since the updating formulas of Eq. (7) are recursive, they can be used also for a continuous updating of the parameters. In such cases it is, however, necessary to introduce a weighting of old inputs and outputs. The loss function of Eq. (6) puts equal weight on all data. A measurement collected a long time ago is as important as the latest measurement. Newer measurements can be given more weight by changing the loss function of Eq. (6) to
Estimator Estimation of process models can be done in many different ways. Summaries of methods and their properties can be found in Ljung (6), S oderstr om and Stoica (7), and Johansson (8). Here only the recursive least squares (RLS) method will be discussed. Dene the vectors where is the forgetting factor. Since the weights are exponentially decaying, the resulting algorithm is called recursive least squares with exponential forgetting. The updating formulas are only slightly modied into
The vector contains the unknown process parameters, while the vector contains known old inputs and outputs of the process. The process model can now be written as
In the following we will use the recursive least squares algorithm, with or without exponential forgetting, to illustrate the properties of STRs. Design Methods The nal step in the construction of an STR is the design procedure. The basic STRs are based on the certainty equivalence principle. This implies that the process parameter estimates obtained from the estimator are used as if they are the true ones. The design principles can, however, be extended to include also the uncertainty of the estimates, given by the P matrix. This leads to so-called cautious or dual controllers. Two different design principles will be discussed in detail, pole-placement design and minimum variance control. In-depth treatments of the design methods can be found in om and Wittenmark (9). Astr Pole-Placement Design. We will now discuss how the parameters in the controller in Eq. (4) can be determined using the method of pole placement for the design of the controller. The closed-loop system is dened by Eq. (5). The closed-loop characteristic polynomial is thus
The least squares method, rst stated in Gauss (10), im , which plies that the estimate of should be chosen as minimizes the loss function
(0) and the unGiven an initial value of the parameters certainty of the parameter estimate P(0), it is possible to derive a recursive solution to the least squares problem. The parameter estimate can be updated recursively using
This is called the recursive least squares algorithm. The estimate at time k is obtained as an update of the estimate at time k 1. The correction term depends on the latest process output, which is compared with the predicted output based on the parameter estimate at time k 1. The matrix P(k) can be interpreted as an estimate of the uncertainty of the parameter estimate at time k. The statistical interpretation can be made rigorous by making assumptions about the disturbances that are acting on the system. The recursive least squares method is well suited for process parameter estimation when there are no disturbances or when a white noise process is added to the righthand side of Eq. (2). For other noise or disturbance assumptions, there are variants of the recursive least squares method that can be used.
where A*c is given as a specication by the designer. The key idea is now to nd the controller polynomials R* and S* that fulll this equation. Equation (8) is called a diophantine equation. The desired closed-loop system from the reference signal to the output dened by Eq. (3) requires that the following condition must hold
This design procedure is called model-following design, and also pole-placement design, if the poles only are specied.
Whether model following can be obtained depends on the model, the process, and the complexity of the controller. The characteristic polynomial of Eq. (8) will, in general, have higher degree than the model polynomial A m. This implies that there must be a polezero cancellation in Eq. (9). The consequences of this will now be discussed. The B* polynomial of the process is rst factored into
variance controller is obtained by solving the diophantine equation
and using the control law
where B+ corresponds to the process zeros that can be cancelled in the design. These zeros must be located inside the unit circle. The zeros corresponding to B , which are not allowed to be canceled, must then be a factor of B m, which must have the form
Also, linear quadratic gaussian controllers can be interpreted as solving a special form of the diophantine equa om and Wittenmark (9). tion, see Astr Design of Self-Tuning Regulators The design of STRs can be summarized by the following procedure: Specications. Determine the class of controller by determining the specications on the closed-loop system. Estimation. Estimate the process parameters using, for instance, the recursive least squares algorithm of Eq. (7). Design Procedure. Determine the controller parameters using the estimated process parameters as if they are the correct ones. The controller design is usually reduced to the solution of an equation such as the diophantine Eq. (8). Control. Update the parameters of the controller, for instance, in the form of the controller in Eq. (4). The estimation, design, and control steps are done at each sampling interval. In some situations, it may be sufcient to update the estimation at a slower rate than the rate of the control loop. The behavior of the basic indirect selftuning algorithm will be described by an example. Example 1: Indirect Deterministic Self-Tuning Regulator Assume that the open-loop process is described by the continuous-time system
Since B+ is canceled, it must be a factor of A c . The closedloop characteristic polynomials is thus of the form
The polynomial A o is called the observer polynomial, and can be interpreted as the dynamics of a state observer. The observer polynomial inuences, for instance, how fast the system will recover after a disturbance. A o is determined by the designer and should be a stable polynomial. Since B+ is a factor of B* and A c , it follows from Eq. (10) that it also is a factor of R*, which implies that
and the diophantine equation reduces to
Finally, the polynomial T* is given by
The design procedure can now be summarized into: Data. Given the process polynomials A*, B* = B B , and the observer polynomial A o Step 1. Solve the diophantine equation with respect to R* and S*. Step 2. The controller is given by Eq. (4) with R* = R* B+ and T* = A o Bm . The diophantine equation can always be solved if there are no common factors between the A* and B* polynomials and if the controller polynomial has sufciently many parameters. Minimum Variance Control. Most design procedures can be interpreted as a pole-placement or model-following design. For instance, the minimum variance controller can easily be formulated in this form. The minimum variance controller is a controller that minimizes the variance of the output from the process. In this case, we add a disturbance term C*(q1 )e(k) on the right-hand side of Eq. (2), where C* is a stable polynomial and e(k) is white noise. The minimum
+
The process has an integrator and a time constant of 1 s. There are no disturbances acting on the system and the specications are that the controlled system should be able to follow constant reference signals without too much overshoot. Sampling the system with the sampling interval h = 0.5 s gives the sampled-data description
There is a process zero in b1 /b0 = 0.85. The zero is inside the stability boundary, but it is still decided not to cancel the zero. Let the desired closed-loop system be
Figure 2. Process output and input when an indirect self-tuning regulator is used to control the process in Example 1. The specications are changed at time 100. The reference signal is shown as a dashed curve.
Figure 3. Parameter estimates corresponding to the simulation in Fig. 2. Upper diagram: a 1 (full) and a 2 (dashed), lower diagram: 0 (full) and b 1 (dashed). The true parameter is shown by dashedb dotted lines.
This corresponds to a continuous-time system with natural frequency = 1 and a damping of = 0.7 sampled with the sampling period h = 0.5. The gain K is chosen such that the steady-state gain from the reference signal to the output is equal to one, that is, B m (1)/Am (1) = 1. The controller solving the design problem will have the structure
The design specication is to minimize the variance of the output signal. Minimum variance control is equivalent to predicting the output signal d steps ahead and choosing the control signal such that the predicted value is equal to zero, or any other desired set point value. The prediction horizon should be equal to d, which is the delay in the process. From Ref. 5 or Ref. 9 it follows that the output of Eq. (13) can be written as
Figure 2 shows the output and the control signal when the process is controlled by a self-tuning controller. The reference signal is a square wave. It is seen that the output behaves well already at the second change of the reference signal. At time 100 the design specications are changed and the damping is changed from = 0.7 to = 1. The closed-loop response is immediately changed. The process model has four unknown parameters, b0 , b1 , a1 , and a2 . These parameters are estimated using the RLS algorithm, and the estimated process parameters are shown in Fig. 3. The example shows that the STR can nd good controller parameters very quickly and that the design parameters can be changed. The transient in the beginning depends on the choice of initial values in the estimator. Direct Self-Tuning Regulators The self-tuning algorithm described above relies on a separation between the estimation and the design. The design step is repeated at each sampling instant. It can, on some occasions, be desirable to avoid the computations done in the design step, for instance, because of computing time limitations. One way to do this is to convert the indirect STR into a direct STR. This implies that the controller parameters are estimated instead of the process parameters. How to do this reparameterization will be illustrated on the minimum variance controller. Let the system to be controlled be described by
where F* and G* are obtained from the diophantine Eq. (11). The predicted output d steps ahead is given by the second and third terms on the right-hand side of Eq. (14). The prediction error is given by the rst term on the right-hand side of Eq. (14). The prediction error is a moving average stochastic process that is independent of the predicted output. The predicted output is zero if the control law is chosen according to Eq. (12). Using the underlying design principle, it has been possible to reparameterize the model of Eq. (13) such that the reparameterized model explicitly contains the controller parameters. The controller parameters then can be estimated directly. Using the minimum variance controller, the closed-loop system becomes
The idea behind the basic direct STR is to estimate the parameters in the prediction model
and use the controller
The estimated parameters are thus the same as the controller parameters, and the design step has been eliminated.
Elimination of Disturbances We will now see how the inuence of disturbances can be reduced by introducing integrators and by using feedforward. Introduction of Integrators. Consider the process
Figure 4. The accumulated loss function V(k) when the direct self-tuning algorithm (full) and the optimal minimum variance controller (dashed) are used on the process in Example 2.
0 when the self-tuning algoFigure 5. The controller gain s 0 /I rithm is used (full). The gain of the optimal minimum controller is shown as a dashed line.
Example 2: Direct Minimum Variance Self-Tuning Algorithm Assume that the open-loop process is described by the sampled-data model
which is a slight variant of Eq. (2). The signal v(k) is an input load disturbance. If this is, for instance, a step, then there needs to be an integrator in the controller to eliminate the inuence of this disturbance. There are several ways to cope with this in an STR. One way is to estimate the magnitude of the disturbance and to compensate for it in the controller. To do so, the tuning has to be active all the time since the disturbance may change over time. A recommended way is to introduce an integrator directly into the controller. This can be done by postulating that the R* polynomial contains the factor 1 q1 . This can be done in the direct as well as in the indirect algorithms. In the indirect algorithm, it is necessary to modify the estimator since the disturbance will change the relations between the inputs and outputs. Load disturbances such as steps have a particularly bad inuence on the estimated model in the low-frequency range. Let the disturbance be modeled as
where e(k) is white noise with variance 1. The time delay in the system is d = 1. Estimate the parameters r0 and s0 in the model
where e(k) is a pulse, a set of widely separated pulses, or white noise. For instance, a step disturbance is generated by
and use the controller
The model can now be described as
The optimal minimum variance controller is given by u(k) = 0.2y(k), which is a proportional controller. Using this controller gives the output y(k) = e(k), that is, the output should be white noise with a variance of 1. One way to compare the optimal and the self-tuning regulators is to compare the accumulated loss functions
Introduce the ltered signals yf (k) = A d y(k) and uf (k) = A d u(k). We thus get
The new model has the equation error e(k) instead of v(k). The process model can now be estimated from Eq. (17). Based on the estimated model the controller design is done by solving the diophantine equation
The slope of the accumulated loss function is an estimate of the variance of the output. Figure 4 shows the loss function when the self-tuning algorithm and when the optimal minimum variance controller is used. After a short initial transient, the slopes of the loss functions are the same, which indicates that the STR has converged to the optimal minimum variance controller. This can also be seen by looking at the gain of the controller shown in Fig. 5.
and using the controller
The controller now contains the factor A d , which will eliminate the inuence of the disturbance v. In the direct minimum variance self-tuning algorithm an integrator can be introduced by changing the model of Eq. (16) and estimating the controller parameters from
where
u(k) = u(k) u(k 1) and using the controller
which contains an integrator. Feedforward from Measurable Disturbances. On many occasions it is possible to measure some of the disturbances acting on the system. A typical example is control of indoor temperatures. By measuring the outdoor temperature also it is possible to use this signal to compensate for changing outdoor temperatures before the disturbance has inuenced the process too much. One way to introduce feedforward in STRs is exemplied with the direct algorithm. The estimated model is changed from Eq. (15) to
where vm (k) is the measurable disturbance. The controller is now
results have lately been rened and extended [see Well om and Wittenmark (5)]. stead and Zarrop (4) and Astr One important theoretical aspect is the inuence of unmodeled dynamics. Unmodeled dynamics are present if the estimator is trying to t a too-simple model to the data. The unmodeled dynamics may cause severe stability problems, which must be avoided by introducing counter measures such as careful ltering of the signals in the STR. This type of problem has successfully been analyzed using averaging theory. It is important that a controller is robust against assumptions and choices of controller parameters. Much theoretical research has been devoted to make STRs and adaptive controllers more robust. This work has resulted in practical rules of thumb for their implementation (see Ref. 5). Robust design methods are complementary to selftuning and adaptive control. In robust control one xed controller is designed to cope with a variety of processes. By using tuning and adaptation the parameters of the controller are instead tuned to adjust to the present process dynamics. PRACTICAL ISSUES AND IMPLEMENTATION
The rst part of the controller is the feedback from the measurement y(k) and the second is the feedforward from the measurable disturbance vm (k). Feedforward is, in general, very useful in STRs because to make effective feedforward, it is necessary to have a good model of the process. By combining the measurement of the disturbance and the selftuning property of the controller it is possible to eliminate much of the disturbance before it reaches the output of the process. SOME THEORETICAL PROBLEMS The previous section described the basic ideas of STRs. Self-tuning regulators are inherently nonlinear. The nonlinearities are due to the estimation part and the changing parameters in the controller. This makes the analysis of STRs very difcult. The STRs contain two feedback loops and it is necessary to investigate the stability and convergence properties of the closed-loop systems. This is a difcult question because of the interaction between the two feedback loops. One way to circumvent this problem is to make a time separation between the two loops. The controller loop is assumed to be fast compared to the updating loop. This makes it possible to use averaging theory to analyze the updating loop on a much longer time-scale. This approach has made it possible to derive results concerning stability and convergence of STRs. om and Wittenmark (2) showed how to characterize Astr the stationary properties of STRs, that is, the properties if and when the parameter estimation has converged. The algorithms were used in a number of applications before several of the theoretical problems were solved. Goodwin, Ramadge, and Caines (3) gave the rst results showing when the algorithm converges and that the closed-loop system remains stable during the estimation phase. These
Some problems in the implementation of STRs are discussed briey in this section. Self-tuning regulators as well as adaptive controllers will run unattended on the processes. It is therefore very important that there be a good safety net around the self-tuning algorithm. There are many aspects of STR implementation that are important for implementations of digital controllers in om and Wittenmark (9)]. Some important general [see Astr issues for STRs are Organization of the computer code Sampling and ltering Antireset windup Design calculations Excitation Safety nets It is important that the computer code be organized so that as little delay as possible is introduced by the controller. In STRs this usually implies that the estimation and the design calculations are done after the controlled signal is sent out to the process. The latest measurement is thus used in the computation of the control signal. The estimation and the design are then performed, which implies that the controller parameters are based on estimates from the previous sampling instant. This is usually no drawback since the estimated parameters are changing very little between samples, after the initial transient. In all sampled-data controllers it is important that the sampling interval be chosen properly. The sampling interval should be chosen in relation to the desired closed-loop behavior. A common rule of thumb is that there should be four to ten samples per rise time of the closed-loop system. It is also necessary to lter the analog signals before they are sampled. The reason is the aliasing effect, which
implies that all frequencies over the Nyquist frequency /h, where h is the sampling period, will be interpreted as a lower frequency signal after the sampling. These lters are called antialiasing lters. In the design of the controllers it is important also to incorporate the dynamics of the antialiasing lters since they introduce a phase lag in the system. The dynamics of the antialiasing lters will automatically be included in the estimated dynamics when self-tuning or adaptive controllers are used. It may be necessary only to increase the order of the estimated model to incorporate the lters into the estimated dynamics. The indirect STRs contain a design calculation that normally involves the solution of a diophantine equation such as Eq. (8). This equation has no solution if the A* and B* polynomials have a common factor that is not also a factor in A c . This also implies that the solution of the diophantine equation is a numerically ill-conditioned problem if there are almost common factors in A* and B*. These polynomials are obtained through estimation and there is no guarantee that there are no common factors. The factors that are close must thus be eliminated before solving the diophantine equation. Parameter estimation is a crucial element of STRs. The estimation is relatively simple for processes with disturbances and set-point changes that excite the process all the time. If there is not enough excitation of the process, it is necessary to introduce a logical condition in the algorithm that ensures that controller parameters are not changed when there is no excitation of the process. The design of a good safety net for an STR is a difcult task that requires thorough knowledge of the details of the algorithms and an understanding of where difculties may occur. Experience shows that a good safety net normally occupies much more code than the basic controller algorithm.
BIBLIOGRAPHY
1. R. E. Kalman, Design of self-optimizing control systems. ASME Trans., 80: 468478, 1958. om, B. Wittenmark, On self-tuning regulators. Auto2. K. J. Astr matica, 9: 185199, 1973. 3. G. C. Goodwin, P. J. Ramadge, P. E. Caines, Discrete-time multivariable adaptive control, IEEE Trans. Autom. Control, AC25: 449456, 1980. 4. P. E. Wellstead, M. B. Zarrop, Selftuning Systems: Control and Signal Processing, Chichester, U.K.: Wiley, 1991. om, B. Wittenmark, Adaptive Control, 2nd ed. Read5. K. J. Astr ing, MA: Addison-Wesley, 1995. 6. L. Ljung, System IdenticationTheory for the User. Englewood Cliffs, NJ: Prentice-Hall, 1987. 7. T. S oderstr om, P. Stoica, System Identication. Hemel Hempstead, U.K.: Prentice-Hall International, 1988. 8. R. Johansson, System Modeling and Identication. Englewood Cliffs, NJ: Prentice-Hall, 1993. om, B. Wittenmark, Computer-Controlled Systems, 9. K. J. Astr 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1997. 10. K. F. Gauss, Theoria motus corposum coelestium, (1809). English translation, Theory of the Motion of the Heavenly Bodies. New York: Dover, 1963.
BJORN WITTENMARK
Lund University Lund, Sweden
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Singularly Perturbed Systems Standard Article Petar V. Kokotovic1, Joe H. Chow2, Hassan K. Khalil3 1University of California, Santa Barbara, Santa Barbara, CA, 2Rensselaer Polytechnic Institute, Troy, NY, 3Michigan State University, East Lansing, MI, Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1038 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (225K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Time-Scale Properties of the Standard Model Examples Stability Analysis Composite Feedback Control Applications to Large Power Systems Further Reading Keywords: two-time-scale systems; model order reduction; slow manifold; boundary layer; relaxation oscillation; multiple time scales; singular perturbation; stability properties under perturbations About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
302
SINGULARLY PERTURBED SYSTEMS
+ u C
+ v
Figure 1. RLC circuit.
to neglect it and replace the second-order RLC equation (1) by the rst-order RC equation +v = u RC v (2)

Many models of dynamic systems contain small parameters multiplying some of the time derivatives. When such small parameters are neglected, the dynamic order of the model is usually reduced, as illustrated by the series RLC circuit LCv + RCv +v = u (1)
in Fig. 2. Neglecting several such parasitic parameters (small time constants, masses, moments of inertia, etc.) often leads to a signicant simplication of a high-order model. To validate such simplications, we must examine whether, and in what sense, a lower-order model approximates the main phenomenon described by the original high-order model. To see the issues involved, consider the RLC circuit in Eq. (1) with u 0. Its free transients are due to the amount of energy stored in C and L, that is, the initial conditions v(0) and v (0), respectively. The simplied model in Eq. (2) disregards the transient due to v (0), that is, the dissipation of energy stored in the inductance L. When L is small, this transient is fast, and after a short initial time, the RC equation (2) provides an adequate description of the remaining slow transient due to the energy stored in C. The RLC circuit in Eq. (1) with a small L is a two-timescale system, and the RC circuit in Eq. (2) is its slow timescale approximation. In higher-order models, several small parameters may cause a multi-time-scale phenomenon, which can be approximated by nested two-time-scale models. In this article we consider only the two-time-scale systems. In this example, a parameter perturbation from L 0 to L 0 has resulted in a model order reduction. Such parameter perturbations are called singular, as opposed to regular perturbations, which do not change the model order. For example, if instead of L, the small parameter is R, then its perturbation from R 0 to R 0 leaves the order of the RLC equation (1) unchanged. The resulting undamped sinusoidal oscillation is due to both v(0) and v (0). In the engineering literature of the past 30 years, singular perturbation techniques and their applications have been discussed in hundreds of papers and a dozen of books. This article presents only the basic singular perturbation tools for reduced-order modeling and systematic approximation of twotime-scale systems. Our main sources are the textbook by Kokotovic, Khalil, and OReilly (1) and the IEEE collection of
+ u C
+ v
in Fig. 1, where v is the capacitor voltage and u is the applied voltage. If the inductance L is very small, then it is common
Figure 2. RC circuit.
303
benchmark papers edited by Kokotovic and Khalil (2). These books and the references therein are recommended for further reading. TIME-SCALE PROPERTIES OF THE STANDARD MODEL Several examples in the next section will show that a common state-space model of many two-time-scale systems is x = f (x, z, , t ), z = g(x, z, , t ), x(t0 ) = x0 z (t0 ) = z0 (3a) (3b)
clear that we cannot expect z to converge to its quasi-steady state z unless certain stability conditions are satised. To analyze the stability of the fast transient, we perform the change of variables y z (x, t), which shifts the quasisteady state of z to the origin. Then Eq. (3b) becomes
y = g(x, y + (x, t ), , t )
f (x, y + (x, t ), , t ), t x y(t0 ) = z0 (x0 , t0 )
(7)
where x Rns, z Rnf, and 0 is the small singular perturbation parameter. The parameter represents the small time constants and other parasitics to be neglected in the slow time-scale analysis. The rate of change of z in Eq. (3b) is of order 1/ , that is, z O(1/ ), which means that z exhibits a fast transient. When this fast transient settles, the longer-term behavior of x and z is determined by the quasi-steady-state equation g(x, z, 0, t ) = 0 (4)
Let us note that y may remain nite even when tends to zero and y tends to innity. We introduce a fast time variable by setting dy d 1 dy = ; hence, = dt d dt and use 0 as the initial value at t t0. The fast time variable (t t0)/ is stretched: if tends to zero, tends to innity even for nite xed t only slightly larger than t0. In the scale, system (7) is represented by
where the bar indicates that this equation is obtained by setting 0 in Eq. (3b). This equation will make sense only if it has one or several distinct (isolated) roots z = (x, t ) (5)
dy = g(x, y + (x, t ), , t ) f (x, y + (x, t ), , t ), d t x y(0 ) = z0 (x0 , t0 ) (8)

In the fast time-scale , the variables t and x are slowly varying because t t0 and x x(t0 ). Setting 0 freezes these variables at t t0 and x x0 and reduces Eq. (8) to the autonomous system dy = g(x0 , y + (x0 , t0 ), 0, t0 ), d y(0 ) = z0 (x0 , t0 ) (9)
for all x and z of interest. If this crucial requirement is satised, for example, when det(g / z) 0, then we say that system (3) is a standard model. The substitution of Eq. (5) into Eq. (3a) results in the reduced model = f (x, (x, t ), 0, t ), x x (t0 ) = x0 (6)
which has equilibrium at y 0. The frozen parameters (x0, t0) in Eq. (9) depend on the given initial state and initial time. Tikhonovs Theorem
If Eq. (4) has several distinct roots as shown in Eq. (5), then each of them leads to a distinct reduced model as shown in Eq. (6). The singular perturbation analysis determines which of these models provides an O() approximation of the slow phenomenon in system (3). When, and in what sense, will x(t), z(t) obtained from Eqs. (6) and (5) be an approximation of the true solution of system (3)? To answer this question, we examine the variable z, which has been excluded from the reduced model in Eq. (6) by z (x, t). In contrast to the original variable z, which starts at t0 from a prescribed z0, the quasi-steady state z is not free to start from a prescribed value, and there may be a large discrepancy between z(t0) (x0, t0) and the prescribed initial state z0. Thus, z(t) cannot be a uniform approximation of z. The best we can expect is that the approximation z z(t) O() will hold on an interval excluding t0, that is, for t [tb, tf ] where tb t0. On the other hand, it is reasonable to expect the approximation x x(t) O() to hold uniformly for all t [t0, tf ] because x(t0) x(t0). If the error z z(t) is indeed O() over [tb, tf ], then it must be true that during the initial (boundary-layer) interval [t0, tb] the variable z approaches z. Let us remember that the speed of z can be large since z g / . In fact, having set 0 in Eq. (3b), we have made the transient of z instantaneous whenever g 0. It is
In our investigation of the stability of the origin of the system in Eq. (9), we should allow the frozen parameters to take any values in the domain of interest. Therefore, we rewrite the system in Eq. (9) as dy = g(x, y + (x, t ), 0, t ) d (10)
where (x, t) are treated as xed parameters. We refer to the system in Eq. (10) as the boundary-layer system and assume its exponential stability, uniform in the frozen parameters; that is, y( ) k y(0 ) e (11)
for some positive constants k and . Furthermore, we assume that y(0) belongs to the region of attraction of the origin. Under these conditions, a fundamental result of singular perturbation theory, called Tikhonovs Theorem, guarantees that the approximations x = x(t ) + O( ) (t / ) + O( ) z = (x(t ), t ) + y (12a) (12b)
304
() is the solution of the hold uniformly for t [t0, tf ], where y system in Eq. (9). Moreover, given any tb t0, the approximation z = (x(t ), t ) + O( ) (13)
which (x, ) must satisfy for all x of interest and all (0, *]. This is a partial differential equation, which, in general, is difcult to solve. However, its solution can be approximated by the power series (x, ) = 0 (x ) + 1 (x ) +
2
holds uniformly for t [tb, tf ]. Local exponential stability of the boundary-layer system can be guaranteed with the eigenvalue condition Re g (x, (x, t ), 0, t ) z c < 0 (14)
2 (x ) +
(21)
where the functions 0(x), 1(x), . . ., can be found by equating the terms with like powers in . To this end, we expand f and g as power series of f (x ) + (22) z 1 g (x ) + (23) g(x, 0 (x ) + 1 (x ) + ) = g(x, 0 (x )) + z 1 f (x, 0 (x ) + 1 (x ) + ) = f (x, 0 (x )) + where all the partial derivatives are evaluated at x and z 0(x). We substitute Eqs. (22) and (23) into Eq. (20). The terms with 0 yield g(x, 0 (x )) = 0, that is, (x, 0 ) = 0 (x ) (24)
for all (x, t) in the domain of interest, where denotes the eigenvalues and c is a positive constant. Alternatively, it can be veried by a Lyapunov analysis if there is a Lyapunov function W(x, y, t) that depends on (x, t) as parameters and satises c1 y
2
W (x, y, t ) c2 y
(15)
2
W g(x, y + (x, t ), 0, t ) c3 y y
(16)
over the domain of interest, where c1 to c3 are positive constants independent of (x, t). Slow Manifold In the state-space Rns Rnf of (x, z), the equation g(x, z, 0, t) 0 forces x and z to lie in an ns-dimensional quasi-steadystate manifold M, explicitly described by Eq. (5). It can be shown that, under the conditions of Tikhonovs Theorem, there exists an * 0 such that for all (0, *], the system in Eq. (3) possesses an integral manifold M that is invariant: whenever x(t0), z(t0) M, then x(t), z(t) M for all t [t0, tf ]. The slow manifold M is in the -neighborhood of the quasisteady-state manifold M. We will characterize M in the special case when f and g in the system in Eq. (3) do not depend on t and : x = f (x, z ) z = g(x, z ) (17a) (17b)
which is the quasi-steady-state manifold M. Equating the 1 terms, we get g 0 (x ) (x ) = f (x, 0 (x )) z 1 x (25)
For the standard model, det(g / z) 0, that is, g / z is nonsingular, so that 1 (x ) = g z

1
0 (x ) f (x, 0 (x )) x
(26)
This recursive process can be repeated to nd the higherorder terms in Eq. (21). The fast off-manifold variable is y = z (x, ) In the x,y coordinates, the system in Eq. (17) becomes (27)
x = f (x, (x, ) + y ) y = g(x, (x, ) + y ) f (x, (x, ) + y ) x
(28a) (28b)
A derivation of slow manifolds for systems with f and g also dependent on t and is given in Ref. (26). We will seek the graph of M in the explicit form M : z (x, ). The existence of M gives a clear geometric meaning to the slow subsystem of the full-order model in Eq. (17): it is the restriction of the model in Eq. (17) to the slow manifold M, given by x = f (x, (x, )) (18)
In these coordinates, the slow manifold M is simply y 0, that is, the equilibrium manifold of Eq. (28b). The geometry of a third-order system in Eq. (17) with x R2 and z R1 is illustrated in Fig. 3. Starting from an off-manifold initial conz z(0)
To nd M, we differentiate the manifold z (x, ) with respect to t d (x, ) = x z = dt x

M
(19)
x2 x(0) x1
Figure 3. Trajectory converging to a slow manifold.
and, upon the multiplication by and the substitution for x and z , we obtain the slow manifold condition f (x, (x, )) = g(x, (x, )) x (20)
305
dition (x(0), z(0)), the state trajectory rapidly converges to M and then slowly evolves along M. Linear Two-Time-Scale Systems The manifold condition in Eq. (20) is readily solvable for linear two-time-scale systems x = A11 x + A12 z + B1 u, z = A21 x + A22 z + B2 u, x(t0 ) = x0 z (t0 ) = z0 (29a) (29b)
applied to Eq. (32) results in the block-diagonal system
A A12 L = 11 y 0 +
0 A22 + LA12
y (39)
B1 H (B2 + LB1 ) u B2 LB1
with the initial condition
where A22 is nonsingular, which corresponds to det(g / z) 0, and u Rm is the control input vector. The change of variables y = z + L ( )x (30)
(t0 ) x H (z0 + Lx0 ) = 0 y(t0 ) z0 + Lx0
(40)
where the nf ns matrix L() satises the matrix quadratic equation A21 A22 L + LA11 LA12L = 0 (31)
For sufciently small, Eq. (38) admits a unique solution H() that can be expressed as
1 H ( ) = A12 A 22 + O ( )
(41)
The solution H() can also be computed iteratively as

1 1 Hk+ 1 = A12 A 22 + ((A11 A12 L )Hk + Hk LA12 )A22 , 1 H0 = A12 A 22
transforms the system in Eq. (29) into a block-triangular system
(42)
x A11 A12 L = y 0
A12 A22 + LA12
x B1 + u y B2 LB1 (32)
with the initial condition
If L is available from the recursive formula (36), we can use Lk1 instead of L in Eq. (42). From the block-diagonal form in Eq. (39), it is clear that the slow subsystem of Eq. (29) is approximated to O() by = A 0 + B 0 u, (t0 ) = x0 (43)
x(t0 ) x0 = y(t0 ) z0 + Lx0
(33)
Note that Eq. (31) is the slow manifold condition in Eq. (20) for linear systems. Given that A22 is nonsingular, the implicit function theorem implies that Eq. (31) admits a solution L() for sufciently small. Furthermore, an asymptotic expansion of the solution to Eq. (31) is given by
1 2 1 2 L ( ) = A 22 A21 + A22 A21 (A11 A12 A22 A21 ) + O ( )
1 1 where A0 A11 A12A 22 A21 and B0 B1 A12A22 B2. The fast subsystem is approximated to O() by
y = A22 y + B2 u,
1 y(t0 ) = z0 A 22 A21 x0
(44)
(34)
We can readily verify that for 0, Eq. (31) reduces to A21 A22 L(0 ) = 0 (35)
Thus as 0, the slow eigenvalues of Eq. (29) are approximated by (A0), and the fast eigenvalues are approximated by (A22)/ . It follows that if Re(A0) 0 and Re(A22) 0, then there exists an * 0 such that (29) is asymptotically stable for all (0, *]. Furthermore, if the pair (A0, B0) and the pair (A22, B2) are each completely controllable (stabilizable), then there exists an * 0 such that Eq. (29) is completely controllable (stabilizable) for all (0, *]. EXAMPLES Example 1. An RLC Circuit To complete our introductory example, we represent the RLC circuit in Eq. (1) using the state variables x v and z v :
whose solution is the rst term in Eq. (34). Furthermore, to solve for the higher-order terms, an iterative scheme
1 1 Lk + 1 = A 22 A21 + A22 Lk (A11 A12 Lk ), 1 L0 = A 22 A21
(36)
can be used. Although the fast variable y in the triangular system in Eq. (32) is now decoupled from the slow variable x, the slow variable x is still driven by the fast variable y. To remove this inuence, the change of variables = x H ( )y (37)
x =z z = z 1 (x u ) RC
(45a) (45b)
where the ns nf matrix H() satises the linear matrix equation (A11 A12 L )H H (A22 + LA12 ) + A12 = 0 (38)
where is the small time constant L / R. The unique solution of the quasi-steady-state equation (5) is z (x u)/(RC), which yields the reduced-order model = x + u RC x (46)
306
Speed (rad/s)
As expected, this is the RC equation (2). The boundary-layer system Eq. (9) for y z (x u)/(RC) is dy = y, d y(t0 ) = z (t0 ) + 1 (x(t0 ) u(t0 )) RC (47)
52.6 52.5 52.4 52.3 52.2 0 0.1 0.2 0.3 Time (s) 0.4 0.5
Example 2. A dc Motor A common model for dc motors, shown in Fig. 4, under constant eld excitation, consists of a mechanical torque equation and an equation for the electrical transient in the armature circuit, namely, J = Ki TL = K Ri + u Li (48a) (48b)
Current (A)
Its solution y ey(t0) e(R / L)ty(t0) approximates the fast transient neglected in the slow subsystem in Eq. (46). Tikhonovs Theorem is satised, because the fast subsystem in Eq. (47) is exponentially stable.
3.45
3.40 0 0.1 0.2 0.3 Time (s) 0.4 0.5
Figure 5. dc motor step response.
where i, u, R, and L are the armature current, voltage, resistance, and inductance, respectively, J is the combined moment of inertia of the motor and the load, is the angular speed, TL is the load torque, and K is a motor design constant such that Ki and K are, respectively, the motor torque and the back emf (electromotive force). We consider the case in which the electrical time constant e L / R is much smaller than the mechanical time constant m JR / K2 (3). Dening e / m, x , and z i, we rewrite Eq. (48) as
x =
1 R z TL m K J 1 1 K x u z+ z = m R m m R
(49a) (49b)
The parameters of a 1 hp (746 W) dc motor with a rated speed of 500 rpm (52.3 rad/s) are R 7.56 , L 0.055 H, K 4.23 Vs/rad, and J 0.136 kg m2. At the rated steady state condition, TL 14.3 N m, i 3.38 A, and u 246.8 V. The time constants are e 0.0073 s and m 0.115 s, resulting in 0.063. The response to a 1 V step increase in u of the full model in Eq. (48) (solid curves) and the slow subsystem in Eqs. (51) and (52) (dashed curves) is shown Fig. 5. Note that x is a good approximation of x. Initially there is a fast transient in z. After this boundary layer has decayed, z becomes a good approximation of z. The fast electrical transient is approximated by the boundary-layer system m dy = y, d y (0 ) = z (0 ) u(0 ) Kx(0 ) R (53)
Setting 0, we obtain from Eq. (49b) 0 = K x Rz + u Thus the quasi-steady state of z is uKx z= R (51) (50)
which has the unique solution y = e / m y(0 ) = et / e y(0 ) (54)
Example 3. Multiple Slow Subsystems To illustrate the possibility of several reduced-order models, we consider the singularly perturbed system x = x2 (1 + t )/z z = [z + (1 + t )x] z [z (1 + t )] (55a) (55b)
which, when substituted in Eq. (49a), yields the slow mechanical subsystem = x + m x m 1 u T K J L (52)
where the initial conditions are x(0) 1 and z(0) z0. Setting 0 results in 0 = [z + (1 + t )x] z [z (1 + t )] (56)
L J TL
which has three distinct roots z = (1 + t )x; z = 0; z = 1+t (57)
+ u
Consider rst the root z (1 t)x. The boundary-layer system in Eq. (10) is dy = y[y (1 + t )x][y (1 + t )x (1 + t )] d (58)
Figure 4. dc motor.
307
Taking W(y) y2 /2, it can be veried that W satises the inequalities in Eqs. (15) and (16) for y (1 t) x. The reduced system = x, x x (0 ) = 1 (59)
ds ks
ms
Car body
has the unique solution x(t) et for all t 0. The boundarylayer system with t 0 and x 1 is dy = y (y 1 )(y 2 ), d y (0 ) = z0 + 1 (60)
bs
mu du kt dr
Tire
() for z0 and has a unique exponentially decaying solution y 1 1 a, that is, for z0 a 0 where a 0 can be arbitrarily small. Consider next the root z 0. The boundary-layer system in Eq. (10) is dy = [y + (1 + t )x] y [ y (1 + t )] d (61)
Road surface Reference
Figure 7. Quarter-car model.
By sketching the right-hand side function, it can be seen that the origin is unstable. Hence, Tikhonovs Theorem does not apply to this case and we rule out this root as a subsystem. Furthermore, x is not dened at z 0. Finally, the boundary-layer system for the root z 1 t is dy = [ y + (1 + t ) + (1 + t )x][y + (1 + t )] y d (62)
full model (solid curves) and two for each reduced model (dashed curves). Example 4. A Quarter-Car Model A simplied quarter-car model is shown in Fig. 7, where ms and mu are the car body and tire masses, ks and kt are the spring constants of the strut and the tire, and bs is the damper constant of the shock. The distances ds, du, and dr are the elevations of the car, the tire, and the road surface, respectively. From Newtons Law, the balance of forces acting on ms and mu results in the modeling equations
As in the rst case, it can be shown that the origin is exponentially stable uniformly in (x, t). The reduced system = x2 , x ( 0 ) = 1 x (63)
s d u ) + ks (ds du ) = 0 s + bs (d ms d u d s ) + ks (du ds ) + k (du dr ) = 0 u + bs (d mu d t
(65a) (65b)
has the unique solution x(t) 1/(1 t) for all t [0, 1). Notice that x(t) has a nite escape time at t 1. However, Tikhonovs Theorem still holds for t [0, tf] with tf 1. The boundary-layer system, with t 0 and x 1, dy = (y + 2 )(y + 1 )y , d y (0 ) = z0 1 (64)
In a typical car, the natural frequency kt / mu of the tire is much higher than the natural frequency ks / ms of the car body and the strut. We therefore dene the parameter = ks /ms = kt /mu k s mu k t ms (66)
() for z0 has a unique exponentially decaying solution y a 0. In summary, only two of the three roots in Eq. (57) give rise to valid reduced models. Tikhonovs Theorem applies to the root (1 t)x if z0 0 and to the root 1 t if z0 0. Figure 6 shows z for four different values of z0 of the
2 z 1 0 1 2 0
The mass-spring system in Eq. (65) is of interest because it cannot be transformed into a standard model without an dependent scaling. From Eq. (66), the tire stiffness kt O(1/ 2) tends to innity as 0. For the tire potential energy kt(du dr)2 /2 to remain bounded, the displacement du dr must be O(), that is, the scaled displacement (du dr)/ must remain nite. Thus to express Eq. (65) in the standard singularly perturbed form, we introduce the slow and fast variables as
x=
ds du , s d
z=
(du dr )/ u d
(67)
0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Time (s)
r as the disturbance input. The resulting model is and u d

x = A11 x + A12z + B1 u z = A21 x + A22z + B2 u (68a) (68b)
Figure 6. Response of system in Eq. (55) illustrating two slow subsystems.
308
where
A11 = A21 = B2 =
and
0 ks /ms 0 ks /ms 1 0
1 , bs /ms 0 , bs /ms
A12 = A22 =
0 0
1 , bs /ms
B1 =
0 0
0 ks /ms
1 , bs /ms (69)
Following the input signal VR, the voltage E modulates the supply voltage VB(x) to control Efd. The supply voltage VB(x) is a function of x and is typically within the range of 48 per unit on the base eld voltage. Under normal operating conditions, the product of the rectier gain and the supply voltage KMVB(x) is very high. This motivates us to dene as = 1 KMV B 1 (73)
k s ms k t mu
(70)
where VB is a constant of the order of magnitude of VB(x), that is, the ratio (x) VB(x)/ VB O(1). Using Efd as the state variable instead of E, Eq. (72) is rewritten as
The parameters of a typical passenger car are ms 504.5 kg, mu 62 kg, bs 1,328 Ns/m, ks 13,100 N/m, kt 252,000 N/m, and 0.65. In this case, 0.08 and the time scales of the body (slow) and the tire (fast) are well separated. To illustrate the approximation provided by the two-timescale analysis, the slow and fast eigenvalues of the uncorrected subsystems in Eqs. (43) and (44) are found to be 2.632 j6.709 and 10.710 j62.848, respectively, which are within 4% of the eigenvalues 2.734 j7.018 and 9.292 j60.287 of the full-order model. If a higher accuracy is desired, the series expansion in Eq. (34) can be used to add the rst-order correction terms to the diagonal blocks of Eq. (39), resulting in the approximate slow eigenvalues 2.705 j6.982 and fast eigenvalues 9.394 j60.434, and thus reducing the errors to less than 0.5%. Example 5. A High-Gain Power Rectier Many modern control systems include power electronic rectiers as actuators. An example is a static excitation system that controls the eld voltage Efd of a synchronous machine shown in Fig. 8 (4). The synchronous machine is modeled as x = f (x, Efd ) VT = h(x ) (71a) (71b)
= 1 + K K V (x ) TM (x, E ) E TM E fd G M B fd fd VB (x ) + KMVB (x )VR
(74)
where (x, Efd) [VB(x)/ x]x is bounded. Using , Eq. (74) becomes = KG (x ) + E fd TM VB (x ) (x, Efd ) Efd + (x ) V TM R (75)
which yields, as 0, the quasi-steady state E fd = 1 V KG R (76)
In a typical rectier system, KG 1, KM 7.93, and TM 0.4 s. For VB 6, we obtain 0.021. The time-scales are well separated, which allows us to achieve high accuracy with singular perturbation approximations. Example 6. Slow Manifold in a Synchronous Machine We now proceed to illustrate the use of the slow manifold concept as a modeling tool. In most cases, the solution of the manifold condition in Eq. (20) is evaluated approximately as a power series in . However, for a synchronous machine model, the slow manifold, which excludes stator circuit transients, can be calculated exactly, as shown by Kokotovic and Sauer (5). The synchronous machine model with one damper winding in the quadrature axis is
where x is the machine state vector including the ux variables and the scalar output VT is the generator terminal voltage. Here we focus on the exciter system which, from Fig. 8, is described by
= E K K E + K V TM E M G fd M R Efd = VB (x )E
where TM is a time constant and KM and KG are gains.
(72a) (72b)
d = s dt 2H d 1 1 = Tm + s dt Lq Ld Tdo Tqo dEq dt =
(77a) d q + 1 1 E + q Eq (77b) Lq d d Ld (77c) (77d) (77e) (77f)
L Ld Ld E d d + Efd Ld q Ld
KG VR + KM 1 + sTM E V B( x ) Efd Synchronous machine VT
Lq Lq dEd Lq = Ed q dt Lq Lq
Ra Ra 1 d d = d + E + q + V sin s dt Ld Ld q s 1 d q Ra Ra = q Ed + V cos s dt Lq Lq s d
Figure 8. Static excitation system.
where , , and H are the generator rotor angle, speed, and inertia, respectively, (Ed , Eq ), (d, q), (Td o, Tq o), (Ld, Lq), and
309
(Ld , Lq ) are the d- and q-axis voltages, ux linkages, opencircuit time constants, synchronous reactances, and transient reactances, respectively, Ra is the stator resistance, Tm is the input mechanical torque, Efd is the excitation voltage, and s is the system frequency. In the model shown in Eq. (77), the slow variables are , , Ed , and Eq , and the fast variables are d and q. The singular perturbation parameter can be dened as 1/ s. If the stator resistance is neglected, that is, Ra 0, it can be readily veried that the slow manifold condition in Eq. (20) gives the exact slow invariant manifold d = V cos , q = V sin (78)
3 A 2 1 0 C D 0 x 0.5 1
z
A B
1 2
D 1.5 2
3 2 1.5 1 0.5
These expressions can be substituted into Eqs. (77a)(77d) to obtain a fourth-order slow subsystem. If the initial condition [d(0), q(0)] is not on the manifold shown in Eq. (78), then using the fast variables yd = d V cos , we obtain the fast subsystem yq = q + V sin (79)
Figure 9. Phase portrait of the Van der Pol oscillator.
dyd = yq , yd (0 ) = d (0 ) V cos (0 ) dt s dyq = yd , yq (0 ) = q (0 ) + V sin (0 ) dt s
(80a) (80b)
than boundary) layers, from B to D and from C to A is clear from this phase portrait. The relaxation oscillation forming the limit cycle A-B-D-C consists of the slow motions from A to B and D to C, connected with fast jumps (layers) from B to D and from C to A. When is increased to a small positive value, this limit cycle is somewhat deformed, but its main character is preserved. This observation is one of the cornerstones of the classical nonlinear oscillation theory (7), which has many applications in engineering and biology.
where the state appears as a time-varying coefcient. When the stator resistance Ra is nonzero, the slow manifold condition can no longer be solved exactly. Instead, the leading terms in the power series expansion in Eq. (21) can be computed to obtain any desired approximation of the slow manifold. Example 7. Van der Pol Oscillator A classical use of the slow manifold concept is to demonstrate the relaxation oscillation phenomenon in a Van der Pol oscillator, modeled in the state-space form as (6)
STABILITY ANALYSIS We consider the autonomous singularly perturbed system in Eq. (17). Let the origin (x 0, z 0) be an isolated equilibrium point and the functions f and g be locally Lipschitz in a domain that contains the origin. We want to analyze stability of the origin by examining the reduced and boundary-layer models. Let z (x) be an isolated root of 0 g(x, z) dened in a domain D1 Rn that contains x 0, such that (x) is continuous and (0) 0. With the change of variables y z (x), the singularly perturbed system is represented in the new coordinates as
x =z z = x + z
1 3 3z
(81a) (81b)
x = f (x, y + (x )) y = g(x, y + (x )) f (x, y + (x )) x
(84a) (84b)
For small, the slow manifold is approximated by g(x, z ) = x + z 1 z3 = 0 3 (82)
which is shown as the curve ABCD in Fig. 9. For the roots z (x) on the branches AB and CD g = 1 z2 < 0 z (83)
and the eigenvalue condition of Tikhonovs Theorem is satised because z2 1. Therefore, the branches AB and CD are attractive, that is, trajectories converging to these two branches will remain on them, moving toward either the point B or C. However, the root on the branch BC is unstable because z2 1; hence, this branch of the slow manifold is repulsive. Figure 9 shows vertical trajectories converging toward AB and CD, because 0. The mechanism of two interior (rather
The reduced system x f (x, (x)) has equilibrium at x 0, and the boundary-layer system dy / d g(x,y (x)) has equilibrium at y 0. The main theme of the two-time-scale stability analysis is to assume that, for each of the two systems, the origin is asymptotically stable and that we have a Lyapunov function that satises the conditions of Lyapunovs Theorem. In the case of the boundary-layer system, we require asymptotic stability of the origin to hold uniformly in the frozen parameter x. Viewing the full singularly perturbed system (84) as an interconnection of the reduced and boundary-layer systems, we form a composite Lyapunov function candidate for the full system as a linear combination of the Lyapunov functions for the reduced and boundary-layer systems. We then proceed to calculate the derivative of the composite Lyapunov function along the trajectories of the full system and verify, under reasonable growth conditions on f and g, that it is negative denite for sufciently small .
310
Let V(x) be a Lyapunov function for the reduced system such that V 2 f (x, (x )) 1 1 (x ), x x D1 (85)
The right-hand side of the last inequality is a quadratic form in (1(x), (2(y)). The quadratic form is negative denite when d (1 d )1 2 > 1 [(1 d )1 + d2 ]2 4 (93)
where 1(x) is a positive denite function. Let W(x, y) be a Lyapunov function for the boundary-layer system such that W 2 g(x, y + (x )) 2 2 ( y ), y (x, y ) D1 D2 (86)
For any d, there is an d such that Eq. (93) is satised for d. The maximum value of d occurs at d* 1 /(1 2) and is given by
1 2 1 + 1 2
(94)
where D2 Rm is a domain that contains y 0, and 2(y) is a positive denite function. We allow the Lyapunov function W to depend on x because x is a parameter of the system and Lyapunov functions may, in general, depend on the systems parameters. Because x is not a true constant parameter, we must keep track of the effect of the dependence of W on x. To ensure that the origin of the boundary-layer system is asymptotically stable uniformly in x, we assume that W(x, y) satises W1 ( y ) W (x, y ) W2 ( y ), (x, y ) D1 D2 (87)
It follows that the origin is asymptotically stable for all *. Note that the functions 1(x) and 2(x) and the interconnection conditions in Eqs. (90) and (91) should be carefully constructed so that * may not be unnecessarily conservative. As an illustration of the preceding Lyapunov analysis, consider the second-order system x = f (x, z ) = x x3 + z z = g(x, z ) = x z (95a) (95b)
for some positive denite continuous functions W1 and W2. Now consider the composite Lyapunov function candidate (x, y ) = (1 d )V (x ) + dW (x, y ), 0<d<1 (88)
where the constant d is to be chosen. Calculating the derivative of along the trajectories of the full system in Eq. (84), we obtain
which has a unique equilibrium point at the origin. Let y z (x) z x. For the reduced system x x3, we take V(x) x4, which satises Eq. (85) with 1(x) x3 and 1 1. For the boundary-layer system dy / d y, we take W(y) y2, which satises Eq. (86) with 2(y) y and 2 1. As for the interconnection conditions in Eqs. (90) and (91), we have V [ f (x, y + (x )) f (x, (x ))] = x3 y 1 2 x W 2 f (x, y + (x )) = y(x3 + y ) 1 2 + 2 y (96) (97)
d W V f (x, (x )) + g(x, y + (x )) = (1 d ) x y V [ f (x, y + (x )) f (x, (x ))] + (1 d ) x W W +d f (x, y + (x )) x y x
(89)
We have represented the derivative as the sum of four terms. The rst two terms are the derivatives of V and W along the trajectories of the reduced and boundary-layer systems. These two terms are negative denite in x and y, respectively, by the inequalities in Eqs. (85) and (86). The other two terms represent the effect of the interconnection between the slow and fast dynamics, which is neglected at 0. Suppose these terms satisfy the interconnection conditions V [ f (x, y + (x )) f (x, (x ))] 1 1 (x )2 ( y ) x W W x y x (90)
2 f (x, y + (x )) 2 1 (x )2 (y ) + 2 (y ) (91)
for some nonnegative constants 1, 2, and . Using the inequalities in Eqs. (85), (86), (90), and (91), we obtain
Note that W / x 0. Hence, Eqs. (90) and (91) are satised with 1 2 1. Therefore, the origin is asymptotically stable for * 0.5. Because all the conditions are satised globally and (x, y) (1 d)V(x) dW(y) is radially unbounded, the origin is globally asymptotically stable for 0.5. The preceding two-time-scale stability analysis can be extended to the nonautonomous singularly perturbed system in Eq. (3). For sufciently smooth f , g, and , it can be shown that if the origin of the reduced system is exponentially stable, and the origin of the boundary-layer system is exponentially stable, uniformly in (t, x), then the origin of the full system in Eq. (3) is exponentially stable for sufciently small . The same conditions ensure the validity of Tikhonovs Theorem for all t t0. Note that the earlier statement of Tikhonovs Theorem, which does not require exponential stability of the reduced system, is valid only on a compact time interval [t0, tf] for a given tf. COMPOSITE FEEDBACK CONTROL The impetus for the systematic decomposition of the slow and fast subsystems in singularly perturbed systems can be readily extended to the separate control design of the slow and fast dynamics. As will be shown, the crucial idea is to compensate for the quasi-steady state in the fast variable.
1 (x ) 2 ( y )
1 2 (1
(1 d )1 d )1 1 2 d 2
1 2 (1
d )1 d 2 1 (x ) 2 ( y )
1 2 d 2
(92)
311
Consider the nonlinear singularly perturbed system x = f (x, z, u ) z = g(x, z, u ) (98a) (98b)
have the desired properties. Thus the composite control in Eq. (99) is
1 u = G 0 x + G 2 [z + A 22 (A21 + B2 G0 )x] = G1 x + G2 z
(111)
and suppose the equation 0 g(x, z, u) has a unique root z (x, u) in a domain D that contains the origin. A separated slow and fast design is succinctly captured in the composite control u = us + uf where us is a slow control function of x us =
s (x )
where
1 1 G1 = (Im + G2 A 22 B2 )G0 + G2 A22 A21
(112)
(99)
(100)
and uf is a fast control function of both x and z uf =

f (x, z )
(101)
Applying the control in Eq. (99) to the full model in Eq. (98), we obtain x = f (x, z, z = g(x, z,
s (x ) s (x )
+ +
f (x, z )) f (x, z ))
(102a) (102b)
When Eq. (111) is applied to the full-order model in Eq. (29), the slow and fast dynamics of the closed-loop system are approximated to O() by the slow and fast subsystems in Eqs. (109) and (110), respectively. If the pairs (A0, B0) and (A22, B2) are completely controllable, the results here point readily to a two-time-scale poleplacement design in which G0 and G2 are designed separately to place the slow eigenvalues of A0 B0G0 and the fast eigenvalues of (A22 B2G2)/ at the desired locations. Then the eigenvalues of the closed-loop full-order system will approach these eigenvalues as tends to zero. The composite feedback control is also fundamental in near-optimal control design of linear quadratic regulators for two-time-scale systems. Consider the optimal control of the linear singularly perturbed system in Eq. (29) to minimize the performance index J (u ) = where 1 2
0
The fast control f (x, z) must guarantee that z (x, s(x)) is a unique solution to the equation 0 = g(x, z,
s (x )
(qT q + uT Ru ) dt ,
R>0
(113)
f (x, z ))
(103)
in the domain D. Furthermore, we require the fast control to be inactive on the manifold in Eq. (103), that is,
f (x, (x, s (x )))
q (t ) = C1 x(t ) + C2 z (t )
(114)
=0
(104)
Then the slow and fast subsystems become, respectively, x = f (x, (x, z = g(x, z,
s (x )), s (x )) f (x, z ))
Following the slow and fast subsystem decomposition in Eqs. (43) and (44), we separate q(t) into its slow and fast components as q (t ) = qs (t ) + qf (t ) + O( )) where qs (t ) = C0 + D0 us with
1 C0 = C1 + C2 A 22 A21 , 1 D0 = C2 A 22 B2
(105) (106)
(115)
s (x ) +
To obtain controllers so that the equilibrium (x 0, z 0) is asymptotically stable, s(x) must be designed so that a Lyapunov function V(x) satisfying Eq. (85) can be found for the slow subsystem in Eq. (105), and f (x, z) must be designed so that a Lyapunov function W(x, z) satisfying Eq. (86) can be found for the fast subsystem in Eq. (106). Furthermore, the interconnection conditions corresponding to Eqs. (90) and (91) must be satised, so that a composite Lyapunov function similar to Eq. (88) can be used to establish the asymptotic stability of the equilibrium. Specializing the composite control design to the linear singularly perturbed system in Eq. (29), we design the slow and fast controls as
s f
(116)
(117)
and qf (t ) = C2 y (118)
From the subsystems in Eqs. (43) and (44) and the decomposition in Eq. (115), the linear quadratic regulator problem in Eq. (113) can be solved from two lower-order subproblems. Slow Regulator Problem Find the slow control us for the slow subsystem in Eqs. (43) and (116) to minimize
= G0 x = G0 + O ( )
1 A 22 (A21 x
(107)
= G 2 [z +
+ B22 G0 x )]
, G2 y
(108)
such that the closed-loop subsystems in Eqs. (43) and (44) to O() = (A0 + B0 G0 ) y = (A22 + B2 G2 )y (109) (110)
Js (us ) =
1 2 1 = 2
T (qT s qs + us Rus ) dt ,
R>0 (119)
T T T T ( T C0 C0 + 2uT s D0 C0 + us R0 us ) dt
312
where R0 = R + D T 0 D0 (120)
applied to the system in Eq. (29) achieves an O() approximation of Jopt, that is, J (us ) = Jopt + O( ) (129)
If the triple (C0, A0, B0) is stabilizable and detectable (observable), then there exists a unique positive-semidenite (positive-denite) stabilizing solution Ks of the matrix Riccati equation
1 T 1 T T 0 = Ks (A0 B0 R 0 D0 C0 ) (A0 B0 R0 D0 C0 ) Ks 1 T 1 T + Ks B0 R 0 B0 Ks C0 (I D0 R0 N0 )C0
(121)
In Theorem 1.1, an asymptotic expansion exists for the solution to the matrix Riccati equation associated with the full linear regulator problem. Theorem 1.3 is one of the robustness results with respect to fast unmodeled dynamics, that is, if the fast dynamics is asymptotically stable, a feedback control containing only the slow dynamics would not destabilize the fast dynamics. APPLICATIONS TO LARGE POWER SYSTEMS
and the optimal control is us =

1 T R 0 (D0 C0
+ BT 0 Ks )
= G0
(122)
Fast Regulator Problem Find the fast control uf for the fast subsystem in Eqs. (44) and (118) to minimize
Jf (uf ) =
1 2 1 = 2
T (qT f qf + uf Ruf ) dt ,
R>0 (123)
T (yT C2 C2 y + uT f Ruf ) dt
If the triple (C2, A22, B2) is stabilizable and detectable (observable), then there exists a unique positive-semidenite (positive-denite) stabilizing solution Kf of the matrix Riccati equation
1 T 0 = Kf A22 AT 22 Kf + Kf B2 R B2 Kf C2 C2
In this section, we analyze a large power system as an example of time-scales arising in an interconnected systems. A power system dispersed over a large geographical area tends to have dense meshes of power networks serving heavily populated areas and many fewer transmission lines interconnecting these urban centers. When such a system is subject to a disturbance, it is observed that groups of closely located machines would swing coherently at a frequency that is lower than the frequency of oscillation within the coherent groups. Singular perturbation techniques have been successfully applied to these large power networks to reveal this two-timescale behavior (9). Consider the linearized electromechanical model of an n-machine power system in the second-order form with damping neglected = K M (130)
(124)
and the optimal control is uf = R 1 BT 2 Kf y = G22 y The following results are from Reference (8). Theorem 1. 1. If the triples (C0, A0, B0) and (C2, A22, B2) are stabilizable and detectable (observable), then there exists an * 0 such that for all (0, *], an optimal control exists for the linear regulator problem (113) with an optimal performance Jopt. 2. The composite control in Eq. (111)
1 1 T T u c = [ ( I R 1 B T 2 Kf A22 B2 )R0 (D0 C0 + B0 Ks ) 1 1 T + R 1 B T 2 Kf A22 A21 ]x R B2 Kf z
(125)
where Rn is the machine rotor angle vector, M is the diagonal matrix of machine inertias, and K is the stiffness matrix determined by the network impedances. Assume that the system in Eq. (130) has r tightly connected areas, with the connections between the areas being relatively fewer. In this case, we decompose K into K = KI + KE (131)
where K I is the stiffness matrix due to the impedances internal to the areas, and K E is the stiffness matrix due to the impedances external to the areas and scaled by the small parameter that represents the ratio of the external to the internal connection strength. For illustrative purposes, we let r 2, with r1 machines in area 1 and r2 machines in area 2. Arranging the machines in area 1 to appear rst in , K I has a block-diagonal structure
I I K I = block-diag(K1 , K2 )
(126)
(132)
applied to the system in Eq. (29) achieves an O( ) approximation of Jopt, that is,
2
J (uc ) = Jopt + O( 2 ) 3. If A22 is stable, then the slow control in Eq. (122) us =
1 T R 0 (D0 C0
(127)
A particular property of the ri ri matrix K I i , i 1, 2, is that each of the rows in K I i will sum to zero. If 0, this conservation property results in a slow mode in each area. When 0, the slow mode from each area will interact to form the low-frequency interarea oscillatory mode. To reveal this slow dynamics, we dene a grouping matrix
BT 0 Ks )x
(128)
U=
1r 1 0
0 1r 2
(133)
313
where 1ri is an ri 1 column vector of all ones. Using U, we introduce the aggregate machine angles weighted according to the inertia as the slow variables a = (U T MU ) 1U T M = C (134)
and the difference angles with respect to the rst machine in each area as the fast variables d = G where (135)
G=
1r 1 1 0
Ir
1 1
0 1r 1
2
0 Ir
2 1
(136)
Noting that CM1K I 0 and K IU 0, that is, C is in the left null space of M1K I and U is in the right null space of K I, the system in Eq. (130) in the new variables become
a = CM 1 K EU a + CM 1 K E G+ d d = GM
1
(137a)
1
K EU a + (GM
K I G+ + GM
K E G+ )d (137b)
where G GT(GGT)1. The system in Eq. (137) clearly points to the two-time-scale behavior in which the right-hand side of Eq. (137a) is O(), indicating that a is a slow variable. The method can readily be extended to systems with r 2 areas. Based on this time-scale interpretation, a grouping algorithm using the slow eigensubspace has been proposed to nd the tightly connected machines if they are not known ahead of time. Then the areas whose internal dynamics are of less interest can be aggregated into single machine equivalents to capture only the slow dynamics. This concept, together with some more recently developed algorithms, has been implemented in computer software to reduce large power systems to smaller models suitable for stability analysis and control design (10). FURTHER READING Singular perturbation techniques have been successfully applied to the analysis and design of many control systems other than those discussed in this article. For our concluding remarks, we briey comment on some of these applications as extensions of the results already discussed. The two-time-scale properties can also be used to characterize transfer functions of singularly perturbed systems (11). In a linear discrete-time two-time-scale system, the slow dynamics arise from the system eigenvalues close to the unit circle, whereas the fast dynamics are a result of those eigenvalues close to the origin. The two-time-scale analysis for continuous-time systems can be readily extended to discrete-time singularly perturbed systems (12). An application of the stability results is to determine the robustness of a control design. For example, Khalil (13) shows a simple example where a static output feedback designed without considering the parasitic effects would lead to an instability in the fast dynamics. Another result shows that in adaptive control, the rate of adaptation must be sufciently slow so that the unmodeled fast dynamics would not cause destabilization (14).
The composite control has also been applied to solve optimal regulator problems for nonlinear singularly perturbed systems (15). Recently, composite control results for H optimal control of singularly perturbed systems have been obtained (16, 17). The composite control idea can also be used to establish the time scales in a closed-loop system induced by a high-gain control (18). Singular perturbation methods also have signicant applications in ight control problems (19, 20). For example, twopoint boundary-value problems arising from trajectory optimization can be solved by treating the fast maneuvers and the slower cruise dynamics separately. The slow coherency and aggregation technique is also applicable to other large-scale systems, such as Markov chains (21) and multi-market economic systems (22). These systems belong to the class of singularly perturbed systems in the nonstandard form, of which an extended treatment can be found in Ref. 23. A topic not covered here is the ltering and stochastic control of singularly perturbed systems with input noise. As 0, the fast dynamics will tend to a white noise. Although the problem can be studied in two-time-scales, the convergence of the optimal solution requires that the noise input in the fast dynamics be either colored or asymptotically small (24). Another topic not covered is the control of distributed parameter systems possessing two-time-scale properties. Averaging and homogenization techniques are also a class of twotime-scale methods (25). More developments are expected in this area. In more complex singularly perturbed systems, jump behaviors may arise not only at the end points but also in interior layers. Reference 27, beside being an introductory text to singular perturbations, contains a detailed treatment of such phenomena. It also contains a historical development of singular perturbations. The singular perturbation results presented in this article represent developments over a period of three decades and contribute to the advances of modern control theory. As new control problems are proposed and new applications are discovered for systems with time-scales, we expect that singular perturbation methods will also be extended accordingly to provide simpler models to gain useful design insights into these new problems. BIBLIOGRAPHY
1. P. K. Kokotovic, H. K. Khalil, and J. OReilly, Singular Perturbation Methods in Control: Analysis and Design, London: Academic Press, 1986. 2. P. K. Kokotovic and H. K. Khalil (eds.), Singular Perturbations in Systems and Control, New York: IEEE Press, 1986. 3. A. E. Fitzgerald, C. Kingsley, and A. Kusko, Electric Machinery, 3rd ed., New York: McGraw-Hill, 1973. 4. IEEE Committee Report, Excitation system models for power system stability studies, IEEE Trans. Power Appar. Syst., PAS-100: 494509, 1981. 5. P. V. Kokotovic and P. W. Sauer, Integral manifold as a tool for reduced-order modeling of nonlinear systems: A synchronous machine case study, IEEE Trans. Circuits Syst., 36: 403410, 1985. 6. H. K. Khalil, Nonlinear Systems, 2nd ed., Upper Saddle River, NJ: Prentice-Hall, 1996.
314
SKIN EFFECT
7. J. K. Hale, Ordinary Differential Equations, Huntington, NY: Krieger Publisher Co., 1980. 8. J. H. Chow and P. V. Kokotovic, A decomposition of near-optimum regulators for systems with slow and fast modes, IEEE Trans. Autom. Control, AC-21: 701705, 1976. 9. J. H. Chow et al., Time-Scale Modeling of Dynamic Networks with Applications to Power Systems, Berlin: Springer-Verlag, 1982. 10. J. H. Chow et al., Inertial and slow coherency aggregation algorithms for power system dynamic model reduction, IEEE Trans. Power Syst., 10: 680685, 1995. 11. H. K. Khalil, Output feedback control of linear two-time-scale systems, IEEE Trans. Autom. Control, AC-32: 784792, 1987. 12. D. Naidu and A. Rao, Singular Perturbation Analysis of Discrete Time Control Systems, Lecture Notes in Mathematics, Berlin: Springer-Verlag, 1985. 13. H. K. Khalil, On the robustness of output feedback control methods to modeling errors, IEEE Trans. Autom. Control, AC-28: 524 528, 1983. 14. B. D. Riedle and P. V. Kokotovic, Integral manifold of slow adaptation, IEEE Trans. Autom. Control, AC-31: 316323, 1986. 15. J. H. Chow and P. V. Kokotovic, A two-stage Lyapunov-Bellman feedback design of a class of nonlinear systems, IEEE Trans. Autom. Control, AC-26: 656663, 1981. 16. Z. Pan and T. Basar, H-optimal control for singularly perturbed systems, Part I: Perfect state measurements, Automatica, 29: 401423, 1993. 17. Z. Pan and T. Basar, H-optimal control for singularly perturbed systems, Part II: Imperfect state measurements, IEEE Trans. Autom. Control, 39: 280299, 1994. 18. K. D. Young, P. V. Kokotovic, and V. I. Utkin, A singular perturbation analysis of high gain feedback systems, IEEE Trans. Autom. Control, 22: 931938, 1977. 19. A. J. Calise, Singular perturbation methods for variational problems in aircraft control, IEEE Trans. Autom. Control, 21: 345 353, 1976. 20. H. J. Kelley, Aircraft maneuver optimization by reduced order approximations, In C. T. Leonides, (ed.), Control and Dynamics Systems, New York: Academic Press, 131178, 1973. 21. F. Delebecque and J. P. Quadrat, Optimal control of Markov chains admitting strong and weak interactions, Automatica, 17: 281296, 1981. 22. G. Peponides and P. V. Kokotovic, Weak connections, time scales and aggregation of nonlinear systems, IEEE Trans. Autom. Control, 28: 729735, 1983. 23. H. K. Khalil, Feedback control of nonstandard singularly perturbed systems, IEEE Trans. Autom. Control, 34: 10521060, 1989. 24. A. Bensoussan, Singular perturbation results for a class of stochastic control problems, IEEE Trans. Autom. Control, 26: 1071 1080, 1981. 25. A. Bensoussan, J. L. Lions, and G. C. Papanicolaou, Asymptotic Analysis for Periodic Structures, New York: North-Holland, 1981. 26. V. A. Sobolev, Integral manifolds and decomposition of singularly perturbed systems, Systems and Control Letters, 5: 169179, 1984. 27. R. E. OMalley, Jr., Singular Perturbation Methods for Ordinary Differential Equations, Berlin: Springer-Verlag, 1991.
SINUSOIDAL STEADY STATE. See NETWORK ANALYSIS,

SINUSOIDAL STEADY STATE.
PETAR V. KOKOTOVIC
University of California, Santa Barbara
JOE H. CHOW
Rensselaer Polytechnic Institute
HASSAN K. KHALIL
Michigan State University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Software for Control System Analysis and Design: Symbol Manipulation Standard Article D. Nei1, I. M. Y. Mareels1, S. T. Glad2, M. Jirstrand3 1The University of Melbourne, Parkville, Victoria, Australia 2Linkping University, Linkping, Sweden 3MathCore AB, Linkping, Sweden Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1039 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (438K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Grbner Basis Method Differential Algebra In Control Quantifier Elimination About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION

The computer revolution has radically changed every area of engineering, and control/systems engineering is not an exception. Indeed, computers have become essential tools in modeling, analysis, design, and implementation of control systems. In particular, they enable the design engineer to tackle problems of ever increasing complexity (1). The topic of this article is the effectiveness of symbolic computation software so-called computer algebra, in the analysis and design of control systems. Symbolic computation software should be distinguished from numerical computation software. The former performs exact computation and can manipulate whole mathematical (symbolic) expressions, whereas the latter is limited to approximate computation based on numerical expressions. MATLAB (The Mathworks, Inc.) and SCILAB (Copyright INRIA; freely available via http://www-rocq.inria.fr/scilab/ are popular numerical software packages in the control engineering community. Such software assists in a wide variety of control engineering tasks, from modeling to real-time control implementation. The simulation facilities offered in these packages make them very popular in educational programs. Similarly, there are a number of readily available symbolic computation software packages. Mathematica (Wolfram Research, Inc.), Maple (Wiferloo Maple Inc.), REDUCE ( c Anthony C. Hearn), DERIVE (Software House Inc., Texas Instruments), Macaulay, COCOA, (freely available via http://cocoa.dima.unige.it and MACSYMA (Macsyma, Inc.) are some of the more popular ones. Most of these software packages incorporate numerical as well as symbolic computation so that the user can resort to numerical analysis of symbolic expressions if required. Observe also that MATLAB, through its Symbolic Math Toolbox, adds to its numerical core symbolic manipulation capability. In general the distinction between purely numerical and symbolic computation software begins to blur, the trend being to offer both capabilities in the same software environment. In general, a particular control problem can be approached using numerical and/or symbolic software. More precisely, all computational problems can be classied as (2 p. 275): (1) Purely Numerical Problems Problems that can be handled only using numerical software and for which there is no symbolic solution, such as the computation of roots of a univariate polynomial. (2) Numerically Reducible Problems Problems for which both symbolic and numerical solutions exist, such as nding the greatest common divisor of a set of univariate polynomials. (3) Numerically Irreducible Problems Problems that can be handled only using symbolic software, such as the computation of Gr obner based for multivariate polynomials. From the above classication it is clear that a familiarity with symbolic and numerical software can benet the control/systems engineer enormously. In this article, the power and usefulness of symbolic software are brought to the fore. It is of course understood that numerical software is an equally indispensable tool in the control engineers toolbox, since, in general, the control designer needs to resort to both symbolic and numerical software to successfully carry out control design.
1
At rst contact, symbolic software is reminiscent of university-level engineering analysis and calculus. It offers the exibility, excitement, and expectation of computing exactly with a completely abstract representation of the problem at hand. Unfortunately, it shares the same drawback, in that the problems that can be dealt with are limited in their complexity. Nevertheless, using symbolic computation, software computations that involve cumbersome algebraic expressions that are far beyond error-prone manual manipulations become routine. As a result, symbolic computation software has signicantly pushed the boundaries of what can be regarded as an analytically tractable mathematical problem, and so it has become an invaluable tool in the control designers toolbox for modeling, design, and analysis of control systems. A plethora of symbolic computation algorithms targeting control system analysis and design are available. Symbolic computation is useful in almost every step of control system modeling and the analysis and controller design of linear and nonlinear control systems. In particular, symbolic manipulation of bond graphs and multidimensional Laplace transforms, which are important tools in the modeling of nonlinear control systems, are facilitated by computer algebra. The analysis of qualitative and quantitative system properties under parameter variations is supported by symbolic computation through sensitivity and bifurcation analysis. Robust control problems in either linear or nonlinear systems are facilitated by computer algebra software. The important geometric approach to nonlinear control design (e.g., computing a linearizing feedback or determining zero dynamics) is enabled by commercially available symbolic software. A more elaborate overview of other symbolic software algorithms used in modeling, analysis, and design of control systems is provided in 2. In this article, polynomial control system analysis and design are targeted, and in particular three important methods are discussed: Gr obner bases, differential algebra (in particular, Ritts algorithm), and quantier elimination. In essence, these methods deal with polynomials. Computation of Gr obner bases is implemented in most commercial symbolic software packages such as Maple and Mathematica. On the other hand, Ritts algorithm is not as yet commercially available, but software implementations may be obtained from the specialists working in the eld. Quantier elimination has been implemented in Mathematica. Polynomial systems are a natural generalization of linear systems; they possess a nice algebraic structure, and their analysis is amenable to computer algebra. Moreover, as polynomials are universal approximators, polynomial control models can be used as valid models for almost any physical system. Furthermore, the restriction to polynomial systems is not severe, since any nonlinear function that satises a polynomial differential or algebraic equation, such as trigonometric functions, can be handled by introducing articial states; see 3 and 4 for more details. The state-space model of a polynomial control system is given by
Here x n , y p , u m represent respectively the state, the observed output, and the manipulated control input of the system. The operator is either the derivative x(t) = dx(t)/dt for continuous-time systems (when t ) or the forward difference x(t) = x(t + 1) for discrete-time systems (when t ). The symbols , , , and denote respectively the sets of real, rational, natural, and complex numbers. The vector functions f and h are assumed to have entries that are polynomial functions in all their variables. Moreover, for computational reasons the polynomials are assumed to have rational coefcients. Continuous-time (or discrete-time) linear systems
form a special subclass of polynomial control systems (1). A wide range of systems can be adequately modeled using Eq. (1). Indeed, systems as diverse as biochemical reactors, grain dryers, satellites, robots, and airplanes have been successfully modeled using polynomial systems (1) (see, e.g., 5). In the context of polynomial control systems, problems like determining equilibria, estimating a domain of attraction of a stable equilibrium, and locating approximately periodic solutions, as well as testing controllability and observability properties, reduce naturally to the analysis of sets of algebraic polynomial equations. Such problems can be addressed using the Gr obner basis method. In any modeling exercise, the elimination of variables from sets of differential and algebraic equations plays a crucial role. This problem also arises naturally in the context of determining if two state representations are equivalent. Differential algebra and Ritts algorithm are very useful in this context. A range of different control problems can be formulated as questions involving quantiers there exists and for all, and this naturally leads to the so-called quantier elimination algorithms.
The Grobner Basis Method

Central objects in the Gr obner basis theory are polynomial ideals and afne varieties (6). Let p1 ,. . ., ps be multivariate polynomials in the variables x1 ,. . . , xn whose coefcients are in the eld k. For the collection of ordered n-tuples of elements of k the notation kn is used. The variables x1 ,. . . , xn are considered as place markers in the polynomial. The notation p1 ,. . . , ps k[x1 ,. . . , xn ] is adopted. The afne variety (or variety) dened by the s polynomials p1 ,. . . , ps is the collection of all solutions in kn of the system of equations
Formally, the variety is
For instance, a straight line, a parabola, an ellipse, a hyperbola, and a single point are all examples of varieties in 2 . The polynomial ideal I that is generated by p1 ,. . . , ps is a set of polynomials obtained by combining these polynomials through multiplication and addition with other polynomials: formally,
The polynomials pi , i = 1,. . . , s, form a basis for the ideal I. A very useful interpretation of a polynomial ideal I is in terms of the equations (3). On multiplying pi by arbitrary polynomials gi k[x1 ,. . . , xn ] and adding the products, the implication of Eq. (3) is that f = g1 p1 + + gs ps = 0, and of course f I. Hence, the ideal I = p1 ,. . . , ps contains all the polynomial consequences of the equations (3). A notion at the core of the Gr obner basis method is that of the monomial ordering (a monomial is a polynomial consisting of a single term), since it introduces an appropriate extension of the notion of the leading term and the leading coefcient familiar from univariate polynomials to multivariate polynomials. One can dene many different monomial orderings [lexicographic, graded lexicographic, graded reverse lexicographic, etc. (6)], but to be specic, consider the so-called lexicographic, or lex, ordering. Let , be two n-tuples of integers [ = (1 ,. . ., n ), = (, . . . , n ) n . The n-tuple is said to succeed (in lex ordering), denoted as , if in the vector difference = (1 1 ,. . ., n n ) the leftmost nonzero entry is positive. Obviously, it
is possible to dene n! different lex orderings for polynomials in n variables. For the polynomial f = 2x1 3 x2 x3 3 + 4x1 3 x3 5 , using the lex ordering x1 x2 x3 the monomial x1 3 x2 x3 3 succeeds the monomial x1 3 x3 5 , as the multidegrees of the monomials satisfy (3,1,3) (3,0,5). With this ordering, the leading coefcient and the leading term are respectively LC(f ) = 2 and LT(f ) = 2x1 3 x2 x3 3 . Using the alternative lex ordering x1 x2 x1 , the leading term becomes LT(f ) = 4x1 3 x3 5 . In general, an ideal I does not have a unique basis, but given any two different bases p1 ,. . . , ps and g1 ,. . . , gm of I, the varieties V(p1 ,. . . , ps ) and V(g1 ,. . . , gm ) are equal. In other words, a variety only depends on the ideal generated by its dening equations. Some bases of an ideal may be simpler than other bases. Intuitively, if all the polynomials in a given basis of an ideal have a degree that is lower than the degree of any other polynomial in the ideal with respect to a particular monomial ordering, then this basis is in some sense the simplest basis. In particular, a Gr obner basis of an ideal (for a given monomial ordering) has such a property and can be thought of as the simplest or canonical basis. Given an ideal I and a monomial ordering, denote the set of leading terms of elements of I as LT(I). The ideal generated by elements of LT(I) is denoted LT(I) . In general, the ideal generated by the leading terms of a particular ideal I is not the same as the ideal generated by the leading terms of the polynomials in a basis for that particular ideal I. A Gr obner basis is a special basis for which this property holds, and it is formally dened as the set of polynomials g1 ,. . . , obner bases, the user species a monomial gm for which LT(I) = LT(g1 ),. . . , LT(gm ) . When computing Gr ordering; different monomial orderings produce different Gr obner bases. Given a monomial ordering, the two most important properties of Gr obner bases are: obner basis. Furthermore, any Gr obner (1) Every ideal I k[x1 ,. . . , xn ], other than the trivial ideal 0 , has a Gr basis of an ideal I is a basis for I. (2) Given an ideal I k[x1 ,. . . , xn ], other than the trivial ideal 0 , a Gr obner basis of I can be computed in a nite number of algebraic operations. The rst algorithm for computation of Gr obner bases, published in the 1960s, is attributed to B. Buchberger (6). Since then a number of improvements have been reported and the algorithm has been implemented in most commercial symbolic software packages. Buchbergers algorithm generalizes two well-known algorithms: Gauss elimination for sets of multivariate linear algebraic equations, and Euclids algorithm for computing the greatest common divisor of a set of univariate polynomials. Solving Sets of Multivariate Polynomial Equations. Gr obner bases facilitate solving a set of multivariate polynomial equations (3) in the same way as the Gauss elimination algorithm facilitates solving a set of linear algebraic equations. Indeed, in a given lex ordering the Gr obner basis has a triangular structure reminiscent of the triangular structure in Gauss elimination. If the Gr obner basis is given by {1}, then the system of polynomial equations has no solution. Equilibria for Polynomial Systems. Consider a polynomial system without inputs x(t) = f (x(t)). A state x n is called an equilibrium of this system if x(0) = x implies that x(t) x t. Equilibria for polynomial systems are therefore obtained as solutions of a set of polynomial equations of the form f (x) = 0 or f (x) = x for continuous-time or discrete-time systems respectively. Gr obner bases facilitate nding all equilibria. Example: The equilibria of the polynomial system
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION are obtained as solutions of the set of polynomial equations
which is hard to solve. The Gr obner basis for the ideal p1 , p2 , p3 using the lexicographic ordering x1 is computed using Maple:
x2
x3
By construction the equations gi = 0, i = 1, 2, 3, 4, have the same solutions as Eq. (6). But obviously the polynomials gi have a much simpler structure than the polynomials pi . Indeed, g4 depends only on x3 , and this allows g4 = 0 to be solved numerically for x3 . The solutions can be substituted into g2 = 0 and g3 = 0 to obtain polynomials in x2 only that can then be solved numerically. This process of back-substitution can be continued until all solutions are found. Observe that a basis for an ideal does not have a xed number of basis polynomials, and indeed the simpler Gr obner basis contains more polynomials than the original basis. As may be observed, it even contains more polynomials than there are variables.
Periodic Solutions for Polynomial Systems. Consider the system f (dn y/dtn , . . . , dy/dt, y, t) = 0, where
y can be thought of as the output of a closed-loop control system. Let f be a polynomial in all its variables. To approximately compute a periodic solution y(t) of this system, the method of harmonic balancing can be used. It can be shown that under mild conditions an approximate periodic solution implies the existence of an exact periodic solution close to the approximate one (7). A truncated Fourier series can be considered as a candidate approximate periodic solution y(t) = k = N N Ck e jkt , ck , where ck and c k are complex conjugates. Postulating that y(t) is a solution leads to a set of polynomial equations, because f is polynomial. Using the Gr obner basis method with lex ordering, this set of equations can be solved for the unknowns c0 , c1 ,. . . , cN and to obtain y(t). In practice the actual solution may not be so important, and it often sufces to know that a periodic solution exists and to know its oscillation frequency . This can be achieved by nding the Gr obner basis of the polynomial equations with any lex ordering where is the lowest-ranking variable. + y = 0, and postulate a periodic solution y(t) Example: Consider the van der Pol equation (8) y a (1 by2 ) y = , where ck = ckr + cki j, k = 0, 1, 2, 3, and j = . Substituting y(t) into the van der Pol differential equation and equating the coefcients on the left-hand and right-hand sides of the equation leads to
and c0 = 0, c2 = 0, and c1i = 0. The coefcients a and b are regarded as variables. To simplify the computations, obner basis of the ideal p1 , p2 , p3 , p4 with the lex ordering c3 c3i p1 is divided by c1 , and p2 by ac1 . A Gr c3 b a contains seven polynomials of nontrivial complexity (42 pages of Maple output). It contains one polynomial in and a only:
Obviously, for a = 0 the van der Pol oscillator is a simple harmonic oscillator with frequency = 1. In the presence of nonlinear damping a = 0, the solution of g7 = 0, which continues = 1, can be expanded as
and using Maple, the coefcients i are obtained:
Grobner Bases and Elimination. Gr obner bases can be used to eliminate some variables of interest from a set of polynomial equations, as illustrated by Example 1, where x1 and x2 are eliminated from g4 in Eq. (7). The variables that are ranked higher in the lex ordering are eliminated rst. This feature of Gr obner bases can be used to obtain an estimate of the domain of attraction of equilibria for polynomial systems. Estimation of the Domain of Attraction. If the closed-loop control system is stable, it is of interest to determine those initial conditions from which the solutions converge to a specied equilibrium. The collection of all the initial conditions that converge to the same equilibrium is called the domain of attraction of that particular equilibrium. Estimating domains of attractions is a hard problem. One way of obtaining a (conservative) estimate for the domain of attraction is to use Lyapunov functions. It is a standard result in Lyapunov theory that if x = 0 is an equilibrium point for the continuous-time system d/dt x = f (x), if D n is a domain containing x = 0, and if W : D is a continuously differentiable function such that W (0) = 0 and for all x D {0} one has W (x) > 0 and W /x f (x) < 0, then x = 0 is asymptotically stable. Given such a Lyapunov function, consider the sets = {x n : W /xf (x) < 0} and Bd = {x n : W (x) d}. If Bd for some d > 0, then the set Bd is an estimate for the domain of attraction. For polynomial systems with a polynomial Lyapunov function W , Gr obner bases can be used to compute Bd systematically. Indeed, it is feasible to construct the largest such set by nding d such that Bd is as large as possible and still inside . For polynomial systems with polynomial Lyapunov functions, W (x) d and W /x f (x) are polynomials, and hence the boundaries of the sets Bd and are varieties. At the points where the varieties V(W d) and V (W /xf (x)) touch each other,
the gradients of W and variables is obtained: are parallel. Using this information, a system of n + 2 polynomial equations in n + 2
computing a Gr obner basis for the above system of equations, where the variable d has the least rank in the lex ordering, a polynomial equation in d only is obtained. The least positive solution to this equation is the best value of d for which Bd , and this yields in turn the best estimate for the domain of attraction that could be obtained with the particular Lyapunov function W .
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION Example: Consider the system (8)
with the Lyapunov function W (x) = 3x2 1 + 4x1 x2 + 4x2 2 . The polynomials (8) are in this case
and using the lexicographic ordering x2 x1 d, the Gr obner basis of the ideal p1 , p2 , p3 , p4 is computed. It contains the polynomial g(d) = 4d4 147d3 + 786d2 + 2048d. The approximate solutions of g(d) = 0 are 1.9223, 0, 8.9657, 29.707. The smallest positive value of d for which there is a solution to the system of equations pi = 0 is 8.9657. The corresponding estimate for the domain of attraction is therefore {x 2 : W (x) 8.9657}.
Equality of Ideals. A special type of a Gr obner basis can be used to decide if two ideals are the same or not. This is the so-called reduced Gr obner basis. A reduced Gr obner basis for an ideal I is a Gr obner basis G for I such that LC(p) = 1 for all p G, and for all p G, no monomial of p lies in LT(G {p}) . The main property of reduced Gr obner bases is that given an arbitrary ideal I = 0 and a particular monomial ordering, I has a unique reduced Gr obner basis. Hence, two ideals J 1 and J 2 are the same if and only if their reduced Gr obner bases G1 and G2 are the same (G1 and G2 must be computed with the same monomial ordering). Most commercial computer algebra systems, such as Maple and Mathematica, provide nite algorithms for computing reduced Gr obner bases. Analysis of Discrete-Time Polynomial Systems. Fundamental control-theoretic concepts such as controllability or observability can be reduced to the problem of computing maximal control-invariant varieties. This is well known for linear systems (computation of control-invariant subspaces), but it holds equally well for discrete-time polynomial systems of the form x(t + 1) = f (x(t),u(t)). For such systems, a variety V is controlinvariant if f (V ,u) V for all possible control inputs u. The computation of the maximal control-invariant subset of a given variety V can be completed in nitely many operations. Consider the dening ideal of the variety V ; say J 1 = g1,1 ,. . ., g1,m1 , where gj,k [x]. If the variety corresponding to J 1 were control-invariant, then g1,k f (x,u) 0 for all u and all k = 1,. . . , m1 . The polynomials g1,k f (x,u) can be viewed as polynomials in u with coefcients in [x]. Denote the collection of all these coefcients as g2,k , k = 1, 2,. . . , m2 , and the corresponding ideal as J 2 = g2,1 ,. . . , g2 ,m2 . Invariance would imply that J 1 = J 2 , and if this is not the case, then obviously J 1 J 2 and the corresponding varieties satisfy V 1 V 2 . This process can be continued to construct an ascending chain of ideals (or descending chain of varieties) J 1 J 2 J 3 . This chain must terminate in nitely many steps (6). That is, there exists an integer N such that J N 1 J N = J N +1 = . The variety V (J N ) is the maximal control-invariant subset of V (J 1 ). The check whether J k = J k+1 can be completed via the computation of reduced Gr obner bases for J k and J k+1 .
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION Observability. The discrete-time polynomial system
as said to be observable if for each pair of initial states = , there exists an integer N and an input sequence U N = {u(0),. . . , u(N 1)} such that the solutions starting at and produce different outputs after N steps, that is h(x(N , , U N )) = h(x(N , , U N )). The polynomial h(x(N , , U N )) h(x(N , , U N )) can be regarded as a polynomial in elements of U N with coefcients in [, ]. A chain of ideals J k is constructed using these coefcients, and obner at each step the condition J k = J k+1 is tested. It can be shown that if J N for some N has a reduced Gr basis {1 1 ,. . . ,n n }, then J N +1 = J N and the system is observable. The above-discussed algorithm for computing invariant sets streamlines these computations and allows a systematic determination of the integer N . Example: Consider the simple Wiener system (9)
The system consists of a linear dynamical system and a quadratic static output nonlinearity. The ideal J 1 = 1 2 1 2 and the output equation (12) are used to construct the following ideals:
Using the lex ordering 1 2 1 2 , the reduced Gr obner basis for J 4 is G4 = {1 1 , 2 2 }, and therefore the system (12) is observable with N = 4.
A Brief Overview of the Literature and Related Problems. Gr obner bases are useful for a range of other control problems, such as the inverse kinematic problem and motion planning in robotics (6), the computation of the switching surfaces in the solution of time optimal control problems (10,11), identiability, inputoutput equivalence of different state-space realizations, normal forms and zero dynamics (12), analysis of hybrid control systems, computation of limit cycles for discrete-time polynomial systems, observability of continuoustime polynomial systems, and forward accessibility of discrete-time polynomial systems. In control design for linear systems, the linear functional observer problem, the model-matching problem, the deterministic identication problem, and the disturbance decoupling problem play a central role. All these problems can be seen as the characterization of a maximally control-invariant subspace of a given subspace of the state space (2). In control theory this problem is known as the cover problem. The cover problem can be solved via a computation of elementary divisors of matrix pencils, which in turn leads to multilinear equations
where the unknowns are the basis vectors of the invariant subspace. The Gr obner basis method facilitates the analytic solution of this problem. The elegance of this approach stems from the fact in one single algorithm all possible (typically innitely many) control-invariant subspaces can be explicitly characterized. The degrees of freedom established in this process can then be further exploited to optimize other desirable system properties, such as sensitivity. We have focused on the use of Gr obner bases for commutative rings over innite elds, that is, nding Gr obner bases in the context of polynomials with rational coefcients. Two important related areas are Gr obner bases for commutative rings over nite elds and Gr obner bases in the context of noncommutative rings. The former is of interest in the context of discrete-event dynamic systems (13,14) and coding theory in communications. The latter is useful for control-theoretic problems involving polynomial matrices (15). In particular, in 15 a new extra term in the expansion of the classical state-feedback optimal control problem in the singular perturbation form was obtained using noncommutative Gr obner bases. An important drawback of Buchbergers algorithm is that even with the best known versions of the algorithm, it is easy to generate examples for which the computation of a Gr obner basis requires inordinate amounts of computer resources (time and/or memory). The main bottlenecks are that the total degrees of the intermediate polynomials that the algorithm generates may be quite large and that the coefcients of the Gr obner basis may be complicated rational numbers. This may be the case even if the original ideal generators are polynomials of small degrees with small integer coefcients. In general, the intermediate polynomials observed in the computation of a Gr obner basis can have total degrees of the order of 22d , where d is the total degree of the ideal generators. Although this appears to be a rather negative result, it typies the worst-case scenario. It appears that the running time and the storage requirements seem to be much more manageable on average. It is important to emphasize that different monomial orderings may produce very different computational times, and some experimentation with the ordering may yield signicant reductions in computation time.
Differential Algebra In Control

In the 1980s differential algebra was applied to control problems (16). Differential algebra can be used to transform a polynomial system from one representation to another. In general differential algebra plays an important role in the realization theory of nonlinear control systems. In the context of control, the problems that lend themselves to differential algebra are varied: the determination of observability, identiability, the calculation of zero dynamics, regulator computations, tracking control, etc. Differential-Algebraic Tools. Differential algebra provides tools for dealing with systems of polynomial differential equations. In this algebra the derivative operation occupies center stage. A multivariate polynomial in variables y1 ,. . . , yN and their derivatives is called a differential polynomial in y1 ,. . . , yN . For instance, f (dn y/dtn , . . . , dy/dt, y) is a differential polynomial in y if f is a polynomial in all its variables. The concept of ranking is introduced for differential polynomials. It is very similar to the concept of monomial ordering for polynomials. Ranking is a total ordering of all variables and their derivatives. Examples involving two variables are
and
10
where denotes is ranked lower than. Any ranking is possible provided it satises two conditions:
for all variables u and y, all nonnegative integers and v, and all positive integers . The highest-ranking variable or derivative of a variable in a differential polynomial is called the leader. The ranking of variables gives a ranking of differential polynomials. They are simply ranked as their leaders. If two have the same leader, they are considered as polynomials in their leader and the one of lower degree is ranked lower. Let A, B be two differential polynomials, and let A have the leader v. Then B is said to be reduced with respect to A if there is no derivative of v in B and if B has lower degree than A when both are regarded as polynomials in v. A set
of differential polynomials is called autoreduced if all the Ai are pairwise reduced with respect to each other. Normally autoreduced sets are ordered so that A1 ,. . ., Ap are in increasing rank. Autoreduced sets are ranked as follows. Let A = A1 ,. . . , Ar and B = B1 ,. . . , Bs be two ordered autoreduced sets. A is ranked lower if either there is an integer k, 0 k min(s, r), such that
or else if r > s and
A characteristic set for a given set of differential polynomials is an autoreduced subset such that no other autoreduced subset is ranked lower. The separant SA of a differential polynomial A is the partial derivative of A with respect to the leader, while the initial I A is the coefcient of the highest power of the leader in A. If a differential polynomial f is not reduced with respect to another differential polynomial g, then either f contains some derivative of the leader ug of g or else f contains ug to a higher power. In the former case one can differentiate g a suitable number (say ) of times and perform a pseudodivision to remove that derivative, giving a relation
where S is the separant of g and R does not contain the highest derivative of ug that is present in f . If f contains ug to a higher power, a pseudodivision of f by g can be performed to obtain
where I is the initial of g, and R is reduced with respect to g.
11
Observability. In control theory a system is called observable if it is possible to compute the state from inputs and outputs. Using differential algebra, the observability question can be answered in a constructive manner for continuous-time polynomial systems (1).
Example: Consider the system
which is in state-space form. Suppose an inputoutput description (i.e., a description directly relating u and y) is called for. The original set of equations is equivalent to
It is now possible to eliminate the derivative of x2 by forming
From this construction it follows that p = 0 whenever the equations of (19) are satised. The last equation of (19) can be replaced by p = 0 to get the system description
From (Eq. 20) it follows that every solution of Eq. (19) also solves Eq. (21). If, moreover, it is known that x2 = 0, then the converse is also true, and Eqs. (19) and (21) are equivalent. It is now possible to form
and it readily follows that
is equivalent to (Eq. 21) if also u = 0. Finally form
to conclude that, provided ux2 = 0, Eq. (19) is equivalent to the following system description:
12
The leftmost equation is an inputoutput relation, while the middle and right equations show how x1 and x2 can be computed from the input and output. This establishes the observability, and more. Using the terminology of differential algebra, the state-space form is an autoreduced set under the ranking
while Eq. (25) is an autoreduced set under the ranking
Autoreduced sets thus give a generalization of several of the standard forms for dynamic systems.
Global Identiability. A polynomial system that is parametrized with a (constant) parameter as
as said to be globally identiable if, given input and output measurements u(t) and y(t) over an interval t [0,T ], there is only one constant value of that satises Eq. (26). Global identiability of polynomial systems (1) can be checked in a constructive manner using the tools from differential algebra. Moreover, conditions on u and y under which the system is not globally identiable can also be obtained, as is illustrated below. Example: Consider the system
where is a constant parameter. Can be identied by observing the output y? Using the notation p = y + 2 y + 2 y, compute the expression
It follows readily that is uniquely determined by y (and its derivatives), provided that along the output trajectory
This is thus an excitation condition that guarantees global identiability.
The Algorithms of Ritt and Seidenberg. The concepts of the previous section were introduced in the 1930s by the American mathematician Ritt, (17). He devised an algorithm by which it is possible to start with an arbitrary number of differential polynomials, introducing a suitable ranking of the variables and performing successive operations of the type described by Eqs. (14), (15) to arrive at an equivalent representation in the form of an autoreduced set of equations. This set has the property that an f belonging to the original differential polynomials can be reduced to zero using (Eqs. 14) and (15) for different gs belonging to the set. If certain factorization conditions are met, then Ritt showed that the generated autoreduced set is a characteristic set, not only for the explicitly generated polynomials, but also for an innite set of polynomials constituting the
13
so-called differential ideal. Ritt used his algorithmic procedure to produce an extensive theory for polynomial systems of differential equations. Later a more systematic algebraic treatment was given by Kolchin (18). The theory was also extended to partial differential equations (18) and difference equations (19). From the viewpoint of practical calculations, a major drawback of Ritts algorithm is the factorization it requires, since this is a task of high computational complexity. Seidenberg (20) has proposed an algorithm for deciding the solvability of systems of equations and inequations. The algorithm uses (Eqs. 14), (15) repeatedly to eliminate variables and reduce the problem to the single-variable case. As in the simple example above, equivalence of the original and nal sets of equations requires the separant S and initial I of (Eqs. 14), (15) to be nonzero. To handle this, Seidenbergs algorithm splits the problem into different subproblems, considering the cases S = 0, S = 0 (or I = 0, I = 0) together with the original equations and inequations. By repeated splitting a tree of subproblems is generated. Each subproblem is nally reduced to a single-variable problem for which solvability can be decided. In recent years several variations of Ritts and Seidenbergs algorithms have been implemented in symbolic manipulation languages like Maple or Mathematica. Other Control Problems. Differential-algebra calculations can be used to compute the zero dynamics of a system. This dynamics is obtained by restricting the output to be zero. By adding the equation y = 0 and using Ritts algorithm with a suitable ranking, it is possible to obtain a description of the resulting dynamics. In a similar manner a regulator description can be obtained. Suppose a differential equation describing the desired response from reference signal to output is given. Using this equation together with the system dynamics as the input to Ritts algorithm, it is possible to arrive at an equation containing the control signal together with the reference and output signals. This equation can be interpreted as a description of the regulator, although it can be difcult to implement due to its implicit nature in the general case.
Quantier Elimination
Quantier elimination is a method for rewriting formulae that include quantiers such as for all () and there exists () in an equivalent form without the quantiers and the quantied variables. According to a theorem by Tarski (21), it is always possible to eliminate the quantied variables in formulae consisting of logical combinations of multivariate polynomial equations and inequalities. Tarski provided a constructive proof in the late 1940s, but the corresponding algorithm has such complexity that it is impractical for most problems. In the mid 1970s Collins (22) presented a new method, the so-called cylindrical algebraic decomposition, which exhibits much better complexity. Since then the algorithmic development has made signicant progress (23,24, 25). Nevertheless, cylindrical algebraic decomposition and quantier elimination are known to be inherently complex (26). It is therefore of importance to identify classes of problems for which the computational complexity is much lower, and for which specialized algorithms can be developed (27,28). Implementations of cylindrical algebraic decomposition and quantier elimination are available in for example Mathematica (29). Depending on the specic nature of the problem posed, different algorithms are used in order to minimize the computational burden. The algorithmic developments are ongoing. Introductions to cylindrical algebraic decomposition and quantier elimination can be found in Ref. 30. An extensive bibliography covering early papers can be found in Ref. 31, and a survey of the algorithmic development of cylindrical algebraic decomposition is given in Ref. 32. An early application of quantier elimination techniques in control theory was made by Anderson et al. (33). This contribution predates the introduction of efcient computational tools, and so it was of theoretical interest only. With the availability of the experimental software QEPCAD (23), the number of papers related to control increased. Now, as these methods are implemented in widely available symbolic computation software packages like Mathematica, the control practitioner can explore their potential. Systems of Real Algebraic Equations and Inequalities. In the context of polynomial equations with real coefcients, inequalities arise naturally, for example to express when a quadratic polynomial has
14
real solutions. The need of using not only equations but also inequalities in describing computational problems triggered the development of methods for doing symbolic computations with so-called semialgebraic sets. A semialgebraic set is a generalization of an algebraic set or variety, as it is the solution set of a system of real algebraic equations and inequalities. An algebraic expression in variables {x1 ,. . . , xn } is an expression constructed with {x1 ,. . . , xn } and rational numbers, using addition, multiplication, rational powers, and algebraic numbers and functions. A system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } is a logical combination of equations and inequalities with both sides being algebraic expressions in {x1 ,. . . , xn }. The notation (and), (or), (implies) for Boolean operators is used. The following is an example of a system of real algebraic equations and inequalities in the variables a, b, x, y:
Alternatively, a semialgebraic set can be characterized as a set obtained by nitely many unions, intersections, and complementations of sets of the form {x | f (x) 0}. Here f is a multivariate polynomial with rational coefcients. Semialgebraic sets are thus closed under projection, union, and intersection. As Gr obner bases provide a way to replace a system of multivariate polynomials by a simpler equivalent set, there are systematic ways to simplify a system of real algebraic equations and inequalities. It can be shown that the set of solutions of any system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } can be written as a disjunction of a nite number of cylindrical parts of the form
In the above expression, stands for one of <, , and =; and f i and gi are either , , or algebraic expressions in the variables {x1 ,. . . , xi i } that are real-valued for all tuples of real numbers {a1 ,. . . , ai i } satisfying
The method of rewriting a real algebraic system as a disjunction of the above form is called cylindrical algebraic decomposition; see Refs. 22,30,32. Observe the triangular nature of the resulting system. Example: An example of a script in Mathematica for computing a cylindrical algebraic decomposition of an ellipsoid in 3 is
15
Here the functions f i and gi can be expressed in terms of simple algebraic operations and radicals. More typically the full generality offered by algebraic functions is required:
In Mathematica algebraic functions are represented by special Root objects. In the above case the placeholder variable #1 is a function of x and y. The last argument enumerates the branches of the algebraic function, which in this case are the rst two roots according to Mathematicas enumeration scheme of the 12 possible complex roots. A quantied system of real algebraic equations and inequalities in variables {x1 ,. . . , xn } is an expression of the form
where the Qi s are the quantiers or , and S is a system of real algebraic equations and inequalities in {x1 ,. . . , xn ; y1 ,. . . , ym }. According to Tarskis theorem the solution set of quantied systems of real algebraic equations and inequalities is a semialgebraic set, which means that QS always can be rewritten as an equivalent expression without any quantiers and quantied variables. The process of eliminating quantied variables from such systems is called quantier elimination. The yi s in QS are called bounded variables, and the xi s free variables. Problems without free variables are called decision problems. Example: The proof of the well-known inequality between the arithmetic and the geometric mean of two real positive numbers can be formulated as a decision problem:
The notation for quantiers in Mathematica is
where Q is either or , var is a single variable or list of variables, and cond gives additional conditions on the variables such as their domain and/or semialgebraic constraints. The command to perform quantier elimination in Mathematica is called Resolve.
16
Projection of semialgebraic sets is equivalent to quantier elimination on existential quantiers. Consider for example the projection of the ellipsoid in Example 7 onto the xy plane:
Set inclusion for semialgebraic sets is another useful property that can be formulated as a decision problem: A B is the decision problem xA B:
Sets of Equilibria. As illustrated before, Gr obner bases are useful to determine equilibria for polynomial systems. In polynomial control systems it is of importance to describe the steady-state solutions: the set of reachable equilibria, for admissible input values. Characterizing the set of reachable equilibria in state space is a quantier elimination problem.
Example: Add a control input to the last equation in Example 1. Let this input be constrained to the interval 1 u 1. Compute the projection of the set of corresponding equilibria onto the x1 x2 plane.
Stability. To determine the local stability of equilibria of dynamic systems the stability tests of Routh and Hurwitz (34) may be used. These stability tests give explicit conditions on the coefcients of the characteristic equation of the Jacobian of the dynamical system at equilibrium in terms of a number of polynomial inequalities. Hence the RouthHurwitz test can be handled using cylindrical algebraic decomposition and quantier elimination. A less well-known, but equivalent stability test is the LienardChipart criterion (34), which has the advantage over the Routh criterion that it involves polynomial inequalities of lower degree. Given a polynomial in [s] a0 sn + a1 sn 1 + + an 1 s + an , ai > 0, the LienardChipart criterion states that all its zeros have strictly negative real parts if and only if the following inequalities holds:
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION where Di , i = 1,. . . , n, are the so-called Hurwitz determinants of order i, which are dened as follows:
17
where ak = 0 for k > n. Example: Consider the following polynomial system with real parameters a and b. Determine for which values of a and b the origin is a stable equilibrium (this is an essential step in any bifurcation analysis of a dynamical systemthe classication of possible asymptotic behaviors of the system):
For zero input u = 0 the zero state is an equilibrium. Linearizing the system around (x,u) = (0,0) and computing the characteristic polynomial of the resulting state transition matrix gives
Compute the solution set corresponding to the LineardChipart stability inequalities for this polynomial:
Example: Consider the problem of stabilizing in a unit-feedback scheme an unstable system G, using a lead compensator F that is parametrized by the parameters B and b, where
The LienardChipart inequalities for the closed-loop system are
18
These can be rewritten using cylindrical algebraic decomposition to either
or
depending on the choice of the order for b and B. Using quantier elimination, questions like For which values of b can it be guaranteed that for all values of 5 < B < 10 the closed loop system is stable? can be answered:
Numerically this corresponds to 0.65359 < b < 1.34641.
Nonlinear Tracking. Tracking a particular output is an important control objective. There are many control design methods that deal with this problem, mainly based on optimization techniques. However, most (analytic) methods do not take the ever present constraints on control signals or states into account. As a consequence the resulting control system must be validated, often through extensive simulations. For a large class of nonlinear tracking problems, quantier elimination can be used to decide if there exists a solution to a given tracking problem subject to the constraints of interest. Consider the continuous-time polynomial control system
where x n , u m and each component of f is a polynomial in all its variables. Furthermore, the input value u(t) is required to belong to some semialgebraic set U :
called the admissible control set. Let
be a rationally parametrized curve in the state space, i.e.,
where g : is a rational function and the orientation of is dened by increasing values of s. For the state of the system (27) to follow the curve , there has to be an admissible control u such that the vector eld f (xp , u) is tangent to at each xp on . Since the curve is parametrized in terms of s, this
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN: SYMBOL MANIPULATION tangency requirement can be formulated as
19
where > 0 ensures that the state follows in only one direction. The tracking problem is thus equivalent to deciding if the following formula is true or false:
The above problem becomes a quantier elimination problem if f or g include some free (design) parameters. Example: Consider the polynomial control system
and the curve = {x 2 |x1 = s x2 = s3 + 1.5s2 0 s 1}. Is it possible to steer the system state along using control signals constrained to 1 u 1? Using Mathematica, this can be resolved as follows:
The tracking problem has a solution as long as 0 a 1.12769.
20
Consider now a = 1, and compute the limits on u such that the above tracking problem has a solution:
For the tracking problem to have a solution it sufces that 0.98077 u 0.98077.
Multiobjective Feedback Design. Many objectives in linear system design are formulated in terms of frequency-domain inequalities. Let the plant be represented by G(s, p) and the controller by F (s, q). Both G and F are assumed to be scalar-valued rational functions of the Laplace variables s, real plant parameter p, and real controller parameter q. It is now possible to write many robust design problems in a form suitable for quantier elimination. Stability. The unit-negative-feedback closed-loop system consisting of plant and controller is asymptotically stable if and only if all zeros of the rational function 1 + G(s, p)F (s, q) have strictly negative real part. This can be converted into polynomial inequalities in p and q using the LineardChipart criterion. Tracking Error. The tracking error at the output of the unit-negative-feedback loop of plant and controller is governed by the so-called sensitivity transfer function
Acceptable levels of the tracking error can be specied by the inequality |S(iw)| < T , 0 w w1 , which can be rewritten as a real algebraic inequality in the variables w, p, q, and T . Through quantier elimination it can be veried that for the class of systems p P a single stabilizing controller q exists that meets the sensitivity objective. In essence any linear control design question formulated in frequency domain leads naturally to a quantier elimination or cylindrical algebraic decomposition problem. More importantly, symbolic manipulation software is up to the task of systematically approaching these control design questions in practical and moderately complex situations. Related Problems and Other Control Applications. There is a vast array of problems in control that can be posed as quantier elimination problems or systems of multivariate polynomial inequalities, whose solutions can be described by cylindrical algebraic decomposition. For example, quantier elimination has been used to investigate stability, stabilizability, and controllability of discrete-time polynomial systems (35,36), stability and stabilizability of switched polynomial systems and unstable zero dynamics via switching (30,37), frequency-domain design (38), and multiobjective robust control design (39). Quantier elimination has also been used for design of nonlinear control systems for nonlinear aircraft dynamics (40) and for robust nonlinear feedback design (41).
21
The generality of the quantier elimination method comes at the cost of very high computational complexity. Quantier elimination software tools are not a panacea allowing one to approach every control problem by brute force. The challenge in successfully applying quantier elimination software tools lies in formulating the problems in such a way that their inherent structure is maximally exploited. Most importantly, instead of applying quantier elimination to a complete problem, there are often simplications that can be carried out so that only a small problem (with far fewer variables) has to be handled by quantier elimination. Control designers should start focusing on providing the right heuristics and tools, built on their experience, in order to make general-purpose quantier elimination tools better attuned to control problems. In parallel, algorithm designers now focus on developing specialized methods as components of a general quantier elimination package in order to exploit structure and to handle complexity. For instance, in Mathematica (25) there are a modied simplex linear optimization algorithm for linear systems of equations and inequalities with inexact or rational coefcients, linear quantier elimination for equations and inequalities with exact coefcients according to 27, preprocessing by linear equation solvers, Gr obner bases, special care and simplication for variables appearing at most linearly in the systems, a simplied cylindrical algebraic decomposition algorithm for generic solutions (24), and quantier elimination by partial cylindrical algebraic decomposition (23) for the general case. Conclusion. The possibilities offered through symbolic (and numeric) software in order to address control problems are as varied and exciting as the control problems themselves. This article just presents the tip of an iceberg. Dealing with complexity in control design is the main issue. For control algorithm developers there is now more than ever a compelling need to focus on reliable software algorithms that allow nonspecialists to approach a control problem with condence. Software tools that provide information to the user about the likelihood of success and the difculties in a particular solution are called for to make this a reality.
BIBLIOGRAPHY
1. F. Cucker L. Blum S. Smale M. Shub Complexity and Real Computation, Berlin: Springer-Verlag, 1998. 2. N. Munro (ed.) Symbolic Methods in Control System Analysis and Design, Institution of Electrical Engineers, 1999. 3. L. A. Rubel M. F. Singer A differentially algebraic elimination theorem with applications to analog computability in the calculus of variations, Proc. Amer. Math. Soc. 94: 653658, 1985. 4. P. Lindskog Methods, algorithms and tools for system identication based on prior knowledge, PhD thesis 436, Department of Electrical Engineering, Link oping University, Sweden, 1996. 5. D. Ne si c Dead-beat control for polynomial systems, PhD thesis, RSISE, Australian National University, Canberra, Australia, 1996. 6. J. Little D. Cox D. OShea Ideals, Varieties and Algorithms, Berlin: Springer-Verlag, 1992. 7. A. Mees Dynamics of Feedback Systems, Chichester: Wiley, 1981. 8. K. Forsman Constructive commutative algebra in nonlinear control, PhD thesis, Department of Electrical Engineering, Link oping University, Link oping, Sweden, 1997. 9. D. Ne si c A note on observability tests for general polynomial and simple WienerHammerstein systems, Syst. Control Lett., 35: 219227, 1998. 10. S. T. Glad An algebraic approach to bangbang control, European Control Conference, ECC 95, Rome, 1995, Vol. 4, pp. 28922895. 11. T. Georgiou U. Walther A. Tannenbaum Computational algebraic geometry and switching surfaces in optimal control, Proc. Conf. Decision Control, Phoenix, AZ, 1999, pp. 47244729. 12. H. Fortell Algebraic approaches to normal forms and zero dynamics, PhD thesis, Department of Electrical Engineering, Link oping University, Link oping, Sweden, 1995. 13. R. Germundsson Symbolic systems, PhD thesis, Department of Electrical Engineering, Link oping University, Link oping, Sweden, 1995. 14. J. Gunnarsson Symbolic methods and tools for discrete event dynamic systems, PhD thesis, Department of Electrical Engineering, Link oping University, Link oping, Sweden, 1997. 15. F. D. Kronewitter III Non-commutative computer algebra, PhD thesis, University of California, San Diego, 2000.
22
16. M. Fliess S. T. Glad An algebraic approach to linear and nonlinear control, in H. L. Trentelman and J. C. Willems (eds.), Essays on Control: Perspectives in the Theory and Its Applications, Boston: Birkhauser, 1993. 17. J. F. Ritt Differential Algebra, New York: American Mathematical Society, 1950. 18. E. R. Kolchin Differential Algebra and Algebraic Groups, New York: Academic Press, 1973. 19. R. M. Cohn Difference Algebra, Huntington, NY: R. E. Krieger, 1979. 20. A. Seidenberg An elimination theory for differential algebra, University of California Publications in Mathematics, New Series, 1956, pp. 3166. 21. A. Tarski A Decision Method for Elementary Algebra and Geometry, 2nd ed. University of California Press, 1948. 22. G. E. Collins Quantier elimination for real closed elds by cylindrical algebraic decomposition, Second GI Conf. Automata Theory and Formal Languages, Kaiserslauten, Lecture Notes in Comput. Sci., 33 Berlin: Springer-Verlag, 1975, pp. 134183. 23. G. E. Collins H. Hong Partial cylindrical algebraic decomposition for quantier elimination, J. Symbolic Comput. 12: 299328, 1991. 24. S. McCallum An improved projection for cylindrical algebraic decomposition, in B. F. Caviness and J. R. Johnson (eds.), Quantier Elimination and Cylindrical Algebraic Decomposition, Monographs in Symbolic Computation, Berlin: Springer-Verlag, 1998, pp. 242268. 25. A. Strzebonski Solving algebraic inequalities with Mathematica version 4, Mathematica J.7(4): 525541, 2000. 26. S. Basu R. Pollack M.-F. Roy On the combinatorial and algebraic complexity of quantier elimination, Assoc. Comput. Mach., 43(6): 10021045, 1996. 27. R. Loos V. Weispfenning Applying linear quantier elimination, Comput. J. 5(36): 450461, 1993. 28. A. Strzebonski An algorithm for systems of strong polynomial inequalities, Mathematica J. 4(4): 7477, 1994. 29. S. Wolfram The Mathematica Book, 4th ed., Champaign, IL: Wolfram Media, Cambridge University Press, 1998. 30. M. Jirstrand Constructive methods for inequality constraints in control, PhD thesis, Department of Electrical Engineering, Link oping University, Link oping, Sweden, May 1998. 31. D. S. Arnon A bibliography of quantier elimination for real closed elds, J. Symbolic Comput. 5(12): 267274, 1988. 32. G. E. Collins Quantier elimination by cylindrical algebraic decompositiontwenty years of progress, in B. F. Caviness and J. R. Johnson (eds.), Quantier Elimination and Cylindrical Algebraic Decomposition, Berlin: Springer-Verlag, 1998. 33. B. Anderson N. Bose E. Jury Output feedback stabilization and related problemssolution via decision methods, IEEE Trans. Autom. Control, AC-20: 5365, 1975. 34. F. R. Gantmacher Matrix Theory, Vol. II, Chelsea: New York, 1960. 35. D. Ne si c I. M. Y. Mareels Stabilizability and stability for implicit and explicit polynomial systems: a symbolic computation approach, Eur. J. Control 5: 3243, 1999. 36. D. Ne si c I. M. Y. Mareels Dead-beat controllability of polynomial systems: symbolic computation approaches. IEEE Trans. Autom. Control 43: 162175, 1998. 37. D. Ne si c M. Jirstrand Stabilization of switched polynomial systems, in IMACS Conf. on Applications of Computer Algebra (ACA98), Prague, 1998. 38. P. Dorato W. Yang C. T. Abdallah Quantier elimination approach to frequency domain design, in S. Tarbouriech and G. Garcia (eds.), Control of Uncertain Systems with Bounded Inputs, Lecture Notes in Control and Information Sciences, 227, Berlin: Springer-Verlag, 1997, pp. 165172. 39. P. Dorato W. Yang C. Abdallah Robust multi-objective feedback design by quantier elimination, J. Symbolic Comput. 24(2): 153159, 1997. 40. M. Jirstrand Nonlinear control system design by quantier elimination, J. Symbolic Comput. 24(2): 137152, 1997. 41. P. Dorato D. Famularo C. T. Abdallah W. Yang Robust nonlinear feedback design via quantier elimination theory, Int. J. Robust Nonlinear Control, 9: 817822, 1999.
IC D. NES I. M. Y. MAREELS The University of Melbourne S. T. GLAD Link oping University M. JIRSTRAND MathCore AB
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Software for Control System Analysis and Design, Singular Value Decomposition Standard Article Paul Van Dooren1 1Universit Catholique de Louvain, Centre for Systems Engineering and Applied Mechanics, Louvain-la-Neuve, Belgium Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1040 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (198K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Singular Value Decomposition Numerical Background Numerical Robustness of Singular Values Applications in Systems and Control Concluding Remarks About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
464
SOFTWARE FOR CONTROL SYSTEM ANALYSIS AND DESIGN, SINGULAR VALUE DECOMPOSITION
extends to two spaces A and B of dimension m spanned by the orthonormal columns of two given matrices A and B: A A = Im , B B = Im The denition uses now the singular value decomposition of the inner product
The singular value decomposition (SVD) goes back to the beginning of this century. In a paper of Beltrami (1) it was shown for the rst time that any n n matrix A can be diagonalized via orthogonal row and column transformations. For the more general case of an n n complex matrix A, the result says that there exist unitary matrices U and V of dimension n n and a real diagonal matrix diag1, . . ., n such that A = U V where 1 n 0. If A is real then U and V are also real. This implies indeed that U and V diagonalize A since U AV = The decomposition is in nature close to an eigenvalue decomposition, which was well known at the time. But this new decomposition is also very different since singular values are always positive real, whereas eigenvalues are in general complex. Also, the transformations are unitary in this decomposition, whereas in the eigenvalue decomposition they are just nonsingular, and hence can be quite badly conditioned (see the next section for a more detailed discussion). The use of this new decomposition was not apparent from the very beginning, but nowadays it has become an invaluable tool in several application areas such as statistics, signal processing, and control theory. The rst important property that was observed is the perturbation result for the singular values of a matrix A. If A is a small perturbation of the matrix A, then its singular values i are perturbed by an amount that can be bounded by the norm of the perturbation A. The fact that the sensitivity of singular values to perturbations is rather low makes them a good candidate for measuring certain variations in an observed phenomenon or in a model for it, and this is also its principal use in engineering applications. More formally, one can show that several matrix norms can actually be expressed in terms of its singular values. The most important ones are the 2-norm and the Frobenius norm:
A B = U V BV, then the columns AU and B Indeed, if one takes A of A and B span the same spaces and still are orthonormal: A = Im , B = Im B A Moreover, B = A = diag{1 , . . ., m }
and from this diagonal form one can dene the canonical angles between the spaces A and B as cos i i, i 1, . . ., m. The second important property is that the singular value decomposition yields a direct construction of best lowerrank approximation to a given matrix A. Let us indeed rewrite the SVD in its dyadic form:
n
A=
i= 1
i ui v i
Then the matrices

r
Ar =
i= 1
i ui v i
are rank r matrices and the error . Ar = A Ar =

n i= r+ 1
i ui v i
Ax 2 . = max = 1 (A ), x =0 x 2
. =
i, j
|ai, j |2 =
i2
i
But the singular values are also used to measure angles. The well-known formula for the angle between two real vectors a and b of norm 1 . aT a = 1, bT b = 1, cos = |aT b|
has obviously norm Ar2 r1, which is the minimal norm among all possible rank r approximations (see Ref. 2 and the section entitled Robustness in Systems and Control). This leads to the important concept of the A rank of a matrix, dened in terms of the machine accuracy of the computer used, and the norm A2 of the given matrix. For A A2 one denes the A rank of A as the smallest rank r of Ar within A distance of A. It turns out that this is the most reliable way to recognize rank deciency of a given matrix and hence it is an important tool in engineering. In several applications the order of the system identied from observed measurements and the minimality of a constructed model indeed amount to rank determination problems (see the section entitled Applications in Systems and Control). A nal important feature of the SVD is that it puts the matrix A in a diagonal form under orthogonal (or unitary) transformations of its columns and rows, and these transformations have good numerical properties. Interpreting y Ax as a mapping from a space X to a space Y , we have thus transformed the coordinate systems in both spaces (by a wellbehaved transformation) such that the mapping becomes diagonal. It is obvious that this coordinate system will reveal
465
special properties of the original mapping since now it is decoupled in a set of scalar equations. We will see that in the context of dynamical systems this coordinate system plays a fundamental role in what are called balanced realizations (see the section entitled Balanced Realization).
NUMERICAL BACKGROUND The importance of the SVD is strongly tied to numerical aspects. For this reason, we rst give a very brief discussion of numerical stability and conditioning, which play a very important role in the study of numerical algorithms. For more details we refer to standard textbooks such as Refs. 3 and 4. We also choose the example of the singular value decomposition to introduce the relevant concepts. Let A be an arbitrary m n matrix. Then it is well known that there always exist unitary matrices U and V such that U A = R , A V = C 0 0 (5)
THE SINGULAR VALUE DECOMPOSITION Before we give more details about its basic properties, we rst recall the singular value decomposition in its most general form. Theorem 1. Let A be a m n complex matrix of rank r. Then there exist unitary matrices U and V of dimensions m m and n n, respectively, such that A = U V where (1)
0 Rm n 0
(2)
and r diag1, . . ., r with 1 r 0. If A is real then U and V are also real. The numbers 1, . . ., r together with r1 0, . . ., min(m,n) 0 are called the singular values of A. The columns ui, i 1, . . ., m of U are called the left singular vectors of A and the columns vi, i 1, . . ., n of V are called the right singular vectors of A. They also appear in the dyadic decomposition of A:
r
where R and C have, respectively, r linearly independent rows and columns. This implies, of course, that r is the rank of the matrix A. We call such transformations a row and column compression of the matrix A, respectively, and R and C are said to be of full row rank and full column rank, respectively. These decompositions can, for example, be computed with the singular value decomposition Eqs. (1) and (2). It is easy to verify that U*A and AV yield, respectively, a row and a column compression of the matrix A. In this new coordinate system, the kernel and image of the map U*AV are also simple to express. Indeed,
Im U AV = Im
Ir , 0
Ker U AV = Im
0 In r
As a consequence we also have (2)
Im A = Im U
Ir = Im U1 , 0
Ker A = Im V
0 = Im V2 In r
A=
i= 1
i ui v i
(3)
which is nothing but an alternative way of writing A UV*. Written in block form, this becomes A = U1
rV1
where U1 is the submatrix of the rst r columns of U and V2 is the submatrix of the last n r columns of V. The computation of the preceding decomposition is, of course, subject to rounding errors. Denoting computed quantities by an overbar, we generally have, for some error matrix A, . A =A+ A=U V (6)
(4)
where U1 and V1 are the submatrices of U and V, respectively, containing their rst r columns. This decomposition can be viewed as a more compact form of writing Eqs. (1) and (2). It is also called a rank factorization of A since the factors have the rank r of A as at least one of their dimensions. The proof of the preceding theorem is based on the eigendecomposition of the Hermitian matrices AA* and A*A. From the SVD one can indeed see that AA = U
T
Hence, the computed decomposition does not correspond exactly to the given matrix A but rather to a perturbed version A A. When using the SVD algorithm available in the literature (5,6), this perturbation A can be bounded by A cA
A
= cA
(7)
U ,
A A = V
where T and T are clearly diagonal. Hence the left singular vectors are the eigenvectors of AA*, the right singular vectors are the eigenvectors of A*A, and the nonzero singular values are the square roots of the nonzero eigenvalues of both AA* and A*A. Deriving Theorem 1 from these connections is in fact quite simple, but we refer to Ref. 2 for the details.
where is the machine accuracy and cA is some known polynomial expression in m and n (3). Very often, this is a rough upper bound and one prefers to replace cA by some statistical A, usually close to 1. The error A induced by this estimate c algorithmcalled the backward error because it is interpreted as an error on the data thus has roughly the same norm as the input error in generated when reading in the data A in the computer. When such a bound exists for the perturbation A induced by a numerical algorithm, it is called backward stable. We can make this denition more rigorous by considering a function X f (A) with data A and solution X.
466
If the computed solution X f (A) satises X f (A A) and A A, then the algorithmic implementation f ( ) of the function f ( ) is said to be backward stable. Notice that backward stability does not warrant any bounds on the errors in the results U, , and V. This depends indeed on how perturbations on the data (namely A) affect the resulting decomposition (or the differences U U U, , and V V V). In other words, it depends on the sensitivity of the function f ( ) with input A and solution X. This sensitivity is commonly measured by the condition of f at A:
methods available to compute the SVD in a backward stable manner, that is, such that the backward error A satises Eqs. (6) and (7). Moreover, it is known that the singular values of any matrix A are in fact well conditioned: [ f ( A )] = 1 (10)
This is derived from the variational properties of singular values (2,3) and leads to the following theorem. Theorem 2. Let A be an arbitrary m n complex matrix and A an arbitrary perturbation of the same dimensions. Then the corresponding SVD of A and A A satisfy the following strict bounds:
2
. [ f (A )] = lim sup
0 AA =
X X ,
X = f (A ),
X = f (A ) (8)
Notice that we have not specied what norms are used in this denition, but in principle one can use different norms in the data and solution spaces (7). From this denition it is clear that the condition number [f (A)] is some sort of derivative of the function X f (A) that we want to compute. When [f (A)] is innite, the problem of determining X f (A) from A is ill posed (as opposed to well posed). When [f (A)] is nite and relatively large (or relatively small, the problem is said to be ill conditioned (or well conditioned). Further details can be found in Ref. 7. It is important to note that backward stability is a property of an algorithm, while conditioning is associated with a problem and the specic data for that problem. The errors in the result depend on both the stability of the algorithm used and the conditioning of the problem solved. A good algorithm should therefore be backward stable since the size of the errors in the result is then mainly due to the condition of the problem and not due to the algorithm. An unstable algorithm, on the other hand, may yield a large error even when the problem is well conditioned. We point out that if f (A) has a Taylor expansion around A, then we can write f (A ) = f (A ) + x f (A )(X X ) + O( X X
2
A 2,
This result is also proven using variational inequalities and we refer again to Ref. 2 for a proof. Suppose now that the computed singular values i are such that 1 r >
A
r+ 1 min(m,n )
Then it is reasonable to assume that r+ 1 = = min(m,n ) = 0 The level of accuracy of the computed singular values is indeed A (which is a very small quantity) and hence only 1 to r are guaranteed to be nonzero. While in general there may be little justication for setting the small A singular values equal to zero, it is indeed a sound choice in several applications. A typical example is the determination of Im A and Ker A, which, for example, plays an important role in leastsquares solutions of the equation Ax b. Notice that orthonormal bases for Im A and Ker A are given by the columns of U1 and V2 dened earlier. The condition of Im A and of Ker A is thus connected to the sensitivity of the transformations U and V of the SVD. Consider, for example, the computation of Im A (where we assume m n). As the distance function between two spaces X and Y we use the gap (X , Y ) PX PY 2, where PS is the orthogonal projector on the space S . If A has full column rank n, then
1 [Im A] = n
(9)
Setting X f (A) and X f (A) and taking norms, we then have X X x f (A ) A A + O ( X X

2
) [ f ( A )] A A
This is a very powerful inequality, which indicates that forward errors X X are bounded in norm by the sensitivity [f (A)] and the backward error A A. Forward errors depend thus on two factors: the sensitivity of the problem and the backward error induced by the algorithm, and these two factors multiply each other in the preceding bound. Bounds of the type in Eq. (7) are obtained by an error analysis of the algorithm used; see, for example, Ref. 8. The condition of the problem is obtained by a sensitivity analysis; see, for example, Refs. 4 and 8. NUMERICAL ROBUSTNESS OF SINGULAR VALUES One of the most important features of the singular value decomposition is that the singular values can be computed in a numerically reliable manner. There are indeed numerical
(11)
as illustrated by the following example (1 1, 2 a 1):
1 . 0 A= 0 0 . A=A+
0 a ; 0 0
1 0 A= 0 0
0 . 0 A= 0 0 0 c s 0 0 s c 0
0 0 ; 0 0 1 0 0 0 0 0 1
0 a 0 0
with a a2 2, c a / a, s / a. The second basis vector of Im A is rotated in Im A over an angle , where sin s,
467
and one easily checks that (Im A, Im A) s. Therefore lim

0
(Im A, Im A) 1 1 = = 2 a
The fact that the singular values have a low sensitivity to perturbations does of course not mean that every algorithm will compute them to high accuracy. The link with the eigenvalue problems AA* and A*A indicates that there can be no nite algorithm for computing singular values. An early iterative procedure was actually based on these connected Hermitian eigenvalue problems, but they have been shown to be unreliable because of the intermediate construction of squared matrices (3). It was shown in Refs. 5 and 6 that the unitary transformations U and V of the decomposition can be constructed via an iterative procedures that works directly on A to give the SVD. This algorithm rst computes unitary matrices U1 and V1 such that B U1*AV1 is in bidiagonal form, that is, only the elements on its diagonal and rst superdiagonal are non-zero. Then it uses an iterative procedure to compute unitary matrices U2 and V2 such that U2*BV2 is diagonal and non-negative. The SVD dened in Eqs. (1) and (2) is then given by U*BV, where U U1U2 and V V1V2. The computed U and V are unitary to approximately the working precision, and the computed singular values can be shown to be the exact is for A A, where A / A is a modest multiple of . Other alternative methods to compute the singular values of a matrix A were proposed later and are based on Jacobilike methods (9). They have been shown to have speed and accuracy comparable to the Golub-Kahan algorithm (10,11). As a consequence of the discussion in the preceding section, the singular values are thus computed with small absolute error. More recent results suggest that in particular cases the singular values of matrices can sometimes be computed to high relative accuracy as well (1214). We nally remark that although the singular value decomposition is the most reliable method for determining the numerical rank of a given matrix, it is considerably more expensive than, for example, the QR factorization with column pivoting, which can usually give equivalent information with less computation (15). APPLICATIONS IN SYSTEMS AND CONTROL The problems considered in this article arise in the study of dynamical systems that can be modeled as state-space models: x (t ) = Ax(t ) + Bu(t ) y(t ) = Cx(t ) + Du(t ) (12)
In the subsections that follow, we survey a number of problems from systems and control theory that rely heavily on the singular value decomposition. We shall only discuss the numerical aspects here; for the system theoretical background, we refer the reader to the systems and control literature. Impulse Response Realization Let H(z) be an p m transfer function of a discrete-time causal system, and let its impulse response be given by H (z ) =
i= 0
Hi z i
(14)
The realization problem is to nd the transfer function H(z) in state-space description, H (z ) = D + C(zIn A ) 1 B (15)
when the impulse response sequence Hi is given. In the sca approximation lar case, this problem is related to the Pade problem, for which fast methods exist (see Refs. 16 and 17 for approach is in a survey). In Ref. 16, it is shown that the Pade fact unstable, and it is better to consider a more general technique based on matrix decompositions of the Hankel matrix:
H k,l
H2 = . . . Hk
H1
H2
.. .
.. . .. .
Hl . . . . . . Hk+ l 1
(16)
Here k and l are upper bounds for the minimal dimension n of the state-space realization Eq. (15) of H(z). From the expansion of H(z) in powers of z1 one nds that H0 = D, Hi = CAi 1 B, i = 1, . . ., k + l 1
and therefore Hk,l can be factorized as follows: H k,l = C l O k where
.. .
. C l = [B AB
Al 1 B],
C . CA Ok = ... CAk 1
(17)
Here, x(t) is an n-vector of states, u(t) is an m-vector of controls or inputs, and y(t) is an p-vector of outputs. The standard discrete-time analog of Eq. (12) takes the form xk+ 1 = Axk + Buk yk = Cxk + Duk (13)
This implies that Hk,l has at most rank n and a simple argument proves that Hk,l will have exactly rank n. Since determining the order of the system requires a rank determination, it is natural to use here the SVD Eq. (4): H k,l = U1
T nV1
Typically the matrices A, B, C, and D are real. In case the results for real matrices are different we will explicitly state it.
For the construction of the triple A, B, C, let L = U1

1 /2 n ,
R=
1 /2 T n V1 ,
H k,l = L R
(18)
468
and partition these left and right factors as follows:
L=
L1 C1 = , L2 C2
R = [B 1
R 2 ] = [R 1
B2 ]
(19)
So the problem is solved as soon as the states xi are determined. But those depend on the choice of coordinates chosen i Txi; then for the state-space model. Replace indeed xi by x Eq. (22) becomes the related equation
where C1 and C2 have p rows and B1 and B2 have m columns. From the two ways [Eqs. (17) to (19)] of writing the factorization of Hk,l, one derives then that C = C1 , B = B1
x 2 y1
or
x 3 y2
A x l = yl 1 C
B D
x 1 u1
x 2 u2
x l 1 ul 1
(24)
and that A can be solved from the overdetermined systems L1 A = L2 , AR1 = R2
Tx2 y1
Tx3 y2 =
Txl yl 1 TB D Tx1 u1 Tx2 u2 Txl 1 ul 1 (25 )
TAT 1 CT 1
The particular choice of factors L and R makes the realization unique and we shall see that it is also linked to so-called balanced realizations later on (18). This realization algorithm based on the singular value decomposition of Hk,l was rst given in Refs. 19 and 20. Realization from Input/Output Data Very often one does not have access to the impulse response of the system but only to a sequence of inputs ui and corresponding outputs yi. In such cases a novel algorithm was derived in Ref. 21 based on the following Hankel matrix:
So each sequence of states X1,l = [x1 x2 xl ] (26)
H 1:k,l
z 2 = . . . zk
z1
z2
.. .
.. . .. .
zl . . . . . . zk + l 1
can only be expected to be known up to an invertible row transformation corresponding to the particular coordinate system of the reconstructed model A, B, C, D. This row transformation T leaves the row space of X1,l unchanged. Also the rank condition for Eq. (22) to be solvable implies that Eq. (23) must be full rank n since this is a submatrix of the righthand side matrix in Eq. (22). The row space of X1,l is therefore n-dimensional. This row space can now be found with the aid of the singular value decomposition based on the following theorem, proved in Ref. 21. Theorem 3. Dene . Xk+ 1,l = [xk+ 1 xk + 2 xk + l ] (27)
.. .
where
and the Hankel matrices H1:k,l and Hk1:2k,l as before, then
ui zi = yi
T T Im [X T k + 1 , l ] = Im [H 1:k , l ] Im [H k + 1:2 k , l ]
(28)
(20)
provided the input/output (I/O) data are persistently exciting and k n, l (m p)k. In practice, due to perturbations on the data, the row spaces typically do not intersect. An approximate intersection, using the singular value decomposition or some rank revealing QR decomposition, has thus to be constructed. A possible implementation of this idea is the following decomposition:
We start by noting that the problem would be much simpler if the sequence of states xk would be known as well. From Eq. (13), rewritten as
xk + 1 = yk
A C
B D
xk uk
(21)
one derives immediately the concatenated form:
x2 y1
x3 y2
A xl = C yl 1
B D
x1 u1
x2 u2
xl 1 ul 1
H 1:k,l H k+ 1:2k,l
I 0
0 Q
A11 A21
A12 03
01 02 V T A33
(29)
(22)
where [A11A12] has full column rank equal to the rank of H1:k,l [which equals r mk n under the assumption of persistence of excitation (21)] A33 has full row rank, which must be smaller than r if an intersection is to be detected A21 has full column rank equal to the dimension of the intersection, hence n denotes an arbitrary matrix
Under the assumption of persistence of excitation one shows that the right data matrix in Eq. (22) has full column rank n m and has thus a right inverse. Equivalently, Eq. (22) can be solved in a least-squares sense for the evolution matrix
E=
A C
B D
(23)
469
The order in which this decomposition is constructed is as follows. First the transformation V T is constructed to compress the columns of H1:k,l, yielding the trailing zero matrix 01. Then the rows of the trailing bottom matrix are compressed with the transformation Q, yielding 02 and a full row rank A33. Then V T is updated to yield the full column rank matrix A21 and the trailing zero matrix 03. Notice that all three steps involve a rank factorization that can be done with the singular value decomposition (or any other rank revealing factorization). The center matrix in this decomposition has a form that trivially displays the intersection of row spaces of the top and bottom parts, namely,
T AT A21 11 0 Im AT Im 12 0 0
In = Im 0 AT 0 33
shows that these matrices are positive semidenite and that they allow one to detect whether or not the realization is minimal by computing the rank of these matrices (18). This suggests the use of the SVD of both Gc and Go in order to nd minimal realizations of the system, provided a given system A, B, C, D is not necessarily minimal (see also the next section). But here we want to focus on another application of the SVD in this context. If one applies a state-space transforma, B , Tx then the system triple transforms to A tion x x , D TAT1, TB, CT1, D and the Gramians to G c C o TTGoT1. One shows then that T can be choTGcTT and G sen such that both new Gramians are equal and diagonal: c = TGc T T = G o = T T Go T 1 =G (32)
Because of the transformation V T in Eq. (29) one derives that
In T Im(H T ) Im ( H ) = V Im 0 1:k , l k+ 1; 2k,l 0

that is, the rst n rows of V T are a representation of Xk1,l. From this we can now construct A, B, C, D as explained in Eqs. (22) to (25). Balanced Realization In the preceding section we pointed out that the realization problem from an impulse response or from input/output data is only dened up to a state-space transformation T, which in principle can be chosen arbitrarily. Is there a particular coordinate system that should be chosen for some reason, and if so, how can we construct it? We develop here the concept of balanced realization that is based on the singular value decomposition and has several appealing properties. For this we rst need to dene the controllability Gramian Gc and observability Gramian Go of a system. For the continuous-time system equation (12) these are dened as follows:
which is exactly the balanced coordinate system. In order to construct T one starts from the Cholesky factorization of Gc and Go: Gc = L 1 L T 1, Go = L T 2 L2 (33)
where L1 and L2 are both lower triangular. One then computes the singular value decomposition of the upper trianguT lar matrix LT 1 L2 :
T T LT 2 L1 = U V
(34)
using, for example, the efcient algorithm described in Ref. 1 1 1/2 23. Then, dening T 1/2UTL and T1 L 1 2 V , one checks that
1 T 1 1 1 T 1 T = L 1 (V U )L2 = L1 (L1 L2 )L2 = I
(35)
and
c = TGc T T = G
1 /2
1 T T U T L 1 ( L 1 L 1 )L 1 U 1 /2
1 /2
=
1 /2
o = T T Go T 1 = G
T T 1 L 2 ( L 2 L 2 )L 2 V
. Gc ( T ) =
0
(e At B )(e At B )T dt ,
. Go ( T ) =
0
(Ce At )T (Ce At ) dt (30)
while for the discrete-time systems they are dened as follows:
. Gc ( K ) =
K 1 k=0
(Ai B )(Ai B )T ,
. Go ( K ) =
K 1 k=0
(CAi )T (CAi )
(31)
In this new coordinate system one can associate the diagonal element i of with the unit vectors ei (zero everywhere except a 1 in position i): i gives then the energy needed to control the state ei as well as the energy observed from this state. Since both these are equal the state-space system is said to be balanced (see Ref. 18 for more details). It turns out that in addition to an energy interpretation, this coordinate system has useful properties. If a singleinput/single-output system is stable and if one uses the innite-horizon Gramians, then the evolution matrix of the balanced realization
. = E
A c
b d
(36)
An intuitive interpretation (18) is that the controllability Gramian Gc measures the amount of energy needed to control the states of the system using an input sequence of a certain duration, while the observability Gramian Go measures the amount of energy one can observe from the state in an output sequence of a particular duration. Typically K and T are chosen to be innite if the system is stable, because then these Gramians can be computed efciently using Lyapunov equations (22). For a given realization A, B, C, D one easily
is symmetric up to some sign changes, that is, there exists a S SE T. As a consediagonal matrix S of 1 such that E quence of this, one shows that such systems have low sensitivity to roundoff propagation (24). The sign symmetry is not preserved for multi-input/multi-output systems or when considering nite-horizon Gramians, but nevertheless the robustness properties are (25).
470
While the singular values can be interpreted as a measure for the energy that is being transferred from the input space to the state space as well as from the state space to the output space, they also play an important role in constructing approximate models that preserve this energy as well as possible. Since the singular values are ordered decreasingly one can partition the balanced evolution matrix as follows:
A C
B D
A 11 . A = 21 C
1
A 12 A
22
C 2
B 1 B 2 D
(37)
11, B 1, D will be a good approxima 1, C and the subsystem A tion of the original system in the sense that it keeps the states that were responsible for the largest part of the energy transfer (i.e., the largest singular values). A more formal approximation measure is the so-called H norm, which can be bounded as a function of the balanced singular values if the balancing was based on innite-horizon Gramians. This socalled balanced truncation problem is also related to that of optimal approximation in the Hankel norm (26,27) and has led to a renewed interest in the partial realization problem and related topics (28). Finally, we point out here that the realization algorithm described in the section entitled Impulse Response Realization for discrete-time systems in fact constructs immediately a balanced realization. Let the matrices Cl and Ok be dened as in Eq. (17). Then
trices since they contain powers of the matrix A, which could lead to a considerable buildup of rounding errors (see Ref. 30). It is well known (see e.g., Ref. 29) that the innite-horizon Gramians, Gc() and Go(), have the same rank as Cn and On, respectively. Since the latter can be computed as the solution of Lyapunov equations, this seems a possible alternative, but it turns out to be a sensitive roundabout as well (31). A third way is to nd an appropriate coordinate system for the pair A, B or A, C such that the rank of the matrices in Eq. (41) becomes apparent. Since observability and controllability are dual to each other we discuss this for controllability only. The following theorem proposes such a coordinate change, which is orthogonal and based on a succession of singular value decompositions (31). Theorem 4. There always exists an orthogonal state-space ] has the form A transformation U such that [UTBUTAU] [B
] = A [B . =
Bc 0 X1 0 . . . . . . 0
Ac 0 A1 , 1 X2 .. .
Ac A1 , 2 A2 , 2 .. . .. . A1 , k . . . . . . Ak , k 0 A1 , k + 1 . . . . . . Ak , k + 1 Ak + 1 , k + 1
(42)
..
Xk 0
Cl CT l = OT kOk =
l 1 i= 0 k1 i= 0
. (Ai B )(Ai B )T = Gc (l ), (38) . (CAi )T (CAi ) = Go (k )
where Ai,i, i 1, . . ., k, are ri ri matrices, and Xi, i 1, . . ., k, are ri ri1 matrices of full row rank ri (with r0 m). The matrices Xi are constructed recursively as the result or row compressions, using, for example, the SVD (31). In this new coordinate system, one easily sees that the controllability n UTCn has the form matrix C
and from the choice of factorization, Eq. (18), it follows that Cl =

1 /2 T n V1 ,
O k = U1
1 /2 n
(39)
Since U1 and V1 have orthonormal columns, one obtains
C lC T l = OT k Ok =
1 /2 T n V1 V1 1 /2 T n U1 U1
1 /2 n = 1 /2 = n
n, n
(40)
Controllability and Observability The concepts of controllability and observability play a fundamental role in systems and control theory. A system in the form of Eq. (12) is indeed a minimal representation of the input/output behavior of the system if and only if it is both controllable and observable. The same holds for a discretetime system, Eq. (13), except that there one talks about reachability rather then controllability. The conditions for controllability (reachability) and observability are equivalent to the following rank conditions, respectively (29), rank C n = n, rank O n = n (41)
n = [B B n1 B ] A ...A C X1:1 . 0 . X1:2 . . . . .. . = . 0 . . . . . . . . X1:k 0 0 0
. . . . . . 0
. . . . . . 0
(43)
where X1:i is the product X1 . . . Xi. Since these products have n full row rank ri by construction, the factorization Cn UC has a second factor that is row compressed, and the result thus follows. The controllable subspace is dened as the space spanned by the columns of Cn. It follows from Eq. (43) that k this space has dimension nc i1 ri and that
n = Im ImC
In c 0
In principle one could use the SVD to check these rank conditions, but it is not recommended to construct these ma-
n this space is spanned in the original coordiSince Cn UC nate system A, B by the rst nc columns of U. The matrix pair Ac, Bc is shown to be controllable, and the eigenvalues
471
of Ac are called the uncontrollable modes of the pair A, B. The indices ri are the so-called controllability indices of the pair A, B (31). A dual form of the staircase algorithm applied to A, C yields a dual result:
geneous systems x = Ax, xk+ 1 = Axk (47)
V T AV CV
Ao = 0 0
Ao Co
(44)
where the subsystem Ao, Co is observable. If no is the number of rows or columns of Ao then the rst no columns of V span Ker On, which is called the unobservable subspace of the pair A, C. Combining the preceding decompositions, one constructs an orthogonal state-space transformation yielding a trans, B , D which has the form , C formed system A
A C B D =
A11 0 0 0
A12 A22 0 C2
A13 A23 A33 C3
B1 B2 0 D
(45)
and where the subsystem A22, B2, C2, D is minimal, that is, both observable and controllable (reachable). Moreover, the , B , D and A22, B2, C2, D are equal. , C transfer functions of A This form is closely related to the Kalman decomposition and for its construction we refer to Ref. 31. In addition to the controllable and unobservable subspaces of a system, there are other spaces that play a fundamental role in the control of systems modeled as Eqs. (12) and (13). Two other fundamental objects in the so-called geometric system theory (32) are the supremal (A, B)-invariant and controllability subspaces contained in a given subspace. As shown in Refs. 31 and 33 they can also be computed via a matrix recurrence based on a sequence of SVDs constructing an orthogonal basis for the relevant spaces. The role of the SVD in these staircase algorithms is not only the reliable rank determination of the subsequent steps, but at the same time the singular values allow one to assess the sensitivity of the computed bases (31). Robustness in Systems and Control In the last decade, there has been a signicant growth in the theory and techniques of robust control. These developments mainly center around two concepts: H (34) and the structured singular value (35). They both provide a framework for synthesizing robust controllers for linear systems, in the sense that they achieve a desired system performance in the presence of a signicant amount of uncertainty in the system. In this section, we rst focus on H techniques. The H norm of a stable rational transfer matrix H(s) (continuoustime) or H(z) (discrete-time) is dened as
which are assumed to be stable against Cs, that is, the eigenvalues of the matrix A are in a region Cs of the complex plane, which is the open left-half plane for a continuous-time system and the open unit disk for a discrete-time system, respectively. The complex stability radius measures the robustness of system stability for complex perturbations. This radius is dened as the norm of the smallest complex perturbation such that the perturbed matrix A BC becomes unstable (where B nm, C pn, and hence mp). For A BC to be unstable, it must have at least one eigenvalue in the complement of Cs. It is important to note that although Cs is convex, the set of all Cs-stable matrices St M: (M) Cs, where (M) denotes the spectrum of M, is nonconvex, as well as its complement Unst of Cs-unstable matrices. The stability radius r therefore measures the distance of a stable matrix A to the nonconvex set Unst. By continuity of the spectrum of a matrix versus perturbations on its entries, the stability radius is clearly equal to the distance from a stable matrix A to an optimal matrix A BC lying on Unst. Indeed, when a matrix A BC passes the boundary Unst, at least one of its eigenvalue must also cross Cs. The boundary Unst in the matrix space describes matrices with at least one eigenvalue in Cs. Therefore, the stability radius can be written as rC (A,B,C ) = inf{
2:
(A + B C ) Cs = }
Consider a parametrization of the boundary Cs by a real variable , such as Cs j, or Cs ej, [0, 2]. The stability radius can then be rewritten as
rC (A,B,C ) = inf [inf{

C s C s
2: 2:
det(I A B C ) = 0}] det(I C(I A ) 1 B ) = 0}] (48)
= inf [inf{
the second equality resulting from the stability of the initial matrix A (I A is invertible for Cs) and from the fact that det(I XY) 0 det(I YX) 0. The following classical result allows us to simplify this expression considerably and is based on the singular value decomposition M UV*. Theorem 5. One has the inequality
C n n
inf {
2:
det (I
1 M ) = 0} max [M ]
If there are no constraints on , the bound is attained for 1 vmax max. maxu* Combining this with Eq. (48), it follows that
1 rC (A,B,C ) = inf max [C(I A ) 1 B] C s
H (s )
. = sup max[H ( j )],
H (z )
. = sup max [H (e j )]
[0 , 2 ]
(46)
where max[ ] denotes the largest singular value of a (complex) matrix. We explain how this quantity comes about by starting from a basic robustness problem. Consider the homo-
C s
sup max[C(I A ) 1 B]
472
which is the H norm inverse of the rational transfer function H() C(I A)1B. This is therefore reduced to a nonconvex optimization problem on Cs, parametrized by a real parameter . Efcient iterative methods are available for computing this norm (36,37), and are based on the relationship between the singular values of H( j) and the imaginary eigenvalues of a Hamiltonian matrix obtained from a state-space realization of H() (38). This result is then used to develop a quadratically convergent algorithm for computing the H norm of a transfer function. Structured Singular Values In Theorem 5, it was stated that the lower bound can actually be met when there are no constraints on the perturbation . But will be in general complex, since the matrix M is in general complex [even for transfer functions H() with real coefcient matrices A, B and C]. The problem becomes more involved when one imposes to be real. It was shown in Ref. 39 that r (A,B,C ) = where, for any M pm,
Computing D (M) is a difcult numerical problem. It is a nonconvex optimization problem and its complexity can be nonpolynomial for certain norms (see Ref. 41). One approach, which is computationally rather demanding, is to formulate the problem as a nondifferentiable convex optimization problem involving the maximum singular value of a matrix obtained from M. A more efcient scheme is given in Ref. 42 and uses several smooth optimization problems that do not involve any eigenvalue or singular value computations. The computational complexity of the problem of computing D (M) has prompted several researchers to look for bounds that are easier to compute (43,44). CONCLUDING REMARKS In this paper we have given several uses of the singular value decomposition in analysis and design problems of systems and control. We have considered computational issues and useful properties of this decomposition as well, such as diagonalization, norms, and sensitivity. The list given here is far from complete. Closeness problems (45,46) and canonical forms (31,47) are just a few examples. We expect the number of applications to grow also in the future because of the serious interdisciplinary effort that is under way between the communities of the numerical linear algebra eld on the one hand and of the systems and control eld on the other hand. BIBLIOGRAPHY
1. E. Beltrami, Sulle funzioni bilineari, Giornali di Mathematiche, 11: 98106, 1873. 2. P. Lancaster and M. Tismenetsky, The Theory of Matrices, 2nd ed., New York: Academic Press, 1985. 3. G. H. Golub and C. F. Van Loan, Matrix Computations, 2nd ed., Baltimore, MD: Johns Hopkins Univ. Press, 1989. 4. N. Higham, Accuracy and Stability of Numerical Algorithms, Philadelphia, PA: SIAM, 1995. 5. G. H. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, SIAM J. Numer. Anal., 2: 205224, 1965. 6. G. H. Golub and C. Reinsch, Singular value decomposition and least squares solutions, Numer. Math., 14: 403420, 1970. 7. J. R. Rice, A theory of condition, SIAM J. Numer. Anal., 3: 287 310, 1966. 8. J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford, England: Oxford University Press, 1965. 9. E. G. Kogbetliantz, Solution of linear equations by diagonalization of coefcient matrix, Quart. Appl. Math., 13: 123132, 1955. 10. C. Paige and P. Van Dooren, On the quadratic convergence of Kogbetliatzs algorithm for computing the singular value decomposition, Lin. Alg. Appl., 77: 301313, 1986. 11. J.-P. Charlier and P. Van Dooren, On Kogbetliantzs SVD algorithm in the presence of clusters, Lin. Alg. Appl., 95: 135160, 1987. 12. J. Demmel and W. Kahan, Accurate singular values of bidiagonal matrices, SIAM J. Sci. Stat. Comput., 11: 873912, 1990. 13. V. Fernando and B. Parlett, Accurate singular values and differential qd algorithms, Numer. Math., 67: 191229, 1994. 14. G. Golub, K. Slna, and P. Van Dooren, Computing the Singular Values of Products and Quotients of Matrices, in M. Mooren and B. DeMoor (eds.), SVD in Signal Processing III, Algorithms, Architectures and Applications, Amsterdam: Elsevier, 1995.
C s
sup [H ()]
(49)
(M ) = inf 2
(0 , 1]
Re M 1 Im M
Im M Re M
!
(50)
This becomes now an optimization problem in two real parameters and , but in Ref. 39 it is shown that the function Eq. (50) is unimodal in . In Ref. 40 an efcient algorithm is then derived to compute this real stability radius based on ideas of Ref. 36. The computation of the real stability radius can be considered a special case of the more general structured stability radius. Structured singular values (35) have been introduced to provide a perturbation bound for structured uncertainties in control system analysis and design. Therefore, the structured singular value approach can be viewed as a complement for the H approach. In a linear system with multiple independent norm-bounded perturbations, it is always possible by rearranging the system to isolate the perturbations as a single large block-diagonal perturbation . Then, denoting the transfer function from the collective outputs of the perturbation to their inputs by M(s), the stability problem reduces to ensuring that det(I M) 0 at all frequencies and for all allowable . Notice that is again not arbitrary anymore and therefore Theorem 5 does not apply. The largest singular value of that theorem has thus to be replaced by the so-called largest structured singular value, which is dened as follows. Let the set of allowable perturbations be denoted by D nn and be dened as
D ={
r c = block diag(1 Ik , . . ., r p Ik p , 1 Ik
1
C 1 , . . .,
C r ):
ir R, ic C ,
p+1
c , . . ., q Ik p + q ,
C i
C c i c i }
(51)
The largest structured singular value of an n n complex matrix M is then dened as 0 if det (I M ) = 0 for all D 1 D (M ) = (52) min max ( ) otherwise
{ D : det(I M )= 0 }
SOFTWARE HOUSES 15. T. F. Chan, Rank revealing QR factorizations, Lin. Alg. Appl., 88/ 89: 6782, 1987. 16. L. S. de Jong, Towards a formal denition of numerical stability, Numer. Math., 28: 211220, 1977. problem, 17. A. Bultheel, Recursive algorithms for the matrix Pade Math. Comput., 35: 875892, 1980. 18. B. C. Moore, Principal component analysis in linear systems: Controllability, observability, and model reduction, IEEE Trans. Autom. Control., AC-26: 1731, 1981. 19. H. Zeiger and A. McEwen, Approximate linear realizations of given dimension via Hos algorithm, IEEE Trans. Autom. Control, AC-19: 153, 1974. 20. S. Kung, A New Identication and Model Reduction Algorithm via Singular Value Decompositions, Proc. 12th Asilomar Conf. Circuits Syst. Comp., 1978, pp. 705714. 21. M. Moonen et al., On- and off-line identication of linear statespace models, Int. J. Control, 49: 219232, 1989. 22. R. Patel, A. Laub, and P. Van Dooren, Numerical Linear Algebra Techniques for Systems and Control, Piscataway, NJ: IEEE, 1993. 23. M. T. Heath et al., Computing the singular value decomposition of a product of two matrices, SIAM J. Sci. Stat. Comput., 7: 1147 1159, 1986. 24. C. Mullis and R. Roberts, Synthesis of minimum roundoff noise xed point digital lters, IEEE Trans. Circuit Syst., CAS-23: 551 562, 1976. 25. M. Gevers and G. Li, Parametrizations in Control, Estimation and Filtering Problems, London: Springer, 1993. 26. V. Adamjan, D. Arov, and M. Krein, Analytic properties of Schmidt pairs for a Hankel operator and the generalized SchurTakagi problem, Mat. USSR Sbornik, 15: 3173, 1971. 27. K. Glover, All optimal Hankel norm approximations of linear multivariable systems, and their L error bounds, Int. J. Control, 39: 11151193, 1984. 28. A. Antoulas, New results on the algebraic theory of linear systems: The solution of the cover problems, Lin. Alg. Appl., 50: 1 45, 1983. 29. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1980. 30. C. Paige, Properties of numerical algorithms related to computing controllability, IEEE Trans. Autom. Control, AC-26: 130 138, 1981. 31. P. Van Dooren, The generalized eigenstructure problem in linear system theory, IEEE Trans. Autom. Control, AC-26: 111129, 1981. 32. W. M. Wonham, Linear Multivariable Control: A Geometric Approach, 2nd ed., New York: Springer, 1979. 33. V. C. Klema and A. J. Laub, The singular value decomposition: Its computation and some applications, IEEE Trans. Autom. Control, AC-25: 164176, 1980. 34. G. Zames, Feedback and optimal sensitivity: Model reference transformations, multiplicative semi-norms, and approximate inverses, IEEE Trans. Autom. Control, AC-26: 301320, 1981. 35. J. C. Doyle, Analysis of feedback systems with structured uncertainties, IEE Proc., Pt. D, 129: 242250, 1982. 36. S. Boyd, V. Balakrishnan, and P. Kabamba, A bisection method for computing the H norm of a transfer matrix and related problems, Math. Control Signals Syst., 2: 207219, 1989. 37. N. A. Bruinsma and M. Steinbuch, A fast algorithm to compute the H norm of a transfer matrix, Syst. Control Lett., 14: 287 293, 1990. 38. R. Byers, A bisection method for measuring the distance of a stable matrix to the unstable matrices, SIAM J. Sci. Stat. Comput., 9: 875881, 1988.
473
39. L. Qiu et al., A formula for computation of the real stability radius, Automatica, 31: 879890, 1995. 40. J. Sreedhar, P. Van Dooren, and A. L. Tits, A Level-Set Idea to Compute the Real Hurwitz-Stability Radius, Proc. 34th IEEE Conf. Dec. Control, New Orleans, 1995, WA05. 41. O. Toker and H. Ozbay, On the NP-Hardness of the Purely Complex Computation, Analysis/Synthesis, and Some Related Problems in Multi-Dimensional Systems, Proc. 1995 Amer. Control Conf., 1995, pp. 447451. 42. M. K. H. Fan and A. L. Tits, Characterization and efcient computation of the structured singular value, IEEE Trans. Autom. Control, AC-31: 734743, 1986. 43. P. M. Young, M. P. Newlin, and J. C. Doyle, Practical Computation of the Mixed Problem, Proc. 1992 Amer. Control Conf., Chicago, 1992, pp. 21902194. 44. C. T. Lawrence, A. L. Tits, and P. Van Dooren, A Fast Algorithm for the Computation of an Upper Bound on the -norm, Proc. 13th IFAC World Congr., San Francisco, 1996, vol. H, pp. 5964. 45. R. Eising, Between controllable and uncontrollable, Syst. Control Lett., 4: 263264, 1984. 46. D. Boley, Computing rank-deciency of rectangular matrix pencils, Syst. Control Lett., 9: 207214, 1987. 47. M. M. Konstantinov, P. Hr. Petkov, and N. D. Christov, Invariants and canonical forms for linear multivariable systems under the action of orthogonal transformation groups, Kybernetika, 17: 413424, 1981.
PAUL VAN DOOREN

Catholique de Louvain, Universite Centre for Systems Engineering and Applied Mechanics
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Stability Theory, Asymptotic Standard Article Halit Eren1 and Alexander Lowe1 1Curtin University of Technology, Australia, Bentley, Australia Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1041 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (306K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Stability Criteria Bounded-InputBounded-Output Stability Asymptotic Stability RouthHurwitz Criterion Nyquist Criterion The Root Locus The Riccati Equation Lyapunov Stability Robust Stability Exponential Stability About Wiley InterScience | About Wiley | Privacy | Terms & Conditions
Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
STABILITY THEORY, ASYMPTOTIC
341

STABILITY CRITERIA Automatic control is an essential part of engineering and science. It nds applications in many areas from space vehicles and missiles to industrial processes and medicine. Automatic control devices, laboratory equipment, design and analysis tools, and complete automatic processes and systems are offered by many companies, some of which are listed in Table 1. Basically, a control system consists of interconnected components that achieve a desired response. In order to meet the objectives effectively, the system must be understood fully and properly modeled mathematically. When the system is mathematically represented, it may be designed appropriately and the performance may be examined and analyzed. For the performance analysis, many methods are available. For example, the classic control theory is the earliest and one of the most established methods, mainly applied in simple systems. Although a nonlinear approach is available, in classic control theory, the foundations of analysis are mainly based on linear system theory. The linear system approach assumes a causeeffect relationship between the components of the system and expresses this relationship as differential equations.
342
Table 1. List of Manufacturers Automated Applications Inc. 680 Flinn Ave. Unit 36 Moorpark, CA 93021 Tel: 800-893-4374 Fax: 805-529-8630 Automation Technologies International 17451 W. Dartmoor Drive Grayslake, IL 60030-3014 Tel: 708-367-3347 Fax: 708-367-1475 Capitol Technologies, Inc. 3613 Voorde Drive South Bend, IN 46628 Tel: 219-233-3311 Fax: 219-233-7082 CEI Automation 15250 E. 33rd Place Aurora, CO 80011 Tel: 303-375-0050 Fax: 303-375-1112 Control Engineering Company 8212 Harbor Springs Road Harbor Springs, MI 49740 Tel: 800-865-3591 Design Technology Corporation 5 Suburban Park Drive Billerica (Boston), MA 01821 Tel: 508-663-7000 Fax: 508-663-6841 Fata Automation 37655 Interchange Drive Farmington, MI 48335 Tel: 810-478-9090 Fax: 810-478-9557 FTI International, Inc. Hashikma St. Ind. Zone P.O.B. 87 Kfar Saba, Israel Tel: 052-959152-4 Fax: 052-959162 Kuntz Automation Engineering 402 Goetz Street, Dept. 7 Santa Ana, CA 92707 Tel: 714-540-7370 Fax: 714-540-6287 The Math Works, Inc. 24 Prime Park Way Natic, MA 01760-1500 Tel: 508-647-7000 Fax: 508-647-7001 Munck Automation Technology 161-T Enterprise Drive Newport News, VA 23603 Tel: 804-887-8080 Fax: 804-887-5588 Portech Pathnder Operations 1610 Fry Avenue, Dept. T Canon City, CO 81212 Tel: 800-959-0688 Fax: 719-269-1157 Prime Automation, Inc. 1321 Capital Drive Rockford, IL 61109-3067 Tel: 815-229-3800 Fax: 815-229-5491
where r(t) reference input e(t) error signal c(t) output signal G forward path gain or process transfer function H feedback gain Eliminating the error e(t) and rearranging Eqs. (1) and (2) gives the closed-loop gain M= G c(t ) = r(t ) 1 + GH
or in the Laplace transform domain, M (s ) = G (s ) C (s ) = R (s ) 1 + G ( s )H ( s ) (3)
Once the system performance is stated as differential equations in the time domain, Laplace transforms are commonly, used for frequency analysis. Some examples of Laplace transforms are given in Table 2. A control system can be an open loop or closed loop. Figure 1 illustrates an open-loop system in which a controller controls the process without using any feedback. In this case, the output is not compared with the input; therefore, deviations of the output from the desired value cannot be automatically corrected. This method nds limited application since it does not lead to fully automatic systems. In a closed-loop system, the actual output is compared with a desired reference input by a suitably arranged feedback mechanism. Figure 2 illustrates a single-inputsingle-output feedback control system. In this system, a prescribed relationship of one system variable to another is maintained by comparing the two and using the difference as a means of control. Using system modeling and mathematical representations, a closed loop control system with a single-input single-output may be represented as shown in Fig. 3. In this case, the relationship between the input and output of the single-input single-output system can be expressed as e(t ) = r(t ) c(t )H c(t ) = e(t )G (1) (2)
The main effect of the feedback is that it reduces the error between the reference input and the system output, thus forcing the system output to follow or track the reference input. It also affects the forward gain G by a factor of 1/(1 GH). This effect is one of the main subjects of study in classic control theory. For example, when GH 1 the gain M will be innite; hence, C will increase without bound, thus leading to unstable conditions. In many cases, the control systems are much more complex than a single-inputsingle-output arrangement. They may have many inputs and outputs and controlled variables that are interrelated as shown in Fig. 4. These systems necessitate a multivariable control system approach for analysis and design. However, the feedback concept of the single-input single-output linear system will be the main concern of this article. A control system needs to be designed carefully with a suitable conguration with clearly identied specications to achieve the desired performance. In this process, the identication of key parameters, mathematical representation of the system, and system analysis play important roles. After having mathematically represented the system, it is possible to employ analytical tools to describe the characteristics of the feedback control system. Important characteristics, such as the transient and steady state performance, frequency response, sensitivity, and robustness can be studied in detail. When these characteristics are known, the desired response can be obtained by adjusting the system parameters. Therefore, a good understanding and effective use of stability theory in control systems is very important. If the system is not stable, it will display an erratic and destructive response and will get out of bounds and disintegrate. The transient response of a system is related to its stability. Typical responses times of second-order systems are illustrated in Fig. 5. In this system, the outputs are bounded by the decaying oscillations. Therefore, a stable system may be dened as a system with bounded response. If the oscillations increase with time, the system is said to be unstable. The stability of a dynamic system can be described by its response to an input disturbance. The output response can be either decreasing, increasing, or neutral, giving an indication of sta-
343
Table 2. Laplace Transform Table Time Domain Function u(t) (unit step) t t n, for any positive integer n eat eat ebt ba n
1 2
Laplace Transform 1 s 1 s2 n! s n1 1 sa 1 (s a)(s b) 2 n s 2 n s 2 n

2
ent sin(n1 2 t )
sin(n t ) cos(n t ) 1
1 2
n s 2 2 n s s 2 2 n e
n t
sin(n1 2 t ) where tan1
1 2
2 n s(s 2 n s 2 n)
2
1 cos(n t ) 1 et/T 1 [sin(n t ) n t cos(n t )] 2n 1 2T (1 2T )et/T
2 n s(s 2 n)
2
1 s(1 Ts) s2 2 (s 2 n)
2
1 s 2(1 Ts)2
bility as dened by stability criteria. There are three types of stability of control systems: the bounded-inputboundedoutput (BIBO) stability, asymptotic stability, and marginal stability (stability in the sense of Lyapunov). BOUNDED-INPUTBOUNDED-OUTPUT STABILITY A system is stable if a bounded input yields to a bounded output. When the closed-loop transfer function of a linear system is expressed as a Laplace transform, the stability may be dened in terms of the locations of the poles of the system in the complex plane or the s-plane. An s-plane is shown in Fig. 6 indicating the right half plane and left half plane. Take a single-inputsingle-output system and express the input and
output relationship as
M (s ) =
C (s ) = R (s )
K
U k = 1 (s
L i = 1 (s
+ zi )
+ pk )
R 2 j = 1 (s
+ 2 j s + a 2 s + 2 + 2 ) j j j (4)
where the denominator of M(s) is the characteristic equation of the system. The roots of the characteristic equation are the poles of the closed-loop system. The time response of the output is a function of the roots of this characteristic equation.
Desired response Comparator Controller Process
Output
Reference input or Controller desired response Process
Output
Measurement
Figure 2. A closed-loop control system. In a single-inputsingle-output system, the output of the system is compared with the input, and any deviation between the two is corrected by suitably designed controllers. However, the use of feedback can lead to instability. Closed-loop control arrangements are used extensively in automatic processes and devices.
Figure 1. An open-loop control system. An open-loop control system does not compare the output and the input; therefore, any deviations between the two cannot be corrected automatically. This system is applicable only in simple cases in which the process characteristics are fully known and the outputs from the desired values are not all very important.
344
r(t) +
e(t) G
c(t)
2.0 1.8
H
Figure 3. Block diagram of a closed-loop system. A closed-loop system may be represented mathematically by a forward loop transfer function G and a feedback loop transfer function H. The relation between the input and output can be expressed in terms of these two terms in the form of a system transfer function, which is extremely useful in system design and analysis.
1.6 1.4 1.2 1.0 0.8 0.6 Step input

c(t)
For example, the time response of this system for an impulse function may be written as
U
0.4 0.2 0 1 2 3 4 5 6 7 nt 8 9 10 11 12 13
c(t ) =
k=1
(Ak e Pk ) +
R j=1
Bje
j t
sin( j t )
(5)
Left-Half-Plane Poles To clarify the important concept of locations of poles on the splane, as used in stability analysis, see Eq. (4). As we can see in this equation, the poles of the closed-loop system may be real or complex, and simple or repeated. It is often convenient to plot the poles and zeros of the closed-loop transfer function on the s-plane. The s-plane can be considered to be in three parts, the right half plane (RHP), the left half plane (LHP), and the pure imaginary axis or j-axis. If a pole lies inside the open LHP, then the pole has a negative real part. If it lies inside the closed RHP, then it has a positive or zero repeated real part. Consider poles 1/(s p)n on the s-plane. For a real p, a portion of the time domain response of the system will be proportional to c(t ) 1/n! t n 1 e pt (6)
Figure 5. Time response of a second-order system. The output of a second-order system for a unit step input contains a transient and steady state response. The sinusoidals frequency and amplitude components depend on the natural frequency and the damping ratio of the system. If the oscillation increases without bound, the system is said to be unstable. The stability can be related to the locations of the poles on the s-plane.
2. The time response of a pole approaches a nonzero constant as t if and only if the pole is simple and located at s 0. As indicated earlier, in order to obtain a bounded response to a bounded input, the poles of the closed-loop system must be in the left-hand portion of the s-plane. That is, pk 0 and j 0 so that the exponential terms epkt and ejt decay to zero as the time goes to innity. A necessary and sufcient condition is that a feedback system is stable if all the poles of the system transfer function have negative real parts. If the characteristic equation has simple roots on the imaginary axis ( j) with all other roots on the left half plane, the steady state output is sustained oscillations for a bounded input. If the input is sinusoidal with a frequency equal to the magni-
If p 0, it lies on the RHP, and its response increases exponentially. If it is at the origin, p 0 and simple, its response is a step function. When p 0 and repeated with multiplicity n 2, then its response approaches innity as t . If the poles are in the LHP or p 0, then the response ept approaches zero as t . Therefore, 1. The time response of a pole, simple or repeated, approaches zero as t if and only if the pole lies inside the open-loop LHP or has a negative real part.
Inputs (desired responses)
Imaginary axis j
s-plane
LHP stable
RHP unstable Real axis
Controllers
Process
Outputs
Measurement
Figure 4. A multivariable control system. Many control systems have multiple inputs and multiple outputs. In these cases, using the multivariable control theory and matrix approach is applicable. In the design and analysis, most of the theories developed for singleinputsingle-output systems can still be used.
Figure 6. A complex plane. The response of control system depends on the locations of poles and zeros of the characteristic equation on the complex plane also known as the s-plane. The poles in the closedloop transfer function on the right half plane lead to instability because the exponential term in the time domain representation increases as time increases. The poles in the LHP indicate stable conditions, whereas poles on the imaginary axis may lead to stable or unstable conditions. In this case, repetition of roots on the imaginary axis and the inputs must be taken into account.
345
tude of the j-axis pole, the output becomes unbounded. This is called marginal stability because only certain bounded inputs cause the output to become unbounded. For an unstable system, the characteristic equation has at least one root in the right half of the s-plane, that is at least one of the exponential terms epkt and/or ejt will increase indenitely as the time increases. Repeated j-axis roots will also result in an unbounded output for any input. ASYMPTOTIC STABILITY In general, the output response of a linear time-invariant system may be divided into two components. 1. The forced response is the part of the response that has the same form as the input. 2. The natural response is the part of the response that follows a form, which is dictated by the poles of the characteristic equation. In some cases, the investigation of stability by using only the transfer function M(s) is not sufcient. Hence, the nature of the input signal must be taken into account. For example, a plant output c(t) is said to track or follow the reference input r(t) asymptotically if
t
H (s ) =
i= 1
F (s, pi , ri ) K1 K2 Kr + + + (s pi )r i (s pi ) (s pi )r i 1 (8)
F (s, pi , ri ) =
where there are n sets of poles, located at s pi, each of multiple of ri. The impulse response may be written in terms of the system poles by taking the inverse Laplace transform of H(s). The general expression is
n
h(t ) =
i= 1 pt
f (t , pi , ri )
(9)
f (t , p, r ) = e (k1t r 1 + k2t r 2 + + 1 )
The behavior of h(t) is dictated by the behavior of f (t, p, r). As t , f (t, p, r) becomes dominated by the epttr1 term, so the behavior of h(t) as t becomes large may be investigated by the following limit:
L = lim (t r 1 |e pt | )
t
= lim
t r1 e
(10)
lim |c(t ) r(t )| 0
(7)
where p j. The limit is in the innity divided by innity indeterminate form. Applying LHopitals rule r 1 times results in
Suppose that the transfer function of the overall control system is M(s); if M(s) is not stable the system cannot follow any reference signals. If M(s) is stable, in order for the system to be asymptotically stable, it is an additional requirement that the system be capable of following all inputs. This is important because, in some cases, the output may be excited by nonzero initial conditions such as noise or disturbance. As a result, the stability conditions may be generalized as follows. 1. The system is stable if the natural response approaches zero as t . 2. The system is unstable if the natural response grows without bound as t . 3. The system is marginally stable or marginally unstable if the natural response neither grows nor decays as t . 4. The system is stable if bounded inputs result in bounded outputs. 5. The system is unstable if bounded inputs result in unbounded outputs. Here, in order to explain asymptotic stability and to lay a rm background for the following theories, a rigorous mathematical approach may be introduced. To observe the natural response h(t) of a linear time-invariant system, a dirac delta function (impulse) (t) may be applied to the input to give the system internal energy upon which to act. The dirac delta function has a rectangular shape with a height of 1/ and a width of . is made vanishingly small so that the function has innite height and zero width and unit area. The ensuing response is the natural response. Its Laplace transform is identical to the transfer function of the system, which can be written in the general partial fraction form as
|L| = lim = lim

t
( r 1 )! r 1 e t ( r 1 )! t e r1
(11)
There are several regions of interest. 1. 0. The et term forces the limit to approach innity. 2. 0. The et term forces the limit to approach zero. 3. 0, r 1. In this case, we have a zero divided by zero indeterminate form with three independent variables. The solution is obtained by allowing the limit (, r 1, t) (0, 0, ) to be approached from an arbitrary trajectory.
L= =
( , r 1 , t ) (0 , 0 , )
lim
( r 1 )! t e r1 (12)
1 k e 1 =C
where k depends of the trajectory of approach and C is a bounded constant. 4. 0, r 1. This time we have a limit with two independent variables, (, t) (0, ).
L=
( r 1 )! t e ( , t ) (0 , ) r1 ( r 1 )! k = lim e 0 r1 lim
(13)
where k depends on the trajectory of approach.
346
We can now summarize the stability of a system depending of the location of the poles. Pole Locations Poles on the left half plane ( 0) only. Any pole on the right half plane ( 0), or pole of multiplicity greater than one on the j-axis ( 0 and r 1). Any pole on the j-axis of multiplicity equal to one ( 0 and r 1). Stability The natural response approaches zero, so the system is stable. The natural response approaches innity, so the system is unstable. The natural response approaches neither zero nor innity. It is, however, bounded. The system is called marginally stable or marginally unstable.
functions are listed, we are left with what is known as Rouths array, which is the tabular technique presented next. The RouthHurwitz criterion may be expressed as follows. There are two necessary, but not sufcient, conditions for no RHP poles. 1. All the polynomial coefcients ai must have the same sign. The coefcients are determined by cross-multiplication of roots pi. If two particular coefcients were of opposite sign, it would mean that one cross multiplication yielded a positive result whereas another yielded a negative result. This is possible only if there exist at least two pi of opposite sign, which means that one of them must be on the right half plane. 2. No ai can be zero. Cancellation of terms in the crossmultiplication implies one LHP pole and one RHP pole. If either of these two criteria is violated, it is immediately clear that the system is unstable. Otherwise, further analysis is required by the formation of the Rouths array. Lets express Eq. (14) in the following form: P ( s ) = a n sn + a n 1 sn 1 + a n 2 sn 2 + + a 1 s + a 0 (15)
Consider the case of a pole of multiplicity equal to one on the j-axis, if the input were to be a sinusoid of a frequency equal to the distance of this pole from the origin. This would have the same effect on the total response as if the input were zero and the system had a pole of multiplicity equal to two on the j-axis. The output would then approach innity even though the inputs were bounded. Consequently, a bounded input function that will produce an unbounded output exists. A system classied as marginally stable under the asymptotic stability denition is, therefore, classied as unstable under the bounded-inputbounded-output denition. ROUTHHURWITZ CRITERION To determine the stability of a system, we need to know whether any poles are located in the RHP. It is always possible to calculate these pole locations by direct computational methods, but it is not necessary. For determining system stability, it is enough just to know whether there are any poles on the RHP or not. This can be investigated using the Routh Hurwitz criterion. A generalized nth-order characteristic polynomial may be represented as
n
To form the Rouths array, the highest order coefcient an, followed by every second coefcient is listed in the rst row, labeled sn. The rest of the coefcients are listed in the second row, labeled sn1. More rows are added all the way down to s0 as illustrated.
sn sn 1 sn 2 sn 3 . . . . . . s1 s0
an an 1 bn 1 = cn 1 . . . . . . xn 1 a0 an 4 an 5 an 1 an 4 an an 5 an 1 b n 1 an 5 an 1 b n 5 = bn 1 bn 5 = cn 5 . . . . . . an 1 an 6 an an 7 an 1 b n 1 an 7 an 1 b n 7 = bn 1 an 1 an 2 an an 3 an 1 b n 1 an 3 an 1 b n 3 = bn 1
P (s ) =
i= 0 n
a i si (14) (s + pi )
i= 0
=k
an 2 an 3 bn 3 = cn 3 . . . . . . xn 3
where ai are the polynomial coefcients, k is a constant and s pi are the roots of the polynomial. The RouthHurwitz criterion is based on the Mikhailov criterion, which states that if a system is characterized by an nth order polynomial P(s), then it is necessary and sufcient for stability that the following condition be satised. The contour traced in the P(s) domain by P( j), 0 , must proceed counterclockwise around the origin and limarg[P( j)] must tend toward (n /2). If the Mikhailov criterion is applied algebraically to the generalized form of P(s) given previously, then the RouthHurwitz criterion results are based on the determinants of the coefcients. From these determinants, it is possible to derive a set of polynomials, known as subsidiary functions. If the coefcients of these
To simplify manual calculation of the table, it is useful to note that multiplication of any row by a positive constant will not affect the end result.
347
The number of sign changes in the rst column gives the number of poles located on the right half plane. As an example, take the polynomial P ( s ) = 3s 3 + s 2 + 4s + 2 The Routh array is constructed as follows: (16)
cients resulting from the derivative of the auxiliary polynomial. Consider the following polynomial: P(s ) = (s2 + 4 )(s + 1 )(s + 3 ) = s4 + 4s3 + 7s2 + 16s + 12 We proceed to build a Routh array as (19)
s3 s2 s
1
3 1 2 2
4 2 0
s4 s3 s
2
1 4 3 0
7 16 12 0
12 0
s0
s1
There are two changes of sign in the rst column, which means that there are two poles located on the RHP. We have P(s ) = 3 (s + 0.476 )(s 0.0712 + 1.18 j )(s 0.0712 1.18 j ) (17) which conrms the fact that there are two poles on the right half plane. Special Cases In forming Rouths array, there are three special cases that need further consideration. First Case. The rst column of a row is zero, but the rest of the row is not entirely zero. Take the following polynomial as an example. P ( s ) = s 5 + s 4 + 2s 3 + 2s 2 + 3s + 4 (18)
The auxiliary polynomial is formed and differentiated.
A(s ) = 3s2 + 12 A ( s ) = 6s
(20)
We may then replace the s1 row and proceed.
s4 s3 s s
2
1 4 3 6 12
7 16 12 0
12 0
s1
0
When the zero appears, replace it with a variable , to complete the table. Then take the limit as 0, both from above and below, to determine if there are any sign changes.
0+ s5 s4 s
3
0 + + +
1 1 2+ 1 4
2
2 2 1 4 2 +1 0
3 4
+ + + + +
Because there are no changes in sign in the rst column, there are no roots on the RHP. The presence of this row of zeros, however, indicates that the polynomial has an even polynomial as a factor. An even polynomial has only terms with even powers of s. It has the property that all its roots are symmetrical about both the real and the imaginary axis. Consequently, an even polynomial must have either roots in both the left and right half planes or only on the j-axis. In this case, there are no right half plane roots, so they must be located on the j-axis. In addition, the auxiliary polynomial A(s) is the same even polynomial that caused the row of zeros, so we can tell that these roots are located at s 2j. Third Case. There is a repeated root on the j-axis. The RouthHurwitz criterion indicates the existence of roots on the imaginary axis, but it does not indicate whether they are of multiplicity greater than one, which is essential knowledge if the distinction between marginal stability and instability is required. Take the following polynomial as an example: P(s ) = (s + 1 )(s2 + 4 )2 = s5 + s4 + 8s3 + 8s2 + 16s + 16 (21)
s2 s1 s0
1 4
The table shows that there are two changes of sign in the rst column, regardless of whether approaches zero from above or below in this case. Consequently, there are two roots in the right half plane. The poles are located at s1 0.6672 1.1638j, s2 0.5983 1.2632j and at s3 1.1377, conrming the result. Second Case. A whole row consists of zeros only. When an entire row of zeros is encountered in row sm, an auxiliary polynomial of order m 1 is formed by using the sm1 row as the coefcient and by skipping every second power of s. The row containing zeros is then replaced with the coef-
Auxillary polynomial s5 s4 s s
3
Derivative
Final table entry 1 1 8 8 16 16 16 16
1 1 0 4 0 16
8 8 16 16
16 16 s + 8s + 16
4 2
4s + 16s
3
4 4 8 16
s2
1
4s + 16
2
8s
s0
348
Even though none of the signs in the rst column have changed sign, there are two roots located at s 2j and two at s 2j. A system having P(s) as a characteristic equation must be considered unstable, even though the Routh Hurwitz algorithm did not predict it. Routh developed this criterion in 1877. In 1893, Hurwitz, apparently unaware of Rouths work, developed a similar technique based on determinants, from which the Routh Hurwitz criterion is derivable. In 1892, Lyapunov developed a more general technique that is applicable to both linear and nonlinear systems, called the direct method of Lyapunov.
Im
=
s-plane
R= Re
NYQUIST CRITERION Given the characteristic equation of a system, the Routh Hurwitz criterion enables a system analyst to determine whether or not the system is stable without actually solving the roots of the characteristic equation. Unfortunately, the method still requires the characteristic equation, which may be somewhat cumbersome, to be derived. The Nyquist criterion offers a graphical method of solution based on the openloop transfer function, thereby saving some algebraic manipulation. In addition, Nyquists method is quite capable of handling pure time delays, which Rouths method and the root locus method can handle only clumsily, at best. The Nyquist criterion is based on the following principal argument. Suppose a contour 1 is traced arbitrarily in the splane as shown in Fig. 7(a). If each point s, comprising 1 were to be transformed by a polynomial function of s, L(s), then a new contour 2 would result in the L(s)-plane, as illustrated in Fig. 7(b). Provided that 1 does not pass through any poles or zeros of L(s), the contour 2 does not encircle the origin. The principal argument relates the number of times that the new contour 2 encircles the origin to the number of poles and zeros of L(s) encircled by 1. In other words, 2 encircles the origin by the difference between the number of poles and number of zeros in contour 1 N = ZP (22)
Figure 8. The Nyquist contour. If a contour is traced on the s-plane covering the entire RHP in the clockwise direction and if the number of zeros of G(s)H(s) are greater than number of poles then the corresponding contour on the G(s)H(s) will encircle the origin at least once in the same direction. The poles of G(s)H(s) can usually be determined easily from the mathematical model. The number of zeros of G(s)H(s) can be determined by the Nyquist plot.
where N is the number of encirclements of the origin by 2, Z is the number of zeros of L(s) encircled by 1, and P is the number of poles of L(s) encircled by 1. A positive N indicates that 2 and 1 both travel in the same direction (i.e., clockwise or counterclockwise), whereas negative N indicates opposite directions. Some examples of contours 2 and 1 and encirclements are given in the section dedicated for Nyquist. Interested readers should refer to this section. The Nyquist Contour Consider the transfer function of a closed-loop system
s-plane
1
jv G(s)-plane
C (s ) G (s ) = R (s ) 1 + G ( s )H ( s ) G (s ) = 1 + L (s )
From this equation, the following points should be clear.
(23)
(a)
(b)
1. The poles of 1 L(s) are the poles of L(s), the open-loop transfer function. This makes identication of the poles of 1 L(s) possible simply by inspection in most cases. 2. Most importantly, the zeros of 1 L(s) are the poles of C(s)/ R(s), the closed-loop transfer function. The problem of determining the closed-loop pole locations may then be reduced to determining the zeros of 1 L(s). To do this, the Nyquist contour, which covers the entire right half s-plane, as shown in Fig. 8, is used. It consists of a section along the j-axis covering and a semicircle described by s , /2 arg(s) /2. The diagram also shows how the contour skips around simple (not repeated) poles of L(s) located on the j-axis to avoid discontinuities
Figure 7. Contours in the s-plane and G(s)H(s)-plane. Every closed contour on the s-plane traces a closed contour on the G(s)H(s)-plane. If there are any poles or zeros (but not equal in numbers) of G(s)H(s) in the contour in the s-plane, the contour in the G(s)H(s) will encircle the origin at least once. If the number of poles of G(s)H(s) inside the contour in the s-plane is greater than zero, the contour in the G(s)H(s)-plane goes in the opposite direction of the contour on the s-plane. If the zeros are greater than poles, the contours are in the same direction.
349
when the contour is mapped onto the L(s) plane. The direction of the contour has been arbitrarily drawn as clockwise. If the Nyquist contour is mapped onto the 1 L(s)-plane, we would nd that the resulting contour would encircle the origin N Z P times (in the clockwise sense). It should be emphasized that Z is the variable under investigation because it concerns the poles of the closed-loop system. The requirement for stability is that 1 L(s) contain no zeros on the right half plane, or Z 0. That is to say, if the (clockwise) Nyquist contour were mapped onto the 1 L(s)-plane, it is a requirement for closed loop stability that the resulting contour encircle the origin counterclockwise exactly the same number times as the number of poles of L(s) in the RHP. The contour shown in Fig. 8 skips to the right around jaxis poles. Consequently, these j-axis poles are not considered to be right-half-plane poles. It is perfectly feasible for the contour to skip to the left around these poles, in which case they should be included in the count of right-half-plane poles. It is emphasized that the poles of L(s) are easily obtainable. A further renement of the Nyquist criterion is that it is unnecessary to plot the contour on the 1 L(s)-plane and observe the number of encirclements about the origin. The plot on the L(s) plane is in fact identical to that of the 1 L(s)-plane, except that it is shifted left by one unit. It will therefore sufce to plot the contour on the L(s)-plane and observe the number of encirclements about the Cartesian point (1, 0). Simplied Nyquist Plot Suppose that the function 1 L(s) contains P poles on the RHP, P poles on the j-axis, and Z zeros on the right half plane and that two contours are to be mapped onto the L(s)plane. 1. The Nyquist contour skips to the right around the P poles on the j-axis. When mapped on the L(s)-plane, it is found to encircle the Cartesian point (1, 0) point N1 times. 2. The Nyquist contour skips to the left around the P poles on the j-axis. When mapped on the L(s)-plane, it is found to encircle the Cartesian point (1, 0) point N2 times. Each contour may be considered to consist of three sections, each contributing a certain number of turns about the Cartesian point (1, 0). 1. The section consisting of , excluding the skips around j-axis poles. Because of symmetry about the real axis in both the contour and the location of poles and zeros, this section may be divided into two halvesthe positive imaginary axis and the negative imaginary axis, each contributing NA turns. 2. The section consisting of the innite semicircle, contributing NB turns. If the order of the numerator of L(s) is less than or equal to the denominator, then as s , L(s) corresponds to a point on the real axis or an encirclement of the origin. The contribution to the number of turns about the Cartesian point (1, 0) in either case is, therefore, NB 0.
3. The skips around the j-axis poles. Because the two contours skip around these poles in opposite directions, if contour 1 were to contribute NC turns, then contour 2 would contribute NC turns. Combining all these sections, N1 = 2NA + NB + NC N2 = 2NA + NB NC (24) (25)
From the principal argument, it is also known that N1 Z P; therefore, N2 = Z P P (26)
Eliminating N1, N2, and NC and realizing that NB 0, we nd that
NA = or
2Z 2P P 4 P = ZP 2
(27)
where is the angle of rotation (in the clockwise sense) about the point (1, 0) when the line 0, 0 is mapped onto the L(s)-plane, where s j. For stability, we require that Z 0, from which the modied form of the Nyquist stability criterion may be expressed as = P+ P 2
(28)
That is to say, if the open-loop transfer functions frequency response is plotted on polar coordinates and is found to encircle the Cartesian point (1, 0) in a counterclockwise direction by an angle of exactly (P P /2) radians. In this case, where P is the number of open-loop transfer function poles on the right half plane and P is the number of open-loop transfer function poles on the imaginary axis, the closed-loop system is stable. For the illustration of Nyquist stability criterion, lets take an example of a system having a open-loop transfer function G(s)H(s) of
G ( s )H ( s ) = L ( s ) 30 (s + 1 )(s + 2 )(s + 3 ) 30 = 3 s + 6s2 + 11s + 6 = (29)
The Nyquist plot of L(s) of Eq. (29) can be obtained in a number of ways (e.g., polar plots) by substituting s j. By calculating the real and imaginary components of L(j), the Ny-
350
4 3 2
Imag axis
NG / DG and H(s) NH / DH. Equation (30) then reduces to kNG DH C (s ) = R (s ) DG DH + kNG NH (31)
1 0 1 2 3 4 2 1 0 1 2 3 4 5
Real axis
Figure 9. A typical Nyquist plot. This is the plot of a third-order system and hence it traces three quadrants. The curve cuts the real axis on the negative side. If the gain is increased sufciently, the curve will encircle the 1 point hence indicating instability. This means that at least one of the roots of the characteristic equation, poles of the closed loop system, will be on the right half of the s-plane.
Equation (31) reveals that the zeros of the closed-loop system are independent of k and correspond to the zeros of G(s) and the poles of H(s). However, as k 0, there is pole/zero cancellation of the H(s) pole term DH, and as k there is pole/ zero cancellation of the G(s) zero term NG. The location of the closed-loop poles, or the roots of the characteristic equation, is the subject of the remaining discussion. The root locus is a plot in the s-plane of the poles of the closed-loop transfer function as the parameter k varies from 0 to . From Eq. (1), it should be clear that these poles correspond to the zeros of the 1 kG(s)H(s) denominator. The root locus is therefore a plot in the s-plane of Eq. (31) kG(s )H (s ) = 1 Equation (32) may be expressed in its polar form as
u i = 1 Ai v i = 1 Bi v
(32)
1 k
(33a) (33b)
quist plot of Eq. (29) may be plotted as shown in Fig. 9. It can be seen that the contour does not encircle the point (1, 0), so the system is stable. Further examples of Nyquist plots are given in Fig. 10. From the Nyquist plots, it is possible to nd phase and gain margins of the system. The gain margin is dened to be the amount of gain that can be allowed before the system becomes unstable, and the phase margin is the angle at unity gain. It is also possible to nd the phase crossover frequency c and the gain crossover frequency g either from the graph or mathematically. From the graph the phase and gain margins of the preceding example are 25 and 6 dB, respectively. It is also possible to design the system to obtain desired responses by varying the margins. THE ROOT LOCUS Often engineers want to see how changes in some parameters such as loop gain will affect the performance and the stability of a system. The root locus is a widely practiced method in this regard. It gives information about how the closed-loop poles of the system vary as the parameter in question is changed. This is particularly useful in determining the range the parameter may cover while keeping the system stable. As discussed previously, the relative stability of a system is largely determined by the location of poles, which the root locus approach clearly conrms. Formulation of Root Locus Figure 11 shows a block diagram of a system with a variable loop gain. It has a closed-loop transfer function given by kG(s ) C (s ) = R (s ) 1 + kG(s )H (s ) (30)
i
i= 1 i= 1
i = ( 1 + 2n )
where Ai is the distance between a point on the root locus and the ith zero of the loop transfer function L(s); Bi is the distance between a point on the root locus and the ith pole; i is the angle about the ith zero from the positive real axis to a point on the root locus; i is the angle about the ith pole from the positive real axis to a point on the root locus; u is the number of zeros in the loop transfer function; v is the number of poles; k is the loop gain; and n is any integer. It should be evident that Eq. (33b) determines whether a point is on the root locus, and Eq. (33a) just determines the value of k corresponding to that point. It is always possible to solve Eq. (32) for an array of values for k, but that would be too time consuming. Evans developed a set of rules for sketching the root locus, reducing the problem to a few simple calculations. Figure 12 shows the root locus of a typical system with an open-loop transfer function given by
T (s ) =
k s(s + 2 )(s + 5 ) k = 3 s + 7s2 + 10s
(34)
Further examples of root locus are given in Fig. 10. The Root Locus Method of Evans A number of rules may be applied to sketch the root locus. The root locus starts with k 0 at the poles of G(s)H(s) and nishes with k at the zeros of G(s)H(s). This can be seen from the magnitude condition |G(s )H (s )| = 1 k (35)
In order to investigate how G(s) and H(s) contribute poles and zeros to the closed loop system, it is informative to let G(s)
351
Transfer function K (s 1 + 1)
Imag axis
Nyquist plot
Root locus
Imag axis
1
1
0 Real axis Real axis
Imag axis
Imag axis
K (s 1 + 1)(s 2+ 1)
1
2
1
1
Imag axis
Imag axis
K (s 1 + 1)(s 2+ 1) (s 3 + 1)
1 0
3
1
2
1
1
Imag axis
Imag axis
K s(s 1 + 1)
1
1
Imag axis
Imag axis
K s(s 1 + 1)(s 2 + 1)
1 1
2 1
Real axis
0 Real axis
Figure 10. Examples of Nyquist and root locus plots. The stability of control systems can be determined by various methods as exemplied here. In obtaining these examples, a popular software called MATLAB was used.
As k approaches zero, the magnitude of the loop transfer function becomes innite, corresponding to a pole. For k becoming innitely large, the loop transfer function becomes innitesimally small, corresponding to a zero. Actually, inspection of Eq. (31) reveals that the poles of H(s) and zeros of G(s) never actually appear as poles of the
R(s) + k G(s) C(s)
closed-loop transfer function because of pole/zero cancellation for k 0 and k . This point should be kept in mind when designing systems with the root locus; for very high and very low gains, there may be signicant pole/zero cancellation. The zeros of G(s)H(s) include both the nite zeros found in the denominator terms and the innite zeros at s caused by a denominator of higher order than the numerator. The result is that there are always the same number poles and zeros and that the root locus will always have enough zeros at which to terminate, be they nite or innite. The root locus plot is symmetrical about the real axis. All physically realizable transfer functions have real coefcients. Transfer functions with real coefcients always produce complex poles and zeros in conjugate pairs, which means that if Eq. (32) locates a point s j as being on the root locus, then s j must also be on the root locus.
H(s)
Figure 11. Block diagram of a closed-loop system with variable k. In many systems, one of the parameters of the system is varied to achieve the desired response. In this case, the variation of k in the forward path will relocate the roots of the characteristic equation on the s-plane. The suitable locations of the roots lead to appropriate system design.
352
10 8 6 4
Imag axis
shown in Fig. 13, plotted for G(s)H(s) (s 1)(s 3)(s2 4s 8) . All breakaway points s must satisfy the following conditions:
2 0 2 4 6 8
d G ( s )H ( s ) = 0 ds arg[G(s )H (s )] = (1 + 2n )
(37a) (37b)
10 10
0 2 Real axis
10
For real values of s, Eq. (37a) implies Eq. (37b), but for complex s, there is no such implication. If there are n poles involved in a breakaway point, then there are always n branches entering and n branches leaving. The angle between the entering and leaving branches is given by = n (38)
Figure 12. The root locus of a system with a characteristic equation 1 ks(s 2)(s 5). This is a typical example of root locus. Roots start from poles of the characteristic equation when k 0 and approaches zeros as k . In this example, all three zeros are at . At some value of k, the root loci crosses the imaginary axis to the RHP, thus indicating unstable conditions. With the aid of root locus a suitable value of k can be determined to locate the roots at the desired points on the s-plane.
A point on the real axis is on the root locus if and only if there is an odd number of poles and zeros on the righthand side of it. The angular contribution to Eq. (33b) of a pole or zero on the real axis to the left of a point on the real axis is always zero. In the case of a complex conjugate pair of poles or zeros, if one member of the pair contributes an angle of then the other will contribute 2 . The total contribution from the pair is then 2 0 rad. Similarly, any pole or zero to the right of a point on the real axis will end up contributing rad to the angular equation. Consequently, an odd number of poles or zeros is required to satisfy Eq. (33). Branches terminating at innite zeros approach an asymptotic line that is described by
Angle of departure from a complex root. The angle at which a branch leaves a pole or arrives at a zero may be determined by assuming a point s innitesimally close to the singularity. Because s is innitesimally close to the pole or zero, the angular contributions to the angle Eq. (33b) from all the other poles and zeros are known, and the only unknown quantity is the contribution from the pole or zero in question. This angle, the angle of departure, is easily found by Eq. (33b). Imaginary axis intersection. This is a very important point to know because it reveals the value of k that will result in a marginally stable closed loop system. Forming a Routh table with the unknown k as a parameter and then solving for k to give a row of zeros in the table is one of the most common methods. For higher-order systems, the table may become too cumbersome. In such a situation, it may be more desirable to solve Eq. (32) as G( j )H ( j ) = 1 k (39)
n =
( 2n + 1 ) vu v p A = i = 1 i vu
(36a)
u i= 1 zi
6 4
(36b)
Imag axis
2 0 2 4 6 6 4 2 0 Real axis 2 4 6
where n is the angle between the positive real axis and the nth asymptote; (A, 0) is the point at which the asymptotes intersect the real axis; pi is the ith open-loop transfer function pole location; zi is the ith open-loop transfer function zero location; v is the number of openloop transfer function poles; and u is the number of zeros. Applying these rules will provide a reasonable sketch of the root locus. There are several signicant points on the sketch that may be of interest to locate in terms of their precise location and the value of k required to achieve them. Breakaway points. These are the points at which multiple roots meet and then diverge. This most commonly occurs on the real axis, but it may occur anywhere, as
Figure 13. Breakaway points. The root locus plotted for G(s)H(s) (s 1)(s 3)(s2 4s 8) shows typical breakaway points at which multiple roots meet and then diverge. The breakaways generally occur on the real axis, but they may occur anywhere. In this example, a breakaway has happened on the real axis where as two others have taken place on the s-plane, in which the corners of the two root loci have met.
353
R(s)
1 (s + 1)(s + k)
C(s)
ing that there is a known solution x1, then the equation may be reduced to a rst-order linear equation by letting x = x1 + 1 u (44)
Figure 14. A representation of a variable component other than the root gain. In this case, one of the open-loop poles is the variable. This can be handled by forming an equivalent loop transfer function to construct the root locus.
Taking the derivative with respect to t results in dx dx1 1 du = 2 dt dt u dt Substituting Eqs. (44) and (45) into Eq. (41) gives (45)
In this case, the root locus is solved for an imaginary axis intercept, s j. System Parameters Other Than Gain In many situations the loop gain is not the parameter that is variable. It may be that the position of the open loop poles is the variable, as in Fig. 14. The way to handle this is to form an equivalent loop transfer function for the purpose of constructing the root locus as 1 C (s ) = R (s ) (s + 1 )(s + k ) + 1 (40)
dx1 1 du 1 2 + P(t ) x1 + dt u dt u
But since it is known that
= Q(t ) x2 1 +
1 2x 1 + 2 u u
+ R(t ) (46)
dx1 + P(t )x1 = Q(t )x2 1 + R(t ) dt Eq. (46) reduces to du + [2x1 (t )Q(t ) P(t )]u = Q(t ) dt
(47)
(48)
After some algebraic manipulation, Eq. (40) may be expressed in the form of
C (s ) = R (s )
1 s2 + s + 1 s+1 1+k 2 s +s+1 G (s ) = 1 + kG(s )H (s )
which is a linear rst-order differential equation and is simple to solve particularly if P(t), Q(t), and R(t) are constants, as would the case if Eq. (41) were to describe a time-invariant system. The Matrix Riccati Differential Equation Consider the dynamic system given by the state-space description as x (t ) = Fx(t ) + Gu(t ) y(t ) = Hx(t ) (49)
(41)
where G(s) 1/(s2 s 1) and H(s) s 1. The root locus may now be constructed in the normal manner. It may also occur that there are two parameters that are variable. Then the root locus may be represented by a set of contours or a surface plots. THE RICCATI EQUATION Many matrix equations naturally arise in linear control system theory. One of the most applied equations is the Riccati equation, which can be expressed as dx + P(t )x = Q(t )x2 + R(t ) dt (42)
where x(t) is the state matrix (n by 1), u(t) is the control matrix (q by 1), and y(t) is the matrix (p by 1) of output variables to be controlled and F, G, and H are matrices of appropriate dimensions that characterize the system. In the optimal control applications, the object of optimal control is to nd u(t) over an interval t [t1, t2] such that some cost function is optimized. One of the popularly used cost function is the quadratic cost function that can be generalized by F (t1 , t2 , T ) =
t2 t1
[y (t )y(t ) + u (t )u(t )] dt + x (t2 )Tx(t2 ) (50)
The equation was rst developed and applied by Count Riccati and Jacopo Francesco in the early eighteenth century. In recent times, the Riccati equation nds wide application, particularly in the area of optimal control and ltering. In these applications, the matrix Riccati equation depicts a system of Riccati equations given by (t ) + X(t )A(t ) D(t )X(t ) B(t )X(t ) + C(t ) = 0 X Solution of the Riccati Equation A closed form solution to Eq. (42) cannot be guaranteed depending of the functions P(t), Q(t), and R(t). However, assum(43)
where T is a constant, real, symmetric (T T) and nonnegative denite (T 0) matrix. It can be proven that there exists a unique optimal control for nite t2 t1 0, which has the form of u(t ) = G P(t1 , t2 , T )x(t ) where P(t1, t2, T) can be described by (t ) + P(t )F + F P(t ) P(t )GG P(t ) + H H = 0 P with the terminal condition that P(t2) T. (52) (51)
354
Correlation of Eq. (52) together with Eq. (43) reveals that the optimum control problem does indeed reduce to the problem of solving the Riccati matrix equation with constant, real A, B, and C matrices and D A. Furthermore, because any matrix UU is a symmetric, nonnegative denite matrix for U with real elements, it follows that B and C in are symmetric and nonnegative denite matrices, which are necessary conditions for solutions. Solution of the Riccati Matrix Differential Equation The following equation gives the form of the Riccati equation of interest applied in optimal control problems: (t ) + X(t )A A X(t ) X(t )BX(t ) + C = 0 X (53)
which corresponds to Eq. (55). Applying Eq. (56), the nal solution to Eq. (57) is found to be
x(t ) =
(0.3534 + 0.1464T )e1.414t + (0.3534 + 0.8536T )e1.414t (0.8536 0.3536T )e1.414t + (0.1464 + 0.3536Te 1.414t (62)
As expected, x(t2) x(0) T. Riccati Algebraic Equation and Innite Horizon In regulating control systems, it is not convenient to restrict control to a nite time period, t [t1, t2]. Even though it is possible to let t1 approach negative innity, it is customary to let t2 approach innity. Assuming that the system is time invariant [i.e., the matrices A, B and C in Eq. (53) are constant], the two intervals yield the same result. Suppose that (A, B) are stabilizable. Then there must exist some stabilizing control that is not necessarily optimal. The resulting cost function associated with this stabilizing control will then dominate the optimal cost function, and it must be nite. Consequently, the solution of the Riccati equation with innite horizon must be bounded. The solution X(t), as t approaches innity, will either approach a constant value or become periodic, depending on the system and the value of T chosen. The value of X(t) for t in the case where it is constant may be found by substituting (t) 0 into Eq. (53) to give the Riccati algebraic equation. X XA + A X XBX + C = 0 (63)
with B and C matrices being symmetric and nonnegative denite. The solution of Eq. (53) may be found by setting up the linear Hamiltonian matrix differential system as (t ) U A (t ) = C V B A U(t ) V(t ) (54)
where V(t2) TU(t2), and T is as dened in Eq. (50). Then if the solution of Eq. (54) may be found by using Eq. (53) U(t ) w11 = V(t ) w21 w12 w22 U(t2 ) V(t2 ) (55)
and w11 w12T is invertible, then the solution of Eq. (53) is given by X(t ) = (w21 + w22 T )(w11 + w12 T ) 1 As an example, consider dx + 2x x 2 + 1 = 0 dt (57) (56)
If A, B, and C are 1 by 1 matrices (i.e., there are just scalar numbers), the solution of Eq. (63) is a trivial task. The general solution is a little more complex, involving Jordan chains. A Jordan chain of the matrix H is a set of vectors, x1, x2, . . ., xn such that Hx1 = x1 Hxj = xj + xj1 (64)
which corresponds to A [1], B [1], and C [1] in Eq. (53). The associated linear Hamiltonian matrix is then given by u 1 = v 1 which has the solution 1 u 1 e1.414t =a e 1.414t + b 0.414 v 2.414 (59) 1 1 u v (58)
where is an eigenvalue of H and x1 is the associated eigenvector. Equation (63) has a solution for X if and only if there exists a set of Jordan chains of
H=
A C
B A
given by x1, x2, . . ., xn. If we let xi = and U = [u1 un ] and ui vi
For some arbitrary constants a and b, the solution becomes u (0 ) = a + b v(0 ) = 2.414a 0.414b Also, Eq. (59) may be expressed as (60)
u 0.8536e1.414t + 0.1464e 1.414t = v 0.3534e1.414t + 0.3534e 1.414t 0.3536e1.414t + 0.3536e 1.414t 0.1464e1.414t + 0.8536e1.414t u (0 ) v (0 )
(61)
V = [v 1 v n ] then the solutions of Eq. (63) are given by X VU1.
355
We can verify this by using it to derive the quadratic formula. Let ax2 + bx + c = 0 (65)
values, hence achieved an equilibrium state. If the system is in the equilibrium state, that is, no states are varying in time, the equilibrium state may be described by e = f (xe ) = 0 X (72)
where A b /2, C c, and B a, to form the Hamiltonian matrix, b 2 c a

b 2
In order to seek solutions to Eq. (71), Lyapunov introduced a continuously differentiable scalar function V(x) with the following properties. 1. Positive denite if V(0) 0 for all t t0 and V(x) 0 for all t t0. 2. Positive semidenite if V(x) 0 for all x. 3. Negative denite or negative semidenite if V(x) positive denite or positive semidenite. These conditions ensure that V is positive if any state is different from zero but equals zero when the state is zero. These conditions ensure that V is a smooth function and the trajectory does not expand indenitely but rather is drawn to the origin. This can be explained with the aid of Fig. 15. Lyapunov stability states that an equilibrium state xe of a dynamic system is stable if for every 0, there exists a 0, where depends only on , such that (x0 xe) results (x(t; x0) xe) for all t t0. This statement of stability in the sense of Lyapunov indicates that if an equilibrium state is stable, the trajectory will remain within a given neighborhood of the equilibrium point if the initial state is close to the equilibrium point. Likewise, an equilibrium state xe of a dynamic system is unstable if there exists an , such that a corresponding value cannot be found. From the preceding explanations, asymptotic stability may be dened. An equilibrium state xe of a dynamic system is asymptotically stable if a. it is Lyapunov stable. b. there is a number a 0 such that every motion starting within a in the neighborhood of xe converges to xe as t .
H=
(66)
which has two eigenvalues given by
b ac 2 b2 4ac = 2
(67)
The eigenvector associated with the eigenvalue of 0 is 1 w = b (68) 2a a The solution of Eq. (65) is then found as
x=
b 2a 2a b b2 4ac = 2a
(69)
which is the familiar quadratic equation. For 0, there exists one Jordan chain, 1 x1 = b 2a 0 x2 = 1 a giving one solution, x b /2a.
(70)
LYAPUNOV STABILITY Lyapunov studied the question of motion, basing his theories on the nonlinear differential equations of motion. His equations for linear motion are equivalent to Rouths criterion. The Lyapunovs theorem determines the stability in the small region about an equilibrium point. The stability in the large may be determined by the global stability techniques. In this article, stability in the small will be introduced, and some references will be made to global stability methods. Lyapunov considers the stability of general systems described by ordinary differential equations expressed in the state-variable form as = f (x ) X (71)
Unstable
x=0 Asymptotically stable
xo
Stable
Figure 15. Lyapunov stability criteria. The equilibrium state xe is stable if for every 0, there exists a 0, where depends only on , such that the trajectory remains within a given neighborhood of the equilibrium point. Likewise, an equilibrium state xe of a dynamic system is unstable if there exists an , such that a corresponding value cannot be found.
Suppose that in a system all states have settled to constant
356
A simplied version of Lyapunovs rst theorem of stability, Ax is a valid Eq. (71), may be explained. Suppose that X model about the equilibrium point xe and the roots of the characteristic equation may be expressed in matrix form as sI A = 0 (73)
where Q = (FT P + PF ) (78)
All eigenvalues of A have negative real parts if for any symmetric positive denite matrix N, the Lyapunov equation AT M + AM = N (74)
has a symmetric positive denite solution. For example, suppose that Eq. (71) can be expressed as = Ax(t ) X (75)
Its solution is x(t) eAt x(0). If the eigenvalues of A are k1, k2, and k3, then every component of x(t) is a linear combination of ek1t, ek2t, and ek3t. These time functions will approach zero as t if and only if ki has negative real parts. Thus it can be concluded that any nonzero initial state will approach to zero only if A is stable. This can be generalized as follows. 1. If the characteristic values all have negative real parts, the equilibrium point is asymptotically stable. 2. If at least one of the values has a positive real part, the equilibrium point is unstable. 3. If one or more characteristic values have zero real parts, with all other values having negative real parts, the system stability cannot be determined with the current method. Testing the stability by considering the linear part is referred to as Lyapunovs rst or indirect method. Using the Lyapunov function directly on the nonlinear equations themselves is called the second or direct method. The argument is as follows. Lyapunov showed that in the case of nonlinear systems Eq. (71) may be extended as = Fx + g(x ) X (76)
For any positive Q, the solution of P of the Lyapunov equation is positive if and only if all the characteristic roots of F have negative real parts. That is, if a system matrix F is given, it is possible to select a positive Q, solve the Lyapunov equation in n(n 1)/2 unknowns, and test to see if P is positive by looking at the determinants of the n principal minors. From this, the stability may be determined from the equations without either solving them or nding the characteristic roots. The study of nonlinear systems is vast, here only the basic principles of the methods have been discussed. Also, Lyapunov methods are applied in many diverse areas of control engineering; therefore, it is impossible to cover them all here. Interested readers should refer to the reading list given at the end of this article.
ROBUST STABILITY Consider a linear time-invariant feedback system with a plant transfer function G(s) and a compensator with Gc(s) cascaded as shown in Fig. 16. In many applications, the plant model will not accurately represent the actual physical system because of (1) unmodeled dynamics and time delays, (2) changes in equilibrium points, (3) nonlinear characteristics of the plant, (4) noise and other disturbance inputs, and (5) parameter drift. The aim of a robust system is to assure that performance is maintained in spite of model inaccuracies and parameter changes. The closed-loop transfer function of the system in Fig. 16 may be written as M (s ) = G ( s )G c ( s ) C (s ) = R (s ) 1 + G ( s )G c ( s ) (79)
The sensitivity of the system to changes in G(s) or Gc(s) can be expressed by the sensitivity function S= 1 M/M = G/G 1 + G ( s )G c ( s ) (80)
where g(x) contains all the higher powers of x. If g(x) goes to zero faster than x does, then the system is stable if all the roots of F are strictly inside the left half plane and will be unstable if at least one root is in the right-half plane. For the system with roots in the left half plane and on the imaginary axis, the stability depends on the terms in the function g. For global stability analysis of linear constant systems, quadratic functions are often used. Consider the function V xT Px, where P is a symmetric positive matrix. The V is the sum of squares of xi. In general, if P is positive, we can nd a matrix T such that P TTT and V zi, where z Tx. For the derivative of V the chain rule can be used as
R(s) +
C(s) G c( s ) G(s)
H(s)
Figure 16. Block diagram of a closed-loop control system for robust stability analysis. In the many mathematical representation of systems, a full account of all affected parameters may not be taken into consideration because of unmodeled dynamics and time delays. Also, during the operations, the equilibrium points may change, parameters may drift, and noise and disturbances may become signicant. The aim of a robust system is to assure that performance is maintained in spite of model inaccuracies and parameter changes.
V = dxT Px/dt = xT Px + xT Px = xT (FT P + PF )x = xT Qx (77)
STABILITY THEORY: INCLUDING SATURATION EFFECTS
357
G(s) H(s)-plane
Imaginary
EXPONENTIAL STABILITY The study of exponential signals (eat) is important in linear system analysis. They contain a variety of signals such as constants, sinusoids, or exponentially decaying or increasing sinusoids. A system with an n-dimensional state model is said to be an exponential system if its state-transition matrix (t, ) can be written in matrix exponential form (t , ) = e
(t , )
0 1
Real
(82)
Figure 17. An example of a closed-loop system resulting from parameter drifts in Nyquist plots. This diagram indicates that because of uncertainties in modeling and changes in parameters the gain and phase margins may be altered. These alterations may lead to unstable conditions if these margins are close to critical values.
As can be seen from Eq. (80), the sensitivity function has the same characteristic equation [1 G(s)Gc(s)] as the closed-loop transfer function M(s). For sensitivity to be small, it is necessary to have a high value for loop gain L(s) G(s) Gc(s). The high gain is obtained at high frequencies of L( j). But as we know, high gain could cause instability and poor responsiveness of M(s). Now, the design problem becomes a matter of selecting Gc(s) such that the closed-loop sensitivity is small, and the closed-loop transfer function has a wide bandwidth. At the same time, the desired gain and phase margins must be achieved. The stability of the control system depends on the openloop transfer function L(s) G(s)Gc(s). Because of the uncertainties outlined here, the transfer function may be written as L ( s ) = G c ( s )[G ( s ) + G ( s )] (81)
where (t, ) is an n n matrix function of t and . A sufcient condition for the system to be uniformly exponentially stable is that the eigenvalues of the of the n n matrix (1/ t) (t, ) be bounded as functions of t and have real parts v for all t and for some v 0 and . In many applications the stochastic components and random noises are included in the dynamical system models. The stochastic aspects of the model are used to capture the uncertainty about the environment in which the system is operating. The analysis and control of such systems involve evaluating the stability properties of the random dynamical systems. The stability of the system can be studied by stochastic stability approaches.
BIBLIOGRAPHY
1. W. S. Levine, The Control Handbook, Boca Raton, FL: CRC Press, 1996. 2. K. Ogata, Modern Control Engineering, 3rd ed., Upper Saddle River, NJ: Prentice-Hall, 1997. 3. B. J. Kuo, Automatic Control Systems, 6th ed., Englewood Cliffs, NJ: Prentice-Hall, 1991. 4. K. Watanabe, Adaptive Estimation and Control: Partitioning Approach, Hertfordshire, UK: Prentice-Hall, 1992.
HALIT EREN ALEXANDER LOWE

Curtin University of Technology, Australia
For system stability, Nyquists stability condition must always be satised. That is the 1 point must not be encircled by the L( j) under any circumstances. An example of uncertainty in a typical Nyquist plot resulting from G(s) is illustrated in Fig. 17. To guarantee stability, a safe gain and phase margin must be ensured. Many methods are available to deal with the robustness of the system including classical methods linked to the use various compensators and PID controllers. The H technique is one method that nds extensive application in robust control design and analysis. In general, feedback reduces the effect of disturbances and moderate modeling errors or parameter changes in the control system. In the presence of disturbances and sensor noises, systems are designed such that they keep the tracking errors and outputs small for disturbance inputs. In order to achieve this, the sensitivity to modeling errors and sensor noise must be made small, thus making the system robust. In this case, the plant output will follow any reference input asymptotically even if there are variations in the parameters of disturbance and noise. Briey, it can be said that the system is more robust if it can tolerate larger perturbations in its parameters.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Stability Theory, Nonlinear Standard Article Hassan K. Khalil1 and Steven W. Shaw2 1Michigan State University, East Lansing, MI 2Michigan State University, East Lansing, MI Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1042 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (270K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Steady-State Behavior Stability of Equilibrium Points Bifurcation Theory Further Reading About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
390
STABILITY THEORY, NONLINEAR
as time becomes large. This article deals with the latter notion of stability. An asymptotically stable response is the basis of a steady-state behavior whereby other responses asymptotically approach the steady-state response. A steady-state response can be as simple as a constant (time-invariant) response, or it can be a periodic one. These, as well as other more complicated steady-state responses, are described in the rst section. In our study of dynamical systems, we model the system by a nite number of coupled rst-order ordinary differential equations
x 1 = f 1 (t , x1 , . . ., xn ) x 2 = f 2 (t , x1 , . . ., xn ) . . . . . .
x n = f n (t , x1 , . . ., xn )
where x i denotes the derivative of xi with respect to the time variable t. We call the variables x1, x2, . . ., xn the state variables. They represent the memory that the dynamical system has of its past. They are usually chosen as physical variables that represent the energy-storing elements. For example, in an RLC electrical circuit, the state variables could be voltages across capacitors and currents through inductors, while in a spring-mass-damper mechanical system, the state variables could be positions and velocities of moving masses. We usually use vector notation to write the above equations in a compact form. Dene x1 f 1 (t , x ) x2 f 2 (t , x ) x = . , f (t , x ) = . . . . . xn f n (t , x ) and rewrite the n rst-order differential equations as an ndimensional rst-order vector differential equation x = f (t , x ) which we call the state equation, and x is referred to as the state. The response of the system due to initial state x0 at initial time t0 is the solution of the differential equation x f (t, x) subject to the initial condition x(t0) x0. This solution is unique provided the function f is locally of Lipschitz character in the domain of interest D; that is, every point in D has a neighborhood D0 and a nonnegative constant L such that the Lipschitz condition f (t , x ) f (t , y ) L x y is satised for all x and y in D0. x is a measure of the length of the vector x in the n-dimensional state space (the space of n n-dimensional real vectors), and is dened by x2 i xi2. The locus of the solution x(t) in the state space is usually referred to as a trajectory. A special case of the state equation arises when the function f does not depend explicitly on t; that is, x f(x), in which case the system is said to be autonomous; otherwise it is said to be nonautonomous. In the rst section, we introduce the most common forms of steady-state responses, namely, equilibrium points (constant

Stability analysis plays a central role in systems engineering. There are two general notions of stability that arise in the study of dynamical systems: input-output stability and stability of a particular response or a set of responses. In the rst notion, the system is viewed as a map from the space of input signals to the space of output signals. It is said to be stable if an input that is well behaved in some sense (e.g., signals with nite amplitude or energy) will always produce an output that is well behaved in the same sense. In the second notion, the input to the system is xed so that the response of the system over time is determined solely by the initial state of the system at the initial time. Such a response is said to be stable if other responses starting at nearby initial states stay nearby; otherwise it is unstable. It is said to be asymptotically stable if all responses starting at nearby initial states not only stay nearby, but also approach this particular response
391
solutions), limit cycles (periodic solutions), tori, and strange attractors (chaos). The most effective stability analysis tools are available for the case of equilibrium points, which we treat in the second section. We make the notions of stability, instability, and asymptotic stability, introduced earlier, precise and present Lyapunovs method and illustrate it by examples. We treat the special case of linear systems and show how stability of an equilibrium point of a nonlinear system can be studied by linearizing the system about this point. We use the center-manifold theorem to treat the critical case when linearization fails and end the section with an extension of Lyapunovs method to nonautonomous systems. An important issue in the analysis of dynamical systems is the effect of changes in the systems parameters on its behavior. Smooth changes in the systems behavior are usually studied via sensitivity analysis tools, but when the change in the parameters results in a change in the qualitative behavior of the system, like the disappearance of an equilibrium point or a limit cycle, it is studied via bifurcation theory, which we introduce in the nal section.
STEADY-STATE BEHAVIOR The steady-state behavior of a system is described by the asymptotic nature of solutions as time becomes large. For nonlinear systems, this will depend on the system as well as on the initial conditions provided. The possible types of steadystate behavior are more varied than one might think; they include the well-known constant time behavior (asymptotically stable equilibria) and periodic time behavior (asymptotically stable limit cycles), as well as more complicated behaviors, such as multiperiodic behavior (asymptotically stable tori) and chaos (strange attractors). We begin with some denitions that are general enough to capture this range of possibilities, and then provide some examples. The steady-state behavior of a system takes place on a subset of the state space called an attractor. The key ingredients for dening an attractor A are the following: (1) If a solution is started in A , it never leaves A . That is, A is an invariant set, dened by saying that for each x(0) x0 A , x(t) A t. (2) Solutions started sufciently close to A will approach A as t . That is, A is locally attractive. (3) A feature of an attractor A is that it contains a solution that comes arbitrarily close to every point in A at some time. This implies that A is minimal in the sense that there are no subsets of A that satisfy conditions (1) and (2). The domain of attraction (or region, or basin, of attraction) for an attractor A is dened to be the set of initial conditions in the state space that are asymptotic to A as t . It can (at least formally) be constructed by considering a neighborhood UA of A that is used in proving its asymptotic stability, and taking t0x(t) x(0) UA . This simply starts solutions in a neighborhood in which one knows they will approach A in forward time and runs time backward. In this way all solutions that will approach A are collected. Except in simple problems it is impossible to determine the domain of attraction, although parts of it can often be estimated using Lyapunov methods, as described in the next section. It is important to realize that a nonlinear system may possess multiple attractors of various types. The domains of attraction for different attractors must, of course, be distinct. Typically, differ-
ent domains of attraction are separated by solutions that are asymptotic to saddle-type invariant sets (that is, invariant sets that are generally unstable but that have some stable directions). The most well-known type of attractor is an asymptotically stable equilibrium point of an autonomous system, x f (x). An equilibrium point for this system is a point x such that f (x) 0, representing a constant, time-invariant, steady-state response. Obviously the solution for the system initiated at such a point is simply x(t) x, and x is invariant and minimal. The matter of its stability is considered in detail in the following section. The next simplest type of attractor is represented by a stable closed trajectory in the state space, called a limit cycle, . A solution x(t) that lies on is necessarily periodic, since by starting from a point on , the solution takes time T to traverse the trajectory and return to the starting point, after which the motion continually repeats. Thus, x(t T) x(t) for all points on , where T is the period of the limit cycle. The frequency content of a limit cycle is composed of a fundamental harmonic plus multiples of the fundamental. Limit cycles can arise in autonomous or nonautonomous systems, examples of which follow. A simple autonomous system that possesses a limit cycle is the following: r = r(1 r 2 ), = (1)
which is expressed in polar coordinate form. This two-dimensional system has an invariant set (r, ): r 1, [0, 2), a circle, which attracts all solutions except for the unstable equilibrium at r 0, and on which solutions wind around with constant speed leading to a period T 2 / . Furthermore, since all points on the circle are visited by every solution started on it, it satises all the conditions for an attractor. Such a closed trajectory can, of course, exist in higher-order systems as well. Another example of an asymptotically stable periodic attractor is offered by the simple linear equation x = x + cos (t ) (2)
which has a known steady-state solution of the form xss(t) A cos(t ), which is obviously asymptotically stable for 0 since the transient decays to zero. In order to view this steady state as a limit cycle, one considers the extended state space, constructed by supplementing the preceding equation and replacing t by in Eq. with the trivial equation (2). (This renders the system autonomous.) The extended state space, (x, ), is a cylinder, shown in Fig. 1, on which the
=
Figure 1. Limit cycle in a cylindrical state space.
392
steady-state solution is a closed trajectory with period T 2 / . While both of these examples have circular limit cycles around which solutions move at constant speed, this, of course, is not generally the case. A steady-state response that is composed of multiple, noncommensurate frequencies corresponds to an asymptotically stable torus in the state space. Roughly speaking, each angular coordinate on the torus has an associated frequency. (Note that a limit cycle can be viewed as a one-dimensional torus.) A simple example of an asymptotically stable torus is given by a state model that is a simple generalization of that used for the rst limit-cycle example, r = r(1 r 2 ), 1 = 1 , 2 = 2 (3)
Tori can exist in Euclidean spaces of dimension three and higher. It is easy to show that the r 0 solution is unstable and the solution with r 1, 1 1t 10, 2 2t 20 is asymptotically stable. Here 1 and 2 represent the frequencies of the steady-state response. Note that if 1 / 2 is rational, then every solution on the torus is periodic and closed, representing a one-parameter family of periodic responses. In contrast, when 1 / 2 is irrational, the torus will be covered by a single solution (a dense solution) for any initial condition. Thus, the torus satises condition (3) for an attractor only in the incommensurable case. Also, note that in more general examples, the various rotation speeds are not constant on the torus, and the tori can be highly distorted. The response of a torus is generally composed of a set of discrete frequencies that include 1 and 2, as well as various linear combinations of them that result from nonlinear interactions. Also, one can encounter tori with more than two frequencies. A chaotic steady-state response corresponds to a complicated set in the state space known as a strange attractor. While chaos is observed in many simulations and experiments, it is virtually impossible to prove the existence of a strange attractor for a given system model. The essence of these difculties lies in the fact that there exist extremely complicated invariant sets in such systems, and it is not possible to prove that an asymptotically stable periodic solution does not exist nearby. However, these subtle issues fall outside the main topic of this article. The response of a strange attractor has a broadband frequency content, which is rather unexpected for a deterministic system. Note that the complexity of these attractors is related to their dimensionality. The simplest, an equilibrium, has dimension 0, the limit cycle has dimension 1, and a torus with N frequencies has dimension N. It is interesting to note that a chaotic attractor, if it exists, will have a noninteger, or fractal, dimension due to the rich structure of the invariant manifold on which it exists. The difculties associated with determining the stability of various types of invariant sets is similarly related to their dimensionality. For equilibria many analysis techniques exist, as described in detail in the following. Techniques also exist for limit cycles, tori, and chaos, but can rarely be applied without computational tools. STABILITY OF EQUILIBRIUM POINTS We consider the autonomous system x f (x), where the components of the n-dimensional vector f (x) are locally Lipschitz
functions of x, dened for all x in a domain D n that contains the origin x 0. Suppose the origin is an equilibrium point of x f (x); that is, f (0) 0. Our goal is to characterize and study the stability of the origin. There is no loss of generality in taking the equilibrium point at the origin, for any equilibrium point x 0 can be shifted to the origin via the change of variables y x x. The equilibrium point x 0 of x f (x) is stable, if for each 0, there is () 0 such that x(0) implies that x(t) , for all t 0. It is said to be asymptotically stable if it is stable and can be chosen such that x(0) implies that x(t) approaches the origin as t tends to innity. When the origin is asymptotically stable, the domain of attraction is dened as the set of all points x such that the solution of x f (x) that starts from x at time t 0 approaches the origin as t tends to . When the domain of attraction is the whole space n, we say that the origin is globally asymptotically stable. Lyapunovs Method In 1892, Lyapunov introduced a method to determine the stability of equilibrium points without solving the state equation. Let V(x) be a continuously differentiable scalar function dened in D. A function V(x) is said to be positive denite if V(0) 0 and V(x) 0 for every x 0. It is said to be positive semidenite if V(x) 0 for all x. A function V(x) is said to be negative denite or negative semidenite if V(x) is positive denite or positive semidenite, respectively. The derivative (x) of V along the trajectories of x f (x) is given by V n i1(V / xi)x i (V / x)f (x), where V / x is a row vector whose ith component is V / xi. Lyapunovs stability theorem states that the origin is stable if there is a continuously differentiable positive denite function (x) is negative semidenite, and it is asymptotiV(x) so that V (x) is negative denite. A function V(x) satiscally stable if V fying the conditions for stability is called a Lyapunov function. The surface V(x) c, for some c 0, is called a Lyapunov surface or a level surface. Using Lyapunov surfaces, Fig. 2 makes the theorem intuitively clear. It shows Lyapunov surfaces for decreasing constants c3 c2 c1 0. The condi 0 implies that V(x(t)) decreases along the trajectory tion V x(t). Therefore, when a trajectory crosses a Lyapunov surface V(x) c, it moves inside the set V(x) c and can never come 0, the trajectory moves from one Lyapuout again. When V nov surface to an inner Lyapunov surface with a smaller c. As c decreases, the Lyapunov surface V(x) c shrinks to the origin, showing that the trajectory approaches the origin as 0, we cannot be time progresses. If we only know that V
c3 c2 V(x) = c1
c1 < c2 < c3
Figure 2. Level surfaces of a Lyapunov function.
393
sure that the trajectory will approach the origin, but we can conclude that the origin is stable since the trajectory can be contained inside any neighborhood of the origin by requiring the initial state x(0) to lie inside a Lyapunov surface contained in that neighborhood. (x) is only negative semidenite, we may still be When V able to conclude asymptotic stability of the origin if we can (x) show that no solution can stay identically in the set V 0, other than the trivial solution x(t) 0. Under this condition, V(x(t)) must decrease toward 0, and consequently x(t) 0 as t . This extension of the basic theorem is usually referred to as the invariance principle. Lyapunov functions can be used to estimate the domain of attraction of an asymptotically stable origin, that is, to nd sets contained in the domain of attraction. If there is a Lyapunov function that satises the conditions of asymptotic stability over a domain D and if V(x) c is bounded and contained in D, then every trajectory starting in V(x) c remains in V(x) c and approaches the origin as t . Thus, V(x) c is an estimate of the domain of attraction. If the Lyapunov function V(x) is radially unbounded that is, x implies that V(x) , then any point x n can be included in the bounded set V(x) c. Therefore, the origin is globally asymptotically stable if there is a continuously differentiable, radially unbounded function V(x) such that for all x n, V(x) is posi (x) is either negative denite or negative tive denite and V semidenite but no solution can stay identically in the set (x) 0 other than the trivial solution x(t) 0. V Lyapunovs method is a very powerful tool for studying the stability of equilibrium points. However, there are two drawbacks to the method of which the reader should be aware. First, there is no systematic method for nding a Lyapunov function for a given system. In some cases, there are natural Lyapunov function candidates like energy functions in electrical or mechanical systems (see Example 2). In other cases, it is basically a matter of trial and error. Second, the conditions of the method are only sufcient; they are not necessary. Failure of a Lyapunov function candidate to satisfy the conditions for stability or asymptotic stability does not mean that the origin is not stable or asymptotically stable. Example 1. Consider the second-order system x 1 = x1 + x2 1 x2 , x 2 = x1 x2
by
(x ) = 2 (ax + bx )x V 2 1 2 1 + 2 (bx1 + dx2 )x = 2 (ax1 + bx2 )(x1 + x2 1 x2 ) + 2 (bx1 + dx2 )(x1 x2 )

Choosing b 0 yields (x ) = xT Qx + 2ax3 x V 1 2 where
Q=2
a 0.5d
0.5d d
The matrix Q is positive denite when ad d2 /4 0. Choose a d 1. Near the origin, the quadratic term xTQx domi3 (x) is negative dex2. Thus, V nates the fourth-order term 2x1 nite and the origin is asymptotically stable. Notice that the origin is not globally asymptotically stable since there are other equilibrium points. We can use V(x) to estimate the domain of attraction of the origin. The function V(x) is positive denite for all x. We need to determine a domain D about the (x) is negative denite and a set V(x) c origin where V D, which is bounded. We are interested in the largest set V(x) c that we can determine, that is, the largest value for the constant c. Using the inequalities xTQx min(Q)x2 3 2 (x) x2 x2 x1 2x1x2 x4, we see that V x2 and 2x1 4 x . Hence V(x) is negative denite in the domain x 1. We would like to choose a positive constant c such that V(x) c is a subset of this domain. Since xTPx min(P)x2 x2, we can choose c 1. Thus, the set V(x) 1 is an estimate of the domain of attraction. Example 2. A simple pendulum moving in a vertical plane can be modeled by the second-order differential equation = mg sin kl ml where l is the length of the rod, m is the mass of the bob, is the angle subtended by the rod and the vertical line through the pivot point, g is the acceleration due to gravity, and k is as the state a coefcient of friction. Taking x1 and x2 variables, we obtain the state equation x 1 = x2 , x 2 = a sin x1 bx2
The system has three equilibrium points at (0,0), (1,1), and (1,1). We want to study the stability of the origin (0,0). We take the quadratic function
2 T V (x ) = ax2 1 + 2bx1 x2 + dx2 = x Px
where
P=
a b
b d
as a Lyapunov-function candidate. For V(x) to be positive denite, we must have a 0, d 0, and ad b2 0. The derivative of V(x) along the trajectories of the system is given
where a g / l 0 and b k / m 0. The case b 0 is an idealized frictionless pendulum. To nd the equilibrium points, we set x 1 x 2 0 and solve for x1 and x2. The rst equation gives x2 0 and the second one gives sin x1 0. Thus, the equilibrium points are located at (n, 0), for n 0, 1, 2, . . . . From the physical description of the pendulum it is clear that the pendulum has only two equilibrium positions corresponding to the equilibrium points (0,0) and (,0). Other equilibrium points are repetitions of these two positions that correspond to the number of full swings the pendulum would make before it rests at one of the two equilibrium positions. Let us use Lyapunovs method to study the stability of the equilibrium point at the origin. As a Lyapunov-function candidate, we use the energy of the pendulum, which is de-
394
ned as the sum of its potential and kinetic energies, namely, V (x ) =

0 x1
Linear Systems The linear time-invariant system x Ax has an equilibrium point at the origin. The equilibrium point is isolated if and only if det(A) 0. Stability properties of the origin can be characterized by the locations of the eigenvalues of the matrix A. Recall from linear system theory that the solution of x Ax for a given initial state x(0) is given by x(t) exp(At)x(0) and that for any matrix A there is a nonsingular matrix P (possibly complex) that transforms A into its Jordan form; that is, P 1 AP = J = block diag[J1 , J2 , . . ., Jr ] where Ji is the Jordan block associated with the eigenvalue i of A. Therefore,
a sin y dy +
1 2
x2 2 = a (1 cos x1 ) +
1 2
x2 2
The reference of the potential energy is chosen such that V(0) 0. The function V(x) is positive denite over the domain 2 x1 2. The derivative of V(x) along the trajectories of the system is given by (x ) = ax V 1 sin x1 + x2 x 2 = bx2 2 (x) 0 and we can conWhen friction is neglected (b 0), V clude that the origin is stable. Moreover, V(x) is constant during the motion of the system. Since V(x) c forms a closed contour around x 0 for small c 0, we see that the trajectory will be conned to one such contour and will not approach the origin. Hence the origin is not asymptotically stable. On the other hand, in the case with friction (b 0), 2 (x) bx2 V 0 is negative semidenite and we can conclude (x) is only negative semthat the origin is stable. Notice that V (x) 0 for x2 0 idenite and not negative denite because V irrespective of the value of x1. Therefore, we cannot conclude asymptotic stability using Lyapunovs stability theorem. Here comes the role of the invariance principle. Consider the set (x) 0 x2 0. Suppose that a solution of the state V equation stays identically in this set. Then x2 (t ) 0 x 2 (t ) 0 sin x1 (t ) 0 Hence, on the segment x1 of the x2 0 line, the (x) 0 condition only at the origin system can maintain the V x 0. Noting that the solution is conned to a set V(x) c and, for sufciently small c, V(x) c x1 , we conclude that no solution can stay identically in the set V(x) c x2 0 other than the trivial solution x(t) 0. Hence, the origin is asymptotically stable. We can also estimate the domain of attraction by the set V(x) c where c minx1 V(x) 2a is chosen such V(x) c is a closed contour contained in the strip x1 . Example 3. Consider the system x 1 = x2 , x 2 = g1 (x1 ) g2 (x2 )
exp (At ) = P exp(Jt )P 1 =
mi
t k 1 exp (it )Rik
(4)
i= 1 k= 1
where mi is the order of the Jordan block associated with the eigenvalue i. If one of the eigenvalues of A is in the open right-half complex plane, the corresponding exponential term exp(it) in Eq. (4) will grow unbounded as t . Therefore, to guarantee stability we must restrict the eigenvalues to be in the closed left-half complex plane. But those eigenvalues on the imaginary axis (if any) could give rise to unbounded terms if the order of the associated Jordan block is higher than 1, due to the term tk1 in Eq. (4). Therefore, we must restrict eigenvalues on the imaginary axis to have Jordan blocks of order 1. For asymptotic stability of the origin, exp(At) must approach 0 as t . From Eq. (4), this is the case if and only if Re(i) 0, i. When all eigenvalues of A satisfy Re(i) 0, A is called a Hurwitz matrix. The origin of x Ax is asymptotically stable if and only if A is a Hurwitz matrix. The asymptotic stability of the origin can be also investigated using Lyapunovs method. Consider a quadratic Lyapunov-function candidate V(x) xTPx where P is a real symmetric positive-denite matrix. The derivative of V along the trajectories of x Ax is given by (x ) = xT Px V +x T Px = xT (PA + AT P )x = xT Qx where Q is a symmetric matrix dened by
where g1( ) and g2( ) are locally Lipschitz functions and saty isfy gi(0) 0, ygi(y) 0 y 0, i 1, 2, and 0 g1(z) dz , as y . The system has an isolated equilibrium point at the origin. The equation of this system can be viewed as a generalized pendulum equation with g2(x2) as the friction term. Therefore, a Lyapunov-function candidate may be taken x 2 , which is as the energylike function V(x) 01 g1(y) dy x2 2 positive denite in and radially unbounded. The derivative of V(x) along the trajectories of the system is given by (x ) = g (x )x + x [g (x ) g (x )] = x g (x ) 0 V 1 1 2 2 1 1 2 2 2 2 2 (x) is negative semidenite. Note that V (x) 0 imThus, V plies x2g2(x2) 0, which implies x2 0. The only solution that can stay identically in the set x 2x2 0 is the trivial solution x(t) 0. Thus, by the invariance principle, the origin is globally asymptotically stable.
PA + AT P = Q
(5)
If Q is positive denite, we can conclude that the origin is asymptotically stable; that is, A is a Hurwitz matrix. Suppose we start by choosing Q as a real symmetric positive denite matrix, and then solve Eq. (5) for P. If Eq. (5) has a positive denite solution, then again we can conclude that the origin is asymptotically stable. Equation (5) is called the Lyapunov equation. It turns out that A is a Hurwitz matrix if and only if for any given positive denite symmetric matrix Q there exists a positive denite symmetric matrix P that satises the Lyapunov equation (5). Moreover, if A is a Hurwitz matrix, then P is the unique solution of Eq. (5). Equation (5) is a linear algebraic equation that can be solved by rearranging it in the form Mx y, where x and y are dened by stacking the elements of P and Q in vectors.
395
Almost all commercial software programs for control systems include commands for solving the Lyapunov equation. Linearization Consider the nonlinear system x f (x) and suppose that f (x) is continuously differentiable for all x D n. The Jacobian matrix f / x is an n n matrix whose (i, j) element is f i / xj. Let A be the Jacobian matrix evaluated at the origin x 0. By applying the mean-value theorem to each component of f , it can be shown that f (x) Ax g(x), where g(x) / x 0 as x 0. Suppose A is a Hurwitz matrix and let P be the solution of the Lyapunov equation (5) for some positive denite Q. Taking V(x) xTPx, it can be shown that (x) is negative denite in the neighborhood of the origin. V Hence, the origin is asymptotically stable if all the eigenvalues of A have negative real parts. Using some advanced results of Lyapunov stability, it can be also shown that the origin is unstable if one (or more) of the eigenvalues of A has a positive real part. This provides us with a simple procedure for determining stability of the origin of a nonlinear system by calculating the eigenvalues of its linearization about the origin. Note, however, that linearization fails when Re(i) 0 for all i, with Re(i) 0 for some i. Example 4. The pendulum equation has two equilibrium points at (0,0) and (,0). Let us investigate stability of each point using linearization. The Jacobian matrix is given by
The Center-Manifold Theorem When the Jacobian of an equilibrium point has one or more eigenvalues with zero real parts and all other eigenvalues with negative real parts, the stability of the equilibrium cannot be ascertained from linearization. In these cases, the local nonlinear nature of the system will dictate the stability of the equilibrium, and the center-manifold theorem allows one to determine precisely the nature of the nonlinear terms that determine the stability. The main idea behind this technique is that the critical behavior occurs in a low-dimensional invariant manifold of the state space, one with dimension equal to the number of eigenvalues with zero real parts. The stability in the other dimensions is dominated by the exponential behavior associated with the eigenvalues that have negative real parts, but the nonlinear coupling between the marginal and asymptotically stable modes can play a critical role in determining stability. The center-manifold theorem makes these ideas precise. We begin with the following motivating example: y = zy y 3 , z = z + ay 2 (6)
f 1 x1 f = f x 2 x1
f1 0 x2 = a cos x1 f2 x2
1 b
To determine stability of the origin we evaluate the Jacobian at x 0, to obtain
A=
f x
x=0
0 a
1 b
Here, from the linearization point of view, the z dynamics are asymptotically stable and y is neutral. Based on this, one might be tempted to make the assumption that z 0, and therefore y is governed by y y3, and thus the origin is asymptotically stable. However, as is shown in an example below, this is incorrect; the stability of the origin is dictated by the sign of (a 1). The problem with the naive assumption made previously is that z approaches something small, but nonzero, and the correction, which stems from the nonlinear coupling terms and is captured by the center manifold, is crucial for determining stability. The development of the center manifold technique begins with the autonomous system x f (x), which has an equilibrium at x 0. The Jacobian A is dened as before and the state equation is written as x Ax g(x), where g(x) f (x) Ax contains terms that are essentially nonlinear about the origin. In order to split the dynamics into linearly asymptotically stable and neutral parts, the linear part of the equation is put into real Jordan form via a matrix P, as follows:
The eigenvalues of A are 1,2 b /2 b2 4a. For all positive values of a and b, the eigenvalues satisfy Re(i) 0. Hence, the equilibrium point at the origin is asymptotically stable. In the absence of friction (b 0), both eigenvalues are on the imaginary axis. In this case we cannot determine the stability of the origin through linearization. We have seen before that in this case the origin is a stable equilibrium point, as determined by an energy Lyapunov function. To determine the stability of the equilibrium point at (,0), we evaluate the Jacobian at this point. This is equivalent to performing a change of variables z1 x1 , z2 x2 to shift the equilibrium to the origin, and then evaluating the Jacobian f / z at z 0.
J = P 1 AP =
A1 0
0 A2
(7)
where all eigenvalues of A1 have zero real parts and all eigenvalues of A2 have negative real parts. The coordinate transformation
y =x z
then puts the state equation into the split form y = A1 y + g1 ( y, z ), z = A2 z + g2 ( y, z ) (8)
= f A x
x1 = , x2 =0
0 a
1 b
are 1,2 b /2 b2 4a. For all a The eigenvalues of A 0 and b 0, there is one eigenvalue in the open right-half plane. Hence, the equilibrium point at (,0) is unstable.
The z component of the dynamics of this system is dominated by the relatively fast linear system z A2z, whereas the y dynamics are slower than any exponential order. The key to the center-manifold technique is to capture the small, but crucial, coupling effects correctly in the nonlinear terms.
396
A center manifold for this system is simply a smooth invariant manifold of the form z h(y) with h(0) 0 and (h / y)(0) 0. Under some smoothness conditions on the nonlinear terms in Eq. (8) (that are inherited from the original equation), a local center manifold exists, although it is not in general unique. The power of the center manifold is that it can be used to reduce the dimensionality of the problem, as follows. By restricting the system dynamics to the center manifold one obtains y A1y g1(y, h(y)), referred to as the reduced system. The center manifold theorem states that if the origin of the reduced system is asymptotically stable (unstable), then the origin of the original system x f (x) is likewise asymptotically stable (unstable). The construction of the center manifold can be carried out as follows. We take the time derivative of z h(y) to obtain z [h(y)/ y]y . Equation (8) is used to substitute in for z and y , and z is replaced everywhere by h(y). This leads to A2h(y) g2(y, h(y)) [h(y)/ y][A1y g1(y, h(y))]. This equation for h(y), which must satisfy the conditions h(0) 0 and h / y(0) 0, is generally impossible to solve. However, since only local information is needed for stability considerations, an approximation for h(y) can be obtained by assuming a series expansion for h(y), substituting it into the equation, and matching coefcients, as demonstrated in the forthcoming Example 5. Once the expansion form for h(y) is determined and the expanded version of the reduced equation is in hand, various techniques can be employed for determining the stability of the reduced system. In general, this task is made much easier due to the lower dimensionality of the reduced system. Example 5. Consider the system given in Eq. (6). Here A1 0, A2 1, g1(y,z) yz y3, and g2(y,z) ay2. The center manifold is assumed to be of the form h(y) c1y2 c2y3 . This is substituted into the equation for h(y) and expanded in powers of y, and the coefcients of y2, y3, etc., are gathered and solved. This leads to the result that c1 a and c2 0. Therefore, h(y) ay2 O(y4). [We use the notation f (y) O(yp) when f (y) kyp for sufciently small y.] The reduced system is given by taking the equation for y and simply replacing z by the expansion for h(y), resulting in y (a 1)y3 O(y5). Thus, the conclusion is reached that for a 1 0, x 0 is an asymptotically stable equilibrium point for system Eq. (6), while for a 1 0 the origin is unstable. For a 1 0 no conclusions regarding stability can be drawn without considering higher-order expansions. Nonautonomous Systems Suppose the origin x 0 is an equilibrium point of the nonautonomous system x f (t, x); that is, f (t, 0) 0 for all t 0. For nonautonomous systems we allow the Lyapunov-function candidate V to depend on t. Let V(t, x) be a continuously differentiable function dened for all t 0 and all x D. The derivative of V along the trajectories of x f (t, x) is given by (t, x) V / t (V / x) f (t, x). If there are positive-denite V functions W1(x), W2(x), and W3(x) such that W1 (x ) V (t , x ) W2 (x ), (t , x ) W (x ) V 3 (9)
independent of the initial time t0. Such uniformity annotation is not needed with autonomous systems since the solution of an autonomous state equation starting at time t0 depends only on the difference t t0, which is not the case for nonautonomous systems. If inequalities in Eq. (9) hold globally and W1(x) is radially unbounded, then the origin is globally uniformly asymptotically stable. Example 6. Consider the nonautonomous system x 1 = x1 g(t )x2 , x 2 = x1 x2
where g(t) is continuously differentiable and satises 0 g(t) for all t 0. The system has an equilibg(t) k and g rium point at the origin. Consider a Lyapunov-function candi2 2 date V(t, x) x1 [1 g(t)]x2 . The function V satises the inequalities
2 2 2 x2 1 + x 2 V (t , x ) x 1 + (1 + k )x 2
The derivative of V along the trajectories of the system is given by = 2 x2 + 2 x x [2 + 2 g(t ) g (t )] x2 V 1 2 1 2 (t), we have 2 2g(t) g (t) 2 Using the bound on g 2g(t) g(t) 2. Therefore
2 x 2 + 2 x x 2 x 2 = xT V 1 2 1 2
2 1
1 x = xT Qx 2
The matrix Q is positive denite. Hence the origin is uniformly asymptotically stable. Since all inequalities are satis2 2 ed globally and x1 x2 is radially unbounded, the origin is globally uniformly asymptotically stable. BIFURCATION THEORY The term bifurcation, strictly speaking, refers to the splitting of a whole into two parts. While this is relevant to its meaning in dynamical systems, it has taken on a much broader denition. The general bifurcation problem deals with qualitative changes in system behavior as parameters are varied in a quasistatic manner. The simplest example is the case when a parameter is varied in such a manner that the real part of an eigenvalue of the Jacobian at an equilibrium point changes sign, corresponding to the change in stability of the equilibrium. There are two generic ways in which such transitions can occur. The rst is a real eigenvalue passing through zero; the second is a complex conjugate pair of eigenvalues passing through the imaginary axis. While linearization and the center-manifold theory allow one to determine the stability of the equilibrium point, the larger question looms as to what changes take place near the equilibrium through such a transition. Questions such as these are the basic motivation behind bifurcation theory. Consider the system x f (x, ), where represents a system parameter. The technical denition of a bifurcation is as follows: A bifurcation is said to occur at 0 if the state space for 0 is not topologically equivalent to that for 0. It is said that 0 is the bifurcation value of the parameter. The
for all t 0 and all x D, then the origin is uniformly asymptotically stable, where uniformly indicates that the denition of stability and the convergence of x(t) to zero are
397
reason for emphasizing topological equivalence in the state space is that one is interested in transitions that cause qualitative changes in the structure of the system response. For example, a smooth increase in the amplitude of a limit cycle is not a bifurcation, but the disappearance of a limit cycle is. A very important feature of bifurcation theory is that it allows for one to build a catalog of the generic qualitative changes systems may undergo as parameters are varied. For some simple, but widely encountered cases, this classication is complete. However, there are many important issues that remain unresolved. At the end of this section we will further address these classications. The rst convenient classication of bifurcations is to separate them into local and global. Local bifurcations describe changes in a neighborhood of an equilibrium, whereas global bifurcations involve nonlocal changes. (It is interesting to note that the study of local bifurcations in which two parameters are varied requires knowledge of global bifurcations.) We focus our attention on the analysis of local bifurcations and demonstrate some global bifurcations at the end of the discussion. Center-manifold theory is a powerful tool for analyzing these local bifurcations. It, along with normal-form theory (described briey later), allows one to reduce local bifurcations to their simplest possible forms, facilitating the classication mentioned above. To x the ideas, consider the original system x f (x, ) augmented by the trivial dynamics of the pa 0. Clearly the dynamics are linearly neutral rameter and have a zero eigenvalue (the case of multiple parameters goes through in the same manner). The center-manifold theory described earlier is redeveloped by carrying along the 0. The result is that the slow dynamics now equation contain y and and the center manifold is of the form z h(y, ). The procedure is carried out as before, with the key observation that is considered as a state variable, that is, terms such as y are taken to be nonlinear. [The equation for 0, with h(y, ) simplies to that of the original case since carried along in a straightforward manner.] This results in a reduced system describing the y dynamics that depends on 0. The equations are valid for the parameter, along with a range of values near the bifurcation value, and therefore this process allows for the unfolding of the behavior of the system about the bifurcation. Example 7. Consider the following parameterized version of the system given in Eq. (6): = 0, y = y + zy y 3 , z = z + ay2 (10)
y a >1 a <1
Figure 3. Subcritical pitchfork bifurcation (left); supercritical pitchfork bifurcation (right).
much more. It is seen that the approximate reduced equation has equilibria at y 0 and /(1 a). It is not difcult to show that the nonzero equilibria have the following properties: for a 1 they exist for 0 and are unstable when they exist, while for a 1 they exist for 0 and are asymptotically stable when they exist. Thus, for a 1 there exist two unstable equilibria near the asymptotically stable origin for 0, and they collapse onto the origin and disappear at 0, leaving the origin unstable for 0. This subcritical pitchfork bifurcation is shown in Fig. 3. The case for a 1, a supercritical pitchfork bifurcation, is shown in the same gure, from which the reader can infer the dynamic behavior as is varied through zero. This example points out some important features of local bifurcations. First, bifurcations that involve n eigenvalues with zero real parts require analysis of an nth-order dynamical system, obtained by reduction to the parametrized center manifold. Second, one can easily extend the ideas to include more than one parameter. The selection of the proper parameters to uncover all critical behavior near a bifurcation is a subtle matter that involves techniques beyond the scope of this article. Third, the analysis of this example is very straightforward, as there is only one nonlinear term in the approximate reduced equation. In more complicated problems several nonlinear terms may occur, rendering the stability or bifurcation analysis much more difcult. In such cases one can use normal-form theory to simplify the problem. This technique, which shares many similarities with feedback linearization, involves a systematic sequence of coordinate changes that remove as many nonlinear terms as is possible for the given system. This technique, when the bifurcation parameters are included, allows one to systematically reduce entire classes of problems into generic forms. This produces the classication scheme described above. Here we offer the results of this classication for the simplest bifurcations. As an eigenvalue of an equilibrium passes through zero, there are three generic things that can occur. The most general is the saddle-node bifurcation, in which a pair of equilibria merge and annihilate one another. If all other eigenvalues of the equilibrium of interest are stable, then the bifurcation diagram as the parameter is varied is as shown in Fig. 4. If
Here A1 0 (the 2 2 zero matrix), A2 1, g1(, y, z) (0, y yz y3)T, and g2(, y, z) ay2. Note that is treated as a state variable. The center manifold is assumed to be of the form h(, y) b1y2 b2y b32 . This is substituted into the equation for h(, y) and expanded in powers of y and and the coefcients of y2, y, 2, etc., are gathered and solved. This leads to the result that b1 a, b2 0, and b3 0. Therefore h(, y) ay2 , and the reduced system is given by taking the equation for y and simply replacing z by the expansion for h(, y), resulting in y y (a 1)y3 , which is valid in some neighborhood of (, y) (0,0). Clearly, for 0 the stability of the origin of the original system is dictated by the sign of ; this is known from linearization. The center-manifold results conrm this, but offer
Figure 4. Saddle-node bifurcation (left); transcritical bifurcation (right).
398
STACK SIMULATION
the equilibrium persists through the instability, a transcritical bifurcation, shown in Fig. 4, generally occurs. When a special symmetry in the problem exists, the pitchfork bifurcation encountered in the preceding example can occur. These cover the generic bifurcations involving a single eigenvalue going through zero. When a complex conjugate pair of eigenvalues passes through the imaginary axis, something more interesting happens. First, the center-manifold reduction for this problem leads to a dynamical system consisting of two equations. When expressed in polar coordinates these are r br2 . The radial variable, r ar3 and representing the amplitude of the oscillation, undergoes a pitchfork bifurcation (although r 0 is meaningless) that is super- (a 0) or sub- (a 0) critical as is increased through zero. The angular variable simply describes rotation at a nominal frequency of with a small amplitude-dependent shift arising from nonlinear effects. This is the Hopf bifurcation. The result of this bifurcation is shown in Fig. 5, and it results in the birth of stable limit cycles as an asymptotically stable equilibrium goes unstable (for a 0) or in the merging of an asymptotically stable equilibrium with an unstable limit cycle (for a 0). This completes the list of bifurcations that generically occur as a single parameter is varied and an equilibrium point changes stability. Center-manifold and normal-form methods also exist for analyzing the bifurcations that occur when limit cycles change stability. These problems can be handled by dening map near the limit cycle and studying and using a Poincare the generic bifurcations of xed points of maps. It is found that limit cycles can undergo saddle-node, transcritical, and pitchfork bifurcations that are essentially the same as shown in Figs. 3 and 4. The analogy to the Hopf bifurcation, the Neimark-Sacker bifurcation, involves a limit cycle changing stability and merging with a two-dimensional torus around it (in a super- or subcritical manner). However, this situation is complicated by the fact that resonances can occur between the two frequencies of the torus, and this can lead to secondary nonlinear phenomena. In addition, limit cycles have an additional bifurcation, the period-doubling bifurcation, in which a limit cycle changes stability and merges with another limit cycle that has a period twice that of the original. Again, this can be super- or subcritical in nature. Before leaving the topic of local bifurcations it should be noted that if several parameters are being simultaneously varied, one can encounter situations in which several eigenvalues simultaneously have zero real parts. Similarly, one may nd that some critical terms in a given normal form change sign as parameters are varied (for example, the parameter a in the Hopf bifurcation). The bifurcation analysis of such problems is extremely rich and often exceedingly complicated, but some classications along these lines have been carried out for two and three parameter systems. Global bifurcations result in qualitative changes in the state space that cannot be described as local to any equilib-
Figure 6. Saddle connection global bifurcation.
rium. Perhaps the simplest interesting example of a global bifurcation is the saddle connection shown in Fig. 6. Here, as a parameter is varied in the two-dimensional state space, a stable limit cycle moves toward and eventually merges with a saddle point, forming a saddle loop. As the parameter is pushed beyond this point, the limit cycle disappears. Note that this occurs without any changes to the stability types of the saddle point or the limit cycle. Global bifurcations involving saddle connections in systems of dimension three and higher are the source of chaos in dynamical systems. FURTHER READING For further reading on the Lyapunov stability and its applications, we refer the reader to Refs. 1 and 2. As for the steadystate behavior of dynamical systems, chaos, and bifurcations, we recommend Refs. 35. BIBLIOGRAPHY
1. H. K. Khalil, Nonlinear Systems, 2nd ed., Upper Saddle River, NJ: Prentice Hall, 1996. 2. M. Vidyasagar, Nonlinear Systems Analysis, 2nd ed., Englewood Cliffs, NJ: Prentice Hall, 1993. 3. S. H. Strogatz, Nonlinear Dynamics and Chaos, Reading, MA: Addison-Wesley, 1994. 4. J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, 2nd ed., New York: Springer Verlag, 1986. 5. D. K. Arrowsmith and C. M. Place, An Introduction to Dynamical Systems, Cambridge, UK: Cambridge University Press, 1990.
HASSAN K. KHALIL STEVEN W. SHAW

Michigan State University
STABILIZATION. See BILINEAR SYSTEMS.
Figure 5. Hopf bifurcation.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Stability Theory: Including Saturation Effects Standard Article J. J. D'Azzo1 and M. Pachter2 1Air Force Institute of Technology (AFIT/ENG), WrightPatterson AFB, OH 2Air Force Institute of Technology (AFIT/ENG), WrightPatterson AFB, OH Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1043 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (444K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Lyapunov Stability Criterion Quadratic Forms Linearization (Jacobian Matrix) Second Method of Lyapunov Location of Poles and Stability Design Using the Root Locus Frequency Response Closed-Loop Frequency Response Nonlinear Stability and Control Control Concept
Linear Quadratic Tracking (LQT) Maximal Output Admissible Sets Discrete-Time Reference Governor Static Admissibility and Invariance Suboptimal Approach Conclusion About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
357

The basic concept of the stability of dynamical systems is carefully developed, and its application to the design of feedback control systems is presented. Classical stability theories for linear and nonlinear feedback control systems are discussed and the Lyapunov and RouthHurwitz stability criteria are developed. Next, attention is given to conventional control design and to frequency-domain methods: The root locus and Nyquist stability criteria are presented, and the Bode plot and Nichols chart-based methods for stability and for degree of stability determination are discussed. The emphasis then shifts to the stability of control systems comprising linear plants and actuation elements which are subject to saturation. Then, nonlinear controllers are called for. Moreover, the broader question of bounded input and bounded output stability is addressed. Thus, a comprehensive time-domain
358
design methodology for nonlinear tracking controllers for actuator saturation effects mitigation, is presented. LYAPUNOV STABILITY CRITERION Stability is a very important characteristic of a system. If a system is linear and time-invariant, several criteria such as the Nyquist and Routhian stability criteria may be applied to determine stability. However, for nonlinear and time-varying systems, the Lyapunov direct method is applicable. Its advantage is that it is not necessary to solve the system equation, which is generally very difcult. The system with zero input is dened by = f ( x, t ) x (1)
and contains the n-dimensional response vector x x1, x2, . . ., xn. The vector f (x, t) is a function of x and t. The solution of Eq. (1) is a function of the initial condition x(0) and the initial time t0 and is given by x = (t , x(0 ), t0 ) (2)
region . Note that with this denition it is not necessary for the trajectory to approach the equilibrium point. It is necessary only for the trajectory to stay within the region . This permits the existence of a continuous oscillation about the equilibrium point. The state-space trajectory for such an oscillation is a closed path called a limit cycle. The performance specications for a control system must be used to determine whether or not a limit cycle can be permitted. The amplitude and frequency of the oscillation may inuence whether it represents acceptable performance. When limit cycles are not acceptable, more stringent restraints must be chosen to exclude their possible existence (25). Limit cycles are usually characterized as stable or unstable. If a limit cycle is stable, it means that trajectories in the state space on either side will approach the limit cycle. An unstable limit cycle is one in which the trajectories on either side diverge from the limit cycle. These trajectories may approach other limit cycles or equilibrium points. In the case that there is an input u(t), the system is dened by = f(x, u, t ) x (5)
The length of the vector x can be measured by the Euclidean norm (1)
2 2 1 /2 x = ( x2 1 + x2 + + xn )
(3)
The system of Eq. (1) is uniformly stable (2) when there is a nite positive constant such that, for any x(0) and t0, the solution of Eq. (2) satises the condition x x0 , t t0 (4)
where x0 represents the equilibrium solution as t . The vector x0 may be a point or it may be a continuous oscillation (limit cycle). Figure 1 represents the graphical plots of several possible solutions of Eq. (2) for an initial condition x(0) and a nite nal value x0. Stability in the Sense of Lyapunov Consider a region in the state space enclosing an equilibrium point x0. This equilibrium point is stable provided that there is a region (), which is contained within such that any trajectory (3) starting in the region does not leave the
In linear systems, stability is a system characteristic which is independent of any initial condition x(0) and/or the input magnitude u(t). However, in nonlinear systems the stability of the system may depend on the initial condition and/or the magnitude of the input (4). This case is typical for control systems and is considered later in this article. An important case is a system described by bounded-inputbounded-output (BIBO) stability. In such a system, either linear or nonlinear, the output response to any bounded input is also bounded. For a linear time-invariant (LTI) system, the necessary and sufcient condition for stability is that (5,6) the integral of the system weighting function or impulse response must be nite. This is expressed by |h(t )| dt < (6)
An equivalent condition for an LTI system is that all eigenvalues (3) are located in the left-half plane. In that case, the transient response associated with those eigenvalues decrease with time, and the output response therefore approaches the particular solution which is determined by the input. Asymptotic Stability An equilibrium point is asymptotically stable if, in addition to being stable in the sense of Lyapunov, all trajectories approach the equilibrium point. This means that the perturbation solution x*(t) approaches 0 as time t approaches innity. This is the stability denition usually used in control-system design. Figure 1 shows trajectories in the state plane illustrating the general principle of Lyapunov stability (3), asymptotic stability, and instability. The trajectory a starting at the initial state point x(0) and remaining within the region meets the conditions for Lyapunov stability. This trajectory is closed, indicating a continuous output oscillation or limit cycle. Trajectory b terminates at the equilibrium point x0; thus
b c x(0) x0
Figure 1. State-plane trajectories indicating (a) Lyapunov stability; (b) asymptotic stability; and (c) instability.
359
it represents asymptotic stability. Trajectory c leaves the region and therefore indicates instability. When the region includes the entire state space, the denitions of Lyapunov and asymptotic stability are said to apply in a global sense. The stability or instability of a linear system is global because any initial state yields the same stability determination. A stable linear system is globally asymptotically stable. The technique for linerarizing nonlinear differential equations in the neighborhood of their singularities or equilibrium points is presented in the section entitled Linearization (Jacobian Matrix). The validity of determining stability of the unperturbed solution near the singular points from the lin earized equations was developed independently by Poincare and Lyapunov in 1892. Lyapunov (4,7) designated this as the rst method. This stability determination is applicable only in a small region near the singularity and results in stability in the small. The section on linearization considers Lyapunovs second method, which is used to determine stability in the large. This larger region may include a nite portion, or sometimes the whole region, of the state space.
This can be written more compactly as

n n
f ( x, y ) =
i= 1 j = 1
a i j xi y j a12 a22 an 2 y1 a1 n y a2 n 2 ann yn
a11 a = [x1 x2 xn ] 21 an 1 = xT Ay = x, Ay
(11)
The matrix A is called the coefcient matrix of the bilinear form, and the rank of A is called the rank of the bilinear form. A bilinear form is called symmetric if the matrix A is symmetric. Quadratic Form. A quadratic form V is a real homogeneous polynomial in the real variables x1, x2, . . ., xn of the form
n n
V =
i= 1 j = 1
a i j xi x j
(12)
QUADRATIC FORMS Some of the techniques used in determining stability of control systems and for optimizing their response utilize scalar functions expressed in quadratic form. The necessary background for expressing functions in quadratic form is developed in this section. Then some important properties of quadratic forms are presented. Conjugate Matrix. The elements of a matrix may be complex quantities. For example, a matrix A may have the elements aij ij jij. A conjugate matrix B has elements with the same real component and with imaginary components of the opposite sign; that is, bij ij jij. This conjugate property is expressed by B = A (7)
where all aij are real. This is the special case of Eq. (11) where x y. The quadratic form V can therefore be expressed as the inner product V (x ) = xT Ax = x, Ax (13)
A homogeneous polynomial can always be expressed in terms of a symmetric matrix A. In the expansion of Eq. (12) the cross-product terms (i j) all have the form (aij aji)xixj. Choosing aij aji makes the matrix A for the quadratic form symmetric. This is illustrated in Eq. (14).
2 2 V ( x ) = x2 1 4x2 + 5x3 + 6x1 x2 20x2 x3 1 3 0 = x T 3 4 10 x = xT Ax 0 10 5
(14)
Inner Product. The inner product is also called a scalar (or dot) product since it yields a scalar function. The scalar product of vectors x and y is dened by
x, y = ( x ) T y = y T x = x 1 y1 + x2 y2 + + xn yn
(8)
When x and y are real vectors, the inner product becomes x, y = x 1 y 1 + x 2 y 2 + + x n y n (9)
The rank of the matrix A is called the rank of the quadratic form. If the rank of A is r n, the quadratic form is singular. If the rank of A is n, the quadratic form is nonsingular. A nonsymmetric matrix can be converted into an equivlent symmetric matrix by replacing all sets of elements aij and aji by the average value (aij aji)/2. The principal diagonal will be preserved. Length of a Vector. The length of a vector x is called the Euclidean norm and is denoted by x. It is dened as the square root of the inner product x, x. For real vectors it is given by x = x, x = x2 + x2 + + x2 n 1 2 (15)
Bilinear Form. A scalar homogeneous expression containing the product of the elements of vectors x and y is called a bilinear form in the variables xij and yij. When they are both of order n, the most general bilinear form in x and y is
A vector can be normalized so that its length is unity. In that case it is called a unit vector. The unit vector may be denoted by x and is obtained by dividing each element of x by x: = x x x (16)
f (x, y ) = a11 x1 y1 + a12 x1 y2 + + a1n x1 yn + a21 x2 y1 + a22 x2 y2 + + a2n x2 yn + + an1 xn y1 + an2 xn y2 + + ann xn yn (10)
Principal Minor. A principal minor of a matrix A is obtained by deleting any row(s) and the same numbered col-
360
umn(s). The diagonal elements of a principal minor of A are therefore also diagonal elements of A. The determinant A is classied as a principal minor, with no rows and columns deleted. The number of principal minors can be determined from a Pascal triangle. The number of principal minors for a matrix A of order n is: 1 for n 1, 3 for n 2, 7 for n 3, and 15 for n 4, . . .. Leading Principal Minor. There are n leading principal minors, formed by all the square arrays within A that contain a11. Starting with [a11], the next square array is formed by including the next row and column. This process is continued until the matrix A is obtained. The leading principal minors are given by
= |a11 | a11 = a21 a31 a12 a22 a32 a13 a23 a33
12
a11 a21
a12 a22 (17)
123
12 n
= |A|
The subscripts of are the rows and columns of A used to form the minor. The minor contains the common elements of these rows and columns. Deniteness and Semideniteness. The sign deniteness for a scalar function of a vector, such as V(x), is dened for a spherical region S about the origin described by x K (a constant equal to the radius of S). The function V(x) and all V(x)/ xi for i 1, 2, . . ., n must be continuous within S. The deniteness of a quadratic form is determined by analyzing only the symmetric A matrix. If A is given as a nonsymmetric matrix, it must rst be converted to a symmetric matrix. Positive Denite (PD). A scalar function, such as the quadratic form V(x) x, Ax, is called positive denite when V(x) 0 for x 0 and V(x) 0 for all other x K. The positive denite condition requires that A 0; thus, the rank of A is equal to n. The rank of A may be determined by putting it in hermite normal form. When a real quadratic form V(x) xTAx is positive denite, then the matrix A is also called positive denite. The matrix A has the property that it is positive denite iff there exists a nonsingular matrix H such that A HTH. The deniteness of the matrix A can be determined by reducing this matrix to diagonal form by means of a congruent transformation such that the diagonal matrix B PTAP. The matrix P may be formed by using the following elementary transformations: 1. Interchanging the ith and jth rows and interchanging the ith and jth columns. 2. Multiplying the ith row and column by a nonzero scalar k. 3. Addition to the elements of the ith row of the corresponding elements of the jth row multiplied by a scalar k and addition to the ith column of the jth column multiplied by k. If the principal diagonal of the matrix B contains only positive, nonzero elements, then the matrix A is positive denite. Another method of diagonalizing the A matrix is to determine a modal matrix T (8). In that case the diagonalized ma-
trix contains the eigenvalues obtained from I A 0. In order for the matrix A to be positive denite, all the eigenvalues must be positive. An alternate method of determining positive deniteness is to calculate all the leading principal minors of the matrix A. If all the leading principal minors of A are positive, the real quadratic form is positive denite. (This is sometimes called the Sylvester theorem.) Note that when the leading principal minors are all positive, then all the other principal minors are also positive. Positive Semidenite (PSD). The quadratic form V(x) is called positive semidenite when V(x) 0 for x 0 and V(x) 0 for all x K. The function V(x) is permitted to equal zero at points in S other than the origin (i.e., for some x 0), but it may not be negative. In this case A 0, the rank of A is less than n, and all of the eigenvalues of A are positive. The positive semidenite quadratic form can there2 2 2 fore be reduced to the form y 1 y2 yr , where r n. When V(x) is positive semidenite, then the matrix A is also called positive semidenite. The matrix A is positive semidenite iff all its characteristic values are 0. In order to be positive semidenite, all the principal minors (9) must be nonnegative. Negative Denite (ND) and Negative Semidenite (NSD). The denitions of negative denite and negative semidenite follow directly from the denitions above when the inequalities are applied to A. If a matrix A does not satisfy the conditions for positive deniteness or positive semideniteness, then A is checked for these conditions. If A satises the positive denite or positive semidenite conditions, then A is said to be negative denite or negative semidenite, respectively. Indenite. A scalar function V(x) is indenite if it assumes both positive and negative values within the region S described by x K. If A is not positive denite, positive semidenite, negative denite, or negative semidenite, then it is said to be indenite. Note that when A, in general form, has both positive and negative elements along the principal diagonal, it is indenite. When A is transformed to a diagonal form, it is indenite if some of the diagonal elements are positive and some are negative. This means that some of the eigenvalues are positive and some are negative. Example 1. Use the principal deniteness of 1 1 A = 1 1 1 1 minors to determine the
1 1 0
(18)
The leading principal minors are evaluated to check for positive deniteness:
= 1,
12
1 1
1 = 0, 1
123
1 = 1 1
1 1 1
1 1 = 0
=0
The conditions for positive deniteness are not satised. Next the remaining principal minors are evaluated to check for positive semideniteness.
13 2
1 1 = 1, =
1 = 1, 0
23 3
1 1 =0 =
1 = 1 0
361
Since the principal minors are not all nonnegative, the matrix A is not positive semidenite. Similarly, A is not negative denite or negative semidenite; therefore A is indenite. Example 2. Transform A in Eq. (18) to the diagonal Jordan form and check its deniteness. The characteristic equation is
Saddle point
x2 Node Saddle point
1 I A | | = 1 1
1 1 1
1 1 = 3 22 2 = 0
x1
(19)
The eigenvalues are 1 2.732, 2 0.732, and 3 0. Thus the matrix A is indenite since
Separatix
2.732 = 0 0
0 0.732 0
0 0 0
Figure 3. Phase portrait for Eq. (20).
LINEARIZATION (JACOBIAN MATRIX) (35,7,10) A linear system with no forcing function (an autonomous system) and with A 0 has only one equilibrium point x0. A nonlinear system, on the other hand, may have more than one equilibrium point. This is easily illustrated by considering the unity-feedback angular position-control system shown in Fig. 2. The feedback action is provided by synchros which generate the actuating signal e sin(i o). With no input, i 0, the differential equation of the system is o + K sin o = 0 o + a (20)
in a Taylor series about one of the equilibrium points x0. Assuming that x* is restricted to a small neighborhood of the equilibrium point, the higher-order terms in the Taylor series may be neglected. Thus, the resulting linear variational state equation is f1 f1 f1 x xn 1 x2 f2 f2 f 2 x1 x2 xn = x x = Jx x (23) fn fn fn x1 x2 xn
x= x 0
This is obviously a nonlinear differential equation because of the term sin o. With the phase variables x1 o and x2 o, the corresponding state equations are x 1 x 1 = x2 , x 2 = K sin x1 ax2 (21)
where f i is the ith row of f (x) and Jx f / xT is called the Jacobian matrix and is evaluated at x0. For the system of Eq. (21) the motion about the equilibrium point x1 x2 0 is represented by
The slope of the trajectories in the phase plane (25) is obtained from N= x 2 K sin x1 ax2 = x 1 x2 (22)
= x
0 x 1 = K x 2
1 a
x 1 = Jx x x 1
(24)
In the general case the unforced state equation is x f (x). Since equilibrium points x0 exist at x f (x0) 0, the singularities are x2 0 and x1 k, where k is an integer. The system therefore has multiple equilibrium points. In a small neighborhood about each of the equilibrium points, a nonlinear system behaves like a linear system. The states can therefore be written as x x0 x*, where x* represents the perturbation or state deviation from the equilibrium point x0. Each of the elements of f (x) can be expanded
For these linearized equations the eigenvalues are 1,2 a /2 (a /2)2 K. This equilibrium point is stable and is either a node or a focus, depending upon the magnitudes of a and K. A node denotes an overdamped response and a focus denotes an underdamped response about the equilibrium point. For motion about the equilibrium point x1 , x2 0, the state equation is
= x
0 K
1 x a
(25)
f ( e)
K s (s + a )
Synchro
Figure 2. A nonlinear feedback control system.
The eigenvalues of Jx are 1,2 a /2 (a /2)2 K. Thus, one eigenvalue is positive and the other is negative, and the equilibrium point represents an unstable saddle point. The motion around the saddle point is considered unstable since every point on all trajectories, except on the two separatrices, moves away from this equilibrium point. A phase-plane portrait for this system can be obtained by the method of isoclines (25). From Eq. (22) the isocline equation is x2 K sin x1 /(N a). The phase portrait is shown in Fig. 3. Note that the linearized
362
equations are applicable only in the neighborhood of the singular points. Thus, they describe stability in the small. In analyzing the system performance in the vicinity of equilibrium, it is usually convenient to translate the origin of the state space to that point. This is done by inserting x x0 x* into the original equations. With the origin at x0, the variations from this point are described by x*. SECOND METHOD OF LYAPUNOV The second, or direct, method of Lyapunov provides a means for determining the stability of a system without explicitly solving for the trajectories in the state space. This is in contrast to the rst method of Lyapunov, which requires the determination of the eigenvalues from the linearized equations about an equilibrium point. The second method is applicble for determining the behavior of higher-order systems which may be forced or unforced, linear or nonlinear, time-invariant or time-varying, and deterministic or stochastic. Solution of the differential equation is not required. The procedure requires the selection of a scalar energy function V(x), which is tested for the conditions that indicate stability. When V(x) successfully meets these conditions, it is called a Lyapunov function. The principal difculty in applying the method is in formulating a correct Lyapunov function because, for asymptotically stable systems, the failure of one function to meet the stability conditions does not mean that a true Lyapunov function does not exist. This difculty is compounded by the fact that the Lyapunov function is not unique. Nevertheless, there is much interest in the second method (10,11). In order to show a simple example of a Lyapunov function, consider the system of Fig. 2 represented by Eq. (20) which o 0, o n, where n is has multiple equilibrium points at an integer. The proposed function is the sum of the kinetic and potential stored energies, given by o ) = 1 2 + K (1 cos o ) V (o , 2 o (26)
ellipsoid, in the state space with its center at the origin. The entire state space is lled with such nonintersecting closed surfaces, each representing a different positive value of V(x). (x) is negative for all points in the state space, it When V means that all trajectories cross the closed surfaces from the outside to the inside and eventually converge at the equilibrium point at the origin. A qualitatively correct Lyapunov function is shown in Fig. 2 2 4 for a second-order system. The function V(x1, x2) x1 x2 is positive denite and is represented by the paraboloid surface shown. The value V(x1, x2) ki (a constant) is represented by the intersection of the surface V(x1, x2) and the plane z ki. The projection of this intersection on the x1, x2 plane is a closed curve, an oval, around the origin. There is a family of such closed curves in the x1, x2 plane for different values of ki. The value V(x1, x2) 0 is the point at the origin; it is the innermost curve of the family of curves representing different levels on the paraboloid. The gradient vector of V(x) is dened by V (x ) x1 V (x ) gradient V (x ) = V (x ) = x2 (28) . . . V (x ) xn (x) along any trajectory is The time derivative V
(x ) = V (x ) dx1 + V (x ) dx2 + + V (x ) dxn V x1 dt x2 dt xn dt = V , x = [ V ( x )] T x
(29)
o, except at This function is positive for all values of o and the equilibrium points, where it is equal to zero. The rate of change of this energy function along any phase-plane trajectory is obtained by differentiating Eq. (26) and using Eq. (20):
2 (o , o ) = o o = a o o + K (sin o ) V
(x) can be evaluated without It is important to note that V knowing the solution of the system state equation x f (x). The gradient V(x1, x2) describes the steepness between adjacent levels of V(x1, x2). The function V(x1, x2) is negative denite in Fig. 4except at the origin, where it is equal to zero. The state-plane trajectory crosses the ovals for successively smaller values of V(x). Therefore, the system is asymptoti-
(27)
z = V(x)
is negative along any trajectory for all values The value of V o / do of the phaseof o, except o 0. Note that the slope d plane trajectories [see Eq. (22)] is innite along the line repre o 0, except at the equilibrium points o n. sented by o 0 does not represent an equilibrium, exThus, the line cept at o n, and it is not a trajectory in the state plane. Since the energy stored in the system is continuously decreasing at all points except at the equilibrium points, the equilibrium at the origin and at even multiples of o n is asymp is NSD. totically stable. This V This example demonstrates that the total system energy may be used as the Lyapunov function. When the equations of a large system are given in mathematical form, it is usually difcult to dene the energy of the system. Thus, alternate Lyapunov functions must be obtained. For any system of order n, a positive, constant value of the proper positive denite Lyapunov function V(x) represents a closed surface, a hyper-
x2
V(x) = const = k1 x1
0 V(x) State plane trajectory
Figure 4. A positive denite function V(x1, x2) and projections on the x1, x2 plane.
363
cally stable, and V(x) is a proper Lyapunov function. The concepts described above may be summarized in the following theorem, which provides sufcient, but not necessary, conditions for stability. Theorem 1. Lyapunov asymptotic stability. A system x f (x, t), where f (0, t) 0 for all t, is asymptotically stable in the vicinity of the equilibrium point at the origin if there exists a scalar function V(x) such that: 1. V(x) is continuous and has continuous rst partial derivatives in a region S around the origin. 2. V(x) 0 for x 0. [V(x) is PD.] 3. V(0) 0. (x) 0 for x 0. [V (x) is ND.] 4. V Conditions 13 ensure that V(x) is positive denite. Therefore, V(x) k is a closed surface within the region S. Condi (x) is negative denite, and thus any tration 4 means that V jectory in S crosses through the surface V(x) k from the outside to the inside for all values of k. Therefore, the trajectory converges on the origin where V(0) 0. (x) 0 for x 0 can be relaxed in The condition that V Theorem 1 under the proper conditions. Condition 4 can be (x) 0; that is, V (x) is negative semidenite. changed to V (x) is not This relaxed condition is sufcient, provided that V equal to zero at any solution of the original differential equation except at the equilibrium point at the origin. A test for (x) 0 into the this condition is to insert the solution of V state equation x f (x) to verify that it is satised only at the equilibrium point. Also, if it can be shown that no trajectory can stay forever at the points or on the line, other than the 0, then the origin is asymptotically staorigin, at which V ble. This is the case for the system of Fig. 2 as described at the beginning of this section. For linear systems there is only one equilibrium point which is at the origin; therefore it is (x) to be negative semidenite. Theorem 1 may sufcient for V also be extended so that it is applicable to the entire state space. In that case the system is said to have global stability, or stability in the large. Including these conditions results in the following theorem: Theorem 2. Lyapunov global asymptotic stability. A system is globally asymptotically stable if there is only one stable equilibrium point and there exists a scalar function V(x) such that: 1. V(x) is continuous and has continuous rst partial derivatives in the entire state space. 2. V(x) 0 for x 0. 3. V(0) 0. 4. V(x) at x . (x) 0. 5. V (x) 0, except at x 0, or any locus in the 6. Either V (x) 0 is not a trajectory of the state space where V system. Conditions 13 ensure that V(x) is positive denite. Condition 4 is satised when V(x) is positive denite (i.e., it is closed) in the entire state space. When V(x) goes to innity as
any xi , then V(x) ki is a closed curve for any ki. Conditions 5 and 6 mean that V(x) is continuously decreasing along any trajectory in the entire plane and ensures that the system is asymptotically stable. In order to check for global stability, it is necessary to select V(x) so that conditions 14 are all satised. Finding a proper Lyapunov function V(x) means that the system is stable, but this is just a sufcient and not a necessary condition for stability. The fact that a function V(x) has not been found does not mean that it does not exist and that the system is not stable. Application of the Lyapunov Method to Linear Systems The second method of Lyapunov is applicable to time-varying and nonlinear systems. However, there is no simple general method of developing the Lyapunov function. Methods have been developed for many such systems, and the literature in this area (4) is extensive. The remaining applications in this chapter are restricted to linear systems. Since linear systems may have only one equilibrium point, the stability or instability is necessarily global in nature. The RouthHurwitz stability criterion is available for determining the stability of linear systems. However, the necessity of obtaining the characteristic polynomial can be a disadvantage for higher-order systems. Thus, the following material is presented to develop familiarity with the more general second method of viewing stability. Condence in the second method of Lyapunov can be developed by showing that the results are identical to those obtained with the RouthHurwitz method. The approach presented is to select a V(x) which is positive (x). The coefcients of V(x) and denite and to evaluate V those restraints on the system parameters are then deter (x) negative denite or negative semimined which make V denite. Consider the linear, unity feedback system presented in Fig. 5 with r(t) 0. Example G (s ) = K , s (s + a ) a>0 (30)
The differential equation for the actuating signal, when r 0, is e + ae + Ke = 0 (31)
When phase variables with x1 e are used, the state equations are x 1 = x2 , x 2 = Kx1 ax2 (32)
A simple Lyapunov function which is positive denite is V (x ) = 1 1 1 p x2 + p x2 = xT Px 2 1 1 2 2 2 2 (33)
R(s) +
E(s)
G(s)
Y(s)
Figure 5. Unity-feedback linear system.
364
where p1 0 and p2 0. Its derivative is
(x ) = p x x V 2 = p1 x1 x2 p2 Kx1 x2 ap2 x2 1 1 1 + p 2 x2 x 2 p1 p2 K 0 T 2 = xT p p K x = x Nx 1 2 ap2 2
(34)
(x) is always negative semidenite if N is The function V negative semidenite. The matrix N is negative semidenite if all the principal minors of N are positive or zero.
2
= ap2 0 = ( p1 p2 K )2 0 4
(35) (36)
For example, a simple diagonal matrix such as N I or N 2I is positive denite. A positive semidenite matrix N can be chosen which contains all zero elements except one positive element in any position along the principal diagonal. Step 2. Determine the elements of P by equating terms in Eq. (40). Since P is symmetric, this requires the solution of n(n 1)/2 equations. Step 3. Use the Sylvester theorem to determine the deniteness of P. Step 4. Since N is selected as positive denite (or positive semidenite), the necessary and sufcient condition for asymptotic stability (instability) is that P be positive denite (negative denite). The Sylvester conditions for the positive deniteness of P are the same as the RouthHurwitz stability conditions. The equivalence is derived in Ref. 12. Linear systems which are asymptotically stable are globally asymptotically stable. Example. Examine the conditions of asymptotic stability for the second-order dynamic system x + a1 x + a0 x = 0 With phase variables the coefcient matrix is (41)
12
Equation (35) is satised since it is required that both a and p2 be greater than zero. Since the left side of Eq. (36) cannot be positive, only the equality condition p1 p2K 0 can be satised, which yields p1 p2K. Since p1 0 and p2 0, this 2 (x) ap2x2 requires that K 0. Then Eq. (34) yields V , which is negative semidenite. The Lyapunov global asymptotic stability theorem (Theorem 2) is satised since, by condi (x) 0 holds tion 4, V(x) as x . The condition V along the x1 axis, where x2 0 and x1 has any value. A way (x) being negative semidenite is sufcient of showing that V for global asymptotic stability is to show that the x1 axis is not a trajectory of the system state equations [see Eq. (32)]. The rst equation yields x 1 0 or x1 c. The x1 axis can be a trajectory only if x2 0 and x 2 0. But the second state equation yields x 2 Kx1 Kc 0. This is a contradiction since it requires x1 c 0. Therefore, the x1 axis is not a trajectory, and the system is asymptotically stable. Using Lyapunovs second method, the necessary condition for stability is shown above to be K 0. This result is readily recognized as being correct, either from the Routh stability criterion or from the root locus. Lure Method Another design approach is the use of a procedure developed by Lure (cited in Ref. 12). Consider the unexcited system represented by the state equation in which A is of order n: = Ax x (37)
A=
0 a0
1 a1
(42)
The symmetric matrix P and a chosen positive semidenite matrix N are
p11 P= p 12
p12 p22 ,
N=
2 0
0 0
(43)
The identity in Eq. (40) yields three simultaneous equations: The solution of these equations yields
n11 = 2 = 2a0 p12 n12 = 0 = p11 + a1 p12 + a0 p22 n22 = 0 = 2 p12 + 2a1 p22 p12 1 = , a0 p22 1 = , a0 a1 p11 a + a2 1 = 0 a0 a1 (44)
The quadratic Lyapunov function V(x) is expressed in terms of the symmetric matrix P by V (x ) = x Px
T
(38)
The necessary conditions for P to be positive denite are obtained by applying the Sylvester theorem: p11 = a0 + a2 1 > 0, a0 a1 p11 p22 p2 12 = a0 >0 a2 a2 0 1 (45)
The time derivative of V(x) is (x ) = x T Px + xT Px V = xT (AT P + PA )x = xT Nx where the symmetric matrix N is given by N = (AT P + PA ) = 2 (PA)sym The following procedure is used: Step 1. Select an arbitrary symmetric matrix N of order n which is either positive denite or positive semidenite. (40) (39)
The second equation requires that a0 0. Using this condition in the rst equation produces the necessary condition a1 0. These are obviously the same conditions that are obtained from the RouthHurwitz conditions for stability, as shown in a later section. Krasovskii (cited in Ref. 4) has shown that a similar approach may be used with a nonlinear system. However, only sufcient conditions for local asymptotic stability in the vicinity of an equilibrium point may be obtained. For the Lyapu-
365
j s = jd s = b + jc 2 s=a 1 2* s = b jc s = jd s=0 3 s=e 5 6* s = f jg 4* 4 s = f + jg 3
s plane
Poles of the response transform located at the origin or on the imaginary axis that are not contributed by the forcing function R(s) are undesirable in a control system. Poles in the right half-plane result in transient terms that increase with time. Such performance characterizes an unstable system. Therefore, poles in the right-half s plane are not permissible. Routh Stability Criterion The stability characteristic of a linear time-invariant system is determined from the systems characteristic equation (13). A system is asymptotically stable if and only if all roots of the characteristic equation lie to the left of the imaginary axis. Rouths stability criterion provides a means for determining stability without evaluating the roots of this equation. The response transform Y(s) has the general form given by Eq. (47), where R(s) is the driving transform. Although the inverse Laplace transform of Y(s) can be performed, the polynomial Q(s) must rst be factored. Computer and calculator programs are readily available for obtaining the roots of a polynomial (18). Figure 6 and Table 1 show that stability of the response y(t) requires that all zeros of Q(s) have negative real parts. Since it is usually not necessary to nd the exact solution when the response is unstable, a simple procedure to determine the existence of zeros with positive real parts is needed. If such zeros of Q(s) with positive real parts are found, the system is unstable and must be modied. Rouths criterion is a simple method of determining the number of zeros with positive real parts without actually solving for the zeros of Q(s). Note that zeros of Q(s) are poles of Y(s) (15,16). The characteristic equation is Q ( s ) = b n sn + b n 1 sn 1 + b n 2 sn 2 + + b 1 s + b 0 = 0 (48)
Figure 6. Location of poles in the s plane. (Numbers are used to identify the poles.)
(x) nov function V(x) xTPx, the time derivative is V xTNx, where
T N = Jx P + PJx
(46)
The matrix Jx is the Jacobian evaluated at the equilibrium point. Selecting P I may often lead to a successful determination of the conditions for asymptotic stability in the vicinity of the equilibrium point. This extension to nonlinear systems shows the generality of the method. LOCATION OF POLES AND STABILITY The dynamic equations of a system are often developed in linearized form. However, if they are nonlinear, it is possible to obtain a linearized approximation as described in the section entitled Linearization (Jacobian Matrix). In terms of the Laplace transform variable s, the response transform Y(s) for the linear representation can be expressed, in general, as the ratio of two polynomials. The response transform Y(s) has the form
Y (s ) =
a w sw + a w 1 sw 1 + + a 1 s + a 0 P (s ) R (s ) = R (s ) Q (s ) sn + b n 1 sn 1 + + b 1 s + b 0 P (s ) (47) R (s ) = (s s1 )(s s2 ) (s sk ) (s sn )
where R(s) is the system input and the values of the poles s1, s2, . . ., sn may be real or occur in conjugate complex pairs. The stability and the corresponding response of a system can be determined from the locations of the poles of the response transform Y(s) in the s plane. The possible positions of the poles are shown in Fig. 6, and the responses associated with the poles are given in Table 1. These poles are the roots of the characteristic equation Q(s) 0.
If the b0 term is zero, divide by s to obtain the equation in the form of Eq. (48). The bs are real coefcients, and all powers of s from sn to s0 must be present in the characteristic equation. A necessary but not sufcient condition for stable roots is that all the coefcients in Eq. (48) be positive. If any coefcients other than b0 are zero or if the coefcients do not all have the same sign, then there are pure imaginary roots or roots with positive real parts and the system is unstable. It is therefore unnecessary to continue if only stability or instability is to be determined. When all the coefcients are present and positive, the system may or may not be stable because there still may be roots on the imaginary axis or in the righthalf s plane. Rouths criterion is mainly used to determine stability. In special situations it may be necessary to determine the actual number of roots in the right-half s plane. For these situations the procedure described in this section can be used. The coefcients of the characteristic equation are arranged in the pattern shown in the rst two rows of the following Routhian array. These coefcients are then used to evaluate the rest of the constants to complete the array.
Table 1. Relation of Response to Location of Poles Position of Pole 1 22* 3 44* 5 66* Form of Response Ae at Ae bt sin(ct ) A A sin(dt ) Ae et Ae ft sin( gt ) Characteristics Damped exponential Exponentially damped sinusoid Constant Constant-sinusoid Increasing exponential (unstable) Exponentially increasing sinusoid (unstable)
sn | b n s s s
n1 n2 n3
bn 2 bn 3 c2 d2
bn 4 bn 5 c3
bn 6 bn 7
(49)
| bn 1 | c1 | d1
| s1 | j 1 s0 | k 1
366
The constants c1, c2, c3, and so on, in the third row are evaluated as follows: c1 = bn 1 bn 2 bn bn 3 bn 1 (50) (51)
Theorem 2. A zero coefcient in the rst column: When the rst term in a row is zero but not all the other terms in that row are zero, the following methods (14) can be used: 1. Substitute s 1/ x in the original equation; then solve for the roots of x with positive real parts. The number of roots x with positive real parts will be the same as the number of s roots with positive real parts. 2. Multiply the original polynomial by the factor (s 1) which introduces an additional negative root. Then form the Routhian array for the new polynomial. The rst method is illustrated in the following example: Q ( s ) = s 4 + s 3 + 2s 2 + 2s + 5 (56)
bn bn 5 b b c2 = n 1 n 4 bn 1
This pattern is continued until the rest of the cs are all equal to zero. Then the d row is formed by using the sn1 and sn2 rows. These contants are d1 = c1 b n 3 b n 1 c2 c1 (52) (53)
b n 1 c3 c b d2 = 1 n 5 c1
The Routhian array is
This is continued until no more d terms are present. The rest of the rows are formed in this way, down to the s0 row. The complete array is triangular, ending with the s0 row. Notice that the s1 and s0 rows contain only one term each. Once the array has been found, Rouths criterion states that the number of roots of the characteristic equation with position real parts is equal to the number of changes of sign of the coefcients in the rst column. Therefore the system is stable if all terms in the rst column have the same sign. Reference 17 shows that a system is unstable if there is a negative element in any position in any row. The following example illustrates this criterion: Q(s ) = s + s + 10s + 72s + 152s + 240
5 4 3 2
s4 | s |
3 2
1 1 0
2 2 5
5 (57)
s |
The zero in the rst column prevents completion of the array. The following methods overcome this problem. Method 1. Letting s 1/ x and rearranging the polynomial gives Q ( x ) = 5x 4 + 2x 3 + 2x 2 + x + 1 The new Routhian array is (58)
(54)
The Routhian array is formed by using the procedure described above:
s4 | s3 | s2 | s1 |
5 2 1 5 2
2 1 2
s5 | s4 | s3 | s2 | s1 | s0 |
1 1 62 70.6 122.6 240
10 72 88 240
152 240 (55)
(59)
s |
0
In the rst column there are two changes of sign, from 1 to 62 and from 62 to 70.6; therefore Q(s) has two roots in the right-half s plane. Note that this criterion gives the number of roots with positive real parts but does not tell the values of the roots. If Eq. (54) is factored, the roots are s 3, s2,3 1 j3, and s4,5 2 j4. This conrms that there are two roots with positive real parts. The Routh criterion does not distinguish between real and complex roots. Theorem 1. Division of a row: The coefcients of any row may be multiplied or divided by a positive number without changing the signs of the rst column. The labor of evaluating the coefcients in Rouths array can therefore be reduced by multiplying or dividing any row by a constant. This may result, for example, in reducing the size of the coefcients and therefore simplifying the evaluation of the remaining coefcients.
There are two changes of sign; therefore there are two roots of x in the right-half s plane. The number of roots of s with positive real parts is also two. This method does not work when the coefcients of Q(s) and of Q(x) are identical. Method 2 Q1 (s ) = Q(s )(s + 1 ) = s5 + 2s4 + 3s3 + 4s2 + 7s + 5 (60)
The reader may obtain the Routhian array which has two changes of sign in the rst column, so there are two zeros of Q(s) with positive real parts. Thus, the same result is obtained by both methods. An additional method is described in Ref. 20. Theorem 3. When all the coefcients of one entire row are zero, the procedure is as follows: 1. The auxiliary equation can be formed from the preceding row, as shown below.
367
2. The Routhian array can be completed by replacing the all-zero row with the coefcients obtained by differentiating the auxiliary equation. 3. The roots of the auxiliary equation are also roots of the original equation. These roots occur in pairs and are the negative of each other. Therefore, these roots may be imaginary (complex conjugates) or real (one positive and one negative), may lie in quadruplets (two pairs of complex-conjugate roots), and so on. Consider the system which has the characteristic equation q (s ) = s4 + 2s3 + 11s2 + 18s + 18 = 0 (61)
characteristic equation, which is Q(s ) = s4 + 7s3 + 15s2 + (25 + K )s + 2K (66)
The coefcients must all be positive in order for the zeros of Q(s) to lie in the left half of the s plane, but this is not a sufcient conditon for stability. The rst column of the Routhian array permits evaluation of precise boundaries for K. It is left as an exercise for the reader to show that the closedloop system is stable for 0 K 28.1. DESIGN USING THE ROOT LOCUS As described in the section entitled Location of Poles and Stability, the stability of linear time-invariant (LTI) systems is determined by the locations of the system poles. Any pole in the right-half plane leads to an exponentially increasing transient which therefore means that the system is unstable. An effective method for designing a feedback control is by use of the root locus. Figure 7 shows a block diagram of a singleinput single-output feedback system expressed in terms of the Laplace transform variable s. G(s) is the forward transfer function representing the dynamics of the plant. Also, H(s) is the feedback transfer function representing the dynamics in that path. The feedback signal B(s) is compared with the command input signal R(s), and their difference is called the actuating signal. The overall closed-loop transfer function is C(s)/ R(s), where R(s) is the control system input and C(s) is the controlled output signal. For the root-locus method the open-loop transfer function can be put in the form G ( s )H ( s ) = K (s + a1 ) (s + ah ) (s + aw ) sm (s + b1 )(s + b2 ) (s + bc ) (s + bu ) (67)
s4 | s3 | s | s2 | s2 | s1 |
3
1 2 1 2 1 0
11 18 9 18 9
18
(After dividing the s3 row by 2)
(62)
(After dividing the s2 row by 2)
The presence of a zero row for s1 indicates that there are roots that are the negatives of each other. The next step is to form the auxiliary equation from the preceding row, which is the s2 row. The highest power of s is s2, and only even powers of s appear. Therefore the auxiliary equation is s2 9 0. The roots of this equation are s j3. These are also roots of the original equation. The presence of imaginary roots indicates that the output includes a sinusoidally oscillating component. For a sinusoidal input with the frequency corresponding to the imaginary root, the response is unbounded. Thus, the system with imaginary roots is considered unstable. To complete the Routhian array, the auxiliary equation is differentiated and is 2s + 0 = 0 (63)
where ah and bc may be real or complex numbers and may lie in either the left-half or right-half s plane. The value of K may be either positive or negative. Equation (67) can be rewritten as G ( s )H ( s ) = K K (s z1 ) (s zw ) = m sm ( s p 1 ) ( s p u ) s
w h = 1 (s zh ) u c = 1 (s pc )
The coefcients of this equation are inserted in the s1 row, and the array is then completed:
(68)
s1 | s0 |
2 9
(64)
Since there are no changes of sign in the rst column, there are no roots with positive real parts. In feedback systems, the ratio of the output to the input does not have an explicitly factored denominator (see the section entitled Design Using the Root Locus). An example of such a function is P (s ) K (s + 2 ) Y (s ) = = R (s ) Q (s ) s(s + 5 )(s2 + 2s + 5 ) + K (s + 2 ) (65)
where indicates a product of terms. The degree of the numerator is w and that of the denominator is m u n. The zh are the zeros and the pc are the poles of G(s)H(s). In the form shown in Eq. (68), with the coefcients of s all equal to unity, the K is dened as the loop sensitivity. The underlying principle of the root locus is that the poles of the control ratio C(s)/ R(s) are related to the zeros and poles of the open-loop transfer function G(s)H(s) and to the loop
R(s) + B(s)
E(s)
G(s)
C(s)
The value of K is an adjustable parameter in the system and may be positive or negative. The value of K determines the location of the poles and therefore the stability of the system. It is important to know the range of values of K for which the system is stable. This information must be obtained from the
H(s)
Figure 7. Block diagram of a feedback system.
368
sensitivity K. This is shown as follows. Let G (s ) = Thus C (s ) A (s ) G (s ) = M (s ) = = R (s ) B (s ) 1 + G ( s )H ( s ) where D D + N1 N2 B ( s ) 1 + G ( s )H ( s ) = 1 2 D1 D2 Rationalizing Eq. (70) gives C (s ) N1 D2 P (s ) = M (s ) = = R (s ) D1 D2 + N1 N2 Q (s ) (72) (70) N1 (s ) , D1 ( s ) H (s ) = N2 (s ) D2 ( s ) (69)
B(s)H(s) given in Eq. (67) as |K | = |sm | |s p1 | |s pu | |s z1 | |s zw | (79)
= =
(angles of denominator terms) (angles of numerator terms) (1 + 2h )180 h360 for K > 0 for K < 0 (80)
Design Application: Root Locus Procedures (71) The procedure for obtaining the root locus is to rst put the open-loop transfer function G(s)H(s) into the form shown in Eq. (68). The poles and zeros are then plotted in the s j plane. Then the angle condition given in Eq. (80), for both K 0 and K 0, is used to obtain the complete root locus. Then the root locus can be calibrated in terms of the loop sensitivity K by using the magnitude condition given in Eq. (79). The dominant roots are those that contribute the most to the overshoot and the settling time of the closed-loop system response. Selection of the dominant complex roots on the root locus is based on the characteristics of a simple second-order system with a unit step input (8). The peak overshoot Mp is related to the damping ratio by
From Eqs. (71) and (72) it is seen that the zeros of B(s) are equal to the poles of M(s), and they determine the form of the closed-loop system response. The degree of the numerator of B(s) is equal to m u. Therefore B(s) has n m u nite zeros. The roots of B(s) 0, which is the characteristic equation of the closed-loop system, must satisfy the equation B ( s ) 1 + G ( s )H ( s ) = 0 These roots must satisfy the equation G ( s )H ( s ) = K (s z1 ) (s zw ) = 1 sm ( s p 1 ) ( s p u ) (74) (73)
M p = 1 + exp
1 2
(81)
A specication based on Mp yields a required value of . Roots with a specied value of occur at the intersection of the root locus with a straight line drawn at an angle drawn from the negative real axis, where = cos 1 (82)
Thus, as the loop sensitivity K assumes values from zero to innity, the open-loop transfer function must always be equal to 1. The corresponding values of s which satisfy Eq. (74) for any value of K are the poles of M(s). The plots of these values of s are dened as the root locus of M(s). They satisfy the following conditions: For K 0: Magnitude condition: Angle condition: |G(s )H (s )| = 1 (75)
An alternate specication is based on the desired settling time Ts. The real part of the desired complex root is given by = number of time constants Ts (83)
\G(s)H (s) = (1 + 2h)180,

For K 0: Magnitude condition: Angle condition:
h = 0, 1, 2, . . .
(76)
The transient coefcient et decays to, and stays within a specied percentage of, the nal value, in the time Ts. The latter is expressed as the number of time constants obtained from Eq. (83). For example, using 2% of the nal value as the criterion requires four time constants for the selected root. Root Locus Construction Rules The following properties facilitate the drawing of the root locus:
|G(s )H (s )| = 1
(77)
\G(s)H (s) = h360,
h = 0, 1, 2, . . .
(78)
Thus, the root locus method provides a plot of the variation of each of the poles of C(s)/ R(s) in the complex s plane as the loop sensitivity K is varied from zero to innity. All the angles are measured as positive in the counterclockwise sense. Since G(s)H(s) usually has more poles than zeros, it is convenient to express the angle condition for the generalized form of
Rule 1. The number of branches of the root locus is equal to the number of poles of the open-loop transfer function. Rule 2. For positive values of K, the root locus exists on those portions of the real axis for which the total number of real poles and zeros to the right is an odd number. For negative values of K, the root locus exists on those portions of the real axis for which the total number of
369
real poles and zeros to the right is an even number (including zero). Rule 3. The root locus starts (K 0) at the open-loop poles and terminates (K ) at the open-loop zeros or at innity. Rule 4. The angles of the asymptotes of the root locus branches that end at innity are determined by a
A root-locus digital-computer program (8,21) will produce an accurate calibrated root locus. This considerably simplies the work required for the system design. By specifying for the dominant roots or Km, a computer program can determine all the roots of the characteristic equation. Root Locus Example. Given here is the unity feedback system with
h360 for K < 0 (1 + 2h )180 for K > 0 [Number of poles of [Number of zeros of G ( s )H ( s )] G ( s )H ( s )]
(84)
G (s ) =
Rule 5. The real-axis intercept of the asymptotes is o =

n c = 1 Re ( pc )
nw
w h = 1 Re(zh )
K1 s(s/25 + 1 )(s2 /2600 + s/26 + 1 ) 65,000K1 = s(s + 25 )(s2 + 100s + 2600 ) K = 4 s + 125s3 + 5,000s2 + 65,000s
(87)
(85) where K 65,000K1. Specication: Find C(s)/ R(s) with 0.5 for the dominant roots (roots closest to the imaginary axis). 1. The poles of G(s)/ H(s) are plotted on the s plane in Fig. 8. The values of the poles are: s 0, 25, 50 j10, 50 j10. The system is completely unstable for K 0. Therefore, this example is designed only for the condition K 0. 2. There are four branches of the root locus. Since there are no zeros, all branches end at innity for K . 3. The locus exists on the real axis between 0 and 25. 4. The angles of the asymptotes are = (1 + 2h )180 = 45 , 135 4
Rule 6. The breakaway point for the locus between two poles on the real axis (or the break-in point for the locus between two zeros on the real axis) can be determined by taking the derivative of the loop sensitivity K with respect to s. Equate this derivative to zero and nd the roots of the resulting equation. The root that occurs between the poles (or the zeros) is the breakaway (or break-in) point. Rule 7. For K 0 the angle of departure from a complex pole is equal to 180 minus the sum of the angles from the other poles plus the sum of the angles from the zeros. Any of these angles may be positive or negative. For K 0 the departure angle is 180 from that obtained for K 0. For K 0 the angle of approach to a complex zero is equal to the sum of the angles from the poles minus the sum of the angles from the other zeros minus 180. For K 0 the approach angle is 180 from that obtained from K 0. Rule 8. The imaginary-axis crossing of the root locus can be determined by forming the Routhian array for the closed-loop characteristic equation. Equate the s1 row to zero and form the auxiliary equation from the s2 row. The roots of the auxiliary equation are the imaginaryaxis crossover points. Rule 9. The selection of the dominant roots of the characteristic equation is based on the specications that give the required system performance; that is, it is possible to evaluate , d, and , which are used to select the location of the desired dominant roots. The loop sensitivity for these roots is determined by means of the magnitude condition. The remaining roots are then determined to satisfy the same magnitude condition. Rule 10. For those open-loop transfer functions for which w n 2, the sum of the closed-loop roots is equal to the sum of the open-loop poles. Thus, once the dominant roots have been located,
n n
5. The real-axis intercept of the asymptotes is o = 0 25 50 50 = 31.25 4
jw K s3 55.9 + j18 K
= 0.5
s1
s plane 6.6 + j11.4
50 + j10
K=0
K=0
sa
60 K=0
o
50 j10 s4 K=0
25
s2
pj =
j=1 j=1
rj
(86)
Figure 8. Root locus for
can be used to nd one real or two complex roots. Factoring known roots from the characteristic equation can also simplify the work of nding the remaining roots.
G ( s )H ( s ) =
65, 000K1 s(s + 25 )(s2 + 100s + 2600 )
370
6. The breakaway point sa on the real axis between 0 and 25 is found from the derivative dK / ds 0, using Eq. (79) to obtain K(s). The result is dK = 4s3 + 375s2 + 10,200s + 65,000 = 0 ds This yields sa 9.15. 7. The angle of departure from the pole 50 j10 is obtained from 0 + 1 + 2 + 3D = (1 + 2h )180 where the angles are measured from each open-loop pole to the pole 50 j10. This yields 3D 123.1. 8. Using Eq. (72), the closed-loop transfer function is
14. With a unit step input, R(s) 1/ s, the output response can be obtained from a CAD program (8,21):
c(t ) = 1 + 1.4e 6.6t sin(11.4t 143.8 ) + 0.2e 55.9t sin(18.0t 123.7 )
(90)
The gures of merit which describe the signicant step response characteristics are ts 0.64 s, tp 0.31 s, Mp 1.56, and c(t)ss 1.00. The root locus design method ensures more than system stability. It also permits selection of the closed-loop system poles that best satisfy the desired performance specications.
C (s ) 65,000K1 FREQUENCY RESPONSE = 4 (88) 3 R (s ) s + 125s + 5,100s2 + 65,000s + 65,000K1 The objective in using feedback is threefold. The rst purpose The Routhian array for the denominator of C(s)/ R(s), is to achieve stability and/or a specied degree of stability which is the characteristic polynomial, is margin. The second goal is to reduce the sensitivity to parameter variation. The third goal is disturbance rejection. The s4 | 1 5100 65,000K1 previous section introduces the root locus method of design. The closed-loop poles are assigned in the left-half plane so s3 | 1 520 (After division by 125 ) that the performance is stable, and they are located on the 1 14.2K1 (After division by 4580 ) s2 | root locus in positions that best achieve the desired performance criteria. This section introduces the frequency res1 | 520 14.2K1 sponse methods that ensure closed-loop system stability by 0 s | 14.2K1 application of the Nyquist criterion and meet desired gures of merit. The gain of the forward transfer function is the adPure imaginary roots exist when the s1 row is zero. justable parameter which is used in the design. The feedback This occurs when K1 520/14.2 36.6. The auxiliary control system is shown in Fig. 7. For a linear time-invariant 2 equation is s 14.2K1 0. The imaginary roots are (LTI) system, the open-loop transfer function in the s domain s j14.2K1 j22.8. is shown in Eq. (68), and the closed-loop transfer function is 9. Additional points on the root locus are found by locat- represented by Eqs. (70) and (72). ing points that satisfy the angle condition The forward frequency transfer function is written in the generalized form: \s + \(s + 25 ) + \(s + 50 j10 ) + \(s + 50 + j 10 ) = (1 + 2m )180
The root locus is shown in Fig. 8. 10. The radial line for 0.5 is drawn on the graph of Fig. 8 at the angle cos1 cos1 0.5 60. The dominant complex roots obtained from the graph are s1,2 = 6.6 j 11.4 11. Applying the magnitude condition of Eq. (79) yields K = 65, 000K1 = |s| |s + 25| |s + 50 j 10| |s + 50 + j 10| For s 6.6 j11.4, K 598,800 and K1 9.25. 12. The other roots are evaluated to satisfy the magnitude conditon K 598,800. The other roots of the characteristic equation are s = 55.9 j 18.0 13. The overall closed-loop transfer function is C (s ) 598,800 = R (s ) (s + 6.6 j 11.4 )(s + 55.9 j 18 ) (89)
G( j ) =
Km (1 + j T1 )(1 + j T2 ) (1 + j Tw ) m 2 )( j )2 ] ( j ) (1 + j Ta )[1 + (2 /n ) j + (1/n (91)
= Km G ( j )
where m denes the system type, Km is the gain constant, and G( j) is the forward transfer function with unity gain. There are two forms typically used for the plot of G( j). In the rst category is the Nyquist plot which is the output input ratio in polar coordinates. The Nyquist stability criterion is applied to this plot to determine the closed-loop system stability. The second category is the pair of Bode plots. One plot is the magnitude in decibels versus the logarithm of frequency, and the second plot is the angle versus the logarithm of frequency. Typically, semilog graph paper is used so that it is not necessary to obtain the logarithm of frequency. The data from the Bode plot is then plotted on a Nichols plot in rectangular coordinates: log magnitude versus angle. The Nichols chart is a set of contours which are drawn on the Nichols plot and is used to both ensure stability and to adjust the gain in order to meet the closed-loop specications. A digital computer CAD program (8,21) provides considerable assistance in obtaining these plots and performing the design.
371
Direct Polar-Plot Characteristics To obtain the direct polar plot of a systems forward transfer function, the following characteristics are used to determine the key parts of the curve. Step 1. The forward transfer function has the general form shown in Eq. (91). For this transfer function the system type is equal to the value of m and determines the portion of the polar plot representing the lim0G( j) 0(w m u)90. The low-frequency polar-plot characteristics (as 0) of the different system types are summarized in Fig. 9. The angle at 0 is m(90). The arrow on the polar plots indicates the direction of increasing frequency. Step 2. The high-frequency end of the polar plot can be determined as follows:
+
smooth one in which the angle of G( j) continuously decreases as goes from 0 to . With time constants in the numerator, and depending upon their values, the angle may not continuously vary in the same direction, thus creating dents in the polar plot. Step 6. As is seen later in this chapter, it is important to know the exact shape of the polar plot of G( j) in the vicinity of the 1 j0 point and the crossing point on the negative real axis. Nyquists Stability Criterion A system designer must be sure that the closed-loop system he designs is stable. The Nyquist stability criterion (22,23) provides a simple graphical procedure for determining closedloop stability from the frequencyresponse curves of the openloop transfer function G( j)H( j). The application of this method in terms of the polar plot is covered in this section; application in term of the log magnitudeangle (Nichols) diagram is covered in a later section. The closed-loop transfer function of the system is given in Eqs. (70) and (72). The characteristic equation formed from the denominator of this closed-loop transfer function is B ( s ) = 1 + G ( s )H ( s ) = D1 D2 + N1 N2 =0 D1 D2 (94)
lim G( j ) = 0\(w m u )90
(92)
Note that since the degree of the denominator of Eq. (91) is usually greater than the degree of the numerator, the high-frequency point ( ) is approached (i.e., the angular condition) in the clockwise sense. The plot ends at the origin, tangent to the axis determined by Eq. (92). Tangency may occur on either side of the axis. Step 3. The asymptote that the low-frequency end approaches, for a Type 1 system, is determined by taking the limit as 0 of the real part of the transfer function. Step 4. The frequencies at the points of intersection of the polar plot with the negative real axis and the imaginary axis are determined, respectively, by setting Im[G( j )] = 0 Re[G( j )] = 0 (93)
Step 5. If there are no frequency-dependent terms in the numerator of the transfer function, the curve is a
270 Type 3 0+
180 0+
1 + j 0
K0
=+
Type 2
=0
For a stable system the roots of the characteristic equation must not lie in the right-half s plane or on the j axis. Note that the numerator and denominator of B(s) have the same degree. The poles of the open-loop transfer function G(s)H(s) are the poles of B(s). Since the denominator of C(s)/ R(s) is the same as the numerator of B(s), the condition for stability may therefore be restated as: For a stable system, none of the zeros of B(s) can lie in the right-half s plane or on the imaginary axis. Nyquists stability criterion relates the number of zeros and poles of B(s) that lie in the right-half s plane to the polar plot of G(s)H(s). In this analysis it is assumed that the control systems range of operation is conned to the linear region. This yields a set of linear differential equations which describe the dynamic performance of the systems. Because of the physical nature of feedback control systems, the order of the denominator D1D2 is equal to or greater than the order of the numerator N1N2 of the open-loop transfer function G(s)H(s). This means that limsG(s)H(s) 0 or a constant. A rigorous mathematical derivation of Nyquists stability criterion is based on complex variable theory. A qualitative approach to Nyquists stability criterion is presented for the special case that B(s) is a rational fraction. The characteristic function B(s) given by Eq. (94) can be rationalized, factored, and then written in the form B (s ) = P (s ) (s Z1 )(s Z2 ) (s Zn ) = (s p1 )(s p2 ) (s pn ) Q (s ) (95)
0+ Type 1 90
Type 0
Figure 9. A summary of direct polar plots of different types of systems.
where Z1, Z2, . . ., Zn are the zeros and p1, p2, . . ., pn are the poles. The poles pi are the same as the poles of the open-loop transfer function G(s)H(s) and include the s term for which p 0, if it is present.
372
j O s plane p2
(s p
2)
Generalizing Nyquists Stability Criterion Consider now a closed contour Q which encloses the whole right-half s plane (see Fig. 11), thus enclosing all zeros and poles of B(s) that have positive real parts. As a consequence of the theory of complex variables used in the formal derivation, the contour Q must not pass through any poles or zeros of B(s). When the results of the preceding discussion are applied to the contour Q, the following properties are noted:
Q
Q Z1 (s Z1) 0 (s Z3) Z3
) p3
(s)
Z4
p4
p1
(s
p3
(s Z2) Z2
p3
Figure 10. A plot of some poles and zeros of Eq. (95).
1. The total number of clockwise rotations of B(s) due to its zeros is equal to the total number of zeros ZR in the right-half s plane. 2. The total number of counterclockwise rotations of B(s) due to its poles is equal to the total number of poles PR in the right-half s plane. 3. The net number of rotations N of B(s) 1 G(s)H(s) about the origin is equal to its total number of poles PR minus its total number of zeros ZR in the right-half s plane. N may be positive (ccw), negative (cw), or zero. The essence of these three properties can be represented by the equation N = Change in phase of [1 + G(s )H (s )]/2 = PR ZR (96)
In Fig. 10 some poles and zeros of a generalized function B(s) are drawn on the s plane. Also, an arbitrary closed contour Q is drawn which encloses the zero Z1. To the point O on Q, whose coordinates are s j, are drawn directed line segments from all the poles and zeros. The lengths of these directed line segments are given by s Z1, s Z2, s p1, s p2, and so on. Not all the directed segments from the poles and zeros are indicated in the gure, because they are not necessary to proceed with this development. As the point O is rotated clockwise once around the closed contour Q, the directed segment s Z1 rotates through a net clockwise angle of 360. All the other directed segments rotate through a net angle of 0. Thus, referring to Eq. (95), it is seen that the clockwise rotation of 360 for s Z1 is simultaneously realized by the function B(s) for the enclosure of the zero Z1 by the path Q. Consider now a larger closed contour Q which includes the zeros Z1, Z2, and Z3 and the pole p5. As a point O is rotated clockwise once around the closed curve Q, each of the directed line segments from the enclosed pole and zeros rotates through a net clockwise angle of 360. Since the angular rotation of the pole is experienced by the characteristic function B(s) in its denominator, the net angular rotation realized by Eq. (95) must be equal to the net angular rotations due to the pole p5 minus the net angular rotations due to the zeros Z1, Z2, and Z3. Therefore, for this case, the net number of rotations N experienced by B(s) 1 G(s)H(s) for the clockwise movement of point O once about the closed contour Q is equal to 2; that is, N = (number of poles enclosed) (number of zeros enclosed) = 1 3 = 2 where the minus sign denotes clockwise (cw) rotation. Also, for any closed path that may be chosen, all the poles and zeros that lie outside the closed path each contribute a net angular rotation of 0 to B(s) as a point is moved once around this contour.
where counterclockwise rotation is dened as being positive and clockwise rotation is negative. In order for the characteristic function B(s) to realize a net rotation N, the directed line segment representing B(s) (see Fig. 12) must rotate about the origin 360N degrees, or N complete revolutions. Solving Eq. (96) for ZR yields ZR = PR N (97)
For a stable system, B(s) can have no zeros ZR in the righthalf s plane. It is therefore concluded that, for a stable system, the net number of rotations of B(s) about the origin must be counterclockwise and equal to the number of poles PR that lie in the right-half plane. In other words, if B(s) experiences a net clockwise rotation (i.e., if N is negative), this indicates that ZR PR, where PR 0, and thus the closed-loop system is unstable. If there are zero net rotations, then ZR PR and the system may or may not be stable, according to whether PR 0 or PR 0.
j s=+j s = re j
s plane Q +
s=j
Figure 11. The contour that encloses the entire right-half s plane.
373
270 B(s) = [1+ G(s) H(s)] B(s) 180
270
B(s)
1 + j0
180
0 1 + j0
G(s) H(s)
90 (a)
90 (b)
Figure 12. A change of reference for B(s).
Obtaining a Plot of B(s). Figures 12(a) and 12(b) show a plot of B(s) and a plot of G(s)H(s), respectively. By moving the origin of Fig. 12(b) to the 1 j0 point, the curve is now equal to 1 G(s)H(s), which is B(s). Since G(s)H(s) is known, this function is easily plotted, and then the origin is moved to the 1 point to obtain B(s). In general, the open-loop transfer functions of many physical systems do not have any poles PR in the right-half s plane. In this case, ZR N. Thus, for a stable system the net number of rotations about the 1 j0 point must be zero when there are no poles of G(s)H(s) in the right-half s plane. If the function G(s)H(s) has some poles in the right-half s plane and the denominator is not in factored form, then the number PR can be determined by applying Rouths criteiron to D1D2. The Routhian array gives the number of roots in the right-half s plane by the number of sign changes in the rst column. Analysis of Path Q. Nyquists criterion requires that the whole right-half s plane must be encircled to ensure the inclusion of all poles or zeros in this portion of the plane. In Fig. 11 the entire right-half s plane is included by the closed path Q which is composed of the following two segments: 1. One segment is the imaginary axis from j to j. 2. The other segment is a semicircle of innite radius that encircles the entire right-half s plane. The portion of the path along the imaginary axis is represented mathematically by s j. Thus, replacing s by j in Eq. (95) and letting take on all values from to gives the portion of the B(s) plot corresponding to that portion of the closed contour Q which lies on the imaginary axis. A requirement of the Nyquist criterion is that limsG(s)H(s) 0 or a constant. Therefore, as the point O moves along the segment of the closed contour represented by the semicircle of innite radius, the corresponding portion of the B(s) plot is a xed point. As a result, the movement of point O along only the imaginary axis from j to j results in the same net rotation of B(s) as if the whole contour Q were considered. In other words, all the rotation of B(s) occurs while the point O goes from j to j along the imaginary axis. Effect of Poles at the Origin on the Rotation of B(s). Some transfer functions G(s)H(s) have an sm in the denominator. Since no poles or zeros can lie on the contour Q, the contour shown in Fig. 11 must be modied to the contour shown in
Fig. 13(a). Consider the transfer function G ( s )H ( s ) = K1 s(1 + T1 s )(1 + T2 s ) (98)
The point O is rst moved along the negative imaginary axis from s j to a point where s j 0 90 becomes very small; that is, s j. Then the point O moves along a semicircular path of radius s ej in the right-half s plane, with a very small radius until it reaches the positive imaginary axis at s j j0 0 90. From here the point O proceeds along the positive imaginary axis to s j. Letting the radius approach zero, 0, for the semicircle around the origin, ensures the inclusion of all poles and zeros in the right-half s plane. To complete the plot of B(s), the effect of moving point O on this semicircle around the origin must be investigated. For the semicircular portion of the path Q represented by s ej, where 0 and 90 90, Eq. (98) becomes G ( s )H ( s ) = K K1 K K = 1 = 1 e j = 1 e j s e j (99)
where K1 / as 0, and goes from 90 to 90 as the directed line segment s goes counterclockwise from 90 to 90. Thus, in Fig. 13(b), the points G(s)H(s) for 0 and 0 are joined by a semicircle of innite radius in the rst and fourth quadrants. Figure 13(b) shows the completed contour of G(s)H(s) as the point O moves along the
j j
= 0
1 2 3
j0 + s = e j j0
7 6 5 4 3 12 Q
1 + j0 B(s)
=+ =
5 6 (b)
G(s)H(s) j (a)
= 0+
Figure 13. (a) The contour Q which encircles the right-half s plane. (b) Complete plot for Eq. (98).
374
modied contour Q of Fig. 13(a) in the s plane in the clockwise direction. When the origin is moved to the 1 j0 point, the curve becomes B(s). The plot of B(s) in Fig. 13(b) does not encircle the 1 j0 point; therefore N 0. Also, from Eq. (98), PR 0. Thus, using Eq. (97), ZR 0 and the system is stable. If the gain K1 is increased so that the intersection of G(s)H(s) with the negative real axis occurs to the left of the 1 point, then N 2 and Z 2, with the result that the closed-loop system is unstable. Nyquist Stability: Type 2 System. Transfer functions that have the term sm in the denominator have the general form, as 0, G ( s )H ( s ) = Km = sm Km m e jm = Km
m
be determined by drawing the line radiating from the 1 j0 point (see Fig. 14) and noting one cw and one ccw crossing; thus, the sum of these crossings is N 0. Like the previous example, this system can be made unstable by increasing the gain sufciently for the G(s)H(s) plot to cross the negative real axis to the left of the 1 j0 point. When G( j)H( j) Passes Through the 1 j0 Point. When the curve of G( j)H( j) passes through the 1 j0 point, the number of encirclements N is indeterminate. This corresponds to the condition where B(s) has zeros on the imaginary axis. A necessary condition for applying the Nyquist criterion is that the path encircling the specied area must not pass through any poles or zeros of B(s). When this condition is violated, the value for N becomes indeterminate and the Nyquist stability criterion cannot be applied. Simple imaginary zeros of B(s) mean that the closed-loop system will have a continuous steady-state simusoidal component in its output which is independent of the form of the input. In addition, a sinusoidal input with the frequency equal to that of the imaginary zero produces an unbounded output. Therefore, this condition is considered unstable. Nichols Plot and Stability The log magnitude of the open-loop transfer function G( j)H( j), abbreviated as Lm, is dened as 20 times the logarithm to the base 10: Lm G( j ) = 20 log10 |G( j )| dB (102)
e jm =
Km
m
e jm
(100)
where m 1, 2, 3, 4, . . .. With the reasoning used in the preceding example, it is seen from Eq. (100) that, as s moves from 0 to 0, the plot of G(s)H(s) traces m clockwise semicircles of innite radius about the origin. For example, if m 2, then, as goes from /2 to /2 in the s plane with radius, , G(s)H(s) experiences a net rotation of (2)(180) 360. Since the polar plots are symmetrical about the real axis, it is only necessary to determine the shape of the plot of G(s)H(s) for a range of values of 0 . The net rotation N of the plot for the range of is twice that of the plot for the range of 0 . Consider the system G ( s )H ( s ) = K2 (1 + T4 s ) s2 (1 + T1 s )(1 + T2 s )(1 + T3 s ) (101)
where T4 T1 T2 T3. Figure 14 shows the mapping of G(s)H(s) for the contour Q of the s plane. The word mapping, as used here, means that for a given point in the s plane there corresponds a given value of G(s)H(s) or B(s). The presence of the s2 term in the denominator of Eq. (101) results in a net rotation of 360 in the vicinity of s 0, as shown in Fig. 14. For the complete range of frequencies the net rotation is zero; thus, since PR 0, the system is stable. The value of N can
1 0
1 + j0
=+ =
B(s)
G(s)H(s) Radial line
PR = 0 N=0 ZR = 0 Stable system
Figure 14. The complete polar plot for Eq. (101).
where the Lm has the units of decibels (dB). The Nichols plot is dened for the open-loop transfer function G( j)H( j) as having the Lm on the vertical axis and the angle on the horizontal axis. This is an alternate to the polar plot and can be used to determine closed-loop system stability by use of the Nyquist criterion. The corresponding polar and Nichols plots for the transfer function of Eq. (98) are shown in Figs. 15(a) and 15(b). The data for plotting these curves can be obtained by use of a computer-aided design (CAD) program (8,21). The log-magnitudeangle diagram is drawn by picking for each frequency the values of log magnitude and angle. The resultant curve has frequency as a parameter. The curve for the Eq. (98) is drawn in Fig. 15(b). Note that the point 1 j0 on the polar plot becomes a series of points on the Nichols plot, having the values (0 dB, k), where k takes on all odd integer values. Changing the gain raises or lowers the Lm G( j) curve without changing the angle characteristics. Increasing the gain raises the curve, and analysis of the polar plot has shown that this reduces stability. The log-magnitudeangle diagram for G(s)H(s) can be drawn for all values of s on the contour Q of Fig. 13(a). The resultant curve for minimum-phase systems is a closed contour. Nyquists criterion can be applied to this contour by determining the number of points having the values 0 dB and odd multiples of 180, which are enclosed by the curve of G(s)H(s). This number is the value of N which is used in the equation ZR N PR to determine the value of ZR. As an example, consider a control system whose transfer function is given by Eq. (98). Its log-magnitudeangle diagram, for the contour Q, is shown in Fig. 16. From this gure it is seen that the value of N is zero. Thus ZR N PR 0, and the system
375
1
Phase margin (+)
Lm G ( j ) (dB) Lm (dB)
c
Gain margin, Lma (+)
1 a 1
1 +
0 dB
360
270
+
180
(+) G( j)
Gain margin (+)
90
0 Angle
90 135 180 225 270 Phase margin, (+)
=
(a) (b)
Figure 15. Frequency response plots: (a) Polar plot, (b) Nichols plot, and (c) Bode plots of G( j).
(c)
is stable. The log-magnitudeangle contour for a non-minimum-phase system does not close; thus it is difcult to determine the value of N. For these cases the polar plot is easier to use to determine stability. It is not necessary for minimum-phase systems to obtain the complete log-magnitudeangle contour to determine stability. Only that portion of the contour is drawn representing G( j)H( j) for the range of values 0 . The stability is then determined from the position of the curve of G( j)H( j) relative to the (0 dB, 180) point. In other words, the curve is traced in the direction of increasing frequencythat is, walking along the curve in the direction of increasing frequency. The system is stable if the (0 dB, 180) point is to the right of the curve. This is a simplied rule of thumb which is based on Nyquists stability criterion for a minimum-phase system. A conditionally stable system is one in which the curve crosses the 180 axis at more than one point. Figure 17 shows the transfer-function plot for such a system with two stable and two unstable regions. The gain determines whether the system is stable or unstable.
Gain Margin and Phase Margin Stability from the Nichols Plot The absolute stability of an LTI closed-loop system can be determined by applying the Nyquist stability theorem, using the representation of the open-loop transfer function G( j) as a polar plot or as Nichols plot (log magnitude versus angle). Some measures of degree of stability can be expressed in terms of gain margin and phase margin. The following quantities are used to express these stability measures: Gain Crossover. This is the point on the plot of the transfer function at which the magnitude of G( j) is unity [Lm G( j) 0 dB]. The frequency at gain crossover is called the phase-margin frequency . Phase Margin. This is 180 plus the negative trigonometrically considered angle of the transfer function at the gain-crossover point. It is designated as the angle , which can be expressed as 180 , where G( j ) is negative. Phase Crossover. This is the point on the plot of the transfer function at which the phase angle is 180. The fre-
This portion of contour represents the semicircle of radius of contour Q
= 0+
+
+ dB + dB
= 0 0+
0 dB 270 180 90 0 dB +90 +180 +270
= +180
G(s) Contour dB This portion of contour represents the semicircle of infinite radius r of contour Q
=
Figure 16. The log-magnitudeangle contour for the minimum-phase system of Eq. (98).
376
Lm (dB)
Lm (dB)
to the gain margin. However, the phase margin gives a better estimate of damping ratio, and therefore of the transient overshoot of the closed-loop system, than the gain margin. The phase-margin frequency , phase margin , crossover frequency c, and the gain margin Lm a are readily identied on the Nichols plot, as shown in Fig. 15(b). Bode Plots (Logarithmic Plots)
0 180
0 180 180
Lm (dB)
Lm (dB)
0 180
(a)
(b)
(c)
(d)
The Bode plots consist of two components. One plot is for the log magnitude [Lm G( j) 20 log G( j)] versus log , and the second plot is for the angle of G( j) versus log . The Bode plots are often used to represent the open-loop transfer function. The log magnitude of G( j) converts the operations of multiplication and divisoin to addition and subtraction, respectively. Each factor of a transfer function [see Eq. (91)], plotted versus log , has a distinctive characteristic which is readily recognized. Constant Km. The Lm Km is a constant which is independent of frequency. Thus, it is a horizontal line and serves to raise or lower Lm G( j), when it is larger than or smaller than unity, respectively. j Factor. The Lm G( j) has a positive slope of 20 dB per decade when this factor appears in the numerator. Its slope is negative when it appears in the denominator of G( j). The transfer function type is related to the low frequency slope of the Lm plot. A zero slope indicates a Type 0 system, a slope of 20 dB/decade indicates a Type 1 system, and so on. 1 jT Factor. The Lm(1 jT) has a corner frequency at cf 1/ T. At frequencies below cf the asymptote is the 0 dB line. Above cf the asymptote is a straight line that passes through 1/ T at zero dB, with a slope of 20 dB/decade. When (1 jT) appears in the numerator of G( j), the slope of the asymptote is positive. When it appears in the denominator, the slope of the asymptote is negative. Quadratic Factor [1 2j / n ( j / n)2]. The log magnitude of the quadratic factor has a corner frequency at cf n. At frequencies below cf the asymptote is the 0 dB line. Above cf the asymptote is a straight line that passes through cf at 0 dB with a slope of 40 dB/decade. When the quadratic factor is in the denominator, the slope of the asymptote is negative. When it appears in the numerator, the slope of the asymptote is positive. Because this quadratic factor has the damping ratio as an additional variable, there is a family of log magnitude plots which depend on the value of . The gain margin and the phase margin angle can be determined from the Bode plots, as shown in Fig. 15(c). Adjusting the gain results in raising or lowering the log magnitude curve, without changing the angle curve of G( j). This permits the designer to change the phase margin frequency , the phase margin angle , and the gain margin. For a stable closed-loop system, it is necessary for both the phase margin angle and the gain margin to be positive. Experimental Determination of Transfer Function The log-magnitudephase-angle diagram is of great value when the mathematical expression for the transfer function
Figure 17. Log-magnitudeangle diagram for a conditionally stable system: (a, c) stable; (b, d) unstable.
quency at which phase crossover occurs is called the gain-margin frequency c. Gain Margin. The gain margin is the factor a by which the gain must be changed in order to produce instability. Expressed in terms of the transfer function at the frequency c, it is |G(c )|a = 1 (103)
On the polar plot of G( j), the value at c, as shown in Fig. 15(a), is G( jc) 1/ a. In terms of the Lm in dB, the gain margin is Lm a = Lm G( j c ) (104)
This is shown in Fig. 15(b) as the amount by which the plot of G( j) must be raised so that it goes through the 0 dB, 180 (1 j0) point. These quantities are illustrated in Fig. 15 on both the polar and the Nichols plots. Note the algebraic sign associated with these quantities as marked on the curves. Both cases shown on Figs. 15(a) and 15(b) represent stable systems. The phase margin angle is the amount of phase shift at the frequency that would just produce instability. The phase margin for minimum-phase systems must be positive for a stable system, whereas a negative phase margin means that the system is unstable. It can be shown (8) that the phase margin angle is related to the effective damping ratio of the system. Satisfactory response is usually obtained with a phase margin of 45 to 60. As an individual gains experience and develops his own particular technique, the desirable value of to be used for a particular system becomes more evident. This guideline for system performance applies only to those systems where closed-loop behavior is that of an equivalent second-order system. The gain margin must be positive when expressed in decibels (greater than unity as a numeric) for a stable system. A negative gain margin means that the system is unstable. The damping ratio of the closed-loop system is also related
377
of a given system is not known. The magnitude and angle of the ratio of the output to the input can be obtained experimentally for a steady-state sinusoidal input signal at a number of frequencies. These data values are used to obtain the exact log-magnitudeangle diagram. Asymptotes are drawn on the exact log magnitude curve, using the fact that their slopes must be multiples of 20 dB/decade. From these asymptotes, the system type and the approximate time constants are determined. Thus, in this manner, the transfer function of the system can be synthesized (8, 24). Care must be exercised in determining whether any zeros of the transfer function are in the right-half s plane. A system that has no open-loop zeros in the right-half s plane is dened as a minimum-phase system (8). A system that has open-loop zeros in the right-half s plane is a non-minimum-phase system. The stability is determined by the location of the poles and does not affect the designation of minimum or non-minimum phase. The angular variation for poles or zeros in the right-half s plane is different from those in the left-half plane. For this situation, one or more terms in the transfer function have the form 1 Ts and/or 1 As Bs2. As an example, consider the functions 1 jT and 1 jT. The log magnitude plots of these functions are identical, but the angle plot for the former, as varies from 0 to , goes from 0 to 90, whereas for the latter it goes from 0 to 90. Therefore, care must be exercised in interpreting the angle plot to determine whether any factors of the transfer function lie in the right-half plane. CLOSED-LOOP FREQUENCY RESPONSE Specications for closed-loop system performance are often stated either in terms of the time response characteristics with a specied input or in terms of the frequency response. The time response gures of merit with a unit step input may include: peak overshoot, Mp; peak time, tp; nal value, yss; and settling time, ts. Additional gures of merit that may be specied include: system type, m; gain constant Km; and duplicating time, tD (time to rst reach the nal value). For systems of higher than second order, an effective damping ratio eff may be used to compare the response to that of a simple second-order system. The settling time ts is affected principally by the dominant closed-loop poles, with modifying effects due to the other poles and zeros. There is, therefore, an effective damping ratio eff . An alternate or supplementary way of identifying performance specications is in terms of frequency response characteristics. For a simple second-order system, frequency response characteristics consist of a set of gures of merit which are the maximum value Mm and the frequency m at which the maximum value occurs. The relationships for a simple second-order system are Mm = 1 , 2 1 2 m = n 1 2 2 (105)
Therefore, designing for given values of Mm and m results in effective values of eff and n. These, in turn, yield corresponding values of the time domain gures of merit which include Mp, d, and, for a 2% settling time, ts = 4 , n tp = n 1 2 (107)
When G( j) is plotted in a polar (Nyquist) plot, there exists a family of constant M closed-loop contours, where M C( j)/ R( j). These contours are a set of circles with specied centers and radii (8): x0 = M2 , M2 1 y 0 = 0, r0 = M M2 1 (108)
Therefore, the gain constant Km can be selected which makes the plot of G( j) tangent to a desired value Mm. In accordance with Eq. (105), this results in a corresponding set of values of and n. This produces a set of afliated values of Mp, d, ts, and tp, as given by Eqs. (106) and (107). The polar Nyquist plot therefore not only provides information on closed-loop stability, but also provides a design process for determining degree of stability. This degree of stability provides information on the time response characteristics. The constant M circles on the polar plot can be transformed to the Nichols (Lm versus angle plot) plot. Then, adjusting the gain Km raises or lowers the plot of G( j). This procedure can be used to make the plot of Lm G( j) tangent to the desired M curve on the Nichols chart. The frequency at the point of tangency determines the value of m. Equations (105)(107) determine the time response characteristics. NONLINEAR STABILITY AND CONTROL In this article the stability of dynamical systems is treated in a control theoretic context. Thus, the stability of feedback control systems is discussed. This is in the tradition of the classical Nyquist stability method in linear control theory covered in the preceding sections, where, based on information about the open-loop plant, the stability of the closed-loop feedback control system is being ascertained. In the remaining sections of this article the more realistic problem of the stability of constrained feedback control systems is considered. Specically, the stability of closed-loop feedback control systems which consist of linear plants driven by actuation elements which are subject to hard saturation constraints is addressed. Obviously, under conditions of small perturbations, the hard actuator constraints are not active and stability is then determined according to the rst method of Lyapunov using conventional linear analysis methods. Attention is now given to the treatment of high amplitude maneuvers and large perturbations away from trim, namely, the treatment of the stability in the large of control systems comprised of linear plants and actuators which are subject to saturation. Even though the primary interest is in stability, a complete analysis and a mathematically rigorous treatment of the stability problem in a control context is undertaken by embedding the stability problem into the more general framework of tracking control. Thus, the problem of tracking a zero exogeneous command signal naturally reduces to the investi-
These values are correlated to the peak value Mp and time response frequency of oscillation d in the time domain by M p = 1 + exp 1 2 , d = n 1 2 (106)
378
gation of the feedback control systems stability. Moreover, actuator saturation is addressed by utilizing a nonlinear dual-loop control architecture, where (internally modied) reference signals are generated by the feedback controller, even in the absence of a nonzero exogeneous command signal. Indeed, the feedback controller responds to the exogeneous reference signal and to the current state measurement; and the ill effects of saturation caused by large excursions in the state can be mitigated by injecting into an inner-loop linear controller a modied (nonzero) reference signal that tries to pull the control system out of saturation. This situation, where the state excursions are large, can arise even in the case where the exogeneous reference signal vanishes for a period of timein which case the feedback control system is, strictly speaking, a regulator. Finally, in the case where the feedback control system is nonlinear, the question of input/ output [viz., bounded input, bounded output (BIBO)] stability becomes dominant. Whereas it is well known that asymptotically stable linear control systems are BIBO stablethat is, stability with respect to perturbations in the initial state also guarantees the input/output stability of the control system, and vice versathe same cannot be said about nonlinear control systems. Hence, the response of the nonlinear control system to exogeneous reference signals needs to be considered and the broader issue of BIBO stability of tracking control systems must be addressed. In the remaining sections of this article, the design of nonlinear tracking controllers for the mitigation of actuator saturation effects is discussed. Evidently, the development of a nonlinear controller synthesis methodology which addresses the conicting requirements of high-amplitude dynamic reference signal tracking and regulation in the face of hard actuator displacement and rate constraints, actuator dynamics, and unstable open-loop plants is sorely needed. At the same time, in many applications (e.g., in ight control), full-state feedback can be assumed. Current analysis and design methods can address one, or some, of the above requirements, but, taken together, these requirements represent, at best, a difcult task, in particular with respect to satisfying the inherent tradeoffs between tracking control/regulation performance and closed-loop nonlinear system input/output and asymptotic stability. The objective here is to critically examine the state of the art and point the reader to a workable tracking control paradigm which, from the outset, also acknowledges actuator constraints. The adverse effects of controller-induced state and/or control constraint violation on closed-loop system performance and stability are well documented for both stable and unstable open-loop plants, namely, degraded system performance, limit cycling, and unstable system response. In addition, in feedback control systems with open-loop unstable plants (e.g., in modern ight control) and where feedback is used for stabilization, saturation can cause the feedback loop to be opened and immediately induce instabilityas opposed to more benign open-loop stable plants, where feedback does not play such a critical role. Moreover, in ight control, and for properly designed aircraft, actuator rate saturation usually occurs before actuator deection saturation. There is much interest with regard to constrained control in the recent literature, and actuator saturation is a topic of active research in control theory. Thus, numerous controller design and modication methodologies exist which address
linear time-invariant (LTI) systems with state and/or control constraints (11,2448). Much attention has been given to regulation and set-point control, as opposed to tracking control. Also, most of the work fails to address actuator rate saturation or, for that matter, actuator dynamics. Notable exceptions concerning tracking control are Refs. 30, 33, 34, 46, and 51, and rate saturation is explicitly considered in Refs. 30, 38, and 46. In most cases, open-loop stable plants are assumed and conditions are devised for global stability. In these investigations the compensator is allowed to send infeasible control signals to the actuator. In the landmark treatise (50), feedback control systems with amplitude constrained actuators and stable open-loop plants are considered. Necessary and sufcient frequency-domain conditions, which are a generalization of the Nyquist criterion, are given for global stability in the face of windup. Many of the proposed methodologies can be classied as anti-windup methods (26,35,44): The goal of anti-windup controllers is to avoid or delay actuator saturation-induced windup of linear dynamic compensation. While these methods may prevent or reduce windup and improve closed-loop system performance, they do not fully prevent constraint violation, namely, actuator saturation, which is the root cause of windup. Ad hoc anti-windup compensators (26,33,35,37,39,44,52), dont perform well in the case of openloop unstable plants with actuator displacement and rate constraints. Finally, unstable open-loop plants are specically addressed in Refs. 28, 30, 34, 46, and 51. State and control constraint mitigation strategies based on saturation avoidance focus on constraint violation prevention. Recent work on saturation avoidance methods has produced several important results based on the concepts of output admissibility and static admissibility which are applicable to the problem of tracking control (11,29,30,3449). Gilberts discrete-time reference governor (DTRG) (3132) is representative of these methods. Another interesting method is the linear quadratic tracking (LQT) concept (4042), which uses a receding horizon optimal control paradigm to avoid downstream state and control constraint violations. In this article, and motivated by problems in modern ight control, the synergetic consideration of stability and tracking control of open-loop unstable plants, with actuators subject to rate and amplitude saturation, using state feedback, is undertaken. Since Refs. 31, 32, and 4042 are representative of the state of the art with respect to constrained tracking control, their results are reviewed here. A viable feedback control concept is presented in the section entitled Control Concepts. The section entitled Linear Quadratic Tracking (LQT) discusses the LQT methodology for saturation avoidance and improved tracking performance developed in Refs. 4042. The section entitled Maximal Output Admissible Sets discusses the concept of maximal output admissible sets which play a crucial role in the synthesis of BIBO tracking control systems. The section entitled Discrete-Time Reference Governor is devoted to Gilberts DTRG, which is illustrated using a scalar example. In the section entitled Static Admissibility and Invariance the same scalar example is used to investigate more general invariance-based control methods with guaranteed BIBO stability and also yield improved tracking performance; some ne points are illustrated. A suboptimal approach is presented in the section entitled Suboptimal Approach, which is followed by concluding remarks in the section entitled Conclusion.
379
Limiter r N r kr + + uc u Plant = (A, B) x C y
kx
Figure 18. Current control system with open-loop limiter.
CONTROL CONCEPT Figures 1821 show examples of nonlinear control system architectures for linear systems with control constraints. Figure 18 shows the typical nonlinear control system architecture currently used in (ight control) practice: A simple open-loop limiter is used to restrict the reference signal and help mitigate the adverse effects of actuator saturationcaused windup. Figure 19 is a general nonlinear feedback control system architecture where the nonlinear controller, G, computes the commanded control signal based on the current plant state and the current, and possibly past, values of the reference signal. The controller is cognizant of the saturation level of the downstream actuator which is located at the plant input. Hence, the controller-generated signal, uc, may attain its limits, and by doing so the plant is driven to its full capacity. Figure 20 shows a specic example of a general nonlinear controllers architecture. A nonlinear dual-loop control system is envisaged where the controllers nonlinearity is conned to a feedback limiter, N, which performs a nonlinear scaling of the reference signal. The inner loop controller is linear and the nonlinear element N is contained in the outer loop, as illustrated in Fig. 20. Furthermore, the inner-loop linear controller is the result of a linear design, wherein it is assumed that the actuator operates in the linear regime. Hence, the inner loop is a linear control system. The nonlinear element N in the outer loop is a (pilot) command limiter which employs feedback. The nonlinear element N scales the reference signal such that the commanded control signal uc does not violate the actuator imposed control constraints in the inner loop: The feedback (pilot) command limiter generates a modied reference signal r which drives the inner loop in such a way that saturation in the downstream actuator is precluded, and strictly linear action ensues in the inner loop. The outer loops nonlinearity N renders transparent the actuators nonlinearity in the inner loop, and therefore the feedback control system shown in Fig. 20 is equivalent to the simpler feedback control system of Fig. 21, where the inner loop is linear. In conclusion, in Figs. 20 and 21 the linear inner loop controller is designed to meet small signal tracking and stability specications, and the nonlinear element N in the outer loop is designed to yield good tracking performance and
BIBO stability. Obviously, the design challenge is to maintain tracking performance and the BIBO stability guarantee of the dual loop nonlinear control system, for a relatively large class of exogeneous reference signals, and for a sufciently large set of initial states. Clearly, the control schemes shown in Figs. 20 and 21 are more advanced than the simple control scheme shown in Fig. 18, and are discussed in this article. LINEAR QUADRATIC TRACKING (LQT) It is convenient to address the actuator constrained tracking control problem using a time-domain, receding-horizon, optimal control formulation. Now, application of optimal control methods to the tracking problem requires a priori knowledge of the dynamic reference signal for all time. This also applies to linear quadratic (LQ) optimal control. Also, knowledge of the reference signal ahead in time is required, irrespective of whether the control signal is, or is not, constrained. Of course, this a priori knowledge does not exist, and in Refs. 4042, tracking control is addressed using reference signal extrapolation and implementing a receding-horizon optimal control methodology in which the exogeneous (pilot) reference signal is predicted over the optimization horizon, based on current and near past pilot reference signal values. Specically, polynomial extrapolation and interpolation is used to obtain the reference signal over the predetermined optimization horizon. Then, within each receding horizon window (RHW), the refer , is known, so that the LQ optimal control ence signal vector, r methodology may be applied to each of the nite-horizon control problems. The resulting nite-horizon LQ optimal control problem is solved within each RHW. In this way the indenite-horizon control problem is broken up into an open-ended sequence of nite-horizon control problems. In a discrete-time formulation the optimization window is of length N ( the number of samples within each RHW). Specically, an inner-loop linear quadratic controller, designed for good small-signal performance, minimizes a quadratic cost functional over the optimization horizon N. Thus, suppose that the bare plant is x(k + 1 ) = Ax(k ) + Bu(k ), x ( 0 ) = x0 , k = 0, 1, . . ., N 1
uc
Plant = (A, B)
x
Figure 19. Nonlinear control system.
380
G r + + uc
kr
Plant = (A, B)
x kx
Figure 20. Two-loop control concept.
and the tracked output signal is y(k + 1 ) = Cx(k + 1 ) The cost functional is
N
manded reference signal at time now, and is not a predicted value. Hence, in the current optimization window, LQ optimal control returns the optimal control time history u ( i ) = k i x x0 + k i r r 1 , i = 0, 1, . . ., N 111.1 (109)
J (u ) =
[Q(r(k + 1 ) y(k + 1 ))2 + Ru2 (k )]

k=0
In particular, at time 0, at the start of the window, u*(0) k0xx0 k0rr1. In the sequel, the notation used is kx k0 x , so that u ( 0 ) = k x x0 + k r r 1 (110) kr k0 r
The Q and R weights establish the tracking performance/control effort tradeoff in the inner (linear) loop and determine the small signal performance of the control system. Fullstate feedback is assumed. , LQ optimal control reFor a given reference sequence r turns the optimal control time history u* [u0 *, u* 1 , . . ., u* ], obtained within each RHW as a linear function of the N1 2, . . ., r [r1, r N], and the initial complete reference vector r plant state, x0, where r1 is the currently commanded reference signal. Now, polynomial extrapolation and interpolation 2, . . ., r N, the reference signals prediction, linear in yields r the data. Specically, if, for example, a simple ZOH extrapolation strategy is used, it is shown in Ref. 40 that the predicted 2, . . ., r is linear N] is linear in r1; that is, r reference vector [r in r1, which ultimately yields the optimal control sequence [u* 0 , u* 1 , . . ., u* N1] linear in r1, the current pilot-demanded reference signal, and x0. Note that r1 is the actual pilot de-
Both kx and kr are provided by the solution of the nite horin zon LQ optimal control problem. The row vector kT x R corresponds to the gain derived from the solution, over the nite optimization horizon, of the Riccati difference equation; this same Riccati equation is also associated with the solution of the nite horizon regulation problem. Moreover, one can deviate from the optimal tracking control solution in the RHW and instead, once kx has been determined, choose the gain kr such that asymptotic tracking of a step reference command r is enforced. Of course, if integral action is used to obtain a type-one system, then asymptotic tracking is achieved for any kr.
G r + + uc = u
kr
Plant = (A, B)
x kx
Figure 21. Saturation mitigation concept.
381
Furthermore, since the optimal control time history u* is a linear function of r1 and x0, the actuator amplitude constraints of, say, umax, can be readily transformed into constraints on the reference signal at time now, r1: In the RHW umax ki x x0 + ki r r1 umax , i = 0, 1, . . ., N 1
tion algorithm is reapplied and the open-loop LQ optimal control problem is solved from the beginning. Hence, r1 (or r1 ) is the only reference value actually tracked; and within each RHW, u* 0 is the only control which is actually applied to the system, so that feedback action is achieved. Now, the windows local time instant 0 corresponds to the current time k. Thus, at time k, the optimal control signal u* k satises u (k ) = kx x (k ) + kr r (k + 1 ) and since the nonlinear element N in the outer loop causes 1, the exogeneous reference signal rk1 to be replaced by rk which is then sent to the inner loop, the actual optimal control signal is u (k ) = kx x (k ) + kr r (k + 1 ) (112)
there is nothing that can be done about the plants initial state x0. Hence, for example, for kr 0 the above inequalities yield the explicit constraints on the exogeneous reference signal:
0iN 1
max
umax ki x x0 ki r
r1
0iN 1
min
umax ki x x0 ki r
Next, differencing Eq. (109) yields an explicit expression for the actuator rate, namely,
u i
1 (u ui 1 ) T i 1 ( k x + k i r r 1 k i 1 x x0 k i 1 r r 1 ) = T ix 0 1 [ ( k i x k i 1 x )x 0 + ( k i r k i 1 r )r 1 ] = T
Similar to the derivation for actuator amplitude constraints, the above equations are manipulated to yield an additional explicit bound on the reference signal at time now, r1,
Many of the constraints on rk1 are induced by constraints on k2, subsequent elements of the predicted reference vector (r k3, . . ., r kN). But these reference signals are not actually r realized, so enforcement of these downstream constraints may lead to an overly conservative scaling of rk1. Thus, the designer may wish to specify feasibility criteria which do not necessarily include all the constraints in the RHW. An important special case entails the enforcement of actuator constraints at time now only, whereupon the inequalities from above are reduced to
r1 max
r1
1iN 1
max
Tu max (ki x ki 1 x )x0 ki r ki 1 r
umax ki x x0 u k x x0 , 1 ki r kr
Tu max Tu max (113)
, Tu max
u 1 k x x0 kr r1 min Tu max (ki x ki 1 x )x0 ki r ki 1 r ,
umax ki x x0 u k x x0 , 1 r1 min ki r kr
1iN 1
u 1 k x x0 + kr
Tu max (111)
Finally, if rk1 satises the applicable constraints, then N is transparent and clearly rk 1 rk1, that is, tracking performance during small-signal operation is not sacriced. Then, the closed-loop linear system, which translates the exogeneous r command into aircraft responses, is given by x(k + 1 ) = Acl x(k ) + Bcl r(k + 1 ) y(k + 1 ) = Cx(k + 1 ) where Acl A Bkx, Bcl Bkr, and the actuator displacement is u (k ) = kx x (k ) + kr r (k + 1 ) In general, the nonlinear LQT control law depends on (1) the choice of the Q and R weights for small-signal performance, (2) the reference vector prediction method, (3) the selection of the applicable feasibility criteria for saturation avoidance, and (4) the modied reference signal optimization criterion. So far, only saturation avoidance was considered for modied reference signal feasibility determination. BIBO stability is only guaranteed under certain additional restrictive conditions. Thus, the design for guaranteed BIBO stability of the control system shown in Figs. 20 and 21 is now undertaken. MAXIMAL OUTPUT ADMISSIBLE SETS In Ref. 32 the concept of maximal output admissible set is employed to develop reference governors N for discrete-time
where u* 1 is the control signal from the last window. Hence, actuator constraints are easily transformed into constraints on the current reference signal. Saturation avoidance is guaranteed, provided that the reference signal at time now, r1, satises the above inequalities. Actuator displacement and rate constraints impose constraints on the inner closed-loops exogenous reference signal. Thus, an additional outer control loop is employed which performs a nonlinear modication of the reference signal from the pilot, such that the downstream actuator constraints in the inner loop are satised. Now, if r1 causes the violation of any of the constraints, it may be optimally scaled such that the modied reference signal, r1 , satises all constraints; here, optimally scaled means that in the outer loop r1 is chosen so that, for example, r1 r1 is minimized, subject to r1 satisfying the actuator constraints. In accordance with the receding-horizon modus operandi, at each time step the optimization window is shifted forward in time, a new state is attained, and upon receipt of a new reference signal from the pilot the reference signal extrapola-
382
systems. The discrete-time reference governor (DTRG) developed in Ref. 32 is a nonlinear dynamical element, that is, it is a rst-order lag lter with a variable bandwidth parameter, [0, 1], which scales the reference signals increments so that the controlled systems constraints are not violated. These constraints characterize the maximal output admissible set. By forming a modied reference signal such that the discrete-time systems state update satises these constraints, both saturation avoidance and BIBO stability are enforced: BIBO stability also requires that the maximal output admissible set be bounded. In the sequel, the DTRGbased approach to tracking control is discussed. More importantly, the construction of the maximal output admissible set is presented. This is a critical step in the development of more general BIBO tracking control systems, in both the discretetime and the continuous-time settings. The LTI discrete-time system
Example 2. The Set Y is a Cone. This is the situation where there are one-sided control constraints, for example, 0 u(t)in which case C nonnegative orthant in m set of vectors in m with non-negative entries. Then O (Acl , Ccl , Y ) = {0} iff 1. The pair (Acl, S) is observable, where the n-column matrix S is such that Sx 0 implies that Cclx Y, and 2. Acl does not have an eigenvector v which corresponds to a nonnegative eigenvalue of Acl such that Cclv Y. For a proof, see, for example, Ref. 45. Of major interest is the case where the constraint set Y is a polyhedron, namely, Y = {y
p
x(t + 1 ) = Ax(t ) + Bu(t ), x (0 ) = x u(t )

m n
: f i ( y ) 0,
i = 1, 2, . . ., s}
(118)
(114)
t = 0, 1, 2, . . . ( = I + )
That is, the s functions f i(y), f i : p are linear in y, with f i(0) 0. Obviously O (Acl , Ccl , Y ) = {x
n
is considered. Initially, the regulation problem is exclusively analyzed and the prespecied small-signal linear-state feedback control law is u(t ) = kx x(t ) Also, assume there are constraints on both the state and control vectors, including linear combinations of them. Then, with appropriate choices of matrices C and D, along with a set Y, these rather general constraints may be represented as y(t ) = Cx(t ) + Du(t ) Y
p
: f i (Ccl At cl x ) 0,
i = 1, 2, . . ., s, and t I + }
(119)
(115)
Let Acl A Bkx and Ccl C Dkx. Then, Eqs. (114) and (115) become x(t + 1 ) = Acl x(t ), x(0 ) = x,
p
t I+
y(t ) = Ccl x(t ) Y
(116)
Although Y is a polyhedron, Eqs. (117) and (119) each represent an innite number of constraints. However, if there ex ists a nite t* i I for each inequality in (119) for which the constraints associated with the ith inequality constraint are inactive for t t* i , let t* 1 i s maxt* i . Then O(Acl, Ccl, Y) is characterized by a nite number (st*) of inequality constraintsin this case, O(Acl, Ccl, Y) is said to be nitely determinedand determination of whether a particular initial state vector, x, is an element of O(Acl, Ccl, Y) involves evaluation of a nite set of linear inequalities. Moreover, it may transpire that the constraints associated with one or more of the inequalities, f i, that dene Y are inactive for all t I. Thus, let S* denote the set of inequality constraints that are active for some t I, namely, S* 1, 2, . . ., s. Then, O(Acl, Ccl, Y) may be written as O (Acl , Ccl , Y ) = {x
n
Hence, the saturation avoidance problem has been transformed into a feasibility problem consisting of an unforced LTI discrete-time system with an output constraint. In Ref. 31 the maximal output admissible set associated with system (116) is dened as the set of all initial states x n such that the unforced closed-loop linear systems response does not violate the system output constraints for all t I. Thus, the maximal output admissible set is the largest set of initial states, x n, that is positively invariant w.r.t. system [see Eq. (116)]. Evidently, the maximal output admissible set is O (Acl , Ccl , Y ) = {x
n
: f i (Ccl At cl x ) 0,
t = 0, 1, . . ., ti , and i S }
(120)
: Ccl At cl x Y
t I+ }
(117)
Example 1. The Output Constraint Set Y 0. The maximal output admissible set is then the subspace of unobservable states that corresponds to the pair (Acl, Ccl), namely, O = (N(Ccl ))A
cl
This is the largest subspace contained in the null space of the matrix Ccl which is invariant under Acl.
In the discrete-time case under consideration, sufcient conditions for the existence of a nite t*, namely, sufcient conditions for the nite determination of O, are as follows: (1) Acl is asymptotically stable, (2) 0 int(Y), (3) Y is bounded, and (4) the pair (Acl, Ccl) is observable. Here, int( ) denotes the interior of the set. The concept of maximal output admissible sets is only applicable to unforced systems. Thus, it is not directly applicable to the tracking problem without some modication. In Ref. 32 the concept of maximal output admissible sets is adapted for use with the tracking problem by using a nonlinear element in the DTRG which has rst-order dynamics; that is, a rst-order lag lter with a variable bandwidth parameter [0, 1] is used to prelter the exogenous reference signal. The closed-loop system state vector is then augmented with the prelter state, namely, the modied reference signal, r. The end result is an augmented system whose exogenous input
383
can be turned off by setting 0. Thus, consider the tracking control problem x(t + 1 ) = Ax(t ) + Bu(t ), u(t ) = kx x(t ) + kr r(t ) with state and control constraints y(t ) = Cx(t ) + Du(t ) Y
p
x(t )
, u(t )
(121)
(122)
and where the control constraint set Y is given by Eq. (118). The ensuing closed-loop system is x(t + 1 ) = Acl x(t ) + Bcl r(t ) y(t ) = Ccl x(t ) + Dcl r(t ) Y
p
(123)
where, as before, Acl A Bkx, Ccl C Dkx, and Bcl Bkr, Dcl Dkr. To transform this problem into one which allows use of the concept of maximal output admissible sets, the exogenous reference signal, r, is preltered by the rstorder lag lter given by r (t + 1 ) = r (t ) + (r(t ), xg (t )) r(t ) r (t ) (124)
inequality constraintsthen at each time-step the upper limit imposed on the scalar (r(t), xg(t)) by each linear inequality constraint is given by a simple algebraic formula. Thus, at each time-step (r(t), xg(t)) [0, 1] may be chosen so that it satises the minimum of the upper limits and is easily computed on-line. Hence, a tracking controller could easily be synthesize; moreover, the output vector is guaranteed to be bounded. Unfortunately, application of the concept of maximal output admissible sets to the tracking control problem generally results in an augmented dynamics matrix Ag which is only Lyapunov stable. Thus, O(Ag, Cg, Y) is generally not nitely determined. The reason O(Ag, Cg, Y) is not nitely determined stems from the fact that the unforced response of a Lyapunov stable linear system does not decay to the origin, unlike that of an asymptotically stable linear system. In fact, the unforced response of the Lyapunov stable linear system does converge to an a priori unknown equilibrium point. In general, the equilibrium point will not be reached in a nite number of time-steps, and thus t* cannot be bounded. The following will be useful in resolving this problem. The set of statically admissible inputs, Rs, for the system dened by Eq. (123), is the set of constant inputs for which the associated equilibrium point is admissible. That is, R s = {r
m
where (r(t), xg(t)) [0, 1]; and in Eq. (119) r(t) is replaced by the lters output r(t) (the modied reference signal). Then, the augmented state is
: yss Y }
xg =
r x
(125)
where yss is the steady-state response. yss Hor, where Ho Ccl(I Acl)1 Bcl Dcl. Hence, an equivalent expression for Rs is R s = {r
m
: Ho r Y }
(127)
and the augmented system dynamics and the output constraints are given by
xg (t + 1 ) = Ag xg (t ) + Bg (r(t ), xg (t )) r(t ) [I y(t ) = Cg xg (t ) Y

where
p
0]xg (t ) (126)
The set of statically admissible states, Xs, for the closedloop system dened by Eq. (123), is the set of initial states for which there exists a statically admissible reference signal such that the ensuing trajectory does not violate the systems state and control constraints for all time. That is, Xs = {x
n
: r Rs
s.t.
y(t ) Y
t I+ }
(128)
Ag =
I Bcl
0 , Acl
Bg =
I , 0
Cg = [Dcl
Ccl ]
For the augmented system dened by Eq. (126), the set of statically admissible states, Xgs, is Xgs = O (Ag , Cg , Y ) (129)
From Eq. (124) notice that if (r(t), xg(t)) 1, the exogenous reference signal, r(t), is passed through unmodied, but with a one time-step delay. More importantly, if (r(t), xg(t)) 0 the current modied reference signal, r(t), remains unchanged, and Eq. (126) becomes an unforced system with output constraints. Thus, if at time t and for all initial states xg() O(Ag, Cg, Y), (r(), xg()) can be chosen such that the updated state xg( 1) O(Ag, Cg, Y), it is possible to guarantee that the controlled systems state and control constraints will not be violated in the future. Moreover, assume that O(Ag, Cg, Y) is bounded. Then because 0 is always an option and because the new reference signal is chosen such that it does not remove the state of the plant from the compact set of admissible states, the BIBO stability of tracking control systems which employ the DTRG is guaranteed. If Y is given by Eq. (118) and O(Ag, Cg, Y) is nitely determinedthat is, it is characterized by a nite set of linear
Similar to the maximal output admissible set, both Xs and Xgs are positively invariant sets. Xs is the projection of Xgs onto the plane r 0. Now, recall that the reason O(Ag, Cg, Y) is not nitely determined is that an innite number of timesteps is required to reach the unknown equilibrium point. However, if it is somehow known that the unforced system given by Eq. (126) will converge to an equilibrium point xg int(Xgs), then only the transient response for a nite number of time-steps needs to be considered, until the peak-to-peak magnitude of constrained quantity oscillations decay sufciently. If r is restricted to values in int(Rs) and if x(0) Xs, then x(t) Xs and xg(t) int(Xgs) for all t I. Moreover, the set int(Xgs) int( O(Ag, Cg, Y)) is nitely determined. Now, dene Y() Y by Y() y : f i(y) , i 1, , s, where 0 minf j(0): i 1, , s. Then, to restrict r to values in int(Rs), which results in xg int(Xgs), append
384
the additional constraints f i ([Ho 0]xg ) , i = 1, . . ., s (130)
cable constraints are x (k + 1 ) = ad x (k ) + b d r (k ) y(k ) = u(k ) = cx(k ) + dr (k ) Y R Y = {y : f i ( y ) 0, i = 1, 2} (137) (138) (139)
to those that dene O(Ag, Cg, Y). Denoting int(Xgs) by X gs yields
Xgs = {xg
n+m
: f i (Cg At g xg ) 0, t = 0, . . ., ti , i S ,
and f j ([Ho
0]xg ) , j = 1, . . ., s}
(131)
where ad 0.999, bd 9.995 104, c kx 3, d kr 1, f 1(y) y 1, and f 2(y) y 1. r(k) is the feasible reference signal. The reference governor is r (k + 1 ) = r (k ) + (k )[r(k ) r (k )] (140)
Now, X gs is characterized by a nite set of inequality constraints. Moreover, the upper limit imposed on (r(t), xg(t)) [0, 1] by each inequality, so that xg(t 1) X gs, is given by a simple formula. Thus, if xg(0) X gs, and at each time-step we choose (r(t), xg(t)) [0, 1] such that it satises the minimum of the upper limits imposed by all inequality constraints in Eq. (131), then xg(t) X gs for all t I . The nite determination of the maximal statically admissible (invariant) set is exclusively an artifact of discrete-time dynamics. The maximal output admissible invariant sets of continuous-time systems are not polyhedral. DISCRETE-TIME REFERENCE GOVERNOR
Combining the closed-loop system and reference governor dynamics results in the augmented second-order system xg (k + 1 ) = Ag xg (k ) + Bg (k )(r(k ) [1 0]xg (k )) y(k ) = Cg xg (k ) Y where the augmented state is (141) (142)
xg ( k ) = and
The concept of a statically admissible set and the discretetime reference governor (DTRG) are illustrated. Specically, a tracking controller is synthesized for a constrained scalar control system. The continuous-time control system is given by x = ax + bu, x ( 0 ) = xo , 1 u 1 (132)
r (k ) x (k ) 0 0.999 3]
Ag = Bg =
1 9.995 10 4 1 , 0
Cg = [1
and the small-signal, linear control law is u = kx x + kr r Combining Eqs. (132) and (133) gives x = acl x + bkr r , acl = a + bkx (134) (133)
Now the maximal output admissible set for the augmented system is concerned with the homogeneous system [(k) 0] and is dened as O = {x
: f i (Cg Ak g xg ) 0, k = 0, . . ., ki , i S }
(143)
Now, choose kr such that asymptotic tracking is enforced, namely, kr = a a kx = cl b b
k xg) 0, are inactive for k k* where the constraints, f i(CgAg i and i S* 1, 2. Since the augmented system is only Lyapunov stable, k* i may be unbounded, so a nitely determined approximation, O , to the maximal output admissible set is needed. As noted in the section entitled Maximal Output Admissible Sets, O is obtained by restricting the feasible input, r, to values that are statically admissible with respect to the reduced constraint set Y() y : f i(y) , i 1, 2, where 0 minf i(0): i 1, 2. Thus, the additional constraint is added:
Then, the inner loop in Figs. 20 and 21 is x = acl (x r ) and a u = kx x cl r b (136) Then, (135) where
r W0 = {r : H0 r Y ( )}
(144)
H0 = d + c(1 ad ) 1 bd = 1.9985
(145)
Application of the DTRG to the system represented by Eqs. (132) and (133) requires the consideration of the equivalent discrete-time closed-loop system. For example, with a 2, b 1, kx 3, kr 1, and a sampling interval of T 0.001 s, the equivalent discrete-time closed-loop system and the appli-
O = {x f j ([H0
: f i (Cg Ak g xg ) 0, k = 0, . . ., ki , i S ,
0]xg ) , j = 1, 2}
(146)
and k* i is guaranteed to be bounded (32). Notice that O is k composed of two sets of constraints. The rst set ( fi(CgAgxg)
385
0, k 0, . . ., k* i , i S*) deals with saturation of the transient response, and the second set ( fj([H0, 0]xg) , j 1, 2) deals with saturation in steady state. Now, Algorithm 3.2 of Ref. 31 may be used to determine k* i and S*. In this case S* is empty. That is, the inequalities associated with the transient saturations are inactive. This should be expected because the closed-loop system of Eqs. (137) and (138) is a stable rst-order (over damped) system. Thus, Eq. (146) becomes O = { f j ([H0 0]xg ) , j = 1, 2} (147)
over, the DTRG is now replaced by a static nonlinearity N whose synthesis is outlined. The exibility afforded by this less conservative and more general approach makes it possible to consider two separate tracking control concepts. The rst tracking control concept is minrr r subject to the control constraints. The second control concept for the selection of r attempts to drive the output x to r as quickly as possible, subject to the control constraints. A continuous-time derivation is given. Control Concept 1: Choose r such that r r is minimized subject to the control constraints. Assuming kx 0, acl 0, and b 0 results in the explicit (nonlinear) control law b bkx b bkx r x+ r x if a a a a cl cl cl cl bk b bkx b x x if r x r (x, r ) = acl acl acl acl b bkx b bkx if r x+ a x+ a a a
cl cl cl cl
Now, with 0.05, Eqs. (3.10) and (3.15) of Ref. 32 may be used to generate (k), namely, (k ) = min{1 (k ), 2 (k )} where (148)
min 1, 0.95 H0 r (k ) H0 (r(k ) r (k )) 1 (k ) = 1,
if H0 (r(k ) r (k )) > 0 if H0 (r(k ) r (k )) 0 (149)
(152)
and substituting Eqs. (152) into (135) results in the closedloop system bkx b bkx b x+ r x acl (x r ) if a a a a cl cl cl cl bkx b if r x x = ax + b (153) acl acl bkx b if r x+ ax b acl acl The saturation avoidance control law (152) does not guarantee BIBO stability in the case of an open-loop unstable plant. However, the desired BIBO stability can be obtained through an additional invariance requirement. A bounded set XI X x: x is invariant with respect to the system given by Eqs. (132), (133), and (153) if and only if on the boundary of XI, xx 0. Thus, if a bounded invariant set is characterized as XI = x : r such that 1 u = kx x acl r 1 and xx 0 b
and
min 1, 0.95 + H0 r (k ) H0 (r (k ) r(k )) 2 (k ) = 1
if H0 (r (k ) r(k )) > 0 if H0 (r (k ) r(k )) 0 (150)
In this case, the end result of Eqs. (148)(150) is to limit the feasible reference signal such that 0.47536 r 0.47536 (151)
While the above reference governor avoids saturation and affords BIBO stability, it is somewhat conservative in that it restricts the feasible reference signal, r, such that at all times it is always statically admissible. The BIBO stable tracking controller developed in Refs. 4042 employs a static nonlinearity N; it is nevertheless related to Gilberts DTRG (32) in that it also includes static admissibility of the modied reference signal in the feasibility criteria. However, there are certainly cases where a modied exogenous reference input that is not statically admissible over a nite time interval would not necessarily result in saturation. Thus, as discussed in the next section, the achievable tracking performance can be enhanced, while at the same time the constraint violation is avoided and BIBO stability is guaranteed. STATIC ADMISSIBILITY AND INVARIANCE The scalar constrained control system is revisited and the application of statically admissible and more general (maximal output admissible) invariant sets to achieve saturation avoidance and BIBO stability enforcement is demonstrated. The construction of these invariant sets is a crucial step in the design of tracking control laws. Insights into the critical role played by the above-mentioned invariant sets permit focusing on design for tracking performance, while at the same time guaranteeing the BIBO stability of the control system. More-
then, by restricting x to x XI, BIBO stability is guaranteed. First, consider the case of an open-loop unstable plant, a 0. For (bkx / acl)x b / acl r (bkx / acl)x b / acl, the result is r r, and x acl(x r). Hence, r x results in x 0, and r x results in x 0. Second, for r (bkx / acl)x b / acl we have x ax b. In this case x b / a results in x 0, and x b / a results in x 0. Finally, for r (bkx / acl)x b / acl we have x ax b. Therefore, x b / a results in x 0, and x b / a results in x 0. This is summarized in Fig. 22, where the directions of the arrows represent the sign of x in the Cartesian product space, V, dened by V = {v
2
: v = [r, x]T , r, x
From Fig. 22 and the above discussion it is clear that if the control law allows x b / a the system will diverge due to the constraint on the control signal, u. Also, if the system ever
386
kx x b acl b a x=r b a 1 kx
acl r=1 b
subject to the control constraints in Eq. (133). Thus,

a kx x cl r = 1 b
If r > x, then r max(x ) ) If r < x, then r min(x If r = x, then r min(x 2 )

From Eq. (135) and the assumption that acl 0, this results in
1 kx
b a
If r > x, then max r If r < x, then min r If r = x, then min (x r )2 (156) bkx b bkx b x+ r x acl acl acl acl
b a b acl
subject to
Figure 22. Cartesian product space, V, showing invariant region for the scalar system.
achieves either of the equilibrium points v [b / a, b / a]T, it becomes stuck. Thus, in addition to control law (152), there must be a limit on x such that x XI = x : b b + x a a (154)
Hence, the explicit control law is nally obtained bkx b x , if r > x a a cl cl b bkx x+ , if r < x a a cl cl b b if r = x and x r = r a a bkx b b if r = x and x < a x a a cl cl bkx b b if r = x and x > a x+ a a
cl cl
(157)
then saturation will be avoided, and the system output will be bounded. Hence, the control law [see Eq. (152)] is modied as follows:
r b a r (x, r ) = b + a bk x x a cl bk x a x+
cl
b acl b acl
bkx b bkx b x+ r x acl acl acl acl b b and + x a a b b if x > and r a a b b if x < + and r + a a bkx b b if r > x and x acl acl a bkx b b if r x+ and x + acl acl a (155) if
and substituting Eq. closed-loop system, ax + b ax b x = 0 ax + b ax b
(157) into Eq. (135) results in the
if r > x if r < x b b x a a b if r = x, and x < a b if r = x, and x > a if r = x, and (158)
Now, under the explicit (nonlinear) control law [see Eq. (155)], x XI is enforced, the system will not become stuck at x b / a, and saturation avoidance plus invariance in a bounded subset of the state space yield the BIBO stability guarantee. A similar analysis for the case of an open-loop stable plant (a 0) shows that the saturation avoidance set is XI 1. Hence, the simpler control law [see Eq. (152)] yields global BIBO stability in this case. Control Concept 2: Choose r so as to either maximize or minimize x , based on the sign of r x, and subject to the control constraints, therefore maximizing the instantaneous reduction in tracking error. Also, if r x, then minimize x 2,
If the open-loop system is stable (i.e., a 0), then control law (157) results in a globally BIBO stable closed-loop system. However, if the open-loop system is unstable (a 0), then, as before, x must be restricted such that x XI, where XI is given by Eq. (154). In this case, control law [see Eq. (157)] is modied as follows: bkx b b x if r > x, and x acl acl a b b bkx if r < x, and x + a x+ a a cl cl b b (159) r = r, r = x if + x a a b b if r > x, and x > a a b b + if r < x, and x < + a a Now, x XI is enforced, the system will not become stuck at x b / a, and BIBO stability is achieved.

0.5 0.45 0.4 0.35 r(t) = h(t 0.5) h(t 1.5) 0.3 x 0.25 DTRG CC1 CC2
387
The scalar controlled output variable y must track the exogeneous reference signal r. The point of departure is LQT. Hence, a tracking linear control law is specied; the simplest 1 such control law is u kxx krr, where kr 1/ cA cl b and Acl A bkx. Hence, u = kx x and the linear inner-loop is 1 r 1b cA cl
0.2 0.15 0.1 0.05 0 0 0.5 1 1.5 2 2.5 Time (sec) 3 3.5 4
x = Acl x
1 br 1b cA cl
The set of rest points of the closed-loop system is the one1 1 dimensional subspace x (1/ cA cl b)Acl br, r . Now, 1 u 1 yields the constraint on statically admissible r:
Figure 23. Comparison of responses to a statically inadmissible input.
1 b1 kx A cl 1b cA cl
r1
Notice that while it is necessary to enforce x XI for the above control laws when the open-loop plant is unstable, neither Eq. (155) nor Eq. (159) requires that r be statically admissible; that is, r b / a is allowed when x int(XI). This is in contrast to Gilberts DTRG, and the globally BIBO stable LQT controller constructed in Refs. 32 and 40, which restrict the feasible reference signal to statically admissible values. By relaxing the requirement that the modied reference signal be statically admissible, the tracking performance is enhanced. Simulation results are shown in Figs. 23 and 24. These concepts are applied to higher-order plants and ight control in Refs. 42, 43, and 49. SUBOPTIMAL APPROACH A single-input continuous-time control-constrained plant is considered: x = Ax + bu, y = cx 1 u 1
Thus, the set of statically admissible reference signals is 1 1 rs r rs, where rs cA cl b /(1 kxAcl b), and the set of admissible rest points is the segment Xr = {x | x = 1 A 1 br, rs r rs } 1 b cl cA cl
BIBO stabilty enforcement requires the construction of certain invariant sets in the state space. The maximal statically admissible set Xs is characterized as follows: The rest state which corresponds to the constant reference signal r is xr = 1 A 1 br, 1 b cl cA cl rs r rs
It is convenient to use the perturbation state x : e; x xr and scale r : e r / rs whereupon the dynamics are transformed into x = Acl x (160)
and the saturation avoidance constraint is transformed into 1 + r kx x 1 + r (161)
0.45 0.4 0.35 0.3 x 0.25 0.2 0.15 0.1 0.05 0 0 0.5 1 1.5 Time (sec) 2 2.5 r(t) = 0.3h(t 0.5) 0.3h(t 1.5) DTRG CC1 CC2
Now, x r, rs r rs, and determine the largest set contained in the slab in Eq. (161) which is invariant under Eq. (160); denote this maximal output admissible set by Xs(r). Finally, the maximal statically admissible set Xs is Xs = 1r1 Xs (r ) Proposition. The set Xs Rn is bounded iff the pair (A, kx) is observable. Remark. The solution of well-posed inner-loop optimal control problems automatically renders the pair (A, kx) observable. Hence, the set Xs will be bounded and BIBO stability can be enforced. In general, the maximal statically admissible set Xs is compact and convex. However, Xs is not polyhedral; that is, it is
Figure 24. Comparison of responses to a statically admissible input.
388
not nitely determined and, moreover, is rather difcult to construct. In the two-dimensional case (n 2), it is possible to obtain a closed-form representation of Xs: The boundaries of the r cross-sections Xs(r) are trajectories of the unforced system; the boundaries of Xs are the envelopes of the above mentioned family of trajectories. For the control constrained system,
It is now fairly easy to synthesize the outer-loop nonlinearity N, namely, the nonlinear control law r r (x, r). For example, consider the Control Strategy 1: r is chosen to minimize r r, subject to the saturation avoidance constraint 1 kx x provided that xT Px < 1 k x P 1 k T x 1 r 1 1b cA cl
x (t ) =
0 2
1 0 x(t ) + u(t ) 1 2 2.5]x(t ) + 8r(t ) 1
y(t ) = [1 0]x(t ) 1 u(t ) = [9

For x such that
Xs is shown in Fig. 25. In higher dimensions, suboptimal (i.e., smaller statically admissible or invariant) sets, which are easy to characterize, are certainly useful. In engineering applications, guaranteeing BIBO stability for a reasonably large but bounded set of initial states which contains the origin is often sufcient. In this respect, the following holds. Let P be the real symmetric positive denite solution of the Lyapunov equation AT cl P + PAcl = Q
xT Px =
1 k x P 1 k T x
r must satisfy the additional invariance-enforcing inequality r 2 bT Px xT Qx 1b cA cl
where Q is a somewhat arbitrary, real, symmetric, and positive denite matrix. The ellipsoid Es = x | xT Px 1 k x P 1 k T x
Feasibility is guaranteed, by construction. Hence, a dual-loop tracking controller as shown in Fig. 20 can be constructed. For all measurable exogeneous reference signals r and for all initial states in the ellipsoid Es, good small-signal performance is achieved, high-amplitude signals are optimally tracked, and BIBO stability is guaranteed. CONCLUSION The basic concept of the stability of dynamical systems is carefully developed, and its application to the design of feed-
is statically admissible. Furthermore, Xr Es, provided that 1 2 1 T T 1 T (cA cl b) (kxP kx )(b (Acl ) b).
Xs = U(Xs (r)) for rs r rs 1
0.5
1.5 1
X2
0.5
0.5
X2
0.5
1
0.5
0 X1 (a)
0.5
1.5 1.5
0.5
0 X1 (b)
0.5
1.5
Figure 25. (a) Maximal statically admissible set. (b) Comparison of Xs and the suboptimal ellipse.
389
back control systems is presented. Classical stability theories for linear and nonlinear feedback control systems are discussed and the Lyapunov and RouthHurwitz stability criteria are developed. Linearization of nonlinear dynamics in terms of the Jacobian matrix is presented. Attention is given to conventional control design and to frequency-domain methods: The root locus and Nyquist stability criteria are developed and, in addition, the Bode plot and Nichols chart-based methods for stability, and degree of stability determination, are discussed. The emphasis then shifts to the stability of control systems comprised of linear plants and actuators which are subject to saturation. The state of the art with respect to nonlinear controller synthesis methods that address tracking control in the presence of actuator constraints is examined. The saturation avoidance approach is pursued. Special attention is given to tracking performance and to the BIBO stability of the closed-loop nonlinear control system. Results based on the concept of invariant setsnamely, maximal output admissible sets, maximal statically admissible sets, and the linear quadratic tracking (LQT) control methodare presented. These results are applied to the synthesis of a dual-loop nonlinear control structure. The on-line computational requirements are modest. Good tracking performance of highamplitude command signals is achievable, small-signal performance is preserved, and BIBO stability is guaranteed provided that the command signals are sufciently smooth, and the initial state is contained in a predetermined invariant set. BIBLIOGRAPHY
1. B. Porter, Synthesis of Dynamical Systems, London: Thomas Nelson, 1969. 2. W. J. Rugh, Linear System Theory, Englewood Cliffs, NJ: Prentice-Hall, 1993. 3. J. J. DAzzo and C. H. Houpis, Linear Control System Analysis and Design, 3rd ed., New York: McGraw-Hill, 1988. 4. F. Csaki, Modern Control Theories, Budapest: Akademiai Kiado, 1972. 5. T. Kailath, Linear Systems, Englewood Cliffs, NJ: Prentice-Hall, 1980. 6. H. M. James, N. B. Nichols, and R. S. Phillips, Theory of Servomechanisms, Radiation Laboratory Series, Vol. 25, New York: McGraw-Hill, 1974. 7. J. E. Gibson, Nonlinear Automatic Control, New York: McGrawHill, 1963. 8. J. J. DAzzo and C. H. Houpis, Linear Control System Analysis and Design, 4th ed., New York: McGraw-Hill, 1995. 9. K. N. Swamy, On Sylvesters criterion for positive semidenite matrices, IEEE Trans. Autom. Control, AC-18: 306, 1973. 10. K. Ogata, Modern Control Engineering, Englewood Cliffs, NJ: Prentice-Hall, 1970. 11. N. Minorsky, Theory of Nonlinear Systems, New York: McGrawHill, 1969. 12. R. E. Kalman and J. E. Bertram, Control system analysis and design via the second method of Liapunov, J. Basic Eng., 80: 371 400, 1960. 13. E. A. Guillemin, The Mathematics of Circuit Analysis, New York: Wiley, 1949. 14. V. Singh, Comments on the Routh-Hurwitz Criterion, IEEE Trans. Autom. Control, AC-26: 612, 1981.
15. B. Porter, Stability Criteria for Linear Dynamical Systems, New York: Academic, 1968. 16. K. J. Khatwani, On Routh-Hurwitz criterion, IEEE Trans. Autom. Control, AC-26: 583584, 1981. 17. S. K. Pillai, On the -method of the RouthHurwitz criterion, IEEE Trans. Autom. Control, AC-26: 584, 1981. 18. D. M. Etter, Engineering Problem Solving with MATLAB, Engelwood Cliffs, NJ: Prentice-Hall, 1993. 19. V. Krishnamurthi, Implications of Routh stability criteria, IEEE Trans. Autom. Control, AC-25: 554555, 1980. 20. F. R. Gantmacher, Applications of the Theory of Matrices, New York: Wiley-Interscience, 1959. 21. MATLAB Version 4, Natic, MA: The MathWorks, Inc., 1993. 22. H. Nyquist, Regeneration Theory, Bell Syst. Tech. J., 11: 126 147, 1932. 23. H. W. Bode, Network Analysis and Feedback Amplier Design, Princeton, NJ: Van Nostrand, 1945, Chapter 8. 24. T. F. Schubert, Jr., A quantitative comparison of three Bode straight-line phase approximations for second-order, underdamped systems, IEEE Trans. Educ., 40: 135138, 1997. 25. A. A. Andronow, A. A. Vitt, and S. E. Chaikin, Theory of Oscillations, Reading, MA: Addison-Wesley, 1966. 26. R. S. Baheti, Simple anti-windup controllers, in Proc. 1989 Amer. Control Conf., Vol. 2, 1989, pp. 16841686. 27. D. S. Bernstein, Optimal nonlinear, but continuous feedback control of systems with saturating actuators, in Proc. 32rd Conf. Decision Control, December 1993, pp. 25332537. 28. D. S. Bernstein, Actuator saturation and control system performance, in Proc. 32rd Conf. Decision Control, 1993, pp. 24202421. 29. P. J. Campo and M. Morari, Robust control of processes subject to saturation nonlinearities, Comput. Chem. Eng., 14 (4/5): 343 358, 1990. 30. P. R. Chandler, M. J. Mears, and M. Pachter, A hybrid LQR/LP approach for addressing actuator saturation in feedback control, in 1994 Conf. Decision Control, Dec. 1416, 1994, pp. 38603867. 31. E. G. Gilbert and K. T. Tan, Linear systems with state and control constraints: The theory and application of maximal output admissible sets, IEEE Trans. Autom. Control, AC-36: 1008 1020, 1991. 32. E. G. Gilbert, I. Kolmanovsky, and K. T. Tan, Nonlinear control of discrete-time linear systems with state and control constraints: A reference governor with global convergence properties, in Proc. 33rd Conf. Decision Control, December 1994, pp. 144149. 33. R. Hanus, M. Kinnaert, and J. L. Henrotte, Conditioning technique, a general anti-windup and bumpless transfer method, Automatica, 729739, 1987. 34. R. A. Hess and S. A. Snell, Flight control design with actuator saturation: SISO systems with unstable plants, in 33rd Aerospace Sciences Meeting and Exhibit, AIAA, paper No 95-0336, January 1995; also published in the AIAA J. Guidance and Control, 19 (1): 191197, 1996, and see also the Errata in AIAA J. Guidance and Control, 19 (6): 1200. 35. I. Horowitz, Feedback systems with rate and amplitude limiting, Int. J. Control, 40: 12151229, 1984. 36. P. Kapasouris, M. Athans, and G. Stein, Design of feedback control systems for unstable plants with saturating actuators, in IFAC Symp. Nonlinear Control Syst. Design, June 1989, pp. 302307. 37. P. Kapasouris and M. Athans, Multivariable control systems with saturating actuators antireset windup strategies, in Proc. 1985 Amer. Control Conf., 1985, pp. 15791584. 38. P. Kapasouris and M. Athans, Control systems with rate and magnitude saturation for neutrally stable uncertain plants, in Proc. 29th Conf. Decision Control, Vol. 6, 1990, pp. 34073409.
390
39. N. J. Krikelis and S. K. Barkas, Design of tracking systems subject to actuator saturation and integrator windup, Int. J. Control, 39: 667683, 1984. 40. R. B. Miller, A New Approach to Manual Tracking Flight Control with Amplitude and Rate Constrained Dynamic Actuators, Ph.D. dissertation, Air Force Institute of Technology, March 1997. 41. R. B. Miller and M. Pachter, Manual tracking control with amplitude and rate constrained actuators, in Proc. 1996 Conf. Decision Control, December 1996, pp. 31593164. 42. R. B. Miller and M. Pachter, Manual control with saturating actuators, in Proc. 1997 Eur. Control Conf., July 14, 1997. 43. R. B. Miller and M. Pachter, Maneuvering ight control with actuator constraints, AIAA J. Guidance, Control Dynamics, 20 (4): 729734, 1997. 44. C. Moussas and G. Bitsoris, Adaptive constrained control subject to both input-amplitude and input-velocity constraints, in Proc. IEEE Int. Conf. Syst., Man Cybern., Vol. 4, 1993, pp. 601606. 45. M. Pachter and D. H. Jacobson, Observability with a conic observation set, IEEE Trans. Autom. Control, AC-24: 632633, 1979. 46. M. Pachter, P. R. Chandler, and M. Mears, Control reconguration with actuator rate saturation, in Proc. Amer. Control Conf., June 1995, pp. 34953499. 47. M. Pachter, P. R. Chandler, and L. Smith, Velocity vector roll control, in 1996 AIAA Guidance, Navigation Control Conf., San Diego, CA, July 1996, AIAA paper No. 96-3867. 48. M. Pachter, P. R. Chandler, and L. Smith, Maneuvering ight control, in 1997 Amer. Control Conf., Albuquerque, NM, June 46, 1997. 49. M. Pachter and R. B. Miller, Manual ight control with saturating actuators, IEEE Control Syst. Magazine, 18: 1019, 1997. 50. V. M. Popov, The Hyperstability of Control Systems, New York: Springer, 1973. 51. A. A. Rodriguez and Y. Wang, Saturation Prevention Strategies for an Unstable Bank-To-Turn (BTT) Missile: Full Information, Ph.D. dissertation, Department of Electrical Engineering, Arizona State University, Tempe, AZ. 52. A. G. Tsirukis and M. Morari, Controller design with actuator constraints, in Proc. Conf. Decision Control, 1992, pp. 26232628.
J. J. DAZZO M. PACHTER
Air Force Institute of Technology (AFIT/ENG)
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Stochastic Optimization, Stochastic Approximation and Simulated Annealing Standard Article James C. Spall1 1The Johns Hopkins University, Laurel, MD Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1044 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (206K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are RobbinsMonro Stochastic Approximation Stochastic Approximation with Gradient Approximations Based on Function Measurements Simultaneous Perturbation Stochastic Approximation Simulated Annealing Concluding Remarks Acknowledgements About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
STOCHASTIC OPTIMIZATION, STOCHASTIC APPROXIMATION AND SIMULATED ANNEALING
529

Optimization problems are central to human existence. Individuals and organizations are often faced with making tradeoffs between different factors in order to obtain desirable outcomes, and the problem of choosing these factors in some best way is the essence of the optimization problem. Some of the earliest manifestations occurred at the dawn of civilization when humans developed strategies for societal organization and for obtaining food and shelter. Optimization continues today throughout society in the design, operation, control, and evaluation of modern systems. Formal optimization is associated with the specication of a mathematical objective function and a collection of factors (or parameters) that can be adjusted to optimize the objective function. In particular, one can formulate an optimization problem as follows: Find that solves min L( )
C
(1)
where L : Rp R1 represents some loss function to be minimized, represents the p-dimensional vector of adjustable parameters, and C Rp represents a constraint set dening the allowable values for the parameters . (Note that a maximization problem can be trivially recast as the minimization problem in Eq. (1) by applying a minus sign to the objective function.) Our focus in this article will be problems for which represents a vector of continuous parameters; this is in contrast to discrete problems such as how many items X do we need to optimize performance? Discrete optimization is a large subject unto itself, and will not be considered in any detail here (see, e.g., Ref. 1 for a detailed discussion of this subject from a deterministic perspective). Further, we are inJ. Webster (ed.), Wiley Encyclopedia of Electrical and Electronics Engineering. Copyright # 1999 John Wiley & Sons, Inc.
530
terested in the case where L is sufciently complicated that it is not possible to obtain a closed-form analytical solution to Eq. (1). This is far and away the most common setting for large-scale optimization problems encountered in practice. Hence to solve for * in Eq. (1), one uses an iterative algorithm: a step-by-step method for moving from an initial guess at * to a nal value that is expected to be closer to the true * than the initial guess. One of the major distinctions in optimization is between global and local optimization. All other factors being equal, one would always want the global optimal solution to Eq. (1), that is, the * that provides a lower value of L than any other value of C. However, in practice this solution is often not available and one must be satised with obtaining a local solution. For example, L may be shaped so that there is a clearly dened (local) minimum point over a broad region of the allowable space C while there is a very narrow spike at a distant point such that the trough of this spike is lower than any point in the broad region. Hence, the local solution is better than any nearby , but may not be the best possible . Because of the inherent limitations of the vast majority of optimization algorithms, it is usually only possible to ensure that an algorithm will approach a local minimum with a nite amount of resources being put into the optimization process. However, since the local minimum may still yield a signicantly improved solution (relative to no formal optimization process at all), the local minimum may be a fully acceptable solution for the resources available (human time, money, computer time, etc.) to be spent on the optimization. Most of this article will focus on algorithms that are only guaranteed to yield a local optimum; however, we will also consider one type of algorithm (simulated annealing) that aims to nd a global solution from among multiple local solutions. Our focus in this article is the study of iterative algorithms where 1. there is random noise in the measurements of L (and its derivatives if relevant) and/or 2. there is a random choice in computing the search direction as the optimization algorithm iterates toward a solution. The above two characteristics contrast with classical deterministic optimization, where it is assumed that one has perfect information about the loss function (and its derivatives, if relevant) and that this information is used to determine the search direction in a deterministic manner at every step of the algorithm. In many practical problems one will not have perfect information about the loss function due to inevitable noise effects. A common example is where it is desired to minimize some mean squared error in the performance of some system (e.g., the tracking error in a robot manipulation problem). Only rarely will one be able to compute the mean squared error (or its derivatives); rather, one might be able to get a specic observation of the squared error, but this differs (sometimes very signicantly) from the mean squared error. Relative to point 2 above, it is sometimes benecial to deliberately introduce randomness into the search process as a means of speeding convergence and making the algorithm less sensitive to modeling errors. Although the introduction of randomness may seem counterproductive, it is well known to have benecial effects in some settings (another case where
this is true is in numerical integration, where Monte Carlo methods can be much more efcient in high-dimensional problems than deterministic quadrature approaches). However, deterministic optimizationwhich includes linear and nonlinear programming and such well-known methods as steepest descent, NewtonRaphson, and conjugate gradientprovides a useful starting point for the study of stochastic methods. In particular, many (though certainly not all) of the methods in both deterministic and stochastic optimization for the continuous problems of interest in this article rely in some manner on the gradient vector of the loss function with respect to the parameters: L 1 . . g ( ) . L p Then for local unconstrained optimization (i.e., C Rp), a necessary condition for optimization when L is a continuously differentiable nonlinear function is that * satises g ( ) = 0 (2)
(constrained problems can also be formulated in this way through, e.g., the use of a penalty function added to the basic loss function to penalize violations of the constraints). The proof of this result follows from simple Taylor series arguments showing that if Eq. (2) is not true then one can move in some direction that reduces the value of L. Many optimization algorithms are based on Eq. (2), thereby converting the general optimization setting of Eq. (1) to the problem of nding a root to the equation g() 0. This article will focus on two broadand popularclasses of stochastic optimization algorithms: stochastic approximation and simulated annealing. There are many other stochastic optimization algorithms that we are not considering: notably, genetic algorithms, evolutionary strategies, evolutionary programming, and various types of iterative random search. Many references on these other approaches are available to the interested reader. Among these are Ref. 2 for genetic algorithms, Ref. 3 or the journal Evolutionary Computation for genetic algorithms and other evolutionary methods, and Refs. 4, 5, or 6 for random search. The next section of this article reviews the core stochastic approximation (SA) algorithm that is based on direct (but usually noisy) measurements of the gradient vector g(); this is the well-known RobbinsMonro SA algorithm. Then follows an overview of SA when only measurements of the loss function L() are available [not measurements of the gradient g()]. The subsequent section analyzes in greater detail one of the gradient-free SA methodssimultaneous perturbation SAintroduced previously. Then a review of simulated annealing is given. The nal section is a brief summary putting these algorithms into perspective. ROBBINSMONRO STOCHASTIC APPROXIMATION Background and Algorithm Form We now discuss the well-known RobbinsMonro stochastic approximation (RMSA) algorithm, which is a gradient-based
531
stochastic optimization algorithm (sometimes referred to as a stochastic gradient method) for a wide variety of nonlinear problems. This subsection will introduce the basic algorithm form. The following two subsections discuss some of the theoretical properties related to convergence and asymptotic distributions. Then we will summarize three extensions to the basic algorithm form, and nally summarize how the RMSA algorithm is implemented in several different nonlinear applications. The prototype stochastic optimization application for the RMSA algorithm is the problem of nding a root to the equation g() 0 based on (noisy) measurements of g(). Let Y() represent the measurement of g(). [Although this article is written in the language of optimization, many of the ideas carry over directly to the general root-nding context as well, where g() represents the function for which a zero is to be found.] RMSA was introduced in a famous 1951 paper (10) and has spawned a large number of follow-on papers. The algorithm has the form k + 1 = k akY ( k ) (3)
where ak is a nonnegative gain sequence that must satisfy k represents the escertain conditions (discussed below) and timate of * at the kth iteration. Since the deterministic term L/ does not equal the stochastic term Y, the SA algorithm is fundamentally different from the well-known deterministic steepest descent algorithm. However, there is an intuitive connection, since E(Y) L/ under standard RMSA conditions [typically under the relatively modest regularity conditions justifying the interchange of a derivative and an (expectation) integral] (see, e.g., Ref. 7 or 8). A variation on the basic form in Eq. (3) is to include a projection operator, say C, that automatically maps solutions outside the constraint set C back to the nearest point within C. Kushner and Yin (98) treat this approach extensively. In such a case, Eq. (3) becomes k + 1 =
C [ k
gence of the iterate, which implies convergence in probability. Blum (11) was apparently the rst to give conditions for a.s. convergence. Neither of a.s. and mean-square convergence is implied by the other, but a.s. is stronger than in probability.] Many sufcient conditions have been given over the years for a.s. convergence of the SA algorithm in Eq. (3). Ruppert (12) and Rustagi (13, Chap. 9), for example, discuss conditions that have largely evolved out of the statistics perspective. Ljung (14), Kushner and Clark (15), Kushner and Yin (98), and Benveniste et al. (16, Chap. I.2) discuss a somewhat different set of conditions that have largely grown out of the engineering perspective. Central to the latter approach is the denition of an underlying ordinary differential equation (ODE) that roughly emulates the SA algorithm in Eq. (3) for large k and as the random effects disappear. It turns out that the convergence properties of this deterministic differential equation are closely related to the stochastic convergence properties of Eq. (3). Lai (17, Sec. 2) provides a nice intuitive explanation of this differential-equation-based approach. Probably the most famous of the convergence conditions for RMSA are those on the gain sequence ak. The conditions provide a careful balancing between wanting to damp out the noise effects as we get near the solution * (ak 0) and avoiding premature (false) convergence of the algorithm (k0ak ). The scaled harmonic sequence a/ (k 1), a 0, is the best-known example of a gain sequence that satises the gain conditions (and, as discussed in the next subsection, is an optimal choice with respect to the theoretical rate of convergence, although in practice other decay rates may be superior in nite samples). Usually some numerical experimentation is required to choose the best value of the scale factor that appears in the decaying gain sequence. Other conditions important for convergence relate to the smoothness of g(), the relative magnitude of the noise, and the position of the initial condition. Asymptotic Normality of RobbinsMonro Stochastic Approximation and Choice of Gain Sequence We discussed above the issue of convergence of the iterate k. This is of central importance in any optimization algo rithm. Also of importance is the probability distribution of the iterate (which, recall, is a random vector in our stochastic optimization context). Having knowledge of the distribution provides key insight into two main aspects of the algorithm: (1) error bounds for the iterate and (2) guidance in the choice of the optimal gain ak so as to minimize the likely deviation of k from *. Unfortunately, the theory governing the asymptotic distribution of the SA iterate is rather difcult. This is to be expected, given the nonlinear transformations arising from the k) forms the basis for k1, which in recursion (3): a value Y( k1) (generally a nonlinturn is the point of evaluation for Y( ear mapping) in the next iteration. General results on the asymptotic distribution of SA iterates are given by Fabian (18). His work is a generalization of previous asymptotic distribution results for SA by Sacks (19). He shows that, under appropriate regularity conditions, k ) N (0, ) k/2 (
dist
a kY ( k )]
We will not treat this form further in this article. Convergence of the RobbinsMonro Stochastic Approximation Algorithm As with any optimization algorithm, it is of interest to know k will converge to * as k gets large. In whether the iterate fact, one of the strongest aspects of SA is the rich convergence theory that has been developed over many years. Researchers and analysts in many elds have noted that if they can show that a particular stochastic optimization algorithm is a form of SA algorithm, then it may be possible to establish formal convergence where otherwise that might have remained an open question. In neural networks, for example, White (9) was apparently the rst to use this idea to show convergence of certain forms of the well-known backpropagation algorithm. Note that since we are in a stochastic context, convergence is in a probabilistic sense. In particular, the most common form of convergence established for SA is in the almost sure (a.s.) (or with probability one) sense. [A historical note: Robbins and Monro (10) showed conditions for mean-square conver-
(4)
532

dist
as k , where denotes convergence in distribution (see Ref. 20, Chap. 3, or any other graduate-level probability text for a formal denition of this type of convergence), is some covariance matrix that depends on the Hessian matrix of L() (at *), N(0, ) represents a multivariate normal distribution with mean 0 and covariance , and governs the decay rate for the SA gain sequence ak [e.g., ak a /(k 1)]. Ruppert (12) also discusses this result. Various special cases of this result dealing with the situation 1 are presented in Refs. 13 (pp. 258259), 21 (pp. 7178), and 12. The intuk * will be approxiitive interpretation of Eq. (4) is that mately normally distributed with mean 0 and covariance matrix / k for k reasonably large. k Equation (4) implies that the rate at which the iterate approaches * is k /2. This follows because a random vector with the distribution N(0, ) on the right-hand side of Eq. (4) is well behaved (i.e., not degenerate 0 or in magnitude), k * must be decaying at a rate k /2 to balance the and /2 k blowup factor on the left-hand side of Eq. (4). Under standard conditions on ak (see preceding subsection), the k to * is maximized at 1. rate of convergence of In practical nite-sample problems, however, the choice of 1 is not generally recommended. Most analysts and researchers nd a lower value of yields superior nite-sample behavior. (This is a fact that is well known, but not widely documented in the literature because it contradicts the theoretical result suggesting the optimality of 1; nevertheless, in the authors experience and in the experience of all SA implementers he has consulted, a lower value of is generally preferable. Othereffectively equivalentways exist for slowing down the decay rate of ak so that it acts like 1 in nite samples; see, e.g., the practically oriented paper of Okamura et al. (22) for estimation in nite-impulse-response adaptive lters.) The intuitive reason for the desirability of 1 is that a slower decay provides a larger step size in the iterations with large k, allowing the algorithm to move in bigger steps towards the solution. This observation is a practical nite-sample result, as the asymptotic theory showing optimality of 1 is unassailable. In fact, in many applications, a constant step size ( 0) is used. Typical applications involve adaptive tracking or control problems where * is changing in time. The constant gain provides enough impetus to the algorithm to keep up with the variation in *, whereas if a decaying gain were used, too little weight would be applied to the current input information to allow for the algorithm to track the solution. Such constant-gain algorithms are also frequently used in neural network training even when there is no variation in the underlying * (9,23). Constant-step-size SA algorithms will generally not formally converge. (However, a partial convergence theory is possible for constant gains. This is typically based on limiting arguments as the gain magnitude gets small. Essentially, one is able to show that the iterate from a constantgain algorithm will approach the optimal to within some error that decreases as the gain magnitude is made smaller. See, e.g., Refs. 2428.) Also note that the limiting distribution for the standardized SA iterate [analogous to the left-hand side of Eq. (4)] is not generally normal with constant step size (29,26). Extensions of Standard RMSA This section discusses some extensions to the basic RMSA framework presented in Eqs. (3)(4). In particular, we con-
sider: (1) the setting where the observed gradient Y includes a state vector that evolves as is being updated, (2) methods of algorithm acceleration by improved choice of the gain sequence, (3) iterate averaging for SA as a means for accelerating convergence, and (4) the setting where the loss function L may change with time. Joint Parameter and State Evolution. In our rst generalizak) by Y( k, xk), where xk represents a state tion, we replace Y( vector related to the system being optimized. The book by Benveniste et al. (16) is devoted to this framework, which was apparently rst considered by Ljung (14) and extended by Metivier and Priouret (30). It is typically assumed that xk evolves online according to a set of certain Markov transition k [as given by probabilities. The convergence properties of Eq. (3)] then depend on the properties of these transition probabilities. Benveniste et al. (16) discuss several applications of this framework in the context of telecommunications, fault detection, and signal processing. One of the common Markov representations of the evolution of the state is via the k)xk B( k)wk, where A and B linear state equation xk1 A( are appropriately dimensioned matrices and wk is a sequence of independent, identically distributed random vectors (see, e.g., Ref. 14). Then the behavior of the Markov chain, and the k, can be tied directly to the stabilassociated convergence of ity of this linear process using well-known results in linear systems theory. A specic application of this form is in temporal difference learning for function approximation (99). The goal here is to approximate the cost-to-go function associated with many time-series prediction and control problems. The cost-to-go function measures the expected future cost associated with specic policies (e.g., control strategies) for a dynamic process, given that the process is currently in a particular state. When the state evolves according to a Markov chain, temporal difference learning can be cast as an RMSA form with joint parameter and state evolution. Adaptive Estimation and Second-Order Algorithms. There are a large number of methods for adaptively estimating the gain sequence (or a multivariate analog of the gain) to enhance the convergence rate of RM-type SAs. Some of these are built on stochastic approximation analogs of the famous Newton Raphson deterministic optimization algorithm (e.g., Ref. 31). It is knownsee, e.g., Ref. 16, Sec. 3.2that the asymptotically optimal gain for RMSA is a matrix given by H(*)1 / k, where H() represents the Hessian matrix of L(). This is identical to deterministic optimization, except that we now have the decay factor k included to cope with the stochastic effects. Unfortunately, this is largely of theoretical interest only, since in practice one does not know either * or the Hessian as a function of . It also is an asymptotic result, and, as discussed in the preceding subsection, optimality for practical nite-sample analysis may impose other requirements. Nevertheless, this asymptotic result provides a type of ideal in designing adaptive SA algorithms. Perhaps the rst adaptive technique is that of Kesten (32), k1 k in a where one looks at the signs of the difference scalar process as a means of designing an adaptive gain sequence ak (unlike the approaches described below, this approach does not explicitly use the connection with the Hessian matrix as mentioned above). If there are frequent sign
533
changes, this is an indication that the iterate is near the solution, while if signs are not changing we have an indication that the iterate is far from the solution. This forms the basis for an adaptive choice of the gain ak, whereby a larger gain is used if there are no sign changes and a smaller gain is used if the signs change frequently. Kesten (32) established a.s. convergence with such a scheme. A multivariate extension of the Kesten idea is described in Delyon and Juditsky (33), including theoretical justication of the extension via a.s. convergence of the iterates. Ljung and Soderstrom (34, Sec. 2.4, Sec. 3.4, and Chap. 4) and Wei (35) discuss stochastic analogs of the Newton Raphson search in the context of parameter estimation for particular (possibly linear) models. [These represent extensions of the rst paper on adaptive Hessian estimates for scalar problems (100).] In so doing, Refs. 34 and 35 demonstrate, via the method of minimizing prediction error, that a batch version of the problem of nding a * satisfying Eq. (2) (where all the data are processed simultaneously) can be converted to the recursive form (where the data are processed one at a time) using only the current (instantaneous) input. The scalar gain ak is then replaced in their formulation by a matrix that approximates the (unknown) true inverse Hessian matrix corresponding to the current data points contribution to the loss function. Ruppert (36) describes an approach where the Hessian is estimated by taking nite differences of a gradient measurement. The gradient used by Ruppert differs slightly from the standard RM gradient in Eq. (3) in that he converts the basic problem from one of minimizing L() to one of minimizing L / 2 (note that this yields the same * when there is a unique minimum). Spall (102) presents a more efcient approach to general Hessian estimation based on the simultaneous perturbation idea discussed below. Iterate Averaging. An important and relatively recent development in SA is the concept of iterate averaging. Like many good ideas, this one is simple and, in some problems, can be effective. The idea was jointly described by Polyak and Juditsky (37) and Ruppert (Ref. 12, based on a 1988 internal technical report). There are several variations, but the basic k as our current best estimate of * after idea is to replace k iterations with the average
convergence that previously was possible only with knowledge of the Hessian H(*), as discussed under Adaptive estimation and second-order algorithms above. This result is available for any valid gain satisfying the above-mentioned additional condition. Hence, in principle, iterate averaging greatly reduces one of the traditional banes of SA implementationnamely, choosing the gain sequence in some optimal way. Variations on the above basic iterate averaging formulation are readily available. One obvious variation is to not average in the rst few iterations, but rather start the average at some N 0 or else use only a sliding window of the last n N (say) measurements. In practical nite-sample implementations, such modications are likely to help, since the rst few iterations tend to produce the poorest estimates. In the sliding-window approach, formal asymptotically optimal normality can be retained if the window length grows with time (see, e.g., Ref. 38). A further modication to the basic k) in approach is to use the averaged value k (together with a modied form of the RMSA iteration [instead of k alone on the right-hand side of the basic form (3)]. This is referred to as the feedback approach in Ref. 39, and can be shown to sometimes yield further improvement. In practice, however, the results on iterate averaging are more mixed than the above would suggest. Numerical studies by the author have shown that a well-chosen ak sequence will yield results superior to that possible by averaging (see also Ref. 40, p. 57, and Ref. 41 for some cautionary notes). Once again, this appears to be a consequence of the nite-sample properties of practical problems. More study is required to understand the full capabilities of iterate averaging in practice. Time-Varying Loss Functions. A nal generalization of the RobbinsMonro recursion that we discuss is one where the loss function (and corresponding gradient with respect to ) varies with k. This problem is treated in Refs. 42 and 43. The basic idea is that, while the loss function may change shape with k, it is assumed that the underlying minimum * is either constant for all k or xed in the limit as k (even though the loss function may change shape indenitely). The two references mentioned above show a.s. convergence in these cases for the scalar- setting; the proofs have to be changed from those commonly seen in SA to accommodate the time-varying loss. Figure 1 depicts a time-varying loss function Lk() for a scalar parameter. This gure shows a case where the loss function and the minimizing parameter value both change with time, but one where the optimal parameter value converges to a limiting point * . This is a situation for which the above-mentioned theory would apply. Applications of RobbinsMonro Stochastic Approximation RMSA has been applied in a large number of engineering (and other) problems, often under another name. This subsection provides a summary of several applications together with some references for further study. Neural network (NN) training via the well-known backpropagation algorithm has long been recognized as an application of the RMSA algorithm, two of the publications discussing this connection being Ref. 9 and Ref. 44 (pp.
k (k + 1 ) 1
k j=0
(5)
j summands is computed as in Eq. (3). The where each of the singular achievement of the above references was to show that k1/2(k *) is asymptotically normally distributed with mean 0 and a covariance matrix that is as small as possible in a matrix sense. These references establish this optimality of Eq. (5) for gain sequences satisfying the standard conditions plus the conditions ak1 / ak 1 o(ak) [with o( ) implying a term that goes to 0 faster than the argument]. This important additional condition implies that ak must decay at a rate slower than the optimal rate of 1/ k for the individual k (as discussed in the preceding subsection). The imiterates plications of this result are quite strong: namely, that one can use the standard algorithm in Eq. (3) together with the simple averaging in Eq. (5) to achieve the same optimal rate of
534
L1( )
L2( )
L3( )
Lk+1( ) Lk( )
Time index, k

Figure 1. Example of time-varying loss functions with limiting minimum *.
190199). The essence of the backpropagation algorithm is to systematically compute (via the chain rule applied to the neural network structure) the gradient g() [or its noisy measurement Y()] for use in an algorithm identical to RMSA (it appears that the authors of some of the earlier publications in this areae.g., Ref. 45were unaware of the connection with RMSA). A popular area for application of RMSA is in recursive estimation for linear models, especially as applied in signal processing, fault detection, and adaptive control [e.g., Benveniste et al. (16, Chap. I.1), Ljung et al. (21, Chap. III), and Solo and King (101, Chap. 5)]. Several variations of the resulting algorithms exist, most notably the least mean squares (LMS) and recursive least squares (RLS) algorithms. The essential idea in such algorithms is that the data are processed sequentially as they arrive according to a formula that weights each new data point in light of the assumed linear relationship between the input and output on the one hand and the data that have already arrived on the other. This contrasts with traditional linear regression methods where all the data are assumed to be available at the outset and the processing is done in batch form. Given the dynamic nature of problems in signal processing, fault detection, adaptive control, and certain other areas, this recursive processing has obvious advantages. Further, it can be shown that the recursive algorithms yield an estimate at time t that is close to that which would result from batch processing if all the data up to time t were available at the outset (when t is reasonably large). Another popular area for application of RMSA is in simulation-based optimization, especially in the context of discreteevent dynamic systems such as queuing networks. Here the goal may be to optimize some design aspect of a system (e.g., the position of certain machines on a factory oor) by running experiments via computer simulations. A systematic approach to simulation-based optimization has been adapted under the rubric of perturbation analysis, which was introduced in modern form by Y. C. Ho and his colleagues in the late 1970s. In this method, one aims to get a gradient estimate Y() at any value based on only one or a small number of simulation runs. Given the stochastic nature of the simulation, this gradient estimate will also only be a stochastic estimate of the true gradient L / . Some references on simulation-based optimization in the context of RMSA are Refs. 46, 47, 8, 48, and 49.
The nal application we mention here is in image restoration. Here the task is to recover a true image of a scene from a recorded image of the scene where the recorded image is typically corrupted by noise and/or otherwise degraded from the original image. The essential reason for using RMSA rather than more conventional signal processing and deconvolution techniques is the ability to adapt to nonlinearities in the process. Some recent references that discuss implementations of RMSA in the context of image restoration are Refs. 50 and 51. Abreu et al. (50) are concerned with removing impulse noise that may be due, for example, to noisy sensor or transmission errors in collecting the image data. They note that nonlinear methods have often proven superior to linear methods for this problem. Cha and Kassam (51) describe an approach to image restoration based on a type of NN called the radial basis function network (e.g., Ref. 6, Sec. 9.5), where RMSA is used to train the weights of this form of function approximator. Their use of the radial basis function network is as a spatially invariant lter taking measurement data as input and producing estimates of the pixel-by-pixel gray levels in the image. STOCHASTIC APPROXIMATION WITH GRADIENT APPROXIMATIONS BASED ON FUNCTION MEASUREMENTS Introduction and Contrast of Gradient-Based and Nongradient Algorithms There has been a growing interest in stochastic optimization algorithms that do not depend on direct gradient information or measurements. Rather, these algorithms are based on an approximation to the gradient formed from (generally noisy) measurements of the loss function. This interest has been motivated, for example, by problems in the adaptive control and statistical identication of complex systems, the optimization of processes by large Monte Carlo simulations, the training of recurrent neural networks, the recovery of images from noisy sensor data, and the design of complex queuing and discreteevent systems. Overall, such algorithms exhibit certain convergence properties of the RobbinsMonro gradient-based algorithms considered above while requiring only loss-function measurements. A main advantage of such algorithms is that they do not require the detailed knowledge of the functional relationship between the parameters being adjusted (optimized) and the loss function being minimized that is required in gradient-based algorithms [in particular, they do not need the Y() term in the RM recursion of Eq. (3)]. Such a relationship can be notoriously difcult to develop in some areas (e.g., nonlinear feedback controller design or system optimization via large-scale simulation), while in other areas (such as highdimensional statistical parameter estimation) there may be large computational saving in calculating a loss function rather than a gradient. In contrast, the approaches based on gradient approximations require only conversion of the basic output measurements to sample values of the loss function, which does not require full knowledge of the system input output relationships. Examples of approximation-based methods using loss-function measurements only are given below; such methods include, as an early prototype, the Kiefer Wolfowitz nite-difference SA algorithm (52). Because of the fundamentally different information needed in implementing these gradient-based (RM) and gradient-free
535
algorithms, it is difcult to construct meaningful methods of comparison. As a general rule, however, the gradient-based algorithms will converge faster than those using loss-function-based gradient approximations when speed is measured in number of iterations. Intuitively, this is not surprising given the additional information required for the gradient-based algorithms. In particular, based on asymptotic theory, the optimal rate of convergence measured in terms of the deviation of the parameter estimate from the true optimal parameter vector is of order k1/2 for the gradient-based algorithms (see the subsection on asymptotic normality above) and of order k1/3 for the algorithms based on gradient approximations, where k represents the number of iterations (52). (Exceptions to this maximum rate of convergence for the nongradient algorithms are discussed in Refs. 5355 and Ref. 56, where special cases are presented that achieve a rate arbitrarily close to, or equal to, k1/2.) In practice, of course, many other factors must be considered in determining which algorithm is most appropriate for a given circumstance. Three examples of why this is true are: (1) in cases where it is not possible to obtain reliable knowledge of the system inputoutput relationships, the gradientbased algorithms may be either infeasible (if no system model is available) or undependable (if a poor system model is used); (2) the total cost to achieve effective convergence depends not only on the number of iterations required, but also on the cost needed per iteration, which is typically greater in gradientbased algorithms (this cost may include greater computational burden, additional human effort required for determining and coding gradients, experimental costs for model building such as labor, materials, and fuel, etc.); and (3) the rates of convergence are based on asymptotic theory, and may not be representative of practical convergence rates in nite samples. For these reasons, one cannot say in general that a gradient-based search algorithm is superior to a gradient-approximation-based algorithm even though the gradient-based algorithm has a faster asymptotic rate of convergence (and with simulation-based optimization such as perturbation analysis, as discussed above, requires only one system run per iteration, while the approximation-based algorithm may require multiple system runs per iteration). As a general rule, however, if direct gradient information is conveniently and reliably available, it is generally to ones advantage to use this information in the optimization process. The focus in this section is on the case where such information is not readily available. Spall (57) presents a summary of historical contributions in the area of optimization with algorithms based on gradient approximations using only loss-function measurements. The summary below is a much condensed version of that review. Background As in the introduction consider the problem of minimizing a (scalar) differentiable loss function L(), where Rp, p 1, and where the optimization problem can be translated into nding the minimizing * such that the gradient g() 0. It is assumed that measurements of L() are available at various values of (actually, most of the algorithms have the weaker requirement of only requiring measurements of the difference of two values of the loss function, as opposed to measuring the loss functions themselves). These measurements may or may not include added noise. No direct mea-
surements of g() are assumed available, in contrast to the RM framework. The recursive procedure we consider is in the general SA form: k + 1 = k ak g k ) k ( (6)
k) is the estimate of the gradient L / at the iter k( where g k based on the above-mentioned measurements of the loss ate function. Under appropriate conditions, the iteration in Eq. (6) will converge to * in some stochastic sense, usually a.s. (see, e.g., Ref. 53 or 15). Typical convergence conditions are similar to those mentioned above for the RMSA algorithm. The essential part of Eq. (6) is the gradient approximation k). We discuss below two forms that have attracted atten k( g tion. We let y( ) denote a measurement of L( ) at a design level represented by the dot [i.e., y( ) L( ) noise], and ck be some (usually small) positive number [if the noise has mean 0, then y( ) Q( ) as dened in the section on RMSA]. One-sided gradient approximations involve measurements k) and y( k perturbation) while two-sided gradient apy( k perturproximations involve measurements of the form y( bation). The two general forms of gradient approximations are: k is perFinite Difference (FD) (52,58). Each component of turbed one at a time, and corresponding measurements y( ) are obtained; each component of the gradient estimate is formed by differencing the corresponding y( ) values and then dividing by a difference interval. This is the standard approach to approximating gradient vectors and is motivated directly from the denition of a gradient as a vector of p partial derivatives, each constructed as the limit of the ratio of a change in the function value over a corresponding change in one component of the argument vector. Typically, the ith k) (i 1, 2, . . ., p) for a two-sided FD k( component of g approximation is given by
k ) = ki ( g k + ck ei ) y ( k ck ei ) y ( 2c k
where ei denotes a vector with a one in the ith place and zeros elsewhere (an obvious analog holds for the onesided version; likewise for the SP form below). Simultaneous Perturbation (SP) (59,60). All elements of k are randomly perturbed together to obtain two mea k) is formed k( surements y( ), but each component of g from a ratio involving the individual components in the perturbation vector and the difference in the two corresponding measurements. For two-sided SP, we have k ) = ki ( g k + ck y (
k)
k ck y (
ki
k)
2c k
where the distribution of the user-specied random perturbations for SP, k (k1, k2, . . ., kp)T, satises conditions mentioned in the next section. The algorithm [Eq. (6)] with one of the gradient approximations will be referred to as FDSA or SPSA, as appropriate. [An approach in the same spirit as SPSA, called random di-
536
rections SA, is discussed in Refs. 61, 15, and 62, but it is shown in Ref. 63 that SPSA will generally have a lower asymptotic mean squared error than RDSA for the same number of measurements y( ).] Note that the number of loss-function measurements y( ) needed in each iteration of FDSA grows with p, while with SPSA only two measurements are needed, independent of p. This, of course, provides the potential for SPSA to achieve a large saving (over FDSA) in the total number of measurements required to estimate when p is large. This potential is only realized if the number of iterations required for effective convergence to * does not increase in such a way as to cancel the measurement savings per gradient approximation at each iteration. The next section will discuss this efciency issue further, demonstrating when this potential can be realized. SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION Introduction The preceding section motivated the interest in techniques for recursive optimization that rely on measurements of the loss function only, not on measurements (or direct calculations) of the gradient (or higher-order derivatives) of the loss function. The focus was on two such techniques in the stochastic approximation setting: nite-difference (KieferWolfowitz) SA, and simultaneous perturbation SA. This chapter will focus on SPSA for reasons of its relative efciency. Recent applications of SPSA are described in Refs. 64 and 65 (queuing systems), 66 (industrial quality improvement), 67 (pattern recognition), 68 (neural network training), 69 and 70 (adaptive control of dynamic systems), 71 (statistical model parameter estimation and fault detection), 72 (sensor placement and conguration), and 73 (vehicle trafc management). As discussed in the preceding section, SPSA is based on a highly efcient and easily implemented simultaneous perturbation approximation to the gradient: this gradient approximation uses only two loss-function measurements, independent of the number of parameters being optimized. This contrasts, for example, with the standard (two-sided) nitedifference approximation associated with the well-known KieferWolfowitz SA algorithm, which uses 2p function measurements to approximate the gradient. The fundamental (and perhaps surprising) theoretical result in Ref. 60 (Sec. 4) is:
Under reasonably general conditions, SPSA and KieferWolfowitz nite-difference-based SA (FDSA) achieve the same level of statistical accuracy for a given number of iterations even though SPSA uses p times fewer function evaluations than FDSA (since each gradient approximation uses only 1/ p the number of function evaluations).
and efciency of SPSA. The subsection after that summarizes a reference dealing with guidelines for practical implementation of the algorithm. The subsection after that discusses some extensions to the basic SPSA algorithm, including some relatively recent results on a second-order version of SPSA that emulates the NewtonRaphson algorithm of deterministic optimization while still only requiring loss-function measurements (i.e., no gradient or Hessian information). Numerical studies of SPSA are available in many of the references (see, e.g., Ref. 60, 66, 64, 69, 70, or 63). Basic Assumptions and Supporting Theory Once again, the goal is to minimize a loss function L() over C Rp. The SPSA algorithm works by iterating from an initial guess of the optimal , where the iteration process depends on the above-mentioned simultaneous perturbation approximation to the gradient g(). The form of the SPSA gradient approximation was presented above. Spall (74,60) presents sufcient conditions for convergence k * a.s.) using the differential equaof the SPSA iterate ( tion approach discussed in the RMSA section in the context of the RM algorithm. Because of the different form of the input, the conditions here are somewhat different from those of the RM approach. In particular, we must impose conditions on both gain sequences (ak and ck), the user-specied distribution of k, and the statistical relationship of k to the measurements y( ). We will not repeat the conditions here, since they are available in Refs. 74 and 60 (with later extensions in Refs. 7577). The main conditions are that ak and ck both go to 0 at rates neither too fast nor too slow, that L() is sufciently smooth (several times differentiable) near *, and that the ki are independent and symmetrically distributed about 0 with nite inverse moments E(ki1) for all k, i. One particular distribution for ki that satises these latter conditions is the symmetric Bernoulli 1 distribution; two common distributions that do not satisfy the conditions (in particular, the critical nite-inverse-moment condition) are the uniform and the normal. Although the convergence result for SPSA is of some independent interest, the most interesting theoretical results in Ref. 60, and those that best justify the use of SPSA, are the asymptotic efciency conclusions that follow from an asymptotic normality result. In particular, under some minor additional conditions in Ref. 60 (Proposition 2), it can be shown that
dist k ) k / 2 ( N (, )
as k
(7)
This theoretical result has been conrmed in many numerical studies, even in cases where p is on the order of several hundred or thousand. The subsection below discusses this result further. Of course, this result will not always hold in nitesample practice, because it is derived from asymptotic theory; further, the asymptotic theory is based on conditions that may be violated in some practical applications (the general conditions are similar to those for the RMSA algorithm). The next subsection summarizes the problem setting and discusses some of the theory associated with the convergence
where 0 depends on the choice of gain sequences (ak and ck), depends on both the Hessian and the third derivatives of L() at *, and depends on the Hessian matrix at * (note that in general 0, in contrast to many well-known asymptotic normality results in estimation). Given the restrictions on the gain sequences to ensure convergence and asymptotic normality, the fastest allowable value for the rate of conver k to * is k1/3. This contrasts with the fastest allowgence of able rate of k1/2 for the RMSA algorithm. Hence, one measure of the value of the gradient information in RM is the increase in rate of convergence. There are exceptions to this result, and cases arise where the SPSA rate can be made either arbitrarily close to the RM rate of k1/2 (following the logic of Ref. 53) or the same as the RM rate (e.g., the use in simulation-
537
based optimization with common random numberssee Ref. 56). Spall (60, Sec. 4) uses the asymptotic normality result in Eq. (7) (together with a parallel result for FDSA) to establish the relative efciency of SPSA. This efciency depends on the shape of L(), the values for ak and ck, and the distributions of the ki and measurement noise terms. There is no single expression that can be used to characterize the relative efciency; however, as discussed in Refs. 60 (Sec. 4) and 63, in most practical problems SPSA will be asymptotically more efcient than FDSA. For example, if ak and ck are chosen as in the guidelines mentioned in the subsection below, then by k *2) in equating the asymptotic mean squared errors E( SPSA and FDSA, we nd no. of measurements of L( ) in SPSA 1 no. of measurements of L( ) in FDSA p (8)
(the standard algorithm can be coded in 12 or fewer lines). In addition, this reference provides some practical guidelines for choosing the gain coefcients ak and ck. The values recommended in these guidelines differ from the asymptotically theoretical optimal values in a reection of the realities of nite-samples analysis (e.g., it is recommended in practice that the gains ak and ck decay at rates slower than the asymptotically optimal 1/ k and 1/ k1/6, respectively). Furthermore, theoretical guidelines, such as discussed in Fabian (53) and Chin (63), are not generally useful in practical applications, since they require the very information on the loss function and its gradients that is assumed unavailable. Further Results and Extensions to the Basic Algorithm Sadegh and Spall (79) consider the problem of choosing the best distribution for the vector k. Based on asymptotic distribution results, it is shown that the optimal distribution for the components of k is symmetric Bernoulli. This simple distribution has also proven effective in many nite-sample practical and simulation examples. Some extensions to the basic SPSA algorithm above are reported in the literature. For example, its use in feedback control problems, where the loss function changes with time, is given in Refs. 69, 70. The former reference also reports on a gradient smoothing idea (analogous to momentum in the neural network literature) that may help reduce noise effects and enhance convergence (and gives guidelines for how the smoothing should be reduced over time to ensure convergence). Alternatively, it is possible to average several SP gradient approximations at each iteration to reduce noise effects (at the cost of additional function measurements); this is discussed in Ref. 60. An implementation of SPSA for global minimization is discussed in Ref. 80 [for the case where there are multiple minimums at which g() 0]; this approach is based on a stepwise (slowly decaying) sequence ck (and possibly ak). The problem of constrained (equality and inequality) optimization with SPSA is considered in Refs. 81 and 65 using a projection approach. A one-measurement form of the SP gradient approximation is considered in Ref. 82; although it is shown in that reference that the standard two-measurement form will usually be more efcient (in terms of the total number of loss-function measurements to obtain a given level of accuracy in the iterate), there are advantages to the onemeasurement form in real-time operations where the underlying system dynamics may change too rapidly to get a credible gradient estimate with two successive measurements (and, analogously to the perturbation analysis and related approaches in simulation-based optimization for discrete-event systems, this would allow for a one-run gradient estimate, but without requiring the gradient information needed in the IPA-type approaches). An accelerated form of SPSA is reported in Refs. 83 and 102. This approach extends the SPSA algorithm to include second-order (Hessian) effects with the aim of accelerating convergence in a stochastic analog to the deterministic NewtonRaphson algorithm. Like the standard (rst-order) SPSA algorithm, this second-order algorithm is simple to implement and requires only a small numberindependent of p of lossfunction measurements per iteration (no gradient measurements, as in standard SPSA). In particular, only four measurements are required to estimate the loss-function gradient and inverse Hessian at each iteration (though one additional
as the number of loss measurements in both procedures gets large. This result implies that the p-fold saving per iteration (gradient approximation) translates directly into a p-fold saving in the overall optimization process. Figure 2 is an illustration of the relative efciency of SPSA and FDSA in an adaptive control problem related to wastewater treatment. The specic scenario is close to that described in Ref. 70. The plot is showing the mean deviation (in a rootmean square, RMS sense) of the output of a treatment plant from some specied target values for water cleanliness and methane gas byproduct, the goal, of course, being to minimize the RMS error. We see that on an iteration-by-iteration basis, the SPSA and FDSA approaches yield similar RMS values. However, the cost per iteration in FDSA [measured in number of L() measurements] is 412 ( p) times that of SPSA. This leads to the very large overall cost savings shown in the upper right box of the gure. In fact, one iteration of FDSA takes over ve times the number of loss measurements than are taken in all 80 iterations of SPSA. The performance of the two algorithms in Fig. 2 is very consistent with the theoretical result in Eq. (8) above. Implementation of Simultaneous Perturbation Stochastic Approximation Reference 78 includes a step-by-step summary of the implementation of SPSA together with a listing of MATLAB code
1.6 Total number of L( )measurements 1.2
Normalized L ( )
SPSA FDSA
160 65,920
0.8 SPSA 0.4 FDSA Minimum achievable long-run value
20
40 Iterations
60
80
Figure 2. Relative performance of SPSA and FDSA in wastewater treatment system.
538
measurement is recommended as a check on algorithm behavior). The algorithm is implemented with two simple parallel recursions: one for and one for the Hessian of L(). The recursion for is a stochastic analog of the well-known NewtonRaphson algorithm of deterministic optimization. The recursion for the Hessian matrix is simply a recursive calculation of the sample mean of per-iteration Hessian estimates that are formed using SP-type ideas. SIMULATED ANNEALING Introduction and Motivation from the Physics of Cooling This section continues in the spirit of the preceding two in considering algorithms that do not require direct gradient information. One of the algorithms that has attracted considerable attention is simulated annealing (SAN). SAN was originally developed for discrete optimization problems, but more recently has found application in continuous optimization problems of the type emphasized in the previous sections. One of the main virtues of SAN is that, in principle, this algorithm addresses the difcult global optimization problem discussed in the introduction. The algorithm is designed to traverse local minima en route to a global minimum to L(). Further, since the method can address both discrete and continuous optimization problems, there is no need to assume the existence of a loss-function gradient (much less compute it). The term annealing comes from analogies to the cooling of a liquid or solid. A central issue in statistical mechanics is analyzing the behavior of substances as they cool. At high temperatures, molecules have much mobility, but as the temperature decreases this mobility is lost and the molecules may tend to line themselves in a crystalline structure. This structure is the minimum-energy state for the system. Note the qualier may: temperature alone does not govern whether the substance has reached a minimum-energy state. To achieve this state, the cooling must occur at a sufciently slow rate. If the substance is cooled at too rapid a rate, an amorphous (or polycrystalline) state may be reached that is not a minimum-energy state of the substance. The principle behind annealing in physical systems is the slow cooling of substances to reach the minimum-energy state. In optimization, the analogy to a minimum-energy state for a system is a minimized value of the loss function. The technique of SAN attempts to capture mathematically the process of controlled cooling associated with physical processes, the aim being to reach the lowest value of the loss function in the face of possible local minima. As with the physical cooling process, whereby temporary higher-energy states may be reached as the molecules go through their alignment process, SAN also allows temporary increases in the loss function as the learning process captures the information necessary to reach the global minimum. A more thorough explanation of the analogy between SAN and physical cooling is given in, e.g., Ref. 84. It is clear that the critical component of an SAN algorithm is the mathematical analog of the rate of cooling in physical processes. As with all other stochastic optimization algorithms, the choice of this algorithm- and problem-specic cooling schedule (analogous to the gains in stochastic approximation) has a strong effect on the success or failure of SAN. A primary distinction between SAN and the majority of other optimization approaches (including those discussed in
earlier sections) is the willingness to give up the quick gain of a rapid decrease in the loss function by allowing the possibility of temporarily increasing it. SAN derives this property from the Boltzmann (or Gibbs) probability distribution of statistical mechanics, describing the probability of system having a particular energy state: P(energy = x ) = cT exp x cb T (9)
where cT 0 is a normalizing constant, cb 0 is known as the Boltzmann constant, and T is the temperature of the system. Note that at high temperatures, the system is more likely to be in a high-energy state than at low temperatures. The optimization analogy derives from the fact that even at a low temperature (equivalent to the optimization algorithm having run for some time with a decreasing temperature), there is some nonzero probability of reaching a higher energy state (i.e., higher level of the loss function). So the SAN process sometimes goes uphill, but the probability of this decreases as the temperature is lowered. Hence, there is the possibility of getting out of a local minimum in favor of nding a global minimum, and this possibility is especially prominent in the early iterations when the temperature is high. It was Metropolis et al. (85) who rst introduced the idea of the BoltzmannGibbs distribution into numerical analysis through constructing a means for simulation of a system at some xed temperature. In particular, if a system is in some current energy state Ec, and some system aspects are changed to make the system potentially achieve a new energy state En, then the Metropolis simulation always has the system go to the new state if Enew Ecurr. On the other hand, if Enew Ecurr, then the probability of the system actually going to the new state is exp Enew Ecurr cb T (10)
This expression is known as the Metropolis criterion. After a large number of such decisions and outcomes, the system eventually reaches an equilibrium where the system state is governed by the BoltzmannGibbs distribution in Eq. (9). This is predicated on the system being at the xed temperature T. Kirkpatrick et al. (84) were apparently the rst to use this criterion for optimization together with the idea of a changing temperature that decays according to an annealing schedule (exponentially in their case). Geman and Geman (86) go beyond the informal annealing guidelines of Kirkpatrick et al. (84) and establish conditions on the annealing schedule for formal convergence of their form of the SAN algorithm iterate to the global minimum. Algorithm Form Using the principles discussed in the preceding subsection, we now present the general SAN algorithm form. There are variations depending on how one wants to implement the annealing schedule and how one performs the sampling required for generating a new candidate point. As before, we consider the minimization of some loss function L(), C Rp. Since the user has control over T, one can (without loss of generality) take cb 1 in the Metropolis criterion (10). Below are listed the general sequence of steps in SAN when noise-free measurements of L are available:
539
Step 1. Choose an initial temperature T and set of current parameter values curr; determine L(curr). Step 2. Randomly determine a new value of , new, that is close to the current value, and determine L(new). Step 3. Compare the two L values via the Metropolis criterion (10): Let L(new) L(curr). Accept new if 0. Alternatively, if 0, accept the new point new only if a uniform (0, 1) random variable U (generated by Monte Carlo) satises U exp( / T). Step 4. Repeat steps 2 and 3 for some period until either the budget of function evaluations allocated for that T has been used or the system reaches some state of equilibrium. Step 5. Lower T according to the annealing schedule, and return to step 2. Continue the process until the total budget for function evaluations has been used or some indication of convergence is satised (analogous to the system being frozen in its minimum-energy state). The specics of implementation for the ve steps above can vary greatly. In the annealing schedule, Kirkpatrick et al. (84), Press et al. (87 p. 452), and Brooks and Morgan (88) discuss the case where T decays geometrically in the number of cooling phases (number of times T is lowered according to step 5). Geman and Geman (86) present conditions such that if T decreases at the rate 1/log k (where k is the number of algorithm iterations) then the probability distribution for the iteration converges to a uniform distribution over all possible global minimum points. Szu and Hartley (89) present some formal arguments justifying a faster rate of decay for T: namely, having T decay at a rate 1/ k. Another area for different implementations is in step 2, where a new value is generated randomly. Probably the most common form of perturbation for continuous optimization is to add a p-dimensional Gaussian random variable to the current value curr (e.g., Ref. 6, p. 183). Alternative perturbations include the approach based on changing only one component of at a time (88), the approach using spherically uniform perturbations (90), and the approach using a multivariate Cauchy distribution to change all components (89,62). Aside from the general variations in implementation above, SAN is critically dependent on the specic values for various algorithm parameters and decision criteria. In particular, these include the initial temperature T, the specic distribution parameters chosen for the perturbation distribution (e.g., the covariance matrix for a Gaussian perturbation), the specic parameter(s) associated with the decay of T (e.g., the in Tnew Told, where Tnew and Told are the new and old temperatures if one is using a geometric decay), and the criterion for determining when to lower the temperature (e.g., the maximum allowable number of function evaluations before a lowering of T). There appears to be no fully satisfactory way to accommodate noisy function measurements in SAN, although some work on this aspect of the discrete (combinatorial) optimization problem has been carried out in Ref. 91. (One of the particularly appealing features of the stochastic approximation methods in the preceding three sections is that they handle noisy function measurements in an essentially seamless manner. Namely, the forms of the algorithms do not have to be altered, the convergence theory applies in the noisy case, and there is a gradual degradation in performance as the noise
level increases from 0, in contrast with the sudden large degradation in approaches such as SAN that use explicit decision criteria based on loss-function measurements). The primary difculty arises in the critical decision step 3, where curr is changed or not changed according to the value of L(new) L(curr) together with the random sampling associated with the Metropolis criterion. With noisy function measurements, the value of , now equal to y(new) y(curr) instead of L(new) L(curr), will be altered from its underlying true value according to noise-free function measurements. In fact, even a modest level of noise will frequently alter the sign of , which is likely to change the decision regarding the acceptance or rejection of the new point new. The obvious means by which one can attempt to cope with noise is to average several function measurements y( ) at each of the values curr and new when performing the comparison at step 3. However, this may dramatically increase the cost of optimization (specically, the total number of function evaluations required), since a large amount of averaging will often be required to effectively remove the noise, especially in the region around a local or global minimum where the function may be relatively at. An alternative way is to alter the acceptance criterion to accept the inherent errors that will be made with the noisy measurements. Hence we replace the criteria 0 or 0 with or , where is the function measurement noise standard deviation (or, more likely, an estimate of it) and the number of multiples of the standard deviation that will be tolerated. The rationale for such a change is that, while we are willing to accept a new that temporarily increases the loss function (a basic aspect of SAN), we are less willing to forgo a new that decreases the loss function. Changing the unconditional acceptance criterion from 0 to will allow for a greater number of cases where L(new) L(curr) even though y(new) y(curr) due to the noise. With 2, one can be sure (through the Chebyshev inequality of probability) that most such cases will be caught, although at the expense of letting some additional new values that increase the loss function be accepted. Evaluation of Simulated Annealing Although many positive studies regarding SAN have been reported in the literature (e.g., Refs. 9294), the author is unaware of any formal theoretical analysis comparing SAN with the gradient-free SA algorithms discussed in the preceding two sections. (Note that some connections of SAN with SA have been explored in Refs. 95 and 96. The basic idea there is, beginning with the basic SA recursion using either gradients or function measurements, to add a Monte Carlo-generated Gaussian noise term bkWk to the recursion, where bk 0 and Wk is the Gaussian random vector. This term is similar to SAN in that it will force the iterate out of local minima under some conditions that are established in the cited papers.) However, Fabian (97), Chin (103), Styblinski and Tang (62), and the author (together with several students in a graduate class he taught) have conducted some numerical comparisons. Fabian (97), Chin (103), and Styblinski and Tang (62) compare different forms of SAN against several random and deterministic search algorithms for problems with noise-free measurements of the loss function. The SAN algorithms generally compared poorly with the competing algorithms in these two papers. The authors studies involved four different loss functions: p 2 and p 20 fourth-order polynomials,
540
and the Example 1 (p 2, fourth-order polynomial) and Example 6 (p 10, trigonometric) functions in Ref. 62. The studies of the author considered both noise-free and noisy measurements of the four loss functions and compared SAN with simple random search mentioned in Ref. 7 (pp. 186189) and the global version of SPSA outlined in Ref. 80. Although SAN was sometimes superior to the random search method, the global SPSA method was signicantly more efcient than SAN in all cases considered, and, even more dramatically, was sometimes convergent when SAN seemed unable to obtain a solution anywhere near the true optimal (this was most apparent in the Example 6 problem of Ref. 62, which had a very large number of local minima, one of which always seemed to form a trap for SAN). In order to have a fair comparison in performing these studies, the algorithm coefcients (e.g., the gains for the SA algorithms and the decay rate and other coefcients mentioned above for SAN) were tuned to approximately optimize performance for the competing algorithms. Since one must be careful in drawing conclusions beyond the specic cases in any numerical analysis, the above should not indicate that SAN is always a poor performer. Rather, these results should be a cautionary note in view of some of the positive results reported elsewhere. Certainly, SAN does have a role in the area of global optimization, as evidenced by its popularity and positive performance in some challenging problems. Furthermore, the above studies were for problems with smooth (differentiable) loss functions, and SAN (unlike most SA algorithms) has the ability to deal with nonsmooth loss functions.
CONCLUDING REMARKS This article has surveyed several important algorithms with stochastic optimization. The focus was on algorithms within the stochastic approximation and simulated annealing classes. Broadly speaking the gradient-based (Robbins Monro) SA algorithms are most appropriate when enough prior information is available about the system so that it is possible to determine the gradient of the observed loss function. It is expected that when such information is available, this type of algorithm may be the most efcient means for optimization within the stochastic setting, especially when coupled with one of the algorithm acceleration techniques such as iterate averaging or second-order methods. However, it was also pointed out that in many applications, it is difcult or impossible to obtain the gradient information needed for the application of the gradient-based methods. Then one is faced with carrying out the optimization using only the (possibly noisy) measurements of the loss function itself. In that context, we emphasized three different approaches: nite-difference and simultaneous perturbation SA (both of which are based on approximating the unknown gradient vector) and simulated annealing. The FDSA method is the oldest and best-known of these approaches, and can work in a variety of applications. The newer SPSA approach will, however, usually be more efcient than FDSA, with the gain in efciency being approximately equal to the dimension of the vector being optimized (i.e., the SPSA algorithm will use only onepth the number of function evaluations to obtain the same level of statistical accuracy as FDSA). Unlike the SA algorithms, simulated annealing is focused on global (as opposed
to local) optimization problems. Simulated annealing is based on an intriguing analogy to the cooling of materials and the achievement of an optimal state for the material by cooling neither too fast nor too slow. While some positive experience has been reported with optimization by simulated annealing, it appears that there exist more efcient algorithms for many problems. Although stochastic optimization has the potential for treating a broader class of problems than many traditional deterministic techniques, their application may sometimes be a challenge. A problem common to all stochastic optimization techniques (including those not discussed in this article, such as the genetic algorithm, sequential random search, and evolutionary programming) is that values must be specied for the algorithms tunable coefcients. All stochastic optimization techniques have such coefcients (the gains in SA, terms associated with the cooling schedule and probability of accepting a step in simulated annealing, etc.). These coefcient values are typically problem-dependent and can have a profound effect on the performance of an algorithm. Stochastic optimization is playing an ever larger role in optimization, as it allows for the treatment of problems such as global optimization and noisy loss-function evaluations that arise frequently in areas such as network analysis, neural network training, image processing, and nonlinear control. It is expected that the role of stochastic optimization will continue to grow as modern systems increase in complexity and as population growth and dwindling natural resources force tradeoffs that were previously unnecessary. Stochastic optimization allows for the treatment of a broader range of problems than possible with only standard deterministic methods. The algorithms of this article provided a sampling of several important stochastic optimization methods. ACKNOWLEDGEMENTS This work was partially funded by U.S. Navy Contract N00024-97-C-8119 and the JHU/APL Independent Research and Development Program. The author appreciates the comments of two anonymous reviewers and G. Cauwenberghs, V. Fabian, M. Fu, J. L. Maryak, B. T. Polyak, M. A. Styblinski, and S. Yakowitz on an earlier version of parts of this article. BIBLIOGRAPHY
1. G. L. Nemhauser and L. A. Wolsey, Integer and Combinatorial Optimization, New York: Wiley, 1988. 2. K. S. Tang et al., Genetic algorithms and their applications, IEEE Sig. Process. Mag., November 1996, pp. 2237. 3. T. Back, Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, and Genetic Algorithms, Oxford, UK: Oxford University Press, 1996. 4. F. J. Solis and J. B. Wets, Minimization by random search techniques, Math. Oper. Res., 6: 1930. 5. A. A. Zhiglavsky, Theory of Global Random Search, Boston: Kluwer Academic, 1991. 6. J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-fuzzy and Soft Computing, Upper Saddle River, NJ: Prentice-Hall, 1997. 7. W. H. Fleming, Functions of Several Variables, New York: Springer Verlag, 1977. 8. P. Glasserman, Gradient Estimation via Perturbation Analysis, Boston: Kluwer, 1991.
STOCHASTIC OPTIMIZATION, STOCHASTIC APPROXIMATION AND SIMULATED ANNEALING 9. H. White, Some asymptotic results for learning in single hidden layer feedforward neural networks, J. Amer. Statist. Assoc., 84: 10031013, 1989. 10. H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist., 22: 400407, 1951. 11. J. R. Blum, Approximation methods which converge with probability one, Ann. Math. Statist., 25: 382386, 1954. 12. D. Ruppert, Stochastic approximation, in B. K. Ghosh and P. K. Sen (eds.), Handbook of Sequential Analysis, New York: Marcel Dekker, 1991, pp. 503529. 13. J. S. Rustagi, Optimization Techniques in Statistics, New York: Academic Press, 1994. 14. L. Ljung, Analysis of recursive stochastic algorithms, IEEE Trans. Autom. Control, AC-22: 551575, 1977. 15. H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, New York: Springer Verlag, 1978. 16. A. Benveniste, M. Metivier, and P. Priouret, Adaptive Algorithms and Stochastic Approximations, Berlin: Springer Verlag, 1990. 17. T. L. Lai, Stochastic approximation and sequential search for optimum, in L. M. Le Cam and R. A. Olshen (eds.), Proc. Berkeley Conf. Honor of Jerzy Neyman and Jack Kiefer, Vol. II, Belmont, CA: Wadsworth, 1985. 18. V. Fabian, On asymptotic normality in stochastic approximation, Ann. Math. Statist., 39: 13271332, 1968. 19. J. Sacks, Asymptotic distribution of stochastic approximation procedures, Ann. Math. Statist., 29: 373405, 1958. 20. R. G. Laha and V. K. Rohatgi, Probability Theory, New York: Wiley, 1979. 21. L. Ljung, G. Pug, and H. Walk, Stochastic Approximation and Optimization of Random Systems, Basel: Birkha user, 1992. 22. A. Okamura, T. Kirimot, and M. Kondo, A new normalized stochastic approximation algorithm using a time-shift parameter, Electron. Commun. Jpn., Part 3, 78: 4151, 1995. 23. C. M. Kuan and K. Hornik, Convergence of learning algorithms with constant learing rates, IEEE Trans. Neural Net., 2: 484 489, 1991. 24. O. Macchi and E. Ededa, Second-order convergence analysis of stochastic adaptive linear ltering, IEEE Trans. Autom. Control, AC-28: 7685, 1982. 25. H. J. Kushner and H. Huang, Asymptotic properties of stochastic approximations with constant coefcients, SIAM J. Control Optim., 19: 87105, 1983. 26. G. C. Pug, Stochastic minimization with constant step size: Asymptotic laws, SIAM J. Control Optim., 24: 655666, 1986. 27. G. G. Yin, Asymptotic optimal rate of convergence for an adaptive estimation procedure, in T. E. Duncan and B. Pasik-Duncan (eds.), Stochastic Theory and Adaptive Control, New York: Springer Verlag, 1992, pp. 480489. 28. G. G. Yin and K. Yin, Passive stochastic approximation with constant step size and window width, IEEE Trans. Autom. Control, 41: 90106, 1996. 29. S. Mukherjee and T. L. Fine, Online steepest descent yields wieghts with nonnormal limiting distributions, Neural Comp., 8: 10751084, 1996. 30. M. Metivier and P. Priouret, Applications of a Kushner and Clark lemma for general classes of stochastic algorithms, IEEE Trans. Inf. Theory, IT-30: 14401451, 1984. 31. J. E. Dennis and R. B. Schnabel, A view of unconstrained optimization, in G. L. Nemhauser et al. (eds.), Optimization (Handbooks in OR and MS, Vol. 1) New York: Elsevier, 1989, pp. 172. 32. H. Kesten, Accelerated stochastic approximation, Ann. Math. Statist., 29: 4159, 1958. 33. B. Delyon and A. Juditsky, Accelerated stochastic approximation, SIAM J. Control Optim., 3: 868881, 1993.
541
34. L. Ljung and T. Soderstrom, Theory and Practice of Recursive Identication, Cambridge, MA: MIT Press, 1983. 35. C. Z. Wei, Multivariate adaptive stochastic approximation, Ann. Statist., 15: 11151130, 1987. 36. D. Ruppert, A NewtonRaphson version of the multivariate RobbinsMonro procedure, Ann. Statist., 13: 236245, 1985. 37. B. T. Polyak and A. B. Juditsky, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30: 838 855, 1992. 38. H. J. Kushner and J. Yang, Stochastic approximation with averaging: Optimal rates of convergence for general processes, SIAM J. Control Optim., 31: 10451062, 1993. 39. H. J. Kushner and J. Yang, Stochastic approximation with averaging and feedback: Rapidly convergent on-line algorithms, IEEE Trans. Autom. Control, 40: 2434, 1995. 40. I. J. Wang, Analysis of stochastic approximation and related algorithms, Ph.D. Thesis, Purdue University School of Electrical Engineering, 1996. 41. J. L. Maryak, Some guidelines for using iterate averaging in stochastic approximation, in Proc. IEEE Conf. Decision Control, San Diego, CA, 22872290, 1997. 42. C. A. Goodsell and D. L. Hanson, Almost sure convergence for the RobbinsMonro process, Ann. Probab., 4: 890901, 1976. 43. S. N. Evans and N. C. Weber, On the almost sure convergence of a general stochastic approximation procedure, Bull. Austral. Math. Soc., 34: 335342, 1986. 44. B. Kosko, Neural Networks and Fuzzy Systems, Englewood Cliffs, NJ: Prentice-Hall, 1992. 45. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning internal representations by error propagation, in Parallel Distributed Processing, Vol. 1, Cambridge, MA: MIT Press, 1986, pp. 318362. 46. R. Suri, Perturbation analysis: The state of the art and research issues explained via the Gl/G/1 queue, Proc. IEEE, 77: 114 137, 1989. 47. P. LEcuyer and P. W. Glynn, Stochastic optimization by simulation: Convergence proofs for the Gl/G/1 queue in steady state, Management Sci., 40: 15621578, 1994. 48. G. C. Pug, Optimization of Stochastic Models: The Interface between Simulation and Optimization, Boston: Kluwer, 1996. 49. M. Fu and J.-Q. Hu, Conditional Monte Carlo: Gradient Estimation and Optimization Applications, Norwell, MA: Kluwer, 1997. 50. E. Abreu et al., A new efcient approach for the removal of impulse noise from highly corrupted images, IEEE Trans. Image Proc., 5: 10121025, 1996. 51. I. Cha and S. A. Kassam, RBFN restoration of nonlinearity degraded images, IEEE Trans. Image Proc., 5, 964975, 1996. 52. J. Kiefer and J. Wolfowitz, Stochastic estimation of a regression function, Ann. Math. Statist., 23: 462466, 1952. 53. V. Fabian, Stochastic approximation, in J. J. Rustigi (ed.), Optimizing Methods in Statistics, New York: Academic Press, 1971, pp. 439470. 54. P. Glasserman and D. D. Yao, Some guidelnes and guarantees for common random numbers, Management Sci., 38: 884908. 55. P. LEcuyer and G. Yin, Budget-dependent convergence rate of stochastic approximation, SIAM J. Optim., 8: 217247, 1998. 56. N. L. Kleinman, J. C. Spall, and D. Q. Naiman, Simulationbased optimization with stochastic approximation using common random numbers, Management Sci., submitted. 57. J. C. Spall, Developments in stochastic optimization algorithms with gradient approximations based on function measurements, in J. D. Tew, M. S. Manivannan, D. A. Sadowski, and A. F. Seila (eds.), Proc. Winter Simulation Conf., 1994, pp. 207214. 58. J. R. Blum, Multidimensional stochastic approximation methods, Ann. Math. Statist., 25: 737744, 1954.
542
STOCHASTIC OPTIMIZATION, STOCHASTIC APPROXIMATION AND SIMULATED ANNEALING 79. P. Sadegh and J. C. Spall, Optimal random perturbations for multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Control, 43: in press. 80. D. C. Chin, A more efcient global optimization algorithm based on Styblinski and Tang, Neural Netw., 7, 573574, 1994. 81. P. Sadegh, Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation, Automatica, 33: 889892, 1997. 82. J. C. Spall, A one-measurement form of simultaneous perturbation stochastic approximation, Automatica, 33: 109112, 1997. 83. J. C. Spall, Accelerated second-order stochastic optimization using only function measurements, Proc. IEEE Conf. Decision Control, 1997, pp. 14171424. 84. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Optimization by simulated annealing, Science, 220: 671680, 1983. 85. N. Metropolis et al., Equation of state calculations by fast computing machines, J. Chem. Phys., 21: 10871092, 1953. 86. S. Geman and D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., PAMI-6: 721741, 1984. 87. W. H. Press et al., Numerical Recipes in C, 2nd ed., Cambridge: Cambridge University Press, 1992. 88. S. P. Brooks and B. J. T. Morgan, Optimization using simulated annealing, Statistician, 44: 241257, 1995. 89. H. Szu and R. Hartley, Fast simulated annealing, Phys. Lett. A, 122: 157162, 1987. 90. I. O. Bohachevsky, M. E. Johnson, and M. L. Stein, Generalized simulated annealing for function optimization, Technometrics, 28: 209217, 1986. 91. S. Gelfand and S. K. Mitter, Simulated annealing with noisy or imprecise energy measurements, J. Optim. Theor. Appl., 62: 4962, 1989. 92. P. J. M. Van Laarhoven, Theoretical and Computational Aspects of Simulated Annealing, CWI Tract 51, Amsterdam: Center for Mathematics and Computer Science, 1988. 93. R. Azencott (ed.), Simulated Annealing: Parallelization Techniques, New York: Wiley, 1992. 94. R. V. V. Vidal (ed.), Applied Simulated Annealing, Berlin: Springer Verlag, 1993. 95. S. Gelfand and S. K. Mitter, Simulated annealing type algorithms for multivariate optimization, Algorithmica, 6: 419 436, 1991. 96. S. Gelfand and S. K. Mitter, Metropolis-type annealing algorithms for global optimization in Rd, SIAM J. Control Optim., 31: 111131, 1993. 97. V. Fabian, Simulated annealing simulated, Comput. Math. Appl., 33: 8194, 1997. 98. H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications, New York: Springer-Verlag, 1997. 99. J. Tsitsiklis and B. Van Roy, An analysis of temporal-difference learning with function approximation, IEEE Trans. Autom. Control, 42: 674690, 1997. 100. J. H. Venter, An extension of the RobbinsMonro procedure, Ann. Math. Statist., 38: 181190, 1967. 101. V. Solo and X. Kong, Adaptive Signal Processing Algorithms, Englewood Cliffs, NJ: Prentice-Hall, 1995. 102. J. C. Spall, Adaptive stochastic approximation by the simultaneous perturbation method, IEEE Trans. Autom. Control, submitted, 1998. 103. D. C. Chin, The simultaneous perturbation method for processing magnetospheric images, Opti. Eng., submitted, 1998.
59. J. C. Spall, A stochastic approximation technique for generating maximum likelihood parameter estimates, in Proc. Amer. Control Conf., 1987, pp. 11611167. 60. J. C. Spall, Multivariate stochastic approximation using a simultaneous peturbation gradient approximation, IEEE Trans. Automat. Control, 37: 332341, 1992. 61. B. T. Polyak and Y. Z. Tsypkin, Pseudogradient adaptation and training algorithms, Automation Remote Control, 34: 377397, 1973. 62. M. A. Styblinski and T. S. Tang, Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing, Neural Netw., 3: 467483, 1990. 63. D. C. Chin, Comparative study of stochastic algorithms for system optimization based on gradient approximations, IEEE Trans. Syst. Man. Cybern. B, 27: 244249, 1997. 64. S. D. Hill and M. C. Fu, Transfer optimization via simultaneous perturbation stochastic approximation, in C. Alexopoulos, K. Kang, W. R. Lilegdon, and D. Goldsman (eds.), Proc. Winter Simulation Conf., 1995, pp. 242249. 65. M. C. Fu and S. D. Hill, Optimization of discrete event systems via simultaneous perturbation stochastic approximation, Trans. Inst. Ind. Eng., 29: 233243, 1997. 66. F. Rezayat, On the use of an SPSA-based model-free controller in quality improvement, Automatica, 31: 913915, 1995. 67. Y. Maeda, H. Hirano, and Y. Kanata, A learning rule of neural networks via simultaneous perturbation and its hardware implementation, Neural Netw., 8: 251259, 1995. 68. G. Cauwenberghs, Analog VLSI autonomous systems for learning and optimization, Ph.D. Dissertation, California Institute of Technology, 1994. 69. J. C. Spall and J. A. Cristion, Nonlinear adaptive control using neural networks: Estimation based on a smoothed form of simultaneous perturbation gradient approximation, Statist. Sinica, 4: 127, 1994. 70. J. C. Spall and J. A. Cristion, A neural network controller for systems with unmodeled dynamics with applications to wastewater treatment, IEEE Trans. Syst. Man. Cybern. B, 27: 369 375, 1997. 71. A. Alessandri and T. Parisini, Nonlinear modeling and state estimation in a real power plant using neural networks and stochastic approximation, in Proc. Amer. Control Conf., 1995, pp. 15611567. 72. P. Sadegh and J. C. Spall, Optimal Sensor Conguration for Complex Systems, in Proc. Test Technol. Symp. (sponsored by U.S. Army TECOM), http://www.atc.army.mil/ tecom/tts/ proceed/optsenr.html, 1996. 73. D. C. Chin and R. H. Smith, A trafc simulation for mid-Manhattan with model-free adaptive signal control, in Proc. Summer Comput. Simulation Conf., (sponsored by the Society for Computer Simulation) 1994, pp. 296301. 74. J. C. Spall, A stochastic approximation algorithm for large-dimensional systems in the KieferWolfowitz setting, in Proc. IEEE Conf. Decision Control, 1988, pp. 15441548. 75. H. F. Chen, T. E. Duncan, and B. Pasik-Duncan, A stochastic approximation algorithm with random differences, in Proc. 13th IFAC World Congr., Vol. H, 1996, pp. 493496. 76. I.-J. Wang and E. K. P. Chong, A deterministic analysis of simultaneous perturbation stochastic approximation, in Proc. 30th Conf. Inf. Sci. Syst., 1996, pp. 918922. 77. J. Dippon and J. Renz, Weighted means in stochastic approximation of minima, SIAM J. Control Optim., 35, 18111827, 1997. 78. J. C. Spall, Implementation of the simultaneous perturbation method for stochastic optimization, IEEE Trans. Aerosp. Electron. Syst., 34: in press.
JAMES C. SPALL
The Johns Hopkins University
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Stochastic Systems Standard Article Bozenna PasikDuncan1 1University of Kansas Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1045 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (210K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Preliminary Results The Filtering Problem Stochastic Control Least Squares and Continuous-Time Stochastic Adaptive Control Weighted Least Squares and Continuous-Time Adaptive LQG Control About Wiley InterScience | About Wiley | Privacy | Terms & Conditions Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
STOCHASTIC SYSTEMS
543
STOCHASTIC SYSTEMS
Many control systems in practice are subject to imperfectly known disturbances which may be taken as random. Such disturbances have been ignored in the study of deterministic control problems. In many control systems some unknown parameters occur. The system behavior depends on the parameters and the fact that the value of the parameters is unknown makes the system unknown. Some crucial information concerning the system control is not available to the controller, and this information should be learned during the systems performance. The problem described is the problem of adaptive control. The adaptive control problem can be considered the identication problem and the control problem. A solution to an adaptive control problem will be understood as a solution to the identication problem and to the control problem. We focus on control and adaptive control of continuous-time linear stochastic systems. The theory of adaptive control of continuous-time linear stochastic systems has been recently developed as an important application of the stochastic control theory in engineering, biology, economics, nance, and manufacturing systems. Continuous-time linear systems are an important and commonly used class of systems (1,2). It is assumed that the models evolve in continuous time rather than discrete time because this assumption is natural for many models and it is important for studying discrete time models when the sampling rates are large and for analyzing numerical round-off errors. Stochastic systems are described by linear stochastic differential equations. The general approach to adaptive control described here exhibits a splitting or separation of the problems of identication of the unknown parameters and adaptive control. Maximum likelihood, least squares, or weighted least squares estimators are used to identify the unknown constant parameters (3,4,5). These estimates are given recursively and are strongly consistent, which means that the family of estimates converges to the true value of the parameter with the probability one. It turns out that for some cases the weighted least squares estimator is strongly consistent whereas the least squares estimator is not. The adaptive control constructed by the so-called certainty equivalence principle, that is, the optimal stationary control, is computed by replacing the unknown true parameter values by the current estimates of these values. Because the optimal stationary controls are continuous functions of the unknown parameters, the selftuning property is veried, which means that asymptotically adaptive control using the estimate of the unknown parameter is as good as optimal control if we know the system. It is also shown that the family of average costs using the control from the certainty equivalence principle converges to the optimal average cost. This veries the self-optimizing property. In describing a stochastic control model, the kind of information available to the controller at each instant plays an important role. Several situations are possible: (1) The controller has no information during the systems operation. In this case a function of time is chosen as a control. These controls are often called open loop as distinct from closed-loop or feedback controls, in which the actual value of control input at time t is a function of the observation at time t (6). The distinction between open-loop and feedback control is fundamental. An open-loop control is deterministic whereas a feedback law determines a random control process.
An open-loop control is formally a special case of a closed-loop control. For a deterministic system the reverse also holds. The important point is that this is not true for stochastic systems. (2) The controller knows the state of the system at each time t. We call this the case of complete observations. (3) The controller has partial knowledge of the system states. This is called the case of partial observations. For a deterministic control model, the state of the system at any time t can be deduced from the initial data and the control used up to time t by solving the corresponding differential equation. Thus observing the current state of the system at each time t does not really give more information than knowing the initial data. For stochastic control systems, there are many paths which the system states follow given the control and the initial data. In the stochastic case, the best system performance depends on the information available to the controller at each time t. When we consider continuous-time stochastic control problems, the system states are described by the stochastic process that changes according to a stochastic differential equation. We need to be aware that to deal with more realistic mathematical models of the situation we allow for some randomness in some of the coefcients of an ordinary differential equation (7). Randomness arises when we consider real life situations with some disturbances or noise. These disturbances or noise are modeled by a stochastic process, often by the so-called Brownian motion or Wiener process (8,9), which we dene later. Stochastic control problems have been difcult to solve explicitly (10,11,12). One of the fundamental results of the stochastic control problem, where the optimal control is obtained explicitly, is the linear quadratic Gaussian (LQG) problem (5,6,13). It is called linear because the system is described by a linear difference equation (in the discrete-time case) or by a linear differential equation (in the continuous-time case). It is called quadratic because the cost criterion to be minimized or maximized is quadratic. It is quadratic in a state and in a control. It is called Gaussian because the noise is modeled by a Gaussian process. Another stochastic control problem that has been solved explicitly is the portfolio selection and consumption model known as Mertons model (14) and described in Example 2 below. A linear quadratic Gaussian problem and a portfolio selection and consumption model were the two of a very few stochastic control problems that have been solved explicitly. Recently Duncan and Upmeir (15) provided a pretty wide class of stochastic control problems that are solved explicitly. The stochastic control problems are the control of Brownian motion in noncompact symmetric spaces by a drift vector eld. In solving stochastic control problems the basic difculty lies in solving the so-called HamiltonJacobiBellman (HJB) equation (11). To solve this equation it is assumed that the solution, the value function, is smooth. Typically the solution is not smooth enough to satisfy the HJB equation in a classical sense. The theory of viscosity solutions, rst introduced by M. G. Grandall and P. L. Lions, provides a convenient framework in which to study HJB equations. See Ref. 16 on the theory of viscosity solutions. Stochastic calculus provides powerful mathematical tools for solving stochastic control problems and stochastic adap-
544
STOCHASTIC SYSTEMS
tive control problems. The mathematical tools of stochastic calculus most commonly used are stochastic integral (17), s differential rule, and martingales (7,8). These elements Ito of stochastic calculus together with the limit theorems of the probability theory, the theory of stochastic processes (18,19,20,21,22), and the theory of stochastic differential equations are extremely useful in proving the results of stochastic control and stochastic adaptive control of continuous time. This is another motivation for considering stochastic control problems in continuous time rather than in discrete time. There has been always a greater chance of stronger results because those mathematical techniques are more sophisticated and because of that, more powerful. Stochastic adaptive control problems are applications of stochastic control theory (23,24,25,26). Because we use the certainty equivalence control as an adaptive control, it means that we need the optimal control given explicitly or an almost optimal control (27). The stochastic adaptive control problems with discrete time have been under investigation for a long time. Many references are in the books in Refs. 5, 6, 23, and 28, and in a few others (17,24,29). Continuous-time stochastic adaptive control problems have been investigated relatively recently. Many results have been obtained under conditions difcult to verify. The goal of further investigation was to provide results which could be obtained under the same assumptions as stochastic control problems, that is, controllability and observability which have been always considered the most natural conditions for optimal control (30). The results presented here are obtained under these kinds of conditions for linear systems. Corresponding results for nonlinear systems do not yet exist. Stochastic adaptive control problems with continuous time are under intensive investigation. It is important to mention that the problems of stochastic adaptive control previously described are problems of stochastic adaptive control with complete observation of the state. Let us describe the adaptive control scheme used here. The control as feedback gains is obtained from solving an innitetime, quadratic cost control problem by replacing the correct values of parameters in this solution by the estimates of parameters at time t to obtain the feedback gain at time t. The estimates of the parameters are the least squares estimates or the weighted least squares estimates. The least squares estimate based on the observations of the state until time t is used as the correct value of the parameter to solve the innite-time control problem by solving the algebraic Riccati equation. This method gives a feedback gain at each time t and therefore a control policy. Problems of continuous-time stochastic adaptive control with partial observations in a general setting remain still open. Partial results have been provided by Bertrand (31). For discrete time see Ref. 32 and references therein. Many problems with partial observations of a state can be solved by the ltering theory (33). To recognize the differences among ltering problems, control problems, and adaptive control problems, let us consider some simple examples. Example 1. Filtering problem (7,11,34). Suppose that we would like to improve our knowledge about the solution of a differential equation so we observe X(s) at times s t. Let us imagine that the X(t) is the charge at time t at a xed point
in an electric circuit. However, due to inaccuracies in our measurements we do not really measure X(s) but a perturbed version of it: Y(s) X(s) noise. The ltering problem is, what is the best estimate of X(t) satisfying Y(s) X(s) noise based on the observations Y(s) where s t? Initially the problem is to lter the noise away from the observations optimally. In 1961 Kalman and Bucy (33) proved what is now known as the Kalman-Bucy lter. Basically the lter gives a procedure for estimating the state of a system which satises a linear differential equation, based on a series of observations with a noise. Example 2. Stochastic control problem. Portfolio Selection and Consumption Model. Let X(t), t 0 be the wealth of an individual at time t who invests his wealth in two types of assets: the safe asset has the return rate r and the risky asset has the average return rate . The wealth X(t) at time t changes according to the stochastic differential equation dX (t ) = r[1 U1 (t )]X (t ) dt + U1 (t )X (t )[ dt + dW (t )] U2 (t ) dt where U1(t) is the fraction of the wealth invested in the risky asset at time t and U2(t) is the consumption rate at time t; r, , are constants with r and 0; and (W(t), t 0) is a real-valued standard Wiener process. The controls are naturally constrained as 0 U1(t) 1 and U2(t) 0. The stochastic control problem is to maximize the expected discounted total utility I (U ) = Ey
s
exp(t )F [U2 (t )] dt ,
y = W (s )
where F(u) u with 0 1 is the utility function and 0 is the discount rate, Example 3. Stochastic adaptive control problem. Portfolio Selection and Consumption Model. Consider the situation described in Example 2. For the adaptive control problem, it is assumed that is an unknown parameter such that [a1, a2] with r a1. The adaptive control procedure in this adaptive setting is to dene the control at a time t, that is, the portfolio selection and the consumption rate, using the optimal innite-time control, where the estimate of the unknown parameter at time t is used for the unknown parameter. Example 4. Stochastic adaptive control problem. Manufacturing Model (35). Consider a manufacturing system that produces n distinct part types using m identical machines. Let U(t) n denote the vector of productions rates, X(t) n the vector of total inventories/backlogs, and Z(t) n the vector demands. These processes are related by the following stochastic differential equation: dX (t ) = U (t ) dt dZ(t ), X ( 0 ) = x0 R n
The demand Z( ) is given by the following stochastic differential equation: dZ(t ) = Z dt + dW (t ) Z (0 ) = z0
STOCHASTIC SYSTEMS
545
where Z is a vector of unknown constants, 0 is a small parameter, is a given n n matrix, and [W(t), t 0] is a standard n-valued Wiener process dened on a complete probability space (, F , P). The manufacturing system under consideration consists of machines subject to breakdown and repair. Let M 0, 1, . . ., m denote the set of machine total capacity states, and let the process [(t), t 0] where (t) M denotes the total capacity process for the manufacturing system. Because only a nite amount of production capacity is available at any given time t, an upper bound is imposed on the production rate U(t). For example, in the one-dimensional case (n 1), the production constraint is 0 U(t) (t). The cost function J is dened by J [U ( )] = E
0
Let us dene now the unfair games in which we expect to lose (supermartingale) and to win (submartingale) Denition 2 A stochastic process (T(t), t 0) is a supermartingale with respect to the increasing family of -algebras (F t, t 0) if it satises properties (1) and (2) of Denition 1 and (3) if s t, then E [X (t )|Fs ]
6 X (s )
a.s.
(2)
(X(t), t 0) is a submartingale with respect to the increasing family of -algebras (F t, t 0) if it satises properties (1) and (2) of Denition 1 and (3) if s t, then E [X (t )|Fs ]
e t G[X (t ), U (t )] dt
> X (s )
a.s.
(3)
where G is the running cost of inventory/backlog and production and 0 is the discount rate. The problem is to nd an admissible control U( ) that minimizes J[U( )]. A noise that appeared in the above example is described by a Wiener process. A Wiener process plays a very important and exceptional role in stochastic control theory. It is useful for modeling disturbances which corrupt transmitted information and also for modeling controlled system disturbances. A Wiener process also occurs in the KalmanBucy lter (33). It became as a model of Brownian motion, very popular in physics since Einsteins and Smoluchowskis work, but it is not well known that it appeared for the rst time in the scientic literature as a model of a price process in Bacheliers Ph.D. dissertation in 1900. PRELIMINARY RESULTS We shall assume that (, F , P) is a probability space and that F t, t 0 is a nondecreasing, right-continuous family of sub--algebras of F . We assume that the ltration (F t) is complete with respect to P, that is, each F t contains all the P-null sets of F . Martingales as Important Tools in Proofs (8,9) A stochastic process is a parametrized collection of random variables. We consider collections (and so processes) for which the parameter t takes values in positive real numbers. It is common to call this kind of process a continuous-time stochastic process. Let us start by introducing the concept of martingale. The intuitive idea of martingale is related to the description of a fair game: we are in fact in a situation in which the fortune expected after a certain time is exactly equal to what we have. Then we dene this fair game. Denition 1 A stochastic process (X(t), t 0) is a martingale with respect to the increasing family of -algebras (F t, t 0) if (1) X(t) is F t measurable; (2) EX(t) for every t 0; (3) if s t, then E [X (t )|Fs ] = X (s ) a.s. (1)
where a.s. stands for almost surely. We are interested in the martingale theory because of its strong connection with stochastic calculus and because of the properties that the processes dened previously have. The rst property that we illustrate is the DoobKolmogorov inequality that tells how (and when) one can get information about a family of ordered, random variables based on the knowledge of the last one: Theorem 3. Let (Y(t), F t, t [0, T]) be a submartingale (with right continuous sample paths). Then
t [0 , T ]
sup Y (t )
E [Y + (T )] >x 6 1 x
(4)
where x 0 and Y(T) max(Y(T), 0). One of the consequences of this result is the martingale convergence theorem which is commonly used in proving convergence results for stochastic adaptive control problems. Theorem 4. Let (Y(t), F t, t [0, T]) be a nonnegative submartingale. If supt0E[Y(t)] , then
t
lim Y (t ) = Y
a.s.
(5)
and E[Y] . The Wiener Process Used for Modeling the Noise Denition 5. The stochastic process (W(t), t 0) is a Brownian motion (or a Wiener process) if (1) W(0) 0; (2) for t 0, W(t s) W(t) is a random variable that is normally distributed with mean zero and variance 2s where 2 is a constant; and (3) the process has independent increments (i.e., if 0 t1 t2 tn, then W(t2) W(t1), W(t3) W(t2), . . ., W(tn) W(tn1) are mutually independent random variables). If 2 1, then the process is called standard Brownian motion. The following result provides important information about the quadratic variation of the Wiener process:
546
STOCHASTIC SYSTEMS
Theorem 6. Consider a family of partitions Pnn 1 of [0, T] n n n where Pn tin0 i n, 0 t0 t1 tn T with Pn Pn1 for all n and such that Pnn 1 is dense in [0, T] (i.e., limn [supi(tin1 tin)] 0). Let W(t), t 0] be a standard Brownian motion process. Then
s sense) satisfactorily. We start by dening the gration (in Ito integral for simple processes: Denition 10 Let F t be the -algebra generated by [W(s), s t]. We call (t), t [0, T] a simple process, if there is a partition 0 t0 t1 tn T such that (t ) = j t [t j , t j + 1 ) (11)
n1 n
lim
i= 0
n 2 [W (tin + 1 ) W (ti )] = T
a. s.
(6)
and
n1 n
n1 i0
and j is measurable with respect to F tj.

2
lim E
i= 0
n i1
[W (tin +1 )
n i
W (tin )]2
2
=0
(7)
Dene the stochastic integral for simple processes when t [tj, tj1) as
t j1
that is (W(t ) (W(t )) converges to t both almost surely and in L2. It follows from this result that almost every sample path of a Brownian motion process has innite arc length on every nonempty closed interval. Moreover it follows that there is not a closed interval in which Brownian motion is differentiable (otherwise by the mean value theorem we would have convergence to zero). Much more than this is true: Theorem 7. Almost all sample paths of the Wiener process are nowhere differentiable.
(s ) dW (s ) =
0 i= 0
i [W (ti+ 1 ) W (t j )] + j [W (t ) W (t j )] (12)
Note that the integral is linear and continuous (almost surely) in t. We want to preserve these properties and at the same time we want to extend the class of functions that we can integrate. To do this, we need the following result: Proposition 11. Let (t), t [0, T] be a process adapted to F t, t [0, T] such that
T
2 (s ) ds < a.s.
0
(13)
Let us give another important result about the sample path behavior of the Wiener process: Theorem 8. The strong law of large numbers for Brownian motion: W (t ) = 0 a.s. lim t t (8)
Then there is a sequence of simple processes n(t), t [0, T] such that for every 0
T n
lim P
0
|n (t ) (t )|2 dt >
=0
(14)
If (W(t), t 0) is a Brownian motion process with continuous sample paths and we let F t be the -algebras generated by [W(s), s 0], it follows from the independence of increments property of Brownian motion that W(t), F tt [0, T] is a square integrable martingale with continuous sample paths. The converse of this statement is true: Theorem 9. If W(t), F t, t [0, T] is a square integrable martingale with continuous sample paths such that for t s E {[W (t ) W (s )]2 | Fs } = t s then W(t), t [0, T] is a Brownian motion process. Stochastic Integrals In this section we dene the stochastic integral
t
Now if we dene Yn as the integral of the simple process , that is,

t
Yn (t ) =
0
(s ) dW (s )
(15)
then using Proposition 11, the family Yn converges to a process Y uniformly in probability. Note that the uniform convergence and the continuity of sample paths for Yn imply the continuity of the sample paths of the process Y. Now we can dene the stochastic integral as follows:
t
(9)
0
(s ) dW (s ) = Y (t )
(16)
The construction does not depend on the sequence of simple processes used, and the integral is unique up to indistinguishability, that is, if
t
(s ) dW (s )
0
(10) and
(s ) dW (s ) = Y (t )
0
(17)
The unbounded variation property of Brownian motion sample paths described in the previous section does not allow us to use the denition of the RiemannStieljes integral. Nevertheless by proceeding similarly we can dene stochastic inte-
(s ) dW (s ) = Z(t )
0
(18)
STOCHASTIC SYSTEMS
547
then PY(t) Z(t) for all t [0, T] 1. The following result is the strong law of large numbers for stochastic integrals (22). Theorem 12. Let (W(t), t 0) be a standard Wiener process, and let F t be the -algebra generated by (W(s), s t). Let ( f(t), t 0) be a random process adapted to (F t, t 0) such that for 0 T
T
We mentioned previously that we shall present the differential rule that we must use in stochastic calculus. This rule . was introduced by Ito s Lemma). For i 1, 2, . . ., k let ( fi(t), t Theorem 14. (Ito [0, T]) and (gi(t), t [0, T]) be two processes adapted to (F t, t [0 t]) such that
T
f (t ) dt < a.s.
2 0
(19)
f 2 (s ) ds < a.s.
0 T
(26) (27)
and
0
g2 (s ) ds < a.s.
0
f 2 (t ) dt = a.s.
(20)
and let F : k1 be twice continuously differentiable in the rst k variables and continuously differentiable in the last variable. Let dXi (t ) = g j (t ) dt + f i (t ) dW (t ) (28)
Then
T
f (t ) dW (t )
T 0
lim
0 T
= 0 a.s. f (t ) dt
2
(21)
and Y (t ) = F [X1 (t ), . . ., Xk (t ), t ] Then (29)
An interesting feature of the stochastic integral is that it does not satisfy the ordinary calculus rules of integration. A standard example of this situation is the fact that
t
dY (t ) =
i= 1
Fi dXi (t ) +
1 2
k i, j = 1
Fij f i (t ) f j (t ) + Fk+ 1 dt
(30)
W (s ) dW (s ) =
0
t W 2 (t ) 2 2
(22)
where
This is a consequence of the fact that Brownian motion has nonzero quadratic variation (Theorem 6). We see in the next section the differential rule that is to be used in stochastic calculus. Stochastic Differential Equations (7) In the previous section we introduced the concept of stochastic integration. Now we can use it to dene stochastic differential equations. In fact we say that X(t), t [0, T] is a solution for the stochastic differential equation dX (t ) = a[t , X (t )] dt + [t , X (t )] dW (t ), X (0 ) = X0 (23) and
Fi = Fij =
F [X (t ), . . ., Xk (t ), t ] Xi 1 2F [X (t ), . . ., Xk (t ), t ] Xi X j 1
Fk+ 1 = i, j 1, . . ., k.
F [X (t ), . . ., Xk (t ), t ] t 1
if for every t [0, T] it satises the following equation

t t
THE FILTERING PROBLEM (7) (24) Example 1 previously given is a special case of the following general ltering problem: Suppose that the state X(t) n of a system at time t is given by a stochastic differential Eq. (23). We assume that the observations H(t) m are performed continuously and are of the form (t ) H (t ) = c[t , X (t )] + [t , X (t )]W (31)
X (t ) = X0 +
a[u, X (u )] du +
0 0
[u, X (u )] dW (u )
We have the following result about the existence and uniqueness of the solution: Theorem 13. If a(t, x) and (t, x) are continuous in t and satisfy the Lipschitz condition in x uniformly in t, that is, for every t [0, T] |a(t , x ) a(t , y )| + | (t , x ) (t , y )|
6 K |x y|,
KR (25)
where c : n1 m, : n1 mr are functions satisfying (t) denotes r-dimensional white noise. If we Eq. (25) and W introduce
t
then the solution to the stochastic differential Eq. (23) exists and it is unique (up to indistinguishability).
Z(t ) =
0
H (s ) ds
(32)
548
STOCHASTIC SYSTEMS
then dZ = c[t , X (t )] dt + [t , X (t )] dW (t ) Z (0 ) = 0 (33)
where (t )]2 } S(t ) = E {[X (t ) X satises the (deterministic) Riccati equation
where W(t) is an r-dimensional Wiener process. Note that if H(s) is also known for 0 s t, then Z(s) is also known for 0 s t and vice versa. So no information is lost or gained by considering Z(t) as our observations instead of H(t), but this allows us to obtain a well-dened mathematical model of the situation. The ltering problem is the following: Given the observations Z(s) satisfying Eq. (33) for 0 s (t) of the state X(t) of the system t, what is the best estimate X Eq. (23) based on these observations? (t) is based on the observaBy saying that the estimate X tions Z(s) : s t, we mean that
G2 (t ) dS = 2F (t )S(t ) 2 S2 (t ) + C2 (t ), dt D (t ) S(0 ) = E {(X0 E [X0 ] )2 }
(40)
Example 5. Estimation of a parameter. Suppose we want to estimate the value of a (constant) parameter based on observations Z(t) satisfying the model dZ(t ) = X (t ) dt + (t ) dW (t ) where X(t), (t) are known functions. In this case the stochastic differential equation for is of course d = 0 so the Riccati equation for S(t) E[ (t)]2 is X (t )S(t ) dS = t (t ) which gives
2
(t ) is Ft -measurable, X
where Ft is the -algebra generated by {Z(s ), s 6 t }
(34)
(t) is the best such estimate, we mean that By saying that X
(t )|2 dP = E [|X (t ) X (t )|2 ] |X (t ) X = inf{E [|X (t ) = Y |2 ] : Y Y }

where Y := {Y : Rn ; Y L2 (P ) and Y is Ft -measurable}, (36)
(35)
1 S(t ) = S 0 +
t 0
X (s )2 (s ) 2 ds
where L2(P) L2(, P) and (, F , P) is a probability space. One of the most important results is the following: (t ) = E [X (t )|Ft ] X This is the basis for the general FujisakiKallianpurKunita equation of ltering theory. See, for example, Ref. 34. Here we concentrate on the linear case, which allows an explicit solution in terms of a stochastic differential equation (t) (the KalmanBucy lter): for X In the linear ltering problem the system and observation equations have the following form: (linear system) dX (t ) = F (t )X (t ) dt + C(t ) dW (t ); F (t ) Rnn , C(t ) Rn p (linear observations) dX (t ) = G(t )X (t ) dt + D(t ) dV (t ); G(t ) Rmn , D(t ) Rmr (38) (37)
and the KalmanBucy lter is given by d( t) = X (t )S(t ) [dZ(t ) = X (t )( t )] (t )2
This can be written as

1 S 0 + t 0
X (s )2 (t ) 2 d( t ) + X (t )2 (t ) 2 dt = X (t ) (t ) 2 dZ(t )
We recognize the left-hand side as d so

1 + 0 S 0 1+ S 0 t 0 t 0
1 S 0 + t 0
X (s )2 (t ) 2 ds ( t)
Theorem 15. The One-Dimensional KalmanBucy Filter. (t) E[X(t)F t] of the one-dimensional linear The solution X ltering problem
X (s ) (s ) 2 dZ(s ) X (s )2 (t ) 2 ds
(linear system)
dX (t ) = F (t )X (t ) dt + C(t ) dW (t );
F (t ), C(t ) R
(linear observations) dZ(t ) = G(t )X (t ) dt + D(t ) dV (t );
This estimate coincides with the maximum likelihood esti1 mate in the classical estimation theory if S 0 0. See (22). For more information about estimates of drift parameters in diffusion, see the next section. Theorem 16. The Multidimensional KalmanBucy Filter. The solution (t ) = E [X (t )|Ft ] X
G(t ), D(t ) R
satises the stochastic differential equation
(t ) = F (t ) dX = E [X ] X 0 0
G(t )S(t ) G2 (t )S(t ) X (t ) dt + dZ(t ) D2 (t ) D2 (t )
(39)
STOCHASTIC SYSTEMS
549
of the multidimensional linear ltering problem (linear system ) dX (t ) = F (t )X (t ) dt + C(t ) dW (t ); F (t ) Rnn , C(t ) Rn p (linear observations ) dZ(t ) = G(t )X (t ) dt + D(t ) dV (t ); G(t ) Rmn , D(t ) Rmr satises the stochastic differential equation
tion of the following stochastic differential equation: dX (t ) = a t , X (t ), U (t )] dt + t , X (t ), U (t ) dB(t ) X (0 ) = x (45)
to minimize the objective function (also called the cost functional)
J (x, U ) = Ex
0
(t ) = [F SGT (DDT ) 1 G]X (t ) dt + SGT (DDT ) 1 dZ(t ) : dX ( 0 ) = E [X ] X 0

where (t )][X (t ) X (t )]T Rnn S(t ) := E [X (t ) X satises the matrix Riccati equation
L s, X (s ), U (s ) ds + X (T )
(46)
where the admissible choices of the controls are all of the smooth functions of the observations of X(t). We dene the HamiltonJacobiBellman equation of dynamic programming as W + min AsU W + L(s, X , U ) = 0 s (47)
dS T T T = FS + SF DG (DDT ) 1 GS + CC dt S(0 ) = E {X (0 ) E [X (0 )]}{X (0 ) E [X (0 )]}T

The condition on D(t) mr is now that D(t)D(t)T is invertible for all t and that [D(t)D(t)T]1 is bounded on every bounded t-interval. A similar solution can be found for the more general situation
where W(T, x) Es,x[X(T)] is the value function and (A t, t 0) is the innitesimal generator of the semigroup associated with the stochastic differential Eq. (45). The following result is known as the verication theorem Theorem 17. Let W(s, y) be a solution of Eq. (47) such that W C 2(Q) and it is continuous on the closure of Q, (we say W is a smooth function). Then (1) W(s, y) J(s, y, U) for any admissible control U and any initial condition (s, y) Q; and (2) if U* is an admissible control such that AsU W + L(s, y, U ) = min AsU W + L(s, y, v )
v U
dX (t ) = [F0 (t ) + F1 (t )X (t ) + F2 (t )Z(t )] dt + C(t ) dW (t ) (observations) dZ(t ) = [G0 (t ) + G1 (t )X (t ) + G2 (t )Z(t )] dt + D(t ) dV (t ) (system)

where X(t) , Z(t) and B(t) [W(t), V(t)] is (n m)dimensional Brownian motion with appropriate dimensions on the matrix coefcients. See (34), which also treats the nonlinear case. For various applications of ltering theory see Ref. 33.
n m
(48)
for all (s, y) Q, then W (s, y ) = J (s, y, U ) Remark. U* is an optimal control. The Linear Stochastic Gaussian Problem (7,11) Suppose that the state X(t) of the system at time t is given by the following linear stochastic differential equation: (49)
STOCHASTIC CONTROL (7,10,11,36) Let (X(t), t 0) be the solution of the stochastic differential Eq. (23) with X(0) x. Let f be a bounded, continuous function, and let T be the operator dened by (Tt f )(x ) = Ex { f [t , X (t )]} This operator satises the semigroup property Tt (Ts f )(x ) = (Tt + s f )(x ) If we differentiate this operator, lim
h0
dX (t ) = H (t )X (t ) + M (t )U (t ) dt + (t ) dB(t ) t
> s; X (s) = x
(50)
and the cost has the form
(41)
J u (s, x ) = E s,x
t1 s
X (t )T C(t )X (t ) + U (t )T D(t )U (t ) dt + X (t1 )T RX (t1 ) s 6 t1 (51 )
(42)
Tt + h Tt = At h
(43)
where A t is an operator that acts on f in the following way: At f (x ) = a(t , x ) 1 f 2 f (x ) + 2 (t , x ) 2 (x ) x 2 x (44)
(A t, t 0) is called the innitesimal generator of the semigroup (Tt, t 0). Consider the problem of controlling the solu-
where all the coefcients H(t) nn, M(t) nk, (t) nm, C(t) nn, D(t) kk, and R nn, are t-continuous and deterministic. We assume that C(t) and R are symmetric, nonnegative-denite and D(t) is symmetric, positive-denite, for all t. We also assume that t1 is a deterministic time. [B(t), t 0] is a standard Brownian motion. Then the problem is to choose the control u u[t, X(t)] so that it minimizes Ju(s, x). We may interpret this as follows: The aim is to nd a control u which makes X(t) ( denotes the norm) small quickly and so that the energy used (uT Du) is small. The sizes of C(t) reects the cost of large values
550
STOCHASTIC SYSTEMS
of X(t), whereas the size of D(t) reects the cost (energy) of applying large values of u(t). In this case the HJB equation for W(s, x) becomes
0 = inf F v (s, x ) + (LvW )(s, x )

v
W + inf xT C(t )x + vT D(t )v + v s 1 + 2 2W ( T )ij xi x j i, j = 1

n
(H (t )x + M (t )v )i
i= 1
W xi (52)
for s < t1
and W (t1 , x ) = xT Rx We nd a solution W of Eqs. (52)(53) of the form W (t , x ) = xT S(t )x + a(t ) (54) (53)
tion of the unknown parameters and adaptive control. Maximum likelihood (or equivalently least squares) estimates are used to identify unknown constant parameters. These estimates are given recursively and are strongly consistent. The adaptive control is usually constructed by the so-called certainty equivalence principle, that is, the optimal stationary controls are computed by replacing the unknown true parameter values by the current estimates of these values. Because the optimal stationary controls are continuous functions of the unknown parameters, the self-tuning property is veried. The family of average costs using the control from the certainty equivalence principle converges to the optimal average cost. This veries the self-optimizing property. A model for the adaptive control of continuous-time linear stochastic systems with complete observations of the state is described by the following stochastic differential equation: dX (t ) = A()X (t ) + BU (t ) dt + dW (t ) where X(t) n, U(t) m.
p
(59)
where S(t) St nn is symmetric, nonnegative-denite, a(t) are both a(t), and S(t) are continuously differentiable w.r.t. t (and deterministic). The following u (t , x ) = D(t ) 1 M (t )T S(t )x, t < t1 (55)
A() = A0 +
i= 1
i Ai
(60)
is an optimal control, and the minimum cost is given by W (s, x ) = xT S(s )x +

s t1
tr( T S )t dt ,
s < t1
(56)
where S(t) satises the corresponding Riccati type equation from linear ltering theory. This formula shows that the extra cost due to the noise in the system is given by a (s ) =
s t1
tr( T S )t dt
Ai L (n) i 0, . . ., p, B L (m, n), (W(t), t ) is a standard n-valued Wiener process and X0 a n. It is assumed that (5.A1) A p is compact and A . (5.A2) (A(), B) is reachable for each A . (5.A3) The family (Ai, i 1, . . ., p) is linearly independent. Let (F t, t ) be a ltration such that Xt is measurable with respect to F t for all t and (W(t), F t, t ) is a Brownian martingale. The ergodic, quadratic control problem for Eq. (59) is to minimize the ergodic cost functional 1 lim sup J (X0 , U, , t ) t t where
t
(61)
The Separation Principle The separation principle (5,18) states that if we only have partial knowledge of the state X(t) of the system, that is, if we have only noisy observations (t ) dZ(t ) = g(t )X (t ) dt + (t ) dB (57)
J (X0 , U, , t ) =
QX (s ), X (s ) + PU (s ), U (s ) ds
0
(62)
at our disposal, then the optimal control u*(t, ) (required to be F t-adapted, where F t is the -algebra generated by Z(r) : r t), is given by (t )() u (t , ) = D(t ) 1 M (t )T S(t )X (58)
and t (0, ), X(0) X0, Q L (n); P L (m) are selfadjoint and P1 exists; [X(t), t ] satises Eq. (59); and [U(t), t ] is adapted to (F t, t ). It is well known (33) that, if is known, then there is an optimal linear feedback control such that U (t ) = KX (t ) (63)
(t) is the ltering estimate of X(t) based on the obserwhere X vations Z(r) : r t, given by the KalmanBucy lter. Comparing with Eq. (55), we see that the stochastic control problem in this case splits into a linear ltering problem and a deterministic control problem. LEAST SQUARES AND CONTINUOUS-TIME STOCHASTIC ADAPTIVE CONTROL (2) The general approach to adaptive control described here exhibits a splitting or separation of the problems of identica-
where K P1B*V and V is the unique, symmetric, nonnegative-denite solution of the algebraic Riccati equation VA + AV VB P 1 BV + Q = 0 (64)
For an unknown the admissible adaptive control policies [U(t), t ] are linear feedback controls t , X (u ), u U (t ) = K (t )X (t ) = K
6t
X (t )
(65)
STOCHASTIC SYSTEMS
551
where [K(t), t 0] is an L (n, m)-valued process that is uniformly bounded and there is a xed 0 such that (K(t), t 0) is measurable with respect to (Xu, u t ) for each t and [K(t), t [0, )] is a deterministic function. For such an adaptive control, it is elementary to verify that there is a unique strong solution of Eq. (59). The delay 0 accounts for some time required to compute the adaptive control law from the observation of the solution of Eq. (59). Let (U(t), t 0) be an admissible adaptive control and let [X(t), t 0] be the associated solution of Eq. (59). Let A (t) (t) [a [aij(t)] and A ij(t)] be L (p)-valued processes such that
t
where (X(t), t 0) is the solution of Eq. (59) with the admissible adaptive control [U(t), t 0] and 0 K and V is the solution of the algebraic Riccati Eq. (64) with 0. Then lim inf
T
1 J (X0 , U, 0 , T ) T
> tr V
a.s.
(71)
If U is an admissible adaptive control U(t) K(t)X(t) such that

t
lim K (t ) = k0
a.s.
(72)
aij (t ) = a ij (t ) =
Ai X (s ), A j X (s ) ds
where k0 P1B*V, then

T
aij (t ) aii (t )
lim
1 J (X0 , U, 0 , T ) = tr V T
a.s.
(73)
To verify the strong consistency of a family of least squares estimates, it is assumed that (t )| > 0 a.s. lim inf | det A
t
Corollary. Under the assumptions of the Proposition, if Eq. (72) is satised, then Eq. (69), and (70) are satised. The previous results can be combined for a complete solution to the stochastic adaptive control problem Eqs. (59, 61). Theorem 20. Assume that (5.A15.A4) are satised. Let [ (t), t 0] be the family of least squares estimates where (t) is the minimizer of Eq. (66). Let [K(t), t 0] be an admissible adaptive control law such that K (t ) = P 1 BV ( t )
The estimate of the unknown parameter vector at time t, (t), for t 0 is the minimizer for the quadratic functional of , L(t, ), given by
t
L(t , ) =
0
[A() + BK (s )]X (s ), dX (s ) 1 2
t 0
(66) [A() + BK (s )]X (s ) ds

2
where U(s) K(s)X(s) is an admissible adaptive control. The following result (31) gives the strong consistency of these least squares estimators. Theorem 18. Let (K(t), t 0) be an admissible adaptive feedback control law. If (5.A15.A4) are satised and 0 A , the interior of A , then the family of least squares estimates [ (t), t 0] where (t) is the minimizer of Eq. (66), is strongly consistent, that is, P 0 lim ( t ) = 0 = 1
t
where V() is the solution of Eq. (64) for A . Then the family of estimates ( (t), t 0) is strongly consistent.
t
lim K (t ) = k0
a.s.
(74)
where k0 P1B*V(0), and lim 1 J (X0 , U, 0 , T ) = tr V T a.s. (75)
(67)
WEIGHTED LEAST SQUARES AND CONTINUOUS-TIME ADAPTIVE LQG CONTROL (30) Introduction The linear, Gaussian control problem with an ergodic, quadratic cost functional is probably the most well-known ergodic control problem. Because the optimal control is easily computed and the existence of an invariant measure follows directly from the stability of the optimal system, it is a basic problem to solve for stochastic adaptive control. For discretetime linear systems it has been studied extensively, especially for ARMAX models (see Refs. 6, 28, 3740 for many references). Although this adaptive control problem has been less studied for continuous-time linear systems, it is nonetheless an important problem as a model for physical systems that naturally evolve in continuous time and as an approximation for discrete-time sampled systems when the sampling rate is high. The adaptive control problem is solved using only the natural assumptions of controllability and observability. The weighted least squares scheme is used to obtain the conver-
where 0 is the true parameter vector. The family of estimates [ (t), t 0] can be computed recursively because this process satises the following equation d( t ) = A 1 (t ) A (t )X (t ), dX (t ) A[( t )]X (t ) dt BU (t ) dt (68) where (t)x, y (Aix, y) i 1, . . ., p. Now the performance of some admissible adaptive controls is described. Proposition 19. Assume that (5.A15.A4) are satised and that
t
lim
1 V X (t ), X (t ) = 0 a.s. t 1 t
t
(69) (70)
lim sup
t
|X (t )|2 ds < a.s.

0
552
STOCHASTIC SYSTEMS
gence of the family of estimates (self convergence), and the scheme is modied by a random regularization to obtain the uniform controllability and observability of the family of estimates. A diminishing excitative white noise is used to obtain strong consistency. The excitation is sufcient to include the identication of unknown deterministic linear systems. This approach eliminates some other assumptions previously used that are unnecessary for the control problem for a known system and are often difcult to verify. Furthermore, this approach eliminates the need for random switchings or resettings which often occur in previous work. Weighted Least Squares Identication Let [X(t), t 0] be the process satisfying the stochastic differential equation dX (t ) = AX (t ) dt + BU (t ) dt + D dW (t ) (76)
A function f is slowly increasing if it is positive, increasing, and f (x2) 0[f (x)] as x . formula to [XT(t)X(t), t 0], it is elemenApplying the Ito tary to verify (31) that for D 0
t
lim r(t ) = a.s.
If D 0, then the diminishing excitation used subsequently implies this result. The dependence of [(t), t 0] on [a(t), t 0] and the dependence of [a(t), t 0] on f are suppressed for notational convenience. The following result describes some basic properties of the WLS algorithm. Proposition 21. Let [(t), P(t) t 0] satisfy Eqs. (80)(81). The following properties are satised: (1) (2)
0
where X(0) X0, X(t) n, U(t) m, [W(t), t 0] is an pvalued standard Wiener process, and [U(t), t 0] is a control from a family specied subsequently. The random variables are dened on a xed, complete probability space (, F , P), and there is a ltration (F t, t 0) dened on this space that is specied subsequently. It is assumed that A and B are unknown. The following assumption is made: (A1) (A, B) is controllable. To describe the identication problem in a standard form, let = [A B] and (t ) = so that Eq. (76) is rewritten as dX (t ) = (t ) dt + D dW (t ) X (t ) U (t )
(t )|2 < a.s. sup |P 1/2 (t )

t 0
>
(85) (86) (87)
(t )(t )|2 dt < a.s. a(t )|

t
(3)
lim (t ) =
a.s.
(t) (t) and is a matrix-valued random where variable. (t, x) as Dene

(t , x ) = (t ) P1/2 (t )x and
(77)
(78)
(t , x ) = [A(t , x )
B(t , x )]
(79)
A well-known test for controllability of (A, B) is the positivity of F where
A family of weighted least squares (WLS) estimates [(t), t 0] is given by
F (t , x ) = det
n1 i= 0
Ai (t , x )B(t , x )B (t , x )Ai (t , x )
T T dP(t ) = a(t )P(t )(t ) T (t )P(t ) dt

1 a(t ) = f [r(t )]
t
d (t ) = a(t )P(t )(t )[dX (t ) (t ) (t ) dt ]
(80) (81) and similarly a test for observability of (A, C) is the positivity of G where
where (0) is arbitrary, P(0) 0 is arbitrary
G(t , x ) = det
(82) |(s )|2 ds (83)
n1 i= 0
AiT (t , x )CT CAi (t , x )
r(t ) = e +
0
and f .
F = f | f : R+ R+ , f is slowly increasing
and
c
dx < for some c > 0 x f (x )
(84)
The linear transformation C is known. For the adaptive control problem, C Q1/2 where Q1 determines the quadratic 1 form of the state in the cost functional. A random search method ensures uniform controllability and observability of a family of estimates. Let (n, n )be a sequence of independent, identically distributed M (n m,n)-valued random variables that is independent of [W(t), t 0] so that each random variable n is uniformly distributed in the unit ball or the unit sphere for a norm of the matrices. Dene a sequence of M (n m,n)-valued random
STOCHASTIC SYSTEMS
553
variables by induction as follows:
If (t) is given by
1 T (t ) = A(t ) B(t )Q 2 B (t )R(t )
0 = 0 k = k k 1 if f (k, k ) otherwise
> ( 1 + ) f ( k, k 1 )
and
1 T ( ) = A ( ) B ( )Q 2 B ( )R ( )
where (0, 2 1) is xed and f (k, x) F(k, x)G(k, x). By the compactness of the unit ball or the unit sphere and the continuity of F(k, ) and G(k, ) for k , it follows that the sequence (k, k )terminates after some random integer so there is a such that = lim k
k
then
t
lim
(t ) =
( )
and (t) and () are stable a.s. The lagged certainty equivalence control is
1 T U (t ) = Q 2 B (t )R(t )X (t )
a.s.
(91)
(t), t 0] as Dene a family of estimates [ (t ) = k where k and k = (k ) P1/2 (k )k (89) t (k, k + 1] (88)
It is called lagged because B(t) and R(t) depend on X(s) and j for s t and j t. This follows easily by induction recalling (t), t 0), that is, the construction of ( [B(t ), R(t )] = [B(0 ), R(0 )] for t [0, 1], [B(t ), R(t )] = (Bk , Rk ) for t (k, k 1], and k [Ak, Bk] is given by Eq. (89). To obtain strong consistency for the family of estimates (t), t 0), a diminishing excitation is added to the adaptive ( control Eq. (91), that is, dU (t ) = Lk dX (t ) + k dV (t ) for t (k, k 1] and k where U(0) is arbitrary,
1 Lk = Q 2 B ( k )R ( k )
(t), t 0] converges It is clear from Eqs. (81) and (87) that [ a.s. For notational simplicity the dependence on (k, k ) in Eq. (88) is suppressed so that (t ) = [A(t ) B(t )] (t), t 0) is the family of estimates given Theorem 22. If ( by Eq. (88), (A, B) in Eq. (76) is controllable, and (A, Q1/2 1 ) is observable, then for any admissible control [U(t), t 0], (t), t 0] has the following properties a.s.: [ (1) self-convergence, (2) uniform controllability and observability, and (3) semiconsistency. Adaptive Control Consider the following ergodic cost functional for the system Eq. (76): J (U ) = lim sup
T
(92)
and log k 2 k = k for k 1. The process [V(t), t 0] is an m-valued standard Wiener process that is independent of [W(t), t 0] and (k, k ). As noted before, it easily follows that there is a matrixvalued random variable L such that
k
1 T
T 0
[X T (t )Q1 X (t ) + U T (t )Q2U (t )] dt
(90)
lim Lk = L
a.s.
where [U(t), t 0] is an admissible control, Q2 0, and Q1 0. The following assumption is made: (A2) (A, Q1/2 1 ) is observable. The stochastic algebraic Riccati equation
1 T AT (t )R(t ) + R(t )A(t ) R(t )B(t )Q 2 B (t )R(t ) + Q1 = 0
The sub--algebra F t is the P-completion of [X0, W(s), j, V(s); s t, j t]. The state X(t) is augmented with the control U(t) as follows:
(t ) = Fk = Gk = (t ) =
X (t ) U (t ) A Lk A D Lk D W (t ) V (t ) B Lk B 0 k I
has a unique, random, symmetric, positive solution R(t) a.s. (t). Because the solution of the algebraic where [A(t) B(t)] Riccati equation is a smooth function of the parameters of the equation, there is a symmetric, positive L (n)-valued random variable R() such that
t
lim R(t ) = R( )
a.s.
554
STOCHASTIC SYSTEMS 3. B. Bercu, Weighted estimation and tracking for ARMAX models, SIAM J. Control Optim., 33: 89106, 1995. 4. V. Borkar and A. Bagchi, Parameter estimation in continuoustime stochastic processes, Stochastics, 8: 193212, 1982. 5. M. H. A. Davis and R. B. Vinter, Stochastic Modelling and Control, London: Chapman and Hall, 1985. 6. P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identication and Adaptive Control, Englewood Cliffs, NJ: Prentice-Hall, 1986. 7. B. ksendal, Stochastic Differential Equations: An Introduction with Applications, 4th ed., New York: Springer, 1995. 8. I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, New York: Springer-Verlag, 1988. 9. S. Karlin and H. Taylor, A First Course in Stochastic Processes, 2nd ed., New York: Academic Press, 1975. 10. A. Bensoussan, Stochastic Control by Functional Analysis Method, Amsterdam, The Netherlands: North Holland, 1982. 11. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer Verlag, 1975. 12. J. Zabczyk, Lectures in Stochastic Control, Control Theory Centre Report No. 125, Univ. Warwick Control Theory Centre. stro 13. K. J. A m, Introduction to Stochastic Control, New York: Academic Press, 1970. 14. R. C. Merton, Continuous-Time Finance, Cambridge, MA: Blackwell, 1990. 15. T. E. Duncan and H. Upmeier, Stochastic control problems and spherical functions on symmetric spaces, Trans. Amer. Math. Soc., 347: 10831130, 1995. 16. W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, New York: Springer Verlag, 1993. 17. H. P. McKean, Stochastic Integrals, New York: Academic Press, 1969. 18. R. J. Elliott, Stochastic Calculus and Applications, New York: Springer Verlag, 1982. 19. I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes, New York: Springer Verlag, 1979. and H. P. McKean, Diffusion Processes and Their Sample 20. K. Ito Paths, New York: Springer Verlag, 1965. 21. R. S. Liptser and A. N. Shiryayev, Statistics of Random Processes, New York: Springer Verlag, 1977, vol. I. 22. R. S. Liptser and A. N. Shiryayev, Statistics of Random Processes, New York: Springer Verlag, 1978, vol. II. stro m and B. Wittenmark, Adaptive Control, 2nd ed., 23. K. J. A Reading, MA: AddisonWesley, 1995. 24. T. E. Duncan and B. PasikDuncan (eds.), Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, Springer Verlag, 1992, vol. 184. r and P. E. Caines (eds.), Topics in Stochastic Systems: 25. L. Gerencse Modelling, Estimations, and Adaptive Control, Lecture Notes in Control and Information Sciences, New York: Springer Verlag, 1991, vol. 161. 26. P. R. Kumar, A survey of some results in stochastic adaptive control, SIAM J. Control Optim., 23: 329380, 1985. 27. T. E. Duncan, B. PasikDuncan, and L. Stettner, Almost selfoptimizing strategies for the adaptive control diffusion processes, J. Optim. Theor. Appl., 81: 479507, 1994. 28. H. F. Chen and L. Guo, Identication and Stochastic Adaptive Control, Boston: Birkha user, 1991. 29. W. Levine (ed.), The Control Handbook, Boca Raton, FL: CRC Press, Piscataway, NJ: IEEE Press, 1996. 30. T. E. Duncan, L. Guo, and B. PasikDuncan, Adaptive continuous-time linear quadratic Gaussian control, IEEE Trans. Autom. Control, submitted.
The stochastic differential equation for the augmented state process ((t), t 0) is d(t ) = Fk (t ) dt + Gk d (t ) for t (k, k 1]. The following result shows that the family of regularized, weighted, least squares estimates is strongly consistent using the lagged certainty equivalence control with diminishing excitation. (t), t 0] be the family of estimates Proposition 23. Let [ given by Eq. (88) using the control Eq. (92) in (Eq. 76). Then
t
(t ) = lim
a.s.
where = [A B] Remark. It should be noted that the conditions of this proposition are satised if D 0 so that the identication of deterministic systems is included in this result. (t), t 0] is strongly conBecause the family of estimates [ sistent, the self-optimality of the diminishingly excited lagged certainty equivalence control Eq. (92) can be veried. Theorem 24. Let (A1) and (A2) be satised for the stochastic system dX (t ) = AX (t ) dt + BU (t ) dt + D dW (t ) with the cost functional Eq. (90) where A and B are unknown. If the admissible adaptive control
dU (t ) = Lk dX (t ) + k dV (t )
1 Lk = Q 2 B ( k )R ( k )
is used, then J (U ) = inf J (U ) = tr (D RD )

U
a.s.
where R is the unique, positive, symmetric solution of the algebraic Riccati equation
1 A R + RA RBQ 2 B R + Q1 = 0
and J (U ) = lim sup

T
1 T
T 0
[X (t )Q1 X (t ) + U (t )Q2U (t )] dt
a.s.
BIBLIOGRAPHY
1. V. Borkar, Optimal Control of Diffusion Processes, New York: Wiley, 1989. 2. T. E. Duncan and B. PasikDuncan, Adaptive control of continuous-time linear stochastic systems, Math. Control, Signals, Syst., 3: 4560, 1990.
STOCK MARKETS 31. P. Bertrand, Adaptive control of partially observed linear stochastic systems, Stochastics, 54: 2151, 1995. 32. T. E. Duncan, B. PasikDuncan, and L. Stettner, Adaptive control of a partially observed discrete time Markov processes, Appl. Math. Optim., 37: 269293, 1998. 33. R. E. Kalman and R. S. Bucy, New results in linear ltering and prediction theory, Trans. ASME, Ser. D, J. Basic Eng., 83: 95 107, 1961. 34. G. Kallianpur, Stochastic Filtering Theory, New York: Springer Verlag, 1980. 35. G. Yin and Q. Zhang (eds.), Mathematics in Stochastic Manufacturing Systems, Lecture Notes in Applied Mathematics, Providence: Amer. Math. Soc., 1997, vol. 33. 36. H. J. Kushner, Stochastic Stability and Control, New York: Academic Press, 1967. 37. P. E. Caines, Linear Stochastic Systems, New York: Wiley, 1988. 38. G. Goodwin, P. Ramadge, and P. Caines, Discrete time stochastic adoptive control, SIAM J. Control Optim., 19: 829853, 1981. stro 39. G. C. Goodwin, K. A m, and P. R. Kumar (eds.), Adaptive Control, Filtering and Signal Processing, IMA Volumes in Mathematics and Its Applications 74, New York: Springer, 1995. 40. G. Chen, G. Chen, and S. H. Hua, Stochastic Control Systems, Boca Raton, FL: CRC Press, 1995.
555
BOZENNA PASIKDUNCAN
University of Kansas
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

System Identification Standard Article Lennart Ljung1 1Linkping University, Linkping, Sweden Copyright 1999 by John Wiley & Sons, Inc. All rights reserved. DOI: 10.1002/047134608X.W1046 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (287K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are The Problem Background and Literature Outline Displaying the Basic Ideas: Arx Models and the Linear Least Squares Method Model Structures I: Linear Models Model Structures Ii: Nonlinear Black-Box Models General Parameter Estimation Techniques Special Estimation Techniques for Linear Black-Box Models Data Quality Model Validation and Model Selection Back to Data: The Practical Side of Identification
SYSTEM IDENTIFICATION
263
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
20
40
60
80
100
120
140
160
180
The process of going from observed data to a mathemathical model is fundamental in science and engineering. In the control area, this process has been termed system identication, and the objective is then to nd dynamical models (difference or differential equations) from observed input and output signals. Its basic features are, however, common with general model building processes in statistics and other sciences. System identication covers the problem of building models of systems when insignicant prior information is available and when the systems properties are known, up to a few parameters (physical constants). Accordingly, one talks about black-box and gray-box models. Among black-box models, there are familiar linear models such as ARX and ARMAX, and among nonlinear black-box models we have, for example, articial neural networks (ANN). THE PROBLEM The area of system identication begins and ends with real data. Data are required to build and to validate models. The result of the modeling process can be no better than what corresponds to the information contents in the data. Look at two data sets: Example 1 An Unstable Aircraft Figure 1 shows some results from test ights of the new Swedish aircraft, JAS-Gripen, developed by SAAB Military Aircraft AB, Sweden. The problem is to use the information in these data to determine the dynamical properties of the aircraft for ne-tuning regulators, for simulations, and so on. Of particular interest are the aerodynamical derivatives. Example 2 Vessel Dynamics Figure 2 shows data from a pulp factory. They are collected from one of the buffer vessels. The problem is to determine the residence time in the vessel. The pulp spends about 48 h total in the process, and knowing the residence time in the different vessels is important in order to associate various portions of the pulp with the different chemical actions that have taken place in the vessel at different times. (The -num-
Figure 1. Results from test ights of the new Swedish aircraft JASGripen, developed by SAAB Military Aircraft AB, Sweden. From above (a) Pitch rate, (b) Elevator angle, (c) Canard angle, (d) Leading edge ap.
ber is a quality property that in this context can be seen as a marker allowing us to trace the pulp.) So, the bottom line of these examples is that we have collected inputoutput data from a process or a plant, and we need to extract information from these to nd out (something about) the process dynamical properties. BACKGROUND AND LITERATURE System identication has its roots in standard statistical techniques, and many of the basic routines have direct interpretations as well-known statistical methods such as least squares and maximum likelihood. The control community took an active part in the development and application of these basic techniques to dynamic systems right after the birth of modern control theory in the early 1960s. Maximum likelihood estimation was applied to different equations (ARMAX models) by Ref. 1, and thereafter, a wide range of estimation techniques and model parameterizations ourished. By now, the area is well matured with established and well understood techniques. Industrial use and application of the techniques has become standard. See Ref. 2 for a common software package. The literature on system identication is extensive. For a practical user oriented introduction, we may mention (3). Texts that go deeper into the theory and algorithms include Refs. 4 and 5. A classical treatment is Ref. 6. These books all deal with the mainstream approach to system identication, as described in this article. In addition, there is substantial literature on other approaches, such as set membership (compute all those models that reproduce the observed data within a certain given error bound), estimation of models from given frequency response measurement (7), on-line model estimation (8), non-parametric frequency domain methods (9), etc. To follow the development in the
264
25 20 15 10 5 0 100 200
Output #1
The problem can be expressed as nding that model in the candidate set, that best describes the data, according to the criterion, and then evaluate and validate that models properties. To do this we need to penetrate a number of things: 1. First, we give a preview of the whole process, as applied to the simplest set of candidate models. 2. Then, at some length, we display and discuss the most common sets of candidate models used in system identication. In general terms, a model will be a predictor of the next output y(t) from the process, given past observations Zt1, and parameterized in terms of a nite-dimensional parameter vector : y (t | ) = g(, Zt 1 ) (1)
300 Input #1
400
500
600
20 15 10 0 100 200 300 400 500 600
3. We then discuss the criterion of t for general model sets. This will have the character VN ( ) = y(t ) y (t | )
2
(2)
80 60 40 20 0 0 200 150 100 50 0 0 100 200 300 400 500 600

Figure 2. From the pulp factory at Skutska r, Sweden. The pulp ows continuously through the plant via several buffer tanks. From above: (a) the -number of the pulp owing into a buffer vessel, (b) the -number of the pulp coming out from the buffer vessel, (c) ow out from the buffer vessel, and (d) level in the buffer vessel.
100
200
300
400
500
600
We also discuss how to nd the best model (minimize the criterion) and how to assess its properties. 4. We shall describe special methods for linear black-box models. This includes frequency analysis, spectral analysis and so called subspace methods for linear statespace models. 5. We then turn to the practical issues of system identication, to assure good quality of the data by proper experiment design, how to decide upon a good model structure, and how to deal with the data.
DISPLAYING THE BASIC IDEAS: ARX MODELS AND THE LINEAR LEAST SQUARES METHOD The Model We shall generally denote the systems input and output at time t by u(t) and y(t), respectively. Perhaps the most basic relationship between the input and output is the linear difference equation y(t ) + a1 y(t 1 ) + . . . + an y(t n ) = b1 u(t 1 ) + . . . + bm u(t m ) (3)
eld, the IFAC series of Symposia on System Identication [Budapest, Hungary (1991), Copenhagen, Denmark (1994), Fukuoka, Japan (1997)] is also a good source.
OUTLINE The system identication procedure is characterized by four basic ingredients: 1. The observed data 2. A set of candidate models 3. A criterion of t 4. Validation
We have chosen to represent the system in discrete time, primarily since observed data are always collected by sampling. It is thus more straightforward to relate observed data to discrete time models. Nothing prevents us, however, from working with continuous time models: we shall return to that later. In Eq. (3), we assume the sampling interval to be one time unit. This is not essential but makes notation easiser. A pragmatic and useful way to see Eq. (3) is to view it as a way of determining the next output value given previous observations y(t ) = a1 y(t 1 ) . . . an y(t n ) + b1 u(t 1 ) + . . . + bm u(t m ) (4)
265
For more compact notation, we introduce the vectors = [a1 , . . ., an , b1 , . . ., bm ]T (t ) = [y(t 1 ) . . . y(t n )u(t 1 ) . . . u(t m )] With these, Eq. (4) can be rewritten as y(t ) = T(t ) To emphasize that the calculation of y(t) from past data [Eq. (4)] indeed depends on the parameters in , we shall rather call this calculated value y (t) and write y (t | ) = T(t ) The Least Squares Method Now suppose for a given system that we do not know the values of the parameters in , but that we have recorded inputs and outputs over a time interval 1 t N: ZN = {u(1 ), y(1 ), . . ., u(N ), y(N )} (8) (7)
T
(5) (6)
Once the vectors (t) are dened, the solution can easily be found by modern numerical software, such as MATLAB, see Ref. 2. Example 3 First-Order Difference Equation Consider the simple model y(t ) + ay(t 1 ) = bu(t 1 ) This gives us the estimate according to Eqs. (5), (6) and (13)
a N y2 (t 1 ) y(t 1 )u(t 1 ) = y(t 1 )u(t 1 ) u2 (t 1 ) bN
y(t )y(t 1 ) y(t )u(t 1 )
All sums are from t 1 to t N. A typical convention is to take values outside the measured range to be zero. In this case, we would thus take y(0) 0. The simple model in Eq. (3) and the well-known least squares method in Eq. (13) form the archetype of system identication. Not only that, they also give the most commonly used parametric identication method and are much more versatile then perhaps perceived at rst sight. In particular, one should realize that Eq. (3) can directly be extended to several different inputs [this just calls for a redenition of (t) in Eq. (6)] and that the inputs and outputs do not have to be the raw measurements. On the contrary, it is often most important to think over the physics of the application and come up with suitable inputs and outputs for Eq. (3), formed from the actual measurements. Example 4 An Immersion Heater Consider a process consisting of an immersion heater immersed in a cooling liquid. We measure v(t): the voltage applied to the heater r(t): the temperature of the liquid y(t): the temperature of the heater coil surface Suppose we need a model for how y(t) depends on r(t) and v(t). Some simple considerations based on common sense and high school physics (semiphysical modeling) reveal the following: The change in temperature of the heater coil over one sample is proportional to the electrical power in it (the inow power) minus the heat loss to the liquid. The electrical power is proportional to v2(t). The heat loss is proportional to y(t) r(t).
An obvious approach is then to select in Eqs. (3) through (7) (t) as well as possible to so as to t the calculated values y the measured outputs by the least squares method: min VN (, ZN )
(9)
where
V (, ZN ) =
1 N
(y(t ) y (t | ))2
t =1 N
(10) (y(t ) (t ) )
T 2
1 = N
t =1
N: We shall denote the value of that minimizes Eq. (9) by

N = arg min VN (, ZN )
(11)
(arg min means the minimizing argument, i.e., that value of which minimizes VN.) Since VN is quadratic in , we can nd the minimum value easily by setting the derivative to zero:
0=
which gives
d 2 V (, ZN ) = d N N
(t )(y(t ) T (t ) )
t =1
(t )y(t ) =
t =1 t =1
(t ) T (t )
(12)
This suggests the model y(t ) = y(t 1 ) + v2 (t 1 ) (y(t 1 ) r(t 1 ))
or
N 1 N
which ts into the form
N =
t =1
(t ) (t )
T t =1
(t )y(t )
(13)
y(t ) + 1 y(t 1 ) = 2 v2 (t 1 ) + 3 r(t 1 ))
266
This is a two input (v2 and r) and one output model and corresponds to choosing (t ) = [y(t 1 ) v2 (t 1 ) r(t 1 )]T in Eq. (7). Some Statistical Remarks Model structures, such as Eq. (7) that are linear in , are known in statistics as linear regression, and the vector (t) is called the regression vector (its components are the regressors). Regress here alludes to the fact that we try to calculate (or describe) y(t) by going back to (t). Models such as Eq. (3), where the regression vector (t) contains old values of the variable to be explained y(t) are then partly auto-regressions. For that reason, the model structure in Eq. (3) has the standard name ARX-model (auto-regression with extra inputs). There is rich statistical literature on the properties of the N under varying assumptions (10). So far, we have estimate just viewed Eqs. (9) and (10) as curve-tting. Later, we shall deal with a more comprehensive statistical discussion, which includes the ARX model as a special case. Some direct calculations will be done in the following subsection. Model Quality and Experiment Design Let us consider the simplest special case, that of a nite impulse response (FIR) model. That is obtained from Eq. (3) by taking n 0: y(t ) = b1 u(t 1 ) + . . . bm u(t m ) (14)
N 0, since e has zero mean. The estimate is consethat E quently unbiased. Here, E denotes mathematical expectation. N T We can also form the expectation of N, that is the covariance matrix of the parameter error. Denote the matrix within brackets by RN. Take expectation with respect to the white noise e. Then, RN is a deterministic matrix, and we have
2 y (t | ) 2 since the double sum collapses to RN. We have, thus, computed the covariance matrix of the esti N. It is determined entirely by the input properties and mate the noise level. Moreover, dene R = lim 1 R N N (19)
This will be the covariance matrix of the input, that is, the i j-element of R is Ruu (i j ) = Eu(t i )u(t j ) If the matrix R is non-singular, we nd that the covariance matrix of the parameter estimate is approximately (and the approximation improves as N ) PN = 1 R N (20)
A number of things follow from this. All of them are typical of the general properties to be described later The covariance decays like 1/ N, so the parameters approach the limiting value at the rate 1/ N. The covariance is proportional to the noise-to-signal ratio. That is, it is proportional to the noise variance and inversely proportional to the input power. The covariance does not depend on the inputs or noises signal shapes, only on their variance/covariance properties. Experiment design, that is, the selection of the input u, aims at making the matrix R1 as small as possible. Note that the same R can be obtained for many different signals u. MODEL STRUCTURES I: LINEAR MODELS Output Error Models Starting from Eq. (3), there is actually another, quite different, way to approach the calculation of good values of ai and bi from observed data in Eq. (8). Equation (3) describes a linear, discrete-time system with transfer function G (z ) = b1 zn 1 + b2 zn 2 + . . . + bm zn m z n + a1 z n 1 + . . . + an (21)
Suppose that the observed data really have been generated by a similar mechanism
0 y(t ) = b0 1 u (t 1 ) + . . . bm u (t m ) + e (t )
(15)
where e(t) is a white noise sequence with variance , but otherwise unknown. (That is, e(t) can be described as a sequence of independent random variables with zero mean values and variances .) Analogous to Eq. (7), we can write this as y(t ) = T (t )0 + e(t ) (16)
We can now replace y(t) in Eq. (13) by the above expression and obtain
N 1 N
N =
t =1 N
(t ) T (t )
t =1 1
(t )y(t )
N N
=
t =1
(t ) T (t )
t =1
(t ) T (t )0 +
t =1
(t )e(t )
or
N 0 = N =
(t ) T (t )
t =1 t =1
(t )e(t )
(17)
Suppose that the input u is independent of the noise e. Then, and e are independent in this expression, so it is easy to see
(assuming n m) Here, z is the z-transform variable, and one may simply think of the transfer function G as a shorthand notation for the difference in Eq. (3).
267
We shall here use the shift operator q as an alternative for the variable z in Eq. (21). The shift operator q has the properties qu(t ) = u(t + 1 ) (22)
Noise Models and Prediction Filters A linear, nite-dimensional dynamical system can be described by the equation y(t ) = B (q ) u(t ) A (q ) (26)
(just as multiplying a z-transform by z corresponds to a time shift). Given only an input sequence {u(t ), t = 1, . . ., N } we could then calculate the output for system in Eq. (21) by running u as input to this system y (t | ) = G(q )u(t ) Example 5 A First-Order System Consider the system y(t + 1 ) + ay(t ) = bu(t ) The output according to Eq. (23) is then obtained as y (t | ) =
b u(t ) = b (a )k 1 u(t k ) q+a k=1
See Eqs. (21)(22). Based on Eq. (26) we can predict the next output from previous measurements either as in Eq. (23) y (t | ) = or as in Eqs. (4) and (7): y (t | ) = (1 A(q ))y(t ) + B(q )u(t ) (28) B (q ) u(t ) A (q ) (27)
(23)
Which one shall we choose? We can make the discussion more general by writing for Eq. (26) y(t ) = G(q, )u(t ) (29)
to indicate that the transfer function depends on the (numerator and denominator) parameters [as in Eq. (5)]. We can multiply both sides of Eq. (29) by an arbitrary stable lter W(q, ) giving W (q, )y(t ) = W (q, )G(q, )u(t ) (30)
or y (t + 1| ) + ay (t | ) = bu(t ) (24)
Then, we can add y(t) to both sides of the equation and rearrange to obtain y(t ) = (1 W (q, ))y(t ) + W (q, )G(q, )u(t ) We assume that the lter W starts with a 1: (31)
Notice the essential difference between Eqs. (23) and (7). (t) using both past measured inIn Eq. (7), we calculated y puts and also past measured outputs y(t k). In Eq. (23), (t) is calculated from past inputs only. As soon as we use y data from a real system [that does not exactly obey Eq. (13)], there will always be a difference between these two ways of obtaining the computed output. Now, we could of course still say that a reasonable estimate of is obtained by minimizing the quadratic t,
W ( q, ) = 1 + w 1 q 1 + w 2 q 1 + w 2 q 2 + . . . so that 1 W(q, ) actually contains a delay. We thus obtain the predictor y (t | ) = (1 W (q, ))y(t ) + W (q, )G(q, )u(t ) (32)
N = arg min
1 N
[ y(t ) y (t | )]2
t =1
(25)
(t) is computed according to Eq. (23). Such an even when y estimate is often called an output-error estimate since we have formed the t between a purely simulated output and (t) according to Eq. (23) is the measured output. Note that y not linear in , so the function to be minimized in Eq. (25) is not quadratic in . Hence, some numerical search schemes N in Eq. (25). Most often have to be applied in order to nd in practice, a GaussNewton iterative minimization procedure is used. It follows from the discussion that the estimate obtained by Eq. (25) will in general differ from the one from Eq. (9). What is the essential difference? To answer that question, we will have to discuss various ways of perceiving and describing the disturbances that act on the system.
Note that this formulation is now similar to that of Eq. (28). We see that the method used in Eq. (27) corresponds to the choice W(q, ) 1, while the procedure in Eq. (28) is obtained for W(q, ) A(q). Now, does the predictor in Eq. (32) depend on the lter W(q, )? Well, if the inputoutput data are exactly described by Eq. (29) and we know all relevant initial conditions, the (t), repredictor in Eq. (32) produces identical predictions y gardless of the choice of stable lters W(q, ). To bring out the relevant differences, we must accept the fact that there will always be disturbances and noise that affect the system, so instead of Eq. (29), we have a true system that relates the inputs and outputs by y(t ) = G0 (q )u(t ) + v(t ) for some disturbance sequence v(t). So Eq. (32) becomes y (t | ) = { (1 W (q, ))G0 (q ) + W (q, )G(q, )}u(t ) + (1 W (q, ))v(t ) (33)
268
Now, assume that there exists a value 0, such that G(q, 0) G0(q). Then the error of the above prediction becomes (t , ) = y(t ) y (t |0 ) = W (q, )v(t ) (34)
To make this error as small as possible, we must thus match the choice of the lter W(q, 0) to the properties of the noise v(t). Suppose v(t) can be described as ltered white noise v(t ) = H0 (q )e(t ) (35)
That is, we assume the system (plant) dynamics and the noise model to have common poles and no numerator dynamics for (t) will be the noise. Its main feature is that the predictor y linear in the parameters according to Eq. (11) or Eq. (7). We can make Eq. (40) more general by allowing also numerator dynamics. We then obtain the parameterization G ( q, ) = C (q ) B (q ) , H ( q, ) = A (q ) A (q ) (41)
where e(t) is a sequence of independent random variables. Here, we assume H0(q) to be normalized, so that H0(q) 1 h1q1 . . . . Then, it is easy to see from Eq. (34) that no lter W(q 0) can do better than 1/ H0(q), since this makes the prediction error (t, 0) equal to the white noise source e(t). All this leads to the following summarizing conclusion (which is the only thing one needs to understand from this section). 1. In order to distinguish between different predictors, one has to introduce descriptions of the disturbances that act on the process. 2. If the inputoutput description is assumed to be y(t ) = G(q )u(t ) + H (q )e(t ) (36)
The effect of the numerator C is that the current predicted value of y will depend upon previous predicted values, not just measured values. This is known as an ARMAX model, since the C(q)-term makes the noise model a moving average of a white noise source. Also, Eq. (41) assumes that the dynamics and the noise model have common poles, and is, therefore, particularly suited for the case where the disturbances enter together with the input, early in the process, so to speak. The output error (OE) model we considered in Eq. (23) corresponds to the case G ( q, ) = B (q ) , H ( q, ) = 1 F (q ) (42)
where e(t) is a white noise source, then the natural predictor of y(t) given previous observations of inputs and outputs will be y (t | ) = [1 H 1 (q )] y(t ) + H 1 (q )G(q )u(t ) (37)
[We use F in the denominator to distinguish the case from Eq. (40).] Its unique feature is that the prediction is based on past inputs only. It also concentrates on the model dynamics and does not bother about describing the noise. We can also generalize this model by allowing a general noise model G ( q, ) = C (q ) B (q ) , H ( q, ) = F (q ) D (q ) (43)
This predictor gives the smallest possible error if e(t) indeed is white noise. 3. Since the dynamics G(q) and the noise model H(q) are typically unknown, we will have to work with a parameterized description y(t ) = G(q, )u(t ) + H (q, )e(t ) (38)
The corresponding predictor is then obtained from Eq. (37): y (t | ) = [I H 1 (q, )] y(t ) + H 1 (q, )G(q, )u(t ) (39)
We may now return to the question we posed earlier. What is the practical difference between minimizing Eqs. (10) and (25)? Comparing Eq. (23) with Eq. (29), we see that this predictor corresponds to the assumption that H 1, i.e., that white measurement noise is added to the output. This also means that minimizing the corresponding prediction error, Eq. (25), will give a clearly better estimate, if this assumption is more or less correct. Linear Black-Box Model Parameterization The model parameterization in Eq. (38) contains a large number of much-used special cases. We have already seen that the ARX model in Eq. (3) corresponds to G ( q, ) = 1 B (q ) , H ( q, ) = A (q ) A (q ) (40)
This particular model parameterization is known as the BoxJenkins (BJ) model, since it as suggested in the well-known book by Box and Jenkins (6). It differs from the ARMAX-model in Eq. (42) in that it assigns different dynamics (poles) to the noise characteristics from the inputoutput properties. It is thus better suited for cases where the noise enters late in the process, such as measurement noise. See Fig. 3. One might wonder why we need all these different model parameterizations. As has been mentioned in the text, each has its advantages which can be summarized as follows ARX. Gives a linear regression, very simple to estimate ARMAX. Gives reasonable exibility to the noise description, assumes that noise enters like the inputs OE. Concentrates on the inputoutput dynamics BJ. Very exible, assumes no common characteristics between noise and inputoutput behavior. Physically Parameterized Linear Models So far, we have treated the parameters only as vehicles to give reasonable exibility to the transfer functions in the general linear model in Eq. (38). This model can also be arrived at from other considerations.
269
e u B
If we select the state variables
1 A
ARX
x(t ) = we obtain the state space form
y(t ) y (t )
e u B F
OE
x =
0 1 x+ 0 a y = (1 0 ) x + v
0 u b
(45)
e C
where v denotes disturbances and noise. In this case, we thus have
= A( ) =
a b 0 0 1 a B( ) = 0 b (46)
1 A
ARMAX
C = (1 0 )
The parameterization reects our insight that the system contains an integration, but is in this case not directly derived from detailed physical modeling. Basic physical laws would, in this case, have given us how depends on physical constants, such as resistance of the wiring, amount of inertia, friction coefcients, and magnetic eld constants.
e C D u B F y
BJ
Now, how do we t a continuous-time model in Eq. (44a) to sampled observed data? If the input u(t) has been piecewise constant over the sampling interval u(t ) = u(kT ) kT t < (k + 1 )T then the states, inputs, and outputs at the sampling instants will be represented by the discrete time model
Figure 3. Linear black-box model structures.
Consider a continuous time state space model x (t ) = A( )x(t ) + B( )u(t ) y(t ) = C( )x(t ) + v(t ) (44a) (44b)
x((k + 1 )T ) = A( )x(kT ) + B( )u(kT ) y(kT ) = C( )x(kT ) + v(kT )

where
(47)
Here, x(t) is the state vector and typically consists of physical variables (such as positions and velocities, etc). The state space matrices A, B, and C are parameterized by the parameter vector , reecting the physical insight we have into the process. The parameters could be physical constants (resistance, heat transfer coefcients, aerodynamical derivatives, etc.) whose values are not known. They could also reect other types of insights into the systems properties. Example 6 An Electric Motor Consider an electric motor with the input u being the applied voltage and the output y being the angular position of the motor shaft. A rst but reasonable approximation of the motors dynamics is as a rst-order system from voltage to angular velocity, followed by an integrator: G (s ) = b s (s + a )
A( ) = eA ( )T ,
B( ) =
0
eA ( ) B( ) d
(48)
If the input is not piecewise constant, other sampling rules than Eqs. (47) and (48) will apply. This follows from solving Eq. (44) over one sampling period. We could also further model the added noise term v(kT) and represent the system in the innovations form
x((k + 1 )T ) = A( )x(kT ) + B( )u(kT ) + K ( )e(kT ) y(kT ) = C( )x(kT ) + e(kT )
(49)
where e(kT) is white noise. The step from Eqs. (47) to (49) is really a standard Kalman lter step: x will be the one-step ahead predicted Kalman states. A pragmatic way to think about it is as follows: In Eq. (47), the term v(kT) may not be white noise. If it is colored, we may separate out that part of v(kT) that cannot be predicted from past values. Denote this part by e(kT): it will be the innovation. The other part of
270
v(kT), the one that can be predicted, can then be described as a combination of earlier innovations, e(T), k. Its effect on y(kT) can then be described via the states by changing them from x to x, where x contains additional states associated with getting v(kT) from e(T), k . Now, Eq. (49) can be written in inputoutput form as (let T 1) y(t ) = G(q, )u(t ) + H (q, )e(t ) with (50)
where (t ) = (Z t 1 ) (55)
Let the dimension of be d. As before, we shall call this vector the regression vector, and its components will be referred to as the regressors. We also allow the more general case that the formation of the regressors is itself parameterized: (t ) = (Z t 1 , ) (56)
G(q, ) = C( )(qI A( )) 1 B( ) H (q, ) = I + C( )(qI A( )) 1 K ( )
(51)
We are, thus, back at the basic linear model in Eq. (38). The parameterization of G and H in terms of is, however, more complicated than the ones we discussed earlier. The general estimation techniques, model properties [including the characterization in Eq. (85)], algorithms, etc. apply exactly as described in the section on general parameter estimation techniques. From these examples, it is also quite clear that non-linear models with unknown parameters can be approached in the same way. We would then typically arrive at a structure x (t ) = f (x(t ), u(t ), ) y(t ) = h(x(t ), u(t ), ) + v(t ) (52)
which we for short write (t, ). For simplicity, the extra argument will, however, be used explicitly only when essential for the discussion. The choice of the nonlinear mapping in Eq. (53) has thus been reduced to two partial problems for dynamical systems: 1. how to choose the nonlinear mapping g() from the regressor space to the output space (i.e., from Rd to Rp) 2. how to choose the regressors (t) from past inputs and outputs The second problem is the same for all dynamical systems, and it turns out that the most useful choices of regression vectors are to let them contain past inputs and outputs, and possibly also past predicted/simulated outputs. The regression vector will thus be of the character in Eq. (6). We now turn to the rst problem. Nonlinear Mappings: Possibilities Now, let us turn to the nonlinear mapping g(, ) (57)
In this model, all noise effects are collected as additive output disturbances v(t) which is a restriction, but also a very helpful (t) as the simulated output resimplication. If we dene y sponse to Eq. (52), for a given input, ignoring the noise v(t), everything to be said about parameter estimation, model properties, and so on is still applicable. MODEL STRUCTURES II: NONLINEAR BLACK-BOX MODELS In this section, we shall describe the basic ideas behind model structures that have the capability to cover any nonlinear mapping from past data to the predicted value of y(t). Recall that we dened a general model structure as a parameterized mapping in Eq. (1): y (t | ) = g(, Z t 1 ) (53)
which for any given maps from Rd to Rp. For most of the discussion, we will use p 1; that is, the output is scalarvalued. At this point, it does not matter how the regression vector (1, . . ., d)T was constructed. It is just a vector that lives in Rd. It is natural to think of the parameterized function family as function expansions: g(, ) = k gk () (58)
We shall consequently allow quite general nonlinear mappings g. This section will deal with some general principles for how to construct such mappings, and will cover articial neural networks as a special case. See Refs. 11 and 12 for recent and more comprehensive surveys. Nonlinear Black-Box Structures Now, the model structure family in Eq. (53) is really too general, and it turns out to be useful to write g as a concatenation of two mappings: one that takes the increasing number of past observations Zt1 and maps them into a nite dimensional vector (t) of xed dimension and one that takes this vector to the space of the outputs: y (t | ) = g(, Z t 1 ) = g((t ), ) (54)
We refer to gk as basis functions, since the role they play in Eq. (58) is similar to that of a functional space basis. In some particular situations, they do constitute a functional basis. Typical examples are wavelet bases (see below). We are going to show that expansion in Eq. (58), with different basis functions, plays the role of a unied framework for investigating most known nonlinear black-box model structures. Now, the key question is: How to choose the basis functions gk? The following facts are essential to understand the connections between most known nonlinear black-box model structures: All the gk are formed from one mother basis function, that we generically denote by (x). This function (x) is a function of a scalar variable x.
271
Typically, gk are dilated (scaled) and translated versions of . For the scalar case d 1, we may write gk () = gk (, k , k ) = (k ( k )) (59)
used methods for constructing multivariable basis functions from single-variable basis functions. 1. Tensor Product. Given d single-variable functions of the different components j of a d-dimensional vector , h1(1), . . ., hd(d) (identical or not), the tensor product construction of the corresponding function from Rd is then given by their product. In the present case, this means that the basis functions are constructed from the scalar function as
d
We, thus, use k to denote the dilation parameters and k to denote translation parameters. A Scalar Example: Fourier Series Take (x) cos(x). Then Eqs. (58) and (59) will be the Fourier series expansion, with k as the frequencies and k as the phases. Another Scalar Example: Piecewise Constant Functions Take as the unit interval indicator function:
gk () =
j=1
j (k ( j kj ))
(64)
(x ) =
1 0
for 0 x < 1 else
(60)
2. Radial Construction. For any single-variable function , the radial construction of multivariable basis function of Rd has the form gk () = gk f (, k , k ) = ( k
k )
and take, for example, k k, k 1/ , and k f (k). Then Eqs. (58) and (59) give a piecewise constant approximation of any function f . Clearly, we would have obtained a quite similar result by a smooth version of the indicator function, for example the Gaussian bell:
2 1 ( x ) = e x / 2 2
(65)
where k denotes any chosen norm on the space of the regression vector . The norm could typically be a quadratic norm
2 k
(61)
= T k
(66)
A Variant of the Piecewise Constant Case Take to be the unit step function
(x ) =
0 for x < 0 1 for x 0
(62)
with k as a possibly k-dependent positive denite matrix of dilation (scale) parameters. In simple cases, k may be just scaled versions of the identity matrix. 3. Ridge Construction. Let be any single-variable function. Then for all k Rd, k R, a ridge function is given by
T gk () = gk (, k , k ) = (k + k ), Rd
(67)
We, then, just have a variant of Eq. (60), since the indicator function can be obtained as the difference of two steps. A smooth version of the step, like the sigmoid function (x ) = (x ) = 1 1 + e x (63)
will, of course, give quite similar results. Classication of Single-Variable Basis Functions. Two classes of single-variable basis functions can be distinguished depending on their nature: Local basis functions are functions having their gradient with bounded support, or at least vanishing rapidly at innity. Loosely speaking, their variations are concentrated to some interval. Global basis functions are functions having a nonvanishing gradient for all arguments. Clearly, the Fourier series is an example of a global basis function, while Eqs. (60), (61), (62) and (63) are all local functions. Construction of Multivariable Basis Functions. In multidimensional case (d 1), gk are multivariable functions. In practice, they are often constructed from the single-variable function in some simple manner. Let us recall the three most often
The ridge function is thus constant for all in the subspace Rd : T k constant. As a consequence, even if the mother basis function has local support, the basis functions gk will have unbounded support in this subspace. The resulting basis could be said to be semiglobal, but the term ridge function is more precise. Approximation Issues. For any of the described choices, the resulting model becomes
en
g(, ) =
k=1
k (k ( k ))
(68)
with the different exact interpretations of the argument k( k) just discussed. The expansion is entirely determined by the scalar valued function (x) of a scalar variable x the way the basis functions are expanded to depend on a vector The parameterization in terms of can be characterized by three types of parameters: the coordinates the scale or dilation parameters the location parameters
272
A key issue is how well the function expansion is capable of approximating any possible true system g0(). There is rather extensive literature on this subject. For an identication oriented survey, see, for example, Ref. 12. The bottom line is easy: For almost any choice of (x)except being a polynomialthe expansion in Eq. (68) can approximate any reasonable function g0() arbitrarily well for sufciently large n. It is not difcult to understand this. It is sufcient to check that the delta functionor the indicator function for arbitrarily small areascan be arbitrarily well approximated within the expansion. Then, clearly, all reasonable functions can also be approximated. For local with radial construction, this is immediate: By scaling and location, an arbitrarily small indicator function can be placed anywhere. For the ridge construction, one needs to show that a number of hyperplanes dened by and can be placed and intersect so that any small area in Rd is cut out. The question of how efcient the expansion is, that is, how large n is required to achieve a certain degree of approximation is more difcult and has no general answer. We may point to the following aspects: If the scale and location parameters and are allowed to depend on the function g0 to be approximated, then the number of terms n required for a certain degree of approximation is much less than if k, k; k 1, . . . is an a priori xed sequence. For the local, radial approach, the number of terms required to achieve a certain degree of approximation of a p times differentiable function is proportional to n 1 (d / p ) (69)
hyperplane structure is obtained (15). Nearest Neighbors or Interpolation. By selecting as in Eq. (60) and the location and scale vector k, k in structure Eq. (65), such that exactly one observation falls into each cube, the nearest neighbor model is obtained: just load the input-output record into a table, and for a , given , pick the pair (y ) for closest to the given ; y is the desired output estimate. If one replaces Eq. (60) by a smoother function and allows some overlapping of the basis functions, we get interpolation-type techniques such as kernel estimators. Fuzzy Models. Fuzzy models based on fuzzy set membership belong to the model structures of class Eq. (58). The basis functions gk then are constructed from the fuzzy set membership functions and the inference rules using the tensor approach Eq. (64). The exact relationship is described in Ref. 11. Estimating Nonlinear Black-Box Models The model structure is determined by the following choices: the regression vector (typically built up from past inputs and outputs) the basic function the number of elements (nodes) in the expansion Eq. (58). Once these choices have been made, y (t) g((t), ) is a well dened function of past data and the parameters . The parameters are made up of coordinates in expansion Eq. (58) and from location and scale parameters in the different basis functions. All the algorithms and analytical results of the next section can thus be applied. For neural network applications, these are also the typical estimation algorithms used, often complemented with regularization, which means that a term is added to the criterion Eq. (74) that penalizes the norm of . This will reduce the variance of the model, in that spurious parameters are not allowed to take on large, and mostly random, values (11). For wavelet applications, it is common to distinguish be (t) (i.e., the tween those parameters that enter linearly in y coordinates in the function expansion) and those that enter non-linearly (i.e., the location and scale parameters). Often, the latter are seeded to xed values, and the coordinates are estimated by the linear least squares method. Basis functions that give a small contribution to the t (corresponding to nonuseful values of the scale and location parameters) can then be trimmed away (pruning or shrinking). GENERAL PARAMETER ESTIMATION TECHNIQUES In this section, we shall deal with issues that are independent of model structure. Principles and algorithms for tting models to data, as well as the general properties of the estimated models, are all model-structure independent and equally well applicable to, say, ARMAX models and neural network models. The section is organized as follows. The general principles for parameter estimation are outlined. Then, we deal with the
It thus increases exponentially with the number of regressors. This is often referred to as the curse of dimensionality. Connection to Named Structures. Here we briey review some popular structures; other structures related to interpolation techniques are discussed in Refs. 11 and 12. Wavelets. The local approach corresponding to Eqs. (58, and 65) has direct connections to wavelet networks and wavelet transforms. The exact relationships are discussed in Ref. 11. Loosely, we note that via the dilation parameters in k, we can work with different scales simultaneously to pick up both local and not-so-local variations. With appropriate translations and dilations of a single suitably chosen function (the mother wavelet), we can make expansion Eq. (58) orthonormal. This is discussed extensively in Ref. 12. Wavelet and Radial Basis Networks. The choice in Eq. (61) without any orthogonalization is found in both wavelet networks (13) and radial basis neural networks (14). Neural Networks. The ridge choice in Eq. (67) with given by Eq. (63) gives a much-used neural network structure through the one hidden layer feedforward sigmoidal net. Hinging Hyperplanes. If instead of using the sigmoid function we choose V-shaped functions (in the form of a higher-dimensional open book), Briemans hinging
273
asymptotic (in the number of observed data) properties of the models, while algorithms are described in the last section. Fitting Models to Data Earlier, we showed several ways to parameterize descriptions of dynamical systems. This is actually the key problem in system identication. No matter how the problem is approached, the bottom line is that such a model parameterization leads to a predictor y (t | ) = g(, Z t 1 ) (70)
choose L (q ) = 1 and ( ) = log f e ( ) (76)
The MLE has several nice statistical features and, thus, gives a strong moral support for using the outlined method. Another pleasing aspect is that the method is independent of the particular model parameterization used (although this will affect the actual minimization procedure). For example, the method of back propagation often used in connection with N neural network parameterizations amounts to computing in Eq. (74) by a recursive gradient method. We shall deal with these aspects later. Model Quality An essential question is, of course, what properties will the estimate resulting from Eq. (74) have. These will naturally depend on the properties of the data record ZN dened by Eq. (8). It is, in general, a difcult problem to characterize the N exactly. One normally has to be content with the quality of N as the number of data, N, tends asymptotic properties of to innity. It is an important aspect of the general identication method in Eq. (74) that the asymptotic properties of the resulting estimate can be expressed in general terms for arbitrary model parameterizations. The rst basic result is the following one: N where = arg min E (
F (t , ))
that depends on the unknown parameter vector and past Zt1 [see Eq. (8)]. This predictor can be linear in y and u. This, in turn, contains several special cases both in terms of blackbox models and physically parameterized ones, as was discussed earlier. The predictor could also be of general, nonlinear nature, which was also discussed. In any case, we now need a method to determine a good value of , based on the information in an observed, sampled data set in Eq. (8). It suggests itself that the basic leastsquares like approach in Eqs. (9) through (11) still is a natu (t) is a more general ral approach, even when the predictor y function of . A procedure with some more degrees of freedom is the following one: (t), form the 1. From observed data and the predictor y sequence of prediction errors, (t , ) = y(t ) y (t | ), t = 1, 2, . . ., N (71)
as N
(77)
2. Possibly lter the prediction errors through a linear lter L(q),

F (t , )
(78)
= L(q ) (t , )
(72)
so as to enhance or depress interesting or unimportant frequency bands in the signals. 3. Choose a scalar valued, positive function ( ) so as to measure the size or norm of the prediction error: (
F (t , ))
(73)
4. Minimize the sum of these norms: N = arg min VN (, ZN )
(74)
That is, as more and more data become available, the estimate converges to that value * that would minimize the expected value of the norm of the ltered prediction errors. This is in a sense the best possible approximation of the true system that is available within the model structure. The expectation E in Eq. (78) is taken with respect to all random disturbances that affect the data, and it also includes averaging over the input properties. This means, in particular, that * will make y (t*) a good approximation of y(t) with respect to those aspects of the system that are enhanced by the input signal used. The second basic result is the following one. If (t, *) is N approximately white noise, then the covariance matrix of is approximately given by N )( N )T E ( [E (t ) T (t )] 1 N (79)
where
VN (, ZN ) =
1 N
(
t =1
F (t , ))
(75)
where =E (t ) =
2
This procedure is natural and pragmatic; we can still think of (t). It also has several it as curve-tting between y(t) and y statistical and information theoretic interpretations. Most importantly, if the noise source in the system [like in Eq. (38)] is supposed to be a sequence of independent random variables e(t) each having a probability density function f e(x), then Eq. (74) becomes the maximum likelihood estimate (MLE) if we
(t , )
(80) (81)
d y (t | )| = d
Think of as the sensitivity derivative of the predictor with respect to the parameters. Then Eq. (79) says that the covari N is proportional to the inverse of the covariance matrix for
274
ance matrix of this sensitivity derivative. This is a quite natural result. Note that for all these results, the expectation operator E can, under most general conditions, be replaced by the limit of the sample mean; that is,
Measures of Model Fit Some quite general expressions for the expected model t, that are independent of the model structure, can also be developed. Let us measure the (average) t between any model in Eq. (70) and the true system as V ( ) = E y(t ) y (t | )
2
E (t ) T (t ) lim
1 N
(t ) T (t )
t =1
(82)
(87)
The results in Eqs. (77) through (81) are general and hold for all model structures, both linear and nonlinear ones, subject only to some regularity and smoothness conditions. They are also fairly natural and will give the guidelines for all user choices involved in the process of identication. See Ref. 4 for more details around this. Characterization of the Limiting Model in a General Class of Linear Models.. Let us apply the general limit result in Eqs. (77)(78) to the linear model structure in Eq. (38). If we choose a quadratic criterion () 2 (in the scalar output case), then this result tells us, in the time domain, that the limiting parameter estimate is the one that minimizes the ltered prediction error variance (for the input used during the experiment.) Suppose that the data actually have been generated by y(t ) = G0 (q )u(t ) + v(t ) (83)
Here, expectation E is over the data properties [i.e., expectation over Z with the notation Eq. (8)]. Recall that expectation also can be interpreted as sample means as in Eq. (82). Before we continue, let us note the very important aspect that the t V will depend not only on the model and the true system but also on data properties, like input spectra, possible feedback, etc. We shall say that the t depends on the experimental conditions. N is a random variable The estimated model parameter because it is constructed from observed data that can be described as random variables. To evaluate the model t, we N) with respect to the estimathen take the expectation of V( tion data. That gives our measure N ) FN = EV ( (88)
In general, the measure FN depends on a number of things: The The The The model structure used number of data points N data properties for which the t V is dened N properties of the data used to estimate
Let u() be the input spectrum and v() be the spectrum for the additive disturbance v. Then, the ltered prediction error can be written
F (t , ) =
L (q ) [ y(t ) G(q, )u(t )] H ( q, ) L (q ) = [(G0 (q ) G(q, ))u(t ) + v(t )] H ( q, )
The rather remarkable fact is that if the two last data properties coincide, then, asymptotically in N (4, Chap. 16),
(84)
FN V N ( ) 1 +
dim N
(89)
By Parsevals relation, the prediction error variance can also be written as an integral over the spectrum of the prediction error. This spectrum, in turn, is directly obtained from Eq. (84), so the limit estimate * in Eq. (78) can also be dened as
= arg min +

G0 ( ei ) G ( ei , ) 2
u ()
L ( ei ) 2 d H ( ei , ) 2 (85)
v ()
L ( ei ) 2 / H ( ei , ) 2 d
If the noise model H(q, ) H*(q) does not depend on [as in the output error model Eq. (42)], the expression in Eq. (85) thus shows that the resulting model G(ei, *) will give that frequency function in the model set that is closest to the true one, in a quadratic frequency norm with weighting function Q() =
u ()
Here, * is the value that minimizes the expected criterion in Eq. (78). The notation dim means the number of estimated parameters. The result also assumes that the criterion function () 2, and that the model structure is successful in the sense that F(t) is approximately white noise. Despite the reservations about the formal validity of Eq. (89), it carries a most important conceptual message: If a model is evaluated on a data set with the same properties as the estimation data, then the t will not depend on the data properties, and it will depend on the model structure only in terms of the number of parameters used and of the best t offered within the structure. 0(tt 1) The expression can be rewritten as follows: Let y denote the true one step ahead prediction of y(t), and let W ( ) = E y 0 (t |t 1 ) y (t | ) and let = E y(t ) y 0 (t |t 1 )
2 2
(90)
L ( ei ) 2 / H ( ei )
(86)
(91)
This shows clearly that the t can be affected by the choice of prelter L, the input spectrum u, and the noise model H . *
Then is the innovations variance, that is, that part of y(t) that cannot be predicted from the past. Moreover, W(*) is the
275
bias error, that is, the discrepancy between the true predictor and the best one available in the model structure. Under the same assumptions as above, Eq. (89) can be rewritten as FN + W ( ) + dim N (92)
The gradient is, in the general case, a matrix with dim rows and dim y columns. It is well known that gradient search for the minimum is inefcient, especially close to the minimum. Then, it is optimal to use the Newton search direction R 1 ( )VN ( ) where (97)
The three terms constituting the model error then have the following interpretations: is the unavoidable error, stemming from the fact that the output cannot be exactly predicted, even with perfect system knowledge. W(*) is the bias error. It depends on the model structure and on the experimental conditions. It will typically decrease as dim increases. The last term is the variance error. It is proportional to the number of estimated parameters and inversely proportional to the number of data points. It does not depend on the particular model structure or the experimental conditions. Algorithmic Aspects In this section, we shall discuss how to achieve the best t between observed data and the model, that is, how to carry out the minimization of Eq. (74). For simplicity, we here assume a quadratic criterion and set the prelter L to unity:
R( ) = VN ( ) = 1 + N
N
d 2VN ( ) 1 = d 2 N
(t , ) T (t , )
t =1
2 ( y(t ) y (t | )) 2 y (t | ) t =1
(98)
The true Newton direction will, thus, require that the second derivative 2 y (t | ) 2 be computed. Also, far from the minimum, R() need not be positive semidenite. Therefore, alternative search directions are more common in practice: Gradient Direction. Simply take Ri = I (99)
1 VN ( ) = 2N
y(t ) y (t )
t =1
(93)
GaussNewton Direction. Use
No analytic solution to this problem is possible unless the (t) is linear in , so the minimization has to be done model y by some numerical search procedure. A classical treatment of the problem of how to minimize the sum of squares is given in Ref. 16. Most efcient search routines are based on iterative local search in a down-hill direction from the current point. We then have an iterative scheme of the following kind: ( i ) i R 1 g (i+ 1 ) = i i (94)
Ri = Hi =
1 N
N t =1
(i ) ) T (t , (i ) ) (t ,
(100)
LevenbergMarquard Direction. Use Ri = Hi + I (101)
(i) is the parameter estimate after iteration number i. Here, The search scheme is thus made up of the three entities:
i step size (i)) i an estimate of the gradient VN ( g Ri a matrix that modies the search direction Search Directions. The basis for the local search is the gradient
where Hi is dened by Eq. (100). Conjugate Gradient Direction. Construct the Newton direction from a sequence of gradient estimates. Loosely, think of V N as constructed by difference approximation of d gradients. The direction in Eq. (97) is, however, constructed directly, without explicitly forming and inverting V . It is generally considered (16) that the GaussNewton search direction is to be preferred. For ill-conditioned problems, the LevenbergMarquard modication is recommended. Local Minima. All properties of the estimate that we have discussed really refer to the global minimum of the criterion. A fundamental problem with minimization tasks like Eq. (9) is that VN() may have several or many local (non-global) minima, where local search algorithms may get caught. There is no easy solution to this problem. It is usually well worth the effort to nd a good initial value (0) where to start the iterations. Other than that, only various global search strategies are left, such as random search, random restarts, simulated annealing, and the genetic algorithm.
VN ( ) =
where
1 dVN ( ) = d N
( y(t ) y (t | )) (t , )
t =1
(95)
(t , ) =
y (t | )
(96)
276
SPECIAL ESTIMATION TECHNIQUES FOR LINEAR BLACK-BOX MODELS An important feature of a linear, time invariant system is that it is entirely characterized by its impulse response. So if we know the systems response to an impulse, we will also know its response to any input. Equivalently, we could study the frequency response, which is the Fourier transform of the impulse response. In this section, we shall consider estimation methods for linear systems that do not use particular model parameterizations. First, we shall consider direct methods to determine the impulse response and the frequency respone by simply applying the denitions of these concepts. Then, spectral analysis for frequency function estimation will be discussed. Finally, a recent method to estimate general linear systems (of given order, by unspecied structure) will be described. Transient and Frequency Analysis Transient Analysis. The rst step in modeling is to decide which quantities and variables are important to describe what happens in the system. A simple and common kind of experiment that shows how and in what time span various variables affect each other is called step-response analysis or transient analysis. In such experiments, the inputs are varied (typically one at a time) as a step: u(t) u0, t t0; u(t) u1, t t0. The other measurable variables in the system are recorded during this time. We, thus, study the step response of the system. An alternative would be to study the impulse response of the system by letting the input be a pulse of short duration. From such measurements, information of the following nature can be found: 1. The variables affected by the input in question. This makes it easier to draw block diagrams for the system and to decide which inuences can be neglected. 2. The time constants of the system. This also allows us to decide which relationships in the model can be described as static (that is, they have signicantly faster time constants than the time scale we are working with. 3. The characteristic (oscillatory, poorly damped, monotone, and the like) of the step responses, as well as the levels of static gains. Such information is useful when studying the behavior of the nal model in simulation. Good agreement with the measured step responses should give a certain condence in the model. Frequency Analysis. If a linear system has the transfer function G(q), and the input is u(t ) = u0 cos kT, (k 1 )T t kT (102)
If the system is driven by the input [see Eq. (102)] for a certain u0 and 1 and we measure y0 and from the output signal, it is possible to determine the complex number G(ei1T) using Eqs. (104)(105). By repeating this procedure for a number of different , we can get a good estimate of the frequency function G(eiT). This method is called frequency analysis. Sometimes, it is possible to see or measure u0, y0, and directly from graphs of the input and output signals. Most of the time, however, there will be noise and irregularities that make it difcult to determine directly. A suitable procedure is then to correlate the output with cos t and sin t. Estimating the Frequency Response by Spectral Analysis Denitions. The cross spectrum between two (stationary signals u(t) and y(t) is dened as the Fourier transform of their cross covariance function, provided this exists:
yu ()
Ryu ( )ei
(106)
where Ryu() is dened by Ryu ( ) = Ey(t )u(t ) (107)
The (auto) spectrum u() of a signal u is dened as uu(), that is, as its cross spectrum with itself. The spectrum describes the frequency contents of the signal. The connection to more explicit Fourier techniques is evident by the following relationship
u ()
= lim
1 U () N N
(108)
where UN is the discrete time Fourier transform

N
UN () =
t =1
u(t )eit
(109)
The relationship in Eq. (108) is shown, for example, in Ref. 3. Consider now the general linear model: y(t ) = G(q )u(t ) + v(t ) (110)
It is straightforward to show that the relationships between the spectra and cross spectra of y and u (provided u and v are uncorrelated) is given by
yu () y ()
= G ( ei )
2
u ()
(111)
v ()
= G ( ei )
u ()
(112)
then the output after possible transients have faded away will be y(t ) = y0 cos (t + ), where y0 = G ( e i T ) u 0 = arg G(ei T ) (104) (105) for t = T, 2T, 3T, . . . (103)
It is easy to see how the transfer function G(ei) and the noise spectrum v() can be estimated using these expressions, if only we have a method to estimate cross spectra. Estimation of Spectra. The spectrum is dened as the Fourier transform of the correlation function. A natural idea would then be to take the transform of the estimate
N ( ) = 1 R yu N
y(t )u(t )
t =1
(113)
277
That will not work in most cases, though. The reason could N yu be described as follows: The estimate R () is not reliable for large since it is based on only a few observations. These bad estimates are mixed with good ones in the Fourier transform, thus creating an overall bad estimate. It is better to introduce a weighting, so that correlation estimates for large lags carry a smaller weight:
about 2 radians/time unit (118)
N yu () =
N ( ) w ( )e i R yu
(114)
This means that details in the true frequency function that are ner than this expression will be smeared out in the estimate. It is also possible to show that the estimates variances satisfy (i) 0.7 Var G N and Var N v () 0.7 N
2 v ()
This spectral estimation method is known as the Blackman Tukey approach. Here, w() is a window function that decreases with . This function controls the trade-off between frequency resolution and variance of the estimate. A function that gives signicant weights to the correlation at large lags will be able to provide ner frequency details (a longer time span is covered). At the same time, it will have to use bad estimates, so the statistical quality (the variance) is poorer. We shall return to this trade-off in a moment. How should we choose the shape of the window function w()? There is no optimal solution to this problem, but the most common window used in spectral analysis is the Hamming window:
v () u ()
(119)
(120)
1 k 1 + cos 2 w ( k ) = 0 w ( k ) =
k < k
(115)
[Variance here refers to taking expectation over the noise sequence v(t).] Note that the relative variance in Eq. (119) typically increases dramatically as tends to the Nyquist frequency. The reason is that G(i) typically decays rapidly, while the noise-to-signal ratio v()/ u() has a tendency to increase as increases. In a Bode diagram, the estimates will thus show considerable uctuations at high frequencies. Moreover, the constant frequency resolution in Eq. (118) will look thinner and thinner at higher frequencies in a Bode diagram due to the logarithmic frequency scale. See Ref. 3 for a more detailed discussion. Choice of Window Size The choice of is a pure trade-off between frequency resolution and variance (variability). For a spectrum with narrow resonance peaks, it is thus necessary to choose a large value of and accept a higher variance. For a more at spectrum, smaller values of will do well. In practice, a number of different values of are tried. Often, we start with a small value of and increase it successively until an estimate is found that balances the trade-off between frequency resolution (true details) and variance (random uctuations). A typical value for spectra without narrow resonances is 20 30. Subspace Estimation Techniques for State Space Models A linear system can always be represented in state space form: x(t + 1 ) = Ax(t ) + Bu(t ) + w(t ) y(t ) = Cx(t ) + Du(t ) + e(t ) (121)
From the spectral estimates u, y, and yu obtained in this way, we can now use Eq. (111) to obtain a natural estimate of the frequency function G(ei):
( ei ) = G N
N yu () N u ()
(116)
Furthermore, the disturbance spectrum can be estimated from Eq. (112) as
N N v () = y ()
N yu () N u ()
(117)
To compute these estimates, the following steps are performed: 1. Collect data y(k), u(k), k 1, . . ., N. 2. Subtract the corresponding sample means from the data. This will avoid bad estimates at very low frequencies. 3. Choose the width of the lag window w(k). N N N y u yu 4. Compute R (k), R (k), and R (k) for k according to Eq. (113). N N N y u yu 5. Form the spectral estimates (), (), and () according to Eq. (114) and analogous expressions. 6. Form Eq. (116) and possibly also Eq. (117).
N N and w are Quality of the Estimates. The estimates G formed entirely from estimates of spectra and cross spectra. Their properties will, therefore, be inherited from the properties of the spectral estimates. For the Hamming window with width , it can be shown that the frequency resolution will be
We assume that we have no insight into the particular structure, and we would just estimate any matrices A, B, C, and D, that give a good description of the input-output behavior of the system. This is not without problems, among other things because there are an innite number of such matrices that describe the same system (the similarity transforms). The coordinate basis of the state-space realization, thus, needs to be xed. Let us for a moment assume that not only are u and y measured, but also the sequence of state vectors x. This would, by the way, x the state-space realization coordinate basis. Now, with known u, y, and x, the model in Eq. (121) becomes a linear regression: the unknown parameters, all of
278
the matrix entries in all the matrices, mix with measured signals in linear combinations. To see this clearly, let
Y (t ) = = (t ) = E (t ) =
x(t + 1 ) y(t ) A C B D
quoted earlier are not directly applicable. Experience has shown, however, that condence intervals computed according to the general asymptotic theory are good approximations. One may also use the estimates obtained by a subspace method as initial conditions for minimizing the prediction error criterion [see Eq. (74)]. DATA QUALITY It is desirable to affect the conditions under which the data are collected. The objective with such experiment design is to make the collected data set ZN as informative as possible with respect to the models to be built using the data. A considerable amount of theory around this topic can be developed and we shall here just review some basic points. The rst and most important point is the following one 1. The input signal u must be such that it exposes all the relevant properties of the system. It must, thus, not be too simple. For example, a pure sinusoid u(t ) = A cos t will only give information about the systems frequency response at frequency . This can also be seen from Eq. (85). The rule is that the input must contain at least as many different frequencies as the order of the linear model to be built. To be on the safe side, a good choice is to let the input be random (such as ltered white noise). It then contains all frequencies. Another case where the input is too simple is when it is generated by feedback such as u(t ) = Ky(t ) If we would like to build a rst-order ARX model (125)
x(t ) u(t ) w(t ) e(t )
Then, Eq. (121) can be rewritten as Y (t ) = (t ) + E (t ) (122)
From this, all the matrix elements in can be estimated by the simple least squares method, as described earlier. The covariance matrix for E(t) can also be estimated easily as the sample sum of the model residuals. That will give the covariance matrices for w and e, as well as the cross covariance matrix between w and e. These matrices will, among other things, allow us to compute the Kalman lter for Eq. (121). Note that all of the above holds without changes for multivariable systems, that is, when the output and input signals are vectors. The only remaining problem is where to get the state vector sequence x. It has long been known, for example (17,18), that all state vectors x(t) that can be reconstructed from inputoutput data in fact are linear combinations of the components of the n k-step ahead output predictors y (t + k|t ), k = {1, 2, . . ., n} (123)
where n is the model order (the dimension of x). See also Appendix 4.A in Ref. 4. We could then form these predictors and select a basis among their components:
x(t ) = L
y (t + 1|t ) . . . y (t + n|t )
(124)
y(t ) + ay(t 1 ) = bu(t 1 ) + e(t ) we nd that for any given , all models such that a + bK = will give identical inputoutput data. We can, thus, not distinguish between these models using an experiment with Eq. (125). That is, we can not distinguish between any combinations of a and b if they satisfy the above condition for a given . The rule is If closed-loop experiments have to be performed, the feedback law must not be too simple. It is to be preferred that a set-point in the regulator is being changed in a random fashion. The second main point in experimental design is 2. Allocate the input power to those frequency bands where a good model is particularly important. This is also seen from the expression in Eq. (85).
The choice of L will determine the basis for the state-space realization and is done in such a way that it is well condi (t kt) is a linear function of u(s), tioned. The predictor y y(s), 1 s t and can efciently be determined by linear projections directly on the input output data. (There is one complication in that u(t 1), . . ., u(t) k should not be predicted, even if they affect y(t) k).) What we have described now is the subspace projection approach to estimating the matrices of the state-space model in Eq. (121), including the basis for the representation and the noise covariance matrices. There are a number of variants of this approach. See among several references, for example, Refs. 19 and 20. The approach gives very useful algorithms for model estimation and is particularly well suited for multivariable systems. The algorithms also allow numerically very reliable implementations. At present, the asymptotic properties of the methods are not fully investigated, and the general results
279
If we let the input be ltered white noise, this gives information on how to choose the lter. In the time domain, it is often useful to think like this: Use binary (two-level) inputs if linear models are to be built; this gives maximal variance for amplitude-constrained inputs. Check that the changes between the levels are such that the input occasionally stays on one level so long that a step response from the system has time, more or less, to settle. There is no need to let the input signal switch so quickly back and forth that no response in the output is clearly visible. Note that the second point is really just a reformulation in the time domain of the basic frequency domain advice: let the input energy be concentrated in the important frequency bands. A third basic piece of advice about experiment design concerns the choice of sampling interval. 3. A typical good sampling frequency is 10 times the bandwidth of the system. That corresponds roughly to 57 samples along the rise time of a step response. MODEL VALIDATION AND MODEL SELECTION A Pragmatic Viewpoint The system identication process has, as we have seen, these basic ingredients The set of models The data The selection criterion Once these have been decided upon, we have, at least implicitly, dened a model: The one in the set that best describes the data according to the criterion. It is thus, in a sense, the best available model in the chosen set. But is it good enough? It is the objective of model validation to answer that question. Often, the answer turns out to be no, and we then have to go back and review the choice of model set, or perhaps modify the data set. See Fig. 4. How do we check the quality of a model? The prime method is to investigate how well it is capable of reproducing the behavior of a new set of data (the validation data) that was not used to t the model. That is, we simulate the obtained model with a new input and compare this simulated output. One may then use ones eyes or numerical measurements of t to decide if the t in question is good enough. Suppose we have obtained several different models in different model structures (say a fourth-order ARX model, a second-order BJ model, a physically parameterized one, and so on) and would like to know which one is best. The simplest and most pragmatic approach to this problem is then to simulate each one of them on validation data, evaluate their performance, and pick the one that shows the most favorable t to measured data. (This could indeed be a subjective criterion!)
Construct experiment, collect data Data Filter data? Polish and present data Data Choose model structure Fit model to data Model
Validate model Data not OK Model structure not OK No Accept model? Yes
Figure 4. The identication loop.
The Bias-Variance Trade-off The heart of the model structure selection process is to handle the trade-off between bias and variance, as formalized by Eq. (92). The best model structure is the one that minimizes FN, the t between the model and the data for a fresh data setone that was not used for estimating the model. Most procedures for choosing the model structures are also aiming at nding this best choice. Cross Validation. A very natural and pragmatic approach is cross validation. This means that the available data set is N1 that is used to estisplit into two parts, estimation data, Zest mate the models: N = arg min VN (, ZN1 ) est
1 1
(126)
N2 for which the criterion is evaluated: and validation data, Zval
= V ( N , ZN 2 ) F N N val
1 2 1
(127)
N will be an unbiHere, VN is the criterion in Eq. (75). Then, F ased estimate of the measure FN, dened by Eq. (88), which was discussed at length in the previous section. The procedure would the be to try out a number of model structures N . and choose the one that minimizes F 1 Such cross validation techniques to nd a good model structure have an immediate intuitive appeal. We simply check if the candidate model is capable of reproducing data it hasnt yet seen. If that works well, we have some condence in the model, regardless of any probabilistic framework that might be imposed. Such techniques are also the most commonly used ones. A few comments could be added. In the rst place, one could use different splits of the original data into estimation
280
and validation data. For example, in statistics, there is a common cross validation technique called leave one out. This means that the validation data set consists of one data point at a time but successively applied to the whole original set. In the second place, the test of the model on the validation data does not have to be in terms of the particular criterion [see Eq. (127)]. In system identication, it is common practice to simulate (or predict several steps ahead) the model using the validation data and then visually inspect the agreement between measured and simulated (predicted) output. Estimating the Variance ContributionPenalizing the Model Complexity. It is clear that the criterion in Eq. (127) has to be evaluated on the validation data to be of any use; it would be strictly decreasing as a function of model exibility if evaluated on the estimation data. In other words, the adverse effect of the dimension of shown in Eq. (92) would be missed. There are a number of criteria, often derived from entirely different viewpoints, that try to capture the inuence of this variance error term. The two best known ones at Akaikes Information Theoretic Criterion, AIC, which as the form (for Gaussian disturbances)
mercial packages for identication available, such as Mathworks System Identication Toolbox (2), Matrixxs System Identication Module (23) and PIM (24). They all have in common that they offer the following routines: Handling of data, plotting, and so on: Filtering of data, removal of drift, choice of data segments, etc. Non-parametric identication methods: Estimation of covariances, Fourier transforms, correlation- and spectralanalysis, etc. Parametric estimation methods: Calculation of parametric estimates in different model structures. Presentation of models: Simulation of models, estimation and plotting of poles and zeros, computation of frequency functions, and plotting Bode diagrams, etc. Model validation: Computation and analysis of residuals N)). Comparison between different models proper((t, ties, etc. The existing program packages differ mainly in various user interfaces and by different options regarding the choice of model structure according to C above. For example, MATLABs Identication Toolbox (2) covers all linear model structures discussed here, including arbitrarily parameterized linear models in continuous time. Regarding the user interface, there is now a clear trend to make it graphically oriented. This avoids syntax problems and relies more on click and move, at the same time as tedious menu-labyrinths are avoided. More aspects of CAD tools for system identication are treated in Ref. 25. How to Get to a Good Model? It follows from our discussion that the most essential element in the process of identicationonce the data have been recordedis to try out various model structures, compute the best model in the structures using Eq. (38), and then validate this model. Typically, this has to be repeated with quite a few different structures before a satisfactory model can be found. While one should not underestimate the difculties of this process, the following simple procedure to get started and gain insight into the models could be suggested: 1. Find out a good value for the delay between input and output, for example, by using correlation analysis. 2. Estimate a fourth order linear model with this delay using part of the data, and simulate this model with the input and compare the models simulated output with the measured output over the whole data record. In MATLAB language, this is simple
(, ZN ) = 1 + 2 dim V N N
1 N
N 2 t =1
(t , )
(128)
and Rissanens Minimum Description Length Criterion, MDL, in which dim in the expression above is replaced by log N dim (21,22). N is then to be minimized both with respect The criterion V to and to a family of model structures. The relation to the expression in Eq. (89) for FN is obvious. Residual Analysis The second basic method for model validation is to examine the residuals (the leftovers) from the identication process. These are the prediction errors N ) = y(t ) y N ) (t ) = (t , (t | that is, what the model could not explain. Ideally, these should be independent of information that was at hand at time t 1. For example, if (t) and u(t ) turn out to be correlated, then there are things in y(t) that originate from N). (t u(t ) but have not been properly accounted for by y The model has then not squeezed out all relevant information about the system from the data. It is good practice to always check the residuals for such (and other) dependencies. This is known as residual analysis. A basic reference for how to perform this is Ref. 10. BACK TO DATA: THE PRACTICAL SIDE OF IDENTIFICATION Software for System Identication In practice, system identication is characterized by some quite heavy numerical calculations to determine the best model in each given class of models. This is mixed with several user choices, trying different model structures, ltering data, and so on. In practical applications, we will thus need good software support. There are now many different com-
z = [y u]; compare(z,arx(z(1:200,:),[4 4 1]));

If the model/system is unstable or has integrators, use prediction over a reasonable large time horizon instead of simulation. Now, either of two things happen: The comparison looks good. Then, we can be condent that with some extra worktrying out different orders
281
and various noise modelswe can ne-tune the model and have an acceptable model quite soon. The comparison does not look good. Then we must do further work. There are three basic reasons for the failure. 1. A good description needs higher order linear dynamics. This is actually, in practice, the least likely reason, except for systems with mechanical resonances. One then obviously has to try higher order models or focus on certain frequency bands by band pass ltering. 2. There are more signals that signicantly affect the output. We must then look for what these signals might be, check if they can be measured, and if so, include them among the inputs. Signal sources that cannot be traced or measured are called disturbances, and we simply have to live with the fact that they will have an adverse effect on the comparisons. 3. Some important nonlinearities have been overlooked. We must then resort to semiphysical modeling to nd out if some of the measured signals should be subjected to nonlinear transformations. If no such transformations suggest themselves, one might have to try some nonlinear black-box model, like a neural network. Clearly, this advice does not cover all the art of identication, but it is a reasonable rst approximation. Example 7 Aircraft Dynamics Let us try the recipe on the aircraft data in Fig. 1. Picking the canard angle only as the input, estimating a fourth order model based on the data points 90 to 180, gives Fig. 5. (We use 10-step ahead prediction in this example since the models are unstable, as they should be; JAS has unstable dynamics in this ight case.) It does not look good. Let us try alternative 2: More inputs. We repeat the procedure using all three
20
40
60
80
100
120
140
160
180
Figure 6. As Fig. 5 but using all three inputs.
inputs in Fig. 1. That is, the model is computed as

arx([y u1 u2 u3], [4 4 4 4 1 1 1])
on the same data set. The comparison is shown in Fig. 6. It looks good. By further ne-tuning, as well as using model structures from physical modeling, only slight improvements are obtained. Example 8 Buffer Vessel Dynamics Let us now consider the pulp process of Fig. 2. We use the number before the vessel as input and the -number after the vessel as output. The delay is preliminarily estimated to 12 samples. Our recipe, where a fourth order linear model is estimated using the rst 200 samples and then simulated over the whole record gives Fig. 7. It does not look good.
6 4 2 0 2 4 6 8
0 20 40 60 80 100 120 140 160 180
100
200
300
400
500
600
Figure 5. Dashed line: actual pitch rate. Solid line: 10 step ahead predicted pitch rate, based on the fourth order model from canard angle only.
Figure 7. Dashed line: -number after the vessel, actual measurements. Solid line: simulated -number using the input only and a fourth order linear model with delay 12, estimated using the rst 200 data points.
282
SYSTEM INTERCONNECTS 9. D. Brillinger, Time Series: Data Analysis and Theory, San Francisco: Holden-Day, 1981. 10. N. Draper and H. Smith, Applied Regression Analysis, 2nd ed, New York: Wiley, 1981, 2nd ed. 11. J. Sjo berg et al., Nonlinear black-box modeling in system identication: A unied overview, Automatica, 31 (12): 16911724, 1995. 12. A. Juditsky et al., Nonlinear black-box modeling in system identication: Mathematical foundations, Automatica, 31 (12): 1724 1750, 1995. 13. Q. Zhang and A. Benveniste, Wavelet networks, IEEE Trans. Neural Networks, 3: 889898, 1992. 14. T. Poggio and F. Girosi, Networks for approximation and learning. Proc. IEEE, 78: 14811497, 1990. 15. L. Breiman, Hinging hyperplanes for regression, classication and function approximation, IEEE Trans. Inf. Theory, 39: 999 1013, 1993.
6 4 2 6 0 4 6 8
50
100
150
200
250
300
350
400
Figure 8. Same as Fig. 7 but applied to resampled data.
16. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Englewood Cliffs, NJ: Prentice-Hall, 1983.
17. J. Rissanen, Basis of invariants and canonical forms for linear dynamic systems, Automatica, 10: 175182, 1974. 18. H. Akaike, Stochastic theory of minimal realization, IEEE Trans. Autom. Control, AC-19: 667674, 1974. 19. P. V. Overschee and B. DeMoor, Subspace Identication of Linear Systems: Theory, Implementation, Application, Boston, MA: Kluwer, 1996. 20. W. E. Larimore, System identication, reduced order ltering and modelling via canonical variate analysis, Proc. 1983 Amer. Control Conf., San Francisco, 1983. 21. H. Akaike, A new look at the statistical model identication, IEEE Trans. Autom. Control, AC-19: 716723, 1974. 22. J. Rissanen, Modelling by shortest data description, Automatica, 14: 465471, 1978. 23. MATRIXx Users Guide, Santa Clara, CA: Integrated Systems Inc., 1991. 24. I. D. Landau, System Identication and Control Design Using P.I.M. Software, Englewood Cliffs, NJ: Prentice-Hall, 1990. 25. L. Ljung, Identication of linear systems, in E. D. Linkens (ed.), CAD for Control Systems, New York: Marcel Dekker, 1993, Chap. 6, pp. 147165.
Some reection shows that this process indeed must be non-linear (or time-varying): the ow and the vessel level denitely affect the dynamics. For example, if the ow was a plug ow (no mixing), the vessel would have a dynamics of a pure delay equal to vessel volume divided by ow. Let us thus resample the date accordingly, that is, so that a new sample is taken (by interpolation from the original measurement) equidistantly in terms of integrated ows divided by volume. In MATLAB terms, this will be z = [y,u]; pf = ow./level; t =1:length(z) newt = table1([cumsum(pf),t],[pf(1)sum:(pf)] ); newz = table1([t,z], newt); We now apply the same procedure to the resampled data. This gives Fig. 8. This looks good. Somewhat better numbers can then be obtained by ne-tuning the orders. BIBLIOGRAPHY
stro 1. K. J. A m and T. Bohlin, Numerical identication of linear dynamic systems from normal operating records, IFAC Symp. Self-Adapt. Syst., Teddington, England, 1965. 2. L. Ljung, The System Identication Toolbox: The Manual, Natick, MA: MathWorks Inc., 1995, 4th ed. 3. L. Ljung and T. Glad, Modeling of Dynamic Systems, Englewood Cliffs, NJ: Prentice-Hall, 1994. 4. L. Ljung, System IdenticationTheory for the User, Englewood Cliffs, N.J: Prentice-Hall, 1987. 5. T. So derstro m and P. Stoica, System Identication, London: Prentice-Hall Int., 1989. 6. G. E. P. Box and D. R. Jenkins, Time Series Analysis, Forecasting and Control, San Francisco: Holden-Day, 1970. 7. J. Schoukens and R. Pintelon, Identication of Linear Systems: A Practical Guideline to Accurate Modeling, London: Pergamon, 1991. 8. L. Ljung and T. So derstro m, Theory and Practice of Recursive Identication, Cambridge, MA: MIT Press, 1983.
LENNART LJUNG
Linko ping University
SYSTEM IDENTIFICATION, MULTIDIMENSIONAL. See MULTIDIMENSIONAL SIGNAL PROCESSING.
}{{}}
HOME
G
ABOUT US
G
CONTACT US
G
HELP
G

Voltage-to-Frequency Converters Standard Article Fernando Vidal-Verd1, Rafael Navas-Gonzlez1, Manuel Delgado-Restituto2, Angel Rodrguez-Vzquez2,3 1Dpto. de Electrnica, Universidad de Mlaga, Campus de Teatinos, Complejo Tecnolgico, Mlaga, SPAIN 2Institute of Microelectronics of Seville, CNM-CSIC, Edificio CICA-CNM, C/ Tarfia sn, Seville, SPAIN 3School of Engineering, University of Seville, Seville, SPAIN Copyright 2007 by John Wiley & Sons, Inc. DOI: 10.1002/047005011X.W2299 Article Online Posting Date: December 27, 1999 Abstract | Full Text: HTML PDF (1080K)
G G
Browse this title

G

H
H H
Abstract
The sections in this article are Introduction Basic Concepts and Models for Oscillator Design Converters Based on Multivibrators Schmitt Trigger Type Converters Design Issues and Limitations of Converters Based on Multivibrators Frequency to Voltage Conversion Applications Acknowledgment About Wiley InterScience | About Wiley | Privacy | Terms & Conditions
Copyright 1999-2008John Wiley & Sons, Inc. All Rights Reserved.
VOLTAGE-TO-FREQUENCY CONVERTERS
INTRODUCTION The Voltage-to-Frequency Converters (VFCs) belong to the category of the oscillator circuits. The function of an oscillator is to produce periodic time-varying signals with only dc excitations 1 . These signals may have quite diverse waveform shapes such as triangular, square, impulsive, exponential, sinusoidal, etc. For some oscillator applications, the shape of the generated signal is an important design criteria. For instance, in the production of periodic sweep voltages for display systems; or in the generation of quasi-sinusoidal signals for testing and instrumentation purposes. In other applications, the important design criteria is the signal frequency, while the spectral purity of the signal remains secondary. This is for instance the case of the oscillators used to generate clock signals for timing purposes. And this is also the case of VFCs. The function of VFCs is to convert the value of an analog input voltage or current into an output frequency; i.e. they must generate signals of any shape whose frequency is a faithful representation of the analog input data. Thus, they also belong to the category of the data converter circuits, whose function is to convert the information among different domains with minimum degradation of the information itself. From this point of view, the VFCs have some similarities with the Analog-to-Digital Converters (ADCs), whose function is to convert an analog data into a digital word with the minimum error possible. In the case of VFCs the domain of the transformed information is the frequency of a signal, instead of a digital word, and the aim is to obtain the smallest possible error in the voltage-tofrequency mapping. The design criteria for VFCs include the following (14): Dynamic Range, i.e. the range of input voltage values where the transformation takes place. A related magnitude is the frequency range the range of output frequencies which corresponds to the input voltage range. 60dB and larger dynamic ranges 2 are commonplace. Linearity of the voltage-to-frequency map, f 0 = F (vin ). This map is required to be highly linear, F (vin ) = k vin + E(vin ) (1)
signal frequency remains essentially unchanged. Then, at the receiver front-end, the analog data is recovered from the signal frequency by using a Frequency-to-Voltage Converter (FVC). From the construction point of view, the VFCs are in the same category as the so-called Volt-age-ControlledOscillators (VCOs), in the sense that the circuit structures employed to realize VFCs and VCOs are actually quite similar. However, the typical applications of VCOs usually demand low precision in the voltage-to-frequency map and, in some cases, high spectral purity. In this chapter, we will start by presenting a broad overview of the different circuit structures that qualify to generate periodic signals with voltage-controlled frequency. Then, in the remaining of the chapter we will focus on those which are better suited to achieve accuracy in the volt-age-to-frequency mapping. BASIC CONCEPTS AND MODELS FOR OSCILLATOR DESIGN Circuit synthesis is the process of interconnecting circuit components so as to realize some targeted behavior. The rst step for systematic circuit synthesis is to identify mathematical descriptions for the targeted behavior. In the case of oscillators, the target is to obtain a steadystate cyclic solution with only dc excitations. For VCOs and VFCs, the timing of these cyclic solutions must also be electronically controllable. The simpler the better; this is a basic motto for electronic circuit design. In the case of oscillators, two ineludible ingredients are dynamics and nonlinearity. Thus, oscillator circuits must contain capacitors and/or inductors together with nonlinear elements. Strictly speaking, the dynamics must be at least of second-order typemeaning that the corresponding circuit requires at least two energystorage elements. With regards to the nonlinearity, the simplest one is the N-like shape of Figure 1(a), which can be approximated by a series expansion, (x) = NI x + NE x3 (2)
where NI and NE are real parameters which can be used to t the required function shape. Alternatively, Figure 1(a) can be approximated by the piecewise-linear curve depicted in Figure 1(b), where the tting is realized through the real parameters NI and NE . Basic Oscillator Model The basic oscillator model is built by combining the ingredients above into dx x = y (x) dt (3) dy =x y dt Figure 2(a) shows a conceptual realization of this equation using two integrators, one adder and one block with nonlinear transfer function (x) all realizable in electronic form. Figure 2(b) and (c) show realizations using capacitors, inductors and nonlinear resistors; it is easy to conrm that Figure 2(b) maps onto Eq. (3) for x = v, y = RiL , x = RC, y = L/R and assuming
with error E(vin ) much smaller than 1% of full scale. Scale Factor Accuracy and Stability. The parameter k in Eq. (1) must be externally set for given voltage-tofrequency gain. After its setting, the gain must remain stable with temperature 100ppm/ C , aging, and changes of the power supply 0.1%/V. Such requirements are related to the main application area of the VFCs, namely the encoding of information for telemetry or remote control. In these applications, the analog data is converted into frequency at the emitter frontend by using a VFC. The generated signal is then transmitted through an optical or radio link a process in which the
Voltage-to-Frequency Converters
Figure 1. N-like shaped characteristics necessary to implement oscillator circuits and associated piecewise-linear approximation.
Figure 2. Conceptual realization of the basic oscillator model in Eq. (3) (a) and LC realizations for x = v, y = RiL , y = L/R (b) and for x = RiL , y = vC , x = L/R, y = CR (c).
that the resistor has a voltage-controlled characteristic iR = (v); similarly, Figure 2(c) maps onto Eq. (3) for x = RiL , y = vc , x = L/R, y = CR and assuming that the resistor has a current-controlled characteristic v = (iR). The model of Eq. (3) generates stable oscillationscalled limit cyclesdue to the dynamic equilibrium between two opposite forces, one expansive, the other contractive. The expansive force happens for |x|< due to the negative slope of the nonlinearity in this regionsee Figure 1(b) 3 . This negative slope can only be realized through an active element which injects energy into the system thereby making |x| increase with time. On the other hand, the contractive force occurs for |x|> due to the positive slope of the nonlinearity in this region. Because of this positive slope, the system dissipates energy and, consequently, |x| decreases with time. The actual trajectories in the transient evolution towards the limit cycle depend on the values of the parameters x , y , NI , NE and . The shape of the limit cycle orbit and the frequency of the motion are also dependent on these values. Finding closed relationships among the shape and frequency, on the one side, and the parameter values, on the other, is not even possible in the more general case. However, for our purposes here it sufces to study the approximated solutions in two particular cases. First of all, we nd convenient rewriting Eq. (3) in terms of a normalized time variable = t/o with o = x y . It yields dx = [y (x)] d dy 1 = x d
responds to 1, and the relaxation oscillator casewhich corresponds to 1. Harmonic Oscillator Case This corresponds to 1, i.e. to y x . For a better understanding of what happens in this case, it is convenient to recast Eq. (4) into the following scalar form, d2x ddx + +x=0 (5) d 2 dxd obtained after differentiating the top expression in Eq. (4) and using the bottom expression to substitute dy/d . Assume rst that the nonlinearity is null. Then d/dx = 0 and the solution of Eq. (5) for initial conditions x(0) = x0 and dx/d | =0 = 0 is x( ) = x0 cos( ), and hence, x(t ) = x0 cos(0 t ) (6) where o 1/0 . Obviously, such a solution could not be kept in practice due to the unavoidable energy dissipationwhich is why nonlinearity is needed for practical oscillator design. However, because 1, the second term in Eq. (5) is negligible and at the limit cycle x(t) is quasi-sinusoidal; x(t ) cos(0 t ) with amplitude for xed and 0 depending on NI , NE and . The set of drawings in Figure 3 illustrate the harmonic oscillation case. On the one hand, they are intended to highlight the dynamic equilibrium process underlying the onset of oscillations. On the other, they give insight on the inuence of the model parameters. All the drawings are for x = 10, y = 0.1 and = 1. The three drawings at the top are for NE = 1 and NI = 1. The one on the left in this subset shows the solution trajectories in the x-y plane for different initial conditions. The opposed expansive and contractive forces are highlighted by this gure. Their equilibrium results in the system evolving towards a limit cycle from any initial condition located either inside or outside it.
(4)
where = y /x .The two particular cases of interest occur for extreme values of the parameter ; they are respectively caalled the harmonic oscillator casewhich cor-
Figure 3. Trajectories, signals and spectrum for the oscillator model in the harmonic oscillation case for x = 10, y = 0.1 and = 1. The three drawings at the top are for NE = 1 and NI = 1, those in the center row are for NE = 1 and NI = 0.2, the drawings at the bottom were obtained for NE = 1 and NI = 3. We see that the oscillation frequency is practically the same in the three cases (basically set by 0 = x y ). We also observe that the larger NI , the larger the signal amplitude and distortion, while the dynamic evolution toward the limit cycle is faster.
The waveform at the center in the top set of gures shows x(t) at the steady-state. Because thisxt() steady-state is obtained as the result of a nonlinear equilibrium process, this waveform is not a pure sinusoidal, but contains harmonics. The spectrum on the right shows these harmonics. The three drawings in the center row in Figure 3 are for NE = 1 and NI = 0.2 . Here the frequenc is practically the same as in the previous case, although the amplitude and distortion of are x(t) now smaller. On the other hand, the evolution from inside the limit cycle towards the limit cycle itself is much slower. However, the last set of drawings, for NE = 1 and NI = 3, disply, again, practically the same frequency but much larger amplitude and distortion, and a very fast evolution to the limit cycle. The following observations on the inuence of parameters can be made regarding the harmonic oscillator case: The oscillation frequency is basically set by 0 = x y . The voltage control of this frequency is hence
achieved through electronic tuning of the time constants x and y For the LC circuits of Figure 2(b) and (c), the frequency can be made voltage-controlled by resorting to the use of tunable capacitors or inductors (1524). On the one hand, for given values of x , y , NE and , the larger NI , the larger the signal amplitude and the signal distortion. Also, the faster the dynamic evolution towards the limit cycle from the inside. On the other hand, for given values of x , y , NI and , the smaller NE , the smaller the signal distortion, although the speed of the dynamic evolution towards the limit cycle from the inside does not change with NE . Finally, the value of inuence either the signal amplitude, but has no inuence on either the signal distortion or the speed of the evolution towards the limit cycle.
Figure 4. (a) Representation of the basic oscillator model in Figure 2(a) as the interconnection of a linear dynamic block and a nonlinear block and (b) general harmonic oscillator model with these blocks. The phase shift of the linear block is null at the oscillation frequency, and the oscillation occurs at the amplitude for which the loop gain is just unity.
General Harmonic Oscillator Model Figure 2(a) can be redrawn as depicted in Figure 4(a), consisting of the interconnection of a linear dynamic block with transfer function in the s-domain given by H (s ) = s x X (s ) = 2 NE Z(s) s + s x +
1 1 x y
(7)
and a nonlinear block with transfer characteristics (x). Note that the linear block acts as a bandpass lter with resonant frequency 0 1/0 and quality factor Q = 1/(NE ).Note also that the gain of this lter at the resonant frequency is 1/NE , and that the nonlinear block gains NE + NI inside its inner linear region. Assume x(t ) cos(t ) . The system can oscillate, provided that the amplitude and the phase of this signal is maintained while the signal is being transmitted across the loop. Consider rst the phase. On the one hand, the nonlinearity is static, and does not produce any phase shift. On the other hand, the phase of the linear block is null only for = 0 which is why the oscillation actually happens at this frequency. Consider now the amplitude. On the one hand, for > , the loop has a gain of 1 + NI /NE , larger than unity, so that the amplitude grows while the signal transverses the loop. On the other hand, for > , the gain is compressed due to the saturation nonlinearity. The compression increases with until a critical amplitude value 0 is reached where the effective gain is unity. Figure 4(b) shows the block diagram of a generic harmonic oscillator model. Based on the explanations made for Figure 4(a) the following statements are made for this generic model: The phase shift of the linear block must be frequencydependent and null at a single frequency = 0 , which is the osocillation frequency. The sharper the phase-vs-frequency characteristics of the linear block around this zero-phase frequency, the larger the accu-
racy of the oscillation frequency setting. Consider now the oscillation amplitude. On the one hand, the loop gain at the zero-phase frequency must be larger than unity for low amplitudes. This ensures that the oscillations will start. On the other hand, the compression exercised by the nonlinear block must render the loop gain smaller than unity for large amplitude values. The oscillation occurs at the amplitude for which the compressed loop gain at the zero-phase frequency is just unity. This can be applied to the ring oscillator structure of Figure 5(a)commonly employed in 10 VCOs. It is composed of N identical stages, Neach with linearized transfer function (22) , F (s) = Gm R 1 + sRC (8)
connected into a feedback loop. On the one hand, note that each stage produces a phase shift of atan(RC); the zero-phase frequency is hence readily calculated as, f0 = 1 tan {3 + (1)N } 2RC 2N (9)
On the other hand, the nonlinearity, and hence the control of the oscillation amplitude, is inherent to the active devices used to realize the transconductors. For the simple circuits typically employed in these transconductors, such as for instance those in Figure 5(b), the transconduc-tance decreases with the signal amplitude. Hence, the condition for the generation of oscillations is Gm R > 1. Relaxation Oscillator Case It corresponds to in Eq. (4) atbeing much larger than unity, i.e. to x y . In the limit, as , its factor in the top equation of Eq. (4) becomes negligible and hence, y = (x) (10)
Figure 5. (a) Ring oscillator structure and its linearized model as a specic example of Figure 4. (b) Simple typical circuit realizations for the tunable transconductors.
Figure 6. The limit cycle in the relaxation case ts with the nonlinear function (x) in (a) as depicted in (b), where we assume that the horizontal transitions occur instantaneously. The bottom drawing (c) shows the limit cycle, the x(t) waveform and its spectrum for x = 0.1, y = 10, = 1 and NE = NI = 1.
meaning that the shape of the limit cycle ts with the nonlinear function (x), drawn with a thick solid trace in Figure 6(a). This piece of information has to be complemented with the analysis of the solution trajectories of Eq. (4) the different linear pieces of (x) : Let us consider rst the inner
piece, where y = NI x. Analysis obtains, x( ) 1 [y(0) NI x(0)]eNI NI y( ) [y(0)]e NI
(11)
valid for |x|<. Note that in the limit, fo , y( ) remains practically constant while the system evolves in this inner region. Quite on the contrary, x(t) changes at very high speed; either decreasing, for y(0) > NI x(0), or increasing,
for y(0) < NI x(0). The arrows drawn with thin solid lines in Figure 6(a) indicate the direction and sense of the trajectories in this inner region. From this picture, it is readily inferred that the nonlinearity inner piece is not part of the limit cycle. Let us now consider the outer pieces, where y = NE x , where the minus is for the left-hand piece, the plus for the right-hand one, and is the absolute value of the ordinate at the origin. Analysis obtains a fast transient across a quasi-horizontal trajectory followed by a slow asymptotic behavior whose trajectory is described by 1 ( ) 1 NE x( ) [y(0) ( )]e NE (12) 1 ( ) x( ) [y(0) ( )]e NE Note that this slow asymptotic behavior results in trajectories which t exactly with the corresponding outer piece of the nonlinear function. As a summary of these considerations, the limit cycle can be built as shown in Figure 6(b), where we assume that the horizontal transitions occur instantaneously. Thus, the oscillation frequency is basically determined by the time invested in making the asymptotic evolutions on the outer pieces of the nonlinearity, which can be calculated from Eq. (12) as Tc = 2y NE ln + NI NI (13)
These summary considerations are conrmed by the simulations depicted in Figure 6(c), which shows the limit cycle, the x(t) waveform and its spectrum for x = 0.1, y = 10, = 1 and NE = NI = 1. Let us at this point explore the implications of the conditions leading to the relaxation oscillator case on the LC circuits of Figure 2(b) and (c). These conditions are met by making C 0 in the former case; and by making L 0 in the latter. For circuit implementation purposes, these negligible elements can be considered parasitics and, hence, the relaxation oscillator can be built by simply connecting either an inductor or a capacitor to a nonlinear resistor. Figure 7 shows these realizations, where the nonlinear resistor is N-shaped for Figure 7(a), and S-shaped for Figure 7(b). CONVERTERS BASED ON MULTIVIBRATORS Since the frequency in oscillators can be controlled by means of tunable capacitors, resistors or inductors, any oscillator can be used to translate a voltage into a frequency, provided that the voltage to be converted is that used to tune the devices (1524). However, in the case of harmonic oscillators, these mechanisms do not provide a good enough linear dependence of the output frequency on the input voltage nor temperature stability for most voltage-to-frequency applications. They are used instead in PLL (phase locked loops) applications, many allowing nonlinearities up to several tens of percent and temperature induced variations as long as the tuning range accommodates them (23). The jitter in these circuits is also lower
than in relaxation oscillators, which make them more appropriate for timing applications (11). Multivibrators have usually poor timing accuracy for frequencies above a few hundred kHzs because of the random uctuations of the threshold levels and charging and discharging currents and their increasing importance with respect to shorter clock periods (1), but they provide the best linearity and temperature and supply stability at medium-low frequencies, as well as wide control and signal ranges (8). Since the latter are features required for voltage-to-frequency conversion, most VFCs on the market are based on multivibrators, thus we will focus on this approach in the following. Implementations of VFCs are usually based on capacitors as energy storage elements, and the nonlinear resistor in Figure 7(b) is commonly implemented as depicted in Figure 8(a). The current switches in Figure 8(a) are set to the Ch or Dh position by the reversal block, corresponding to the charging phase and the discharging phase respectively. During the charging phase, the current iC enters the capacitor input node (+) and the voltage vC grows, while it decreases during the discharging phase, wherein the current iD leaves the capacitor input node. The circuit operation consists in the succession of these two phases in a cyclic behavior. The period of the cycle is determined on the one hand by the value of iC and iD and the capacitance C associated to the timing capacitor, which set the rate for the voltage vC to change inside each phase, and on the other hand by the reversal block, which senses vC and turns the switches to the Ch or Dh positions. Assuming constant values for the currents iC and iD , the time intervals corresponding to the charging and discharging phases are (see Figure 8(c)) , tC = C(vCH vCL ) iC tD = C(vCH vCL ) iD (14)
where vCH and vCL are the top and bottom limits respectively of the voltage swing in the capacitor terminals. In the specic case of having i = ic = iD , the time invested in a cycle is Tc = tC + tD = 2C(vCH vCL ) i (15)
where we have supposed that the delay associated with the reversal block and the current switches is negligible, which corresponds to the assumption x y in the previous section. Actually, the expression for Tc above is also derived from Eq. (13) by making NE ,which provides Tc = 4y NI (16)
for the time invested in completing the cyclic diagram in Figure 8(b) (see also Figure 6). Note that Eq. (16) yields Eq. (15) if NI is obtained from Figure 8(a) as NI = vCH vCL 2iR (17)
and taking into account that y = RC. The VFCs currently on the market can be classied into two major categories that follow the previous approach, the Schmitt trigger type converters and the charge balancing type converters. The main difference between both types
Figure 7. Simplied rst-order abstractions for a relaxation oscillator. The conditions leading to the relaxation oscillator case on the LC circuits of Figure 2(b) and Figure 2(c) are met by making C -> 0 in the former case (a) and L 0 in the latter (b).
Figure 8. Realization of relaxation oscillators based on Figure 7(b). The reversal block controls the switches to alternatively charge and discharge the capacitor (a). The associated cyclic diagram corresponding to the general case in Figure 6 is shown in (b), while (c) depicts the resultant waveform.
lies in the role that the reversal block plays. In the former, the reversal block works like a Schmitt trigger, which senses vC (t) and turns the current switch to the position Ch or Dh any time it reaches the low vCL and high vCH threshold values respectively. Thus, the reversal block imposes the thresholds vCH and vCL , and sets directly the voltage swing (vCH vCL ) in the CLacitor during the circuit operation. Eq. (14) applies here and the frequency of the waveform associated to vC is controlled by means of voltage controlled current sources, thus making iC = KC vin and iD = KD vin obtain f = 1 KC KD vin = t C + tD (KC + KD)C(vCH vCL ) (18)
lower threshold vCL . At this time, the switch is turned to Ch by the reversal block and the capacitor is charged but now for a xed period T0 . Thus, the voltage vC changes an amount given by vC = (i0 iD )T0 iC T0 = C C (19)
Once this charging phase is concluded, the switch is set to Dh by the reversal block and the capacitor is discharged by iD until the lower threshold is reached again, hence the voltage vC falls an amount of vC and the duration of this phase is tD = C vC i0 = ( 1)T0 iD iD (20)
where vin is the input voltage that is converted into the frequency f. In the particular case of having KC = KD , we get a symmetrical triaKC=KDngular shaped periodic signal. On the other hand, in the charge-balancing type converters, the reversal block does not impose lower and upper limits or thresholds to the voltage drop in the capacitor, but a xed duration of the charging phase T0 . These circuits also are built to fulll iC = i0 iD , where i0 is a constant current. The basic working principles are similar to those in the Schmitt trigger converters, and the circuit evolves in a cycle with successive charging and discharging phases. Let us begin in a certain point inside the discharging phase. The voltage vC ramps down until it equals a
since T0 and i0 are constant values, the frequency is changed by tuning iD in Eq. (20). If iD = KD vin , we obtain the following expression for the frequency of vC (t), f = iD KD vin 1 = = tC + tD i0 T0 i0 T0 (21)
In the following we will show implementations of these two classes of converters and we will discuss design issues and non idealities.
Figure 9. (a) Direct implementation of a Schmitt trigger type VFC converter, where the reversal block in Figure 8 is implemented by a Schmitt trigger circuit.(b) Two realizations of the Schmitt trigger block.
SCHMITT TRIGGER TYPE CONVERTERS Direct Implementation Figure 9(a) shows a straightforward implementation of a Schmitt trigger type converter with a grounded capacitor. Two possible realizations of the Schmitt trigger block are also depicted in Figure 9(b) (14). In the single-comparatortype Schmitt trigger on the left of Figure 9(b), for very low values of vi the comparator output is low and the current switch is open, thus the voltage reference at the inverting input is viH = VCC R2 + R3 R1 + R2 + R3 (22)
with supply voltage is 2%/V. Emitter-coupled multivibrator Instead of a grounded capacitor, the circuit in Figure 10 uses a oating one which is charged and discharged by means of the bias current sources iC and iD . Basically, the circuit xes the voltage at one terminal of the capacitor, for instance, the emitter of Q3 , to a nearly constant voltage VM , while the other terminal is connected to the current source iC that charges it. Eventually, the voltage in the latter drops enough to allow the base-to-emitter voltage of Q2 to reach the transistor threshold, which causes the voltage at the base of Q3 to fall and the transistor is turned off. The voltage at the emitter of Q2 is now xed to two baseto-emitter drops below the supply voltage and the capacitor is being discharged by iD . Since the circuit is symmetrical, the process is repeated periodically. Let us split the cycle in periods related to important events in order to derive the expression of the output wave frequency: Figure 10(b) shows the circuit working somewhere in the cycle with the transistor Q2 and the diode D1 in the cut-off region, which is indicated by using a white trace. Consider the clamping block in Figure 10(a) formed by one diode and one resistor in parallel. The voltage drop across this block is depicted in Figure 10(a), as well as the response to a step in the current ID . As long as Q2 is off, the voltage across the block equals zero if we neglect the base and leakage currents. Since Q1 and Q3 are on, the voltage at node B is clamped to VM = VCC 2 VB Eon approximately. On the other hand, the current source iC is charging the capacitor with a constant current, thus the voltage at the emitter of Q2 falls to a lower threshold VL beyond which Q2 is not longer cut-off. Since D2 and Q4 are on, such a limit is given by the equation, VL = VCC (VD2 + VBEQ4 + VBEQ2 ) VCC 3 VB Eon (25) Once the voltage at node A reaches VL , the transistor Q2 turns on and its collector current makes the voltage drop across the left clamping block rise to VBEcon , thus the base of Q1 drops to VCC VB Eon and the emitter of Q1 follows this voltage, hence the voltage at the base of Q3 is now VCC 2VB Eon . Since its emitter was xed
For increasing values of the input voltage, eventually the reference in Eq. (22) is crossed and the comparator output changes and turns the switch on, thus the resistor R3 is short circuited. Hence, if the input voltage decreases at this point, the new reference voltage at the inverting input of the comparator is viL = VCC R2 R1 + R2 (23)
Note that Eq. (22) and Eq. (23) constitute the high and low threshold voltages, respectively, of the single-comparator Schmitt trigger. The main drawback of this circuit is that the parasitic capacitors at the nodes between the resistors have to be charged and discharged every time the switch is closed or opened, which is a time-consuming task and causes errors that limit the high frequency capability of the circuit. A better behavior is obtained by using two comparators and xed voltage references to dene the threshold values. The right part of Figure 9(b) shows the dual-comparator Schmitt trigger which is based on this approach and whose threshold voltages are determined by the resistor ladder as R3 R2 + R3 viL = VCC viH = VCC (24) R1 + R2 + R3 R1 + R2 + R3 Besides other general limitations that will be seen later, the basic disadvantage of this strategy is the poor supply voltage stability, due to the dependence of the threshold values on VCC . Table 1 shows the typical performance parameters associated to the NE5664 , which follows the direct approach in Figure 9(a) and whose maximum drift
Voltage-to-Frequency Converters Table 1. Typical performance parameters of the NE566 Maximum operating frequency Sweep range Frequency drift with supply voltage Frequency drift with temperature FM distortion (10%deviation) 1MHz 10:1 0.2 %/V 600 ppm/ C 0.4%
Table 2. Typical performance parameters of the AD537 JD Frequency range Nonlinearity (fmax = 100kHz) Error drift with temperature (0 to 70 C) Error drift with supply voltage (fmax < 100kHz) 0kHz to 150kHz 0.15% 50 ppm/ C 0.01%/V
Table 3. Typical performance parameters of the AD VFC32KN Frequency range Nonlinearity (fmax = 500kHz) Error drift with temperature (0 to 70 C) Error drift with supply voltage (max) 0kHz to 500kHz 0.05% 75 ppm/ C 0.015% of FSR%
Table 4. Typical performance parameters of the AD652 JP Gain Temperature Coefcient fCLOCK = 200kHz fCLOCK = 4MHz Linearity Error fCLOCK = 200kHz fCLOCK = 4MHz Power Supply Rejection Ratio 25ppm/ C 25ppm/ C 0.002% 0.02% 0.001 %/V
Figure 10. Emitter-coupled multivibrator. The clamping block in (a) causes a voltage drop of value VBEon for a current ID VBEon /R, that makes the basis voltage of Q2 fall, thus it is cut-off and the capacitor is charged through . The situation reiCmains until the voltage drop at the emitter of Q2 is low enough to turn on Q2 , thus the clamping block at the top-left corner cuts-off Q3 and the situation is reversed. The process is repeated cyclically.
to the same voltage, the transistor Q3 turns off. Note that at this point, the voltage drop across the capacitor is VC = VB VA = VBEon Since Q3 is cut off, its collector current vanishes and the voltage drop across the right clamping block goes down to zero. As a consequence, the voltage at node A is pulled up to the value VCC 2VB Eon . On the other hand, since the switching time is very short, the capacitor remains charged and the voltage increment is translated into an increment in the voltage at node B, which reaches the value,
Note that the situation is symmetrical to that in the rst point, but now the voltage at nodeA remains constant while the current source iD discharges the capacitor, which is equivalent to the initial point above. From the previous discussion we conclude that in each charging or discharging phase, the voltage at one capacitor terminal A or B remains constant, while the other changes from the voltage VH to VL . It is then easy to derive the voltage increment in the capacitor, vC = vCH vCL = (VH VM ) (VL VM ) = 2VBEon (27)
VH = VCC VBEon
(26)
Since the problem is equivalent to that treated in the previous section, equation Eq. (18) is also valid here and the
10
frequency of oscillation is given by the expression, f = where KC =KD =K. Improved emitter-coupled multivibrator Since the base-to-emitter voltage has a strong temperature coefcient, the circuit in Figure 10 has as a main drawback the dependence of Eq. (28) on the temperature. One strategy to compensate this dependence consists of making that the current that charges and discharges the capacitor is dependent on the temperature for cancelling the global dependence (14). Such an approach achieves circuits whose output frequency does not change with the temperature for vin =0, while there is a dependence for values of vin different from zero. Another strategy tries to make vC = vCH vCL equal to a quantity very stable under temperature variations. That is the strategy followed in the circuit of Figure 11, which basically works in the same way as the previous one in Figure 10. The main difference consists in the change of the voltage values used to clamp the collectors of Q1 and Q2 . For this purpose, the simple diodes in Figure 10 are substituted by a more complex circuitry depicted at the top of Figure 11. In addition to the current sources, the circuit uses two reference voltages to implement the strategy: a very stable voltage reference VR , and the reference voltage VCC 2VBEon generated by D5 and D6 . In addition, one diode bridge derives the voltages at the collectors of Q1 and Q2 from these references. Figure 11 illustrates the same situation as in Figure 10, where Q1 was cut-off, the voltage at node B was xed, and the capacitor was being charged by iC . The same process is going on now in Figure 11 where the devices in white are off, but the voltages at the collectors of Q1 and Q2 are set to VCC VBEon and VCC VR VBEon , respectively, by the above referred top circuitry. Taking into account the voltage drops across the emitter followers as well as that in Q2 , the voltage at B is VM = VCC 3 VBEon (29) Kvin 4CVBEon (28)
deviations introduced by the voltage-to-current converter at input. The other major class of commercial voltage-tofrequency converters are based on a charge balancing system. As said in Section 3, these circuits are characterized by having a xed length in time in the charging phase. We can classify them into two major categories, those based on one-shot timers and those which are synchronized to a clock signal. Charge balancing converters based on one-shot timers Figure 12 illustrates a typical implementation of these converters, whose working principle we now describe. Suppose that vC vin , then the comparator provides a high output. As a consequence, the one-shot timer output rises and the current switch is set to the Ch position. This behavior of the one-shot continues under the start-up conditions or in the case of a change in the input voltage, and the frequency of the timer output signal is zero. Once the condition vC > vin is reached, the one-shot is reset and the current switch is set to the Dh position, thus the capacitor is discharged through R. However, as soon as the condition vC vin is fullled again, the timer is triggered and the switch is turned to Ch again to allow the current ow into the integrator. This time the one-shot timer operates as usual and generates a pulse of xed duration tos , thus a packet of charge is injected in the RC integrator and vC ramps upward by an amount dvc (i0 iD ) vC = tos = tos (33) dt C which is enough to make vC > vin , thus the one-shot is reset and the capacitor is discharged again until vC reaches vin . If vC vC , we can approximate iD vin /R, which is a constant current that discharges C, thus the time tD is given by tD | vC |
C | dv dt
tos ((i0 iD )/C)

iD C
= tos (
i0 R 1) vin
(34)
and the voltage atA beyond which Q1 is not off anymore is VL = VCC VR 3 VBEon (30)
The cycle is repeated again as soon as vC equals vin , thus the output of the one-shot timer is a periodic signal whose frequency is f = 1 vin = tos + tD tos i0 R (35)
When this limit is reached, the situation is reversed, Q1 is turned on and Q2 is turned off. At this point, the voltage across the capacitor is vC = VB VA = VM VL = VR , and the nal voltage at B after the switching will be VH = VM + VR = VCC 3VBEon + VR (31)
Now from Eqs. (29), (30) and (31) vC = vCH vCL = (VH VM ) (VL VM ) = 2VR and Eq. (18) reduces to f = K Vin 4CVR (32)
The frequency of oscillation can now be stable if VR is designed carefully to have a low temperature coefcient, which is described later. Table 2 shows some typical parameters of the AD537JD (2). The data refer to the current-tofrequency converter block without taking into account the
Note that this coincides with Eq. (21) for KD = 1/R and T0 = tos . The circuit in Figure 12 has two main drawbacks. First, the change of vC causes an error in the charging current due to the nite output impedance of the current source that generates it, as well as in the discharging current iD , which is not constant as supposed above. Second, the passive RC integrator is very slow, especially for large input voltage changes, which require a large amount of charge to be injected in or extracted from the capacitor. Both aspects can be improved by using an active integrator, as Figure 13 depicts (327). The negative feedback provides low impedance at the inverting input terminal of the integrator, whose voltage remains nearly constant, thus the error in the charging and discharging currents is reduced. Equations Eq. (33)-Eq. (35) are valid in Figure 13(b), but now iD = vin /R exactly. With respect to Figure 13(a), vC
11
Figure 11. Improved emitter-coupled multivibrator. The clamping circuitry is redesigned to make the voltage variations at the capacitor depend on a voltage (VR ) very stable against temperature changes, thus the frequency drift with temperature is improved.
Figure 12. Basic diagram of a charge-balancing converter based on a one-shot timer. The charging phase has xed length in time, which is the width of the pulse generated by the one-shot timer (tos ) when it is triggered by the comparator. For vC vC , the average voltage at the comparator inverting input is vin , thus iD vin /R. The average current injected in the bottom integrator must equal iD and depends on the rate the one-shot is triggered with, hence this rate depends on vin .
Figure 13. Improved converters based on one-shot timers. The use of active integrators instead of the passive one in Figure 12 achieves smaller errors and improves the dynamic response.
diminishes in the reset mode, thus the derivative in Eq. (33) is negative, as is the voltage increment, but Eq. (34) and Eq. (35) are valid. In both cases, the output current of the op amp in the integrator is the same for the charging and the discharging modes, which improves the dynamic
response because this minimizes the transients. In addition, Figure 13(a) provides high impedance for the input terminal, which can be useful for applications with low load requirements. Finally, note that the capacitance of the integrating capacitor does not appear in Eq. (35), thus it
12
does not inuence directly the output frequency. However, it certainly inuences the shape of vC (t), because if C is too small, vC in Eq. (33) could become too large and exceed the linear range of the integrator. In (16), a nonlinearity error of 0.024% FS (f= 10Hz to 11 kHz) is reported for the approach in Figure 12, while it is reduced to 0.006% FS for the improved converter in Figure 13(b). The input voltage offset Voff of the comparator in Figure 12 and the integrators in Figure 13 introduces an error Vo f f f = | | in the expected off frequency - an effect that tos i0 R can be signicant, especially for low frequencies, in terms of relative error. On the other hand, precision and low temperature coefcient current sources to derive io are generated on-chip by means of a voltage band-gap reference and a precision voltage-to-current conversion circuit (see Section 7), while further reections and replications should also be carried out by cascode current mirrors. Nevertheless, accuracy as well as high frequency capability of these converters is primarily determined by the precision of the one-shot section. Figure 14 shows the simplest form of an exponentialramp-type one-shot timer (14). The principle of operation is as follows. As long as the ip-op is reset (low input), the voltage at the output Q is high and the current switch (which can be implemented in the simplest way by a grounded transistor) is open, thus the voltage drop in the capacitor is zero. If the circuit is triggered by a high input, the outputs Q and Q of the ip-op are set to a high and a low voltage respectively, thus the switch is turned off and the capacitor is charged through R, and vCos grows exponentially with a time constant of RCos . However, as soon as the reference voltage at the non-inverting input (which is determined by the voltage divider formed by Ra and Rb ) is reached, the comparator output changes to a low voltage value and the ip-op is reset, hence the switch is closed and the capacitor is discharged quickly. Simple calculations on the circuit formed by the supply voltage source, R and Cos provides tos = RCos ln(1 + Rb ) Ra (36)
one bistable and one AND logic gate, which works as follows. If the integrator output crosses the threshold vref , the comparator places a high value at the bistable input (suppose that Q is high). However, the output does not follow it immediately as in Figure 13(b), but it is synchronized with the clock by the ip-op and changes at the next clock rising edge. At the same time, the output complement (low value) is put at the ip-op input by the feed-back loop, thus the output will go back down at the next rising edge of the clock and the duration of the output pulse is just one clock cycle. Note that the circuit operates in the charging phase as long as the output is high, thus TCLK replaces tos in the equations Eq. (33), Eq. (34) and Eq. (35), which remain valid. However, since the output is now synchronized, its frequency must be a multiple of the clock frequency , f = 1 1 = tD 0R TCLK + TCLK iS ( TCLK ) TCLK [1 + iS ( iv 1)] in (37)
The voltage divider formed by Ra and Rb is usually implemented on-chip and the accuracy obtained for the ratio Rb /Ra is within 1%, and does not depe1nd on the temperature. Thus, the precision, as well as the temperature stability in the expected tos are mainly determined by the resistor R and the capacitor. However, for small values of tos , the time delays in the input comparator, the ip-op and the switch limit the accuracy. In addition, for low frequencies the leakage currents in the capacitor and the switch introduce an error in the timing prediction. Table 3 shows some typical performance parameters of the monolithic converter ADVFC32KN (4). Synchronized charge balancing type converters Further improvements can be achieved by replacing the one-shot timer by circuitry clocked by a precise oscillator, usually a crystal one, which determines the charging phase duration (5). Figure 15 shows the circuit which results from substituting the one-shot timer with a block composed of
where is in Eq. (37) means the next superior integer of the argument. Table 4 shows some typical performance parameters of the synchronized converter AD652JP(5). In addition to other design issues that will be discussed in the next section, Figure 15 has a specic problem to solve due to the lack of synchronism between the input voltage and the clock. The latter could cause that the comparator output changes at the same time as the clock arrives at the D ip-op. Since the latter are usually designed to work synchronously, a setup time must be respected for the input signal to change before the clock edge, otherwise there could be metastability problems that do not guarantee a correct value of the op output, thus tC can be shorter or larger and the circuit does not work properly. Is it possible to avoid this problem by building the D ip-op with two ip-ops in cascade or with a ip-op and a latch in cascade (5), like the right part of Figure 15 shows, where the clock edge for both ops is delayed. The simplest way to have delayed edges for both ops consists in using the clock falling edge to trigger the rst ip-op, while the second op is triggered by the rising edge, thus they are delayed half a clock cycle. With this strategy, the metastability of the rst ip-op has ended by the time the second ip-op is clocked, and the pulse width of the output is a clock cycle, thus there is not error in tC . Sigma-delta modulators Figure 16 shows a rst-order sigma-delta modulator. It basically consists in a loop with a 1-bit analog to digital converter, which is the latched comparator, and a 1-bit digital to analog converter in the feedback path. Provided the loop has enough gain, the difference between the analog input and the D/A converter output is minimized and the average value of the D/A output must approach that of the input signal. Similarly, the averaging of the comparator output provides a digital word that represents the analog input. This averaging is described later as part of the processing required to obtain a digital word from the frequency output of a VFC (digital processing in Section 7), thus sigmadelta modulators can be viewed as synchronized chargebalancing voltage to frequency converters (the charge is injected into or removed from the integrator for the DAC
13
Figure 14. Exponential-ramp-type one-shot timer
Figure 15. Synchronized charge balancing converter. The one-shot timer is replaced by externally clocked circuitry. The charging phase length is now xed by the clock period, which is supposed very accurate.
Figure 16. Sigma-delta modulator as synchronized charge-balancing voltage to frequency converter.
output to track the input voltage). In fact, the transfer function of the AD7740 (6) is f = 0.1 f CLK + 0.8(vin /vre f ) f CLK . Since sigma-delta modulators can be built currently with cheap CMOS technology, the main advantage of this approach is the cost. The AD7741 (7) with 0.012% Nonlinearity at 2.75MHz of FoutMax costs ve times less than the synchronous AD652 with 0.02% Nonlinearity at 2MHz of FoutMax. In addition, they need fewer external components.
rived without taking into account the deviations from the ideal case. Let us now show some real implementations of formerly ideal blocks and discuss deviations from the ideal case and their consequences in the output wave shape. Implementation of the Voltage Controlled Current Sources Many VFCs described above use voltage controlled current sources to implement the charging and discharging currents in Figure 8, where iC = KC vin and iD = KD vin . such proportional relationship is obtained readily with a resistor, but a transistor is needed to isolate the control node from the output, which is depicted in Figure 17(a). Such a circuit provides the reference current , vin vBEon (38) iB R which acts as charging or discharging current. However, note that this circuit provides a current that is not proportional to vin , but contains an offset that equals (vBEon /R + ire f =
DESIGN ISSUES AND LIMITATIONS OF CONVERTERS BASED ON MULTIVIBRATORS Although some design aspects relative to the design and non-idealities have already been discussed in previous sections, for more clarity, some others regarding all the converters based on relaxation oscillators are examined here. Specically, note that Eq. (15) and Eq. (18) have been de-
14
Figure 17. Controlled current source implementation
iB ). Besides modifying the intended proportional relationship, the added term has a strong temperature coefcient, since the base to emitter voltage varies with the temperature as 2mV/ C. A common alternative for eliminating the dependence of iref on VBEon places the transistor in a negative feedback loop as Figure 17(b) illustrates. If we neglect the base current, the output reference current for this circuit is ire f = A vBEon vin (A + 1)R (A + 1)R (39)
Dependence of the Frequency on Temperature and Voltage Supply Variations The temperature stability of the converter output frequency depends on many factors, some of which have already been discussed above. First, in order to improve the temperature stability we should avoid circuits like that in Figure 10 whose output frequency depends directly on parameters like VBEon that has a large temperature coefcient. Other circuits described above must still be designed carefully to avoid drift. First, voltage references and charging currents have a large inuence on the temperature stability. For instance, the voltage source VR in the equation Eq. (32) must be designed to compensate the temperature dependence. Figure 19(a) shows an example circuit able to generate a reference voltage VR with a low temperature coefcient (13). Such a circuit takes advantage of the different sign of the temperature coefcients associated to the voltage-to-emitter drop in a BJT and the thermal voltage VT . The box in Figure 19(b) encloses a circuit able to generate a current that is proportional to VT . As long as the voltage at both output terminals A1 and A2 is the same, i.e. VA1 = VA2 , the current I2 is given by the equation I2 = (VBEQ1 VBEQ2 ) VT nI1 = ln( ) R2 R2 I2 (41)
Note that for A 1 the term depending on VBEon vanishes and the reference current equals ire f vin R (40)
which is proportional to vin . The higher the value of the input amplierA, the larger the accuracy of Eq. (40). In addition, a MOS transistor can replace the bipolar one in Figure 17(b) to reduce the error introduced by neglecting the base current, due to the high impedance associated to the gate of MOS transistors. Finite output resistance of the constant currents The current source depicted in Figure 17(b) still has a non-ideal behavior that can introduce deviations in the expected output current. Specically, since the collector current depends on the collector-to-emitter voltage drop because of the B JT base length modulation, the current iref varies with vo and this causes an error if the output node directly drives the terminals of the capacitor in Figure 8, because the voltage drop in the capacitor changes as a consequence of the circuit operation. We can model this dependence on the output node voltage by means of a resistor in parallel with an ideal current source whose value is the expected one. This is illustrated in Figure 18, where the Thevenin equivalent is also depicted. If the circuit is now connected to the timing capacitor, the voltage vC does not follow a linear behavior with t, but a well-known exponential behavior. As a consequence, the time to charge or discharge the capacitor an amount vC is longer than expected, which is indicated in Figure 18(a) by means of the error t . For the circuits described above, this means longer periods or lower frequencies than those in the ideal case. In order to improve the output impedance of the current source, several cascode congurations have been reported (13). Some usual ones are depicted in Figure 18(b), where the current i ref equals iref , but the output resistance the circuit in Figure 17.
Two possible approaches to force VA1 =VA2 are illustrated in Figure 19(b). The circuit at the top of Figure 19(b) uses a negative feedback loop with an operational amplier that ensures zero differential input, thus VA 1=VA2 . Taking into account that the voltage drop across resistors R1 and R3 is the same, we can derive the following expression for VR in Figure 19(b) VR = VBEQ1 + R3 VT nR3 ln( ) R2 R1 (42)
Another strategy uses cascode transistors to get VA 1=VA2 . This is the approach followed by the circuit at the bottom of Figure 19(b), where the reference voltage is VR = VBEQ3 + R3 VT ln(n) R2 (43)
The reference voltage in Eq. (42) and Eq. (43) is made independent of the temperature variations by choosing properly the circuit and device parameters to achieve dVR |T =T0 = 0 (see (13) for more details). dT On the other hand, stable current sources can be obtained by driving the circuit in Figure 17(b) with a stable voltage source like that in Fig. 19, which is the strategy followed by many commercial VFCs like the AD537(2).
15
Figure 18. Current source nite output resistance: timing error (a) and common cascode strategies to increase the output resistance (b).
Figure 19. Band-gap reference voltage. (a) The voltages VBE and VT have temperature VTcoefcients of opposite sign, thus they compensate the global dependence of VR .(b) Two realizations of this approachVR.
Another element that affects the temperature stability is the dependence on the temperature of the Schmitt trigger thresholds, specically that of the difference vC = vCH vCL in Eq. (15). We have already discussed the particular case of the emitter-coupled circuit in Figure 11. With regard to the direct implementation in Figure 9, the reference voltages and the comparator offsets are the main sources of drift. Contributions to global drift are in both cases a few tens of ppm /C. In the case of the charge-balancing converters, note that vC is determined mainly by the charging and discharging currents and by the parameter T0 in Eq. (21). The drift of T0 in converters based on one-shot timers is due mainly to the resistance R and the capacitance Cos in Eq. (36) and Figure 14, which are usually external in most commercial VFCs. Finally, the stability of T0 is obviously improved when such a parameter is determined by a precise external clock as in the synchronous converters. On the other hand, frequency drift with the supply voltage is also mainly due to variations of the charging and discharging currents as well as that of vC . Band-gap references like those described above also provide supply independent biasing.
input stage, bias currents have to be provided from the input to the sensing nodes. Such currents are usually in the range of a few hundred nano-amperes and add to or subtract from charging and discharging currents. MOS transistors can be used instead, to implement almost innite input impedance thanks to the isolation provided by the gate oxide. In addition, junction leakage or dielectric absorption currents can also be added to or subtracted from charging currents, thus also affecting the output frequency. However, these currents are commonly at a low nanoampere range. Hence, errors due to these base and leakage currents can be usually tolerated as long as the charging and discharging currents are in the range of 10A (14). Finite Switching Times The switching times in Figure 8 have been neglected in the computation of the output frequency in Eq. (18). However, they are not zero in real implementations and limit the frequency capability of the converter. The elements that contribute to the delay in switching are mainly the length of the feedback path and the delays of the blocks along it, while the gain of the active devices, the parasitic capacitances and the internal current levels determine such delays. Output Interfacing The output stage of the commercial VFCs is designed to allow easy interfacing to all digital logic families. Hence, BJT open collector or both uncommitted collector and emit-
Finite Sensing Node Impedance and Leakage Currents Note that Figure 8 assumes innite impedance in the reversal block sensing input nodes. Real implementations however do not usually present such a feature. Specically, if the reversal block is built with bipolar transistors in the
16
ter interfaces are provided. External pull-up resistors and proper biasing complete the interface and determine the rise times and output logic levels that are adequate for each specic application. Internally, some interface circuitry is needed to drive the base of the output BJT for circuits like that on the right of Figure 11 (2), while it is not necessary in one-shot based converters like those in (3, 25), whose one-shot output drives it directly. Synchronous converters also may use a one-shot timer to drive the output transistor, which allows the pulse width of the frequency output to be controlled by means of an external capacitor. External Component Selection Timing capacitors in the converters described above, as well as resistors in Figs. 12,14 and 15 are usually external components in commercially available VFCs. Special care must be taken to select these components in order to preserve performance. High linearity and low temperature coefcients for the resistors and the capacitors are required. In addition, the latter should not leak, since dielectric absorption can affect the linearity and offset of the transfer function. FREQUENCY TO VOLTAGE CONVERSION The complementary operation of that carried out by the voltage to frequency converters is the frequency to voltage conversion, which is performed by the Frequency to VoltageConverters (FVCs). This task can be often implemented by the same ICs that perform the V/F conversion, and it is applicable in areas like telemetry, motor speed control, as well as to build interfaces for sensors whose output is in the form of a variable frequency or pulse train (14, 19). Figure 20(a) shows how this conversion can be accomplished by means of an integrator and a current switch that is driven by the frequency signal in the so called pulse integrating FVCs. The switch can also be driven by the output of a one-shot timer (see Figure 20(a)), which reduces the integration time, thus allowing small increments in the output voltage without needing too large capacitors. The average output voltage in Figure 20(a) can be calculated by taking into account that in the steady state the whole current will ow through the resistor R thus vout = Ri0 tos f in (44)
associated to the integrator = RC, thus the response time is slow. This is a common trade-off in the design of these circuits. One strategy to reduce the ripple while keeping the dynamic response consists in ltering the output. As reported in (20), a passive RC lter exhibits low ripple at all frequencies, but it is slow. The lowest ripple at high frequencies is achieved by an active lter, which also provides much faster step response than the RC lter. Finally, is it possible to cascade a second active lter on the converters output to reduce the ripple at moderate frequencies. Another approach to convert a frequency into a voltage is implemented by the phase-locked-loop type FVCs (21). Figure 20(b) shows a phase-locked loop (PLL), which basically works as follows. The frequency and phase detector provides an output voltage which is proportional to the difference between the output and input signal phases. After ltering, this voltage is converted back into the output frequency by a V/F converter. The feedback loop locks the phase difference between both input and output signals. The lter output voltage is proportional to the deviations of the frequency of the input signal, thus the circuit also performs a frequency-to-voltage conversion. This converter operates over a wide frequency range of 1 or 2 or 3 decades and responds quickly to frequency changes while it does not have ant inherent ripple. Finally, pulse-integrating converters are open-loop versions of the charge balancing type converters in Section 5, thus their accuracy, linearity and stability are basically the same as those obtainable from their related VFCs. With respect to Figure 20(b), the high linearity provided by the VFC, which is not required for most PLL applications, is exploited here to get an ultra-linear FVC. APPLICATIONS This section describes some typical applications of the voltage-to-frequency converters, which we classify in ve major elds: telemetry and remote control, isolation, digital processing, communication and signal processing, and articial neural networks. Telemetry and remote control Data acquisition of remote sensors is often not possible without transforming the dc sensor output signal, because it is physically unreachable or because of the vulnerability of dc signals to be degraded when they have to travel long distances through noisy channels (1019). Once converted to frequency signals, they can be transmitted by wires, radio or ber optic links and processed digitally or converted back into a dc signal to drive an actuator. The latter is performed by a frequency-to-voltage converter , Isolation Some applications require galvanic isolation of the sensor and the data processing unit in order to prevent unreasonable current leakage, or to prevent damage due to high bias voltages which are often of several volts. Typical isolation interfaces use magnetic or optical couplers (10, 17) like those depicted in Figure 22.
where i0 tos fin is the average current injected by thi0tosne top current source in the integrator. A simpler passive integrator could be used instead of the active one in Figure 20(a), but the latter reduces the error caused by the nite output impedance of the current source (this can also be accomplished by means of cascode transistors (20)) and speeds up the converter response time. Since the current is injected in the integrator in packets of size i0 tOs , the output voltage has an ac ripple riding on it whose peak-to-peak value is ( vout ) p p = io tos C (45)
It is possible to reduce the ripple by increasing the capacitance C in Eq. (45), but it also will enlarge the time constant
17
Figure 20. Frequency-to-Voltage converters. (a) Pulse-integrating converters, where the charge packets are injected in the integrator at a rate determined by the input frequency. (b) Phase-locked-loop type converters, where the low-pass lter output voltage in a PLL is proportional to the deviations of the input signal frequency.
Figure 21. Telemetry with VFC. The voltage signal from a sensor is translated into a frequency signal to be transmitted through twisted pair, ber optic or radio links. On the other side, the signal can be converted back into a voltage to drive an analog display or an actuator, or can be processed digitally.
Figure 22. Isolation with VFCs. The frequency signal from a VFC can be transmitted through magnetic (a) or optic (b) couplers to implement galvanic isolation, thus preventing damage due to high bias voltages, or unreasonable current leakage.
Digital Processing The output frequency signal from a VFC can be easily translated into a digital code in order to be processed, stored or displayed by conventional digital circuits and systems (928). Actually, this procedure implements an A/D converter while transforming the dc input of the VFC into a digital word. Figure 23(a) shows the simplest way to carry out this A/D conversion, where a counter, which is only enabled for a given interval TG named gate time, counts the transitions of the input signal. Thus, after this known period, the counter stores the decimal value N = f TG , which encodes the input voltage in a binary number. Another ap-
proach is illustrated in Figure 23(b) and uses two counters, one to count the transitions of the input signal, and the other to act as a timer that generates an interruption to a microcomputer that reads the content of the rst counter and resets it. In both cases, if FS = f max f min is the full fmin frequency range, the resolution of the A/D converters will be n = log 2 (FS TG ) (46)
where n is the number of bits of the output digital word. Hence, the larger the gate time, the larger the resolution, which is also limited by the resolution of the VFC. Commercial VFCs provide resolutions as high as 13 bits. Note
18
Figure 23. Digital processing of the output frequency signal of a VFC. The transitions in a xed time interval TG (TG ) are counted and the average frequency is computed.
that the input frequency must remain constant during the gate time for proper conversion, otherwise an average of the input frequency is computed. In addition, since the input signal and the clock are not synchronized, one cycle of the frequency signal can be lost or counted if the transition is just after or before the edge of the clock signal, which causes an error of 1 in the result stored in the counter. Communication and Signal Processing Since voltage-to-frequency converters are actually VCOs, they can be used in phase-locked loops in applications such as jitter reduction, skew suppression, frequency synthesis, clock recovery or FM modulation (23). However, special care must be taken in order to reduce the jitter and enhance the spectral purity of the output signal, like the use of differential mode in signal and control paths and guard rings to isolate the noise sources. High linearity is not a key issue in PLL applications, except that described in the Section 6, where nonlinearity degrades the loop stability but can be of several tens of percent. Higher linearity is required for FM modulation, where distortion in the detected signal must be below 1%. Figure 20(b) depicts a PLL that uses a voltage-to-frequency converter as a voltage controlled oscillator.
ACKNOWLEDGMENT This work was supported in part by the spanish government under project TEC2006-12376-C02-01/MIC. BIBLIOGRAPHY
1. A. A. Abidiand R. G. Meyer, Noise in relaxation oscillators, IEEE Journal of Solid-State Circuits, Vol. CS-18, pp. 794802, Dec. 1983. 2. AD537 Integrated circuits voltage-to-frequency converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www.analog.com/en/ 3. AD650 Voltage-to-frequency and frequency-to-voltage converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www. analog. com/en/ 4. ADVF32 Monolithic non-synchronous Voltage-to-frequency and frequency-to-voltage converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, http:www. analog. com/en/ 5. AD652 Monolithic synchronous voltage-to-frequency converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www. analog. com/en/ 6. AD7740 3V/5V Low Power, Synchronous Voltage-to-Frequency Converter, Voltage-to-frequency converter in Analog to Digital Converters in Product Catalogue, www. analog. com/en/ 7. AD7741/AD7742 Single and Multichannel, Synchronuos Voltage-to Frequency Converters. Voltage-to-frequency converter in Analog to Digital Converters in Product Cata- logue, www.analog.com/en/ 8. M. Banu, MOS oscillators with multi-decade tuning range and gigahertz maximum speed, IEEE Journal of Solid-State Circuits, Vol. CS-23, pp. 474479, April 1988. 9. L. N. Beard, Use of a voltage-to-frequency converter to measure the integral JEdt, American Journal of Physics, Vol. 57, Iss. 5, pp. 457476, 1989. 10. R. Carrand J. Yang, Current-to-frequency converter for electron multipliers, Review of Scientic Instruments, Vol. 60, Iss. 7, pp. 23212323, 1989. 11. L. M. DeVito, A versatile clock recovery architecture and monolithic implementation, Monolithic phase-locked loops and clock recovery circuits, Editor B. Razavi, IEEE Press, New York 1996.
i In practice, the excitations may not strictly be dc, but ac signals whose frequency is smaller, commonly much smaller, than that of the signal generated by the circuit. ii The dynamic range can be expressed in a number of equivalent bits b by using the following expression DR = 6.02 b + 1.76. Thus, 60dB dynamic range corresponds to an equivalent resolution of 9.7bits. iii In the electronic implementations of Figure 2(b) and (c), the negative slope is caused by a negative resistance. This negative resistance injects energy into the two other circuit components, namely the inductor and the capacitor. Because these latter elements do not dissipate energy, but simply store it, the total energy becomes increased and this manifests itself as an increase of the signal amplitude. On the other hand, the positive slope is caused by a positive resistance which dissipates part of the energy stored in the energy-storage part of the circuit. At the equilibrium there is balance between the energy injected and the energy dissipated, the circuit energy remains unchanged and the movement is due to a continuous interchange of this energy between the inductor and the capacitor.
iv This device is obsolete, but its data are still a good example of the performance obtained with this implementation.
Voltage-to-Frequency Converters 12. S. W. Glowacki, A current-to-frequency converter, Nuclear Instruments and Methods, Iss. 186, pp. 549551, NorthHolland Publishing Company 1981. 13. P. R. Grayand R. G. Meyer, Analysis and Design of Analog Integrated Circuits, 2nd Edition. Ed. John Wiley & Sons. Singapore 1984. 14. A. B. Grebene, Bipolar and MOS analog integrated circuit design, John Wiley & Sons, New York 1984. 15. K. W. Kobayashi et al., A novel self-oscillating HEMT-HBT cascode VCO-Mixer using an active tunable inductor, IEEE Journal of Solid-State Circuits, Vol.33, No. 6, June 1998. 16. LM231A/LM231, LM331A/LM331 Precision voltage-tofrequency converters, National Semiconductor Other Analog Product Catalogue www.national.com/catalog/ 17. J-F Loude, Current-to-frequency isolation converter, Journal of Physics E-Scientic Instruments, Vol.18,Iss. 10, pp. 824826, 1985. 18. T. D. Moore, S. Turner, C.C. Speake, A capacitance displacement transducer with large dynamic-range, good linearity and frequency read-out, Measurement Science & Technology, Vol. 4,Iss. 12,pp. 15191521, 1993. 19. R. Pallas-Arenyand ` J. G. Webster, Sensors and Signal Conditioning, John Wiley and Sons, New York 1991. 20. R. A. Pease, Application Note C: V/F converter ICs handle frequency-to-voltage needs,National Semiconductor Application Notes, http://www.national.com/apnotes/Voltage-toFrequencyConverters v1. html 21. R. A. Pease, Application Note 210: New Phase-LockedLoops Have Advantages as Frequency to Voltage Converters (and more),National Semiconductor Application Notes, http://www. national. com/apnotes/Voltage-toFrequencyConverters v1. html 22. B. Razavi, Analysis, Modeling and Simulation of Phase Noise in Monolithic Voltage-Controlled Oscillators, Proc. CICC, pp. 323326, May 1995. 23. B. Razavi, Design of Monolithic phase-locked loops and clock recovery circuits- a tutorial, Monolithic phase-locked loops and clock recovery circuits, Editor B. Razavi, IEEE Press, New York 1996.
19
24. M. Soyuerand R. G. Meyer, High-frequency phase-locked loops in monolithic bipolar technology, IEEE Journal of Solid-State Circuits, Vol. CS-24, pp. 787795, June 1989. 25. VFC32 Voltage-to-frequency and frequency-to-voltage converter, Texas Instruments www.ti.com. 26. VFC110 High-frequency voltage-to-frequency converter, Texas Instruments, www.ti.com. 27. VFC320 Voltage-to-frequency and frequency-to-voltage converter, Texas Instruments www.ti.com. 28. W. S. Wilburn, A high-accuracy beam current integrator for fast spin-ip experiments, Nuclear Instruments and Methods in Physics Research Section A-Accelerators Spectrometers Detectors and Associated Equipment, Vol.394, Iss. 12, pp. 257260, 1997.
1 FERNANDO VIDAL-VERDU 1 RAFAEL NAVAS-GONZALEZ MANUEL DELGADO-RESTITUTO2 2,3 ANGEL RODRIGUEZVAZQUEZ
1 Dpto.
de Electrnica, Universidad de Mlaga, Campus de Teatinos, Complejo Tecnol ogico, Malaga, Spain 2 Institute of Microelectronics of Seville, CNM-CSIC, Edicio CICA-CNM, C/ Tara sn, Seville, Spain 3 School of Engineering, University of Seville, Seville, Spain

John G. Webster Editor 04.automatic Control 4 1999

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

John G. Webster Editor 04.automatic Control 4 1999

Uploaded by

Copyright:

Available Formats

04 Automatic Control

file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20...RONICS%20ENGINEERING/04.Automatic%20Control.htm (1 of 3)18.06.2008 19:59:52

file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20...RONICS%20ENGINEERING/04.Automatic%20Control.htm (2 of 3)18.06.2008 19:59:52

file:///N|/000000/0WILEY%20ENCYCLOPEDIA%20OF%20...RONICS%20ENGINEERING/04.Automatic%20Control.htm (3 of 3)18.06.2008 19:59:52

Home / Engineering / Electrical and Electronics Engineering

Wiley Encyclopedia of Electrical and Electronics Engineering

Browse this title

Search this title Enter words or phrases

Advanced Product Search Search All Content Acronym Finder

then also the feedback system x (t ) = Ax(t ) B [Cx(t ), t ] (4)

w(t )[v(t ) w(t )] dt

for all T and all y = Fu

ABSTRACT DATA TYPES

Home / Engineering / Electrical and Electronics Engineering

Wiley Encyclopedia of Electrical and Electronics Engineering

Browse this title

Search this title Enter words or phrases

Advanced Product Search Search All Content Acronym Finder

Fig. 1. Controller structure with adjustable controller gains.

Fig. 2. Gain scheduling.

Fig. 3. Indirect adaptive control.

Fig. 4. Direct adaptive control.

Fig. 5. Model reference control.

On-Line Parameter Estimation

The constants ai , bi are the plant parameters. We can express yn as

In a similar way we can express the plant dynamics as follows:

ADAPTIVE CONTROL based on (t) is given by

and the estimation error is given by

> 0 is a constant matrix and Pc = PT c > 0 satises the algebraic equations

= to make V 0 is now obvious, namely, for The choice for

ADAPTIVE CONTROL which gives

> 0 is a scaling matrix that we refer to as the adaptive gain. We have

which yields the so-called nonrecursive least-squares algorithm

We can show that P, satisfy the differential equations

where ns is designed to satisfy

T Now T T = T T + T T = T , and therefore,

where Gp (s) is expressed in the form

which is expressed in the same form as Eq. (32), that is,

Reference Model Assumptions.

ADAPTIVE CONTROL where

the matching equation Gc (s) = W m (s) becomes

and Rm be as specied in Eq. (34). Then

whose state-space realization is given by

Adaptive Pole Placement Control

whose characteristic equation

has order 2n + q 1. The objective now is to choose P, L such that

where Sl is the Sylvester matrix of Qm Rp , Zp of dimension 2(n + q) 2(n + q),

The tracking error e1 = yp ym is given by

, P pointwise in time, or the algebraic equation for L

Adaptive Control of Nonlinear Systems

[kn , kn 1 , . . ., k1 ]T . Then the error system in Eq. (59) becomes

Step 2. Using the above denitions of 1 , 2 , we have that

ADAPTIVE CONTROL Substituting Eq. (70) into Eq. (69) yields

Under the above control law, the closed-loop dynamics become

Parameter update law:

By differentiating V with respect to time, we obtain that

ADAPTIVE CONTROL where: One has

s is hysteresis-switching variable dened as follows:

lim t s(), where t.

is a symmetric positive denite design matrix and PC is dened as follows (15):

PETROS A. IOANNOU ELIAS B. KOSMATOPOULOS University of Southern California

Home / Engineering / Electrical and Electronics Engineering

Wiley Encyclopedia of Electrical and Electronics Engineering