598

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 4, APRIL 2004

TABLE I STEADY STATE PERFORMANCE METRICS FOR  = 0:2

cost criterion, because of which a significant contribution to the value function comes from the initial stages as costs accrued from later stages get discounted heavily. Next, in Table I, we use the rates obtained after convergence of our algorithm for various values of T , for computing certain steady state performance measures of interest. We use these rates to compute the steady-state mean queue length E [q ], average rate of the controlled c source 3 , long run average cost J 3 , variance of queue length V ar (q) and the stationary probability (P 3 ) that the queue length is in the region = f(B=2) 0 2; . . . ; (B=2) + 2g. The previous metrics are obtained using another simulation that runs for 30 000 T -instants using standard Monte-Carlo estimates. We computed these metrics in order to get an idea of the steady-state system performance (in addition) using our algorithm. From Table I, it is clear that the steady state performance degrades as T increases in both cases. For instance, V ar(q ) and J 3 increase while P 3 decreases. Note that the average rate for all cases when u = 0:2 is close to 1.8 which ensures almost complete bandwidth utilization in all cases. V. CONCLUSION In this note, we developed a two timescale gradient search based actorcritic algorithm for solving infinite horizon MDPs with finite-state and compact action spaces under the discounted cost criterion. The algorithm was theoretically shown to converge to a locally optimal policy. We showed numerical experiments using a continuous time queueing model for rate-based flow control. The algorithm is found to be computationally efficient with the computational effort growing only linearly with the number of states. ACKNOWLEDGMENT The first author would like to thank Prof. V. S. Borkar for many helpful comments on an earlier version of this manuscript which led, in particular, to the relaxation of a technical assumption. REFERENCES
[1] A. Barto, R. Sutton, and C. Anderson, Neuron-like elements that can solve difficult learning control problems, IEEE Trans. Syst., Man, Cybern., vol. 13, pp. 835846, Dec. 1983. [2] D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996. [3] O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Control Optim., vol. 36, pp. 12931314, 1998.

[4] H.-T Fang, H.-F Chen, and X.-R. Cao, Recursive approaches for single sample path based Markov reward processes, Asian J. Control, vol. 3, no. 1, pp. 2126, 2001. [5] S. B. Gelfand and S. K. Mitter, Recursive stochastic algorithms for global optimization in , SIAM J. Control Optim., vol. 29, no. 5, pp. 9991018, 1991. [6] M. W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks, vol. 2, pp. 331349, 1989. [7] V. R. Konda and V. S. Borkar, Actor-critic like learning algorithms for Markov decision processes, SIAM J. Control Optim., vol. 38, no. 1, pp. 94123, 1999. [8] V. R. Konda and J. N. Tsitsiklis, Actor-critic algorithms, SIAM J. Control Optim., vol. 42, no. 4, pp. 11431166, 2003. [9] H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer-Verlag, 1978. [10] P. Marbach and J. N. Tsitsiklis, Simulation-based optimization of Markov reward processes, IEEE Trans. Automat. Contr., vol. 46, pp. 191209, Feb. 2001. [11] J. Neveu, Discrete Parameter Martingales. Amsterdam, The Netherlands: North Holland, 1975. [12] R. Permantle, Nonconvergence to unstable points in urn models and stochastic approximations, Ann. Probab., vol. 18, pp. 698712, 1990. [13] J. C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Automat. Contr., vol. 37, pp. 332341, Mar. 1992. [14] R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. [15] J. N. Tsitsiklis, Asynchronous stochastic approximation and Q-learning, Mach. Learn., vol. 16, pp. 185202, 1994. [16] C. Watkins and P. Dayan, Q-learning, Mach. Learn., vol. 8, pp. 279292, 1992.

On the Solvability of Extended Riccati Equations

Nikita E. Barabanov and Romeo Ortega

AbstractThe KalmanYakubovichPopov lemma, which gives necessary and sufficient conditions for solvability of matrix LureRiccati equations, is a milestone in modern control theory. There are, however, important and general extensions of this lemma that have not been studied yet. Starting with the absolute stability theory with semidefinite frequency domain function, we generalize here this lemma to the sign indefinite casea control research that is motivated by new problems on passivity and theory. Index TermsRicatti equations, stability of nonlinear systems.

I. INTRODUCTION The note is devoted to the problem of solvability with respect to matrices H , h of following matrix equations arising in control theory:

HA + A3 H + Q = h3 0h + G 3 HB1 + L1 = h3 0 3 HB2 + L2 = 0

(1)

Manuscript received September 7, 2001; revised March 12, 2003 and November 4, 2003. Recommended by Associate Editor C. Oara. N. E. Barabanov is with the Department of Mathematics, North Dakota State University, Fargo, ND 58105 USA (e-mail: nikita.barabanov@ndsu.nodak.edu). R. Ortega is with the Supelec, Laboratoire des Signaux et Systmes, 91192 Gif-sur-Yvette, France (e-mail: ortega@lss.supelec.fr). Digital Object Identifier 10.1109/TAC.2004.825628

0018-9286/04$20.00 2004 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 4, APRIL 2004

599

where A, B1 , B2 , 01 , L1 , and L2 are given matrices of dimensions n 2 n, n 2 m1 , n 2 m2 , m1 2 m1 , m1 2 n, and m2 2 n, respectively. Matrices 0 and Q are Hermitian, matrix 0 is nonsingular. Hermitian matrices H and G of dimension n 2 n, and matrix h of dimension m1 2 n are to be found. Without loss of generality we assume that matrix col(B2 ; L3 ) has full-column rank. 2 The rank and signature of the matrix G is defined as follows. Consider a vector identity which is equivalent to (1)

absolute stability of systems in sector [0; 1) may be formulated as a problem of solvability of

HA + A3 H = 0 gg3 HB + C = 0

(5)

2x3 H (Ax + B1 u1 + B2 u2 ) 3 + x3 Qx + 2x3 (L1 u1 + L2 u2 ) + u1 0u1 3 = (u1 + 001 (B1 H + L1 )x)3 0 01 (B 3H + L1 )x) + x3 Gx 2 (u 1 + 0 1

with respect to Hermitian matrix H and vector g . It is essential here that vector g may be nonzero. In case of G = 0 we have g = 0 and if A is a Hurwitz matrix and C 6= 0 such a solution does not exist. However, in case  (i! ) = RefC (A 0 i!I )01 Bg > 0 for almost all ! there exists a solution to (5), and system is absolutely stable. II. MATRIX PENCIL

(2)

for all x 2 Rn , u1 2 Rm , u2 2 Rm . Extend this identity to the complex vector space. That is, for any real ! such that det(A 0 i!I ) 6= 0 denote x! (u1 ; u2 ) = (i!I 0 A)01 (B1 u1 + B2 u2 ), and define an (m1 + m2 ) 2 (m1 + m2 )-Hermitian function (i!) such that

The matrix pencil

R 0 J0
plays a crucial role in studying (1) (see [8] and [9]), where

(6)

u1 3 u (i!) 1 u2 u2

 x! (u1 ; u2 )3 Qx! (u1 ; u2 )

+ 2Re(x! (u1 ; u2 )3 (L1 u1 + L2 u2 )) 3 + u1 0u1 :

3 3 0Q 0A3 0L1 0L2 R= 3 L1 B1 0 0

J0 =

B1

B2

If (1) has a solution, then

3 L2 B2 I 0 0 0 0 I 0 0

u1 3 u (i!) 1 u2 u2 3  (u1 + 001 (B1 H + L1 )x! (u1 ; u2 ))3 0 3 2 (u1 + 001 (B1 H + L1)x! (u1 ; u2 )) 3 Gx! (u1 ; u2 ): + x ! (u 1 ; u 2 )

Introduce the 2n 2 2n-matrix

0 0 0 0 0 0 0 0
J= I

(7)

0I

(3)

Denote by n+ (! ), n0 (! ), n0 (! ) the numbers of positive, zero and negative eigenvalues of matrix  (i! ). The function  is clearly rational. Hence, its signature sig ( (i! )) = (n+ (! ); n0 (! ); n0 (! )) is constant for sufficiently big numbers ! . Denote by n+ (0), n0 (0) the numbers of positive and negative eigenvalues of matrix 0. Equation (3) at infinity implies sig ( (i1)) = (n+ (0); m2 ; n0 (0)) because x! (u1 ; u2 ) ! 0 as ! ! 1. Denote n2 = n+ (! ) 0 n+ (0), n1 = n0 (!) 0 n0 (0) for sufficiently big ! . From (3), it follows that minimal rank of matrix G is equal to n1 + n2 , and in this case G has exactly n2 positive, n1 negative and n 0 n1 0 n2 zero eigenvalues. Therefore, we claim

(2n + m1 + m2 ) 2 J1 = diagfJ; I; I g, J2 = J1 J0 . Then, the pencil J1 R is Hermitian: Both matrices J1 R and iJ2 are Hermitian. Assume (1) has a solution H , h. Under these conditions I I 0 H H G R 0 h = 0 (A 0 B 1 h ) 0 0 :
and the

0 (2n + m1 + m2 )-matrices

0 iJ2

(8)

sig(G) = (n1 ; n 0 n1 0 n2 ; n2 ):

(4)

Problem Statement: Find out conditions for the existence of a solution to system (1) with restriction (4). Describe all such solutions. Problem of solvability of system (1) arises in the theories of absolute stability, quadratic optimization, and H1 control, see [3] for some motivational problems and a list of references. For the case of feedback nonlinear systems satisfying integral quadratic constraints (IQCs) the set of coefficients in system (1) may be arbitrary. The problem can not be reduced in general to the problem of spectral factorization of matrix  (see [5] and [6]). Indeed, matrix  may be equal to zero identically (for example, for Popov absolute stability criterion), while (1) may (or may not) have a solution. Equation (1) were studied recently in a number of papers and monographs (see [6][9]), but with essential additional condition G = 0. In case of n1 + n2 > 0, which usually occur when m2 > 0, such equations with G = 0 are completely different from the equations arising in absolute stability and H1 control theories, which are the main application of this algebraic problem. For example, circle criterion for

Definition 1: An 2 k-matrix Z is called (R; J0 )-invariant with defect p if J0 RZ = RZ and rank(J0 Z; RZ ) 0 rank(J0 Z )) = p. Definition 2: A subspace P of Euclidean space R2n+m is called (R; J0 )-invariant with defect p if there exists a basis fz1 ; . . . ; zk g of this subspace such that matrix Z = fz1 ; . . . ; zk g is (R; J0 )-invariant with defect p. Matrices and subspaces, which are (R; J0 )-invariant with defect 0 are called (R; J0 )-invariant. A subspace P is called J0 -orthogonal if u3 J0 v = 0 for all vectors u, v 2 P . A matrix Z is J0 -orthogonal if Z 3 J0 Z = 0. To clarify the main result of this note we consider first the wellstudied case when there is no matrix G in (1). A. Case m2

0 (2n + m1 + m2 )

to matrix algebraic Riccati equation

=0 Then, n1 = n2 = 0, there is no matrix G in (1), and (1) is equivalent

3 3 HA + A3 H + Q 0 (HB1 + L1 )001 (B1 H + L1 ) = 0:
(9)

In this case there exists a well-known [8], [9] correspondance between n-dimensional (R; J0 )-invariant J2 -orthogonal subspaces and solutions of (1). Namely, for any solution H , h of (1) the space spanned

600

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 4, APRIL 2004

by (col(I; H; 0h)), that we denote in the sequel by Lin(1), is n-dimensional (R; J0 )-invariant and J2 -orthogonal. Vice versa, if a space P is n-dimensional (R; J0 )-invariant and J2 -orthogonal, and a matrix Z of basis vectors z1 ; . . . ; zn of P takes the form Z = col(X; 9; Y ), where X , 9 are n 2 n-matrices and X is invertible, then matrices H = 9X 01 , h = 0Y X 01 are a solution to (1). To find out n-dimensional (R; J0 )-invariant and J2 -orthogonal subspaces one should study Jordan blocks of the canonical representation of pencil (6). Consider J -Hamiltonian matrix associated with Riccati equation (9)
R0

0 0 0

A B1 001 L1 Q L1 001 L1

0B1 001 B1 0(A 0 B1 001 L1 )3

Relation between Lagrangian subspaces of matrix R0 (i.e., n-dimenstional R0 -invariant J -orthogonal subspaces of Euclidean space R2n ) and solutions to (9) is well-known [5], [8]. If H is a solution to (9), then Lin(col(I; H )) is the Lagrangian subspace of matrix R0 . If Z is matrix of basis vectors of a Lagrangian subspace of matrix R0 , and Z = col(X; 9) with square matrices X , 9, then provided that X is nonsingular, matrix H = 9X 01 is a solution to (9). All Lagrangian subspaces may be found using proper division of Jordan blocks of matrix R0 (see [8] for comprehensive details). There is a one-to-one correspondance between Lagrangian subspaces of matrix R0 and n-dimensional (R; J0 )-invariant J2 -orthogonal subspaces: if Lin((X; 9)) 3 is a Lagrangian subspace of matrix R0 , then Lin((X; 9; 001 (B1 H + L1 ))) is an n-dimensional (R; J0 )-invariant J2 -orthogonal subspace, and vice versa. Such subspaces correspond to the same division of Jordan blocks of pencil (6) as for the case of Lagrangian subspaces of matrix R0 . Now, consider the general case. B. General Case We assume that number m2 may be nonzero. Consider the canonical representation of pencil (6) [5], [10]. Denote by Sm m 2 m-matrix with entries i;m+10j ; let Tm denote the m 2 m-matrix with entries i;m+20j , where i;j is the Kronecker function. There exists a nonsingular (2n + m1 + m2 ) 2 (2n + m1 + m2 )-matrix V with complex coefficients such that
V J 1 (R

where M2p is simply S2p with the entry (pj +1; pj +1) position changed from 1 to 0, and pj is nonnegative integer. Blocks of Type 1 correspond to generalized pure imaginary eigenvalues of pencil (6). Blocks of Type 2 correspond to generalized eigenvalues of pencil (6), which have nonzero real part. Blocks of Type 3 correspond to generalized eigenvalues of pencil (6) at infinity. At last, blocks of Type 4 are called Kronecker blocks. There always exist m1 scalar blocks Bj () = f1 g of Type 3. We j assume that they are the first m1 blocks of Type 3 in representation (10). Denote by nJ , n1 + m1 and nK the numbers of blocks of (Type 1 + Type 2), Type 3, and Type 4, respectively. Denote by NJ , N1 , NK the sums of dimensions of blocks of corresponding Types. Then m2 = n1 + nK , NJ + N1 + NK = 2n + m1 + m2 , and the number of blocks of Type 3 of dimension bigger than one is equal to n1 + n2 [5]. In case m2 = 0 there are no Kronecker blocks, and there are only m1 scalar blocks of Type 3. Hence, NJ = 2n, and we can find out n-dimensional (R; J0 )-invariant J2 -orthogonal subspaces using blocks of Types 1 and 2 in a similar way as we use Jordan blocks for the case of Riccati equation (9). If m2 > 0 and there is a block of Type 3 or 4 of dimension bigger than one, than N1 + NK > m1 + m2 , and hence NJ < 2n. The biggest dimension of a subspace, which is J2 -orthogonal and which contains only generalized root eigenvectors of pencil (6) is equal to NJ =2 < n. Therefore, there is no n-dimensional (R; J0 )-invariant J2 -orthogonal subspaces composed, as in the case of m2 = 0, only from root vectors of generalized eigenspaces of pencil (6). What kind of vectors should stand instead of these missing vectors? In this note, it is shown that such vectors should be taken from first halves of Kronecker blocks and blocks corresponding to generalized eigenvalues at infinity. First it is necessary to study some properties of representation (10). III. PRELIMINARY RESULTS Lemma 1: The dimensions of blocks Sp 0 Tp , corresponding to generalized eigenvalues of pencil (6) at infinity, are odd. Proof: Consider any such a block, and denote by Dj the submatrix of matrix V 01 corresponding to this block
J 1 (R

0 iJ0)V = diagfB ()g

j

(10)

0 iJ0) =  D3 (S 0 T )D :
j j p p j

where each block Bj has one of the following canonical forms. Type 1:
B j ( )

=  [(a
f j

0  )S + T ]
p p j

is an pj 2 pj -block, with f j integer. Type 2:

B j ( ) =

= 61, a

is real, and pj is positive

(a
j

0  )S + T
p j

(a
p j

0  )S + T
0
p j

is an 2pj Type 3:

2 2p -block, with a 6= a , and p

B j (  ) =  j [S p

is positive integer.

3 Then, J1 J0 = ij Dj Tp Dj . Left hand side of this equation is real. Hence, right-hand side is also real. But if pj is even, then for the (pj =2 + 1)th column d of matrix Dj we have ij d3 d is real, which contradicts nonsingularity of matrix V 01 . Denote by Vj (Uj ), j = 1; . . . ; n1 + nK , column blocks of matrix V (respectively, (V 3 J1 )01 ) corresponding to blocks Bj () in (10) of Types 3 and 4. Denote by (2qj + 1), j = 1; . . . ; n1 + nK , the numbers of columns of matrices Vj . Recall that q1 = 1 1 1 = qm = 0. Denote by vj;0 ; . . . ; vj;2q (uj;0 ; . . . ; uj;2q ) columns of matrices Vj (respectively, Uj ). Recall that signature of matrix G in the solution of (1) is equal to (n 1 ; n 0 n 1 0 n 2 ; n 2 ) . Lemma 2: Assume that H , h, G is a solution to the (1). Then matrix
Z

= col(I; H; 0h; 0)

0 T ]
p j

is an pj 2 pj -block, with 1 j Type 4:

= 61, and p
p

is positive integer.

Bj () = [M2p

0 T2 ]

is (R; J0 )-invariant with defect n1 + n2 . Proof: Follows from definitions and (8). Denote by PJ the union of generalized root spaces corresponding to finite eigenvalues of pencil (6). Denote by V0 a matrix which columns are vectors vj;k for all j = 1; . . . ; n1 + nK such that qj > 0, and all k = 1 ; . . . ; qj .

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 4, APRIL 2004

601

Lemma 3: Assume that H , h, G is a solution to the (1). Then, the following conditions hold. 1) Number NJ is even; there exists an NJ =2-dimensional (R; J0 )-invariant J2 -orthogonal subspace PJ 0 of PJ , such that J0 PJ 0  Lin(col(I; H; 0; 0)). 2) We have J0 V0  Lin(col(I; H; 0; 0)). 3) The numbers of blocks of Type 3 such that qj > 0 and j = 01 (j = 1) is equal to n1 (respectively, n2 ). Proof: Equation (10) imply (R 0 iJ0 )Vj = Uj Bj (). For blocks of Type 3, it means that

Rv J0 v J0 v Rv 2
Therefore

1 0 = j uj;2q 1 j;0 = 0; Rvj;1 = j uj;2q 01 1 j;1 = (0i)j uj;2q ; . . . 1 uj;0 ; J0 vj;2q = (0i)1 uj;1 : j; q = j j
j;

J0 v 0 = 0; Rv 0 = (0i)J0 v 1 ; . . . Rv 2 01 = (0i)J0 v 2 :
j; j; j; j; q j; q

For blocks of Type 4, we have similar relations

J0 v 0 = 0; Rv 0 = (0i)J0 v 1 ; . . . Rv = (0i)J0 v ; Rv = 0:
j; j; j; j;q j;q j;q

Denote Z = col(I; H; 0h; 0), F with qj > 0, we have

j; j;

= col(I; H; 0; 0). For blocks B ()

j j;

Theorem 1: For any n-dimensional (R; J0 )-invariant J2 -orthogonal matrix W with defect n1 + n2 there exists a NJ =2-dimensional (R; J0 )-invariant J2 -orthogonal subspace PJ 0 of PJ such that J0 Lin(W) = J0 (Lin(V0 ) + PJ 0 ). . Proof: Similar to proof of Lemma 3 and, hence, omitted. The set of all NJ =2-dimensional (R; J0 )-invariant J2 -orthogonal subspace PJ 0 of PJ may be found, counted and described like Lagrangian subspaces of Hamiltonian matrix R0 [8], [9]. The only difference concerns matrix V0 , which does not depend on these subspaces. Theorem 2: Let matrix Y consists of basic vectors of an NJ =2-dimensional (R; J0 )-invariant J2 -orthogonal subspace of PJ . Split (2n + m1 + m2 ) 2 n-matrix (Y; V0 ) = K as follows: K = col(X; 9; Y1 ; Y2 ), where X and 9 are n 2 n-matrices. Assume that matrix X is nonsingular. Then, matrices H = 9X 01 , h = 0Y1 X 01 are a solution to the (1) with some matrix G, which has signature (n1 ; n 0 n1 0 n2 ; n2 ). Proof: Matrix K is J0 -orthogonal. Hence, matrix H is Hermitian. There exists a Hermitian n 2 n-matrices G0 with signature (n1 ; n 0 n1 0 n2 ; n2 ) such that (0; 0; I; I)RK = 0 and K 3 J2 RK = G0 . From the definition of matrices R, K it 3 is easy to see that HA + A3 H + Q 0 h3 0h = X 01 G0 X 01 , 3 + h3 0 = 0, HB2 + L2 = 0. 3 . HB1 + L1 Theorem 3: Assume that H , h, G is a solution to (1) such that rank(G)  n1 + n2 . Then, there exists a NJ =2-dimensional (R; J0 )-invariant J2 -orthogonal subspace PJ 0 of PJ such that Lin(col(I; H; 0; 0) = J0 (Lin(V0 ) + PJ 0 ), and signature of G is equal to (n1 ; n 0 n1 0 n2 ; n2 ). Proof: Follows from Lemma 3. V. CONCLUSION The problem of solvability of special matrix equations arising in the theories of absolute stability and H1 control has been formulated and solved. The technique of matrix pencil occurs to be useful to solve the problem. It is shown that any solution corresponds to some maximal invariant orthogonal subspace of root space of a particular matrix pencil corresponding to finite generalized eigenvalues. The description of such subspaces is similar to the description of all Lagrangian subspaces of a Hamiltonian matrix. The additional matrix involved in getting the solutions is the same for all such subspaces and is determined by Kronecker blocks and blocks corresponding to eigenvalues at infinity. To put our contributions in perspective it is interesting to compare them with the important results of [5]. First of all, while [5] is primarily devoted to the spectral factorization problem, in our note we are interested in the solution of RiccatiLure equations which, in the authors opinion, is more general than a spectral factorization problem. For example, in case of circle criterion with the class of nonlinearities described by strict sector inequalities 0 < f(s; t)=s <  for all s, and matrix of linear part with pure imaginary eigenvalues the frequency function to be factorized (function 8 in [5]) can be equal to zero, while the problem of solvability of Riccati equations may, or may not, have a solution. At a more technical level, Lemma 1 of our note states that there are no blocks of even dimension, which correspond to generalized eigenvalues at infinity. In fact the statement is not true if we consider general complex matrix pencils. In this note, the pencil is real, and the statement is true. However, in [5] no difference is made between real and complex cases. Hence, there is no such a statement in it. REFERENCES
[1] N. E. Barabanov, G. A. Leonov, A. H. Gelig, A. S. Matveev, V. B. Smirnova, and A. L. Fradkov, Frequency domain theorem (Yakubovich-Kalman lemma) in the control theory, Automat. Remote Control, no. 10, pp. 340, 1996.

(Rv 0 )3 J2 Z = (Rv 0 )3 J1 Z = v3 0 J1 RZ = 0:
3 Hence, vj;1 col(0H; I; 0; 0) = 0 and J0 vj;1 2 Lin(J0 Z). Consider a chain vj;1 ; vj;2 ; . . . ; vj;s such that J0 vj;k 2 Lin(J0 Z) for all k = 1; . . . ; sj , but not for k = sj + 1. Vectors J0 vj;k are linearly independent and belong to J2 -self-orthogonal subspace Lin(F ). Hence, sj  qj . Each chain ends up with vector vj;s such that J0 Rvj;s = Rvj;s . The number of such chains is equal to s + n1 + n2 , and vectors ffJ0 vj;k gk=1 gn=m n are linearly independent. j +1 Hence, there is no vector v such that J0 v is linearly independent s n +n , J0 v 2 Lin(F ), J0 Rv = Rv , and with ffJ0 vj;k gk=1 gj =m +1 Rv 62 Lin(F ). The space PJ 0 = PJ \ Lin(col(I; H; 0h; 0)) occurs to be (R; J0 )-invariant and J2 -orthogonal. Hence, its dimension can not exceed [NJ =2]. If sj < qj for some j , then the number of vectors s n + ffJ0 vj;k gk=1 gj=m n+1 is less than [(N1 + NK 0 m1 0 m2 )=2]. Due to the property stated before, there is no other vectors in the space Lin(F ). Then, the dimension of the space Lin(F ) is less than [NJ =2] + [(N1 + NK 0 m1 0 m2 )=2]  n. The contradiction proves that NJ is even, dim(PJ 0 ) = NJ =2, and sj = qj for all j . Consider a (2n + m1 + m2 ) 2 n-matrix W , which columns are s n + basic vectors of space PJ 0 and vectors ffvj;k gk=1 gj =m n . Then, +1 3 J2 RW = diagfl g, where l 2 f01; 0; 1g rank(W ) = n, and W and the number of ones (minus ones) is equal to the number r1 of positive (respectively, number r2 of negative) values 1 corresponding j to blocks of Type 3 with qj > 0. On the other hand signature of W 3 J2 RW coincide with signature of F 3 J2 Rcol(I; H; 0h; 0), which is equal to signature of G. Hence, r1 = n1 and r2 = n2 .
IV. MAIN RESULT Consider again a subspace PJ of generalized root vectors corresponding to finite generalized eigenvalues of pencil (6), and matrix V0 which consists of first halves of matrices, corresponding to blocks of Types 3 and 4 with dimension bigger than 1.

602

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 49, NO. 4, APRIL 2004

[2] N. E. Barabanov, The state space extension method in the theory of absolute stability, IEEE Trans. Automat. Contr., vol. 45, pp. 23352338, Dec. 2000. [3] N. Barabanov and R. Ortega, Matrix pencils and extended algebraic Riccati equations, Eur. J. Control, vol. 8, no. 3, pp. 251264, 2002. [4] D. J. Clements and K. Glover, Spectral factorization via Hermitian pencils, Linear Alg. Applicat., vol. 122124, pp. 797846, 1989. [5] D. J. Clements, A state space approach to indefinite spectral factorization, SIAM J. Matrix Anal. Applicat., vol. 21, no. 3, pp. 743767, 2000. [6] V. Ionescu and C. Oara, Generalized continuous-time Riccati theory, Linear Alg. Applicat., vol. 232, pp. 111130, 1996. [7] W. -W. Lin, V. Mehrmann, and H. Xu, Canonical forms for hamiltonian and symplectic matrices and pencils, Linear Alg. Applicat., vol. 302303, pp. 469533, 1999. [8] P. Lancaster and L. Rodman, Algebraic Riccati Equations. Oxford, U.K.: Oxford Univ. Press, 1995. [9] G. Freiling, V. Mehrmann, and H. Xu, Existence, uniqueness, and parametrization of lagrangian invariant subspaces, SIAM J. Matrix Anal. Applicat., vol. 23, no. 4, pp. 10451069, 2002. [10] R. C. Tompson, The characteristic polynomial of a principal subpencil of an Hermitian matrix pencil, Linear Alg. Applicat., vol. 14, pp. 135177, 1976.

An Analysis and Design Method for Uncertain Variable Structure Systems With Bounded Controllers
Han Ho Choi
AbstractWe present a method for estimating the asymptotic stability region(ASR) of uncertain variable structure systems with bounded switching feedback controllers. Using linear matrix inequalities(LMIs) we estimate the ASR and we show the exponential stability of the closed-loop control system in the estimated ASR. We also give a simple LMI-based method for designing switching surfaces that will make the estimated ASR big. Finally, we give numerical examples in order to show the effectiveness of our method. Index TermsAsymptotic stability region (ASR), bounded control, linear matrix inequality (LMI), switching surface, uncertain systems, variable structure system (VSS).

bounded because of physical constraints. Recently, the problem of estimating the asymptotic stability region (ASR) of uncertain VSSs with bounded switching feedback controllers is considered in [5] and [19]. In [19], the notion of combining different Lyapunov functions along with a state transformation has been used. Using different Lyapunov functions, three domains of attraction have been found and combined to improve the estimated ASR. Therefore, the improved regions do not have convex boundaries, and because the method of [19] is based on the subsystem matrix norms, a certain degree of conservativeness on the estimated ASR is imposed. In [5], stronger stability results, the exponential stability of the closed-loop control system, than just only asymptotic stability results of [19] have been derived by using a state transformation and a decoupling matrix. By using a numerical example, Choi and Chung [5] have shown that for a certain class of uncertain VSSs with bounded controllers less conservative estimates of the ASR can be obtained via the method of [5] rather than via the method of [19] under the same sets of conditions. Because both results given in [5] and [19] have used state transformation matrices, they are indirect and complex, thus they do not seem to be useful in designing a sliding surface. On the other hand, various Lyapunov-based methods that can be applied to the problem of stability analysis and/or control design for linear systems with input saturation nonlinearities have been developed in [8][13] and the references therein. However, it is unlikely that these methods are directly applicable to either estimating the ASR of uncertain VSSs with bounded switching feedback controllers or designing switching surfaces which will make the estimated ASR big. Considering these facts, we propose a simple method for estimating the ASR. We obtain an estimate of the ASR by using LMIs. Because our method does not require state transformation it is direct. Moreover, since the LMIs are very easy to handle in both analysis and design, our method can easily be used for switching surface synthesis. By using numerical examples we show that our method can be better than the previous results. II. PROBLEM STATEMENTS AND BACKGROUND RESULTS We will consider the uncertain systems that can be represented by the following dynamical equation [5], [19]:

x(t) = Ax(t) + B [u(t) +  (t)] _

I. INTRODUCTION The variable structure system (VSS) theory is often used in controlling of incompletely modeled or uncertain systems. In the VSS, the control structure around the system is intentionally changed by using a high-speed switching feedback control to obtain a desired system behavior or response, from which the term VSS arises. Using the switching feedback control law the VSS drives the system trajectory onto a specified and user-chosen surface, which is termed the sliding surface or the switching surface, and maintains the trajectory on this switching surface for all subsequent time. The central feature of the VSS is the so-called sliding mode on the switching surface within which the system remains insensitive to internal parameter variations and extraneous disturbance satisfying so-called matching condition [4], [17]. In practical implementation, the control input variables are

(1)

Manuscript received April 16, 2003; revised September 28, 2003. Recommended by Associate Editor Z. Lin. This work was supported by the Dongguk University Research Fund. The author is with the Department of Electrical Engineering, Dongguk University, Seoul 100-715, Korea (e-mail: hhchoi@dongguk.edu). Digital Object Identifier 10.1109/TAC.2004.825630

where x(t) 2 Rn is the state, u(t) 2 Rm is the control,  (1) : R+ ! Rm is the lumped uncertainty, A and B are constant matrices with appropriate dimensions, and the following assumptions are satisfied. A1) There exists a known nonnegative constant  such that k(t)k   for all t. A2) The input matrix B has full rank m < n.  A3) The control u has a known upper norm bound kuk  u.  A4) The difference u 0  =  is positive,  > 0. A5) The pair (A; B ) is stabilizable. Now, define the linear switching surface as
= fx :  (x) = Sx = 0g where S 2 Rm2n with full rank m. By referring to the previous results [5], [7], [19], one can see that it is reasonable to assume that the switching surface parameter matrix S satisfies the following properties. SB is nonsingular. For brevity, we assume SB > 0. P1) P2) After the system trajectory is already stably restricted to the linear switching surface Sx = 0 by using an appropriately chosen stabilizing feedback control law, the reduced (n 0 m)th-order sliding mode dynamics is asymptotically stable. It should be noted that the independence of the sliding mode dynamics from the control arises from the nonsingularity of SB and P1) is nec-

0018-9286/04$20.00 2004 IEEE