INr. r. coNrRoL. 1988.vol-. 48. No.

3- 949 959

Remarks on the periodic orbits of the matrix Riccati equation IlNRIQUE SERNIKTAndARISTOTLEARAPOSTATHISI
We investigatecertain questionsconcerning the periodic structure of the matrix Riccatidifferentialequation with constantcoe{icients. closed-formcxpression A for is the periodic solution.s obtained for both the casesinvolving distinct br repeated eigenvaluesin the associated linear hamiltonian system. Previous results are extendedby establishing that periodic solutionsare bounded ifand only ifthe span of their range does not intersectthe orthogonal complement ol' the controllable subspace the associated of linear systcm.

1. Introduction effort has beendevotedto the study of pcriodic In thc past few years,considerable optimization problemsand periodic control in general.There are severalreasonslor appear in nature (Speyerand Evans 1984)and this: many examplcsclfcyclic processes it seemsthat their performancecan be improved by using cyclic opcration; also periodic controls are easier to implement than arbitrary time-varying controls; in addition, under certain conditions, periodic controls provide better performanccfor the systemthan the optimal stcadystatccontrol. All of thesereasons lead to thc study clf optimal periodic processcs. Among the research problems in periodic control, thosc associatcdwith the periodic solutions of the matrix Riccati differential equation (RDE) are of special significance. The study of the RDE is worthwhile sinceit appearsin many important applicationsof optimal control, numerical analysis,estimati<lnthcory, frltering and other fields.Understandingthe pcriodic structure of the RDll might also prove to bc very bcncficialto the constructionof numerical algorithms.When seekinga constant solution of the RDE one should avoid bcing trapped near a periodic orbit, otherwise will be very slow. In addition, knowledgeof the periodic solutionsof the convergence RDE will provide us with a better understandingof its phaseportrait. Our interest lies in the real, symmctric, periodic solutions of the matrix Riccati d i ffe rc n ti a l q u a ti o n e p (d : - A 1' p(t)- p(t)A + p(t)R p(t)-e (r.l ) where .4, R and Q are n x n real matrices and R and Q are symmetric. As is well k n o w n , (1 .1 )i s a s s o c i a te d th the l i near probl em wi

*11,""):l:, _;]til]
':lln -;]

(1.2)

The hamiltonian matrix

(1.3)

Received October1987. 20 of and ComputerEngincering, tJniversity Texasat Austin, The of t Department Electrical Austin, Texas 78712. U.S.A.

950 satisfies

E. Sernik and A. Arapostathis

IIJ,+ J,H r:0

where./, is the 2n x 2n matrix

J^:l

fo

,,-l

L- 1, 0 l

(14 )

that the exponential the n. with /, denoting identitymatrix of dimension It follows, (Hr) is symplectic, i.e. matrix exp (Ht))r:7, exp(Hr)"/,(exp are by linearsystem related of The solutions the RDE and the associated
P (t) : Y (t)X - t(t)

(1.5)

(16)

Clearly, if a solution of the RDE is periodic, then there cxists a t > 0 such that the linear system(1.2) satisfies solution of the associated

,,"" span [;l:i]: t;g]

( 1.7)

if and X(r) is non-singularfor 0 < t < r, then Y(t)X 1(r)is Conversely, ( 1.7)is satisfied a periodic solution of thc RDE. One may view the RDE as an equation on the Lagrange Grassmann manifold, a technique introduced by Hermann and Martin (1981)for the study ofperiodic orbits, dating back to the work ofSchneider (1973). manifold ariseshere as the natural compactificationof the The Lagrangc-Grassmann spaceof real symmetricmatricesS(n) and in this extendedphasespacecvery solution (1.7),may be viewed as a periodic solution of of (1.2),having fullrank and satisfyinC the RDIr.'I'his conventionis going to be adopted throughout this paper.Also, we refer to those solutions that lie in S(n) as the bounded periodic solutions of the RDE. The early work on periodic solutions of the RDE concentratedon the study of (Bekir and Bucy l976,Hewer 1975). In time-varyingsystems with periodiccoefficients the study of the time invariant problem, the work of Nishimura and Kano ( 1980)has been the starting point of numerous results.The methodology of Martin (1980) and H e rma n n a n d Ma rti n (1 9 81)w as adoptedby S hayman(1983a, b), w ho embarkedon an extensivestudy of the phaseportrait of the RDE. T'he purpose of this communication is twofold. First, following the approach of for Hermann and Martin, we presenta closed-formexpression the periodic orbits of the RDE. Not only does therc seemto be growing interest in closed-form solutions for the problem (Wilcox and Harten 1984;Sasagawa 1985)but, in addition, thesehelp to establish additional properties of the orbits, that are hard to obtain otherwise. we and sufficientconditions for the periodic solution to be Secondly, derive necessary (1982),Marti n (1981)and by b o u n d e d .T h i s q u e s ti o nh as beenaddressed S asagaw a Shayman(1983a). We extend their resultsand resolvethis question in Theorem 3.1. The resultsin this paper are basedon the following mild assumption.

Periodic orbits of matrix Riccati equation

951

Assumption The RDE (1.1)hasa constant solution. (Coppel The structureof the equilibriaof the RDE has beenstudiedextensively l9l4,Lancaster and Rodman1980, Gohberg al. 1986). is wellknown,that if K is et It a constant solutionof (1.1), matrix FI can be reduced, a similarity via the transformation, to the form

fe H:L

RK

-(/-RI()'l
(1.8)

Hence, without loss of generality, as far as studying the periodic orbits is concerned, w e m a y re p l a c e(1 .1 )b y P(/): - A " | P G) P (t)A + P (0R P (4

In $ 2 we briefly review the approach of Hermann and Martin and we analyse the two-dimensionalproblem, which seemsto be very popular becauseof its simplicity and the geometric insight it has to offer. The method generalizesin a straightforward manner, as we seein $ 3, and it can also be extended to include the case of repeated in eigenvalucs the hamiltonian matrix ($ 4). Periodic orbits and the Lagrange-Grassmann manifold A key step in the study of the RDE is the identificationof the phasespaceof ( 1.2) with the spaceof n-dimensional subspaces R2n,which is most commonly known as of the Grassmann manifold G(2n, n). Since we are interested in periodic solutions of the RDE, it is clear that we should restrict our attention to those subspaces R2' of the of form span I 2,

It is relativelysimpleto deducethat a subspace of R2' will have this property if and W only if for all pairs of vectorsx, y W, xr J ny :0, whereJ, is the matrix in ( 1.4).These subspacesof R2'are called lagrangian. Observe that, by (1.5), once the initial condition of (1.2) is lagrangian,then the correspondingorbit is also lagrangian. Consider a subspaceof R2n of the form

Ixl - 1 is a symmetric matrix (provided X l, such that YX Lvl

is non-singular).

* : r0 " " [;
where K is n x k and L, M are nx(nand only if

:]

(2.1)

k) matrices.It follows, thatW is lagrangianif MTL

LTM: K rM:0

(2.2a\ (2.2b)

In order for a lagrangian subspaceto generatea non-trivial periodic orbit of the RDE it must be exp (Hr)-invariant but not Il-invariant. This observation leads to a very simple rule for obtaining periodic orbits, which we next investigatefor n:2. Consider the equation (1.8), with n:2. Supposethat Ahas a pair of complex eigenvalues jb, with b *0. Let n:Le't.e"f be a square real matrix, where e:e' a*

952

E. Sernik and A. AraPostathis

A: a + jb. If we define with of i je,, eC2is an eigenvector .4 associated the eigenvalue

F:( Er ) - ' :lf' :.f")

(2.3)

to of jf,,is clearlya left eigenvector ,4, corresponding the eigenvalue thenf:f,* X: a - jb. We also define G : lg' :.g"l according to AG - RF: -GAr
where

(2.4)

n:[j, o"f
AE : Elt, Fr A: lt Fr

(2.5)

that Observe

(2.6)
(2.7)

and AFrG + FrGAr: FrRF In addition,(2.7)impliesthat FrG is a symmetricmatrix. If we let

trll l-cos w( 4 : l . I (4 1
L s rn
by then the lagrangian manifold containing the periodic orbit is spanned

(2.8)

[r'(o) GW(r)1. p+x2 v1o,t):l o FW(y,1)

condition(2.2b), that yields subjectto the lagrangian : g [Ew(0)]rFw(]): cos(0 + 71 if The periodicorbit is bounded and only if [Ew(g)i Gw(y)]is invertibleand,in to (2.10), this is equivalent [Fw(y)]rGw(l) + 0, for all 7 e R
+FrG is sign definite

(2.e)

(2.r0)
viewof

(2.1 1)

is Shayman(1983a) has shown that a periodic orbit of the Riccati equation pair (in the casewhereR : BBr). We providedthat (,4,B) is a controlfible bounded, lemma' We ui" goingio showthat this conditionis alsonecessary. needthe following

Lemma2.1 that the matrix Suppose

,:l::x)

Periodic orbits of matrix Riccati equation has distinct eigenvalues. Then, the solution of the Lyapunov equation A X + X A T:B B T is non-singular, i.e. det X * 0, if and only if the pair (A, B) is controllable.

953

Proof The pair (,4,B) is not controllable thereexists vectorel 0suchthat iff a
eeN

[";"] whereN denotes null-space. the Hence, AXe* XAre: BBre+(A+ )"l)Xe:O which implies that,sinceFI hasdistincteigenvalues, Xe:0. Conversely, suppose that X is singular. z *0, with z e N(X). Consider Let AXz * XArz: BBrz The first term on the left-hand sideof (2.12) zero,and is 0:zrXArz:zrBBrz implies Bt'z:0+XArz:O Hence, z e N(x), i.e.N(x) is ,4r-invariant. Ar Therefore, N(x) contains eigenvector an e of Ar. It follows that Bre: 0, implying that (A, B) is not controllable. n In view ol(2.7),it is a directconsequence Lemma2.1 that the periodicorbit is of bounded the pair (,4,B) is controllable. is of interest note that onecan express iff It to the periodicorbit in the closed-form expression: e.l2)

.I P (4 :

cos( - br ) sin( - br ) lo, sin21- br ; I fcos(-br)sin(-br) r -arrl ( ( lcos -br) sin -br)lFrofto' fsinl- br ) l

(cosz bt)

(2.1 3)

z-Dimensional problem In this section we investigate the general n-dimensional problem, under the assumption that H has distinct eigenvalues.Every periodic solution of the Riccati equation can be constructed as follows. Let {a,+ jb,}i:, be a collection of n pairs of complex conjugate eigenvalues .4, whose imaginary parts (necessarily of non-zero) are rationally related and r denote the least integral multiple of nfbr, ..., nlb^. Extending the notation of the previous section,given eigenvectorse, : e,i* je,i,of ,4, correspondl i n g to th e e i g e n v a l u e7 i :a i + j bi ,i :1,..., nr, w e form the mati i x

3.

(3 1)

954

E. Sernik and A. Araoostathis

and selectleft eigenvectorsf : f',+ jf'! , corresponding to the conjugate eigenvalues, l"i: ai * jbi, such that if

(3.2)
then

E: F,: I

( 3.3)

Augment the collection of the linearly independentvectors er, 8,, i: l, ..., m, by adding eigenvectors 1, ..., un-26 of ,4 so as to construct a basisfor C', and let ur, ..., u un-r^denote the correspondingleft eigenvectors. For an arbitrary permutation o on { 1 ,2 ,...,n -2 m } a n d a n i ntegerk,0< k { n-2m,l et

(3.4)
and Au., A." denotethe diagonal matricesformed by the eigenvalues corresponding to the vectors in E,, F", respectively. we proceed to construct an exp (FIr)-invariant, n-dimensionalsubspace R2'. of Let

(A A , : dia g r, . . . .A . ) wh e re ,: l A
and

t' L -b ' 'tl at]

(3 5)

By c o n s tru c ti o n ,th e m a tr i ces E : [E .:8,] e R " x(zn+ k\ and F : s a ti s fY A E : E Le, Fr,4 : A r Fr Si m i l a rl y ,w e d e fi n eG : [G.i G,] e R rx(tr-k) by

: ;" : n' l-A," ol n. l-no" ol L ^,_l' L;" ^,1

(3 . 6 )
p '" ( n - f t ) [F. lF,] e

(37 ) (38)

Fora rear vector 0:l.t

'"rr'tt*;,;:f,t

bethe2m x

m matrix

0 fco so r I sin o , 0

... ...

I W(01:l
|

o coss,
0 sin0,

o
0

o I 0 I

tl

l::l
0

I |

(3e)

I o

... coso. l

... sin g .I

Then, if we let E(0) : lE,:.E,W(Q)] and similarly define F( .) and G( .), the span of th e 2 n x n m a tri x

,tr,rl:l

t(u) crrtl
r(Y)-l

L o

(3.10)

Periodic orbits of matrix Riccati equation is an exp (Hz)-invariant subspace, all 9, 7 e R'. Also, F and G satisfy for AF FTG + FTGAT: FTRF

955

(3 . 1 r)

: FrG : GrF. Thus,E(0)rF(7) 0 and therefore, the of since eigenvalues FI aredistinct, is a necessary sufficient and condition for the span of V(0,y) to be lagrangian. This reduces the constraints to cos(0r- 7;): 0, i: 1,...,m

(3.r2)

In obtaining closed-form a solution theperiodic for orbit,we will usethefollowing matrix inversion lemma. Lemma 3.1 Let

v:l

l-xl [r l:l L YI L O

*l
J

L1

( 3.r 3)

where X and Y are square matrices and suppose that K has full rank and Mr K : O. Then, a necessary and sufficient condition for X to be invertible is det (MrL) * 0. In a d d i ti o n
V- t _

[t r ' r t ' K ' ( /

L

- L(Mr \-'r')l L

(Mr 1 ; - r1 r1 t

(3.l 4)

and hence Y X -t: M(Mr7.)-r 14r

(3.15)

In order to compute the periodic orbit, one calculatesthe evolution of F(y) and G(y) under the hamiltonian system.Then, applying Lemma 3.1,with

M : lF, exp( - A|.r) : F, exp( - AI AW$l L: lG,exp( - A["r): C,exp( - LI t)W(y)l

we obtain P(t) : F(y - bt)lFr(y - bt)G(v - bt)l- 1 Fr(y - bt\ w h e re b : l b r b ^ f,. Concerning the boundednessof the periodic orbit observe that

(3.16)

(3.17)

F (0): Ftfr/(o), c'(0): cfr/@)

where

( 3.r 8)

o -l tu@t:l'o

L0 w(0))

(3 . 1 e )

Supposethat R : BBr. By Lemma 2.1, FrG is non-singular if and only if the pair (AI, FrB) is controllable.Following the same argumentsas Shayman (1983a), one showsthat (A;, FrB) being controllableis a sufficientcondition for the orbit in (3.17) to be bounded.Conversely, the proof of Lemma 2.l,FrGz: 0 lor someeigenvector by

956

E. Sernik and A. Arapostathis

z of A|. It is evidentthen, in view of (3.18),that for some to e R, Fr(7 - bt)G(y - bto) is singular and hencethe periodic orbit is not bounded. Note that the span of the range of the periodic solution in (3.17) is equal to span (F), which we call the rotating subspace associated with the periodic orbit. With C o d e n o ti n g th e c o n tro l l abi l i ty matri x l B :.A B :....' .A " -tB ], i t can be seen, usi ng (3.7),that the pair (A;, FrB) is controllable if and only if

N(G I F ): 0 is in Thisobservationsummarized thefollowine theorem.

Theorem3.I

(3.20)

The periodic solution of the Riccati equation is bounded if and only if the associatedrotating subspacedoes not contain any vectors which are orthogonal to the controllable subspace. As a final comment,observethat wheneverm> l, the periodicorbit is not isolated. This is due to the fact that there is a continuum of parametersy, that satisfy the lagrangian constraint (3.12).In general,then, one obtains a continuum of periodic orbits that comprisean n-dimensional torus. 4. Periodic structure with repeatedeigenvalues As was pointed out by Martin (1980),the presence repeatedeigenvalues II of of complicatesthe periodic structure and the problem is not analysedin detail in the most recent studies.The complexity of the problem arises from the coupling of repeatedeigenvalues the sameJordan block. Hence,without loss of generality,we in consider the case where ,4 is a 2r x 2r cyclic matrix, having a pair of conjugate A eigenvalues, and i, that are roots of multiplicity r of its minimal polynomial. With the eigenvalueA : a + jb we associatean eigenvect eo : e'o-f .je'[as well as or a c h a i n o f g e n e ra l i z e d i g e nvectots e' ,+j e' ;' ,i : 1,..., r - 1, that are sel ected as e e,: so to satisfy Ai + t:i e,* r* e;, i :0,...,r-l (41)

In analogy to the developmentin the previous section,we define

(4.2)
and F:(E r)-1 Let A denote the 2r x 2r matrix
/4 1\

^ : ;I ^:l: L oo
[n t o- l
where A is given in (2.5) and define G by ,4G - R F:

(4 4)

^l

-GA r

(4.s)

Periodicorbits of matrix Riccatiequation Corresponding a pair of realvectors to 0:l0o in R', we form a 4r x2r matrix V(O,y), given by 4-r]t and y: [70

957 ],-r]r

few@ ct{r(?)-l v1 o ' r ) :l o F W( y) )

whereW(,) is a 2r x r matrix of the form (3.9), Observe that

(4.6)

w (L o -b t ) t w(L t -b t )
: exp(AI)EW(O) exp(at)E

f}

w(O , -r-b t l

o
0

w(o,-bt)ft^u,-,-r,l lto.rl
0 w(O,-r-bt)

therefore, span( EW(0))is exp(,4r)-invariant 0. : 0o(mod n), i: l, ..., r - 1.Note, iff oncemore,that AFTG+ FTGAT: FTRF (4.8)

FrG is symmetric, while the lagrangian conditionIEW(fl)rfW67:g and hence, yields cos(0,-lr-r-r):0, i:0, 1,...,r-1 (4.9)

generality, 0i:nl2andy,-0,i:0, Thus,withoutlossof let ...,r- 1.Inthismanner, span I(0, y) is an exp(Ilr)-invariant lagrangian periodic subspace the associated and orbit of the RDE can be computed, utilizing Lemma 3.1,to yield P(4: Fq(-dtq(-4tFtGq(-r)l-lq(-r)'F' where q(tl : w(bt)K,(t) and 100 tl0 K,(/): (4.12) (4.11) (4.10)

ti-, t)_, ( ' - lX O1) t

Note that (4.10)simplifies to p(t) :71411- bt)lw(- bt)rFrcw(-bt)l-

rw(-bt)rFr

(4.13)

Finally, following the analysisin the previoussection,it is the casehere,that with R : BBr, the periodicorbit is boundedif and only if the.pair (.4,B) is controllable.

958 Example Consider

E. Sernik and A" Arapostathis

l- z I
A:l

r
-2 0*2 0

I
0

o
I 1 -2

| -1 l0 I L 0

II
l
-fsinrcos fsin2r fisintcost -$sin2r

-1

and fet R be the 4 x 4 identitv matrix and O :0. Usins the techniques established in this section,we obtain

P (r):?i I
5.

l- - cost t I I sint cosr

|cos2t | l -| s i n rc o s r

sin t cos, -sin2 r

-{ si ntcost j si n2r

fcos2r -|sinrcost -ficos2 t $sinrcost

Concluding remarks Using classicaleigenvectormethods,aided by geometricalinsight, we presented certain aspects the periodic structureof the RDE. We gave a closed-formsolution, of that can be computed directly from the eigenvectors the associatedhamiltonian of matrix and we analysedthe caseof repeated eigenvalues. concludewith a remark We on the role of eigenvalues the imaginary axis. on Let P(t) be a periodic solution of the RDE. Note that
Ft

P(r): exp( -,4rr) P(0)exp( - Ail + | exp[ - A'(t _s)] P(s)RP(s) [ -,4(r - s)] ds exp
JO

Supposenow, in order to simplify matters, that all the eigenvalues H lie on the of imaginaryaxis.Then, if R is positive-semidefinite, null-space R must contain the the of ra n g e o f P (r), fo r a l l re R . L et { er,...,ex) be an orthonormal basi s for N (R ) and considerthe matrix S : [er:.er.:....:.nof It follows that P (t) :5P 1115r where P-(r)satisfies the linear equation

: P(r) -s',arsP(r) P(r)s'.4s

As the above analysisshows, the presenceof eigenvalues the imaginary axis is on responsible the existence a continuum of periodic solutionsthat are solutionsof for of a linear systemof lower dimension.

AcrNowlrocMENT This researchwas supported in part by the Air Force Office of Scientific Research under Grant AFOSR-86-0029and in part by the National Science Foundation under Grants ECS-8307547 and ECS-8412100.

Periodic orbits of matrix Riccati equation

9s9

RnrBRENces Brrn, E., and Bucy, R. S., 1976,Stochastics,2, I. Corrrr, W. A., 1974, Bull. Aust. nath. Soc., i0,'tll. p., GoHnrnc,I., Ler.rc,c,srEn,and Ropr'rnN, 19g6, L., srAM J. contror optim.,24, 1323. HrnuaNN, R., and MnRnN, c. F., l9gr, proc. 20th I.E.E.E.coi/.ii6"r*ion and contror, pp. 645-648. Hewsn,G., 1975, SIAM J. Control,13,1235. LrNcasreR, and RoorrlaN, 19g0, J. Controt,32,2g5. P., L., Int. MenrrN, c. F., 1980, Geometrical Methodsforthe.Theoiyof Linear sysrems, edited by c. I. Byrnesand c. F. Martin (Dordreiht Reidet),pp.'r95-2r2;irtt, syrrr^s contror Lett.,l, 127. NIsHrrrauna, and KnNo, H., 1980, T., I.E.E.E.Trans.autom.Control,25,749. Ses'rc'rwn,T,1982, I'E'E.E-Trans.autom.connor,27,977;lggi:;;iitth.comput.,16,l4r. ScnNupER, 1973, C, Math. Syst.Theory,7, ZEl. SueyuaN, A., 1983 Myth. Syst. M. a, ineory,16,267;19g3b, proc.22ndI.E.E.E.Conf.on Decision Control,pp. 1393-1395. and Spryen, L., and EvaNS, f., tSA+, J. R. I.E.E.E.Trans.autom.Control,29, l3g. Wtrcox, R. M., and Henrnn, L. p., 19g4, Appl. Math. Comput., iq;.' tl,