-
Proceedings of the 30th Conference
-
on Declslon and Control
Brighton, England December 1991
SUBSPACE ALGORITHMS FOR THE STOCHASTIC
IDENTIFICATION PROBLEM1
Peter Van Overschee2 Bart De Moor
ESAT ,
Department of Electrical Engineering , Katholieke Universiteit Leuven
Kardinaal Mercierlaan 94 , 3001 Leuven (Heverlee) , Belgium
tel: 32/16/220931
email: vanoversQesat.ku1euven.ac.be , demoorQesat.kuleuven.ac.be
Abstract
In this paper, we derive a new algorithm to consistently iden-
tify stochastic state space models from given output data without
forming the covariance matrix and using only semi-infinite block
Hankel matrices. The algorithm is based on the concept of princi-
pal angles and directions. We describe how they can be calculated
with only QR and QSVD decompositions. We also provide an
interpretation of the principal directions as states of a non-steady
state Kalman filter.
1. I n t r o d u c t i o n
..
Let Yk E @, k = 0,1,. ,K be a data sequence that is generated
by the following system :
Zk+1 = AZk + Wk , Yk = czk t vk (1)
where Zk E s"
is a state vector. Both wk E 8" and vk E 8' are
assumed to be zero mean, white, gaussian noise sequences with
covariance matrix 4:
It is assumed that the stochastic process is stationary, i.e. :
E[zk] = 0 and E[X~Z:] = E, where the state covariance matrix
C is independent of the time le. This implies that A is a stable
matrix.
The central problem discussed in this paper is the identification
of a state space model from the data Yk and the determination of
the noise covariance matrices.
The main contributions of this paper are the following :
- Since the pioneering papers by Akaike [l],canonical correla-
tions (which were introduced by Hotelling [5] in the statisti-
cal community) have been used as a mathematical tool in the
stochastic realization problem. In this paper we show how
the approach by Akaike and others (e.g. [2],[7]) boils down to
applying canonical correlation analysis to two matrices that
are double infinite. A careful analysis reveals the nature of
this double infinity and we manage to reduce the canonical
correlation approach to a semi-infinite matrix problem, i.e.
only the number of columns needs to be very large while the
number of (b1ock)rows remains sufficiently small.
'The research reported in this paper was partially supported by the Bel-
gian Program on Interuniversity Attraction Poles initiated by the Belgian
State Science Policy Programming (Prime Minister's Office) and the Eurw
pean Community Research Program ESPRIT, Basic Research Action nr. 3280
and by a research project supported by Philips. The scientific responsibility
is assumed by its authors.
'Research Assistant of the Belgian National Fund for Scientific Research
8Research Associate of the Belgian National Fund for Scientific Research
'The expected value operator is denoted by E[.].
.OO 0 1991 IEEE
CH3076-7/91/0000-1321$01 1321
TI-IO 1O:OO
fax: 32/16/221855
- In order to find the state space model, we derive a finite-
dimensional vector sequence which, in the case of double
infinite block Hankel matrices, would be a valid state se-
quence of the stochastic model. For the semi-infinite matrix
problem, the sequence is not an exact state sequence but
it corresponds to the output of j (+ w) non-steady state
Kalman filters that have used a finite number i of output
data as input to form an estimate of the state sequence.
The state space model is then derived from the sequences by
solving a least squares problem.
- It is a common belief that ARMA model identification requires
non-linear estimation methods, requiring iterative solution
techniques [9]. In that framework, the main problems arise
from convergence difficulties and/or the existence of local
minima. With the correlation approach, this ARMA mod-
elling is reduced to the solution of an eigenvalue problem, for
which numerically robust and always convergent algorithms
exist.
- We derive a numerically robust square root algorithm, that
mainly uses QR-decomposition and SVD of the triangular
factors and is compeletely data driven instead of covariance
driven.
This paper is organised as follows : In section 2, we summarize
the main properties of linear time-invariant stochastic processes.
In section 3, we'll describe how the projection of past and futsnre
block Hankel matrices of output data, leads to the definition of
the state sequences. From these sequences, it is easy to derive the
stochastic state space model. This will be done in section 4. In
section 5 we'll explore the relationship with the non-steady state
Kalman filter. In section 6 we'll introduce our main mathematical-
geometrical tool : principal angles and directions. We'll also de-
scribe a numerical robust computation scheme. In section 7, we'll
describe a new, numerically robust algorithm to calculate the state
space model of the stochastic system directly from output data.
An illustrative examples is given in section 8.
2. Linear time - invariant stochastic processes
In this section, we'll summarize the main properties of linear time
invariant stochastic processes.
It is common to assume in (1) that Zk and V k , w k are mutually
independent and hence
E[ZkV:] =0 and E[ZkW:] =0
Then we find the following Lyapunov equation for the state co-
def
variance matrix C = E ( Z k Z i )
C = AEAt +Q (2)
Defining the output covariance matrices A; def
for : = CCC'
Ai = CN-'G
= E(yk+iyL) we find
+ R. Defining G d ~ Af C C ~+ S, we get :
(3)
= [
I C

'A
CAi-1
\

( Ai-'G Ai-ZG ... G )

A-, = G'(N-')'C' i = 1 , 2 , .. . (4) = Ui C, (say)
The model (1)can be converted into a so-called forward innovation
model and a backward innovation model (see e.g. [ 1 2 ] ) . If we now compute the orthogonal projection of the row space of
The forward innovation model is obtained by applying a Kalman y I / ~ i - l (the future) onto the row space of & / i - l (the past), we'll
filter to the stochastic system (I), which results in : find that this projection is an n dimensional subspace of the j
dimensional subspace, as can be seen from :
zk+l = AZk + Wf , Yk = CZk + U:
Ej[X/zi-1Yi/i-iI ( E j I Y o / i - ~ Y ~ / i - i IY) -ol/ i - 1
where the forward noise sequences w: and U: are given by w [ =
K f v f and .
= Y k - CZk Here K f is the Kalman gain : K f =
= uj c; L,:' YOli-1 (9)
( G - A P C ' ) ( h o - C P C t ) - ' and P is the forward state covari-
ance matrix, which can be determined from the forward Riccati where we define
equation :
...
1
A0 A-1 A-2 Ai-i
P = APA' + ( G - A P C t ) ( A o- C P C ' ) - ' ( G - APC')'
The backward innovations model can be described as follows :
(5) Li dzf E j [ Y o / i - l Y i / i - l=]
= Ej[yI/zi-1Y?zi-i]
( ...
Ai-1
...
Ai-z
... ...
Ai-3 ...
...
A0

rk-1 = A'rk + Wk , Y k = G'rk + U;

The last equality is due to stationarity. Hence, a basis for the
with : w i = KbvL and vf = yk - G ' T ~ Here
. Kb is the backward n-dimensional projection of the row space of yI/z;-l onto that of
Kalman gain : Kb = (C' - A'N-'G)(Ao - GtN-'G)-' and N is YO/,-^ is formed by the rows of the n x j matrix 2; :
the backward state covariance matrix, which can be determined
from the backward Riccati equation : 2
1 = CiLfl&/i-l (10)
N = A'NA + (C' - A'NG)(Ao - G t N G ) - l ( C t- A'NG)' (6) In a similar way, we find for the orthogonal projection of the row
In [8] a robust algorithm to solve these Riccati equations based on space of YO/^-^ onto the row space of yI/2i-l :
the Schur decomposition is presented.
E j K / i - l K ~ z i - 1 1(Ej[yI/2i-1Y"/i-iI)-1KlZi-1

3. Orthogonal projections = c,t 0: L,Y' yI/2;-1. (11)

In this section, we'll start to describe an identification scheme that Hence, a basis for the n-dimensional projection of the row space
is based on geometrical insights and that obtaines consistent esti- of YO/,-^ onto that of K,z,-1 is formed by the rows of the n x j
mates with semi-infinite block Hankel matrices. This last fact is matrix W , :
in contrast with previously described similar algorithms [1],[2],[7] wi = u:L,:1K/2i-1 (12)
that needed double infinite block Hankel matrices.
The conclusion is that both the orthogonal projections of the row
In what follows, output block Hankel matrices of the form
space of +i-1 onto that of Yo/,-1and the other way around, are

&/i-1 =
i
Yo
Y1
Y1
Yz
.. .
Yz
...
Y3
. . . . . . . .. . . .
Yi-1 Yi Yi+l
Yj-2
Yj-1
...
Yi+j-3

will play a crucial role. The first subscript denotes the time index
of the upper left element while the second subscript is the time
Yj-1

Yi+j-Z
Yj
.. .
I finite dimensional subspaces of dimension n, with a lot of struc-
ture. Of course, in an infinite dimensional setting (i.e. i -+ 00 and
j -i m) this result has been known since the pioneering work of
Akaike [ l ] . In this framework and also the one presented in [12],
those finite dimensional orthogonal projections can be considered
as states of the stochastic process. As we'll see in section 5 , for fi-
nite i, these sequences are not the real states, but only an optimal
index of the bottom left element. For all output block Hankel ma-
Kalman prediction of the states based upon the i previous outputs.
trices, the number of columns will be j and throughout, we assume
that j -+ 00.
To arrive at a consitent identification scheme, we also need to
Another important concept arrises with the estimation of the co- +
define the ( i l)'h projections as :
variance matrices. We assume that they can be estimated its
1 j-1 - The orthogonal projection of the row space of K+l/z;-l onto
Ai = iimj+m[Y Yk+iY:] (7) that of Yo/;which is
3 k=O

for which we will use the notation E j [ C i I i Y k + i & ] , with an obvious K+l/zi-1Y,;iL::1K/i = ~ i - l c i + l L : ~ 1 y O / i
definition of Ej[.] = limj-, $[.I. The ensamble average can be A basis for the row space of this orthogonal projection is
replaced by a time average due to ergodicity. generated by the rows of
We now find easily for the correlation matrix between the future
block output Hankel and the past block output Hankel matrix : zi+l = ci+lL::iYO/i (13)

1322
 A basis for the row space of this projection is generated by
the rows of
w;+1= u:+lL;;lY-l,2i-l
Figure 1 illustrates these projections graphically
8
In this section, we'll show that the sequences 2; and Z;+l can be
(14)
considered as the outputs of a bank of non-steady state Kalman
: +
filters after i respectively i 1 time steps.
The proof consists of two parts : 1/ The state covariance matrices
C; = Ej(Z;Zj) and C;+1 = Ej(Z;+lZ:+l) are two iterates of the

&[
O t difference Riccati equation associated with the forward model and
+
2/ Z;+1 can be formed from 2; as : Z;+, = AZ; K;(y; - C Z ; )
with K; equal to the Kalman gain.
1/ From the definition of the sequences 2; and Z;+l we find easily :

C; = Ej(Z;Z:) = C;LclCf
Xi+' = Ej(Zi+lzt+,) = Ci+lLy+!lC:+,

If we rewrite C,+1 in function of C; with the help of the matrix

Figure 1 : Definition of the vector sequences Z;, W,, and W;+l. inversion lemma, we get (see [4]) :
The thick lines represent the span of the data, the thin lines rep-
resent the projections. +
= AE;A~ ( A C ~ -
CG~ ) ( A ~- C C ; C ~ ) - ' ( C C-
~GA'~) (18)
which is one step of the forward difference Riccati equation. The
4. HOWt o find A, C, G a n d A,,
Kalman gain associated with this equation is at time step i :
In this section, we'll show how the system matrices A , C, G and A. K; = (G - AC;Ct)(Ao - CC;Ct)-'
can be identified consistently, by solving a set of linear equations
in least squares sense. 2/ We'll prove now that Z;+1 is formed out of 2; with the help of
Define the matrices PZ and 2, such that Z;+l = PzZ; 2, where +
this Kalman gain.
by definition we take Z,Zj = 0. Postmultiplying this equation First, we can write (with Ni and M; unknown) : Z;+' = N;Zi +
M y ; , and by postmultiplying this equation with y,"/,-, and taking
with Zj we find theleast squares solution PZ : PZ = Z;+lZ,t(Z;Z,!)-l
Let us now prove that PZ = A . the expected value, we find : A = N; M;C, which leads to : +
+
Z;+l = AZ; M;(y;- CZ;). If we postmultiply this equation with
&+l = PzZ; 2, + y: and take the expected value we get : G = AC;Ct M;(Ao - +
+ C;+lL:~lEj[Yo/;Y,;i-l]LTICj = PzC;LT'Ej[Yo/;-lYk/;-,ILr'CtCC;Ct) So M; = ( G - AC;Ct)(Ao- CC;C')-' = K; which is ex-
actly the Kalman gain.
+ C;+IL:-~Ej[&/;Y,tI;_,JLf'C: = Pz C;L;'C;
0 ... 0 So, if we start from 2, = 0 and CO = 0, we find :

+ c;+1[ .; );I
I
e.; );. L;'C: = PzC;L;lC,t z1= &yo = GA&, = clL;ly0

Through induction, this clearly proves that the sequence 20,21,

+ A C;L;'C;' = Pz CjL7'C;' .. ., Z;, Z;+1 is a sequence of subsequent optimal Kalman state
+ A=& estimates. This has to be interpreted as follows : the Ph col-
The conclusion is that A is the least squares solution of umn of 2; is the optimal prediction of the state zl+;-' based on
the measurements yl-l,yl,. . .,yl+;-2 and with initial conditions :
$
; /lZ;+l- P z Z ; ( ( i as j 4 00 (15)
21-1 = 0, E ( Z ~ - ~ Z ; =
- ~ 0.
) Figure 2 illustrates this graphically.
where 1 1 . 1 1 ~ is the F'robeniusnorm. Let us now show that the matrix So, in this way every column of 2; represents an optimal state esti-
C can also be determined from 2; by solving the least squares mate. And we can interprete the columns of 2;as the outputs of a
problem. bank of j Kalman filters that estimate j optimal states in parallel.
:
9 II( ~i Y ~ + I. . . Yi+j-1 ) - PcZ~II;
20 [ 0 .** 0 ...
Define Pc and 2, by 0 1

y = ( yi yi+1 * . yi+j-1 ) = P c Z ~ +
where by definition Z,Zj = 0. Then from postmultiplying y =
+
PcZi 2, with y,"/,-, and taking the expected value, we find :
C = Pc. The preceeding derivation could also be applied to the
n-dimensional projection of the past onto the future, which is given
by the matrix W; (l2),which will lead to the matrices A and G. It
is straightforward to derive that : At is the least squares solution
of
min IIW;+l - PwWiIl; (16) Figure 2 : Interpretation of the sequence 2;as Kalman filter state
pw
estimates based upon i measurements.
and Gt is the least squares solution of
yk I[( yi-1 Y; * * yi+j-2 - P~wiIli (17)
8. Orthogonal projections a n d
A0 can then be found as : A0 = E j ( C t ' , ykyk) principal angles and directions.

1323
In this section, we briefly review orthogonal projections and the where
corresponding matriz algebra. We ezplain the notion of principal
angles and directions between subspaces.
rob = rank ( )
6.1 Orthogonal projections
Given two matrices U E RP"J and V E RqxJ, where j 2 The original formulation of the QSVD (which used to be called
maz(p,q) and r a n k ( U ) = p and r a n k ( V ) = q. Then, the row the generalized SVD, but see also [3]) is due to Paige [ll]and Van
space of UV'(VV')-'V is the orthogonal projection of the row Loan [13].
space of U onto that of V while the rowspace of VUt(UUt)-'U is
the orthogonal projection of the row space of V onto that of U. We are now ready to derive the following Theorem (which to our
We can combine these two projections in the matrix knowledge is new) :
U'( UU')-1 UV'( V V ' )-1 v (19) Theorem 1
the row space of which is the orthogonal projection of the row space Consider the QR-decomposition of the concatenated matriz
of U onto that of V . The column space of (19) is the orthogonal
projection of the row space of V onto that of U.
6.2 Geometrical motivation
= (g)
( i ) (1:: 12)
The notion of an angle between two vectors can be generalized where both A and B are of full row rank and consider a QSVD
to angles between subspaces by using a recursive procedure. This of the matric pair R;, = USX' and R& = V T X ' , where U, V are
concept goes back to Jordan [6]. Before giving a formal definition, orthonormal, S and T quasi-diagonal and X square nonsingular.
let us first present a verbal recipe. Suppose we are given two Then, the principal angles and directions between the row spaces
matrices U E Rp"j and V E Rqxj. The first principal angle B1 of A and B are given b y
(the smallest one) is obtained as follows : Choose unit vectors
ul E R ( U t ) and v1 E R ( V t ) and minimize the angle between ( Qiu) S ( &ius+ QzVT )'
them (The notation R(.)refers to the column space (range) of the where the nonzero elements of S are the cosines of the principal
matrix between brackets.) Next choose a unit vector 212 E R ( U ' ) angles, the left factor contains the principal directions in the row
orthogonal to u1 and v2 E R ( V t )orthogonal to V I and minimize space of A and the right factor contains the principal directions in
the angle 8 2 between them. This is the second principal angle. the row space of B .
Continue in this way until min(p, q ) angles have been found. This
informal description is now formalized : Proof : Because A and B are of full row rank, we have

Definition 1 Principal angles a n d directions A'A = QIQ:

The principal angles el 5 B2 5 . . . 5 .n/2 between the row spaces +
B ' B = ( Q i U S QZVT)(S'U'Q; T'V'Q;) +
R ( U ' ) and R(V') of two matrices U E R P x j and V E 8PxJand +
It follows from the properties of the QSVD that S'S TtT = I
the corresponding principal directions U ; E R(U') and v; E R ( V )
are defined recursively as
+
and hence &IUS QZVT is an orthonormal matrix. We find that

cosek = m a r vER(vt) U'V = u:vk

A'AB'B = ( Q l U ) S ( Q l U S +QzVT)~
subject to llull = 1 1 ~ 1 1= 1 and for k > 1 : u'u; = 0 and V'V; =0 which is clearly an SVD. It follows from Lemma 1that it contains
..
f o r i = 1 , . ,k - 1 the principal angles and directions. 0

From this definition, it is not too difficult to show that

The major advantage of this result lies in the fact that princi-
Lemma 1 pal angles and directions can basically be computed from blocks
The principal directions and cosines of the angles between the row of the R-factor of a QR decomposition. This R-factor consists of
spaces of two matrices U and V are the singular vectors and values matrices of which the dimensions are small compared to j .
of the m a t h U'UV'V.
7 . Principal angles a n d directions
If both U and V are of full row rank then of course U'UVtV =
and outDut block Hankel matrices.
U'( UU')-1UV'(VV')-1V.
6.3 The QR-decomposition and the QSVD W e have derived in section 4 that the matrices A and C could be
Lemma 2 estimated from the matrices 2; and Z;+1 while A and G could be
Any pair of matrices A E 32""" and B E RPxn can be decomposed estimated from W; and W;+l. I n this section, we show how these
as matrices can be computed starting from the output data.
A= uosQx'and B = U b S b X t First, consider
where uo
E R""", E R p x pare orthonormal, x E R""" is
square non-singular and S, E RmXnand S b E Rp"" have the fol-
lowing quasi-diagonal structure : We know from Lemma 1 that the SVD of (20) delivers the prin-
cipal angles and directions between past and future.
rob - rb To +% - rob f a b - Ta - rob Recall that the rows of 2; form a basis for the projection of the

L
- Tb
rob 0 0 row space of y I / z , - 1 onto the IOW space of Yo/;-1. Consider the
So = +
ro r b - r d 0 D, 0 RQ-decomposition of the output block Hankel matrix :
m - r, 0 0 0
Tab - Tb To + Tb - fob Tab - n - Tab
0 0
Db 0 where all blocks in R are square : li x li. The scalar J? is used
0 I 0 to be conform with the definition of the operator Ej.
1324
Lemma S Without loss of generality, we can set one of these three equal to
Let j -+ 00 and consider the QR-decomposition as in (21). Then the identity matrix (which is equivalent with fixing a certain basis
the following facts hold true : in the state space), say M; = I,,. Then we get :
- The column space of Rzl in the QR-decomposition (21) coincides
with the column space of U ; . Mi-, = ( s , ! ~ l ) - ~ ~ ~ ( x , ? ~ l(23)
)~~(s,!)~~~
- Only n principal angles between the row spaces of Yo/;-1and M+i = ( s;+l)-'/2(x&)+x:(s:y (24)
Y&l are different fiom ~ / 2 .
Proof : It is straightforward to derive that
The notation 2 refers to the l(i - 1) x n matrix obtained from
X,! by omission of its last 1 rows.
Ej[Yi/~i-iYO;;-11(Ej[Yo/;-iY~/;-~])-~yO/;-i
= RziQ:
Let us now clarify how the QSVD allows us to calculate the ma-
It then follows from (9) that the column space of Rzl coincides trices Z;,W;,Z;+l and W;+l and at the same time provide the
with the column space of the extended observability matrix 0;as required similarity transformations.
j + 00. The fact that only n principal angles are different from Recall that Y i ; z i ~ l l ; f ' [ E ~ [ ~ ~ z ~ ~ l ~ ~ ~ ]=] ~W/Z;
l L r lUsing
Yg/i-~
90" follows from the rank deficiency in (20). This enables us to the QR-decomposition and the QSVD we take
estimate the order n of the stochastic system : when j + 00, n is
equal to the number of principal angles different from 7r/2. 0

Next consider the QR-decomposition of the output block Hankel

matrix Yo/z;-1with the following appropriate partitioning (which
is also visualized in Figure 1):
Wi = (S:)l/z( (T:)'v') [ $1
Q:
(26)

Then, it follows from Lemma 3 that the column spaces of

The conclusion is that the combined QR-QSVD approach delivers

respective column spaces of 0,and U;+1. Next, we will w e Can obtain the matrices zi+iand Wi+i by using formulae (13)-
apply Theorem 1 to these three blocks. Hereto we firsts compute (14), but we have also to take into account the similarity transfor-
three QSVDs : mations Mi-1 and M;+1 as in (23)-(24) :

\ I

If we now apply equation (15) to these matrices, we find that A is

( Rzz
R3Z
R4Z
0
R33
&3 &4
)' = K+lT;+lX:+l
Or with equation (16) we find for A: :

(Gt)
)l/zx,! ( ':+l It '?+I ( yi1) t ) ( u;ls: ) (s,!)-l/z
KIT:
With n, the state space dimension from the number of principal (32)
angles different from s / 2 , let X;-1, X i , X ; + l , Ui-1, U;, Ui+l,K-I, The matrix C follows from the least squares solution of () and is
Vi, K+l, &-I, S;, S;+l, Tj-1, T; and T;+1 with a supperscript 1 given by
denote the first n directions and first n diagonal elements of these
matrices. It then follows from Theorem 1that the column spaces of c = ( R31 R32 )U,!(s;')-'" = ( 1
1 0 )Xi(s;')"' (33)
U;-l,U; and U;+l are generated by the columns of X:-l(S:-l)1/2, The second equality shows that C is equal to the first 1 rows of the
X,!(S,!)'/' and X,?+l(S:+l)1/2.This means that there exist three estimated U;.Finally, the matrix G follows from the least squares
similarity transformations, M; and Mi+l so that
solution (17)
x : - l ( s ; ' - l ) ~ ~ ~ M ;=- l u;-1 G = ( Rzi Rzz )U,!S,! (34)
X,!(S,!,'/'M; = U; This last equation could also be written in function of the QSVD's,
but that would make things unnecessary complicated. Observe
X , ? + l ( s , ! + i ) " z ~ i +=
i oi+i
1325
that in these final expressions for A, C and G the orthogonal ma-
trices Ql,Qz,Qs and 9 4 have cancelled out, so we don't need to
calculate them. This observation represents a major reduction in
the computational complexity.
8. An example
Estimates of the state sequence 2; are obtained by taking linear
combinations of the past output matrix YO/;-^ as 2; = QiYo/i-i-
Now, in earlier work [2],[7],[12), the shifted state sequence Zi+1
is computed by applying the same linear combinations Q; to the
shifted Hankel matrix Y1p as Zi,, = QiY1li. While this trick
works for purely deterministic systems (see [lo]), it doesn't work
for stochastic systems for finite i, as our formulas in the previous
section clearly indicate. In general, one can state that for small
i , the bias for the classical approach (same linear combinations)
becomes wurse as some of the poles of A move towards the unit
circle. Only for i + 00, this classical approach will be exact (as
e.g. in [12]). The following easy example illustrates this clearly.
Applying formulae. (10) and (13) to a block Hankel matrix with
respectively one and two block rows (SISO system), gives :
We then find that
If on the other hand we take, as is usually done, the same linear
combinations for 22 as for Z1 (namely GA,'), we find : 2 2 =
GA;'Yip and
PZ = ZzZ:(ZiZ:)-' = A1/Ao = CG/(CZC' + R)
which has nothing to do with A !
Consider the very simple SISO system with only one state : A =
0.9490,B = -0.4424, C = 0.8725,D = -0.9666. Figure 3 shows
the eigenvalues of the identified A matrix (a scalar), in function
of the parameter i ( j = 1000) for the 2 algorithms : Stars (*) :
taking the same linear combinations as described in this section,
0's : ( 0 ) the algorithm as described in section 7. For every i,
100 Monte Carlo experiments were performed. The mean value of
these 100 experiments is plotted for every i on Figure 3 (the exact
value is ofcourse 0.949). Clearly the algorithm described in this
section (which is the one that is normally used in the liturature),
calculates heavily biased solutions for small i. On the other hand,
the new algorithm (of section 7) doesn't suffer at all from this
problem.
OM
0 0 6 * I * * 1982.
0.94
a*t
a91
Figure 3 : Stars (*) : Mean of the estimated eigenvalues in function
of a for the algorithm of section 8. 0's ( 0 ) : The same for the
algorithm of section 7.
9. C o n c l u s i o q
In this paper, we have derived a new algoritm for the consis-
tent identification of stochastic state space descriptions, using only
1326
semi-infinite block Hankel matrices of output measurements. The
explicit formation of the covariance matrix and the double infi-
nite block Hankel matrices are avoided. The algorithm is based
on principal angles and directions. We have described a numerical
robust and fast way to calculate these on the basis of QR and
QSVD decompositions. We have also interpreted the principal
directions as non-steady state Kalman filter states. With a couple
of examples, we have showed that the new algorithm is superior to
the classical canonical correlation algorithms, especially for small
2.
References
[l] Akaike H. Markovian Representation of Stochastic processes by
Canonical Variables. SIAM Journal on Control, 13, 1, pp. 162-
173, January 1975.
[2] A m K.S., Kung S.Y. Balanced Aprozimation of Stochastic Sys-
tems. SIAM Matrix Analysis and Applications, Vol. 11, No l . , pp.
42-68, J ~ ~ u 1990.
~ I Y
(31 De Moor B., Golub G.H. Generalized singular value decomposi-
tions : A proposal for a standardized nomenclature. Numerical
Analysis Project Report 89-04, Department of Computer Science,
Stanford University, April 1989.
[4] Faure P. Stochastic realization algorithms. System identification
: Advances and case studies. Eds. Mehra R. and Lainiotis D.,
Academic Press, 1976.
[5] Hotelling H. Relations between two sets of variates. Biometrika,
V01.28, pp. 321-377, 1936.
[6] Jordan C. Essai sur la gkometrie d n dimensions. Bull. Soc. Math.
France, 3, 1875, pp. 103-174.
[7] Larimore W.E. Canonical Variate Analysis in Identification, Fd-
tering, and Adaptive Control. Proceedings of the 29th Conference
on Decision and Control, Hawaii, December 1990.
[a] Laub A.
A Schur meihod for solving algebraic Riccati equations.
IEEE Transactions on Automatic Control, AC-24, pp. 913-921,
1979.
[9] Ljung L. System Identification : theory for ihe user. Prentice-Hall
Information and System Sciences Series, 1987.
[lo] Moonen M., De Moor B., Vandenberghe L., Vandewde J. On- and
off-line identification of linear state space models. International
Journal of Control, Vol. 49, No. 1, pp. 219-232, January 1989.
[ll] Paige C.C., Saunders M.A. Towards a genedized singular value
decomposition. SIAM J. Numer. Anal., 18, pp. 398-405, (1981).
[12] Pal D. Balanced Stochastic Realization and Model Reduction. Mas-
ter's thesis, Whasington State University, Electrical Engineering,
[13] Van Loan C.F. Genemlizing the singular value decomposition.
SIAM J. Numer. Anal., 13, pp. 76-83, (1976).