We SPECIAL SECTION: COMPUTATIONAL SCIENCE ‘TUTORIALS An introduction to the proper orthogonal decomposition Anindya Chatterjee* Depatmest of Engineering ‘A tutorial Is presented on the Proper Orthogonal Decomposition (POD), which finds applications in computa tionally processing Inrge amounts of high-dimensional data with the alm of obtaining low-dimensional descriptions that capture much of the phenomena of interest. The dis- crete version of the POD, which is the singular value de- composition (SVD) of matrices, is described in some deta, ‘The continuous version of the POD is outlined, Low-rank approximations to data using the SVD are discussed, The SVD and the eigenvalue decomposition are compared. Two geometric Interpretations of the SVD/POD are given, Computational strategles (using standard software) are mentioned. Two numerical examples are provided: one shows low-rank approximations of a surface, and the other demonstrates simple a posteriori analysis of data from 2 simulated vibroimpaet system, Some relevant computer code Is supplied, 1, Introduction ‘ComPUTERS have increased our eapecity to not only sinn- late complicated systems, but also-to collect and analyse large amounts of data. Using personal computers, it is now unremarkable to record data at, say, 20 Kitz for afew hours then process hundreds of millions of data points few quantities of final interest. For example, ex pensive machinery might be instrumented and monitored lover days, with the sole objective of efficiently scheduling ‘maintenance. ‘This article provides an introduetion to the Proper Ontho- gonal Decomposition (POD) which is a powerful and elegant method of data analysis aimed at obtaining tow: dimensional approximate descriptions of high-dimensional processes. The POD was developed by several poople (among the first was Kosambi'), and is also known as Prin cipal Component Analysis, the Karhuen-Loéve Decomposition, and the single value decomposition. The POD has been wed to obtain approximate, low-dimensional descriptions of turbulent fluid flows%, structural vibra tions™, and insect gait’, and has been used for damage de- tection’ to name @ few applications in dynamic systems. ft mals anindva@erah em paueds 08 8 and Mechanis, Penn State Unversity, Univesity Pak, Peansylvans, PA, 16802, USA has also been extensively used in image processing, signal analysis and data compression, For references to the many sources of the POD, for applications of the POD in a variety of fclds, as well as for a nice treatment that complements this tutorial, the reader is encouraged to read Holmes, Lum Jey and Berkooz! (chapter 3). Data analysis using the POD is often conducted to ex: tract ‘mode shapes’ or basis functions, from experimental data of detailed simulations of high-i for subsequent use in Galerkin projections that yield low- dimensional dynamieal models (se ref. 2). This attcle con- centrates on the data analysis aspect, and subsequent ‘duced-order modelling is not discussed. 2. Motivation Suppose we wish to approximate a function 2(x,4) over some domain of interest as a finite sum in the variables: separated form tos Beaco, o withthe reasonable expectation thatthe approximation be- comes exact in the limit as M approaches infinity, except possibly on a set of measure zero (readers unfamiliar wi rmeasuce theory may ignore it if they deat with iite-dimen- sional calculations; snd consult, eg. Rudin” otherwise) ‘While in eq, (1) there is no’ fundamental difference be- tween f and x, we usually think of x as a spatial coordinate (possibly vector-valued) and think of fas a temporal coor dinate, ‘The repeesentation of eg (1) isnot unique. For example, if the domain ofx isa bounded interval X on the real line, then the functions g(x) can be chosen as a Fourier series, or Legendre polynomials, or Chebyshev polynomials, and so fon. For each such choice of a sequence d(x) that forms a basis for some suitable class of functions 2, # (See note 1), the sequence of time-functions a(t) is different. That is, for sines and cosines. «= we = get—one. sequence of funetions ay(0 is different. That is, for sines and cosines we get one sequence of functions ay(), for Legendre polynomials Wve get another, and so on, The POD. Js concetaed with one possible choice of the functions (a). ‘CURRENT SCIENCE, VOL. 78, NO. 7, 10 APRIL 2000,SPECIAL SECTION: COMPUTATIONAL SCIENCE If we have chosen orthonormal basis functions [acne ee one then 2400= [2000 (945, . {., for orthonormal basis funtion, the determination ofthe coefficient function a(?) depends only on gs) and not on the ather gs. ‘What erteria should we use for seleting the functions 42 Onhonormality would be useful. Moreover, while an approximation to any desired accwacy in eq. (1) can always be obtained if 2/ ean be chosen large enough, we may like to choose the 4G) in such a way that the pproximation foreach is a5 good as possible in. least squares sense. That is, we would try to find, onee and for all,asequence of orthonormal functions (2) such thatthe fiat two of these functions give the best possible two-term approximation, the first seven give the best possible sever- term approximation, and soon, These spacial ‘ordered, orthonormal functions are celled the proper arthogonal nodes forthe function 2(%,). With these fune- tions the expression ine (1) i ealled the POD of (0) 3. Theory: Finite-dimensional ease Consider a system where we take measurements of m state variables (these eould be from m strain gauges on a siruc- ture, or m velocity probes in a fig, or mixture of two kinds of probes in a system with flow-induced vibrations, te.) Assume that at W instants of time, we take N sets of simultaneous measurements at these o locations. We a- range out data in an N * m matrix A, suc that element Ay is the measurement from the jth probe taken at the ith tn instant. ‘The m state variables are nor assumed to be measured by transducers that are arranged in some straight line in physi- cal space. We merely assume that the transducers have ‘been numbered for identification; and that their outputs have beon placed side by side in the matrix 4. In the actual physical system these measurements might represent one spatial dimension (e.g. accelerometers on a beam), oF more than one spatial dimension (e.g. pressure probes ina three- dimensional fluid flow experiment). Each physical u ‘ducer may itself measure more than one scalar quantity (e.g, triaxial accelerometers); in such cases, the different sealar time series from the same physical transducer are arranged in different columns of 4. The final result of the data collection is assumed here tobe the > mati 4 ft is common to subtract, from each column of 4, the ‘mean value of that column. Whether or not this is done does not affect the basie calculation, though it affects the ‘CURRENT SCIENCE, VOL. 78, NO. 7, 10 APRIL. 2000, interpretation ofthe result (see section 3.5 below). 3.1 The singular value decomposition We now compute the singular value decomposition (SVD) ‘of the matrix A, which is of the form (fora discussion ofthe SVD, see ref. 8): AsuBy’, % where U is an N20 orthogonal mais, Vis an ann oF thogonal matrix, the superscript 7 indicates matrix tans- pose, and E is an Nm matrix with al elements zero except slong the diagonal. The diagonal elements 2, consist of ‘r= min(¥,m) nonnegative numbers cy, which are arranged in decreasing order, Le, = @z.,.2 Fz 0. The o's are called the singular values of 4 (and also of 4”) and ‘unique. The rank of 4 equals the number of nonzero singu- lar values it has. In the presence of noise, the number of singular values larger than some suitably small fraction of the largest singular value might be taken as the ‘aumerical rk’ Since the singular values are arranged in a specific ‘order, th index & ofthe kth singular value will be called the singular value number (see Figures | e and 2c), 3.2. Correspondence with egs (1) and (2) In eq, (3, let US=Q. Then the matrix Q is Nxm, and V7". Letting qu be the kth column of and vy be the kth column of F, we write out the matrix product as anor" = Soh ® Equation (4) is the dseote form of on. (1). The furtion 6,2) is ropresented ere by the mateix 4. The fune- tion ay) is represented by the colunsn matrix gu. The fune- tion ¢Gs) is represented by the row mati vf, The approx tation of eg. (1) is now exact because the dimension is it. Duet the oxthonormaity ofthe columns oF ¥, eg (2) oesponds to multiplying ea. (4) by one ofthe vs onthe viet 33. Lower-rank approximations 10 A For any 1)))¢0inG.91); subplot (3,2,1) urd (X,T,2) x19 (0,1,0,2,0,4,2.4]) xlabel('x!), ylabel(°t"), zlebel(/2!), titled Actual sucfa ‘CURRENT SCIENCE, VOL. 78, NO. 7,10 APRIL 2000 4% gonore the dave matrix & % plot the surface ") sis il‘SPECIAL SECTION: COMPUTATIONAL SCIENCE tu,s.vlaavacad; % note: the meana ara not aubtracted % trom the colums 4X for rank 1,2 and 3 approxinations 4% compare vith Bq. 6, and folloving text ad evcs 3 ubplot(3,2,ke1) surf (L,1,22) ,arts({0,1,0,2,0.4,2.11) mlabel x'), ylabel('t!), alebel Cs?) title ({/Rank muster (h),’ approxization’]) % plot the singular values, somilog acale xlabel Cmusber"), ylabel *aingular value’), titleC’Singular values of 2°) subplot(3,2,6) eit J drop ali bot first 3 colums of V oven; 4% nove, thin is the transpose of zev, which % —" corrooponds to Aev in the text phot tye(,1),!—? tye(2s 2,12? stacy xlabe1('t?}, "label nodal coordinates’ title('The soda contributions’) egend('t* 924,139) °) % plot nodal coordinates A2 From section 5.2 % data file containing time t, and ye[x,xdot) subplot(2,2,1) plot(t,x(: .4)) % plot dioplecenent of anos 4 nlabel (tine?) ,ylabel('dioplacenent?) title(?(a) divplacenent of uass 4?) axia([0,600,-3,17]) ssubplot(2,2,2 X start another displacenent plot plot (e (400026500) ,(4000:6600,2) ,'~. (40006600) (400016600, 5),’=") zlabeL('tine') jylabel(' displacement”) title? (>) dashedot: muss 2; solid Line: mass 6°) axia((340,456,-3,20]) for wat:10 2G ADex(:)k)-mean(xC,29);—% wenn subtraction ond fu,e,v}ava(x (20016000, % the second argunent (1.0., 0) in used for 4% far-tron-nquare watrices to save tino and menory mote: first 2000 pointa are dropped (transieats) stiag(s) 416 ‘CURRENT SCIENCE, VOL. 18, NO. 7,10 APRIL 2000‘SPECIAL SECTION: COMPUTATIONAL SCIENCE, subplot (2,2,8) sentlogy(s, 0!) 4 plot singular values xlabel (‘singular value musber'),ylabel (aingular value’) titte("e) singular values of soan-subtracted date’) axia((0,11,10-2,1640) supplee(2,2,4) plot (1:20, (5,4), 70~" 10evC 2,24" 121004 9,1 1% plot proper orthogens} modes wlabel "location (nase number) ') ,ylabel /tirat 3 proper orthogonal wodes") ‘title? (@) modes 1 (circles), 2 (otare) and 3 (triangles) xi0((0,11,-0.7,0.71) Notes 1. We assume these functions ae bounded and itereble, In exe periment, meusremants are Bounded and srt integration Is guivlent to summation; andthe wbtstis of iteration an ately be ignored. Heo, 1 stay away ffom Integration theory ‘The fnteresod render may conse. Rudin. 2. Strict peaking, one should sty ‘ank wi most 5 m> Nihon r= Me ther tem egoavsues but only 1 sloglar ‘ler the large 2 eigenetues equal the squats of he N sie- tule values; end the smallest 1 elgeavaoes are al exactly Zero Im 31, then rm, end the m eigenvalues eq the gutes ofthe m singular vues. (Readers my wish to Wore out the details of thee cleslations using = umercal example of a eee 1. Koshi D. D. Indlan Math, Soe, 1943, 7, 76-88, 2, Holmes P., Lmley, J. L,and Barkow, G. Turbulence. Coher- ‘ent Seactres, Dynamical Syeteme and Symmetry, Cambidge ‘Monogt. Mech, Cambridge Univesity Press, 1996, = Cusumano, J.P, Sharky, M. T. and Kimble, BW, Phils Trans. R See. London, See. A, 1994, 347, 421-38. Feary, BF and Kappagaoty, RJ Sotnd Vir, 1998, 218, 60 16, Koulschek, D. Schwing, W., Garcia, Mand Full, Ry 1999 (ranuroipt under propration) Ruta, R. and Siree, C, J. Sound Pir, 1999, 226, 425- a9. Rudin, W., Prniles of Mathematical Analyt, Internationa! Sere in Pare and Applied Mathers, MeGrn Hil, 1976, Sr en 5. Gol, GA, sed Van Loa, C.F Maris Compton, Johns Hopkins University Press, Baltimoce, 1990, 2nd ed. Powe, D. and Sting, D8. G., Integral Equations: 4 Practical Treatment, From Spectral They 10 Applications, Cambie ‘Texts in Applied. Mathematics, Cambridge University Pres, 1990, Ou, B, Chaos in Dynamical Sytem, Cambeldge Univesity res, 1993, Raviedra, B, J. Sound ib, 1999, 219, 189-192 CURRENT SCIENCE, VOL. 78,NO. 7, 10 APRIL. 2000 ar oy