MIT18_335JF10_mid_sol_f08

Attribution Non-Commercial (BY-NC)

4 views

MIT18_335JF10_mid_sol_f08

MIT18_335JF10_mid_sol_f08

Attribution Non-Commercial (BY-NC)

- Solution of Linear System Theory and Design 3ed for Chi-Tsong Chen
- Linear Algebra
- Matlab Compendium - Control
- Qingwen Hu-Concise Introduction to Linear Algebra-Chapman and Hall_CRC (2017)
- Some Notes on Least Squares, QR-factorization, SVD and Fitting
- Lectures 1 - ME 340
- Chomba Bupe - Quora
- Kernel Matlab
- Vector Spaces
- Using Singular Systems Analysis to Characterise the Flow in the Wake of a Model Passenger Vehicle
- Basic Matrix Operations
- mehdi
- MST209_2005solutions
- Project Report
- IEEESP.pdf
- Parte Terza A
- MMA-2018
- 1-s2.0-S0019357716300313-main.pdf
- Vectors Matrices
- ShaweTaylorCristianini_pdf

You are on page 1of 4

Problem 1: Schur, backsubstitution, complexity (20 points)

You are given matrices A (m m), B (n n), and C (m n), and want to solve for an unknown matrix X (m n) solving: AX XB = C. We will do this using the Schur decompositions of A and B. (Recall that any square matrix S can be factorized in the Schur form S = QUQ for some unitary matrix Q and some upper-triangular matrix U .)

(a) AX XB = C = QAUA Q A X XQBUB QB , and hence (multiplying on the left by QA and on the right by QB ), we obtain UA Q A XQB QA XQBUB = QACQB , so A = UA , B = UB , C = Q ACQB , and X = QA XQB . To get X from X , we obtain X = QA X QB .

A mm Xm,: Xm,: B = Cm,: = Xm,: (Amm I B ), of X . To nd this last row, then, we merely need to solve the which is only in terms of the last row Xm ,: system of equations abovesince Amm I B is upper-triangular, we can do this by backsubstitution in O(n2 ) operations. Or, I guess, technically, this is forward substitution because you start with the rst column of B and move right, but whateverits the same thing under a permutation. [Although this is a row-vector problem, we can obviously transpose to get the familiar column-vector problem, T in which case (A mm I B ) is lower-triangular.]

(c) More generally, the j-th row of A X X B = C can be written purely in terms of the j-th and later rows of X , since A is upper-triangular: Aj j X j,: + Aji Xi,: X j,: B = Cj,:

i> j

and hence

i> j

which is again an upper-triangular system of equations. It takes 2(m j)n operations to construct the right-hand side, and O(n2 ) operations to solve by backsubstitution. (d) We have to solve for m rows. Each of them requires an O(n2 ) backsubstitution, for O(mn2 ) operations. 2 There are also m j=1 2(m j )n = O(m n) ops to compute the right-hand sides. Finally, to compute 2 2 X = QA X Q B requires two matrix multiplies, for 2m n + 2mn ops. So, the total complexity is 2 2 3 3 O(m n) + O(mn ), not including the O(m ) + O(n ) time for the Schur factorizations.

Since it is backwards stable (with respect to A and/or b), we obtain an x + x such that (A + A)(x + x) = b + b A(x + x) + A x, where A = O(machine )A and b = O(machine )b. That means that the residual, computed in exact arithmetic, would r = b A(x + x) = A x = A x b. The norm of this is A x + b A x + b = [A x + b]O(machine ). But x = A1 b A1 b, and so we obtain r [ (A) + 1]bO(machine ). However, I didnt specify whether the backwards stability was with respect to A or b; if you only assumed the latter you wouldnt have gotten the (A) term. This is still not quite right, however, if the residual r itself is computed in oating-point arithmetic. In particular, the computation of b Ay in oating-point for any y is also backwards stable with respect 1

to y, so in computing b A(x + x) we obtain b A(x + x + x ) where x = xO(machine ) A1 bO(machine ). Hence, this gives us an additional term A x in the residual, which has magnitude A x (A)bO(machine ). Adding these two sources of error, we obtain a residual whose magnitude proportional to (A)bO(machine ).

(a) CG does not change the component of xn in the nullspace (the span of the zero- eigenvectors). Proof: If we expand x j = i i qi in the eigenvectors qi with some coefcients i , we see im( j) mediately that Ax j = i>k i i qi is in the span of the nonzero- eigenvectors of A; equivalently, it is perpendicular to the nullspace. Hence, the residual r j = b Ax j (which we compute by recurrence in the CG algorithm) is also perpendicular to the nullspace. Since all the residuals are perpendicular to the nullspace, and since the directions d j are linear combinations of the residuals (via Gram-Schmidt), the directions d j are also perpendicular to the nullspace. Hence, when we compute xn = xn1 + n dn1 , (n) (n1) for i k. we do not change the components of x in the nullspace, and i = i (b) Because CG only changes xn in directions perpendicular to the nullspace, it is equivalent to doing CG on the nonsingular problem of Ax = b acting within the column space of A. Since x0 = 0, it initially has no (nonzero) component in the nullspace and hence xn has no component in the nullspace. Hence, i if b = i>k i qi for some coefcients i , it converges to xn i>k i qi . The rate of convergence is determined by the square root of the condition number of A within this subspace, i.e. at worst the convergence requires O( m /k+1 ) iterations, assuming we have sorted the j s in increasing order. (Not including possible superlinear convergence depending on the eigenvalue distribution.) (c) If we choose the initial guess x0 = 0, it will still converge, but it may just converge to a different solutionthe component of x0 in the nullspace has no effect on CG at all, and the component in the column space is just a different starting guess for the nonsingular CG in the subspace. That is, since the component ik i qi of x0 in the nullspace is not changed by CG, we will get (in the notation i above) xn ik i qi + i>k i qi . (d) Just set b = 0 and pick x0 to be a random vector, and from above it will converge to a vector in the nullspace in O( m /k+1 ) iterations at worst.

( j) ( j)

) q1 , in which case the 0 Rayleigh quotient is r(x) = 1 by inspection, and since this is an upper bound for the smallest eigenvalue of A, we are done. Let the smallest- eigensolution of B be B1 = 1 q1 where q 1 q1 = 1. Let x =

( ) 1 b If B were just a real number b, this would be a 2 2 matrix A = , which has eigenvalues 1 b b 1 ( ) 1 for eigenvectors . We would immediately obtained the desired result since B = |b| and A2 is 1 the ratio of the maximum to the minimum eigenvalue. Now, we just want to use a similar strategy for the general case where B is m n, where from the SVD we can write: ( ) ( ) I U V I B A= = . B I V T U I 2

That is, we expect to get combinations of eigenvectors of B. For simplicity, lets start with the case where B is square m m, in which case = T (diagonal) and ( ) U U and V are all m m. In this case, consider the vectors corresponding to the columns of X = . In V this case, ( )( ) ( ) I U V U U U AX = = = X (I ), V U I V V V Since the matrix at right is diagonal, this means that the columns of X are eigenvectors of A, with eigenvalues 1 i where i are the singular values of B (possibly including some zeros from the diagonal of if B is not full rank). These are, moreover, all of the 2m eigenvalues of A. Since A is Hermitian, eigenvalues are the same thing as the singular values, and hence the maximum singular value of A is 1 + max i and the minimum is 1 max i (since we are given that B2 < 1 and hence i < 1), and hence (A) = (1 + max i )/(1 max i ) = (1 + B2 )/(1 B2 ). Q.E.D. What about the case where B is not square? Suppose m > n, in which case U is bigger than V so it doesnt make sense to write X as above. However, there is a simple x. In the denition of X , just pad V with m n columns of zeros to make an n m matrix V0 . Then V V0 is the n n identity matrix plus m n columns of zeros. Then we get ( )( ) ( ) I U V U U 0 U AX = = = X (I 0 ), V0 V T U I V T V0 where 0 is padded with m n columns of zeros to make a diagonal m m matrix, noting that V T = V0 T 0 = V0 0 . The result follows as above. If m < n, the analysis is similar except that we pad U with n m columns of zeros.

We want to minimize (Ax b)W (Ax b). The best thing to do is to turn this into a regular least-squares problem by breaking W in halves and putting half on the left and half on the right. For example, we can compute the Cholesky factorization W = R R, and then we are minimizing (RAx Rb) (RAx Rb), which is equivalent to solving the least-squares problem for RA and Rb. This we could do, e.g., by computing the QR factorization RA = Q R , and then solve R x = Q Rb by backsubstitution. None of these steps has any particular accuracy problems. Of course, there are plenty of other ways to do it. You could also compute W by diagonalizing W = QQ and then using W = Q Q . This might be a bit more obvious if you have forgotten about Cholesky. Again solving the least-squares problem with W A and W b, this works, but is a bit less efcient because eigenproblems take many more operations than Cholesky factorization. We could also write down the normal equations AWAx = AW b, derived from the gradient of (Ax b) W (Ax b) with respect to x. However, solving these directly sacrices some accuracy because (as usual) it squares the condition number of A.

Fall 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

- Solution of Linear System Theory and Design 3ed for Chi-Tsong ChenUploaded byQingyu Meng
- Linear AlgebraUploaded byLã Phụng Tiên
- Matlab Compendium - ControlUploaded byDaniel Ariano
- Qingwen Hu-Concise Introduction to Linear Algebra-Chapman and Hall_CRC (2017)Uploaded byStjepkoKatulić
- Some Notes on Least Squares, QR-factorization, SVD and FittingUploaded bymatteyas
- Lectures 1 - ME 340Uploaded byLodu Cent
- Chomba Bupe - QuoraUploaded bybarber bob
- Kernel MatlabUploaded byivan gonzalez castillo
- Vector SpacesUploaded byAnonymous sJV1DacWRy
- Using Singular Systems Analysis to Characterise the Flow in the Wake of a Model Passenger VehicleUploaded byVyssion
- Basic Matrix OperationsUploaded byAgusPrianto
- mehdiUploaded byThanigai Vel
- MST209_2005solutionsUploaded byrashismart2000
- Project ReportUploaded byArjun VS
- IEEESP.pdfUploaded byali
- Parte Terza AUploaded byPasquale Leone
- MMA-2018Uploaded byNikhil Yenumula
- 1-s2.0-S0019357716300313-main.pdfUploaded bySuntood
- Vectors MatricesUploaded bySanjeev Neo Jaiswal
- ShaweTaylorCristianini_pdfUploaded byMartine Dumont
- Procrustes Distance READMEUploaded byMirQuinto
- 1620141009_ftpUploaded bykramaseshan
- Differential EquationUploaded byTarek Mohamed
- Factor AnalysisUploaded byRajendra Kumar
- 01656700Uploaded byPrashanth Prashu
- Quantum Algorithm for LSOEUploaded byÇalanyalıoğlu
- Chapter 0Uploaded byHassan Ajrouche
- MAT1341_TT2_2012WUploaded byexamkiller
- Chapter 5 SlidesUploaded bySingh Karan
- Unit 7_ Homework-MUBARAK NALF 25.pdfUploaded byom

- Some Useful Linux CommandsUploaded byVIJAYPUTRA
- L111Uploaded byVIJAYPUTRA
- LiiUploaded byVIJAYPUTRA
- InfoUploaded byVIJAYPUTRA
- Phase Margin and Quality FactorUploaded byVIJAYPUTRA
- LorenzUploaded byVIJAYPUTRA
- New Text Document.txtUploaded byVIJAYPUTRA
- LanguageUploaded byVIJAYPUTRA
- Verilog PWM Switch Models2Uploaded byVIJAYPUTRA
- Corrections 1st Print Loop ControlUploaded byVIJAYPUTRA
- New Text Document.txtUploaded byVIJAYPUTRA
- Unix BookUploaded byVasanth
- tcsv_fin.pdfUploaded byVIJAYPUTRA
- Python 2Uploaded byrathod100
- univ - CopyUploaded byVIJAYPUTRA
- PRQA-QAVerifyUploaded byVIJAYPUTRA
- Verilog PWM Switch ModelsUploaded byVIJAYPUTRA
- 00931078Uploaded byVIJAYPUTRA
- cast-2Uploaded byVIJAYPUTRA
- cast-1Uploaded byVIJAYPUTRA
- 30-21028Uploaded byVIJAYPUTRA
- 3xnextbifurUploaded byVIJAYPUTRA
- BAN105_1.2Uploaded byVIJAYPUTRA
- MenuUploaded byVIJAYPUTRA
- Mae 342 Lecture 17Uploaded byVIJAYPUTRA
- 9023Uploaded byVIJAYPUTRA
- Teppo LuukkonenUploaded byVIJAYPUTRA
- Installing Software on Linux — Guide to Linux for BeginnersUploaded byVIJAYPUTRA
- Table of ContentsUploaded byClunche

- Kirchhoffs LawUploaded byraumil
- Bohr.docxUploaded byMuhammad Awais Sultani
- ME3112 Help SheetUploaded byPS Chua
- Lab. Conjugated DyesUploaded byIreneVelado
- Physics of Semiconductor DevicesUploaded byGiorgos Papageorgiou
- Atomic Structure 1 (1)Uploaded byRohman Entino
- Quantum ChemistryUploaded byVijay Pradhan
- marine hydrodynamicsUploaded byAnu Parameswaran
- Fotini Markopoulou- Quantum Gravity and Emergent LocalityUploaded byKuiasm
- vaasrevUploaded byGopalBanik
- Mechanical Behavior of Materials 01 PDFUploaded byjunee
- <HTML><HEAD><TITLE>Access Denied</TITLE></HEAD> <BODY>Your page is blocked due to a KIET IT security policy that prohibits access to Category default <BR> </BODY></HTML>Uploaded byAshutosh Chandel
- UniverseUploaded bymohan
- Gravity and Coulombs LawUploaded byAnshu Girdhar
- DIFFERENTIAL ANALYSIS CFDUploaded byotonbara
- apset 2012 3Uploaded byVeerareddy Vippala
- Newman–Penrose formalismUploaded byAnonymous kg7YBMFH
- matrices and determinants notes.pdfUploaded byayush mittal
- 0691004358 MechanicsUploaded byvishrantanand
- The Laplace transform in control theory.pdfUploaded byflausen
- Chemistry-4-Electron Sructure of Atom Student NoteUploaded byam2030
- 1213midterm.pdfUploaded byEric Yeung
- Cap 2 SolucionarioUploaded byPaul Corella
- EsrUploaded byShweta
- SpectralTheory LNUploaded byRamón Marcelo Bustos Méndez
- Afriat, Alexander and Caccese, Ermenegildo (2009) the Relativity of Inertia and Reality of NothingUploaded byTommaso Cimino
- chapter 2Uploaded byAnonymous bJV1MVb
- Modern Physics for Scientists and Engineers SolutionsUploaded byJosé Pedro Casagrande Trentín
- Engg mathematics - 1 Dec 2012.pdfUploaded byPrasad C M
- On The Darboux Vector Belonging To Involute Curve A Different ViewUploaded byMia Amalia