0 views

Uploaded by FayazKhanPathan

save

- robust synthesis
- Near-synonym Lexical Choice in Latent Semantic
- illum_iccv2001
- Scheinine_2008_Infer Ring Gene Networks- Dream or Nightmare- Part 2- Challenges 4 and 5
- MIT18_06SCF11_Ses3.5sum.pdf
- MIMO Algebra
- Non-square matrix.pdf
- Review sheet
- linear-algebra-and-its-application.129.pdf
- SVD Presentation Updated2
- Final Prep
- third_sem
- MIT18_06SCF11_Ses3.5sum.pdf
- sk rd
- Pellegrino Calladine 1985 Matrix Analysis of Statically and Kinetically Indeterminate Frameworks
- 236899461-Solution-Manual-Linear-Algebra-and-Its-Applications-2ed.pdf
- Cheb2paper_chebfun Para 3 Dimensiones
- Applied Linear Algebra Review
- Math Exam
- Review of Linear Algebra
- 6 2 linear algebra
- 4
- 10.1.1.85
- applied-la-2013.pdf
- Svmtrain.m
- MA ECONOMICS 1ST SEMESTER
- Relay 1
- Systems Modeling and Model Reduction
- Chapter 1 Introduciton to Vectors
- Connor Ch5
- book-englishphonetics.pdf
- Clinical Usefulness of Laser Evoke 2003 Neurophysiologie Clinique Clinical N
- Principles-of-Electronics-V-K-Mehta-and-R-Mehta-pdf.pdf
- Writing_an_Abstract_Update_051112.pdf
- Linear Integrated Circuit 2nd Edition - D. Roy Choudhary.pdf
- Example Expressing Vector Fields with Coordinate Systems.pdf
- Elements of Style
- Shar Quie 2002
- Elements_of_Style.pdf
- Love
- EDC hand written theory notes of ACE.pdf
- CHEP Hospitol Procedure
- Fetel EEG Recordings
- Maxfilter User Guide
- Sound_Spelling_Chart.pdf
- What Are Neurotransmitters
- ElectrocardiographyCircuitDesign.pdf
- Action Potential Modelling of a Single Axon
- CHEP in Normal Subjects
- raspberri_sparkfun
- CHEP as a Valid Means to Study Nociceptive Pathways in Human Subjects
- Gardner Thesis
- EEG-fMRI Fusion_kalman Filter_inverse Problem
- Idioms and Phrases Capsule - PDF.pdf
- Discrete_PID.pdf
- PDF of 190 vocabulary words.pdf
- Thesis Troeman
- system identification.pdf
- Pid Tuning Rules for Second Order Systems
- shcams raspberri pi guide.pdf

You are on page 1of 5

**The singular value decomposition of a matrix is usually referred to as the SVD.
**

This is the ﬁnal and best factorization of a matrix:

A = UΣV T

where U is orthogonal, Σ is diagonal, and V is orthogonal.

In the decomoposition A = UΣV T , A can be any matrix. We know that if A

is symmetric positive deﬁnite its eigenvectors are orthogonal and we can write

A = QΛQ T . This is a special case of a SVD, with U = V = Q. For more general

A, the SVD requires two different matrices U and V.

We’ve also learned how to write A = SΛS−1 , where S is the matrix of n

distinct eigenvectors of A. However, S may not be orthogonal; the matrices U

and V in the SVD will be.

How it works

We can think of A as a linear transformation taking a vector v1 in its row space

to a vector u1 = Av1 in its column space. The SVD arises from ﬁnding an

orthogonal basis for the row space that gets transformed into an orthogonal

basis for the column space: Avi = σi ui .

It’s not hard to ﬁnd an orthogonal basis for the row space – the Gram-

Schmidt process gives us one right away. But in general, there’s no reason

to expect A to transform that basis to another orthogonal basis.

You may be wondering about the vectors in the nullspaces of A and A T .

These are no problem – zeros on the diagonal of Σ will take care of them.

Matrix language

The heart of the problem is to ﬁnd an orthonormal basis v1 , v2 , ...vr for the row

space of A for which

� � � �

A v1 v2 · · · vr = σ1 u1 σ2 u2 · · · σr ur

⎡ ⎤

σ1

� �⎢ σ2 ⎥

= u1 u2 · · · ur ⎢ ⎥,

⎢ ⎥

⎣ . .. ⎦

σr

with u1 , u2 , ...ur an orthonormal basis for the column space of A. Once we

add in the nullspaces, this equation will become AV = UΣ. (We can complete

the orthonormal bases v1 , ...vr and u1 , ...ur to orthonormal bases for the entire

space any way we want. Since vr+1 , ...vn will be in the nullspace of A, the

diagonal entries σr+1 , ...σn will be 0.)

The columns of U and V are bases for the row and column spaces, respec

tively. Usually U �= V, but if A is positive deﬁnite we can use the same basis

for its row and column space!

1

SVD example � � 4 4 We return to our matrix A = . 7 25 The eigenvectors of this matrix will give us the vectors vi . Since V is orthogonal. and the eigenvalues will gives us the numbers σi .. the ui are orthonormal. we do the same thing with AA T . 2 . V and Σ simultaneously. ⎣ . and positive numbers σ1 and σ2 so that the vi are orthonormal. This is a big step toward ﬁnding orthonormal matrices V and U and a di agonal matrix Σ for which: AV = UΣ. we can now ﬁnd V by diagonalizing the symmetric positive deﬁnite (or semideﬁnite) matrix A T A. u1 and u2 in the column space R2 .Calculation � � 4 4 Suppose A is the invertible matrix . and the σi are the scaling factors for which Avi = σi ui .) To ﬁnd U. we multiply both sides by A T = VΣ T U T to get: A T A = VΣU −1 UΣV T = VΣ2 V T ⎡ 2 ⎤ σ1 ⎢ σ22 ⎥ ⎥ T =V⎢ ⎥V . Rather than solving for U. ⎦ σn2 This is in the form QΛQ T . (We choose σi to be the positive square root of λi . we can multiply both sides by V −1 = V T to get: A = UΣV T . We want to ﬁnd vectors v1 −3 3 and v2 in the row space R2 . We start by computing −3 3 � �� � T 4 −3 4 4 A A = 4 3 −3 3 � � 25 7 = . The columns of V are eigenvec tors of A T A and the eigenvalues of A T A are the values σi2 . ⎢ .

−3 3 0 3 2 1/ 2 −1/ 2 We could solve this for U.6 3 . but because Av2 = we 1 −3 2 � � � � 0 1 0 instead have u2 = and U = . � � � � 1 1 Two orthogonal eigenvectors of A T A are and . � � . AA T happens to be diagonal.8 Unit basis vectors of the row and column spaces are v1 = and u1 = . as Professor Strang did in the lecture. −3 3 0 −1 0 3 2 1/ 2 −1/ 2 Example with a nullspace � � 4 3 Now let A = . the SVD of A is: √ Σ A U T � � � � � � � √ V √ � 4 4 1 0 4 2 √0 1/√2 1/√2 = . This has a one dimensional nullspace and one dimen 8 6 sional row and column spaces. � � 4 The row space of A consists of the multiples of . � �� � T 4 4 4 −3 AA = −3 3 4 3 � � 32 0 = . It’s tempting to let u1 = and u2 = 0 � � � � 0 √ 0 . Note that this also gives us a −1 0 −1 chance to double check our calculation of σ1 and σ2 . The column space � � 3 4 of A is made up of multiples of . but for practice we’ll ﬁnd U by ﬁnding orthonor mal eigenvectors u1 and u2 for AA T = UΣ2 U T . let v1 = and v2 = . The nullspace and left nullspace are 8 perpendicular to the row and column spaces. We now have: √ Σ A U T � � � � � � � √ V √ � 4 4 4 2 √0 1/√2 1/√2 = . To get an or 1 −1 � √ � � √ � 1/√2 1/√2 thonormal basis. 0 18 � � 1 Luckily. These have eigen 1/ 2 −1/ 2 values σ12 = 32 and σ22 = 18. Thus. respectively.

8 .� √ � 1/√5 . 4 . These are the “right” bases to use. vr+1 .vn is an orthonormal basis for the nullspace. .vr is an orthonormal basis for the row space. .6 = √1 .. because Avi = σi ui .. . ur+1 ... After ﬁnding unit vectors perpendicular to u1 and v1 (basis vectors for the left nullspace and nullspace. v2 . 2/ 5 � �� � 4 8 4 3 A T A = 3 6 8 6 � � 80 60 = .. u2 .6 −. one eigenvalue must be 0. To compute σ1 we ﬁnd the nonzero eigenvalue of A T A. u1 ..um is an orthonormal basis for the left nullspace. The other must equal the trace..ur is an orthonormal basis for the column space.8 A U Σ VT The singular value decomposition combines topics in linear algebra rang ing from positive deﬁnite matrices to the four fundamental subspaces. . 8 6 5 2 −1 0 0 . 60 45 Because this is a rank 1 matrix. respectively) we see that the SVD of A is: � � � � � √ � � � 4 3 1 2 125 0 . so σ12 = 125. v1 ..

06SC Linear Algebra Fall 2011 For information about citing these materials or our Terms of Use.edu/terms.mit.edu 18. .mit. visit: http://ocw.MIT OpenCourseWare http://ocw.

- robust synthesisUploaded byioncopae
- Near-synonym Lexical Choice in Latent SemanticUploaded byShan
- illum_iccv2001Uploaded bysandeep0029
- Scheinine_2008_Infer Ring Gene Networks- Dream or Nightmare- Part 2- Challenges 4 and 5Uploaded byEnrico Pieroni
- MIT18_06SCF11_Ses3.5sum.pdfUploaded byFilozófus Önjelölt
- MIMO AlgebraUploaded byngduyhieu
- Non-square matrix.pdfUploaded byatto_11
- Review sheetUploaded byninnguyen
- linear-algebra-and-its-application.129.pdfUploaded byraqibapp
- SVD Presentation Updated2Uploaded byShirshendu Roy
- Final PrepUploaded byŞamil Okan
- third_semUploaded byRizwan Islam
- MIT18_06SCF11_Ses3.5sum.pdfUploaded bybarabara
- sk rdUploaded byShashikant Chaudhary
- Pellegrino Calladine 1985 Matrix Analysis of Statically and Kinetically Indeterminate FrameworksUploaded bycolkurtz21
- 236899461-Solution-Manual-Linear-Algebra-and-Its-Applications-2ed.pdfUploaded byBqsauce
- Cheb2paper_chebfun Para 3 DimensionesUploaded byJuan Antonio Zurera Durán
- Applied Linear Algebra ReviewUploaded byEllie
- Math ExamUploaded byPoulaElhamyMondyBasaly
- Review of Linear AlgebraUploaded byRichard Ding
- 6 2 linear algebraUploaded byapi-195736626
- 4Uploaded byMerlin Gilbert Raj
- 10.1.1.85Uploaded byRizka Fajarini
- applied-la-2013.pdfUploaded byImad Siddiqui
- Svmtrain.mUploaded bylizhi0007
- MA ECONOMICS 1ST SEMESTERUploaded byNaresh Sehdev
- Relay 1Uploaded byjimmydafe
- Systems Modeling and Model ReductionUploaded byAnubi Olugbenga Moses
- Chapter 1 Introduciton to VectorsUploaded byLeepsteer Loo
- Connor Ch5Uploaded byAmesh Anvekar

- book-englishphonetics.pdfUploaded byFayazKhanPathan
- Clinical Usefulness of Laser Evoke 2003 Neurophysiologie Clinique Clinical NUploaded byFayazKhanPathan
- Principles-of-Electronics-V-K-Mehta-and-R-Mehta-pdf.pdfUploaded byFayazKhanPathan
- Writing_an_Abstract_Update_051112.pdfUploaded byFayazKhanPathan
- Linear Integrated Circuit 2nd Edition - D. Roy Choudhary.pdfUploaded byFayazKhanPathan
- Example Expressing Vector Fields with Coordinate Systems.pdfUploaded byFayazKhanPathan
- Elements of StyleUploaded byFayazKhanPathan
- Shar Quie 2002Uploaded byFayazKhanPathan
- Elements_of_Style.pdfUploaded byFayazKhanPathan
- LoveUploaded byFayazKhanPathan
- EDC hand written theory notes of ACE.pdfUploaded byFayazKhanPathan
- CHEP Hospitol ProcedureUploaded byFayazKhanPathan
- Fetel EEG RecordingsUploaded byFayazKhanPathan
- Maxfilter User GuideUploaded byFayazKhanPathan
- Sound_Spelling_Chart.pdfUploaded byFayazKhanPathan
- What Are NeurotransmittersUploaded byFayazKhanPathan
- ElectrocardiographyCircuitDesign.pdfUploaded byFayazKhanPathan
- Action Potential Modelling of a Single AxonUploaded byFayazKhanPathan
- CHEP in Normal SubjectsUploaded byFayazKhanPathan
- raspberri_sparkfunUploaded byFayazKhanPathan
- CHEP as a Valid Means to Study Nociceptive Pathways in Human SubjectsUploaded byFayazKhanPathan
- Gardner ThesisUploaded byFayazKhanPathan
- EEG-fMRI Fusion_kalman Filter_inverse ProblemUploaded byFayazKhanPathan
- Idioms and Phrases Capsule - PDF.pdfUploaded byFayazKhanPathan
- Discrete_PID.pdfUploaded byFayazKhanPathan
- PDF of 190 vocabulary words.pdfUploaded byFayazKhanPathan
- Thesis TroemanUploaded byFayazKhanPathan
- system identification.pdfUploaded byFayazKhanPathan
- Pid Tuning Rules for Second Order SystemsUploaded byFayazKhanPathan
- shcams raspberri pi guide.pdfUploaded byFayazKhanPathan