27 views

Original Title: Singular Value Decomposition Example

Uploaded by dev1712

- SSRN-id3308964
- Numerical Linear Algebra with Applications Syllabus
- Free
- Chapter 4
- Computational Techniques
- 5650_1MatrixAlgebra
- Anastasia Doikou- A note on the boundary spin s XXZ chain
- Rational Expectations
- PartI-Problem Set Matrices
- Ms Capp Math
- Compre
- c Bo 9780511624117 a 015
- 00b4952a6fbc9d7486000000
- Daniel Stereo
- ch1
- Solution of Linear System Theory and Design 3ed for Chi-Tsong Chen
- noise ppt
- TRATAMIENTO_DE_AGUAS_RESIDUALES_DE_USO_AGRICOLA_EN.pdf
- taylor series
- UT Dallas Syllabus for engr2300.001.10f taught by Carlos Busso Recabarren (cxb093000, cxb093000)

You are on page 1of 5

Paul Skoufranis

March 11, 2010

We shall explore a concept known as the singular value decomposition of a matrix.

Denition Let A be an n m matrix. The singular values of A are the square roots of the eigenvalues of

m m matrix A

T

A listed according to their algebraic multiplicity. We should note that the eigenvalues of

A

T

A will always be positive numbers so it makes sense to take the square root. It is customary to denote

the singular values by

1

,

2

, . . .,

m

with

1

2

m

.

Theorem Any n m matrix A can be written in the form A = UV

T

where U is an orthogonal n n

matrix, V is an orthogonal mm matrix, and is an n m matrix whose rst r diagonal entries are the

non-zero singular values

1

,

2

, . . .,

r

of A and whose other entries are all zeros. The expression UV

T

is

known as the Singular Value Decomposition of A.

Now we shall go through an example of computing the Singular Value Decomposition (SVD) of a ma-

trix A, why it works, and how we can use the SVD to determine the action of A on the unit circle x

2

+y

2

= 1.

Example Compute the SVD of the matrix

A =

_

_

1 1

1 1

3 0

_

_

and use this decomposition to describe the action of A on the unit circle.

Solution: The rst step in nding the SVD of A is to nd the singular values of A. To do this, we

simply need to compute the eigenvalues of A

T

A. Thus let us rst compute A

T

A;

A

T

A =

_

1 1

3

1 1 0

_

_

_

1 1

1 1

3 0

_

_

=

_

5 2

2 2

_

Therefore the characteristic polynomial of A

T

A is

f

A

T

A

() = det(A

T

AI) = (5 )(2 ) 4 =

2

7 + 6 = ( 1)( 6)

Therefore the eigenvalues of A

T

A are 1 and 6. Thus the singular values of A are

1

= 1 and

2

=

6 (in

general, at this step you would take the square root of each eigenvalue to get a singular value and include

it am() times). This tells us that the matrix will be the 3 2 matrix

=

_

_

6 0

0 1

0 0

_

_

where we place

1

=

2

= 1 in the second column since

6 > 1 (normally, if

1

2

r

> 0 are your non-zero singular values (including repetitions), we would place these

entries along the diagonal of and place zeros elsewhere).

Our next goal is to determine the matrix V . We know that since A

T

A is a symmetric matrix, A

T

A will

have an orthonormal basis of eigenvectors. It turns out that the matrix V will be 22 matrix whose columns

are an orthonormal eigenbasis for A

T

A. Therefore we compute the eigenspaces of A

T

A. We notice that

E

6

= ker(A

T

A6I) = ker

__

1 2

2 4

__

= ker

__

1 2

0 0

__

= span

_

1

5

_

2

1

__

and

E

1

= ker(A

T

AI) = ker

__

4 2

2 1

__

= ker

__

1

1

2

0 0

__

= span

_

1

5

_

1

2

__

Let

v

1

=

1

5

_

2

1

_

and v

2

=

1

5

_

1

2

_

Then we let V be the 2 2 matrix

V =

_

v

1

v

2

=

_

2

5

1

5

2

5

_

where we place the eigenvector for the eigenvalue 6 in the rst column and the eigenvector for the eigenvalue

1 in the second column since we placed

Now our question is how do we nd the orthogonal 3 3 matrix U? Well, if we could nd an orthogonal

3 3 matrix U such that AV = U, we would be done since V

1

= V

T

as V is an orthogonal matrix. Let

us consider the actions of AV and on the standard basis e

1

and e

2

of R

2

. Well

_

1

0

_

=

1

_

_

1

0

0

_

_

_

0

1

_

=

2

_

_

0

1

0

_

_

and

AV

_

1

0

_

= Av

1

AV

_

0

1

_

= Av

2

Therefore, if U were an orthogonal 3 3 matrix such that AV = U and u

1

, u

2

, and u

3

were the columns

of U, then {u

1

, u

2

, u

3

} would be an orthonormal basis of R

3

and

Av

1

= AV

_

1

0

_

= U

_

1

0

_

=

6U

_

_

1

0

0

_

_

=

1

u

1

and

Av

2

= AV

_

0

1

_

= U

_

0

1

_

=

1U

_

_

0

1

0

_

_

=

2

u

2

Thus we should let

u

1

=

1

1

Av

1

=

1

6

_

_

1 1

1 1

3 0

_

_

_

1

5

_

2

1

__

=

1

30

_

_

3

3

2

3

_

_

and

u

2

=

1

2

Av

2

=

1

1

_

_

1 1

1 1

3 0

_

_

_

1

5

_

1

2

__

=

1

5

_

_

1

1

3

_

_

2

If we check, we see that {u

1

, u

2

} are indeed orthonormal vectors. The question is, why does this work?

Well, just using the facts that u

1

=

1

1

Av

1

, u

2

=

1

2

Av

2

, A

T

Av

1

=

2

1

v

1

(since v

1

is an eigenvector for A

T

A

with eigenvalue

2

1

), A

T

Av

2

=

2

2

v

2

(v

2

is an eigenvector for A

T

A with eigenvalue

2

2

), and {v

1

, v

2

} is an

orthonormal set, we see that

u

1

u

1

=

_

1

1

Av

1

_

_

1

1

Av

1

_

=

_

1

1

Av

1

_

T

_

1

1

Av

1

_

=

1

2

1

v

T

1

A

T

Av

1

=

1

2

1

v

T

1

(A

T

Av

1

)

=

1

2

1

v

T

1

(

2

1

v

1

)

= v

T

1

v

1

= v

1

v

1

= 1

and

u

2

u

2

=

_

1

2

Av

2

_

_

1

2

Av

2

_

=

_

1

2

Av

2

_

T

_

1

2

Av

2

_

=

1

2

2

v

T

2

A

T

Av

2

=

1

2

2

v

T

2

(A

T

Av

2

)

=

1

2

2

v

T

2

(

2

2

v

2

)

= v

T

2

v

2

= v

2

v

2

= 1

and

u

1

u

2

=

_

1

1

Av

1

_

_

1

2

Av

2

_

=

_

1

1

Av

1

_

T

_

1

2

Av

2

_

=

1

2

v

T

1

A

T

Av

2

=

1

2

v

T

1

(A

T

Av

2

)

=

1

2

v

T

1

(

2

2

v

2

)

=

2

1

v

T

1

v

2

=

2

1

v

1

v

2

= 0

3

which is want we wanted.

However, to nd an orthogonal matrix using u

1

and u

2

, we need a third orthonormal vector as we need

U to be a 3 3 matrix and we only have two orthonormal vectors (in some cases, you will not need this

step). To nd u

3

, write

u

3

=

_

_

a

b

c

_

_

Then we want a

2

+ b

2

+ c

2

= 1 so u

3

is unit length, and we want

0 = u

1

u

3

=

1

30

(3a + 3b + 2

3c)

and

0 = u

2

u

3

=

1

5

(a b +

3c)

By solving a system of linear equations, we obtain that

_

_

1

1

0

_

_

where R is the set of all solutions to the equations 0 =

1

30

(3a +3b +2

3c) and 0 =

1

5

(a b +

3c).

Therefore, if we let u

3

=

1

2

_

_

1

1

0

_

_

, u

3

is a normal vector that is orthogonal to u

1

and u

2

. Thus {u

1

, u

2

, u

3

}

is an orthonormal basis for R

3

. Then, if we let

U =

_

u

1

u

2

u

3

=

_

_

3

30

1

5

1

2

3

30

1

5

1

2

2

10

5

0

_

_

then U is an orthogonal matrix and A = UV

T

as desired. If you are not condent in your computations,

you could easily check your answer by multiplying this out.

Now we desire to determine the action of A on the unit circle. To see this, we will rst apply V

T

to the

unit circle, then to that image, and then we will apply U. The following is a diagram representing the

actions of each map:

4

In the diagram, we start with the top left diagram. The vector v

1

and its images are shown in blue and the

vector v

2

and its images are shown in red. First we know that V take e

1

to v

1

and e

2

to v

2

. Since V

T

= V

1

as V is an orthogonal matrix, V

T

takes v

1

to e

1

and v

2

to e

2

. Then adds a z-axis and dilates the x-axis by

6. Then U rotates the whole diagram in an awkward manner (you probably do not need to know exactly

what this rotation is but, it takes the x-axis to the line through u

1

, it takes the y-axis to the line through

u

2

, and it takes the z-axis to the line through u

3

). It is hard to see what the curve in the nal diagram looks

like. The blue line goes to the point

30

(3, 3, 2

1

5

(1, 1,

3).

5

- SSRN-id3308964Uploaded bySuresh Panda
- Numerical Linear Algebra with Applications SyllabusUploaded bySankalp Chauhan
- FreeUploaded byGaurav Kumar Singh
- Chapter 4Uploaded byHamed Nikbakht
- Computational TechniquesUploaded byAriel Alejandro Acosta
- 5650_1MatrixAlgebraUploaded byYagya Raj Joshi
- Anastasia Doikou- A note on the boundary spin s XXZ chainUploaded byPo48HSD
- Rational ExpectationsUploaded bykshitijsaxena
- PartI-Problem Set MatricesUploaded byAbhijit
- Ms Capp MathUploaded bySubhajit Das
- CompreUploaded byKriti Goyal
- c Bo 9780511624117 a 015Uploaded bystefan
- 00b4952a6fbc9d7486000000Uploaded byquantack
- Daniel StereoUploaded byVan Duc
- ch1Uploaded byspamjimteh
- Solution of Linear System Theory and Design 3ed for Chi-Tsong ChenUploaded byadambose1990
- noise pptUploaded bypriyankar
- TRATAMIENTO_DE_AGUAS_RESIDUALES_DE_USO_AGRICOLA_EN.pdfUploaded byJuan
- taylor seriesUploaded byIan Ridzuan
- UT Dallas Syllabus for engr2300.001.10f taught by Carlos Busso Recabarren (cxb093000, cxb093000)Uploaded byUT Dallas Provost's Technology Group
- case3_1Uploaded bySandip Gaikwad
- lay-optUploaded byduyhoang2004
- MIT18 06S10 Final AnswersUploaded byandreaskailina
- CSD Project ReportUploaded byAkash
- Analyzing biosignals using the R freewareUploaded byergoshaun
- Integrable and NonintegrableUploaded bydapias09
- Algebra of Matrices Question Bank Maths lUploaded bydadajee420
- D1077d01Uploaded byalissonguimaraes
- Modern Control TheoryUploaded byA.Subburayan
- NUMERICAL SYNTHESIS DESIGN OF COUPLED RESONATOR FILTERSUploaded byabaidullah

- ijwin02Uploaded bydev1712
- Borel SigmaUploaded bydev1712
- 9 Laplace z TableUploaded bydev1712
- supremum and inﬁmumUploaded byJessica Henderson
- 4.4 BodeUploaded bydev1712
- ProbabilityUploaded bydev1712
- Laplace TablesUploaded bydev1712
- Bhagvat GitaUploaded bySupS.Peace
- random variableUploaded bydev1712
- BER and SNRUploaded byshervinsh
- 8.7 BesselUploaded bydev1712
- Important TablesUploaded bydev1712
- ChessUploaded bydev1712
- GradientUploaded bydev1712
- 7.2 TransientUploaded bydev1712
- OFDM_NoRestrictionUploaded bySirag Mhgoob Sirag
- Curl and DivergenceUploaded bydev1712
- Digital Signal ProcessingUploaded byzsyed92
- bode_plotUploaded bydev1712
- wimax generalUploaded bySurya6556
- deconvLevinEtalCVPR09Uploaded byAvisha N Avisena
- Capacity and v-blast Techniques for Mimo Wireless ChannelUploaded bydev1712
- 9.5 FIR FiltersUploaded bydev1712
- Musical Touch BellUploaded byprofessinoal
- Basic Concepts in MatlabUploaded byimad.akhdar
- discrete Time Fourier MethodsUploaded bydev1712
- 6.3 Transfer FuncUploaded bydev1712
- 10.1.1.112Uploaded byAmit Patel
- 10.1.1.112Uploaded byAmit Patel
- LatexUploaded byapi-3720088

- 1331638332.949Components of Forces 1-9Uploaded byMohammed Abu-Zayed
- MathematicsUploaded byamjad
- Fundamentals of Number SystemUploaded byamitsaxena17
- Rademacher's TheoremUploaded byKandryck Chow
- 23_3_even_odd_fns.pdfUploaded byUnknown
- Cal1 IU SLIDES(2ndSem09-10) Chapter1 SVUploaded byVu Sang
- Cartesian ProductUploaded byChaste Diez
- Metric SpacesUploaded byNeelima Kandula
- Ex 4 4 FSC Part1 M ShahidUploaded byutkarsh mishra
- Sec3.3Uploaded byTheGreatPostulio
- 16S1_CH3101_MD_Week 3_V2 0Uploaded byEunike Regina Febby
- Complex Trigonometric FunctionsUploaded by09polkmnpolkmn
- Design and Analysis of Propeller Blade Geometry using the PDE MethodUploaded byAlexEcnom
- 3d Projection and Mapping Seminar ReportUploaded byHimani Bhati
- Scheme of Work Maths Stage 9Uploaded bySue Adames de Velasco
- articol Gerresten.pdfUploaded byRobert Pop
- fdp-4th.pdfUploaded byChristianStefan
- MODULE12 Matrices.docUploaded byCorey Lopez
- 107SOL I-F1Uploaded byOggy22
- EE Assign 2Uploaded byBawonda Isaiah
- 12 Vector Techniques_noPWUploaded byHaitham Haitham
- Math as and GS Good Questions YeahUploaded byBob Steve
- chap_8_3Uploaded bychrizzdM
- Unit-4 Basis and DimensionUploaded byChandradeep Reddy Teegala
- Hw1 Sol Amcs202Uploaded byFadi Awni Eleiwi
- 5francisco Maiz Jimenez Version 12bunUploaded byarg7math
- Strichartz_1977_Restrictions of Fourier Transforms to Quadratic Surfaces and Decay of Solutions of Wave Equations.Uploaded byJin Wang
- Brahma GuptaUploaded bybantyseth
- Munkres - Elements of Algebraic Topology - 0201045869 - OptUploaded byJose Luis Farro Quesquen
- Thesis uUploaded byTony Ware