This action might not be possible to undo. Are you sure you want to continue?

BooksAudiobooksComicsSheet Music### Categories

### Categories

### Categories

### Publishers

Scribd Selects Books

Hand-picked favorites from

our editors

our editors

Scribd Selects Audiobooks

Hand-picked favorites from

our editors

our editors

Scribd Selects Comics

Hand-picked favorites from

our editors

our editors

Scribd Selects Sheet Music

Hand-picked favorites from

our editors

our editors

Top Books

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Audiobooks

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Comics

What's trending, bestsellers,

award-winners & more

award-winners & more

Top Sheet Music

What's trending, bestsellers,

award-winners & more

award-winners & more

P. 1

Singular Value Decomposition Example|Views: 1|Likes: 0

Published by dev1712

See more

See less

https://www.scribd.com/doc/143889585/Singular-Value-Decomposition-Example

05/27/2013

text

original

Paul Skoufranis

March 11, 2010

We shall explore a concept known as the singular value decomposition of a matrix.

Deﬁnition Let A be an n ×m matrix. The singular values of A are the square roots of the eigenvalues of

m× m matrix A

T

A listed according to their algebraic multiplicity. We should note that the eigenvalues of

A

T

A will always be positive numbers so it makes sense to take the square root. It is customary to denote

the singular values by σ

1

, σ

2

, . . ., σ

m

with σ

1

≥ σ

2

≥ · · · ≥ σ

m

.

Theorem Any n × m matrix A can be written in the form A = UΣV

T

where U is an orthogonal n × n

matrix, V is an orthogonal m×m matrix, and Σ is an n ×m matrix whose ﬁrst r diagonal entries are the

non-zero singular values σ

1

, σ

2

, . . ., σ

r

of A and whose other entries are all zeros. The expression UΣV

T

is

known as the Singular Value Decomposition of A.

Now we shall go through an example of computing the Singular Value Decomposition (SVD) of a ma-

trix A, why it works, and how we can use the SVD to determine the action of A on the unit circle x

2

+y

2

= 1.

Example Compute the SVD of the matrix

A =

_

_

1 −1

1 −1

√

3 0

_

_

and use this decomposition to describe the action of A on the unit circle.

Solution: The ﬁrst step in ﬁnding the SVD of A is to ﬁnd the singular values of A. To do this, we

simply need to compute the eigenvalues of A

T

A. Thus let us ﬁrst compute A

T

A;

A

T

A =

_

1 1

√

3

−1 −1 0

_

_

_

1 −1

1 −1

√

3 0

_

_

=

_

5 −2

−2 2

_

Therefore the characteristic polynomial of A

T

A is

f

A

T

A

(λ) = det(A

T

A−λI) = (5 −λ)(2 −λ) −4 = λ

2

−7λ + 6 = (λ −1)(λ −6)

Therefore the eigenvalues of A

T

A are 1 and 6. Thus the singular values of A are σ

1

= 1 and σ

2

=

√

6 (in

general, at this step you would take the square root of each eigenvalue λ to get a singular value σ and include

it am(λ) times). This tells us that the matrix Σ will be the 3 ×2 matrix

Σ =

_

_

√

6 0

0 1

0 0

_

_

where we place σ

1

=

√

6 in the ﬁrst column and σ

2

= 1 in the second column since

√

6 > 1 (normally, if

σ

1

≥ σ

2

≥ · · · ≥ σ

r

> 0 are your non-zero singular values (including repetitions), we would place these

entries along the diagonal of Σ and place zeros elsewhere).

Our next goal is to determine the matrix V . We know that since A

T

A is a symmetric matrix, A

T

A will

have an orthonormal basis of eigenvectors. It turns out that the matrix V will be 2×2 matrix whose columns

are an orthonormal eigenbasis for A

T

A. Therefore we compute the eigenspaces of A

T

A. We notice that

E

6

= ker(A

T

A−6I) = ker

__

−1 −2

−2 −4

__

= ker

__

1 2

0 0

__

= span

_

1

√

5

_

2

−1

__

and

E

1

= ker(A

T

A−I) = ker

__

4 −2

−2 1

__

= ker

__

1 −

1

2

0 0

__

= span

_

1

√

5

_

1

2

__

Let

v

1

=

1

√

5

_

2

−1

_

and v

2

=

1

√

5

_

1

2

_

Then we let V be the 2 ×2 matrix

V =

_

v

1

v

2

¸

=

_

2

√

5

1

√

5

−

1

√

5

2

√

5

_

where we place the eigenvector for the eigenvalue 6 in the ﬁrst column and the eigenvector for the eigenvalue

1 in the second column since we placed

√

6 in ﬁrst column of Σ and

√

1 in the second column of Σ.

Now our question is how do we ﬁnd the orthogonal 3 ×3 matrix U? Well, if we could ﬁnd an orthogonal

3 ×3 matrix U such that AV = UΣ, we would be done since V

−1

= V

T

as V is an orthogonal matrix. Let

us consider the actions of AV and Σ on the standard basis e

1

and e

2

of R

2

. Well

Σ

_

1

0

_

= σ

1

_

_

1

0

0

_

_

Σ

_

0

1

_

= σ

2

_

_

0

1

0

_

_

and

AV

_

1

0

_

= Av

1

AV

_

0

1

_

= Av

2

Therefore, if U were an orthogonal 3 × 3 matrix such that AV = UΣ and u

1

, u

2

, and u

3

were the columns

of U, then {u

1

, u

2

, u

3

} would be an orthonormal basis of R

3

and

Av

1

= AV

_

1

0

_

= UΣ

_

1

0

_

=

√

6U

_

_

1

0

0

_

_

= σ

1

u

1

and

Av

2

= AV

_

0

1

_

= UΣ

_

0

1

_

=

√

1U

_

_

0

1

0

_

_

= σ

2

u

2

Thus we should let

u

1

=

1

σ

1

Av

1

=

1

6

_

_

1 −1

1 −1

√

3 0

_

_

_

1

√

5

_

2

−1

__

=

1

√

30

_

_

3

3

2

√

3

_

_

and

u

2

=

1

σ

2

Av

2

=

1

1

_

_

1 −1

1 −1

√

3 0

_

_

_

1

√

5

_

1

2

__

=

1

√

5

_

_

−1

−1

√

3

_

_

2

If we check, we see that {u

1

, u

2

} are indeed orthonormal vectors. The question is, why does this work?

Well, just using the facts that u

1

=

1

σ1

Av

1

, u

2

=

1

σ2

Av

2

, A

T

Av

1

= σ

2

1

v

1

(since v

1

is an eigenvector for A

T

A

with eigenvalue σ

2

1

), A

T

Av

2

= σ

2

2

v

2

(v

2

is an eigenvector for A

T

A with eigenvalue σ

2

2

), and {v

1

, v

2

} is an

orthonormal set, we see that

u

1

· u

1

=

_

1

σ

1

Av

1

_

·

_

1

σ

1

Av

1

_

=

_

1

σ

1

Av

1

_

T

_

1

σ

1

Av

1

_

=

1

σ

2

1

v

T

1

A

T

Av

1

=

1

σ

2

1

v

T

1

(A

T

Av

1

)

=

1

σ

2

1

v

T

1

(σ

2

1

v

1

)

= v

T

1

v

1

= v

1

· v

1

= 1

and

u

2

· u

2

=

_

1

σ

2

Av

2

_

·

_

1

σ

2

Av

2

_

=

_

1

σ

2

Av

2

_

T

_

1

σ

2

Av

2

_

=

1

σ

2

2

v

T

2

A

T

Av

2

=

1

σ

2

2

v

T

2

(A

T

Av

2

)

=

1

σ

2

2

v

T

2

(σ

2

2

v

2

)

= v

T

2

v

2

= v

2

· v

2

= 1

and

u

1

· u

2

=

_

1

σ

1

Av

1

_

·

_

1

σ

2

Av

2

_

=

_

1

σ

1

Av

1

_

T

_

1

σ

2

Av

2

_

=

1

σ

1

σ

2

v

T

1

A

T

Av

2

=

1

σ

1

σ

2

v

T

1

(A

T

Av

2

)

=

1

σ

2

σ

2

v

T

1

(σ

2

2

v

2

)

=

σ

2

σ

1

v

T

1

v

2

=

σ

2

σ

1

v

1

· v

2

= 0

3

which is want we wanted.

However, to ﬁnd an orthogonal matrix using u

1

and u

2

, we need a third orthonormal vector as we need

U to be a 3 × 3 matrix and we only have two orthonormal vectors (in some cases, you will not need this

step). To ﬁnd u

3

, write

u

3

=

_

_

a

b

c

_

_

Then we want a

2

+ b

2

+ c

2

= 1 so u

3

is unit length, and we want

0 = u

1

· u

3

=

1

√

30

(3a + 3b + 2

√

3c)

and

0 = u

2

· u

3

=

1

√

5

(−a −b +

√

3c)

By solving a system of linear equations, we obtain that

α

_

_

1

−1

0

_

_

where α ∈ R is the set of all solutions to the equations 0 =

1

√

30

(3a +3b +2

√

3c) and 0 =

1

√

5

(−a −b +

√

3c).

Therefore, if we let u

3

=

1

√

2

_

_

1

−1

0

_

_

, u

3

is a normal vector that is orthogonal to u

1

and u

2

. Thus {u

1

, u

2

, u

3

}

is an orthonormal basis for R

3

. Then, if we let

U =

_

u

1

u

2

u

3

¸

=

_

¸

_

3

√

30

−

1

√

5

1

√

2

3

√

30

−

1

√

5

−

1

√

2

2

√

10

√

3

√

5

0

_

¸

_

then U is an orthogonal matrix and A = UΣV

T

as desired. If you are not conﬁdent in your computations,

you could easily check your answer by multiplying this out.

Now we desire to determine the action of A on the unit circle. To see this, we will ﬁrst apply V

T

to the

unit circle, then Σ to that image, and then we will apply U. The following is a diagram representing the

actions of each map:

4

In the diagram, we start with the top left diagram. The vector v

1

and its images are shown in blue and the

vector v

2

and its images are shown in red. First we know that V take e

1

to v

1

and e

2

to v

2

. Since V

T

= V

−1

as V is an orthogonal matrix, V

T

takes v

1

to e

1

and v

2

to e

2

. Then Σ adds a z-axis and dilates the x-axis by

√

6. Then U rotates the whole diagram in an awkward manner (you probably do not need to know exactly

what this rotation is but, it takes the x-axis to the line through u

1

, it takes the y-axis to the line through

u

2

, and it takes the z-axis to the line through u

3

). It is hard to see what the curve in the ﬁnal diagram looks

like. The blue line goes to the point

√

6

√

30

(3, 3, 2

√

3) and the red line goes to

1

√

5

(−1, −1,

√

3).

5

ijwin02

Chess

Probability

Important Tables

BER simulation using MATLAB

Bhagavat Geeta

Borel Sigma

Probability

random variable

supremum and infimum

7.2 Transient

9 Laplace z Table

Laplace Tables

8.7 Bessel

4.4 Bode

6.3 Transfer Func

9.5 FIR Filters

deconvLevinEtalCVPR09

22_ofdm2

Capacity and v-blast Techniques for Mimo Wireless Channel

Musical Touch Bell

Basic Concepts in Matlab

Cognitive Radio Brain Empowered

Cognitive Radio Brain Empowered

Digital Signal Processing

- Read and print without ads
- Download to keep your version
- Edit, email or read offline

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

CANCEL

OK

You've been reading!

NO, THANKS

OK

scribd

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->