25 views

Uploaded by Raúl Ruiz

Quadratic forms

- math 8 course syllabus
- Model analysis theory
- NX Nastran Theoretical Manual
- Practical Linear Algebra
- Image Analysis and Pattern Recognition for Remote Sensing with Algorithms in ENVI/IDL
- RepetitoriumMultivariat
- Jornal Charmode
- Data Analytics Courses
- ch07_9
- Linear Systems Errata
- Revision for Java Programming I Basic
- 477577_Chapter_11
- 1010projectqsp19 1
- Colored Image Watermarking Technique Based on HVS using HSV Color Model
- FTEE_77_70
- Hermitian matrix.pdf
- Composite
- Sing Value
- Ybus
- Aieee Solution 2011

You are on page 1of 32

Stephen Boyd

Lecture 15

Symmetric matrices, quadratic forms, matrix

norm, and SVD

eigenvectors of symmetric matrices

quadratic forms

inequalities for quadratic forms

positive semidefinite matrices

norm of a matrix

singular value decomposition

151

suppose A Rnn is symmetric, i.e., A = AT

fact: the eigenvalues of A are real

to see this, suppose Av = v, v 6= 0, v Cn

then

v T Av = v T (Av) = v T v =

n

X

i=1

but also

T

v T Av = (Av) v = (v) v =

|vi|2

n

X

i=1

|vi|2

Symmetric matrices, quadratic forms, matrix norm, and SVD

152

fact: there is a set of orthonormal eigenvectors of A, i.e., q1, . . . , qn s.t.

Aqi = iqi, qiT qj = ij

in matrix form: there is an orthogonal Q s.t.

Q1AQ = QT AQ =

A = QQT =

n

X

iqiqiT

i=1

Symmetric matrices, quadratic forms, matrix norm, and SVD

153

Interpretations

replacements

A = QQT

x

QT

QT x

QT x

Ax

resolve into qi coordinates

scale coordinates by i

reconstitute with basis qi

Symmetric matrices, quadratic forms, matrix norm, and SVD

154

or, geometrically,

rotate by QT

diagonal real scale (dilation) by

rotate back by Q

decomposition

A=

n

X

iqiqiT

i=1

155

example:

1/2

3/2

3/2 1/2

T

1

1

1

0

1

1

1

1

=

0 2

2 1 1

2 1 1

A =

q2q2T x

x

q1

q1q1T x

1q1q1T x

q2

2q2q2T x

Ax

156

suppose v1, . . . , vn is a set of linearly independent eigenvectors of A:

Avi = ivi,

kvik = 1

then we have

viT (Avj ) = j viT vj = (Avi)T vj = iviT vj

so (i j )viT vj = 0

for i 6= j, i 6= j , hence viT vj = 0

in this case we can say: eigenvectors are orthogonal

in general case (i not distinct) we must say: eigenvectors can be

chosen to be orthogonal

Symmetric matrices, quadratic forms, matrix norm, and SVD

157

Example: RC circuit

i1

v1

c1

in

vn

resistive circuit

cn

ck v k = ik ,

i = Gv

thus v = C 1Gv where C = diag(c1, . . . , cn)

note C 1G is not symmetric

Symmetric matrices, quadratic forms, matrix norm, and SVD

158

use state xi =

civi, so

x = C 1/2v = C 1/2GC 1/2x

where C

1/2

we conclude:

eigenvalues 1, . . . , n of C 1/2GC 1/2 (hence, C 1G) are real

eigenvectors qi (in xi coordinates) can be chosen orthogonal

eigenvectors in voltage coordinates, si = C 1/2qi, satisfy

C 1Gsi = isi,

sTi Csi = ij

159

Quadratic forms

a function f : Rn R of the form

f (x) = xT Ax =

n

X

Aij xixj

i,j=1

in a quadratic form we may as well assume A = AT since

xT Ax = xT ((A + AT )/2)x

((A + AT )/2 is called the symmetric part of A)

uniqueness: if xT Ax = xT Bx for all x Rn and A = AT , B = B T , then

A=B

Symmetric matrices, quadratic forms, matrix norm, and SVD

1510

Examples

kBxk2 = xT B T Bx

Pn1

i=1

(xi+1 xi)2

kF xk2 kGxk2

sets defined by quadratic forms:

{ x | f (x) = a } is called a quadratic surface

{ x | f (x) a } is called a quadratic region

1511

suppose A = AT , A = QQT with eigenvalues sorted so 1 n

xT Ax = xT QQT x

= (QT x)T (QT x)

n

X

i(qiT x)2

=

i=1

n

X

(qiT x)2

i=1

= 1kxk2

i.e., we have xT Ax 1xT x

Symmetric matrices, quadratic forms, matrix norm, and SVD

1512

nxT x xT Ax 1xT x

q1T Aq1 = 1kq1k2,

1513

suppose A = AT Rnn

we say A is positive semidefinite if xT Ax 0 for all x

denoted A 0 (and sometimes A 0)

A 0 if and only if min(A) 0, i.e., all eigenvalues are nonnegative

not the same as Aij 0 for all i, j

we say A is positive definite if xT Ax > 0 for all x 6= 0

denoted A > 0

A > 0 if and only if min(A) > 0, i.e., all eigenvalues are positive

Symmetric matrices, quadratic forms, matrix norm, and SVD

1514

Matrix inequalities

we say A is negative semidefinite if A 0

we say A is negative definite if A > 0

otherwise, we say A is indefinite

matrix inequality: if B = B T Rn we say A B if A B 0, A < B

if B A > 0, etc.

for example:

A 0 means A is positive semidefinite

A > B means xT Ax > xT Bx for all x 6= 0

Symmetric matrices, quadratic forms, matrix norm, and SVD

1515

if A B and C D, then A + C B + D

if B 0 then A + B A

if A 0 and 0, then A 0

A2 0

A 6 B,

B 6 A

1516

Ellipsoids

E = { x | xT Ax 1 }

is an ellipsoid in Rn, centered at 0

s1

s2

1517

1/2

qi, i.e.:

eigenvalues determine lengths of semiaxes

note:

in direction q1, xT Ax is large, hence ellipsoid is thin in direction q1

in direction qn, xT Ax is small, hence ellipsoid is fat in direction qn

Symmetric matrices, quadratic forms, matrix norm, and SVD

1518

suppose A Rmn (not necessarily square or symmetric)

for x Rn, kAxk/kxk gives the amplification factor or gain of A in the

direction x

obviously, gain varies with direction of input x

questions:

what is maximum gain of A

(and corresponding maximum gain direction)?

what is minimum gain of A

(and corresponding minimum gain direction)?

how does gain of A vary with direction?

Symmetric matrices, quadratic forms, matrix norm, and SVD

1519

Matrix norm

the maximum gain

kAxk

x6=0 kxk

is called the matrix norm or spectral norm of A and is denoted kAk

max

kAxk2

xT AT Ax

T

max

=

max

=

(A

A)

max

2

2

x6=0 kxk

x6=0

kxk

p

so we have kAk = max(AT A)

min kAxk/kxk =

x6=0

min(AT A)

1520

note that

AT A Rnn is symmetric and AT A 0 so min, max 0

max gain input direction is x = q1, eigenvector of AT A associated

with max

min gain input direction is x = qn, eigenvector of AT A associated with

min

1521

35 44

44 56

1 2

example: A = 3 4

5 6

AT A =

then kAk =

0.620

0.785

0.785 0.620

90.7

0

0 0.265

0.620

0.785

0.785 0.620

T

max(AT A) = 9.53:

0.620

0.785
= 1,

2.18

0.620

A

=
4.99
= 9.53

0.785

7.78

1522

min gain is

min(AT A) = 0.514:

0.785

0.620
= 1,

A

0.46

0.785

0.14

=

= 0.514

0.620

0.18

kAxk

9.53

0.514

kxk

1523

n1

consistent

with

vector

norm:

matrix

norm

of

a

R

is

p

max(aT a) = aT a

scaling: kaAk = |a|kAk

triangle inequality: kA + Bk kAk + kBk

definiteness: kAk = 0 A = 0

norm of product: kABk kAkkBk

1524

decomposition (SVD) of A:

A = U V T

where

A Rmn, Rank(A) = r

U Rmr , U T U = I

V Rnr , V T V = I

= diag(1, . . . , r ), where 1 r > 0

Symmetric matrices, quadratic forms, matrix norm, and SVD

1525

A = U V T =

r

X

iuiviT

i=1

vi are the right or input singular vectors of A

ui are the left or output singular vectors of A

1526

AT A = (U V T )T (U V T ) = V 2V T

hence:

vi are eigenvectors of AT A (corresponding to nonzero eigenvalues)

i =

kAk = 1

1527

similarly,

AAT = (U V T )(U V T )T = U 2U T

hence:

ui are eigenvectors of AAT (corresponding to nonzero eigenvalues)

i =

v1, . . . vr are orthonormal basis for N (A)

1528

Interpretations

A = U V T =

r

X

iuiviT

i=1

V Tx

V T x

Ax

compute coefficients of x along input directions v1, . . . , vr

scale coefficients by i

difference with eigenvalue decomposition for symmetric A: input and

output directions are different

Symmetric matrices, quadratic forms, matrix norm, and SVD

1529

u1 is highest gain output direction

Av1 = 1u1

1530

example: consider A R44 with = diag(10, 7, 0.1, 0.05)

input components along directions v1 and v2 are amplified (by about

10) and come out mostly along plane spanned by u1, u2

input components along directions v3 and v4 are attenuated (by about

10)

kAxk/kxk can range between 10 and 0.05

A is nonsingular

for some applications you might say A is effectively rank 2

1531

resolve x along v1, v2: v1T x = 0.5, v2T x = 0.6, i.e., x = 0.5v1 + 0.6v2

now form Ax = (v1T x)1u1 + (v2T x)2u2 = (0.5)(1)u1 + (0.6)(0.5)u2

v2

u1

x

v1

Ax

u2

1532

- math 8 course syllabusUploaded byapi-351773730
- Model analysis theoryUploaded bystructural vinay
- NX Nastran Theoretical ManualUploaded byMSC Nastran Beginner
- Practical Linear AlgebraUploaded byel_greco1982
- Image Analysis and Pattern Recognition for Remote Sensing with Algorithms in ENVI/IDLUploaded byCynthia
- RepetitoriumMultivariatUploaded byPeter Paul
- Jornal CharmodeUploaded byYonagi Mbah
- Data Analytics CoursesUploaded bystuart max
- ch07_9Uploaded byVSRI1993
- Revision for Java Programming I BasicUploaded byRyan Tang
- 477577_Chapter_11Uploaded byRamnik Jain
- FTEE_77_70Uploaded byMariya Pavlova
- Linear Systems ErrataUploaded bymxjoe
- 1010projectqsp19 1Uploaded byapi-401184949
- Hermitian matrix.pdfUploaded bymars
- CompositeUploaded byPraveen Vundrajavarapu
- Colored Image Watermarking Technique Based on HVS using HSV Color ModelUploaded byIDES
- Sing ValueUploaded byTruong Cai
- YbusUploaded byarunabh bhattacharya
- Aieee Solution 2011Uploaded byArun Prasath Ramamoorthy
- 0000707Uploaded byR Vignesh Babu
- 318Uploaded byFahad Ahmad Khan
- S2-18-ET ZC413-L7.pptxUploaded byAasheesh Chander Agrawal
- ch8_4Uploaded bySujith Surapaneni
- 12.pdfUploaded byTalin Machin
- 9231_s10_qp_11Uploaded byArinj Jaruwatanachai
- 030107probTheoryUploaded bygioigoi
- Utah hw1 solns.pdfUploaded bycschmm
- Enhanced ESPRIT Algorithm for Optimal Wind Turbine Fault RecognizingUploaded bySaad Chakkor
- 1609.01552Uploaded byAnonymous lqvgUsVck

- 1697Uploaded byAntony Boston Michael
- Brother QL 570 printer / labelerUploaded byFred Seibert
- Custom PC - August 2014 UKUploaded byqinlao
- Deutz Accessories CatalogueUploaded byВалентин Димитров
- 0000518241-DB Connect Oracle DBUploaded byMohamed Ezzat
- Ga TutorialUploaded byrajmeha
- List of Posts for Which MoST Test is Postponed.pdfUploaded byKhawaja Rameez
- AccutronMAQS_2013Uploaded byEnrique Alonso
- Basic Guidelines to Product SketchingUploaded byOlajide DI
- My Future Objectives by Sander KausUploaded bysander_kaus
- ARRI Digital - Signal Processing - 2009-09-03Uploaded byTruong Minh
- Diction and ToneUploaded byTerentiux
- Proposal Geotechnical for Block Caving.docUploaded byFrans Pepuho
- Schindler Handrail MaintenanceUploaded bya_salehi
- Example - Positioning StatementsUploaded byAtul Kathuria
- 2e85f9517dc2bf164e_htm62xganUploaded byShowwei Chu
- Karpen's Pile - Hypothesis - Circ_O1Uploaded byJacek Rafał Tatko
- DEVELOPING NEW MARKET OFFERINGSUploaded byJay Tyagi
- 2111287Uploaded byJorge A. López
- Mon Taj Grav Mag InterpretationUploaded byrenzoquim
- 03 ZF Sachs Product Information PC a Automated Manual Transmission en eBookUploaded bysherisha
- dcdUploaded byDaniel Chen
- French Bulletin IATSE 638Uploaded byIATSE
- ch41.pdfUploaded byRodrigo Silva Quirino
- AVEVA E3D 2.1 STUTUS CONTROL TRAINING CHAPTER 7Uploaded byEmavwodia Solomon
- CS8383 OOPS Lab Manual.pdfUploaded byMohammed Sufiyan
- Vacuum Ejector SystemUploaded byfunshare
- Asme Section II a Sa-276Uploaded byAnonymous GhPzn1x
- GeorgiAmmaSchultz_RecognizingHandAndFingerGesturesWithIMUBasedMotionAndEMGBasedMuscleActivitySensingUploaded byPeter Mueller
- Database RefreshUploaded byNst Tnagar