Chap 7.

Diagonalization and Quadratic

Forms
 7. Diagonalization and Quadratic Forms
 7.1 Orthogonal Matrices
 7.2 Orthogonal Diagonalization
 7.3 Quadratic Forms
 7.4 Optimization Using Quadratic Forms
 7.5 Hermitian, Unitary, and Normal Matrices

2
 7.1 Orthogonal Matrices
❑ Orthogonal Matrices

3
Example 2

4
Orthogonal Matrices

5
Orthogonal Matrices

6
Orthogonal Matrices as Linear Operators

7
Change of Orthonormal Basis

8
 7.2 Orthogonal Diagonalization
❑ The Orthogonal Diagonalization Problem

❑ Recall “similar” in Chap 5, p. 18

9
Orthogonal Diagonalizable

10
 Conditions for Orthogonal Diagonalizability

❑ Every symmetric matrix with real entries is orthogonally

diagonalizable.

11
Properties of Symmetric Matrices

12
Orthogonal Diagonalization

13
Example 1

14
Example 1 (Cont)

15
Spectral Decomposition

16
Example 2

17
Example 2 (Cont)

18
 7.3 Quadratic Forms
❑ Linear Form

❑ Quadratic Form

❑ quadratic form associated with A.

▪ For diagonal A

19
Example 1

20
Change of Variable in a Quadratic Form

21
Principle Axes Theorem

22
Example 2

23
 Quadratic Forms in Geometry
❑ A conic section or conic is a curve that results by
cutting a double-napped cone with a plane

24
 Quadratic Forms in Geometry
❑ central conic

❑ central conic in standard position

25
Identifying Conic Sections

26
 Positive Definite Quadratic Forms

❑ A quadratic form for which xTAx ≥ 0 if x ≠ 0 is called

positive semidefinite, and one for which xTAx ≤ 0 if x ≠
0 is called negative semidefinite.
27
Example 4

28
Classifying Conic Sections Using Eigenvalues

29
 Identifying Positive Definite Matrices
❑ kth principal submatrix

30
Example 5

31
 7.4 Optimization Using Quadratic Forms
❑ Constrained Extremum Problems
▪ Problem of finding the maximum and minimum values of

32
Example 1

33
Example 2

34
 7.5 Hermitian, Unitary, and Normal Matrices
❑ Real Matrices Versus Complex Matrices
▪ real matrices: matrices whose entries must be real
▪ complex matrices: matrices whose entries may be either real numbers
or complex numbers

❑ Hermitian and Unitary Matrices

35
Hermitian Matrices

36
Unitary Matrices

37
 Eigenvalues and Eigenvector of Hermitian
Matrices
❑ In Section 5.3 (p. 57)

38
Unitary Matrices

39
Example 5

40
 Unitary Diagonalizability

❑ Orthogonally diagonalizable in pp. 9-10

41
42
Example 6

43
 Skew-Symmetric and
Skew-Hermitian Matrices
❑ A square real matrix A is said to be skew-symmetric if
AT = −A.
❑ A square complex matrix A is said to be skew-Hermitian
if A∗ = −A.

44
 Normal Matrices
❑ The real symmetric matrices are the only orthogonally
diagonalizable matrices.

❑ Normal Matrix A

❑ A square complex matrix A is unitarily diagonalizable if

and only if it is normal.

▪ Normal matrices include the Hermitian, skew-Hermitian, and unitary

matrices in the complex case and the symmetric, skew-symmetric, and
orthogonal matrices in the real case.

▪ The nonzero skew-symmetric matrices are particularly interesting

because they are examples of real matrices that are not orthogonally
diagonalizable but are unitarily diagonalizable.

45
 A Comparison of Eigenvalues
❑ Hermitian matrices have real eigenvalues.
❑ The eigenvalues of a skew-Hermitian matrix are either
zero or purely imaginary (have real part of zero)
❑ The eigenvalues of unitary matrices have modulus 1.

46
Eigenvalue Decomposition and
Fourier Transform
 Orthonormal Basis
❑ Basis {v1 , v 2 , , vN }
▪ A set of linearly independent vectors that, in a linear combination, can
represent every vector in a given vector space
▪ For any x  C
N

N
x =  an v n
n =1

x = a1 v1 + a2 v 2
❑ Orthonormal basis {v1 , v 2 , , vN }
1, i = j
v v j =  ij = 
H
a2 = v 2H x
0, i  j
i

v2
▪ For any x C N
v1
N
x =  an v n an = v nH x a1 = v1H x
n =1

48
 Fourier Series & Fourier Transform
❑ Fourier series (for periodic signal)
▪ Period T, frequency f0=1/ T

1
x(t ) = 
n =−
an e j 2 nf 0t
an =
T 
T
x(t )e − j 2 nf0t dt

❑ Fourier transform
 
x(t ) =  X ( f )e j 2 ft
df X ( f ) =  x(t )e − j 2 ft dt
− −

❑ cf) Orthonormal basis representation

N
x =  an v n an = v nH x
n =1

49
 Eigenvalue Decomposition (EVD)
❑ Linear system
x A y = Ax
input system output

❑ Eigenvalue & Eigenvector

▪ An eigenvector is a vector that preserves its direction after
transformation
Av =  v
eigenvector eigenvalue
1 
0  
A[ v1 v 2 v N ] = [ v1 v 2 vN ]  2 
 
 
 N 
AV = VΛ → A = VΛV −1
50
 Eigenvalue Decomposition (EVD)
❑ Unitary matrix
UU H = U H U = I U −1 = U H

❑ Normal matrix
AA H = A H A
▪ Eigenvectors form an unitary matrix. VV H = V H V =I
▪ Normal matrix can be orthogonally diagonalizable.
A = VΛV H
1   v1H 
0   H
= [ v1 v 2 vN ]  2   v2 
  
  H
 N   v N 
N
=  n v n v nH Spectral decomposition
n =1
51
 Linear Transformation
❑ Linear transformation
x A y = Ax
N
A =  n v n v nH
N n =1

x =  an v n
n =1
y = Ax y = 1a1 v1 + 2 a2 v 2
N N
=  n v n v H
n a m vm
n =1 m =1 x = a1 v1 + a2 v 2

2 a2
N N
=  n am v n ( v nH v m )
n =1 m =1 a2 = v 2H x v2
N N
=  n am v n nm v1
n =1 m =1
N
a1 = v1H x
=  n an v n
n =1
1a1

52
 Linear Transformation
❑ Linear-time invariant (LTI) system: Filter
x(t ) H(f) y (t )

y (t ) =  h( ) x(t −  ) d
−

▪ Fourier series Fourier transform





x (t ) = an e j 2 nf0t x(t ) =  X ( f )e j 2 ft df
n =− −

bn = H (nf 0 )an Y( f ) = H ( f )X ( f )
 
y (t ) =  bn e j 2 nf 0 t
y (t ) =  Y ( f )e j 2 ft df
−
n =−


=  H ( f ) X ( f ) e j 2  ft d f
=  H (nf )a e
n =−
0 n
j 2 nf 0t
−

▪ cf) linear transform

N N
x =  an v n y = Ax =  n an v n
N
A =  n v n v nH
n =1 n =1 n =1

53
 Linear Transformation
❑ Eigenfunction
▪ An eigenfunction is a function that preserves its shape after
transformation
x (t ) = e j 2 ft

y (t ) =  h( ) x (t −  )d
−
 e j 2 ft H(f) H ( f )e j 2 ft
=  h( )e j 2 f ( t − ) d
−

= e j 2 ft  h( )e − j 2 f  d
−

= H ( f )e j 2 ft

▪ e j 2 ft is an eigenfunction of a linear system and H(f) is an eigenvalue

54
 Power and Energy Conservation
❑ Orthonormal basis
N N N
x =  an v n x = x x = a v a
2 H * H
n n m vm
n =1 n =1 m =1
N N N N
=  a a v v m =  an* am nm
*
n m
H
n
n =1 m =1 n =1 m =1
N 2

=  an
n =1

❑ Fourier series Fourier transform





x(t ) = an e j 2 nf0t x(t ) =  X ( f )e j 2 ft df
−
n =−


 
=  x(t ) 2dt = 
2 2
x(t ) an X ( f ) 2df
− −
n =−

55