Outline of the week

1 Complex adjoint and special matrices

2 Nature of the eigenvalues
3 Orthogonality of the eigenvectors
4 Spectral theorem for special matrices
5 Examples of diagonalization
6 Applications of spectral theorem
7 Selected problems from tutorial sheets

February 6, 2017 1/1

Complex adjoint

We now study some important classes of matrices for which the eigenvalue
problem is well understood and solvable.

Definition 1 (Complex adjoint)

For an m × n matrix with complex entries, we define its (complex) adjoint
by taking its transpose and conjugating its entries. Thus
A = [ajk ] =⇒ A∗ = ĀT = [brs ] where brs = āsr , (1, 1) ≤ (r , s) ≤ (n, m).

If the entries are real, then we just get the transpose.

Not to be confused with AdjA defined earlier. Its usage is almost zero in
practice. ∗-adjoint is heavily used.

February 6, 2017 2/1

Elementary properties of complex adjoint

Theorem 2
The complex adjoint satisfies the following properties:
(A + B)∗ = A∗ + B ∗ .
(λA)∗ = λ̄A∗ .
(AB)∗ = B ∗ A∗ .

∗
A−1 = (A∗ )−1 for invertible A.
A∗∗ = A.
Remarks:
• Except for the second property, all others are just like for transpose.

• First and second can be combined as (λA + µB)∗ = λ̄A∗ + µ̄B ∗ . It is

almost like linearity and is known as conjugate linearity.

February 6, 2017 3/1

Special matrices: hermitian and symmetric, skew hermitian
and skew symmetric

Definition 3 (Hermitian and symmetric matrices)

• An n × n square matrix with complex entries is called hermitian if it
equals its own adjoint i.e. A∗ = A.
• An n × n square matrix with real entries is called symmetric if it equals
its own transpose i.e. AT = A.

We observe that the property of being hermitian already means symmetric

for the real matrices.
There was no need to define it separately.
Exercise:Show that in a hermitian matrix,the diagonal entries are reals.

February 6, 2017 4/1

Special matrices: skew hermitian and skew symmetric

Definition 4 (Skew-hermitian and skew-symmetric matrices)

• A square matrix with complex entries is called skew-hermitian if it equals
the negative of its own adjoint i.e. A∗ = −A.
• An n × n square matrix with real entries is called skew symmetric if
AT = −A.

Once again the property of being skew hermitian already means skew
symmetric for the real matrices.
There was no need to define it separately.

Exercise: Show that the diagonal entries of a skew-hermitian matrix lie in

iR.
Show that A is skew-hermitian if and only if iA is hermitian.

February 6, 2017 5/1

Unitary/Orthogonal matrices

Definition 5 (Unitary/orthogonal)
• An n × n complex matrix A is called unitary if it satisfies
A∗ A = AA∗ = In .
• An n × n real matrix A is called orthogonal if it satisfies
AT A = AAT = In .

• Unitarity also automatically becomes orthogonality for real matrices.

Exercise: Show that | det A| = 1 if A is unitary or orthogonal.

Exercise: Show that if A is unitary or orthogonal, its columns form an
orthonormal basis of Cn or Rn . What about the rows?

February 6, 2017 6/1

Nature of eigenvalues

Theorem 6
(a) The eigenvalues of a hermitian matrix lie in R.
(b) The eigenvalues of a skew-hermitian matrix lie in iR.
(c) The eigenvalues of a unitary matrix lie in U(1) = {z ∈ C||z| = 1}
-the unit circle in the complex plane. .

Proof:

Av = λv =⇒ v∗ Av = λv∗ v =⇒ λkvk2 = (A∗ v)∗ v.


∗ ∗
(Av) v if A = A

∴ λkvk2 = (−Av)∗ v if A∗ = −A

 −1 ∗
(A v) v if A∗ = A−1

February 6, 2017 7/1

Nature of eigenvalues, contd.
In other words,

∗
(λv) v if A is hermitian

λkvk2 = (−λv)∗ v if A is skew hermitian

 −1 ∗
(λ v) v if A is unitary

kvk 2 if A is hermitian
λ̄
 
2 2 if A is skew hermitian
kvk
λ 
 = −λ̄ kvk

 −1 
λ̄  kvk 2 if A is unitary


λ̄ if A is hermitian

=⇒ λ = −λ̄ if A is skew hermitian

 −1
λ̄ if A is unitary

=⇒ λ ∈ R, iR, or U(1) respectively.

February 6, 2017 8/1

Exercises

Classify
 and find the eigenvalues:   
4 1 − 3i 4 1 3i 2+i
(a) , (b) , (c) ,
1 + 3i 7 1 7 −2 + i −i
   √   √ 
0 2 1 i
√ 3 1 i
√ − 3
(d) , (e) , (f ) ,
−2 0 2 3 i 2 3 i
 √ 
1 1 3
(g ) √ .
2 3 −1

February 6, 2017 9/1

Answers

(a) Hermitian, {9, 2}.

( √ )
11 ± 13
(b) Symmetric, .
2
(c) Skew-hermitian, {4i, −2i}.
(d) Skew-symmetric, {±2i}.
( √ )
± 3+i
(e) Unitary, .
2
( √ )
1 ± 3i
(f) Orthogonal, .
2
(g) Orthogonal and symmetric, {±1}.

February 6, 2017 10 / 1
Orthogonality of eigenvectors
We have something much better than just linear independence of
eigenvectors in the case of special matrices.

Theorem 7 (Orthogonality of eigenvectors)

Let λ1 , λ2 , ..., λk be distinct eigenvalues of a special matrix
(hermitian/skew-hermitian/unitary) A. Let v1 , v2 , ..., vk be a choice of the
corresponding eigenvectors in Cn . Then {v1 , v2 , ..., vk } is an orthogonal
set in Cn .
In other words, hv` , vj i = vj∗ v` := v̄jT v` = 0 ∀j 6= `.

We first prove a lemma:

Lemma 8
If A is a special matrix then

Av = λv ⇐⇒ A∗ v = λ̄v.

February 6, 2017 11 / 1
Proofs

Proof of the lemma:

A∗ = A, − A, A−1 =⇒ λ̄ = λ, − λ, λ−1 respectively.
Hence in short, A∗ v = λ̄v and conversely.

Proof of the theorem:

Av` = λ` v` =⇒ vj∗ Av` = λ` vj∗ v`

=⇒ (A∗ vj )∗ v` = λ` vj∗ v` =⇒ (λ̄j vj )∗ v` = λ` vJ∗ v`
=⇒ λj vJ∗ v` = λ` vJ∗ v` =⇒ vJ∗ v` = 0 if λj 6= λ` i.e. if j 6= `.

Example 9
 
4 1 − 3i
Find the eigenvalues and the eigenvectors of .
1 + 3i 7

February 6, 2017 12 / 1
Example

Example 10
 
4 1 − 3i
Find the eigenvalues and the eigenvectors of .
1 + 3i 7
Solution: The matrix is hermitian and λ = 9, 2 can be seen easily.
For λ = 9:
   
−5 1 − 3i E21 ( 1+3i
5 ) −5 1 − 3i
A − 9I2 = −→
1 + 3i −2 0 0
 
1 − 3i
=⇒ v1 = is a choice.
5
 
3i − 1
Similarly for λ = 2, v2 = is a choice.
2
v1∗ v2 = (1 + 3i)(3i − 1) + (5)(2) = 0 is easy to see. [1.0]

February 6, 2017 13 / 1
Example

Example 11
 
0 3 0
Find the eigenvalues and the eigenvectors of −3 0 4 . The matrix
0 −4 0
is skew-symmetric and λ = 0, ±5i can be seen easily.
For λ = 0:
   
0 3 0 −3 0 4
P12
A = −3 0 4 −→ 0 3 0
0 −4 0 0 −4 0
   
−3 0 4 4
E32 (4/3)
−→  0 3 0 =⇒ v1 = 0 is a choice.
0 0 0 3

February 6, 2017 14 / 1
Example contd.

Similarly, for λ = 5i:

   
−5i 3 0 −5i 3 0
E21 (3i/5)
A − 5iI3 =  −3 −5i 4  −→  0 −16i/5 4 
0 −4 −5i 0 −4 −5i
 
−5i 3 0
M2 (5/4)
−→  0 −4i 5 
0 −4 −5i
 
−5i 3 0
E32 (i)
−→  0 −4i 5
0 0 0
 
−3
=⇒ v2 = −5i  is a choice.
4

February 6, 2017 15 / 1
Example contd

 
−3
Similarly for λ = −5i, v3 =  5i  is a choice.
4
Note that v3 is just the conjugate of v2 as expected.
Easy to verify that v1∗ v2 = v2∗ v3 = v3∗ v1 = 0.

February 6, 2017 16 / 1
Spectral theorem

We now state the very important:

Theorem 12 (Spectral theorem-1)

Every special complex matrix A (hermitian, skew-hermitian or unitary)
admits an orthonormal basis in Cn consisting of eigenvectors of A.

Another version is:

Theorem 13 (Spectral theorem-2)
Given a special complex matrix A (hermitian, skew-hermitian or unitary),
there exists a unitary matrix U such that U ∗ AU = diag{λ1 , λ2 , ..., λn }
where λj are eigenvalues of A, each repeated as many times as its
algebraic multiplicity.

February 6, 2017 17 / 1
Spectral theorem contd.

The spectral theorem claims that for the special matrices the geometric
and algebraic multiplicities coincide i.e. each eigenvalue is defect-free.
Suppose that the eigen-basis consists of the vectors {v1 , v2 , ..., vn } in Cn ,
then G-S process produces an orthonormal basis {u1 , u2 , ..., un }
which continues to be an eigen-basis of A.
Hence if U = [u1 , u2 , ..., un ], then U is a unitary matrix and
U ∗ AU = diag{λ1 , λ2 , ..., λn }.
Spectral theorem for real symmetric matrices:
If A is real symmetric n × n, then one can sharpen the spectral theorem
since the eigenvalues are all real. The eigen-basis, therefore, can be found
in Rn itself. In this case the matrix U will be real orthogonal O say, and
OT AO = diag{λ1 , λ2 , ..., λn }. [1.5]

February 6, 2017 18 / 1
Applications, quadratic forms

Definition 14 (Quadratic form)

A homogeneous real second degree polynmial in n variables is called a real
quadratic form.

Thus it looks like

n
X
q(x) ≡ q(x1 , x2 , ..., xn ) = ajk xj xk , ajk = akj ∈ R.
j,k=1
X X
In practice we write it as q(x) = ajj xj2 + 2 ajk xj xk .
j j<k
It is useful to look at the real symmetric matrix A = [ajk ] of the
coefficients of Q(x) and verify that

Q(x) = xT Ax.

February 6, 2017 19 / 1
Quadratic forms contd.

By spectral theorem, there is an orthogonal O such that

OT AO = diag{λ1 , λ2 , ..., λn }.

Make change of variables

u = OT x
or EQUIVALENTLY substitute

x = Ou.

We get

Q̃(u) ≡ Q(Ou) = λ1 u12 + λ2 u22 + · · · + λn un2 .

Observe that the change of variables or the substitution is a rotation of

the standard coordinate axes.
February 6, 2017 20 / 1
Normal form of a quadratic surface

Let Q(x) be a quadratic form. The (hyper-)surface Q(x) = c is called a

quadratic (hyper-)surface in Rn . After a rotation of the coordinate axes, it
can be brought into the form

λ1 u12 + λ2 u22 + · · · + λn un2 = c.

If c 6= 0, we re-write the surface as

λ1 2 λ2 2 λn
u1 + u2 + · · · + un2 = 1,
c c c
c2
which is known as the normal form of Q = c. The quantities are the
λ2j
squares of lengths of the semi-axes of the surface.
If c = 0, there is no normal form. We deal with it in an ad-hoc manner.

February 6, 2017 21 / 1
Applications, Example of a quadratic surface

Example 15
Find the lengths of the semi-axes
√ and the type of the quadric surface
x + 2 3xz + 4yz − 3z 2 = −25.
2

Also find the line along which the shortest axis lies.

Solution:    √ 
x 1 0 3
The quadric is [x y z]A y  = −25 where A = √0 0 2  .
z 3 2 −3
3
The eigenvalue equation is −DA (λ) = λ + 2λ −2
√ 10λ + 4 = 0
=⇒ λ = 2, −2 ± 6. (All real)
Normal form is:
√ √
2 2 6−2 2 6+2 2
− u − v + w = 1.
25 25 25

February 6, 2017 22 / 1
Example contd.

Semi-axes (lenghts of) are:

5 5 5
√ , p√ , p√ .
2 6−2 6+2

Type: Hyperboloid of 1-sheet (with elliptic sections)

To find the directions of the semi-axes, we must solve for the eigenvectors
or equivalently find the orthogonal matrix O so that
√ √
OT AO = diag{2, −2 + 6, −2 − 6}.

The axes lie along the columns of O.

February 6, 2017 23 / 1
Example contd.

√
For the shortest axis, find eigenspace for the eigenvalue − 6 − √ 2 of A- the
largest in absolute value.
√ Thus √ solve for the null space of A + ( 6 + 2).
√3 − 2
Computations give  6 − 2  as a choice for an eigenvector. Hence the
−1
shortest axis lies along a line in space with parametric equations:
√ √ √
x = ( 3 − 2)t ≈ .3178t, y = ( 6 − 2)t ≈ .4495t, z = −t.

February 6, 2017 24 / 1
Example, a vibrating 2-spring system

Example 16
(A vibrating 2-spring system) Given the system of ODE

y100 = −5y1 + 2y2

y200 = 2y1 − 2y2 ,
 
y1 (t)
Find the solution vector y(t) = .
y2 (t)

Solution:  
−5 2
The ODE is y00
= Ay where A = . A has eigenvalues −1, −6
2 −2
   
1 1 1 2
with eigenvectors √ and √ respectively.
  5
2 5 −1
1 1 2
O=√ .
5 2 −1
February 6, 2017 25 / 1
Example 2, contd.

Put u(t) = OT y(t) to obtain the DECOUPLED system:

 00   
u1 (t) −u1 (t)
= .
u200 (t) −6u2 (t)

The well known  general

 solution is 
u1 (t) A cos
√ t + B sin t
√
u(t) = = and y(t) = Ou(t) solves the
u2 (t) C cos 6t + D sin 6t
given system.

The solutions form a vector space of 4 dimensions. How can you find a
basis? Which solutions are periodic?

February 6, 2017 26 / 1
Example 2, contd.

√ √
       
1 1 2 2
Answer:A basis is cos t , sin t , cos 6t , sin 6t
2 2 −1 −1
There are exactly two 2-dimensional spaces of periodic solutions. They are
d2
the eigenspaces E−1 and E−6 of the linear map 2 acting on the vector
dt
space of all the solutions of the system.

Remark: The de-coupling of the system shows that the 2 spring system in
series can be modelled by superimposition of two orthogonal simple
harmonic restoring forces acting on a particle in a plane.

Thus abstractly the complicated motion of compression and expansion of

the two springs can be ’seen’ via Lissajous figures traced by a single
particle.

February 6, 2017 27 / 1
Applications, the second derivative test in two variables

Let f (x, y ) be a function of two variables and P = (x0 , y0 ) be a critical

point of f i.e. fx (x0 , y0 ) = 0 = fy (x0 , y0). 
f f

The real symmetric matrix (D 2 f ) = xx xy is the ”second

P fyx fyy P
derivative” of f at P.
The second derivative test says that
 
P is a strict local minimum if (D 2 f ) (P) > 0 and fxx (P) > 0.
 
P is a strict local maximum if (D 2 f ) (P) > 0 and fxx (P) < 0.
 
P is a (strict) saddle point if (D 2 f ) (P) < 0.
 
The test fails if (D 2 f ) (P) = 0.

February 6, 2017 28 / 1
Applications, theory of the second derivative test

Theorem 17
 
a b
Let A = be a symmetric 2 × 2 real matrix and let and λ and µ be its
b d
eigenvalues. Then

(1) |A| > 0, a > 0 ⇐⇒ |A| > 0, d > 0 ⇐⇒ tr2 A > 0, trA > 0
⇐⇒ λ, µ > 0(both eigenvalues are positive).
(2) |A| > 0, a < 0 ⇐⇒ |A| > 0, d < 0 ⇐⇒ tr2 A > 0, trA < 0
⇐⇒ λ, µ < 0(both eigenvalues are negative).

(3) |A| = tr2 A < 0 ⇐⇒ λµ < 0(eigenvalues are of opposite signs).

(4) If |A| = 0, at least one eigenvalue is 0. Moreover, both
eigenvalues are zero if only if A = [O]-the zero matrix.

February 6, 2017 29 / 1
Applications, second derivative test in two variables
Proof: ad − b 2 = |A| = λµ. Rest is exercise. For (1) and (2) also need
a + d = λ + µ.
The case |A| = 0 is particularly boring. No wonder that the test fails.
Example 18
 
−4.6 7.2
Let H = be the matrix of the second partial derivatives at a
7.2 −0.4
stationary point of some function f (x, y ). Show that the stationary point
is a saddle and find the directions of steepest ascent and descent of the
surface z = f (x, y ) at the point.

Solution:The determinant |H| = 1.84 − (7.2)2 = −50 < 0. Hence the

stationary point is a saddle point. The eigenvalues of H are roots of the
characteristic polynomial

D(t) = t 2 + 5t − 50.

February 6, 2017 30 / 1
Example contd.

The eigenvalues of H are −10 and 5.

The corresponding unit eigenvectors are

   
0.8 0.6
± and ±
−0.6 0.8

respectively.
The steepest descent is parallel to the line 3x + 4y = 0, while the steepest
ascent is parallel to 4x − 3y = 0.

The two directions are mutually perpendicular (m1 m2 = −1). [2.0]

February 6, 2017 31 / 1
G-S process; an example in Cn

Example 19
Orthonormalize the ordered set in C5 :

{[1, ı, 0, 0, 0], [0, 1, ı, 0, 0], [0, 0, 1, ı, 0], [0, 0, 0, 1, ı]}

5
X
relative to the unitary inner product hv, wi = vw∗ = vj w̄j .
j=1
Use Bessel’s inequality to find whether [1, ı, 1, ı, 1] is in the (complex)
linear span of the above set or not.

February 6, 2017 32 / 1
G-S process; example contd.
Solution: {[1, ı, 0, 0, 0], [0, 1, ı, 0, 0], [0, 0, 1, ı, 0], [0, 0, 0, 1, ı]}.

v1 = w1 = [1, ı, 0, 0, 0]
−ı 1
w2 = [0, 1, ı, 0, 0] − [1, ı, 0, 0, 0] = [ı, 1, 2ı, 0, 0]
2 2
1
Similarly, w3 = [−1, ı, 1, 3ı, 0]
3
1
and w4 = [−ı, −1, ı, 1, 4ı].
4
The corresponding orthonormal (unitary) set is

1 1
√ [1, ı, 0, 0, 0], √ [ı, 1, 2ı, 0, 0],
2 6

1 1
√ [−1, ı, 1, 3ı, 0], √ [−ı, −1, ı, 1, 4ı] .
2 3 2 5

February 6, 2017 33 / 1
G-S process; example contd.

Next,
OTO1 H kvk2 = k[1, ı, 1, ı, 1]k2 = 5.

 |hv, u1 i|2 = |hv, u2 i|2 = 2/3,


2,
OTO2 H,
|hv, u3 i|2 = 4/3, |hv, u4 i|2 = 4/5.


The sum is 2 + 2/3 + 4/3 + 4/5 = 24/5 < 5,

hence can not be in the linear span.

February 6, 2017 34 / 1
Similarity
Example 20
   
1 1 2 λ
Show that ∼ . (∼ denotes similarity).
1 1 0 0
Solution: For both the matrices tr = 2 and det = 0. Hence both have the
same characteristic polynomial t 2 − 2t with simple eigenvalues 0, 2. Hence
both are ∼ diag{2, 0}.
Since similarity is an equivalence relation, the two matrices are mutually
similar too.
     
1 1 1 1 2 λ
Notes: For A = a diagonalizer is X = and for B = a
1 1 1 −1 0 0
 
1 −λ
diagonalizer is Y = .
0 2
    
−1 1 1 1 λ/2 1 (λ + 1)/2
Let P = XY = = . Then
1 −1 0 1/2 1 (λ − 1)/2
B = P −1 AP
is easy to verify.
February 6, 2017 35 / 1
Similarity

Example 21
Let A = [ajk ] be an n × n matrix.
Show that à = [(−1)j+k ajk ] is similar to A.
Solution: Let X = diag{−1, 1, ..., (−1)j , ..., (−1)n }. Observe that X = X −1 .
Then
XA = [(−1)j ajk ].
The j th row of A gets multiplied by the j th entry of X . Similarly,

XAX −1 = XAX = [(−1)j ajk ]X = [(−1)j (−1)k ajk ] = Ã.

The k th column of XA gets multiplied by the k th entry of X . Hence the result.

Note that this means that for a square matrix A, M(A) ∼ C(A). i.e. the
matrix of minors is similar to the matrix of co factors.

February 6, 2017 36 / 1
Jordan canonical form

Example 22
 
1 1 −1
Let A = −2 −1 2  . Show that A has only one eigenvalue λ with
2 0 −3
multiplicity 3 but only one eigenvector u, say. Solve successively, for
Av = λv + u and Aw = λw + v to find an ordered basis {u, v, w} so that
for X = [u, v, w] we have
 
λ 1 0
X −1 AX =  0 λ 1  .
0 0 λ

February 6, 2017 37 / 1
Jordan can. form contd.

D(λ) = −(λ + 1)3 =⇒ λ = −1, −1, −1. Solving for  the

 eigenvectors we
1
get only one (upto scalar multiples), namely u = −1 . Next on solving
1
Av = λv + u = −v + u, we have
     
2 1 −1  1 2 1 −1  1
 
     
(A + I)+ = −2 0 2  −1 7→  0 1 1  0 .
     
     
2 0 −2  1 0 0 0  0
 

 
1/2
Hence can take v =  0 - a particular solution.
0

February 6, 2017 38 / 1
Example contd.

Likewise Aw = λw + v yields
     
2 1 −1  1/2 2 1 −1  1/2
 
     
(A + I)+ = −2 0 2  0 → 7 0 1 1  1/2 .
     
     
2 0 −2  0 0 0 0  0
 

 
0
Hence can take w = 1/2- a particular solution.
0
 
1 1/2 0
X = [u v w] = −1 0 1/2
1 0 0

February 6, 2017 39 / 1
  
0 0 1
Lastly, X −1 = 2 0 −2 and
0 2 2
   
−1 1 0 λ 1 0
X −1 AX =  0 −1 1  =  0 λ 1  .
0 0 −1 0 0 λ

Remark: v and w are called generalized eigenvectors of A. They require

more effort by A − λI to kill them. Thus (A − λI)u = 0, but
(A − λI)v = u 6= 0. Make one more attempt

(A − λI)[(A − λI)v)] = (A − λI)u = 0.

Thus two stabs kill v. Likewise, w needs three stabs to get killed.
Theorem: For every n × n matrix A there exists a basis of Rn ( or Cn ) consisting
of generalized eigenvectors.
Proof: Out of syllabus. Statement itself is out of syllbus.

February 6, 2017 40 / 1