EE547: Fall 2018

Lecture 4: Eigenvalues and Eigenvectors

Lecturer: L.J. Ratliff

Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications.
They may be distributed outside this class only with the permission of the Instructor.

Chapter 3 & 5 of Linear Alg Done Right; App A, B of C&D

4.1 Eigenvalues and Eigenvectors

Remark 1.1 (Why do we care? ) Eigenvalues and eigenvectors are important for
• assessing stability; recall the example
ẋ = −λx
whose solution is x(t) = x0 e−λt and if λ > 0 solution decays to zero, otherwise it blows up. λ is an
’eigenvalue’ for this scalar system.
• as we will see, it is also important for assessing controllability and observability

Let A ∈ Cn×n . If there exists α ∈ C and v ∈ Cn , v 6= 0, such that

Av = λv

then λ is an eigenvalue of A and v is an eigenvector of A.

Note that if v is an eigenvector, then any nonzero multiple of v is an eigenvector.

Definition 1.1 (Spectrum.) The set of all eigenvalues of A is known as the spectrum of A, denoted by
σ(A). The spectral radius of A is defined as

ρ(A) = max{|λ|| λ ∈ σ(A)}

The definition of eigenpairs (x, λ)—i.e. Ax = λx—is equivalent to

(A − λI)x = 0

Thus, λ is an eigenvalue of A iff the linear system (A − λI)x = 0 has a nontrivial—i.e. x 6= 0—solution.

This, in turn, is equivalent to

det(A − λI) = 0.
Therefore we define the so-called characteristic polynomial of A.

Definition 1.2 (Characteristic Polynomial.)

χA (λ) = det(A − λIn ) = α0 + α1 λ + α2 λ2 + · · · + αn λn

The eigenvalues of A are the roots of the characteristic polynomial:

χA (λ) = det(A − λIn ) = α0 + α1 λ + α2 λ2 + · · · + αn λn

4-1
4-2 Lecture 4: Eigenvalues and Eigenvectors

where In is the n × n matrix identity. The characteristic polynomial is an n-degree polynomial.

Fact 1.1 λ is an eigenvalue of A iff χA (λ) = 0.

Example 1.1 It is well known that even real matrices can have complex eigenvalues. For instance,
 
0 1
A=
−1 0

has a characteristic polynomial

 
z −1
χA (z) = det = z2 + 1
1 z

so that its eigenvalues are λ1,2 = ±i with associated eigenvalues

   
1 i
x1 = , x2 =
i 1

Fact 1.2 if A is symmetric (or Hermitian), then all its eigenvalues are real. That is, if A = A∗ = ĀT
(aij = āji ).

Let λ1 , λ2 , . . . , λp be the distinct eigenvalues of A. Then

det(A − λIn ) = (λ1 − λ)m1 (λ2 − λ)m2 · · · (λp − λ)mp

Pp
where i=1 mi = n and mi is the algebraic multiplicity of λi .
Hence, given λ1 , . . . , λp , the coefficients αi are determined by solving the above equation.

Suppose we define
χA (X) = α0 I + α1 X + α2 X 2 + · · · + αn X n

Theorem 1.1 (Cayley-Hamilton.) For any square matrix A with elements in a field F ,

χA (A) = 0

i.e.
0 = α0 I + α1 A + α2 A2 + · · · + αn An

4.2 Range and Null Spaces: Rank and Nullity

Remark 2.1 (Why do we care? ) Range and null spaces are going to be important for characterizing
the reachable/controllable and observable subspaces.

Consider the linear map A : U → V where dim(U ) = n and dim(V ) = m. Let A be its matrix representation.

4.2.1 Range and Null Space with Matrix

For a m × n matrix (linear operator) A : Cn → Cm , the range space or image of A is defined to be

R(A) = {w ∈ Cm | w = Av, v ∈ Cm } ⊂ Cm
Lecture 4: Eigenvalues and Eigenvectors 4-3

and the null space or kernel of A is defined to be

N (A) = {v ∈ Cn | Av = 0} ⊂ Cn

Fact 2.1 The range space and the null space are both linear subspaces. As a consequence, they have
well-defined dimensions denoted by dim(R(A)) and dim(N (A)).

Proof. Let v1 and v2 be any elements of N (A), and let α1 , α2 ∈ C. We have

A(α1 v1 + α2 v2 ) = α1 Av1 + α2 Av2 = α1 0W + α2 0W = 0W

and thus
α1 v1 + α2 v2 ∈ N (A)
which shows that N (A) is a subspace.

Now consider w1 , w2 ∈ R(A). Then there must exist v1 , v2 ∈ Cm such that

w1 = Av1
w2 = Av2

Now consider α1 w1 + α2 w2 for some α1 , α2 ∈ C. We have

α1 w1 + α2 w2 = α1 Av1 + α2 Av2 = A(α1 v1 + α2 v2 )

Since α1 v1 + α2 v2 ∈ Cm , it follows that α1 w1 + α2 w2 ∈ R(A) and thus R(A) is a subspace.

Definition 2.1 (Rank.) The rank of a matrix A ∈ F m×n is dim(R(A)).

We use the notation rank(A) or rk(A) for the ‘rank’ of A.

Definition 2.2 (Nullity.) The nullity of A ∈ F m×n is dim(N (A))

We use the notation nullity(A).

Example 2.1 For example, consider the linear map T : R4 → R2 defined by

T (x, y, z, w) = (0, 0)

For this map,

1. The image (i.e. range) of T is the set {(0, 0)}, because T outputs (0, 0) on every input.
2. The null space (i.e. kernel) of T is all of R4 because T sends every element of R4 to (0, 0)
Similarly, consider the map T : R4 → R defined such that

T (x, y, z) = x + y + z

DIY Exercise. show this map is linear.

1. What is the image?
sol. The image is all of R. This is because on input

T (a, 0, 0) = a + 0 + 0 = a, ∀ a ∈ R

Therefore, we can get any real number as an output of T . What do we call maps like this? surjective
4-4 Lecture 4: Eigenvalues and Eigenvectors

2. What is the kernel/null space?

sol. The null space of T is the collection of all triples (a, b, c) such that

T (a, b, c) = a + b + c = 0.

In other words, if we solve for c in terms of the other two variables, it’s the collection

{(a, b, −a − b) ∈ R3 | a, b ∈ R} ⊂ R3

Given the notion of ‘null space’, we can define an eigenbasis for a linear map A.

Definition 2.3 (Eigenbasis.) An eigenbasis corresponding to A is a basis for V consisting entirely of

eigenvectors for A.

To find an eigenbasis, you find a basis for each eigenspace of A where the null space of A − λI is called the
eigenspace associated with λ. That is, the vector space Eλ of all eigenvectors corresponding to λ:

Eλ = span{x| Ax = λx, λ ∈ C}

The collection of all these basis vectors for an eigenbasis for A.

Example 2.2 Let (v, λ) be an eigenvalue/eigenvector pair for A ∈ Cn×n . The nullity of λI − A is called
the geometric multiplicity of λ. In particular, the null space of λI − A (or the eigenspace of A for λ) is
the space of all eigenvectors of A for λ and the zero vector. Its dimension is the geometric multiplicity of
λ.

Note: It is called geometric because it refers to the dimension of a particular space. On the other hand,
the algebraic multiplicity of λ is the number of times λ is a root of the characteristic polynomial for A—
i.e. det(A − λI) = 0. We will return to these concepts as they are important for understanding stability as
well as characterizing controllability and observability.

Example 2.3 Consider  

1 0 0
A = 1 1 1
0 0 1
so that χA (z) = (z − 1)3 . Thus, λ = 1 is an eigenvalue (in fact, the only one) of A with algebraic
multiplicity 3. To determine its geometric multiplicity we need to find the associated eigenvectors. Thus
we solve (A − λI)x = 0 for the special case of λ = 1. That is
 
0 0 0
(A − I) = 1 0 1 x = 0
0 0 0

Hence x1 = −x3 from the second row or

     
α 1 0
x =  β  = α  0  + β 1
−α −1 0

and thus the geometric multiplicity of λ = 1 is only 2 since all vectors in the null space can be written in
terms of the two basis vectors given.

The following are some very useful theorems in linear algebra.

Fact 2.2 A linear transformation T : V → W is injective if and only if ker(T ) = {0}.

Lecture 4: Eigenvalues and Eigenvectors 4-5

Definition 2.4 (Injectivity ) A function f : X → Y is injective if, for every pair x1 6= x2 of elements of
X, f (x1 ) 6= f (x2 ).

Proof. Suppose T is injective. Since T (0) = 0, if T (v) = 0, then v = 0. Hence,

v ∈ ker(T ) =⇒ v = 0

i.e., ker(T ) = {0}. Conversely, suppose that ker(T ) = {0}. If T (v) = T (w), then T (vw) = 0, and hence
vw = 0. So v = w. Hence,
T (v) = T (w) =⇒ v = w
i.e., T is injective.

Theorem 2.1 (Rank-Nullity Theorem.) For a linear map A : F n → F m , the following holds

dim R(A) + dim N (A) = n

Proof. DIY Exercise

Proof Sketch. Note that N (A) ⊂ F n while R(A) ⊂ F m . Consider a basis for N (A) where dim N (A) = k.
Complete that basis (i.e. generate n − k linearly independent vectors independent of the original basis for
N (A)). Consider the representation of x w.r.t. this completed basis.
To prove the Rank-Nullity theorem, you need to know that you can extend a basis of a subspace to a basis
for the whole space.

Lemma 2.1 Let (U, F ) be a finite-dimensional linear space with dim U = n. Suppose {u1 , u2 , . . . , um } is
a set of m linearly independent vectors. There exists n − m additional vectors {ũm+1 , . . . , ũn } such that

{u1 , u2 , . . . , um , ũm+1 , . . . , ũn }

forms a basis for U —i.e. any set of linearly independent vectors can be extended to a basis.

Proof. If m = n, no extension is needed. Hence we assume m < n. It follows that {u1 , . . . , um } is not a
basis for U so there must exists a v ∈ U such that
m
X
αi ui 6= v, ∀ {αi }m
i=1 ⊂ F
i=1

Then,
{u1 , . . . , um , v}
is linearly independent so that we can take ũm+1 = v. The lemma follows by induction.

Fact 2.3 Let A ∈ F m×n be a matrix. The rank-nullity theorem gives us that

0 ≤ rankA ≤ min{m, n}

Hence, rankA is equal to the

a. maximum number of linearly independent column vectors of A, and
b. maximum number of linearly independent row vectors of A
4-6 Lecture 4: Eigenvalues and Eigenvectors

That is, the following fundamental result from linear algebra holds:

rankA = rowrankA = colrankA

where row rank is the row rank of A (i.e. maximum number of linearly independent row vectors of A) and
col rank is the column rank of A (i.e. maximum number of linearly independent column vectors of A).
The column rank of A is the dimension of the column space of A, while the row rank of A is the dimension
of the row space of A.

Theorem 2.2 (Sylvester’s Inequality.) Let A ∈ F m×n and B ∈ F n×p , then AB ∈ F m×p and

rank(A) + rank(B) − n ≤ rank(AB) ≤ min{rank(A), rank(B)}.

Proof. DIY Exercise. Hint: Let W be the co-domain of A and let A|R(B) : R(B) → W be the restriction of
A to R(B). Apply the rank-nullity theorem to A|R(B) .

Remark 2.2 (Why important? ) Sylvester’s inequality and the rank-nullity theorem are the main
tools to prove the following fact, which is very useful for our analysis since we will see that transformation,
e.g., to the controllable canonical form does not change key facts about the system which allow us to
analyze its properties.

Fact 2.4 Rank and nullity are invariant under equivalence (i.e. if two matrices are equivalent via similarity
transformation, then they have the same rank and nullity).

Fact 2.5 In fact, two similar matrices A and Ā = P −1 AP have the same spectrum (i.e. same eigenvalues).
This is important because we can transform our system and preserve key properties like controllability,
observability, stability.

Fact 2.6 Similar matrices have the same characteristic polynomial, eigenvalues, algebraic and geometric
multiplicities. The eigenvectors, however, are in general different.

Theorem 2.3 (A) matrix A ∈ Cm×m is nondefective—i.e., has no defective eigenvalues meaning the
geometric and algebraic multiplicites are equal—if and only if A is similar to a diagonal matrix, i.e.,

A = XΛX −1

where  
X = x1 x2 ··· xm
is the matrix formed with the eigenvectors of A as its columns, and Λ = diag(λ1 , . . . , λm ).

we will talk more about this when we discuss Jordan form, but essentially there is a full set of eigenvectors.