Eigenvalues and Eigenvectors

Philippe B. Laval

KSU

Fall 2015

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 1 / 14

Introduction

We de…ne eigenvalues and eigenvectors.

We discuss how to compute them.
We present their main properties.
We …nish with two applications, each could be a …nal project.

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 2 / 14

De…nitions

De…nition
A scalar is called an eigenvalue or a characteristic value of A if there
is a nontrivial solution x of Ax = x. Such a vector x is called an
eigenvector of A corresponding to the eigenvalue .

Let us note a few things:

1 Note that the zero vector x is always a solution of Ax = x, it is why
we are looking for nontrivial vectors x.
2 An eigenvalue can be 0, but not an eigenvector.

De…nition
The eigenspace of A corresponding to the eigenvalue is the set of all
eigenvectors of A corresponding to .

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 3 / 14

Computation
Recall, we are trying to solve Ax = x.

Ax = x ()Ax x=0
() Ax Ix = 0
() (A I)x = 0

Two things can happen.

1 (A I ) is invertible that is det (A I ) 6= 0. The only solution is
the trivial solution. This is not of interest to us since we are seeking a
nontrivial solution.
2 (A I ) is not invertible that is det (A I ) = 0. This is the only
case in which we can hope to …nd a solution.

De…nition
det (A I ) is a polynomial of degree n. It is called the characteristic
polynomial.
Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 4 / 14
Computation
1 det (A I ) is a polynomial of degree n, hence it has n roots, real
and/or complex. Some of which may be repeated.
2 This means that if A is an n n matrix then it will have n
eigenvalues, call them 1 ; 2 ; :::; n . Some may be repeated.
3 If an eigenvalue appears only once in the list, it is called simple.
4 If an eigenvalue appears k > 1 times in the list, we say that has
multiplicity k.
5 If 1 ; 2 ; :::; k (k n) are the simple eigenvalues in the list, with
corresponding eigenvectors x(1) ; x(2) ; ::; x(k ) , then the eigenvectors are
linearly independent.
6 If is an eigenvalue with multiplicity k > 1 then will have
anywhere from 1 to k linearly independent eigenvectors.
7 If x is an eigenvector corresponding to then kx is also an
eigenvector corresponding to . This means that eigenvectors are
de…ned up to a constant.
Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 5 / 14
Computation

Example
1 3
Find the eigenvalues and eigenvectors of A =
1 5

Example
0 1
2 1 1
Find the eigenvalues and eigenvectors of A = @ 0 0 2 A
0 3 5

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 6 / 14

Eigenvalues and MATLAB
The MATLAB function to get the eigenvalues of a matrix is eig. It can be
used di¤erent ways; we only show a few here. For a complete list, type
help eig within MATLAB.
1 Given an n n matrix A, eig (A) will display the eigenvalues of A.
Each eigenvalue will be printed as many times as its multiplicity.
2 Given an n n matrix A, s = eig (A) will …nd the eigenvalues of A
and store them into the n 1 vector s. As above, each eigenvalue will
appear as many times as its multiplicity.
3 Given an n n matrix A, [V D] = eig (A) will …nd the eigenvalues
and eigenvectors of A. The eigenvectors of A will be stored in V as
column vectors. So, V is in fact a matrix. The eigenvalues of A will
be stored on the diagonal of D, the remaining entries of D being
zeros. The eigenvalues will appear in the same order as the
eigenvectors. Note that MATLAB will …nd eigenvectors which are
unit vectors (magnitude 1).

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 7 / 14

Eigenvalues and MATLAB

Example
0 1
0 0 2
Test the eig function with A = @ 1 2 1 A.
1 0 3

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 8 / 14

Properties of Eigenvalues and Eigenvectors

We list some important properties. Their proof can be found in any linear
algebra book.
It will be useful to remember some properties of determinants.
1 1
det A =
det (A)
det (AB) = det (A) det (B)
det AT = det (A)
det (cA) = c n det (A)
Powers of a matrix: If Ax = x then
A2 x = A (Ax) = A ( x) = Ax = 2 x. In general,

An x = n
x

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 9 / 14

Properties of Eigenvalues and Eigenvectors

Eigenvalues of a triangular or diagonal matrix. Remembering that

the determinant of a triangular or a diagonal matrix is the product of
its entries on the diagonal, we see that the characteristic polynomial
Yn
of such a matrix is (aii ) hence the eigenvalues are aii for
i =1
i = 1::n.
Similar Matrices: Recall that two matrices A and B are similar if
there exists an invertible matrix P of the same size such that
B = PAP 1 . Two similar matrices have the same eigenvalues.
1
is an eigenvalue of A if and only if is an eigenvalue of A 1 .
If = a + bi is an eigenvalue of A with eigenvector v then = a bi
is also an eigenvalue of A and its corresponding eigenvector is the
conjugate of v.
Symmetric Matrices always have real eigenvalues.
Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 10 / 14
Properties of Eigenvalues and Eigenvectors

Diagonalization. Suppose that v1 ; v2 ; :::; vn are linearly independent

vectors and 1 ; 2 ; :::; n are their corresponding eigenvalues. De…ne
P = [v
01 v2 ::: vn ] (note this1is an n n matrix) and
1 0 0
B . . . C
B 0
B 2 . .. CC then AP = PD that is A = PDP 1 or
D=B . . .. C
@ . . . . . 0 A
0 0 n
1
D = P AP which means that the eigenvalues of A and D are the
same. We can make the following important conclusions:
If P is as de…ned, then P 1 AP is a diagonal matrix.
det D = det P 1 AP = det P 1 det (A) det (P) = det (A) hence the
determinant of a matrix is the product of its eigenvalues. Similarly, the
trace of a matrix is the sum of its eigenvalues.
The above implies that a matrix is invertible if and only if none of its
eigenvalues is zero.

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 11 / 14

Properties of Eigenvalues and Eigenvectors

Powers Matrix Revisited: If A = PDP 1 then

A2 = PDP 1 PDP 1 = PD 2 P 1 . Similarly, An = PD n P 1.

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 12 / 14

Applications

An important application of eigenvalues and eigenvectors is with

solving systems of …rst order di¤erential equations.
Google’s page ranking algorithm uses a lot of linear algebra, including
eigenvalues and eigenvectors. Here is a paper by Bryan and Leise on
Google’s PageRank algorithm.
Eigenvalues for face recognition (eigenfaces). Here is the paper which
started it all by Turk and Pentland.

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 13 / 14

Assignment

1 In the last section of this document, read and understand the paper
on Google page ranking. This could be a potential project.
2 In the last section of this document, read and understand the paper
on eigenfaces. This could be a potential project.

Philippe B. Laval (KSU) Eigenvalues and Eigenvectors Fall 2015 14 / 14