L05 Eigenvalues and Linear Transformation

Lecture 5
Eigenvalues and
Linear Transformation
Dr. Norhazwani Md Yunos
Department of Intelligent Computing & Analytics

Faculty of Information & Communication Technology
Universiti Teknikal Malaysia Melaka
Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 1 / 47

Lecture Contents
Lecture Contents:
1 Eigenvalues and Eigenvectors

Introduction
Computation of Eigenvalues and Eigenvectors
Diagonalization of Symmetric Matrices
2 General Linear Transformation

Some Linear Transformations on R2
Composition of Transformation
Application

Eigenvalues and Eigenvectors
Lecture Contents:

Introduction

Application

Eigenvalues and Eigenvectors Introduction
In this subtopic, we will study on classes of scalars and vectors known as

“eigenvalues” and “eigenvectors”.
The idea first appeared in the study of rotational motion, but was later
used to classify various kinds of surfaces and to describe solutions of
certain differential equations.
In the early 1900s, it was applied to matrices and matrix transformations,
and today it has applications in such diverse fields as computer graphics,
mechanical vibrations, heat flow, population dynamics, quantum
mechanics, and economics, to name just a few.

Eigenvalues and Eigenvectors
Definition of Eigenvalue and Eigenvector:

If A is an n × n matrix, then a nonzero vector x in Rn is called an eigenvector
of A if Ax is a scalar multiple of x, i.e.,
Ax = λx
for some scalar λ. The scalar λ is called an eigenvalue of A, and x is said to

be an eigenvector corresponding to λ.

Example 1:
The vector
1
x=
2
is an eigenvector of
3 0
8 −1
corresponding to the eigenvalue λ = 3, since

3 0 1 3
Ax = = = 3x
8 −1 2 6

Eigenvalues and Eigenvectors Computation of Eigenvalues and Eigenvectors
Computation of Eigenvalues
In Example 1, we observed that λ = 3 is an eigenvalue of the matrix

3 0
8 −1
but, how to find it?


3 0
8 −1
but, how to find it?
Theorem:
If A is an n × n matrix, then λ is an eigenvalue of A if and only if it satisfies
the equation
det(λI − A) = 0
This is called the characteristic equation of Aa .
a
The system of equations (λI − A) = 0 has nontrivial solutions


3 0
8 −1
but, how to find it? → the answer is using the characteristic equation.
Theorem:
If A is an n × n matrix, then λ is an eigenvalue of A if and only if it satisfies
the equation
det(λI − A) = 0
This is called the characteristic equation of Aa .
a
The system of equations (λI − A) = 0 has nontrivial solutions

Revisited Example 1:
Determine the eigenvalues of

3 0
A=
8 −1
Solution:
Based on characteristic equation of A;
Step 1 (Form the matrix λI − A):

1 0 3 0
λI − A = λ −
0 1 8 −1

λ 0 3 0
= −
0 λ 8 −1

λ−3 0
=
−8 λ + 1

Step 2 (Take the determinant of λI − A):

λ−3 0
det = (λ − 3)(λ + 1) − 0(−8)
−8 λ + 1
= (λ − 3)(λ + 1)

Step 2 (Take the determinant of λI − A):

λ−3 0
det = (λ − 3)(λ + 1) − 0(−8)
−8 λ + 1
= (λ − 3)(λ + 1)
Step 3 (Let det(λI − A) = 0):
(λ − 3)(λ + 1) = 0
λ = 3 and λ = −1
This shows that the eigenvalues of A are λ = 3 and λ = −1.

Thus, in addition to the eigenvalue λ = 3 as stated in Example 1, we have
discovered a second eigenvalue λ = −1.

When the determinant det(λI − A) is expanded, the characteristic

equation of A takes the form
λn + c1 λn−1 + . . . + cn = 0
where the left side of this equation is a polynomial of degree n in which

the coefficient of λn is 1.
The polynomial
p(λ) = λn + c1 λn−1 + . . . + cn
is called the characteristic polynomial of A.

For example, recall Example 1, det(λI − A) = (λ − 3)(λ + 1), hence the
characteristic polynomial of A is λ2 − 2λ − 3, which is a polynomial of
degree 2.

Computation of Eigenvectors
In Revisited Example 1, we had derived the eigenvalues of the matrix

3 0
A=
8 −1
are λ = 3 and λ = −1.

Next, we would like to derive the eigenvector of the matrix A associated with
the eigenvalues, respectively.
By definition,
x is an eigenvector of A corresponding to an eigenvalue λ if and only if
(λI − A)x = 0.

Revisited Example 1:
Given λ = 3 and λ = −1. Determine the eigenvector of

3 0
A=
8 −1
Solution:
For λ = 3:
(λI − A)x = 0
Thus,
3 0 3 0 x1 0
− = 1 1
0 3 8 −1 x2 0 2 s
x= =s 2

0 0

x1 0 s 1
=
−8 4 x2 0 Therefore,
1
Solving this system (nontrivial 2
x=
homogenous system), give 1
1
x1 = x2 , x2 = any number s.
2
For λ = −1:
(λI − A)x = 0 Thus,

−1 0 3 0 x1 0
− = 0 0
0 −1 8 −1 x2 0 x= =s
s 1
−4 0 x1 0
=
−8 0 x2 0 Therefore,

Solving this system (nontrivial 0
x=
homogenous system), give 1
x1 = 0, x2 = any number s.

Eigenvalues and Eigenvectors Diagonalization of Symmetric Matrices
Diagonalization
Products of the form P−1 AP in which A and P are n × n matrices and P
is invertible will be our main topic of study in this section.
There are various ways to think about such products, one of which is to
view them as transformations
A → P−1 AP
in which the matrix A is mapped into the matrix P−1 AP.
These are called similarity transformations.
Such transformations are important because they preserve many
properties of the matrix A.
For example, if we let B = P−1 AP, then A and B have the same
determinant since
det(B) = det(P−1 AP) = det(P−1 ) det(A) det(P)
1
= det(A) det(P) = det(A)
det(P)
Definition 1:
If A and B are square matrices, then we say that B is similar to A if there is an
invertible matrix P such that B = P−1 AP.
Note that, if B is similar to A, then it is also true that A is similar to B.

Because diagonal matrices have such a simple form, it is natural to
inquire whether a given n × n matrix A is similar to a matrix of this type.
Definition 2:
A square matrix A is said to be diagonalizable if it is similar to some diagonal
matrix, i.e., if there exists an invertible matrix P such that P−1 AP = diagonal
matrix D. In this case, the matrix P is said to diagonalize A.

A Procedure for Diagonalizing an n × n Matrix
Step 1. Find n linearly independent eigenvectors of A, says p1 , p2 , . . . , pn . If

there is a total of n vectors, then the matrix is diagonalizable, and if the
total is not equal to n, then it is not.
Step 2. Form the matrix P having p1 , p2 , . . . , pn as its column vectors.
Step 3. The resulting matrix multiplication of P−1 AP will be a diagonal

matrix, D, whose successive diagonal entries are the eigenvalues
λ1 , λ2 , . . . , λn that correspond to the successive columns of P.

Example 2:
Let
3 0
A= .
8 −1
Find the nonsingular matrix P and the diagonal matrix D such that
AP = PD (rewrite from D = P−1 AP)
Solution:
From Example 1, we had derived that λ1 = 3 and λ2 = −1, while the
eigenvector associated with λ1 is
1
x1 = 2
1
and the eigenvector associated with λ2 is

0
x2 = .
1
Step 1:
We had derived that 1
2 0
x1 = and x2 = .
1 1
are two linearly independent eigenvectors of A.

Step 1:
2 0
x1 = and x2 = .
1 1
Step 2:
From Step 1., the matrix P is
1
2 0
P= .
1 1

Step 1:
2 0
x1 = and x2 = .
1 1
Step 2:
From Step 1., the matrix P is
1
2 0
P= .
1 1
Step 3:
The diagonal matrix D is

λ1 0 3 0
D= = .
0 λ2 0 −1
* You may check using AP = PD formula.

Example 3:
Show that the following matrix is not diagonalizable:
 
1 0 0
A =  1 2 0
−3 5 2
Solution:
Step 1:
Find the eigenvalues and eigenvectors.

Example 3:
 
1 0 0
A =  1 2 0
−3 5 2
Solution:
Step 1:
The characteristic polynomial of A is

λ − 1 0 0
0 = (λ − 1)(λ − 2)2

det(λI − A) = −1 λ − 2
3 −5 λ − 2
so the characterictis equation is
(λ − 1)(λ − 2)2 = 0
and the distinct eigenvalues of A are λ1 = 1 and λ2 = 2.

Example 3:
 
1 0 0
A =  1 2 0
−3 5 2
Solution:
Step 1:
The eigenvector associated with The eigenvector associated with
λ1 = 1 is λ2 = 2 is
 1   
8 0
x1 = − 18  x2 = 0

1 1

Example 3:
 
1 0 0
A =  1 2 0
−3 5 2
Solution:
Step 1:
The eigenvector associated with The eigenvector associated with
λ1 = 1 is λ2 = 2 is
 1   
8 0
x1 = − 18  x2 = 0
1 1
Since A is a 3 × 3 matrix and there are only two eigenvectors in total, hence A
is not diagonalizable.
General Linear Transformation
Lecture Contents:

Introduction

Application

Definition:
If L : V → W is mapping from a vector space V to a vector space W, then L is
called a linear transformation from V to W if the following two properties
hold for all vectors u and v in V and for all scalars k:
1 L(u + v) = L(u) + L(v)
2 L(ku) = kL(u)
In the special case where V = W, the linear transformation L is called a linear

operator on the vector space V.

Example 4:
Let V be a 2 × 2 matrix and W be a polynomial of degree 3, P3 . Define L to
be the linear function from V to W.
Solution:
Let
a b
L = at3 + bt2 + ct + d
c d

Example 4:
Solution:
Let
a b
L = at3 + bt2 + ct + d
c d
First property, shows that L(u + v) = L(u) + L(v):

a1 b1 a2 b2
L + = (a1 + a2 )t3 + (b1 + b2 )t2 + (c1 + c2 )t + (d1 + d2 )
c1 d1 c2 d2

a b1 a2 b2
L 1 +L = a1 t3 + a2 t3 + b1 t2 + b2 t2 + c1 t + c2 t + d1 + d2
c1 d1 c2 d2
= (a1 t3 + b1 t2 + c1 t + d1 ) + (a2 t3 + b2 t2 + c2 t + d2 )
= L(u) + L(v)

Example 4:
Solution:
Let
a b
L = at3 + bt2 + ct + d
c d
Second property, shows that L(ku) = kL(u):

a b
L k = kat3 + kbt2 + kct + kd
c d

ka kb
L = kat3 + kbt2 + kct + kd
kc kd

a b
kL = k(at3 + bt2 + ct + d)
c d

General Linear Transformation Some Linear Transformations on R2
Some Linear Transformation on R2
Let’s look at some linear transformations on the plane R2 .

We’ll look at several kinds of operators on R2 including scaling,
reflection, rotation and projection.

Scaling: Contraction and Expansion
(b) Contraction
(a) Original picture
(c) Expansion

Contraction:
L : R2 → R2 is define by

u1 u
L(u) = L = r 1 = ru, 0<r<1
u2 u2
Expansion:

u1 u
L(u) = L = r 1 = ru, r>1
u2 u2

Example 5:
Find the image of the triangle with vertices (−1, −1), (3, 0) and (0, 3) when it
is scaled to 200% from the original size.
Solution:
Scaling to 200%, means expansion with r = 2. Hence, L : R2 → R2 is define
by
xi x rxi
L(u) = l =r i =
yi yi ryi
Thus,

−1 −2 3 6 0 0
L = , L = , L =
−1 −2 0 0 3 6

Reflection
(a) Original picture (b) Reflection with respect to x−axis


u1 u
L(u) = L = A 1 = u0
u2 u2
where A is a matrix 2 × 2 which varies depending on the plane of reflection,

as follows:

1 0
A= reflection with respect to x−axis
0 −1

−1 0
A= reflection with respect to y−axis
0 1

0 −1
A= reflection with respect to y = −x
−1 0

Example 6:
Find the image of the reflection with respect to x−axis for the triangle with
vertices (−1, 4), (2, 6) and (3, 1).
Solution:
For reflection with respect to x−axis,

xi 1 0 xi xi
L(u) = L = =
yi 0 −1 yi −yi

Example 6:
vertices (−1, 4), (2, 6) and (3, 1).
Solution:

xi 1 0 xi xi
L(u) = L = =
yi 0 −1 yi −yi
Thus,

−1 −1 2 2
L = , L = ,
4 −4 6 −6

3 3
L =
1 −1

Example 6:
vertices (−1, 4), (2, 6) and (3, 1).
Solution:

xi 1 0 xi xi
L(u) = L = =
yi 0 −1 yi −yi
Thus,

−1 −1 2 2
L = , L = ,
4 −4 6 −6

3 3
L =
1 −1

Rotation
(c) Rotation of 90◦

(a) Original picture (b) Rotation of 10◦


u1 u
L(u) = L = A 1 = u0
u2 u2
where A is a matrix 2 × 2 describes a rotation of the plane by an angle of θ,

cos θ − sin θ
A=
sin θ cos θ
For example, when θ = 90◦ ,
cos 90◦ − sin 90◦

0 −1
A= =
sin 90◦ cos 90◦ 1 0

Example 7:
Find the image of the 90◦ rotation for the triangle with vertices (0, 0), (1, 0)
and (1, 1).
Solution:
For 90◦ rotation, L : R2 → R2 is define by

xi 0 −1 xi −yi
L(u) = L = =
yi 1 0 yi xi

Example 7:
and (1, 1).
Solution:

xi 0 −1 xi −yi
L(u) = L = =
yi 1 0 yi xi
Thus,

0 0 1 0
L = , L = ,
0 0 0 1

1 −1
L =
1 1

Example 7:
and (1, 1).
Solution:

xi 0 −1 xi −yi
L(u) = L = =
yi 1 0 yi xi
Thus,

0 0 1 0
L = , L = ,
0 0 0 1

1 −1
L =
1 1

Projection
(b) Projection to x−axis

(a) Original picture

u1 u
L =A 1
u2 u2
where A is a matrix 2 × 2 describes a projection, such as,

1 0
A= projection from 2-space onto the x−axis
0 0
Example 8:
Find the image of the projection from 2-space onto the x−axis for the triangle
with vertices (0, 1), (−2, 3) and (4, 5).
Solution:
For the projection from 2-space onto the x−axis, L : R2 → R2 is define by

xi 1 0 xi x
L = = i
yi 0 0 yi 0
Thus,

0 0 −2 −2
L = , L = ,
1 0 3 0

4 4
L =
5 0

Example 8:
Find the image of the projection from 2-space onto the x−axis for the triangle
with vertices (0, 1), (−2, 3) and (4, 5).
Solution:
For the projection from 2-space onto the x−axis, L : R2 → R2 is define by

xi 1 0 xi x
L = = i
yi 0 0 yi 0
Thus,

0 0 −2 −2
L = , L = ,
1 0 3 0

4 4
L =
5 0

General Linear Transformation Composition of Transformation
Suppose that we have two linear transformations induced by the matrices
A and B,
LA : Rn → Rk and LB : Rk → Rm
If we take out x out of Rn and first apply LA to x and then apply LB to the
result, we will have a transformation from Rn to Rm .
This process is called composition of transformations and is denoted as
(LA ◦ LB )(x) = LB (LA (x))
Note that, the order is important! The first transformation to be applied is
on the right and the second is on the left.
Since both of our original transformations were linear, then
(LA ◦ LB )(x) = LB (LA (x)) = LB (Ax) = BA(x)
In other words, the composition LA ◦ LB is the same as multiplication by
BA. The composition will be a linear transformation provided that the
two original transformation were also linear.
Example 9:
Let LA : R2 → R2 be a linear transformation defined by

x x
L =A
y y
where A is a matrix that shows a reflection with respect to y−axis, and

LB : R2 → R2 be a linear transformation defined by

x x
L =B
y y
where B is a matrix that shows a reflection with respect to x−axis.

Determine the matrix induced by the composition of reflection with respect to
the y−axis followed by reflection with respect to the x−axis.

Solution:
Recall,

−1 0
A= reflection with respect to y−axis, and
0 1

1 0
B= reflection with respect to x−axis
0 −1
So, the matrix induced by the composition of the linear transformations is

1 0 −1 0 −1 0
LA ◦ LB = BA = =
0 −1 0 1 0 −1

General Linear Transformation Application
Application: Cryptography
Suppose we want to send the following message to our friend,

MEET TOMORROW
For the security, we first code the alphabet as follows:
A B C ... X Y Z
1 2 3 ... 24 25 26
Thus, the code message is
M E E T T O M O R R O W
13 5 5 20 20 15 13 15 18 18 15 23
The sequence
13 5 5 20 20 15 13 15 18 18 15 23
is the original code message.

To encrypt the original code message, we can apply a linear transformation to

original code message. Let
L : R3 → R3 , L(x) = Ax
where  
1 2 3
A = 1 1 2
0 1 2
Next, break the original message into 4 vectors first, i.e.,
       
13 20 13 18
 5  , 20 , 15 , 15
5 15 18 23

and use the linear transformation to obtain the encrypted code message:
        
13 13 1 2 3 13 38
L  5  = A  5  = 1 1 2  5  = 28 ,
5 5 0 1 2 5 15
        
20 20 1 2 3 20 105
L 20 = A 20 = 1 1 2 20 =  70  ,
15 15 0 1 2 15 50
        
13 13 1 2 3 13 97
L 15 = A 15 = 1 1 2 15 = 64 and
18 18 0 1 2 18 51
        
18 18 1 2 3 18 117
L 15 = A 15 = 1 1 2 15 =  79 
23 23 0 1 2 23 61
Then, we can send the encrypted message code:

38 28 15 105 70 50 97 64 51 117 79 61
Suppose our friend wants to encode the encrypted message code. Our friend
can find the inverse of a matrix A,
 −1  
1 2 3 0 1 −1
A−1 = 1 1 2 =  2 −2 −1
0 1 2 −1 1 1

then
      
38 0 1 −1 38 13
A−1 28 =  2 −2 −1 28 =  5  ,
15 −1 1 1 15 5
      
105 0 1 −1 105 20
A−1  70  =  2 −2 −1  70  = 20 ,
50 −1 1 1 50 15
      
97 0 1 −1 97 13
−1  
A 64 = 2 −2 −1 64 = 15 and
   
51 −1 1 1 51 18
      
117 0 1 −1 117 18
−1 
A 79 = 2
  −2 −1  79 = 15
 
61 −1 1 1 61 23
Thus, our friend can find the original message code:

13 5 5 20 20 15 13 15 18 18 15 23
via the inverse of a matrix A.
Similarly, if we receive the following message code from our friend

77 54 38 71 49 29 68 51 33 76 48 40 86 53 52
and we know the message from our friend transformed by the same linear
transformation
 
1 2 3
L : R3 → R3 , L(x) = Ax = 1 1 2 x
0 1 2
First break the message into 5 vectors,

         
77 71 68 76 86
54 , 49 , 51 , 48 , 53
38 29 33 40 52

and the original message code can be obtained by

      
77 0 1 −1 77 16
−1  
A 54 = 2 −2 −1 54 = 8  ,
    
38 −1 1 1 38 15
      
71 0 1 −1 71 20
A−1 49 =  2 −2 −1 49 = 15 ,
29 −1 1 1 29 7
      
68 0 1 −1 68 18
A−1 51 =  2 −2 −1 51 =  1  ,
33 −1 1 1 33 16
      
76 0 1 −1 76 8
A−1 48 =  2 −2 −1 48 = 16 , and
40 −1 1 1 40 12
      
86 0 1 −1 86 1
A−1 53 =  2 −2 −1 53 = 14
52 −1 1 1 52 19
Hence, the original message from our friend is:
16 8 15 20 15 7 18 1 16 8 16 12 1 14 19
P H O T O G R A P H P L A N S
PHOTOGRAPH PLANS

We have covered:

Introduction

Application

Next...
Introduction to Matlab

L05 Eigenvalues and Linear Transformation

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

L05 Eigenvalues and Linear Transformation

Uploaded by

Copyright:

Available Formats

Lecture 5

Department of Intelligent Computing & Analytics

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 1 / 47

1 Eigenvalues and Eigenvectors

2 General Linear Transformation

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 2 / 47

1 Eigenvalues and Eigenvectors

2 General Linear Transformation

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 3 / 47

In this subtopic, we will study on classes of scalars and vectors known as

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 4 / 47

Eigenvalues and Eigenvectors

Definition of Eigenvalue and Eigenvector:

for some scalar λ. The scalar λ is called an eigenvalue of A, and x is said to

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 5 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 6 / 47

In Example 1, we observed that λ = 3 is an eigenvalue of the matrix

but, how to find it?

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 7 / 47

In Example 1, we observed that λ = 3 is an eigenvalue of the matrix

but, how to find it?

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 7 / 47

In Example 1, we observed that λ = 3 is an eigenvalue of the matrix

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 7 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 8 / 47

Step 2 (Take the determinant of λI − A):

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 9 / 47

Step 2 (Take the determinant of λI − A):

Step 3 (Let det(λI − A) = 0):

This shows that the eigenvalues of A are λ = 3 and λ = −1.

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 9 / 47

When the determinant det(λI − A) is expanded, the characteristic

where the left side of this equation is a polynomial of degree n in which

is called the characteristic polynomial of A.

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 10 / 47

In Revisited Example 1, we had derived the eigenvalues of the matrix

are λ = 3 and λ = −1.

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 11 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 13 / 47

Note that, if B is similar to A, then it is also true that A is similar to B.

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 15 / 47

A Procedure for Diagonalizing an n × n Matrix

Step 1. Find n linearly independent eigenvectors of A, says p1 , p2 , . . . , pn . If

Step 2. Form the matrix P having p1 , p2 , . . . , pn as its column vectors.

Step 3. The resulting matrix multiplication of P−1 AP will be a diagonal

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 16 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 18 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 18 / 47

* You may check using AP = PD formula.

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 19 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 19 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 19 / 47

1 Eigenvalues and Eigenvectors

2 General Linear Transformation

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 20 / 47

General Linear Transformation

In the special case where V = W, the linear transformation L is called a linear

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 21 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 22 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 22 / 47

Dr. Wanie MY BITI 1213 L05: Eigenvalues & Linear Trans 22 / 47

Some Linear Transformation on R2

Let’s look at some linear transformations on the plane R2 .