Course Name: Engineering Statistics and Linear Algebra(ESLA)

MODULE-5
Determinants and Eigen Values and Eigen Vector
Faculties Handling: -
1. Mrs. Anjani , 2. Mrs. Laxmi G
Asst Professor, Senior Asst Professor,
Dept. of ECE, Dept. of ECE,
MITE, Moodabidri. MITE, Moodabidri

1
 Course Name: Engineering Statistics and Linear Algebra(ESLA)

OUTLINE
• The Determinant of a Matrix
• Evaluation of a Determinant using Elementary Row Operations
• Properties of Determinants
• Application of Determinants: Cramer’s Rule
• Class Exercise
• Eigenvalues and Eigenvectors
• Diagonalization
• Symmetric Matrices and Orthogonal Diagonalization
• Application of Eigenvalues and Eigenvectors
• Principal Component Analysis

2
 Engineering Statistics and Linear Algebra(ESLA )
[As per Choice Based Credit System (CBCS) scheme]
SEMESTER – IV

Subject Code 18EC44 CIE Marks 40

Number of Lecture 03 SEE Marks 60

Hours/Week

Total Number of 40 Exam Hours 03

Lecture Hours (8 Hours per
Module)

Dept.of ECE,MITE,Moodabidri 3
 Course Name: Engineering Statistics and Linear Algebra(ESLA)

SYLLABUS Revised Bloom’s

Taxonomy(RBT) Level
Determinants: Properties of L1- Remembering
Determinants, Permutations and
Cofactors. L2- Understanding
(Refer Chapter 4, Text 2) L3- Applying
Eigenvalues and Eigen vectors:
Review of Eigenvalues and
Diagonalization of a Matrix,
Special Matrices (Positive
Definite, Symmetric) and their
properties, Singular Value
Decomposition.
(Refer Chapter 5, Text 2)

4
Department of Electronics and Communication, MITE, Moodabidri
 Course Name: Engineering Statistics and Linear Algebra(ESLA)

COURSE OUTCOME
Course Outcomes(CO)
Identify and associate random variables and distributive functions associated to
CO.1
Communication events.

Understand two variable expectations, transformation and Joint probabilities for multiple
CO.2
random variables and application exercises to Chi-square Random Variables, Student-T
Random Variables, and Cauchy and Rayleigh Random Variables.
Interpret Random process, Correlation functions and analyze with effect of noise.
CO.3

Understand the concept of vector spaces, linear independence, basis and dimension, and
CO.4
Orthogonality and apply these concepts to various vector spaces and subspaces.

Demonstrate by way of simulation or emulation the ease of analysis employing

CO.5
determinants, Eigen values, Diagonalization of matrix and Singular Value Decomposition.

Department of Electronics and Communication, MITE, Moodabidri 5

Determinants

3.6
※ The determinant is NOT a matrix operation
※ The determinant is a kind of information extracted from
a square matrix to reflect some characteristics of that
square matrix
※ For example, this chapter will discuss that matrices with
a zero determinant are with very different characteristics
from those with non-zero determinants
※ The motives to calculate determinants are to identify the
characteristics of matrices and thus facilitate the
comparison between matrices since it is impossible to
investigate or compare matrices entry by entry
※ The similar idea is to compare groups of numbers
through the calculation of averages and standard
deviations
※ Not only the determinant but also the eigenvalues and
eigenvectors are the information that can be used to
identify the characteristics of square matrices
3.7
 The Determinant of a Matrix
• The determinant of a 2 × 2 matrix:
 a11 a12 
A  
 21 22 
a a

 det( A)  | A |  a11a22  a21a12

 Note:
1. For every SQUARE matrix, there is a real number associated with this matrix
and called its determinant
2. It is common practice to omit the matrix brackets

 a11 a12  a 11 a 12
a  
 21 a22  a 21 a 22
3.8
• Historically speaking, the use of determinants arose from the recognition of
special patterns that occur in the solutions of linear systems:

a11 x1  a12 x2  b1

a21 x1  a22 x2  b2
b1a22  b2 a12 b2 a11  b1a21
 x1  and x2 
a11a22  a21a12 a11a22  a21a12

 Note:
1. x1 and x2 have the same denominator, and this quantity is called the determinant
of the coefficient matrix A
2. There is a unique solution if a11a22 – a21a12 = |A| ≠ 0

3.9
• Ex. 1: The determinant of a matrix of order 2
2 3
 2(2)  1(3)  4  3  7
1 2

2 1
 2(2)  4(1)  4  4  0
4 2

0 3/ 2
 0(4)  2(3 / 2)  0  3  3
2 4

 Note: The determinant of a matrix can be positive, zero, or negative

3.10
 Minor of the entry aij: the determinant of the matrix obtained by deleting the i-th
row and j-th column of A

a11 a12  a1( j 1) a1( j 1)  a1n

 
a( i 1)1  a( i 1)( j 1)
M ij 
a( i 1)1  a( i 1)( j 1)
 
a n1  an ( j 1)

a( i 1)( j 1)  a( i 1) n
※ Mij is a real number
a( i 1)( j 1)  a( i 1) n
 
an ( j 1)  ann

• Cofactor of aij:
Cij  ( 1)i  j M ij ※ Cij is also a real number

3.11
• Ex:
 a11 a12 a13 
A  a21 a22 a23 
 
a31 a32 a33 
a12 a13 a11 a13
 M 21  M 22 
a32 a33 a31 a33

 C21  ( 1) 21 M 21   M 21 C22  ( 1) 22 M 22  M 22

     
     
 Notes: Sign pattern for cofactors. Odd positions (where i+j is  
odd) have negative signs, and even positions (where i+j is even)      
have positive signs. (Positive and negative signs appear      
alternately at neighboring positions.)  
     
     
3.12  
• Theorem 3.1: Expansion by cofactors
Let A be a square matrix of order n, then the determinant of A
is given by
n
(a) det( A) | A |  aij Cij  ai1Ci1  ai 2Ci 2   ainCin
j 1

(cofactor expansion along the i-th row, i=1, 2,…, n)

or
n
(b) det( A) | A |  aij Cij  a1 j C1 j  a2 j C2 j   anj Cnj
i 1

(cofactor expansion along the j-th column, j=1, 2,…, n)

※The determinant can be derived by performing the cofactor expansion

3.13
along any row or column of the examined matrix
• Ex: The determinant of a square matrix of order 3

 a11 a12 a13 

A  a21 a22 a23 
 
a31 a32 a33 

 det( A)  a11C11  a12C12  a13C13 (first row expansion)

 a21C21  a22C22  a23C23 (second row expansion)
 a31C31  a32C32  a33C33 (third row expansion)
 a11C11  a21C21  a31C31 (first column expansion)
 a12C12  a22C22  a32C32 (second column expansion)
 a13C13  a23C23  a33C33 (third column expansion)

3.14
• Ex 3: The determinant of a square matrix of order 3
0 2 1 
A   3 1 2   det( A)  ?
 4 0 1 
Sol:
11  1 2 1 2 3 2
C11  (1)  1 C12  (1)  (1)(5)  5
0 1 4 1
1 3 3 1
C13  (1) 4
4 0
 det( A)  a11C11  a12C12  a13C13
 (0)(1)  (2)(5)  (1)(4)
 14
3.15
• Alternative way to calculate the determinant of a square matrix of order 3:

Subtract these three products

 a11 a12 a13  a11 a12 a13 a11 a12

A   a21 a22 a23  a21 a22 a23 a21 a22
 a31 a32 a33  a31 a32 a33 a31 a32
Add these three products

 det( A) | A | a11a22 a33  a12 a23a31  a13a21a32  a31a22 a13

 a32 a 23 a11  a33a21a12

3.16
• Ex: Recalculate the determinant of the square matrix A in Ex 3

–4 0 6
0 2 1  0 2
A   3 1 2  3 1
 4 0 1  4 0
0 16 0

 det( A) | A | 0  16  0  (4)  0  6  14

※ This method is only valid for matrices with the order of 3

3.17
• Ex 4: The determinant of a square matrix of order 4

 1 2 3 0
 1 1 0 2 
A  det( A)  ?
0 2 0 3 
 
3 4 0  2

3.18
Sol:
det( A)  (3)(C13 )  (0)(C23 )  (0)(C33 )  (0)(C43 )
 3C13
1 1 2
 3(1)13 0 2 3
3 4 2
 2 1 1 2 22  1 2 23  1 1 
 3(0)( 1)  ( 2)( 1)  (3)( 1) 
 4  2 3  2 3 4 
 30  ( 2)(1)( 4)  (3)( 1)( 7)
 (3)(13)
 39
※ By comparing Ex 4 with Ex 3, it is apparent that the computational effort for
the determinant of 4×4 matrices is much higher than that of 3×3 matrices. In
3.19
the next section, we will learn a more efficient way to calculate the determinant
• Upper triangular matrix :
All entries below the main diagonal are zeros
 Lower triangular matrix :
All entries above the main diagonal are zeros
 Diagonal matrix :
All entries above and below the main diagonal are zeros

Ex:

a11 a12 a13   a11 0 0  a11 0 0 

 0 a22 a23  a21 a22 0   0 a22 0 
0 0 a  a a a  0 0 a 
 33   31 32 33   33 

upper triangular lower triangular diagonal

3.20
• Theorem 3.2: (Determinant of a Triangular Matrix)

If A is an n  n triangular matrix (upper triangular, lower triangular, or diagonal),

then its determinant is the product of the entries on the main diagonal. That is

det( A) | A | a11a22a33 ann

※ Based on Theorem 3.2, it is straightforward to obtain that det( I )  1

※ On the next slide, I only take the case of upper triangular matrices for
example to prove Theorem 3.2. It is straightforward to apply the
following proof for the cases of lower triangular and diagonal
matrices

3.21
 Pf: by Mathematical Induction Suppose that the theorem is true for any upper
triangular matrix U of order 𝑛– 1, i.e.,
| U | a11a22a33 a( n1)( n1)
Then consider the determinant of an upper triangular matrix A of order n by the
cofactor expansion across the n-th row

| A | 0Cn1  0Cn 2   0Cn ( n 1)  annCnn  ann (1)2 n M nn  ann M nn

Since Mnn is the determinant of a (n–1)×(n–1) upper triangular matrix by deleting

the n-th row and n-th column of A, we can apply the induction assumption to write

| A | ann M nn  ann (a11a22 a( n 1)( n 1) )  a11a22 a( n 1)( n 1)ann

3.22
  Ex 6: Find the determinants of the following triangular matrices

 1 0 0 0 0
2 0 0 0  0 3 0 0 0
 4 2 0 0 
(a) A   (b) B 0 0 2 0 0
 5 6 1 0  
1 3 3  0 0 0 4 0
 5  0 0 0 0  2
Sol:

(a) |A| = (2)(–2)(1)(3) = –12

(b) |B| = (–1)(3)(2)(4)(–2) = 48

3.23
 Keywords :
 determinant
 minor
 cofactor
 expansion by cofactors
 upper triangular matrix
 lower triangular matrix
 diagonal matrix

3.24
3.2 Evaluation of a Determinant Using Elementary Row Operations
 The computational effort to calculate the determinant of a square matrix with a large
number of n is unacceptable. In this section, I will show how to reduce the
computational effort by using elementary operations

• Theorem 3.3: Elementary row operations and determinants

Let A and B be square matrices

(a) B  Ii , j ( A)  det( B)   det( A) (by mathematical induction

(b) B  Mi( k ) ( A)  det( B)  k det( A) (straightforward)

(c) B  Ai(,kj) ( A)  det( B)  det( A) (by combining Thm. 3.4 and 3.5)

 Notes: The above three properties remains valid if elementary column operations are
performed to derive column-equivalent matrices (These results will be used in Ex 5
on Slide 3.25)
3.25

Ex: 1 2 3 
A  0 1 4   det( A)  2
1 2 1 

 4 8 12  A1  M 1(4) ( A)
A1   0 1 4   det( A1 )  8
 det( A1)  4 det( A)  (4)( 2)  8
1 2 1 

0 1 4 
A2  I 1, 2( A)
A2  1 2 3   det( A2 )  2
 det( A2)   det( A)  ( 2)  2
1 2 1 

1 2 3 ( 2)
A3  A1,2 ( A)
A3   2 3 2   det( A3 )  2
 det( A3)  det( A)  2
 1 2 1  3.26
Row reduction method to evaluate the determinant
1. A row-echelon form of a square matrix is either an upper triangular matrix or a matrix
with zero rows
2. It is easy to calculate the determinant of an upper triangular matrix (by Theorem 3.2) or a
matrix with zero rows (det = 0)


Ex: Evaluation a determinant using elementary row operations

2  3 10 
A  1 2  2  det( A)  ?
0 1  3
Sol: • Notes:
2 3 10 1 2 2
det( A1 )   det( A)
det( A)  1 2A 2  2 3A110
I1,2
 det( A)   det( A1 )
0 1 3 0 1 3 3.27
 1 2 2 1 2 2
( 1 )
( 2 ) 1

  0 7 14 
 (  1) (
A ) 0 1 2 A
A1,2 M2 7

2 (1/ 7) 3
0 1 3 0 1 3
1 2 2
( 1)
 A4
 7 0 1 2  7(1)  7
A2,3

0 0 1

• Notes:
det( A2 )  det( A1 )
1 1
det( A3 )   det( A2 )  det( A2 )  det( A3 )
7 (1/ 7)
det( A4 )  det( A3 )
3.28
 Comparison between the number of required operations for the two kinds of methods
to calculate the determinant

Cofactor Expansion Row Reduction

Order n Additions Multiplications Additions Multiplications

3 5 9 5 10

5 119 205 30 45

10 3,628,799 6,235,300 285 339

※ When evaluating a determinant by hand, you can sometimes save steps by

integrating this two kinds of methods (see Examples 5 and 6 on the next
three slides)
3.29
• Ex 5: Evaluating a determinant using column reduction and cofactor
expansion

 3 5 2
A   2  4  1
 
  3 0 6 
Sol:

3 5 2 (2)
AC1,3
3 5 4
det( A)  2 4 1  2 4 3
3 0 6 3 0 0
3 1 5 4
 (3)(1)  (3)(1)(1)  3
4 3
※ ACi(,is
k ) the counterpart column operation to the row operation
Ai(,kj )
j
3.30
• Ex 6: Evaluating a determinant using both row and column reductions and
cofactor expansion

 2 0 1 3  2
 2 1 3 2  1
A   1 0 1 2 3
 3 1 2 4  3
 1 1 3 2 0
Sol:

2 0 1 3 2 2 0 1 3 2
2 1 3 2 1 A(1) ( 1)
A2,5
2 1 3 2 1
2,4

det( A)  1 0 1 2 3  1 0 1 2 3
3 1 2 4 3 1 0 5 6 4
1 1 3 2 0 3 0 0 0 1
2 1 3 2
2 2 1 1 2 3
 (1)(  1)
1 5 6 4
3.31 3 0 0 1
 8 1 3 2
( 3) 8 1 3 0 0 5
8 1 2
(1)
AC4,1 A2,1
3
  (1)(  1) 4  4 8 1 2 = 8 1 2
13 5 6 4
13 5 6 13 5 6
0 0 0 1
1 3 8 1
 5(  1)
13 5
 (5)(27)
 135

3.32
• Theorem 3.4: Conditions that yield a zero determinant
If A is a square matrix and any of the following conditions is true, then det(A) = 0

(a) An entire row (or an entire column) consists of zeros

(Perform the cofactor expansion along the zero row or column)

(b) Two rows (or two columns) are equal

(c) One row (or column) is a multiple of another row (or column)
(For (b) and (c), based on the mathematical induction, perform the cofactor
expansion along any row or column other than these two rows or columns)

 Notes: For conditions (b) or (c), you can also use elementary row or column operations
to create an entire row or column of zeros and obtain the results by Theorem 3.3
※ Thus, we can conclude that a square matrix has a determinant of zero if and
only if it is row- (or column-) equivalent to a matrix that has at least one row
(or column) consisting entirely of zeros
3.33
 Ex:

1 2 3 1 4 0 1 1 1
0 0 0 0 2 5 0 0 2 2 2 0
4 5 6 3 6 0 4 5 6

1 4 2 1 2 3 1 8 4
1 5 2 0 4 5 6 0 2 10 5  0
1 6 2 2 4 6 3 12 6

3.34
 3.3 Properties of Determinants
 Theorem 3.5: Determinant of a matrix product

det(AB) = det(A) det(B)

(Verified by Ex 1 on the next slide)
• Notes:
(1) det( A1 A2 An )  det( A1 ) det( A2 ) det( An ) (by using Thm. 3.5 iteratively)
det(Ei , j A)  det(Ei , j )det( A) and det(Ei , j A)   det( A)  det(Ei , j )  1

(2) det(Ei( k ) A)  det(Ei( k ) )det( A) and det(Ei( k ) A)  k det( A)  det(Ei( k ) )  k
 det(E ( k ) A)  det(E ( k ) )det( A) and det(E ( k ) A)  det( A)  det(E ( k ) )  1
 i, j i, j i, j i, j

(3) det( A  B)  det( A)  det( B)

a11 a12 a13 a11 a12 a13 a11 a12 a13
(4) a21  b21 a22  b22 a23  b23  a21 a22 a23  b21 b22 b23
a31 a32 a33 a31 a32 a33 a31 a32 a33
(There is an example to verify this property on Slide 3.33) (Note that this property
3.35
is also valid for any rows or columns other than the second row)
 Ex 1: The determinant of a matrix product

1  2 2  2 0 1 
A  0 3 2 B  0  1  2
1 0 1  3 1  2

Find |A|, |B|, and |AB|

Sol:

1 2 2 2 0 1
| A | 0 3 2  7 | B | 0  1  2  11
1 0 1 3 1 2

3.36
 1  2 2 2 0 1  8 4 1 
AB  0 3 2 0  1  2  6  1  10
1 0 1 3 1  2 5 1  1 

8 4 1
 | AB | 6 1 10  77
5 1 1

 Check:
|AB| = |A| |B|

3.37
 Ex:

1 2 2 1 2 2 1 2 2
?
A0 3 2  1 1 2 1 2 0 BC
1 0 1 1 0 1 1 0 1

Pf:

2 1 2 2 2 2 1 2 23 1 2
| A | 0( 1)  3(1)  2(1)
0 1 1 1 1 0
2 1 2 2 2 2 1 2 23 1 2
| B | 1( 1)  1(1)  2(1)
0 1 1 1 1 0
2 1 2 2 2 2 1 2 23 1 2
| C | 1( 1)  2( 1)  0( 1)
0 1 1 1 1 0
3.38
 Theorem 3.6: Determinant of a scalar multiple of a matrix

If A is an n × n matrix and c is a scalar, then

det(cA) = cn det(A)
(can be proven by repeatedly use the fact that if B  M i( k ) ( A)  B  k A )

• Ex 2:
 10 20 40  1 2 4
A   30 0 50  , if 3 0 5  5, find |A|
 20 30 10  2 3 1

Sol:
 1  2 4 1 2 4
A  10 3 0 5  A  10 3 0 5  (1000)(5)  5000
3

 
  2  3 1 2 3 1 3.39
 Theorem 3.7: (Determinant of an invertible matrix)

A square matrix A is invertible (nonsingular) if and only if det(A)  0

)
(Pf:
If A is invertible, then AA–1 = I. By Theorem 3.5, we can have |A||A–1| = |I|. Since
( ) |I| = 1, neither |A| nor |A–1| is zero
Suppose |A| is nonzero. It is aimed to prove A is invertible.
By the Gauss-Jordan elimination, we can always find a matrix B, in reduced
row-echelon form, that is row-equivalent to A
1. Either B has at least one row with entire zeros, then |B| = 0 and thus |A| = 0
since |Ek|…|E2||E1||A| = |B|. →←
2. Or B = I, then A is row-equivalent to I, and by Theorem 2.15 (Slide 2.59), it can
be concluded that A is invertible

3.40
• Ex 3: Classifying square matrices as singular or nonsingular

0 2  1 0 2  1
A  3  2 1  B  3  2 1 
3 2  1 3 2 1 

Sol:

A 0  A has no inverse (it is singular)

B  12  0  B has inverse (it is nonsingular)

3.41
 Theorem 3.8: Determinant of an inverse matrix
1
If A is invertible, then det( A1 ) 
det( A)
(Since AA1  I , then A A1  1)
 Theorem 3.9: Determinant of a transpose

If A is a square matrix, then det( AT )  det(A)

(Based on the mathematical induction, compare the cofactor expansion along a row of A
and the cofactor expansion along the corresponding column of AT)

• Ex 4: 1 0 3  (a) A1  ? (b) AT  ?

A  0  1 2
2 1 0
Sol:
1 0 3 1 1
 A1  
 | A | 0  1 2  4 A 4
2 1 0 AT  A  4 3.42
• The similarity between the noninvertible matrix and the real number 0

Matrix A Real number c

Invertible det( A)  0 c0

1 1
A1 exists and det( A1 )  c 1 exists and c 1 =
det( A) c

Noninvertible det( A)  0 c0

A1 does not exist c 1 does not exist
 1 1 1  1 1 1 
 det( A )    c = = 
 det( A) 0  c 0

3.43
 Equivalent conditions for a nonsingular matrix:

If A is an n × n matrix, then the following statements are equivalent

(1) A is invertible
(2) Ax = b has a unique solution for every n × 1 matrix b (Thm. 2.11)
(3) Ax = 0 has only the trivial solution (Thm. 2.11)
(4) A is row-equivalent to In (Thm. 2.14)

(5) A can be written as the product of elementary matrices (Thm. 2.14)

(6) det(A)  0 (Thm. 3.7)
※ The statements (1)-(5) are collected in Theorem 2.15, and the
statement (6) is from Theorem 3.7
3.44
• Ex 5: Which of the following system has a unique solution?

(a) 2 x2  x3  1
3 x1  2 x2  x3  4
3 x1  2 x2  x3  4
(b)
2 x2  x3  1
3 x1  2 x2  x3  4
3 x1  2 x2  x3  4

3.45
Sol:
(a) Ax  b (the coefficient matrix is the matrix A in Ex 3)
A  0 (from Ex 3)
 This system does not have a unique solution

(b) Bx  b (the coefficient matrix is the matrix B in Ex 3)

B  12  0 (from Ex 3)
 This system has a unique solution

3.46
 3.4 Applications of Determinants
 Matrix of cofactors of A:

C11 C12  C1n 

C C  C 
Cij    21 22 2n 
Cij  (1)i  j Mij
    
 
Cn1 Cn 2  Cnn 
(The definitions of cofactor Cij and minor Mij of aij can be reviewed on Slide 3.6)

 Adjoint matrix of A:

 C11 C21 Cn1 

C C22 Cn 2 
adj( A)  Cij    12
T

 
 
C1n C2 n Cnn 
3.47
• Theorem 3.10: The inverse of a matrix expressed by its adjoint matrix

1 1
A  adj( A)
det( A)
Pf: If A is an n × n invertible matrix, then

Consider the product A[adj(A)]

 a11 a12 a1n 

   C11 C j1 Cn1  det( A) 0 0 
  C12 C j2 Cn 2   0 det( A) 0 
  ai1 ai 2 ain   
    
  C   
 1n C jn Cnn   0 det( A) 
ann  
0
 an1 an 2

The entry at the position (i, j) of A[adj(A)]

det( A) if i  j
ai1C j1  ai 2C j 2   ainC jn  
 0 if i  j
3.48
Consider a matrix B similar to matrix A except that the j-th row is
replaced by the i-th row of matrix A
a11 a12 a1n

ai1 ai 2 ain ※ Since there are two

 det( B)   0 identical rows in B,
ai1 ai 2 ain according to Theorem 3.4,
det(B) should be zero

an1 an 2 ann

Perform the cofactor expansion along the j-th row of matrix B

 det( B)  ai1C j1  ai 2C j 2   ainC jn  0
(Note that Cj1, Cj2,…, and Cjn are still the cofactors for the entries of the j-th row)
A-1

 1 
 A[adj( A)]  det( A) I  A  adj( A)   I
 det( A)  3.49
 Ex: For any 2×2 matrix, its inverse can be calculated as follows

a b 
A   det( A)  ad  bc,
c d 

 C11 C21   d b 
adj( A)     
 12
C C 22    c a 

1 1  d  b
A  1
adj( A) 
det  A ad  bc   c a 

3.50
• Ex 2:
 1 3 2 (a) Find the adjoint matrix of A
A   0  2 1 
(b) Use the adjoint matrix of A to find A–1
 1 0  2
Sol:

 Cij  (1)i  j M ij
2 1 0 1 0 2
 C11    4, C12    1, C13    2,
0 2 1 2 1 0

3 2 1 2 1 3
C21    6, C22    0, C23    3,
0 2 1 2 1 0

3 2 1 2 1 3
C31    7, C   1, C33    2.
2 1
32
0 1 0 2
3.51
 cofactor matrix of A  adjoint matrix of A

 4 1 2 4 6 7
 
Cij  6 0 3 adj( A)  Cij   1 0 1 
T

7 1 2  2 3 2 

 inverse matrix of A ※ The computational effort of this

method to derive the inverse of a
1
adj( A) (det  A
1 matrix is higher than that of the
A   3) G.-J. E. (especially to compute the
det  A cofactor matrix for a higher-order
square matrix)
4 6 7  43 2 7
3 
 13 1 0 1   13 0 1
※ However, for computers, it is
   3
 easier to implement this method
than the G.-J. E. since it is not
2 3 2  23 1 3
2
 necessary to judge which row
operation should be used and the
 Check: AA1  I only thing needed to do is to
3.52
calculate determinants of matrices
• Theorem 3.11: Cramer’s Rule
a11 x1  a12 x2   a1n xn  b1
a21 x1  a22 x2   a2 n xn  b2 A(i) represents the i-th
 Ax  b column vector in A

an1 x1  an 2 x2   ann xn  bn
 x1   b1 
x  b 
where A   aij    A(1) A(2) ( n)
A ,  x   2
, b   2
nn    
   
 n
x bn 
Suppose this system has a unique solution, i.e.,
a11 a12 a1n
a21 a22 a2 n
det( A)  0

an1 an 2 ann 3.53

By defining Aj   A(1) A(2) A( j 1) b A( j 1) A( n) 

 a11 a1( j 1) b1 a1( j 1) a1n 

a a2( j 1) b2 a2( j 1) a2 n 

21

 
 
 an1 an ( j 1) bn an ( j 1) ann 

(i.e., det( Aj )  b1C1 j  b2C2 j   bnCnj )

det( Aj )
 xj  , j  1, 2, ,n
det( A)

3.54
• Pf:
Ax = b (det( A)  0)
1 1
x A b  adj( A)b (according to Thm. 3.10)
det( A)
 C11 C21 Cn1   b1 
C Cn 2  b2 
1  12 C22

det( A)   
  
C1n C2 n Cnn  bn 

 b1C11  b2C21   bnCn1 

  bnCn 2 
1 b1C12  b2C22 

det( A)  
 
b1C1n  b2C2 n   bnCnn  3.55
  x1   b1C11  b2C21   bnCn1 
x  b C  b C   b C 
1
  2   1 12 2 22 n n2 

  det( A)  
   
 n
x b C 
 1 1n 2 2 n
b C   bn nn 
C
 det(A1 ) / det( A) 
 det(A ) / det( A)  (On Slide 3.49, it is already derived that
 2 
  det( Aj )  b1C1 j  b2C2 j   bnCnj )
 
det(An ) / det( A) 

det( Aj )
 xj  , j  1, 2, ,n
det( A)
3.56
• Ex 4: Use Cramer’s rule to solve the system of linear equation
 x  2 y  3z  1
2x  z  0
3x  4 y  4 z  2
Sol:

1 2 3 1 2 3
det( A)  2 0 1  10 det( A1 )  0 0 1 8
3 4 4 2 4 4
1 1  3 1 2 1
det( A2 )  2 0 1  15, det( A3 )  2 0 0  16
3 2 4 3 4 2
det( A1 ) 4 det( A2 )  3 det( A3 )  8
x  y  z 
det( A) 5 det( A) 2 det( A) 5 3.57
 Keywords
• matrix of cofactors
• adjoint matrix
• Cramer’s rule: Cramer

3.58
Class Exercise
 Determinants
If A = aisij a square matrix of order 1,

then |A| = | a11 | = a11

 a11 a12 
If A = a 
is a square matrix of order 2, then
 21 a 22 

a11 a12
|A| = = a11a22 – a21a12
a21 a22
 Example
4 -3
Evaluate the determinant :
2 5

4 -3
Solution : = 4 × 5 - 2 × -3  = 20 + 6 = 26
2 5
 Solution
a11 a12 a13 
a a22is aa23 
If A =  21 square
 matrix of order 3, then
a31 a32 a33 

a11 a12 a13

a a23 a21 a23 a21 a22
| A |= a21 a22 a23 = a11 22 - a12 + a13
a32 a33 a31 a33 a31 a32
a31 a32 a33
[Expanding along first row]

= a11  a22 a33 - a32 a23  - a12 a21a33 - a31a23  + a13 a21a32 - a31a22 

  a11a22 a33  a12 a31a23  a13 a21a32   a11a23 a32  a12 a21a33  a13 a31a22 
 Example
2 3 -5
Evaluate the determinant : 7 1 -2
-3 4 1
Solution :

2 3 -5
1 -2 7 -2 7 1
7 1 -2 =2 -3 + -5
4 1 -3 1 -3 4
-3 4 1

[Expanding along first row]

= 2 1 + 8  - 3 7 - 6  - 5 28 + 3 
= 18 - 3 - 155
= -140
 Minors
-1 4
If A =  , then
2 3

M11 = Minor of a11 = 3, M12 = Minor of a12 = 2

M21 = Minor of a21 = 4, M22 = Minor of a22 = -1

 Minors
 4 7 8
If A = -9 0 0  , then
 2 3 4

M11 = Minor of a11 = determinant of the order 2 × 2 square

sub-matrix is obtained by leaving first
row and first column of A
0 0
= =0
3 4

4 7
Similarly, M23 = Minor of a23 = =12 -14 = -2
2 3

4 8
M32 = Minor of a32 = = 0+72
etc. = 72
-9 0
 Cofactors
Cij = Cofactor of aij in A = -1
i+ j
Mij ,

where Mij is minor of aij in A

 Cofactors (Con.)
 4 7 8
A = -9 0 0 
 2 3 4

0 0
C11 = Cofactor of a11 = (–1)1 + 1 M11 = (–1)1 +1 =0
3 4

4 7
C23 = Cofactor of a23 = (–1)2 + 3 M23 =  2
2 3

4 8
C32 = Cofactor of a32 = (–1)3 + 2M32 = -
etc. = - 72
-9 0
Value of Determinant in Terms of Minors
and Cofactors
a11 a12 a13 
If A = a21 a22 a23  , then
a31 a32 a33 

3 3
A    1 i j
aijMij   aijCij
j1 j1

= ai1Ci1 + ai2Ci2 + ai3Ci3 , for i =1 or i = 2 or i = 3

 Properties of Determinants
1. The value of a determinant remains unchanged, if its
rows and columns are interchanged.

a1 b1 c1 a1 a2 a3
a2 b2 c2 = b1 b2 b3 i.e. A  A '
a3 b3 c3 c1 c2 c3

2. If any two rows (or columns) of a determinant are interchanged,

then the value of the determinant is changed by minus sign.

a1 b1 c1 a2 b2 c2
a2 b2 c2 = - a1 b1 c1  Applying R2  R1 
a3 b3 c3 a3 b3 c3
 Properties (Con.)
3. If all the elements of a row (or column) is multiplied by a
non-zero number k, then the value of the new determinant
is k times the value of the original determinant.

ka1 kb1 kc1 a1 b1 c1

a2 b2 c2 = k a2 b2 c2
a3 b3 c3 a3 b3 c3

which also implies

a1 b1 c1 ma1 mb1 mc1

1
a2 b2 c2 = a2 b2 c2
m
a3 b3 c3 a3 b3 c3
 Properties (Con.)
4. If each element of any row (or column) consists of
two or more terms, then the determinant can be
expressed as the sum of two or more determinants.

a1 + x b1 c1 a1 b1 c1 x b1 c1
a2 + y b2 c2 = a2 b2 c2 + y b2 c2
a3 + z b3 c3 a3 b3 c3 z b3 c3

5. The value of a determinant is unchanged, if any row

(or column) is multiplied by a number and then added
to any other row (or column).

a1 b1 c1 a1 + mb1 - nc1 b1 c1
a2 b2 c2 = a2 + mb2 - nc2 b2 c2  Applying C1  C1 + mC2 - nC3 
a3 b3 c3 a3 + mb3 - nc3 b3 c3
 Properties (Con.)
6. If any two rows (or columns) of a determinant are
identical, then its value is zero.

a1 b1 c1
a2 b2 c2 = 0
a1 b1 c1

7. If each element of a row (or column) of a determinant is zero,

then its value is zero.

0 0 0
a2 b2 c2 = 0
a3 b3 c3
Properties (Con.)
a 0 0
 
8 Let A = 0 b 0 be a diagonal matrix, then
0 0 c 

a 0 0
A =0 b 0  abc
0 0 c
 Row(Column) Operations
Following are the notations to evaluate a determinant:

(i) Ri to denote ith row

(ii) Ri  Rj to denote the interchange of ith and jth
rows.
(iii) Ri  Ri + lRj to denote the addition of l times the
elements of jth row to the corresponding elements
of ith row.
(iv) lRi to denote the multiplication of all elements of
ith row by l.

Similar notations can be used to denote column

operations by replacing R with C.
 Evaluation of Determinants
If a determinant becomes zero on putting
x =  , then  x -   is the factor of the determinant.

x 5 2
For example, if Δ = x2 9 4 , then at x = 2
x3 16 8

  ,0because C1 and C2 are identical at x = 2

Hence, (x – 2) is a factor of determinant . 
Sign System for Expansion of Determinant

Sign System for order 2 and order 3 are given by

+ – +
+ –
, – + –
– +
+ – +
 Example-1
Find the value of the following determinants
42 1 6 6 -3 2
(i) 28 7 4 (ii) 2 -1 2
14 3 2 -10 5 2

Solution :

42 1 6 6×7 1 6
i  28 7 4 = 4×7 7 4
14 3 2 2×7 3 2

6 1 6
=7 4 7 4  Taking out 7 common from C1 
2 3 2

= 7×0  C1 and C3 are identical

=0
Example –1 (ii)
6 -3 2
(ii) 2 -1 2
-10 5 2

3    2   3 2
  1   2  1 2
5   2  5 2

3 3 2
 ( 2) 1 1 2 Taking out  2 common from C1 
5 5 2
 ( 2)  0  C1 and C2 are identical
0
 Example - 2
1 a b+c
Evaluate the determinant 1 b c+a
1 c a+b
Solution :

1 a b+c 1 a a+b+c
1 b c+a = 1 b a+b+c  Applying c3  c2 +c3 
1 c a+b 1 c a+b+c

1 a 1
=  a+b+c  1 b 1  Taking  a+b+c  common from C3 
1 c 1

= a + b + c × 0  C1 and C3 are identical

=0
 Example - 3
a b c
Evaluate the determinant:
a2 b2 c2
bc ca ab
Solution:

a b c
We have a2 b2 c2
bc ca ab

(a-b) b-c c
= (a-b)(a+b) (b - c)(b+c) c2 Applying C1  C1 - C2 and C2  C2 - C3 
-c(a-b) -a(b - c) ab

1 1 c
 Taking  a-b  and b - c  common 
= (a-b)(b - c) a+b b+ c c2  
 from C and C respectively 
-c -a ab 1 2
 Solution Cont.
0 1 c
= (a-b)(b - c) -(c - a) b+ c c2  Applying c1  c1 - c2 
-(c - a) -a ab

0 1 c
= -(a-b)(b - c)(c - a) 1 b+c c2
1 -a ab

0 1 c
= -(a-b)(b - c)(c - a) 0 a+b+c c2 - ab Applying R 2  R 2 -R 3 
1 -a ab

Now expanding along C1 , we get

(a-b) (b-c) (c-a) [- (c2 – ab – ac – bc – c2)]
= (a-b) (b-c) (c-a) (ab + bc + ac)
 Example-4
Without expanding the determinant,
3x+y 2x x
3
prove that 4x+3y 3x 3x = x
5x+6y 4x 6x
Solution :
3x + y 2x x 3x 2x x y 2x x
L.H.S = 4x +3y 3x 3x = 4x 3x 3x + 3y 3x 3x
5x +6y 4x 6x 5x 4x 6x 6y 4x 6x

3 2 1 1 2 1
= x3 4 3 3 + x2 y 3 3 3
5 4 6 6 4 6

3 2 1
= x3 4 3 3 + x2 y×0  C1 and C2 are identical in II determinant 
5 4 6
 Solution Cont.
3 2 1
= x3 4 3 3
5 4 6

1 2 1
= x3 1 3 3  Applying C1  C1 - C2 
1 4 6

1 2 1
= x3 0 1 2  Applying R 2 R 2 -R1 and R 3 R 3 -R 2 
0 1 3

= x3 ×(3-2) Expanding along C1 

=x3 = R.H.S.
 Example -5
1 ω3 ω5
Prove that : ω3 1 ω 4 = 0 , where w is cube root of unity.
ω5 ω5 1
Solution :

1 ω3 ω5 1 ω3 ω3 .ω2
L.H.S = ω3 1 ω4 = ω3 1 ω3 .ω
ω5 ω5 1 ω3 .ω2 ω3.ω2 1

1 1 ω2
= 1 1 ω  ω3 =1
ω2 ω2 1

= 0 = R.H.S.  C1 and C2 are identical

 Example-6
x+a b c
Prove that : a x+b c = x2 (x+a+b+c)
a b x+C
Solution :

x+a b c x+a+b+c b c
L.H.S= a x+b c = x+a+b+c x+b c
a b x+C x+a+b+c b x+c
 Applying C1  C1 +C2 +C3 

1 b c
=  x+a+b+c  1 x+b c
1 b x+c
 Taking  x+a+b+c  common from C1 
 Solution cont.
1 b c
=(x+a+b+c) 0 x 0
0 0 x
 Applying R 2  R 2 -R1 and R 3  R 3 -R1 

Expanding along C1 , we get

(x + a + b + c) [1(x2)] = x2 (x + a + b + c)
= R.H.S
 Example -7
Using properties of determinants, prove that
b+ c c + a a+b
c + a a+b b+ c =2(a+b+ c)(ab+bc + ca- a2 -b2 - c2 ).
a+b b+ c c + a

Solution :
b+c c+a a+b
L.H.S = c+a a+b b+c
a+b b+c c+a

2(a+b+c) 2(a+b+c) 2(a+b+c)

= c+a a+b b+c  Applying R1  R1 +R 2 +R 3 
a+b b+c c+a

1 1 1
=2(a+b+c) c+a a+b b+c
a+b b+c c+a
 Solution Cont.
0 0 1
= 2(a+b+c) (c -b) (a- c) b+c  Applying C1  C1 - C2 and C2  C2 - C3 
(a- c) (b - a) c +a

Now expanding along R1 , we get

2(a+b+c) (c -b)(b- a)-(a- c)2 

=2(a+b+c) bc -b2 - ac+ab -(a2 +c2 - 2ac)

=2(a+b+c) ab+bc+ac- a2 -b2 - c2 

=R.H.S
 Example - 8
Using properties of determinants prove that

x+4 2x 2x
2x x+4 2x =(5x+4)(4- x)2
2x 2x x+4

Solution :

x+4 2x 2x 5x + 4 2x 2x
L.H.S = 2x x+4 2x = 5x + 4 x + 4 2x Applying C1  C1 +C2 +C3 
2x 2x x+4 5x + 4 2x x+4

1 2x 2x
= (5x + 4) 1 x + 4 2x
1 2x x+4
 Solution Cont.
1 2x 2x
=(5x + 4) 0 -(x - 4) 0  Applying R 2  R 2 -R1 and R 3  R 3 -R 2 
0 x-4 -(x - 4)

Now expanding along C1 , we get

(5x+4) 1(x - 4)2 - 0

=(5x+4)(4- x)2

=R.H.S
 Example -9
Using properties of determinants, prove that

x+9 x x
x x+9 x =243 (x+3)
x x x+9

Solution :

x+9 x x
L.H.S= x x+9 x
x x x+9

3x +9 x x
= 3x +9 x +9 x  Applying C1  C1 +C2 +C3 
3x +9 x x +9
 Solution Cont.
1 x x
=(3x+9) 1 x+9 x
1 x x+9

1 x x
= 3  x +3 0 9 0  Applying R 2 R 2 -R1 and R 3  R 3 -R 2 
0 -9 9

= 3(x +3) 81 Expanding along C1 

=243(x +3)
= R.H.S.
 Example -10
(b + c)2 a2 bc
Show that (c + a)2 b2 ca = (a2 +b2 + c2 )(a-b)(b - c)(c - a)(a+b + c)
(a+b)2 c2 ab

Solution :

(b+c)2 a2 bc b2 +c2 a2 bc
L.H.S.= (c+a)2 b2 ca = c2 +a2 b2 ca  Applying C1  C1 - 2C3 
(a+b)2 c2 ab a2 +b2 c2 ab

a2 +b2 +c2 a2 bc
 a2 +b2 +c2 b2 ca  Applying C1  C1 +C2 
a2 +b2 +c2 c2 ab

1 a2 bc
=(a2 +b2 +c2 ) 1 b2 ca
1 c2 ab
 Solution Cont.
1 a2 bc
=(a2 +b2 +c2) 0 (b- a)(b+a) c(a-b)  Applying R 2 R 2 -R1 and R 3  R 3 -R 2 
0 (c-b)(c+b) a(b- c)

1 a2 bc
=(a2 +b2 +c2 )(a-b)(b- c) 0 -(b+a) c
0 -(b+c) a

=(a2 +b2 +c2 )(a-b)(b - c)(-ab - a2 +bc+c2 ) Expanding along C1 

=(a2 +b2 +c2 )(a-b)(b- c) b c - a + c - ac+a

=(a2 +b2 +c2 )(a-b)(b - c)(c - a)(a+b+c)=R.H.S.

 Applications of Determinants
(Area of a Triangle)
The area of a triangle whose vertices are

(x1, y1 ), (x2 , y2 ) and (x3, y3 ) is given by the expression

x1 y1 1
1
Δ= x2 y2 1
2
x3 y3 1

1
= [x1 (y2 - y3 ) + x2 (y3 - y1 ) + x3 (y1 - y2 )]
2
 Example
Find the area of a triangle whose
vertices are (-1, 8), (-2, -3) and (3, 2).

Solution :
x1 y1 1 -1 8 1
1 1
Area of triangle = x2 y2 1 = -2 -3 1
2 2
x3 y3 1 3 2 1

1
= -1(-3 - 2)- 8(-2 - 3)+1(-4 + 9)
2

1
= 5+ 40 +5 = 25 sq.units
2
 Condition of Collinearity of Three Points
If A (x1 , y1 ), B (x2 , y2 ) and C (x3 , y3 ) are three points,

then A, B, C are collinear

 Area of triangle ABC = 0

x1 y1 1 x1 y1 1
1
 x2 y2 1 = 0  x2 y2 1 = 0
2
x3 y3 1 x3 y3 1
 Example
If the points (x, -2) , (5, 2), (8, 8) are collinear,
find x , using determinants.

Solution :

Since the given points are collinear.

x -2 1
5 2 1 =0
8 8 1

 x 2- 8 - -25- 8  +1  40-16  = 0

 -6x-6+24=0

 6x =18  x =3
 Solution of System of 2 Linear Equations
(Cramer’s Rule)
Let the system of linear equations be

a1x +b1y = c1 ... i

a2 x +b2 y = c2 ... ii

D1 D2
Then x = , y= provided D  0,
D D

a1 b1 c1 b1 a1 c1
where D = , D1 = and D2 =
a2 b2 c2 b2 a2 c2
 Cramer’s Rule (Con.)
Note :
1 If D  0,

then the system is consistent and has unique solution.

2  If D = 0 and D1 = D2 = 0,

then the system is consistent and has infinitely many

solutions.

3 If D = 0 and one of D1, D2  0,

then the system is inconsistent and has no solution.

 Example
Using Cramer's rule , solve the following system of
equations 2x-3y=7, 3x+y=5

Solution :
2 -3
D= = 2+9 =11  0
3 1

7 -3
D1 = =7+15=22
5 1

2 7
D2 = =10-21=-11
3 5

D0
D1 22 D -11
 By Cramer's Rule x = = = 2 and y = 2 = = -1
D 11 D 11
Solution of System of 3 Linear Equations
(Cramer’s Rule)
Let the system of linear equations be
a1x +b1y + c1z = d1 ... i

a2 x +b2 y + c2 z = d2 ... ii

a3x +b3y + c3z = d3 ... iii

D1 D2 D3
Then x = , y= , z= provided D  0,
D D D

a1 b1 c1 d1 b1 c1 a1 d1 c1
where D = a2 b2 c2 , D1 = d2 b2 c2 , D2 = a2 d2 c2
a3 b3 c3 d3 b3 c3 a3 d3 c3

a1 b1 d1
and D3 = a2 b2 d2
a3 b3 d3
 Cramer’s Rule (Con.)
Note:

(1) If D  0, then the system is consistent and has a unique

solution.

(2) If D=0 and D1 = D2 = D3 = 0, then the system has infinite

solutions or no solution.

(3) If D = 0 and one of D1, D2, D3  0, then the system

is inconsistent and has no solution.

(4) If d1 = d2 = d3 = 0, then the system is called the system of

homogeneous linear equations.

(i) If D  0, then the system has only trivial solution x = y = z = 0.

(ii) If D = 0, then the system has infinite solutions.

 Example
Using Cramer's rule , solve the following system of
equations
5x - y+ 4z = 5
2x + 3y+ 5z = 2
5x - 2y + 6z = -1

Solution :

5 -1 4 = 5(18+10) + 1(12-25)+4(-4 -15)

D= 2 3 5 = 140 –13 –76 =140 - 89
5 -2 6 = 51  0

5 -1 4 = 5(18+10)+1(12+5)+4(-4 +3)
D1 = 2 3 5 = 140 +17 –4
-1 -2 6 = 153
 Solution Cont.
5 5 4 = 5(12 +5)+5(12 - 25)+ 4(-2 - 10)
D2 = 2 2 5 = 85 + 65 – 48 = 150 - 48
5 -1 6 = 102

5 -1 5
= 5(-3 +4)+1(-2 - 10)+5(-4-15)
D3 = 2 3 2 = 5 – 12 – 95 = 5 - 107
5 -2 -1 = - 102

D0
D1 153 D 102
 By Cramer's Rule x = = = 3, y = 2 = =2
D 51 D 51
D3 -102
and z = = = -2
D 51
 Example
Solve the following system of homogeneous linear equations:
x + y – z = 0, x – 2y + z = 0, 3x + 6y + -5z = 0

Solution:

1 1 - 1
 
We have D = 1 -2 1  = 1 10 - 6  - 1 -5 - 3  - 1 6 + 6 
3 6 - 5 
= 4 + 8 - 12 = 0

 The system has infinitely many solutions.

Putting z = k, in first two equations, we get

x + y = k, x – 2y = -k
 Solution (Con.)
k 1
D1 -k -2 -2k + k k
 By Cramer's rule x = = = =
D 1 1 -2 - 1 3
1 -2
1 k
D2 1 -k -k - k 2k
y= = = =
D 1 1 -2 - 1 3
1 -2

These values of x, y and z = k satisfy (iii) equation.

k 2k
x= , y= , z = k , where k  R
3 3
 Eigen values and Eigen vectors
Eigenvalues and Eigenvectors
Diagonalization
Symmetric Matrices and Orthogonal Diagonalization
Application of Eigenvalues and Eigenvectors
Principal Component Analysis

7.108
 Eigenvalues and Eigenvectors
• Eigenvalue problem (one of the most important problems in the linear
algebra):
If A is an nn matrix, do there exist nonzero vectors x in Rn such that Ax is a
scalar multiple of x?

(The term eigen value is from the German word Eigen wert, meaning
“proper value”)
 Eigenvalue and Eigenvector:
A: an nn matrix
l: a scalar (could be zero) ※ Geometric Interpretation
x: a nonzero vector in Rn y
Ax = l x

Eigenvalue

Ax  lx
x

Eigenvector
x
• Ex 1: Verifying eigenvalues and eigenvectors
2 0  1  0
A  x1    x 2   
0  1 0  1 
Eigenvalue
※ In fact, for each eigenvalue, it
 2 0  1   2  1  has infinitely many eigenvectors.
Ax1          2    2x1 For l = 2, [3 0]T or [5 0]T are
 0 1 0   0  0 both corresponding
eigenvectors. Moreover, ([3 0] +
Eigenvector
[5 0])T is still an eigenvector.
The proof is in Thm. 7.1.
Eigenvalue
 2 0  0   0  0
Ax 2          1    (1)x 2
 0 1 1   1 1 

Eigenvector
• Thm. 7.1: The eigenspace corresponding to l of matrix A
If A is an nn matrix with an eigenvalue l, then the set of all eigenvectors of l
together with the zero vector is a subspace of Rn. This subspace is called the
eigenspace of l

Pf:
x1 and x2 are eigenvectors corresponding to l

(i.e., Ax1  l x1 , Ax 2  l x 2 )
(1) A( x1  x 2 )  Ax1  Ax 2  l x1  l x 2  l (x1  x 2 )
(i.e., x1  x 2 is also an eigenvector corresponding to λ)
(2) A(cx1 )  c( Ax1 )  c(l x1 )  l (cx1 )
(i.e., cx1 is also an eigenvector corresponding to l )
Since this set is closed under vector addition and scalar multiplication, this set
is a subspace of Rn according to Theorem 4.5
• Ex 3: Examples of eigenspaces on the xy-plane
For the matrix A as follows, the corresponding
eigenvalues are l1 = –1 and l2 = 1:
  1 0
A 
 0 1 
Sol:
For the eigenvalue l1 = –1, corresponding vectors are any vectors on the x-axis

 x   1 0   x    x   x  ※ Thus, the eigenspace

A           1   corresponding to l = –1 is the x-
0  0 1 0  0  0 axis, which is a subspace of R2

For the eigenvalue l2 = 1, corresponding vectors are any vectors on the y-axis

 0   1 0   0   0  0 ※ Thus, the eigenspace

A           1  corresponding to l = 1 is the y-
 y   0 1  y   y   y axis, which is a subspace of R2
※ Geometrically speaking, multiplying a vector (x, y) in R2 by the matrix A
corresponds to a reflection to the y-axis, i.e., left multiplying A to v can
transform v to another vector in the same vector space

 x   x 0    x 0
Av  A    A         A    A  
 y  0  y   0  y
 x  0   x
 1    1     
0  y   y 
• Thm. 7.2: Finding eigenvalues and eigenvectors of a matrix AMnn
Let A be an nn matrix.

(1) An eigenvalue of A is a scalar l such that det(l I  A)  0

(2) The eigenvectors of A corresponding to l are the nonzero
solutions of
(l I  A)x  0
 Note: follwing the definition of the eigenvalue problem
Ax  l x  Ax  l Ix  (l I  A)x  0 (homogeneous system)
(l I  A)x has
0 nonzero solutions for x iff det(l I  A)  0
(The above iff results comes from the equivalent conditions on Slide 4.101)
 Characteristic equation of A:

det(l I  A)  0
 Characteristic polynomial of AMnn:

det(l I  A)  (l I  A)  l n  cn1l n1   c1l  c0

• Ex 4: Finding eigenvalues and eigenvectors
2  12
A 
 1  5 

Sol: Characteristic equation:

l 2 12
det(l I  A) 
1 l 5
 l 2  3l  2  (l  1)(l  2)  0
 l  1,  2
Eigenvalue: l1  1, l2  2
  3 12   x1  0 
(1) l1  1  (l1 I  A)x       
 1 4   x2  0 
 3 12  G.-J. E. 1 4 
    
  1 4   0 0 
 x1   4t   4 
       t  , t  0
 x2   t   1 

 4 12   x1   0 
(2) l2  2  (l2 I  A)x       
 1 3   x2   0 
 4 12  G.-J. E. 1 3
    
  1 3   0 0 
 x1  3s   3
       s , s  0
 x2   s  1 
• Ex 5: Finding eigenvalues and eigenvectors
Find the eigenvalues and corresponding
eigenvectors for the matrix A. What is the
dimension of the eigenspace of each eigenvalue?

2 1 0
A  0 2 0 
 
 0 0 2 
Sol: Characteristic equation:

l 2 1 0
lI  A  0 l 2 0  (l  2)3  0
0 0 l 2
Eigenvalue: l 2
The eigenspace of λ = 2:

0 1 0   x1  0 
(l I  A)x  0 0 0   x2   0 
0 0 0   x3  0 
 x1   s  1 0
 x2    0   s  0   t 0  , s , t  0
       
 x3   t  0 1
 1   0  
     

    
s 0 t 0 s , t  R  : the eigenspace of A corresponding to l  2
  0  1  
     

Thus, the dimension of its eigen space is 2

• Notes:
(1) If an eigenvalue l1 occurs as a multiple root (k times) for the characteristic
polynominal, then l1 has multiplicity k
(2) The multiplicity of an eigenvalue is greater than or equal to the dimension of
its eigen space. (In Ex. 5, k is 3 and the dimension of its eigen space is 2)
• Ex 6：Find the eigenvalues of the matrix A and
find a basis
for each of the corresponding eigenspaces
1 0 0 0 
0 1 5  10
A 
1 0 2 0 
1 0 0 3 
Sol: Characteristic equation:

l 1 0 0 0 ※ According to the previous

slide, the dimension of the
0 l 1 5 10 eigenspace of λ1 = 1 is at
lI  A 
1 0 l 2 0 most to be 2
1 0 0 l 3 ※ For λ2 = 2 and λ3 = 3, the
demensions of their
 (l  1) 2 (l  2)(l  3)  0 eigenspaces are at most to
be 1
Eigenvalues: l1  1, l2  2, l3  3
 0 0 0 0   x1  0 
0 0 5 10   x2  0 
(1) l1  1  (l1 I  A)x   
 1 0 1 0   x3  0 
    
 1 0 0 2   x4  0 
 x1   2t   0   2
   s 
G.-J.E. x 1   0 
  2      s    t   , s, t  0
 x3   2t  0  2 
       
 x4   t  0  1 
 0    2  
    
 1   0  
 ,  is a basis for the eigenspace
0  2   corresponding to l1  1
0  1  
 
※The dimension of the eigenspace of λ1 = 1 is 2
 1 0 0 0   x1  0 
0 1 5 10   x2  0 
(2) l2  2  (l2 I  A)x   
 1 0 0 0   x3  0 
    
 1 0 0 1  x4  0 

 x1   0  0
  5t 
G.-J.E. x 5
  2     t  , t  0
 x3   t  1 
     
 4  
x 0 0
 0  
  
 5  is a basis for the eigenspace
 
 1   corresponding to l2  2
 0 
 
※The dimension of the eigenspace of λ2 = 2 is 1
 2 0 0 0   x1  0 
0 2 5 10   x2  0 
(3) l3  3  (l3 I  A)x   
 1 0 1 0   x3  0 
    
 1 0 0 0   x4  0 
 x1   0  0
   5t 
G.-J.E. x  5 
  2     t  , t  0
 x3   0  0
     
 x4   t  1

 0  
  
  5  is a basis for the eigenspace
 
  0   corresponding to l3  3
  1  
 
※The dimension of the eigenspace of λ3 = 3 is 1
 • Thm. 7.3: Eigenvalues for triangular matrices
If A is an nn triangular matrix, then its eigenvalues are the entries on its
main diagonal

 Ex 7: Finding eigenvalues for triangular and diagonal matrices

 1 0 0 0 0 
 0 2 0 0 0
 2 0 0   
 
(a) A   1 1 0  (b) A   0 0 0 0 0 
 
 5 3 3  0 0 0 4 0 
 0 0 0 0 3
Sol:
l 2 0 0
(a) l I  A  1 l 1 0  (l  2)(l  1)(l  3)  0
5 3 l  3 ※According to Thm. 3.2, the
determinant of a triangular
 l1  2, l2  1, l3  3 matrix is the product of the
entries on the main diagonal
(b) l1  1, l2  2, l3  0, l4  4, l5  3
 • Eigenvalues and eigenvectors of linear transformations:

A number l is called an eigenvalue of a linear transformation

T : V  V if there is a nonzero vector x such that T (x)  l x.
The vector x is called an eigenvector of T corresponding to l ,
and the set of all eigenvectors of l (together with the zero
vector) is called the eigenspace of l

※ The definition of linear transformation functions should be introduced in Ch 6

※ Here I briefly introduce the linear transformation and its some basic
properties
※ The typical example of a linear transformation function is that each
component of the resulting vector is the linear combination of the
components in the input vector x
 An example for a linear transformation T: R3→R3

T ( x1 , x2 , x3 )  ( x1  3x2 ,3x1  x2 , 2 x3 )
• Theorem: Standard matrix for a linear transformation

Let T : R n  R n be a linear trtansformation such that

 a11   a12   a1n 
a  a  a 
T (e1 )   21  , T (e 2 )   22  , , T (e n )   2 n  ,
     
     
 n1 
a  an 2   ann 
where {e1 , e 2 , , e n } is a standard basis for R n . Then an n  n
matrix A, whose i -th column correspond to T (ei ),
 a11 a12 a1n 
a a22 a2 n 
A  T (e1 ) T (e 2 ) T (e n )    21 ,
 
 
 an1 an 2 ann 
satisfies that T (x)  Ax for every x in R n . A is called the
standard matrix for T (T的標準矩陣)
 • Consider the same linear transformation T(x1, x2, x3) = (x1 + 3x2, 3x1 +
x2, –2x3)

1  1  0   3 0  0 
 T (e1 )  T ( 0  )   3 , T (e 2 )  T ( 1 )  1  , T (e3 )  T ( 0 )   0 
0  0  0  0  1   2 
 Thus, the above linear transformation T is with the following corresponding
standard matrix A such that T(x) = Ax

1 3 0  1 3 0   x1   x1  3 x2 
A   3 1 0   Ax   3 1 0   x2   3 x1  x2 
0 0 2  0 0 2   x3   2 x3 
※ The statement on Slide 7.18 is valid because for any linear transformation T: V →V,
there is a corresponding square matrix such that T(x) = Ax. Consequently, the
eignvalues and eigenvectors of a linear transformation T are in essence the
eigenvalues and eigenvectors of the corresponding square matrix A
 Ex 8: Finding eigenvalues and eigenvectors for standard matrices

Find the eigenvalues and corresponding eigenvectors for

1 3 0  ※ A is the standard matrix for T(x1, x2,
A  3 1 0  x3) = (x1 + 3x2, 3x1 + x2, –2x3) (see
Slides 7.19 and 7.20)
0 0 2 
Sol:

l  1  3 0 
lI  A    3 l  1 0   (l  2) (l  4)  0
2
 
 0 0 l  2
 eigenvalues l1  4, l2  2

For l1  4, the corresponding eigenvector is (1, 1, 0).

For l2  2, the corresponding eigenvectors are (1,  1, 0)
and (0, 0, 1).
• Transformation matrix A' for nonstandard bases
Suppose B is the standard basis of R n . Since the coordinate matrix of a vector
relative to the standard basis consists of the components of that vector, i.e.,
for any x in R n , x = [x]B , the theorem on Slide 7.19 can be restated as follows.

T (x)  Ax  T (x) B  A  x B , where A  T (e1 ) B T (e 2 ) B T (en )B 

is the standard matrix for T or the matrix of T relative to the standard
basis B
The above theorem can be extended to consider a nonstandard basis B ', which
consists of {v1 , v 2 , , vn}

T (x)B '  A '  xB ' , where A '  T ( v1 )B ' T ( v 2 )B ' T ( v n ) B ' 
is the transformation matrix for T relative to the basis B '
※ On the next two slides, an example is provided to verify numerically that this
extension is valid
• EX. Consider an arbitrary nonstandard basis B ' to be {v1,
v2, v3}= {(1, 1, 0), (1, –1, 0), (0, 0, 1)}, and find the
transformation matrix T (x)B '  A ' xB '
such that A ' corresponding to the same
linear transformation T(x1, x2, x3) = (x1 + 3x2, 3x1 + x2, –2x3)
 1   4 4  1   2  0
   
T ( v1 )B '  T ( 1  )    4    0  , T ( v 2 )B '  T (  1 )    2    2  ,
 0    0  B '  0    0    0  B '  0 
B' B'

 0  0 0

 
T ( v3 )B '  T ( 0  )    0    0 
 1    2  B '  2 
B'

4 0 0 
 A '   0 2 0 
 0 0 2 
• Consider x = (5, –1, 4), and check that T (x)  A 'x B' B'

corresponding to the linear transformation T(x1,

x2, x3) = (x1 + 3x2, 3x1 + x2, –2x3)
 5  2 8 5 2
 
T (x)B '  T (  1 )   14    6  ,  x B '   1   3  ,
  4    8 B '  8  4  B '  4
B'

4 0 0  2  8 
 A '  x B '   0 2 0   3    6   T ( x) B '
 0 0 2   4   8
 For a special basis 𝐵′ = 𝐯1 , 𝐯2 , … , 𝐯𝑛 , where 𝐯𝑖 ’s are eigenvectors of the standard matrix 𝐴,
𝐴′ is obtained immediately to be diagonal due to
𝑇 𝐯𝑖 = 𝐴𝐯𝑖 = 𝜆𝑖 𝐯𝑖
and
𝜆𝑖 𝐯𝑖 𝐵′ = 0𝐯1 + 0𝐯2 + ⋯ + 𝜆𝑖 𝐯𝑖 + ⋯ + 0𝐯𝑛 𝐵′ = 0 ⋯ 0 𝜆𝑖 0 ⋯ 0 𝑇

Let B ' be a basis of R 3 made up of three linearly independent eigenvectors

of A, e.g., B '  {v1 , v 2 , v 3}  {(1, 1, 0), (1,  1, 0), (0, 0, 1)} in Ex. 8

Then A ', the transformation matrix for T relative to the basis B ', defined as
[[T ( v1 )]B ' [T ( v 2 )]B ' [T ( v3 )]B ' ] (see Slide 7.22), is diagonal, and the main
diagonal entries are corresponding eigenvalues (see Slides 7.23)
for l1  4 for l2 2 4 0 0
B '  {(1, 1, 0), (1,  1, 0), (0, 0, 1)} A'  0  2 0 
 
Eigenvectors of A  0 0  2 
Eigenvalues of A
 Keywords
• eigenvalue problem
• Eigen value
• Eigen vector
• characteristic equation
• characteristic polynomial
• Eigen space
• multiplicity
• linear transformation
• diagonalization
 7.2 Diagonalization
• Diagonalization problem
For a square matrix A, does there exist an invertible matrix P such that P–1AP is
diagonal?

 Diagonalizable matrix

Definition 1: A square matrix A is called diagonalizable if there exists an

invertible matrix P such that P–1AP is a diagonal matrix (i.e., P diagonalizes A)
Definition 2: A square matrix A is called diagonalizable if A is similar to a
diagonal matrix

※ In Sec. 6.4, two square matrices A and B are similar if there exists an invertible
matrix P such that B = P–1AP.
 Notes:
This section shows that the eigenvalue and eigenvector problem is closely related to
the diagonalization problem
• Thm. 7.4: Similar matrices have the same eigenvalues
If A and B are similar nn matrices, then they have the same eigenvalues

Pf:
1 For any diagonal matrix in the
A and B are similar  B  P AP form of D = λI, P–1DP = D
Consider the characteristic equation of B:

l I  B  l I  P 1 AP  P 1l IP  P 1 AP  P 1 (l I  A) P
 P 1 l I  A P  P 1 P l I  A  P 1P l I  A
 lI  A
Since A and B have the same characteristic equation, they are with the same
eigenvalues

※ Note that the eigenvectors of A and B are not necessarily identical

• Ex 1: Eigenvalue problems and diagonalization programs
1 3 0 
A  3 1 0 
 
 0 0  2 
Sol: Characteristic equation:
l 1 3 0
l I  A  3 l 1 0  (l  4)(l  2) 2  0
0 0 l2

The eigenvalues : l1  4, l2  2, l3  2

1 
 
(1) l  4  the eigenvector p1  1 
 0 
 1 0
   
(2) l  2  the eigenvector p 2   1 , p 3  0 
 0  1 
1 1 0  4 0 0 
P  [p1 p 2 p3 ]  1 1 0  , and P 1 AP  0 2 0 
0 0 1   0 0 2 
 Note: If P  [p 2 p1 p3 ]
 1 1 0  2 0 0 
  1 1 0   P 1 AP   0 4 0 
 0 0 1   0 0 2 
※ The above example can verify Thm. 7.4 since the eigenvalues for both A and P–1AP
are the same to be 4, –2, and –2
※ The reason why the matrix P is constructed with the eigenvectors of A is
demonstrated in Thm. 7.5 on the next slide
• Thm. 7.5: Condition for diagonalization
An nn matrix A is diagonalizable if and only if it has n linearly
independent eigenvectors

※ If there are n linearly independent eigenvectors, it does not imply that there are n
distinct eigenvalues. In an extreme case, it is possible to have only one eigenvalue with
the multiplicity n, and there are n linearly independent eigenvectors for this eigenvalue

※ On the other hand, if there are n distinct eigenvalues, then there are n linearly
independent eigenvectors (see Thm. 7.6), and thus A must be diagonalizable
Pf: ( )
Since A is diagonalizable, there exists an invertible P s.t. D  P 1 AP
is diagonal. Let P  [p1 p 2 p n ] and D  diag (l1 , l2 , , ln ), then

 l1 0 0
0 l 0 
PD  [p1 p 2 pn ]  2

 
 
0 0 ln 
 [l1p1 l2p 2 lnp n ]
 AP  PD (since D  P 1 AP )
[ Ap1 Ap 2 Ap n ]  [l1p1 l2p 2 lnp n ]
 Api  li pi , i  1, 2, ,n
(The above equations imply the column vectors pi of P are eigenvectors
of A, and the diagonal entries li in D are eigenvalues of A)
Because A is diagonalizable  P is invertible
 Columns in P, i.e., p1 , p 2 , , p n , are linearly independent
(see Slide 4.101 in the lecture note)
Thus, A has n linearly independent eigenvectors
( )
Since A has n linearly independent eigenvectors p1 , p 2 , p n with
corresponding eigenvalues l1 , l2 , ln (could be the same), then
 Api  li pi , i  1, 2, ,n
Let P  [p1 p 2 pn ]
AP  A[p1 p 2 p n ]  [ Ap1 Ap 2 Ap n ]
 [l1p1 l2p 2 lnp n ]
l1 0 0
0 l 0 
 [p1 p 2 pn ]  2
 PD
 
 
0 0 ln 
Since p1 , p 2 , , p n are linearly independent
 P is invertible (see Slide 4.101 in the lecture note)
AP  PD  P 1 AP  D
 A is diagonalizable
(according to the definition of the diagonalizable matrix on Slide 7.27)

※ Note that p 's are linearly independent eigenvectors and the diagonal
i

entries li in the resulting diagonalized D are eigenvalues of A

• Ex 4: A matrix that is not diagonalizable
Show that the following matrix is not diagonalizable
1 2 
A 
 0 1 
Sol: Characteristic equation:
l 1 2
lI  A   (l  1) 2  0
0l 1
The eigenvalue l1  1, and then solve (l1 I  A)x  0 for eigenvectors

0 2  1 
l1 I  A  I  A     eigenvector p1   
0 0  0
Since A does not have two linearly independent eigenvectors, A is not
diagonalizable
• Steps for diagonalizing an nn square matrix:
Step 1: Find n linearly independent eigen vectors p1 , p 2 , pn for A with
corresponding eigen values l1 , l2 , , ln

Step 2: Let P  [p1 p 2 pn ]

l1 0
Step 3:
 0
1
 0 l2  0
P AP  D   
   
 0 0  ln 
where Api  li pi , i  1, 2, ,n
• Ex 5: Diagonalizing a matrix
 1  1  1
A 1 3 1
 
 3 1  1
Find a matrix P such that P 1 AP is diagonal.
Sol: Characteristic equation:

l 1 1 1
l I  A  1 l 3 1  (l  2)(l  2)(l  3)  0
3 1 l 1
The eigenvalues : l1  2, l2  2, l3  3
 1 1 1 1 0 1   x1  0 
l1  2  l1 I  A   1 1 1 G.-J. E.
 0 1 0   x2   0 
 3 1 3  0 0 0   x3  0 
 x1   t   1
 x    0   eigenvector p   0 
 2   1  
 x3   t   1 

 3 1 1  1 0  14   x1  0 
l2  2  l2 I  A   1 5 1 
G.-J. E.
 0 1 14   x2   0 
 3 1 1 0 0 0   x3  0 

 x1   14 t  1
 x     1 t   eigenvector p   1
 2  4  2  
 x3   t   4 
 2 1 1 1 0 1   x1  0 
l3  3  l3 I  A   1 0 1  G.-J. E.
 0 1 1  x2   0 
 3 1 4  0 0 0   x3  0 
 x1   t   1
 x    t   eigenvector p   1 
 2   3  
 x3   t   1 

 1 1 1
P  [p1 p 2 p 3 ]   0 1 1  and it follows that
 1 4 1 
2 0 0
P 1 AP   0 2 0 
 0 0 3 
 Note: a quick way to calculate Ak based on the diagonalization technique

 l1 0 0 l1k 0 0
0 l   
0  0 l2k 0
(1) D   2
 D 
k
   
   k
0 0 ln   0 0 ln 

(2) D  P 1 AP  D k  P 1 AP P 1 AP P 1 AP  P 1 Ak P
repeat k times

l1k 0 0
 
k 1  0 l2k 0
A  PD P , where D 
k k
 
 k
 0 0 ln 
• Thm. 7.6: Sufficient conditions for diagonalization
If an nn matrix A has n distinct eigenvalues, then the corresponding
eigenvectors are linearly independent and thus A is diagonalizable according
to Thm. 7.5.

Pf:
Let λ1, λ2, …, λn be distinct eigenvalues and corresponding eigenvectors be x1,
x2, …, xn. In addition, consider that the first m eigenvectors are linearly
independent, but the first m+1 eigenvectors are linearly dependent, i.e.,

xm1  c1x1  c2 x2   cm xm , (1)

where ci’s are not all zero. Multiplying both sides of Eq. (1) by A yields

Ax m 1  Ac1x1  Ac2 x 2   Acm x m

lm 1x m 1  c1l1x1  c2 l2 x 2   cm lm x m (2)
On the other hand, multiplying both sides of Eq. (1) by λm+1 yields

lm1xm1  c1lm1x1  c2lm1x2   cm lm1xm (3)

Now, subtracting Eq. (2) from Eq. (3) produces

c1 (lm1  l1 )x1  c2 (lm1  l2 )x 2   cm (lm1  lm )x m  0

Since the first m eigenvectors are linearly independent, we can infer that all coefficients
of this equation should be zero, i.e.,

c1 (lm1  l1 )  c2 (lm1  l2 )   cm (lm1  lm )  0

Because all the eigenvalues are distinct, it follows all ci’s equal to 0, which contradicts our
assumption that xm+1 can be expressed as a linear combination of the first m eigenvectors.
So, the set of n eigenvectors is linearly independent given n distinct eigenvalues, and
according to Thm. 7.5, we can conclude that A is diagonalizable
 Ex 7: Determining whether a matrix is diagonalizable

1  2 1 
A  0 0 1
 
0 0  3

Sol: Because A is a triangular matrix, its eigenvalues are

l1  1, l2  0, l3  3

According to Thm. 7.6, because these three values are distinct, A is

diagonalizable
• Ex 8: Finding a diagonalized matrix for a linear transformation
Let T : R 3  R 3 be the linear transformation given by
T (x1 ,x2 ,x3 )  (x1  x2  x3 , x1  3x2  x3 ,  3x1  x2  x3 )
Find a basis B ' for R 3 such that the matrix for T relative
to B ' is diagonal
Sol:
The standard matrix for T is given by

 1 1 1
A   1 3 1 
 3 1 1
From Ex. 5 you know that λ1 = 2, λ2 = –2, λ3 = 3 and thus A is
diagonalizable. So, similar to the result on Slide 7.25, these
three linearly independent eigenvectors found in Ex. 5 can be
used to form the basis B '. That is
B '  {v1 , v 2 , v3}  {(1, 0, 1), (1,  1, 4), ( 1, 1, 1)}

The matrix for T relative to this basis is

A '  [T ( v1 )]B ' [T ( v 2 )]B ' [T ( v 3 )]B ' 

2 0 0
  0 2 0 
 0 0 3 

※ Note that it is not necessary to calculate A ' through the above equation.
According to the result on Slide 7.25, we already know that A ' is a diagonal
matrix and its main diagonal entries are corresponding eigenvalues of A
 Symmetric Matrices and Orthogonal Diagonalization

• Symmetric matrix
A square matrix A is symmetric if it is equal to its transpose:

A  AT
 Ex 1: Symmetric matrices and nonsymetric matrices

0 1  2
A 1 3 0 (symmetric)
 
 2 0 5
4 3
B
3 1
(symmetric)

3 2 1
C  1  4 0 (nonsymmetric)
 
1 0 5
• Thm 7.7: Eigenvalues of symmetric matrices
If A is an nn “symmetric” matrix, then the following properties are true
(1) A is diagonalizable (symmetric matrices (except the matrices in the form of A =
aI, in which case A is already diagonal) are guaranteed to have n linearly
independent eigenvectors and thus be diagonalizable)
(2) All eigenvalues of A are real numbers
(3) If l is an eigenvalue of A with the multiplicity to be k, then l has k linearly
independent eigenvectors. That is, the eigenspace of l has dimension k

※ The above theorem is called the Real Spectral Theorem, and the set of
eigenvalues of A is called the spectrum of A
• Ex 2:
Prove that a 2 × 2 symmetric matrix is diagonalizable
a c 
A
 c b 
Pf: Characteristic equation:

l a c
lI  A   l2  (a  b)l  ab  c 2  0
c l b
As a function in l, this quadratic polynomial function has a
nonnegative discriminant as follows
(a  b) 2  4(1)(ab  c 2 )  a 2  2ab  b 2  4ab  4c 2
 a 2  2ab  b 2  4c 2
 (a  b) 2  4c 2  0  real-number solutions
(1) (a  b) 2  4c 2  0

 a  b, c  0
a c  a 0 
A    itself is a diagonal matrix
 c b 0 a
※ Note that in this case, A has one eigenvalue, a, whose multiplicity is 2,
and the two eigenvectors are linearly independent

(2) (a  b) 2  4c 2  0

The characteristic polynomial of A has two distinct real roots,

which implies that A has two distinct real eigenvalues.
According to Thm. 7.6, A is diagonalizable
• Orthogonal matrix
A square matrix P is called orthogonal if it is invertible and
P 1  PT (or PPT  PT P  I )
 Thm. 7.8: Properties of orthogonal matrices
An nn matrix P is orthogonal if and only if its column vectors
form an orthonormal set
Pf: Suppose the column vectors of P form an orthonormal set, i.e.,

P  p1 p2 pn  , where pi  p j  0 for i  j and pi  pi  1

 p1T p1 p1T p 2 p1T p n   p1  p1 p1  p 2 p1  p n 
 T  
 p p p T
2 p2 p 2T p1  p 2  p1 p 2  p 2 p 2  p1 
P P
T 2 1
  In
   
 T   
p n p1 p n p 2
T
p n p n  p n  p1 p n  p 2
T
pn  pn 

It implies that P–1 = PT and thus P is orthogonal

• Ex 5: Show that P is an orthogonal matrix.
 13 2 2 
 2 
3 3

P 5 1
5
0
 2 4 5 
3 5 3 5 3 5

Sol: If P is a orthogonal matrix, then P 1  PT  PPT  I

 1 2 2   13 2 2  1 0 0 
 3 3 3
 5 3 5
  I
PPT   25 1
5
0   32 1
5
4
3 5
   0 1 0 
 2 4
 2  0 0 1 
 3 5 3 5
  3 
5 5
3 5
0 3 5
  1   2   2 
 3   3  
3

Moreover, let p1   5  , p 2   5  , and p3   0  ,
2 1

 2   4   5 
 3 5   3 5  3 5 
we can produce p1  p 2  p1  p3  p 2  p 3  0 and p1  p1 
p 2  p 2  p3  p3  1

So, {p1 , p 2 , p3} is an orthonormal set (These results are

consistent with Thm. 7.8)
  Thm. 7.9: Properties of symmetric matrices
Let A be an nn “symmetric” matrix. If l1 and l2 are distinct eigenvalues of A, then their
corresponding eigenvectors x1 and x2 are orthogonal. (Thm. 7.6 states that for “any”
matrix, eigenvectors corresponding to distinct eigenvalues are linearly independent)

Pf:
l1 (x1  x2 )  (l1x1 )  x2  ( Ax1 )  x2  ( Ax1 )T x2  (x1T AT )x2
because A is symmetric
 (x1T A)x2  x1T ( Ax2 )  x1T (l2 x2 )  x1  (l2 x2 )  l2 (x1  x2 )
The above equation implies (l1  l2 )(x1  x 2 )  0, and because
l1  l2 , it follows that x1  x 2  0. So, x1 and x 2 are orthogonal
※ For distinct eigenvalues of a symmetric matrix, their corresponding
eigenvectors are orthogonal and thus linearly independent to each other
(Theorem 5.10 states that orthogonality implies linear independence)
※ Note that there may be multiple x1’s and x2’s corresponding to l1 and l2
 Orthogonal diagonalization

A matrix A is orthogonally diagonalizable if there exists an orthogonal matrix P

such that P–1AP = D is diagonal

• Thm. 7.10: Fundamental theorem of symmetric matrices

An nn matrix A is orthogonally diagonalizable and has real eigenvalues if and
only if A is symmetric

Pf:

( )
A is orthogonally diagonalizable
 D  P 1 AP is diagonal, and P is an orthogonal matrix s.t. P 1  PT
 A  PDP 1  PDPT  AT  ( PDPT )T  ( PT )T DT PT  PDPT  A
( )
See the next two slides
 Orthogonal diagonalization of a symmetric matrix:

Let A be an nn symmetric matrix.

(1) Find all eigenvalues of A and determine the multiplicity of each
※ According to Thm. 7.9, eigenvectors corresponding to distinct eigenvalues are
orthognoal
(2) For each eigenvalue of multiplicity 1, choose the unit eigenvector
(3) For each eigenvalue of the multiplicity to be k  2, find a set of k
linearly independent eigenvectors. If this set {v1, v2, …, vk} is not
orthonormal, apply the Gram-Schmidt orthonormalization process
It is known that G.-S. process is a kind of linear transformation, i.e., the
produced vectors can be expressed as c1 v1  c2 v 2   ck v k (see Slide 5.55),
i. Since Av1  l v1 , Av 2  l v 2 , , Av k  l v k ,
 A(c1 v1  c2 v 2   ck v k )  l (c1 v1  c2 v 2   ck v k )
 The produced vectors through the G.-S. process are still eigenvectors for l
ii. Since v1 , v 2 , , v k are orthogonal to eigenvectors corresponding to other
different eigenvalues (according to Thm. 7.9), c1 v1  c2 v 2   ck v k is also
orthogonal to eigenvectors corresponding to other different eigenvalues.
(4) The composite of steps (2) and (3) produces an orthonormal set of
n eigenvectors. Use these orthonormal and thus linearly
independent eigenvectors as column vectors to form the matrix P.
i. According to Thm. 7.8, the matrix P is orthogonal
ii. Following the diagonalization process on Slide 7.35, D = P–1AP
is diagonal
Therefore, the matrix A is orthogonally diagonalizable
• Ex 7: Determining whether a matrix is orthogonally diagonalizable
Symmetric Orthogonally
matrix diagonalizable
1 1 1
A1  1 0 1
 
1 1 1
 5 2 1
A2   2 1 8
 
  1 8 0 
3 2 0
A3  
2 0 1
0 0 
A4  
0  2
• Ex 9: Orthogonal diagonalization
Find an orthogonal matrix P that diagonalizes A.
 2 2 2 
A   2 1 4 
 2 4 1
Sol:
(1) lI  A  (l  3) 2 (l  6)  0
l1  6, l2  3 (has a multiplicity of 2)
v1
(2) l1  6, v1  (1,  2, 2)  u1   ( 13 , 2
3 , 2
3 )
v1
(3) l2  3, v 2  (2, 1, 0), v3  (2, 4, 5) ※ Verify Thm. 7.9 that
v1·v2 = v1·v3 = 0

orthogonal
※If v2 and v3 are not orthogonal, the Gram-Schmidt Process should be
performed. Here we simply normalize v2 and v3 to find the
corresponding unit vectors
v2 v3
u2   ( 25 , 1
5
, 0), u3   ( 325 , 3
4
5
, 5
3 5
)
v2 v3

 13 2 2    6 0 0
  2 15 3 5
  P 1 AP   0 3 0
P3 4

5 3 5  
 23 0 5   0 0 3 
 3 5 
u1 u 2 u3

※ Note that there are some calculation error in the solution of Ex.9 in
the text book
 Keywords
• symmetric matrix
• orthogonal matrix
• orthonormal set
• orthogonal diagonalization
 Applications of Eigenvalues and Eigenvectors
 The rotation for quadratic equation: ax2 + bxy + cy2 + dx + ey + f = 0

 Ex 5: Identify the graphs of the following quadratic equations

(a) 4 x 2  9 y 2  36  0 (b) 13x 2  10 xy  13 y 2  72  0

Sol:

x2 y 2
(a) In standard form, we can obtain 2  2  1.
3 2

※ Since there is no xy-term, it is easy

to derive the standard form and it
is apparent that this equation
represents an ellipse.
(b) 13x 2  10 xy  13 y 2  72  0
※ Since there is a xy-term, it is difficult to identify the graph of this equation.
In fact, it is also an ellipse, which is oblique on the xy-plane.

※ There is a easy way to identify the graph of

quadratic equation. The basic idea is to rotate
the x- and y-axes to x’- and y’-axes such that
there is no more x’y’-term in the new quadratic
equation.
※ In the above example, if we rotate the x- and y-
axes by 45 degree counterclockwise, the new
( x ') 2 ( y ') 2
quadratic equation 2  2  1 can be
3 2
derived, which represents an ellipse apparently.

※ In Section 4.8, the rotation of conics is achieved by changing basis,

but here the diagonalization technique based on eigenvalues and
eignvectors is applied to solving the rotation problem
 Quadratic form
ax2 + bxy + cy2 is the quadratic form associated with the quadratic equation ax2 +
bxy + cy2 + dx + ey + f = 0

 a b / 2
 Matrix of the quadratic form: A 
 b / 2 c 
※ Note that A is a symmetric
 x matrix
If we define X =  , then XTAX= ax2 + bxy + cy2 . In fact, the quadratic equation can
 y
be expressed in terms of X as follows

X T AX   d e X  f
 Principal Axes Theorem
 For a conic whose equation is ax2 + bxy + cy2 + dx + ey + f = 0, the rotation to
eliminate the xy-term is achieved by X = PX’, where P is an orthogonal matrix that
diagonalizes A. That is,

1 l1 0 
P AP  P AP  D  
T
 ,
 0 l2 
where λ1 and λ2 are eigenvalues of A. The equation for the rotated conic is given by

l1 ( x ')2  l2 ( y ')2   d e PX   f  0
Pf:
According to Thm. 7.10, since A is symmetric, we can conclude that there exists
an orthogonal matrix P such that P–1AP = PTAP = D is diagonal
Replacing X with PX’, the quadratic form becomes

X T AX  ( PX )T A( PX )  ( X )T PT APX 

 ( X )T DX   l1 ( x)2  l2 ( y)2
※ It is obvious that the new quadratic form in terms of X’ has no x’y’-term, and
the coefficients for (x’)2 and (y’)2 are the two eigenvalues of the matrix A
 x  x   x  x 
※ X  PX       v1 v 2     xv1  yv 2  Since   and   are
 y  y  y  y 
the orignal and new coodinates, the roles of v1 and v 2 (the eigenvectors
of A) are like the basis vectors (or the axis vectors) in the new coordinate
system
 Ex 6: Rotation of a conic
Perform a rotation of axes to eliminate the xy-term in the following quadratic
equation

13x 2  10 xy  13 y 2  72  0
Sol:
The matrix of the quadratic form associated with this equation is

13 5
A 
 5 13 
The eigen values are λ1 = 8 and λ2 = 18, and the corresponding eigenvectors are

1  1
x1    and x 2   
1 1
After normalizing each eigenvector, we can obtain the orthogonal matrix P as
follows.
※ According to the results on p.
 1 1  268 in Ch4, X=PX’ is
 2 2  equivalent to rotate the xy-
cos 45  sin 45  coordinates by 45 degree to
P  
 1 1   sin 45 cos 45  form the new x’y’-coordinates,
 2 2 which is also illustrated in the
figure on Slide 7.62

Then by replacing X with PX’, the equation of the rotated conic is

8(x)2  18( y) 2  72  0,

which can be written in the standard form
( x) 2 ( y) 2
2
 2 1
3 2
※ The above equation represents an ellipse on the x’y’-plane
 In three-dimensional version:
ax2 + by2 + cz2 + dxy + exz + fyz
is the quadratic form associated with the equation of quadric surface: ax2 + by2 + cz2
+ dxy + exz + fyz + gx + hy + iz + j = 0

 Matrix of the quadratic form:

 a d /2 e/2
A   d / 2 b f / 2  ※ Note that A is a symmetric
matrix
 e / 2 f / 2 c 
If we define X = [x y z]T, then XTAX= ax2 + by2 + cz2 + dxy + exz + fyz, and the quadratic
surface equation can be expressed as

X T AX   g h i  X  j
 Keywords
• quadratic form
• principal axes theorem
 Principal Component Analysis
 Principal component analysis
 It is a way of identifying the underlying patterns in data
 It can extract information in a large data set with many variables and
approximate this data set with fewer factors
 In other words, it can reduce the number of variables to a more manageable set

 Steps of the principal component analysis

 Step 1: Get some data
 Step 2: Subtract the mean
 Step 3: Calculate the covariance matrix
 Step 4: Calculate the eigenvectors and eigenvalues of the
covariance matrix
 Step 5: Deriving the transformed data set
 Step 6: Getting the original data back
 Step 1: Step 2:
x* y* x y
2.5 2.4 0.69 0.49
0.5 0.7 -1.31 -1.21
2.2 2.9 0.39 0.99
1.9 2.2 0.09 0.29
3.1 3.0 demeaned
 X T   x y
1.29 1.09
2.3 2.7  0.49 0.79
2.0 1.6 0.19 -0.31
1.0 1.1 -0.81 -0.81
1.5 1.6 -0.31 -0.31
1.1 0.9 -0.71 -1.01
1.81 1.91 0 0

Step 3:

  x T
   xT x xT y 
 
var( X )  E  XX   E   T 
T T
 x y   E  T 
  y   y x yT y 
 var( x) cov( x, y )   0.616556 0.615444 
    0.615444 0.716556   A
 cov( x , y ) var( y )   
  Step 4: Calculate the eigenvectors and eigen values of the covariance matrix A

 0.67787   0.73518 
l1  1.284028, v1 =   l2  0.049083, v 2 =  
 0.73518   0.67787 

1. The two eigenvectors are perpendicular

v1 (orthogonal) to each other according to
Thm. 7.9 (In fact, they are orthonormal
here)
v2 2. v1 eigenvector (corresponding to the
largest eigenvalue 𝜆1 ) is just like a best-
fit regression line
3. v2 seems less important to explain the
data since the projection of each node
on the v2 axis is very close to zero
4. The interpretation of v1 is the new axis
which retains as much as possible the
interpoint distance information (or
said the variance information) that was
contained in the original two
dimensions
※ The intuition behind the principal component analysis:
(1) Total variance of series of x and y = variance of x + variance of y
= sum of the main diagonal entries in the covariance matrix A
= 0.616556+0.716556 = 1.33311 (The series x, which are the coordinate values
on the x-axis, explains 0.616556/1.33311 = 46.25% of total variance)
(2) Consider P = [v1 v 2 ] and X = PX . (According to the Principal Axes Theorem
on Slide 7.65, it is equivalent to transform x- and y -axes to be v1 - and v 2 -axes,
i.e., the data is the same but with different coordinate values X  on the v1 v 2 -plane.)
 X   P 1 X  PT X  (X )T  X T P, where (X )T  [ x y] and X T  [ x y ]
l 0  (It also implies that the new
 var((X )T ) = var(X T P)  PT var( X T ) P  PT AP  D   1  series of x’ and y’ are
 0 l2  independent)
(3) Total variance of transformed series of x and y (called principal components) in X 
= variance of x + variance of y
= sum of the main diagonal entries in the covariance matrix var((X )T ) = l1  l2
(4) A property for eigenvalues: Trace(A) =  li , which means that after the transformation,
the total variance remains the same. In this case, l1  l2  1.284028  0.049083  1.33311.
(5) The new series x, which are the coordinate values on the v1 -axis, explains l1 /(l1  l2 )
 1.284028 /(1.284028  0.049083)  96.32% of total variance
  Step 5: Deriving the transformed data set: (X )T  X T P
v  0.67787 v21  0.73518
Case 1: P   v1 v 2    11  Case 2: Set y '  0 on purpose
v12  0.73518 v22  0.67787 
 v  0.67787 v21  0.73518 (X )T   x y
(X )T   x y   x y   11 
v12  0.73518 v22  0.67787    v11 x  v12 y 0
  v11 x  v12 y v21 x  v22 y    0.67787 x  0.73518 y 0 
  0.67787 x  0.73518 y  0.73518 x  0.67787 y 

x’ y’ x’ y’
-0.82797 -0.17512 -0.82797 0
1.77758 0.14286 1.77758 0
-0.99220 0.38437 -0.99220 0
-0.27421 0.13042 -0.27421 0
-1.67580 -0.20950 -1.67580 0
( X )T  -0.91295 0.17528 ( X )T  -0.91295 0
0.09911 -0.34982 0.09911 0
1.14457 0.04642 1.14457 0
0.43805 0.01776 0.43805 0
1.22382 -0.16268 1.22382 0
0 0 0 0

1.284028 0  1.284028 0 
 var((X )T ) =   var((X )T ) = 
 0 0.049083  0 0 
 Step 6: Getting the original data back:

X T  ( X )T P1 ( ( X )T PT )  original mean, where P = [v1 v2 ]

 v  0.67787 v12  0.73518
 x y    x y v11  0.73518     v11 x  v21 y v12 x  v22 y
 21 v22 0.67787 
  0.67787 x  0.73518 y  0.73518 x  0.67787 y
case 1 case 2
x* y* x* y*
2.5 2.4 2.37 2.52
0.5 0.7 0.61 0.60
2.2 2.9 2.48 2.64
1.9 2.2 2.00 2.11
3.1 3 2.95 3.14
2.3 2.7 2.43 2.58
2 1.6 1.74 1.84
1 1.1 1.03 1.07
1.5 1.6 1.51 1.59
1.1 0.9 0.98 1.01
1.81 1.91 1.81 1.91

※ We can derive the original data set if ※ Although when we derive the transformed data, only v1
we take both v1 and v2 and thus x’ and thus only x’ are considered, the data gotten back is
and y’ into account when deriving still similar to the original data. That means, x’ can be a
the transformed data common factor almost able to explain both series x and y
 v1

v2

※ If only the principal component x’ is considered in the Principal Component Analysis, it is

equivalent to project all points onto the v1 vector
※ It can be observed in the above figure that the projection onto v1 vector can retains as
much as possible the interpoint distance information (variance) that was contained in the
original series of (x, y)
 Factor loadings: the correlations between the principal components (F1 = x’ and F2
= y’) and the original variables (x1 = x and x2 = y) (Note that x and y should be
expressed as the linear combination of x’ and y’ for deriving factor loadings)

vij li
lij   Fi x j 
s.d . j
0.67787 1.284028 0.73518 0.049083
l11   xx   0.97824 l21   yx   0.20744
0.785211 0.785211

 Factor loadings are used to identify and interpret the unobservable principal
components

0.73518 1.284028 0.67787 0.049083

l12   xy   0.98414 l22   yy   0.177409
0.846496 0.846496

※ The factor loadings of x’ on x and y are close to 1 in absolute levels, which

implies the principal component x’ can explain x and y quite well and thus x’
can viewed as an index to represent the combined information of x and y
※ Questionnaire analysis: salary vs. personality and learned-courses
information
i. If there are five variables (the results of five problems), x1 to x5, x1 to x3 are
about personality information of the respondent and x4 and x5 are about
the course information which this respondent had learned
ii. Suppose there are two more important principal components x1’ and x2’
(x3’ , x4’ , and x5’ are less important principal components), and the
principal component x1’ is highly correlated with x1 to x3 and the principal
component x2’ is highly correlated with x4 and x5
iii. x1’ can be interpreted as a general index to represent the personal
characteristic, and x2’ can be interpreted as a general index associated with
the course profile that the respondent had learned
iv. Study the relationship between the salary level and the personal-
characteristic index x1’ and education index x2’
Pamela Leutwyler
  7  2  14 
 
A  5  4  7 
 5  1  10 
 

Find the eigenvalues and eigenvectors

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[ ]  (5)[ ]  (5)[ ]

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[l2  14l  33]  (5)[ ]  (5)[ ]

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[l2  14l  33]  (5)[2l  6]  (5)[ ]

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[l2  14l  33]  (5)[2l  6]  (5)[14l  42]

next
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[l2  14l  33]  (5)[2l  6]  (5)[14l  42]

l3  14l2  33l
 7l2  98l  231
 10l  30
 70l  210
next
 l3  7l2  15l  9
  7  2  14  l  7  2  14 
   
A  5  4  7  lI  A    5 l  4  7 
 5  1  10   5  l  
   1 10 

l  7  2  14 
 
det(lI  A)  det  5 l  4 7 
 5  l  
 1 10 

(l  7)[l2  14l  33]  (5)[2l  6]  (5)[14l  42]

l3  14l2  33l
 7l2  98l  231
 10l  30
 70l  210
characteristic polynomial next
 l3  7l2  15l  9
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 

l3  14l2  33l
 7l2  98l  231
 10l  30
 70l  210
characteristic polynomial next
 l3  7l2  15l  9
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 7 15 9

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9
1
1 8

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9
1 8
1 8 23

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9
1 8 23
1 8 23 31

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

1 1 7 15 9
1 8 23
1 8 23 31
This is not zero. 1 is not a root.

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3
1 4

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3 -12
1 4 3

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3 -12 -9
1 4 3 0

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3 -12 -9
1 4 3 0
This is zero. -3 is a root.

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3 -12 -9
1 4 3 0
l  7l  15l  9  (l  3)(l  4l  3)
3 2 2

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10  potential rational roots:1,-1,3,-3,9,-9
 

synthetic division:

-3 1 7 15 9
-3 -12 -9
1 4 3 0
l  7l  15l  9  (l  3)(l  4l  3)
3 2 2

 (l  3)(l  3)(l  1)  0 next

  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

synthetic division:

-3 1 7 15 9
-3 -12 -9
1 4 3 0
l  7l  15l  9  (l  3)(l  4l  3)
3 2 2

 (l  3)(l  3)(l  1)  0 next

  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

To find an eigenvector belonging to the repeated root –3,

consider the null space of the matrix –3I - A

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

To find an eigenvector belonging to the repeated root –3,

consider the null space of the matrix –3I - A

  10 2 14   1  .2  1.4 
   
 3I  A    5 1 7  reduces to  0 0 0 
 5 1 7  0 0 
   0 

The 2 dimensional null space of this

matrix has basis =
 1   7 
   
 5 ,  0 
 0   5  next
   
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

To find an eigenvector belonging to the repeated root –1,

consider the null space of the matrix –1I - A

  8 2 14  1 0  2
   
 1I  A    5 3 7  reduces to  0 1  1 
5 1 9  0 0 0 
   

The null space of this

matrix has basis =
 2 
 
 1 
 1  next
 
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

The eigenvectors are: 1 7  2

    
 5 ,  0  1
0  5 1
    

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

The eigenvectors are: 1 7  2

    
 5 ,  0  1
0  5 1
    

1
3 0 0   1 7 2   7  2  14  1 7 2 
      
 0  3 0    5 0 1   5  4  7  5 0 1 
 0    0 5 1   5  1  10  0 5 1 
 0 1     

next
  7  2  14  l  7l  15l  9  0
3 2

 
A  5  4  7 
 5  1  10 
 
The eigenvalues are: -3, -3, -1

The eigenvectors are: 1 7  2

    
 5 ,  0  1
0  5 1
    

1
3 0 0   1 7 2   7  2  14  1 7 2 
      
 0  3 0    5 0 1   5  4  7  5 0 1 
 0    0 5 1   5  1  10  0 5 1 
 0 1     
P –1 A P
diagonal matrix
that is similar
to A
 For more details
• Prof. Gilbert Strang’s course videos:

• http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-
2005/VideoLectures/index.htm

• Esp. the lectures on eigenvalues/eigenvectors, singular value

decomposition & applications of both. (second half of course)

• Online Linear Algebra Tutorials:

• http://tutorial.math.lamar.edu/AllBrowsers/2318/2318.asp

• http://www.cliffsnotes.com/study_guide/Determining-the-
Eigenvectors-of-a-Matrix.topicArticleId-20807,articleId-20804.html
 Course Name: Engineering Statistics and Linear Algebra(ESLA)

Innovative teaching and Learning

methods:
• Beyond the syllabus: simulation using Linear
Algebra Toolkit
http://www.math.odu.edu/~bogacki/cgi-
bin/lat.cgi

220
Software
List of numerical analysis software
• Several programming languages use numerical linear algebra
optimisation techniques and are designed to implement
numerical linear algebra algorithms. These languages
include MATLAB, Analytica, Maple, and Mathematica.

• Other programming languages which are not explicitly designed

for numerical linear algebra have libraries that provide
numerical linear algebra routines and
optimisation; C and Fortran have packages like Basic Linear
Algebra Subprograms and

• LAPACK, python has the library NumPy, and Perl has the Perl
Data Language.

• Many numerical linear algebra commands in R rely on these

more fundamental libraries like LAPACK.