Chapter 2 - Numerical Methods in Linear Algebra

Ho Chi Minh City University of Technology (HCMUT), VNU-HCM
CHAPTER 2
NUMERICAL METHODS IN LINEAR ALGEBRA
Dr. Le Thanh Long

ltlong@hcmut.edu.vn
Department of Machine Design - Faculty of Mechanical Engineering

OUTLINE
2.1. Matrices and Determinants

2.2. System of linear equations
2.3. Eigenvalues and Eigenvectors


1. Matrices
A matrix A with size m x n is a rectangular
table of numbers, which contains m rows and
n columns.
 a11  a1 j  a1n 
 
      
A   ai1  aij  ain 
 
      
a  amj  amn 
 m1
The numbers are called the entries.

3


1. Matrices
Diagonal matrix
1 0 00
Diagonal matrix is the square matrix in  
 0 4 0 0
which all the entries outside the main A
0 0 1 0 
diagonal are all zero.  
0 0 0 2
Identity matrix
1 0 0 0
A square matrix with 1’s on the main  
 0 1 0 0
diagonal and zeros elsewhere is called an A
identity matrix of order n and is denoted by 0 0 1 0
 
I or I 0 0 0 1
4


1. Matrices
Transpose of a matrix
If = ( ) × is any m x n matrix, then the transpose of A,
denoted by = ( ) × is defined to be the n x m matrix
that results by interchanging the rows and columns of A.
 a11 a12  a1n   a11 a21  am1 

   
 a21 a22  a2 n  T  a12 a22  am 2 
A ,A 
           
   
 am1 am 2  amn   a1n a2 n  amn 


1. Matrices
Row-echelon form matrix
1) A matrix is in row-echelon form if it has the following
 1 4 1   1 0 0
   
C   0 2 2  C   1 2 0 
0 0 3   2 2 3 
   
2) The first non-zero element of each row is called the leading

entry of this row.


1. Matrices
Rank of a matrix
• We denote A → B to show that B is the matrix that

results from A by performing some of the Elementary
Row Operations.
• If × → × , where B is in row-echelon form, then

the rank of the matrix A is the number of non-zero rows
of matrix B and is denoted by r(A).


1. Matrices
Properties
If A, B, C are the matrices with the same size, then:
1) A + B = B + A
2) A + (B + C) = (A + B) + C
3) α A + B = αA + αB, ∀α ∈ K
4) α + β A = αA + βA, ∀α, β ∈ K
5) A + 0 = 0 + A = A
8


2. Determinants
Definition
If = is a square matrix, then the determinant of A is
a number. We denote it by det(A) or |A|
Cofactor of entry a :
ℎ
C = (−1) ℎ − ℎ −
= (−1)


2. Determinants
Determinant’s properties
1) det( ) = det 2) det(AB) = det(A).det(B)
3) | = | 4) | |=| |
5) A has a row (or column) equal to 0 then |A| = 0
6) A has 2 row (or column) proportional to each other then |A| = 0
7) det(A+B) ≠ det(A) + det(B)
10


2. Determinants
a) Evaluating determinants by the inductive method.
The determinant of the square matrix can be computed by a
cofactor expansion along any row or column. That is:
A = (a ) => |A| = a .
a a
A= a a => |A| = a C +a C =a a -a a
...
a … a
A= … … … ⇒ A =a C +a C + ... + a C
… … …
11


2. Determinants
a) Evaluating determinants by the inductive method.
Example 2.1:  1 2 3 
 
Find the determinant det(A) of A   2 3 0
Solution: 3 2 4 

det(A) = a C +a C +a C
=1C +2C -3C
3 0 2 0 2 3
= 1 −1 + 2 −1 - 3 −1
4 3 3 4 2 3
= 12 – 16 + 15 = 11 12


2. Determinants
Example 2.2: Evaluating the area of the triangle
2.5 1 1
1 
  1 5
S  abs  AB, AC   abs 3 2 1 
2 2 4
1 3 1
Note: abs(x) is the absolute value of x
13


2. Determinants
b) Evaluating determinants by cofactor expansion
The determinant of the square matrix can be computed by a
cofactor expansion along any row or column.
Example 2.3:
 1 2 1 
Find the determinant det(A) of A   2 1 3 
 0 0 3 
 
Cofactor expansion along the third row:
1 2
det(A) = − 3(−1) = −3 −3 = 9
2 1
14


2. Determinants
b) Evaluating determinants by cofactor expansion
Example 2.4:  2 3 3 2
3 0 1 4 
Find the determinant det(B) of B  
2 0 3 2
Solution:  
4 0 1 5 
Cofactor expansion along the second row:
3 1 4
det(B) = -3(−1) −2 3 2
4 −1 5
Cofactor expansion along the first row:
3 2 −2 2 −2 3
det(B) = 3 3(−1) + 1 (−1) + 4(−1)
−1 5 4 5 4 −1
= 3(51+18-40) = 87 15


2. Determinants
c) Evaluating determinants by row reduction.
• Select one element, using elementary row operation to
eliminate all the entries in the same column of the selected one.
• Using cofactor expansion along the above column.
Example 2.5: Find the determinant of matrix I?
3 2 1 1 3 2 1 1
r3  2 r1
2 3 2 0 r4  r1 2 3 2 0
I 

3 1 4 2 3 5 2 0
4 1 3 1 1 1 4 0
2 3 2 2 3 2
5 8
1 3 5 2   5 8 0  2  30
5 5
1 1 4 5 5 0
16


3. Inverse of a matrix
If = is any n x n matrix and is cofactor of entry , then:
T
 C11  C1 j  C1n   C11  Ci1  Cn1 
 
             


PA   Ci1  Cij  Cin    C1 j  Cij  Cnj 
   
             
 Cn1
  Cnj  Cnn   C1n  Cin  Cnn 
Is called the adjoint of A and is denoted by adj(A) or

A square matrix A is invertible if and only if det(A) ≠ 0 and then
1 1
A  PA
det A
17


Using the adjoint to find an inverse matrix
• Step 1: Evaluate det(A) to test for Invertibility
• Step 2: Find adjoint of A
Where = (−1)
• Step 3: =
18


3. Inverse of a matrix 1 1 1 
Example 2.6: Find the inverse of the matrix A   2 3 1 
 
 3 4 0 
Solution:
det(A) = -2 ≠ 0 => A is invertible
11 3 1 1 2 2 1 1 3 2 3
C11  ( 1)  4; C12  (1)  3; C13  (1)  1
4 0 3 0 3 4
C = 4; C = −3; C = -1; C = −2; C = 1; C =1
 4 4 2 
1 1  
A1  PA  3 3 1
| A| 2  
 1 1 1 
19


Properties
• If A, B are invertible, then P = P P
Since
(AB) = B A ⇒ = .
• If A is invertible and α ≠ 0, then
(αA) = A .Since ( A ).(αA)=I
• If A is invertible, then A , A are also invertible and
(A ) = A; (A ) = (A ) . Indeed,
A . A = I, (A ) . A = (A. A ) = I = I
20

2.2 System of linear equations

 A  X    B 
1. Substitution method
By using the substitution method, you must find the value of one
variable and then substitute that variable into the other equation.
Example 2.7: Solve these system equations.
 x1  x2  3  x1  x2  3  x1  x2  3  x1  8
   
3 x1  4 x2  4 3( x2  3)  4 x2  4  x2  5  x2  5
 2 x1  x2  7  x2  7  2 x1  x2  7  2 x1  x1  2
   
 x1  4 x2  10  x1  4(7  2 x1 )  10  x1  2  x2  3
21


2. Gaussian Elimination
A   A | B
• Step 1: Perform elementary row operations to simplify
matrix ̅ to row-echelon form matrix.
• Step 2: Solve and analyze:
• If r(A) = r(A) = n: The system has unique solution.
• If r(A) = r(A) < n: The system has infinitely many solutions.
• If r(A) ≠ r(A): The system has no solution.
22


2. Gaussian Elimination
Example 2.8:  x1  5 x2  4 x3  7
Solve the system by Gaussian elimination 2 x1  9 x2  x3  4
3 x  11x  7 x  17
Solution:  1 2 3
1 5 4 7  1 5 4 7  1 5 4 7 
     
 2 9  1 4 
  0 1  9 18 
  0 1 9 18 
 3 11 7 17   0 4 19 38   0 0 17 34 
 x1  5 x2  4 x3  7  1
=> The system has unique solution.  x2  9 x3  18  0
 x  34 /17  2
 3 23


2. Gaussian Elimination mx1  x2  x3  1

Example 2.9: Solve the system by Gaussian elimination  x1  mx2  x3  m
 2
 x1  x2  mx3  m
Solution:
 m 1 1 1   1 m 1 m  1 m 1 m  1 m 1 m 
     2 2   2 
 1 m 1 m 
  m 1 1 1 
  0 1  m 1  m 1  m 
  0 1  m m  1 m  m 
2 2 2 2 2 2 2
 1 1 m m   1 1 m m   0 1  m m  1 m  m  0 1  m 1  m 1  m 
1 m 1 m 
 
 0 1  m m 1 m2  m 
0 0 (1  m)(m  2) (1  m)(1  m) 2 
If m = 1 => r(A) = r(A) = 1 < 3: Infinitely many solutions.
If m = -2 => r(A) ≠ r(A) = 2: no solution.
If m ≠ 1;-2 => r(A) = r(A) = 3 = n: unique solution 24


3. Cramer’s Rule (Only use when the number of variables is
equal to the number of equations)
det Ai | A1 | |A | |A |
xi  ; i  1...n Example: x1  ; x2  2 ; x3  3
det A | A| | A| | A|
det(A) ≠ 0: The system has a unique solution.
det(A) = 0:
+ det(A ) ≠ 0: The system has no solution.
+ det(A ) = 0: The system has many solutions
(Use Gaussian Eli. to solve)
25


3. Cramer’ Rule
 x1  2 x2  3 x3  6
Example 2.10: Solve the system by Cramer’s Rule 
 2 x1  x2  x3  2
Solution: 3 x  x  2 x  2
 1 2 3
1 2 3  6 2 3   | A1 | 30
A   2 1 1   A  30; A1   2 1 1   A1  30 x
 1 | A |  30  1

 3 1 2   2 1 2  
 | A | 30
  x2  2  1
 | A | 30
1 6 3  1 2 6 
 | A3 | 30
A2   2 2 1   A2  30; A3   2 1 2   A3  30  x3   1
 | A | 30
 3 2 2   3 1 2 
The system has a unique solution.
26


3. Cramer’ Rule  mx1  x 2  x3  1
Example 2.11: Solve the system by Cramer’s Rule 
 x1  mx 2  x3  m
 2
Solution: x
 1  x 2  mx 3  m
m 1 1 
A   1 m 1    d e t ( A )  ( m  1) 2 ( m  2 )
 1 1 m 
 1 1 1  m 1 1  m 1 1 
A 1   m m 1  ; A 2   1 m 1  ; A 3   1 m m 
 m 2 1 m   1 m2 m   1 1 m 2 
det( A1 )   ( m  1) 2 ( m  1); det( A2 )  ( m  1) 2 ; det( A3 )  ( m  1) 2 ( m  1) 2
27


3. Cramer’ Rule
Example 2.11 Solution (cont):
det(A) ≠ 0 => m ≠ 1; m ≠ -2: The system has unique solution
det A1 (m  1) 2 ( m  1) m 1
x1   2

det A (m  1) (m  2) m2
det A2 (m  1) 2 1
x2   
det A (m  1) 2 (m  2) m  2
det A3 (m  1) 2 (m  1) 2 (m  1) 2
x3   2

det A (m  1) (m  2) m2
det(A) = 0 => m = 1; m = -2:
m = -2: det(A ); det A ; det A ≠0: No solution
m = 1: det(A ) = det(A ) = det(A ) = 0: Solve by Gaussian.
28


3. Cramer’ Rule
Example 2.11 Solution (cont):
With m = 1. Then:
r(A) = r(A) < n The system has many solutions
 x1  x2  x3  1

 x1  x2  x3  1  x1  x2  x3  1
x  x  x  1
 1 2 3
 x1  1    

 x2   ; ,   R
x  
 3
29


4. Cholesky Decomposition AX  B
T
Symmetric positive definite matrix A can be decomposed into A  C .C
Elements of C are determined by:
ai1
c11  a11 ; ci1  (2  i  n)
c11
i 1
cii  aii   cik2 ;(1  i  n)
k 1
j 1
1
cij  (aij   cik c jk );(1  j  n)
c jj k 1
CY  B  Y  C 1 B
C T X  Y  X  (C T ) 1Y
30


4. Cholesky Decomposition  2 2 3  x1   7 
Example 2.12: AX  B; A   2 5 4  ; X   x2  ; B   12 
Solution:  3 4 5   x3   12 
A is symmetric matrix
1  2  0;  2  6  0;  3  1  0 => A is positive definite matrix
 c11 0 0   c11 c21 c31   2 2 3
A  C.C T   c21 c22 0  .  0 c22 c32    2 5 4 
 c31 c32 c33   0 0 c33   3 4 5 
2 2 3 3
c11.c11  2  c11  2; c11.c21  2  c21     2; c11.c31  3  c31  
c11 2 c11 2
2 2 2
c21  c22  5  c22  5  c21  52  3
1 1 1
c21.c31  c22 .c32  4  c32  (4  c21c31 )  (4  3) 
c22 3 3
c312  c32
2 2
 c33 2
 c33  5  c31 2
 c32  5  9 / 2  1/ 3  1/ 6 31


4. Cholesky Decomposition
Example 2.12 Solution (cont)
 2 0 0   2  2 3 / 2 
  T  
C  2 3 0  ;C   0 3 1/ 3 
   
 3 / 2 1/ 3 1/ 6   0 0 1 / 6 
1
 2 0 0   7   7/ 2 
1
   12    5 / 3 
CY  B  Y  C B    2 3 0     
   12   1/ 6 
 3 / 2 1/ 3 1/ 6   
1
 2  2 3 / 2   7/ 2   3 
T T 1
     
C X  Y  X  (C ) Y   0 3 1/ 3   5 / 3    2 
    1
 0 0 1/ 6   1/ 6   
32

2.3 Eigenvalues and Eigenvectors

Definition:
Let A be an n x n matrix, a scalar λ is called eigenvalue of A if there
exists a nonzero x in such that
Ax=λx
The vector x is called an eigenvector corresponding to λ.
Ax x
x
O
Ax
33


• Ax=λx may be written:
Ax – λx = 0
Given
(A – λ )x = 0
• Solving equation A – λ = 0 for λ leads to all
eigenvalues of A.
• On expending the determinant A – λ , we get a
polynomial in λ
• This polynomial is called the characteristic polynomial
of A
• The equation A – λ = 0 is called the characteristic
equation of A.
34


Example 2.13: Finding eigenvalues and eigenvectors of the matrix
 4 6 
A 
Solution:  3 5 
Firstly, derive the characteristic polynomial of A
 4 6  1 0   4   6 
A   I2        
 3 5   0 1   3 5   
Now solve the characteristic equation of A
A   I 2  (4   )(5   )  18   2    2
2  1  2
    2  0  
2  1
The eigenvalues of A are 2 and -1
The corresponding eigenvectors are found by using these values of λ
in the equation (A – λ )x = 0. There are many eigenvectors
corresponding to each eigenvalue. 35


Example 2.13:
Solution (cont):
6 6 0      1 
• λ =2    x       
 3 3 0   1
3 6 0   2    2 
• λ =−    x    
 3 6 0    1
The corresponding eigenvectors of λ = 2 and λ = −1 are vectors

−1 −2
have the form of and β , respectively.
1 1
36


Example 2.14: Google’s PageRank
Original idea of PageRank
Figure 2.3.1: A simple Internet web containing 6 pages.

The author of P1 thinks pages 2,4,5 and 6 have good content and links to
them. The author of P2 only likes P3, P4 so only links from her page to
them…Google engineers assumed each of these pages is related in some way
to the other page, since there is at least one link to and from each page in the
web. Their task was to find most important page for a particular search query.
37


Google’s use of eigenvalues and eigenvectors
P1 has 4 outgoing links. So in the 1st column of “link
matrix”, we place value ¼ in each of rows 2,4,5,6. The rest
of the rows in column 1 have value 0. We continue process
for the rest of 6 pages.
 0 0 0 0 1/ 2 0 
1/ 4 0 0 0 0 0 

 0 0 0 0 0 0
A 
1/ 4 1/ 2 0 0 1/ 2 0 
1/ 4 1/ 2 1 1 0 1
 
1/ 4 0 0 0 0 0
38


Next, to find the eigenvalues.
  0 0 0 1/ 20
1/ 4  0 0 0 0 

 0 0  0 0 0
A  I   
1/ 4 1/ 2 0  1/ 2 0 
1/ 4 1/ 2 1 1  1 
 
1/ 4 0 0 0 0  
4 3 2
5  
 6   
8 4 8
   0.72031; 0.13985  0.39240 j;0;1
39


We can only use non-negative, real values of λ => λ=1.

Using Wolfram|Alpha, the corresponding eigenvector is:
v = 4 1 0.5 5.5 8 1
As page 5 has the highest PageRank (8), it is the most

“important” and will appear at the top of the search results
40

41

Chapter 2 - Numerical Methods in Linear Algebra

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 2 - Numerical Methods in Linear Algebra

Uploaded by

Copyright:

Available Formats

Ho Chi Minh City University of Technology (HCMUT), VNU-HCM

Dr. Le Thanh Long

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

The numbers are called the entries.

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

 a11 a12  a1n   a11 a21  am1 

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

2) The first non-zero element of each row is called the leading

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

• We denote A → B to show that B is the matrix that

• If × → × , where B is in row-echelon form, then

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

1) det( ) = det 2) det(AB) = det(A).det(B)

5) A has a row (or column) equal to 0 then |A| = 0

6) A has 2 row (or column) proportional to each other then |A| = 0

7) det(A+B) ≠ det(A) + det(B)

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

=1C +2C -3C

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Is called the adjoint of A and is denoted by adj(A) or

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

• Step 2: Find adjoint of A

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

C = 4; C = −3; C = -1; C = −2; C = 1; C =1

Department of Machine Design - Faculty of Mechanical Engineering

2.1. Matrices and Determinants

Department of Machine Design - Faculty of Mechanical Engineering

2.2 System of linear equations

Department of Machine Design - Faculty of Mechanical Engineering

2.2 System of linear equations

Department of Machine Design - Faculty of Mechanical Engineering

2.2 System of linear equations

Department of Machine Design - Faculty of Mechanical Engineering

2.2 System of linear equations

If m = -2 => r(A) ≠ r(A) = 2: no solution.