Chap3 SYSTEM LINEAR EQUATION

SYSTEM OF LINEAR EQUATIONS
Dau The Phiet

HCMUT-OISP
TP. HCM — 2020.
Dau The Phiet (BK TPHCM) SYSTEM OF LINEAR EQUATIONS TP. HCM — 2020. 1 / 40
Problem
In this chapter, we will solve system of linear

equations:


 a 11 x 1 + a 12 x 2 + . . . + a 1i x i + . . . + a 1n x n = b1



 .................................... ... ...
ai 1 x1 + ai 2 x2 + . . . + ai i xi + . . . + ai n xn = bi (1)




 .................................... ... ...
a n1 x 1 + a n2 x 2 + . . . + a ni x i + . . . + a nn x n = bn

It can be written by:
AX = b
where A = (ai j )n×n , |A| 6= 0 then system has an unique

solution.
Problem
UPPER TRIANGULAR MATRIX
 
a 11 a 12 . . . a 1n
 0 a . . . a 2n 
22
U =  ..
 
 . .
.. . . . ... 
0 0 . . . a nn
Problem
LOWER TRIANGULAR MATRIX
 
a 11 0 0 0
 a
 21 a 22 . . . 0

L= ..

 . .. . . . .. 
. . 
a n1 a n2 . . . a nn
Method LU factorization
LU FACTORIZATION METHOD
Factorize matrix A = LU , where U is an upper

triangular matrix and L is a lower triangular matrix
where 1s are on the main diagonal.( Doolittle’s
method )
LU FACTORIZATION METHOD
Factorize matrix A = LU , where U is an upper

triangular matrix and L is a lower triangular matrix
where 1s are on the main diagonal.( Doolittle’s
method )
Ax = b ⇔ LU x = b
FORMS OF MATRIX L AND U
 
1 0 0 0
 ` 1 ... 0 
 21
L =  ..

 . .. . . . ... 
. 
`n1 `n2 ... 1
 
u 11 u 12 . . . u 1n
 0 u 22 . . . u 2n 
U =  ..
 
 . .. . . . .. 
. . 
0 0 . . . u nn
THE FORMULA FOR FINDING L AND U

 u1 j = a1 j (1 É j É n)
ai 1


ì 1 = (2 É i É n)





 u 11
iP
−1
 u ij = a ij − ì k u k j (1 < i É j )

 k=1
jP−1
 µ ¶
 1
 ì j = ì k u k j

ai j − (1 < j < i )


ui j k=1
LEADING PRINCIPAL MINORS
DEFINITION
Given a square matrix A , then leading principal
minors D k are determinants of submatrices which are
generated from the upper left k × k corner of A
LEADING PRINCIPAL MINORS
DEFINITION
Given a square matrix A , then leading principal
minors D k are determinants of submatrices which are
generated from the upper left k × k corner of A
EXAMPLE
 
a 11 a 12 a 13
Given A = a21 a22 a23 
 
a 31 a 32 a 33
¯ ¯
¯a
¯ 11 a 12 a 13 ¯
¯ ¯ ¯
¯a 11 a 12 ¯
Then D 1 = |a11 |, D 2 = ¯¯ ¯, D 3 = ¯¯a 21 a 22 a 23 ¯¯
a 21 a 22 ¯ ¯ ¯
¯a 31 a 32 a 33 ¯
THE MAIN DIAGONAL OF MATRIX U

Dk
u kk =
D k−1
where D 1 = a11 (a11 6= 0)
EXAMPLE
Using Doolittle’s method, find matrices L and U in the
factorization of
 
1 2 3
A = 3 −6 9
 
2 1 4
EXAMPLE
 
1.012 −2.132 3.104
Given A = −2.132 4.096 −7.013.
 
3.104 −7.013 0.014
Determine l 32 of matrix L in the factorization A = LU
EXAMPLE
 
1/3 1/2 −1/4
Given A = 1/5 2/3 3/8 
 
2/5 −2/3 5/8
Find U11 +U22 +U33 of matrix U in the factorization
A = LU of matrix A .
Cholesky factorization
DEFINITION
A square matrix is called symmetric if A = A T
DEFINITION
A square matrix is called symmetric if A = A T
DEFINITION
A square matrix is called positive definite if all its
leading principal minors are positive
EXAMPLE
 
1 1 −1
Show that the following matrix A =  1 2 0  is
 
−1 0 4
symmetric and positive definite.
EXAMPLE
 
1 1 −1
Show that the following matrix A =  1 2 0  is
 
−1 0 4
symmetric and positive definite.
THEOREM (CHOLESKY )
If square matrix A is symmetric and positive definite,
then there exist an non-singular matrix B such that
A = B B T , where B is lower triangular matrix.
   
b 11 0 0 b 11 b 21 b 31
T
A = B.B =  b 21 b 22 0  .  0 b 22 b 32 
   
b 31 b 32 b 33 0 0 b 33
   
b 11 0 0 b 11 b 21 b 31
T
A = B.B =  b 21 b 22 0  .  0 b 22 b 32 
   
b 31 b 32 b 33 0 0 b 33
 p a i1

 b 11 = a 11 , b i 1 = , (2 É i É n)



 s b 11
iP
−1


bi i = ai i − b i2k , (1 < i É n)

 k=1
jP
−1
µ ¶
1



 bi j = ai j − b i k b j k , (1 < j < i )


bj j k=1
EXAMPLE
 
1 1 −1
Factorize by Cholesky matrix A =  1 2 0 
 
−1 0 4
EXAMPLE
 
1 1 −1
Factorize by Cholesky matrix A =  1 2 0 
 
−1 0 4
EXAMPLE
 
4 −3 0
Given A =  −3 4 −2 , determine b11 + b22 + b33 in
 
0 −2 4
the factorization A = B B T .
EXAMPLE
Determine all values of α such
 that matrix
 A can be
4 4 α
factorized by Cholesky A =  4 6 2  .
 
α 2 7
Norm of vector and matrix Vector norms
DEFINITION
A vector norm in Rn . is a function, denoted by ||X ||
with the below properties
1
∀X ∈ Rn , ||X || Ê 0, ||X || = 0 ⇔ X = 0
2
∀X ∈ Rn , ∀λ ∈ R, ||λX || = |λ|.||X ||
3
∀X , Y ∈ Rn , ||X + Y || É ||X || + ||Y ||.
DEFINITION
1-norm and ∞-norm of a vector X are defined by:
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
DEFINITION
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
||X ||∞ = max{|x 1 |, |x 2 |, . . . , |x n |} = max |x k |.
k=1,n
DEFINITION
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
||X ||∞ = max{|x 1 |, |x 2 |, . . . , |x n |} = max |x k |.
k=1,n
Norm of vector and matrix Matrix norms
DEFINITION
1-norm and ∞-norm of a matrix A are defined by:
n
|a i j |− sum along columns
P
||A||1 = max
1É j Én i =1
DEFINITION
n
P
||A||1 = max
1É j Én i =1
n
|a i j |− sum along rows
P
||A||∞ = max
1Éi Én j =1
DEFINITION
n
P
||A||1 = max
1É j Én i =1
n
|a i j |− sum along rows
P
||A||∞ = max
1Éi Én j =1
EXAMPLE

2 −1 4
Given A =  5 3 2  . Find 1-norm and ∞-norm of
 
6 −7 3
A.
.
EXAMPLE
 
1/2 1/3 −1/3
Given A = 1/4 3/5 2/3 . Compute number
 
3/4 2/5 −3/7
k = ||A|| × ||A −1 || by 1-norm and ∞-norm.
Iterative method Basic concepts
DEFINITION
Consider a sequence of vectors (X (m) )∞
m=0 where
X (m)
∈ R . We say that (X )m=0 converges to a vector
n (m) ∞
X when m → ∞ if and only if
||X (m) − X || → 0 when m → +∞
Iterative method Strictly diagonally dominant matrix
DEFINITION
The n × n matrix is called strictly diagonally
dominant matrix when
n
X
|a i i | > |a i j |
j =1
j 6=i
Iterative method Strictly diagonally dominant matrix
DEFINITION
The n × n matrix is called strictly diagonally
dominant matrix when
n
X
|a i i | > |a i j |
j =1
j 6=i
EXAMPLE
   
7 2 0 6 4 −3
Consider 2 matrices 3 5 −1 and  4 −2 6 
   
0 5 6 −3 0 1
Iterative method Iterative techniques
Convert AX = b to equivalent system X = T X +C , for

some fixed matrix T and vector C . After an initial
vector X (0) is selected, the sequence of approximate
solution is defined by
X (m) = T X (m−1) +C
CONDITION FOR CONVERGENCE
THEOREM
If ||T || < 1, then the sequence is generated by
X (m) = T X (m−1) +C converges to the exact solution X of
system AX = b
CONDITION FOR CONVERGENCE
THEOREM
If ||T || < 1, then the sequence is generated by
X (m) = T X (m−1) +C converges to the exact solution X of
system AX = b
THEOREM (ERROR ESTIMATION)

(m) ||T ||m
1
Priori error: ||X − X || É .||X (1) − X (0) ||
1 − ||T ||
2
Posteriori error:
||T ||
||X (m) − X || É .||X (m) − X (m−1) ||
1 − ||T ||
THEOREM
Iterative methods are convergent if and only if matrix
A in system AX = b is strictly diagonally dominant
matrix.
Iterative method Jacobi’s method
JACOBI’S METHOD
We will
 split A in form   
a 11 a 12 . . . a 1n a 11 0 . . . 0
 a
 21 a 22 . . . a 2n   0 a 22 . . . 0
  
A= = −

 ... ... ... ...   ... ... ... ... 
a n1 a n2 . . . a nn 0 0 . . . a nn
   
0 0 ... 0 0 −a 12 . . . −a 1n
 −a
21 0 ... 0   0
  0 . . . −a 2n 
 −
 
 ... ... ... ...   ... ... ... ...
 

−a n1 −a n2 . . . 0 0 0 ... 0
= D − L −U .
CONSTRUCTING T AND C
Since A = D − L −U ,
AX = B
⇔(D − L −U )X = B
⇔(D)X = (L +U )X + B
⇔X = D −1 (L +U )X + D −1 B.
We denote T j = D −1 (L +U ) và C j = D −1 B, then the

iterative formula is X (m) = T j X (m−1) +C j , m = 1, 2, . . .
EXPLICIT FORM OF JACOBI’S METHOD
Solving x i from the it h equation (assume ai i 6= 0)

 
n
1
x i(m) − a i j x m−1 + b i  .
 X 
= j
ai i 
j =1

i 6= j
EXAMPLE
Given system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
With X = 3. Using Jacobi’s method, find
 
5
approximate solution X (3) and priori error of X (3) with
∞-norm.
EXAMPLE
Given system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
 
5
approximate solution X (3) and priori error of X (3) with
∞-norm.
 
1.9092
Ans: X (3) = 3.1949
 
EXAMPLE
Given system

 11.5x 1 + 2.5x 2 − 1.25x 3 = 1

0.19x 1 + 4x 2 − 1.2x 3 = 2

 2.4x − 1.3x + 12.3x = 3
1 2 3
 
0
(0)
 
0
approximate solution X (3) and posteriori error of X (3)
with 1-norm.
EXAMPLE
Given system

 5.4x 1 + 0.05x 2 − 1.25x 3 = 1.2

0.08x 1 + 4x 2 − 1.2x 3 = 2.4

 2.4x 1 − 1.3x 2 + 10x 3 = 3.5
 
1
(0)
With X = 1, how many steps are nessecary to
 
1
archieve the priori error less than 10−3 with ∞-norm.
EXAMPLE
½ µ ¶
5.4x 1 + 0.05x 2 = 1.2 (0) 1
Given system With X = .
0.08x 1 + 4x 2 = 2.4 2
Find the approximate solution such that posteriori
error less than 10−2 with 1-norm.
Iterative method Gauss-Seidel’s method
GAUSS-SEIDEL’S METHOD
In Jacobi’s method, the components X (m−1) are

used to compute all the components X i(m) (i=1..n)
of vector X (m)

of vector X (m)
In Gauss-Seidel’s method, for i > 1, the
components X 1(m) ,.... X i(m)
−1
have already been
computed and are expected to be better
approximations to the exact solution X (m) than
X 1(m−1) ,.... X i(m−1)
−1

of vector X (m)
In Gauss-Seidel’s method, for i > 1, the
components X 1(m) ,.... X i(m)
−1
have already been
computed and are expected to be better
approximations to the exact solution X (m) than
X 1(m−1) ,.... X i(m−1)
−1
Then in computing X (m) , we will use these most
recently calculated values
THE EXPLICIT FORM OF GAUSS-SEIDEL’S METHOD
Ã !
iX −1 n
1
x i(m) = − ai j x m a i j x m−1
X
j − j + bi .
ai i j =1 j =i +1
MATRIX FORM
Let T g = (D − L)−1U ,C g = (D − L)−1 B , we get the

iterative method
X (m) = T g X (m−1) +C g , m = 1, 2, . . .
ERROR ESTIMATION
Posteriori
||T g ||
||X (m) − X || É .||X (m) − X (m−1) ||
1 − ||T g ||
ERROR ESTIMATION
Posteriori
||T g ||
||X (m) − X || É .||X (m) − X (m−1) ||
1 − ||T g ||
Priori
(m)
||T g ||m
||X − X || É .||X (1) − X (0) ||
1 − ||T g ||
EXAMPLE
½
15x 1 − 6x 2 = 5
Given the system with
−5x 1 + 8x 2 = 5
X (0) = [0.3, 0.2]T , compute approximatie solution X (3)
by Gauss-Seidel and its priori error with ∞-norm.
EXAMPLE
Given the system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
With X = 3, using G-S method to find approx
 
5
(3)
solution X , its posteriori error with 1-norm.
EXAMPLE
Given the system
½
4x 1 + 0.24x 2 = 8
0.09x 1 + 3x 2 = 9
µ ¶
1
With X (0) = , find approximate solution x n such
2
that kx n − x n−1 k∞ < 10−3

Chap3 SYSTEM LINEAR EQUATION

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chap3 SYSTEM LINEAR EQUATION

Uploaded by

Copyright:

Available Formats

SYSTEM OF LINEAR EQUATIONS

Dau The Phiet

TP. HCM — 2020.

In this chapter, we will solve system of linear

It can be written by:

where A = (ai j )n×n , |A| 6= 0 then system has an unique

UPPER TRIANGULAR MATRIX

LOWER TRIANGULAR MATRIX

Factorize matrix A = LU , where U is an upper

Factorize matrix A = LU , where U is an upper

FORMS OF MATRIX L AND U

THE FORMULA FOR FINDING L AND U

LEADING PRINCIPAL MINORS

LEADING PRINCIPAL MINORS

THE MAIN DIAGONAL OF MATRIX U

X when m → ∞ if and only if

||X (m) − X || → 0 when m → +∞

Convert AX = b to equivalent system X = T X +C , for

CONDITION FOR CONVERGENCE

CONDITION FOR CONVERGENCE

THEOREM (ERROR ESTIMATION)

We denote T j = D −1 (L +U ) và C j = D −1 B, then the

EXPLICIT FORM OF JACOBI’S METHOD

Solving x i from the it h equation (assume ai i 6= 0)

In Jacobi’s method, the components X (m−1) are

In Jacobi’s method, the components X (m−1) are

In Jacobi’s method, the components X (m−1) are

THE EXPLICIT FORM OF GAUSS-SEIDEL’S METHOD

Let T g = (D − L)−1U ,C g = (D − L)−1 B , we get the

You might also like