System Linear Equation

SYSTEM OF LINEAR EQUATIONS
Hoang Hai Ha
HCMUT-OISP
Email: hoanghaiha@hcmut.edu.vn
September 17, 2021
Hoang Hai Ha (BK TPHCM) SYSTEM OF LINEAR EQUATIONS September 17, 2021 1 / 49
Problem
In this chapter, we will solve system of linear equations:



 a 11 x 1 + a 12 x 2 + . . . + a 1i x i + . . . + a 1n x n = b 1



 .................................... ... ...
ai 1 x1 + ai 2 x2 + . . . + ai i xi + . . . + ai n xn = bi (1)




 .................................... ... ...
a n1 x 1 + a n2 x 2 + . . . + a ni x i + . . . + a nn x n = b n

It can be written by:

AX = B
where A = (a i j )n×n , X ∈ Rn is unknown vector. We
consider only when |A| 6= 0 then system has an unique
solution.
Problem
U PPER TRIANGULAR MATRIX
 
a 11 a 12 . . . a 1n
 0 a 22 . . . a 2n 
A=
 
.. .. . . . .. 
 . . . 
0 0 . . . a nn
All entries below main diagonal are zero
Problem
L OWER TRIANGULAR MATRIX
 
a 11 0 0 0

 a 21 a 22 . . . 0 

.. .. . . .
. ..
 
 . . 
a n1 a n2 . . . a nn
All entries above main diagonal are zero
Method LU factorization
LU FACTORIZATION METHOD
Factorize matrix A = LU , where U is an upper triangular

matrix and L is a lower triangular matrix where 1s are on
the main diagonal.( Doolittle’s method )
LU FACTORIZATION METHOD
Factorize matrix A = LU , where U is an upper triangular

matrix and L is a lower triangular matrix where 1s are on
the main diagonal.( Doolittle’s method )
AX = B ⇔ LU x = b
(
LY =B
⇔
UX =Y
F ORMS OF MATRIX L AND U
 
1 0 0 0
 `21 1 ... 0 
L=
 
.. .. ... .. 
 . . . 
`n1 `n2 . . . 1
 
u 11 u 12 . . . u 1n
 0 u 22 . . . u 2n 
U =
 
.. .. . . . .. 
 . . . 
0 0 . . . u nn
T HE FORMULA FOR FINDING L AND U

 u 1 j = a 1 j (1 É j É n)
ai 1


ì 1 = (2 É i É n)





 a 11
iP
−1
 u ij = a ij − ì k u k j (1 < i É j )

 k=1
jP−1
 µ ¶
 1
 ì j = ì k u k j

ai j − (1 < j < i )


ui j k=1
E XAMPLE 2.1
Factorize
 matrix
 A = LU by Doolittle’s method, where
1 1 −1
 31 2
2
4
3 
A = 5 3 8 
2 −2 5
5 3 8
E XAMPLE 2.1
Factorize
 matrix
 A = LU by Doolittle’s
 method, where
1 1 −1
1 0 0
 31 22 34  3
A =  5 3 8  ⇒ L =  5 1 0,

2 −2 5 6
5 3 8 5
l 32 1
E XAMPLE 2.1
Factorize
 matrix
 method,
 where
1 1 −1 1 1 −1
3 2 4
1 0 0 3 2 4
1 2 3  3
A =  5 3 8  ⇒ L =  5 1 0, U = 0 u 22 u 23 
  
2 −2 5 6
5 3 8 5
l 32 1 0 0 u 33
u 22 = a 22 − 53 · 12 = 11
30
E XAMPLE 2.1
Factorize
 matrix
 method,
 where
1 1 −1 1 1 −1
3 2 4
1 0 0 3 2 4
1 2 3  3
A =  5 3 8  ⇒ L =  5 1 0, U = 0 u 22 u 23 
  
2 −2 5 6
5 3 8 5
l 32 1 0 0 u 33
u 22 = a 22 − 53 · 12 = 11
30
u 23 = a 23 − 53 · −1
4
= 21
40
E XAMPLE 2.1
Factorize
 matrix
 method,
 where
1 1 −1 1 1 −1
3 2 4
1 0 0 3 2 4
1 2 3  3
A =  5 3 8  ⇒ L =  5 1 0, U = 0 u 22 u 23 
  
2 −2 5 6
5 3 8 5
l 32 1 0 0 u 33
u 22 = a 22 − 53 · 12 = 11
30
u 23 = a 23 − 53 · −1
4
= 21
1 6 1
¡ ¢40
l 32 = a 32 − u22 5 · 2 =
E XAMPLE 2.1
Factorize
 matrix
 method,
 where
1 1 −1 1 1 −1
3 2 4
1 0 0 3 2 4
1 2 3  3
A =  5 3 8  ⇒ L =  5 1 0, U = 0 u 22 u 23 
  
2 −2 5 6
5 3 8 5
l 32 1 0 0 u 33
u 22 = a 22 − 53 · 12 = 11
30
u 23 = a 23 − 53 · −1
4
= 21
1 6 1
¡ ¢40 −38
l 32 = a 32 − u22 5 · 2 = = 11
.
E XAMPLE 2.1
Factorize
 matrix
 method,
 where
1 1 −1 1 1 −1
3 2 4
1 0 0 3 2 4
1 2 3  3
A =  5 3 8  ⇒ L =  5 1 0, U = 0 u 22 u 23 
  
2 −2 5 6
5 3 8 5
l 32 1 0 0 u 33
u 22 = a 22 − 53 · 12 = 11
30
u 23 = a 23 − 53 · −1
4
= 21
1 6 1
¡ ¢40 −38
l 32 = a 32 − u22 5 · 2 = = 11 .
u 33 = a 33 − 56 · −1
¡ ¢ 241
4
+ l 32 u 23 = 88
L EADING PRINCIPAL MINORS
D EFINITION 2.1
Given a square matrix A, then leading principal minors
D k are determinants of submatrices which are generated
from the upper left k × k corner of A
L EADING PRINCIPAL MINORS
D EFINITION 2.1
Given a square matrix A, then leading principal minors
D k are determinants of submatrices which are generated
from the upper left k × k corner of A
E XAMPLE 2.2
 
a 11 a 12 a 13
Given A = a 21 a 22 a 23 
 
a 31 a 32 a 33
¯ ¯
¯a a a ¯
¯
¯a 11 a 12 ¯
¯ ¯ 11 12 13 ¯
Then D 1 = a 11 , D 2 = ¯¯ ¯, D 3 = ¯a 21 a 22 a 23 ¯¯
¯
a 21 a 22 ¯ ¯ ¯
¯a 31 a 32 a 33 ¯
T HE MAIN DIAGONAL OF MATRIX U

Dk
u kk =
D k−1
where D 1 = a 11 (a 11 6= 0)
E XAMPLE 2.3
Using Doolittle’s method, find matrices L and U in the
factorization of  
1 2 3
A = 3 −6 9 
 
2 1 4.
E XAMPLE 2.4
 
1.012 −2.132 3.104
Given A = −2.132 4.096 −7.013. Determine
 
3.104 −7.013 0.014
u 11 + u 22 + u 33 in the factorization A = LU .
Cholesky factorization
D EFINITION 3.1
A square matrix is called symmetric if A = A T .
D EFINITION 3.1
A square matrix is called symmetric if A = A T .
D EFINITION 3.2
A square matrix is called positive definite if all its leading
principal minors are positive.
E XAMPLE 3.1
 
1 1 −1
Show that the following matrix A =  1 2 0  is
 
−1 0 4
symmetric and positive definite.
E XAMPLE 3.1
 
1 1 −1
Show that the following matrix A =  1 2 0  is
 
−1 0 4
symmetric and positive definite.
T HEOREM 3.1 (C HOLESKY )

If square matrix A is symmetric and positive definite, then
there exist an non-singular matrix B such that A = B B T ,
where B is lower triangular matrix.
F ORMULA OF B
 p ai 1

 b 11 = a 11 , b i 1 = , (2 É i É n)



 s b 11
iP−1


bi i = ai i − b i2k , (1 < i É n)

 k=1
j −1
µ ¶
1


 P
 bi j = ai j − b i k b j k , (1 < j < i )


bj j k=1
F ORMULA OF B
 p ai 1

 b 11 = a 11 , b i 1 = , (2 É i É n)



 s b 11
iP−1


bi i = ai i − b i2k , (1 < i É n)

 k=1
j −1
µ ¶
1


 P
 bi j = ai j − b i k b j k , (1 < j < i )


bj j k=1
M AIN DIAGONAL OF B
s
Dk
b kk = , k = 1..n
D k−1
where D k are leading principal minors of A and
p
D 1 = a 11 .
E XAMPLE 3.2
Factorize
 by Cholesky’s
 method the matrix
1 1 −1
A =  1 2 0 .
 
−1 0 4
E XAMPLE 3.2
Factorize
 by Cholesky’s
 method the matrix
1 1 −1
A =  1 2 0 .
 
−1 0 4
E XAMPLE 3.3
 
4 −3 0
Given A =  −3 4 −2 , determine b 11 + b 22 + b 33 in the
 
0 −2 4
factorization A = B B T .
E XAMPLE 3.4
Determine all values of α such
 that matrix
 A can be
4 4 α
factorized by Cholesky A =  4 6 2  .
 
α 2 7
Norm of vector and matrix Vector norms
V ECTOR NORM
Vector norm is a way to determine distance between two

vectors in Rn .
D EFINITION 4.1
A vector norm in Rn is a function, denoted by ||X || with
the below properties
1 ∀X ∈ Rn , ||X || Ê 0, ||X || = 0 ⇔ X = 0
2 ∀X ∈ Rn , ∀λ ∈ R, ||λX || = |λ|.||X ||
3 ∀X , Y ∈ Rn , ||X + Y || É ||X || + ||Y ||.
V ECTOR NORM
Vector norm is a way to determine distance between two

vectors in Rn .
D EFINITION 4.1
A vector norm in Rn is a function, denoted by ||X || with
the below properties
1 ∀X ∈ Rn , ||X || Ê 0, ||X || = 0 ⇔ X = 0
2 ∀X ∈ Rn , ∀λ ∈ R, ||λX || = |λ|.||X ||
3 ∀X , Y ∈ Rn , ||X + Y || É ||X || + ||Y ||.
D ISTANCE
Distance between two vectors X and Y is kX − Y k.
D EFINITION 4.2
1-norm and ∞-norm of a vector X are defined by:
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
D EFINITION 4.2
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
||X ||∞ = max{|x 1 |, |x 2 |, . . . , |x n |} = max |x k |.
k=1,n
D EFINITION 4.2
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
||X ||∞ = max{|x 1 |, |x 2 |, . . . , |x n |} = max |x k |.
k=1,n
D EFINITION 4.2
n
P
||X ||1 = |x 1 | + |x 2 | + . . . + |x n | = |x k |.
k=1
||X ||∞ = max{|x 1 |, |x 2 |, . . . , |x n |} = max |x k |.
k=1,n
E XAMPLE 4.1
Given X = (1, 2, 3, −5)T . ||X ||1 = 1 + 2 + 3 + 5 = 11 and
||X ||∞ = max{1, 2, 3, 5} = 5
Norm of vector and matrix Matrices norms
M ATRICES NORM
Matrix norm of a square matrix A is a real-valued

function, denoted by kAk, satisfies the properties below:
kAk ≥ 0
M ATRICES NORM

kAk ≥ 0
kAk = 0 ⇔ A is zero matrix.
M ATRICES NORM

kAk ≥ 0
kαAk = |α|kAk, ∀α ∈ R.
M ATRICES NORM

kAk ≥ 0
kA + B k ≤ kAk + kB k
M ATRICES NORM

kAk ≥ 0
kA + B k ≤ kAk + kB k
kAB k ≤ kAkkB k
M ATRIX NORM IS ASSOCIATED TO VECTOR NORM
T HEOREM 1
If k.k is a vector norm on Rn , then
kAk = max kAxk

kxk=1
is a matrix norm.
T HEOREM 4.1
1-norm and ∞-norm of a matrix A are computed by:
n
P
||A||1 = max |a i j |− absolute column sum ⇒ find
1É j Én i =1
max.
T HEOREM 4.1
n
P
1É j Én i =1
max.
n
P
||A||∞ = max |a i j |− absolute row sum ⇒ find max.
1Éi Én j =1
T HEOREM 4.1
n
P
1É j Én i =1
max.
n
P
||A||∞ = max |a i j |− absolute row sum ⇒ find max.
1Éi Én j =1
E XAMPLE 4.2
 
2 −1 4
Given A =  5 3 2  . Find 1-norm and ∞-norm of A.
 
6 −7 3
.
Norm of vector and matrix Condition number
Let system AX = B , if there is a small change in input

values, we will see how solution changes.

E XAMPLE 4.3
µ ¶
1 2
Consider system AX = B with A = and
1 2.01
µ ¶
3
B= .
3.01

E XAMPLE 4.3
µ ¶
1 2
1 2.01
µ ¶ µ ¶
3 1
B= . Solution is X = .
3.01 1

E XAMPLE 4.3
µ ¶
1 2
1 2.01
µ ¶ µ ¶
3 1
B= . Solution is X = .Now we are considering
3.01 1
µ ¶ µ ¶
3 −17
system A Xe = Be with Be = . Solution is Xe = .
3.1 10
⇒ There is a big change in output values for a small
change in input value(vector B ).
C ONDITION NUMBER
Let consider two systems with Ax = B and A xe = Be,
C ONDITION NUMBER

We have kx − xek = kA −1 (B − Be)k ≤ kA −1 kkB − Bek
C ONDITION NUMBER

kB | = kAxk ≤ kAkkxk, so if x 6= 0 and B 6= 0 ⇒ 1
kxk
≤ kAk
kB k
C ONDITION NUMBER

kxk
≤ kAk
kB k
kx − xek kB − Bek
⇒ ≤ kAkkA −1 k
kxk kB k
C ONDITION NUMBER

kxk
≤ kAk
kB k
kxk kB k
⇒ δxe ≤ kAkkA k ×δBe
−1
| {z }
k(A)
If k(A) ≈ 1 then δxe ≤ δBe (We say: system is stable).
C ONDITION NUMBER

kxk
≤ kAk
kB k
kxk kB k
⇒ δxe ≤ kAkkA k ×δBe
−1
| {z }
k(A)
If k(A) ≈ 1 then δxe ≤ δBe (We say: system is stable). If k(A)

is large then a small change in B may cause a big change
in x(system is not stable).
D EFINITION 4.3
Number k = kAkkA −1 k is called condition number of
matrix A.
D EFINITION 4.3
matrix A.
E XAMPLE 4.4
µ ¶µ ¶ µ ¶
1 2 x1 3
Let system = . Compute k ∞ (A) and
1 2.01 x 2 3.01
k 1 (A) for this system.
D EFINITION 4.3
matrix A.
E XAMPLE 4.4
µ ¶µ ¶ µ ¶
1 2 x1 3
1 2.01 x 2 3.01
µ ¶
−1 201 −200
Ans: A = ,
−100 100
D EFINITION 4.3
matrix A.
E XAMPLE 4.4
µ ¶µ ¶ µ ¶
1 2 x1 3
1 2.01 x 2 3.01
µ ¶
−1 201 −200
Ans: A = , k ∞ (A) ≈ 1207.02
−100 100
,
D EFINITION 4.3
matrix A.
E XAMPLE 4.4
µ ¶µ ¶ µ ¶
1 2 x1 3
1 2.01 x 2 3.01
µ ¶
−1 201 −200
Ans: A = , k ∞ (A) ≈ 1207.02
−100 100
,k 1 (A) = 1207.01.
E XAMPLE 4.5
 
1/2 1/3 −1/3
Given A = 1/4 3/5 2/3 . Compute k ∞ (A) and k 1 (A).
 
3/4 2/5 −3/7
Round the result to 4 decimal places.
E XAMPLE 4.5
 
1/2 1/3 −1/3
 
3/4 2/5 −3/7
 
−55 1 133
 6 6 18
A −1 =  85
8
5
8
−175 
24 
−49 7 91
8 8 24
E XAMPLE 4.5
 
1/2 1/3 −1/3
 
3/4 2/5 −3/7
 
−55 1 133
6 6 18
A −1 =  85 5 −175 

8 8 24 
−49 7 91
8 8 24
kAk∞ = 140 , kA k∞ = 445
221 −1
24
⇒ k ∞ ≈ 29.2694(Round up)
E XAMPLE 4.5
 
1/2 1/3 −1/3
 
3/4 2/5 −3/7
 
−55 1 133
6 6 18
A −1 =  85 5 −175 

8 8 24 
−49 7 91
8 8 24
kAk∞ = 140 , kA k∞ = 445
221 −1
24
⇒ k ∞ ≈ 29.2694(Round up)
kAk1 = 32 , kA −1 k1 = 311
12
⇒ k 1 = 38.875
Iterative methods Basic concepts
D EFINITION 5.1
Consider a sequence of vectors (X (m) )∞
m=0 where X
(m)
∈ Rn .
We say that (X (m) )∞
m=0 converges to a vector X when
m → ∞ if and only if
||X (m) − X || → 0 when m → +∞
I TERATIVE METHOD
Convert AX = B to equivalent system X = T X +C , for

some fixed matrix T and vector C . After an initial vector
X (0) is selected, the sequence of approximate solutions is
defined by
X (m) = T X (m−1) +C
S TRICTLY DIAGONALLY DOMINANT MATRIX
D EFINITION 5.2
The n × n matrix is called strictly diagonally dominant
matrix when
n
X
|a i i | > |a i j |
j =1
j 6=i
S TRICTLY DIAGONALLY DOMINANT MATRIX
D EFINITION 5.2
The n × n matrix is called strictly diagonally dominant
matrix when
n
X
|a i i | > |a i j |
j =1
j 6=i
E XAMPLE 5.1
   
7 2 0 6 4 −3
Consider 2 matrices 3 5 −1 and  4 −2 6 
   
0 5 6 −3 0 1
C ONDITION FOR CONVERGENCE
T HEOREM 5.1
If matrix A is strictly diagonally dominant, then the
sequence is generated by X (m) = T X (m−1) +C converges to
the exact solution X of system AX = B
C ONDITION FOR CONVERGENCE
T HEOREM 5.1
If matrix A is strictly diagonally dominant, then the
sequence is generated by X (m) = T X (m−1) +C converges to
the exact solution X of system AX = B
T HEOREM 5.2 (E RROR ESTIMATES )

(m) ||T ||m
1 A priori error: ||X − X || É .||X (1) − X (0) ||
1 − ||T ||
||T ||
2 Posteriori error: ||X (m) − X || É .||X (m) − X (m−1) ||
1 − ||T ||
Iterative methods Jacobi’s method
M ATRIX FORM OF J ACOBI ’ S METHOD
We consider matrix A which is strictly diagonally

dominant(S.D.D).
 split A in form  
We will 
a 11 a 12 . . . a 1n a 11 0 . . . 0
 a
 21 a 22 . . . a 2n   0 a 22 . . . 0 
  
A= = −
 ... ... ... ...   ... ... ... ... 
a n1 a n2 . . . a nn 0 0 . . . a nn
   
0 0 ... 0 0 −a 12 . . . −a 1n
 −a
21 0 ... 0    0
 0 . . . −a 2n 
 −
 
 ... ... ... ...   ... ... ... ... 
 
−a n1 −a n2 . . . 0 0 0 ... 0
= D − L −U .
C ONSTRUCTING T AND C BY J ACOBI ’ S METHOD
Since A = D − L −U ,
Since A = D − L −U ,AX = B ⇔ (D − L −U )X = B
Since A = D − L −U ,AX = B ⇔ (D − L −U )X = B
⇔ (D)X = (L +U )X + B
⇔ X = D −1 (L +U )X + D −1 B.
We denote T = D −1 (L +U ) and C = D −1 B , then the
iterative formula is
X (m) = T X (m−1) +C , m = 1, 2, . . . .
E XPLICIT FORM OF J ACOBI ’ S METHOD
Solving x i from the it h equation ( a i i 6= 0)

 
n
1
x i(m) = − a i j x (m−1) + b i  .
 X 
j
ai i 
j =1

i 6= j
Where x i(m) is the i t h component of m t h vector, x (m−1)

j
is
the j t h component of (m − 1)t h vector.
E XAMPLE 5.2
½
6.3x 1 + 2.1x 2 = 8
Given system and initial vector
1.2x 1 + 4.5x 2 = 9
µ ¶
(0) 0.5
X = . Using Jacobi’s method to determine
1.6
approximate solution X (3) . Estimate its posteriori error
with 1-norm.
µ ¶ µ ¶
0 −1/3 80/63
T= ,C = .
−4/15 0 2
µ ¶
(1) 0.73651
X = ,
1.86667
µ ¶ µ ¶
0 −1/3 80/63
T= ,C = .
−4/15 0 2
µ ¶ µ ¶
(1) 0.73651 (2) 0.64762
X = ,X = ,
1.86667 1.80360
µ ¶ µ ¶
0 −1/3 80/63
T= ,C = .
−4/15 0 2
µ ¶ µ ¶ µ ¶
(1) 0.73651 (2) 0.64762 (3) 0.66864
X = ,X = ,X =
1.86667 1.80360 1.82730
µ ¶ µ ¶
0 −1/3 80/63
T= ,C = .
−4/15 0 2
µ ¶ µ ¶ µ ¶
(1) 0.73651 (2) 0.64762 (3) 0.66864
X = ,X = ,X =
1.86667 1.80360 1.82730
kT k1 = 1/3, kX − X k1 ≈ 0.04472, then ∆ X (3) ≈ 0.0224
(3) (2)
E XAMPLE 5.3
Given system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
With X = 3. Using Jacobi’s method, find approximate
 
5
solution X and priori error of X (3) with ∞-norm.
(3)
E XAMPLE 5.3
Given system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
With X = 3. Using Jacobi’s method, find approximate
 
5
solution X and priori error of X (3) with ∞-norm.
(3)
Matrix A is S.D.D?
Matrix A is 
S.D.D? 
0 −0.06 0.02
Matrix T = −0.03 0 0.05 ,
 
−0.01 0.02 0
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
X (1) = T X (0) +C ,
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
 
1.92
(1) (0) (1)
X = T X +C , ⇒ X = 3.19
 
5.04
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
 
1.92
(1) (0) (1)
X = T X +C , ⇒ X = 3.19
 
5.04
X (2) = T X (1) +C ,
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
 
1.92
(1) (0) (1)
X = T X +C , ⇒ X = 3.19
 
5.04
 
1.9094
X (2) = T X (1) +C , ⇒ X (2) = 3.1944
 
5.0446
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
 
1.92
(1) (0) (1)
X = T X +C , ⇒ X = 3.19
 
5.04
 
1.9094
X (2) = T X (1) +C , ⇒ X (2) = 3.1944
 
5.0446
X (3) = T X (2) +C ,
Matrix A is 
S.D.D?   
0 −0.06 0.02 2
Matrix T = −0.03 0 0.05 , C = 3
   
−0.01 0.02 0 5
 
1.92
(1) (0) (1)
X = T X +C , ⇒ X = 3.19
 
5.04
 
1.9094
X (2) = T X (1) +C , ⇒ X (2) = 3.1944
 
5.0446
 
1.909228
(3) (2) (3)
X = T X +C , ⇒ X = 3.194948
 
5.044794
E STIMATE OF ERROR
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
We need to find ∆ X (3) (A priori) with ∞ norm.
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08= q.
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08= q.
q3
∆ X (3) = kX (1) − X (0) k∞ ,
1−q
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08= q.  
−0.08
q3
∆ X (3) = kX (1) − X (0) k∞ , X (1) − X (0) =  0.19 
 
1−q
0.04
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08= q.  
−0.08
q3
∆ X (3) = kX (1) − X (0) k∞ , X (1) − X (0) =  0.19 
 
1−q
0.04
⇒ kX (1) − X (0) k∞ = 0.19
E STIMATE OF ERROR
     
0 −0.06 0.02 1.92 2
 (1)   (0)  
T = −0.03 0 0.05, X = 3.19, X = 3.

−0.01 0.02 0 5.04 5
kT k∞ = 0.08= q.  
−0.08
q3
∆ X (3) = kX (1) − X (0) k∞ , X (1) − X (0) =  0.19 
 
1−q
0.04
⇒ kX (1) − X (0) k∞ = 0.19 ⇒ ∆ X (3) ≈ 0.0002.
E XAMPLE 5.4
½ µ ¶
5.4x 1 + 0.05x 2 = 1.2 1
Given system With X (0) = .
0.08x 1 + 4x 2 = 2.4 2
Using Jacobi’s method, find approximate solution X (3) and
posteriori error of X (3) with 1-norm.
Iterative methods Gauss-Seidel’s method
G AUSS -S EIDEL’ S METHOD
In Jacobi’s method, the components x i (m−1) are used to

compute all the components x i(m) (i=1..n) of vector X (m)

E XAMPLE 5.5
½ µ ¶
15x 1 − 2x 2 = 6 (0) 0.9
Given system with X = .
3x 1 + 11x 2 = 7 0.8

E XAMPLE 5.5
½ µ ¶
15x 1 − 2x 2 = 6 (0) 0.9
Given system with X = .
3x 1 + 11x 2 = 7 0.8
80
 
µ ¶
 171  0.4678
Exact solution X̄ =  29  ≈
0.5088
57
µ ¶ µ ¶ µ ¶
(1) 0.50667 (2) 0.45212 (3) 0.46642
X = ,X = ,X =
0.39090 0.49818 0.51306
In Gauss-Seidel’s method, for i > 1, the components

x 1(m) ,....x i(m)
−1
have already been computed and are
expected to be better approximations to the exact
solution x (m) than x 1(m−1) ,....x i(m−1)
−1
In Gauss-Seidel’s method, for i > 1, the components

x 1(m) ,....x i(m)
−1
have already been computed and are
expected to be better approximations to the exact
solution x (m) than x 1(m−1) ,....x i(m−1)
−1
Then in computing
(m)
x i , we will use these most recently calculated values
T HE EXPLICIT FORM OF G AUSS -S EIDEL’ S METHOD
Ã !
iX
−1 n
1
x i(m) = − a i j x (m) a i j x (m−1)
X
j
− j
+ bi .
ai i j =1 j =i +1
E XAMPLE 5.6
½
15x 1 − 2x 2 = 6
Given system linear equations . With
3x 1 + 11x 2 = 7
µ ¶
(0) 1.5
X = , find approximate solution X (2) by using G-S
1
method.
S OLUTION
6 + 2x 2(m−1)


 x1(m)
 =
15(m) ⇒
 (m) 7 − 3x 1
x

=
2
11
E XAMPLE 5.6
½
15x 1 − 2x 2 = 6
3x 1 + 11x 2 = 7
µ ¶
(0) 1.5
1
method.
S OLUTION
6 + 2x 2(m−1)


 x1 (m)
 =
15(m) ⇒
 (m) 7 − 3x 1
x

=
2
µ ¶11
0.53333
X (1) = ,
0.49091
E XAMPLE 5.6
½
15x 1 − 2x 2 = 6
3x 1 + 11x 2 = 7
µ ¶
(0) 1.5
1
method.
S OLUTION
6 + 2x 2(m−1)


 x1 (m)
 =
15(m) ⇒
 (m) 7 − 3x 1
x

=
2
µ ¶11 µ ¶
0.53333 0.46545
X (1) = ,X (2) = .
0.49091 0.50942
MATRIX FORM
Let T = (D − L)−1U ,C = (D − L)−1 B , we get the iterative

method
X (m) = T X (m−1) +C , m = 1, 2, . . .
MATRIX FORM

method
X (m) = T X (m−1) +C , m = 1, 2, . . .
Consider
µ example
¶ in previous
µ slide,
¶ we have
15 0 0 2
D −L = and U = ,
3 11 0 0
MATRIX FORM

method
X (m) = T X (m−1) +C , m = 1, 2, . . .
Consider
µ example
¶ in previous
µ slide,
¶ we have
15 0 0 2
D −L = and U = ,then
3 11 0 0
µ ¶
−1 0 2/15
T = (D − L) U = and
0 −2/55
MATRIX FORM

method
X (m) = T X (m−1) +C , m = 1, 2, . . .
Consider µ example
¶ in previous
µ slide,
¶ we have
15 0 0 2
D −L = and U = ,then
3 11 0 0
µ ¶
−1 0 2/15
T = (D − L) U = and
0 −2/55
µ ¶
−1 2/5
C = (D − L) B = ⇒ computation is like in
29/55
Jacobi’s method.
E XAMPLE 5.7
½
15x 1 − 6x 2 = 5
Given the system with
−5x 1 + 8x 2 = 5
X (0) = [0.3, 0.2]T , compute approximate solution X (3) by
Gauss-Seidel and its priori error with ∞-norm.
E XAMPLE 5.7
½
15x 1 − 6x 2 = 5
−5x 1 + 8x 2 = 5
S OLUTION
E XAMPLE 5.7
½
15x 1 − 6x 2 = 5
−5x 1 + 8x 2 = 5
S OLUTION
µ ¶ µ ¶ µ ¶
0.413333 0.686667 0.755
1 X (1) = , X (2) = , X (3) =
0.883333 1.054167 1.096875
S OLUTION
µ ¶
0 0.4
2 T= , ⇒ kT k∞ = 0.4 = q
0 0.25
q3
3 ∆ X (3) ≤ kX (1) − X (0) k∞
1−q
µ ¶
(1) (0) 0.113333
4
X −X = ⇒ kX (1) − X (0) k∞ = 0.683333
0.683333,
S OLUTION
µ ¶
0 0.4
2 T= , ⇒ kT k∞ = 0.4 = q
0 0.25
q3
3 ∆ X (3) ≤ kX (1) − X (0) k∞
1−q
µ ¶
(1) (0) 0.113333
4
X −X = ⇒ kX (1) − X (0) k∞ = 0.683333
0.683333,
5
∆ X (3) ≈ 0.072889
E XAMPLE 5.8
Given the system

 4x 1 + 0.24x 2 − 0.08x 3 = 8

0.09x 1 + 3x 2 − 0.15x 3 = 9

 0.04x − 0.08x + 4x = 20
1 2 3
 
2
(0)
With X = 3, using G-S method to find approx solution
 
5
(3)
X , its posteriori error with 1-norm.
S OLUTION
 
1.9092
(3)
X = 3.1950
 
5.0448
S OLUTION
 
1.9092
(3)
X = 3.1950
 
5.0448
Error : ∆ X (3) ≈ 1.24 × 10−5
E XAMPLE 5.9
Given the system
½
4x 1 + 0.24x 2 = 8
0.09x 1 + 3x 2 = 9
µ ¶
(0) 1
With X = , find approximate solution x n such that
2
kx n − x n−1 k∞ < 10−3 by using G-S method.

System Linear Equation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

System Linear Equation

Uploaded by

Copyright:

Available Formats

SYSTEM OF LINEAR EQUATIONS

September 17, 2021

In this chapter, we will solve system of linear equations:

It can be written by:

U PPER TRIANGULAR MATRIX

All entries below main diagonal are zero

L OWER TRIANGULAR MATRIX

All entries above main diagonal are zero

Factorize matrix A = LU , where U is an upper triangular

Factorize matrix A = LU , where U is an upper triangular

F ORMS OF MATRIX L AND U

T HE FORMULA FOR FINDING L AND U

L EADING PRINCIPAL MINORS

L EADING PRINCIPAL MINORS

T HE MAIN DIAGONAL OF MATRIX U

T HEOREM 3.1 (C HOLESKY )

Vector norm is a way to determine distance between two

Vector norm is a way to determine distance between two

Matrix norm of a square matrix A is a real-valued

Matrix norm of a square matrix A is a real-valued

Matrix norm of a square matrix A is a real-valued

Matrix norm of a square matrix A is a real-valued

Matrix norm of a square matrix A is a real-valued

M ATRIX NORM IS ASSOCIATED TO VECTOR NORM

kAk = max kAxk

Let system AX = B , if there is a small change in input

Let system AX = B , if there is a small change in input

Let system AX = B , if there is a small change in input

Let system AX = B , if there is a small change in input

Let consider two systems with Ax = B and A xe = Be,

Let consider two systems with Ax = B and A xe = Be,

Let consider two systems with Ax = B and A xe = Be,

Let consider two systems with Ax = B and A xe = Be,

Let consider two systems with Ax = B and A xe = Be,

If k(A) ≈ 1 then δxe ≤ δBe (We say: system is stable).

Let consider two systems with Ax = B and A xe = Be,

If k(A) ≈ 1 then δxe ≤ δBe (We say: system is stable). If k(A)

||X (m) − X || → 0 when m → +∞

Convert AX = B to equivalent system X = T X +C , for

S TRICTLY DIAGONALLY DOMINANT MATRIX

S TRICTLY DIAGONALLY DOMINANT MATRIX

C ONDITION FOR CONVERGENCE

C ONDITION FOR CONVERGENCE

T HEOREM 5.2 (E RROR ESTIMATES )

M ATRIX FORM OF J ACOBI ’ S METHOD

We consider matrix A which is strictly diagonally

C ONSTRUCTING T AND C BY J ACOBI ’ S METHOD

C ONSTRUCTING T AND C BY J ACOBI ’ S METHOD

C ONSTRUCTING T AND C BY J ACOBI ’ S METHOD

E XPLICIT FORM OF J ACOBI ’ S METHOD

Solving x i from the it h equation ( a i i 6= 0)

Where x i(m) is the i t h component of m t h vector, x (m−1)

G AUSS -S EIDEL’ S METHOD

In Jacobi’s method, the components x i (m−1) are used to

G AUSS -S EIDEL’ S METHOD

In Jacobi’s method, the components x i (m−1) are used to

G AUSS -S EIDEL’ S METHOD

In Jacobi’s method, the components x i (m−1) are used to

In Gauss-Seidel’s method, for i > 1, the components

In Gauss-Seidel’s method, for i > 1, the components

T HE EXPLICIT FORM OF G AUSS -S EIDEL’ S METHOD

Let T = (D − L)−1U ,C = (D − L)−1 B , we get the iterative