PART THREE:

LINEAR SYSTEMS OF EQUATIONS

Linear Algebraic
Equation Solvers

Gauss LU Factorization Iterative

Elimination & Matrix Inversion Methods

Cramer’s rule
LU Decomposition Gauss-Seidel
Naive Gauss
Matrix Inverse Newton Raphson
(Pivoting)

1
 Linear Algebraic
Equation Solvers

Gauss LU Factorization Iterative

Elimination & Matrix Inversion Methods

Cramer’s rule
LU Decomposition Gauss-Seidel
Naive Gauss
Matrix Inverse Newton Raphson
(Pivoting)

LU Decomposition and
Matrix Inversion
We have countered with Naive-Gauss elimination
to solve the algebraic system. It involve forward
elimination and back-substitution.
Time-consuming of forward elimination in Naive-
Gauss elimination will be replaced by
decomposition technique in LU decomposition.
LU decomposition may also help to calculate the
matrix inverse.

2
LU- Factorization & Matrix inversion

 Gauss elimination designed to solve [A] {x} ={b}

 It becomes inefficient when solving equations with same
coefficients [A], but with different right hand side constants {b}
 LU Factorization separate the time-consuming elimination of
Matrix [A] from the manipulations of the right-hand-side {b}.
 LU Factorization is very useful when the vector of variables {x} is
estimated for different parameter vectors {b} since the forward
elimination process is not performed on {b}

LU Decomposition
Given the system of linear algebraic
equation in matrix form.
AX   B
In LU decomposition, [A] will be
decomposed into upper triangle form, [U]
and lower triangle form , [L] to give the
values of {B}.
 1 0 0  a11 a12 a13 
L   f 21 1 0 U    0 a22 a23 
 f 31 f 32 1  0 0 a33  

3
 2 phases on solving equation using LU
decomposition:
1) LU decomposition step
2) Substitution step
Steps for using the LU decomposition method:
1. Arrange the linear algebraic equations
given into matrix form:
AX   B
2. Decomposed [A] into lower and upper
triangular matrices, [L] and [U].
L U X   B 

3. [L] is used to generate an intermediate

vector, {D} using:
L { D }  B 
4. {D} is then used to calculate {X} using:

U { X }  D 

(As shown in example 10.2)

4
LU Factorization
[ A]{ x }  {b}  0 Assume there is a
forawrd elimination lower diagonal matrix :
produce upper triangular system:
1 0 0
 u11 u12 u13   x1   d1   
[ L]   l 21 1 0
    
 0 u 22 u 23   x2    d 2   l 31 l32 1 
 0 0 u33   x3   d 3 
[U ]{ x }  {d }  0 Such that
[ L]{[U ]{ x }  {d }}  [ A]{ x }  {b}

[ L][U ]  [ A]
[ L]{d }  {b}

LU Decomposition [ A]{ x }  {b}

Factorization
[U ] [ L]
i 1
d i  bi   l ij b j for i = 1,2,...,n
j 1 [ L] { d }  { b }
forward
substitution
xn  d n / ann {d }
n
di  u ij xj
[U ] { x }  {d }
j  i 1
xi 
uii
for i = n - 1,n - 2,...,1 backward
{ x} substitution

5
 How to decompose [A] into [L] and [U] ?

1. We are going to have [A] in the form of:

 a11 a12 a13 
A  a21 a22 a23 
 a31 a32 a33 
a21
2. Multiply 1st row by a and use 2nd row to
11
substrate the result to eliminate a21.
a21
f 21 
a11

a31
1st
3. Multiply row by and use 3rd row to
a11
substrate the result to eliminate a31.
a31
f 31 
a11


a32
4. Multiply modified 2nd
row by and use 3rd row

a22
to substrate the result to eliminate a’32.

a32
f 32 
a22

6
5. After the elimination we will get the matrix [U]:
 a11 a12 a13 
U    0 a22 a23 
 0 0 a33  

While the [L]:

 1 0 0
L   f 21 1 0
 f 31 f 32 1

Lower Triangular Matrix [L]

[ A]{ x}  {B}  0 a21

f 21 
a11 a12 a13   x1   b1  a11
a    
a a23   x2   b2  a31 
a32
 21 22  f 31  f 32 
a31 a32 a33   x3  b3  a11 a22

a11 a12 a13 

[U ]   0 a22/ / 
a23 1 0 0
  [ L]   f12 1 0
 0 0 //
a33   
 f 31 f 32 1
[ L][U ]  [ A]

7
 Example
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
 25 5 1
 64 8 1
 
144 12 1

 25 5 1 
 Row1
Row 2     (64)   0  4.8  1.56

 25 
144 12 1 
25 5 1 
 Row1  
Row 3     (144)   0  4.8  1.56 
 25 
 0  16.8  4.76

 25 5 1 
 0 4.8 1.56
 
 0 16.8 4.76

 25 5 1 
 Row 2   
Row 3     ( 16.8)   0 4.8 1.56 
  4.8 
 0 0 0.7 

 25 5 1 
U    0 4.8 1.56
 0 0 0.7 

8
 Finding the [L] matrix
Using the multipliers used during the Forward Elimination Procedure

a 21 64
1 0 0  21    2.56
From the first   a11 25
step of forward   21 1 0
a 31 144
elimination   31  32 1   31    5.76
a11 25

From the second  25 5 1 

step of forward  0 4.8 1.56  a 32 16.8
   32    3.5
elimination a 22 4.8
 0 16.8 4.76 

 1 0 0
 L   2.56 1 0 
 5.76 3.5 1 

Does LU    A ?

 1 0 0 25 5 1 
LU   2.56 1 0  0  4.8  1.56 
  
5.76 3.5 1  0 0 0.7 

9
LU factorization with Gauss Elimination: Example 10.1

 3 0.1 0.2 
 A   0.1 7 0.3 
 0.3 0.2 10 
3 0.1 0.2 
After forward elimination we had U   0 7.00333 0.293333 

 
 0 0 10.0120 
The factors employed to obtain the upper triangular matrix can be
assembled into lower triangular matrix.

0.1 0.3 0.19

f 21  , f 31  , f 32 
3 3 7.00333

Thus the lower triangular matrix is:

 1 0 0
 0 
 L   0.033333 1
 0.100000 0.0271300 1 

Therefore:
 1 0 0  3 0.1 0.2 

[ A]  [ L][U ]   0.033333 1 0   0 7.00333 0.293333 
 
 0.100000 0.0271300 1   0 0 10.0120 

10
Substitution steps: Example 10.2
 3 0.1 0.2   x1   7.85 
 0.1    
 7 0.3   x2    19.3 
 0.3 0.2 10   x3   71.4 
decomposition
3 0.1 0.2 
U    0 7.00333 0.293333

 0 0 10.0120 
 1 0 0
 0 
 L   0.033333 1
 0.100000 0.0271300 1 

Forward substitution  Ld   b

 1 0 0   d1   7.85 
 0.033333 1 0   d    19.3 
  2  
 0.100000 0.0271300 1   d 3   71.4 
  

d1  7.85, d 2  19.3  0.033333d1  19.5617

d 3  71.4  0.100000d1  0.0271300d 2  70.0843
backward substitution U  x  d 
3 0.1 0.2   x1   7.85 
 0 7.00333 0.293333   x    19.5617   x1   3 
   2      
 0 0 10.0120   x3   70.0843 
    x2    2.5 
 x   7.00003 
 3  

11
 Matrix Inverse
Matrix inverse [A]-1 can be calculated numerically
using LU decomposition.
Steps for using the LU decomposition method
for matrix inverse [A]-1:
1. Decompose [A] into lower and
upper triangular matrices, [L] and [U].
2. [L] is used to generate an intermediate
vector, {d} using:

 L { d }  b 
(23)

3. Use different value of {b} for each column of

matrix inverse:

1  0 0
     
b 1  0 b 2  1  b 3  0
0 0 1 
     

(1st Column) (2nd Column) (3rd Column)

4. {d} is used to calculate {x} using:
U { x }  d 

(24)

12
 Matrix Inverse
 A A   A  A   I 
1 1

Column by column, solve for aji

1 0
 A a1i1   0
   A a2i1  1 
 
 
1st column  0  2nd column  0 
of [A]-1
 
of [A]-1

0
 A1  a1i1  a2i1  a3i1 
 A a3i1   0
 
3rd column 1
of [A]-1
 
(25)

Example 10.3
 3 0.1 0.2 
 A   0.1 7 0.3
 0.3 0.2 10 

3 0.1 0.2   1 0 0
U    0 7.00333 0.293333
 
 L   0.033333 1 0 
 0 0 10.0120   0.100000 0.0271300 1 

(26)

13
 1   1  U  {a1i1 }  d   0.33249 
 L {d }1  0 {d }1   0.0333
  1
1  
{a1i }   0.00518
0  0.1009   0.1008 
     

 0 U  {a2 i }  d 2  0.04944 

1
 0
 L {d }2  1 {d }2   1   
{a2i1 }   0.142903 
 0  0.02713   0.00271 
     

 0  U  {a3 i }  d 3  0.006798 
1
0
 L {d }3  0 {d }3   0   
{a3i1 }   0.004183 
1 1.0000   0.099880 
     

(27)

 3 0.1 0.2 
 A   0.1 7 0.3
 0.3 0.2 10 

a 
1
1i a  1
2i a 
1
3i

 0.33249 0.004944 0.006798 

 A   0.00518 0.142903 0.004183 
1

 0.1008 0.00271 0.099880 

 A   A  I
1

(28)

14