CE2407 Engineering and Uncertainty Analysis

Lecture Notes 3
LU Decomposition
(Matrix Factorization)
Kevin Kuang Sze Chiang
Department of Civil and Environmental Engineering
Room E2-04-11, Tel: 6516 4683, Email: ceeksck@nus.edu.sg

Lecture 3 PG1
Overview of LU Decomposition and Strategy

4 3  1  x1   5.7 
 5 1 3   x    2 .8  A x= b
   2  
1  1 2   x3   3.8 
A x b
4 3 1  x1    5.7 
0  2.75  x   9.925 
 4 .25   2
=
 
0 0  0.4545 x3   1.0909

Gaussian Elimination

 1 0 0 4 3 1 
A= L U =  1 . 25 1 0   0  2 . 75 4 . 25 

 0 . 25 0 . 634 1   0 0  0 . 46 
L U
Lecture 3 PG9
Lecture 3: Matrix Factorization

A another method to solve simultaneous equations is through matrix

factorization or triangular decomposition method

A square matrix A can be rewritten in the form of a product of a lower-triangular

matrix L and an upper-triangular matrix U, where A=LU.

1 2 3 
 
If A = 3 5 8 
4 9 10
l11 0 0 u11 u12 u13 
l 0  0 u u23 
A =  21 l22  22
l31 l32 l33   0 0 u33 

L U
Lecture 3 PG10
Matrix Factorization

 l11 0 0 u11 u12 u13 

l l 22 0  0 u 22 u 23 
A=  21 
 l31 l32 l33   0 0 u 33 

L U

 l11u11 l11u12 l11u13 

l u l 21u12  l 22 u 22 l 21u13  l 22 u 23 
=  21 11 
l31u11 l31u12  l32 u 22 l31u13  l32 u 23  l33 u 33 

Note that, for L and U, the elements occur in the major diagonal in each
case. These are related in the product and whatever values we choose
to put for u11, u22, u33… then the corresponding values of l11,l22,l33…will be
determined and vice versa since the values of each element in the
matrix is given in the question. i.e. e.g. l11u11=1, l11u12=2 etc.

Lecture 3 PG11
Matrix Factorization Uii=1 (Crout vs Doolittle)

For convenience, we put u11, u22, u33…=1

 l11 l11u12 l11 u13 

 
Then A = LU = l 21 l 21u 12  l 22 l 21u13  l 22 u 23 
l31 l31u12  l32 l31u13  l32 u 23  l33 

1 2 3 
 3 5 8 
Given A= 
 4 9 10 

l11  1; l11 u 12  2  u 12  2 ; l11 u 13  3  u 13  3

l 21  3; l 22   1; u 23  1 So, here we have worked all the
l 31  4 ; l 32  1; l 33   3 values of the elements in L and U.

Lecture 3 PG12
Matrix Factorization Uii=1 (Crout vs Doolittle)

 l11 l11u12 l11 u13 

 
Then A = LU = l 21 l 21u 12  l 22 l 21u13  l 22 u 23 
l31 l31u12  l32 l31u13  l32 u 23  l33 

 2  First write l11,l21,l31 then u12, u13

1 4

 3

 5
Etc.


Lecture 3 PG13
Matrix Factorization Uii=1 (Crout vs Doolittle)

Substitute all these values back into the upper and lower triangular matrices
and obtain

1 0 0  1 2 3
A= L U =
 3  1 0   0 1 1
  
4 1  3 0 0 1
L U
Thus we have factorized matrix A into a lower triangular matrix, L, and
an upper triangular matrix, U.
Theorem: If Gaussian Elimination can be performed on a linear system
Ax=b without row interchanges, then the matrix A can be factored into the
product of lower-triangular matrix L and upper-triangular matrix U, A = LU.
So, how do we make use of matrix factorization to help us solve a system of
linear equations?

Lecture 3 PG14
Matrix Factorization- Worked Example 1 (3x3) Uii=1 (Crout)

x1  2 x2  3 x3  16
S3 3 x1  5 x2  8 x3  43
4 x1  9 x2  10 x3  57

1 2 3   x1  16 
3 5 8  x  =  43
   2  
4 9 10  x3  57 
A x b
1 2 3  1 0 0  1 2 3
3
A= LU =  5 8  = 3  1 0  0 1 1 as before
 4 9 10  4 1  3 0 0 1

1 0 0  1 2 3  x1  16 
 3  1 0   0 1 1  x  =  43
    2  
4 1  3 0 0 1  x3  57 
Lecture 3 PG15
Matrix Factorization- Worked Example 1 Uii=1 (Crout)

1 0 0  1 2 3  x1  16 
 3  1 0   0 1 1  x  =  43
    2  
4 1  3 0 0 1  x3  57 

Y
1 0 0  y1  16  y1  16
   y  =  43
Solving for Y,
 3 1 0   2   y2  5
4 1  3  y3  57  y3  4

1 2 3  x1  16 x1  2
0 1 1  x    5 
Solving for X,
  2    x2  1
0 0 1  x3   4  x3  4
#

Lecture 3 PG16
Matrix Factorization- Worked Example 2 (3x3) Uii=1 (Crout)

x1  3 x2  2 x3  19
S3 2 x1  x2  x3  13
4 x1  2 x2  3 x3  31

1 3 2  x1  19
2 1 1   x   13 Ax=b
  2   
4 2 3  x3  31
 l11 0 0 1 u12 u13 
l l 22 0  0 1 u 23  Let uii=1
A=LU=  21 
 l31 l32 l33   0 0 1 

l11 l11u12 l11u13  1 3 2 

  = 2 1 1 
A = LU = l21 l21u21  l22 l21u13  l22u23   
l31 l31u12  l32 l31u13  l32u23  l33  4 2 3

Lecture 3 PG17
Matrix Factorization- Worked Example 2 Uii=1 (Crout)

Working out all the values,

1 0 0 1 3 2 
 2  5 0   0 1 3 / 5
A=LU=   
4  10 1 0 0 1 

1 0 0 1 3 2   x1  19
Given:  2  5 0   0 1 3 / 5    
  x2   13
Ax=b
 
4  10 1 0 0 1   x3  31

1 0 0  y1  19  y1  19
2  5 0  y   13 y    5 
   2    2  
4  10 1  y3  31  y3   5 

Lecture 3 PG18
Matrix Factorization- Worked Example 2 Uii=1 (Crout)

1 3 2   x1  19
 0 1 3 / 5 x    5 
   2  
0 0 1   x3   5 

x1  3
x2  2
x3  5 #

Lecture 3 PG19
Matrix Factorization- Worked Example ( 4x4) Uii=1 (Crout)

x1  2 x2  x3  3 x4  9
2 x1  x2  3 x3  2 x4  23
S4
3 x1  3 x2  x3  x4  5
4 x1  5 x2  2 x3  2 x4  2

1 2  1 3  x1   9 
2  1 3
 2  x2   23 
 Ax=b
3 3 1 
1 3  x  5
    
4 5  2 2   x4    2 

1 2  1 3 l11 0 0 0 1 u12 u13 u14 

l 0 1 u24 
2  1 3 2  l22 0 0   u 23
A=LU=  =
21

3 3 1 1 l31 l32 l33 0 0 0 1 u34 

     
4 5  2 2 l41 l42 l43 l44  0 0 0 1

Lecture 3 PG20
Matrix Factorization- Worked Example 3 ( 4x4) Uii=1 (Crout)

l11 l11u12 l11u13 l11u14 

l 
A=  21 l21u12  l22 l21u13  l22u23 u14l21  l22u24 
l31 l31u12  l32 l31u13  l32u23  l33 l31u14  l32u24  l33u34 
 
l41 l41u12  l42 l41u13  l42u23  l43 l41u14  l42u24  l43u34  l44 

1 0 0 0  1 2 1 3
2  5 0 0  0 1  1 4 / 5 
LU=  
3  3 1 0  0 0 1  28 / 5
  
 4  3  1  66 / 5  0 0 0 1 

1 0 0 0  1 2 1 3   x1   9 
2  5 0 0  0 1  1 4 / 5   x2   23 
     Ax=b
3  3 1 0  0 0 1  28 / 5  x3   5 
     
 4  3  1  66 / 5 0 0 0 1   x4    2 
Lecture 3 PG21
Matrix Factorization- Worked Example 3 ( 4x4) Uii=1 (Crout)

1 0 0 0  1 2 1 3   x1   9 
2  5 0 0  0 1  1 4 / 5   x2   23 
    
3  3 1 0  0 0 1  28 / 5  x3   5 
     
 4  3  1  66 / 5  0 0 0 1   x4    2 

1 0 0 0   y1   9   y1   9 
2  5 0 0   y   23   y   1 
   2     2   
3  3 1 0   y3   5   y3   25
        
 4  3  1  66 / 5  y 4   2 y
 4  5 
1 2 1 3   x1   y1   9   x1   1 
0  x    2
 1  1 4 / 5  x   y   1 
 2   
 2   2    
0 0 1  28 / 5  x3   y3   25  x3   3 
           
0 0 0 1   x4   y 4   5   x4   5  #
Lecture 3 PG22
Matrix Factorization- Using a previous matrix
Uii=1 (Crout)
used in GE lectures

Now, let’s take look at a same 3x3 matrix we have encountered

and solved using Gaussian Elimination.
4 3  1  x1   5.7
5 1 3   x    2.8 
Given   2   
1  1 2   x3   3.8 
Here, let’s decompose this matrix using LU decomposition Crout’s approach (i.e. Uii=1)

4 0 0  1 0.75  0.25   x1    5.7 

 5  2.75 
 0  0 1  1.5454   x2    2.8 
1  1.75  0.4545  0 0 1   x3   3.8 

Leave it to you to work this out based on Uii=1 as in previous examples.

The full worked solution is provided in the last 3 pages of this set of notes

Lecture 3 PG23
Matrix Factorization- INSTEAD OF Uii=1, USE lii=1 (Dolittle1)

Here, let’s decompose this matrix using LU decomposition Dolittle’s approach (i.e. lii=1)

4 3  1  x1   5.7 
Given 5 1 3   x    2.8 
  2   
1  1 2   x3   3.8 

FACTORIZATION of matrix A (with Doolittle Decomposition, lii=1):

Doolittle METHOD 1: MATRIX EXPANSION (AS BEFORE)

Doolittle METHOD 2: EXPLOIT Gaussian Elimination RESULTS (SHORT CUT)

Answers: #

Lecture 3 PG24
METHOD 1: MATRIX EXPANSION (AS BEFORE)
Matrix Factorization- INSTEAD OF Uii=1, USE lii=1 (Dolittle1)

4 3  1 Using the same matrix expansion

A  5 1 3  (manual method) to obtain the elements
one at a time….
1  1 2 

L U

1 0 0 u11 u12 u13  lii=1

u11=4, u12=3 u13=-1

A  l21 1 0  0 u22 u 23  4l21  5 l31 

a31 1
  0.25
u11 4
l31 l32 1  0 0 u33 
u22=-2.75, u23=4.25
 1 0 0  4 3 1   1  0.75
l32   0.6363
A  1.25 1 0 0  2.75 4.25   2.75
u33=-0.4545
0.25 0.6363 1 0 0  0.4545
a21 5
l21    1.25
u11 4

Lecture 3 PG25
Summary of GE and LU (Crout) and LU (Dolittle) methods

4 3  1  x1   5.7 
Given 5 1 3   x2    2.8 
1  1 2   x3   3.8 

4 3 x1   5.7  Gaussian

1 
x  
0  2.75 4.25   Elimination via
   
2 = 9.925  Echelon/
0 0 0.4545
x3 1.0909 Reduced
Form
4 0 0  1 0.75  0.25   x1   5.7 LU Decomposition via
5  2.75 0  0 1  1.5454  x    2.8  Crout (using uii=1 and
   2    by matric expansion)-
long-winded
1  1.75  0.4545 0 0    
1   x3   3.8  
(1) LU Decomposition via
 1 0 0  4 3  1   x1   5.7 Dolittle (using lii=1 and
1.25 1 0  0  2.75 4 . 25   x    2.8  by matric expansion)-
long-winded
   2   
0.25 0.6363 1 0 0  0.4545  x3   3.8  Is there a
short-cut ?
Lecture 3 PG26
METHOD 2: EXPLOIT GE RESULTS (SHORT CUT) lii=1

L U

l11 0 0  u11 u12 u13 

A  l21 l22 0   0 u22 u 23 
l31 l32 l33   0 0 u33 

In this method, make use of the Gaussian Elimination results obtained earlier.

(a) elements of L are actually the coefficients used before.

(b) elements of U are the echelon form (upper triangular matrix) of the matrix A,

ALL THESE INFORMATION WERE ALREADY AVAILABLE FROM THE GE PART.

So we can easily and quickly (short cut) factorize A using into L U

Lecture 3 PG27
METHOD 2: EXPLOIT GE RESULTS (SHORT CUT) lii=1

4 3  1 4 3 1  5.7 
 
A  5 1 3 

A  0  2.75 4.25 9.925 
1  1 2  0 0  0.4545  1.0909
From Gaussian Elimination, we have determined the echelon form of A
We want to express A as LU
L U

l11 0 0  u11 u12 u13 

A  l21 l22 0   0 u 22 u 23 
l31 l32 l33   0 0 u33 

1 0 0  4 3 1 
??? A  l21 1 0 0  2.75 4.25 
l31 l32 1 0 0  0.4545
Lecture 3 PG28
METHOD 2: EXPLOIT GE RESULTS (SHORT CUT)
RECALL from Gaussian Elimination Method

4 3  1  5 . 7  E1
 
SO….
Aˆ   A , b    5 1 3 2 .8  E 2
 1 1 2 3 . 8  E 3
Notice the choice of constant used
What can you say?

4 3 1  5.7  E1
5  
E2  E1  E2  0  2.75 4.25 9.925 E2
4
1
E3  E1  E3 0 1.75 2.25 5.225 E3
4

4 3 1  5.7  E1
 
-1.75 0  2.75 4.25 9.925  E2
E3  E 2  E3 0 0  0.4545  1.0909 E3
-2 .75

Lecture 3 PG29
 Matrix Factorization- [A] factorized as below….
Those ratios from GE are used here to form L (shor cut)
5/4=1.25
1/4 = 0.25
1.75/2,75= 0.6363
L U
1 0 0  4 3  1   x1   5.7
   
A  l21 1 0 0  2.75 4.25   x2    2.8 
l31 l32 1 0 0  0.4545  x3   3.8 
Do a check!
They are the  1 0 0  4 3  1   x1   5.7
1.25 1 0  0  2.75 4 .25   x    2.8 
same    2   
DoLittle 0.25 0.6363 1 0 0  0.4545  x3   3.8 
Method 1!
The intermediate calculation

 1 0 0  y1   5.7 y1   5 .7
1.25   y    2.8 
 1 0  2    y2  9 . 925
0.25 0.6363 1  y3   3.8  y3   1 . 0909
Lecture 3 PG30
Matrix Factorization- via Doolittle Decomposition lii=1

4 3  1   x1    5.7 
0  2.75 4 .25   x    9.925 
  2   
0 0  0.4545  x3   1.0909

x1   0 .9
x2  0 .1
x3  2 .4

Lecture 3 PG31
Summary of GE and LU (Crout) and LU (Dolittle) methods

4 3  1  x1   5.7 
Given 5 1 3   x2    2.8 
1  1 2   x3   3.8 

4 3 x1   5.7  Gaussian

1 
x  
0  2.75 4.25   Elimination via
   
2 = 9.925  Echelon/
0 0 0.4545
x3 1.0909 Reduced
Form
4 0 0  1 0.75  0.25   x1   5.7 LU Decomposition via
5  2.75 0  0 1  1.5454  x    2.8  Crout (using uii=1 and
   2    by matric expansion)-
long-winded
1  1.75  0.4545 0 0    
1   x3   3.8  
(1) LU Decomposition via
 1 0 0  4 3  1   x1   5.7 Dolittle (using lii=1 and
1.25 1 0  0  2.75 4 . 25   x    2.8  by matric expansion)-
long-winded
   2   
0.25 0.6363 1 0 0  0.4545  x3   3.8  (2) LU Decomposition
via Dolittle (using GE
results) Short-cut

Lecture 3 PG32
Lecture 3 PG33
Solution for Exercise (uii=1)- Matrix Factorization with Crout
Decomposition

4 3  1  x1    5 . 7 
5 3   x 2    2 . 8 
Given  1
 1 1 2   x 3   3 . 8 

We want to express A = L U  l11 0 0 1 u12 u13 

l l 22 0  0 1 u 23 
 21  Let uii=1
 l31 l32 l33   0 0 1 

l11 l11u12 l11u13  4 3  1

 
Then A = LU = l21 l21u12  l22 l21u13  l22u23   5 1 3 
l31 l31u12  l32 l31u13  l32u23  l33  1  1 2 

l11  4; l11u12  3  u12  0.75; l11u13  1, u13  0.25

l21  5; l21u12  l22  1  l22  2.75 ; l21u13  l22 u23  3, u23  1.5454
l31  1; l31u12  l32   1 l32  1.75 ; l31u13  l32u23  l33  2 ,  l33  0.4545
Lecture 3 PG34
Solution for Exercise (uii=1)- Matrix Factorization with Crout
Decomposition
L U
4 0 0  1 0.75  0.25   1 0 0  4 3 1 
5  2.75  0 1  1 . 5454  1.25 1  
0 0  2.75 4.25 
A =
 0   

0.25 0.6363 1 0 0  0.4545
1  1.75  0.4545  0 0 1 
Compared to when lii=1
4 3  1  x1    5 . 7 
5 3   x 2    2 . 8 
given  1
 1 1 2   x 3   3 . 8 

4 0 0  1 0 .75  0 .25   x1    5 .7 
5  0
  2 .75 0   1  1 .5454   x 2    2 .8 
 1  1 .75  0 .4545   0 0 1   x3   3 .8 

4 0 0   y1   5.7 y1   1 . 425
5  2 .75 0   y    2.8  y2   3 . 609
   2  
 1  1 .75  0 .4545   y3   3.8  y3  2 .4
Lecture 3 PG35
Solution for Exercise (uii=1)- Matrix Factorization with Crout
Decomposition

1 0 .75  0 .25   x1    1 .425 

0 1  1 .5454   x 2     3 .609 

 0 0 1   x3   2 .4 

x1   1 . 425  0 . 75 ( 0 . 1 )  0 . 25 ( 2 . 4 )   0 . 9
x2   3 . 609  1 . 5454 ( 2 . 4 )  0 . 1
x3  2 .4

We arrive at the same values for x as that obtained via Doolittle

Decomposition. Compare

Lecture 3 PG36