LU decomposition Method

Gauss elimination becomes inefficient when solving equations with the same coefficients for [A]
but with different bs.
LU decomposition separates the time consuming elimination of [A] form the manipulation of {b} .
Hence, the decomposed [A] could be used with several {b} s in an efficient manner.
LU decomposition is based on the fact that any square matrix [A] can be written as a product of
two matrices as:
[A]=[L][U]
Where [L] is a lower triangular matrix and [U] is an upper triangular matrix.

Crouts method
To illustrate the Crouts method for LU decomposition, consider the following 33 matrix:

a11
a
21
a31

a12
a 22
a32

a13 l11 0
a23 = l 21 l 22
a33 l31 l32

0
0
l33

1 u12
0 1

0 0

u13
u 23
1

Hence

a11
a
21
a31

a12
a 22
a32

a13 l11
a23 = l21
a33 l31

(l11u12 )
(l21u12 + l22 )
(l31u12 + l32 )

(l11u13 )

(l21u13 + l22u23 )
(l31u13 + l32u23 + l33 )

Elements of the matrices [L] and [U] can be found by equating the two above matrices:

l11 = a11 ; l 21 = a 21 ; l31 = a31

a12 a12
=
l11u12 = a12 , hence u12 =
l
a11
11

l 21u12 + l 22 = a 22 , hence l 22 = a 22 l 21u12

l31u12 + l32 = a32 , hence l32 = a32 l31u12

l11u13 = a13 , hence u13 = a13 = a13

l11 a11

l u + l u = a , hence u = a 23 l 21u13
23
23
21 13 22 23
l 22

l31u13 + l32 u 23 + l33 = a33 , hence l33 = a33 l31u13 l32 u 23

For a general nn matrix, the following expressions can be applied to find the LU decomposition
of a matrix [A]:
j 1

lij = aij lik u kj ; ij; i=1,2,,n

k =1

i 1

aij lik u kj
k =1
u ij =
; i<j; j=2,3,,n
lii

and

u ii = 1 ; i=1,2,,n

Note: It is better to follow a certain order when computing the terms of the [L] and [U] matrices.
This order is: li1,u1j; li2,u2j; ; li,n-1,un-1,,j; lnn.
Properties of LU matrices of A=LU
(a) The L matrices: all 1s in diagonal, same multipliers lij as in the elimination matrices
(b) When a row of A starts with zeros, so does that row of L
(c) When a column of A starts with zeros, so does that column of U
Example
Find the LU decomposition of the matrix

25 5 1
[A] = 64 8 1
144 12 1
Solution

[A] = [L][U ]
1
=
21

31

0
1

32

0 u11 u12
0 0 u22

1 0
0

u13
u23

u33

[U ]

The
matrix is the same as found at the end of the forward elimination of Nave Gauss
elimination method, that is

5
1
25

[U ] = 0 4.8 1.56
0
0
0.7


a


a
To find 21 and 31 , find the multiplier that was used to make the 21 and 31 elements zero in
the first step of forward elimination of the Nave Gauss elimination method. It was

64
25
= 2.56

21 =

144
25
= 5.76

32

31 =

To find

, what multiplier was used to make

a32

element zero? Remember

made zero in the second step of forward elimination. The

second step of forward elimination was

[A]

a 32

element was

matrix at the beginning of the

5
1
25
0 4.8 1.56

0 16.8 4.76
So

16.8
4 .8
= 3 .5

32 =

Hence

0 0
1

[L] = 2.56 1 0
5.76 3.5 1
[L][U ] = [A] .
Confirm
0 0 25
5
1
1
[L][U ] = 2.56 1 0 0 4.8 1.56
5.76 3.5 1 0
0
0.7
25 5 1
= 64 8 1
144 12 1
Another example
Perform LU decomposition of the following matrix
10
2
1
-3
-6
2
1
1
5
Multiply the first row by f21 = -3/10 = -0.3 and subtract the result from the second row to eliminate
the a21 term. Then, multiply the first row by f31 = 1/10 = 0.1 and subtract the result from the third
row to eliminate the a31 term. The result is
10
0
0

2
5.4
0.8

1
1.7
5.1

Multiply the second row by f32 = 0.8/(-5.4) = -0.148148 and subtract the result from the third row
to eliminate the a32 term.
10
0
0

2
5.4
0

1
1.7
5.351852

Therefore, the LU decomposition is

0
0 10
2
1
1

[ L][U ] = 0.3
1
0 0 5 .4
1.7
0.1 0.148148 1 0
0
5.351852
Multiplying [L] and [U] yields the original matrix as verified by the following MATLAB session,
>> L = [1 0 0;-0.3 1 0;0.1 -0.148148 1];
>> U = [10 2 -1;0 -5.4 1.7; 0 0 5.351852];
>> A = L*U
A=
10.0000
2.0000 -1.0000
-3.0000 -6.0000 2.0000
1.0000 1.0000 5.0000

Solution of linear equations by LU decomposition

Now to solve the system of linear equations, the initial system can be expressed as:

[A]{x} = {b}
Under the following form

[A]{x} = [L][U ]{x} = {b}

To find the solution {x}, the first a vector {z} can be defined:

{z} = [U ]{x}

[L]{z} = {b}
As [L] is a lower triangular matrix the {z} can be computed starting by z
of {x} can be found using the equation:
Our initial system becomes, then:

until zn. Then the values

{z} = [U ]{x}
as [U] is an upper triangular matrix, it is possible to compute {x} using a back substitution
process starting xn until x1. [You will better understand with an example ]
The general form to solve a system of linear equations using LU decomposition is:

z1 =

b1
l11
i 1

bi lik z k
zi =

k =1

; i = 2,3,..., n

lii

And

xn = z n
n

xi = z i

ik

xk ; i = n 1, n 2,...,2,1

k =i +1

Example
Use the LU decomposition method to solve the following simultaneous linear equations.

25 5 1 a1 106.8
64 8 1 a = 177.2

144 12 1 a 3 279.2
Solution
Recall that

[A][X ] = [C ]
and if

[A] = [L][U ]
then first solving

[L][Z ] = [C ]
and then

[U ][X ] = [Z ]
gives the solution vector [X ] .
Now in the previous example, we showed

[A] = [L][U ]
0 0 25
5
1
1

= 2.56 1 0 0 4.8 1.56

5.76 3.5 1 0
0
0.7
First solve

[L][Z ] = [C ]
0 0 z1 106.8
1
2.56 1 0 z = 177.2

5.76 3.5 1 z 3 279.2

to give

z1 = 106.8
2.56 z1 + z 2 = 177.2

5.76 z1 + 3.5 z 2 + z 3 = 279.2

Forward substitution starting from the first equation gives

z1 = 106.8
z 2 = 177.2 2.56 z1
= 177.2 2.56 106.8
= 96.208
z3 = 279.2 5.76 z1 3.5 z 2
= 279.2 5.76 106.8 3.5 ( 96.208) = 0.76
Hence

z1
[Z ] = z 2
z 3
106.8
= 96.208
0.76
This matrix is same as the right hand side obtained at the end of the forward elimination steps of
Nave Gauss elimination method. Is this a coincidence?
Now solve

[U ][X ] = [Z ]
5
1 a1 106.8
25
0 4.8 1.56 a = 96.208

0
0
0.7 a3 0.76
25a1 + 5a 2 + a3 = 106.8
4.8a 2 1.56a3 = 96.208
0.7 a3 = 0.76
From the third equation

0.7 a3 = 0.76
0.76
a3 =
0.7
= 1.0857
Substituting the value of

a3 in the second equation,

4.8a 2 1.56a3 = 96.208

a2 =

96.208 + 1.56a3
4 .8
96.208 + 1.56 1.0857
=
4.8
= 19.691

Substituting the value of a 2 and a3 in the first equation,

25a1 + 5a 2 + a3 = 106.8
a1 =

106.8 5a 2 a3
25
106.8 5 19.691 1.0857
=
25
= 0.29048

Hence the solution vector is

a1 0.29048
a = 19.691
2

a3 1.0857
Inverse of a square matrix using LU decomposition?
A matrix [B ] is the inverse of [ A] if

[A][B] = [I ] = [B][A].
How can we use LU decomposition to find the inverse of the matrix? Assume the first column of

[B] (the inverse of [A] ) is

[b11 b12 ... ... bn1 ]T
Then from the above definition of an inverse and the definition of matrix multiplication

b11 1
b 0
[A] 21 =
 

bn1 0
Similarly the second column of [B ] is given by

b12 0
b 1
[A] 22 =
 

bn 2 0
Similarly, all columns of [B] can be found by solving n different sets of equations with the
column of the right hand side being the n columns of the identity matrix.
Example
Use LU decomposition to find the inverse of

25 5 1
[A] = 64 8 1
144 12 1
Solution
Knowing that

[A] = [L][U ]
7

0 0 25
5
1
1

= 2.56 1 0 0 4.8 1.56

5.76 3.5 1 0
0
0.7
1
We can solve for the first column of [ B ] = [A] by solving for
25 5 1 b11 1
64 8 1 b = 0

21
144 12 1 b31 0
First solve

[L][Z ] = [C ] ,
that is

0 0 z1 1
1
2.56 1 0 z = 0

2
5.76 3.5 1 z 3 0
to give

z1 = 1
2.56 z1 + z 2 = 0
5.76 z1 + 3.5 z 2 + z 3 = 0
Forward substitution starting from the first equation gives

z1 = 1
z 2 = 0 2.56 z1
= 0 2.56(1)
= 2.56
z 3 = 0 5.76 z1 3.5 z 2
= 0 5.76(1) 3.5( 2.56)
= 3 .2
Hence

z1
[Z ] = z 2
z 3
1
= 2.56
3.2
Now solve

[U ][X ] = [Z ]
that is

5
1 b11 1
25
0 4.8 1.56 b = 2.56

21

0
0
0.7 b31 3.2
8

25b11 + 5b21 + b31 = 1

4.8b21 1.56b31 = 2.56

0.7b31 = 3.2
Backward substitution starting from the third equation gives

3 .2
0 .7
= 4.571
2.56 + 1.56b31
b21 =
4 .8
2.56 + 1.56(4.571)
=
4 .8
= 0.9524
1 5b21 b31
b11 =
25
1 5(0.9524) 4.571
=
25
= 0.04762

b31 =

Hence the first column of the inverse of [A] is

b11 0.04762
b = 0.9524
21

b31 4.571
Similarly by solving

25 5 1 b12 0
64 8 1 b = 1

22
144 12 1 b32 0
gives

b12 0.08333
b = 1.417
22

b32 5.000

and solving

25 5 1 b13 0
64 8 1 b = 0

23
144 12 1 b33 1 gives

b13 0.03571
b = 0.4643
23

b33 1.429

Hence

[A]

0.04762 0.08333 0.03571

= 0.9524
1.417
0.4643
4.571
5.000
1.429