You are on page 1of 8

Numerical Methods for Eng [ENGR 391] [Lyes KAEM !

""#]
$$$% L& decom'osition Method
Gauss elimination becomes inefficient when solving equations with the same coefficients for [A]
but with different bs.
LU decomposition separates the time consuming elimination of [A] form the manipulation of { } b .
Hence the decomposed [A] could be used with several { } b s in an efficient manner.
LU decomposition is based on the fact that an! square matri" [A] can be written as a product of
two matrices as#
[A]$[L][U]
%here [L] is a lower triangular matri" and [U] is an upper triangular matri".
$$$%1% (rout)s method
&o illustrate the 'routs method for LU decomposition let us start with an e"ample we consider
the (( matri"#
1
1
1
]
1

33 32 31
23 22 21
13 12 11
a a a
a a a
a a a
$
1
1
1
]
1

33 32 31
22 21
11
0
0 0
l l l
l l
l
1
1
1
]
1

1 0 0
1 0
1
23
13 12
u
u u
Hence
1
1
1
]
1

33 32 31
23 22 21
13 12 11
a a a
a a a
a a a
*
( ) ( )
( ) ( )
( ) ( ) 1
1
1
]
1

+ + +
+ +
33 23 32 13 31 32 12 31 31
23 22 13 21 22 12 21 21
13 11 12 11 11
l u l u l l u l l
u l u l l u l l
u l u l l
%e can find therefore the elements of the matrices [L] and [U] b! equating the two above
matrices#

'

+ +

+

+
+


23 32 13 31 33 33 33 33 23 32 13 31
22
13 21 23
23 23 23 22 13 21
11
13
11
13
13 13 13 11
12 31 32 32 32 32 12 31
12 21 22 22 22 22 12 21
11
12
11
12
12 12 12 11
31 31 21 21 11 11
,
,
,
,
,
,
; ;
u l u l a l hence a l u l u l
l
u l a
u hence a u l u l
a
a
l
a
u hence a u l
u l a l hence a l u l
u l a l hence a l u l
a
a
l
a
u hence a u l
a l a l a l
Linear Alge+raic E,uations
52
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
)ooooooooouffffffffff ***.
+or a general nn matri" !ou have to appl! the following e"pressions to find the LU
decomposition of a matri" [A]#

'

1
1
j
k
kj ik ij ij
u l a l , i-, i$1./n

'

ii
i
k
kj ik ij
ij
l
u l a
u
1
1
, i0-, -$!(/n
and 1
ii
u , i$1./n
$$$%!% -olution of e,uations
2ow to solve our s!stem of linear equations we can e"press our initial s!stem#
[ ]{ } { } b x A

Under the following form
[ ]{ } [ ][ ]{ } { } b x U L x A
&o find the solution{ } x we define first a vector { } z #
{ } [ ]{ } x U z
)ur initial s!stem becomes then# [ ]{ } { } b z L
Linear Alge+raic E,uations
53
345)6&A2& 2)&7
As for the (( matri" 8see above9 it is better to follow a certain order when computing the
terms of the [L] and [U] matrices. &his order is# li1u1-, li.u.-, /, lin:1un:1-, lnn.
7"ample
+ind the LU decomposition of the following matri" using 'routs method#
[A]$
1
1
1
]
1

33 32 31
23 22 21
13 12 11
a a a
a a a
a a a
$
1
1
1
]
1

2 2 3
1 3 4
1 1 2
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
As [L] is a lower triangular matri" the { } z can be computed starting b! ;1 until ;n. &hen the values
of { } x can be found using the equation#
{ } [ ]{ } x U z
as [U] is an upper triangular matri" it is possible to compute { } x using a bac< substitution
process starting "n until "1. [=ou will better understand with an e"ample /]
&he general form to solve a s!stem of linear equations using LU decomposition is#
n i
l
z l b
z
l
b
z
ii
i
k
k ik i
i
,..., 3 , 2 ;
1
1
11
1
1

And
1 , 2 ,..., 2 , 1 ;
1

+
n n i x u z x
z x
n
i k
k ik i i
n n
$$$%3% (holes.i)s method for symmetric matrices
Linear Alge+raic E,uations
54
7"ample
>olve the following equations using the LU decomposition#

'

+ +
+
+
15 2 2 3
6 3 4
4 2
3 2 1
3 2 1
3 2 1
x x x
x x x
x x x
2ote on storage of [A], [L], [U]
1: 3n practice the matrices [L] and [U] do not need to be stored separatel!. ?! omitting
the ;eroes in [L] and [U] and the ones in the diagonal of [U] it is possible to store the
elements of [L] and [U] in the same matri".
.: 2ote also that in the general formula for LU decomposition once an element of the
matri" [A] is used it is not needed in the subsequent computations. Hence the
elements of the matri" generated in point 819 above can be stored in [A]
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
3n man! engineering applications the matrices involved will be s!mmetric and defined positive. 3t
is then better to use the 'holes<is method.
3n the 'holes<is method our matri" [A] is decomposed into#
[A]$[U]
&
[U]
where [U] is an upper triangular matri".
&he elements of [U] are given b!#
( )

'


+ +
,
_



,
_

j i u
n i i j and n i u u a
u
u
n i u a u
n j
u
a
u
a u
ij
i
k
kj ki ij
ii
ij
i
k
ki ii ii
j
j
; 0
,..., 2 , 1 ,..., 3 , 2 ;
1
,..., 3 , 2 ;
,..., 3 , 2 ;
1
1
2 / 1
1
1
2
11
1
1
2 / 1
11 11
$$$%/% $n0erse of a symmetric matri1
3f a matri" [A] is square there is another matri" [A]
:1
called the inverse of [A], such as#
[A][A]
:1
$[A]
:1
[A]$[3] , identit! matri"
&o compute the inverse matri" the first column of [A]
:1
is obtained b! solving the problem 8for ((
matri"9#
[ ]{ } { }

'


0
0
1
b x A , second column# [ ]{ } { }

'


0
1
0
b x A , and third column,
[ ]{ } { }

'


1
0
0
b x A
&he best wa! to implement such a calculation is to use LU decomposition.
3n engineering the inverse matri" is of particular interest since its elements represent the
response of a single part of the s!stem to a unit stimulus of an! other part of the s!stem.
$$$%2% Matri1 condition num+er
Linear Alge+raic E,uations
55
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
$$$%2%1% 3ector and matri1 norms
A norm is a real:valued function that provides a measure of the si;e or length of multicomponent
mathematical entities#
+or a vector x

'

n
x
x
x
x
.
.
.
2
1

&he 7uclidean norm of this vector is defined as#


( )
2
1
2 2
2
2
1
...
n
x x x x + + +

3n general Lp @norm of a vector x

is defined as#
P
n
i
P
i P
x L
1
1
)

'

+or a matri", the first and infinit! norms are defined as#
[ ]

n
i
ij
n j
a A
1
1
1
max $ ma"imum column sum
[ ]


n
j
ij
n i
a A
1
1
max
$ ma"imum of row sum.
$$$%2%!% Matri1 condition num+er
&he matri" condition number is defined as#
Linear Alge+raic E,uations
56
2ote
3f the value of 8p9 is increased to infinit! in the above e"pression the value of the L

norm will
tend to the value of the largest component of x

#
i
x L max

Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]


[ ]
1
A A A Cond
+or a matri" [A] we have that# cond [A] 1
and
[ ]
A
A
A Cond
x
x

&herefore the error on the solution can be as large as the relative error of the norm of [A]
multiplied b! the condition number.
3f the precision on [A] is t:digits 81A
:t
9 and 'ond[A]$1A
'
the solution on ["] ma! be valid to onl! t:c
digits 81A
c:t
9.
$$$%4% 5aco+i iteration Method
&he Bacobi method is an iterative method to solve s!stems of linear algebraic equations.
'onsider the following s!stem#

'

+ + + + +
+ + + +
n nn nn n n n
n n
b x a x a x a x a
b x a x a x a x a
...
.
.
...
3 3 2 2 1 1
1 1 3 13 2 12 1 11
&his s!stem can be written under the following form#
( )
( )

'



1 , 1 , 2 2 1 1 1
1 3 13 2 12 1
11
1
...
1
.
.
...
1
n n n n n n n
nn
n n
x a x a x a b
a
x
x a x a x a b
a
x
&he general formulation is#
1
]
1




n
i j j
j ij i
ii
i
x a b
a
x
, 1
1
, i$1./n
Here we start b! an initial guess for "1, "., /, "n and we compute the new values for the ne"t
iteration. 3f no good initial guess is available we can assume each component to be ;ero.
%e generate the solution at the ne"t iteration using the following e"pression#
1
]
1




+
n
i j j
k
j ij i
ii
k
i
x a b
a
x
, 1
1
1
, i$1./n, and for <$1. / convergence
Linear Alge+raic E,uations
57
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
&he calculation must be stopped if#

+
k
i
k
i
k
i
x
x x
1
, is the desired precision.
3t is possible to show that a sufficient condition for the convergence of the Bacobi method is#


>
n
i j j
ij ii
a a
, 1
$$$%#% Gauss6-eidel iteration Method
3t can be seen that in Bacobi iteration method all the new values are computed using the values
at the previous iteration. &his implies that both the present and the previous set of values have to
be stored. Gauss:>eidel method will improve the storage requirement as well as the
convergence.
3n Gauss:>eidel method the values
1 1
2
1
1
,..., ,
+ + + k
i
k k
x x x computed in the current iteration as
well as
k
n
k
i
k
i
x x x ,..., ,
3 2 + +
are used in finding the value
1
1
+
+
k
i
x . &his implies that alwa!s the
most recent a''ro1imations are used during the computation. &he general e"pression is#

1
1
]
1

+
+ +
1
1 1
1 1
1
i
j
n
i j
OLD
k
i ij
NEW
k
j ij i
ii
k
i
x a x a b
a
x

, i$1....n, <$1.( /..
$$$%#%1% $m'ro0ement of con0ergence using rela1ation
6ela"ation is used to enhance convergence the new value is written under the form#
Linear Alge+raic E,uations
58
2ote
: &he Gauss:>eidel method will converge to the correct solution irrespective of the initial
estimate if the s!stem of equations is diagonall! dominant. ?ut in man! cases the solution
will converges even if the s!stem is wea<l! diagonall! dominant.
7"ample
+ind the solution of the following equations using the Gauss:>eidel iteration method#

'

+ +
+
+
32 7 2
2 3 6 2
1 2 5
3 2 1
3 2 1
3 2 1
x x x
x x x
x x x
Numerical Methods for Eng [ENGR 391] [Lyes KAEM !""#]
OLD
i
NEW
i
NEW
i
x x x ) 1 ( +
And usuall! A00.
3f A001 we are using under:rela"ation used to ma<e a non:convergent s!stem converge
or to damp the oscillations.
3f 100. we are using over:rela"ation to accelerate the convergence of an alread!
convergent s!stem.
&he choice of depends on the problem to be solved.
$$$%7% (hoice of the method
1: 3f the equations are to be solved for different right:hand:side vectors a direct method li<e
LU decomposition is preferred.
.: &he Gauss:>eidel method will give accurate solution even when the number of
equations is several thousands 8if the s!stem is diagonall! dominant9. 3t is usuall! twice
as fast as the Bacobi method.
Linear Alge+raic E,uations
59