Faculty of Engineering
Civil Engineering Department
Numerical Analysis
ECIV 3306
Chapter 11
Special Matrices and GaussSiedel
Symmetric Matrices
– Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes.
(
(
(
(
¸
(
¸
=
88 6 39 16
6 9 7 2
39 7 3 1
16 2 1 5
] [ A
• Symmetric matrix:
If a
ij
= a
ji
÷ [A]
nxn
is a
symmetric matrix
Cholesky Decomposition
3
This method is suitable for only symmetric systems
where:
T
ij ji
a a and A A = =
11 12 13 11 11 21 31
21 22 23 21 22 22 32
31 32 33 31 32 33 33
0 0
0 * 0
0 0
a a a l l l l
a a a l l l l
a a a l l l l
( ( (
( ( (
=
( ( (
( ( (
¸ ¸ ¸ ¸ ¸ ¸
11
21 22
31 32 33
0 0
[ ] 0
l
L l l
l l l
(
(
=
(
(
¸ ¸
*
T
A L L =
Cholesky Decomposition
1
1
1
2
1
1, 2, , 1
i
ki ij kj
j
ki
ii
k
kk kk kj
j
a l l
l for i k
l
l a l
÷
=
÷
=
÷ ·
= = ÷
= ÷
¿
¿
1
1
1
2
1
1, 2, , 1
i
ki ij kj
j
ki
ii
k
kk kk kj
j
a l l
l for i k
l
l a l
÷
=
÷
=
÷ ·
= = ÷
= ÷
¿
¿
11
31
31
11
21
21
11 11
L
a
L
L
a
L
a L
=
=
=
2
31
2
32 33 33
22
31 21 32
32
2
21 22 22
L L a L
L
L L a
L
L a L
÷ ÷ =
÷
=
÷ =
For 3x3
matrix,
Once the matrix L has been calculated, L
T
can easily be found and the
forward , backward substitution steps can be followed to find the
solution {X} for any right hand side {B}.
Pseudocode for Cholesky’s LU
Decomposition algorithm (cont’d)
GaussSiedel
• Iterative or approximate methods provide an
alternative to the elimination methods. The Gauss
Seidel method is the most commonly used iterative
method.
• The system [A]{X}={B} is reshaped by solving the
first equation for x
1
, the second equation for x
2
,
and the third for x
3
, …and n
th
equation for x
n
. We
will limit ourselves to a 3x3 set of equations.
GaussSiedel
33
2 32 1 31 3
1
22
3 23 1 21 2
2
11
3 13 2 12 1
1
a
x a x a b
x
a
x a x a b
x
a
x a x a b
x
÷ ÷
=
÷ ÷
=
÷ ÷
=
3 3 33 2 32 1 31
2 3 23 2 22 1 21
1 3 13 2 12 1 11
b x a x a x a
b x a x a x a
b x a x a x a
= + +
= + +
= + +
¬
Now we can start the solution process by choosing
guesses for the x’s. A simple way to obtain initial
guesses is to assume that they are zero. These
zeros can be substituted into x
1
equation to
calculate a new x
1
=b
1
/a
11
.
GaussSiedel
• New x
1
is substituted to calculate x
2
and x
3
. The
procedure is repeated until the convergence
criterion is satisfied:
s
new
i
old
i
new
i
i a
x
x x
c c <
÷
= % 100
,
Gauss Seidal
Graphical Interpretation
Jacobi iteration Method
An alternative approach, called Jacobi iteration,
utilizes a somewhat different technique. This
technique includes computing a set of new x’s on the
basis of a set of old x’s. Thus, as the new values are
generated, they are not immediately used but are
retained for the next iteration.
GaussSiedel
The GaussSeidel method The Jacobi iteration method
Convergence Criterion for GaussSeidel Method
• The gausssiedel method is similar to the technique of
fixedpoint iteration.
• The GaussSeidel method has two fundamental
problems as any iterative method:
1. It is sometimes nonconvergent, and
2. If it converges, converges very slowly.
• Sufficient conditions for convergence of two linear
equations, u(x,y) and v(x,y) are:
1
1
<
c
c
+
c
c
<
c
c
+
c
c
y
v
x
v
y
u
x
u
Convergence Criterion for GaussSeidel
Method (cont’d)
• Similarly, in case of two simultaneous equations, the
GaussSeidel algorithm can be expressed as:
1 12
1 2 2
11 11
2 21
1 2 1
22 22
12
1 2 11
21
1 22 2
( , )
( , )
0
0
b a
u x x x
a a
b a
v x x x
a a
a u u
x x a
a v v
x a x
= ÷
= ÷
c c
= = ÷
c c
c c
= ÷ =
c c
Convergence Criterion for GaussSeidel
Method (cont’d)
Substitution into convergence criterion of two linear
equations yield:
In other words, the absolute values of the slopes must be
less than unity for convergence:
That is, the diagonal element must be greater than the sum
of offdiagonal element for each row.
1 , 1
22
21
11
12
< <
a
a
a
a
21 22
12 11
a a
a a
>
>
¿
=
=
>
n
i j
j
j i ii
a a
1
,
equations n For
GaussSiedel Method Example 1
10
x 2 0 x 3 0 4 71
7
x 3 0 x 1 0 3 19
3
x 2 0 x 1 0 85 7
x
x
x
4 71
3 19
85 7
x
x
x
10 2 0 3 0
3 0 7 1 0
2 0 1 0 3
2 1
3 1
3 2
3
2
1
3
2
1
. . .
. . .
. . .
.
.
.
. .
. .
. .
+ ÷
+ ÷ ÷
+ +
=
=
=
¬
¦
)
¦
`
¹
¦
¹
¦
´
¦
÷ =
¦
)
¦
`
¹
¦
¹
¦
´
¦
(
(
(
¸
(
¸
÷
÷
÷
• Guess x
1
,
x
2
,
x
3
= zero for the first guess
Iter. x
1
x
2
x
3
c
a,1
(%)
c
a,2

(%) c
a,3
 (%)
0 0 0 0   
1 2.6167 2.7945 7.005610 100 100 100
2 2.990557 2.499625 7.000291 12.5 11.8 0.076
Improvement of Convergence Using
Relaxation
• Where ì is a weighting factor that is assigned a
value between [0, 2]
• If ì = 1 the method is unmodified.
• If ì is between 0 and 1 (under relaxation) this is
employed to make a non convergent system to
converge.
• If ì is between 1 and 2 (over relaxation) this is
employed to accelerate the convergence.
( )
1
new new old
i i i
x x x ì ì = · + ÷ ·
GaussSiedel Method Example 2
The system can be rearranged to the following
6x
1
x
2
x
3
= 3
6x
1
+9x
2
+x
3
=40
3x
1
+x
2
+12x
3
=50
GaussSiedel Method Example 2
12
3 50
9
6 40
6
3
2 1
3
3 1
2
3 2
1
x x
x
x x
x
x x
x
÷ +
=
÷ ÷
=
+ +
=
Without Relaxation
GaussSiedel Method Example 2