Iterative Methods for Solving Linear

Systems of Equations
(part of the course given for the 2 grade at BGU, ME)
nd
 Iterative Methods
An iterative technique to solve Ax=b starts with an initial
(k )
approximation x (0) and generates a sequence x k 0  
First we convert the system Ax=b into an equivalent
form x  Tx  c

And generate the sequence of approximation by

x ( k )  Tx ( k 1)  c, k  1,2,3...
.This procedure is similar to the fixed point method
:The stopping criterion x ( k )  x ( k 1)

x (k )
 Iterative Methods (Example)
E1 : 10 x1  x 2  2 x3  6
E2 :  x1  11 x 2  x3  3 x 4  25
E3 : 2 x1  x 2  10 x3  x 4   11
E4 : 3x2  x3  8 x 4  15

We rewrite the system in the x=Tx+c form

1 1 3
x1  x 2  x3 
10 5 5
1 1 3 25
x2  x1  x3  x 4 
11 11 11 11
1 1 1 11
x3  - x1  x 2  x4 
5 10 10 10
3 1 15
x4   x 2  x3 
8 8 8
 .Iterative Methods (Example) – cont
and start iterations with x (0)  (0, 0, 0, 0)
1 ( 0) 1 3
x1(1)  x 2  x3(0)   0.6000
10 5 5
1 ( 0) 1 3 25
x 2(1)  x1  x3(0) - x 4(0)   2.2727
11 11 11 11
1 1 1 11
x3(1)  - x1(0)  x 2(0)  x 4(0)   1.1000
5 10 10 10
3 1 15
x 4(1)  - x 2(0)  x3(0)   1.8750
8 8 8

:Continuing the iterations, the results are in the Table

 The Jacobi Iterative Method

The method of the Example is called the Jacobi iterative

method

 
( k 1)
 aij x j  bi
j 1
j i
xi( k )  , i  1, 2,...., n
aii
Algorithm: Jacobi Iterative Method
The Jacobi Method: x=Tx+c Form
a11 a12  a1n 
a a22  a2n 
 21
A . . . 
 
 . . . 
 an1 an 2  ann 


a11 0...............0  0 ............. .............0  0  a12 ........  a1n 

0 a .......... ...0   a ...................0  0 ................  a 
 22   21   2 n 
 ............................    ..........................   ............................. 
     
...........................0  ...........................  ...................... .  a n -1, n 
 0.................0 a nn    a n1.....  a n, n 1 0  0..... ...................0 
                               
D L U

A  DLU
 The Jacobi Method: x=Tx+c Form
(cont)
and the equation Ax=b can
A  DLU be transformed into

D  L  U x  b
Dx  L  U x  b

x  D1 L  U x  D 1b

Finally
TD 1
L  U  c  D 1b
 The Gauss-Seidel Iterative Method
(k )
The idea of GS is to compute x using most recently
:calculated values. In our example
1 ( k 1) 1 3
x1( k )  x2  x3( k 1) 
10 5 5
1 (k ) 1 3 25
x2( k )  x1  x3( k 1) - x4( k 1) 
11 11 11 11
1 1 1 11
x3( k )  - x1( k )  x2( k )  x4( k 1) 
5 10 10 10
3 1 15
x4( k )  - x2( k )  x3( k ) 
8 8 8
( 0)
Starting iterations with x  (0, 0, 0, 0) , we obtain
 The Gauss-Seidel Iterative Method

 a   a x 
i 1 n
(k ) ( k 1)
 ij j 
x ij j  bi
j 1 j  i 1
xi( k )  , i  1, 2,...., n
aii

(k ) ( k 1)
Gauss-Seidel in x  Tx c form (the Fixed
Point)
Ax  (D  L  U)x  b
D  L x  Ux  b
D  L x (k )  Ux(k 1)  b
Finally x ( k )  D  L 1 U x ( k 1)  D  L 1 b
       
T c
Algorithm: Gauss-Seidel Iterative Method
The Successive Over-Relaxation Method (SOR)

The SOR is devised by applying extrapolation to the

GS metod. The extrapolation tales the form of a
weighted average between the previous iterate and
the computed GS iterate successively for each
component

x (k )
i  x i
(k )
 (1 -  ) x( k 1)
i
(k )
where xi denotes a GS iterate and ω is the
extrapolation factor. The idea is to choose a value of ω
that will accelerate the rate of convergence.
0   1 under-relaxation
1  2 over-relaxation
 SOR: Example
4 x1  3 x 2  24
3 x1  4 x 2  x3  30 Solution: x=(3, 4, -5)
 x 2  4 x3  24