You are on page 1of 12

2.

3

GAUSS – SEIDEL METHOD Once again we consider the system

Ax = y …………….. (I)
(k ) (k ) (k ) In the Jacobi scheme we used the values of x2 , x3 ,L , xn obtained in the kth iteration, in place of x2, x3, ….., xn in the first equation,

a11 x1 + a12 x2 + ..... + a1n xn = y1

to calculate x1( k +1) from
(k ) (k ) (k ) a11 x1( k +1) = − a12 x2 − a13 x3 ..... − a1n xn + y1

Similarly,
(k ) 1 (k ) 2

in
(k ) i −1

the
(k ) i +1

ith
(k ) n

equation,

( i = 2,3,L , n ) ,

we

used

the

values,

x , x ,L , x , x ,L , x from

, in place of x1 , x2 ,L , xi −1 , xi +1 ,L , xn to calculate xi( k +1)

(k ) k) (k ) (k ) aii xi( k +1) = − ai1 x1( k ) − ai 2 x2 − ...... − ai ,i −1 xi(− 1 − ai , i +1 xi +1 − ..... − ain xn + yi LL (*)

What Gauss – Seidel suggests is that having obtained x1( k +1) from the first
( k +1) equation use this value for x1 in the second equation to calculate x2 from

( k +1) (k ) (k ) a22 x2 = − a21 x1( k +1) − a23 x3 − ...... − a2 n xn + y2
( k +1) ( k +1) and use these values of x1( k +1) , x2 , in the 3rd equation to calculate x3 , and so k +1) (k ) k) on. Thus in the equation (*) use x1( k +1) , L , xi(− in place of x1( k ) , x2 , L , xi(− 1 1 to get

the following modification of the i -th equation to calculate xi( k +1) :
( k +1) k +1) k) (k ) (k ) aii xi( k +1) = − ai1 x1( k +1) − ai 2 x2 − ...... − ai ,i −1 xi(− − ai ,i +1 xi(+ 1 1 − ai ,i + 2 xi + 2 ..... − ain xn + yi

In matrix notation we can write this as,

Dx ( k +1) = − Lx ( k +1) − Ux ( k ) + y
which can be rewritten as,

(D + L )x (k +1) = −Ux (k ) + y , and hence
x (k +1 ) = − (D + L ) Ux k + (D + L ) y
−1 −1

65

(see page 60).(II) is the Gauss – Seidel iteration matrix. Of course.Thus we get the Gauss – Seidel iteration scheme as. G = -(D+L)-1U ……. 1  A = 1 2  2 1 2 − 2  1  . 1   0 1 2 0  0 . 1  1   y =  0  0   1  D + L = 1 2  0  −U =  0 0  −2 0 0 2   −1 0   66 . . We have. G ≥ 1 does not mean that the scheme will diverge. Example 3: Let us consider the system x1 + 2x2 – 2x3 = 1 x1 + x2 + x3 = 0 2x1 + 2x2 + x3 = 0 sp < 1.. The acid test for convergence is G We shall now consider some examples. We shall now try to apply the Gauss – Seidel scheme for this system. and ˆ = ( D + L ) −1 y y The scheme converges if and only if G sp < 1. the scheme will converge if G < 1 in some matrix norm. considered in example 1 on page 59. But some matrix norm. and for which the Jacobi scheme gave the exact solution in the 3rd iteration. x(0) initial guess ˆ x (k +1) = Gx ( k ) + y where.

λ2 = 2. We have. −1 0  1  = −1 1  0 −2  0  0 1  G = −( D + L) −1 0 0   0 −2 2  1    U =  −1 1 0    0 0 −1   0 −2 1   0 0 0     From above we get the Gauss – Seidel iteration matrix as. 67 . G sp = 2 >1 Hence the Gauss – Seidel scheme for this system will not converge. Thus λ1 = 0. Thus for this system the Jacobi scheme converges so rapidly giving the exact solution in the third iteration itself whereas the Gauss – Seidel scheme does not converge. λ3 = 2 are the three eigenvalues. Example 4: Consider the system 1 1 x2 − x3 = 1 2 2 x1 + x 2 + x 3 = 0 x1 − − 1 1 x1 − x 2 + x 3 = 0 2 2 Let us apply the Gauss – Seidel scheme to this system. 0  G = 0 0  −2 2 0 2   − 3 2   Since G is triangular we get its eigenvalues immediately. Therefore.(D + L ) Thus. as its diagonal entries.

. (D + L) −1 2  1  =  −1   0  0 1 1 2 0  0 . 1  A= 1  − 1 2  −1 1 −1 2 2 0 1 −1 −1  2 1  ....(*)   1  − 2 1 is the Gauss – Seidel matrix for this sytem....  1  0  −U =  0  0  Thus...  1   0  0  1  1   y =  0  0    1  D+L= 1  − 1  2 .. 1 2 0 0 1  2 −1  .. The Gauss – Seidel scheme is ˆ x (k +1 ) = Gx ( k ) + y x (0 )  0   =  0  0   68 .  0    1  −1 G = − ( D + L ) U =  −1   0  0 1 1 2 0 0  00  10  1 2 0 0 1  2 −1   0   0  ∴G = 0   0  2 − 12 0 1  2  − 3 2  .

where ˆ = (D + L) y −1  1  y =  −1   0  0 1 1 2 01  1       0   0  =  −1 . Hence in this example the Gauss – Seidel scheme 2 x (1 )  0  1   1        ˆ = G  0  +  − 1 =  − 1 = Gx (0 ) + y  0  0   0         1   1      ˆ = G  − 1 +  − 1  +y  0   0      0  = 0   0  2 − 1 2 0        x (2 ) = Gx (1 ) 1 2 − 3 2 − 1 2 1   1   1    −1 +  −1      0   0         1− 1 2  = − 1− 1 2   0   ( ) 69 . Notice that G is upper triangular and hence we readily get the eigenvalues of G as its diagonal entries. Let us now carry out a few steps of the Gauss – Seidel iteration. sp = 1 < 1 . λ1 = 0. λ3 = 1/2. since we have now been assured of convergence. (We shall first do some exact calculations).     10  0   and where G is given (*). λ2 = -1/2. Thus the eigenvalues of G are. Hence G will converge.

..  2 2 2 2     →  − 1 − 1 2 + 1 2 − 1 3 ... + k −1 2 2   k −1   ( ) − 1  1 1 − ... + = − 1 − 2 +    22 2 k −1     0     x (k ) Clearly... here ‘we’ knew ‘a priori’ that the sequence is going to sum up neatly for each component and so we did exact calculation..x (3 ) = Gx ( 2 )  1− 1 + 1 2  2 2  1 1 ˆ + y = − 1− 2 +  22  0  ( )       If we continue this process we get k −1   − ( ) 1 1 1  1− 2 +  2 − .. Of course.  2 2   0     ( ) and by summing up the geometric series we get. x (k )  2 3    →  − 2 3  0    which is the exact solution.. x( k)  1 − 1 + 1 2 − 1 3 + 1 4 ...... If we had not noticed this we still would have carried out the computations as follows: x (1 ) = Gx (0 )  1    ˆ =  − 1 +y  0    as before 70 .

  ∞ = 0 .65625    ˆ =  − 0 . 664062  0   0 .x (2 ) = Gx (1)  0 . x (6 ) − x (5 ) = Gx (5 ) + y   0    0 .6875  +y   0   x (3 ) = Gx (2 ) x (4 ) = Gx (3 ) x (5 )  0 . 666016  =  − 0 . 667969  =  − 0 .03125 x (6 )  0 . 667969  0   0 .001953 x (10 ) x (10 ) − x (9 ) ∞ = 0 .5  +y  0     0 . 666504  0     .671875    ˆ =  − 0 .     .625  +y  0     0 .000488 71 .      . 666504  =  − 0 . = Gx (4 ) + y   0   x (5 ) − x ( 4 ) ∞ = 0 .625    ˆ =  − 0 . 666016  0   0 .003907 x (9 ) x (9 ) − x (8 ) ∞ = 0 . 664062  =  − 0 .      .025625 x (7 ) x ( 7 ) − x (6 ) ∞ = 0 .5    ˆ =  − 0 .007813 x (8 ) x (8 ) − x (7 ) ∞ = 0 .6875    ˆ =  − 0 .65625  .671875  .

λ2 = λ3 = 2 72 .Since now error is < 10-3 we may stop here and take x(10) as our solution for the system. to get. Or we may improve our accuracy by doing more iterations. 666626    (12 ) =  − 0 .000031 < 10 −4 and hence we can take x(14) as our solution within error 10-4. 666687  . 666748  =  − 0 . λ3 = − i 15 4 4 2 2 1 15 + = 16 = 2 4 4 4 1 ∴ λ1 = . 666687    (13 ) =  − 0 . 666626  . λ2 = 1 2 1 1 + i 15 . 666656  =  − 0 . 666748  0   0 . 2 0 1 2 We have the characteristic polynomial of J as λ λI − J = + 1 −1 2 −1 λ −1 2 2 −1 2  λ  1  + 1 =  λ +  λ 2 − + 1 2  2   λ Thus the eigenvalues of J are λ1 = − . x   0         0 .  1   1  2 −1   0   and therefore. x   0        x (14 ) x (14 ) − x (13 ) ∞ = 0 . We have  1  A= 1  − 1 2   0  J =  −1  1  2 −1 1 −1 1 2 2 −1  2 1  . x (11 )  0 . Let us now try to apply the Jacobi scheme for this system. 666656  0   0 .

Thus. in example 3 we had a system for which the Jacobi scheme converged but Gauss – Seidel scheme did not converge. but the Gauss – Seidel scheme converges. 16 λ 4 . it is not ‘correct’ to say that one scheme is better than the other. in general. Example 5: 2x1 – x2 =y1 -x1 + 2x2 – x3 = y2 -x2 + 2x3 –x4 =y3 -x3 + 2x4 = y4 Here 0   2 −1 0   −1 2 −1 0  A= . Let us now consider another example. whereas in example 4 above we have a system for which the Jacobi scheme does not converge. 0 − 1 2 − 1    0  − 0 1 2   is a symmetric tridiagonal matrix. Thus.∴ J sp = 2 which is >1. these two examples demonstrate that. The Jacobi matrix for this scheme is    J =       0 1 0 0 2 1 0 1 0 2 0 1 2 2 0 1 2 0   0   1  2   0   The characteristic equation is.12 λ 2 + 1 = 0 Set λ 2 = α ………………(CJ) 73 . Thus the Jacobi scheme for this system will not converge.

8090 .12α + 1 = 0 ………………(CJ1) ∴ λ is the square root of the roots of (CJ1). ± 0. Hence J sp = 0. Thus the eigenvalues of J are ± 0. The Gauss – Seidel matrix for the system is found as follows: 0 0  2  −1 2 0 (D + L ) =   0 −1 2   0 0 −1  0  0 0  2  0  0 −U =  0  0  1 0 0 0    =      0 1 0 0 1 1 1 1 2 4 8 0  0 1  0  0 1 1 1 2 4 8 0 0 1 1 2 4 0   0   0   1   2 (D + L ) −1 16 74 .8090. and the Jacobi scheme will converge.Therefore 16α2 .3090.

.12λ + 1 = 0 …………. which becomes in this case 16λ4 − 12λ3 + λ 2 = 0. Notice that roots of (CG1) are same as those of (CJ1).6545 < 1 Thus the Gauss – Seidel scheme also converges.6545.....(CG1) Thus one of the eigenvalues of G is 0 (repeated twice). 0.. (C G ) This can be factored as λ 2 (16λ2 − 12λ + 1) = 0 Thus the eigenvalues of G are roots of λ2 = 0 ... Thus....0955... 0. Observe that 75 . ... Thus nonzero eigenvalues of G are squares of eigenvalues of J.   −1 G = − (D + L ) U =       1 0 0 0 2  0 1 1 0 4 2  = 0 1 1 1 8 4 2   1 1 1 0 16 8 4  1 1 1 1          2 4 8 0 1 1 1 2 4 8 0 0 1 1 2 4 16 0   0   0   1   2        0 0 0 0 1 0 0 0 0 1 0 0 0   0  1   0  The characteristic equation of G is λI − G = 0 . G sp = 0.. and 16λ2 . ∴ the nonzero eigenvalues of G are. and two eigenvalues of G are roots of (CG1)...

We shall not go into any further details of this aspect. 76 .G sp = J 2 sp . G sp < J sp Thus the Gauss – Seidel scheme converges faster than the Jacobi scheme. In many class of problems where both schemes converge it is the Gauss – Seidel scheme that converges faster.