Gauss-Siedel Method

Major: All Engineering Majors

Authors: Autar Kaw

(Modified by P. Goel for IDC103)

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
3/25/2010 http://numericalmethods.eng.usf.edu 1
 Gauss-Seidel Method

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
An iterative method.

Basic Procedure :
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Why?
Obviously it isn’t possible to obtain better than machine precision for
a solution of SLEs (using Gaussian Elimination and LU
Decomposition). Actually, the situation is worse for large systems: it
isn’t possible to get close to machine precision in direct methods.

The Gauss-Seidel Method allows the user to control round-off error.

In fact, iterative methods can be used to improve the solution
obtained by direct methods.

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1 non-zero
a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn
First equation, solve for x1
Second equation, solve for x2

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Algorithm
Rewriting each equation
c1 - a12 x 2 - a13 x3 KK - a1n x n
x1 = From Equation 1
a11

c2 - a21 x1 - a23 x3 KK - a2 n xn
x2 = From equation 2
a22
M M M
cn -1 - an -1,1 x1 - an -1, 2 x2 KK - an -1,n -2 xn -2 - an -1, n xn From equation n-1
xn -1 =
an -1,n -1
cn - an1 x1 - an 2 x2 - KK - an , n -1 xn -1
xn = From equation n
ann

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Algorithm
General Form of each equation
n n

c1 - å a1 j x j cn -1 - åa
j =1
n -1, j xj
j =1
j ¹ n -1
x1 =
j ¹1 xn -1 =
a11 an -1,n -1
n
c n - å a nj x j
n
c2 - å a2 j x j
j =1 j =1
j¹n
x2 =
j¹2
xn =
a 22 a nn

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci - å aij x j
j =1
j ¹i
xi = , i = 1,2,K , n.
aii

How or where can this equation be used?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equati ons to solve for
each value of x i.
Important: Remember to use the
é x1 ù most recent value of xi. Which
êx ú means to apply values calculated to
ê 2ú the calculations remaining in the
current iteration.
ê M ú
ê ú
ê xn -1 ú
êë xn úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
x new
-x old
Îa i = i
new
i
´ 100
x i

So when has the answer been f ound?

The iterations are stopped when the absolute relative

approximate error is less than a prespecified tolerance for all
unknowns.

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
The system of equati ons
é 25 5 1ù é a1 ù é106 .8 ù
ê 64 8 1ú êa ú = ê177 .2 ú
ê ú ê 2ú ê ú
êë144 12 1úû êë a3 úû êë 279 .2 úû

é a1 ù é1 ù
Initial Guess: Assume an initial guess of êa ú = ê2 ú
ê 2ú ê ú
êë a3 úû êë5úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Rewriting each equation
106.8 - 5a 2 - a 3
a1 =
25
é 25 5 1ù é a1 ù é106 .8 ù
ê 64 8 1ú ê a ú = ê177 .2 ú 177.2 - 64a1 - a 3
ê ú ê 2ú ê ú a2 =
êë144 12 1úû êë a3 úû êë 279 .2 úû 8

279.2 - 144a1 - 12a 2

a3 =
1

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Applying the initial guess and solving for ai
é a1 ù é1ù 106.8 - 5(2) - (5)
a1 = = 3.6720
êa ú = ê2ú 25
ê 2ú ê ú
êë a3 úû êë5úû 177.2 - 64(3.6720 ) - (5)
a2 = = -7.8510
Initial Guess 8
279.2 - 144(3.6720 ) - 12(- 7.8510 )
a3 = = -155.36
1

When solving for a2, how many of the initial guess values were used?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error

xinew - xiold At the end of the first iteration

Îa i = new
´100
xi
é a1 ù é 3.6720 ù
ê a ú = ê - 7.8510 ú
3.6720 - 1.0000 ê 2ú ê ú
Îa 1 = x100 = 72.76%
3.6720 êë a 3 úû êë - 155 .36 úû

- 7.8510 - 2.0000 The maximum absolute

Îa = x100 = 125.47% relative approximate error is
2
- 7.8510
125.47%

- 155.36 - 5.0000
Îa 3 = x100 = 103.22%
- 155.36

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Iteration #2
Using
é a1 ù é 3.6720 ù the values of ai are found:
êa ú = ê- 7.8510ú 106.8 - 5(- 7.8510 ) - 155.36
ê 2ú ê ú a1 = = 12.056
êë a 3 úû êë - 155.36 úû 25

from iteration #1
177.2 - 64(12.056 ) - 155.36
a2 = = -54.882
8

279.2 - 144(12.056 ) - 12(- 54.882 )

a3 = = -798.34
1

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
12.056 - 3.6720 At the end of the second iteration
Îa 1 = x100 = 69.543%
12.056 é a ù é 12.056 ù
1
êa ú = ê- 54.882ú
ê 2ú ê ú
- 54.882 - (- 7.8510) êë a3 úû êë- 798.54úû
Îa 2 = x100 = 85.695%
- 54.882
The maximum absolute
- 798.34 - (- 155.36 ) relative approximate error is
Îa 3 = x100 = 80.540% 85.695%
- 798.34

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 1
Repeating more iterations, the following values are obtained

Iteration a1 Îa 1 % a2 Îa 2 % a3 Îa 3 %
1 3.6720 72.767 −7.8510 125.47 −155.36 103.22
2 12.056 69.543 −54.882 85.695 −798.34 80.540
3 47.182 74.447 −255.51 78.521 −3448.9 76.852
4 193.33 75.595 −1093.4 76.632 −14440 76.116
5 800.53 75.850 −4577.2 76.112 −60072 75.963
6 3322.6 75.906 −19049 75.972 −249580 75.931
Notice – The relative errors are not dec reasing at any significant rate
é a 1 ù é0.29048ù
Also, the solution is not converging to the true s olution of ê ú ê ú
a
ê 2ú ê = 19 . 690 ú
êëa 3 úû êë 1.0857 úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gaus s-Siedel method: not all
systems of equations will converge.

Is there a fix?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Pitfall
GAUSS-SEIDEL CONVERGENCE THEOREM: If A is
diagonally dominant, then the Gauss-Seidel method
converges for any starting vector x. A sufficient, but
not necessary condition.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n n
aii ³ å aij for all ‘i’ and aii > å aij for at least one ‘i’
j =1 j =1
j ¹i j ¹i

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other c oefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?

é 2 5.81 34 ù é124 34 56 ù
[A ] = êê 45 43 1 úú [ B] = êê 23 53 5 úú
êë123 16 1 úû êë 96 34 129 úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
Given the system of equations The coefficient matrix is:

12 x1 + 3 x2- 5 x3 = 1 é12 3 - 5ù
x1 + 5 x2 + 3x3 = 28 [A] = êê 1 5 3 úú
3x1 + 7 x2 + 13x3 = 76 êë 3 7 13 úû

With an initial guess of Will the solution converge using the

Gauss-Siedel method?
é x1 ù é1ù
ê x ú = ê0 ú
ê 2ú ê ú
êë x3 úû êë1úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
a11 = 12 = 12 ³ a12 + a13 = 3 + - 5 = 8
é12 3 - 5ù
[A] = êê 1 5 3 úú a 22 = 5 = 5 ³ a 21 + a 23 = 1 + 3 = 4
êë 3 7 13 úû
a33 = 13 = 13 ³ a31 + a32 = 3 + 7 = 10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
Rewriting each equation With an initial guess of
é12 3 - 5ù é a1 ù é 1 ù é x1 ù é1 ù
ê 1 5 3 ú ê a ú = ê 28 ú ê x ú = ê0 ú
ê ú ê 2ú ê ú ê 2ú ê ú
êë 3 7 13 úû êë a3 úû êë76 úû êë x 3 úû êë1 úû

1 - 3x 2 + 5 x3 1 - 3(0) + 5(1)
x1 = x1 = = 0.50000
12 12

28 - x1 - 3x3 28 - (0.5) - 3(1)

x2 = x2 = = 4.9000
5 5

76 - 3x1 - 7 x 2 76 - 3(0.50000 ) - 7(4.9000 )

x3 = x3 = = 3.0923
13 13

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000 - 1.0000
Îa 1 = ´ 100 = 100.00%
0.50000

4.9000 - 0
Îa 2
= ´ 100 = 100.00%
4.9000

3.0923 - 1.0000
Îa 3
= ´ 100 = 67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
After Iteration #1
é x1 ù é0.5000 ù
ê x ú = ê 4.9000 ú
ê 2ú ê ú
êë x3 úû êë 3.0923 úû

Substituting the x values into the After Iteration #2

equations
é x1 ù é0.14679 ù
1 - 3(4.9000 ) + 5(3.0923) ê x ú = ê 3.7153 ú
x1 = = 0.14679
12 ê 2ú ê ú
êë x3 úû êë 3.8118 úû
28 - (0.14679 ) - 3(3.0923)
x2 = = 3.7153
5

76 - 3(0.14679 ) - 7(4.900 )
x3 = = 3.8118
13
http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
0.14679 - 0.50000
Îa 1 = ´ 100 = 240.61%
0.14679
3.7153 - 4.9000
Îa 2
= ´ 100 = 31.889%
3.7153
3.8118 - 3.0923
Îa 3
= ´ 100 = 18.874%
3.8118
The maximum absolute relative error after the fi rst iteration is 240.61%

This is much larger than the maxi mum absolute relative error obtai ned in
iteration #1. Is this a problem?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration a1 Îa 1 % a2 Îa 2 % a3 Îa 3 %

1 0.50000 100.00 4.9000 100.00 3.0923 67.662

2 0.14679 240.61 3.7153 31.889 3.8118 18.876
3 0.74275 80.236 3.1644 17.408 3.9708 4.0042
4 0.94675 21.546 3.0281 4.4996 3.9971 0.65772
5 0.99177 4.5391 3.0034 0.82499 4.0001 0.074383
6 0.99919 0.74307 3.0001 0.10856 4.0001 0.00101

é x1 ù é0.99919ù é x1 ù é1 ù
The solution obtained ê x ú = ê 3.0001 ú is close to the exact solution of ê x ú = ê 3ú .
ê 2ú ê ú ê 2ú ê ú
êë x3 úû êë 4.0001 úû êë x3 úû êë4úû

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 3
Given the system of equations

3 x1 + 7 x2 + 13 x3 = 76 Rewriting the equations

x1 + 5 x2 + 3x3 = 28 76 - 7 x2 - 13 x3
x1 =
12 x1 + 3x2 - 5 x3 = 1 3
28 - x1 - 3x3
With an initial guess of x2 =
5
é x1 ù é1 ù
ê x ú = ê0 ú 1 - 12 x1 - 3 x 2
ê 2ú ê ú x3 =
êë x3 úû êë1 úû -5

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained

Iteration a1 Îa 1 % A2 Îa 2 % a3 Îa 3 %
1 21.000 95.238 0.80000 100.00 50.680 98.027
2 −196.15 110.71 14.421 94.453 −462.30 110.96
3 −1995.0 109.83 −116.02 112.43 4718.1 109.80
4 −20149 109.90 1204.6 109.63 −47636 109.90
5 2.0364×105 109.89 −12140 109.92 4.8144×105 109.89
6 −2.0579×105 109.89 1.2272×105 109.89 −4.8653×106 109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
The Gauss-Seidel Method can still be used
é 3 7 13 ù
The coefficient matrix is not
diagonally dominant
[A] = êê 1 5 3 úú
êë12 3 - 5úû
But this is the same set of é12 3 - 5ù
equations used in example #2,
which did converge.
[A] = êê 1 5 3 úú
êë 3 7 13 úû

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equati ons can form a di agonally dominant matrix.

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equati ons

x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9
x1 + 7 x 2 + x3 = 9

Which equation(s) prevents this set of equation from having a

diagonally dominant coefficient matrix?

http://numericalmethods.eng.usf.edu
 Gauss-Seidel Method
Summary

-Advantages of the Gauss-Seidel Method

-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss -Seidel Method

http://numericalmethods.eng.usf.edu
Gauss-Seidel Method

Questions?

http://numericalmethods.eng.usf.edu
 Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit

http://numericalmethods.eng.usf.edu/topics/gauss_seid
el.html
 THE END

http://numericalmethods.eng.usf.edu