Gauss-Siedel Method

Major: All Engineering Majors

Authors: Autar Kaw
(Modified by P. Goel for IDC103)

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates

3/25/2010

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
An iterative method.
Basic Procedure :
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Why?
Obviously it isnt possible to obtain better than machine precision for
a solution of SLEs (using Gaussian Elimination and LU
Decomposition). Actually, the situation is worse for large systems: it
isnt possible to get close to machine precision in direct methods.

The Gauss-Seidel Method allows the user to control round-off error.

In fact, iterative methods can be used to improve the solution
obtained by direct methods.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:

a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2
.
.
.

.
.
.

an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn

If: the diagonal elements are

non-zero

Rewrite each equation solving

for the corresponding unknown

ex:
First equation, solve for x1
Second equation, solve for x2

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
Rewriting each equation

x1 =

c1 - a12 x 2 - a13 x3 KK - a1n x n

a11

From Equation 1

c2 - a21 x1 - a23 x3 KK - a2 n xn
a22

From equation 2

x2 =

M
xn -1 =
xn =

cn -1 - an -1,1 x1 - an -1, 2 x2 KK - an -1,n -2 xn -2 - an -1, n xn

an -1,n -1

cn - an1 x1 - an 2 x2 - KK - an , n -1 xn -1
ann

From equation n-1

From equation n

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form of each equation
n

c1 - a1 j x j
x1 =

j =1
j 1

a11

cn -1 xn -1 =

a 22

j =1
j n -1

n -1, j

xj

an -1,n -1

c n - a nj x j

c2 - a2 j x j
x2 =

a
n

j =1
j2

xn =

j =1
jn

a nn
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form for any row i
n

ci - aij x j
xi =

j =1
j i

aii

, i = 1,2,K , n.

How or where can this equation be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]

x1
x
2
M

xn -1
xn

Use rewritten equati ons to solve for

each value of x i.
Important: Remember to use the
most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error

a i =

-x

new
i

old
i

new
i

100

So when has the answer been f ound?

The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

The system of equati ons

Initial Guess: Assume an initial guess of

25 5 1 a1 106 .8
64 8 1 a = 177 .2

144 12 1 a3 279 .2

a1 1
a = 2
2
a3 5

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Rewriting each equation

25 5 1 a1 106 .8
64 8 1 a = 177 .2

144 12 1 a3 279 .2

106.8 - 5a 2 - a 3
a1 =
25
177.2 - 64a1 - a 3
a2 =
8
279.2 - 144a1 - 12a 2
a3 =
1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Applying the initial guess and solving for ai
a1 1
a = 2
2
a3 5
Initial Guess

a1 =

106.8 - 5(2) - (5)

= 3.6720
25

177.2 - 64(3.6720 ) - (5)

a2 =
= -7.8510
8
a3 =

279.2 - 144(3.6720 ) - 12(- 7.8510 )

= -155.36
1

When solving for a2, how many of the initial guess values were used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Finding the absolute relative approximate error

a i

xinew - xiold
=
100
new
xi

a 1 =

3.6720 - 1.0000
x100 = 72.76%
3.6720

- 7.8510 - 2.0000
=
x100 = 125.47%
- 7.8510

a 3 =

At the end of the first iteration

a1 3.6720
a = - 7.8510
2

a 3 - 155 .36
The maximum absolute
relative approximate error is
125.47%

- 155.36 - 5.0000
x100 = 103.22%
- 155.36
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Using

a1 3.6720
a = - 7.8510
2

a 3 - 155.36

Iteration #2
the values of ai are found:

a1 =

106.8 - 5(- 7.8510 ) - 155.36

= 12.056
25

a2 =

177.2 - 64(12.056 ) - 155.36

= -54.882
8

from iteration #1

a3 =

279.2 - 144(12.056 ) - 12(- 54.882 )

= -798.34
1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Finding the absolute relative approximate error
12.056 - 3.6720
At the end of the second iteration
a 1 =
x100 = 69.543%
12.056
a 12.056
1

a 2 =

a 3 =

- 54.882 - (- 7.8510)
x100 = 85.695%
- 54.882

- 798.34 - (- 155.36 )
x100 = 80.540%
- 798.34

a = - 54.882
2

a3 - 798.54

The maximum absolute

relative approximate error is
85.695%

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 1

Repeating more iterations, the following values are obtained
Iteration

1
2
3
4
5
6

a1
3.6720
12.056
47.182
193.33
800.53
3322.6

a 1 %
72.767
69.543
74.447
75.595
75.850
75.906

a2

a 2 %

a3

7.8510
54.882
255.51
1093.4
4577.2
19049

125.47
85.695
78.521
76.632
76.112
75.972

155.36
798.34
3448.9
14440
60072
249580

a 3 %
103.22
80.540
76.852
76.116
75.963
75.931

Notice The relative errors are not dec reasing at any significant rate

a 1 0.29048
Also, the solution is not converging to the true s olution of

a
=
19
.
690
2

a 3 1.0857
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall

What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gaus s-Siedel method: not all
systems of equations will converge.

Is there a fix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall

GAUSS-SEIDEL CONVERGENCE THEOREM: If A is
diagonally dominant, then the Gauss-Seidel method
converges for any starting vector x. A sufficient, but
not necessary condition.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n

aii aij
j =1
j i

for all i

and

aii > aij

for at least one i

j =1
j i

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Pitfall

Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other c oefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?
2 5.81 34
[A ] = 45 43 1
123 16
1

124 34 56
[ B] = 23 53 5
96 34 129

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

Given the system of equations

12 x1 + 3 x2- 5 x3 = 1

x1 + 5 x2 + 3x3 = 28
3x1 + 7 x2 + 13x3 = 76
With an initial guess of

x1 1
x = 0
2
x3 1

The coefficient matrix is:

12 3 - 5
[A] = 1 5 3
3 7 13
Will the solution converge using the
Gauss-Siedel method?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

Checking if the coefficient matrix is diagonally dominant
12 3 - 5
[A] = 1 5 3
3 7 13

a11 = 12 = 12 a12 + a13 = 3 + - 5 = 8

a 22 = 5 = 5 a 21 + a 23 = 1 + 3 = 4
a33 = 13 = 13 a31 + a32 = 3 + 7 = 10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

Rewriting each equation

With an initial guess of

12 3 - 5 a1 1
1 5 3 a = 28

2
3 7 13 a3 76

1 - 3x 2 + 5 x3
x1 =
12
28 - x1 - 3x3
x2 =
5

76 - 3x1 - 7 x 2
x3 =
13

x1 1
x = 0
2
x 3 1
x1 =

1 - 3(0) + 5(1)
= 0.50000
12

28 - (0.5) - 3(1)
x2 =
= 4.9000
5
76 - 3(0.50000 ) - 7(4.9000 )
x3 =
= 3.0923
13
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

The absolute relative approximate error

0.50000 - 1.0000
a 1 =
100 = 100.00%
0.50000

a
a

4.9000 - 0
=
100 = 100.00%
4.9000
3.0923 - 1.0000
=
100 = 67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

After Iteration #1
x1 0.5000
x = 4.9000
2

x3 3.0923

Substituting the x values into the

equations
x1 =

1 - 3(4.9000 ) + 5(3.0923)
= 0.14679
12

28 - (0.14679 ) - 3(3.0923)
x2 =
= 3.7153
5

x3 =

After Iteration #2
x1 0.14679
x = 3.7153
2

x3 3.8118

76 - 3(0.14679 ) - 7(4.900 )
= 3.8118
13
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

Iteration #2 absolute relative approximate error
a 1 =

0.14679 - 0.50000
100 = 240.61%
0.14679

3.7153 - 4.9000
=
100 = 31.889%
3.7153

3.8118 - 3.0923
=
100 = 18.874%
3.8118

The maximum absolute relative error after the fi rst iteration is 240.61%

This is much larger than the maxi mum absolute relative error obtai ned in
iteration #1. Is this a problem?
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 2

Repeating more iterations, the following values are obtained
Iteration

a1

a 1 %

a2

a 2 %

a3

1
2
3
4
5
6

0.50000
0.14679
0.74275
0.94675
0.99177
0.99919

100.00
240.61
80.236
21.546
4.5391
0.74307

4.9000
3.7153
3.1644
3.0281
3.0034
3.0001

100.00
31.889
17.408
4.4996
0.82499
0.10856

3.0923
3.8118
3.9708
3.9971
4.0001
4.0001

a 3 %
67.662
18.876
4.0042
0.65772
0.074383
0.00101

x1 0.99919
The solution obtained x = 3.0001 is close to the exact solution of
2

x3 4.0001

x1 1
x = 3 .
2
x3 4

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 3

Given the system of equations

3 x1 + 7 x2 + 13 x3 = 76

Rewriting the equations

x1 + 5 x2 + 3x3 = 28

76 - 7 x2 - 13 x3
x1 =
3

12 x1 + 3x2 - 5 x3 = 1
With an initial guess of

x1 1
x = 0
2
x3 1

28 - x1 - 3x3
x2 =
5

1 - 12 x1 - 3 x 2
x3 =
-5
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method: Example 3

Conducting six iterations, the following values are obtained
Iteration

a1

1
2
3
4
5
6

21.000
196.15
1995.0
20149
2.0364105
2.0579105

a 1 %

A2

95.238
0.80000
110.71
14.421
109.83
116.02
109.90
1204.6
109.89
12140
109.89 1.2272105

a 2 %

a3

a 3 %

100.00
94.453
112.43
109.63
109.92
109.89

50.680
462.30
4718.1
47636
4.8144105
4.8653106

98.027
110.96
109.80
109.90
109.89
109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
But this is the same set of
equations used in example #2,
which did converge.

3 7 13
[A] = 1 5 3
12 3 - 5
12 3 - 5
[A] = 1 5 3
3 7 13

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equati ons can form a di agonally dominant matrix.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equati ons

x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9

x1 + 7 x 2 + x3 = 9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss -Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method

Questions?

http://numericalmethods.eng.usf.edu

Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/gauss_seid
el.html

THE END
http://numericalmethods.eng.usf.edu